Query with SQL
Playground
POSTEndpoint:
/api/query/v1/execute/:datasetId
This endpoint executes an SQL query on the specified DataSet and returns the result in the form of a list of objects.
Parameters
| Property Name | Type | Required | Description |
|---|---|---|---|
| datasetId | String | Yes | The ID of the DataSet to query. |
| sql | String | Yes | The SQL statement to execute. |
Example
Response
Dataset Access List
Playground
GET
Endpoint: /api/data/v3/datasources/{dataset_id}/permissions
Description:Retrieves a list of users and groups with access to the dataset, along with their permissions. Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| dataset_id | String | Yes | Unique identifier of the dataset. |
Revoke Dataset Access
Playground
DELETEEndpoint:
/api/data/v3/datasources/:datasetId/permissions/USER/:userId
This endpoint revokes access to a specified DataSet for a given user.
Parameters
| Property Name | Type | Required | Description |
|---|---|---|---|
| datasetId | String | Yes | The ID of the DataSet to revoke access from. |
| userId | String | Yes | The ID of the user to revoke access for. |
Example
Response
Get Metadata
Playground
GETEndpoint:
/api/data/v3/datasources/:datasetId
This endpoint retrieves metadata for a specified DataSet, including the dataset’s properties and optionally requested parts of the metadata.
Parameters
| Property Name | Type | Required | Description |
|---|---|---|---|
| datasetId | String | Yes | The ID of the DataSet to retrieve metadata for. |
| requestedParts | String | No | A comma-separated list of metadata parts to include. |
Example
Response
Append row to Dataset
Playground
POSTEndpoint:
/api/data/v3/datasources/:datasetId/uploads
Description
This endpoint appends a row of values to a specified dataset.Parameters
| Property Name | Type | Required | Description |
|---|---|---|---|
| datasetId | String | Yes | The ID of the dataset to which values will be appended. |
| values | String | Yes | A comma-delimited text string of values to append to the dataset. |
| delimiter | String | No | The delimiter used to split the values (default is comma). |
Example Request
Response
Stream (Connector) - Create
Playground
POSTEndpoint:
/api/data/v1/streams
Description:Creates a new data stream using a connector and specifies configuration details. Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| transport | Object | Yes | Connector transport details, including type and version. |
| configuration | Array | Yes | List of key-value pairs defining stream configurations. |
| account | Object | Yes | Account information (e.g., ID). |
| updateMethod | String | Yes | Update method, e.g., APPEND. |
| dataProvider | Object | Yes | Data provider details, including key. |
| dataSource | Object | Yes | Dataset details, including name and description. |
| advancedScheduleJson | String | No | Schedule details in JSON format. |
Stream (Connector) - Get
Playground
GETEndpoint:
/api/data/v1/streams/{stream_id}
Description:Retrieves configuration details for a specific data stream. Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| stream_id | String | Yes | Unique identifier of the stream. |
Stream (Connector) - Update
Playground
PUTEndpoint:
/api/data/v1/streams/{stream_id}
Description:Updates the configuration of an existing data stream. Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| stream_id | String | Yes | Unique identifier of the stream. |
| transport | Object | Yes | Connector transport details, including type and description. |
| configuration | Array | Yes | List of key-value pairs defining updated configurations. |
| account | Object | Yes | Account information (e.g., ID). |
| updateMethod | String | Yes | Update method, e.g., APPEND. |
| dataProvider | Object | Yes | Data provider details, including key. |
| dataSource | Object | Yes | Dataset details, including name and description. |
| advancedScheduleJson | String | No | Schedule details in JSON format. |
Stream Execution History
Playground
GETEndpoint:
/api/data/v1/streams/{stream_id}/history/aggregate
Description:Retrieves the execution history for a specific data stream, including details about completed operations, errors, and performance metrics. Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| stream_id | String | Yes | Unique identifier of the stream. |
Execute Dataset Stream
Playground
POSTEndpoint:
/api/data/v1/streams/:streamId/executions
Description
This endpoint queries a dataset’s stream ID and executes the dataset stream.Parameters
| Property Name | Type | Required | Description |
|---|---|---|---|
| streamId | String | Yes | The ID of the dataset stream to execute. |
| action | String | Yes | The action to execute on the. “Manual”, for example dataset stream. |
Example Request
Response
createDatasetTag
Playground
POSTEndpoint:
/api/data/ui/v3/datasources/:datasetId/tags
Description
This endpoint creates a tag for the specified dataset.Parameters
| Property Name | Type | Required | Description |
|---|---|---|---|
| datasetId | String | Yes | The ID of the dataset to tag. |
| tag | String | Yes | The tag to assign to the dataset. |
Example Request
Response
Share Dataset with Access permissions
Playground
POSTEndpoint:
/api/data/v3/datasources/:datasetId/share
Description
This endpoint shares a dataset and grants access permissions to specified users.Parameters
| Property Name | Type | Required | Description |
|---|---|---|---|
| datasetId | String | Yes | The ID of the dataset to share. |
| permissions | Array | Yes | A list of owners to add to the dataset. Each owner is an object with id, type, and accessLevel. |
| sendEmail | Boolean | Yes | A flag indicating whether to send an email. |
Example Request
Response
Remove user from dataset
Playground
DELETEEndpoint:
/api/data/v3/datasources/:datasetId/USERS/:userId
Description
This endpoint removes a user from a dataset’s permissions.Parameters
| Property Name | Type | Required | Description |
|---|---|---|---|
| datasetId | String | Yes | The ID of the dataset from which the user will be removed. |
| userId | String | Yes | The ID of the user to remove from the dataset’s permissions. |
Example Request
Response
Remove multiple users from dataset
Playground
DELETEEndpoint:
/api/data/v3/datasources/:datasetId/users
Description
This endpoint removes multiple people from a dataset’s permissions.Parameters
| Property Name | Type | Required | Description |
|---|---|---|---|
| datasetId | String | Yes | The ID of the dataset from which the people will be removed. |
| body | Array | Yes | An array of user objects, each containing id and type properties. |
Example Request
Response
Upload CSV - APPEND or REPLACE
Stage 1 - Get Upload ID
Playground
POSTEndpoint:
/api/data/v3/datasources/:dataset_id/uploads
Description:Generates an
uploadId for appending or replacing data in the dataset. The uploadId is required for subsequent upload stages.
Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| dataset_id | String | Yes | Unique identifier of the dataset. |
Stage 2 - Upload CSV
Playground
PUTEndpoint:
/api/data/v3/datasources/:dataset_id/uploads/:upload_id/parts/:part
Description:Uploads the CSV file data. The dataset is divided into parts for uploading multiple streams simultaneously. Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| dataset_id | String | Yes | Unique identifier of the dataset. |
| upload_id | String | Yes | Upload ID generated in Stage 1. |
| part | Number | Yes | Indicates the part number of the data. |
Stage 3 - Commit and Index
Playground
PUT
Endpoint: /api/data/v3/datasources/:dataset_id/uploads/:upload_id/commit
Description:Finalizes the data upload by committing the uploaded parts and indexing the dataset. Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| dataset_id | String | Yes | Unique identifier of the dataset. |
| upload_id | String | Yes | Upload ID generated in Stage 1. |
| Property Name | Type | Required | Description |
|---|---|---|---|
| action | String | Yes | Accepts APPEND or REPLACE. |
| index | Boolean | Yes | Indicates if indexing should be executed. |
Upload CSV - APPEND with Partitioning
Stage 1 - Get Partition Tag
Playground
POST
Endpoint: /api/data/v3/datasources/{dataset_id}/uploads/?restateDataTag={dataset_partition_id}
Description:Initializes a data upload with partitioning and generates an
uploadId. The uploadId is required for subsequent upload stages. Note that restateDataTag is largely deprecated and retained for backward compatibility.
Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| dataset_id | String | Yes | Unique identifier of the dataset. |
| restateDataTag | String | No | Specifies a partition for UPDATE/REPLACE actions. Defaults to None. |
Stage 2 - Upload CSV
Playground
PUT
Endpoint: /api/data/v3/datasources/{dataset_id}/uploads/{dataset_upload_id}/parts/:part
Description:Uploads data in CSV format. Multiple parts can be uploaded simultaneously for larger datasets. Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| dataset_id | String | Yes | Unique identifier of the dataset. |
| dataset_upload_id | String | Yes | Upload ID generated in Stage 1. |
| part | Number | Yes | Indicates the part number being uploaded. |
Stage 3 - Commit and Index
Playground
PUT
Endpoint: /api/data/v3/datasources/{dataset_id}/uploads/{dataset_upload_id}/commit
Description:Finalizes the data upload by committing all parts and optionally indexing the dataset for query access. Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| dataset_id | String | Yes | Unique identifier of the dataset. |
| dataset_upload_id | String | Yes | Upload ID generated in Stage 1. |
| action | String | Yes | Acceptable values: APPEND, REPLACE. Determines the update method. |
| restateDataTag | String | No | Specifies a partition tag for updating specific partitions. |
| index | Boolean | Yes | Indicates whether the data should be indexed. |
GET Data Versions
Playground
GETEndpoint:
/api/data/v3/datasources/{dataset_id}/dataversions/details
Description:Retrieves detailed information about all data versions associated with the dataset. Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| dataset_id | String | Yes | Unique identifier of the dataset. |
GET Partition List
Playground
GETEndpoint:
/api/query/v1/datasources/{dataset_id}/partition
Description:Retrieves a list of partitions associated with the dataset. Does not include metadata such as row counts. Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| dataset_id | String | Yes | Unique identifier of the dataset. |
- To include row count metadata, use the
/partition/listendpoint. - This endpoint does not differentiate between partitions marked for deletion and those not marked.
Search Partition List
Playground
POSTEndpoint:
/api/query/v1/datasources/{dataset_id}/partition/list
Description:Searches for partitions associated with the dataset. Allows pagination, sorting, and filtering of results. Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| dataset_id | String | Yes | Unique identifier of the dataset. |
| paginationFields | Array | No | Defines sorting and filtering. |
| limit | Number | No | Maximum number of results. Default: 100 |
| offset | Number | No | Number of results to skip. Default: 0 |
Data Version List from Magic
Playground
POSTEndpoint:
/api/dataprocessing/v2/dataflows/data-version-hydrate
Description:Retrieves a list of data versions for datasets processed with the MAGIC engine. Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| dataset_id | String | Yes | Unique identifier of the dataset. |
| engine | String | No | Defaults to MAGIC. |
| limit | Number | No | Maximum number of results. Default: 500 |
| offset | Number | No | Number of results to skip. Default: 0 |
Delete Data Version
Playground
DELETE
Endpoint: /api/query/v1/datasources/{dataset_id}/tag/{dataset_partition_id}/data
Description:Marks the data version associated with a partition tag as deleted. This does not delete the partition tag itself or remove the association between the partition tag and the data version. Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| dataset_id | String | Yes | Unique identifier of the dataset. |
| dataset_partition_id | String | Yes | Identifier of the dataset partition to delete. |
- Use this endpoint to mark data versions as deleted without removing the partition tag.
- To only remove the partition tag, use the “Delete Partition Tag” endpoint.
Delete Partition Tag
Playground
DELETE
Endpoint: /api/query/v1/datasources/{dataset_id}/partition/{dataset_partition_id}
Description:Removes the association of a partition tag with its dataset, ensuring it no longer appears in the partition list. Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| dataset_id | String | Yes | Unique identifier of the dataset. |
| dataset_partition_id | String | Yes | Identifier of the dataset partition to remove. |
- Removing the partition tag does not count against the 400-partition limit in Magic 2.0.
- Partitions against deleted data versions will not appear in the partition list.
Index Dataset
Playground
POST
Endpoint: /api/data/v3/datasources/{dataset_id}/indexes
Description:Indexes data versions for a dataset to make them queryable. If no
dataIds are provided, all complete and unindexed versions are indexed.
Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| dataset_id | String | Yes | Unique identifier of the dataset. |
| dataIds | Array | No | List of data version IDs to index. |
- Sending an empty
dataIdsarray will index all eligible data versions. - Indexing is required for datasets to become accessible for querying.
Create Dataset Copy
Playground
POSTEndpoint:
/api/data/v2/datasources
Description:Creates a new dataset by copying an existing schema and providing user-defined metadata. Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| userDefinedType | String | Yes | Specifies the type of the dataset. Default: api. |
| dataSourceName | String | Yes | Name of the new dataset. |
| schema | Object | Yes | Defines the dataset schema, including column types and names. |
Get Dataset Schema
Playground
GETEndpoint:
/api/query/v1/datasources/{dataset_id}/schema/indexed?includeHidden=false
Description:Retrieves the schema of a dataset, including details about columns, their types, and metadata. Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| dataset_id | String | Yes | Unique identifier of the dataset. |
| includeHidden | Boolean | No | Specifies whether hidden columns are included. Default: false. |
Alter Dataset Schema
Playground
POSTEndpoint:
/api/data/v2/datasources/{dataset_id}/schemas
Description:Modifies the schema of an existing dataset by adding or altering columns. Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| dataset_id | String | Yes | Unique identifier of the dataset. |
| schema | Object | Yes | Updated schema definition. |
Get Cards for a Dataset ID
Playground
GETEndpoint:
/api/content/v1/datasources/{dataset_id}/cards?drill=true
Description:Retrieves a list of cards associated with a dataset. Includes drill paths for cards if
drill is set to true.
Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| dataset_id | String | Yes | Unique identifier of the dataset. |
| drill | Boolean | No | Whether to include drill paths. Default: true. |
Index Dataset
Playground
POST
Endpoint: /api/query/v1/datasources/{dataset_id}
Description:Indexes a dataset to make it available for querying. This process is required after schema changes or data updates. Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| dataset_id | String | Yes | Unique identifier of the dataset. |
Index Dataset Progress
Playground
GET
Endpoint: /api/data/v3/datasources/{dataset_id}/indexes/{index_id}/statuses
Description:
Retrieves the status of a dataset indexing operation for the specified index ID.
Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| dataset_id | String | Yes | Unique identifier of the dataset. |
| index_id | String | Yes | Unique identifier of the index. |
Change Dataset Properties
Playground
PUT
Endpoint: /api/data/v3/datasources/{dataset_id}/properties
Description:
Updates the properties of a dataset, such as the data provider type.
Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| dataset_id | String | Yes | Unique identifier of the dataset. |
| dataProviderType | String | Yes | The type of the data provider. |
Dataset Access List
Playground
GET
Endpoint: /api/data/v3/datasources/{dataset_id}/permissions
Description:
Retrieves a list of users and groups with access to the dataset, along with their permissions.
Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| dataset_id | String | Yes | Unique identifier of the dataset. |
Change Dataset Properties
Playground
PUT
Endpoint: /api/data/v3/datasources/{dataset_id}/properties
Description:
Updates the properties of a dataset, such as the data provider type.
Parameters:
| Property Name | Type | Required | Description |
|---|---|---|---|
| dataset_id | String | Yes | Unique identifier of the dataset. |
| dataProviderType | String | Yes | The type of the data provider. |