Manage Data Sources

Data sources let you store cloud credentials and mirror the contents of an external bucket (ex: Amazon S3, Google Cloud Storage) into your Workspace. You can manage both programmatically with the REST API.

A data source has two parts: a credential that holds the access secret, and a bucket mirror configuration that references a credential by id and describes which files to mirror.

These endpoints are restricted to Workspace owners, or to API keys and OAuth apps granted the matching scopes listed under each operation.

Your api_key must be sent in all requests, either as a query parameter or as a top level attribute in the request body. Secrets in credential responses are always masked.

Credentials

List Credentials

GET /:workspace/credentials

Lists credentials in a Workspace with their secrets masked. Requires the credentials:read scope.

Query

Name	Type	Description	Required
api_key	string	API key of the Workspace
type	string	Filter by credential type

Response

200

{
  "data": [
    {
      "id": "<credential_id>",
      "owner": "<workspace_id>",
      "type": "IAM",
      "name": "My AWS Credential",
      "payload": { "type": "IAM", "accessKeyId": "AKIA...", "secretAccessKey": "****" }
    }
  ]
}

Create a Credential

POST /:workspace/credentials

Creates a credential. Requires the credentials:create scope.

Body

Name	Type	Description	Required
type	string	One of IAM, AssumeRole, AssumeRoleWithWebIdentity, gcs, usernamePassword, apiKey. Must match payload.type.
name	string	Display name (1 to 255 characters)
description	string	Optional description
metadata	object	Optional key/value metadata
payload	object	The credential secret, shaped by type (see below)

Payload shape by type:

type	payload fields
IAM	accessKeyId, secretAccessKey, sessionToken?
AssumeRole	roleArn, externalId, sessionName?
AssumeRoleWithWebIdentity	roleArn, externalId, webIdentity: { audience, roleArn }, sessionName?
gcs	projectId, credentialFile
usernamePassword	username, password
apiKey	apiKey

Example Request

curl --location 'https://api.roboflow.com/<workspace_id>/credentials?api_key=<api_key>' \
--header 'Content-Type: application/json' \
--data '{
    "type": "IAM",
    "name": "My AWS Credential",
    "payload": {
        "type": "IAM",
        "accessKeyId": "<access_key_id>",
        "secretAccessKey": "<secret_access_key>"
    }
}'

Response

201

{ "id": "<credential_id>" }

Delete a Credential

DELETE /:workspace/credentials/:credentialId

Deletes a credential. Requires the credentials:delete scope.

Response

204

No Content. The credential was deleted.

404

Bucket Mirror Configurations

List Configurations

GET /:workspace/bucket-mirror-configs

Lists bucket mirror configurations, each with a summary of its latest sync job. Requires the datasource:bucket-mirror:read scope.

Response

200

{ "data": [ { "id": "<config_id>", "name": "...", "latestJob": { "batchId": "...", "status": "completed" } } ] }

Create a Configuration

POST /:workspace/bucket-mirror-configs

Creates a bucket mirror configuration. The referenced credential must belong to the same Workspace. Requires the datasource:bucket-mirror:write scope.

Body

Name	Type	Description	Required
name	string	Display name
description	string	Optional description
credentialId	string	Id of the credential used to access the bucket
bucket	object	{ type: "s3" | "gcs", bucket, region, endpoint? }
mirrorConfigs	array	At least one mirror rule: { id, settings, globPatterns?, globFilePath? }
requireVpce	boolean	Route access through an AWS PrivateLink endpoint. Defaults to false.

Example Request

curl --location 'https://api.roboflow.com/<workspace_id>/bucket-mirror-configs?api_key=<api_key>' \
--header 'Content-Type: application/json' \
--data '{
    "name": "Product Images",
    "credentialId": "<credential_id>",
    "bucket": { "type": "s3", "bucket": "my-bucket", "region": "us-east-1" },
    "mirrorConfigs": [
        { "id": "images", "globPatterns": ["**/*.jpg"], "settings": {} }
    ]
}'

Response

201

{ "id": "<config_id>" }

Get a Configuration

GET /:workspace/bucket-mirror-configs/:configId

Retrieves a single configuration. Requires the datasource:bucket-mirror:read scope.

Update a Configuration

PUT /:workspace/bucket-mirror-configs/:configId

Updates a configuration. Send only the fields you want to change. Requires the datasource:bucket-mirror:write scope.

Response

200

{ "data": { "id": "<config_id>", "name": "..." } }

Delete a Configuration

DELETE /:workspace/bucket-mirror-configs/:configId

Deletes a configuration. Requires the datasource:bucket-mirror:write scope. Returns 204 No Content.

Validate a Configuration

GET /:workspace/bucket-mirror-configs/:configId/validate

Checks that the credential can list, head, and read objects in the bucket. Requires the datasource:bucket-mirror:read scope.

Response

200

{
  "data": {
    "checks": { "listFiles": true, "headFile": true, "getFile": true, "mirrorConfigs": {} },
    "errors": []
  }
}

Trigger a Sync

POST /:workspace/bucket-mirror-configs/:configId/trigger

Starts a mirror run for the configuration and returns the batch ids created (one per mirror rule). The configuration is validated first, and manual runs are rate limited. Requires the datasource:bucket-mirror:trigger scope.

Response

200

{ "batchIds": ["<config_id>-<mirror_id>-<timestamp>"] }

409

429

Get a Sync Job

GET /:workspace/bucket-mirror-configs/:configId/jobs/:jobId

Retrieves a single sync job by its batchId so you can poll its status after triggering a run. Requires the datasource:bucket-mirror:read scope.

Response

200

{
  "data": {
    "batchId": "<config_id>-<mirror_id>-<timestamp>",
    "status": "running",
    "counters": {},
    "errors": [],
    "bucketContext": {}
  }
}

404