> For the complete documentation index, see [llms.txt](https://docs.roboflow.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.roboflow.com/developer/rest-api/manage-data-sources.md).

# Manage Data Sources

Data sources let you store cloud credentials and mirror the contents of an external bucket (ex: Amazon S3, Google Cloud Storage) into your Workspace. You can manage both programmatically with the REST API.

A data source has two parts: a credential that holds the access secret, and a bucket mirror configuration that references a credential by id and describes which files to mirror.

{% hint style="info" %}
These endpoints are restricted to Workspace owners, or to API keys and OAuth apps granted the matching scopes listed under each operation.
{% endhint %}

Your `api_key` must be sent in all requests, either as a query parameter or as a top level attribute in the request body. Secrets in credential responses are always masked.

## Credentials

### List Credentials

<mark style="color:green;">`GET`</mark> `/:workspace/credentials`

Lists credentials in a Workspace with their secrets masked. Requires the `credentials:read` scope.

**Query**

<table><thead><tr><th>Name</th><th>Type</th><th>Description</th><th data-type="checkbox">Required</th></tr></thead><tbody><tr><td><code>api_key</code></td><td>string</td><td>API key of the Workspace</td><td>true</td></tr><tr><td><code>type</code></td><td>string</td><td>Filter by credential type</td><td>false</td></tr></tbody></table>

**Response**

{% tabs %}
{% tab title="200" %}

```json
{
  "data": [
    {
      "id": "<credential_id>",
      "owner": "<workspace_id>",
      "type": "IAM",
      "name": "My AWS Credential",
      "payload": { "type": "IAM", "accessKeyId": "AKIA...", "secretAccessKey": "****" }
    }
  ]
}
```

{% endtab %}
{% endtabs %}

### Create a Credential

<mark style="color:green;">`POST`</mark> `/:workspace/credentials`

Creates a credential. Requires the `credentials:create` scope.

**Body**

<table><thead><tr><th width="150">Name</th><th width="200">Type</th><th>Description</th><th data-type="checkbox">Required</th></tr></thead><tbody><tr><td><code>type</code></td><td>string</td><td>One of <code>IAM</code>, <code>AssumeRole</code>, <code>AssumeRoleWithWebIdentity</code>, <code>gcs</code>, <code>usernamePassword</code>, <code>apiKey</code>. Must match <code>payload.type</code>.</td><td>true</td></tr><tr><td><code>name</code></td><td>string</td><td>Display name (1 to 255 characters)</td><td>true</td></tr><tr><td><code>description</code></td><td>string</td><td>Optional description</td><td>false</td></tr><tr><td><code>metadata</code></td><td>object</td><td>Optional key/value metadata</td><td>false</td></tr><tr><td><code>payload</code></td><td>object</td><td>The credential secret, shaped by <code>type</code> (see below)</td><td>true</td></tr></tbody></table>

Payload shape by type:

<table><thead><tr><th width="230">type</th><th>payload fields</th></tr></thead><tbody><tr><td><code>IAM</code></td><td><code>accessKeyId</code>, <code>secretAccessKey</code>, <code>sessionToken?</code></td></tr><tr><td><code>AssumeRole</code></td><td><code>roleArn</code>, <code>externalId</code>, <code>sessionName?</code></td></tr><tr><td><code>AssumeRoleWithWebIdentity</code></td><td><code>roleArn</code>, <code>externalId</code>, <code>webIdentity: { audience, roleArn }</code>, <code>sessionName?</code></td></tr><tr><td><code>gcs</code></td><td><code>projectId</code>, <code>credentialFile</code></td></tr><tr><td><code>usernamePassword</code></td><td><code>username</code>, <code>password</code></td></tr><tr><td><code>apiKey</code></td><td><code>apiKey</code></td></tr></tbody></table>

**Example Request**

```
curl --location 'https://api.roboflow.com/<workspace_id>/credentials?api_key=<api_key>' \
--header 'Content-Type: application/json' \
--data '{
    "type": "IAM",
    "name": "My AWS Credential",
    "payload": {
        "type": "IAM",
        "accessKeyId": "<access_key_id>",
        "secretAccessKey": "<secret_access_key>"
    }
}'
```

**Response**

{% tabs %}
{% tab title="201" %}

```json
{ "id": "<credential_id>" }
```

{% endtab %}
{% endtabs %}

### Delete a Credential

<mark style="color:red;">`DELETE`</mark> `/:workspace/credentials/:credentialId`

Deletes a credential. Requires the `credentials:delete` scope.

**Response**

{% tabs %}
{% tab title="204" %}
No Content. The credential was deleted.
{% endtab %}

{% tab title="404" %}
The credential does not exist in this Workspace.
{% endtab %}
{% endtabs %}

## Bucket Mirror Configurations

### List Configurations

<mark style="color:green;">`GET`</mark> `/:workspace/bucket-mirror-configs`

Lists bucket mirror configurations, each with a summary of its latest sync job. Requires the `datasource:bucket-mirror:read` scope.

**Response**

{% tabs %}
{% tab title="200" %}

```json
{ "data": [ { "id": "<config_id>", "name": "...", "latestJob": { "batchId": "...", "status": "completed" } } ] }
```

{% endtab %}
{% endtabs %}

### Create a Configuration

<mark style="color:green;">`POST`</mark> `/:workspace/bucket-mirror-configs`

Creates a bucket mirror configuration. The referenced credential must belong to the same Workspace. Requires the `datasource:bucket-mirror:write` scope.

**Body**

<table><thead><tr><th width="160">Name</th><th width="220">Type</th><th>Description</th><th data-type="checkbox">Required</th></tr></thead><tbody><tr><td><code>name</code></td><td>string</td><td>Display name</td><td>true</td></tr><tr><td><code>description</code></td><td>string</td><td>Optional description</td><td>false</td></tr><tr><td><code>credentialId</code></td><td>string</td><td>Id of the credential used to access the bucket</td><td>true</td></tr><tr><td><code>bucket</code></td><td>object</td><td><code>{ type: "s3" | "gcs", bucket, region, endpoint? }</code></td><td>true</td></tr><tr><td><code>mirrorConfigs</code></td><td>array</td><td>At least one mirror rule: <code>{ id, settings, globPatterns?, globFilePath? }</code></td><td>true</td></tr><tr><td><code>requireVpce</code></td><td>boolean</td><td>Route access through an AWS PrivateLink endpoint. Defaults to <code>false</code>.</td><td>false</td></tr></tbody></table>

**Example Request**

```
curl --location 'https://api.roboflow.com/<workspace_id>/bucket-mirror-configs?api_key=<api_key>' \
--header 'Content-Type: application/json' \
--data '{
    "name": "Product Images",
    "credentialId": "<credential_id>",
    "bucket": { "type": "s3", "bucket": "my-bucket", "region": "us-east-1" },
    "mirrorConfigs": [
        { "id": "images", "globPatterns": ["**/*.jpg"], "settings": {} }
    ]
}'
```

**Response**

{% tabs %}
{% tab title="201" %}

```json
{ "id": "<config_id>" }
```

{% endtab %}
{% endtabs %}

### Get a Configuration

<mark style="color:green;">`GET`</mark> `/:workspace/bucket-mirror-configs/:configId`

Retrieves a single configuration. Requires the `datasource:bucket-mirror:read` scope.

### Update a Configuration

<mark style="color:blue;">`PUT`</mark> `/:workspace/bucket-mirror-configs/:configId`

Updates a configuration. Send only the fields you want to change. Requires the `datasource:bucket-mirror:write` scope.

**Response**

{% tabs %}
{% tab title="200" %}

```json
{ "data": { "id": "<config_id>", "name": "..." } }
```

{% endtab %}
{% endtabs %}

### Delete a Configuration

<mark style="color:red;">`DELETE`</mark> `/:workspace/bucket-mirror-configs/:configId`

Deletes a configuration. Requires the `datasource:bucket-mirror:write` scope. Returns `204 No Content`.

### Validate a Configuration

<mark style="color:green;">`GET`</mark> `/:workspace/bucket-mirror-configs/:configId/validate`

Checks that the credential can list, head, and read objects in the bucket. Requires the `datasource:bucket-mirror:read` scope.

**Response**

{% tabs %}
{% tab title="200" %}

```json
{
  "data": {
    "checks": { "listFiles": true, "headFile": true, "getFile": true, "mirrorConfigs": {} },
    "errors": []
  }
}
```

{% endtab %}
{% endtabs %}

### Trigger a Sync

<mark style="color:green;">`POST`</mark> `/:workspace/bucket-mirror-configs/:configId/trigger`

Starts a mirror run for the configuration and returns the batch ids created (one per mirror rule). The configuration is validated first, and manual runs are rate limited. Requires the `datasource:bucket-mirror:trigger` scope.

**Response**

{% tabs %}
{% tab title="200" %}

```json
{ "batchIds": ["<config_id>-<mirror_id>-<timestamp>"] }
```

{% endtab %}

{% tab title="409" %}
A sync is already in progress for this data source.
{% endtab %}

{% tab title="429" %}
The manual sync cooldown or hourly cap was hit. Wait before triggering another run.
{% endtab %}
{% endtabs %}

### Get a Sync Job

<mark style="color:green;">`GET`</mark> `/:workspace/bucket-mirror-configs/:configId/jobs/:jobId`

Retrieves a single sync job by its `batchId` so you can poll its status after triggering a run. Requires the `datasource:bucket-mirror:read` scope.

**Response**

{% tabs %}
{% tab title="200" %}

```json
{
  "data": {
    "batchId": "<config_id>-<mirror_id>-<timestamp>",
    "status": "running",
    "counters": {},
    "errors": [],
    "bucketContext": {}
  }
}
```

{% endtab %}

{% tab title="404" %}
No job with that id exists for this configuration.
{% endtab %}
{% endtabs %}