> For the complete documentation index, see [llms.txt](https://docs.roboflow.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.roboflow.com/datasets/annotate/annotate/ai-labeling/auto-label.md).

# Auto Label

Auto-label images for use in training models.

## About

{% hint style="info" %}
Auto Label is one of many [AI Labeling](/datasets/annotate/annotate/ai-labeling.md) features. Using this feature will consume [credits](https://docs.roboflow.com/platform/billing-and-plans/credits) at the rates listed on our [credits page](https://roboflow.com/credits).
{% endhint %}

Roboflow Auto Label lets you use a foundation vision model (GPT-6 Astra, SAM 3, or Gemini), Roboflow trained models, or a saved [Workflow](https://docs.roboflow.com/workflows) to automatically label images.

Roboflow Auto Label can use the following sources to identify the objects you specify:

* GPT-6 Astra (Boxes) - GPT-6 Astra draws bounding boxes for the classes you list (object detection only)
* SAM 3 - a foundation model that labels objects from text prompts (object detection and segmentation)
* Gemini (Boxes) - Gemini 3.7 Flash draws bounding boxes for the classes you list (object detection only)
* Models trained in Roboflow ([https://github.com/roboflow/docs/tree/main/models/README.md](https://github.com/roboflow/docs/tree/main/models/README.md "mention"))
  * Note: Only models from the same dataset as your Annotation Batch are currently supported.
* A saved [Workflow](https://docs.roboflow.com/workflows) compatible with your project type

GPT-6 Astra (Boxes) is selected for you on object detection projects. It handles open vocabulary class names, visible text, and object relationships well. Your workspace needs a paid plan to use it, and OpenAI token use is billed on top of the credits per image, so it costs more per image than SAM 3. SAM 3 is selected for you on every other project type.

Auto Label has been used to label millions of images for use in training computer vision models.

If you would rather auto-label on your own hardware, Roboflow also maintains [Autodistill](https://github.com/autodistill/autodistill), an open source framework for auto-labeling image datasets with a range of foundation models.

### When to Use Roboflow Auto Label

You should use Roboflow Auto Label if you need to annotate common objects such as vehicles (i.e. forklifts), people, generic defects (i.e. cracks), and generic products (i.e. vinyl records, bread).

You should not use a foundation model like SAM 3 if you need to identify specific variants of an object. For example, Auto Label cannot distinguish between different types of crack, or identify unique defects in electronics.

## Label Data with Roboflow Auto Label

The Roboflow platform lets you preview how Auto Label will perform on labeling classes of data in your dataset before you run a job on your full batch. After labeling, you can review the results for quality assurance (recommended) and train a model on the labeled dataset.

### Step #1: Upload Data

First, upload data to Roboflow. See our [Upload Images, Videos, and Annotations](/datasets/create-and-upload/adding-data.md) instructions for more information.

<figure><img src="https://2252499186-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNVfRxNsvCh4GOyqrjmzX%2Fuploads%2Fgit-blob-fe3c09375969482e7a921eedaf3d020b21ca43b8%2FScreenshot%202024-03-26%20at%2008.59.56.png?alt=media" alt=""><figcaption><p>Uploading images into Roboflow.</p></figcaption></figure>

### Step #2: Enter Auto Label

Once you have uploaded all of your images, you will be asked how you want to label your images. Select "Auto-Label And Review".

<figure><img src="https://2252499186-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNVfRxNsvCh4GOyqrjmzX%2Fuploads%2Fgit-blob-950d476161ba6ec84080f5e3baeef2181f55ad31%2FScreenshot%202024-03-26%20at%2009.00.42.png?alt=media" alt="" width="353"><figcaption><p>Select "Auto-Label And Review" to open the Roboflow Auto Label interface.</p></figcaption></figure>

### Step #3: Configure Auto Label

The Auto Label labeling interface will appear in which you can configure your auto labeling job.

#### Classes (& Descriptions)

Classes represent the labels you want to assign to objects in the image. Descriptions represent a visual description of the class that the foundation model will use to identify instances of those classes. By default, the description will be the class name.

GPT-6 Astra (Boxes), SAM 3, and Gemini (Boxes) all search with the description and save what they find under the class name. For example, the class "umbrella" with the description "green umbrella" finds green umbrellas and labels them "umbrella". Give each class its own description: Auto Label rejects a preview or a job when two classes share one.

Auto Label works best when labeling common objects with clear visual descriptions. For example, Auto Label will be able to identify the location of an aluminum can on a production line. But, Auto Label will be unable to label images according to specific requirements, such as distinguishing the brand of an aluminum can.

#### Generating Test Results

Once you have configured Auto Label, click "Generate Test Results" to test your classes on a small subset of your dataset. By default, four images are selected.

<figure><img src="https://2252499186-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNVfRxNsvCh4GOyqrjmzX%2Fuploads%2Fgit-blob-f67fac5e50f37beb91c4defd53c49db449550de2%2FScreenshot%202024-03-26%20at%2009.02.20.png?alt=media" alt=""><figcaption><p>The Auto Label interface.</p></figcaption></figure>

### Step #4: Evaluate Roboflow Auto Label Labels

Here are the test results when using the class "aluminum can" on an example image. From here, you can:

<figure><img src="https://2252499186-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNVfRxNsvCh4GOyqrjmzX%2Fuploads%2Fgit-blob-618471f153fa5f3f372ac900efbba9ca950f3c9a%2FScreenshot%202024-03-22%20at%2011.33.02.webp?alt=media" alt=""><figcaption><p>Auto Label annotating aluminum cans.</p></figcaption></figure>

#### Adjust your classes & descriptions

If Auto Label doesn't label images as expected, try testing different descriptions for your classes.

{% hint style="info" %}
All test results are free & don't use any credits.
{% endhint %}

#### Adjust the confidence

The number to the right of each class is represented as (boxes of this class that are shown) / (total boxes of this class). You can adjust the confidence threshold of each class to filter out more or less boxes. Higher confidence means less boxes will be shown.

{% hint style="info" %}
The confidence threshold set here will be the same one used when labeling the entire batch, so make sure it looks right!
{% endhint %}

GPT-6 Astra (Boxes) and Gemini (Boxes) return every box at full confidence, so they have no threshold to set. The slider and the confidence values on preview labels are hidden when you select one of them.

#### Test on different images

To review Auto Label's performance on different images from your batch, click on an image in the "Test images" section on the bottom left. The preview for that image should load immediately without pressing additional buttons.

## Label with a Workflow

Instead of a foundation or trained model, you can auto-label with a saved [Workflow](https://docs.roboflow.com/workflows), letting you use SAHI, model ensembles, and multi-step custom logic to label your batch.

In the Auto Label model dropdown, open the "Workflows" tab and select a workflow. Only workflows compatible with your project type (detection, segmentation, or classification) are listed.

After selecting a workflow, map each of its output classes to a class in your project and set a per-class confidence threshold. A preview runs on sample images so you can confirm the mapping before starting the job.

{% hint style="info" %}
Only the classes you map are applied. Output classes with no mapping are dropped, so map every class you want written back to your project.
{% endhint %}

When the mapping looks right, click "Auto Label With This Workflow" to start the job. It runs in the background and applies the mapped labels to your batch.

### Run Auto Label On Your Batch

If Auto Label labels your images as you expect, click "Auto Label with This Model". Review the summary modal, and click to proceed. It should take a few minutes to label a thousand images.

Once the job starts, you are taken to the Annotate page, where the new Auto Label job is listed alongside your other annotation jobs. Auto Label runs in the background, so you can review results there when it finishes.

## Existing Annotations

If images in your batch are already annotated, the summary modal asks how Auto Label should handle them. The job does not start until you choose one option:

* "Keep Existing Annotations" adds the Auto Label predictions next to the annotations already on each image.
* "Replace Existing Annotations" deletes the existing annotations and saves only the Auto Label predictions.

## HTTP API

You can programmatically start and track auto-labeling jobs using the REST API. Auto Label uses foundation models (SAM 3) or your own trained Roboflow models to automatically annotate images in a batch.

These endpoints require [scoped API keys](https://docs.roboflow.com/reference/authentication/authentication/scoped-api-keys) with the `annotationJob.create` and `annotationJob.read` scopes respectively.

### Create an Auto Label Job

To start an auto-label job, make a POST request:

```bash
curl --location --request POST 'https://api.roboflow.com/${WORKSPACE}/${PROJECT}/autolabel?api_key=${ROBOFLOW_API_KEY}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "batchId": "<BATCH_ID>",
    "modelType": "sam3",
    "ontology": {"a dog": "dog", "a cat": "cat"}
}'
```

You can also omit `ontology` to let the API derive it from your model or dataset classes:

```bash
curl --location --request POST 'https://api.roboflow.com/${WORKSPACE}/${PROJECT}/autolabel?api_key=${ROBOFLOW_API_KEY}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "batchId": "<BATCH_ID>",
    "modelType": "sam3"
}'
```

#### Request Body

| Parameter                     | Type            | Required | Description                                                                                                                                                                                 |
| ----------------------------- | --------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `batchId`                     | string          | Yes      | The ID of the image batch to label.                                                                                                                                                         |
| `modelType`                   | string          | Yes      | Model to use: a catalog id from `GET /:workspace/autolabel/models` (ex: `gpt-6-astra-boxes`, `sam3-rle`, `gemini-boxes`), or `custom_roboflow` with the model id in `modelOptions.modelId`. |
| `ontology`                    | object or array | No       | Maps prompts to class names. See formats below. When omitted, derived automatically (see [Ontology Auto-Derivation](#ontology-auto-derivation)).                                            |
| `numImagesToLabel`            | number          | No       | Number of images to label. Defaults to the full batch.                                                                                                                                      |
| `defaultConfidence`           | number          | No       | Confidence threshold applied to all classes (0-1).                                                                                                                                          |
| `confidenceThresholds`        | object          | No       | Per-class confidence thresholds, e.g. `{"dog": 0.5, "cat": 0.7}`.                                                                                                                           |
| `reviewerEmail`               | string          | No       | Email of a workspace member to assign as reviewer. Defaults to the workspace owner.                                                                                                         |
| `runNMS`                      | boolean         | No       | Whether to run non-max suppression. Defaults to `true`.                                                                                                                                     |
| `modelOptions`                | object          | No       | Additional model-specific options (e.g. `{"modelId": "your-model/1"}` for `custom_roboflow`).                                                                                               |
| `preserveExistingAnnotations` | boolean         | No       | Keep annotations already on the images and add the Auto Label results to them. Defaults to `false`, which replaces them.                                                                    |

#### Ontology Formats

The `ontology` field accepts several formats. All are normalized internally.

**Object (canonical)** - keys are prompts, values are class names:

```json
{"a dog": "dog", "a cat": "cat"}
```

**Array of strings** - each string is used as both prompt and class name:

```json
["dog", "cat"]
```

**Array of objects:**

```json
[{"class": "dog", "prompt": "a dog"}, {"class": "cat", "prompt": "a cat"}]
```

#### Ontology Auto-Derivation

When `ontology` is omitted (or empty), the API will attempt to derive it automatically based on the `modelType`:

| Model Type        | Derivation Source                                                            |
| ----------------- | ---------------------------------------------------------------------------- |
| `custom_roboflow` | Uses the trained classes from the model specified in `modelOptions.modelId`. |
| `sam3`            | Uses the class names defined in the dataset.                                 |

If the dataset has locked annotation classes, the derived ontology is intersected with the allowed classes so no invalid annotations are produced.

The request will fail with a `400` if the ontology cannot be derived, for example when the dataset has no classes or the model classes don't overlap with the dataset's locked classes. Possible `errorType` values:

* `MISSING_MODEL_ID` - `modelOptions.modelId` is required for `custom_roboflow` when ontology is not provided.
* `MODEL_CLASSES_UNAVAILABLE` - The specified model has no classes.
* `NO_MATCHING_CLASSES` - The model's classes don't overlap with the dataset's locked classes.
* `DATASET_HAS_NO_CLASSES` - The dataset has no classes defined (for `sam3`).

#### Response

```json
{
    "jobId": "auto-label-job-uuid",
    "annotationJobId": "annotation-job-uuid",
    "message": "Automatic labeling job created"
}
```

### Get Auto Label Job Status

To check the progress of an auto-label job, make a GET request:

```bash
curl 'https://api.roboflow.com/${WORKSPACE}/autolabel/jobs/${JOB_ID}?api_key=${ROBOFLOW_API_KEY}'
```

Use the `jobId` returned from the create endpoint.

#### Response

```json
{
    "id": "auto-label-job-uuid",
    "status": "pending",
    "modelType": "sam3",
    "projectType": "object-detection",
    "projectId": "project-id",
    "annotationJobId": "annotation-job-uuid",
    "numImages": 100,
    "numImagesSentBack": 0,
    "subjobs": [
        {"id": "subjob-1", "status": "pending"},
        {"id": "subjob-2", "status": "done"}
    ],
    "ontology": {"a dog": "dog"},
    "confidenceThresholds": {"dog": 0.5},
    "startTime": "2026-01-01T00:00:00.000Z"
}
```

## Python SDK

The `roboflow` package labels a batch from Python:

```python
from roboflow import Roboflow

workspace = Roboflow(api_key="YOUR-API-KEY").workspace()
project = workspace.project("my-project")

for model in workspace.autolabel_models()["models"]:
    print(model["id"], model["available"])

job = project.autolabel(
    "<BATCH_ID>",
    model="gpt-6-astra-boxes",
    ontology={"a dog": "dog", "a cat": "cat"},
    reviewer_email="reviewer@example.com",
)

print(project.autolabel_job(job["jobId"])["status"])
```

`ontology` takes the same shapes as the API (see [Ontology Formats](#ontology-formats)) and can be omitted. Pass `preserve_existing_annotations=True` to add labels next to the annotations already on each image instead of replacing them.

`project.autolabel_preview(model, image, ontology=...)` runs one image through a model before you commit a batch to it. `image` accepts an HTTPS URL or a local file path. A preview does not create a job and does not deduct Auto Label credits.

## CLI

The same operations are available from the command line. See [Install and Set Up the CLI](https://docs.roboflow.com/reference/cli/install-and-set-up-the-cli) first.

List the models your Workspace can use:

```bash
roboflow autolabel models
```

Each row carries the `id` you pass to `preview` and `start`, its credits per image, and whether your plan includes it.

Try a model on one image before spending credits on a batch:

```bash
roboflow autolabel preview -p my-project -m sam3-rle \
  --image https://example.com/sample.jpg \
  --ontology '{"a dog": "dog", "a cat": "cat"}'
```

The response summarizes what the model found per class and lists the classes it missed, so you can compare candidates.

Start the job, then track it:

```bash
roboflow autolabel start -p my-project --batch-id <BATCH_ID> -m gpt-6-astra-boxes \
  --ontology '{"a dog": "dog", "a cat": "cat"}' --reviewer reviewer@example.com

roboflow autolabel job <JOB_ID>
```

`start` prints the `jobId` and `annotationJobId`. `job` looks the id up in your default Workspace, so pass the same `-p` when you started the job with a `workspace/project` shorthand.

When a prompt and its class name are the same, repeated `--class` flags are shorter than JSON:

```bash
roboflow autolabel preview -p my-project -m sam3-rle --image sample.jpg --class dog --class cat
```

Options for `start`:

<table data-search="false"><thead><tr><th width="290">Option</th><th>Description</th></tr></thead><tbody><tr><td><code>--batch-id</code></td><td>Batch holding the images to label. Required.</td></tr><tr><td><code>-m</code>, <code>--model</code></td><td>Model id from <code>autolabel models</code>, or a Roboflow model id when you pass <code>--model-type roboflow</code>. Required.</td></tr><tr><td><code>--model-type</code></td><td><code>foundational</code> (default) or <code>roboflow</code> for a model you trained.</td></tr><tr><td><code>--ontology</code>, <code>--class</code></td><td>What to label. Accepts <code>@ontology.json</code> as well as inline JSON.</td></tr><tr><td><code>--num-images</code></td><td>How many images from the batch to label. Defaults to all of them.</td></tr><tr><td><code>--confidence</code></td><td>Confidence threshold applied to every class.</td></tr><tr><td><code>--confidence-thresholds</code></td><td>Per-class thresholds as JSON (ex: <code>'{"dog": 0.5}'</code>).</td></tr><tr><td><code>--no-nms</code></td><td>Turn off non-max suppression.</td></tr><tr><td><code>--reviewer</code></td><td>Email of the reviewer for the resulting annotation job. Must be a Workspace member.</td></tr><tr><td><code>--model-options</code></td><td>Model-specific options as JSON (ex: <code>'{"outputFormat": "polygon"}'</code>).</td></tr><tr><td><code>--preserve-existing</code></td><td>Keep the annotations already on the images and only add new ones.</td></tr></tbody></table>

{% hint style="warning" %}
By default the job replaces the annotations already on the batch images. Pass `--preserve-existing` to keep them.
{% endhint %}

Every command accepts `--json` for structured output.

## MCP Server

Connect your AI agent to the [MCP Server](https://docs.roboflow.com/agents/mcp-server) and it can auto label a batch of images with these tools:

<table data-search="false"><thead><tr><th width="290">Tool</th><th>Description</th></tr></thead><tbody><tr><td><code>autolabel_start</code></td><td>Start a hosted auto label job over a batch of images.</td></tr><tr><td><code>autolabel_job_get</code></td><td>Get per-subjob status and progress for an auto label job.</td></tr></tbody></table>