> For the complete documentation index, see [llms.txt](https://docs.roboflow.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.roboflow.com/datasets/annotate/annotate/ai-labeling/auto-label.md). # Auto Label Auto-label images for use in training models. ## About {% hint style="info" %} Auto Label is one of many [AI Labeling](/datasets/annotate/annotate/ai-labeling.md) features. Using this feature will consume [credits](https://docs.roboflow.com/platform/billing-and-plans#credits) at the rates listed on our [credits page](https://roboflow.com/credits). {% endhint %} Roboflow Auto Label lets you use a foundation vision model (Gemini 4 Argon, Gemini 4 Carbon, GPT-6 Astra, GPT-6.1 Sol, GPT-6.1 Terra, SAM 3, Gemini 3.8 Flash, or Claude Opus 5.5), Roboflow trained models, or a saved [Workflow](https://docs.roboflow.com/workflows) to automatically label images. Auto Label works on object detection, instance segmentation, classification, and keypoint detection projects. It does not label semantic segmentation projects. On a keypoint detection project, Auto Label uses a keypoint detection model you trained in Roboflow. Foundation models and Workflows are not available there. Auto Label skips instances of a class with no [skeleton](/datasets/annotate/annotate/edit-keypoint-skeletons.md). The page warns you about these classes. If none of the model's classes has a skeleton, the job does not start, and the API returns a `400` with `errorType` `MISSING_KEYPOINT_SKELETON`. It matches each predicted keypoint to the skeleton keypoint with the same name, so a keypoint you add or rename after training is saved as not visible. Roboflow Auto Label can use the following sources to identify the objects you specify. Each foundation model is listed once, with the label formats it supports on your project type: * Gemini 4 Argon: Boxes on object detection, Masks or Polygons on instance segmentation * Gemini 4 Argon + SAM 3: Gemini 4 Argon finds each instance, then SAM 3 draws its shape inside every box it found. Masks or Polygons (instance segmentation only) * Gemini 4 Carbon: Boxes (object detection only) * Gemini 4 Carbon + SAM 3: Gemini 4 Carbon finds each instance, then SAM 3 draws its shape inside every box it found. Masks or Polygons (instance segmentation only) * GPT-6 Astra: Boxes on object detection, Masks or Polygons on instance segmentation * GPT-6 Astra + SAM 3: GPT-6 Astra finds each instance, then SAM 3 draws its shape inside every box it found. Masks or Polygons (instance segmentation only) * GPT-6.1 Sol: Boxes on object detection. It is close to GPT-6 Astra in quality, with a lower token cost. On classification projects, GPT-6.1 Sol (Classes) and GPT-6.1 Sol (Multi-Label) pick the classes that match each image from your class names or descriptions. Keep each description short, with no commas. * GPT-6.1 Terra: Boxes on object detection. On classification projects, GPT-6.1 Terra (Classes) and GPT-6.1 Terra (Multi-Label) work like the GPT-6.1 Sol options. Its token price is half of GPT-6.1 Sol's. * GPT-6.1 Terra + SAM 3: GPT-6.1 Terra finds each instance, then SAM 3 draws its shape inside every box it found. Masks or Polygons (instance segmentation only) * SAM 3: labels objects from text prompts. Masks or Polygons (object detection and segmentation) * Gemini 3.8 Flash: Boxes (object detection only) * Claude Opus 5.5: Boxes (object detection only). Try it when classes are unclear or scenes are complex. Its token cost is often higher than Gemini. * CLIP (Classes): CLIP picks the class closest to each image (single-label classification only). It is less accurate than GPT-6.1 Sol, but it works on every plan and uses only Auto Label credits. * Models trained in Roboflow ([https://github.com/roboflow/docs/tree/main/models/README.md](https://github.com/roboflow/docs/tree/main/models/README.md "mention")) * Note: Only models from the same dataset as your Annotation Batch are currently supported. * A saved [Workflow](https://docs.roboflow.com/workflows) compatible with your project type When a model supports more than one format, pick one with the control next to the model name (ex: Masks or Polygons). Roboflow remembers your choice for the project. On an object detection project, masks and polygons train as boxes. The summary modal shows the format you picked. The test results show the shapes the model returns. The format you pick applies when the job saves the labels, so switching format does not run the test again. When Gemini 4 Argon or GPT-6 Astra saves polygons, each separate part of an instance becomes its own polygon, and holes are filled. On paid plans, Gemini 4 Carbon is selected for you on object detection projects, and Gemini 4 Carbon + SAM 3 on instance segmentation projects. On free and trial plans, GPT-6.1 Terra and GPT-6.1 Terra + SAM 3 are selected. Classification projects start on GPT-6.1 Terra on every plan. On instance segmentation projects, SAM 3 is selected when Blocks Governance turns off a block the default needs. Gemini 4 Argon handles open vocabulary class names, visible text, and object relationships well. Gemini token use is billed on top of the credits per image, so it costs more per image than SAM 3. GPT-6 Astra, GPT-6 Astra + SAM 3, GPT-6.1 Sol, GPT-6.1 Terra, and GPT-6.1 Terra + SAM 3 bill OpenAI token use on top of the credits. Gemini 4 Argon + SAM 3, Gemini 4 Carbon, and Gemini 4 Carbon + SAM 3 bill Gemini token use on top of the credits. {% hint style="info" %} Core’s paid monthly credit tiers include access to a Roboflow-provided OpenAI key. You can also use your own workspace OpenAI key. {% endhint %} Auto Label has been used to label millions of images for use in training computer vision models. If your workspace uses [Blocks Governance](https://docs.roboflow.com/workflows/blocks/blocks-governance), a model whose blocks are disabled is still listed, greyed out with the reason, and you cannot select it. Ask a Super User to enable the blocks it needs. If you would rather auto-label on your own hardware, Roboflow also maintains [Autodistill](https://github.com/autodistill/autodistill), an open source framework for auto-labeling image datasets with a range of foundation models. ### When to Use Roboflow Auto Label You should use Roboflow Auto Label if you need to annotate common objects such as vehicles (i.e. forklifts), people, generic defects (i.e. cracks), and generic products (i.e. vinyl records, bread). You should not use a foundation model like SAM 3 if you need to identify specific variants of an object. For example, Auto Label cannot distinguish between different types of crack, or identify unique defects in electronics. ## Label Data with Roboflow Auto Label The Roboflow platform lets you preview how Auto Label will perform on labeling classes of data in your dataset before you run a job on your full batch. After labeling, you can review the results for quality assurance (recommended) and train a model on the labeled dataset. ### Step #1: Upload Data First, upload data to Roboflow. See our [Upload Images, Videos, and Annotations](/datasets/create-and-upload/adding-data.md) instructions for more information.

Uploading images into Roboflow.

### Step #2: Enter Auto Label Once you have uploaded all of your images, you will be asked how you want to label your images. Select "Auto-Label And Review".

Select "Auto-Label And Review" to open the Roboflow Auto Label interface.

### Step #3: Configure Auto Label The Auto Label labeling interface will appear in which you can configure your auto labeling job. #### Classes (& Descriptions) Classes represent the labels you want to assign to objects in the image. Descriptions represent a visual description of the class that the foundation model will use to identify instances of those classes. By default, the description will be the class name. GPT-6 Astra, SAM 3, and Gemini all search with the description and save what they find under the class name. For example, the class "umbrella" with the description "green umbrella" finds green umbrellas and labels them "umbrella". Give each class its own description: Auto Label rejects a preview or a job when two classes share one. A model you trained does not use descriptions. It labels the classes it was trained on. Auto Label works best when labeling common objects with clear visual descriptions. For example, Auto Label will be able to identify the location of an aluminum can on a production line. But, Auto Label will be unable to label images according to specific requirements, such as distinguishing the brand of an aluminum can. #### Generating Test Results Auto Label runs a first preview on a few sample images from your batch. To run it again after you change the model or classes, click "Run again".

The Auto Label interface.

### Step #4: Evaluate Roboflow Auto Label Labels Here are the test results when using the class "aluminum can" on an example image. From here, you can:

Auto Label annotating aluminum cans.

#### Adjust your classes & descriptions If Auto Label doesn't label images as expected, try testing different descriptions for your classes. {% hint style="info" %} All test results are free & don't use any credits. {% endhint %} #### Adjust the confidence Each class has a "Confidence" slider. A higher threshold shows fewer labels. {% hint style="info" %} The confidence threshold set here will be the same one used when labeling the entire batch, so make sure it looks right! {% endhint %} All foundation models except SAM 3 return every label at full confidence, so they have no threshold to set. The slider and the confidence values on preview labels are hidden when you select one of them. GPT-6.1 Sol and GPT-6.1 Terra report their own confidence on classification projects. Its threshold starts at 0.9, and the job keeps a label only when its confidence is above the threshold. API jobs with no threshold also use 0.9. CLIP (Classes) scores each class by similarity, not confidence, so it has no threshold to set. On a single-label classification project, every job keeps only the class with the highest score for each image. On a keypoint detection project, the class threshold decides which instances are kept, and a second slider, "Keypoint Threshold", appears after the test results load. It sets the lowest confidence a keypoint needs to be saved as visible. Keypoints below it are saved as not visible. It starts at 60%, and the job uses the same value. #### Give feedback on labels With a foundation model, you can fix the preview without writing a workflow. Click a label and pick "Wrong class", "Shouldn't detect this", or "Keep this detection". To point out an object the model missed, click "Flag missing object", then click the object. Add details if you want, and click "Update preview". The agent adjusts the labeling workflow and runs the preview again. "Label batch" then labels your batch with the adjusted workflow. You need permission to manage Workflows, and the agent must be on for your Workspace. #### Test on different images Click an image in the strip below the preview to see its labels. Click "Change" to pick other images from your batch. ## Label with a Workflow Instead of a foundation or trained model, you can auto-label with a saved [Workflow](https://docs.roboflow.com/workflows), letting you use SAHI, model ensembles, and multi-step custom logic to label your batch. Select a workflow from the Auto Label model dropdown. Only workflows compatible with your project type (detection, segmentation, or classification) are listed. After selecting a workflow, map each of its output classes to a class in your project and set a per-class confidence threshold. A preview runs on sample images so you can confirm the mapping before starting the job. To build a new workflow instead, click "Chat with Auto Label agent". The agent builds a workflow from your model and classes and shows a preview. Ask it for changes, or click a label or "Flag missing object" in the preview to tell it what is wrong. Click "Use this Workflow" to save the workflow and use it for this batch. The chat is saved for you and this batch, so you can leave and come back. Click "Resume Auto Label agent" to continue it, or "New chat" to start again from the current preview. {% hint style="info" %} Only the classes you map are applied. Output classes with no mapping are dropped, so map every class you want written back to your project. {% endhint %} When the mapping looks right, click "Label batch" to start the job. It runs in the background and applies the mapped labels to your batch. ### Run Auto Label On Your Batch If Auto Label labels your images as you expect, click "Label batch". Review the summary modal, and click to proceed. It should take a few minutes to label a thousand images. The modal itemizes the cost. Auto Label charges a set number of credits per image. Gemini 4 Argon, Gemini 4 Carbon, Gemini 3.8 Flash, Claude Opus 5.5, and workflows that call an outside AI model add usage charges for that model on top of the credits. For a Roboflow-provided model, the modal estimates that usage charge from your recent test results with that model. With no recent test results, it uses the model's average cost per image. Your job can cost more or less. If the total estimate is more than your workspace has left to spend, the job does not start. A model that runs on your workspace's own provider key adds no usage charge in credits. Once the job starts, you are taken to the Annotate page, where the new Auto Label job is listed alongside your other annotation jobs. Auto Label runs in the background, so you can review results there when it finishes. ## Existing Annotations If images in your batch are already annotated, the summary modal asks how Auto Label should handle them. The job does not start until you choose one option: * "Keep Existing Annotations" adds the Auto Label predictions next to the annotations already on each image. * "Merge With Existing Annotations" keeps the existing annotations and adds only the predictions that do not overlap an existing annotation of the same class (IoU of at least 0.5). Use it when some objects are already labeled, so Auto Label does not label them twice. * "Replace Existing Annotations" deletes the existing annotations and saves only the Auto Label predictions. The Python SDK, the CLI and the MCP Server call these modes `preserve`, `merge` and `replace`, and use `replace` when you don't choose one. ## Auto Label Action Recognition Videos In an Action Recognition Project, Auto Label marks when each action happens in the videos of a batch. 1. Open a video batch and choose Auto Label. 2. Select the actions to label. Add a short description of when each action happens to help the model. 3. Set "Sample FPS" (0.5 to 12, default 8). A higher rate catches short actions but costs more. 4. Optional: test your actions on part of one video. Drag the highlighted range on the timeline to pick up to 30 seconds, then click "Run Preview". The segments it finds show in the timeline. The preview costs credits and saves nothing. 5. Click "Annotate All Videos", check the cost, and click "Start Auto Label". Auto Label charges credits for each frame the model sees, plus OpenAI token use. The job replaces the segments already in these videos. You review every segment before it is approved. Each video can be up to 30 minutes long. ## Auto Label From the Agent Ask [Roboflow Agent](https://docs.roboflow.com/agents/roboflow-agent) to label a batch for you. It can use Gemini 4 Argon, Gemini 4 Carbon, GPT-6 Astra, GPT-6.1 Sol, GPT-6.1 Terra, Gemini 3.8 Flash, or Claude Opus 5.5 on object detection projects, SAM 3, a model you trained, or a saved Workflow. When you do not name a model, it uses the same default as the Auto Label page. Your saved class descriptions become the prompts, and classes with no description use the class name. When you ask the Agent for a new class, say what it looks like (ex: a worker in a yellow hard hat). The Agent saves that text as the class description and uses it as the prompt. The Agent labels one image first so you can see what the model finds. It then tells you the credit cost and asks once before it starts the job. The job runs as a background task, and the Agent opens it for review when it finishes. You can also ask the Agent to approve or reject the images in Review. It shows the jobs and image count and waits for you to confirm. Approved images do not go into the dataset until you ask for that as a separate step. ## HTTP API Start and track Auto Label jobs from your own code. A typical run lists the models, previews one or two on a sample image, starts a job on a batch, and polls the job until it is done. These endpoints take a workspace API key with the `annotation-job:create` or `annotation-job:read` scope. They also take an [OAuth access token](https://docs.roboflow.com/reference/authentication/authentication/sign-in-with-roboflow-getting-started), which is how the MCP Server starts a job. A job started with a token runs on the project's billing API key, so it keeps labeling after the token expires. If the app's access is limited to project folders, it can start jobs with foundation models only. Start jobs that use a custom model or a saved Workflow with a workspace API key. ### List Auto Label Models {% openapi src="/files/VivGme2ci6MNqGnErRTb" path="/{workspace}/autolabel/models" method="get" %} [auto-label.yaml](https://2252499186-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNVfRxNsvCh4GOyqrjmzX%2Fuploads%2Fgit-blob-965ca43ed3a8b7f4386dbaaa9ad5b1c9b2a8d47c%2Fauto-label.yaml?alt=media) {% endopenapi %} ### Preview a Model Use a preview to compare models before you start a paid job. {% openapi src="/files/VivGme2ci6MNqGnErRTb" path="/{workspace}/{project}/autolabel/preview" method="post" %} [auto-label.yaml](https://2252499186-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNVfRxNsvCh4GOyqrjmzX%2Fuploads%2Fgit-blob-965ca43ed3a8b7f4386dbaaa9ad5b1c9b2a8d47c%2Fauto-label.yaml?alt=media) {% endopenapi %} ### Create an Auto Label Job The labeled images land in a new annotation job, assigned to `reviewerEmail` for review. When you leave out `ontology`, the job uses your project's classes as prompts. With `custom_roboflow`, it uses the classes the model was trained on, limited to the project's classes if they are locked. On a semantic segmentation project, the request fails with `UNSUPPORTED_PROJECT_TYPE`. On a keypoint detection project, only `custom_roboflow` with a keypoint detection model works. {% openapi src="/files/VivGme2ci6MNqGnErRTb" path="/{workspace}/{project}/autolabel" method="post" %} [auto-label.yaml](https://2252499186-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNVfRxNsvCh4GOyqrjmzX%2Fuploads%2Fgit-blob-965ca43ed3a8b7f4386dbaaa9ad5b1c9b2a8d47c%2Fauto-label.yaml?alt=media) {% endopenapi %} ### Get Auto Label Job Status Poll this endpoint with the `jobId` from the create call until `status` is `done`. A thousand images usually take a few minutes. {% openapi src="/files/VivGme2ci6MNqGnErRTb" path="/{workspace}/autolabel/jobs/{jobId}" method="get" %} [auto-label.yaml](https://2252499186-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNVfRxNsvCh4GOyqrjmzX%2Fuploads%2Fgit-blob-965ca43ed3a8b7f4386dbaaa9ad5b1c9b2a8d47c%2Fauto-label.yaml?alt=media) {% endopenapi %} ## Python SDK The `roboflow` package labels a batch from Python: ```python from roboflow import Roboflow workspace = Roboflow(api_key="YOUR-API-KEY").workspace() project = workspace.project("my-project") for model in workspace.autolabel_models()["models"]: print(model["id"], model["available"]) job = project.autolabel( "", model="gpt-6-astra-boxes", ontology={"a dog": "dog", "a cat": "cat"}, reviewer_email="reviewer@example.com", ) print(project.autolabel_job(job["jobId"])["status"]) ``` `ontology` takes the same shapes as the API (see [Create an Auto Label Job](#create-an-auto-label-job)) and can be omitted. `existing_annotations` sets what the job does with annotations already on each image: `"replace"` (the default), `"preserve"` or `"merge"` (see [Existing Annotations](#existing-annotations)). `project.autolabel_preview(model, image, ontology=...)` runs one image through a model before you commit a batch to it. `image` accepts an HTTPS URL or a local file path. A preview does not create a job and does not deduct Auto Label credits. ## CLI The same operations are available from the command line. See [Install and Set Up the CLI](https://docs.roboflow.com/reference/platform/cli/install-and-set-up-the-cli) first. List the models your Workspace can use: ```bash roboflow autolabel models ``` Each row carries the `id` you pass to `preview` and `start`, its credits per image, and whether your plan includes it. Try a model on one image before spending credits on a batch: ```bash roboflow autolabel preview -p my-project -m sam3-rle \ --image https://example.com/sample.jpg \ --ontology '{"a dog": "dog", "a cat": "cat"}' ``` The response summarizes what the model found per class and lists the classes it missed, so you can compare candidates. Start the job, then track it: ```bash roboflow autolabel start -p my-project --batch-id -m gpt-6-astra-boxes \ --ontology '{"a dog": "dog", "a cat": "cat"}' --reviewer reviewer@example.com roboflow autolabel job ``` `start` prints the `jobId` and `annotationJobId`. `job` looks the id up in your default Workspace, so pass the same `-p` when you started the job with a `workspace/project` shorthand. When a prompt and its class name are the same, repeated `--class` flags are shorter than JSON: ```bash roboflow autolabel preview -p my-project -m sam3-rle --image sample.jpg --class dog --class cat ``` Options for `start`:
OptionDescription
--batch-idBatch holding the images to label. Required.
-m, --modelModel id from autolabel models, or a Roboflow model id when you pass --model-type roboflow. Required.
--model-typefoundational (default) or roboflow for a model you trained.
--ontology, --classWhat to label. Accepts @ontology.json as well as inline JSON.
--num-imagesHow many images from the batch to label. Defaults to all of them.
--confidenceConfidence threshold applied to every class.
--confidence-thresholdsPer-class thresholds as JSON (ex: '{"dog": 0.5}').
--no-nmsTurn off non-max suppression.
--reviewerEmail of the reviewer for the resulting annotation job. Must be a Workspace member.
--model-optionsModel-specific options as JSON (ex: '{"outputFormat": "polygon"}').
--existing-annotationsWhat to do with annotations already on the images: replace (default), preserve or merge. See Existing Annotations.
{% hint style="warning" %} By default the job replaces the annotations already on the batch images. Pass `--existing-annotations preserve` or `--existing-annotations merge` to keep them. {% endhint %} Every command accepts `--json` for structured output. ## MCP Server Connect your AI agent to the [MCP Server](https://docs.roboflow.com/agents/mcp-server) and it can auto label a batch of images with these tools:
ToolDescription
autolabel_startStart a hosted auto label job over a batch of images.
autolabel_job_getGet per-subjob status and progress for an auto label job.
`autolabel_start` replaces the annotations already on the batch images unless the agent sets `existing_annotations` to `preserve` or `merge` (see [Existing Annotations](#existing-annotations)).