CLI Usage

Use the Roboflow CLI to create and manage Batch Processing jobs.

By installing inference-cli you gain access to the inference rf-cloud command, which allows you to interact with Batch Processing and Data Staging — the core components of Roboflow Batch Processing.

Setup

pip install inference-cli
export ROBOFLOW_API_KEY="YOUR-API-KEY-GOES-HERE"

For cloud storage support:

pip install 'inference-cli[cloud-storage]'

If you need help finding your API key, see our authentication guide.

Ingest Data

Images

inference rf-cloud data-staging create-batch-of-images \
  --images-dir <your-images-dir-path> \
  --batch-id <your-batch-id>

Videos

inference rf-cloud data-staging create-batch-of-videos \
  --videos-dir <your-videos-dir-path> \
  --batch-id <your-batch-id>

Batch ID format: Must be lowercase, at most 64 characters, with only letters, digits, hyphens (-), and underscores (_).

Cloud Storage

If your data is already in cloud storage (S3, Google Cloud Storage, or Azure), you can process it directly without downloading files locally.

For images:

inference rf-cloud data-staging create-batch-of-images \
  --data-source cloud-storage \
  --bucket-path <cloud-path> \
  --batch-id <your-batch-id>

For videos:

inference rf-cloud data-staging create-batch-of-videos \
  --data-source cloud-storage \
  --bucket-path <cloud-path> \
  --batch-id <your-batch-id>

The --bucket-path parameter supports:

• S3: s3://bucket-name/path/

• Google Cloud Storage: gs://bucket-name/path/

• Azure Blob Storage: az://container-name/path/

You can include glob patterns to filter files:

• s3://my-bucket/training-data/**/*.jpg — All JPG files recursively

• gs://my-bucket/videos/2024-*/*.mp4 — MP4 files in 2024-* folders

• az://container/images/*.png — PNG files in images folder

Your cloud storage credentials are used only locally by the CLI to generate presigned URLs. They are never uploaded to Roboflow servers.

Generated presigned URLs are valid for 24 hours. Ensure your batch processing job completes within this timeframe.

For large datasets, the system automatically splits images into chunks of 20,000 files each. Videos work best in batches under 1,000.

Signed URL Ingestion

For advanced automation, you can ingest data via signed URLs instead of local files:

• --data-source references-file — Process files referenced via signed URLs.

• --references <path_or_url> — Path to a JSONL file containing file URLs, or a signed URL pointing to such a file.

Reference File Format (JSONL):

{"name": "<unique-file-name-1>", "url": "https://<signed-url>"}
{"name": "<unique-file-name-2>", "url": "https://<signed-url>"}

Signed URL ingestion is available to Growth Plan and Enterprise customers.

Inspect Staged Data

inference rf-cloud data-staging show-batch-details --batch-id <your-batch-id>

Start a Job

Process Images

inference rf-cloud batch-processing process-images-with-workflow \
  --workflow-id <workflow-id> \
  --batch-id <batch-id> \
  --machine-type gpu

Process Videos

inference rf-cloud batch-processing process-videos-with-workflow \
  --workflow-id <workflow-id> \
  --batch-id <batch-id> \
  --machine-type gpu \
  --max-video-fps <your-desired-fps>

Finding your Workflow ID: Open the Workflow Editor in the Roboflow App, click "Deploy", and find the identifier in the code snippet.

By default, processing runs on CPU. Use --machine-type gpu for Workflows with multiple or large models.

Monitor Job Progress

The start command outputs a Job ID. Use it to check status:

inference rf-cloud batch-processing show-job-details --job-id <your-job-id>

Export Results

The job details will include the output batch ID. Use it to export results:

inference rf-cloud data-staging export-batch \
  --target-dir <dir-to-export-result> \
  --batch-id <output-batch-of-a-job>

Webhook Automation

Instead of polling for status, you can use webhooks to get notified when ingestion or processing completes.

Data Ingestion Webhooks

The CLI commands create-batch-of-images and create-batch-of-videos support:

• --notifications-url <webhook_url> — Webhook endpoint for notifications.

• --notification-category <value> — Filter notifications:

  • ingest-status (default) — Overall ingestion process status.

  • files-status — Individual file processing status.

Notifications are delivered via HTTP POST with an Authorization header containing your Roboflow Publishable Key.

Ingest Status Notification

{
    "type": "roboflow-data-staging-notification-v1",
    "event_id": "8c20f970-fe10-41e1-9ef2-e057c63c07ff",
    "ingest_id": "8cd48813430f2be70b492db67e07cc86",
    "batch_id": "test-batch-117",
    "shard_id": null,
    "notification": {
        "type": "ingest-status-notification-v1",
        "success": false,
        "error_details": {
            "type": "unsafe-url-detected",
            "reason": "Untrusted domain found: https://example.com/image.png"
        }
    },
    "delivery_attempt": 1
}

File Status Notification

{
    "type": "roboflow-data-staging-notification-v1",
    "event_id": "8f42708b-aeb7-4b73-9d83-cf18518b6d81",
    "ingest_id": "d5cb69aa-b2d1-4202-a1c1-0231f180bda9",
    "batch_id": "prod-batch-1",
    "shard_id": "0d40fa12-349e-439f-83f8-42b9b7987b33",
    "notification": {
        "type": "ingest-files-status-notification-v1",
        "success": true,
        "ingested_files": [
            "000000494869.jpg",
            "000000186042.jpg"
        ],
        "failed_files": [
            {
                "type": "file-size-limit-exceeded",
                "file_name": "big_image.png",
                "reason": "Max size of single image is 20971520B."
            }
        ],
        "content_truncated": false
    },
    "delivery_attempt": 1
}

Job Completion Webhooks

Add --notifications-url when starting a job:

inference rf-cloud batch-processing process-images-with-workflow \
  --workflow-id <workflow-id> \
  --batch-id <batch-id> \
  --notifications-url <webhook_url>

Job Completion Notification

{
  "type": "roboflow-batch-job-notification-v1",
  "event_id": "8f42708b-aeb7-4b73-9d83-cf18518b6d81",
  "job_id": "<your-batch-job-id>",
  "job_state": "success | fail",
  "delivery_attempt": 1
}

Cloud Storage Authentication

AWS S3 and S3-Compatible Storage

Credentials are detected automatically from:

1. Environment variables:

export AWS_ACCESS_KEY_ID=your-access-key-id
export AWS_SECRET_ACCESS_KEY=your-secret-access-key
export AWS_SESSION_TOKEN=your-session-token  # Optional

1. AWS credential files (~/.aws/credentials, ~/.aws/config)

2. IAM roles (EC2, ECS, Lambda)

Named profiles:

export AWS_PROFILE=production

S3-compatible services (Cloudflare R2, MinIO, etc.):

export AWS_ENDPOINT_URL=https://account-id.r2.cloudflarestorage.com
export AWS_REGION=auto  # R2 requires region='auto'
export AWS_ACCESS_KEY_ID=your-r2-access-key
export AWS_SECRET_ACCESS_KEY=your-r2-secret-key

Google Cloud Storage

Credentials are detected from:

1. Service account key file (recommended for automation):

export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account-key.json

1. User credentials from gcloud CLI (gcloud auth login)

2. GCP metadata service (when running on Google Cloud Platform)

Azure Blob Storage

SAS Token (recommended):

export AZURE_STORAGE_ACCOUNT_NAME=mystorageaccount
export AZURE_STORAGE_SAS_TOKEN="sv=2021-06-08&ss=b&srt=sco&sp=rl&se=2024-12-31"

Account Key:

export AZURE_STORAGE_ACCOUNT_NAME=mystorageaccount
export AZURE_STORAGE_ACCOUNT_KEY=your-account-key

Generate a SAS token via Azure CLI:

az storage container generate-sas \
  --account-name mystorageaccount \
  --name my-container \
  --permissions rl \
  --expiry 2024-12-31T23:59:59Z

Custom Scripts

For advanced use cases, reference scripts for generating signed URL files:

• AWS S3: generateS3SignedUrls.sh

• Google Cloud Storage: generateGCSSignedUrls.sh

• Azure Blob Storage: generateAzureSasUrls.sh

Discover All Options

inference rf-cloud --help
inference rf-cloud data-staging --help
inference rf-cloud batch-processing --help