For the complete documentation index, see llms.txt. This page is also available as Markdown.

Choosing a Deployment Option

Compare Roboflow's deployment options: Serverless Cloud API, Dedicated Deployments, Cloud Streaming, Batch Processing, Self-Hosted Inference, and edge devices. Pick the right option for your use case.

Roboflow offers several ways to run your models and Workflows, from fully managed cloud APIs to inference on your own hardware. The right choice depends on whether your workload is real-time or bulk, how much latency you can tolerate, where your data is allowed to live, and how much infrastructure you want to manage.

Use the tables to compare options at a glance, then follow the decision flow below to narrow down.

Roboflow Cloud

Roboflow runs the infrastructure. Choose Cloud Streaming for managed live-camera Workflows, a request API for individual images, or Batch Processing for stored data.

Managed live video

Cloud Streaming keeps source setup separate from the Workflow job that processes it. Use its quickstart to connect a camera, preview results, deploy processing, and stop it explicitly.

Workload
Choose
Lifecycle and results

Continuous live camera

A preview or deployed Workflow job, with live video, latest JSON in the app, or Workflow sinks. A deployment is not stopped by closing its viewer.

Stored images or videos

An asynchronous job that finishes and produces results. A looping live source is not a finite batch job.

Individual image or Workflow request

A response to each request, rather than a managed camera deployment.

See Cloud Streaming pricing and limits for billing and availability. The deprecated Serverless Video Streaming API has a separate legacy contract; its plan names, regions, and concurrency limit do not describe Cloud Streaming.

Request APIs and batch processing

Where it runs

Roboflow cloud (serverless.roboflow.com)

Roboflow-managed private cloud servers (*.roboflow.cloud)

Roboflow cloud, with infrastructure auto-provisioned per job

GPU

Yes: GPU-accelerated and auto-scaling

CPU (dev-cpu / prod-cpu) or GPU (dev-gpu / prod-gpu)

CPU or GPU, selected per job

Latency profile

Real-time, synchronous. The first request to a model that isn't already loaded incurs a warmup of several seconds, then later requests are fast while the model stays cached.

Consistent, dedicated performance. Deployments auto-pause after a period of inactivity (fixed at 1 hour for dev-cpu / dev-gpu) and resume when you send a request.

Asynchronous, not suitable for real-time. Machines are provisioned when resources are available, typically after a few minutes, with no guaranteed exact start time.

Pricing

Metered per image in credits, at a rate set by the model. See Model pricing.

Pay-per-hour, billed in 1-minute intervals: GPU 1 credit/hour, CPU 0.25 credit/hour. Request-based billing available on request.

CPU and GPU rates differ; GPU jobs are faster but more expensive. See the pricing page.

Region

Contact sales for API region availability.

US-based data centers only.

Plan availability

Core and Enterprise. See pricing.

Workspaces on Roboflow's credit system or Enterprise plans.

Best for

Getting started quickly and scaling automatically for real-time single-image or Workflow inference, when the deployment device has a persistent internet connection.

Predictable or production workloads, resource isolation, and large models that need GPU acceleration (ex: Florence-2, SAM2).

Cost-effective processing of large volumes of stored images and videos with a Workflow.

Self-Hosted

You run inference on your own hardware, so data never has to leave your network.

Where it runs

Your own hardware: edge device, on-prem server, or your own cloud (Docker or pip)

Your edge devices, remotely managed by Roboflow

GPU

CPU, CUDA GPU, or NVIDIA Jetson

Depends on the device (NVIDIA Jetson, x86 with NVIDIA GPU, or Roboflow-supplied hardware)

Latency profile

Local, low latency with no network round-trip; real-time capable on adequate hardware.

Local edge, real-time. Devices need continuous internet access for remote management and monitoring.

Pricing

Free: images and video you run on your own hardware do not use credits. You pay for the machine. TensorRT packages for private models require Enterprise.

Region

Anywhere (your infrastructure).

Your device locations.

Plan availability

All plans. A commercial license to self-host AGPL-3.0 YOLO models and PML 1.0 RF-DETR models (XL and 2XL) is an Enterprise add-on. Apache 2.0 RF-DETR models need no extra license. See Model Licenses. TensorRT packages for private models require Enterprise.

Enterprise only.

Best for

On-prem, edge, air-gapped, or VPC-bound workloads where you want maximum control over environment and latency.

Setting up, deploying, and monitoring a fleet of edge devices at scale.

Capabilities at a glance

This table compares Inference runtimes. For managed live-camera sources and jobs, use the Cloud Streaming guidance above.

Serverless
Dedicated
Self-Hosted

Fine-tuned and pre-trained models

Yes

Yes

Yes

Workflows

Yes

Yes

Yes

Heavy foundation models (SAM2, Florence-2, PaliGemma)

No

Yes

Yes

Cloud-hosted VLMs (GPT, Claude, Gemini blocks)

Yes

Yes

Yes

Video streaming

Use Cloud Streaming for managed live cameras

WebRTC

WebRTC

Custom Python blocks and extra dependencies

No

Yes

Yes

Runs offline

No

No

Yes

Billing

Per image, by model. See Model pricing.

Hourly

Free. Inference does not use credits.

Scale-up behavior differs too: the Serverless Cloud API scales to zero when idle and back up under load with a cold start of a few seconds, while a Dedicated Deployment takes a minute or two to start. Ephemeral dev-cpu and dev-gpu deployments are limited to short sessions and may be evicted when capacity is needed for higher-priority work; the persistent prod-* types have guaranteed capacity and no session limit.

Bring your own cloud

If compliance policies require workloads to stay inside your own infrastructure, you can run Inference on your own AWS, Azure, or GCP account: see Deploy in Your Own Cloud. Inference does not use credits, and you pay your cloud provider for the machine.

Decision flow

  1. Real-time or bulk? If you need results as data arrives - live video, interactive apps, per-request inference - take the real-time path. If you're processing a large backlog of stored images or videos, use Batch Processing.

  2. Sync or async? Real-time inference is synchronous: you send a request and get a response. Batch Processing is asynchronous: you submit a job and collect results when it finishes, so don't depend on an exact start time.

  3. Cloud or on-prem? If your data can leave your network and you want zero infrastructure to manage, use a Roboflow-hosted cloud option. If your workload must stay on-prem, on the edge, or air-gapped, run Self-Hosted Inference.

  4. Managed or self-managed?

Once you've picked an option, see the Production Readiness Checklist for error handling, retries, rate limits, and replica sizing.

Inference vs other deployment stacks

Roboflow Inference overlaps with several other tools. This table summarizes when another stack is the better fit, and what you give up by choosing it.

Inference servers

Tool
How it compares
Choose it if

NVIDIA Triton

A powerhouse for ML experts deploying at scale, focused on extremely optimized pipelines on NVIDIA hardware. It trades simplicity and iteration speed for raw speed, and its model ensembles are more rigid than Workflows.

You are an ML expert on a tightly defined project that values speed on NVIDIA GPUs above all else.

Lightning LitServe

Lightweight, customizable, and task-agnostic (NLP, audio, tabular as well as vision), so it is not as feature-rich for computer vision and has no built-in video streaming or model-chaining abstraction.

You are working on general-purpose ML tasks and want a featureful starting point instead of rolling your own server.

TensorFlow Serving

Good if you are deeply invested in TensorFlow across modalities. It can be complex to set up and lacks table-stakes features like pre- and post-processing, which often need custom code.

The TensorFlow ecosystem matters a lot to you and you are willing to do the legwork.

TorchServe

The PyTorch equivalent, optimized for serving PyTorch models across vision, NLP, tabular data, and audio. Designed for large-scale cloud deployments and light on vision-specific features like video streaming.

You want to scale and customize PyTorch model deployment and do not need vision-specific functionality.

FastAPI or Flask

Rolling your own. Inference's own HTTP interface is built on FastAPI, so starting from scratch means reinventing solved problems.

Your main goal is learning the intricacies of building an inference server.

Edge deployment

Tool
How it compares
Choose it if

Edge Impulse

Focused on very low-power edge devices and embedded systems, and uniquely good with microcontrollers. Its TinyML focus makes it less suited to video processing and modern state-of-the-art models, and it has no Workflows equivalent.

You are building an IoT or wearable device that cannot run more powerful models.

NVIDIA DeepStream

NVIDIA's platform for highly optimized video pipelines using TensorRT and CUDA, targeting many of the same problems as Inference. It has a steep learning curve, requires familiarity with NVIDIA tooling, and is not open source.

You are an expert willing to invest heavily in optimizing a single project where throughput is the primary objective.

Last updated

Was this helpful?