Run a Model Locally

Run Roboflow models on your own hardware with a Docker inference server, in five minutes.

Roboflow Inference is the free, open-source engine behind Roboflow's hosted models. You can run the same models on your own computer: start the inference server in a Docker container, then send it images over HTTP or with the Python Inference SDK. Running it yourself keeps your images on your own machine and gives you full control over your setup.

Run a model on your machine

HTTP (Python)

1

Start the inference server

inference server start launches the Inference Server in a Docker container and picks the right version for your hardware: CPU, NVIDIA GPU, or Jetson. Docker must be installed and running.

pip install inference-cli && inference server start

The server listens on http://localhost:9001. For device-specific setup and troubleshooting, see the Inference install guide.

2

Install the dependencies

supervision draws the results and brings in cv2 and numpy:

pip install supervision

3

Run the model

Send the image to your local server and run a ready-made RF-DETR model. Ready-made models like this one don't need an API key:

import base64
import cv2
import numpy as np
import requests
import supervision as sv

content = requests.get("https://media.roboflow.com/quickstart/cars.jpg").content
image = cv2.imdecode(np.frombuffer(content, np.uint8), cv2.IMREAD_COLOR)
image_b64 = base64.b64encode(content).decode("utf-8")

result = requests.post(
    "http://localhost:9001/coco/38",  # coco/38 == rfdetr-nano
    data=image_b64,  # raw base64 in the body
    headers={"Content-Type": "application/x-www-form-urlencoded"},
).json()
detections = sv.Detections.from_inference(result)
print(f"Found {len(detections)} objects")

annotated = sv.BoxAnnotator().annotate(image.copy(), detections)
annotated = sv.LabelAnnotator().annotate(annotated, detections)
cv2.imwrite("output.jpg", annotated)

SDK (Python)

The first request downloads the model, so it can take a few seconds. Later requests are fast.

RF-DETR detections from the local inference server

Process a video

To run a model on a video, you send it to the server and get each frame back with the boxes already drawn, as it plays. If your computer can't keep up, the server skips frames to stay in real time. This uses the Python SDK and a Workflow: a saved setup that runs the model and draws the results for you.

You can also run a model on a video over HTTP by sending one frame at a time, the same way as the image example above (no streaming add-on needed). This is slower: each frame has to be sent, processed, and drawn before the next one starts, so there's a lot of waiting in between. The SDK streams the whole video and works on several frames at once, so it keeps up much better.

1

Install the SDK

Streaming video needs the webrtc add-on:

pip install "inference-sdk[webrtc]"

2

Run the model on a video

import cv2
from inference_sdk import InferenceHTTPClient
from inference_sdk.webrtc import VideoFileSource
import supervision as sv

client = InferenceHTTPClient.init(api_url="http://localhost:9001")

# Download+cache video
source = VideoFileSource("https://media.roboflow.com/quickstart/cars.mp4")
# Uses webrtc for optimized video streaming
session = client.webrtc.stream(
    source=source,
    model_id="rfdetr-nano"
)

# on_frame runs on the main thread, so cv2.imshow is safe
@session.on_frame
def show(frame, data):
    det = sv.Detections.from_inference(data)
    img = sv.BoxAnnotator().annotate(frame, det)
    img = sv.LabelAnnotator().annotate(img, det)
    cv2.imshow("RF-DETR", img)
    if cv2.waitKey(1) == ord("q"):
        session.close()

session.run()
cv2.destroyAllWindows()

Build richer pipelines (tracking, filtering, zones, notifications) visually in the Workflows editor, then run them the same way. For edge and video-stream deployment, see Deploy a Workflow.

Other options

• Prefer a managed endpoint without running hardware? Use a Dedicated Deployment or the Serverless Hosted API.

• Need on-premises, air-gapped, or Kubernetes deployment? See Enterprise Deployment.

• Compare every option in Deploy a Model or Workflow.