> For the complete documentation index, see [llms.txt](https://docs.roboflow.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.roboflow.com/deployment/self-hosted/self-hosted.md). # Self-Hosted Deployment [Inference](https://github.com/roboflow/inference) is an open source computer vision deployment hub. It serves models and Workflows, manages video streams, and optimizes inference for CPUs and GPUs. Self-host it when you need local processing, control over latency and resources, or offline deployment. The Apache 2.0 licensed core also powers Roboflow's hosted APIs. {% hint style="info" %} Self-hosting means you manage the infrastructure. If you would rather Roboflow run the servers, see [Dedicated Deployments](/deployment/roboflow-cloud/dedicated-deployments.md) or the [Serverless Cloud API](/deployment/roboflow-cloud/serverless-api.md), and the full [comparison of options](/deployment/choosing-a-deployment.md). {% endhint %} ## Pick a path There are three ways to run models on your own hardware. Most projects use the Inference Server.
Cover image
Inference ServerA Docker container that serves models and Workflows over HTTP./files/gPh1WrManZHUqduvUkSB/pages/poyMxVV4pCc72cNp4VZ3
Inference LibraryThe inference Python package for running models in your process./files/rNJ3e52thrZfeIVJDxlK/pages/3rftTy0RN7BsAWZkpsQf
Other SDKsRun models in a web browser, on iOS, or on embedded devices./files/ClRvbuvHypC9cxfNfisG/pages/fOp4F0aWgXlS6xZfcslE
Use the server when more than one client or language needs predictions, when you want models isolated from your application dependencies, or when you deploy to edge devices.
Roboflow Inference architecture diagram

Where Inference sits between your application, your models, and the Roboflow platform

## Run model locally For most projects, run Inference Server in Docker and send requests with `inference-sdk`. The SDK is a Python HTTP client that connects your application to an Inference Server. Use Inference Library when you need to load and run models directly inside your Python process. Both paths accept the same `model_id` values, so you can switch between them later. ### Model IDs The `model_id` parameter can be: * A [pre-trained model alias](/models/pretrained-aliases.md), for example `rfdetr-small` or `rfdetr-large` * Your own [fine-tuned model from Roboflow](https://app.gitbook.com/s/wdr4k0gUcsVnXVoafYcQ/train/model-ids), for example `my-project/1` * A [Universe model](https://docs.roboflow.com/datasets/universe/universe/find-a-model-on-universe), for example `soccer-players-xy9vk/2` Fine-tuned models and Universe models require an [API key](https://docs.roboflow.com/reference/authentication/authentication/find-your-roboflow-api-key). {% tabs %} {% tab title="Inference Server" icon="docker" %} ### Install Start the server with the [Inference CLI](https://docs.roboflow.com/reference/inference/inference-cli). It detects your hardware and pulls the right Docker image with secure defaults: ```bash pip install inference-cli && inference server start ``` Then install the HTTP client: ```bash pip install inference-sdk ``` For hardware requirements, per-device guides, and manual `docker run` commands, see [Install Inference Server](/deployment/self-hosted/inference-server/install.md). The same client also works against the [Serverless Cloud API](/deployment/roboflow-cloud/serverless-api.md) and [Dedicated Deployments](/deployment/roboflow-cloud/dedicated-deployments.md): only `api_url` changes. ### Run inference ```python from inference_sdk import InferenceHTTPClient, InferenceConfiguration image = "https://media.roboflow.com/inference/people-walking.jpg" client = InferenceHTTPClient( api_url="http://localhost:9001", # your self-hosted server api_key="YOUR_API_KEY", ).configure(InferenceConfiguration(api_key_transport="header")) results = client.infer(image, model_id="rfdetr-small") ``` The `api_key_transport="header"` setting sends the key only as an `Authorization: Bearer` header, keeping it out of URLs and logs. It requires an inference server on release 1.5.0 or newer; use `api_key_transport="both"` while you still call older servers. See [API key transport](https://docs.roboflow.com/reference/inference/inference-sdk/configuration#api-key-transport). Swap `api_url` for `https://serverless.roboflow.com` to use the [Serverless Cloud API](/deployment/roboflow-cloud/serverless-api.md) instead, with no other code changes. See the [Inference SDK reference](https://docs.roboflow.com/reference/inference/inference-sdk) for details. ### Visualize results Install [Supervision](https://supervision.roboflow.com): ```bash pip install -U supervision ``` ```python import supervision as sv from inference_sdk import InferenceHTTPClient, InferenceConfiguration image = sv.load_image_from_url("https://media.roboflow.com/inference/people-walking.jpg") client = InferenceHTTPClient( api_url="http://localhost:9001", api_key="YOUR_API_KEY", ).configure(InferenceConfiguration(api_key_transport="header")) results = client.infer(image, model_id="rfdetr-medium") detections = sv.Detections.from_inference(results) annotated_image = sv.BoxAnnotator().annotate(scene=image, detections=detections) annotated_image = sv.LabelAnnotator().annotate(scene=annotated_image, detections=detections) sv.plot_image(annotated_image) ``` {% endtab %} {% tab title="Inference Library" icon="python" %} ### Install Install the `inference` package into your own Python environment: ```bash pip install inference ``` If you have an NVIDIA GPU, install `inference-gpu` instead, matching the index URL to the CUDA version installed in your OS: ```bash pip install --extra-index-url https://download.pytorch.org/whl/cu124 inference-gpu ``` See [Inference Library](/deployment/self-hosted/inference-library.md) for backend extras and GPU setup details. ### Run inference ```python from inference import get_model image = "https://media.roboflow.com/inference/people-walking.jpg" model = get_model(model_id="rfdetr-small") results = model.infer(image) ``` `get_model()` downloads and caches the model weights on first use, then runs inference locally. See the [Inference Python Package reference](https://docs.roboflow.com/reference/inference/inference-python) for details. ### Visualize results Install [Supervision](https://supervision.roboflow.com): ```bash pip install -U supervision ``` ```python import supervision as sv from inference import get_model image = sv.load_image_from_url("https://media.roboflow.com/inference/people-walking.jpg") model = get_model(model_id="rfdetr-medium") results = model.infer(image)[0] detections = sv.Detections.from_inference(results) annotated_image = sv.BoxAnnotator().annotate(scene=image, detections=detections) annotated_image = sv.LabelAnnotator().annotate(scene=annotated_image, detections=detections) sv.plot_image(annotated_image) ``` {% endtab %} {% endtabs %} ![People walking, annotated with detections](https://storage.googleapis.com/com-roboflow-marketing/inference/people-walking-annotated.jpg) {% hint style="warning" %} Be careful not to expose your API key to external users. Do not embed it in a public-facing frontend app; proxy the request through your own backend instead. {% endhint %} You can run a [Workflow](https://docs.roboflow.com/workflows) the same way, on the server or in your own process: see [Deploy a Workflow](https://docs.roboflow.com/workflows/deploy/deploy-a-workflow). {% hint style="info" %} TensorRT-optimized model packages for private models are only available on [Enterprise plans](/deployment/self-hosted/enterprise.md) when running Inference outside the Roboflow platform. Public models include TensorRT packages on all plans. {% endhint %}