> For the complete documentation index, see [llms.txt](https://docs.roboflow.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.roboflow.com/deployment/self-hosted/inference-server.md).

# Inference Server

The Inference Server is a standalone microservice that wraps the [`inference` Python package](/deployment/self-hosted/inference-library.md). It exposes HTTP endpoints for images and a WebRTC endpoint for video streams. One server can serve multiple clients and run the same models and [Workflows](https://docs.roboflow.com/workflows) as Roboflow's hosted APIs. It is the recommended way to self-host: see [Pick a path](/deployment/self-hosted/self-hosted.md#pick-a-path) for how it compares to running the library directly.

## Where it runs

Self-host the server on your own hardware (Raspberry Pi, NVIDIA GPU, NVIDIA Jetson, or a plain server) with [Docker](/deployment/self-hosted/inference-server/install.md), or in [your own AWS, GCP, or Azure account](/deployment/self-hosted/inference-server/install/cloud.md). Roboflow also runs the same server for you as the [Serverless Cloud API](/deployment/roboflow-cloud/serverless-api.md) and [Dedicated Deployments](/deployment/roboflow-cloud/dedicated-deployments.md): see [Choosing a Deployment Option](/deployment/choosing-a-deployment.md).

Whichever you pick, you talk to it through the [Inference SDK](https://docs.roboflow.com/reference/inference/inference-sdk), because they share one interface: only the `api_url` changes.

## Running with Docker

Before you begin, make sure [Docker is installed](https://www.docker.com/get-started) on your machine. The easiest way to start the Inference Server is with the [Inference CLI](https://docs.roboflow.com/reference/inference/inference-cli):

```bash
pip install inference-cli && inference server start
```

This pulls the appropriate Docker image for your machine, with dependencies pre-installed, and starts the Inference Server on port 9001. Check the server status with:

```bash
inference server status
```

## Manually setting up a Docker container

`inference server start` runs `docker run` under the hood with recommended security settings, caching, and platform-specific options.

If you want to start the container yourself, see the "Manually starting the container" section of your platform's install guide:

* [Linux](/deployment/self-hosted/inference-server/install/linux.md#manually-starting-the-container)
* [Windows](/deployment/self-hosted/inference-server/install/windows.md#manually-starting-the-container)
* [Mac](/deployment/self-hosted/inference-server/install/mac.md#using-docker)
* [Jetson](/deployment/self-hosted/inference-server/install/jetson.md#manually-starting-the-container)
* [Raspberry Pi](/deployment/self-hosted/inference-server/install/raspberry-pi.md#manually-starting-the-container)

Container settings are controlled with environment variables: see [Docker configuration options](/deployment/self-hosted/inference-server/configuration/docker-configuration.md) and the full [environment variable reference](/deployment/self-hosted/inference-server/configuration/environment-variables.md).

## Built-in JupyterLab notebook

Inference Servers ship with a built-in JupyterLab environment, which is the fastest way to experiment during development and testing. It is disabled by default, so start the server with the `--dev` flag to enable it:

```bash
pip install inference-cli
inference server start --dev
```

Then open `http://localhost:9001` in your browser to see the Inference landing page, which links to resources, examples, and the built-in JupyterLab environment. Select "Jump Into an Inference Enabled Notebook" to open JupyterLab in a new tab. It comes preloaded with example notebooks and all the dependencies needed to run Inference.

{% hint style="warning" %}
The `--dev` notebook environment is meant for local development. Do not enable it on a server that is reachable from an untrusted network: see [Securing a Self-Hosted Server](/deployment/self-hosted/inference-server/configuration/security.md).
{% endhint %}

## Stream video

Use the Inference SDK WebRTC client to stream webcams, camera feeds, and video files through a model or Workflow:

```bash
pip install "inference-sdk[webrtc]"
```

Set `api_url="http://localhost:9001"` when you create `InferenceHTTPClient`. See [WebRTC Streaming](https://docs.roboflow.com/reference/inference/inference-sdk/webrtc) for model and Workflow examples, or follow [Video processing with Workflows](https://docs.roboflow.com/workflows/deploy/video-processing) for a task-based guide.

## In this section

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Install Inference Server</strong></td><td>Requirements, per-device install guides, your own cloud, and updating.</td><td><a href="/pages/WvXuFe7ZVKLVB7sSZreN">/pages/WvXuFe7ZVKLVB7sSZreN</a></td></tr><tr><td><strong>Run a Model</strong></td><td>Your first request over HTTP, model IDs, and visualization.</td><td><a href="/pages/e2cNydVwX3NG0MigtWxu">/pages/e2cNydVwX3NG0MigtWxu</a></td></tr><tr><td><strong>Configuration</strong></td><td>Container options, environment variables, security, HTTPS, and telemetry.</td><td><a href="/pages/Gqc7gRKHZhmCcOP7FdD4">/pages/Gqc7gRKHZhmCcOP7FdD4</a></td></tr><tr><td><strong>Architecture</strong></td><td>How requests, video, and Workflows flow through the server.</td><td><a href="/pages/uqpMWzrtQDt8VWlo9dyh">/pages/uqpMWzrtQDt8VWlo9dyh</a></td></tr></tbody></table>