> For the complete documentation index, see [llms.txt](https://docs.roboflow.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.roboflow.com/deployment/self-hosted/inference-server/install.md). # Install Inference Server Pick the installation method that matches your platform. All paths start the server on port **9001**. {% tabs %} {% tab title="Docker (recommended)" %} Docker is the preferred way to run Inference (see [why Docker](/deployment/self-hosted/inference-server/architecture.md#why-docker)). It works on Linux, macOS, Windows, Jetson, and other Docker-capable devices. [Install Docker](https://docs.docker.com/engine/install/) first (plus the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) if you have a CUDA-enabled GPU), then install and run the [Inference CLI](https://docs.roboflow.com/reference/inference/inference-cli): ```bash pip install inference-cli && inference server start ``` This automatically chooses and configures the optimal container for your machine. {% endtab %} {% tab title="Windows (native app)" %} Run an Inference Server on Windows with the native desktop app, with no Docker required. 1. [Download the latest installer](https://github.com/roboflow/inference/releases) and run it. 2. When the install finishes, it offers to launch the Inference Server. 3. To stop the server, close the terminal window it opens. 4. To start it again later, find **Roboflow Inference** in your Start Menu. See [Install on Windows](/deployment/self-hosted/inference-server/install/windows.md) for details and the Docker alternative. {% endtab %} {% tab title="macOS (native app)" %} Run an Inference Server on an Apple Silicon Mac with the native desktop app, with no Docker required. 1. [Download the DMG](https://github.com/roboflow/inference/releases) and open it. 2. Drag the Roboflow Inference app to your Applications folder. 3. Double-click the app in Applications to start the server. See [Install on Mac](/deployment/self-hosted/inference-server/install/mac.md) for details, the Docker alternative, and MPS acceleration. {% endtab %} {% endtabs %} ## Requirements Inference adapts to your machine and runs faster on more powerful hardware. The floor is a 64-bit processor, 4 GB of RAM, and 20 GB of free disk space. [Docker](https://www.docker.com/get-started) is required for the container paths above. | Target | Hardware | OS | Docker image | | ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------- | ------------------------------------------------------------------------------- | | **CPU** | 64-bit CPU, 4 GB RAM, 20 GB free disk. Heavy models (e.g. SAM2) may be too slow to be practical. | Linux, macOS, or Windows 10/11 with WSL 2 | `roboflow/roboflow-inference-server-cpu` | | **GPU** | CUDA-capable NVIDIA GPU with the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) installed. Recommended for larger models and live video. | Linux (or Windows 10/11 with WSL 2) | `roboflow/roboflow-inference-server-gpu` | | **NVIDIA Jetson** | Jetson Orin device (Orin NX 16 GB or above recommended), running JetPack 4.5, 4.6, 5.x, or 6.x. Allow \~10 GB free disk for the image. | JetPack / L4T | `roboflow/roboflow-inference-server-jetson-*` (JetPack-specific, auto-selected) | See [Minimum Requirements](/deployment/self-hosted/inference-server/install/minimum-requirements.md) for the full list of supported and suggested devices. ## Device-specific guides Special installation notes and performance tips by device: * [Linux](/deployment/self-hosted/inference-server/install/linux.md) * [Windows](/deployment/self-hosted/inference-server/install/windows.md) * [Mac](/deployment/self-hosted/inference-server/install/mac.md) * [NVIDIA Jetson](/deployment/self-hosted/inference-server/install/jetson.md) * [Raspberry Pi](/deployment/self-hosted/inference-server/install/raspberry-pi.md) * [Other devices](/deployment/self-hosted/inference-server/install/other.md) * [Deploy in your own cloud](/deployment/self-hosted/inference-server/install/cloud.md) - AWS, Azure, or GCP If you cannot run Docker at all, the [Inference Library](/deployment/self-hosted/inference-library.md) runs models in your own Python process instead of a server. ## Running the container yourself You do not usually pick the image by hand: `inference server start` detects your hardware and runs `docker run` for you with recommended security settings, caching, and platform-specific options. If you would rather manage the container yourself, use the CPU image on a CPU-only host, or the GPU image with `--gpus all` on a CUDA host. {% tabs %} {% tab title="CPU" %} ```bash sudo docker run -d \ --name inference-server \ --read-only \ -p 9001:9001 \ --volume ~/.inference/cache:/tmp:rw \ --security-opt="no-new-privileges" \ --cap-drop="ALL" \ --cap-add="NET_BIND_SERVICE" \ roboflow/roboflow-inference-server-cpu:latest ``` {% endtab %} {% tab title="GPU" %} Install the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) first, then add `--gpus all`: ```bash sudo docker run -d \ --name inference-server \ --gpus all \ --read-only \ -p 9001:9001 \ --volume ~/.inference/cache:/tmp:rw \ --security-opt="no-new-privileges" \ --cap-drop="ALL" \ --cap-add="NET_BIND_SERVICE" \ roboflow/roboflow-inference-server-gpu:latest ``` {% endtab %} {% endtabs %} Your platform's guide has a "Manually starting the container" section with the exact flags for that device. ## Updating Docker images default to the `:latest` tag. To move to the newest server, pull the latest image, or re-run `inference server start`, which pulls it for you: ```bash docker pull roboflow/roboflow-inference-server-gpu:latest ``` For reproducible deployments, **pin a specific version tag** instead of `:latest` so an update never changes behavior unexpectedly, for example `roboflow/roboflow-inference-server-gpu:`. Browse available tags on [Docker Hub](https://hub.docker.com/u/roboflow), and update deliberately by bumping the pinned tag. ## Securing your server A self-hosted server does not enforce authentication, encryption, or network restrictions by default, so securing it is your responsibility. Before exposing it beyond local development traffic, review [Securing a Self-Hosted Server](/deployment/self-hosted/inference-server/configuration/security.md). ## Using your new server Once the server is running, call it over [its HTTP API](/deployment/self-hosted/self-hosted.md#run-a-model) or with the [Inference SDK](https://docs.roboflow.com/reference/inference/inference-sdk). See [Run a model](/deployment/self-hosted/self-hosted.md#run-a-model) for the first request, and [Docker configuration options](/deployment/self-hosted/inference-server/configuration/docker-configuration.md) for tuning the container. ## Enterprise considerations [A Helm chart](https://github.com/roboflow/inference/tree/main/inference/enterprise/helm-chart) is available for enterprise cloud deployments, and enterprise networking solutions that support deployment in OT networks are available on request. Roboflow also offers customized support and installation packages and [a pre-configured Jetson-based edge device](https://roboflow.com/hardware) suitable for rapid prototyping. [Contact the sales team](https://roboflow.com/sales) if you are part of a large organization and want to learn more. See [Enterprise Deployment](/deployment/self-hosted/enterprise.md) for the full feature set.