> For the complete documentation index, see [llms.txt](https://docs.roboflow.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.roboflow.com/deployment/self-hosted/inference-server/configuration/docker-configuration.md). # Docker Configuration Options Inference servers have a number of configurable parameters that you set with environment variables. To set an environment variable with `docker run`, use the `-e` flag: ```bash docker run -it --rm -e ENV_VAR_NAME=env_var_value -p 9001:9001 --gpus all roboflow/roboflow-inference-server-gpu:latest ``` This page covers the options you are most likely to change. For the complete list, see [Environment Variables](/deployment/self-hosted/inference-server/configuration/environment-variables.md). ## Networking **`HOST`**: string (default `0.0.0.0`). Sets the host address used by HTTP interfaces. **`PORT`**: integer (default `9001`). Sets the port used by HTTP interfaces. **`ALLOW_ORIGINS`**: string (default `*`). Sets the `allow_origins` property on the CORS middleware used with FastAPI for HTTP interfaces. Multiple values can be provided separated by a comma, for example `ALLOW_ORIGINS=orig1.com,orig2.com`. ## Inference behavior **`CLASS_AGNOSTIC_NMS`**: boolean (default `False`). Sets the default non-maximum suppression (NMS) behavior for detection models (object detection, instance segmentation, and similar). When `True`, NMS is class agnostic, so overlapping detections from different classes may be removed based on the IoU threshold. When `False`, only overlapping detections from the same class are considered for removal. **`MAX_CANDIDATES`**: integer (default `3000`). The maximum number of candidates for detection. **`MAX_DETECTIONS`**: integer (default `300`). The maximum number of detections returned by a model. **`FIX_BATCH_SIZE`**: boolean (default `False`). When `True`, the batch size is fixed to the maximum batch size configured for this server. **`MAX_ACTIVE_MODELS`**: integer (default `8`). The maximum number of models the internal model manager keeps in memory at one time. By default the model queue removes the least recently accessed model when making space for a new one. **`NUM_WORKERS`**: integer (default `1`). The number of workers used by HTTP interfaces. ## CLIP model options **`CLIP_VERSION_ID`**: string (default `ViT-B-16`). Sets the OpenAI CLIP version used by all `/clip` routes. Available versions are `RN101`, `RN50`, `RN50x16`, `RN50x4`, `RN50x64`, `ViT-B-16`, `ViT-B-32`, `ViT-L-14-336px`, and `ViT-L-14`. **`CLIP_MAX_BATCH_SIZE`**: integer (default `8`). Sets the max batch size accepted by the CLIP model inference functions. ## Model cache **`MODEL_CACHE_DIR`**: string (default `/tmp/cache`). Sets the container path for the root model cache directory. **`TENSORRT_CACHE_PATH`**: string (default: the value of `MODEL_CACHE_DIR`). Sets the container path to the TensorRT cache directory. Setting this path together with a mounted host volume reduces the cold start time of TensorRT-based servers. ### Persistent model cache By default model weights are stored inside the container at `/tmp/cache` and are **lost on container restart or system reboot**. For production deployments, mount a persistent host volume to preserve downloaded weights: ```bash # Create a persistent cache directory on the host mkdir -p /var/lib/roboflow/cache # Run the container with a persistent cache docker run -d \ -p 9001:9001 \ -v /var/lib/roboflow/cache:/tmp/cache \ -e MODEL_CACHE_DIR=/tmp/cache \ roboflow/roboflow-inference-server-cpu:latest ``` Things to keep in mind: * The host path should be on persistent storage, not in `/tmp`. * The mounted directory needs appropriate permissions for the container user (typically UID 1000 or root, depending on the image). * A persistent cache lets you pre-populate weights before deployment and keeps them across container updates. See [Offline weights download](https://docs.roboflow.com/reference/inference/inference-python/offline-weights) for more on pre-downloading and caching weights. ## HTTPS / TLS **`ENABLE_HTTPS`**: boolean (default `False`). When set, the Inference Server serves traffic over HTTPS instead of HTTP, reading the certificate and private key from `SSL_CERTFILE` and `SSL_KEYFILE`. **`SSL_CERTFILE`**: string (default `/etc/inference/certs/server.crt`) and **`SSL_KEYFILE`**: string (default `/etc/inference/certs/server.key`). Paths to the PEM-encoded certificate and private key inside the container. The defaults are convenient mount points, so usually you only need to bind your cert and key into `/etc/inference/certs/` and set `ENABLE_HTTPS=true`. **`SSL_KEYFILE_PASSWORD`**: string (optional). Set this if your private key is encrypted. **`SSL_CA_CERTS`**: string (optional). Set this to a CA bundle when you need client certificate verification (mTLS). Full walkthrough with self-signed certificates: [Serving Inference over HTTPS](/deployment/self-hosted/inference-server/configuration/https.md). ## Secure Gateway **`SECURE_GATEWAY`**: string (default unset). Sets the address of a [Roboflow Secure Gateway](/deployment/self-hosted/enterprise/secure-gateway.md) for air-gapped deployments. Roboflow API and model download traffic is routed through this proxy. Traffic that cannot be proxied is disabled or rerouted as follows: * The Inference version check (which calls `api.github.com`) is force-disabled: `DISABLE_VERSION_CHECK` is set to `True` even if explicitly configured otherwise. * If `WORKFLOWS_STEP_EXECUTION_MODE=remote` is combined with `WORKFLOWS_REMOTE_API_TARGET=hosted`, step execution falls back to `local` with a warning, because the hosted Roboflow inference endpoints cannot be reached through the gateway proxy. To keep remote execution, use `WORKFLOWS_REMOTE_API_TARGET=self-hosted` **and** point `LOCAL_INFERENCE_API_URL` (default `http://127.0.0.1:9001`) at an Inference server reachable inside the gateway perimeter. * Third-party integrations (Google Vision and Gemini direct-key paths, Stability AI, Twilio media upload, webhooks to external hosts) are not proxied and will fail unless the gateway network permits them. The legacy `LICENSE_SERVER` environment variable is still accepted but deprecated.