> For the complete documentation index, see [llms.txt](https://docs.roboflow.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.roboflow.com/deployment/self-hosted/inference-server/configuration/environment-variables.md).

# Inference Server Environment Variables

Environment variables that control a self-hosted Roboflow Inference server - execution providers, caching, Workflows, Roboflow API retries, telemetry, HTTPS, and security.

Inference server behavior is controlled by a set of environment variables. Every variable is defined in [`inference/core/env.py`](https://github.com/roboflow/inference/blob/main/inference/core/env.py); the ones below are the variables that need more explanation.

Pass them to the container with `-e` (see [Docker configuration options](/deployment/self-hosted/inference-server/configuration/docker-configuration.md) for the most commonly changed settings):

```bash
docker run -it --rm -e ENV_VAR_NAME=env_var_value -p 127.0.0.1:9001:9001 roboflow/roboflow-inference-server-cpu:latest
```

{% hint style="info" %}
These variables configure the Inference server. The Roboflow CLI and Python SDK read a different, smaller set of variables (`ROBOFLOW_API_KEY`, `ROBOFLOW_CONFIG_DIR`, and others): see [Environment Variables](https://docs.roboflow.com/reference/environment-variables) in the Reference section.
{% endhint %}

For the source and release scope of recent security settings, see [Security Configuration Migration](/deployment/self-hosted/inference-server/configuration/security-migration.md#release-status).

## Execution and models

<table data-search="false"><thead><tr><th>Variable</th><th>Description</th><th>Default</th></tr></thead><tbody><tr><td>ONNXRUNTIME_EXECUTION_PROVIDERS</td><td>List of execution providers in priority order. A warning is displayed if a provider is not supported on your platform.</td><td>See <a href="https://github.com/roboflow/inference/blob/main/inference/core/env.py">env.py</a></td></tr><tr><td>RUNS_ON_JETSON</td><td>Whether Inference runs on a Jetson device. Set to True in all Docker builds for the Jetson architecture.</td><td>False</td></tr><tr><td>MODEL_VALIDATION_DISABLED</td><td>Makes model loading faster by skipping the trial inference.</td><td>False</td></tr><tr><td>SAM2_MAX_EMBEDDING_CACHE_SIZE</td><td>Number of SAM2 embeddings held in GPU memory. Each embedding takes 16777216 bytes.</td><td>100</td></tr><tr><td>SAM2_MAX_LOGITS_CACHE_SIZE</td><td>Number of SAM2 logits held in CPU memory. Each logit takes 262144 bytes.</td><td>1000</td></tr><tr><td>DISABLE_SAM2_LOGITS_CACHE</td><td>Disables caching of SAM2 logits. Useful for debugging or to minimize memory usage, at the cost of slower repeated similar requests.</td><td>False</td></tr></tbody></table>

## `inference-models` backend

<table data-search="false"><thead><tr><th>Variable</th><th>Description</th><th>Default</th></tr></thead><tbody><tr><td>USE_INFERENCE_MODELS</td><td>Selects the inference-models backend; images may override it.</td><td>False in the core library</td></tr><tr><td>ALLOW_INFERENCE_MODELS_DIRECTLY_ACCESS_LOCAL_PACKAGES</td><td>Allows local model packages, including their code. Incompatible with online model authorization. See <a href="/deployment/self-hosted/inference-server/configuration/model-security.md">Model Package Security</a>.</td><td>False</td></tr><tr><td>ALLOW_INFERENCE_MODELS_UNTRUSTED_PACKAGES</td><td>Allows downloaded packages classified as untrusted. Separate from direct local loading; see <a href="/deployment/self-hosted/inference-server/configuration/security-migration.md#model-package-trust">image migration defaults</a>.</td><td>False in the core library; image-dependent</td></tr><tr><td>MAX_INFERENCE_MODELS_CACHE_SIZE_MB</td><td>Enables the inference-models cache watchdog. When set above 0, the watchdog prunes model artifacts (oldest and biggest first) to prevent the system running out of disk space over time. Only applies when USE_INFERENCE_MODELS=True.</td><td>-1</td></tr><tr><td>INFERENCE_MODELS_CACHE_WATCHDOG_INTERVAL_MINUTES</td><td>Frequency of inference-models cache watchdog cycles. The minimum is 15 minutes.</td><td>60</td></tr><tr><td>ENABLE_CUDA_MEMORY_RECLAMATION_WATCHDOG</td><td>Enables a background daemon that periodically returns cached-but-unused CUDA memory to the driver via torch.cuda.empty_cache(). PyTorch's caching allocator retains freed device blocks in its own pool and never releases them on its own, so on a long-running server the high-water mark of concurrent or batched inference is sticky and reserved VRAM only grows. This watchdog reclaims that slack on a fixed interval; live allocations are untouched. It does not prevent an OOM caused by a genuinely oversubscribed concurrent peak. Only meaningful with USE_INFERENCE_MODELS=True on CUDA.</td><td>False</td></tr><tr><td>CUDA_MEMORY_RECLAMATION_WATCHDOG_INTERVAL_SECONDS</td><td>Interval between reclamation cycles of the CUDA memory watchdog. The minimum is 5 seconds (lower values are clamped up). Only applies when ENABLE_CUDA_MEMORY_RECLAMATION_WATCHDOG=True.</td><td>300</td></tr></tbody></table>

## Workflows and video

<table data-search="false"><thead><tr><th>Variable</th><th>Description</th><th>Default</th></tr></thead><tbody><tr><td>ENABLE_WORKFLOWS_PROFILING</td><td>Allows the server to return Workflows profiler traces to the client.</td><td>False</td></tr><tr><td>WORKFLOWS_PROFILER_BUFFER_SIZE</td><td>Size of the profiler buffer: the number of consecutive Workflows Execution Engine run(...) invocations traced in the buffer.</td><td>64</td></tr><tr><td>WORKFLOWS_DEFINITION_CACHE_EXPIRY</td><td>Number of seconds to cache Workflow definitions returned by get_workflow_specification(...).</td><td>900 (15 minutes)</td></tr><tr><td>ENABLE_STREAM_API</td><td>Enables the integrated video management API. The standard CPU, GPU, and TensorRT images set this to True, as do the JetPack 5.1.1 and 6.2.0 images; slim images and the JetPack 6.0.0 and 7.1.0 images do not. Check your pinned image.</td><td>False in the core library; True in the images listed</td></tr><tr><td>STREAM_API_PRELOADED_PROCESSES</td><td>Target number of warm idle video workers, which reduces worker start time on GPU. Images may override it.</td><td>0 in the core library</td></tr><tr><td>STREAM_MANAGER_MAX_ACTIVE_PIPELINES</td><td>Cap on managed pipeline processes, including idle workers. Requests that need a new worker beyond the cap are rejected. Raised to STREAM_API_PRELOADED_PROCESSES if that is higher. See <a href="/deployment/self-hosted/inference-server/configuration/video-configuration.md">Video Configuration</a>.</td><td>8</td></tr><tr><td>STREAM_MANAGER_MAX_RAM_MB</td><td>Total RAM budget for the stream manager and its workers. New workers are refused when the projected total would exceed it. This is an estimate-based guard, not a hard memory limit.</td><td>Not set (no RAM ceiling)</td></tr><tr><td>ALLOW_UNSAFE_GSTREAMER_PIPELINES</td><td>Bypasses media-reference validation for trusted raw pipelines in the pending runtime-hardening build. See <a href="/deployment/self-hosted/inference-server/configuration/security-migration.md#video-source-validation">migration scope</a>.</td><td>False in that build</td></tr><tr><td>ALLOW_CUSTOM_PYTHON_EXECUTION_IN_WORKFLOWS</td><td>Allows Custom Python to execute locally. Does not disable Modal execution.</td><td>True</td></tr><tr><td>WORKFLOWS_CUSTOM_PYTHON_EXECUTION_MODE</td><td>Selects local or Modal Custom Python execution. See <a href="/deployment/self-hosted/inference-server/configuration/security-migration.md#offline-custom-python">offline migration behavior</a>.</td><td>local</td></tr><tr><td>ALLOW_WEBHOOK_WORKFLOWS_SINK_TO_NON_GLOBAL_ADDRESSES</td><td>Allows the Webhook Sink to connect to loopback, private, link-local, and other non-global addresses. Set to False when untrusted callers can choose webhook destinations. Redirects and HTTP(S) proxies are rejected for either value. See <a href="/deployment/self-hosted/inference-server/configuration/security.md#restrict-webhook-sink-destinations">server security</a>.</td><td>True</td></tr></tbody></table>

## GPU tensor pipeline

On a capable NVIDIA GPU, Workflows can keep image data on the GPU from video decode through model inference to output, instead of moving it through the CPU. This lowers latency for GPU-heavy video Workflows. It needs `USE_INFERENCE_MODELS=True`, a GPU with compute capability 7.5 or higher (ex: T4, RTX 20-series and newer; not V100), and the `onnx.gpu` or Jetson Docker image. Without a matching GPU and image, Inference falls back to normal CPU-based execution.

<table data-search="false"><thead><tr><th>Variable</th><th>Description</th><th>Default</th></tr></thead><tbody><tr><td>ENABLE_TENSOR_DATA_REPRESENTATION</td><td>Turns on the GPU tensor pipeline: GPU-resident Workflows execution and hardware-accelerated (NVDEC) video decoding.</td><td>False</td></tr><tr><td>WORKFLOWS_IMAGE_TENSOR_DEVICE</td><td>Torch device Workflow tensors are placed on when the tensor pipeline is enabled.</td><td>cuda if available, else cpu</td></tr><tr><td>VIDEO_SOURCE_BUFFER_SIZE</td><td>Number of decoded frames buffered per video source.</td><td>8 when the tensor pipeline is enabled, 64 otherwise</td></tr><tr><td>VIDEO_SOURCE_ADAPTIVE_BACKPRESSURE</td><td>Drops buffered frames based on buffer state instead of estimated frame rate.</td><td>Matches ENABLE_TENSOR_DATA_REPRESENTATION</td></tr><tr><td>WORKFLOWS_ENFORCE_DENSE_INSTANCE_MASKS</td><td>Returns dense instance segmentation masks instead of RLE-encoded ones from tensor pipeline models.</td><td>False</td></tr><tr><td>WORKFLOWS_SAM_VIDEO_MASK_REPRESENTATION</td><td>Mask format (rle or dense) used by SAM video tracking blocks in the tensor pipeline.</td><td>rle</td></tr><tr><td>DISABLE_GSTREAMER_VIDEO_SOURCES</td><td>Disables GStreamer-based video sources, forcing standard CPU decoding.</td><td>False</td></tr></tbody></table>

## Roboflow API connectivity

<table data-search="false"><thead><tr><th>Variable</th><th>Description</th><th>Default</th></tr></thead><tbody><tr><td>TRANSIENT_ROBOFLOW_API_ERRORS</td><td>Comma-separated list of HTTP codes from the Roboflow API that should be retried (GET endpoints only).</td><td>Not set</td></tr><tr><td>TRANSIENT_ROBOFLOW_API_ERRORS_RETRIES</td><td>Number of times transient errors (connection errors and transient HTTP codes) are retried (GET endpoints only).</td><td>3</td></tr><tr><td>TRANSIENT_ROBOFLOW_API_ERRORS_RETRY_INTERVAL</td><td>Delay between retries of transient Roboflow API errors (GET endpoints only).</td><td>3</td></tr><tr><td>RETRY_CONNECTION_ERRORS_TO_ROBOFLOW_API</td><td>Whether connection errors to the Roboflow API should be retried (GET endpoints only).</td><td>False</td></tr><tr><td>ROBOFLOW_API_REQUEST_TIMEOUT</td><td>Timeout in seconds (integer) for requests to the Roboflow API.</td><td>Not set</td></tr><tr><td>API_PROXY_BASE_URL</td><td>Base URL used for Roboflow API proxy requests to apiproxy/* endpoints. Set this to a direct heavy-API Cloud Run service root to bypass Firebase Hosting timeouts for long-running third-party proxy calls.</td><td>Value of API_BASE_URL</td></tr><tr><td>DISABLE_VERSION_CHECK</td><td>Disables the Inference version check that runs in a background thread. Force-set to True (overriding an explicit False) when SECURE_GATEWAY is configured, because api.github.com is unreachable behind the gateway.</td><td>False</td></tr><tr><td>SECURE_GATEWAY</td><td>Address of a <a href="/deployment/self-hosted/enterprise/secure-gateway.md">Roboflow Secure Gateway</a> for air-gapped deployments (legacy alias: LICENSE_SERVER). Routes Roboflow API and model download traffic through the gateway proxy, force-disables the version check, and falls back to local Workflow step execution when remote plus hosted is configured. See <a href="/deployment/self-hosted/inference-server/configuration/docker-configuration.md#secure-gateway">Docker configuration options</a>.</td><td>Not set</td></tr></tbody></table>

## Monitoring and telemetry

<table data-search="false"><thead><tr><th>Variable</th><th>Description</th><th>Default</th></tr></thead><tbody><tr><td>ENABLE_PROMETHEUS</td><td>Enables the Prometheus /metrics endpoint. See <a href="/deployment/self-hosted/inference-server/configuration/telemetry.md">Telemetry</a>.</td><td>True for the Docker Hub images</td></tr><tr><td>DOCKER_SOCKET_PATH</td><td>Path to the Docker daemon socket mounted into the container. When provided, enables polling Docker container stats from the daemon socket. See <a href="/deployment/self-hosted/inference-server/configuration/telemetry.md#docker-container-metrics">Telemetry</a>.</td><td>Not set</td></tr><tr><td>METRICS_ENABLED</td><td>Controls Roboflow <a href="/deployment/monitoring-and-analytics/model-monitoring.md">Model Monitoring</a>.</td><td>True</td></tr><tr><td>MODEL_MONITORING_CACHE_BACKEND</td><td>Cache backend for model-monitoring pingback data. Use default to follow the normal cache selection (Redis when REDIS_HOST is configured, otherwise memory), or memory to force process-local buffering and keep Redis off the inference hot path.</td><td>default</td></tr></tbody></table>

## HTTPS

<table data-search="false"><thead><tr><th>Variable</th><th>Description</th><th>Default</th></tr></thead><tbody><tr><td>ENABLE_HTTPS</td><td>Toggles HTTPS for the Inference server. When True, the server reads SSL_CERTFILE and SSL_KEYFILE and serves traffic over TLS. See <a href="/deployment/self-hosted/inference-server/configuration/https.md">Serving Inference over HTTPS</a>.</td><td>False</td></tr><tr><td>SSL_CERTFILE</td><td>Path to a PEM-encoded TLS certificate served when ENABLE_HTTPS=True.</td><td>/etc/inference/certs/server.crt</td></tr><tr><td>SSL_KEYFILE</td><td>Path to the PEM-encoded TLS private key paired with SSL_CERTFILE.</td><td>/etc/inference/certs/server.key</td></tr><tr><td>SSL_KEYFILE_PASSWORD</td><td>Encrypted-key passphrase for Uvicorn. Unsupported by the parallel Gunicorn launcher.</td><td>Not set</td></tr><tr><td>SSL_CA_CERTS</td><td>Client CA bundle for HTTPS. See <a href="/deployment/self-hosted/inference-server/configuration/https.md#mutual-tls">mTLS enforcement and release scope</a>.</td><td>Not set</td></tr></tbody></table>

## Authentication and input security

<table data-search="false"><thead><tr><th>Variable</th><th>Description</th><th>Default</th></tr></thead><tbody><tr><td>WORKSPACES_WHITELISTED_FOR_LOCAL_DEPLOYMENT</td><td>Allowed Workspace slugs for inference, Workflow, and integrated pipeline requests. See <a href="/deployment/self-hosted/inference-server/configuration/security.md#enforce-authentication">authentication and exemptions</a>.</td><td>Not set</td></tr><tr><td>MODELS_CACHE_AUTH_ENABLED</td><td>Checks a caller's access to a Roboflow model online. Rejects direct local package loading in that mode.</td><td>False</td></tr><tr><td>ALLOW_OFFLINE_MODEL_CACHE_AUTH_BYPASS</td><td>Explicit authorization bypass required when offline mode and model cache authorization are both enabled. For trusted single-tenant deployments only.</td><td>False</td></tr></tbody></table>

Variables that control custom Python execution and URL image fetching (`ALLOW_CUSTOM_PYTHON_EXECUTION_IN_WORKFLOWS`, `ALLOW_URL_INPUT`, `ALLOW_URL_TO_NON_GLOBAL_ADDRESSES`, `VALIDATE_IMAGE_URL_REDIRECTS`, and others) are documented in [Securing a Self-Hosted Server](/deployment/self-hosted/inference-server/configuration/security.md) and [Accepted Input Formats](/deployment/self-hosted/inference-server/configuration/input-formats.md).