Inference Server Environment Variables

Environment variables that control a self-hosted Roboflow Inference server - execution providers, caching, Workflows, Roboflow API retries, telemetry, HTTPS, and security.

Inference server behavior is controlled by a set of environment variables. Every variable is defined in inference/core/env.py; the ones below are the variables that need more explanation.

Pass them to the container with -e (see Docker configuration options for the most commonly changed settings):

docker run -it --rm -e ENV_VAR_NAME=env_var_value -p 9001:9001 roboflow/roboflow-inference-server-cpu:latest

These variables configure the Inference server. The Roboflow CLI and Python SDK read a different, smaller set of variables (ROBOFLOW_API_KEY, ROBOFLOW_CONFIG_DIR, and others): see Environment Variables in the Reference section.

Execution and models

Variable	Description	Default
ONNXRUNTIME_EXECUTION_PROVIDERS	List of execution providers in priority order. A warning is displayed if a provider is not supported on your platform.	See env.py
RUNS_ON_JETSON	Whether Inference runs on a Jetson device. Set to True in all Docker builds for the Jetson architecture.	False
MODEL_VALIDATION_DISABLED	Makes model loading faster by skipping the trial inference.	False
SAM2_MAX_EMBEDDING_CACHE_SIZE	Number of SAM2 embeddings held in GPU memory. Each embedding takes 16777216 bytes.	100
SAM2_MAX_LOGITS_CACHE_SIZE	Number of SAM2 logits held in CPU memory. Each logit takes 262144 bytes.	1000
DISABLE_SAM2_LOGITS_CACHE	Disables caching of SAM2 logits. Useful for debugging or to minimize memory usage, at the cost of slower repeated similar requests.	False

inference-models backend

Variable	Description	Default
USE_INFERENCE_MODELS	Selects the inference-models backend.	False
MAX_INFERENCE_MODELS_CACHE_SIZE_MB	Enables the inference-models cache watchdog. When set above 0, the watchdog prunes model artifacts (oldest and biggest first) to prevent the system running out of disk space over time. Only applies when USE_INFERENCE_MODELS=True.	-1
INFERENCE_MODELS_CACHE_WATCHDOG_INTERVAL_MINUTES	Frequency of inference-models cache watchdog cycles. The minimum is 15 minutes.	60
ENABLE_CUDA_MEMORY_RECLAMATION_WATCHDOG	Enables a background daemon that periodically returns cached-but-unused CUDA memory to the driver via torch.cuda.empty_cache(). PyTorch's caching allocator retains freed device blocks in its own pool and never releases them on its own, so on a long-running server the high-water mark of concurrent or batched inference is sticky and reserved VRAM only grows. This watchdog reclaims that slack on a fixed interval; live allocations are untouched. It does not prevent an OOM caused by a genuinely oversubscribed concurrent peak. Only meaningful with USE_INFERENCE_MODELS=True on CUDA.	False
CUDA_MEMORY_RECLAMATION_WATCHDOG_INTERVAL_SECONDS	Interval between reclamation cycles of the CUDA memory watchdog. The minimum is 5 seconds (lower values are clamped up). Only applies when ENABLE_CUDA_MEMORY_RECLAMATION_WATCHDOG=True.	300

Workflows and video

Variable	Description	Default
ENABLE_WORKFLOWS_PROFILING	Allows the server to return Workflows profiler traces to the client.	False
WORKFLOWS_PROFILER_BUFFER_SIZE	Size of the profiler buffer: the number of consecutive Workflows Execution Engine run(...) invocations traced in the buffer.	64
WORKFLOWS_DEFINITION_CACHE_EXPIRY	Number of seconds to cache Workflow definitions returned by get_workflow_specification(...).	900 (15 minutes)
ENABLE_STREAM_API	Enables the video management API. The standard CPU, GPU, and TensorRT images set this to True, as do the JetPack 5.1.1 and 6.2.0 images; slim images and the JetPack 6.0.0 and 7.1.0 images do not.	False outside the images that set it
STREAM_API_PRELOADED_PROCESSES	How many idle video workers are warmed up. This reduces worker start time on GPU.	0

Roboflow API connectivity

Variable	Description	Default
TRANSIENT_ROBOFLOW_API_ERRORS	Comma-separated list of HTTP codes from the Roboflow API that should be retried (GET endpoints only).	Not set
TRANSIENT_ROBOFLOW_API_ERRORS_RETRIES	Number of times transient errors (connection errors and transient HTTP codes) are retried (GET endpoints only).	3
TRANSIENT_ROBOFLOW_API_ERRORS_RETRY_INTERVAL	Delay between retries of transient Roboflow API errors (GET endpoints only).	3
RETRY_CONNECTION_ERRORS_TO_ROBOFLOW_API	Whether connection errors to the Roboflow API should be retried (GET endpoints only).	False
ROBOFLOW_API_REQUEST_TIMEOUT	Timeout in seconds (integer) for requests to the Roboflow API.	Not set
API_PROXY_BASE_URL	Base URL used for Roboflow API proxy requests to apiproxy/* endpoints. Set this to a direct heavy-API Cloud Run service root to bypass Firebase Hosting timeouts for long-running third-party proxy calls.	Value of API_BASE_URL
DISABLE_VERSION_CHECK	Disables the Inference version check that runs in a background thread. Force-set to True (overriding an explicit False) when SECURE_GATEWAY is configured, because api.github.com is unreachable behind the gateway.	False
SECURE_GATEWAY	Address of a Roboflow Secure Gateway for air-gapped deployments (legacy alias: LICENSE_SERVER). Routes Roboflow API and model download traffic through the gateway proxy, force-disables the version check, and falls back to local Workflow step execution when remote plus hosted is configured. See Docker configuration options.	Not set

Monitoring and telemetry

Variable	Description	Default
ENABLE_PROMETHEUS	Enables the Prometheus /metrics endpoint. See Telemetry.	True for the Docker Hub images
DOCKER_SOCKET_PATH	Path to the Docker daemon socket mounted into the container. When provided, enables polling Docker container stats from the daemon socket. See Telemetry.	Not set
METRICS_ENABLED	Controls Roboflow Model Monitoring.	True
MODEL_MONITORING_CACHE_BACKEND	Cache backend for model-monitoring pingback data. Use default to follow the normal cache selection (Redis when REDIS_HOST is configured, otherwise memory), or memory to force process-local buffering and keep Redis off the inference hot path.	default

HTTPS

Variable	Description	Default
ENABLE_HTTPS	Toggles HTTPS for the Inference server. When True, the server reads SSL_CERTFILE and SSL_KEYFILE and serves traffic over TLS. See Serving Inference over HTTPS.	False
SSL_CERTFILE	Path to a PEM-encoded TLS certificate served when ENABLE_HTTPS=True.	/etc/inference/certs/server.crt
SSL_KEYFILE	Path to the PEM-encoded TLS private key paired with SSL_CERTFILE.	/etc/inference/certs/server.key
SSL_KEYFILE_PASSWORD	Passphrase used to decrypt SSL_KEYFILE when the private key is encrypted.	Not set
SSL_CA_CERTS	Path to a CA bundle used when client certificate verification (mTLS) is required.	Not set

Authentication and input security

Variable	Description	Default
WORKSPACES_WHITELISTED_FOR_LOCAL_DEPLOYMENT	Comma-separated list of Roboflow workspace URL slugs allowed to execute requests against this server. When set, every request (apart from the docs, landing page, and health, liveness, and metrics endpoints) is authorized with a Roboflow API key.	Not set

Variables that control custom Python execution and URL image fetching (ALLOW_CUSTOM_PYTHON_EXECUTION_IN_WORKFLOWS, ALLOW_URL_INPUT, ALLOW_URL_TO_NON_GLOBAL_ADDRESSES, VALIDATE_IMAGE_URL_REDIRECTS, and others) are documented in Securing a Self-Hosted Server and Accepted Input Formats.