Model Weights Download
How the inference package downloads and caches model weights, how to configure a persistent cache directory, and how to pre-download weights.
When deploying Roboflow Inference, model weights are downloaded to your device where inference runs locally. This page covers how the inference package downloads and caches those weights.
Overview
Model weights are downloaded automatically the first time you run inference with a given model. The weights are cached locally on your device, and all inference is performed on-device (not in the cloud).
How it works:
Download model weights to your device while connected to the internet.
Weights are cached locally on your machine.
Run inference on-device using the cached weights.
This approach works across all Roboflow deployment methods and ensures fast, local inference.
Cache location
By default, model weights are cached in /tmp/cache. This directory is cleared on system reboot, which means you need to re-download model weights after each restart.
For production deployments or any scenario where you need weights to persist across reboots, you must configure a persistent cache directory using the MODEL_CACHE_DIR environment variable:
import os
# Set to a persistent directory (not /tmp)
os.environ["MODEL_CACHE_DIR"] = "/home/user/.roboflow/cache"
from inference import get_model
# ... rest of your codeAlternatively, set it system-wide:
Make sure the directory exists and has appropriate permissions:
Native Python API
The native Python API automatically downloads and caches weights when you load a model with get_model().
Pre-downloading weights
Running inference
On a self-hosted Inference Server, you can pre-load weights over HTTP instead: see Model Management.
Best practices
Configure a persistent cache first. Before downloading any weights, configure
MODEL_CACHE_DIRto point to a persistent directory (not/tmp). This is essential for production deployments to avoid losing cached weights on reboot.Pre-download during setup. Download all required model weights during your deployment setup phase to ensure they are cached and ready.
Use a persistent cache in Docker. Always mount a persistent volume when running in Docker containers. Weights stored in the container filesystem are lost on restart.
Verify before deployment. Verify that models are properly cached and that the cache directory persists across reboots before deploying to production.
Document model IDs. Keep a list of all model IDs and versions your application requires for easier pre-caching and troubleshooting.
Consider storage. Model weights can be large (100MB to 1GB+ per model). Ensure sufficient disk space is available in your persistent cache directory.
Test reboot behavior. After caching weights, test that they persist after a system reboot to ensure your cache configuration is correct.
Troubleshooting
Weights disappear after reboot
The default cache location (/tmp/cache) is cleared on reboot. Configure a persistent cache directory as described in Cache location, or use a persistent volume mount for Docker.
Model not found error
Verify the model was actually downloaded (check the cache directory with
ls -lh $MODEL_CACHE_DIR).Ensure you are using the exact same
model_idas when downloading.Check that
MODEL_CACHE_DIRis set correctly if using a custom location.
Permission issues
Ensure the application has read/write permissions to the cache directory:
Last updated
Was this helpful?