> For the complete documentation index, see [llms.txt](https://docs.roboflow.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.roboflow.com/reference/inference/inference-python/offline-weights.md).

# Model Weights Download

When deploying Roboflow Inference, model weights are downloaded to your device where inference runs locally. This page covers how the `inference` package downloads and caches those weights.

{% hint style="info" %}
Looking for the raw `.pt` weights file to run outside the Roboflow Inference ecosystem? See [Download Model Weights](https://docs.roboflow.com/models/model-weights/download-roboflow-model-weights).
{% endhint %}

## Overview

Model weights are downloaded automatically the first time you run inference with a given model. The weights are cached locally on your device, and all inference is performed on-device (not in the cloud).

How it works:

1. Download model weights to your device while connected to the internet.
2. Weights are cached locally on your machine.
3. Run inference on-device using the cached weights.

This approach works across all Roboflow deployment methods and ensures fast, local inference.

{% hint style="warning" %}
**Default cache location.** By default, model weights are stored in `/tmp/cache`, which is **cleared on system reboot**. For production deployments, or any scenario where you need weights to persist across reboots, you must configure a persistent cache directory using the `MODEL_CACHE_DIR` environment variable (see [Cache location](#cache-location) below).
{% endhint %}

{% hint style="info" %}
**Enterprise offline mode.** For enterprise deployments requiring completely disconnected operation, see [Offline Mode](https://docs.roboflow.com/deployment/self-hosted/enterprise/offline-mode). This page focuses on model weights download and caching, while maintaining connectivity for usage tracking, billing, and Workflow updates.
{% endhint %}

## Cache location

By default, model weights are cached in `/tmp/cache`. **This directory is cleared on system reboot**, which means you need to re-download model weights after each restart.

For production deployments or any scenario where you need weights to persist across reboots, you **must** configure a persistent cache directory using the `MODEL_CACHE_DIR` environment variable:

```python
import os
# Set to a persistent directory (not /tmp)
os.environ["MODEL_CACHE_DIR"] = "/home/user/.roboflow/cache"

from inference import get_model
# ... rest of your code
```

Alternatively, set it system-wide:

```bash
export MODEL_CACHE_DIR="/home/user/.roboflow/cache"
```

Make sure the directory exists and has appropriate permissions:

```bash
mkdir -p /home/user/.roboflow/cache
chmod 755 /home/user/.roboflow/cache
```

{% hint style="success" %}
**Docker deployments.** When running Inference in Docker, mount a persistent cache volume to preserve weights across container restarts. See [Docker Configuration](https://docs.roboflow.com/deployment/self-hosted/inference-server/configuration/docker-configuration#persistent-model-cache) for details.
{% endhint %}

## Native Python API

The [native Python API](/reference/inference/inference-python/native-python-api.md) automatically downloads and caches weights when you load a model with `get_model()`.

### Pre-downloading weights

```python
from inference import get_model

# Load model (downloads and caches weights)
model = get_model(
    model_id="rfdetr-base",
    api_key="YOUR_ROBOFLOW_API_KEY"
)
print("Model weights cached!")
```

### Running inference

```python
from inference import get_model

# Uses cached weights for on-device inference
model = get_model(
    model_id="rfdetr-base",
    api_key="YOUR_ROBOFLOW_API_KEY"
)

results = model.infer("path/to/image.jpg")
```

On a self-hosted Inference Server, you can pre-load weights over HTTP instead: see [Model Management](/reference/inference/inference-sdk/model-management.md).

## Best practices

1. **Configure a persistent cache first.** Before downloading any weights, configure `MODEL_CACHE_DIR` to point to a persistent directory (not `/tmp`). This is essential for production deployments to avoid losing cached weights on reboot.
2. **Pre-download during setup.** Download all required model weights during your deployment setup phase to ensure they are cached and ready.
3. **Use a persistent cache in Docker.** Always [mount a persistent volume](https://docs.roboflow.com/deployment/self-hosted/inference-server/configuration/docker-configuration#persistent-model-cache) when running in Docker containers. Weights stored in the container filesystem are lost on restart.
4. **Verify before deployment.** Verify that models are properly cached and that the cache directory persists across reboots before deploying to production.
5. **Document model IDs.** Keep a list of all model IDs and versions your application requires for easier pre-caching and troubleshooting.
6. **Consider storage.** Model weights can be large (100MB to 1GB+ per model). Ensure sufficient disk space is available in your persistent cache directory.
7. **Test reboot behavior.** After caching weights, test that they persist after a system reboot to ensure your cache configuration is correct.

## Troubleshooting

### Weights disappear after reboot

The default cache location (`/tmp/cache`) is cleared on reboot. Configure a persistent cache directory as described in [Cache location](#cache-location), or use a [persistent volume mount for Docker](https://docs.roboflow.com/deployment/self-hosted/inference-server/configuration/docker-configuration#persistent-model-cache).

### Model not found error

* Verify the model was actually downloaded (check the cache directory with `ls -lh $MODEL_CACHE_DIR`).
* Ensure you are using the exact same `model_id` as when downloading.
* Check that `MODEL_CACHE_DIR` is set correctly if using a custom location.

### Permission issues

Ensure the application has read/write permissions to the cache directory:

```bash
chmod -R 755 /path/to/cache
```