Offline Mode

In certain Enterprise Deployments, you may need to take the Roboflow Deployment server offline for a period of time.

With the optional Offline Mode add-on available to enterprise customers, you can configure the Roboflow Inference Server to cache weights for up to 30 days. This allows it to run completely air-gapped or in locations where an Internet connection is not readily available.
To enable Offline Mode, you'll need to create and attach a Docker volume to /tmp/cache on the Inference Server:
CPU
Image: roboflow / roboflow-inference-server-cpu​
sudo docker volume create roboflow
sudo docker run --net=host --env LICENSE_SERVER=10.0.1.1 --mount source=roboflow,target=/tmp/cache roboflow/roboflow-inference-server-cpu
TRT
To use the TRT container, you must first install nvidia-container-runtime.
Image: roboflow / roboflow-inference-server-trt​
sudo docker volume create roboflow
docker run -it --rm -p 9001:9001 --gpus all --mount source=roboflow,target=/tmp/cache roboflow/roboflow-inference-server-trt
The TRT image will only function on NVIDIA hardware with support for TensorRT 22.09.
Jetson TRT 4.5
Your Jetson Jetpack 4.5 will already have https://github.com/NVIDIA/nvidia-container-runtime installed.
Image: roboflow/roboflow-inference-server-trt-jetson​
sudo docker volume create roboflow
docker run -it --rm -p 9001:9001 --runtime=nvidia --mount source=roboflow,target=/tmp/cache roboflow/roboflow-inference-server-trt-jetson
Jetson TRT 5.1
Your Jetson Jetpack 5.1 will already have https://github.com/NVIDIA/nvidia-container-runtime installed.
Image: roboflow/roboflow-inference-server-trt-jetson-5.1.1​
sudo docker volume create roboflow
docker run -it --rm -p 9001:9001 --runtime=nvidia --mount source=roboflow,target=/tmp/cache roboflow/roboflow-inference-server-trt-jetson-5.1.1
Inference Results
The weights will be loaded from the your Roboflow account over the Internet (via the License Server if you have configured it) with SSL encryption and stored safely in the Docker volume for up to 30 days.
Your inference results will contain a new expiration key you can use to determine how long the Inference Server can continue to provide predictions before renewing its lease on the weights via an Internet or License Server connection. Once the weight expiration date drops below 7 days, the Inference Server will begin trying to renew the weights' lease once per hour until a connection to the Roboflow API is successfully made.
{
 "predictions": [
 {
 "x": 340.9,
 "y": 263.6,
 "width": 284,
 "height": 360,
 "class": "example",
 "confidence": 0.867
 }
 ],
 "expiration": {
 "value": 29.91251408564815,
 "unit": "days"
 }
}

Last modified 1mo ago