Inference Server Telemetry

Monitor a self-hosted Roboflow Inference server with Prometheus metrics and Docker container statistics.

Service telemetry provides real-time data on system health, performance, and usage. It enables:

• Monitoring and diagnostics: early detection of issues for quick resolution.

• Performance optimization: identifying bottlenecks to improve efficiency.

• Usage insights: understanding user behavior to guide improvements.

• Security: detecting suspicious activity and ensuring compliance.

• Scalability: predicting and managing resource demands.

The Inference server exposes two sources of telemetry:

• Prometheus metrics

• Docker container metrics provided by the Docker daemon

Prometheus metrics

To enable metrics, set the environment variable ENABLE_PROMETHEUS=True on your container:

docker run -p 9001:9001 -e ENABLE_PROMETHEUS=True roboflow/roboflow-inference-server-cpu

Then use the GET /metrics endpoint to fetch the metrics in Python:

import requests

result = requests.get("http://127.0.0.1:9001/metrics")
result.raise_for_status()

print(result.text)

or with curl:

curl http://127.0.0.1:9001/metrics

/metrics is one of the endpoints that stays unauthenticated even when API-key authentication is enabled, so restrict network access to the server if the metrics are sensitive.

Docker container metrics

Potential security issue. This feature relies on exposing the Docker daemon socket inside the container. That exposes container resource utilization metrics without needing a supervisor service, but it may be considered a security violation. It is disabled by default. Acknowledge the potential security risks before enabling it.

To expose container metrics, run the Inference server container with the Docker socket mounted:

docker run -p 9001:9001 \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -e DOCKER_SOCKET_PATH=/var/run/docker.sock \
  roboflow/roboflow-inference-server-cpu

• The -v line mounts the Docker daemon socket from your host (typically /var/run/docker.sock, but verify your setup) into the container, here also at /var/run/docker.sock.

• The -e line sets DOCKER_SOCKET_PATH to the location of the Docker daemon socket inside the container, matching the mount above.

You can then reach the GET /device/stats endpoint with curl:

curl http://127.0.0.1:9001/device/stats

or with Python:

import requests

result = requests.get("http://127.0.0.1:9001/device/stats")
result.raise_for_status()

print(result.json())

Model-level monitoring

For prediction-level monitoring across deployments (rather than host telemetry), see Model Monitoring. It is controlled on a self-hosted server with the METRICS_ENABLED and MODEL_MONITORING_CACHE_BACKEND environment variables.