Model Monitoring

A guide to model monitoring with Roboflow

Roboflow's Model Monitoring dashboard provides visibility into how your models are performing in production. You can view high level statistics on your models' prediction results and view each individual prediction result.

Note: This feature is only available to workspaces with an Enterprise plan

To view your Model Monitoring dashboard, click the "Monitoring" tab from your workspace view. If you do not see the "Monitoring" tab in your Enterprise workspace, please reach out to Roboflow Support.

Click the "Monitoring" tab to access model monitoring

You will see a summary of statistics on predictions made to all of your models. You can filter by time using one of the buttons in just above the main stats display. To view detailed information on a particular model, click on the model name in the table (near the purple arrow).

Main Model Monitoring dashboard view

The model details dashboard displays information about a specific model.

Model details view

You can view the class distribution and number of detections. The date filters in the top right can be used to filter model details as well.

Model details view

Clicking on the "See All Inferences" button at the top right of the table will navigate to a view where you can see all prediction results for your models. Here you can filter by multiple parameters such as model, class/label, deployment type, and date. Any Custom Metadata that you have configured will show up in the table as well.

All inferences view allows you to filter all inference results

Viewing Prediction Images

Some predictions may contain the actual image used in the inference request. In order to have images on prediction requests, create an Active Learning rule for your model. Any image generated by your Active Learning rule will be inserted into the Model Monitoring prediction and viewable in the dashboard.

Predictions with an image will appear in the dashboard

Clicking on the image will bring up a full screen view. This contains the bounding boxes (or polygons) that were predicted by your model.

Inspect prediction images

To create an Active Learning rule, visit your project's main page and click "Active Learning" at the bottom of the left navigation panel. From here, you can configure your rule. If you would just like to get started quickly, you can just click the "Save" button at the bottom to use our presets.

Note that images generated by Active Learning will count towards the image quota for your workspace.

Alerting

You can subscribe to real time alerting for anomalies with your data. For example, if there is a sudden drop in detections, you can receive an email notification to that effect.

To create an alert, follow the Alerting guide.

Custom Metadata

The model monitoring dashboard can also provide you with data that is related to your specific use case with the Custom Metadata feature. This feature involves injecting custom metadata (via http request) into your inference.

To create a custom metadata field, review the Custom Metadata API documentation.

Edge Deployments

Inferences made with Roboflow's inference container will appear in your dashboard provided your edge device has a stable internet connection. In addition, ensure you set the DEVICE_ID environment variable to something you can easily identify when running the container. Also ensure that the METRICS_ENABLED variable is set to true (default). Edge deployments using the Roboflow License Server are also supported.