Model Monitoring
A guide to Model Monitoring with Roboflow.
Last updated
A guide to Model Monitoring with Roboflow.
Last updated
Roboflow's Model Monitoring dashboard gives you unparalleled visibility into your models, from prototyping, all the way through production. With Model Monitoring, you can view high-level statistics to get insight into how your models are performing over time, or even view individual inference requests, to see how your models perform on edge cases.
Note: This feature is only available to workspaces with an Enterprise plan. To upgrade to an Enterprise plan, please contact Roboflow sales.
To view your Model Monitoring dashboard, click the "Monitoring" tab in your workspace.
Immediately, you will see three statistics pertaining to your models: total requests, average confidence, and average inference time. By default, these statistics will show your data for the last week. However, you can modify the time range using the buttons on top of the statistics.
Under the Models tab, you can select a specific model to view its data.
Here, in addition to the aforementioned statistics, you can view the number of detections for each class in the model, and see its distribution with respect to other classes.
Clicking on the "See All Inferences" button at the top right of the table will navigate you to a new page where you can see all the prediction results for your model. In addition, you will also see any custom metadata that was added to your inferences. To view a subset of your inferences, you can use the filters on the top-right of the table.
Viewing Prediction Images
You will notice that some inferences in the "Recent Inferences" or "All Inferences" views contain an image. Images will be shown for any inference whose image was picked up by Active Learning. Setting a more frequent Active Learning rule will result in more images shown.
Clicking on the image will bring up a full-screen view. This contains the bounding boxes (or polygons) that were predicted by your model.
To create an Active Learning rule, visit your project's main page and click "Active Learning" at the bottom of the left navigation panel. From here, you can configure your rule.
Note that images generated by Active Learning will count towards the image quota for your workspace.
You and other members of your team can subscribe to real-time alerts when issues or anomalies occur with your model. For example, if the confidence of your model suddenly decreases, or your Inference Server goes down, and your model stops running, your team will receive an email notification.
To create an alert, please follow the Alerting Guide.
To attach additional metadata to an inference, you can use Model Monitoring's custom metadata feature. Using custom metadata, you can add information to an inference such as the location of where the image was taken, the expected value of the prediction, and so on. Your custom metadata will show up in the "Recent Inferences" and "All Inferences" views.
To attach custom metadata to an inference result, please see the Custom Metadata API documentation.
For automations and integration into external systems, you can pull Model Monitoring statistics using our API for model monitoring.
Model Monitoring supports inference requests made using Roboflow's Hosted API or the Roboflow Inference Server, granted the Inference Server has internet access. This includes edge deployments which use Roboflow's License Server.
At this time, Model Monitoring does not support inference requests made using the Inference Pipeline, however, we plan to add support in the near future.
