> For the complete documentation index, see [llms.txt](https://docs.roboflow.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.roboflow.com/models/evaluate/evaluate-trained-models.md). # Evaluate Trained Models Use Model Evaluation to explore how your model performs on your test dataset. ## About Model evaluations show: 1. A production metrics explorer, which helps you find the optimal confidence threshold at which to run your model; 2. Model improvement recommendations, which provide suggestions on how you can increase the accuracy of your model; 3. Performance by class, which shows how well your model identifies different classes; 4. A confusion matrix, which you can use to find specific classes on which your model thrives and struggles, and; 5. An interactive vector explorer which lets you identify clusters of images where your model does well or poorly; You can use model evaluation to identify areas of improvement for your model. Model evaluations are automatically run for all versioned models trained on, or uploaded to Roboflow by paid users. It may take several minutes for an evaluation to run for a dataset of a few hundred images, and several hours for large datasets with thousands or more images. The confusion matrix runs your trained model on the images in your test and validation sets and compares its predictions with your annotations (the ground truth). It shows correct predictions, false positives (the wrong class, or an object where none is present), and false negatives. The recommendations are based on it. They cover models with many false negatives or false positives, classes that are often confused, classes that need more data, and test or validation sets that may be too small. ### Optimal Confidence The production metrics explorer computes precision, recall, and F1 at every confidence threshold and recommends an optimal confidence: the threshold with the best trade-off. Once the evaluation completes, that threshold becomes the default for your model's inference requests. If per-class thresholds are available, those apply too, with the global threshold as the fallback for any class without its own value. To override it on a request, pass the `confidence` parameter. ### Median Latency The evaluation shows the median time the Serverless Cloud API takes to answer one request, measured on your test images. The value includes preprocessing and postprocessing time. It does not include network time. Your model lists and model cards show the same value as "Latency (Cloud API)". Latency from a [Neural Architecture Search](/models/train/neural-architecture-search.md) benchmark is measured a different way, so do not compare the two. If your test images hold at least twice as many pixels as the model input, the evaluation adds a recommendation about capture resolution. The server decodes and resizes every image before it runs the model, so frames whose longer side is close to the model input answer faster. Use [Model Comparison](/models/evaluate/compare-trained-models.md) to compare test-set accuracy and latency for models on the same Dataset Version. ### Optimizing for Inference After training, Roboflow compiles your model into the package the Serverless Cloud API serves. The model shows "Optimizing for inference" while this runs, and the evaluation waits for it to finish. This way the evaluation measures the same package that serves your inference requests. Compilation usually adds a few minutes. If it fails, or takes more than 24 hours, the evaluation starts anyway. ### Supported Project Types Model evaluation supports Object Detection, Instance Segmentation, Keypoint Detection, Classification, and Semantic Segmentation projects. For Keypoint Detection, mAP uses Object Keypoint Similarity (OKS) to match predicted keypoints to labeled ones. The evaluation image viewer draws predicted keypoints, but not labeled keypoints. For Semantic Segmentation, the headline metric is mIoU (mean Intersection-over-Union) instead of mAP. All metrics (precision, recall, F1) are computed at the pixel level rather than per-instance. The per-class breakdown shows IoU, precision, recall, F1, and an optimal confidence threshold for each class. Confusion matrix values represent pixel counts rather than object counts. Newer evaluations of segmentation models also report boundary metrics. Boundary IoU compares narrow bands along each mask edge, so it shows edge errors that mask IoU can hide. Instance Segmentation shows a "Mean Boundary Average Precision" chart under "Show Detailed Metrics". Semantic Segmentation shows mBIoU next to mIoU, and a "Boundary IoU" column in the per-class table. ## Web App Open "Models" in your project, select a trained model, and click "View Full Model Evaluation".
In the production metrics explorer, drag the slider to see precision, recall, and F1 at any confidence threshold.
The model improvement recommendations list what to change to increase accuracy.
The performance by class chart counts correct predictions, misclassifications, false negatives, and false positives for each class. Use the "All Classes" dropdown to focus on specific classes.
In the confusion matrix, click a cell to see the images in it (ex: a cell in the "False Positive" column shows images where the model found an object your annotations do not have). Click an image, then switch between "Ground Truth" and "Model Predictions" to compare your annotations with the model's output.
"Vector Analysis" plots your test images by visual similarity and colors each by its F1 score. Click a point, an area, or one of the "Identified Clusters" to see those images, so you can find groups of similar images where the model struggles. ![Vector Analysis with a selected cluster of two images highlighted on the scatter plot and listed by worst F1](https://3448603185-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcmo9mhfIjYrvFFy1U7dk%2Fuploads%2Fgit-blob-f149c384f7b7dacc4a2816045a6c9e5ab8526dc0%2Fmodel-evaluation-vector-analysis.png?alt=media) The class chart and the confusion matrix start at the optimal confidence. Move their "Confidence Threshold" slider to see how the results, precision, and recall change. Click "Improve My Model" to open a [Roboflow Agent](https://docs.roboflow.com/agents/roboflow-agent) chat about this evaluation. The Agent shows the images where the model fails next to their labels and suggests changes to your data or training. It puts images with likely label errors into one review job, so you can check and fix the labels. It asks you before it moves images out of a teammate's job. ## HTTP API The Model Evaluations API returns everything the evaluation page in the app shows. Each panel has its own endpoint. Roboflow runs an evaluation when a training finishes, and you can run one again from the app. Every endpoint needs an API key with the `model-eval:read` scope. Start with [List Model Evaluations](#list-model-evaluations) to find an `evalId`, then poll [Get a Model Evaluation](#get-a-model-evaluation) until `status` is `done`. The panel endpoints return `409` until then. Every panel response repeats `evalId`, `project`, `versionId`, and `modelId`, so you can link to the evaluation in the app at `https://app.roboflow.com/{workspace}/{project}/evaluation/{versionId}`. To compare several models on one version, see [Compare Trained Models](/models/evaluate/compare-trained-models.md#http-api). ### List Model Evaluations Returns a short record for each evaluation, without metrics. Filter by one of `project`, `version`, or `model`; sending more than one returns `400`. To match an evaluation to its model, compare `modelId` with the model ID you use for inference. {% openapi src="/files/ZbzalMzB8cmnv1kQItaq" path="/{workspace}/model-evals" method="get" %} [model-evaluations.yaml](https://3448603185-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcmo9mhfIjYrvFFy1U7dk%2Fuploads%2Fgit-blob-92404e3372138e59de07c239bc165dac2de6cef0%2Fmodel-evaluations.yaml?alt=media) {% endopenapi %} ### Get a Model Evaluation Returns one evaluation. Once `status` is `done`, the response adds `summary` with the headline metrics: `mAP` for detection tasks, `mIoU` for semantic segmentation. {% openapi src="/files/ZbzalMzB8cmnv1kQItaq" path="/{workspace}/model-evals/{evaluationId}" method="get" %} [model-evaluations.yaml](https://3448603185-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcmo9mhfIjYrvFFy1U7dk%2Fuploads%2Fgit-blob-92404e3372138e59de07c239bc165dac2de6cef0%2Fmodel-evaluations.yaml?alt=media) {% endopenapi %} ### Get mAP Results Returns the main metrics for each split, as shown in the metrics per split panel. Read `taskType` first: it tells you whether the splits hold mAP fields or mIoU and pixel-level fields. {% openapi src="/files/ZbzalMzB8cmnv1kQItaq" path="/{workspace}/model-evals/{evaluationId}/map-results" method="get" %} [model-evaluations.yaml](https://3448603185-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcmo9mhfIjYrvFFy1U7dk%2Fuploads%2Fgit-blob-92404e3372138e59de07c239bc165dac2de6cef0%2Fmodel-evaluations.yaml?alt=media) {% endopenapi %} ### Get a Confidence Sweep Returns the curves behind the production metrics explorer: precision, recall, and F1 at each confidence threshold, and the threshold with the best F1. Use it to choose the confidence threshold to deploy with. {% openapi src="/files/ZbzalMzB8cmnv1kQItaq" path="/{workspace}/model-evals/{evaluationId}/confidence-sweep" method="get" %} [model-evaluations.yaml](https://3448603185-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcmo9mhfIjYrvFFy1U7dk%2Fuploads%2Fgit-blob-92404e3372138e59de07c239bc165dac2de6cef0%2Fmodel-evaluations.yaml?alt=media) {% endopenapi %} ### Get Performance by Class Returns the performance by class panel for one split. Each class reports precision, recall, and F1 at its own best threshold. {% openapi src="/files/ZbzalMzB8cmnv1kQItaq" path="/{workspace}/model-evals/{evaluationId}/performance-by-class" method="get" %} [model-evaluations.yaml](https://3448603185-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcmo9mhfIjYrvFFy1U7dk%2Fuploads%2Fgit-blob-92404e3372138e59de07c239bc165dac2de6cef0%2Fmodel-evaluations.yaml?alt=media) {% endopenapi %} ### Get a Confusion Matrix Returns the confusion matrix panel. Each confidence threshold gives a different matrix, so pass `confidence` to match the threshold you deploy with. {% openapi src="/files/ZbzalMzB8cmnv1kQItaq" path="/{workspace}/model-evals/{evaluationId}/confusion-matrix" method="get" %} [model-evaluations.yaml](https://3448603185-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcmo9mhfIjYrvFFy1U7dk%2Fuploads%2Fgit-blob-92404e3372138e59de07c239bc165dac2de6cef0%2Fmodel-evaluations.yaml?alt=media) {% endopenapi %} ### Get Vector Analysis Returns the vector analysis panel: images grouped by visual similarity, with the model's average F1, precision, and recall on each group. Look for clusters with a low `f1Mean` to find kinds of images where the model struggles. To see which images are in a cluster, use [Get Image Predictions](#get-image-predictions). {% openapi src="/files/ZbzalMzB8cmnv1kQItaq" path="/{workspace}/model-evals/{evaluationId}/vector-analysis" method="get" %} [model-evaluations.yaml](https://3448603185-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcmo9mhfIjYrvFFy1U7dk%2Fuploads%2Fgit-blob-92404e3372138e59de07c239bc165dac2de6cef0%2Fmodel-evaluations.yaml?alt=media) {% endopenapi %} ### Get Image Predictions Returns one record per evaluated image, with its counts, metrics, cluster, and confusion entries. Results are paged with `limit` and `offset`. To inspect one image, pass `imageName` or `evalInputId` with `includeGeometry=true`. The record then includes the saved predictions and ground truth, so you can tell whether a failure is a model error or a label error. {% openapi src="/files/ZbzalMzB8cmnv1kQItaq" path="/{workspace}/model-evals/{evaluationId}/image-predictions" method="get" %} [model-evaluations.yaml](https://3448603185-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcmo9mhfIjYrvFFy1U7dk%2Fuploads%2Fgit-blob-92404e3372138e59de07c239bc165dac2de6cef0%2Fmodel-evaluations.yaml?alt=media) {% endopenapi %} ### Get Recommendations Returns the model improvement recommendations panel. Roboflow generates recommendations when a training finishes, and this endpoint only reads them. A finished evaluation without recommendations returns `200` with `generated: false`, not an error. {% openapi src="/files/ZbzalMzB8cmnv1kQItaq" path="/{workspace}/model-evals/{evaluationId}/recommendations" method="get" %} [model-evaluations.yaml](https://3448603185-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcmo9mhfIjYrvFFy1U7dk%2Fuploads%2Fgit-blob-92404e3372138e59de07c239bc165dac2de6cef0%2Fmodel-evaluations.yaml?alt=media) {% endopenapi %} ## MCP Server Connect your AI agent to the [MCP Server](https://docs.roboflow.com/agents/mcp-server) and it can review how a model performed with these tools:
ToolDescription
model_evals_listList model evaluations in the workspace.
model_evals_getGet the top-level summary for one evaluation.
model_evals_get_map_resultsGet per-split mAP results.
model_evals_get_confusion_matrixGet the confusion matrix.
model_evals_get_performance_by_classGet per-class performance metrics for one split.
model_evals_get_recommendationsGet generated recommendations for the evaluation, if available.