> For the complete documentation index, see [llms.txt](https://docs.roboflow.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.roboflow.com/models/evaluate/compare-trained-models.md).

# Compare Trained Models

Compare test-set accuracy and median latency for models on the same Dataset Version.

Compare trained models to find accuracy and speed tradeoffs. The comparison includes models trained on or uploaded to the same Dataset Version.

Comparison requests are read-only. They do not start evaluations.

{% hint style="info" %}
Model Comparison requires Model Evaluation access in your Workspace. Without access, the "Compare" tab shows an upgrade prompt and comparison requests return an error.
{% endhint %}

## About

Each model row contains its metrics, median latency, frontier status, and exclusion reason. Models without evaluations remain in the response.

The comparison uses the latest completed evaluation for each model. It does not fall back to an older eligible evaluation.

The default frontier metric depends on the Project type:

* Object Detection and Instance Segmentation: mAP\@50:95 (`mAP5095`).
* Semantic Segmentation: mIoU (`mIoU`).
* Classification: F1 (`f1`).

Use another value from `availableMetrics` to compare a different metric. API accuracy values are ratios. The CLI displays percentages.

Latency is the median server-side inference time in milliseconds. It includes preprocessing and postprocessing. It excludes network time.

A model is on the Pareto frontier if no model is at least as accurate and as fast, with one strict improvement. Identical measurements can put multiple models on the frontier. Frontier membership is not a recommendation.

## Web App

Open "Compare" under "Models" in the Project sidebar. Use the "Dataset Version" list to select a version.

The chart includes models that have all supported metrics and latency. Select a point to open its model card.

Use the metric selector to change the chart metric and frontier. Detection Projects can include mAP\@50, mAP\@50:95, mAP\@75, Precision, Recall, and F1.

The "Some models need evaluation updates" panel lists models with missing comparison results and explains what each model needs.

* Click "Run Evaluation" for a model that has never been evaluated.
* Click "Update Evaluation" when an evaluation lacks latency or an accuracy metric.
* Click "Retry Evaluation" after a failed run.

Click "Update All" to evaluate all eligible models in the panel. Evaluations already queued or running are excluded.

Before any model has a comparison result, the tab shows the next step instead of the chart (ex: "Train a Model", "View Training", "Run Evaluations", "Update Evaluations"). "Run Evaluations" and "Update Evaluations" replace "Update All" there.

A NAS child without an evaluation links to its NAS training. A NAS child with an existing evaluation can use the evaluation retry flow.

## HTTP API

Read the comparison for one Dataset Version. Requests do not start or update evaluations.

{% openapi src="/files/pqJe3E3GTZ8N5Bx9otyI" path="/{workspace}/model-evals/compare" method="get" %}
[model-comparison.yaml](https://3448603185-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcmo9mhfIjYrvFFy1U7dk%2Fuploads%2Fgit-blob-bf74af8396d73a772ae7e4197559f14286963bad%2Fmodel-comparison.yaml?alt=media)
{% endopenapi %}

## Python SDK

Use Python SDK version 1.5.1 or later to get the same comparison response:

```python
comparison = workspace.compare_model_evaluations(
    "chess-pieces",
    131,
    frontier_metric="mAP5095",
)
```

Use each row's `modelId` as the stable model reference. Use the server's `onFrontier` value instead of rebuilding the frontier.

## CLI

Use `roboflow` version 1.5.1 or later to compare a Project's Dataset Version:

```bash
roboflow --workspace my-workspace eval compare \
  --project chess-pieces \
  --version 131 \
  --frontier-metric mAP5095
```

Add `--json` to return the complete response without display rounding. The key needs the `model-eval:read` scope and Project access.

## MCP

Call the `model_evals_compare` tool with a Project slug, Version number, and optional frontier metric:

```json
{
  "project_id": "chess-pieces",
  "version_number": 131,
  "frontier_metric": "mAP5095"
}
```

The tool uses the current Workspace. Use the returned `onFrontier` values to explain tradeoffs. The tool does not select a model.