> For the complete documentation index, see [llms.txt](https://docs.roboflow.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.roboflow.com/reference/inference/inference-cli/benchmark.md).

# Benchmark Inference

`inference benchmark` offers an easy way to check the performance of Inference in your setup. The command can benchmark both the Inference Server and the `inference` Python package.

{% hint style="success" %}
To see the details of the command, run:

```bash
inference benchmark --help
```

A help guide is also available for each sub-command:

```bash
inference benchmark api-speed --help
```

{% endhint %}

## Benchmarking the `inference` Python package

{% hint style="warning" %}
Running this command requires the `inference` package to be installed.

```bash
pip install inference
```

{% endhint %}

A basic benchmark can be run using the following command:

```bash
inference benchmark python-package-speed \
  -m {your_model_id} \
  -d {pre-configured dataset name or path to directory with images} \
  -o {output_directory}
```

The command runs a specified number of inferences using the given model and saves statistics (including benchmark parameters, throughput, latency, errors, and platform details) in the given directory.

## Benchmarking the Inference Server

{% hint style="info" %}
Before running an API benchmark of your local Inference Server, make sure the server is up and running:

```bash
inference server start
```

{% endhint %}

A basic benchmark can be run using the following command:

```bash
inference benchmark api-speed \
  -m {your_model_id} \
  -d {pre-configured dataset name or path to directory with images} \
  -o {output_directory}
```

The command runs a specified number of inferences using the given model and saves statistics (including benchmark parameters, throughput, latency, errors, and platform details) in the given directory.

This benchmark has more configuration options to support different ways of HTTP API profiling. In default mode, a single client is spawned and it sends one request after another sequentially. This may be suboptimal in specific cases, so you may specify the number of concurrent clients using the `-c {number_of_clients}` option. Each client sends the next request once the previous one is handled.

This option still does not cover all test scenarios. For instance, you may want to send `x` requests each second, which is closer to a production environment where multiple clients send requests concurrently. In this scenario, the `--rps {value}` option can be used (and `-c` is ignored). The value provided in `--rps` specifies how many requests are spawned **each second** without waiting for previous requests to be handled. In I/O intensive benchmark scenarios, we suggest running the command from multiple separate processes and possibly multiple hosts.

See [Inference Benchmarks](/reference/inference/inference-python/benchmarks.md) for published results measured with this command.