> For the complete documentation index, see [llms.txt](https://docs.roboflow.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.roboflow.com/deployment/self-hosted/enterprise/parallel-http-api.md).

# Parallel HTTP API

{% hint style="warning" %}
**Enterprise feature.** This feature is only available to Roboflow Enterprise users. [Contact the sales team](https://roboflow.com/sales) to learn more about Roboflow Enterprise.
{% endhint %}

You can run multiple models in parallel with parallel processing, a version of Roboflow Inference that processes inference requests asynchronously.

Inference Parallel supports the same features as Roboflow Inference, except that it does not support core models (CLIP and SAM).

With Inference Parallel, preprocessing, auto batching, inference, and post-processing all run in separate threads to increase server FPS throughput. Separate requests to the same model are batched on the fly as allowed by `$MAX_BATCH_SIZE`, and response handling then occurs independently. Images are passed via Python's `SharedMemory` module to maximize throughput.

These changes result in as much as a **76% speedup** on one measured workload.

## How to run Inference with parallel processing

{% tabs %}
{% tab title="Bash" %}
First, build the parallel server:

```bash
./inference/enterprise/parallel/build.sh
```

Then run the server:

```bash
./inference/enterprise/parallel/run.sh
```

A message appears in the terminal indicating that the server is running and ready for use.
{% endtab %}

{% tab title="Docker" %}
A container is published on Docker Hub:

```bash
docker pull roboflow/roboflow-inference-server-gpu-parallel:latest
```

If you are pulling a pinned tag, change the `$TAG` variable in `run.sh` to match.
{% endtab %}
{% endtabs %}

## Benchmarks

We evaluated the performance of Inference Parallel on a variety of models from [Roboflow Universe](https://universe.roboflow.com/), comparing it against Inference Server 0.9.5.rc on the same hardware: a computer with eight cores and one GPU. Instance segmentation metrics were calculated with `"mask_decode_mode": "fast"` in the request body. Requests were posted concurrently with a parallelism of 1000.

| Workspace                   | Model                      | Model type            | Split | 0.9.5.rc FPS | 0.9.5.parallel FPS |
| --------------------------- | -------------------------- | --------------------- | ----- | ------------ | ------------------ |
| senior-design-project-j9gpp | nbafootage/3               | object-detection      | train | 30.2 fps     | 44.03 fps          |
| niklas-bommersbach-jyjff    | dart-scorer/8              | object-detection      | train | 26.6 fps     | 47.0 fps           |
| geonu                       | water-08xpr/1              | instance-segmentation | valid | 4.7 fps      | 6.1 fps            |
| university-of-bradford      | detecting-drusen\_1/2      | instance-segmentation | train | 6.2 fps      | 7.2 fps            |
| fy-project-y9ecd            | cataract-detection-viwsu/2 | classification        | train | 48.5 fps     | 65.4 fps           |
| hesunyu                     | playing-cards-ir0wr/1      | classification        | train | 44.6 fps     | 57.7 fps           |

Inference with parallel processing enabled achieved higher FPS on every test. On some models the FPS increase was greater than 10 FPS.