> For the complete documentation index, see [llms.txt](https://docs.roboflow.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.roboflow.com/deployment/self-hosted/inference-library.md).

# Inference Library

The `inference` Python package loads models and runs them inside your own process. There is no container to start and no HTTP request between your code and the model, which makes it the lowest-latency way to self-host and the simplest to embed in an existing Python application.

Use it when your application is Python and runs on the same machine as the model. If several clients, languages, or video streams need predictions, or you want models isolated from your application's dependencies, run the [Inference Server](/deployment/self-hosted/inference-server.md) instead. Both accept the same `model_id` values, so switching later is a small change.

## Install

```bash
pip install inference
```

If you have an NVIDIA GPU, install `inference-gpu` instead:

```bash
pip install --extra-index-url https://download.pytorch.org/whl/cu124 inference-gpu
```

Match `--extra-index-url` to the CUDA version installed in your OS: `https://download.pytorch.org/whl/cu<major><minor>`, for instance `https://download.pytorch.org/whl/cu130` for CUDA 13.0. GPU installation requires CUDA in the OS. See the [Linux](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/) or [Windows](https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/) CUDA installation guide if your environment lacks the dependencies.

Starting with `inference` 1.2.0, the new inference engine (`inference-models`) is the default. It supports several model backends, including TensorRT, and picks the fastest one available for your hardware. `inference` installs what `torch` and `onnx` models need; other backends come from package extras:

```bash
pip install inference-models[trt10]
```

On Windows, CUDA setup has extra steps: see [Install Bare Metal Inference GPU on Windows](/deployment/self-hosted/inference-library/bare-metal-gpu-windows.md).

## Run a model

```python
from inference import get_model

image = "https://media.roboflow.com/inference/people-walking.jpg"
model = get_model(model_id="rfdetr-small")
results = model.infer(image)
```

`get_model()` downloads and caches the model weights on first use, then runs inference locally. The `model_id` can be a pre-trained alias, your own fine-tuned model, or a Universe model: see [model IDs](/deployment/self-hosted/self-hosted.md#model-ids). Fine-tuned and Universe models require an [API key](https://docs.roboflow.com/reference/authentication/authentication/find-your-roboflow-api-key).

## Visualize results

Install [Supervision](https://supervision.roboflow.com) to annotate predictions:

```bash
pip install -U supervision
```

```python
import supervision as sv
from inference import get_model

image = sv.load_image_from_url("https://media.roboflow.com/inference/people-walking.jpg")

model = get_model(model_id="rfdetr-medium")
results = model.infer(image)[0]

detections = sv.Detections.from_inference(results)

annotated_image = sv.BoxAnnotator().annotate(scene=image, detections=detections)
annotated_image = sv.LabelAnnotator().annotate(scene=annotated_image, detections=detections)

sv.plot_image(annotated_image)
```

## Video and Workflows

For most video applications, run an [Inference Server](/deployment/self-hosted/inference-server.md) and stream a model or Workflow to it with the [Inference SDK WebRTC client](https://docs.roboflow.com/reference/inference/inference-sdk/webrtc).

If you intentionally run the Inference Library inside your Python process, `InferencePipeline` can process a webcam, RTSP camera, or video file without a server. This direct-library API gives your process access to frames, custom inference logic, and sinks. See [Inference Pipeline](https://docs.roboflow.com/reference/inference/inference-python/inference-pipeline).

## Going further

* [Inference Python Package reference](https://docs.roboflow.com/reference/inference/inference-python) - the full API surface.
* [Native Python API](https://docs.roboflow.com/reference/inference/inference-python/native-python-api) - load models and run Workflows without the server.
* [Model Weights Download](https://docs.roboflow.com/reference/inference/inference-python/offline-weights) - cache weights for offline and air-gapped hosts.
* [Inference Benchmarks](https://docs.roboflow.com/reference/inference/inference-python/benchmarks) - measured throughput by model and hardware.