> For the complete documentation index, see [llms.txt](https://docs.roboflow.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.roboflow.com/models/supported-models/perception-encoder.md).

# Perception Encoder

Perception Encoder is Meta's vision-language embedding model. It maps images and text into a shared embedding space for similarity search, zero-shot classification, and retrieval.

{% hint style="info" %}
Perception Encoder is not available on the Serverless Cloud API. Run it on a [Dedicated Deployment](https://docs.roboflow.com/deployment/roboflow-cloud/dedicated-deployments) or [self-hosted Inference](https://docs.roboflow.com/deployment/self-hosted/self-hosted).
{% endhint %}

We support three Perception Encoder endpoints:

* `/perception_encoder/embed_image` - embed an image
* `/perception_encoder/embed_text` - embed a string
* `/perception_encoder/compare` - compute similarity between an image and a list of text prompts

## Perception Encoder API

{% stepper %}
{% step %}

### Get your API Key

Create a Roboflow account, find your key on the [Roboflow API settings page](https://app.roboflow.com/settings/api) and make it available to your shell:

```bash
export ROBOFLOW_API_KEY="your-key-here"
```

{% endstep %}

{% step %}

### Install the dependencies

These packages fetch the image and call the API:

```bash
pip install -U requests opencv-python supervision
```

{% endstep %}

{% step %}

### Run the model

The sample below sends an image to `/perception_encoder/embed_image` and prints the embedding shape. Set `URL` to your Dedicated Deployment URL or a local Inference server.

```python
import base64
import os
import cv2
import requests
import supervision as sv

URL = "https://your-deployment.roboflow.cloud"

image = sv.load_image_from_url("https://media.roboflow.com/notebooks/examples/dog.jpeg")

_, buffer = cv2.imencode(".jpg", image)
image_base64 = base64.b64encode(buffer).decode("utf-8")

response = requests.post(
    f"{URL}/perception_encoder/embed_image",
    json={
        "api_key": os.environ["ROBOFLOW_API_KEY"],
        "image": {"type": "base64", "value": image_base64},
    },
)
result = response.json()
embedding = result["embeddings"][0]
print(f"Embedding length: {len(embedding)}")
print(f"First values: {embedding[:5]}")
```

{% endstep %}
{% endstepper %}

The code above prints the embedding shape to the terminal:

```
Embedding length: 1024
First values: [0.0545, -0.0338, -0.0355, -0.0062, 0.0154]
```

## Perception Encoder inference speed

Latency measured with [Roboflow Inference](https://docs.roboflow.com/deployment/self-hosted/self-hosted) on 1x NVIDIA L4, batch size 1, mean after warmup.

<table data-search="false"><thead><tr><th>Model</th><th>Latency (ms)</th></tr></thead><tbody><tr><td><code>perception-encoder</code></td><td>25.2</td></tr></tbody></table>

Measured with `embed_image` on the `PE-Core-L14-336` checkpoint (image embedding only).

{% hint style="info" %}
Set `URL` to match your deployment target:

* `http://localhost:9001` for a local [Inference](https://docs.roboflow.com/deployment/self-hosted/self-hosted) server.
* Your [Dedicated Deployment](https://docs.roboflow.com/deployment/roboflow-cloud/dedicated-deployments) URL for a private endpoint.
  {% endhint %}

## Run Perception Encoder with self-hosted Inference

Perception Encoder can be loaded directly with the [`inference`](https://docs.roboflow.com/deployment/self-hosted/self-hosted) package, which is the fastest path when embedding many images or video frames locally.

{% stepper %}
{% step %}

### Install the package

```bash
pip install "inference[transformers]"
```

Use `inference-gpu[transformers]` on a GPU machine.
{% endstep %}

{% step %}

### Embed and compare locally

```python
from inference.core.utils.postprocess import cosine_similarity
from inference.models import PerceptionEncoder

pe = PerceptionEncoder(model_id="perception_encoder/PE-Core-B16-224")

image_embedding = pe.embed_image("https://media.roboflow.com/inference/people-walking.jpg")
text_embedding = pe.embed_text("a crowd of people walking")

print(cosine_similarity(image_embedding[0], text_embedding[0]))
```

{% endstep %}
{% endstepper %}

### Available checkpoints

`model_id` selects the backbone:

* `perception_encoder/PE-Core-B16-224`
* `perception_encoder/PE-Core-L14-336`
* `perception_encoder/PE-Core-G14-448`

Only the CLIP-style interface is supported; the language-aligned and spatially-aligned Perception Encoder variants are not available yet.

{% hint style="info" %}
Perception Encoder uses the same API shape as [CLIP](/models/supported-models/clip.md): `embed_image`, `embed_text`, and `compare` take the same arguments and return the same response format, so code written against CLIP works with Perception Encoder by changing the model.
{% endhint %}

### Use in Workflows

Perception Encoder is available in [Workflows](https://docs.roboflow.com/workflows) through the **Perception Encoder Embedding Model** block, which generates image or text embeddings without writing code.