> For the complete documentation index, see [llms.txt](https://docs.roboflow.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.roboflow.com/models/supported-models/smolvlm2.md).

# SmolVLM2

SmolVLM2 is a compact vision-language model from HuggingFace. It accepts an image and a text prompt and returns a text response.

{% hint style="info" %}
SmolVLM2 is not available on the Serverless Cloud API. Run it on a [Dedicated Deployment](https://docs.roboflow.com/deployment/roboflow-cloud/dedicated-deployments) or [self-hosted Inference](https://docs.roboflow.com/deployment/self-hosted/self-hosted).
{% endhint %}

## SmolVLM2 API

{% stepper %}
{% step %}

### Get your API Key

Create a Roboflow account, find your key on the [Roboflow API settings page](https://app.roboflow.com/settings/api) and make it available to your shell:

```bash
export ROBOFLOW_API_KEY="your-key-here"
```

{% endstep %}

{% step %}

### Install the dependencies

Install the [Inference SDK](https://docs.roboflow.com/deployment/self-hosted/self-hosted):

```bash
pip install -U inference-sdk supervision
```

{% endstep %}

{% step %}

### Run the model

Set `api_url` to your Dedicated Deployment URL or a local Inference server.

```python
import os
import supervision as sv
from inference_sdk import InferenceHTTPClient

image = sv.load_image_from_url("https://media.roboflow.com/quickstart/dog.jpeg")

client = InferenceHTTPClient(
    api_url="https://your-deployment.roboflow.cloud",
    api_key=os.environ["ROBOFLOW_API_KEY"],
)
result = client.infer_lmm(
    image,
    model_id="smolvlm2",
    prompt="Describe this image briefly.",
    max_new_tokens=64,
)
print(result["response"])
```

{% endstep %}
{% endstepper %}

The code above prints the model response to the terminal:

```
A man is carrying a dog on his shoulders.
```

<figure><img src="/files/bDjHImEVZHKFc9Zvo1Vu" alt=""><figcaption></figcaption></figure>

## SmolVLM2 inference speed

Latency measured with [Roboflow Inference](https://docs.roboflow.com/deployment/self-hosted/self-hosted) on 1x NVIDIA L4, batch size 1, generating exactly 128 tokens with greedy decoding from a fixed prompt. Latency scales with output length, so use tokens/sec to estimate other lengths.

<table data-search="false"><thead><tr><th>Alias</th><th>Latency, 128 tokens (ms)</th><th>Tokens/sec</th></tr></thead><tbody><tr><td><code>smolvlm2</code></td><td>3113</td><td>41</td></tr></tbody></table>

{% hint style="info" %}
Set `api_url` to match your deployment target:

* `http://localhost:9001` for a local [Inference](https://docs.roboflow.com/deployment/self-hosted/self-hosted) server.
* Your [Dedicated Deployment](https://docs.roboflow.com/deployment/roboflow-cloud/dedicated-deployments) URL for a private endpoint.
  {% endhint %}

## Run SmolVLM2 with self-hosted Inference

SmolVLM2 can also be loaded directly with the [`inference`](https://docs.roboflow.com/deployment/self-hosted/self-hosted) package for VQA, document OCR, document VQA, and object counting.

{% stepper %}
{% step %}

### Install the package

```bash
pip install "inference[transformers]"
```

Use `inference-gpu[transformers]` on a GPU machine.
{% endstep %}

{% step %}

### Run the model

```python
from PIL import Image

from inference.models.smolvlm.smolvlm import SmolVLM

model = SmolVLM(api_key="YOUR_API_KEY")

image = Image.open("dog.jpeg")
result = model.predict(image, "How many dogs are in this image?")

print(result)
```

{% endstep %}
{% endstepper %}

### Execution modes in Workflows

When used in a [Workflow](https://docs.roboflow.com/workflows), SmolVLM2 runs in one of two modes:

* **Local execution**: the model runs on your Inference server (GPU recommended).
* **Remote execution**: the model is invoked over HTTP on a remote Inference server through the `infer_lmm()` client method.