> For the complete documentation index, see [llms.txt](https://docs.roboflow.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.roboflow.com/models/supported-tasks/multimodal.md).

# Multimodal

Ask questions about images in plain text and get text answers back, then fine-tune and deploy your own vision-language model on Roboflow.

## What is a multimodal model?

Multimodal models, also called vision-language models (VLMs), take an image and a text prompt and answer in text. The same model can caption an image, answer a question about it (visual question answering), or follow an open prompt (ex: "List every safety issue in this photo"). Some can also return structured output such as boxes or JSON. Use them when the question changes often or is hard to turn into fixed classes.

<figure><img src="https://3448603185-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcmo9mhfIjYrvFFy1U7dk%2Fuploads%2Fgit-blob-0ca7c7502d3b4b2cb7f2078b859080f499b33902%2Fimg-task-multimodal-wide.webp?alt=media" alt="Oranges on a conveyor with the generated caption: Oranges on a conveyor."><figcaption></figcaption></figure>

## Supported multimodal models

Roboflow supports these multimodal models. See the [Multimodal models](https://docs.roboflow.com/models/supported-models.md#multimodal) table for export and deployment support per model.

<table data-search="false"><thead><tr><th>Model</th><th>License</th><th data-type="checkbox">Trainable on Roboflow</th><th>Latest release</th></tr></thead><tbody><tr><td><a href="https://docs.roboflow.com/models/supported-models/cosmos-3-edge.md">Cosmos 3 Edge</a></td><td><a href="https://docs.roboflow.com/models/model-licenses.md#other-models">Upstream license</a></td><td>true</td><td>2026</td></tr><tr><td><a href="https://docs.roboflow.com/models/supported-models/qwen3-5.md">Qwen3.5</a></td><td><a href="https://docs.roboflow.com/models/model-licenses.md#other-models">Upstream license</a></td><td>true</td><td>2026</td></tr><tr><td><a href="https://docs.roboflow.com/models/supported-models/qwen3-vl.md">Qwen3-VL</a></td><td><a href="https://docs.roboflow.com/models/model-licenses.md#other-models">Upstream license</a></td><td>true</td><td>2025</td></tr><tr><td><a href="https://docs.roboflow.com/models/supported-models/perception-encoder.md">Perception Encoder</a></td><td><a href="https://docs.roboflow.com/models/model-licenses.md#other-models">Upstream license</a></td><td>false</td><td>2025</td></tr><tr><td><a href="https://docs.roboflow.com/models/supported-models/smolvlm2.md">SmolVLM2</a></td><td><a href="https://docs.roboflow.com/models/model-licenses.md#other-models">Upstream license</a></td><td>true</td><td>2025</td></tr><tr><td>Qwen2.5-VL</td><td><a href="https://docs.roboflow.com/models/model-licenses.md#other-models">Upstream license</a></td><td>true</td><td>2025</td></tr><tr><td>SmolVLM 256M</td><td><a href="https://docs.roboflow.com/models/model-licenses.md#other-models">Upstream license</a></td><td>true</td><td>2025</td></tr><tr><td><a href="https://docs.roboflow.com/models/supported-models/paligemma2.md">PaliGemma 2</a></td><td><a href="https://docs.roboflow.com/models/model-licenses.md#other-models">Upstream license</a></td><td>true</td><td>2024</td></tr><tr><td><a href="https://docs.roboflow.com/models/supported-models/florence-2.md">Florence 2</a></td><td><a href="https://docs.roboflow.com/models/model-licenses.md#other-models">Upstream license</a></td><td>true</td><td>2024</td></tr><tr><td><a href="https://docs.roboflow.com/models/supported-models/moondream2.md">Moondream2</a></td><td><a href="https://docs.roboflow.com/models/model-licenses.md#other-models">Upstream license</a></td><td>false</td><td>2025</td></tr><tr><td><a href="https://docs.roboflow.com/models/supported-models/yolo-world.md">YOLO-World</a></td><td><a href="https://docs.roboflow.com/models/model-licenses.md">AGPL-3.0</a> (commercial use needs a license, included on Roboflow cloud)</td><td>false</td><td>2024</td></tr><tr><td><a href="https://docs.roboflow.com/models/supported-models/clip.md">CLIP</a></td><td><a href="https://docs.roboflow.com/models/model-licenses.md#other-models">Upstream license</a></td><td>false</td><td>2021</td></tr></tbody></table>

## Fine-tuning a multimodal model

You can fine-tune some multimodal models on Roboflow (ex: Qwen3-VL, Florence 2, PaliGemma 2). The [Multimodal models](https://docs.roboflow.com/models/supported-models.md#multimodal) table shows which ones.

{% stepper %}
{% step %}

### Create a multimodal project

Create a [Project](https://docs.roboflow.com/datasets/create-and-upload/create-a-project) and pick "Multimodal" as the project type.

<figure><img src="https://3448603185-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcmo9mhfIjYrvFFy1U7dk%2Fuploads%2Fgit-blob-5c1c2973605b125b6f909401c24cf657c49a9081%2Fcreate-project-multimodal.webp?alt=media" alt="Create project form with Multimodal selected as the project type and a preview of a multimodal example"><figcaption></figcaption></figure>
{% endstep %}

{% step %}

### Upload and annotate images

[Upload images](https://docs.roboflow.com/datasets/create-and-upload/adding-data), [add prefixes](https://docs.roboflow.com/datasets/annotate/annotate/annotate-multimodal-data) (the prompts your model will answer), then write the answer (suffix) for each prefix on every image in the [annotation editor](https://docs.roboflow.com/datasets/annotate/annotate/annotation-tools). [Label Assist](https://docs.roboflow.com/datasets/annotate/annotate/ai-labeling/model-assisted-labeling) can draft answers from a model for you to review.

<figure><img src="https://3448603185-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcmo9mhfIjYrvFFy1U7dk%2Fuploads%2Fgit-blob-7aaed09426c0bd40ceb0549ec6b3d8c2503e11bc%2Fannotate-multimodal.webp?alt=media" alt="Annotation editor on a pallet manifest image with the prefix JSON and a JSON suffix listing route, pallet number, and products"><figcaption></figcaption></figure>
{% endstep %}

{% step %}

### Generate a dataset version

[Generate a dataset version](https://docs.roboflow.com/datasets/versions/dataset-versions/create-a-dataset-version): a frozen snapshot of your images with a train, valid, and test split, plus any preprocessing and augmentation.
{% endstep %}

{% step %}

### Train a multimodal model

Click "Train" and pick a model that supports fine-tuning. See [Train a Model](https://docs.roboflow.com/models/train/train-a-model.md) for every option.

<figure><img src="https://3448603185-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fcmo9mhfIjYrvFFy1U7dk%2Fuploads%2Fgit-blob-b1e7f5d64887c2d76bddbac9c087e42c5b06c21d%2Ftrain-architectures-multimodal.webp?alt=media" alt="Select Architecture step on the Train page listing Qwen3.5 (Recommended), Qwen3 VL, SmolVLM, Florence 2, Cosmos 3, PaliGemma 2, and Qwen2.5 VL"><figcaption></figcaption></figure>
{% endstep %}
{% endstepper %}

## Deploy a multimodal model

After training, deploy your multimodal model with the [Serverless Cloud API](https://docs.roboflow.com/deployment/roboflow-cloud/serverless-api), a [Dedicated Deployment](https://docs.roboflow.com/deployment/roboflow-cloud/dedicated-deployments), or on your own hardware with [Roboflow Inference](https://docs.roboflow.com/deployment/self-hosted/self-hosted). See [Choosing a Deployment](https://docs.roboflow.com/deployment/choosing-a-deployment) to compare the options.
