SAM3

Use Meta's SAM3 model through our Serverless Hosted API

We support Meta's Segment Anything Model 3 inferencing via our Serverless Hosted API. We offer two different SAM3 endpoints:

• Promptable concept segmentation (PCS), segmentation using text prompts

• [WIP] Promptable visual segmentation (PVS), interactive segmentation using points/boxes

Code sample

Below is a code sample that uses PCS endpoint for SAM3 inferencing. User needs to pass Roboflow's API Key via API_KEY env variable.

import os
import requests
import base64
import cv2
import numpy as np

# From "https://media.roboflow.com/notebooks/examples/dog.jpeg"
image = cv2.imread("./dog.jpeg")

# Encode image as base64
_, buffer = cv2.imencode('.jpg', image)
image_base64 = base64.b64encode(buffer).decode('utf-8')

payload = {
    "image": { "type": "base64", "value": image_base64 },
    "prompts": [
        { "type": "text", "text": "person" },
        { "type": "text", "text": "dog" },
    ],
    "output_prob_thresh": 0.5,
    "format": "polygon",
}

url = "https://serverless.roboflow.com/sam3/concept_segment?api_key=" + os.getenv("API_KEY")
response = requests.post(url, json=payload)
data = response.json()

for key in data:
    print(key) # Should be prompt_results and time

Endpoints

SAM3 PCS (promptable concept segmentation)

post

Concept Segmentation (Text Prompts)

Allows you to segment objects using text prompts.

Image Input: The image field accepts either:

• {"type": "url", "value": "<IMAGE_URL>"} - A publicly accessible image URL

• {"type": "base64", "value": "<BASE64_DATA>"} - Base64 encoded image data

Prompts: Each prompt in the prompts array should have type: "text" and a text field with the object description.

Query parameters

api_keystringRequired

Your Roboflow API Key. Get one at https://app.roboflow.com/settings/api

Body

application/json

formatstringOptional

One of 'polygon', 'rle'

Default: polygon

image_idstringOptional

Optional ID for caching embeddings.

output_prob_threshnumberOptional

Score threshold for outputs.

Default: 0.5

model_idstringOptional

The model ID of SAM3. Use 'sam3/sam3_final' to target the generic base model.

Default: sam3/sam3_final

nms_iou_thresholdnumberOptional

IoU threshold for cross-prompt NMS. If not set, NMS is disabled. Must be in [0.0, 1.0] when set.

Responses

200

Successful Response

application/json

422

Validation Error

application/json

post

/sam3/concept_segment

HTTPcURLJavaScriptPython

POST /sam3/concept_segment?api_key=text HTTP/1.1
Host: serverless.roboflow.com
Content-Type: application/json
Accept: */*
Content-Length: 206

{
  "image": {
    "type": "url",
    "value": "https://media.roboflow.com/notebooks/examples/dog.jpeg"
  },
  "prompts": [
    {
      "type": "text",
      "text": "person"
    },
    {
      "type": "text",
      "text": "car"
    }
  ],
  "output_prob_thresh": 0.5,
  "format": "polygon"
}

{
  "prompt_results": [
    {
      "prompt_index": 0,
      "echo": {
        "prompt_index": 0,
        "type": "text",
        "text": "dog",
        "num_boxes": 0
      },
      "predictions": [
        {
          "masks": [
            [
              [
                345,
                251
              ],
              [
                344,
                252
              ],
              [
                343,
                253
              ]
            ]
          ],
          "confidence": 0.89453125,
          "format": "polygon"
        }
      ]
    }
  ],
  "time": 0.221
}