> For the complete documentation index, see [llms.txt](https://docs.roboflow.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.roboflow.com/models/supported-models/sam3.md). # SAM3 We support Meta's [Segment Anything Model 3](https://github.com/facebookresearch/sam3) inferencing via our [Serverless Cloud API](https://docs.roboflow.com/deployment/roboflow-cloud/serverless-api). We offer two different SAM3 endpoints: {% hint style="info" %} Training a SAM3 model on Roboflow is available on paid [plans](https://docs.roboflow.com/platform/billing-and-plans/plans) that include [usage-based billing](https://docs.roboflow.com/platform/billing-and-plans/credits). From there, you can request access with the "Request Feature" button on the SAM3 architecture to use the feature [training flow](/models/readme.md). Fine-tuned SAM3 models cannot run on the Serverless Cloud API. Deploy them on a [Dedicated Deployment](https://docs.roboflow.com/deployment/roboflow-cloud/dedicated-deployments) or [self-hosted Inference](https://docs.roboflow.com/deployment/self-hosted/self-hosted). The hosted `sam3` endpoints on this page are unaffected. {% endhint %} * [Promptable concept segmentation](#sam3-concept-segmentation-pcs) (**PCS**), which segments every instance of a concept in the image. Concepts are described by text prompts, exemplar boxes, or both. * [Promptable visual segmentation](#sam3-visual-segmentation-pvs) (**PVS**), which interactively segments one object per request from points or a box, in the style of SAM2. Use this table to pick an endpoint:
You haveYou wantUse
A text description (ex: "person")Masks for every matching instance/sam3/concept_segment
A box around one example objectMasks for every similar instance/sam3/concept_segment
Text plus example boxes to include or exclude objectsMasks for every matching instance/sam3/concept_segment
A click or a box on one specific objectA mask for that object only/sam3/visual_segment
Pass your [API key](https://app.roboflow.com/settings/api) as the `api_key` query parameter on every request. ## SAM3 Concept Segmentation (PCS) `POST https://serverless.roboflow.com/sam3/concept_segment` Each entry in `prompts` describes one concept. The response contains one `prompt_results` entry per prompt, each holding every instance found. Requests accept at most 16 prompts. ### Text prompts ```python import os import requests payload = { "image": {"type": "url", "value": "https://media.roboflow.com/inference/people-walking.jpg"}, "prompts": [ {"type": "text", "text": "person"}, {"type": "text", "text": "backpack"}, ], "output_prob_thresh": 0.5, "format": "polygon", # or "rle" } response = requests.post( "https://serverless.roboflow.com/sam3/concept_segment", params={"api_key": os.environ["ROBOFLOW_API_KEY"]}, json=payload, ) for prompt_result in response.json()["prompt_results"]: print(prompt_result["echo"], len(prompt_result["predictions"]), "instances") ``` Images can also be sent inline as `{"type": "base64", "value": ""}`. ### Exemplar box prompts Instead of text, you can prompt with an exemplar: a box around one example object. The model finds every instance that matches the example, not just the boxed object. ```python payload = { "image": {"type": "url", "value": "https://media.roboflow.com/inference/people-walking.jpg"}, "prompts": [ { "type": "visual", "boxes": [{"x": 1409, "y": 705, "width": 112, "height": 183}], "box_labels": [1], } ], "output_prob_thresh": 0.5, "format": "polygon", } ``` Boxes use absolute pixel coordinates. Two formats are accepted: * `{"x": ..., "y": ..., "width": ..., "height": ...}` where `x`, `y` is the top-left corner * `{"x0": ..., "y0": ..., "x1": ..., "y1": ...}` for explicit corners `box_labels` is required when `boxes` is set and must have one entry per box: `1` marks a positive exemplar (find objects like this), `0` marks a negative exemplar (exclude objects like this). ### Combined text and exemplar prompts A single prompt can carry both text and exemplar boxes. This is useful for narrowing a text concept with visual examples, or excluding lookalikes with negative exemplars: ```python payload = { "image": {"type": "url", "value": "https://media.roboflow.com/inference/people-walking.jpg"}, "prompts": [ { "type": "visual", "text": "person", "boxes": [ {"x": 1409, "y": 705, "width": 112, "height": 183}, {"x": 1216, "y": 496, "width": 124, "height": 184}, ], "box_labels": [1, 0], } ], "output_prob_thresh": 0.5, "format": "polygon", } ``` Here the model segments people matching the first (positive) exemplar while suppressing instances similar to the second (negative) exemplar. ## SAM3 Visual Segmentation (PVS) `POST https://serverless.roboflow.com/sam3/visual_segment` PVS segments one specific object indicated by clicks or a box. Use it for interactive, human-in-the-loop mask refinement; use PCS when you want every instance of a concept. ```python import os import requests payload = { "image": {"type": "url", "value": "https://media.roboflow.com/inference/people-walking.jpg"}, "prompts": { "prompts": [ { "points": [{"x": 1465, "y": 796, "positive": True}], "box": {"x": 1465, "y": 796, "width": 112, "height": 183}, } ] }, "multimask_output": False, "format": "json", } response = requests.post( "https://serverless.roboflow.com/sam3/visual_segment", params={"api_key": os.environ["ROBOFLOW_API_KEY"]}, json=payload, ) prediction = response.json()["predictions"][0] print(prediction["confidence"], len(prediction["masks"]), "polygons") ``` A prompt can contain `points`, a `box`, or both: * `points` are absolute pixel coordinates. `"positive": true` includes the clicked region, `false` excludes it. Add more points to refine the mask. * `box` uses center-anchored coordinates: `x`, `y` is the box center, unlike PCS boxes which are top-left anchored. The response contains the single highest-confidence mask for the prompt. `multimask_output` controls how many internal mask proposals the model generates (three when true), but the best proposal is always selected for the response. {% hint style="warning" %} Send one prompt per request. Multiple prompts in one PVS request currently return only one prediction. {% endhint %} For an interactive demo using OpenCV, see this [GitHub Gist](https://gist.github.com/Erol444/4cbc33c6ac52d83c63f6f9d86ca8a7a4), which was used in this video: {% embed url="" %} ## SAM3 inference speed Latency measured with [Roboflow Inference](https://docs.roboflow.com/deployment/self-hosted/self-hosted) on 1x NVIDIA L4, batch size 1, mean after warmup.
ModelLatency (ms)
sam3251.4
Measured with concept segmentation from a single text prompt. ## SAM3 API endpoints ## SAM3 PCS (promptable concept segmentation) > \*\*Concept Segmentation (Text Prompts)\*\*\ > \ > Allows you to segment objects using text prompts.\ > \ > \*\*Image Input\*\*: The \`image\` field accepts either:\ > \- \`{"type": "url", "value": "\"}\` - A publicly accessible image URL\ > \- \`{"type": "base64", "value": "\"}\` - Base64 encoded image data\ > \ > \*\*Prompts\*\*: Each prompt in the \`prompts\` array should have \`type: "text"\` and a \`text\` field with the object description. ```json {"openapi":"3.1.0","info":{"title":"Roboflow SAM3 API","version":"0.64.4"},"servers":[{"url":"https://serverless.roboflow.com"}],"paths":{"/sam3/concept_segment":{"post":{"summary":"SAM3 PCS (promptable concept segmentation)","description":"**Concept Segmentation (Text Prompts)**\n\nAllows you to segment objects using text prompts.\n\n**Image Input**: The `image` field accepts either:\n- `{\"type\": \"url\", \"value\": \"\"}` - A publicly accessible image URL\n- `{\"type\": \"base64\", \"value\": \"\"}` - Base64 encoded image data\n\n **Prompts**: Each prompt in the `prompts` array should have `type: \"text\"` and a `text` field with the object description.","operationId":"sam3_segment_image_sam3_concept_segment_post","parameters":[{"name":"api_key","in":"query","required":true,"schema":{"type":"string","title":"API Key"},"description":"Your Roboflow API Key. Get one at https://app.roboflow.com/settings/api"}],"requestBody":{"required":true,"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Sam3SegmentationRequest"}}}},"responses":{"200":{"description":"Successful Response","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Sam3SegmentationResponse"}}}},"422":{"description":"Validation Error","content":{"application/json":{"schema":{"$ref":"#/components/schemas/HTTPValidationError"}}}}}}}},"components":{"schemas":{"Sam3SegmentationRequest":{"properties":{"image":{"$ref":"#/components/schemas/InferenceRequestImage","description":"The image to be segmented."},"prompts":{"items":{"$ref":"#/components/schemas/Sam3Prompt"},"type":"array","minItems":1,"title":"Prompts","description":"List of prompts (text and/or visual)"},"format":{"type":"string","title":"Format","description":"One of 'polygon', 'rle'","default":"polygon"},"image_id":{"type":"string","title":"Image Id","description":"Optional ID for caching embeddings."},"output_prob_thresh":{"type":"number","title":"Output Prob Thresh","description":"Score threshold for outputs.","default":0.5},"model_id":{"type":"string","title":"Model Id","description":"The model ID of SAM3. Use 'sam3/sam3_final' to target the generic base model.","default":"sam3/sam3_final"},"nms_iou_threshold":{"type":"number","title":"Nms Iou Threshold","description":"IoU threshold for cross-prompt NMS. If not set, NMS is disabled. Must be in [0.0, 1.0] when set."}},"type":"object","required":["image","prompts"],"title":"Sam3SegmentationRequest"},"InferenceRequestImage":{"properties":{"type":{"type":"string","title":"Type","description":"The type of image data provided, one of `url`, `base64`"},"value":{"type":"string","title":"Value","description":"Image data corresponding to the image type, if type = 'url' then value is a string containing the url of an image, else if type = 'base64' then value is a string containing base64 encoded image data."}},"type":"object","required":["type"],"title":"InferenceRequestImage","description":"Image data for inference request.\n\nAttributes:\n type (str): The type of image data provided, one of 'url', 'base64', or 'numpy'.\n value (Optional[Any]): Image data corresponding to the image type."},"Sam3Prompt":{"properties":{"type":{"type":"string","title":"Type","description":"Hint: `text` or `visual`"},"text":{"type":"string","title":"Text","description":"Text prompt describing the object to segment"},"output_prob_thresh":{"type":"number","title":"Output Prob Thresh","description":"Score threshold for this prompt's outputs. Overrides request-level threshold if set."},"boxes":{"items":{"anyOf":[{"$ref":"#/components/schemas/Box"},{"$ref":"#/components/schemas/BoxXYXY"}]},"type":"array","title":"Boxes","description":"Absolute pixel boxes as either XYWH or XYXY entries"},"box_labels":{"items":{"anyOf":[{"type":"integer"},{"type":"boolean"}]},"type":"array","title":"Box Labels","description":"List of 0/1 or booleans for boxes"}},"type":"object","required":["type"],"title":"Sam3Prompt","description":"Unified prompt that can contain text and/or geometry. Absolute pixel coordinates are used for boxes."},"Sam3SegmentationResponse":{"properties":{"prompt_results":{"items":{"$ref":"#/components/schemas/Sam3PromptResult"},"type":"array","title":"Prompt Results","description":"Results for each prompt in the request"},"time":{"type":"number","title":"Time","description":"The time in seconds it took to produce the segmentation including preprocessing"}},"type":"object","required":["prompt_results","time"],"title":"Sam3SegmentationResponse"},"Sam3PromptResult":{"properties":{"prompt_index":{"type":"integer","title":"Prompt Index","description":"Index of the prompt this result corresponds to"},"echo":{"$ref":"#/components/schemas/Sam3PromptEcho","description":"Echo of the original prompt for reference"},"predictions":{"items":{"$ref":"#/components/schemas/Sam3SegmentationPrediction"},"type":"array","title":"Predictions","description":"Segmentation predictions for this prompt"}},"type":"object","required":["prompt_index","predictions"],"title":"Sam3PromptResult"},"Sam3PromptEcho":{"properties":{"prompt_index":{"type":"integer","title":"Prompt Index"},"type":{"type":"string","title":"Type","description":"The prompt type (`text` or `visual`)"},"text":{"type":"string","title":"Text","description":"The text prompt if type is `text`"},"num_boxes":{"type":"integer","title":"Num Boxes","description":"Number of bounding boxes in the prompt"}},"type":"object","title":"Sam3PromptEcho"},"Sam3SegmentationPrediction":{"properties":{"format":{"type":"string","title":"Format","description":"The format of the mask data, either `polygon` or `rle`"},"confidence":{"type":"number","title":"Confidence","description":"Confidence score for this prediction"},"masks":{"items":{"items":{"items":{"type":"number"},"type":"array","minItems":2,"maxItems":2},"type":"array"},"type":"array","title":"Masks","description":"Array of polygons, each polygon is an array of [x, y] coordinate points"}},"type":"object","required":["format","confidence","masks"],"title":"Sam3SegmentationPrediction"},"HTTPValidationError":{"properties":{"detail":{"items":{"$ref":"#/components/schemas/ValidationError"},"type":"array","title":"Detail"}},"type":"object","title":"HTTPValidationError"},"ValidationError":{"properties":{"loc":{"items":{"anyOf":[{"type":"string"},{"type":"integer"}]},"type":"array","title":"Location"},"msg":{"type":"string","title":"Message"},"type":{"type":"string","title":"Error Type"}},"type":"object","required":["loc","msg","type"],"title":"ValidationError"}}}} ``` ## SAM3 PVS (promptable visual segmentation) > \*\*Interactive Segmentation (SAM 2 Style)\*\*\ > \ > SAM 3 also supports interactive segmentation using points and boxes.\ > \ > \*\*Image Input\*\*: The \`image\` field accepts either:\ > \- \`{"type": "url", "value": "\"}\` - A publicly accessible image URL\ > \- \`{"type": "base64", "value": "\"}\` - Base64 encoded image data\ > \ > \> \*\*Note\*\*: NumPy arrays are NOT supported on the serverless API. Use URL or base64 encoding only.\ > \ > \*\*Prompts\*\*: Support point-based prompts with positive/negative clicks for interactive segmentation. ```json {"openapi":"3.1.0","info":{"title":"Roboflow SAM3 API","version":"0.64.4"},"servers":[{"url":"https://serverless.roboflow.com"}],"paths":{"/sam3/visual_segment":{"post":{"summary":"SAM3 PVS (promptable visual segmentation)","description":"**Interactive Segmentation (SAM 2 Style)**\n\nSAM 3 also supports interactive segmentation using points and boxes.\n\n**Image Input**: The `image` field accepts either:\n- `{\"type\": \"url\", \"value\": \"\"}` - A publicly accessible image URL\n- `{\"type\": \"base64\", \"value\": \"\"}` - Base64 encoded image data\n\n> **Note**: NumPy arrays are NOT supported on the serverless API. Use URL or base64 encoding only.\n\n**Prompts**: Support point-based prompts with positive/negative clicks for interactive segmentation.","operationId":"sam3_visual_segment_sam3_visual_segment_post","parameters":[{"name":"api_key","in":"query","required":true,"schema":{"type":"string","title":"API Key"},"description":"Your Roboflow API Key. Get one at https://app.roboflow.com/settings/api"}],"requestBody":{"required":true,"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Sam2SegmentationRequest"}}}},"responses":{"200":{"description":"Successful Response","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Sam2SegmentationResponse"}}}},"422":{"description":"Validation Error","content":{"application/json":{"schema":{"$ref":"#/components/schemas/HTTPValidationError"}}}}}}}},"components":{"schemas":{"Sam2SegmentationRequest":{"properties":{"image":{"$ref":"#/components/schemas/InferenceRequestImage","description":"The image to be segmented."},"image_id":{"type":"string","title":"Image Id","description":"The ID of the image to be segmented used to retrieve cached embeddings. If an embedding is cached, it will be used instead of generating a new embedding. If no embedding is cached, a new embedding will be generated and cached."},"prompts":{"$ref":"#/components/schemas/Sam2PromptSet","description":"A list of prompts for masks to predict. Each prompt can include a bounding box and / or a set of postive or negative points."},"format":{"type":"string","title":"Format","description":"The format of the response. Must be one of 'json', 'rle', or 'binary'. If binary, masks are returned as binary numpy arrays. If json, masks are converted to polygons. If rle, masks are converted to RLE format.","default":"json"},"sam2_version_id":{"type":"string","title":"Sam2 Version Id","description":"The version ID of SAM to be used for this request. Must be one of hiera_tiny, hiera_small, hiera_large, hiera_b_plus","default":"hiera_large"},"multimask_output":{"type":"boolean","title":"Multimask Output","description":"If true, the model will return three masks. For ambiguous input prompts (such as a single click), this will often produce better masks than a single prediction.","default":true},"save_logits_to_cache":{"type":"boolean","title":"Save Logits To Cache","description":"If True, saves the low-resolution logits to the cache for potential future use.","default":false},"load_logits_from_cache":{"type":"boolean","title":"Load Logits From Cache","description":"If True, attempts to load previously cached low-resolution logits for the given image and prompt set.","default":false}},"type":"object","required":["image"],"title":"Sam2SegmentationRequest","description":"SAM2 visual segmentation request."},"InferenceRequestImage":{"properties":{"type":{"type":"string","title":"Type","description":"The type of image data provided, one of `url`, `base64`"},"value":{"type":"string","title":"Value","description":"Image data corresponding to the image type, if type = 'url' then value is a string containing the url of an image, else if type = 'base64' then value is a string containing base64 encoded image data."}},"type":"object","required":["type"],"title":"InferenceRequestImage","description":"Image data for inference request.\n\nAttributes:\n type (str): The type of image data provided, one of 'url', 'base64', or 'numpy'.\n value (Optional[Any]): Image data corresponding to the image type."},"Sam2SegmentationResponse":{"properties":{"prompt_results":{"items":{"$ref":"#/components/schemas/Sam2PromptResult"},"type":"array","title":"Prompt Results","description":"Results for each prompt in the request"},"time":{"type":"number","title":"Time","description":"The time in seconds it took to produce the segmentation including preprocessing"}},"type":"object","required":["prompt_results","time"],"title":"Sam2SegmentationResponse"},"Sam2PromptResult":{"properties":{"prompt_index":{"type":"integer","title":"Prompt Index","description":"Index of the prompt this result corresponds to"},"predictions":{"items":{"$ref":"#/components/schemas/Sam2SegmentationPrediction"},"type":"array","title":"Predictions","description":"Segmentation predictions for this prompt"}},"type":"object","required":["prompt_index","predictions"],"title":"Sam2PromptResult"},"HTTPValidationError":{"properties":{"detail":{"items":{"$ref":"#/components/schemas/ValidationError"},"type":"array","title":"Detail"}},"type":"object","title":"HTTPValidationError"},"ValidationError":{"properties":{"loc":{"items":{"anyOf":[{"type":"string"},{"type":"integer"}]},"type":"array","title":"Location"},"msg":{"type":"string","title":"Message"},"type":{"type":"string","title":"Error Type"}},"type":"object","required":["loc","msg","type"],"title":"ValidationError"}}}} ``` ## Run SAM3 with self-hosted Inference SAM3 can also run on your own hardware, either loaded in-process with the [`inference`](https://docs.roboflow.com/deployment/self-hosted/self-hosted) package or served from a GPU container. ### Run in Docker ```bash docker run -it --rm -p 9001:9001 --gpus=all roboflow/inference-server:latest ``` The server exposes the same `/sam3/concept_segment` and `/sam3/visual_segment` endpoints documented above at `http://localhost:9001`. ### Load the model in Python ```bash pip install "inference-gpu[sam3]" ``` ```python import os os.environ["API_KEY"] = "YOUR_API_KEY" from inference.core.entities.requests.sam3 import Sam3Prompt from inference.models.sam3 import SegmentAnything3 model = SegmentAnything3(model_id="sam3/sam3_final") prompts = [ # Segment every instance of a concept Sam3Prompt(type="text", text="person"), # Box one example object and segment every similar instance. # box_labels: 1 = positive exemplar, 0 = negative exemplar. Sam3Prompt( type="visual", boxes=[Sam3Prompt.Box(x=1409, y=705, width=112, height=183)], box_labels=[1], ), ] response = model.segment_image( image="path/to/your/image.jpg", prompts=prompts, output_prob_thresh=0.5, format="polygon", # or "rle", "json" ) for prompt_result in response.prompt_results: print(prompt_result.echo.text, len(prompt_result.predictions), "instances") ``` Weights download automatically on first use. ### Interactive segmentation in Python `Sam3ForInteractiveImageSegmentation` implements the SAM2-style point and box interface, for human-in-the-loop mask refinement: ```python from inference.models.sam3 import Sam3ForInteractiveImageSegmentation model = Sam3ForInteractiveImageSegmentation(model_id="sam3/sam3_final") embedding, img_shape, image_id = model.embed_image(image="path/to/image.jpg") masks, scores, logits = model.segment_image( image_id=image_id, prompts={"points": [{"x": 500, "y": 400, "positive": True}]}, ) ``` ## Use SAM3 in Workflows Two SAM3 image blocks are available in [Workflows](https://docs.roboflow.com/workflows): * **SAM 3** runs concept segmentation. Enter the classes you want in `class_names` (for example `["person", "vehicle"]`) and the block outputs instance segmentation predictions that other steps can consume. * **SAM 3 Interactive** runs promptable visual segmentation. Supply labeled points (kind `labeled_points`), for example `[{"x": 320, "y": 240, "positive": true}]`, and optionally connect detections from another model to the `boxes` field. Each box becomes a separate prompt, and its class name is forwarded to the predicted mask. ### Video tracking The **SAM3 Video Tracker** block (`roboflow_core/sam3_video@v1`) runs SAM3's streaming concept tracker frame by frame. You provide concepts as text in `class_names`, and the model runs fused detection and tracking on every frame. Objects matching a concept keep a stable `tracker_id`, and, unlike detector-seeded tracking, objects that enter the scene mid-stream are picked up automatically with no re-prompting and no upstream detection model. Each mask carries the concept it matched as its class name and the model's detection score as its confidence (filter with `threshold`, default `0.5`). * **Stateful and local-only.** One tracking session is kept per `video_metadata.video_identifier`. The block requires `WORKFLOWS_STEP_EXECUTION_MODE=local`, a GPU, and a persistent WebRTC session. * **No prompt scheduling.** Concept prompts are registered once per session; the session is re-seeded only when the stream restarts or `class_names` changes. For detector-driven (box-prompted) video tracking, use the SAM2 Video Tracker block on the [SAM2 page](/models/supported-models/sam2.md), which also accepts `sam3trackervideo` as `model_id`. * **Model.** `model_id` defaults to `sam3video`, the HuggingFace transformers port of SAM3 video, which exposes the frame-by-frame streaming interface. The native `sam3` package's video predictor requires the whole video upfront and cannot be used for live streams. ```python from inference_sdk import InferenceHTTPClient from inference_sdk.webrtc import StreamConfig, VideoFileSource WORKFLOW = { "version": "1.0", "inputs": [{"type": "InferenceImage", "name": "image"}], "steps": [ { "type": "roboflow_core/sam3_video@v1", "name": "tracker", "images": "$inputs.image", "class_names": ["person", "forklift"], "threshold": 0.5, }, ], "outputs": [ { "type": "JsonField", "name": "predictions", "selector": "$steps.tracker.predictions", } ], } client = InferenceHTTPClient( api_url="http://localhost:9001", api_key="YOUR_API_KEY", ) session = client.webrtc.stream( source=VideoFileSource("path/to/video.mp4"), workflow=WORKFLOW, config=StreamConfig(data_output=["predictions"]), ) @session.on_data("predictions") def handle_predictions(predictions, metadata): print(predictions) session.run() ``` ## SAM3-3D (beta) SAM3-3D turns a 2D image plus masks into 3D assets: meshes and Gaussian splats. {% hint style="warning" %} SAM3-3D is in beta. It is available only when the `SAM3_3D_OBJECTS_ENABLED` flag is set, requires a GPU with 32 GB or more of VRAM, and runs through the `inference` package or a local Inference server (it is not on the Serverless Cloud API). {% endhint %} Install the dependencies (Python 3.10 recommended): ```bash pip install --no-cache-dir --no-build-isolation -r requirements/requirements.sam3_3d.txt ``` Or build and run the 3D-enabled GPU container: ```bash docker build -t roboflow/roboflow-inference-server-gpu:dev -f docker/dockerfiles/Dockerfile.onnx.gpu.3d . docker run --gpus all -p 9001:9001 roboflow/roboflow-inference-server-gpu:dev ``` **Input.** An RGB image plus `mask_input`, which defines the object regions. Masks are accepted as binary arrays (`(H, W)` or `(N, H, W)`), COCO flat polygons, point-pair polygons, RLE dicts, or an `sv.Detections` object from SAM2 or another segmentation model. **Output.** `mesh_glb` (combined scene mesh, GLB), `gaussian_ply` (combined Gaussian splat, PLY), `objects` (per-object `mesh_glb`, `gaussian_ply`, and `metadata` with rotation, translation, and scale), and `time`. ```python import os os.environ["SAM3_3D_OBJECTS_ENABLED"] = "true" os.environ["SPARSE_ATTN_BACKEND"] = "flash_attn" os.environ["ATTN_BACKEND"] = "flash_attn" from inference import get_model from inference.core.entities.requests.sam3_3d import Sam3_3D_Objects_InferenceRequest model = get_model("sam3-3d-objects", api_key="YOUR_API_KEY") request = Sam3_3D_Objects_InferenceRequest( image={"type": "file", "value": "image.jpg"}, mask_input=mask_polygons, # polygons, binary masks, or RLE ) response = model.infer_from_request(request) if response.mesh_glb is not None: with open("out_mesh.glb", "wb") as f: f.write(response.mesh_glb) for index, obj in enumerate(response.objects): if obj.gaussian_ply is not None: with open(f"out_object_{index}.ply", "wb") as f: f.write(obj.gaussian_ply) ``` Setting `SPARSE_ATTN_BACKEND` and `ATTN_BACKEND` to `flash_attn` speeds up the pipeline. In Workflows, SAM3-3D supports local execution and remote execution through the `sam3_3d_infer()` client method or the `/sam3_3d/infer` endpoint. ## See also * [SAM2](/models/supported-models/sam2.md) - point and box prompted segmentation, plus detector-seeded video tracking. * [Segment Anything (SAM)](/models/supported-models/sam.md) - the original single-object model.