> For the complete documentation index, see [llms.txt](https://docs.roboflow.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.roboflow.com/roboflow/roboflow-jp/deploy/supported-models/sam3.md). # SAM3 Meta の [Segment Anything Model 3](https://github.com/facebookresearch/sam3) の推論を、私たちの [Serverless Hosted API](/roboflow/roboflow-jp/deploy/serverless-hosted-api-v2.md)。SAM3のエンドポイントは2種類あります: {% hint style="info" %} RoboflowでSAM3モデルを学習するには、有料 [プラン](/roboflow/roboflow-jp/billing/plans.md) に含まれる [従量課金](/roboflow/roboflow-jp/billing/credits.md). {% endhint %} * [プロンプト対応の概念セグメンテーション](#concept-segmentation-pcs) (**PCS**)。画像内の概念のすべてのインスタンスをセグメント化します。概念はテキストプロンプト、例示ボックス、またはその両方で記述されます。 * [プロンプト対応のビジュアルセグメンテーション](#visual-segmentation-pvs) (**PVS**）。SAM2のように、ポイントまたはボックスから、1リクエストにつき1つのオブジェクトを対話的にセグメント化します。 この表を使ってエンドポイントを選択します: | あなたが持っているもの | あなたが欲しいもの | 使用する | | -------------------------------- | ------------------ | ----------------------- | | テキスト説明（例: "person"） | 一致するすべてのインスタンスのマスク | `/sam3/concept_segment` | | 1つの例示オブジェクトを囲むボックス | 類似するすべてのインスタンスのマスク | `/sam3/concept_segment` | | オブジェクトを含める、または除外するためのテキストと例示ボックス | 一致するすべてのインスタンスのマスク | `/sam3/concept_segment` | | 特定の1つのオブジェクト上のクリックまたはボックス | そのオブジェクトのみのマスク | `/sam3/visual_segment` | あなたの [APIキー](https://app.roboflow.com/settings/api) を `api_key` を各リクエストのクエリパラメータとして渡してください。 ## 概念セグメンテーション（PCS） `POST https://serverless.roboflow.com/sam3/concept_segment` の各エントリは `prompts` 1つの概念を記述します。レスポンスには1つの `prompt_results` promptごとに1エントリが含まれ、それぞれに見つかったすべてのインスタンスが格納されます。リクエストは最大16個のプロンプトを受け付けます。 ### テキストプロンプト ```python import os import requests payload = { "image": {"type": "url", "value": "https://media.roboflow.com/inference/people-walking.jpg"}, "prompts": [ {"type": "text", "text": "人"}, {"type": "text", "text": "バックパック"}, ], "output_prob_thresh": 0.5, "format": "polygon", # または "rle" } response = requests.post( "https://serverless.roboflow.com/sam3/concept_segment", params={"api_key": os.getenv("API_KEY")}, json=payload, ) for prompt_result in response.json()["prompt_results"]: print(prompt_result["echo"], len(prompt_result["predictions"]), "インスタンス") ``` 画像はインラインでも送信できます: `{"type": "base64", "value": ""}`. ### 例示ボックスのプロンプト テキストの代わりに、例示として1つの例示オブジェクトを囲むボックスを使ってプロンプトできます。モデルは、ボックスで囲まれたオブジェクトだけでなく、その例に一致するすべてのインスタンスを見つけます。 ```python payload = { "image": {"type": "url", "value": "https://media.roboflow.com/inference/people-walking.jpg"}, "prompts": [ { "type": "visual", "boxes": [{"x": 1409, "y": 705, "width": 112, "height": 183}], "box_labels": [1], } ], "output_prob_thresh": 0.5, "format": "polygon", } ``` ボックスは絶対ピクセル座標を使用します。2つの形式が受け付けられます: * `{"x": ..., "y": ..., "width": ..., "height": ...}` ここで `x`, `y` は左上隅です * `{"x0": ..., "y0": ..., "x1": ..., "y1": ...}` 明示的な角指定用 `box_labels` が必要です `boxes` が設定されている場合で、ボックスごとに1件のエントリが必要です: `1` 正の例示を示します（このようなオブジェクトを見つけます）、 `0` 負の例示を示します（このようなオブジェクトを除外します）。 ### テキストと例示を組み合わせたプロンプト 1つのプロンプトにテキストと例示ボックスの両方を含めることができます。これは、視覚的な例でテキスト概念を絞り込んだり、負の例示で似たものを除外したりするのに便利です: ```python payload = { "image": {"type": "url", "value": "https://media.roboflow.com/inference/people-walking.jpg"}, "prompts": [ { "type": "visual", "text": "人", "boxes": [ {"x": 1409, "y": 705, "width": 112, "height": 183}, {"x": 1216, "y": 496, "width": 124, "height": 184}, ], "box_labels": [1, 0], } ], "output_prob_thresh": 0.5, "format": "polygon", } ``` ここでは、モデルは最初の（正の）例示に一致する人をセグメント化し、2つ目の（負の）例示に似たインスタンスを抑制します。 ## ビジュアルセグメンテーション（PVS） `POST https://serverless.roboflow.com/sam3/visual_segment` PVSは、クリックまたはボックスで示された特定のオブジェクト1つをセグメント化します。対話的な人手介在型のマスク調整に使用してください。ある概念のすべてのインスタンスがほしい場合はPCSを使用してください。 ```python import os import requests payload = { "image": {"type": "url", "value": "https://media.roboflow.com/inference/people-walking.jpg"}, "prompts": { "prompts": [ { "points": [{"x": 1465, "y": 796, "positive": True}], "box": {"x": 1465, "y": 796, "width": 112, "height": 183}, } ] }, "multimask_output": False, "format": "json", } response = requests.post( "https://serverless.roboflow.com/sam3/visual_segment", params={"api_key": os.getenv("API_KEY")}, json=payload, ) prediction = response.json()["predictions"][0] print(prediction["confidence"], len(prediction["masks"]), "ポリゴン") ``` プロンプトには `points`、 `box`、またはその両方を含めることができます: * `points` は絶対ピクセル座標です。 `"positive": true` クリックされた領域を含みます、 `false` それを除外します。さらにポイントを追加してマスクを調整します。 * `box` は中心基準の座標を使用します: `x`, `y` はボックスの中心です。PCSのボックスが左上基準なのとは異なります。 レスポンスには、プロンプトに対する信頼度が最も高いマスクが1つ含まれます。 `multimask_output` モデルが内部で生成するマスク候補の数を制御します（trueのときは3つ）が、レスポンスでは常に最良の候補が選択されます。 {% hint style="warning" %} 1リクエストにつき1つのプロンプトを送信してください。現時点では、1回のPVSリクエストで複数プロンプトを送っても予測は1つしか返りません。 {% endhint %} OpenCVを使った対話型デモについては、こちらの [GitHub Gist](https://gist.github.com/Erol444/4cbc33c6ac52d83c63f6f9d86ca8a7a4)をご覧ください。この動画で使用されました: {% embed url="" %} ## エンドポイント ## SAM3 PCS (promptable concept segmentation) > \*\*Concept Segmentation (Text Prompts)\*\*\ > \ > Allows you to segment objects using text prompts.\ > \ > \*\*Image Input\*\*: The \`image\` field accepts either:\ > \- \`{"type": "url", "value": "\"}\` - A publicly accessible image URL\ > \- \`{"type": "base64", "value": "\"}\` - Base64 encoded image data\ > \ > \*\*Prompts\*\*: Each prompt in the \`prompts\` array should have \`type: "text"\` and a \`text\` field with the object description. ```json {"openapi":"3.1.0","info":{"title":"Roboflow SAM3 API","version":"0.64.4"},"servers":[{"url":"https://serverless.roboflow.com"}],"paths":{"/sam3/concept_segment":{"post":{"summary":"SAM3 PCS (promptable concept segmentation)","description":"**Concept Segmentation (Text Prompts)**\n\nAllows you to segment objects using text prompts.\n\n**Image Input**: The `image` field accepts either:\n- `{\"type\": \"url\", \"value\": \"\"}` - A publicly accessible image URL\n- `{\"type\": \"base64\", \"value\": \"\"}` - Base64 encoded image data\n\n **Prompts**: Each prompt in the `prompts` array should have `type: \"text\"` and a `text` field with the object description.","operationId":"sam3_segment_image_sam3_concept_segment_post","parameters":[{"name":"api_key","in":"query","required":true,"schema":{"type":"string","title":"API Key"},"description":"Your Roboflow API Key. Get one at https://app.roboflow.com/settings/api"}],"requestBody":{"required":true,"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Sam3SegmentationRequest"}}}},"responses":{"200":{"description":"Successful Response","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Sam3SegmentationResponse"}}}},"422":{"description":"Validation Error","content":{"application/json":{"schema":{"$ref":"#/components/schemas/HTTPValidationError"}}}}}}}},"components":{"schemas":{"Sam3SegmentationRequest":{"properties":{"image":{"$ref":"#/components/schemas/InferenceRequestImage","description":"The image to be segmented."},"prompts":{"items":{"$ref":"#/components/schemas/Sam3Prompt"},"type":"array","minItems":1,"title":"Prompts","description":"List of prompts (text and/or visual)"},"format":{"type":"string","title":"Format","description":"One of 'polygon', 'rle'","default":"polygon"},"image_id":{"type":"string","title":"Image Id","description":"Optional ID for caching embeddings."},"output_prob_thresh":{"type":"number","title":"Output Prob Thresh","description":"Score threshold for outputs.","default":0.5},"model_id":{"type":"string","title":"Model Id","description":"The model ID of SAM3. Use 'sam3/sam3_final' to target the generic base model.","default":"sam3/sam3_final"},"nms_iou_threshold":{"type":"number","title":"Nms Iou Threshold","description":"IoU threshold for cross-prompt NMS. If not set, NMS is disabled. Must be in [0.0, 1.0] when set."}},"type":"object","required":["image","prompts"],"title":"Sam3SegmentationRequest"},"InferenceRequestImage":{"properties":{"type":{"type":"string","title":"Type","description":"The type of image data provided, one of `url`, `base64`"},"value":{"type":"string","title":"Value","description":"Image data corresponding to the image type, if type = 'url' then value is a string containing the url of an image, else if type = 'base64' then value is a string containing base64 encoded image data."}},"type":"object","required":["type"],"title":"InferenceRequestImage","description":"Image data for inference request.\n\nAttributes:\n type (str): The type of image data provided, one of 'url', 'base64', or 'numpy'.\n value (Optional[Any]): Image data corresponding to the image type."},"Sam3Prompt":{"properties":{"type":{"type":"string","title":"Type","description":"Hint: `text` or `visual`"},"text":{"type":"string","title":"Text","description":"Text prompt describing the object to segment"},"output_prob_thresh":{"type":"number","title":"Output Prob Thresh","description":"Score threshold for this prompt's outputs. Overrides request-level threshold if set."},"boxes":{"items":{"anyOf":[{"$ref":"#/components/schemas/Box"},{"$ref":"#/components/schemas/BoxXYXY"}]},"type":"array","title":"Boxes","description":"Absolute pixel boxes as either XYWH or XYXY entries"},"box_labels":{"items":{"anyOf":[{"type":"integer"},{"type":"boolean"}]},"type":"array","title":"Box Labels","description":"List of 0/1 or booleans for boxes"}},"type":"object","required":["type"],"title":"Sam3Prompt","description":"Unified prompt that can contain text and/or geometry. Absolute pixel coordinates are used for boxes."},"Sam3SegmentationResponse":{"properties":{"prompt_results":{"items":{"$ref":"#/components/schemas/Sam3PromptResult"},"type":"array","title":"Prompt Results","description":"Results for each prompt in the request"},"time":{"type":"number","title":"Time","description":"The time in seconds it took to produce the segmentation including preprocessing"}},"type":"object","required":["prompt_results","time"],"title":"Sam3SegmentationResponse"},"Sam3PromptResult":{"properties":{"prompt_index":{"type":"integer","title":"Prompt Index","description":"Index of the prompt this result corresponds to"},"echo":{"$ref":"#/components/schemas/Sam3PromptEcho","description":"Echo of the original prompt for reference"},"predictions":{"items":{"$ref":"#/components/schemas/Sam3SegmentationPrediction"},"type":"array","title":"Predictions","description":"Segmentation predictions for this prompt"}},"type":"object","required":["prompt_index","predictions"],"title":"Sam3PromptResult"},"Sam3PromptEcho":{"properties":{"prompt_index":{"type":"integer","title":"Prompt Index"},"type":{"type":"string","title":"Type","description":"The prompt type (`text` or `visual`)"},"text":{"type":"string","title":"Text","description":"The text prompt if type is `text`"},"num_boxes":{"type":"integer","title":"Num Boxes","description":"Number of bounding boxes in the prompt"}},"type":"object","title":"Sam3PromptEcho"},"Sam3SegmentationPrediction":{"properties":{"format":{"type":"string","title":"Format","description":"The format of the mask data, either `polygon` or `rle`"},"confidence":{"type":"number","title":"Confidence","description":"Confidence score for this prediction"},"masks":{"items":{"items":{"items":{"type":"number"},"type":"array","minItems":2,"maxItems":2},"type":"array"},"type":"array","title":"Masks","description":"Array of polygons, each polygon is an array of [x, y] coordinate points"}},"type":"object","required":["format","confidence","masks"],"title":"Sam3SegmentationPrediction"},"HTTPValidationError":{"properties":{"detail":{"items":{"$ref":"#/components/schemas/ValidationError"},"type":"array","title":"Detail"}},"type":"object","title":"HTTPValidationError"},"ValidationError":{"properties":{"loc":{"items":{"anyOf":[{"type":"string"},{"type":"integer"}]},"type":"array","title":"Location"},"msg":{"type":"string","title":"Message"},"type":{"type":"string","title":"Error Type"}},"type":"object","required":["loc","msg","type"],"title":"ValidationError"}}}} ``` ## SAM3 PVS (promptable visual segmentation) > \*\*Interactive Segmentation (SAM 2 Style)\*\*\ > \ > SAM 3 also supports interactive segmentation using points and boxes.\ > \ > \*\*Image Input\*\*: The \`image\` field accepts either:\ > \- \`{"type": "url", "value": "\"}\` - A publicly accessible image URL\ > \- \`{"type": "base64", "value": "\"}\` - Base64 encoded image data\ > \ > \> \*\*Note\*\*: NumPy arrays are NOT supported on the serverless API. Use URL or base64 encoding only.\ > \ > \*\*Prompts\*\*: Support point-based prompts with positive/negative clicks for interactive segmentation. ```json {"openapi":"3.1.0","info":{"title":"Roboflow SAM3 API","version":"0.64.4"},"servers":[{"url":"https://serverless.roboflow.com"}],"paths":{"/sam3/visual_segment":{"post":{"summary":"SAM3 PVS (promptable visual segmentation)","description":"**Interactive Segmentation (SAM 2 Style)**\n\nSAM 3 also supports interactive segmentation using points and boxes.\n\n**Image Input**: The `image` field accepts either:\n- `{\"type\": \"url\", \"value\": \"\"}` - A publicly accessible image URL\n- `{\"type\": \"base64\", \"value\": \"\"}` - Base64 encoded image data\n\n> **Note**: NumPy arrays are NOT supported on the serverless API. Use URL or base64 encoding only.\n\n**Prompts**: Support point-based prompts with positive/negative clicks for interactive segmentation.","operationId":"sam3_visual_segment_sam3_visual_segment_post","parameters":[{"name":"api_key","in":"query","required":true,"schema":{"type":"string","title":"API Key"},"description":"Your Roboflow API Key. Get one at https://app.roboflow.com/settings/api"}],"requestBody":{"required":true,"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Sam2SegmentationRequest"}}}},"responses":{"200":{"description":"Successful Response","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Sam2SegmentationResponse"}}}},"422":{"description":"Validation Error","content":{"application/json":{"schema":{"$ref":"#/components/schemas/HTTPValidationError"}}}}}}}},"components":{"schemas":{"Sam2SegmentationRequest":{"properties":{"image":{"$ref":"#/components/schemas/InferenceRequestImage","description":"The image to be segmented."},"image_id":{"type":"string","title":"Image Id","description":"The ID of the image to be segmented used to retrieve cached embeddings. If an embedding is cached, it will be used instead of generating a new embedding. If no embedding is cached, a new embedding will be generated and cached."},"prompts":{"$ref":"#/components/schemas/Sam2PromptSet","description":"A list of prompts for masks to predict. Each prompt can include a bounding box and / or a set of postive or negative points."},"format":{"type":"string","title":"Format","description":"The format of the response. Must be one of 'json', 'rle', or 'binary'. If binary, masks are returned as binary numpy arrays. If json, masks are converted to polygons. If rle, masks are converted to RLE format.","default":"json"},"sam2_version_id":{"type":"string","title":"Sam2 Version Id","description":"The version ID of SAM to be used for this request. Must be one of hiera_tiny, hiera_small, hiera_large, hiera_b_plus","default":"hiera_large"},"multimask_output":{"type":"boolean","title":"Multimask Output","description":"If true, the model will return three masks. For ambiguous input prompts (such as a single click), this will often produce better masks than a single prediction.","default":true},"save_logits_to_cache":{"type":"boolean","title":"Save Logits To Cache","description":"If True, saves the low-resolution logits to the cache for potential future use.","default":false},"load_logits_from_cache":{"type":"boolean","title":"Load Logits From Cache","description":"If True, attempts to load previously cached low-resolution logits for the given image and prompt set.","default":false}},"type":"object","required":["image"],"title":"Sam2SegmentationRequest","description":"SAM2 visual segmentation request."},"InferenceRequestImage":{"properties":{"type":{"type":"string","title":"Type","description":"The type of image data provided, one of `url`, `base64`"},"value":{"type":"string","title":"Value","description":"Image data corresponding to the image type, if type = 'url' then value is a string containing the url of an image, else if type = 'base64' then value is a string containing base64 encoded image data."}},"type":"object","required":["type"],"title":"InferenceRequestImage","description":"Image data for inference request.\n\nAttributes:\n type (str): The type of image data provided, one of 'url', 'base64', or 'numpy'.\n value (Optional[Any]): Image data corresponding to the image type."},"Sam2SegmentationResponse":{"properties":{"prompt_results":{"items":{"$ref":"#/components/schemas/Sam2PromptResult"},"type":"array","title":"Prompt Results","description":"Results for each prompt in the request"},"time":{"type":"number","title":"Time","description":"The time in seconds it took to produce the segmentation including preprocessing"}},"type":"object","required":["prompt_results","time"],"title":"Sam2SegmentationResponse"},"Sam2PromptResult":{"properties":{"prompt_index":{"type":"integer","title":"Prompt Index","description":"Index of the prompt this result corresponds to"},"predictions":{"items":{"$ref":"#/components/schemas/Sam2SegmentationPrediction"},"type":"array","title":"Predictions","description":"Segmentation predictions for this prompt"}},"type":"object","required":["prompt_index","predictions"],"title":"Sam2PromptResult"},"HTTPValidationError":{"properties":{"detail":{"items":{"$ref":"#/components/schemas/ValidationError"},"type":"array","title":"Detail"}},"type":"object","title":"HTTPValidationError"},"ValidationError":{"properties":{"loc":{"items":{"anyOf":[{"type":"string"},{"type":"integer"}]},"type":"array","title":"Location"},"msg":{"type":"string","title":"Message"},"type":{"type":"string","title":"Error Type"}},"type":"object","required":["loc","msg","type"],"title":"ValidationError"}}}} ``` --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.roboflow.com/roboflow/roboflow-jp/deploy/supported-models/sam3.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.