SAM2 Video Tracker

Segment and track objects across video frames with SAM2's streaming camera predictor.

Run Segment Anything 2 on a live video stream frame by frame, keeping per-video temporal memory so object identities are preserved across frames.

Feed box detections from an upstream detector (e.g. a YOLO block) as prompts. The block multiplexes a single SAM2 camera predictor across many video streams by keying state on video_metadata.video_identifier; depending on prompt_mode, it either re-seeds the prompts periodically or simply propagates existing tracks.

Intended for use in a persistent WebRTC session, which delivers one frame at a time and tags each frame with video metadata.

Type identifier

Use the following identifier in step "type" field: roboflow_core/segment_anything_2_video@v1 to add the block as a step in your workflow.

Properties

Name	Type	Description	Refs
name	str	Enter a unique identifier for this step..	❌
model_id	str	Streaming video tracker model id resolved by inference_models. The sam2video family ships four Hiera backbone sizes; small is the default trade-off between speed and quality. sam3trackervideo is SAM3's visually prompted tracker - the same prompt contract with a larger backbone, markedly better at identity retention on long videos and crowded scenes, at higher compute cost..	✅
prompt_mode	str	When to consume boxes as SAM2 prompts. first_frame prompts once per session and then tracks; every_n_frames re-seeds every prompt_interval frames; every_frame re-seeds every frame. On frames where re-seeding does not happen, boxes is ignored and the block simply propagates..	❌
prompt_interval	int	For prompt_mode=every_n_frames: re-prompt every N frames..	✅
threshold	float	Minimum confidence for emitted masks..	✅

The Refs column marks possibility to parametrise the property with dynamic values available in workflow runtime. See Bindings for more info.

Runtime compatibility

soft - runtime hosted_serverless, dedicated_deployment; execution remote; input video : Block keeps per-video state in process memory (keyed by video_metadata.video_identifier). With remote step execution on stateless or multi-replica HTTP runtimes, successive requests may be served by different worker processes, so the state resets between calls and the output is meaningless for tracking / counting / aggregation. Use local step execution in a persistent WebRTC session for stable cross-frame results.

hard - runtime self_hosted_cpu; execution local : Requires a GPU; the streaming SAM2 video model needs CUDA.

soft - input image : Block depends on temporal context from video or repeated-frame workflows. With a still image/photo, there is no meaningful history to track, compare, aggregate, or visualize, so the block provides little or no benefit.

Input and Output Bindings

The available connections depend on its binding kinds. Check what binding kinds SAM2 Video Tracker in version v1 has.

Input and output bindings

• input

  • images (image): The image to infer on..

  • boxes (Union[object_detection_prediction, instance_segmentation_prediction, keypoint_detection_prediction]): Bounding boxes to use as SAM2 prompts. Only read on frames where the block re-prompts (see prompt_mode)..

  • model_id (roboflow_model_id): Streaming video tracker model id resolved by inference_models. The sam2video family ships four Hiera backbone sizes; small is the default trade-off between speed and quality. sam3trackervideo is SAM3's visually prompted tracker - the same prompt contract with a larger backbone, markedly better at identity retention on long videos and crowded scenes, at higher compute cost..

  • prompt_interval (integer): For prompt_mode=every_n_frames: re-prompt every N frames..

  • threshold (float): Minimum confidence for emitted masks..

• output

  • predictions (instance_segmentation_prediction): Prediction with detected bounding boxes and segmentation masks in form of sv.Detections(...) object.

Example JSON definition

{
	    "name": "<your_step_name_here>",
	    "type": "roboflow_core/segment_anything_2_video@v1",
	    "images": "$inputs.image",
	    "boxes": "$steps.object_detection_model.predictions",
	    "model_id": "sam2video/tiny",
	    "prompt_mode": "<block_does_not_provide_example>",
	    "prompt_interval": 30,
	    "threshold": 0.0
	}