Kinds

Kinds are the Workflows type system: the semantic meaning and data representation behind every connection.

In Workflows, some values can’t be set in advance and are only determined during execution. This is similar to writing a function where you don’t know the exact input values upfront - they’re only provided at runtime, either from user inputs or from other function outputs.

To manage this, Workflows use selectors, which act like references, pointing to data without containing it directly.

Selectors

A selector might refer to a named input - for example the input image - like $inputs.image, or to predictions generated by a previous step - like $steps.my_model.predictions.

In the Workflows ecosystem, users focus on data purpose (e.g., “image”) without worrying about its exact format. Meanwhile, developers building workflow blocks need precise data formats. Kinds serve both needs - they simplify data handling for users while ensuring developers work with the correct data structure.

What are the Kinds?

Kinds is Workflows type system with each kind defining:

• name - expressing semantic meaning of the underlying data - like image or point;

• Python data representation - the data type and format that blocks creators should expect when handling the data within blocks;

• optional serialized data representation - defining what is the format of the kind that external systems should use to integrate with Workflows ecosystem - when needed, custom kinds serializers and deserializers are provided to ensure seamless translation;

Using kinds streamlines compatibility: when a step outputs data of a certain kind and another step requires that same kind, the workflow engine assumes they’ll be compatible, reducing the need for compatibility checks and providing compile-time verification of Workflows definitions.

As for now, kinds are such simplistic that do not support types polymorphism - and developers are asked to use unions of kinds to solve that problem. As defining extensive unions of kinds may be problematic, this problem will probably be addressed in Execution Engine v2.

Kinds declared in Roboflow plugins

• *: Equivalent of any element

• ROBOFLOW_MANAGED_KEY: Roboflow-managed key or credential

• bar_code_detection: Prediction with barcode detection

• boolean: Boolean flag

• bytes: This kind represent bytes

• classification_prediction: Predictions from classifier

• contours: List of numpy arrays where each array represents contour points

• detection: Single element of detections-based prediction (like object_detection_prediction)

• detections_overlaps: List of per-pair detection overlap records

• dictionary: Dictionary

• embedding: A list of floating point numbers representing a vector embedding.

• float_zero_to_one: float value in range [0.0, 1.0]

• float: Float value

• image_keypoints: Image keypoints detected by classical Computer Vision method

• image_metadata: Dictionary with image metadata required by supervision

• image: Image in workflows

• inference_id: Inference identifier

• instance_segmentation_prediction: Prediction with detected bounding boxes and segmentation masks in form of sv.Detections(...) object

• integer: Integer value

• keypoint_detection_prediction: Prediction with detected bounding boxes and detected keypoints in form of sv.Detections(...) object

• labeled_points: List of 2D points with positive/negative labels

• language_model_output: LLM / VLM output

• list_of_values: List of values of any type

• numpy_array: Numpy array

• object_detection_prediction: Prediction with detected bounding boxes in form of sv.Detections(...) object

• parent_id: Identifier of parent for step output

• point: Single point in 2D

• prediction_type: String value with type of prediction

• qr_code_detection: Prediction with QR code detection

• rgb_color: RGB color

• roboflow_api_key: Roboflow API key

• roboflow_model_id: Roboflow model id

• roboflow_project: Roboflow project name

• roboflow_solution: Roboflow Vision Events use case identifier

• secret: Secret value

• semantic_segmentation_prediction: Prediction with per-pixel class label and confidence for semantic segmentation

• serialised_payloads: Serialised element that is usually accepted by sink

• string: String value

• timestamp: Timestamp object

• top_class: String value representing top class predicted by classification model

• video_metadata: Video image metadata

• zone: Definition of polygon zone