Create a Dataset Version

Create a dataset version for use in training a model.

About

A version is a point-in-time snapshot of your dataset. We keep these versions since by keeping track of exactly which images, preprocessing, and augmentation steps were used in each iteration of your model, you maintain the ability to reproduce the results. This allows you to scientifically test across various models and frameworks while remaining confident that the results are attributable to the model changes and not due to a bug/change in the data pipeline.

Key Concepts: What are Workspaces & Projects?

Once a version is created, it is frozen in time, which means changes to the project whether that be adding/removing images, annotations, or other data, won't affect versions that were created before.

Web App

How To Create a Dataset Version

To create a dataset version, click "Versions" in the sidebar associated with your Roboflow project. Then, click "Generate New Version".

From this page, you can set a train/test/valid split and specify preprocessing steps and augmentations for your new dataset version.

Once you have specified the preprocessing steps and augmentations you want to apply to your data, click "Generate". This will generate a new dataset version. You can then use this dataset version to train a model in Roboflow. You can also export your dataset for use in training a model manually.

Readjusting Train/Validation/Test Splits

During the version creation process, you can also readjust the balance of your training, validation and test set splitting. To do this, go to "Step 2: Train/Test Split" and click the "Rebalance" button.

The recommended split is 70/20/10 (train/valid/test), a strong starting point for most datasets. Click "Reset" to restore it at any time.

Set the train, validation, and test percentages, then choose how images are reassigned:

• "Move as few as possible" keeps images in their current split and only moves enough to reach the target percentages.

• "Random shuffle" randomly reassigns every image across the splits.

• "Random shuffle per upload batch or annotation job" shuffles each upload batch or completed annotation job separately, so every batch and job matches the target percentages.

You can also scope the rebalance to specific classes. Selecting classes limits moves to images that contain only those classes and switches the method to "Move as few as possible".

For large datasets, the rebalance runs as a background task with per-phase progress. You can dismiss the dialog while it continues, and track it in the Activity Center.

HTTP API

Rebalance Train/Validation/Test Splits

Rebalance a Project's splits by making a POST request to the /:workspace/:project/splits/rebalance endpoint. This changes the splits on the Project itself, so the next version you generate uses the new balance.

The rebalance always runs as a background task. The request returns 202 Accepted with a task ID and a polling URL. Poll the returned url, or call GET /:workspace/asynctasks/:id with the taskId, until the task completes.

The calling API key must have the project:update scope and permission to edit splits in the Workspace.

Example Request

$ curl --location "https://api.roboflow.com/my-workspace/my-project/splits/rebalance?api_key=$ROBOFLOW_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
    "splits": { "train": 70, "valid": 20, "test": 10 }
}'

Parameters

- splits (object) - Required. Target train, valid, and test values, each a non-negative integer. Values are relative, so 70/20/10 means 70%/20%/10%. At least one must be greater than 0.
- mode (string) - "rebalance" (default) moves as few images as possible. "shuffle" randomly reassigns every image.
- grouping (string) - Only used with "shuffle" mode. "global" (default) shuffles the whole Project. "batch" shuffles each upload batch and completed annotation job separately.
- classes (array of strings) - Limit the rebalance to images that contain only these classes. Images with other classes stay in their current split.

Example Response

{
    "taskId": "abc123",
    "url": "https://api.roboflow.com/my-workspace/asynctasks/abc123"
}

Python SDK

A version freezes a snapshot of the project's labeled data with a chosen set of preprocessing and augmentation steps applied. Train against a version, not against the project directly.

Project.generate_version() returns the new version number, which you can pass to project.version(...) to get a Version handle.

import roboflow

rf = roboflow.Roboflow(api_key="YOUR_API_KEY")
project = rf.workspace().project("my-detector")

new_version = project.generate_version(settings={
    "preprocessing": {
        "auto-orient": True,
        "resize": {"width": 640, "height": 640, "format": "Stretch to"},
    },
    "augmentation": {},
})
version = project.version(new_version)

augmentation.image.versions is the maximum version size multiplier, and your plan caps it. Generation fails if you set it higher. See Limiting Augmentations.

Here is a full example of the settings. There is a detailed explanation for each property below.

{
    "versionName": "Raccoons",
    "augmentation": {
        "bbblur": { "pixels": 1.5 },
        "bbbrightness": { "brighten": true, "darken": false, "percent": 91 },
        "bbcrop": { "min": 12, "max": 71 },
        "bbexposure": { "percent": 30 },
        "bbflip": { "horizontal": true, "vertical": false },
        "bbnoise": { "percent": 50 },
        "bbninety": { "clockwise": true, "counter-clockwise": false, "upside-down": false },
        "bbrotate": { "degrees": 45 },
        "bbshear": { "horizontal": 45, "vertical": 45 },
        "blur": { "pixels": 1.5 },
        "brightness": { "brighten": true, "darken": false, "percent": 91 },
        "crop": { "min": 12, "max": 71 },
        "cutout": { "count": 26, "percent": 71 },
        "exposure": { "percent": 30 },
        "flip": { "horizontal": true, "vertical": false },
        "hue": { "degrees": 180 },
        "image": { "versions": 32 },
        "mosaic": true,
        "ninety": { "clockwise": true, "counter-clockwise": false, "upside-down": false },
        "noise": { "percent": 50 },
        "rgrayscale": { "percent": 50 },
        "rotate": { "degrees": 45 },
        "saturation": { "percent": 50 },
        "shear": { "horizontal": 45, "vertical": 45 }
    },
    "preprocessing": {
        "auto-orient": true,
        "contrast": { "type": "Contrast Stretching" },
        "filter-null": { "percent": 50 },
        "filter-tags": { "tag_name": true, "other_tag": false },
        "grayscale": true,
        "isolate": true,
        "remap": { "original_class_name": "new_class_name" },
        "resize": { "width": 200, "height": 200, "format": "Stretch to" },
        "static-crop": { "x_min": 10, "x_max": 90, "y_min": 10, "y_max": 90 },
        "tile": { "rows": 2, "columns": 2 }
    }
}

tile also takes the fixed tile modes: {"mode": "fixed_size", "tileWidth": 640, "tileHeight": 640, "overlap": 5, "padding": "black"} or {"mode": "fixed_aspect", "aspectRatio": 1, "span": "width", "overlap": 5, "padding": "black"}. See Tile.