Collaborate on Labeling

Have the entire team help label your datasets.

Whether you have a small team working on labeling hundreds of images or a large team working on millions, creating a dataset is about more than drawing boxes. A big part of labeling is in the process of getting an image from the real world into a trained model's stored knowledge that involves image collection, storage, organization, selection, assignment, labeling, and review.

Roboflow offers collaborative features that allow you to:

• Divide work between multiple team members by assigning labeling jobs to anyone on the team

• Organize images into batches as you upload them

• Provide image labeling instructions to help guide work and ensure consistency

• Get an at-a-glance view of labeling work in progress

• See a historical timeline of all labeling work

• Revert changes

• Add comments to images and view image comment history

• Send images with labeling issues back to team members for changes

• Approve or reject labels before including them in a dataset

Assign Labeling Jobs

You can divide labeling work among a team or assign it to specific people responsible for labeling. Assigning jobs to individual team members means you won't have to worry about stepping on each others' work if you're online at the same time.

You can choose to assign jobs to one or more people on your team and if you haven’t included a team member to your workspace yet, you can also invite them and assign an labeling job to be completed at the same time.

Provide Labeling Instructions

You can provide instructions to labelers from the Assign Images tab. Click "Add Instructions" to add instructions to a batch before assigning the batch to an labeler. When you have set your instructions, click "Assign Images".

Click "Edit" to assign instructions:

Click "Save Instructions" to save the labeling instructions.

Job Notifications

Once a labeling job has been assigned, a notification will alert your team members when there's work assigned to them.

Labeling Jobs Board

The labeling jobs board gives an at-a-glance view of the current state of your individually assigned jobs as they go through the labeling process.

To view statistics for a particular labeler, specify a value in the Labeler dropdown.

Job Details

Clicking on individual jobs on the labeling jobs board gives a more detailed view of the individual job and its progress. You can quickly see images that still need to be labeled and reassign jobs to different team members as needed.

Managing Individual Images

On the Annotated tab within a job, you can select individual images using checkboxes, drag-to-select, or the "Select all" checkbox in the header. Once you have a selection:

• Click "Send N Images to Unannotated" to move selected images back to the Unannotated tab for re-labeling.

• Click "Add N Images to Dataset" to add only the selected images to the dataset. Unselected annotated images remain in the job.

When adding a partial selection to the dataset, the job stays in its current state with updated counts so labelers can continue working on the remaining images.

These controls are also available in the annotation editor. When viewing an annotated image in an assigned job:

• Use the checkmark button to add the current image to the dataset.

• Use the "Send to Unannotated" button (arrow icon) next to the image switcher to send the current image back for re-labeling.

After either action, the editor advances to the next image automatically.

Review Mode

You can individually approve or reject labeled images and send them back to the labeler for rework when necessary. To do so, click on a batch of images. Then, navigate between the Approved, Rejected, and To Do tabs to view the state of images in a job.