SDK Changes: A new way to queue & review

What is changing?

Labelbox is deprecating QueueMode, meaning all projects will be required to use Batch-based queueing for all projects by the end of Q1 2023. We will release a new SDK version that will automatically set up all new projects with this new functionality.

Labelbox will release the following changes:

Free / EDU / Starter customers can expect the changes on November 21st.

Pro and Enterprise customers can expect the changes on a rolling basis starting December 14th.

Automatic updates

If you do not specify a queueing mode when creating a project, the queueing mode will default to Batches unless you explicitly opt out (set queue mode to Dataset to opt out).
If you do not specify a quality mode at project creation, the quality mode will default to Benchmarks.

Note: Projects created before queue mode deprecation will still use dataset-based queueing. Dataset-based projects will be migrated to batches by the end of Q1 2023.

SDK upgrade required

All new projects will require media type upon creation.
Once you add a batch to a project, you cannot update the Consensus configurations on that batch.
If you enable Consensus for your project, you will be able to set the Data Row priority in the label queue and number of times a Data Row gets labeled at the batch level. New projects created after this date will not support Labeling Parameter Overrides as it will be obsolete.
You will be able to set data row priority, % coverage, and # labels at the batch level (for Consensus projects)
If you want to add a batch of up to 100k data rows, you'll need to upgrade to the latest version of the SDK (see changelog)

By mid-December

You may add your entire dataset to a batch as long as the batch does not exceed 100k data rows. There is no limit to how many batches you can add to a project.

Q1 2023

QueueMode will be removed, meaning you will no longer be presented the option to select Dataset-based queueing via the SDK. Batch mode will automatically be configured for queueing Data Rows to a project.
Labelbox will throw an exception if you attempt to create a project and do not specify the media type.
Labeling Parameter Overrides (LPO) will no longer be supported for any project as it will be replaced by setting priority and number of labels at the batch level.
The old Review step will no longer be supported for any project. This functionality will be replaced by “send to task” within Workflows.
Bulk actions in Workflows will be available.

What action do I need to take?

On the appropriate date, upgrade your Labelbox Python SDK version to begin using batch-based consensus for all new projects.

In mid-Q1 2023, schedule the migration for all of your dataset-based projects created before QueueMode deprecation. Labelbox will provide a detailed migration path for your dataset-based projects. No action required for now.

Why are we making these changes?

Adding an entire dataset to a project’s labeling queue is the old way of queueing Data Rows to a project. Sending batches of Data Rows from a dataset to a project’s labeling queue provides your team with flexibility to build more complex review pipelines for your team.

Where can I learn more?

For more information, visit our Migration guide.