SDK Changes: A new way to queue & review

What is changing?

Labelbox is deprecating QueueMode, meaning all projects will be required to use Batch-based queueing for all projects by the end of Q1 2023. We will release a new SDK version that will automatically set up all new projects with this new functionality.

Labelbox will release the following changes:

Free / EDU / Starter customers can expect the changes on November 21st.

Pro and Enterprise customers can expect the changes on a rolling basis starting December 14th.

Automatic updates

• If you do not specify a queueing mode when creating a project, the queueing mode will default to Batches unless you explicitly opt out (set queue mode to Dataset to opt out).
• If you do not specify a quality mode at project creation, the quality mode will default to Benchmarks.

Note: Projects created before queue mode deprecation will still use dataset-based queueing. Dataset-based projects will be migrated to batches by the end of Q1 2023.

SDK upgrade required

• All new projects will require media type upon creation.
• Once you add a batch to a project, you cannot update the Consensus configurations on that batch.
• If you enable Consensus for your project, you will be able to set the Data Row priority in the label queue and number of times a Data Row gets labeled at the batch level. New projects created after this date will not support Labeling Parameter Overrides as it will be obsolete.
• You will be able to set data row priority, % coverage, and # labels at the batch level (for Consensus projects)
• If you want to add a batch of up to 100k data rows, you'll need to upgrade to the latest version of the SDK (see changelog)

By mid-December

• You may add your entire dataset to a batch as long as the batch does not exceed 100k data rows. There is no limit to how many batches you can add to a project.

Q1 2023

• QueueMode will be removed, meaning you will no longer be presented the option to select Dataset-based queueing via the SDK. Batch mode will automatically be configured for queueing Data Rows to a project.
• Labelbox will throw an exception if you attempt to create a project and do not specify the media type.
• Labeling Parameter Overrides (LPO) will no longer be supported for any project as it will be replaced by setting priority and number of labels at the batch level.
• The old Review step will no longer be supported for any project. This functionality will be replaced by “send to task” within Workflows.
• Bulk actions in Workflows will be available.

What action do I need to take?

On the appropriate date, upgrade your Labelbox Python SDK version to begin using batch-based consensus for all new projects.

In mid-Q1 2023, schedule the migration for all of your dataset-based projects created before QueueMode deprecation. Labelbox will provide a detailed migration path for your dataset-based projects. No action required for now.

Why are we making these changes?

Adding an entire dataset to a project’s labeling queue is the old way of queueing Data Rows to a project. Sending batches of Data Rows from a dataset to a project’s labeling queue provides your team with flexibility to build more complex review pipelines for your team.

Where can I learn more?

For more information, visit our Migration guide.