Table of Contents

Model-assisted labeling

Alex Cota Updated by Alex Cota

Model-assisted labeling workflow

The Model-assisted labeling workflow allows you to import computer-generated predictions (or simply annotations created outside of Labelbox) and load them as editable annotations on an asset. This can be useful for speeding up the labeling process and supporting human labeling efforts.

Model-assisted labeling is only available for Pro and Enterprise customers.

The Model-assisted labeling workflow described in this page is only supported in the new Editor. If you are still using the legacy Editor, please refer to our Legacy editor docs.

Getting started

  1. Make sure you have the proper authentication.
  2. Create a project.
  3. Create your dataset and attach data rows.
  4. Use the sample script below to get the IDs for your data rows.
from labelbox import Client

client = Client(api_key="<LABELBOX_API_KEY>")
project = client.get_project("<PROJECT_ID>")

project.get_dataset("<DATASET_ID>")

for data_row in dataset.data_rows():
data_row.uid
  1. Get the project ontology. When you create your import file, you will need to reference the schemaId for each annotation type in your project's ontology. To get the value for eachschemaId, use this sample script. Keep this ontology handy as you will need to reference the featureSchemaId values from this query in your import file.
def get_ontology(client: Client, project_id: str) -> Dict[str, str]:
result = client.execute("""
query get_ontology($proj_id: ID!) {
project(where: {id: $proj_id}) {
ontology {
normalized
}
}
}
""", {"proj_id": project_id})
return result['project']['ontology']['normalized']
  1. Turn on Model-assisted labeling. When an asset is loaded in the Editor, any predictions for that asset will show up as editable annotations for the user. Imported annotations will appear as long as the following conditions are met:
  • Model-assisted labeling is toggled on under Settings > Automation.
  • The imported annotations are assigned to a Data Row.
  • The asset has not already been labeled in the Labelbox Editor.

You can also turn on Model-assisted labeling for a project programmatically via the Python SDK.

from labelbox import Client

client = Client("<Labelbox_API_Key>")
project = client.get_project("<Project_ID>")

project.enable_model_assisted_labeling(True)

Create the import file

To learn how to format your import file, see MAL import formats.

Import annotations

After you create your import file, the best way to import your annotations is via the Python SDK. Currently, annotations must be imported programmatically as importing via the UI is not supported.

To learn how to bulk import using the upload_annotations method in the Python SDK, see Import annotations.

Wait until the import job is complete before opening the Editor to make sure all annotations are imported properly.

Update/delete imported annotations

When a labeler submits an asset with imported annotations, submitted annotations cannot be overwritten with new imported annotations without being deleted first.* To update annotations with new imported annotations, you will need to:

  1. Delete the Label from the Data Row.
  2. Import the annotations with the same uuids.
  3. Load the asset again to see the updated annotations.

*If you label an asset, delete the Label, then keep the Label as a template, that label template will take precedence over any imported annotations for that asset.

When a labeler skips an asset with imported annotations, the annotations will not be saved to the asset. Next time a user loads the asset in the labeling interface, the imported annotations will be loaded on the asset again.

Refer to deleting an import request in order to delete all annotations imported by a specific request.

End-to-end tutorial

Click here for an end-to-end tutorial that walks you through each step in the Model-assisted labeling workflow.

Was this page helpful?

MAL import formats

Contact