Table of Contents

MAL import formats

Alex Cota Updated by Alex Cota

Use the examples in this page to learn how to format your import files for the Model-assisted labeling (MAL) workflow.

Your import file must be in newline delimited (NDJSON) format. The samples in this document are prettified for legibility. When you prepare your annotations for import, each newline in your NDJSON file should be the entirety of a JSON object.

Images

For images, the Model-assisted labeling workflow supports all annotation types except Dropdown classification and classifications nested within classifications.

Mask annotations

Each mask color in your import file should match the corresponding mask color on the image. Passing a URI/mask color pair for a mask color that doesn’t exist in the image will generate an error and no annotation will be created.

For example, to import in a mask of the white road markings onto the sample image here, you would indicate a color of: [255, 255, 255].

This sample import attaches 3 mask annotations to a single datarow. Before you import, make sure the masks and data row dimensions match. See a full sample import file here.

{ 
"uuid": "45b15f9d-7884-4bb7-ac01-3567e8ed6c36",
"schemaId": "ck68grts29n7w0890wv344dif", # mask 1 schema id
"dataRow": {
"id": "cjxav5aa07r1g0dsq70t9eveg"
},
"mask": {
"instanceURI": "https://api.labelbox.com/masks/feature/ck7a0jw4o0nk80x9o5offz4mc",
"colorRGB": [
255,
255,
255
]
}
}
{
"uuid": "3a95ddcd-3ad0-4dc5-a24e-c05004b4b4d5",
"schemaId": "ck7wi85rnd1050757aac5ba4d", # mask 2 schema id
"dataRow": {
"id": "cjxav5aa07r1g0dsq70t9eveg"
},
"mask": {
"instanceURI": "https://api.labelbox.com/masks/feature/ck7a0jw4o0nk80x9o5offz4mc",
"colorRGB": [
255,
0,
0
]
}
}
{
"uuid": "f8284cbc-ecf3-4363-9e10-138501daf5f7",
"schemaId": "ck7wi85pr1xz6079026yc0hch", # mask 3 schema id
"dataRow": {
"id": "cjxav5aa07r1g0dsq70t9eveg"
},
"mask": {
"instanceURI": "https://api.labelbox.com/masks/feature/ck7a0jw4o0nk80x9o5offz4mc",
"colorRGB": [
0,
0,
0
]
}
}

Where:

  • uuid is a user-generated UUID for each annotation. See the example below for a sample uuid. The following formats are supported:
    • A0EEBC99-9C0B-4EF8-BB6D-6BB9BD380A11
    • {a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11}
    • a0eebc999c0b4ef8bb6d6bb9bd380a11
    • a0ee-bc99-9c0b-4ef8-bb6d-6bb9-bd38-0a11
    • {a0eebc99-9c0b4ef8-bb6d6bb9-bd380a11}
  • schemaId is the ID of the schema that contains all of the information needed for rendering your annotation. To get the value for schemaId, use the sample script below and copy the value for featureSchemaId.
  • dataRow is the ID of the Data Row where you want to attach the imported annotations.
  • instanceURI is the cloud-hosted mask (can be hosted on any cloud provider). If you are importing multiple mask predictions on one Data Row, each mask should reference the same instanceURI.
  • colorRGB is an array of RGB values from 0 to 255 that indicates which color represents each given mask. Only 3-channel RGB colors are supported.

Vector annotations

The vector annotation types supported in Model-assisted labeling are a Bounding box, Polygon, Point, and Polyline. Note: The geometry format in the import file matches the geometry format in the export file for each type. See a full sample import file here.

{
"uuid": "efca0c21-5206-4da6-8cb5-d6ca43649cfa",
"schemaId": "ck67grts29n7x0890atmeiahw",
"dataRow": {
"id": "cjxav4aa07r1g0dsq70t9eveg"
},
"bbox": {
"top": 153,
"left": 34,
"height": 204,
"width": 67
}
}
{
"uuid": "1b5762e9-416c-44cf-9a5f-07effb51f863",
"schemaId": "ck67grts29n7y0890q89jdcyp",
"dataRow": {
"id": "cjxav4aa07r1g0dsq70t9eveg"
},
"polygon": [
{
"x": 2,
"y": 99
},
{
"x": 93,
"y": 5
},
{
"x": 51,
"y": 106
},
{
"x": 176,
"y": 142
}
]
}
{
"uuid": "62e1d949-1c75-47f6-9ea2-e938da17d37c",
"schemaId": "ck68grts29n7z08903nvgaim5",
"dataRow": {
"id": "cjxav5aa07r1g0dsq70t9eveg"
},
"line": [
{
"x": 58,
"y": 148
},
{
"x": 135,
"y": 79
},
{
"x": 53,
"y": 191
}
]
}
{
"uuid": "532953e6-746f-4d74-945d-b4a9c2786479",
"schemaId": "ck68grts29n800890roip3u5d",
"dataRow": {
"id": "cjxav5aa07r1g0dsq70t9eveg"
},
"point": {
"x": 30,
"y": 150
}
}

Where:

  • uuid is a user-generated UUID for each annotation. See the example below for a sample uuid. The following formats are supported:
    • A0EEBC99-9C0B-4EF8-BB6D-6BB9BD380A11
    • {a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11}
    • a0eebc999c0b4ef8bb6d6bb9bd380a11
    • a0ee-bc99-9c0b-4ef8-bb6d-6bb9-bd38-0a11
    • {a0eebc99-9c0b4ef8-bb6d6bb9-bd380a11}
  • schemaId is the ID of the schema that contains all of the information needed for rendering your annotation. To get the value for schemaId, use the sample script below and copy the value for featureSchemaId.
  • dataRow is the ID of the Data Row where you want to attach the imported annotations.

Classifications

You can import Classification annotations on images in either of the following ways:

  • Classification on the image itself (AKA Global classification)
  • Classification within an object annotation on the image (AKA Classification within Object)

There are 3 classification types supported in Model-assisted labeling (Radio, Checklist, and Free-form text). Each Classification type references a schema for the question and a separate schema for each answer option (except Text answers).

When importing Classification annotations on an image, you must include the following information.

  • uuid is a user-generated UUID for each annotation. See the example below for a sample uuid. The following formats are supported:
    • A0EEBC99-9C0B-4EF8-BB6D-6BB9BD380A11
    • {a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11}
    • a0eebc999c0b4ef8bb6d6bb9bd380a11
    • a0ee-bc99-9c0b-4ef8-bb6d-6bb9-bd38-0a11
    • {a0eebc99-9c0b4ef8-bb6d6bb9-bd380a11}
  • schemaId is the ID of the schema that contains all of the information needed for rendering your annotation. To get the value for schemaId, use the sample script below and copy the value for featureSchemaId.
  • dataRow is the ID of the Data Row where you want to attach the imported annotations.
The import format should mirror the export format.

In addition, you should specify the answer schemas. Below are the answer schema details for each Classification type.

Radio

Radio is the only classification type that supports child classifications. The child of a Radio classification can be a Radio, Checklist, or Text classification.

A Radio classification with a child Checklist classification would be structured as follows:

Radio question / Answer / Child Checklist question / Answer

For example:

Is there a car in the image? / Yes / What color is the car? / Blue

This example includes a Radio classification with a child Checklist classification.

{
"schemaId": "ckd11j3yk000c0z0u4xn6dc4r", # radio question schema id
"uuid": "1278daa6-ce64-4363-be24-4fa5eadffb17",
"dataRow": {
"id": "ckd11jg6scq9c0cq43vmh6i07"
},
"answer": {
"schemaId": "ckd11j415000u0z0ubu7ee4w2", #answer schema id
"classifications": [
{
"schemaId": "ckd1295j9006k0z0udz3rh4mp", # checklist question schema id
"answers": {
"schemaId": "ckd1295jk006y0z0u1sk8h075" # answer schema id
},
{
"schemaId": "ckd1295hh006g0z0ucbxgfgec" # answer schema id
}
}
]
}
}

Where:

  • answer is singular because a Radio classification can only have one correct answer.
Checklist

This example includes a Checklist classification with two correct answers specified.

{
"schemaId": "ckd1295hc00640z0uapvm1xbd", #question schema id
"uuid": "fb72782d-f6ed-43ba-8677-77b03197392d",
"dataRow": {
"id": "ckd1299m8cqbs0cq43mju1bvp"
},
"answers": [
{
"schemaId": "ckd1295jn00760z0u01hw4yz5" #answer schema id
},
{
"schemaId": "ckd1295hh006g0z0ucbxgfgec" # answer schema id
}
]
}

Where:

  • answers is plural because Checklist classifications can have multiple correct answers.
Free-form text

This example includes a Free-form text classification. Since the free-form text answer must be an exact match, there is no answer schema to reference and you must enter the correct text answer upon import.

{
"schemaId": "ckd1295hg006c0z0u6x41hx0d", #question schema id
"uuid": "4f1fe322-7b80-49a1-81cb-5914404df378",
"dataRow": {
"id": "ckd1299m8cqck0cq42lsz5khc"
},
"answer": "<correct_text_answer>"
}

Classifications within Objects

For Classifications nested within Objects, the nested classifications are stored in an array under the key "classifications".

This example includes a Radio classification nested within a Bounding box.

{
"uuid": "6e20a5ec-613d-4ecd-8fe5-34e47a05fea8",
"schemaId": "ckd1295hh006e0z0uh2x32i82", # bounding box schema id
"dataRow": {
"id": "ckd1299m8cqc00cq4c1by8hza"
},
"bbox": {
"top": 216,
"left": 144,
"height": 69,
"width": 67
},
"classifications": [
{
"schemaId": "ckd1295j9006k0z0udz3rh4mp", # question schema id
"answer": {
"schemaId": "ckd1295jk006y0z0u1sk8h075" # answer schema id
}
}
]
},

Videos

For videos, the Model-assisted workflow only supports Classifications at the frame-level.

Multi-frame classifications

The Model-assisted labeling workflow supports global classifications for video labeling.

The 2 classification types supported in Model-assisted labeling for videos are Radio and Checklist. In your JSON import, each Classification type should reference a schema for the question and a separate schema for each answer option.

When importing multi-frame classifications, you must include the following information:

  • uuid is a user-generated UUID for each annotation. See the example below for a sample uuid. The following formats are supported:
    • A0EEBC99-9C0B-4EF8-BB6D-6BB9BD380A11
    • {a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11}
    • a0eebc999c0b4ef8bb6d6bb9bd380a11
    • a0ee-bc99-9c0b-4ef8-bb6d-6bb9-bd38-0a11
    • {a0eebc99-9c0b4ef8-bb6d6bb9-bd380a11}
  • schemaId is the ID of the schema that contains all of the information needed for rendering your annotation. To get the value for schemaId, use the sample script below and copy the value for featureSchemaId.
  • dataRow is the ID of the Data Row where you want to attach the imported annotations.
  • start: The index of the first frame you wish to include in the classification.
  • end: The index of the last frame you wish to include in the classification.

The samples below contain the answer schemas for each Classification type.

Checklist

This example includes a Checklist classification on two frame selections.

{
"schemaId": "ckd1295hc00640z0uapvm1xbd", # question schema id
"uuid": "fb72782d-f6ed-43ba-8677-77b03197392d",
"dataRow": {
"id": "ckd1299m8cqbs0cq43mju1bvp"
},
"answers": [
{
"schemaId": "ckd1295jn00760z0u01hw4yz5" # answer schema id
}, {
"schemaId": "ckd1295hh006g0z0ucbxgfgec" #answer schema id
}
],
"frames": [
{
"start": 7,
"end": 13,
},
{
"start": 18,
"end": 19,
}
]
}

Where:

  • answers is plural because Checklist classifications can have more than one correct answer.
Radio

This example includes a Radio classification on two sets of frames. Nested Radio classifications are not supported in the Model-assisted labeling workflow for videos.

This example includes a Radio classification on two frame selections.

{
"schemaId": "ckd1295hc00640z0uapvm1xbd", # question schema id
"uuid": "fb72782d-f6ed-43ba-8677-77b03197392d",
"dataRow": {
"id": "ckd1299m8cqbs0cq43mju1bvp"
},
"answer": {
"schemaId": "ckd1295jn00760z0u01hw4yz5" # answer schema id
},
"frames": [
{
"start": 7,
"end": 13,
},
{
"start": 18,
"end": 19,
}
]
}

Where:

  • answer is singular because a Radio classification can only have one correct answer.

Text

For text, the Model-assisted workflow only supports Named entity recognition.

Named entity recognition

You can speed up the process of labeling text in Labelbox by importing model-assisted Entity annotations. In order to import and attach Entity annotations to a Data Row, your import file should contain the following information:

  • uuid is a user-generated UUID for each annotation. See the example below for a sample uuid. The following formats are supported:
    • A0EEBC99-9C0B-4EF8-BB6D-6BB9BD380A11
    • {a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11}
    • a0eebc999c0b4ef8bb6d6bb9bd380a11
    • a0ee-bc99-9c0b-4ef8-bb6d-6bb9-bd38-0a11
    • {a0eebc99-9c0b4ef8-bb6d6bb9-bd380a11}
  • schemaId is the ID of the schema that contains all of the information needed for rendering your annotation. To get the value for schemaId, use the sample script below and copy the value for featureSchemaId.
  • dataRow is the ID of the Data Row where you want to attach the imported annotations.
  • location: Indicates the set of characters included in a single Entity annotation. Assumes zero-based indexing.
    • start: The index of the first character in the Entity annotation. Assumes start-index inclusion.
    • end: The index of the last character in the Entity annotation. Assumes end-index exclusion (character 128 in the example below would be excluded from the Entity annotation).

This sample attaches a single Entity annotation to a Data Row containing text data.

{ 
"uuid": "9fd9a92e-2560-4e77-81d4-b2e955800092",
"schemaId": "ck8kukafkqx1a0880iczbrqym", # entity schema id
"dataRow": {
"id": "ck1s02fqxm8fi0757f0e6qtdc"
},
"location": {
"start": 67,
"end": 128
}
}

Was this page helpful?

Model-assisted labeling

Webhooks setup

Contact