Table of Contents


Alex Cota Updated by Alex Cota

When you label an image with the Segmentation tool, each annotated pixel in the image belongs to a single class. It is often used to label images for applications that require high accuracy. The output is a mask that outlines the shape of the item in the image.

Configure the Segmentation tool

During project setup, you can use a shared ontology, make a copy of an existing ontology, or create a new ontology.

To learn more about how ontologies work, see Ontology overview.

Setup steps
  1. Create a project.
  2. Select "Editor" as your label editor.
  3. Click "Add object" and name your object.
  4. Select "Segmentation" as your labeling tool.
  5. [OPTIONAL] Configure nested classifications.
  6. Click "Confirm".
  7. Click "Complete setup".

Pen tool

Use the Pen tool to outline items in the image. Hold the cursor down to draw freehand or let the cursor go to draw straight lines between points. Select the (-) icon to draw around an area to erase. The Pen tool is only available when creating Segmentation annotations.

Draw over existing objects

"Draw over existing objects" is on by default. When this tool is enabled, drawing a new Segmentation annotation over an existing annotation will overwrite previously classified pixels. When this tool is disabled, a new Segmentation annotation drawn over an existing annotation will be "drawn behind" the existing annotation.

This tool is designed to significantly speed up labeling time since it is not required to intricately outline around the border of other objects. This tool is only available when creating Segmentation annotations.


For Segmentation annotations with complex boundaries, using the Superpixel tool first may be more efficient than using the Pen tool alone. Superpixel works by calculating segment clusters of similarly colored pixels in the image. The Superpixel tool is only available when you are creating Segmentation annotations.

In the top toolbar is a slider, which allows you to adjust the size of segment clusters from XS to XL. After you have selected a segment cluster size, choose an object from the left Tools panel and use your cursor to select and classify each segment to be included in that Segmentation annotation. You can then adjust the boundaries using the Pen and Eraser tools.

Calculation time for Superpixel segments increases with image size. We advise using images that are no more than 4000 by 4000 pixels.

Creating instances

From the labeling interface, you can use the same class for more than one annotation. For example, if there are 5 fish in an image and you would like to assign the "Fish" class to all five, you can create multiple instances of the "Fish" class.

Follow these steps to create multiple instances of the same object:

  1. Select a class and draw the object.
  2. Select the same class again.
  3. Draw the next instance of the object.

Nested classifications

If you have configured the interface to have nested classifications for any of your objects, the labeler will be presented with classification questions after the annotation of the object.

Label format

When you export your labels as a JSON file, it will look like the sample below. The instanceURI is a URL to the image mask.

"Label": { 
"objects": [
"featureId": "ck9bmetz800340za5k073bm7w",
"schemaId": "ck9blmq1lnlxz08892qqlmxto",
"title": "sample-mask",
"value": "sample-mask",
"color": "#2BFF00",
"instanceURI": ""
"classifications": []

Convert mask to polygon coordinates

If you need to convert your vector masks to polygon coordinates, you can use this script here. You will need to pip install labelbox first. See our Getting started page in our Python docs for installation instructions.

Was this page helpful?

Image classification