Table of Contents

Multi-frame bounding box

Alex Cota Updated by Alex Cota

With multi-frame bounding boxes, you can apply a bounding box to a series of frames and manipulate the size and the position of the bounding box across the frame sequence. You can also add nested radio and checklist classifications to objects.

There are some limitations with this initial release. Please read the following carefully before using multi-frame bounding boxes.

Not supported

  • Text or dropdown classifications
  • Toggle on/off bounding box on a selection of frames
  • Undo/redo actions
  • Change an existing bounding box class
  • Copy/paste a bounding on multiple frames at a time
  • Create multi-frame bounding boxes via the API
  • Exports for mixed media (e.g. videos & images in the same project)

Import your video data

Make sure your video file meets the following criteria before importing.

Max frames per video

2000 frames

Max resolution

1080p

Max frame rate

30 fps

Format

mp4

JSON import format
[
{
"externalId": "file-1.mp4",
"videoUrl": "<URL_TO_MP4_FILE>"
},
{
"externalId": "file-2.mp4",
"videoUrl": "<URL_TO_MP4_FILE>"
}
]

Configure bounding box for video

Project setup
  1. Create a project.
  2. Select "Image editor" as your label editor.
  3. From the "Configure editor" view, click "Add object" and name the object class.
  4. Select "bounding box" as your labeling tool.
  5. Click "Confirm".
  6. Click "Complete setup".
Nested classifications

The nested classification types supported for multi-frame bounding boxes are radio and checklist only. Text and dropdown classification types are not supported.

In the configure editor step, you can create nested classifications within your bounding box objects by clicking the right carrot next to the object. You’ll also have the option to make the classification question(s) required.

How it works

There are three main components to multi-frame bounding boxes:

  • The selected frames to persist the bounding box
  • The size of the bounding box
  • The bounding box position on the frame(s)

A keyframe is a frame on the timeline where a change occurs. Interpolated (tweened) frames the ones between two keyframes. Labelbox uses interpolation to incrementally adjust the size and position of bounding boxes between keyframes.

Select frames

Apply a bounding box at a single keyframe

By default, when you apply a bounding box when a single frame is selected, that bounding box will persist with the same size and position across all subsequent frames until the end of the video.

Apply a bounding box to a set of frames

When you select a set of frames on the timeline and apply a bounding box to the selection, the bounding box will persist with the same size and position across each frame in the selection.

Toggle on/off a bounding box

When an items in the video goes out of frame then reappears 1 or more frames later, use the toggle icon to turn on/off the bounding box for that item.

Toggle on/off is only possible when a single frame is selected. You cannot toggle the bounding box on or off when multiple frames are selected.

Copy/paste a bounding box

When you copy and paste a bounding box from one frame to another, its position and size are preserved. A new bounding box instance is created out of the copied object and, by default, the bounding box will persist until the last frame in the video.

Copy/paste on multiple frames at a time is not supported.

Edit location and size

Update bounding box position and/or size

When you change the position/size of the bounding box in the middle of the sequence, Labelbox uses interpolation (also known as tweening) to incrementally adjust the bounding box to its new position/size. The updated bounding box position/size will persist until the end of the sequence.

Insert bounding box between interpolated frames

When you adjust the size and/or position of a bounding box between two already interpolated keyframes, Labelbox will interpolate the frames to adjust for the newly inserted keyframe.

Delete bounding box

Right click on the bounding box to delete the entire instance of the bounding box in every frame it appears. You can also do this with the backspace key.

Create instances

When a video is loaded in the labeling interface, your labelers can use the same class for more than one annotation with the following steps:

  1. Select the bounding box class from the "Tools" panel.
  2. Select the same bounding box class again.
  3. Draw the next instance of the bounding box.

Export format

When you export Labels from a project, the export file will contain on Label per video file. If your project has multiple video files, all Labels (each containing all the features on a video file) will be included in the export file. Frames without bounding boxes or classifications will not be included in the export file. Each frame will contain an id which corresponds to a lookup dict under features.

{ 
frames: [
{
frameNumber: 5,
features: [
{
id: "${objectWithNestedRadioId}",
geometry: {
type: 'Multipolygon',
coordinates: [[14, 37], [28, 37], [28, 19], [14, 19]]
}
}
]
}
],
features: {
[${objectWithNestedRadioId}]: {
schema: "<SCHEMA_ID>",
kind: "bounding box",
definition: {
color: "green",
name: "car"
}
}
}
}

Was this page helpful?

Video classification

Contact