Table of Contents

Multi-frame bounding box

Alex Cota Updated by Alex Cota

With multi-frame bounding boxes, you can apply a bounding box to a series of frames and manipulate the size and the position of the bounding box across the frame sequence. You can also add nested radio and checklist classifications to objects.

There are some limitations with this initial release. Please read the following carefully before using multi-frame bounding boxes.

Not supported

  • Text or dropdown classifications
  • Toggle on/off bounding box on a selection of frames
  • Undo/redo actions
  • Change an existing bounding box class
  • Copy/paste a bounding on multiple frames at a time
  • Create multi-frame bounding boxes via the API
  • Exports for mixed media (e.g. videos & images in the same project)

Import your video data

First, create your project in Labelbox and ensure your videos are in .mp4 format.

By default, you'll be able to load videos with up to 2000 frames and 30fps. Please contact support if you need to work with larger videos.

You have two options for adding your video files to Labelbox:

  1. Direct upload files via app or via Python SDK.
  2. Import URLs via JSON or CSV. Note: private data will require signed URLs. To see the JSON video import format, see Import via JSON - Video import sample.
    1. Upload JSON or CSV containing URLs via app.
    2. Bulk import URLs in JSON via the GraphQL API.
    3. Bulk import URLs in JSON via the Python API.

Configure bounding box for video

Project setup
  1. Create a project.
  2. Select "Editor" as your label editor.
  3. From the "Configure editor" window, click "Add object" and name the object class.
  4. Select "bounding box" as your labeling tool.
  5. Click "Confirm".
  6. Click "Complete setup".
Nested classifications

The nested classification types supported for multi-frame bounding boxes are radio and checklist only. Text and dropdown classification types are not supported.

In the configure editor step, you can create nested classifications within your bounding box objects by clicking the right carrot next to the object. You’ll also have the option to make the classification question(s) required.

How it works

There are three main components to multi-frame bounding boxes:

  • The selected frames to persist the bounding box
  • The size of the bounding box
  • The bounding box position on the frame(s)

A keyframe is a frame on the timeline where a change occurs. Interpolated (tweened) frames the ones between two keyframes. Labelbox uses interpolation to incrementally adjust the size and position of bounding boxes between keyframes.

Select frames

Apply a bounding box at a single keyframe

By default, when you apply a bounding box when a single frame is selected, that bounding box will persist with the same size and position across all subsequent frames until the end of the video.

Apply a bounding box to a set of frames

When you select a set of frames on the timeline and apply a bounding box to the selection, the bounding box will persist with the same size and position across each frame in the selection.

Toggle on/off a bounding box

When an item in the video goes out of frame then reappears 1 or more frames later, use the toggle icon to turn on/off the bounding box for that item.

Toggle on/off is only possible when a single frame is selected. You cannot toggle the bounding box on or off when multiple frames are selected.

Copy/paste a bounding box

When you copy and paste a bounding box from one frame to another, its position and size are preserved. A new bounding box instance is created out of the copied object and, by default, the bounding box will persist until the last frame in the video.

Copy/paste on multiple frames at a time is not supported.

Edit location and size

Update bounding box position and/or size

When you change the position/size of the bounding box in the middle of the sequence, Labelbox uses interpolation (also known as tweening) to incrementally adjust the bounding box to its new position/size. The updated bounding box position/size will persist until the end of the sequence.

Insert bounding box between interpolated frames

When you adjust the size and/or position of a bounding box between two already interpolated keyframes, Labelbox will interpolate the frames to adjust for the newly inserted keyframe.

Delete bounding box

Right click on the bounding box to delete the entire instance of the bounding box in every frame it appears. You can also do this with the backspace key.

Create instances

When a video is loaded in the labeling interface, your labelers can use the same class for more than one annotation with the following steps:

  1. Select the bounding box class from the "Tools" panel.
  2. Select the same bounding box class again.
  3. Draw the next instance of the bounding box.

Export labels

For instructions on how to access your multi-frame point labels and a sample Multi-frame bounding box label export, see our docs on Label export formats - video.

Was this page helpful?

Multi-frame point