Table of Contents
Updated by Alex Cota
With multi-frame bounding boxes, you can apply a bounding box to a series of frames and manipulate the size and the position of the bounding box across the frame sequence. You can also add nested radio and checklist classifications to objects.
There are some limitations with this initial release. Please read the following carefully before using multi-frame bounding boxes.
- Text or dropdown classifications
- Toggle on/off bounding box on a selection of frames
- Undo/redo actions
- Change an existing bounding box class
- Copy/paste a bounding on multiple frames at a time
- Create multi-frame bounding boxes via the API
- Exports for mixed media (e.g. videos & images in the same project)
Import your video data
First, create your project in Labelbox and ensure your videos are in .mp4 format.
You have two options for adding your video files to Labelbox:
- Direct upload files via app or via Python SDK.
- Import URLs via JSON or CSV. Note: private data will require signed URLs. To see the JSON video import format, see Import via JSON - Video import sample.
Configure bounding box for video
- Create a project.
- Select "Editor" as your label editor.
- From the "Configure editor" window, click "Add object" and name the object class.
- Select "bounding box" as your labeling tool.
- Click "Confirm".
- Click "Complete setup".
The nested classification types supported for multi-frame bounding boxes are radio and checklist only. Text and dropdown classification types are not supported.
In the configure editor step, you can create nested classifications within your bounding box objects by clicking the right carrot next to the object. You’ll also have the option to make the classification question(s) required.
How it works
There are three main components to multi-frame bounding boxes:
- The selected frames to persist the bounding box
- The size of the bounding box
- The bounding box position on the frame(s)
A keyframe is a frame on the timeline where a change occurs. Interpolated (tweened) frames the ones between two keyframes. Labelbox uses interpolation to incrementally adjust the size and position of bounding boxes between keyframes.
Apply a bounding box at a single keyframe
By default, when you apply a bounding box when a single frame is selected, that bounding box will persist with the same size and position across all subsequent frames until the end of the video.
Apply a bounding box to a set of frames
When you select a set of frames on the timeline and apply a bounding box to the selection, the bounding box will persist with the same size and position across each frame in the selection.
Toggle on/off a bounding box
When an item in the video goes out of frame then reappears 1 or more frames later, use the toggle icon to turn on/off the bounding box for that item.
Copy/paste a bounding box
When you copy and paste a bounding box from one frame to another, its position and size are preserved. A new bounding box instance is created out of the copied object and, by default, the bounding box will persist until the last frame in the video.
Edit location and size
Update bounding box position and/or size
When you change the position/size of the bounding box in the middle of the sequence, Labelbox uses interpolation (also known as tweening) to incrementally adjust the bounding box to its new position/size. The updated bounding box position/size will persist until the end of the sequence.
Insert bounding box between interpolated frames
When you adjust the size and/or position of a bounding box between two already interpolated keyframes, Labelbox will interpolate the frames to adjust for the newly inserted keyframe.
Delete bounding box
Right click on the bounding box to delete the entire instance of the bounding box in every frame it appears. You can also do this with the backspace key.
When a video is loaded in the labeling interface, your labelers can use the same class for more than one annotation with the following steps:
- Select the bounding box class from the "Tools" panel.
- Select the same bounding box class again.
- Draw the next instance of the bounding box.