Table of Contents

Video classification

Alex Cota Updated by Alex Cota

This document outlines how to import video data, configure the editor for labeling video frames, navigate the video labeling interface, and export video data.

To request access to Video classification, reach out to us here.

This initial release of video classification supports checklist and radio classification of one or more frames simultaneously. Currently, there are some limitations that we are actively working to fix. Please read the following carefully before using video classification.

  • Only checklist and radio global classifications are supported. Object types (bounding box, polygon, polyline, point, masks), object subclassifications, text classification, dropdown classification, and nested radios are not yet supported. However, you won't be restricted from seeing these unsupported options in the "Configure editor" step.
  • For larger videos, you may experience slow upload times and slow loading times in the labeling interface.
  • Creating video labels via the API is not yet supported.
  • Exports for mixed media is not yet supported. Do not put video data and image data together in a project.
  • Project analytics do not yet support video frames.
  • The import file and export label formats (see below) are subject to change.

How to access Video classification

The video labeling interface is integrated into the existing Image editor. Therefore, the steps for importing video data are the same as importing image data.

Create your project

Step 1: Import your video data

First, create your project in Labelbox. Make sure your video file meets the following criteria before importing.

Max frames per video

2000 frames

Max resolution

1080p

Max frame rate

30 fps

Format

mp4

If your video has more than 2000 frames, please break your video into smaller clips and label the clips separately.

You have three options for importing your data.

a. Import directly

See Direct upload for instructions on importing your video files directly at https://app.labelbox.com/new-data.

b. Import via JSON

Pass a URL to your cloud-hosted mp4 file as the "videoUrl" in the JSON file. You have three options for importing your JSON file to Labelbox.

[
{
"externalId": "file-1.mp4",
"videoUrl": "<URL_TO_MP4_FILE>"
},
{
"externalId": "file-2.mp4",
"videoUrl": "<URL_TO_MP4_FILE>"
}
]

c. Import via CSV

If you want to pass URLs to video files, create a CSV and put all of the mp4 URLs in one column and create an additional optional column for the external IDs. When you import your CSV in the web app, you will be able to indicate the column to label and the column containing the external IDs.

Step 2: Configure the Image editor for video

After you attach your dataset, select Image editor as your labeling interface. Image editor (legacy) will not work.

This early release only supports global classifications of selected video frames. One frame can have 0 or more global classifications. Users are not limited to any number of classifications in the ontology.

When configuring the ontology, make sure to select checklist or radio as your the global classification type. You will not be restricted from seeing the dropdown and text classification types, however, they are not yet supported.

Label video data

The Labelbox labeling interface will have some additional tools for labeling video data.

Zoom bar

Drag the endpoints on the zoom bar to adjust the amount of frames selected in the timeline. You can also drag the selection across the zoom bar to move to a different part in the timeline.

Timeline

The timeline shows the temporal progression of the video clip. You can hover over a frame in the timeline to preview it.

A single frame can have 0 or more classifications. You can assign a classification by selecting one or more frames in the timeline and choosing a classification from the left sidebar.

Note: when you select a set of frames on the timeline, the left sidebar will only show the active classification state when the classification is true for all frames selected.

Below are some helpful shortcuts for selecting frames.

Shift+click

Select a set of consecutive frames on the timeline by clicking the first and last frame in the set.

Cmd/Ctrl+click

Select individual frames on the timeline.

Left/Right arrows

Seek the previous/next frame.

Shift+Left/Right arrows

Add the previous/next frame to the selection.

Export video data

When you export your Labels from a project, the export file will contain one Label per video file. If your project has multiple video files, all Labels (each containing all the features on a video file) will be included in the export file.

Export format

A single video label contains n frames. The export file will only contain frames with classifications, meaning that frames with no classifications will be excluded from the export.

The frames included in the export file will contain an id for each included frame, and the frame id corresponds to a lookup dict under features.

# Example export file
{
frames: [
{
frameNumber: 0,
features: [
{id: "${checkListQuestionId}"},
{id: "${checkListOptionOneId}"}
],
},
],
features: {
[${checkListQuestionId}]: {
schema: "<SCHEMA-ID>",
kind: "checklist question",
definition: {
value: "video_attrs",
title: "Check all that apply to the frame"
},
frames: [0]
},
[${checkListOptionOneId}]: {
schema: "<SCHEMA-ID>",
kind: "checklist option",
definition: {
value: "image_blur",
title: "Blurry"
},
frames: [0]
}
}
}

Was this page helpful?

Multi-frame bounding box

Contact