Pose and motion signal for markerless motion capture AI
Problem
Move.ai's AI application needed to accurately identify people and classifications on a person (e.g. a number on a jersey) and recognize human limbs at different distances and poses, often occluded by limbs or objects.
Solution
Labelbox's video annotation interface and Python SDK to produce the signal, with the Labelbox workforce for rapid throughput.
Result
Labelbox's data engine gave Move.ai faster iterations on its algorithms, helping it move 2x as fast in this domain and accelerate its go-to-market and product launches.
Move.ai builds markerless motion capture from ordinary video. Labelbox produced the pose and spatial signal across multiple models and helped the team iterate 2x faster.
The challenge
High-quality video content — film, games, live broadcast — has been cost-prohibitive, made by studios with the best equipment and animation talent. Move.ai, a London AI company, is changing that by making motion capture and key-point recognition faster, cheaper, and markerless. Its markerless motion capture technology, aim, uses computer vision to extract granular motion data from ordinary video — one client analyzes soccer matches to report each player's strengths, weaknesses, and a coaching plan. Its depth-keying tool, ai key, replaces green screens, changing foreground and background to place a subject in an extended-reality environment. Training those models is hard: the algorithms must track a target across keyframes, identify the people and objects it contacts, extract spatial data, recognize classifications like a jersey number, and recognize human limbs at different distances, poses, and occlusions. With multiple AI products, Move.ai needed training signal across many use cases and algorithms, fast.
The approach
Move.ai used Labelbox to produce the signal for its models and iterate quickly. The video annotation interface and Python SDK let the team produce pose and spatial signal, and Labelbox's Boost workforce delivered a high standard of accuracy with quick turnaround. After producing signal in Labelbox, Move.ai exports to TensorFlow or PyTorch to train, and uses TensorRT for production — on Nvidia RTX GPUs for research and T4s for production.
What Labelbox has massively helped us with is fast iterations on our algorithms, helping us move twice as fast in this domain. The results we received from it are magnificent and their labeling user interface is the best we’ve seen for supporting our annotations efforts.
— Mark Endemano, CEO of Move.ai
The outcome
Labelbox gave Move.ai fast iterations on its algorithms — moving 2x as fast in this domain and accelerating its go-to-market and product launches.
Where this goes
Markerless motion capture is spatial intelligence from raw video. The pose and motion signal that trains it is the same grounding signal robots and embodied models need to understand how bodies move through the world.