Environments for post-training, at scale
RL training gyms & evals for reasoning, tool use, and computer use. Built for the domains where AI creates the most economic value.
RL environments for the hardest problems in AI
Software generated RL environments at scale. Delivering calibrated reward signals and target pass@k optimization across the most valuable knowledge work domains.
The simulation platform for enterprise knowledge work
WorldSim recreates the enterprise environments where agents work, from GitLab and Jira to CRM, email, and chat, using realistic business data. Configurable scenarios generate diverse tasks at scale, from outages to workflow changes.
Agents navigate hundreds of MCP tools across computer and terminal tasks, with evaluation tuned to your pass@k targets and RL infrastructure
The reward signal problem, solved at scale
Building RL environments that produce useful reward signals is hard. You need the right tasks, verification, difficulty progression, and credit assignment. Get any of it wrong and models learn shortcuts instead of real capabilities.
Labelbox’s software helps generate RL environments that capture this expertise, calibrated to your reward goals and pass@k targets, at the scale needed for post training.
Teaching models taste
Robert Pirsig called it Quality: something you can recognize before you can fully define. Preference labels capture that signal. They turn human judgment into structured comparisons across agent trajectories, helping models learn what makes a long horizon response genuinely good, not just technically correct.
Operating across most economically valuable domains
Autonomous AI research
Long-horizon reasoning tasks with intermediate reward signals. Multi-step hypothesis generation, verification, and structured knowledge synthesis. Environments co-designed with domain experts.
Agent coding & software engineering
Long-horizon software tasks across real codebases — debugging production failures, authoring PRs, navigating full SDLC workflows. Agents operate on real code with real consequences, not toy problems.
Multimodal knowledge work
Tasks spanning text, images, charts, documents, and structured data — requiring agents to reason across modalities within a single workflow. Built for the full complexity of real knowledge work.
Voice with agentic tool use
Voice-native agents that reason, plan, and execute tool calls mid-conversation. Tasks designed for the latency, interruption, and context challenges unique to voice interfaces.
Computer use
GUI-based task execution across enterprise software. Verifiable multi-step outcomes in environments that mirror production toolchains.
Cybersecurity
Attack and defense scenarios. CTF-style tasks with programmatic verification. Adversarial edge cases designed to surface brittleness.
Hundreds of AI teams build with Labelbox
How Meta built GIM with Labelbox data to evaluate frontier AI reasoning
Problem
Meta needed a benchmark that remained discriminative as existing LLM evaluations saturated. The team wanted tasks grounded in practical reasoning rather than obscure knowledge or synthetic puzzles, with enough rubric detail to capture partial credit and enough quality control to support a public-private contamination diagnostic.
Solution
Labelbox produced the data foundation for GIM: 820 expert-authored problems across seven cognitive categories, including 229 multimodal items and 528 rubric-graded prompts. The work included original prompt creation, structured scoring criteria, review, annotation, and quality assurance, enabling Meta to calibrate a 2PL IRT model over more than 200,000 prompt-response pairs.
Result
Meta released GIM-615, calibrated item parameters, and an evaluation framework that benchmarked 22 models across 47 reporting configurations. The paper found GIM remains far from saturated, with roughly 20% of items above frontier ability, giving researchers a durable way to compare model capability, thinking budgets, and future systems.
Human preference signal for evaluating LLMs inside Vertex AI
Tracking surgical instruments in video to advance robotic surgery
