Blog

Reflections on NeurIPS 2025: Advancing evaluation and continual learning in AI

Takeaways on the themes and research directions likely to shape the year ahead. We focus on two core areas how to rigorously measure AI capabilities and how to build interactive systems that learn through experience over time.

Implicit Intelligence and Agent‑as‑a-World: Evaluating agents on what users don’t say

Most real-world tasks are underspecified. We introduce Implicit Intelligence to test whether agents can infer hidden constraints, and Agent-as-a-World, a simple YAML framework for simulating environments without brittle, hard-coded worlds.

Bridging insight and innovation: Introducing Labelbox Applied Research

Today we’re launching Labelbox Applied Research with three flagship pillars: Labelbox Evals for unified model evaluation, Labelbox Agents for building reliable and interpretable agents, and Labelbox Robotics (LBRx) for delivering high-quality training data for advanced robotic manipulation.

Announcing R-ConstraintBench: A novel way to stress-test LLM reasoning abilities under interacting constraints

We've released a research paper on R-ConstraintBench, a novel benchmark for evaluating LLM reasoning on realistic resource-constrained project scheduling problems (RCPSP), a well-known NP-complete challenge.

Introducing Labelbox Evaluation Studio: Drive AGI advancements with real-time feedback on model performance

Labelbox Evaluation Studio unlocks a private, real-time platform where top AI teams unlock tailored insights, instantly spot strengths and weaknesses, and accelerate faster frontier model improvements.

Teaching agents to use tools with human supervision: MCP support now available

Meet Labelbox's MMC editor now with MCP support which enables human-in-the-loop evaluation by making it easy to inspect, label, and correct agentic-tool interactions.

Benchmarking deep research agents

Introducing Labelbox’s deep research leaderboard: an open, continuously‑updated scorecard that shows showing how top AI agents like OpenAI, Google, and Anthropic perform on long-form research tasks.

An economic report on the human expertise fueling frontier AI

Our latest report examines the emerging expert economy driving frontier AI, detailing the backgrounds and disciplines of these knowledge workers and their impact on cutting-edge AI systems. It also covers their earnings and the high-skill data crucial for the next stage of AI development.

Building true RL systems: An experiment on solving real business tasks

We tested rubric-based rewards and GRPO on a real-world e-commerce task and found they outperformed sparse rewards by 300%. This helps validate their effectiveness for complex, multi-step business workflows.

Agentic AI: What it takes to build AI that acts

Agentic AI is emerging as a new frontier in autonomy, where models can plan, adapt, and take action independently. In this post we highlight three real-world projects with leading AI labs, from multi step tool use to structured reasoning and dynamic instruction following.