Labelbox

Applied research

Reflections on NeurIPS 2025: Advancing evaluation and continual learning in AI

Ved Sirdeshmukh

Implicit Intelligence and Agent‑as‑a-World: Evaluating agents on what users don’t say

Ashley Wu

Bridging insight and innovation: Introducing Labelbox Applied Research

Announcements

Announcing R-ConstraintBench: A novel way to stress-test LLM reasoning abilities under interacting constraints

Introducing Labelbox Evaluation Studio: Drive AGI advancements with real-time feedback on model performance

Releases

Teaching agents to use tools with human supervision: MCP support now available

Benchmarking deep research agents

An economic report on the human expertise fueling frontier AI

Building true RL systems: An experiment on solving real business tasks

Use cases

Agentic AI: What it takes to build AI that acts

Benchmarking agentic search

Explore the shift from "golden answers" to multi-dimensional rubric evaluations for more precise and scalable AI assessment and development.

Rubric-based evaluations fuel the next wave of RL data

Rubric evaluations: Fueling the next wave of reinforcement learning

Overcome current AI app generator and prompt-to-app limitations with rubric evaluations and human-centric evaluations from Labelbox. 

How to improve AI app generators and prompt-to-app tools

Prompt to production: How to improve AI app generators with rubric evals

Learn how to improve RLVR pipelines with advanced training data to improve frontier AI model performance for complex tasks.

Reinforcement Learning from Verifiable Rewards (RLVR) | Labelbox guide

How to fill your RLVR pipeline with advanced reasoning data

Level up your AI model evaluation with the redesigned Labelbox Multimodal Chat editor, featuring a streamlined form-based layout and new minimap.

Discover the redesigned Labelbox Multimodal Chat Editor

Reinventing AI evaluation: Discover the simplicity of Labelbox's new form-based MMC editor

A new evaluation by Labelbox, the Complex Reasoning Leaderboard, highlights Gemini 2.5 Pro as the current leader in complex reasoning and CoT.

New Complex Reasoning Leaderboard: Gemini 2.5 Pro at the top

New Complex Reasoning Leaderboard: Gemini 2.5 debuts at the top

The new Labelbox Workflow introduces an interactive, node-based editor that makes it easy to create, manage, and visualize multi-step workflows.

Labelbox releases new Workflow experience for reviews

Introducing a powerful, new interactive Workflow editor

Catch up on Labelbox's latest innovations including expanded Leaderboards, the Alignerr Connect launch, and platform advancements empowering the next generation of AI models.

Labelbox Q1 Recap: From agent trajectories to leaderboards 

Q1 spotlight: Accelerating AI development with new products and services

Learn how to use Labelbox's Multimodal Chat Editor for the key tasks of agent training, using its new capabilities to evaluate agent trajectories

Train and evaluate AI agents and trajectories with Labelbox

How to train and evaluate AI agents and trajectories with Labelbox

Labelbox now supports six new AI models, enhancing tasks like speech recognition, multimodal reasoning, coding, and more, helping users automate data tasks.

New AI models in Labelbox: Nova Pro, Gemini 2.0, Claude 3.7, Whisper, & more

Explore the latest AI model benchmarks on Labelbox leaderboards, featuring Imagen 3, Kokoro, AWS Nova Pro, and more across image, speech, and video.

Updated Labelbox Leaderboards: Kokoro, Imagen 3, AWS Nova Pro on the rise

leaderboards

Labelbox integrates a full VS Code for the Web IDE into its platform, delivering a desktop-class coding experience to create fast, quality data.

Labelbox add integrated VS Code environment for coding 

Labelbox unveils integrated VS Code IDE: Generate sophisticated training code quickly

In this blog, learn about two AI lab customers who utilized Labelbox's top-tier AI trainers to drive innovation in their audio and multimodal STEM models. 

Customers transform audio and multimodal AI with Labelbox Services

Build AI

The power of human expertise: Transforming audio and multimodal STEM models with Labelbox Services

This guide teaches you how to generate domain-specific data with the Labelbox data factory to train your AI models on industry-specific knowledge.

Ultimate guide to generating industry-specific AI training data

How to generate industry-specific data for AI training with Labelbox

Labelbox’s latest innovations: New features for coding, audio, talent acquisition, and more

The importance of humans: AI expectations for 2025 and key NeurIPS learnings

Dmytro Apollonin

Code Runner: Secure, scalable code execution for model evaluation

Stanislav Issayenko

Engineering

Bringing AI to the browser: SAM2 for interactive image segmentation

Advance LLM reasoning with advanced fact-checking and prompt rating tools

Generative AI

Leaderboards: Multimodal reasoning now available & updated evaluations for image, speech and video

Michał Jóźwiak

Inside the matrix: A look into the math behind AI

IMPORTANT: Don't change the name of this Tag, it is referenced inside the theme's code in a way that filters out all of the posts that contain it.
If you want to change this name, you should also change it in the loop_index.hbs file in the theme's repository.

Release Notes

Multi-step reasoning: Teach LLMs to think critically

Enhanced video editor adds  deeply nested classifications to improve visibility and efficiency

Announcing Labelbox Alignerr Connect: Discover and hire proven AI experts

Michael Haag

Product spotlight: Delivering  high-quality data at scale for GenAI models

New AI-powered code and grammar critic improve quality and efficiency

Val Brodsky

Faster, cleaner code: Labelbox SDK migrates to Pydantic v2

Efficient LLM fine-tuning with PEFT

Bring your own models to Labelbox with new custom model integration

Our latest feature lets you integrate custom models and candidates into Labelbox with just a few clicks. Use them for model-assisted labeling or model evaluations. 

Integrate custom models into Labelbox 

Tim Kerr

Inversion of control and observability for React codebases

The new Labelbox Monitor dashboard boosts data quality

New Labelbox Monitor dashboard boosts data quality and control

Monitor and optimize: Boosting data quality with new Labelbox workspace Monitor

New Labelbox Monitor dashboard boosts data quality

Labelbox leaderboards: Redefining AI evaluation with private, transparent, and human-centric assessments

Labelbox labeling services at your fingertips

Labeling services at your fingertips: Discover the new Labelbox workflow

New multimodal chat evaluation experience in Labelbox for GenAI

Improve multimodal evaluations with enhanced arena style experience

Redesigned multimodal GenAI chat in Labelbox

Industry: Any

Inside the data factory: How Labelbox produces the highest quality data at scale

Partnerships and technical integrations between 3rd party solutions and Labelbox

Partners

Faster image segmentation with Meta’s Segment Anything 2.0

Labelbox Q2 product spotlight: The data factory for GenAI

Product spotlight: Building the data factory for GenAI and frontier models

Enhanced synthetic data generation with human-in-the-loop supervision using Llama 3.1 and Labelbox

Streamlining large-scale data exports: The evolution of  Labelbox’s Export System to a streamable architecture with fairness in place

Mikiko Bazeley

What the focus on multimodal, generative, and 3D AI means for the future at CVPR 2024

Powerful multimodal support delivers high-quality data for GenAI models

Product spotlight: Native LLM & multimodal support for hybrid evaluation

Seamless LLM human evaluation with LangSmith and Labelbox

Google Cloud partners with Labelbox to offer LLM human evaluation services

Labelbox introduces no-code data pipeline integrations

The dev corner: streamlining deployments at Labelbox with ArgoCD and GitOps

Building computer vision

Labelbox video search powered by Google Gemini Pro Vision

Labelbox Labs - A preview of multimodal data labeling

A pragmatic introduction to model distillation for AI developers

Learn how you can easily evaluate Gemini models, compare them to other powerful foundation models like Open AI’s GPT-4, and choose the best model for your use case with model foundry in Labelbox.

How to leverage Google’s Gemini models in Labelbox foundry for building AI

Use AI

How to leverage Google’s Gemini models in Labelbox Foundry for building AI

Learn how the Texas Rangers Baseball Club used text data to inform player drafting and trading decisions and build a team that secured their first World Series win in November 2023.

How the World Series Champions Texas Rangers built a winning team with AI and analytics

Unleash the full potential of your data in Labelbox with 10x faster performance on large-scale data ingestion uploads. 

10X faster uploads: Labelbox’s data ingestion upgrades and how to use them

Learn how these two methods differ, the pros and cons of each, and how to choose the right method for your fine-tuning project.

RLHF vs RLAIF: Choosing the right approach for fine-tuning your LLM

New in Labelbox: Fine-tune LLMs with our LLM data generation editor, and curate and share data more easily.

Unlock the power of AI: Labelbox’s newest features to use and build better AI

Discover tips and best practices for accelerating your time-to-value when building AI solutions for your retail and e-commerce business. 

How to accelerate time-to-value when building AI for retail and e-commerce

Industry: Retail & e-commerce

Large language models (LLMs) have revolutionized the field of AI and natural language processing (NLP). Dive into six popular LLMs and explore their intended use cases, limitations, and possible real-world use cases.

6 key LLMs to power your text-based AI applications

Building LLMs

Explore six of the most powerful foundation models available to AI builders, the use cases and applications they are best suited for, and how you can explore, test, and leverage them quickly and easily for pre-labeling data and AI development.

6 cutting-edge foundation models for computer vision and how to use them

Explore how labeling quality can be measured and improved with the right observability practices for labeling operations.

How to set up your labeling ops for success with observability

MLOps

With foundation models, such as GPT-4, teams now have a way to automate and accelerate data labeling rather than starting from scratch. Leverage Model Foundry's prompt-based interface to create effective prompts for pre-labeling data with GPT-4. 

How to unlock the full power of GPT-4 for Enterprise AI

How to unlock the full power of GPT-4 for enterprise AI

In this post, we’ll explore how you can easily leverage a large language model (LLM) to identify and extract PII from your datasets with much higher accuracy, efficiency, and speed.

How to use LLMs to detect and extract personal data from AI datasets

Celebrating Labelbox’s latest security advancement: ISO 27001:2022 Certification

Learn to identify and remove personal identifiable information from data before it puts your AI and organization at risk.

How to detect and extract personal information from datasets for AI

Seamlessly integrate Databricks data pipelines with Labelbox

Explore some of the overarching themes from the sessions, demos, and papers of CVPR 2023.

Key trends and innovations in AI development from CVPR 2023

Recapping CVPR 2023: Key takeaways from Labelbox

Learn how to always choose the best and most efficient model for your use case. With Model Foundry, you can test, compare, and evaluate models to confidently select the best performing model. 

How to confidently compare, test, and evaluate models for machine learning

Jay Patel

Learn how to accelerate image segmentation with Segment Anything, Auto Segment 2.0 and Model Foundry 

Accelerate image segmentation with a new AI-powered solution

Learn how to accelerate image segmentation with Model Foundry and Labelbox's latest upgrades to the image editor. 

Manu Sharma

Learn how to use LLMs to instantly generate predictions on product categorization and summarization. 

How to automate labeling for product categorization with LLMs

With the introduction of Model Foundry, we will bring the power of foundation models in Labelbox to perform complex tasks such as data labeling and data enrichment. 

Model Foundry: Automating data tasks with foundation models

Labelbox Model Foundry: Automating data tasks with foundation models

Explore best practices for crafting successful prompts specifically for natural language searches.

5 tips to write more effective prompts for natural language search

Explore & manage data

How to write more effective prompts for natural language search

Learn how reinforcement learning from human feedback can be used to train models on tasks that are harder to specify.

Using reinforcement learning from human feedback to fine-tune large language models

Learn more about how you can organize and automate your data labeling with bulk classifications. 

Pre-label and enrich your data with bulk classifications | Labelbeox

Pre-label and enrich your data with bulk classifications

Pre-label and enrich data with bulk classifications

We put GPT-4 and PaLM - two of the largest and most powerful LLMs developed to date - to the test. Read more to learn how GPT-4 and PaLM perform on zero-shot predictive accuracy and generative ability. 

GPT-4 vs PaLM: Assessing the predictive and generative -performance of LLM models

GPT-4 vs PaLM: Assessing the predictive and generative performance of LLM models

Labelbox's upcoming Auto-Segment 2.0 tool will apply the power of Meta AI’s recently released Segment Anything Foundation (SAM) Model.

Auto-segment 2.0 Coming Soon | Labelbox

The upcoming Auto-Segment 2.0 tool will apply the power of Meta AI’s recently released Segment Anything Foundation (SAM) Model in a workflow that helps users rapidly generate high performance training data for their most complex computer vision use cases.

Coming soon: Auto-Segment 2.0 powered by Meta's Segment Anything Model

Label data for AI

Auto-Segment 2.0 powered by Meta's Segment Anything Model

Get a high-level introduction of how vector similarity search works and how it’s helping teams get access to information faster.

How Vector Similarity Search Works | Labelbox

How vector similarity search works

Learn how to utilize the latest advances in zero-shot and few-shot image classifications using OpenAI CLIP embeddings to better curate your data

Utilizing Few-shot and Zero-shot learning with OpenAI embeddings | Labelbox

Train & fine-tune AI

Utilizing Few-shot and Zero-shot learning with OpenAI embeddings

Kahveh Saramout

To embrace the fifth industrial revolution, manufacturing organizations will need to break down data silos and consolidate their unstructured data into centralized data lakes.

How to prepare your manufacturing organization for the fifth industrial revolution: AI

Industry: Manufacturing

New in Labelbox: Upload custom embeddings, label video data with step interpolation, send model predictions to your labeling project, and more. 

Upload custom embeddings to find similar data, a new way to label video data, and more.

Think about all the situations in your factory where you need humans to watch a process, identify an event, and then make a decision based on SOPs or other protocols. The bulk of this work can be offloaded to large language models (LLMs).

How to leverage foundation models in manufacturing

Make your videos queryable using foundation models

Labelbox expands partnership with Google Cloud to help teams build AI applications faster

Matt Chambers was recognized for his achievements in render farm management system design with an Academy Award during the Scientific and Technical Awards presentation on February 24th.

Blog