The power of human expertise: Transforming audio and multimodal STEM models with Labelbox Services

Leading frontier AI builders leverage domain-and language-specific expertise to differentiate their models across data modalities—including audio, multimodal, image, text, and video—and to train them for more complex tasks. As the capabilities of AI expands, the need for post-training processes like SFT, RLHF, and human evaluation remain strong. These tasks depend on expert human knowledge to guide models toward higher performance.This is where we come in. Labelbox is the AI data factory that delivers high-quality data across all modalities, including specialized domains like STEM, finance, coding, and law—across a wide range of languages for each. 

With Labelbox Labeling Services, we harness our skilled talent network, Alignerr, to source, vet, and onboard custom teams of human experts who can align models and generate new training data with domain-specific knowledge. Labelbox can operate and fully-manage a project with Alignerrs and the Labelbox Platform that generates new training data for our customers, or through Alignerr Connect, customers can browse and select top-tier experts to staff their existing projects and utilize their in-house tools and processes. In this blog, we highlight two recent customers who utilized our top-tier human experts to drive innovation in their cutting-edge audio and multimodal STEM models. 

Driving breakthroughs in the AI audio landscape 

A growing AI audio startup aimed to enhance its voice, speech, and sound models by training with expert-labeled data. They faced challenges due to the subjectivity of labeling large volumes of complex audio. Labelbox addressed this through our platform’s custom audio editor and building a team of trainers with expertise in voice acting and speech. Their background enabled them to label nuanced audio segments with greater accuracy than generalists.

One of the Alignerrs on the project, Jeff K., has a PhD in Theater and Performance Studies. He shared this about his experience on the project:

“Through years of performing and teaching the arts, I've developed a deep understanding of voice dynamics. I have mental checklists for how and where voices change, which makes it natural for me to identify the various emotions in speech and understand their impact on the listener."

This specialized level of human expertise was critical in enabling the startup to create high-quality audio datasets and enhance their AI models, driving the adoption of their advanced audio technology.

Want to explore the full story behind this success? Read more here. 

Improving multimodal reasoning capabilities with STEM experts

A leading AI lab aimed to enhance its large language model (LLM) by identifying weaknesses in K-12 STEM education responses, but they needed a diverse team of STEM experts to create new training data. 

Labelbox’s Labeling Services brought together a team of highly skilled experts with PhDs and Masters in STEM, who were tasked with reviewing multimodal prompts and responses spanning various domains, including natural science, physics, earth science, and language comprehension. Each task included reviewing the model’s response to a question that contained metadata such as question format, subject category, and an associated image URL. 

The AI trainers then had to rate the answers across a number of different areas, including examining the model’s ability to read text, analyze image, and respond correctly. This collaboration helped the lab pinpoint areas for improvement and boost performance with high-quality, domain-specific STEM data.

If you are interested in learning more about this work, read more here. 

Looking to leverage Labelbox’s expert AI trainers?

These two customer stories highlight a few examples of the groundbreaking work Labelbox is performing with frontier model builders. 

If you’d like to learn more about the expert teams we offer and are ready to discuss your data needs, contact our team anytime on how we can help. You can also directly explore profiles of some of our AI trainers here.