A generative audio startup needed to train models on emotion, tone, and speech style. Labelbox's platform produced the expert-graded temporal signal — sourced through its Alignerr network — that made that subjective data trainable.

The challenge

Audio is becoming central to AI: voice interfaces, text-to-speech, speaker intent, audio translation. Training models to understand and generate nuanced audio takes high-quality signal. A fast-growing generative AI startup wanted to improve its voice, text-to-speech, and sound models so they could recognize sentiment and emotion in human speech — and the data had to be framed as commands the model could learn from. Labeling subjective, temporal audio is hard: emotional cues are interpreted differently, situational context shifts meaning, mixed emotions are ambiguous, and human bias creeps in. Add high-precision segmentation, and the startup lacked both the expertise at scale and the tooling to produce this signal.

The approach

Labelbox produced the signal. Through its Alignerr network of expert contributors, the platform captured judgment from people with backgrounds in theater, performing arts, and voice acting — domain expertise particularly suited to identifying shifts in emotion and describing how a voice conveys them. The platform's audio editor — waveforms, custom ontologies with temporal classifications, and millisecond-level timestamps — let contributors segment and label with the precision the task required, with auto-transcription powered by the Whisper model. Contributors identified "interesting" segments defined by high emotion (anger, happiness, disgust) or special speech styles (sarcasm, slurring, whining), then described how each was spoken — writing the descriptions as commands to guide replication of speech pattern, tone, emotion, and style.

Through years of performing and teaching the arts, I've developed a deep understanding of voice dynamics. I have mental checklists for how and where voices change, which makes it natural for me to identify the various emotions in speech and understand their impact on the listener.

— Jeff K., PhD in Theatre and Performance Studies, MFA in Dance

The outcome

The startup chose Labelbox over competing data providers for its ability to produce expert-graded audio signal quickly. With an efficient workflow in place, it got high-quality, differentiated datasets describing emotionally charged and uniquely styled audio, and can now train its models to produce realistic audio — driving adoption of its audio technology. These audio leaders now rely on Labelbox to drive innovation in a competitive market.

Where this goes

Voice and audio are a frontier modality. The grounding signal that teaches a model emotion, tone, and intent comes from domain experts — the real-world signal that separates frontier audio models from the rest.