Higher-quality NLP signal that cut Dialpad's cost per datapoint
Problem
Dialpad's previous annotation provider couldn't meet the data quality its AI projects required. The team tried spending more time reviewing and redesigning labeling projects with the provider, but the lack of quality slowed progress on its AI work.
Solution
Dialpad turned to Labelbox and Boost for higher-quality training signal and less demand on its own data scientists for project design and review. Labelbox let the team collaborate smoothly with contributors and produce higher-quality signal for its AI use cases.
Result
After a year on Labelbox, Dialpad saw a 20% improvement in signal quality and a 41% reduction in labeling costs. Its AI team gained productivity and motivation, with data scientists proactively requesting the signal they need without tying up their own time and resources.
Dialpad's NLP and LLM products do transcription, summarization, and sentiment analysis. Labelbox's platform produced higher-quality training signal that improved accuracy over 20% while cutting cost per data point.
The challenge
Dialpad provides AI-driven customer engagement, communications, and intelligence. Its NLP- and LLM-powered products handle transcription, summarization, sentiment analysis, and more. Building and maintaining them takes large amounts of training signal for custom models, plus subject matter expertise and fine-tuning for LLMs. Dialpad had relied on legacy crowdsourced labeling that produced data fast but not at the quality its models needed. As the team scaled — fine-tuning production models and building specialized ones — labels were often inaccurate or missing. Extra time, special teams, and redesigned projects didn't fix it. Data scientists grew reluctant to even request data.
Our data scientists were less proactive about requesting labeled data because they knew how much effort it was going to be and how much pain it was going to cause,” said Anne Paling, Manager of Data Annotation and Testing at Dialpad.
The approach
Dialpad switched to Labelbox with two requirements: higher-quality signal, and less of its own team's time. Labelbox's software-first approach produced datasets labeled quickly, accurately, and comprehensively, with a transparent process. Labelbox Boost matched the right contributors to each use case, reducing the supervision Dialpad's data scientists had to provide, and kept data private and secure versus crowdsourced labeling.
The outcome
A year in, Dialpad saw major gains in quality, time saved, and team productivity and motivation. Data scientists now proactively request the signal they need and scale AI development faster. Switching also lowered costs, even though Dialpad had been willing to pay more for quality.
Before, we were averaging roughly 29 cents per data point. This year with Labelbox, it’s down 48% to only 15 cents per data point. We essentially labeled more data for less money with an average increase in accuracy of over 20%,” said Paling.
Where this goes
Signal quality is the constraint on every NLP and LLM product. When producing it stops being painful, teams ask for more of it — and the models keep improving.