Product spotlight: Delivering high-quality data at scale for GenAI models

What we’ve been up to

Over the past quarter, we’ve doubled-down on helping AI labs, AI disruptors, and high-tech enterprises create innovative AI models and applications. We’ve introduced a number of new features and services to our AI data factory, all designed to help you create high-quality data at scale. These include a new real-time dashboard, a revamped UI for arena-style model evaluations, and a built-in AI critic for grammar and code. 

Along with new features and services, one of the highlights of this quarter was the introduction of Labelbox leaderboards, an innovative and scientific process for ranking multimodal AI models that goes beyond conventional benchmarks. 

Read on for a recap of all that’s new from Labelbox in the past quarter. We’ll break down three major areas of focus for our product teams:

• Improve GenAI model evaluation: Use the new Labelbox leaderboards to avoid the limitations of traditional benchmarks and use a scientific, human-based approach to evaluating the latest multimodal models. 
• Deliver high-quality data at scale: Explore the new capabilities of the Labelbox data factory and discover why the right combination of software and services is critical to creating high-quality data.. 
• Differentiate your models with human evaluations: Unlock the power of human evaluation with our growing network of Alignerrs. Discover how Labelbox makes it easier than ever to leverage expert feedback and integrate your custom AI models or candidates for optimal performance.  

Meet the new Labelbox leaderboards

In the rapidly evolving landscape of artificial intelligence, traditional benchmarks are no longer sufficient to capture the true capabilities of AI models. At Labelbox, we're excited to introduce our groundbreaking Labelbox leaderboards — an innovative, scientific process to rank multimodal AI models that goes beyond conventional benchmarks.

The Labelbox leaderboards represent a significant advance in AI evaluation, pushing past traditional leaderboards by incorporating expert human evaluations for subjective generative AI models using comprehensive metrics. We are uniquely able to achieve this thanks to our modern AI data factory that combines human experts and our scalable platform with years of operational excellence evaluating AI models.

Leaderboards are now available for Image Generation, Speech Generation, and Video Generation models.

We invite you to:

• Check out the Labelbox leaderboards to explore our latest evaluations across various AI modalities and niche applications.
• Let us know if you have suggestions or want a specific model included in future assessments.

Deliver high-quality data at scale 

At Labelbox, we're committed to providing the tools and methodologies you need to not only generate large volumes of data but to ensure its accuracy, consistency, and overall quality.

Recently, we pulled back the curtain on our own internal data factory, revealing how we leverage a scientific approach to measure and define data quality, even within the subjective realm of Generative AI. This rigorous methodology, combined with our powerful platform features, enables you to confidently scale your data operations without sacrificing quality.

To further empower your pursuit of data excellence, we've introduced a range of new features designed to streamline quality control and provide deeper insights into your data:

• Revamped model comparison UI: Our redesigned model comparison interface offers an intuitive, arena-style experience. Easily compare and contrast different model versions, facilitating data-driven decisions and accelerating model optimization. Learn more.

• Built-in AI critics: Our new AI critic provides automated feedback on text and code, helping to identify errors, inconsistencies, and areas for improvement in new or rewritten responses. This intelligent assistance ensures your data meets the highest standards of quality. See demo. 
• Labelbox Monitor: Gain real-time visibility into your labeling operations across all projects and labelers within a workspace with our Monitor dashboard. Track key metrics, visualize outliers, and proactively address potential issues to maintain data integrity. Learn more. 

With Labelbox, you can confidently scale your data operations while maintaining a steadfast commitment to quality. Our platform provides the tools, insights, and methodologies needed to fuel your AI ambitions with data you can trust.

Differentiate your models with human evaluations

In the competitive landscape of AI, delivering truly exceptional models requires more than just technical prowess. It demands a deep understanding of human needs and expectations. This is where human evaluation becomes essential. By incorporating human feedback into your development workflow, you can ensure your models are not only accurate but also aligned with real-world use cases and user preferences.

Over the past several months, we’ve introduced new capabilities and expanded our network of subject matter experts to help you perform fast and accurate human evaluations. Key updates include: 

• Integrated on-demand services: Quickly and easily request human evaluations directly within the Labelbox platform. Our streamlined workflow allows you to specify your evaluation criteria, select your desired evaluator pool, and receive timely, actionable feedback. Learn more. 
• Expanded Alignerr network: Leverage the expertise of our growing network of Alignerrs, skilled professionals trained to provide high-quality human evaluations. Our diverse Alignerr pool ensures you can find raters with the specific domain knowledge and language proficiency needed for your project. Learn more. 
• Custom model integration: Seamlessly integrate your own custom models or builds into Labelbox for evaluation. This flexibility allows you to leverage our platform's powerful evaluation tools and Alignerr network regardless of your specific model architecture or development environment. Learn more. 

By harnessing the power of human evaluation with Labelbox, you can gain a critical edge in the AI landscape. Refine your models for optimal performance, mitigate biases, and ensure your AI solutions truly meet the needs of your users.

Build with us: Training and guides

To further support our customers in their AI development journey, we've created a wealth of resources available online and on-demand:

Technical guides 

We maintain a series of in-depth guides to help users navigate various aspects of AI development. This quarter we expanded the library of guides with several key additions: 

• Programmatically launch human data jobs for RLHF and evaluation
• Evaluating leading text-to-speech models
• Metrics-based RAG Development with Labelbox
• Unlocking precision: The "Needle-in-a-Haystack" test for LLM evaluation
• A comprehensive approach to evaluating text-to-video models
• A comprehensive approach to evaluating text-to-image models

Engineering insights

Labelbox engineers are unlocking knowledge and sharing their insights for others on our Labelbox blog. These posts reveal techniques on how our team optimized for code efficiency and streamlined performance.

• Faster, cleaner code: Labelbox SDK migrates to Pydantic v2
• Efficient LLM fine-tuning with PEFT
• Inversion of control and observability for React codebases

New interactive product tours 

For anyone who wants the simplest way to experience the Labelbox platform and our new capabilities, our interactive product tours are a perfect fit. 

These interactive tours take only a minute or two to complete. This quarter, we’ve added new interactive demos to showcase our latest innovations in the platform:

• AI-powered code and grammar critic
• Labelbox Monitor dashboard
• New multimodal chat editor UI
• Integrating a custom model
• Boost labeling services

Check them out on our product tours page!

Connect with us: webinars, events & community 

We offer various opportunities for users to connect, learn, and engage with the AI community through informative webinars, industry events, and a thriving user community.

Webinars & events

• Inside the AI data factory: How to produce high-quality data at scale: Watch this on-demand webinar to hear Labelbox engineers share their approach to improving data quality and efficiency in AI development, focusing on precision, accuracy, and workflow optimization.  
• Neural Information Processing Systems Foundation (NeurIPS): Join us in December in Vancouver at NeurIPS. RSVP to the Labelbox-hosted happy hour for a chance to network with top engineers, researchers, and leaders from the AI community. 

Community

And finally, did you know we have a vibrant Community of 500+ users sharing invaluable tips, technical assistance, and insightful How-To guides on building cool projects with custom data? For example, discover how to convert Labelbox annotations to COCO format. Sign up for a free account and join the conversation here!

Conclusion

As we wrap up Q3, we're proud to highlight the significant advancements we've made in our AI data factory to help you consistently create high-quality data at scale.

We've introduced innovative features and services to continue to empower AI labs, AI disruptors, and high-tech enterprises to build cutting-edge AI applications. These features include a real-time performance analytics dashboard, an enhanced UI for arena-style model evaluations, and a built-in AI critic for grammar and code. Additionally, by leveraging our growing network of expert human Alignerrs and innovative Labelbox leaderboards, you can optimize performance with human-centered AI model evaluation.

The future of AI is bright, and Labelbox is leading the charge. As we continue to explore the frontiers of AI development, organizations can anticipate even more innovative features and enhancements from Labelbox’s next-gen data factory.