What we've been up to

At Labelbox, we're empowering organizations to build and deploy cutting-edge AI systems by providing a data-centric AI platform that combines AI-powered labeling automation with human evaluation expertise. 

As the demand for Generative AI (Gen-AI) and Large Language Model (LLM) solutions skyrockets, our new features and partnerships directly support this growth by addressing the challenges users face when evaluating and implementing these technologies.

With the increasing complexity and costs of training state-of-the-art AI models, we recognize the need for high-quality, diverse training data and human evaluation. Our platform supports multimodal data labeling, AI-assisted data generation, and human-centric benchmarking, aligning with the industry's shift towards multimodal AI and the importance of human evaluation in assessing model performance.

In this release, we've focused on enhancing the user experience, optimizing performance, and introducing new features, making us the go-to platform for data labeling, model building, and AI leadership.

Making labeling & discoverability easier than ever

We understand the challenges data labelers and DataOps teams face when efficiently labeling data, comparing model responses, and working with various data types. That's why we've addressed these issues with labeling and annotation (as well as data discoverability and curation) through these new features:

• LLM Human Preference Editor: This powerful tool solves the problem of comparing model responses and deciding on ideal model configurations. With support for markdown and raw modes, labelers can easily parse formatted content and code, ensuring high-quality data for training AI models.
• Video Annotation Improvements: The enhanced video player in Catalog addresses the lack of information on model predictions, pre-labels, and ground truth when curating and evaluating videos. Data labelers can now effectively curate, evaluate, and visualize video data, streamlining the annotation process.
• Enhanced Issue Categorization: We've introduced issue categories to reduce friction in using the Issues feature. Users can now select an existing category or create a new one on the fly, streamlining the issue resolution process and increasing adoption.
• Smart Select and Enhanced Search: We've introduced Smart Select to decouple the sampling process from batch submission and introduce embedding-based cluster sampling. Additionally, natural language and similarity search now support video, audio, and conversational data, making it easier to discover and auto-label data quickly.

These enhancements work together to create a more intuitive and efficient labeling workflow, enabling data labelers to quickly discover and prioritize data for labeling, easily compare model responses, efficiently annotate video data, and resolve issues, ultimately improving data quality and labeling efficiency.

Decreasing time from training to deployment

We're dedicated to helping model builders and developers overcome obstacles in iterating and deploying models, leveraging foundation models for multimodal automation, and easier data experimentation for better models. We’re tackling these challenges by: 

Automating pre-labeling

• Production Inferencing: The introduction of the HTTP REST API solves the problem of limited inference options and the inability to view predictions outside Labelbox. Customers can now invoke inferencing endpoints for Foundry models, enabling powerful and flexible automated inferencing capabilities.
• Model Fine-Tuning: We now offer the ability to fine-tune YoloV8 object detection models with custom features, addressing the limitation of using only foundation models for inference and auto-labeling. This feature will soon expand to support classification and segmentation tasks, as well as fine-tuning on previously fine-tuned models.
• Video Support in Foundry: Foundry, our no-code AI platform, now supports video data, solving the issue of Foundry being limited to only images and text. Users can preview Foundry predictions per-frame, and various models and annotation types are supported.

Enabling model experimentation for data

• Frictionless Model Experiments: Setting up model experiments is now easier than ever, with multiple new workflows introduced on the Labelbox UI. This addresses the challenge of requiring SDK usage to set up experiments with data rows, ground truths, and predictions.
• Custom Metrics at the Feature Level in Model: We now support importing custom metrics per prediction annotation, as well as filtering and sorting by prediction-level custom metrics. This addresses the limitation of only having aggregated custom metrics at the data row level, providing more granularity for effective model error analysis.
• Feature-level Metrics Filtering and Sorting in Model: In addition to custom metrics, we support feature-level auto metrics and custom metrics at the prediction level. This enhancement is crucial for customers to identify problematic predictions and run active learning.

These features work together to create a powerful and streamlined model building workflow within Labelbox, empowering model builders to easily set up experiments, fine-tune models with custom features, deploy them in production, and leverage custom and feature-level metrics for more effective model error analysis and active learning.

Expanding platform for production-grade data & workflow requirements

AI leaders often face challenges in delivering specific AI capabilities or intelligent applications, scaling up AI throughput, and reducing the cost of AI/ML development. All while remaining agile and pivoting with the winds of change. 

We’ve decided to address these pain points through investments in the platform, enabling organizations to overcome limitations in their existing AI/ML operations, better understand large amounts of unstructured data, and efficiently manage data labeling teams and costs.

Supercharging data I/O & the SDK

• No-Code Warehouse Integrations: The introduction of Census integration solves the problem of manually keeping datasets in Labelbox in-sync with tables in data warehouses. Supporting over 25 different data sources, this feature eliminates the need for manual data synchronization, unlocking valuable first-party data for labeling and model training.
• Cloud Buckets Integration: Labelbox now provides a simple way for customers to bulk create assets from their Cloud Buckets (AWS, GCP, Azure). This feature addresses the friction that prevents customers from uploading their data to Labelbox from Cloud Buckets, making it easier to import and work with data stored in these platforms.

• SDK Upsert and Embeddings Support: Labelbox has updated its SDK to include "upsert" functionality, addressing the issue of requiring multiple steps to create and update data rows and associated data. The SDK and web interface now support the manipulation of custom embeddings, and embeddings can be exported for further analysis, enhancing the quality and performance of generative AI models through an iterative process of auto-eval, human review, and feedback incorporation.
• Improved SDK Interface: Labelbox has enhanced its SDK functions to support global keys and data row IDs as inputs, addressing the inconsistency in function signatures and the lack of global key support. This improvement streamlines the SDK user experience, making it easier for customers to check the existence of data rows and create batches based on global keys.

Improved data management

• Data Search Filters: Finding the right data in Catalog is now more intuitive than ever, solving the problem of strict and cumbersome search functionality. Users can quickly search their data by typing anything in the data search filter bar, reducing the steps required to find data quickly and accurately.
• Optimized Performance: Labelbox has made significant improvements to its platform's performance, addressing issues such as limited data row handling in Catalog Embeddings Search, slow embeddings searches, and unreliable routing of data rows in task queues.
• Exports V2 Streaming: Labelbox has removed the technical limitation that prevented customers from exporting over 700k DataRows at any given time. This update streamlines the data export process for larger enterprise customers, regardless of the interface used.

Optimizing workflows

• Async Task Queues: Labelbox has improved the reliability of routing data rows in task queues by making the process asynchronous using Pub Sub. This enhancement is essential for large customers who rely on Task Queues for reviews at scale. The update also ensures a complete workflow history, reducing confusion and escalations.
• Boost Workforce Services: Labelbox's Boost product offers a human labeling and evaluation workforce of subject matter experts, allowing AI leaders to outsource, expand, and scale human evaluation. This service is crucial for enhancing the quality and performance of generative AI models through an iterative process of auto-eval, human review, and feedback incorporation.

These advancements create a more efficient and scalable AI development workflow for AI leaders. Easy data integration, a low-code environment to run foundation models, and automation combined with human expertise streamline the process. When utilized with Boost AI and Workforce services, organizations can deliver intelligent applications faster and scale their AI throughput.

AI leaders will be able to drive innovation across their organizations by reducing the costs associated with getting started with large volumes of unlabeled data through optimized labeling efficiency, increased data labeling capacity, and accelerated AI project deployment. 

Build with us: partnerships, training, and guides

To further support our customers in their AI development journey, we've created a wealth of resources, formed strategic partnerships, and participated in industry events:

Partnerships

We've formed strategic partnerships with leading companies in the AI industry to provide our customers with a more comprehensive and integrated solution.

• Google Cloud: We've partnered with Google Cloud to provide Vertex AI platform customers with an integrated solution for LLM evaluation as a fully managed service. This partnership allows Vertex AI customers to easily launch LLM evaluation jobs, set desired evaluation types and criteria, and receive quality-reviewed results from skilled evaluation professionals. Additionally, a full suite of Labelbox products is now available on the Google Cloud Marketplace, enabling customers to combine AI-assisted labeling with human evaluation and integrate data pipelines with Labelbox using native no-code integrations with Google Cloud's BigQuery, CloudSQL, and Google Sheets.

• LangSmith: Our collaboration with LangSmith offers enterprise-grade LLM monitoring, human evaluation, labeling, and workforce services. LangSmith is a unified developer platform for building, testing, and monitoring LLM applications, allowing companies to harness the power of LLMs while minimizing risks and inefficiencies. The seamless integration of LangSmith and Labelbox enables customers to develop, debug, manage datasets, evaluate, test, monitor, and experiment with their LLM applications more effectively.

These partnerships demonstrate our commitment to working with the best in the industry to deliver cutting-edge AI development solutions to our customers, enabling them to build safe, responsible, and high-performing LLM applications.

Training 

Due to popular demand we’ve started expanding our training platform, Labelbox Academy.

Our customer academy offers comprehensive training modules designed to help you dive into Labelbox like a pro. With in-depth lessons and best practices covering annotation, modeling, and cataloging, Labelbox Academy provides the resources you need to master the platform. In addition, User Enablement streamlines the onboarding process by providing step-by-step guidance for both administrators and labelers. 

Whether you're onboarding a large team or looking to delve deep into a specific feature, Labelbox Academy has you covered. 

Guides 

We've produced a series of in-depth guides to help users navigate various aspects of AI development, grouped by industry and use case:

• General AI Development: Our guides cover a range of topics to help you streamline your AI development processes, from turning LangSmith logs into conversational data to accelerating and automating data labeling with labeling functions.
• Computer Vision (CV): We offer step-by-step tutorials and best practices for various CV tasks, including efficient video labeling, pre-labeling images, accelerating labeling projects with GPT-4V, and building models for worker safety, visual quality inspection, predictive maintenance, and claims automation. Looking to do model distillation for computer vision? That’s covered too. We also provide an end-to-end tutorial on automating medical imaging with AI to help you build powerful models for improved medical imaging detection.
• Natural Language Processing (NLP): Our guides help you leverage AI for NLP tasks, such as improving website search relevance, optimizing task-specific chatbots, automating invoice and document processing, distilling custom LLMs, enhancing RAG chatbot performance, and generating captions and enriching product listings.

By leveraging these resources, partnerships, and events, Labelbox users can enhance their knowledge, skills, and capabilities in AI development, ultimately enabling them to build more successful and impactful AI applications. The collaborations with Google Cloud and LangChain, in particular, offer customers a powerful suite of tools and services to streamline their LLM development, evaluation, and monitoring processes, ensuring the creation of safe, responsible, and high-performing AI systems.

Connect with us: webinars, events & community 

We offer various opportunities for users to connect, learn, and engage with the AI community through informative webinars, industry events, and a thriving user community.

Webinars 

We've hosted a series of webinars featuring industry experts and thought leaders, covering a range of topics related to AI development, such as:

• Automated labeling with any foundation model: From images to inference in less than 20 minutes: In this webinar, we showcase how to automatically transform and prepare unstructured image data, enabling teams to get started with minimal human involvement and speed up existing labeling operations.
• Automated Labeling With Foundation Models: From Text to Intelligent Applications: We demonstrate how to leverage foundation models to automatically label training data, incorporate human evaluation, and develop end-to-end intelligent applications using Foundry.
• How to choose the best computer vision model for your use case: In this webinar, we shed light on best practices for selecting computer vision models best-suited for unique use cases, both for rapid prototyping and building task-specific models.

These webinars provide our users with valuable insights, real-world case studies, and the opportunity to learn from and engage with leading practitioners in the field.

Events 

Labelbox has been actively participating in industry events to share our expertise and insights on AI development:

• Labelbox CEO Manu Sharma spoke at GCP Next: Manu shared real-world use cases and best practices for building Gemini-powered applications that address compelling business use cases and how enterprises are driving transformation with Gemini on Vertex AI.
• How labelbox helps solve critical healthcare problems with AI | "AI in Medical Imaging" meetup: Our Solutions Engineer Kushal explained how Labelbox enables companies in the medical and healthcare domains to build and deploy intelligent applications using AI and medical imaging, covering key features and benefits of the Labelbox platform.

Community

• And finally, did you know we have a vibrant Community of 500+ users sharing invaluable tips, technical assistance, and insightful How-To guides on building cool projects with custom data? For example, discover how to convert LangSmith results to Labelbox conversation data or how to fine-tune your models. Sign up for a free account and join the conversation here!

Through a combination of educational webinars, active participation in industry events, and a dynamic user community, Labelbox empowers its users to stay at the forefront of AI development. By providing valuable insights, real-world case studies, and a platform for knowledge sharing, Labelbox ensures that its users have the resources and support they need to build and deploy cutting-edge AI applications effectively.

Conclusion

Hopefully this sample of our Q1 2024 product release illustrates our dedication to empowering organizations to build and deploy cutting-edge AI systems. 

By addressing key challenges faced by data labelers, model builders, and AI leaders, we've created a more seamless and efficient user experience across our Annotate, Catalog, Model, and Boost products.

The enhancements in labeling tools, model building capabilities, and AI workflow optimization features work together to accelerate innovation in both traditional and generative AI. Data labelers can now work more efficiently and accurately, model builders can deploy models faster and adapt them to specific use cases, and AI leaders can optimize their operations and bring projects to market more quickly.

As we continue to push the boundaries of what's possible in AI development, organizations can expect even more groundbreaking features and enhancements in the future, cementing Labelbox's position as the leading labeling and annotation platform. 

• If you're interested in giving these features a test drive, be sure to sign-in (or sign-up if you're new to Labelbox) here.
• If you'd like to keep up with all the new features we're releasing, keep an eye on our blog and subscribe to our monthly release notes.