Partner Integrations
Google Vertex AI human evaluation
Integrate with Labelbox to add human evaluations to your Vertex AI projects to assess the performance of your models and perform side-by-side comparisons
Human evaluation by Labelbox is now integrated into Vertex AI
Easily request human evaluations directly from Vertex AI. Evaluate a single model, or request side-by-side comparison of two models based on custom evaluation criteria.
Contact us if you are interested in joining our tech preview program.
High-quality human evaluations
Labelbox’s expert labeling services conduct human evaluations of your data within the platform, returning evaluation results directly to your Google Cloud Storage. We currently support summarization and question-answering tasks in English, both for single model and comparative assessments. Contact us if you have additional or more complex needs.
Flexible evaluation criteria
Labelbox offers a range of evaluation criteria, and the rating of your prompts/responses are returned as a JSON file. Default criteria includes groundedness, instruction following, comprehensiveness, and overall quality, rated on a 1 to 5 Likert Scale. You can optionally include a custom criteria to tailor your evaluations.
Robust security & privacy
Benefit from a deep integration that prioritizes your data's security and privacy. Labelbox operates under stringent data protection measures, and your data is transferred securely between Vertex AI and Labelbox, ensuring compliance and peace of mind. Learn more about Labelbox privacy and security.
How Labelbox and Vertex AI integrations works
Transparent pricing & billing
Usage and billing are managed through the Google Cloud Marketplace. Anyone with an active Marketplace agreement can request a human evaluation from the Labelbox platform. Labelbox automatically performs the services, and they are billed at a fixed price per task in Labelbox Units (LBUs). You only pay for completed tasks, even if you cancel a job early.
Fixed pricing per task for quick, self-serve projects
Tasks supported: summarization, question & answer, custom task
Volume discounts automatically. Contact us for large or custom projects
Why thought leaders choose Labelbox
“Labelbox has dramatically improved our model performance for our most critical AI initiatives by tapping into their network of expert labelers and platform for human evaluation. In the past two months, our document intelligence teams are seeing a 2X increase in data quality compared to previous vendors.”
Human data lead
"We switched over from a BPO and have been absolutely blown away by the quality and speed of Labelbox services. Not only that, we thought we would be paying more for the quality but in actuality, we now spend 2-3x less due to the major reduction in wasted spend, annotation duplication, and need for fewer pilot/test runs."
Manager, Annotating & TestingLabelbox runs on Google Cloud and integrates with a number of Google solutions
Vertex AI
Launch human evaluations, seamlessly integrate model training and inferencing on dedicated platforms, and leverage Vertex AI Model Garden within Labelbox Foundry for AI-assisted labeling.
BigQuery and Cloud SQL
Accelerate data pipeline setup, allowing seamless synchronization of data from various sources like BigQuery, Sheets, and SQL databases into Labelbox for efficient labeling and model development.
Google Cloud Storage
Import and synchronize data securely and efficiently from your Cloud Storage buckets into Labelbox via IAM Delegated Access for streamlined AI model development.
Resources
Google Cloud Marketplace
Vertex AI partnership announcement
