Generative AI startup aims to revolutionize the legal industry with their AI agent and advanced training
Problem
A generative AI startup seeks to transform the way plaintiff firms operate and perform case evaluations using highly trained AI agents. The company's key challenge was sourcing accurate, cost-effective, and scalable legal training datasets.They needed help sourcing legal experts who could precisely guide their model in completing industry-specific tasks.
Solution
The startup partnered with Labelbox Labeling Services to quickly assemble a skilled team of professionals with expertise in legal terminology, insurance cases, and medical documents. The task focused on document text extraction, aiming to teach the AI legal agent how to interpret, extract, and process relevant information like a human legal expert.
Result
The company was thrilled with the data quality, quick turnaround, and the ease of collaboration. Behind the scenes, Labelbox seamlessly managed and customized the project. In the end, the startup gained access to critical legal industry expertise and established a process to continue expanding their AI agents’ capabilities.
Introduction
As a leading generative AI legal startup, the company’s mission was to create an AI agent that automates labor intensive tasks done by legal experts such as contract review, risk assessment, compliance checks, and document analysis. Their goal was to help legal professionals efficiently manage legal documents and streamline time-consuming tasks.
In order to make this possible, their AI model required training on industry-specific, complex reasoning that was not readily available in any foundational model. They needed to build out new, unique datasets tailored to their tasks and then complete powerful fine-tuning and post-training methods to teach the model how to be trained on legal industry nuances.
Partnering with Labelbox Labeling Services
Generating and labeling the data internally was challenging, as the startup had a small team and the data volume was growing rapidly. To improve their domain-specific AI agent, this company was looking for a data vendor to provide them with human experts with legal expertise to onboard and label for their tasks.
After evaluating multiple vendors and running a few proof of concepts with them, the company chose Labelbox Labeling Services, powered by Alignerr, to partner with on this project. The company was impressed with the quality of data demonstrated, the turnaround time, and the overall cost, which as an emerging AI company was important.
Creating unique, legal-specific training data
To support the project, we quickly recruited and onboarded a team of labelers with legal backgrounds to tackle the task. In parallel, Labelbox worked with the company to define the detailed instructions for the project and to create a custom ontology to ensure all the necessary training data was captured for each data row.
The company provided a series of prompts that were each linked to numerous multi-page legal documents. The experts were responsible for identifying key information, crafting well-reasoned responses supported by evidence from the documents, and evaluating the model's output for accuracy, safety, and reasoning.
Additionally, a second project was added where the team focused on understanding insurance and medical billing details, ensuring the correct information was extracted.
Expanding opportunities for legal experts
Labelbox found that legal experts, with their background in handling contracts and reviewing lengthy legal documents, were twice as fast and more accurate compared to generalists with no prior experience.
“Training AI models through Alignerr isn't just about data labeling - it's about imparting years of professional judgment and expertise. My background in compliance allows me to help models understand the complex decision-making processes that many legal professionals use every day." - Andrew K, Compliance Analyst
Accelerating the use of AI for legal outcomes
After evaluating two other data service providers, the company chose Labelbox for our ability to deliver high-quality, differentiated human data quickly as well as our advanced data platform that enabled efficient management and creation of custom project ontologies.
Backed by Labelbox’s data factory, the legal AI startup has been able to enhance their model's accuracy and accelerate development, keeping them competitive in the fast-moving market.