Customer reviews have become a critical tool for businesses looking to improve their products, services, and customer satisfaction. In today’s digital world, review sites like Yelp and social media make it easier than ever for customers to share their experiences with the world. Customer care can range in the services and support that businesses provide to their customers before, during, and after purchase. Great customer care can create positive brand experiences that lead to greater loyalty and customer satisfaction. In the ever-evolving world of retail, it also helps keep your business competitive and at the forefront of your customer’s sentiment and desires.
While companies now have access to a wealth of customer feedback data, sifting through all of these reviews can be incredibly time-consuming and manual. By leveraging AI, teams can analyze customer reviews and feedback at scale, to gain insights into common review topics or customer sentiment. This allows businesses to identify common themes and pinpoint areas of improvement to enhance the customer experience.
However, businesses can face multiple challenges when implementing AI for customer care. This includes:
Labelbox is a data-centric AI platform that empowers businesses to transform their customer care through advanced natural language processing. Instead of relying on time-consuming manual reviews, companies can leverage Labelbox’s assisted data enrichment and flexible training frameworks to quickly build AI systems that uncover actionable insights from customer reviews. Tackle unique customer care challenges with AI-driven insights to create more thoughtful and strategic customer interactions.
In this guide, we’ll walk through how your team can leverage Labelbox’s platform to build an NLP model to improve customer care. Specifically, this guide will walk through how you can explore and better understand review topics and classify review sentiment to make more data-driven business decisions around customer care initiatives.
Part 1: Explore and enhance your data
Part 2: Create a model run and evaluate model performance
You can follow along with both parts of the tutorial below in either:
As customer reviews and feedback across channels proliferate, brands want to learn from customer feedback to foster positive experiences. For this use case, we’ll be working with a dataset of customer hotel reviews – with the goal of analyzing the reviews to demonstrate how a hospitality company could gain insight into how their customers feel about the quality of service they receive.
The first step will be to gather data:
For the purpose of this tutorial, we’ve provided a sample open-source Kaggle dataset that can be downloaded.
Please download the dataset and store it in an appropriate location on your environment. You'll also need to update the read/write file paths throughout the notebook to reflect relevant locations on your environment. You'll also need to update all references to API keys, and Labelbox ontology, project, and model run IDs
Once you’ve uploaded your dataset, you should see your text data rendered in Labelbox Catalog. You can browse through the dataset and visualize your data in a no-code interface to quickly pinpoint and curate data for model training.
Explore topics of interest
With your data in Labelbox, you can begin to leverage Catalog to uncover interesting topics to get a sense of what customers are talking about from hotel reviews.
Create and save data slices
If you have a search query that you’re interested in saving or reusing in the future, you can save it as a slice. You can construct a slice by using one or more filters to curate a collection of data rows. Users often combine filters to surface high-impact data and then save the results as a slice.
In this example, we’ve surfaced reviews on the topic of breakfast that all talk about the value and price of the hotel’s breakfast. We can save this as a slice for future reference (“Breakfast_value”) and as we ingest more data that matches the slice’s criteria, they will automatically get filed into the slice.
After we’ve explored our data, we now have a better understanding of what topics exist in our dataset and can create our ontology. Ontologies can be reused across different projects and they are required for data labeling, model training, and evaluation.
To create a new ontology:
1) Navigate to the ‘Schema’ tab
2) Hit ‘Create new ontology’
3) Select the media type that you wish to work with – for this use case ‘Text’
4) Give your ontology a name
5) Add objects and classifications based on you use case
6) Objects are named entities
8) Save and create your ontology
After creating an ontology, you can begin labeling your data to fine-tune or train a model.
With Labelbox, you can label your data in the following ways:
1) Internal team of labelers: your team can start labeling directly in the Labelbox editor, utilizing automation tools and maintaining quality with custom workflows to maintain human-in-the-loop review.
2) External team of expert labelers with Labelbox Boost: leverage our global network of specialized labelers for a variety of tasks.
Workforce Boost provides a collaborative platform for labeling services in a self-serve manner — this is great for teams that don’t have the technical expertise to build a machine learning system yet are looking for an easy-to-use technology to get a quick turnaround on quality training data. You can learn more about our Boost offerings here.
3) Create pre-labels with foundation models
In addition to creating pre-labels for classification projects, you have the ability to send model predictions as pre-labels to your labeling project. This can be done in one of two ways:
Pre-label data with Model Foundry
Model Foundry acts as the copilot to create your training data – instead of going into unstructured text datasets blindly, you can use pre-existing LLMs to pre-label data or pre-tag parts of it, reducing manual labeling efforts and cost.
1) Select data you wish to label in Catalog
3) Hit "Predict with Model Foundry"
3) Choose a foundation model
4) Configure the model’s settings
5) Generate preview predictions
6) Name and submit the model run
7) View the model run in the Model tab to explore results
Once you’ve evaluated and are satisfied with GPT-4’s predictions, you can send them to a labeling project in Labelbox Annotate.
Add a batch to a labeling project as pre-labels
Before you can send these model predictions to a labeling project as pre-labels, you need to create a labeling project.
Create a new labeling project
1) Navigate to the Annotate tab
2) Create a ‘New project’
3) Select the project type – in this case we want to create a ‘Text’ project
4) Name your project
5) Attach your model’s ontology (created in a previous step)
Once you’ve created your labeling project and configured the ontology, head back to the Model tab to send your batch of data with pre-labels to that labeling project.
1) Highlight all data rows of interest
2) Select ‘Manage selection’ > ‘Add batch to project’
3) Select the appropriate project that you created in the above step
4) You can give the batch a priority (from 1-5)
5) Select the appropriate model run of the predictions you wish to send
6) You can explore and select the various tags that have been applied and uncheck those that aren’t of interest
7) Submit the batch
You can now navigate back to your project in Annotate and hit ‘Start labeling’.
Rather than starting from scratch, your internal or external team of labelers can now see predictions from the Model Foundry run. From here, you can validate or edit predictions as necessary and submit data rows to create ground truth labels.
As you begin to progress through your data rows, you’ll notice data rows that are initially marked up and reviewed by labelers in the ‘Initial review’ task (for your reviewers to verify and approve), with all submitted data rows falling into ‘Done’.
You can create customizable, multi-step review and rework pipelines to drive efficiency and automation for your review tasks. Set a review task based on specific parameters that are unique to your labeling team or desired outcome.
Once all data rows have been reviewed and moved to the ‘Done’ step, you can begin the model training process.
In Part 1 of this tutorial, we have looked at how we can leverage Catalog to understand the topics that exist within your dataset and construct an appropriate ontology. To accelerate our initial labeling job, we leveraged Model Foundry as part of our model-assisted labeling pipeline to use pre-labels from GPT-4 to our labeling workforce for validation. Those initial annotations can be exported via a model run and can be used to train or fine-tune a model outside of Labelbox.
In the previous step, we leveraged Model Foundry to create pre-labels that were passed through Annotate for review with human-in-the-loop validation. Now that we have our appropriate annotation data, we can train a series of initial models on sentiment, topic classification, and named entity recognition.
Model training occurs outside of Labelbox. Labelbox Model works with any model training and inference framework, major cloud providers (AWS, Azure, GCS), and any data lake (Databricks, Snowflake).
Bring the trained model’s predictions back into a model run
Once the model has been trained, you can create an inference pipeline that leverages each model to classify different attributes for review. We can then leverage this for two things:
After following the notebook, you’ll be able to compare ground truth (green) versus the model’s predictions (red) for sentiment and topic.
Diagnose model performance with model metrics
In addition to visualizing the difference between model predictions and ground truth, you can click into the ‘Metrics’ view to get a better sense of how your model is performing.
For example, we can click into false negatives or false positives to narrow down situations where there might be false positives – where ‘negative’ sentiment is predicted whereas ground truth sentiment is ‘positive’.
After running error analysis, you can make more informed decisions on how to iterate and improve your model’s performance with corrective action or targeted data selection.
Curate high-impact data to drastically improve model performance
Once you’ve identified an example of a corner-case where the model might be struggling, you can easily leverage Catalog to surface similar unlabeled examples to improve model performance.
Use model predictions as model-assisted labeling pipeline
Once you’ve filtered for and have selected reviews that you wish to label you can "Add batch to project" to send them to your labeling project in Annotate.
1) Name your batch
2) Select your labeling project from the dropdown
3) Include model predictions (from your model run) – this will perform better than the initial GPT-4 run with Model Foundry since it has been trained on your custom data
4) Select or uncheck any predictions as desired
5) Submit the batch
When you return to Labelbox Annotate, you will now see the original batch that we added at the start of the project, as well as the newly added batch ready for labeling.
Rather than starting from scratch, similar to the predictions created by GPT-4 in Model Foundry, your labelers will now see the custom model predictions and validate them with human-in-the-loop review in the same manner. This workflow helps accelerate model iterations, allowing your team to bring in the latest model prediction as pre-labels for your project to reduce the amount of human labeling effort required to create ground truth labels.
With new high-impact data labeled, you can retrain the model and can track model improvements across various runs for comparison and how this has affected model performance.
Customer reviews and feedback data represent an invaluable yet untapped opportunity for businesses. Manually analyzing this growing mountain of data is no longer practical. Instead, forward-thinking companies are turning to AI to efficiently sift through and extract actionable insights from reviews.
Natural language processing can help identify customer sentiment, pain points, and unmet needs. By leveraging AI to tap into this feedback treasure trove, businesses can drive measurable improvements in customer satisfaction, retention, and advocacy. They can refine products, enhance user experiences, and preemptively address concerns.
Labelbox is a data-centric AI platform that empowers teams to iteratively build powerful product recommendation engines to fuel lasting customer relationships. To get started, sign up for a free Labelbox account or request a demo.