Product Updates: July 2018

Labelbox adds Auto Consensus, TensorFlow integration and usability features.

We are very thankful to all of our passionate users who have helped us continuously improve Labelbox. Over the past few months, we’ve made a number of usability improvements. Now, we are taking a major step to help our users easily create and manage training data. Read on to learn about new features in Labelbox.

Auto Consensus

Auto Consensus allows you to quantitatively measure the quality of your training data — this is important because high quality training data produces performant AI. Auto Consensus works in real time so you can take immediate and corrective actions towards improving your training data and model performance.

How does it work?

When Auto Consensus is turned on it automatically distributes the task to the team of labelers such that at least 2 labelers annotate some or all of the same data rows. Auto Consensus configuration can be found under Settings → Quality.

In the example below, user 1 and user 2 were given same instructions to create bounding boxes around damaged areas on the roof. While labels on both images look largely similar, they are only ~45% similar! Not only are the size of the rectangles different but both users have missed annotating some parts of the damaged roof. This kind of inconsistency in training data can yield a poorly performing model.

It would be a lot of work to go through each of the images to look for such inconsistencies and therefore Labelbox automatically provides a precomputed consensus score akin to intersection over union. The data rows that have a consensus score available are identifiable via the stacked square icon on the right.

You can understand the the holistic quality of your labeled data from the consensus histogram available on the project overview screen.

TensorFlow Export Beta

We’ve made it simpler for you to export your training data from Labelbox into Tensorflow. Now you can export your labeled images in Tensorflow’s binary format (.tfrecord). This feature is currently in beta and we’d love to hear how we can further improve this to save you more time.

Re-enqueuing labels

Its now possible to selectively delete labels and send them back to be relabeled. Simply select the labels in activity tab and press “Re-enqueue”.

Visualizing annotation classes (just press “ . ” )

We’ve made reviewing annotations easier by allowing you to view the classes inline with the annotation. Simply press “.” in the labeling interface.

Get Started with Labelbox

Visit www.labelbox.com to evaluate Labelbox for free or speak to one of our team members to find a right solution for your business.