×

A self-retraining data engine for fall-prevention AI

Problem

The VirtuSense team struggled to distribute unstructured data to its labeling team and track progress efficiently. With open-source tools, the lack of immediate support when issues arose was another blocker.

Solution

The VirtuSense team designed a data engine with Labelbox to automate every aspect of the model iteration loop, so the only human involvement required is contributors annotating data.

Result

Since adopting Labelbox, VirtuSense increased the amount of labels created by 5X and has produced over one million labeled assets. False alarm rates fell from roughly 28% to 6%, the daily average number of alerts dropped from seven to five, and average accuracy increased by over 20%.

A self-retraining data engine for fall-prevention AI

VirtuSense detects patient falls seconds before they happen. Labelbox is the data engine that retrains its edge models every few weeks, cutting false alarms from 28% to 6%.

The challenge

VirtuSense founder Deepak Gaddipati started the company after his healthy grandmother fell, broke her hip, and died from complications — no one knew she was a fall risk. By 2014 he founded VirtuSense, an AI system of sensors that monitors patients and alerts caretakers when movements signal a fall risk. Every hour, seven people die from fall-related injuries in the US. VirtuSense is the #1 fall prevention system in the world; in two and a half years it has prevented over 100,000 falls and saved tens of thousands of lives. It places LiDAR sensors in each room, ambulance, or facility; AI models on the edge devices track movement patterns and, per peer-reviewed research, flag potential falls 30-65 seconds before they occur, alerting nearby staff. The system now also detects pressure-ulcer risk. The hard part is trust: nurses who get too many false alarms stop taking alerts seriously — current technology produces 10 to 20 alerts a day, many false. To earn that trust, VirtuSense had to push model accuracy higher, and that took a more diverse, higher-quality training signal it struggled to produce with a mix of internal and open-source tools.

The approach

VirtuSense chose Labelbox and designed an AI data engine to maintain and raise model accuracy — automating every step of the iteration loop so the only human involvement is contributors annotating data. The pipeline: find low-confidence areas by running models on a curated test set of edge cases and rare situations; clean the relevant data of metadata tying it to a sensor, facility, or patient (for confidentiality and to prevent bias); pre-label it with a purpose-built model; import programmatically via the Labelbox Python SDK; have an internal team review and correct in Labelbox; export to the model training system. Trained models are pushed automatically to thousands of edge devices, and VirtuSense retrains every few weeks. Because edge models are too small to pre-label well, VirtuSense built a large model just for pre-labeling, so contributors mostly review and correct.

We were very pleasantly surprised by how easy [Labelbox] is to use. People got the hang of it just like that. And it's not just about how easy it is to label data. We really like its ability to distribute data to multiple people and keep track of what's happening — it gives us a base of operations to help traverse through all that data….but the real beauty of it, the reason we really love Labelbox, is that it integrates so well with our tech. From a thousand-foot view, it looks like an AI labeling engine, which feeds into the deep learning system and spits out trained networks.

— Deepak Gaddipati, Founder & CTO of VirtuSense

The outcome

Since adopting Labelbox, VirtuSense increased the labels it creates by 5X and has produced over one million labeled assets. False alarm rates fell from roughly 28% to 6%, the daily average number of alerts dropped from seven to five, and average accuracy increased by over 20%.

As a CTO, you usually only usually hear about the technologies your team uses when things are going wrong. With Labelbox however, the only time I dealt with it is the day I signed the contract. Since then, my team has been using the technology seamlessly without any issues, and this gives me a lot of peace of mind as a technical and engineering leader.

— Deepak Gaddipati, Founder & CTO of VirtuSense

Where this goes

With its data engine, VirtuSense is expanding beyond LiDAR to detect stroke risk, arrhythmia, and other dangerous conditions. A model that retrains itself on the signal where it's weakest is how safety-critical AI earns — and keeps — clinicians' trust.

Out of all the platforms we explored, Labelbox is the easiest to use and the best for managing labelers and workflows and monitoring performance. Labelbox also keeps offering more interesting and helpful features. I find the Catalog feature very interesting. We haven’t been able to use it yet, but we are planning to update our workflow to include it in the future.

— Hithesh Reddivari, Tech Lead, VirtuSense