NASA JPL's OWLS project builds an onboard model to find signs of life in ocean-world video. Labelbox produced the expert video signal that teaches it life-like motion, set up in a day.

The challenge

Earth's ocean water holds billions of microbes. Researchers have found ice and water vapor on Europa (a moon of Jupiter) and Enceladus (a moon of Saturn). NASA JPL's Ocean World Life Surveyor (OWLS) project is preparing for a possible spacecraft to one of them, carrying microscopes that collect video of water samples. If the footage shows microbes, NASA will have found extraterrestrial life. The catch is downlinking: the moons are so far that returning data is astronomically expensive — less than 0.01% of what's gathered can come back, like summarizing the entire Lord of the Rings trilogy in 57 words. Two compressed 40 MB video samples alone would eat a third of the science data budget. So the MLIA (Machine Learning Instrument Autonomy) team set out to build a model that finds the videos most likely to contain life-like motion, clips the candidates, prioritizes them for downlink, and explains its decisions. It had four requirements:

1. It must fit onboard a spacecraft's computer, which is about as powerful as a mobile processor

2. It should find and track any moving particles

3. It must distinguish life-like motion from drifting or jostling using ML

4. It must be able to efficiently explain what it saw and how it made its decisions

Limited onboard compute and the need for explanations led the team to traditional ML — decision trees and SVMs — rather than deep learning. And recognizing life-like motion is genuinely hard.

While human beings are really good at being able to tell whether or not something is alive by the way it moves, it’s a much more difficult task for a computer,” said Jake Lee, a Data Scientist in the MLIA group at the Jet Propulsion Laboratory. “Different species of microbes move differently according to their environment. There are also free-floating particles that may move randomly and seem organic to a computer.

The approach

The model also has to earn the trust of the scientists who would use it.

At the end of the day, what matters is that we work with the scientists to help them trust the model,” said Lee. “They won’t trust it if it seems like a black box. So we are working closely with them throughout the process and showing them how we’re training the model. Our goal is that, when the model generates results in the end, the people using it know how it produced those results. They know what the algorithm is doing, and what its benefits and limitations are.

So scientists are integrated into the ML pipeline, reviewing outputs and verifying the model tracks the right objects, folding their expertise into each iteration. To build the algorithms, the team needed signal from a large, diverse set of field water samples and lab-grown specimens. Labeling was easy; building and maintaining in-house labeling tools was not — PhD researchers were writing Java GUIs and sorting samples by hand. The team turned to Labelbox for both the platform and workforce, uploading video from Digital Holographic Microscopes. With the video editor and basic training, contributors tracked objects' movement over time and judged whether each might be a live organism, and the Labelbox Workforce annotated quickly so the MLIA team could focus elsewhere.

Labelbox is so easy to use. The documentation is accessible, and the labeling pipeline is straightforward. We just had to upload our data, customize the editor to our exact requirements, and go. We had actually budgeted a week to get it set up, but we were done in a day.

In addition, Labelbox Boost has been a game-changer for us. We needed a workforce that could deliver fast turnaround with quality labels, and a powerful platform that we could use for years to come. Boost exceeded all of our expectations and helped us create significant improvements in our model.

— Jake Lee, Data Scientist, Jet Propulsion Laboratory

The outcome

Labelbox set up about 5x faster than expected — the team had budgeted a week and finished its most manual-intensive tasks in a day — and Boost delivered fast, high-quality signal that drove significant model improvements.

Where this goes

The search for life is an eval problem in deep space: a model that knows what life-like motion looks like, trained on expert-graded signal, and trusted enough to decide what's worth sending home.