What is the best user experience for implementing machine learning in conservation projects?
In my work at Vulcan, I worked on several projects where we leverage new technology for species conservation, for example our wildlife conservation and ocean health efforts. In both of these projects, we work with organizations that are often underfunded and understaffed. They usually have the daunting tasks of patrolling and monitoring large areas of responsibility. In addition, they have to do these tasks with minimal resources.
Using new technology for good
There was a desire to see how we could use new technology, such as aerial imagery or satellites and machine learning, to aid the organizations in their conservation and enforcement tasks.
One of the initiatives that came out of this desire was to use cameras to do persistent surveillance. These would stream pictures and video back to the basecamp to help organizations monitor their protected areas.
Working with the Machine learning team and the software developers, I facilitated the discussion to come up with guidelines for designing the best user experience for flagging and signaling anomalies in the camera feed so that the user had a clear understanding of what was going on. A summary of which is written below.
We found that to be able to convey the intel to the organizations, there were three crucial things that dominated the User Experience:
- Help the user – Be the Oracle - In our conservation species surveillance scenario, I observed first hand that operators would sit and watch video feeds or images to find traces of anomalous behavior. Since users had to pay close attention to a screen for hours, there was a serious risk of operator fatigue. This would lead to missed detections with all kinds of possible consequences.
- Reduce the noise - Our neural network classification worked fine as long as we didn’t ask it too granular questions; it could classify anomalies with a pretty good success rate but it couldn’t distinguish cows from elephants. In our effort to aid the user, the last thing we wanted to do is to replace user fatigue with annoyance over misclassified detections. How could we make it so that the user wasn’t inundated with alerts and notifications, often caused by duplicate sightings from different frames?
- Build trust with the user – be transparent - As accuracy of our detections wasn’t anywhere close to being acceptable, we needed a way of gradually introducing the user to our ML system’s output. We needed to be just granular enough to stay trustworthy without shooting ourselves in the foot through over-promising. Setting the user expectation was crucial.
To combat user fatigue, we used object detection through our neural network so that we could classify detected objects as animal, human or vehicle. These detections could then be communicated in real time to the users so that they would be able to multitask without the fear of missing detections.
We thought to aggregate detections based on a grid system, somewhat configurable by the user. This would generate only one alert for a given grid, where there could be noise from multiple detections in the same location.
The grid would be visible to the user, so they could use the grid to communicate a location from the command center to the rangers in the field. Eg. They could instruct a patrol to go to G17 without having to communicate coordinates.
We toyed with the idea of simply flagging that our system detected an anomaly in our video or image feed. This was based on the classification limitations of our network. In our next milestone we would then starting using a more granular classification scheme. The end game was an actionable system, in which we could maybe even predict a scenario where rangers could be deployed automatically.
If we wanted to build trust and have the user give feedback to improve the system, we needed to make sure users were aware of the underlying engine. We didn’t want to be just a black box. In our results, we needed to make sure the user understood the system so the user would be able to ‘forgive’ false positives. The more the user understood the way the system classified objects, the more empathy we could build for the system. Not only would the user be more forgiving, they would also be more inclined to help the system get better at domain knowledge by providing feedback on erroneous classified images. (For more on this, see post on Human-in-the-loop learning
Detection and classification still has some ways to go. Accuracy in the field is constantly improving. Even so, many users are skeptical of machine learning and most users want to know how a system works before they trust it. Lessons learned from this project could be essential to user adoption in current and future projects.