NRI: Collaborative Research: Scalable Robot Autonomy through Remote Operator Assistance and Lifelong Learning

University Of Texas At Austin, Austin TX

Investigators

Abstract

One of the most significant barriers to the wider adoption of autonomous robotic systems in commercial applications is the challenge of achieving 100% reliable autonomy in unconstrained human environments. One path toward more robust autonomy is to spend more time in research labs improving robot capabilities, delaying deployment until autonomy is entirely robust. Instead, it may be valuable to deploy robots out in the wild and adapt their behavior based on the rare examples, corner cases, and contingencies encountered after deployment in order to achieve near-term, fully reliable autonomy. This approach is specifically motivated by the call center model, in which robots are deployed at end-user sites and contact a remote human operator for assistance whenever an error is encountered. This project develops a system that enables robots to perform lifelong, incremental improvement from remote human assistance with the long-term goal of achieving full autonomy. This research program has significant broader impacts, making personal robots more accessible to everyday people, while also providing opportunities for human-robot interaction that are ideal for educational K-12 programs, as well as undergraduate and graduate education. Towards these goals, novel algorithms, interfaces, and user studies are being developed to advance the state of the art in three key areas related to the call center model: (1) Robust, Multi-Sensory Task Outcome Detection: multimodal techniques for identifying conditions under which to seek assistance or deploy recovery behaviors; (2) Transparency Devices for Situated Awareness: visual and language interface modalities for increasing the situational awareness of the remote operator and allowing for intuitive interaction, leading to more efficient and correct recovery procedures; (3) Low-Level and High-Level Task Model Refinement: lifelong learning techniques for incorporating corrections and recovery procedures into existing task models, as well as active learning methods to collect more targeted data. The proposed approach is being evaluated on a variety of mobile manipulation tasks that a hotel concierge robot might perform, such as delivery tasks or preparing for and cleaning up after a conference banquet.