Interface-Aware Intelligence for Robot Teleoperation and Autonomy

Northwestern University, Evanston IL

Investigators

Abstract

Humans issue control signals to robot systems in contexts ranging from teleoperation to instruction to shared autonomy, and in domains as wide as space exploration to assistive robotics. Whether via overt body movement or electrical signals from the muscles or brain, for a human to issue a control signal to a robot platform requires physical actuation of an interface. Deviations between the true signal intended by the human and that received from the interface—in magnitude, direction, or timing—can have rippling effects throughout a robotics autonomy system. This award will demonstrate both the need for and utility of interface-aware robotic intelligence. The research will explore how the physical source of the human control signal—their physical capabilities, the interface actuation mechanism, signal transmission limitations, could impose an artificial upper limit on the human-robot team synergy and success. This limitation impacts the teleoperation and autonomy of any robot system, but it can be felt acutely within the domain of assistive robotics, where human motor impairment and accessible interface limitations can result in dramatic operational constraints. As part of the project, annual outreach demos at a local museum will educate K-12 students on assistive robotics. Additionally, undergraduate students will be retained on summer internships for advanced research experience in robotics. The characteristics of a particular interface, operated by a specific human, leave an imprint on the control signal that can be mined for information pertinent to the intelligent interpretation of the human’s control command. In this project, novel robot intelligence paradigms will be designed that aim specifically to complement characteristics of, or compensate for degradations in, control signals issued from a known and characterized combination of control interface and human operator. To do so, a framework for interface-awareness that offers a more complete model of the input pathway from human to robot control system will be designed, and within this framework interface-usage interpretations of and techniques to elicit user-defined maps from human input to robot control space will be developed. Extensive user studies will be performed both to motivate and evaluate the efficacy and impact of interface-aware robotic intelligence within two salient application domains dramatically impacted by the choice of interface activation and mapping: shared autonomy, anchored to physically assistive robots operated by persons with motor impairments, and human-to-robot instruction, anchored to robotic arm behavior demonstration. This work holds the potential to innovate human-machine interactions by mining and modeling information already imprinted upon human-issued control signals, and in doing so achieve a higher level of human-machine symbiosis. This project is supported by the cross-directorate Foundational Research in Robotics program, jointly managed and funded by the Directorates for Engineering (ENG) and Computer and Information Science and Engineering (CISE). This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.