CRII: CHS: Identifying When People Need a Robot's Assistance

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

Robot collaborators and assistants have the potential to improve lives by helping people perform physical tasks more safely, quickly, and effectively. For example, wheelchair-mounted assistive robot arms can help people with motor impairments perform activities of daily living (like eating) independently, increasing their self-sufficiency and quality of life. However, robot assistance is limited by the fact that robots cannot always recognize when people want or need help. The goal of this research is to develop algorithms that enable robots to recognize when a person is having difficulty with a physical task, based on their behavior before they reach a failure point, and then provide the necessary assistance to complete the task. This work will draw from psychology to explore how nonverbal behaviors like eye gaze, body posture, and facial expression can reveal people's need for assistance. The project will include a data collection study of nonverbal behavior during robot operation. The nonverbal behavior collected during this study will be open sourced to enable other researchers to draw insights about human behavior during human-robot interactions. The work will improve the usefulness of collaborative and assistive robots and lead to better integration of personal robots in workplaces, homes, and assistive care environments. The main research question in this work is: can robots recognize that a person needs assistance based on their nonverbal behaviors during a human-robot interaction? To investigate this, the project has four goals. Goal 1: Recognize the need for assistance. The work will begin with a large-scale data collection of people's nonverbal behavior (eye gaze, body posture, and facial expressions) during an assistive human-robot manipulation task. Using machine learning approaches, the project team will train predictors on the data that can use nonverbal behavior patterns to recognize when people need assistance. Goal 2: Provide assistance. By monitoring real-time nonverbal behaviors during a human-robot interaction, this system will use the trained predictors from Goal 1 to identify when a person needs help. Once the system predicts that assistance is required, it should be able to provide that assistance seamlessly and in real time using shared autonomy. Goal 3: Evaluate the system. Individual system components will be validated separately, then a full-scale evaluation will be conducted to measure the utility of the implemented system in a real-world assistive human-robot interaction. Goal 4: Create and disseminate an open source data set. A major goal of this project is to collect and share a data set of nonverbal behavior during assistive human-robot interaction, which represents a novel contribution to the field. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.