Data Driven Predictive Auditory Cues for Safety and Fluency in Human-Robot Interaction

Georgia Tech Research Corporation, Atlanta GA

Investigators

Abstract

Most industrial and social robots are not sufficiently aware of their surroundings, which leads to a wide range of injuries. This can hamper fluent and efficient human-robot work. This project focuses on developing, integrating, and testing a set of sound cues for robotic movements, with the goals of enhancing human safety and allowing fluent interaction. The sound cues are created using algorithms which provide rich information about the robots’ current and future actions, alerting humans to potential hazards, and allowing them to prepare and adjust their work space. Such sound cues bear the promise of using aa non-distracting auditory channel to help humans plan their actions and responses to robotic actions. The system is based on music-driven robotic maneuvers using a novel audio generation method that will provide information about the robotic movements. The algorithm used is trained on a newly created dataset of audio clips with risk information. The system can increase safety, fluency and trust building in human-robot interaction in industrial and personal robots, private and public spaces, addressing tasks in manufacturing, training, education, and others. To address this goal the project is divided into four phases: Phase 1 - collection, analysis, labeling and feature extraction of a newly created dataset of audio clips. Phase 2 - development of a novel neural network model that will be trained on the collected dataset in correlation to labeled set of robotic movements. The output of the model will then be fed to a neural network that will generate long-context raw audio conditioned by musical and gestural features. Phase 3 - integration of the generated audio cues into a large set of robotic gestures with the goal of representing robotic motion and future actions. Phase 4 - a comprehensive evaluation study of the sonified robotic gestures for safety and fluency in a variety of human-robot interaction scenarios. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.