EAGER: RI: Enabling Natural Language and Decision Making Capabilities of Robotic Guide Dogs for People with Visual Impairment

Suny At Binghamton, Binghamton NY

Investigators

Abstract

Guide dogs are trained to serve people who have severely limited or no vision by helping them avoid obstacles and other dangers while walking. Research has shown that guide dogs improve the lives of visually impaired people by increasing independence, confidence, companionship, and mobility. However, in the US, only about two percent of the visually impaired people work with guide dogs. Example reasons include high training costs, low graduation rates and the everyday care needed by the dogs. Quadruped robots are increasingly drawing attention from the robotics community because they are more versatile than wheeled robots and can traverse many different terrains. The goal of this project is to provide a low-cost, user-friendly solution for guiding visually impaired people, significantly improving their quality of life. Little research on quadruped robotics has been done to look into the interaction and collaboration aspects of human-quadruped systems. The primary focus of this project is on the human-robot-environment, three-way interaction, and the goal is to enable the following novel capabilities for robotic guide dogs. The first capability is dialog and scene verbalization, which will enable the robotic guide dogs to interact with the visually impaired using natural language. The second capability is intelligent disobedience. For robotic guide dogs, intelligent disobedience requires at least the capabilities of visual scene analysis and commonsense reasoning. For learning and evaluation purposes, this project will produce a dataset that includes 360-degree images and safety labels in different directions. Further, the investigator and team will leverage the state-of-the-art computer vision techniques for making disobedience decisions. The first research aim on the language capabilities will improve the transparency of disobedience decisions in the second research aim. This project will generate the datasets and prototypes towards complete systems and comprehensive evaluations of robotic guide dogs. Guide dog trainers and people with visual impairments will be included and play a key role in this project. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.