CAREER: Robotic Manipulation Using Deep Deictic Reinforcement Learning

Northeastern University, Boston MA

Investigators

Abstract

As robot tasks and environments become more complex, it is getting too challenging to program every detail of the robot's behavior explicitly, by hand. An alternate approach is to learn behaviors through experience, a type of machine learning known as ``reinforcement learning'', where the robot learns through trial and error. Pure trial and error, however, is inefficient, which means it takes the robot a long time to learn. The goal of this project is to enable robots to focus attention on the parts of the environment that lead to effective learning and good generalization to new tasks. A result of this research is the ability of assistive robots in the home, such as an assistive wheelchair equipped with a robotic arm, to learn how to better help the infirm and people with disabilities. This project will develop a new approach to applying deep reinforcement learning (deep RL) to robotic manipulation problems by incorporating deictic representations. A deictic representation encodes state/action relative to a marker that the agent places in the environment. In this project, the marker is a 6-DOF reference frame, placed in a 3-D point cloud, or truncated signed distance function. The robot decides where to place the marker and how it should move relative to that marker by solving a Markov decision process using deep reinforcement learning. Preliminary results suggest that this new method can enable robots to learn control policies that solve complex manipulation problems without the need for precise geometric models of the objects being manipulated. While the method still estimates some elements of object pose implicitly, it does so in a way that generalizes well to novel objects and does not necessarily estimate full object pose unless required by the task. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.