RI: SMALL: Category-Driven Affordance Prediction For Autonomous Robots

Georgia Tech Research Corporation, Atlanta GA

Investigators

Abstract

This research program is developing theory and algorithms that will enable a robot to learn through training and experimentation how to predict object and environmental affordances from sensor data. These affordances determine which actions a robot may perform when interacting with a given object, and thus define the capabilities of the robot at any given time. For example, a doorway affords the possibility to leave one room and enter another, and a handle attached to an object affords the ability to grasp it. The approach being developed leverages a graphical model approach to learn visual categories ? to learn the world contains entities such as doors and handles ? that provide a powerful intermediate representation for affordance prediction and learning. This is in contrast to the classical direct perception approach in which the agent learns a direct mapping from image features to affordances. The models and theory are being validated on two robot platforms and tasks: an outdoor mobile robot performing navigation and pursuit/evasion tasks, and an indoor robot manipulator performing assembly/disassembly tasks. The importance and broader impact of this research lies in empowering robots to actively and effectively learn about its environment given little human training. Because pre-programmed sensing capabilities are typically brittle ? not accounting for the variability of the world in which the robot is actually operating ? and because extensive human training and supervision is too labor intensive, such learning paradigms are essential for the development of robots that operate effectively in the human world.