Robotics: Flexible manipulation without prior shape models

Massachusetts Institute Of Technology, Cambridge MA

Investigators

Abstract

Robots have the potential to improve peoples' daily lives by performing everyday tasks in homes, hospitals, and restaurants. Unlike the carefully designed and accurately modeled environment in a factory, a home is filled with a wide and changing variety of objects, often in messy and unpredictable arrangements. Most current methods for designing intelligent robots depend on having highly accurate models of their environments, which cannot be obtained reliably for our domains of interest. This project will develop strategies for solving robotics problems in environments that are only partially understood in advance, through a combination of reasoning about uncertainty (recognizing, for example, that it doesn't know how full a particular container is), taking actions to gather information (picking up the container to see how heavy it is, or looking inside it), and selecting actions that will work well even in spite of some remaining uncertainty (sweeping objects off a table and into a box can be effective, even when their positions are not well known in advance.) Ultimately, this project will enable robots to be deployed in much less restrictive environments and to provide more robust and flexible assistance to people who need it. To achieve these goals, this project focuses on identifying and estimating the domain information that is crucial for performing a task (for example, how to pick up an object stably or whether placing one object in a certain location would cause a collision). It will design a state-estimation system that combines pre-trained neural-network perception modules (for segmentation, shape completion, grasp prediction, etc.) with classical engineering techniques for multiple hypothesis tracking. This system will be the basis for a task and motion planning system that operates in belief space, enabling it to explicitly plan and execute information-gathering actions. The planner will make an abstract plan that consists of lower-level closed-loop control operations that also take uncertainty into account, by synthesizing appropriate impedance controllers with guard conditions specified in terms of observed torques, tactile percepts, and distance traveled. By applying closed-loop control at both the low and high levels of abstraction, and explicitly modeling and controlling uncertainty, the overall system will demonstrate robust, flexible behavior in complex domains with novel combinations of previously unseen objects. The project will use physical robots that are reasonably priced and widely available and will make all resulting algorithms and implementations freely available through open source software. This project is supported by the cross-directorate Foundational Research in Robotics program, jointly managed and funded by the Directorates for Engineering (ENG) and Computer and Information Science and Engineering (CISE). This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.