A Framework for Manipulation Planning and Execution under Uncertainty in Partially-Known Environments

William Marsh Rice University, Houston TX

Investigators

Abstract

There is a pressing need to make today's robots capable, robust, and efficient during real-world operation. This project focuses on near-future scenarios that require complex long-horizon reasoning with non-trivial constraints on the robot's motion. Examples include a robot operating in a warehouse with a partly automated storage and retrieval system or a robot running experiments in an automated laboratory. Several methods in robotics address the challenges of the above scenarios with explicit and carefully crafted planning domains that model how the robot interacts with the environment. Planning domains provide an abstraction over the world that is essential for Task and Motion Planning (TAMP) methods that plan over long horizons, that is, compute executable complex plans that require many steps or have non-monotonic properties, such as rearranging objects on shelves or fetching reactants for an experiment. This research project is working to develop interpretable TAMP methods with the capability to deal with increasing uncertainty in the environment, while not sacrificing their strengths and providing a structured framework that allows for a meaningful connection with emerging model-free approaches. The project's novelties lie precisely in the development of methodologies that allow the augmentation of TAMP methods with the capability to reason about uncertainty and implicit models. The project's impact is in building the foundations that will enable robots to perform complex tasks such as cleaning a house, helping doctors and nurses, assisting elderly persons, and even performing science-related tasks in the far-off reaches of space. The team is training undergraduate, graduate and postdoctoral students, and pursuing outreach activities including participation to CRA-WP programs. On a technical front, the project supports three aims. The first aim addresses noise in the sensing and actuation of the robot. Factor graphs will be enhanced in a way that allows exploiting inherent structure in long-horizon planning problems to make planning under uncertainty efficient, and loosen assumptions to, e.g., allow efficient use of learned actions. The second aim goes further to consider pathological uncertainty: when there is so little information that there is effectively a gap in the plan. Plans are dynamically modified at execution time to fill these gaps leveraging, among others, learned skills to close gaps. The third aim focuses on augmenting TAMP methods with another source of unknown and difficult-to-model information: human preferences and critiques. The project addresses how implicit and learned representations can be used and accumulated over a system's lifetime. Importantly, the work can fit with any high-level planner, including Satisfiability Modulo Theories solvers and large language models leveraging advances in these domains. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.