RI: Small: Learning Generalizable Abstractions for Fast and Reliable Planning Under Uncertainty

Arizona State University, Scottsdale AZ

Investigators

Abstract

Artificial intelligence (AI) systems such as hospital robots and disaster-recovery support systems need to plan reliably and efficiently to accomplish complex user-desired tasks such as cleaning up a room or creating an effective resource-management strategy. This amounts to models for reasoning over millions of decision points while taking contingencies into account. Current approaches typically use hierarchical world models, which commands actions from a high-level central source to lesser agents, for making this process feasible. However, creating such models requires extensive hand-engineering by domain experts who have prior knowledge of expected tasks – thus undermining the utility and applicability of autonomous systems. This project will develop a new class of algorithms for enabling AI systems to autonomously learn hierarchical world models and high-level actions. For instance, using the approaches to be developed in this project, a disaster-recovery support system would be able to distill its past experience into relational world models with high-level actions (e.g., ``use truck 17 to deliver trauma medication to survivors at site 3'' and ``deploy search and rescue personnel to site 2'') and to reason over them rather than over what to do at each millisecond. The resulting auto-generated abstraction hierarchy will enable AI systems to compute safe and reliable plans for user-desired objectives in stochastic settings. These objectives will be achieved through two broad classes of algorithms for learning abstract world models. Both approaches will address problems involving long-horizons and sparse rewards in stochastic settings where test problems are significantly different from training problems. Bottom-up abstractions will focus on learning abstractions based on unannotated behavior in training scenarios. This thrust will develop algorithms for learning to predict critical regions (sets of salient states) in test problems, and then for inventing high-level actions as moves to and from critical regions. Top-down abstractions will start with an uninformed, coarse abstract representation of the state space and selectively increase the resolution of representation to better explain the ongoing experience. Both paradigms will yield well-defined, interpretable world-models that can be used for scalable and reliable planning in real-world settings. Benchmarks, test scenarios, algorithms and software developed in the project will be made broadly available to the research community in open-source format. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.