RI: Small: Concept Formation in Partially Observable Domains

University Of Maryland Baltimore County, Baltimore MD

Investigators

Abstract

This research focuses on providing artificial intelligence (AI) systems with ways to represent knowledge about a problem domain, by creating descriptions of its observations over time. This work is important because the AI systems can transfer their knowledge from one problem domain to another, enabling them to learn complex behaviors in different environments over time. In addition, the learned representation provides a basis for creating explanations of the agent's behavior, a capability that is becoming increasingly important as AI agents are being applied to more aspects of our daily lives. The resulting learning transfer methods we will create are applicable to a wide variety of problems of interest to the broader AI community, including explainable systems, intelligent wearable computing, and robotic assistants in real world environments. The agents enabled by this work will automatically extract concepts (high-level descriptors) from perceptions, construct a hierarchy of experiences, and record learned behaviors over this structure, by extending existing reinforcement learning methods with these novel representations. Concepts serve as simple, portable, efficient packets of hierarchical knowledge that can be learned in parallel. Our novel contribution, concept-based memory, extends previous work on concept formation to identify useful properties of the domain that are not directly observable in all contexts, expanding the agent's world model and improving performance in partially observable domains. Concept-based memory provides a process for creating multi-layered abstract representations of a domain and the tasks in the domain, enabling learning transfer across multiple tasks, and providing a basis for creating explanations of learned behaviors. Our method for concept formation in reinforcement learning domains, called concept-aware feature extraction (CAFE), produces concept-lattice representations that permit knowledge learned from one task to be applied to a new problem by identifying the appropriate level of generalization for common knowledge between the tasks. We will enable scalability by integrating CAFE with abstract Markov decision processes (AMDPs) and by developing heuristic pruning methods that reduce the branching factor of the concept lattices This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.