CAREER: Learning World Models from Descriptive Language

Oregon State University, Corvallis OR

Investigators

Abstract

Learning to fix a car, operate a new appliance, or use new software through “trial and error” are inefficient and potentially dangerous propositions. To avoid this issue for human learners, tremendous effort has been spent in developing user manuals, field guides, tutorials, and other training documents that provide descriptions of how complex systems function. These allow the reader to bootstrap a useful mental model of a new system before actually interacting with it. Even when this mental model is incomplete, it may still provide a useful starting point that improves the speed and safety with which the reader learns to interact with a new system. For instance, an excavator's cockpit contains a multitude of joysticks, buttons, and pedals to control the excavator itself and its bucket arm. A description of their function would inform a new operator how to begin exploring the system, even if the exact sensitivity of the controls or speed of the motion was not well specified in the text. This is especially compelling compared to an overly eager operator that learns by trial and error without first consulting a manual. This project aims to extend a similar capability to artificially intelligent (AI) systems -- allowing AI systems to more efficiently learn to interact with new systems by reading manuals written in natural language. If realized, this capability will significantly expand the capacity for AI systems to perform digital and physical labor by reducing the cost to develop capable systems for new tasks. The project integrates research with education and outreaches to different groups including underrepresented minorities. From a machine learning perspective, developing AI systems that learn to act in complex interactive environments to achieve high-valued outcomes is often addressed through reinforcement learning techniques. Like our overly eager operator described above, standard methods rely on trial and error to learn. This project will develop a new model-based reinforcement learning paradigm in which natural language descriptions of “how” an environment operates can be leveraged to efficiently learn a predictive model of the environment. This language-guided world model will be capable of simulating the effect of actions and will enable more rapid behavior learning. This project will address three research challenges – 1) developing an empirical testbed for language-guided world model learning that enables systematic investigation of language grounding for descriptive text, 2) designing language-guided world models for both fully-observable and partially-observable settings, and 3) designing mechanisms for these models to adapt to new language over time. These research directions pave the way for artificially intelligent systems to better comprehend and predict complex behaviors described in language and to leverage these descriptions to learn interactive tasks more efficiently. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.