CAREER: Reading To Learn: Language-Guided Machine Learning

Princeton University, Princeton NJ

Investigators

Abstract

Humans have used language for centuries in order to communicate with each other and pass knowledge to successive generations. We learn through a combination of 'doing' things to receive feedback from the world (e.g. feeling pain when we put our finger in the fire) as well as 'reading' about how the world works (e.g. Wikipedia might say 'Fire has the potential to cause pain and physical damage through burning'). Modern artificial intelligence (AI) systems learn new skills predominantly through the former method, using a trial-and-error mechanism that requires comparing their own predictions against human-specified answers or judgements. While this approach has worked for automating a variety of tasks, it requires a large amount of data and computational resources, and is limited to task domains where trial-and-error learning is appropriate due to the low stakes involved. This project will develop techniques for a new paradigm of language-guided machine learning that will enable AI systems to acquire new knowledge and skills by reading relevant text in natural language such as books, manuals and webpages. This will result in robust AI models that require less human effort to train while allowing for better user personalization. Current approaches to efficient machine learning such as domain adaptation, few-shot learning, continual learning and reinforcement learning can only operate over task-specific symbolic or mathematical representations pre-specified by model developers (such as class IDs or hierarchies, dynamics models, reward functions) and do not leverage linguistic knowledge providing the same information. This CAREER project will develop models that can ‘read’ to acquire knowledge from textual sources and incorporate it into a better learning process for different paradigms. This includes supervised classification tasks as well as sequential decision-making where an agent executes several actions in an interactive environment. Models that can automatically acquire new knowledge and skills by reading text (from books, webpages, or human feedback) will require smaller amounts of traditional supervision, generalize better to unseen scenarios, and substantially reduce human effort in model development. The project will achieve this goal by tackling three key directions: (1) enabling language-guided supervised learning by developing a new framework for providing semantic class descriptions, (2) improving efficiency and generalization to new domains in reinforcement learning by leveraging offline textual guidance, and (3) enabling online adaptation of policies using linguistic feedback through human-machine collaboration. These thrusts will open new research directions for machine learning with language guidance and enable better real-world human-machine collaboration. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.