CAREER: Controllable generation for instruction-following language models

Stanford University, Stanford CA

Investigators

Abstract

Instruction-following language models like ChatGPT are beginning to see widespread development, and the ability to understand these systems and control them is critically important to make sure that they benefit society. Despite the success of these language models in generating fluent and convincing-looking outputs, there has been a growing body of work indicating that these systems can generate outputs that are undesirable to users, model creators, and even society at large. This gap between the ability to create models that imitate humans and the inability to have them fulfill specific desiderata (e.g. refuse to generate incorrect information) shows a major deficiency in the ability to precisely control these systems. This project aims to build principled, transparent, and precise methods for controlling language models. To achieve these goals, this project views controllable generation as a viable long-term path to creating instruction-following language models that precisely follow our design goals. Controllable generation offers several benefits. First, it defines a precise statistical modeling problem on which it is possible to build principled methods and rigorous evaluations. Second, it separates the control target from the task, improving transparency by allowing users to see exactly what is being optimized by the model designers. Third, it enables much more precise controls via inference-time methods such as rejection sampling, which strictly enforces the control as a constraint. While controllable generation has major long-term benefits for language models, there also remain significant open problems that must be resolved first, including the difficulty of performing discrete search, the need for specialized training, and the lack of realistic benchmarks of control tasks in the wild. We will address these challenges through a combination of new models (such as diffusion-based models), zero-shot and decoder-based control methods, and a broad benchmark of in-the-wild control behaviors. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.