NSF-BSF: RI: Small: Beyond Binary Variables: Defining, Locating, and Editing Multi-dimensional Features in Large Language Models

Trustees Of Boston University, Boston

Investigators

Abstract

This project expands our understanding of what modern artificial intelligence (AI) systems know, and our ability to precisely control them. Large language models (LLMs), a type of AI, have proven useful on a wide variety of tasks, yet their internal decision-making processes remain largely a mystery. Public understanding of these systems is often based on observing their successes and failures, and there is often an assumption that LLMs represent concepts and reason with them in human-like ways. This project challenges that assumption by looking inside the models themselves. While current research has identified simple, binary concepts, or features that are either “on” or “off”, most real-world concepts are more complex. They can be multi-valued, like educational attainment, which can take values high school, college, professional degree, or among others. Concepts can also exist on a continuous spectrum, like measurements of distance or time. By developing methods to find and understand these more sophisticated internal representations, this work aims to ensure that AI systems are reliable, safe, and can be controlled when needed. The first goal of this project is to move beyond binary concepts when interpreting the inner workings of AI systems, and to directly search for and characterize these multi-valued and continuous concepts. Leveraging this understanding, the second goal of this project will then be to develop tools for applying this understanding to improve AI. For example, if we can find out which parts of an LLM perform logical reasoning, we can directly update them without harming the rest of the model, or directly control them to change how the LLM reasons. To achieve its goals, this award supports research in three main stages. First, the project will develop and test methods for discovering multi-dimensional and continuous features within LLMs. This involves using techniques like sparse dictionary learning and causal analysis to move beyond individual neurons and identify the underlying concepts they represent. Second, the project will characterize the geometric structure of these discovered features to understand how a model organizes related values. For example, are the values of an ordinal variable organized along a line, within a more complex, low-dimensional subspace, or neither? Finally, the project will leverage this structural understanding to create new techniques for precise model control. Instead of simply turning features on or off, these methods will allow for nuanced interventions, such as adjusting a concept’s influence on a model’s reasoning process. The developed tools and insights can enable developers and scientists to fix AI flaws without costly retraining. This democratizes AI research by enabling labs with smaller computational resources to participate in model improvement. The project will also support education by demystifying how AI systems work, fostering more practical and scientifically grounded conversations about their capabilities and limitations. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.