CIF: Small: Foundations of Explainability and Valuation in Scalable AI through Fast Spectral Methods

University Of California-Berkeley, Berkeley CA

Investigators

Abstract

Artificial Intelligence (AI) systems are rapidly advancing in capability, driven by growth in complexity, model size, and training datasets. However, methods to understand the behavior of these models have not scaled commensurately. Furthermore, there remains limited understanding about how important individual data points are to the model behavior. In order to ensure that these models can be trusted and safely deployed in sensitive applications like autonomous navigation, national defense, and medicine, it is critical to develop a more profound understanding of how they work. Greater transparency into the decision-making process of AI models can be used to accelerate scientific discovery in fields like protein design and drug discovery by revealing novel patterns in complex data. Clearer explanations also enable direct model improvement with the goal of building more robust and trustworthy AI systems. This project will develop a powerful method for making AI predictions understandable and explainable using token masking techniques that combine coding theory and signal processing. This project aims to advance the understanding of Artificial Intelligence (AI) models by leveraging the discovery that AI models exhibit sparsity under spectral representations such as the Fourier transform. By employing ideas from channel coding theory for masking, fast spectral methods, and tools from signal processing and error correction coding, the project will develop scalable algorithms to identify significant input interactions and features providing faithful explanations of AI model behavior. The project will also aim to understand the influence of different training data samples on model predictions, as well as the impact of data imbalances on the dynamics of AI applications in order to create well-designed data acquisition mechanisms. In critical applications such as healthcare, these methods will allow one to validate an AI-generated diagnosis by efficiently identifying the most salient features in a patient's medical record. Furthermore, this framework will help identify when a model relies on irrelevant or harmful data features, providing a path to align the model toward more robust and trustworthy outcomes. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.