CAREER: Theoretical Foundations for Probabilistic Models with Dense Random Matrices

Duke University, Durham NC

Investigators

Abstract

Many real-world scientific and engineering applications require sophisticated processing of large and complex data sets. Examples include wireless communications, computational photography, and the training of multilayer networks for classification tasks. In some cases, performance is fundamentally limited by the amount of data. In other cases, the main limitation is the computational complexity of processing algorithms. A major challenge for researchers is to understand these fundamental limits. This project explores these limits by studying probabilistic models that describe the statistical relationship between the data and the unknown quantities of interest (e.g., transmitted message or correct label). The research involves combining ideas from information theory and statistical physics to compute fundamental limits and using these results to design efficient methods with improved performance. The interdisciplinary nature of the research is mirrored in the education activities of this project, which focuses on making connections between engineering, statistical physics, and the information sciences, as well as improving undergraduate education through exploratory data analysis. The key conceptual idea behind this research is that statistical dependencies induced through multiplication by dense random matrices can be understood through connections with simpler models involving additive Gaussian noise. In a recent breakthrough, the investigator showed how ideas from information theory could provide rigorous proofs for behaviors that had been conjectured using the heuristic replica method from statistical physics. Building upon this insight, the research is organized around three thrusts: i) Developing new theoretical methods to provide rigorous and interpretable characterization of fundamental limits; ii) Designing new algorithms for inference, learning, and compression; and iii) Analyzing bi-linear and multi-layer inference problems with applications to deep learning. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.