Deeper Understanding of Mean-field Models

Columbia University, New York NY

Investigators

Abstract

This project addresses the critical need to approximate complex statistical models in the era of big data. The investigator aims to develop a comprehensive understanding of when simpler approximations can be effectively used, and to provide rigorous guarantees for their accuracy. The core challenge lies in the trade-off between model complexity (which is often necessary to capture the nuances of large datasets) and computational feasibility. While complex models offer rich descriptions, their computational demands can be prohibitive. The investigator proposes to explore whether simpler, approximated models can retain the essential features of their complex counterparts while significantly reducing computation time. Graduate students will be involved in this research. The research will concretely examine the validity of naive mean-field variational inference in several key areas where complex models are prevalent, which include (i) High-Dimensional Bayesian Regression (linear and logistic), (ii) Latent Dirichlet Allocation (LDA), (iii) Mixed Membership Models, and (iv) Exponential Random Graph Models (ERGMs). For each of these examples, the investigator plans to develop tailored inference methods, and provide rigorous, quantified bounds on the errors introduced by these approximations. This will offer a clear understanding of the trade-off between simplicity and accuracy. The overarching goal is to equip researchers using naive mean-field based variational inference with concrete guidelines on when and under what circumstances such methods can be reliably applied, and when they might fall short. This will advance the principled application of approximate inference techniques in the context of big data analytics, contributing to more efficient and reliable scientific discovery. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.