Collaborative Research: Partial Priors, Regularization, and Valid & Efficient Probabilistic Structure Learning

North Carolina State University, Raleigh NC

Investigators

Abstract

Modern applications of statistics aim to solve complex scientific problems involving high-dimensional unknowns. One feature that these applications often share is that the high-dimensional unknown is believed to satisfy a complexity-limiting, low-dimensional structure. Specifics of the posited low-dimensional structure are mostly unknown, so a statistically interesting and scientifically relevant problem is structure learning, i.e., using data to learn the latent low-dimensional structure. Because structure learning problems are ubiquitous and reliable uncertainty quantification is imperative, results from this project will have an impact across the biomedical, physical, and social sciences. In addition, the project will offer multiple opportunities for career development of new generations of statisticians and data scientists. Frequentist methods focus on data-driven estimation or selection of a candidate structure, but currently there are no general strategies for reliable uncertainty quantification concerning the unknown structure. Bayesian methods produce a data-dependent probability distribution over the space of structures that can be used for uncertainty quantification, but it comes with no reliability guarantees. A barrier to progress in reliable uncertainty quantification is the oppositely extreme perspectives: frequentists' anathema of modeling structural/parametric uncertainty versus Bayesians' insistence that such uncertainty always be modeled precisely and probabilistically. Overcoming this barrier requires a new perspective falling between these two extremes, and this project will develop a new framework that features a more general and flexible perspective on probability, namely, imprecise probability. Most importantly, this framework will resolve the aforementioned issues by offering new and powerful methods boasting provably reliable uncertainty quantification in structure learning applications. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.