CAREER: Statistical Inference for Bayesian Machine Learning

University Of Chicago, Chicago IL

Investigators

Abstract

With visible successes on a broad range of predictive problems, the role of machine learning (ML) has become increasingly recognized across a wide array of application domains ranging from economics to electronic commerce. In medicine, for instance, machine learning is routinely deployed for image segmentation, registration, computer aided detection and diagnosis, brain function or activity analysis from fMR images, and text analysis of radiology reports using natural language processing. In spite of this nascent ML trend, there has been significant reluctance to delegate decision making entirely to machine intelligence. This has been largely due to the absence of a formal statistical framework for uncertainty quantification and interpretability. This yawning gap between theory and practice presents new exciting research opportunities for theoretical developments that will justify and unleash the potential machine-assisted decision making in real life. This project has two broad objectives. The first one is motivated by the currently unmet demand for theoretical justification of widely used Bayesian machine learning tools. The second objective is developing practicable methodology for interpretable machine learning, which is essential for gleaning insights into the behavior of real-world processes. The research outlined in this project will bridge current conceptual divides between statistics and machine learning by solidifying Bayesian machine-assisted inference as statistically valid so that it can be safely used to tackle complex scientific problems arising in data-rich environments including imaging, personalized medicine, business analytics, marketing and economics. There has been a growing realization of the potential of Bayesian machine learning as a platform that can provide both flexible modeling, accurate predictions as well as coherent uncertainty statements. In particular, Bayesian Additive Regression Trees (BART) has emerged as one of today's most effective machine learning methods under minimal assumptions. BART has already proved itself to be broadly effective at unveiling structure hidden in high dimensional data across a wide variety of contemporary applications. Its theoretical properties for statistical inference, however, have remained unknown. The detailed research agenda of the first three goals aims at obtaining an in-depth theoretical understanding of BART (as well as some aspects of Bayesian deep learning) through the investigation of (1) uncertainty quantification and confidence set constructions, (2) adaptability to spatially inhomogeneous objects, (3) asymptotic normality for causal inference. The completion of these objectives will significantly advance the current frontier of semi-parametric and non-parametric Bayesian theory. The principal investigator will develop new scalable tools for interpretable machine learning which will extend the reach of ML to many new application areas and problem types involving big data. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.