Interpretable, stable and scalable machine learning and statistical inference

University Of Wisconsin-Madison, Madison WI

Investigators

Abstract

In recent years, machine learning and artificial intelligence have been widely deployed across numerous disciplines. Much of the focus of this work has been task-driven (e.g., generating text, predicting outcomes, etc.). However, there remain important follow-up concepts that are more closely aligned with human and project interests and interpretability, such as "How accurate or reliable is the result?" or "How do we interpret this result?" and "What are the next steps?" This project will address these questions using a statistical framework that assesses interpretability and stability for any so-called "black-box" approach typically used in machine learning. The outcomes of this work also lead directly to student training at an undergraduate level, so that students are familiar not only with "how" to implement methods, but also how to interpret outcomes and take next steps in the analysis. Furthermore, all the methods developed are made scalable, applying to extremely large and complex datasets by using computational heuristics known to reduce run-time and storage. The project provides research training opportunities for graduate students. This project will address this issue by focusing on 3 specific thrusts: (1) estimating feature importance for arbitrary algorithms in a reliable and scalable way; (2) performing feature selection by developing a statistical hypothesis testing framework for these variable importance estimates; and (3) providing predictive confidence for predictions in a model-agnostic manner. Thrust (1) and (2) provide interpretability measures for black-box models, while thrust (3) addresses the issue of the reliability of predictions. The unifying theme to all 3 thrusts is providing both scalability and reliability guarantees through mathematical theory, simulations, and real data examples. Through these contributions, the project will address issues of interpretability and human confidence in machine learning algorithms. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.