Game-theoretic statistics and safe anytime-valid inference

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

The majority of statistical inference — a catchall phrase that refers to hypothesis testing, confidence intervals, prediction sets, and other forms of uncertainty quantification — relies rather strongly on probabilistic modeling. However, reality does not always accord with the statistician’s models, especially when it involves non-random but yet uncertain events (like outcomes of sports games) or when the data source is not passive but may have an active role (for example, it may have an incentive not to be detected). The use of traditional probabilistic modeling may result in non-robust methodology, susceptible to being fooled by non-stochastic data. This project will develop broad foundations of a fundamentally different approach to statistical inference that was recently termed “game-theoretic statistical inference.” The project also provides research training opportunities to graduate students. Game-theoretic hypothesis testing is based on a broadly applicable principle of “testing a hypothesis by betting against it” (or testing by betting for short). This is a project to develop a basic theory and methodology for nonparametric problems, where the assumption about the source of the data is minimized. Game-theoretic confidence sequences extend the aforementioned advances in testing to the setting of estimation. Since the data is not typically assumed as stochastic, the target of estimation must be carefully specified and could change with time. The project will develop the basic definitions and methodology for constructing such confidence sets and expound on nonparametric examples. Game-theoretic changepoint detection will directly build on the advances in the aforementioned two directions. Many classical change detection methods assume a parametric (and often i.i.d.) structure. The project will develop change detection methods that work in nonstationary and non-stochastic settings under nonparametric assumptions. This work has a few distinguishing points from classical statistical inference: (1) it is inherently sequential in nature, (2) it is often nonparametric and/or model-free, (3) it freely enables continuous monitoring and updating, and (4) it merges frequentist and Bayesian ideas. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.