Valid Inference when Analytical Models are Approximations

University Of Pennsylvania, Philadelphia PA

Investigators

Abstract

Statistical inferential methods are used to answer questions throughout modern life. For example, what affects the crime rate in a town; which factors are important influences on the housing market; which genes are associated to a certain disease; what are the most important elements to control in order to mitigate climate change? Statistical methods are used to address questions such as these. However, often the statistical data structure and the mathematical model developed for the analysis do not agree. This project arises from a broadly based statistical concern about the mismatch between standard inferential analyses and the statistics of the world they are trying to describe. This research draws a distinction between the statistical models that conventionally describe the correlational relations in experimental and observational data and the inferential models that are used in their analysis. To this end, the project investigates a paradigm in which sampling models are meant to be faithful representations of the real-world structure of the data they are describing. At the same time, the analytical models to be applied to the data are viewed only as approximate descriptions of that reality. The statistical-sampling representations need not match the analytical models, though the two should harmonize in certain important respects. There is a significant disparity between accurate characterization and what is claimed by classical procedures that ignore this distinction. The distinction has been noted by many previous statistical researchers, and various partially adequate approaches have been suggested. Nevertheless, clarifying this distinction in the directions under study and then pursuing the consequences leads to a theory of inference somewhat different from that in common use for relational and observational data. Acknowledging and properly accommodating this duality then leads to new methodology for some important statistical problems. One such new methodology is within the setting of randomized clinical trials in which one wishes to estimate the effect of certain treatment(s) relative to others or to placebo controls. Another is within the setting of semi-supervised learning that occurs in various big-data contexts. The core of the current research is designed for linear analytical models. These involve observations on a vector of explanatory covariates (X-variables) and a numerical dependent variable (Y). The analytical model constructs the best linear approximant of Y as a linear function of the X variables. Virtually no assumptions are made about the (X,Y) pairs in the sample, other than that they form a statistical sample drawn from some unknown joint distribution of (X,Y) pairs and possess desired low-order moments. The notion of "best" is defined in a statistically natural fashion related to minimizing squared prediction error. It follows that the ordinary least squares estimators of parameters still have desirable asymptotic properties. Inference about their (asymptotic) performance can be derived via the standard sandwich estimator. However, a newly derived iterated pairs-bootstrap is shown to give substantially more accurate inferential information for realistic sample sizes. If more information is available about the distribution of X (such as knowledge of its mean and variance) then the usual least-squares solutions can be improved. This observation leads via an indirect path to suggestions that improve the standard methodology for estimating average treatment effect in randomized clinical trials and for producing linear predictions of numerical outcomes in settings of semi-supervised learning. Various additional issues are exposed in the course of the above developments. We also plan to investigate generalizations of the above setting -- for example to models having categorical Y-variables (classification) and to other generalized-linear analytical models. Our earlier research involved post-selection inference in the classical setting in which models for the data and its analysis coincide, and we now intend to pursue analogous issues in the current context in which they do not.