Variable Selection via Measurement Error Modeling

North Carolina State University, Raleigh NC

Investigators

Abstract

Technological advances make it possible to collect and store enormous amounts of data. The implications for how businesses run (online retailing, precision manufacturing), how science is conducted (environmental science, climate monitoring and modeling, astrophysics), and how governments operate (health care delivery, public safety, homeland security) are comparably enormous. However, for many particular uses of massive data sets, not all of the available information is relevant; and a key first step in many big-data explorations is the identification of the most relevant subset of information required to address the particular question at hand. For example, when studying certain diseases, it is essential to first identify the most relevant risk factors and precursors. The more information that is available, the more difficult it is to identify the most relevant subset for a particular purpose, akin to the problem of finding a needle in a haystack. Just as a threshing machine separates the wheat from the chaff, the research in this project will develop statistical methods that separate the relevant information (the wheat) from that information that is not relevant (the chaff), thereby enabling more focused and productive analyses of large data sets. More specifically, the research in this project will develop methods for identifying the subset of information that is most relevant when the data are used to derive a regression/prediction model or algorithm. In this case the problem of separating the wheat from the chaff is the often-studied problem of variable selection. This project will develop a new approach to variable selection that differs conceptually from existing approaches and promises to offer new insights as well as new methodologies. The new approach is based on the intuitive and universally relevant idea that a non-informative variable can be contaminated with noise without a subsequent loss of predictive power; whereas any amount of contamination to an informative predictor necessarily entails a loss of predictive power. Starting from the noise-contamination idea of variable informativeness, the project shows how the theory, methods, and algorithms from the field of measurement error modeling can be used to develop new methods of variable selection applicable across the full spectrum of model- and algorithmic-based prediction methods. Instances of the general strategy will be studied and refined for several particular prediction methods such as: nonparametric regression (based on splines, or kernels, etc.); classification/regression trees; dimension reduction methods (principle components, partial least squares, SIR, etc.); bagged or model-averaged predictors of any type; and ridge regression.