CAREER: High-Dimensional M-Estimation Under Nonstandard Conditions

Cornell University, Ithaca NY

Investigators

Abstract

Modern technology in genomics, medical research and neuroscience generates enormous amounts of data, which calls for reliable statistical analysis tools. While the past decades have witnessed a surge of research activities on the analysis of big data in statistics and data science, the existing statistical tools are not adequate to produce reliable results due to the complexity of the data structure or the manner in which the data are collected in modern applications. The project will develop novel statistical and computational tools to address the emerging challenges in modern big data and implement them within software packages. The project will benefit a broad range of researchers including biologists, epidemiologists, medical doctors and neuroscientists. The research is complemented by an equally important education and outreach plan including designing new undergraduate and graduate courses and recruiting underrepresented minorities into the summer research program. This project will develop a novel computational and statistical framework for high-dimensional M-estimation under two types of nonstandard conditions. In the first project, the principal investigator will consider high-dimensional M-estimation with non-smooth loss functions (e.g. indicator function). The discontinuity of the loss function leads to nonstandard theory and requires new statistical methods equipped with more refined theoretical analysis. In the second project, the principal investigator will consider M-estimation subject to measurement constraints in the sense that the outcomes are only collected in a very small subset of a big dataset. The principal investigator will develop scalable computational algorithms and statistically valid estimation/inference procedures. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.