CIF: Small: A Novel Paradigm of Information Extraction in Big Data Problems

University Of Kentucky Research Foundation, Lexington KY

Investigators

Abstract

This project considers problems related to the increasing volume of big data characterized by ultra-large sample size and ultra-high dimensionality. Such volumes introduce many new and unique challenges to scientists in both computation and statistical inference. Finding and isolating the important information in big data is an extremely difficult challenge. In response to such challenges, the project will develop a novel system that can provide quality analysis of big data. With the investigator's experience in statistical research of theory, methodology, and applications, the project will provide an excellent opportunity for both undergraduate and graduate students to participate in cutting-edge statistical applications and methodology development, and thereby prepare them well for their future careers. By localizing the data, this project develops a system with a coherent collection of novel techniques for estimation, computation, asymptotic studies, and statistical inference overcoming the new challenges for big data. The project will lead to new research directions in sufficient dimension reduction (SDR) and sufficient variable selection (SVS), producing new big data mining tools applicable in a wide range of scientific fields. This work is a major step towards improving the understanding of SDR and SVS, including their advantages and how to overcome their disadvantages to suit for the challenge of big data analysis. This project will also provide scientists a new platform to develop more flexible and efficient methods. The investigator will establish theoretical properties of the proposed estimators and develop new scalable computation algorithms for ultra-size and ultra-high dimensional data with an open-access R package to disseminate the knowledge to the scientific community. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.