Scalable Spectral Methods for Statistical Analysis

Ohio State University, The, Columbus OH

Investigators

Abstract

Recent rapid developments in information technology, fast computation, data collection methods and storage capacity have lead us to an uncharted territory of data analysis.The scientific research community encounters not only massive volume of data, but also datesets with complicated and nontraditional structures. For example, NASA's satellites generate a massive amount of high-dimensional datasets for climate studies and it is critical to pull out important climate features that may help explain climate changes and predict the future. Meanwhile, the advance in communication technology leads to recent surges on the usage and studies of massive networks such as computer networks and social networks. The emergence of these massive data with nontraditional structures presents new challenges in statistical methodology developments, and even more importantly, algorithm innovations. To address these challenges in this proposal the PI intends to develop scalable spectral methods for statistical inference on Euclidian data, non-Euclidian data, and relationship data. These newly emerging nontraditional type of data and problems provide us with great opportunities for exciting scientific discovery. Spectral methods are natural tools for extracting information and knowledge from these nontraditional data. Along with theoretical analysis of the properties and accuracies of these spectral methods, the PI will develop scalable algorithms and test them in real world applications like climate study and network analysis. The project is motivated by real scientific problems and real world applications related with massive datasets with nontraditional data structures. Besides theoretical development in statistical methodology,the PI will work with collaborators in the fields of Atmospheric Science, Computer Science, Meteorology, and Statistics to apply the proposed inference tools to climate change study. This new set of methods and related softwares for inference on massive datasets will greatly help geoscientists and climate modeler in analyzing climate records and calibrating climate models. Collaborating with Computer Scientists and Statisticians, the PI will also develop scalable spectral algorithms for network analysis, with applications to biological networks and social networks. These tools will assist a broad range of scientists, engineers, and business analysts in scientific exploration, technology innovation, and service improvement.