Enhancing Reliable Data Sciences: Statistical Analysis for High-dimensional and Noisy Datasets with Complex Temporal and Nonlinear Structures

University Of California-Davis, Davis CA

Investigators

Abstract

This project develops new statistical and computational methods to enhance the reliability of data analysis in modern, large-scale datasets, particularly in the era of AI. As data science becomes increasingly central to fields such as finance, medicine, and engineering, there is a critical need for tools that can accurately analyze high-dimensional, noisy, and dynamically changing data. This research addresses fundamental challenges related to robustness, adaptivity, and structure in modern algorithms and statistical procedures. The work contributes to scientific advancement by strengthening the theoretical foundations of data analysis and enabling more accurate and interpretable results across complex applications. It also involves the development of open-source software to ensure broad accessibility and reproducibility and supports the training of junior researchers in advanced statistical methodology and computational techniques. The project advances foundational understanding in statistics by integrating random matrix theory, manifold and deep learning, and time series analysis. It focuses on the following main areas: (1) analyzing the robustness of manifold and deep learning algorithms for high-dimensional, noisy, and nonlinear data; (2) developing statistical theory and methods for high-dimensional, nonstationary time series; and (3) combining insights from these two areas to study complex functional time series with nonlinear and temporal structures. The research examines new classes of nonlinear random matrices that arise from modern machine learning and dependent data structures. It also proposes adaptive versions of manifold learning methods and constructs new inferential frameworks suitable for time series datasets. These contributions are supported by theoretical guarantees and disseminated through interdisciplinary collaborations. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.