Theoretical Guarantees of Statistical Methodologies Involving Nonconvex Objectives and the Difference-Of-Convex-Functions Algorithms

Georgia Tech Research Corporation, Atlanta GA

Investigators

Abstract

This project will extend the statistical literature that involves nonconvex optimization to contemporary models. In many contemporary machine learning and/or artificial intelligence applications, deep learning and relevant neural network models are utilized. Extending these theories to other contemporary frameworks can potentially lead to a theoretical foundation for modern techniques such as deep learning. The research project has great potential to make a significant impact on the broad scientific community, who have the needs of performing inferences for their enormous data. Besides scholarly publications and presentations, the research will lead to new teaching modules in statistics and machine learning courses. Ph.D. students will be supported and exposed to asymptotic theory and computational algorithms. New toolboxes will be developed and made available online. Packages are developed so that engineering students (including undergraduates) at Georgia Tech and other universities can use them in their course projects (for example, the undergraduate senior design projects at the School of Industrial and Systems Engineering at Georgia Tech). The PI has organized many influential workshops in the past, including one on the foundation of deep learning, and will continue doing so. Specific aims include the following. The research work will extend the theory on the statistical properties of potentially fully neural network models to some other neural network models under different structures, such as the convolutional neural networks, to explore the relation between the inferential property and the neural network architecture. The project is to derive the theoretical guarantees of statistical estimators that are based on nonconvex optimization in more general settings. The PI will explore the possibility to carrying out similar analysis in neural network-based models. Statistical model selection can be utilized in identification of partial differential equations. The project is to establish the corresponding statistical theory and uncover the related practical implication. A set of open-source software products along with related documentation will be generated, to make our work conveniently reproducible. Existing tools (such as GitHub.com or equivalents) will be utilized to disseminate these tools. The applicability and need of the new methods will be explored in a wide spectrum of application domains. Inference techniques with nonconvex objective functions is a fundamental problem in many contemporary techniques, including the neural network based deep learning methodology. This project will contribute to this research. There are evident societal needs for inference from large datasets, and the results of this project can have many applications. The project will contribute to the statistical literature by exploring a new research frontier in statistical sciences. Our work is interdisciplinary and can bridge the communities of optimization and statistics. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.