Geometrization Approaches toward Understanding Deep Learning

University Of Pennsylvania, Philadelphia PA

Investigators

Abstract

The project will study the theoretical underpinnings of deep learning, a widely successful approach to various data-extensive applications. A comprehensive understanding of deep learning is crucial for the development of principled design and training of deep learning models, ultimately reducing computational burden and human costs. The research will span three foundational and complementary directions: understanding well-trained deep neural networks, examining deep learning training dynamics, and exploring data processing in interior layers of deep learning models. By bridging the gap between the complex training paradigms of modern neural networks and existing theories, the project will demystify these black-box models, making them more interpretable and efficient for a wide range of scientific and engineering applications. The project will offer multiple interdisciplinary opportunities for boosting the professional development of the next generation of statisticians and data scientists. The research activities will focus on three research projects to develop a comprehensive understanding of deep learning from a statistical and mathematical perspective. The first project will analyze symmetric geometries in the final stages of deep learning training, developing novel optimization techniques and statistical insights. The second project will provide a detailed understanding of the dynamics of modern deep learning training, integrating statistical and inferential ideas into the active field of deep learning dynamics. The third project will investigate how deep learning separates data according to class membership across all layers of the neural network, using techniques from random matrix theory, non-convex optimization, and learning theory. The successful completion of these research projects will result in a geometrization of deep learning methodologies. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.