CAREER: Harnessing the Continuum for Big Data: Partial Differential Equations, Calculus of Variations, and Machine Learning

University Of Minnesota-Twin Cities, Minneapolis MN

Investigators

Abstract

Machine learning has broad applications in everyday life, such as self-driving cars, medical image analysis, and speech recognition. Recent years have seen tremendous advances in the ability of machine learning algorithms to replicate and even exceed human performance on many of these tasks. However, we still lack a theoretical understanding of when large scale machine learning will work well to discover interesting patterns and structure in data, and when it will fail to do so and overfit. These problems have recently manifested in adversarial hacking of deep neural networks, which poses risks in sensitive applications where data privacy and security are paramount. The objective of this project is to use the theory of partial differential equations and the calculus of variations to study foundational problems in machine learning and data science, and develop new, more efficient, algorithms founded on strong theoretical principles. The project will advance data science education by developing an annual summer school for high school students on graph-based learning, mentoring high-school students from the University of Minnesota Talented Youth Mathematics Program, developing graduate courses on current research in graph-based learning, and supporting the participation of women and underrepresented groups in all aspects of the project. The overall goal of the project is to use partial differential equation (PDE) continuum limits for discrete machine learning problems to analyze existing algorithms, develop new algorithms with better performance guarantees, and make new connections between machine learning and PDEs. A major focus of the project is graph-based learning, where discrete learning algorithms on graphs can be interpreted as discretizations of continuum PDEs, and properties of those PDEs (e.g., regularity and well-posedness) convey information about the learning problem and can lead to new algorithms founded on strong theoretical principles. In particular, the project will (1) develop and study a new algorithm, called Poisson learning, for graph-based semi-supervised learning at very low labeling rates, (2) draw new connections between stochastic gradient descent (SGD) and viscosity solutions of nonlinear PDEs, showing that SGD has the ability to select good minimizers in ill-posed problems, (3) use ideas from classical elliptic regularity theory to prove interior Lipschitz regularity for solutions of graph-based learning problems on random geometric graphs, (4) prove generalization bounds for graph-based learning in low label-rate regimes, including the active learning setting, and (5) develop a consistency theory for learning problems on directed graphs, such as ranking algorithms. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.