Collaborative Research: CIF: Medium: Learning and Inference in High-Dimensional Models: Rigorous Analysis and Applications

Ohio State University, The, Columbus OH

Investigators

Abstract

A key feature of contemporary signal processing and machine learning problems is their massive scale. Signal processing tasks routinely involve images and videos with millions of pixels, and modern deep-learning methods often involve millions of tunable parameters. Although recent methods, particularly deep learning, have had tremendous practical success in the high-dimensional setting, they are difficult to explain from a theoretical perspective. This project seeks to develop mathematical tools to better understand such estimation and learning problems along the following directions: How does one tractably formulate precise, high-dimensional analyses of contemporary problems; what do those analyses say about the information-theoretic limits of estimation and learning; and how can these limits be approached by practical algorithms? To achieve broader impacts, the project includes dissemination in workshops, coordination with the growing machine learning industry and the development of a new module on data science to be provided to high school teachers. The project builds on the powerful approximate message passing (AMP) framework, an estimation methodology that offers the potential for a rigorous analytic understanding of modern, high-dimensional problems. Since its origin as a method for understanding linear inverse problems in compressed sensing, AMP has had tremendous success in a wide range of estimation and learning tasks. This project aims to extend the AMP framework to contemporary, large-scale learning tasks. The project is organized into three main thrusts: 1) inference with structured bilinear models, 2) learning of multi-layer neural networks, and 3) analysis with Fourier and convolutional operators. In each thrust, the project will develop fundamental mathematical theory and validate the theory on key applications, particular in image processing and statistical learning. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.