CAREER: Information-Theoretic and Statistical Foundations of Generative Models

University Of Maryland, College Park, College Park MD

Investigators

Abstract

Generative machine learning models provide a statistical understanding of data and play an important role in the success of modern machine learning in various application domains including vision, speech, natural languages, and computational biology, among others. Building on the success of deep learning, recent advances in modern generative models hold great promise in revolutionizing various learning methods. Despite this progress, the understanding of some fundamental aspects of these models, required for characterizing their performance guarantees, is still in its infancy. This project aims to elucidate statistical and computational properties of modern generative models by leveraging tools and concepts from information theory, statistics and optimization. This project also includes a comprehensive plan to integrate the research results into an inclusive, diverse and cross-disciplinary educational program at the high school, undergraduate and graduate levels. The overall goal of the research program is to develop a comprehensive and fundamental understanding of the intertwined statistical and computational aspects of modern generative models such as Generative Adversarial Networks (GANs) and Variational AutoEncoders (VAEs). This project aims to make critical advances in proper formulations of generative models for high dimensional distributions, characterizing statistical limits of these models, and developing efficient computational approaches for solving optimization problems involved during their training. This cross-disciplinary project broadens the scope of the prior knowledge on the interplay between information theory and machine learning and creates a tightly connected loop between theory, algorithms and applications in data science. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.