CIF: Small: Towards a Control Framework for Neural Generative Modeling

University Of Illinois At Urbana-Champaign, Urbana IL

Investigators

Abstract

Generative machine learning are neural networks that are trained on input data so as to then generate new data with similar characteristics. In particular, generative machine learning has been used for the creation of images, and recent work has focused on diffusion-based neural networks driven by an image-to-image translation network trained to gradually remove noise. This project adopts control-theory methodologies to provide a theoretical framework for understanding diffusion-based generative machine learning. By framing the operation of diffusion models as an optimal control problem, the investigators seek to establish a foundational link to the domains of partial and stochastic differential equations with the aim to understand generative models in terms of their controllability, expressiveness, computational complexity, and robustness. In contrast to current diffusion models which rely heavily on empirical design with limited theoretical foundation, the project seeks to greatly improve training for generative networks at a reduced computational cost. Given the current widespread interest in generative models in numerous applications, the project has the potential to bridge multiple technical communities, particularly given its theoretical focus. The investigators also plan to include undergraduates in the research endeavors as well as to incorporate ethical and societal ramifications of generative machine learning in their educational activities. This project on a control framework for neural generative modeling will weave together several distinct intellectual strands in the novel context of generative modeling – namely, control of stochastic trajectories and ensembles, control of partial differential equations (PDEs), and classical theories of PDEs for multiscale image analysis. The research program is articulated around three major directions: (1) control of diffusion processes; (2) control in the space of densities; (3) control of image PDEs. The first direction will develop a first-principles framework for generative modeling by drawing on the techniques of optimal control of diffusion processes. The second direction will build on this framework to phrase generative modeling as a control problem in the space of probability densities, potentially bypassing explicit use of stochastic differential equations. This will connect the setting of generative modeling to the problem of optimal control of PDEs of Liouville type that describe the evolution of probability densities under the action of smooth flows. Finally, the third direction will expand the scope of this inquiry to the more general class of PDEs arising from the axioms of multiscale analysis in the context of image processing. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.