Regularized divergences and their gradient flows, generative modeling and structure-preserving learning.

University Of Massachusetts Amherst, Amherst MA

Investigators

Abstract

Generative modeling algorithms underlie many recent and ongoing advances in artificial intelligence, both in popular image and text generation tools as well as scientific applications such as materials design, medical imaging, drug discovery, and cosmology, to name a few. The goal of such algorithms is to learn and construct a model starting from data and available knowledge, and then deploy the learned model to generate new predictions. These predictions are in the form of new data such as new images, new text or even new candidate molecules for drug design. This project involves the development of new mathematical tools from information theory, deep learning, differential equations and probability theory to design, improve, explain and ultimately trust such learning algorithms. The investigators will also apply these new algorithms to merge data sets from different cancer studies, addressing a critical need to improve data analysis by integrating data from the same disease but which are obtained using different studies, technologies, and patient groups. The primary goal of the proposed research is to develop new reliable machine learning algorithms when data is scarce or expensive to obtain. Graduate students will be trained in this field as part of this research project. Probability divergences and metrics are mathematical objects designed to measure discrepancies between different probabilistic models or between models and data and are especially adept in very high-dimensional settings. Divergences need to be carefully designed to construct models which best describe the available data. This project will combine tools from optimal transport, information theory, partial differential equations and deep learning to develop Lipschitz regularized divergences which interpolate between Wasserstein metrics and information-theoretic divergences (e.g. the Kullback-Leibler divergence) and which provide flexible families of loss functions to compare non-absolutely continuous probability measures. In machine learning applications one often needs to build algorithms to model target distributions which are singular, either by their intrinsic nature such as probabilities concentrated on low dimensional structures and/or because they are often only known through data. These new divergences will be combined with deep learning to build gradient flows in a probability space which are capable of transporting any initial distribution to a target data set. These new methods will also be adapted for structure-preserving learning, arising in applications ranging from medicine to the design of new molecules, where data can exhibit symmetries or physical constraints. This additional knowledge will be taken into account in the probability divergence to build structure-preserving generative algorithms in an efficient way. The essential role of structure in generative algorithms will be studied and quantified whenever data is scarce and/or expensive to obtain. One of the demonstration areas of this research is in bioinformatics, where available real datasets, even when they involve the same disease, have low sample size due to budgetary constraints or limited availability of patients e.g., in the case of rare diseases. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.