ATD: Gaussian Fields: Graph Representations and Black-Box Optimization Algorithms

University Of Chicago, Chicago IL

Investigators

Abstract

The increasing amount of data and complexity of models used in science and engineering calls for collaborative and interdisciplinary research at the intersection of computational mathematics, statistics and machine learning. This project will develop novel computational mathematics with the aim of facilitating the statistical analysis of large and unstructured spatial data-sets using Gaussian field methods. Gaussian fields are standard models in statistics and machine learning, but their application to large data sets is notoriously difficult. The investigator will show through mathematical reasoning and practical examples that a standard family of Gaussian fields can be accurately approximated using graphs in such a way that the statistical analysis scales favorably to large data sets. As a result, the benefit of sound statistical modeling through Gaussian fields is made possible in large data regimes of current interest. In addition, the investigator will explore new computationally efficient methods that allow one to combine highly complex models with data. A central part of the project will be the training of graduate students in the Computational and Applied Mathematics (CAM) program at the University of Chicago. To that end, the investigator will i) introduce topics of current research interest in uncertainty quantification and spatial statistics in CAM and Statistics courses, both at the Master’s and PhD levels; ii) serve as PhD and Master's thesis adviser of CAM students; iii) help organize the CAM colloquium and the CAM student colloquium, allowing students to learn from, and interact with, leading experts in spatial statistics, Bayesian inverse problems and graph-based learning; and iv) disseminate the accomplished work through conference and seminar presentations. This project has two research thrusts that share a common theme of pushing forward the use of Gaussian field methods. First, the investigator will develop and analyze graph representations of Gaussian fields for the statistical analysis of discrete and unstructured spatial datasets. Second, the investigator will design and numerically explore novel black-box, derivative free optimization schemes that combine Bayesian optimization with ensemble Kalman methods. The graph representations to be used stem from the stochastic partial differential equation (SPDE) approach to Gaussian fields, one of the breakthroughs in spatial statistics in the last decade. The main idea of the SPDE approach is to define Gaussian fields as the solution to an SPDE and represent the solution using finite elements, a perspective that has inspired many modeling and computational developments. The investigator will introduce and explore graph representations, showing through rigorous analysis and numerical examples that they provide a natural way to generalize the Matérn model to unstructured datasets. The investigator will also demonstrate that graph representations seamlessly unify Gaussian field methods in spatial statistics, graph-based machine learning and Bayesian inverse problems, and will transfer several concrete computational methods and modeling ideas across these three communities. The second research thrust will concern the development of new black-box, derivative free optimization schemes. The investigator will conduct a thorough numerical comparison of existing methods, and will explore new ones. Specific objectives of this project include i) to introduce graph-based covariance models for large and unstructured discrete geospatial datasets, beyond Euclidean settings; ii) to suggest graph representations of a wide family of models in spatial statistics, providing an alternative approach to existing finite element and finite difference representations; iii) to set forward the theoretical foundations of graph representations of Gaussian fields and investigate their use in specific applications; and iv) to develop and numerically test new black-box optimization schemes, exploring their use in a variety of applications. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.