Developments in Gaussian Processes and Beyond: Applications in Geostatistics and Deep Learning

Purdue University, West Lafayette IN

Investigators

Abstract

Gaussian processes have diverse applications in statistics and machine learning and are of great contemporary interest. To give a few examples, they arise in the modeling of spatial data, computer experiments, and in studying the limits of deep neural networks. Key reasons for the appeal of Gaussian processes include their simplicity and wide tractability: the entire process is characterized by just the mean and the covariance functions. Yet, although Gaussian processes are popular with well-developed theoretical and computational properties, there are some distinct limitations in using them. Moreover, there are several situations where Gaussian processes are inappropriate as a modeling choice. New methodology will be developed to address some of these limitations, with wide-ranging implications from spatial statistics to deep learning. Publicly available software development, student mentoring, and broad dissemination of research will have impacts beyond the particular research problems at hand. Key areas of the technical investigation are as follows. The first issue concerns the use of the ubiquitous Matern covariance function. A key benefit of the Matern family is the precise control over the smoothness of the resultant Gaussian processes (GP) realizations. However, the tails of the Matern covariance decay exponentially fast, which is inappropriate in the presence of polynomial dependence. Polynomial covariances such as Cauchy remedy this issue, but at the expense of a loss of control over smoothness, in that, GP realizations using Cauchy covariances are either infinitely differentiable or not at all. The PI will develop a new covariance function that combines the flexibility of the Matern and polynomial covariances. Next, the PI will study the limiting behavior of deep neural networks under global-local horseshoe regularization priors on the weights. The lack of bounded moments necessitates the construction of a new Levy process that can be used to study the limits of neural networks under such priors, thereby aiding uncertainty quantification. The PI will study the theoretical and computational properties of the resultant process. Finally, the PI will use recently developed global-local shrinkage approaches for Bayesian regularization in GP regression, with distinct improvements upon existing methods. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.