New algorithms for Bayesian Computation

Stanford University, Stanford CA

Investigators

Abstract

Bayesian statistics is a highly principled approach to learn about unknown parameters and variables based on observed data. In this approach, existing knowledge about the unknown parameters is represented by the prior distribution. Once new data has been observed, then the prior distribution is updated by the Bayes formula to produce the posterior distribution which represents the updated knowledge about the parameter. Detailed information about the parameters of interest is usually obtained from the posterior distribution through computational inference methods such as Markov Chain Monte Carlo or Importance Sampling. However, these computational inference methods can become inefficient in some situations, such as when the likelihood function is too expensive to evaluate, or when the statistical model is given as a generative model without an explicit likelihood and can only be used to simulate the data. The goal of this project is to develop approaches to Bayesian inference that remain computationally efficient in these situations. The results will enable wider use of Bayesian methods in many areas of science and technology. The project will also contribute to the training of graduate students through their involvement in the performance of the research. Specifically, this project will create new computational tools to address two issues that are challenging for current algorithms, namely, how to sample from the posterior distribution in Hidden Markov Models with continuous variables, and how to design sequential methods for simulation from the posterior distribution even when the likelihood function is not available. Hidden Markov Models are widely used in the engineering and biological sciences, but currently algorithms for Bayesian inference in this model are available only if the variables involved are discrete variables. By creating efficient algorithms for the continuous case, the results of this project will enable engineers and computational biologists to apply these models to a much wider range of problems. The second goal of this project is to develop new tools for approximate Bayesian computation in models with intractable or unknown likelihood functions. These models can arise from many scientific areas such as phylogenetics and computer-based experiments. Currently there is only one available algorithm (the ABC algorithm) for Bayesian inference on this type of models. By developing an extension that can greatly improve the computational efficiency of this algorithm, the research in this project will benefit the aforementioned scientific areas. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.