Simulation-based Inference through Random Features

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

Scientists use simulations to model all kinds of complex systems, from astronomy and ecology, through climatology and finance, to epidemiology and chemical engineering. Once scientists pin down all the parameters in a simulation model, it's easy to run it forward and see what it predicts. Going the other way, from outcomes back to parameters, is usually much harder, but it's the crucial step in fitting models to actual data from the real world, and so knowing which simulations are trustworthy. Good simulation modelers put lots of time and effort into working out what the crucial aspects or "features" of the data are for their models, and then adjusting their models so the simulation output matches those features of real data. This project aims to make such simulation-based inference nearly automatic. Bringing together ideas machine learning and nonlinear dynamics shows that simulation models can be fit by matching about two randomly-chosen features for every parameter. The project will make this into a practical and generic tool for simulation, by adapting the basic method to work with different types of data (time series, spatial, networks, etc.), writing software to automatically pick the features and adjust the model parameters, and developing statistical methods to quantify the uncertainty of the results. This can benefit every area of science and technology that uses simulations. The project will contribute to the training of STEM researchers through the involvement of a post-doctoral fellow and a graduate student. Scientists increasingly express their ideas in generative models which produce fine-grained simulations of the processes they study. Traditional statistical approaches to parameter inference are ill-adapted to such models: just evaluating the likelihood function is usually computationally intractable, never mind optimizing it. Many techniques of simulation-based inference have developed in response, but these typically require picking multiple summary statistics or features, and tuning the generative model's parameters so that summaries calculated on simulations match those calculated on empirical data. Practitioners often spend considerable effort on carefully designing these summaries, aiming to minimize the loss of information from the full data. This project will free simulation-based inference from the need to design informative summaries, by instead using random functions of the data. This draws on two literatures not previously connected to simulation-based inference. One is work in machine learning over the last decade, which has highlighted the power of "random features", showing that predictions based on random functions of high-dimensional data are nearly as good as predictions based on optimal functions of the data. The other is now-classic work in nonlinear dynamics from the 1980s and 1990s, which suggests that 2d+1 random features should suffice to capture d underlying parameters. Bringing these ideas together with the well-established results on simulation-based inference will provide a simple, practical methodology of parameter estimation, uncertainty quantification and hypothesis testing, applicable to a wide range of modern simulation models. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.