RI: Small: Scalable Online Learning with Gaussian Processes

New York University, New York NY

Investigators

Abstract

The modern world is filled with highly complex systems interacting to transport goods and raw materials, to manufacture the tiny components that power phones and laptops, and to assist surgeons in delicate medical procedures. Every day, the operators of these systems must make general decisions like how to schedule workers and deliveries, and specific decisions like how a certain robot in an assembly line should function. In each case, a good decision must account not only for what is known about the environment, but also what is unknown. Sometimes more information should be gathered, and sometimes action must be taken to avoid unlikely but costly mistakes. Moreover, every decision affects the next, and errors and delays in judgement at each step can be propagated and amplified. Scientists rely heavily on computer models to control for unknowns when making decisions, but in many situations the models are simply too slow to be useful. This research will greatly reduce the computational requirements needed for a robust representation of uncertainty, meaning computer models can quantify the effect of uncertainty more quickly and reliably, at a lower cost. In a world where unknowns are carefully modeled, autonomous vehicles are safer, infrastructure is more efficient, and scientific experiments are more informative. Gaussian processes are a gold standard for uncertainty representation. However, the high computational cost of making predictions, after training, has limited their applicability in the sequential decision making frameworks for Bayesian optimization and reinforcement learning, where the quality of uncertainty estimates can have enormous impact. This research develops algebraic methods that exploit advances in hardware design for scalable Gaussian processes in these settings. This work will broaden the applicability of Bayesian optimization methods to general purpose objectives, with crucial scientific impacts such as automating NMR spectroscopy. This research will also enable more realistic assumptions in model-based reinforcement learning, to capture many possible future states of an engineering system, efficient exploration of possible states, and representation of high dimensional state spaces. These features are an important step towards automatic control in complicated engineering systems, such as unmanned vehicles, where data is costly to acquire and safety guarantees are critical. Overall this work will help unlock the potential of probabilistic methods for sequential online decision making, while providing interactive engineering demonstrations in educational settings. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.