Optimality and Robustness in Piecewise-Deterministic Systems

Cornell University, Ithaca NY

Investigators

Abstract

This project focuses on quantifying and actively managing uncertainties resulting from random switches in global environments. An abrupt ecological change, a new disruptive technology, a global economic downturn, or an emerging pandemic – any such game-changer event may transform the planning environment and shift the priorities of decision makers. Reactive planning is often the norm in practice, with the working assumption that global mode switches are too rare and unlikely to take them into account. But if a statistical characterization of such switches is available, using it in strategic planning can significantly improve the performance of the controlled system. Models with these features arise naturally in many research areas including economics, behavioral ecology, public policy, robotic navigation, evolutionary biology, and security applications (e.g., preventing illegal logging or wildlife poaching). PI will develop efficient numerical methods for controlling such systems, focusing on trade-offs between the average-case optimality and robustness (interpreted as minimizing the probability of undesirable outcomes.) This project will support 2 graduate students in each of the first two years and 1 graduate student in the third year. Piecewise-Deterministic Markov Processes (PDMPs) provide an excellent framework for modeling large-scale stochastic perturbations of the global environment. The aleatoric uncertainty due to such perturbations is an important feature of realistic control problems, but until recently it has attracted far less attention in mathematical literature than the diffusive perturbations studied in ``classical'' stochastic optimal control theory. Practitioners often want to model these environment-switch uncertainties as well as time-structured information accumulation patterns present in their applications. Moreover, it may not be enough for them to optimize the expected value of the outcome. They often need to maximize the probability of good outcomes while imposing constraints on the worst-case scenario. To accomplish this, we need to modify the partial differential equations (PDEs) encoding the optimal behavior, and this presents a range of new computational challenges: free boundaries, discontinuities, higher dimensionality of the state space, and larger systems of coupled nonlinear PDEs. We propose to study the trade-offs involved in using such modified models and to develop numerical methods to solve them efficiently. In the PDMP setting, even the traditional risk-neutral approach of optimizing the expected performance can be computationally costly since it involves solving a coupled system of Hamilton-Jacobi-Bellman (HJB) equations. We develop several approaches for decreasing this computational cost by constructing new discretization schemes and leveraging efficient methods previously developed for fully deterministic problems. We also extend our recent approach for optimizing the Cumulative Distribution Function (CDF) of the total cost incurred by a stochastic switching system. This is accomplished by solving a different system of HJB equations on an expanded state space, with ``threshold-optimal'' controls recovered for all starting configurations and all threshold values simultaneously. We further investigate the trade-offs between conflicting optimization criteria and several notions of robustness. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.