Further advances in stability analysis for hybrid adversarial Markov decision processes

University Of California-Santa Barbara, Santa Barbara CA

Investigators

Abstract

Intellectual Merit: The objective of this research is to combine advances in the analysis of non-stochastic hybrid dynamical systems and adversarial Markov decision processes to generate new results for a broad class of hybrid stochastic systems. The salient features of hybrid stochastic systems are that 1) the models combine stochastic and worst-case effects to generate non-unique random processes and 2) solutions combine continuous change with instantaneous change. Most work on hybrid stochastic systems has focused on systems with unique solutions. Nevertheless, the combination of worst-case and stochastic effects is natural and manifests itself in a wide class of engineered systems, including air traffic management systems, financial systems, and networked and logic-based control systems. In recent years, important progress has been made on characterizing robust asymptotic stability for non-stochastic hybrid systems as well as for adversarial Markov decision processes. General invariance principles, converse Lyapunov theorems and relaxed Lyapunov-based sufficient conditions for stability have been codified. In addition, powerful reduction principles and linearization-based results have been extended to hybrid systems. The aim of the current work is to develop analogous results for hybrid adversarial Markov decision processes, which will help pave the way for advances in robust control design for stochastic hybrid systems. Broader Impacts:The proposed developments will enhance the engineering education base by furthering the available graduate curriculum on stochastic and hybrid systems, and by training graduate students. Tutorial workshops at the main control conferences will be made available to graduate students, researchers, and industry personnel. All technical results will be documented in acclaimed international conferences and prestigious systems and control journals. The work will foster collaboration with international researchers, continuing student exchange programs with several international universities. The progress of the proposed work will be applied to areas that are important to the broad population, for example the efficient management of air traffic to enhance throughput in the nation?s aviation system. In addition, early educational opportunities, down to the freshman level via freshman seminars, will be pursued to capture the attention of students early in the curriculum through popular applications of optimization and dynamics.