Collaborative Research: Risk-Averse Control of Markov Systems with Model Uncertainty

Illinois Institute Of Technology, Chicago IL

Investigators

Abstract

This project focuses on mathematical theory and computational methods of decision-making in systems that evolve randomly in time and whose essential characteristics are not precisely known to the observer. The research will address in a coherent way how to model risk in such systems and how to control them within the risk-averse paradigm. This will be accomplished by developing dynamic risk-assessment procedures, called risk filters, and by employing adaptive robust control techniques. The outcome of the project will directly advance and promote the progress of science and engineering, with potential applications in applied areas such as medical sciences, engineering, economics, finance, inventory management and insurance. Special attention will be given to popularizing the proposed research and its impact in these applied fields. In particular, this will be achieved through advising of graduate and undergraduate students, including students from underrepresented groups, presentations at popular, international and local forums, and dissemination of the results via scientific journal and book publications. The classical theory and practice of Markov decision processes have proven to provide a powerful and successful toolkit for generating optimal or sub-optimal decision strategies in situations where the decision maker has access to adequately known (accurate) model of the underlying Markovian dynamical system, and acts so to optimize the expected cumulative cost or reward arising from the decision maker's actions. However, on the one hand, in many decision-making processes the decision maker needs to account for the trade-off between the cumulative award and cumulative risk of the decision. Risk-averse decision criteria underlying this research project and the theory of risk filters are ideally suited for such purposes. On the other hand, it is a typical situation in decision making processes that the model of the underlying Markovian dynamical system is not known exactly. Frequently, such model is a semi-adequate formalization of the underlying Markovian system, in the sense that the structural dynamical features of the system are modeled adequately, but precise knowledge of relevant model parameters is missing. In such cases, we say that the decision maker faces model uncertainty. Part of the proposed research will be devoted to develop methodologies that address this issue through adaptive robust stochastic control framework. Thus, the proposed research addresses in a coherent and novel way two important aspects of decision making in Markov systems: risk-averse decision criteria and model uncertainty. The theory of risk filters will be combined with the adaptive robust control methodology that will lead to novel dynamic programming equations, for which new numerical methods will be established. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.