Constrained Optimization of Markov Decision Processes

Suny At Stony Brook, Stony Brook NY

Investigators

Abstract

The main research objectives of this project are to develop new algorithms and analytical tools for optimization and analysis of broad classes of controlled stochastic systems, known under the name of Markov Decision Processes, when the system performance is characterized by multiple criteria. The objective is to optimize one of the criteria under constraints on other criteria. This project will study models with the criteria of the expected total costs and the expected costs per unit time. It will be focused on finding optimal nonrandomized policies for problems with large state spaces, on developing algorithms for multi-chain problems, and on solving adaptive control problems with unknown parameters. In addition to general Markov Decision Processes, three groups of particular problems will be studied: the so-called multi-armed bandit problems that model many important managerial and operations research problems, call admission, and inventory control. If successful, the results of this project will provide efficient algorithms for computing optimal policies for important classes of stochastic systems. They will also provide descriptions of the structure of optimal policies for several groups of mathematical models of production and service systems. Currently, efficient algorithms and structural results are primarily known for problems with single objective functions. However, real-life applications usually deal with multiple criteria. This project will develop mathematical and engineering tools to control large stochastic systems for which traditional computational methods are intractable. In particular, multi-armed bandit problems to be investigated in this project have important applications to scheduling, pharmaceutical research, project management, and economics.