New Methodologies for Markov Decision Processes and Stochastic Games Motivated by Inventory Control

Suny At Stony Brook, Stony Brook NY

Investigators

Abstract

Inventory control is broadly used in production and service systems and in supply chains to manage operations and improve efficiency and reliability. The analysis and optimization of several important classes of inventory control problems relies on the theory of Markov decision processes, an area of operations research dealing with sequential optimization of stochastic systems. The two major research directions in the theory of Markov decision processes are: (i) to establish the structure of optimal and approximately optimal decisions, and (ii) to develop algorithms for their computation. This project will develop new methodologies for Markov decision processes, including models with incomplete information and risk-sensitive criteria, and for stochastic games. Although motivated by inventory control problems, potential applications of this project's methodological advances include many application areas, in particular to the control of electric storage for power systems. The project will also contribute to the development of human resources in science and engineering. First, it will support Ph.D. students at Stony Brook University including female students. Second, it will create research and educational projects for graduate and undergraduate students including students from underrepresented minority groups. This project will advance solution methodologies for two groups of decision making models: Markov decision processes, including partially observable Markov decision processes, and stochastic games. The initial motivation for solving such problems is inspired by inventory control applications, and this project will also advance the inventory control theory. For Markov decision processes and partially observable Markov decision processes, the project will investigate discounted total cost and average cost objectives. It will also develop methodologies for decision making under risk, robust optimization, incomplete state information, and incomplete knowledge of model parameters. Specifically, the project will establish new results on the validity of optimality equations and inequalities and the structure of optimal policies. It will develop algorithms and investigate their convergence and complexity for problems with classic and nonstandard criteria. For games the project will develop solution methodologies for one-step and sequential stochastic problems with complete and incomplete state observations with possibly unbounded payoffs.