Control of Markov Processes Subject to Qualitative Constraints

University Of Texas At Austin, Austin TX

Investigators

Abstract

Discrete event systems (DESs) are systems evolving according to the occurrence of certain discrete qualitative changes, called events. Examples of events include the arrival of a customer in a queue, the termination of an algorithm in a computer program, the loss of a message packet in a communication network, the breakdown of a machine in a manufacturing system. In this proposal we consider stochastic DESs modeled by Markov processes, and study control problems that satisfy qualitative properties. We outline a program of work aiming to develop a unified framework for qualitative control of stochastic DESs. In particular, we propose to study the control of stochastic systems with general qualitative constraints such as safety, non-blocking, recurrence, and stability; optimal control subject to qualitative constraints; control under complete or partial observations; effects of using deterministic versus randomized policies; and effects of using stationary versus non-stationary and asymptotically stationary control policies. Safety constraints are specified as unit-interval-valued vectors serving as an upper bound for the state probability distribution of the controlled Markov chain. Non-blocking specifications require that the probability of hitting a target set of states stays above a certain minimum value. Recurrence or liveness amounts to the probability of hitting a target set of states infinitely-often being bounded below by a positive constant, while convergence or stability demands that the state probability distribution enters and stays in a `safe' set within a finite number of steps.