New Methods for Studying the Time-Dependent and Steady-State Behavior of Markov Chains

Clemson University, Clemson SC

Investigators

Abstract

Markov chains are mathematical objects used to model random phenomena, such as customer flow within a call center, traffic flow on a highway, as well as protein production within a cell. This project focuses on the development of new methods towards studying, for such a phenomenon, both its long-run behavior, as well as its behavior over moderate time frames. If successful, these methods will provide new insight into the behavior of systems that can be modeled with Markov chains, and could lead to improved methods toward managing such systems. For example, such information could be used to set staffing levels in a call center: a study of its long-run behavior may suffice when customer arrivals are relatively stable over time, but a study over shorter time scales is needed if the arrival rate fluctuates too often. This research plan will provide training for graduate students to prepare them for research-based careers in academia, government or industry, and the results of this study will be published in the appropriate scholarly journals, and incorporated into graduate courses taught by the PI. The methods developed within this project will build on new "random-product representations" recently discovered by the PI, and will be used to analyze both the stationary distribution, as well as the time-dependent distributions of a Markov chain. Examples of the types of chains that will be studied include hysteretic reflected Brownian motion, various types of two-dimensional random walks on the nonnegative quarter-plane, Markovian queueing systems under the influence of an external Markovian environment and other `matrix-geometric' models, as well as other types of Markovian queueing networks not included within the above-mentioned types of chains. The results of this study should lead towards understanding what characteristics of a Markov chain lead to stationary distributions having a "product-form-like" structure. Having such insight should prove useful, as such a distribution often yields analytically tractable performance measures that aid in further understanding the behavior of the underlying Markov chain. The techniques used in this study should draw from many concepts found within the theory of random walks, and the theory of Laplace transforms should play a large role in the study of the time-dependent behavior of the above-mentioned types of Markov chains.