Scaling Limits for some Stochastic Control Problems with Applications to Stochastic Networks

University Of North Carolina At Chapel Hill, Chapel Hill NC

Investigators

Abstract

This research project considers scaling limits of certain controlled stochastic processing networks in heavy traffic. Brownian control problems(BCP) have been proposed as formal diffusion approximations for a broad range of controlled networks.Currently there is a critical lack of general theory that establishes rigorous connections between a network control problem and its associated BCP. The first goal of this research is to establish that for a wide range of control forms, network structures and optimization criteria, the value functions of suitably scaled controlled network models converge to that of the corresponding diffusion control problem. The second goal is the study of qualitative properties of diffusion control problems arising from the above asymptotic analysis. These problems correspond to a family of singular control problems with state constraints in non-smooth domains. The Hamilton-Jacobi-Bellman (HJB) equations for such control problems are a challenging class of degenerate elliptic nonlinear partial differential equations with gradient constraints and somewhat non-standard boundary conditions. Existence, uniqueness and regularity theory for HJB equations for a range of such control problems and cost criteria will be developed. Wellposedness of such equations is a central ingredient in development of numerical schemes for obtaining near optimal controls. Additionally, regularity is key in reading off useful qualitative information on the form of an optimal control. An example of such information is the characterization of an optimally controlled process as a reflected diffusion over a domain determined in terms of a suitable free boundary problem. Such characterization results for singular control problems are some of the most elegant and useful results in the field and their study for problems with state constraints will be a focus of this research. Dynamic control problems that motivate this work arise from a wide range of application areas, such as, telecommunications, manufacturing, service engineering, computing, etc. Control in such networks can take a variety of forms, examples include, scheduling, sequencing, routing and admissions of jobs, and input and processing rate controls. Networks of interest are in general quite complex and thus one seeks tractable approximate models. The overall theme of this research is the development of techniques for obtaining good scheduling policies for general families of such stochastic processing systems using the mathematical theory of diffusion approximations. The work will lead to improved design, stability and regulation of complex manufacturing, communication and computer systems. Research project will support the training of two graduate students and develop international collaborations and with faculty from non-Ph.D. granting institutions.