Ergodic Control of Large-Scale Stochastic Networks
Abstract
Stochastic networks with multiple classes of jobs and multiple parallel server pools arise in a variety of application contexts in data centers, telecommunications, manufacturing, service systems and military logistics and deployment. Scheduling and routing decisions are the most important and challenging tasks to achieve cost-effective, high-quality and reliable system performance. The class of stochastic networks we study is at the frontier of the research subject. Despite the active studies in the past decade, stochastic stability properties of these networks in the Half-Whitt (H-W) regime are only understood to a very limited extent. Since stochastic stability is usually regarded as a prerequisite to study ergodic control problems of queueing networks, ergodic control of these networks in the H-W regime has remained an open problem. Existing methods of ergodic control of diffusions cannot be applied to this class of stochastic networks. This project aims to develop new mathematical tools for ergodic control of limiting processes in diffusion scale in order to address ergodic control problems for a large class of stochastic networks to which existing theory cannot be applied. This research has the following four objectives: (1) To develop a new theoretical framework to study ergodic control of a broad class of diffusions which arise in multiclass multi-pool stochastic networks in the H-W regime, as well as new methods to establish the asymptotic optimality of control policies derived from these limiting diffusions. Existing methods for nonlinear ergodic diffusion control problems assume either that the running cost is near-monotone, or that the controlled diffusion is uniformly stable. Unfortunately, the problems encountered in multiclass networks in the H-W regime do not fall into any one of these two categories. A new structural assumption has been identified in our preliminary work for such networks, which results in a broad class of ergodic diffusion control models which subsumes the near-monotone and uniformly stable frameworks. (2) To develop new stochastic control theory of diffusions with Levy-type jumps arising from multiclass multi-pool networks in a dynamic random environment (especially with service interruptions) in the H-W regime. Ergodic control of queueing networks in dynamic random environments in the H-W regime is an unexplored territory. The associated controlled diffusions in the limit may have jumps, as Levy-type SDEs. The convergence requires unusual Skorohod topologies. Ergodic control of Levy-type SDE has hardly been studied, and existing relevant work is not adequate to deal with the problem at hand. The obstacle lies in the associated semigroup with non-local generators on the entire space. (3) To develop new ergodic control theory for diffusions with time-periodic input data arising from stochastic networks with time-varying demand and capacity allocations in the H-W regime. Many stochastic systems have time-periodic data. Ergodic control of diffusions or diffusions with jumps under time-periodic parameters is also an unexplored territory. The main difficulties lie in developing a suitable convex analytic framework, and understanding the structure of time-periodic invariant measures. (4) To develop relative value iteration algorithms for the classes of ergodic control problems above in order to numerically solve the associated HJB equations, and also to design diffusion-based learning schemes for adaptive control in the presence of unknown parameters.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Feb 14, 2019
- Source ID
- W911NF1710019
Entities
People
- Guodong Pang
Organizations
- Army Contracting Command
- Pennsylvania State University
- United States Army