Numerical Methods in Stochastic Control
Abstract
The second edition of our book 4 on numerical methods in stochastic control has appeared. The book and the methods contained therein are now the standard in the field. It contains the most comprehensive development of numerical algorithms and associated convergence proofs for a large part of the current forms of stochastic control problems in continuous time. The PI's algorithms (and proof techniques) are the algorithms of choice for the bulk of continuous time stochastic control problems. In addition to the broad coverage of the first edition, it gives numerical algorithms and proofs for problems where the variance term is controlled, and for jump-diffusions where the jump is controlled. Important applications of jump control occur, for example, in communications theory. Consider, for example, a system where a server divides its time between several queues whose input processes are bursty, and the individual connections are subject to random breakdown or fading. The control problem is the scheduling of the server and this must be done continuously. A jump increase in the total system workload can occur when some connection breaks down or fades and the work in the available queues is less than the server can handle, but customers continue to arrive at the unavailable queues, so there is undesired idle time. The control policy affects the jump sizes. Traditional methods cannot handle such problems. The standard use of the Poisson measure driven model is no longer adequate, and a general theory is developed. Additionally, the book contains a thorough development of deterministic problems that arise in control and in the calculus of variations, and includes discontinuous or unbounded dynamical terms, with applications to image reconstruction, large deviations, and elsewhere. The algorithms are about the fastest and most stable available, and there are convergence proofs for all of them.
Document Details
- Document Type
- Technical Report
- Publication Date
- Jul 24, 2002
- Accession Number
- ADA414605
Entities
People
- Harold Kushner
- Paul Dupuis
Organizations
- Brown University