Countable State Continuous Time Dynamic Programming with Structure
Abstract
The problem P of maximizing the expected discounted reward earned in a continuous time Markov decision process with countable state and finite action space is considered. (The reward rate is merely bounded by a polynomial). By examining a sequence <P sub N> of approximating problems, each of which is a semi-Markov decision process with exponential transition rate Lambda sub N, Lambda sub N nears infinity, the author is able to show that P is obtained as the limit of the P sub N. The value in representing P as the limit of P sub N is that structural properties present in each P sub N persist, both in the finite and in the infinite horizon problem. Three queueing optimization models illustrating the method are given.
Document Details
- Document Type
- Technical Report
- Publication Date
- Apr 01, 1974
- Accession Number
- AD0782266
Entities
People
- Steven A. Lippman
Organizations
- University of California, Los Angeles