Undiscounted Markov Renewal Programming via Modified Successive Approximations.
Abstract
An efficient class of procedures is described for finding a solution to the functional equations of undiscounted Markov renewal programming. First, for the special case of a single possible policy, the problem is proved equivalent to solving two related ordinary Markov chain problems. This leads to an algorithm for the general problem whose exact form depends on the specification of a decision rule for alternation of two types of iterations. At one extreme the technique is exactly 'policy iteration' with iterative techniques replacing solution of N equations for each improved policy. At the other extreme, the algorithm becomes essentially value iteration, generalizing the method of successive approximations proposed by D. J. White for Markovian decision processes. The latter version of the technique is related to another generalization being currently proposed by Paul J. Schweitzer; the methods being proposed here, however, do not deteriorate when the minimum transition time between states becomes very small. (Author)
Document Details
- Document Type
- Technical Report
- Publication Date
- Jun 01, 1970
- Accession Number
- AD0720299
Entities
People
- Thomas E. Morton
Organizations
- Carnegie Mellon University