SEMI-MARKOVIAN CONTROL SYSTEMS,
Abstract
A discussion is presented of a statistical model useful in analyzing a wide variety of common problems, problems in the areas of maintenance, replacement, marketing, finance, and inventory control. The basis for the present model is a decision model developed for strictly Markov processes, characterized as processes in which the main interest is on state transitions rather than on the time required for a transition. Other types of transition behavior was encouraged by the development of the semi-Markov process which allowed the time between transitions to be a random variable conditional on the transition made.A semi-Markov process has the same limiting state probabilities as an exponential Markov process with the same mean times for transitions, so the decision model can be extended to the semi-Markov process. Some of the results of this development are pointed out and it is shown how this increase in generality does not exact a computational penalty. (Author)
Document Details
- Document Type
- Technical Report
- Publication Date
- Dec 01, 1963
- Accession Number
- AD0434242
Entities
People
- Ronald A. Howard
Organizations
- Massachusetts Institute of Technology