PARTIALLY OBSERVABLE MARKOV PROCESSES.
Abstract
A partially observable Markov process is a model of a discrete time dynamic system which takes into account the effects of imperfect observations and of random system be havior. The model consists of an underlying Markov process with state vector X(n). Direct observations of X(n) are not possible, but a vector Z(n) is observed. The observation Z(n) is related to the state X(n) by a known probability density function. This model is useful in the analysis of a very large class of sequential decision problems. It was shown that a partially observable Markov process is conveniently analyzed by the introduction of the probability density function. This density function was shown to have certain characteristic iterative properties and is referred to as the statistical state of the system. The application of the theory of partially observable Markov processes to the problems of estimation, prediction and smoothing is straightforward. When a general terminal control problem is considered, however, the notion of minimum expected cost turns out to be ambiguous. The concepts of a priori and a posteriori control were introduced to reslove this confusion. (Author)
Document Details
- Document Type
- Technical Report
- Publication Date
- Apr 01, 1964
- Accession Number
- AD0601150
Entities
People
- J. David R. Kramer Jr.
Organizations
- Massachusetts Institute of Technology