Markov Decision Chains.
Abstract
The report is a self contained expository development of policy improvement methods for finding optimal policies under various criteria in Markov decision chains. A (finite) Markov decision chain is a generalization of a finite Markov chain with a distinguished set of stopped states such that whenever the chain is observed in a given state, a decision maker chooses one of finitely many transition probability vectors available in that state and earns a reward depending on the given state and probability vectorchosen. No background in the subject is required, but a knowledge of elementary properties of matrices, infinite series, and finite Markov chains is assumed. The nature of Markov decision chains and their applications to gambling, search, sequential statistical decisions, and inventory control are discussed. (Author Modified Abstract)
Document Details
- Document Type
- Technical Report
- Publication Date
- Feb 20, 1973
- Accession Number
- AD0758652
Entities
People
- Arthur F. Veinott Jr.
Organizations
- Stanford University