Analysis of an Adaptive Control Scheme for a Partially Observed Controlled Markov Chain
Abstract
The authors consider an adaptive finite state-controlled Markov chain with partial state information, motivated by a class of replacement problems. They present parameter estimation techniques based on the information available after actions that reset the state to a known value are taken. They prove that the parameter estimates converge w.p.1 to the true (unknown) parameter, under the feedback structure induced by a certainty equivalent adaptive policy. They also show that the adaptive policy is self-optimizing, in a long-run average sense, for any (measurable) sequence of parameter estimates converging w.p.1 to the true parameter.
Document Details
- Document Type
- Technical Report
- Publication Date
- Dec 01, 1991
- Accession Number
- ADA454807
Entities
People
- Aristotle Arapostathis
- Emmanuel Fernandez-gaucherand
- Steven I Marcus
Organizations
- University of Maryland