Comments on the Sensitivity of the Optimal Cost and the Optimal Policy for a Discrete Markov Decision Process

Abstract

The problem of characterizing the effects that uncertainties and/or small changes in the parameters of a model can have an optimal policies is considered. It is shown that changes in the optimal policy are very difficult to detect even for relatively simple models. By showing for a machine replacement problem modeled by a partially observed, finite state Markov decision process, that the infinite horizon, optimal discounted cost function is piecewise linear, we find formulas to compute the optimal cost and the optimal policy, thus providing a means for carrying out sensitivity analyses. Examples are presented to show the usefulness of the results. Keywords: Algorithms; Stochastic control; Dynamic programming.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Sep 01, 1989
Accession Number
ADA219074

Entities

People

  • Enrique L. Sernik
  • S. I. Marcus

Organizations

  • University of Texas at Austin

Tags

Communities of Interest

  • Materials and Manufacturing Processes

DTIC Thesaurus Topics

  • Air Force
  • Algorithms
  • Classification
  • Computer Programming
  • Computers
  • Dynamic Programming
  • Engineering
  • Equations
  • Intervals
  • Iterations
  • Observation
  • Probability
  • Production
  • Security
  • Sensitivity
  • Structural Properties
  • Uncertainty

Readers

  • Computational Modeling and Simulation
  • Operations Research