NON-DISCOUNTED DENUMERABLE MARKOVIAN DECISION MODELS

Abstract

Countable state, finite action Markovian decision processes are studied under the average cost criterion. The problem is studied by using the known results for the discounted-cost problem. Sufficient conditions are given for the existence of an optimal rule which is of the stationary deterministic type. This rule is shown to be, in some sense, a limit point of the optimal discounted-cost rules. Sufficient conditions are also given for the optimal discounted-cost rules to be epsilon-optimal with respect to the average cost criterion. It is shown that if there is a replacement action then there exists an optimal rule but it may not be of the stationary deterministic type. It is also shown how, in a special case, the average cost criterion can be reduced to the discounted cost criterion. Lastly, an example is given of a process for which there exists an optimal nonstationary rule which is better than any stationary rule.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
May 02, 1967
Accession Number
AD0652797

Entities

People

  • Sheldon M. Ross

Organizations

  • Stanford University

Tags

Communities of Interest

  • C4I
  • Materials and Manufacturing Processes

DTIC Thesaurus Topics

  • Abstracts
  • Air Force
  • Classification
  • Contracts
  • Convergence
  • Dynamic Programming
  • Equations
  • Governments
  • Markov Chains
  • Military Research
  • Probability
  • Security
  • Sequences
  • Stationary
  • Transitions
  • United States
  • United States Government

Fields of Study

  • Mathematics

Readers

  • Mathematical Modeling and Probability Theory.