MATHEMATICAL PROGRAMMING AND THE CONTROL OF MARKOV CHAINS,

Abstract

Linear programming versions of some control problems on Markov chains are derived, and are studied under conditions which occur in typical problems which arise by discretizing continuous time and state systems, or in discrete state systems. Control interpretations of the dual variables and simplex multipliers are given. The formulation allows the treatment of 'state space' like constraints which cannot be handled conveniently with dynamic programming. The relation between dynamic programming on Markov chains, and the deterministic discrete maximum is explored, and some insight is obtained into the problem of singular stochastic controls (with respect to a stochastic maximum principle). (Author)

Document Details

Document Type
Technical Report
Publication Date
May 01, 1970
Accession Number
AD0706910

Entities

People

  • A. J. Kleinman
  • Harold J. Kushner

Organizations

  • Brown University

Tags

DTIC Thesaurus Topics

  • Applied Mathematics
  • Computer Programming
  • Dynamic Programming
  • Interdisciplinary Science
  • Linear Programming
  • Markov Chains
  • Mathematical Programming
  • Mathematics
  • Operations Research
  • Stochastic Control
  • Systems Science

Fields of Study

  • Mathematics

Readers

  • Mathematical Modeling and Probability Theory.
  • Operations Research

Technology Areas

  • Space
  • Space - Spacecraft Maneuvers