CONTROLS LEADING TO OPTIMAL STATIONARY REGIMES,
Abstract
The authors define a controlled Markov chain as one for which the passage probabilities are defined by P(xi sub (n + 1) = X sub (n + 1) (vertical line) xi sub n, d sub n); n = 0,1,... where d sub n is an element which may be chosen from a space D as a function of x sub 0,...,x sub n. A choice of d sub n(x sub 0,...,x sub n) for each n = or > 0 is called a control delta; and it is noted that the process (xi, delta), xi = (xi sub 1, xi sub 2,...), delta = (d sub 1, d sub 2,...) is generally not really Markovian (since the further past has too much influence). The control delta is called Markovian if each of the functions d sub n depends only on x sub n, and homogeneous Markovian if in addition, all d sub n are the same. The authors define admissible controls; they define optimal controls in terms of a loss function W(x, d) where x is the state of the system and d the chosen control. They answer the question of existence of optimal controls and optimal homogeneous Markov controls for the very special case of finite state and control spaces.
Document Details
- Document Type
- Technical Report
- Publication Date
- Sep 19, 1967
- Accession Number
- AD0666741
Entities
People
- A. N. Shiryaev
- O. V. Viskov
Organizations
- National Air and Space Intelligence Center