Transient Analysis and Applications of Markov Reward Processes
Abstract
In this thesis, the problem of computing the cumulative distribution function (cdf) of the random time required for a system to first reach a specified reward threshold when the rate at which the reward accrues is controlled by a continuous time stochastic process is considered. This random time is a type of first passage time for the cumulative reward process. The major contribution of this work is a simplified, analytical expression for the Laplace-Stieltjes Transform of the cdf in one dimension rather than two. The result is obtained using two techniques: i) by converting an existing partial differential equation to an ordinary differential equation with a known solution, and ii) by inverting an existing two-dimensional result with respect to one of the dimensions. The results are applied to a variety of real-world operational problems using one-dimensional numerical Laplace inversion techniques and compared to solutions obtained from numerical inversion of a two-dimensional transform, as well as those from Monte-Carlo simulation. Inverting one-dimensional transforms is computationally more expedient than inverting two-dimensional transforms, particularly as the number of states in the governing Markov process increases. The numerical results demonstrate the accuracy with which the one-dimensional result approximates the first passage time probabilities in a comparatively negligible amount of the time.
Document Details
- Document Type
- Technical Report
- Publication Date
- Mar 01, 2003
- Accession Number
- ADA412909
Entities
People
- Jeffrey A. Sipe
Organizations
- Air Force Institute of Technology