Separation of Nonlinearity and Stochasticity
Abstract
Over 60 years ago, major theoretical results in the control and optimization areas developed the link between the solution of continuous time/space, nonlinear control problems and the solution of associated Hamilton-Jacobi partial differential equations (HJ PDEs) and/or the solution of associated two-point boundary value problems (TPBVPs), including those of the Pontryagin maximum principle. Conversion of the control problem to one of these alternatives was expected to lead to solution of the originating nonlinear control problem. However, it was soon learned that the computational complexity of these formulations was typically prohibitive. The HJ PDE formulation is the more generally applicable of these approaches, particularly in the case of stochastic control problems. However, the application of standard solution methods to these HJ PDE problems is subject to the curse of dimensionality, which limits their successful application to only exceptionally low-dimensional problems. Perhaps the earliest, successful approach to addressing this issue was the max-plus based curse-of-dimensionality-free (CODF) method. The effectiveness of that approach has been validated, extended and improved by multiple research teams cf. [2, 3, 4, 15, 19, 31, 58, 62, 63, 64, 65, 66, 67, 68, 70, 71]. Other research teams have more recently addressed the curse of dimensionality. Examples include [1, 8, 9, 22, 23, 34, 37, 40] among many notable others. Although there has been substantial success on deterministic control problems and their associated rst-order HJ PDEs, there has been less success in the case of stochastic control problems driven by Brownian motion. The issue that arises in adaption of the max-plus based CODF methods to stochastic control problems is a curse of complexity" induced by a necessary application of a max-plus distributive property at each time-step [58, 62]. Related approaches that attempt to attenuate this e ect through other means include those of Akian et al. [2, 3, 4]. A radically different approach to this difficulty has recently come to light. Quite unexpectedly, we have found a means for conversion of certain second-order HJ PDEs to 1st-order HJ PDEs over a set of dual variables. In some cases, this conversion is exact. In the case where the non-quadratic components can be con nected to the zeroth order term, the conversion is exact modulo solution of a linear, non-homogeneous heat equation, which is not difficult.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jan 21, 2022
- Source ID
- FA95502210015XX0
Entities
People
- William M. McEneaney
Organizations
- Air Force Office of Scientific Research
- United States Air Force
- University of California, San Diego