21-000000012: Deep Neural Network Learning Control of Complex Dynamical Systems
Abstract
Deep learning has accelerated progress in reinforcement learning (RL), defining the field of deep RL (DRL), to scale decision-making problems that were previously intractable, i.e., high-dimensional state and action spaces. Deep learning using neural networks (NNs), or deep NN, can be utilized to design efficient controllers with image inputs in the feedback-loop imitating the human brain. Though available deep NN learning schemes employ gradient descent-based weight tuning using RL, the learning process is slow, may incur vanishing gradient, and lack performance guarantees limiting their applicability to real-time environments.To attain optimality faster unlike offline iterative methods, a novel DRL-based hybrid learning scheme, which combines iterative updates within the sampling interval and temporal difference (TD) at the sampling instant, and its use in adaptive dynamic programming (ADP) framework, with image inputs for optimal control is necessary. Theory of deep NNs for optimalcontrol design of intelligent systems is in its infancy.Given the promising results to-date by the investigator, the goal of this study is to provide online DRL-based NN decision schemes with adaptation, optimization, and online learning with application to control, has guaranteed performance, and is supported by a rigorous design and mathematical framework. The objectives are:1. Develop a DRL-based hybrid NN scheme with image inputs. Study deep NN weight convergence and learning effectiveness.2. Investigate DRL-based hybrid NN learning schemes with online tuning using Lyapunov analysis. Examine initial state changes on learning effectiveness and develop lifelong learning.3. Develop a DRL-based optimal hybrid NN control scheme with image inputs using zero-sum game formulation. Study online tuning of deep NN weights and closed-loop stability for lifelong learning. 4. Verify the online hybrid learning schemes in simulation for naval applications. 5. Implement and demonstrate in hardware the DRL-based hybrid NN learning scheme on a naval application. (Optional year if approved for funding)The principal investigator has been quite active in this area and has adequate laboratory facilities to perform this investigation. This research presents an opportunity to deal with a more powerful and unified paradigm of complex learning problems for intelligent systems and envisions a brain-like controller. The approach taken here employs a deep NN as a fundamental block within the RL framework, to provide forward-in-time solutions to learning control problems in real-time environments. The proposal will advance the state of the art in DRL based online decision making for nonlinear systems by providing rigorous mathematical analysis, convergence and performance guarantees. Moreover, by applying the theoretical results to emerging control applications, the DRL based controlwill gain wider acceptance.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Apr 06, 2021
- Source ID
- N000142112232
Entities
People
- S. Jagannathan
Organizations
- Office of Naval Research
- United States Navy
- University of Missouri System