Safe and Resilient Deep Learning-based Optimal Adaptive Tracking with Adversaries (Whitepaper Tracking Number: 23-000004713)
Abstract
Future naval ships are expected to install more challenging shipboard loads with higher electric power demands. These loads that consume significant quantities of energy within a very short time are often referred to as pulsed power loads (PPLs) for ship board integrated power systems (SIPS). The presence of PPL might lead to voltage sags or frequency deviations, load trips, and so on leading to instability as there is a large change in the SIPS dynamics which cannot be precisely modeled. This scenario leads to a challenging power system optimal control problem due to multiple objectives, operational constraints, complex nonlinear system dynamics with system uncertainties which necessitates a novel online learning based safe and resilient tracking scheme that would function despite operational constraints and external inputs/disturbances. Existing offline trained deep neural network-based techniques using gradient based methods do not offer safety, explainable, performance and stability guarantees in the presence of exogenous inputs limiting their ability in real-time environments. In other words, the trustworthiness of such learning-based control schemes is a concern. Deep reinforcement learning (DRL) using neural networks in combination with dynamic programming (DP), and control barrier functions can be utilized to design efficient safe and resilient optimal control schemes. By asserting Shapley#s values, how actions are selected by a control policy can be explained. Theory of trustworthy deep NNs for optimal tracking design of nonlinear systems, in the presence of constraints and adversaries from safety and resilience perspective is in its infancy. The goal of this study is to provide online DRL-based NN optimal adaptive trajectory (OAT) tracking control schemes with adaptation, safety assurances with resiliency, has guaranteed performance, and is supported by a rigorous design and mathematical framework. The objectives are:1. Develop an online explainable hybrid learning scheme using deep NN for OAT, which is mainly for tracking a variety of time-varying trajectories, of uncertain nonlinear dynamic systems. Study explainability of DRL-based control schemes by using Shapley values.2. Examine trustworthiness of OAT of uncertain nonlinear systems with constraints from safe learning perspective using control barrier function (CBF) approach. 3. Investigate online lifelong DRL scheme for actor-critic OAT framework for uncertain nonlinear systems with constraints. 4. Examine the impact of adversaries in such LCS and develop mitigation/resilience of OAT schemes for nonlinear networked control systems (NNCS) with adversarial inputs using control barrier density Lyapunov function (CBDLF).5. Demonstrate the online safe and resilient lifelong deep NN-based OAT scheme in simulation for SIPS and on limited hardware. Development of novel explainable, safe and resilient learning scheme with nonstandard weight tuning for deep NN and its application to power and energy system is utmost importance. The principal investigator has been quite active in this area and has laboratory facilities. This research presents an opportunity to deal with a more powerful and unified paradigm of complex learning problems. The approachtaken here employs a deep NN as a fundamental block within the RL framework, to provide forward-in-time solutions to learning control problems, in particular for explainable, safe and resilient control system, in real-time environments. The proposal will advancethe state of the art in DRL based online decision making for uncertain nonlinear systems by providing rigorous mathematical analysis, trustworthiness, convergence and performance guarantees. Moreover, by applying the theoretical results to emerging power and energy control applications, the DRL based optimal trajectory tracking control will gain wider acceptance.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- May 15, 2024
- Source ID
- N000142412338
Entities
People
- S. Jagannathan
Organizations
- Office of Naval Research
- United States Navy
- University of Missouri System