A Bi-Level Deep Reinforcement Learning based Optimal Control Scheme Design and Experimentation for PEPDS
Abstract
Numerous advanced optimization and control algorithms have been developed for microgrids and shipboard power systems. However, integrating the algorithms separately designed for upper-level energy scheduling and lower-level power control has proven challenging. The existing decoupling between scheduling and control increases complexity and diminishes performance during implementation. Moreover, some solutions suffer from low TRL due to reliance on simplified model-based simulations for testing. To address these issues, a new control scheme needs to be designed to effectively integrate the two levels of operations and undergo hardware experimentation, which is crucial for PEPDS given its aggressive timeline.This proposal introduces a bi-level deep reinforcement learning (DRL)-basedoptimal control scheme for PEPDS, with the primary objective of achieving optimal scheduling and control coordination across diverse subsystems. The approach aims to handle system diversity through both large time-scale energy scheduling and real-time control. Atthe upper level, task-driven optimal schedules will be generated for generation and Energy Storage System (ESS) charging and discharging based on large time-scale constraints. At the lower level, distributed or decentralized optimal control will be employed to further address subsystem diversity in real-time. Subsequently, a hierarchical DRL algorithm will be introduced to optimize the coordination between energy scheduling and power control. Leveraging previous work on model-based control will aid in deep neural networksdesign and communication protocol selection. Comparative study against control theory-based designs will be conducted, and the latest developments in DRL will be incorporated to enhance interpretability, efficiency, and reliability.To evaluate the designed control scheme, both real-time simulation and hardware experimentation will be performed. A detailed PEPDS model will be developed for real-time simulation using the OPAL-RT real-time simulator (RTS). The designed DRL-based algorithms will be tested using the RTS-based distributed online learning platform developed in our lab. Concurrently, a PEPDS testbed will be developed using Wide Bandgap based Power Electronic Building Block (PEBB) modules, DRL-capable Digital Signal Processors, and EtherCAT based network. Continuous efforts will be made to enhance the efficiency and reliability of the software to elevate the TRL of the research outcomes.Over the courseof five years, the PI will actively collaborate with other teams to ensure smooth progress toward the defined objectives of the PEPDS portfolio.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Nov 08, 2024
- Source ID
- N000142412622
Entities
People
- Wenxin Liu
Organizations
- Lehigh University
- Office of Naval Research
- United States Navy