Minimizing Oscillatory Signals in Deep Reinforcement Learning Control of Unmanned Undersea Vehicles
Abstract
Control systems for Unmanned Undersea Vehicles (UUVs) are typically implemented using Proportional Integral Derivative (PID) control systems. PID control systems for UUVs are resource-intensive to tune since they require engineers, marine operators, and ship crew working together to adjust the controller. Furthermore, PID controllers rely heavily on complex dynamical system models that contain assumptions to reduce the computational complexity of the models. The controller's performance may degrade if environmental and external conditions do not fully align with those assumptions. In this work, a Deep Reinforcement Learning (DRL) control system based on the Deep Deterministic Policy Gradient (DDPG) algorithm is studied for a UUV control system. The DDPG algorithm is model-free, meaning that the explicit formulations of the complex dynamical system models are optional to provide optimal performance. The DRL-based control systems are tuned autonomously, reducing the resources needed for manual tuning. Our focus is to study how different Deep Neural Network (DNN) architectures implemented as part of the DDPG agent affect the control signal output by the control system. DNN architectures that minimize undesirable oscillations in the control signals, which could potentially cause physical damage to the UUV, will be of interest. Numerical case studies will be presented.
Document Details
- Document Type
- Technical Report
- Publication Date
- Aug 24, 2021
- Accession Number
- AD1177533
Entities
People
- Alfa Heryudono
- Christopher Hixenbaugh
- Eugene Chabot
Organizations
- University of Massachusetts Dartmouth