Mixed Precision Reinforcement Learning for Control Simulation of Unmanned Undersea Vehicles
Abstract
Control systems for Unmanned Undersea Vehicles (UUVs) are typically implemented using Proportional Integral Derivative (PID) control systems. PID control systems for UUVs are resource intensive to tune because they require several engineers, marine operators, and ship crew to spend time offshore to tune the controller. Furthermore, PID controllers rely on complex dynamic system models that contain assumptions to reduce the computational complexity of the models but degrade performance of the controller if an environmental condition is encountered that conflicts with an assumption. In this study, a Deep Reinforcement Learning control system based on the Deep Deterministic Policy Gradient (DDPG) algorithm is studied for a UUV control system. The DDPG algorithm is model free, meaning that a complex dynamic system model is not needed to learn and provide optimal control performance. Secondly, Deep Reinforcement Learning control systems are tuned autonomously, so this greatly reduces the resources needed for controller tuning. The drawback to Reinforcement Learning is that it is computationally and resource intensive. To improve upon this, this study will investigate how mixed floating point precision with loss scaling can be used to reduce the time and computational resources needed to train the DDPG agent. Numerical case studies will be presented.
Document Details
- Document Type
- Technical Report
- Publication Date
- Aug 20, 2021
- Accession Number
- AD1177529
Entities
People
- Alfa Heryudono
- Christopher Hixenbaugh
- Eugene Chabot
Organizations
- University of Massachusetts Dartmouth