Mixed Precision Deep Reinforcement Learning for Control of Unmanned Undersea Vehicles
Abstract
Deep reinforcement learning (RL) shows promising results for control problems in continuous action spaces. A drawback to deep RL is that it can be very computationally intensive; this is particularly concerning when considering fielding deep RL applications on computationally-constrained and power-constrained autonomous vehicle platforms. Mixed numerical precision methods are an active research area in which progress is being made toward improving the computational efficiency of deep learning methods. Although mixed-precision approaches are well understood for supervised learning tasks, this area is relatively unexplored for deep RL. We aim to fill this gap in the research by presenting a method to improve the computational efficiency of the Deep Deterministic Policy Gradient (DDPG) algorithm using mixed numerical precision and loss scaling. Numerical cases are presented to quantify the performance and computational improvements of DDPG agents trained with mixed precision as compared with single precision in considering the following problems: (1) the pendulum deep RL benchmark problem and (2) continuous control of a complex dynamic system model.
Document Details
- Document Type
- Technical Report
- Publication Date
- Aug 25, 2022
- Accession Number
- AD1200547
Entities
People
- Alfa R. Heryudono
- Christopher J. Hixenbaugh
- Eugene J. Chabot
Organizations
- Naval Undersea Warfare Center
- University of Massachusetts