Combining Deep Reinforcement Learning Control with Novel Vertical Flight Concepts for Robust Ship Based Operation 22-079WP
Abstract
Approved for Public Release: The three overarching goals of the proposed project are to develop (1) novel control algorithms to enable autonomous approach and vertical landing of unmanned aerial systems (UASs) in challenging GPS-denied adversarial environments with wind gusts, delayed feedback, and incomplete and noisy state observation, (2) high-fidelity flight dynamic modeling tools to understand platform response to arbitrary large gusts, and (3) novel vertical take-off and landing (VTOL) UAS concepts that have high gust tolerance during take-off and landing as well as high cruise efficiency. We propose to team with the US Air Force Academy to leverage their extensive experience, and infrastructure for flight testing novel UASs and autonomous multi-platform control. Specific objectives are: Objective 1. Develop robust reinforcement learning algorithms for safe ship landing in adversarial environments: Control algorithms for UASs are typically developed using a simulator model of the real-world system. However, there will be inevitable discrepancies between the simulator of a system and its actual real-world settings, which are collectively known as simulation-to-reality gap (sim-2-real gap). We will develop a class of robust reinforcement learning algorithms using the idea of domain randomization, which are robust against the sim-2-real gap, and provably guarantee optimal performance in a wide variety of real-world conditions. Objective 2: Develop meta-robust RL algorithms for fast adaptation using complex real-time data: A robust controller design is typically conservative as it optimizes the performance against worst-case scenarios and it is typically not adaptive to the available real-time information. At the same time, fast adaptation is extremely useful for the control of UASs operating in complex, dynamic environments. Using the powerful idea of meta-learning, we will develop algorithms that are simultaneously robust and are capable of fast adaptation to thereal-world conditions using the rich and complex real-time sensor data. Objective 3. Develop multi-agent RL algorithms for coordination and control of multiple UASs: While employing multiple UASs may improve the effectiveness of a mission, it requires efficient coordination and control of this multi-agent system. We will develop scalable and real-time multi-agent reinforcement learning algorithms which can provably improve the search-track-land operations through the optimal coordination and control of this multi-agent system. Objective 4. Develop high fidelity flight dynamic modeling to understand UAS platform response to arbitrary large gusts: We will develop a flight dynamic modeling framework to accurately predict the gust response of UAS platforms while exploiting the fact that most UASs can be broken down into three key subsystems # rotors, airframe, and wings. We will develop aerodynamic gust response models of these three subsystems separately in such a way that any arbitrary platform could be modeled by combining these sub-system models and will be utilized for training the RL algorithms in Objectives 1 # 3. Objective 5. Verify the findings from Objectives 1 to 4 using repeatable flight experiments: First phase of flight tests will involve autonomous approach and landing a small scale UAS on a moving platform in different wind scenarios. Subjected to the availability of funds, a second phase of testing could be conducted at the Naval Surface Warfare Center Maneuvering and Seakeeping Basin facility. Objective 6. Develop gust-tolerant and efficient VTOL aircraft concepts: We will develop aircraft concepts that have low bare-airframe gust sensitivity as well as high control authority, which will be complemented by the control laws from Obj. 1 # 3. The improved gust tolerance of these new aircraft designs will be demonstrated through flight dynamic simulations (Obj. 4), wind-tunnel testing, as well as flight-testing sub-scale models (Obj. 5
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- May 15, 2023
- Source ID
- N000142312404
Entities
People
- Benedict Moble
Organizations
- Office of Naval Research
- Texas Engineering Experiment Station
- United States Navy