A Framework for Combining Model-based and Data-driven Control for Autonomous Helicopter Aerial Refueling
Abstract
Aerial refueling (AR) is a fundamental technological enabler for enhancing range and battlespace capabilities of manned and unmanned rotary-wing aircraft for Navy#s Future Vertical Lift Maritime Strike mission. Helicopter air-to-air refueling (HAAR) is a particularly challenging maneuver that demands significant pilot training, skill and workload. Autonomous HAAR holds significant promise for alleviating this need for extensive pilot training, with the potential for improving safety during aerial refueling operations with substantially reduced pilot workload. However, realizing autonomous HAAR faces several barriers: (1) refueling is a precision-contact maneuver with multiple airborne systems in proximity, resulting in stringent safety and positioning requirements, (2) tanker-receiver aerodynamic interactions are complex and difficult to predict, (3) drogue relative position estimation and tanker-drogue coupling results in high control workload, and (4) the slim overlap of acceptable flight speeds for the fixed-wing tanker and rotary wing receiver severely limits the control authority available to the receiver for maneuvering. There is thus a critical need for autonomous control strategies for HAAR that guarantee safe and precise maneuvering, while being robust to model uncertainty, sensor inaccuracies, and environmental disturbances (e.g. gusts and aerodynamic interactions between tanker, receiver and the hose-drogue).To address these barriers, in this proposed effort we will develop a novel control architecture consisting of a model predictive controller (MPC) augmented by a data-driven reinforcement learning (RL) agent that is trained to respond to unmodeled dynamics (e.g., drogue movement) and disturbances (e.g., aerodynamic coupling between the tanker, receiver, and the hose-drogue assembly) during the refueling process. Our prior work has shown that MPC algorithms are well-suited for contact-based maneuvers, which impose time-varying constraints on state and input variables (such as smooth docking, sufficient hose-to-rotor-plane distance). However, models for tanker-receiver-drogue assembly aerodynamic coupling and drogue motion are not well-suited for model-based control design approaches such as MPC. On the other hand, data-driven learning algorithms such as RL can be trained to respond to unmodeled (but systematic) behavior from repeated simulations (or trials) that incorporate high fidelity models of these phenomena. However, unlike model-based design, such data-driven control algorithms typically do not provide safety or stability guarantees. In prior work, we have also developed a new class of RL algorithms where the safety requirements are embedded as constraints. We will leverage these recent advances to create a unified control architecture for these two methodologies (model-based MPC and data-driven RL with safety/stability guarantees) that will enable safe and precise autonomous HAAR. We will validate the proposed methodologies in a high fidelity helicopter AR simulation environment with a generic helicopter dynamical model, aerodynamic tanker-receiver-drogue interaction models, and multibody hose-drogue assembly dynamics. In order to design, implement and validate the proposed methodology for autonomous HAAR, RPI and its partner CTSi will perform the following research tasks: (1) Establish feasibility of using MPC with a plug-in RL module for the pre-contact to contact phase and evaluate suitable control architectures, cost/reward functions and constraints for both components; (2) Develop theoretical and algorithmic machinery required to provide analytical guarantees and bounds of performance for safe and timely contact; (3) Validate algorithms on the full-scale helicopter simulation with realistic aerodynamic disturbances from the tanker, receiver forebody effects and hose-drogue motion dynamical models; and (4) Quantify effect of sensing quality, model accuracy and disturbances on control performance.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Apr 12, 2023
- Source ID
- N000142312377
Entities
People
- Sandipan Mishra
Organizations
- Office of Naval Research
- Rensselaer Polytechnic Institute
- United States Navy