Autonomous Decision-Making Systems under (Adversarial) Distribution Shifts- Learning to Adapt, Generalize and Robustify
Abstract
Widely applicable to autonomous robotics, human-machine teaming, mission-resource coordination in the field, autonomous driving systems, resource optimization and more, Reinforcement learning (RL), an increasingly important machine learning model, is an effective way to model interactive real-time sequential decision-making processes. The idea of designing methods to adapt, generalize or robustify learning under distribution shift in Machine Learning powered decision-making systems is important to avoid catastrophic failures in practice. For instance, consider the task of learning a policy for a next generation combat aircraft to attack enemies. An agent has learned a strategy in a simulator or a previous task on a real-world battlefield. In deployment, however, the real-world battlefield may have changed due to bad weather or enemy mines-traps, so it becomes important to learn a strategy that adapts to the changes. To induce such novel behavior into the future combat aircraft, our technical approach focuses on real-time adaptability of agents in sequential decision-making systems. Critical to many AFOSR applications, we will speed up real-time robust decision-making via RL. We will address the intrinsic difficulty in achieving efficient and effective learning in decision making systems -- the distribution shifts of data samples. We will provide methods to combat against distribution shifts due to (1) policy improvement, (2) domain-task shifts and (3) adversarial perturbations-attacks. The thesis of this proposal is autonomous real-time, robust, and safe decision-making via learning to learn (to speed-up decision-making), learning to adapt (to different tasks), learning to generalize (to new unseen tasks), and learning to robustify (to defense against adversarial perturbations or attacks). The proposed work has the potential to shape applications critical to Air Force Office of Scientific Research, such as autonomous robotics, human-machine teaming, and mission-resource coordination in the field.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Feb 29, 2024
- Source ID
- FA95502310048
Entities
People
- Furong Huang
Organizations
- Air Force Office of Scientific Research
- United States Air Force
- University of Maryland