Trustworthy Real-time Communicative Multi-Agent Reinforcement Learning for Decision-making in Unstab
Abstract
Artificial intelligence has the great potential to augment or amplify intelligence in the service of effective command and control (,C2) over forces in war. In naval warfare, even the best of naval commande,elmed by all highly dynamic areas of the naval warfare they have to keep up with -- including operational goals, the tactical pictur,e, the status of enemy and friendly forces -- to form the basis for their actions. Across a multidomain battle space with military p,lans being executed simultaneously under, on and above the sea, as well as in space and cyberspace, it is pivotal to have real-time,and robust machine-augmented command and control (augmented C2) and decision-making for all agents working collaboratively against t,he enemy. Applicable to autonomous robotics, human-machine teaming, mission-resource coordination in the field, autonomous driving s,ystems, resource optimization and more, Reinforcement Learning is an effective way to model interactive real-time sequential decisio,n-making.As an abstraction of the classified tactical and maneuvering characteristics for submarines and submarine warfare, in this,project, we consider the Lost in the Woods problem in which a prisoner or a group of prisoners (red team) just escaped prison into, the woods nearby. The goal is for the guards (blue team) to find the escapees before they make it to their hideout. Commander s gui,dance to subordinate commands, which is from a more holistic picture of the overall situation but not necessarily a clearer picture,equires modeling of multi-agents and their interactions. We introduce a Multi-agent Communicative Reinforcement Learning (MACRL) fra,mework to model the collaboration of friendly forces (blue team agents including human and machine such as cameras, search parties,,helicopters) against the enemy (red team) under a dynamically changing environment.In MACRL, it is crucial for agents to communicate,. However, just as the fog of war will set in quickly, the situation in the woods is unpredictable -- communications may go down; sa,tellites may be blasted out of space or disrupted; weapons and sensors may fail; and adversaries will not act in the ways we anticip,d the communication channel is also susceptible to cyber-attacks. Commander is likely to have a more holistic picture of the overall, situation, but not necessarily a clearer picture of local conditions at each scene. Therefore, we must enable augmented C2 and deci,sion-making for both the commander and subordinate commands that are capable of being responsive to emerging intelligence, surveilla,nce, and reconnaissance information that differ significantly from expectations and deviating from the plan when conditions require,it.The primary goal of this project is to design real-time, adaptive, and robust decision-making systems for multi-agents in the blu,e team to adaptively combat against the red team and be robust even when the blue agents communication network is under attack. The, thesis of this proposal is autonomous real-time, robust, and safe MACRL-augmented commander and subordinate decision-making via lea,rning to adapt (to emerging intelligence, surveillance, and reconnaissance information that differ significantly from expectations),and learning to robustify (to defense against adversarial communication attacks). We will make fundamental advances for domain adapt,ation in interactive sequential decision-making systems, introduce a framework to model trustworthiness in multi-agent systems and i,mprove the state-of-the-art robustness of decision-making under adversarial attacks.Approved for Public Release.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- May 16, 2022
- Source ID
- N000142212335
Entities
People
- Furong Huang
Organizations
- Office of Naval Research
- United States Navy
- University of Maryland