Graphical Games and Distributed Reinforcement Learning Control in Human- networked Multi-group Societies
Abstract
Military teams operate in complex dynamic networked environments with spatiotemporally diverse threats generated by malicious attacks, functional failures, and human errors. Limited communications restrict the options available to each agent. Existing work in multi-agent systems focuses on consensus and synchronization, generally assumes simplified communication, and rarely include optimality or game-theoretic interactions. The relations between cooperation and conflict among teams can be studied formally using game-theory notions, not yet used in existing cooperative policies. Distributed control policies are fixed and cannot adapt to neighbors behaviors. Reinforcement Learning (RL) to improve responses has not been used fully in existing graph topology synchronization objectives. We propose to study fundamental science for human networks decisions by leveraging our recent discoveries in socio-cognitive Reinforcement Learning, graphical multi-player games, cooperative and adversarial differential games, multi-agent networked control, uncertainty quantification, Bayesian belief updates, and Markov analysis. We will investigate interactions of humans, communication topologies, autonomous agents, adversarial agents, and environments. Applications will be made to interactive decision and control mechanisms for military vehicle convoys. This proposal has three research objectives. Objective 1: Multi-player and Adversarial Graphical Games. We will leverage our background work in Graphical Games to study Multi-player and Adversarial Graphical Games and provide game solutions that accommodate allowed communication topology in networked teams. New classes of non-Nash games on graphs will be studied including min-max games, adversarial team games, and cluster formation games on the coarse structure of graphs. We will bridge Nash graphical games and min-max games by investigating their performances in decoupling agent policies, robustness, and the distance to the Nash optimality. Objective 2: Reinforcement Learning of Multi-agent Graphical Game Solutions in Real Time. We will leverage our background work in RL to develop Bio-inspired Learning methods to learn optimal game solutions online by observing the behaviors of neighbors in real time. New Integral RL algorithms will guarantee convergence to rationalizable optimal Nash responses. New Off-Policy RL algorithms allow agents to observe the actions taken by malicious agents, predict their worst-case activities, and develop optimal defensive responses. Deep Learning techniques in neural networks will be integrated into the structure of Reinforcement Learners to speed up response, use all available data, and provide structured actor-critic topologies. Uncertainty quantification will allow predictable and verifiable control in stochastic settings. Objective 3: Game-theoretic Adaptive Vehicle Convoys. We will develop game-theoretic solutions for mixed convoys with both humans and autonomous vehicles. Epistemic multi-player games for human and autonomous networks will use Bayesian belief updates about the motives of teammates and hidden adversaries to improve situational awareness. Our preliminary results clarify, for the first time, the relations between belief and control using Bayesian games for multi-agent differential dynamical systems on graphs. Uncertain intentions and stochastic spatiotemporal environmental impact will be explored to expedite the decision process. Given the PI s award-winning publication and collaborative record with Army/TARDEC and RDECOM, they are well positioned to address these fundamental research challenges highlighted by ARO s Network Science. UT-Arlington is a Hispanic-serving Institution, the 5th most diversified university in the nation. This research significantly enhances UT-ArlingtonÕs ability to attract and train students for STEM fields of interest to DoD mission.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jul 09, 2020
- Source ID
- W911NF2010132
Entities
People
- Frank Lewis
Organizations
- Army Contracting Command
- United States Army
- University of Texas at Arlington