MURI New Game Theory for New Agents Foundations and Learning Algorithms for Decision-Making Mixed-Agents
Abstract
Modern strategic environments include a diverse set of participants: humans, semi-autonomous systems guided by humans, and autonomous AIs. Complex and multiscale interactions between participants, ubiquitous uncertainty regarding the environment and other participants, and computational limitations and behavioral biases intrinsic to the participants in these environments complicates prediction of outcomes and the design of better systems. Our aim is to develop new approaches resulting in accurate predictions for such environments, as well as leading to the design of learning algorithms and machines that perform better in strategic settings, aid in understanding the underlying causes of observed outcomes, and result in better designed systems. To achieve our aim for this new learning-driven paradigm of systems of heterogeneous agents, we will tackle afresh the grand challenges of game theory#predicting behavior and outcomes in complex strategic settings, designing learning algorithmsthat work well in such complex settings, designing systems that ensure good outcomes with heterogeneous agents, and guiding behavior to outcomes with desirable properties#by building a newgame theory for new agents.To function in real-world complex and dynamic environments which include both human and AI agents, AI agents need to simultaneously and continuously learn and evolve their assessments and their strategies, which we call multi-agent strategic reinforcement learning (MA-SRL). We will develop the MA-SRL framework by generalizing ideas from strategic learning (SL) in traditional games, and computational learning schemes like reinforcement learning (RL).Our goal in developing and analyzing MA-SRL is to transcend SL and RL by building a theory in which AI agents learn simultaneously about the game and other participants. We also seek to include designers who can intervene to steer the system to desirable configurations by changing the payoffs or information structure, and hence, the game environment of the agents. We will achieve this by co-evolving three core elements#A) Models of Strategic Behavior, B) Strategic Learning Algorithms, and C) Learning Outcomes#that are necessary for AI agents to perform effectively and predictably in complex multi-agent systems operating in a variety of environments# static environments, dynamic environments and settings with designers. Throughout, these AI agents will have to respond to human or other types of agents who may be using any strategy of choice, including learned strategies. Since equilibrium outcomes depend on the collective behavior of all participants, we will also include models of and strategies used by these other types of agents.In certain situations, an analyst may be unable to provide accurate predictions about outcomes due to informational or computational limitations. Our project aims to develop tools that can identify such situations and determine additional resources needed to address them. We will also identify scenarios whereit may be possible to make significant predictions about the evolution of the system. In the process we will develop algorithmic solutions with provable performance guarantees for AI-agents to make decisions in complex real-world environments. We will also design games when possible to do so, to steer equilibrium outcomes to desirable configurations.Multi-agent systems with complex strategicinteractions arise in many contexts relevant to the DoD. Our research will provide guidelines and performance predictions for them,as well as algorithms to use for AI-agents that will be employed in them. Some examples are systems with teams of AI-controlled aircraft and human pilots, and platoons of agents operating in hostile territory or assisting in disaster relief. System security goalssuch as systemic risk minimization for financial, economic and infrastructure networks can also be targeted via our research.Approved for Public Release
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Nov 09, 2024
- Source ID
- N000142412742
Entities
People
- Vijay Subramanian
Organizations
- Board of Regents of the University of Michigan
- Office of Naval Research
- United States Navy