Application of Shared Mental Models for Hierarchical Human-AI Teams for Improved Performance and Robustness
Abstract
Prior research has proven the utility of developingand maintaining SMMs to achieve success in decision#making with human#AI dyads. Here both humansand AI agents share common (sub)goals and responsibilities toward the broader scope of themission objectives. However, as most Command and Control (C2) missions comprise a larger numberof interacting individuals operating with varied roles, and responsibilities, creating SMMs willprove increasingly challenging. The goal of this research is to understand and extend the SMM hypothesisby examining the dynamics of humans and AI agents operating within hierarchical decisionmakingteams. We will test for the robustness of the SMM hypothesis in hierarchical mixed#agentteams of greater sizes and task complexities.The proposed work is broadly divided into two major research thrusts with multiple sub#tasks withineach: (1) to understand SMM development in hierarchical human#AI teams operating in a geospatialdecision#making environment, and (2) to assess the impact of maintaining SMMs in hierarchicalhuman#AIteams operating in a complex wargame. In the first research thrust, we will establishexperimental scenarios that allow us to test for humans#AI triads engaging in a decision#makingtask for a disaster relief planning situation. Humans and AI agents will be organized in mixed groupsof three within a hierarchical team structure where each entity will have a different goal and setof responsibilities. This setting will allow us to test for metrics of interest such as the utility ofdecisions made, the speed of decision#making, humans# experience of working within such teams,and ways to improve team performance. We will also identify the role of a team commander infacilitating SMM development by creating shared situational awareness in order to facilitate teamingfor varying task demands.Findings from this thrust will enable us to develop a framework overwhich SMM development in mixed hierarchical teams could be based. Subsequently, in the nextresearch thrust, we will investigate the robustness of SMMs in hierarchical mixed#agent teams inenvironments of greater complexity. We will transition to a complex war game with an in#personexperimental setting that allows us to observeinteractions among multiple humans and AI agentsinteracting throughout the task procedure. We would like to understand how the findings from theprevious research thrust generalize into a more complex task domain, and how SMMs are facilitatedby the team commanders. With greater task and team complexity, we further aim to understandhow additional information in the form of the humans# physiological features can be utilized by theAI agents to develop a better awareness of the humans they interact with. This will help us identifyrelevant features that provide the AI with more useful information to attain teammate awarenessso that SMMs between humans and AI agents can be improved.Through this research, we will (1) extend our knowledge of the impact of SMMs in hierarchicalhuman#AI teamsand the impact it has on the overall decision#making process, (2) establish foundationsfor testing and evaluating SMMs by observinghuman behavior and cognitive states on continuousinteractions with AI agents, and (3) extend the SMM hypothesis for real#time assessments ofhuman#AI teams in increasingly complex en. Approved for Public Release.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Mar 08, 2024
- Source ID
- N000142412135
Entities
People
- Karen Feigh
Organizations
- Georgia Tech Research Corporation
- Office of Naval Research
- United States Navy