Cognitive Models and Strategies for High-Performance Human-AI Teams
Abstract
Mixed human-AI teams will pervade future Army and DoD operations and their relevance to the ultimate Army mission cannot be understated. There is however a limitation in available data, models and theories that explain the dynamic behavior, coordination, and performance of human-AI teams. For example, we do not yet fully understand the dominant socio-cognitive processes that determine the dynamic, adaptive, and learning behavior of human-AI teams. Of especial interest are intellective tasks with uncertainty and limited resources: what are the rational, efficient, or irrational strategies and heuristics that humans tend to adopt in such circumstances? The lack of established theories is important because it leads to a lack of rigorous design principles and guidelines. Useful socio-cognitive models should inform the design of efficient AI agents that improve the overall human-AI team performance. In other words, empirically-validated models and theories are needed to model and build the human-AI teams of the future and to intervene when their performance deteriorates. This proposal s broad objective is the development and experimental validation of a theory of coordination of human-Al teams in complex intellective tasks. We plan to combine fundamental insights and models of team behavior from social sciences with state-of-the-art machine learning and dynamical systems methods. Specifically, our objective include: (1) modeling socio-cognitive structures in human-Al teams, including transacti ve memory systems, influence systems, and prospect theories; (2) identifying leading cognitive processes, heuristics and biases that underlie the formation of socio-cognitive structures and affect the accuracy of human-Al team decision making; (3) designing supervisory/coordinating AI agents in human-AI teams, based on a few leading concepts from applied psychology and machine learning, and testing/validating them in sequential, risky, uncertain decision making tasks; and (4) modeling how human-AI teams cope with limited training data acquired over short sessions, including how they react to various manipulations and intervention schemes. Our technical approach combines human-AI experiments, socio-cognitive theoretical modeling, and mathematical and statistical analysis. Specifically, we propose a sequence of increasingly-complex elaborations of a basic intellective task with various types of AI agents. The task involves decision making under limited resource allocation and uncertainty. In our experiments, AI agents will play the role of expert systems with unknown accuracy and of supervisory/coordinating agents. The proposed human-AI experiment will be executed on our software Platform for Online Group Studies (POGS), that is a general framework for conducting online experiments involving real-time collaboration. The proposed modeling and validation efforts will lead to the advancement of scientific knowledge in the broad field of Socio Cognitive Networks. We envision that our empirically-validated models will shed light onto the general principles describing human-AI teams, including adaptation and learning phenomena as well as their impact on team performance in intellective tasks motivated by Army s applications. We will be able to identify and distinguish efficient high-performance cognitive processes, as compared with heuristic processes that may lead to inaccurate decision making. Our empirically-validated designs of supervisory/coordinating AI agents will provide various baseline designs, inspired by fundamental socio-cognitive concepts and machine learning tools. Our comparative analysis will help the Army implement efficient, robust and reliable human-AI teams.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Sep 08, 2022
- Source ID
- W911NF2210233
Entities
People
- Francesco Bullo
Organizations
- Army Contracting Command
- United States Army
- University of California, Santa Barbara