Development and Empirical Evaluation of a Theoretical Framework for Integrating Adaptive Multimodal Processes to Optimize Outcomes of Human-Agent Teams
Abstract
Although Human-Agent Teaming (HAT) systems of the future will vary greatly in form, function, and in composition, they are all likely to follow the same high level processing cycle which involves sensing the current states of components in the HAT system, assessing how to best adapt to achieve the goals based on those states, and executing that augmentation, which will require coordination among the team members. At the same time, HAT systems should embrace metacognitive monitoring by continuously evaluating their success during each of these three stages, and then updating their internal schema in order to learn from the results of those evaluations. This project views HAT systems through the lens of trust, which is integral to individual and overall system performance. But we know little about how these systems should be built and validated in order to enhance trust and reliance on both human and agent teammembers. We propose a theoretical framework, based on the construct of trust, to guide and collate HAT system research to avoid pitfalls relating to an abundance of disparate Òone-offÓ studies, that might use different trust-based HAT construct definitions, operationalizations, validation standards, and interpretations of results. The framework can be used to guide the development and empirical validation of HAT systems that can: (i) sense trust-related states and processes of heterogeneous HAT systems, (ii) assess why, when, and how to launch interventions (if any) to appropriately enhance trust and reliance, and, hence, overall team effectiveness, and (iii) provide theoretically-guided and empirically-validated evaluation criteria to gauge accuracy of sensing and success of those interventions. We will use the framework to develop a prototype Intelligent Trust Modulation (ITM) system for HATs that uses multimodal sensors to measure human, machine, and team ÔstatesÕ relating to trust, and intelligently selects real-time adaptations of system components to optimize team trust dynamics and team effectiveness. Our prototype system will leverage our suite of wearable and off-body physiological and behavioral sensors and will enable us to conduct empirical studies to test (and iterate upon) the underlying framework. Our approach has three primary objectives: (1) we will develop an initial theoretical framework to suggest appropriate measurable states of interest, system adaptations, and evaluation criteria, each based on relevant basic, computational, and methodological behavioral and social science literatures focused on the constructs of trust and reliance, (2) we will create a proof-of-concept ITM system which will involve functionality for teams with 2-4 members in various configurations and collect data while they engage in complex collaborative tasks, and (3) we will use the ITM system to empirically validate and refine the framework via a series of experiments. The expected impact for the HAT research community is as follows: First, our theoretical framework can be extended beyond our testbed scenarios and used to help suggest appropriate measurable states of interest, system adaptations, and evaluation criteria, for trust-based HAT systems of the future. Second, the architecture and supporting data collections involved in the development of our proof-of-concept ITM system, can guide the development of similar intelligent HAT systems that merge theory and domain knowledge from the social sciences with the computational power of pattern classification and reinforcement learning. Third, the results from our empirical studies can show the research community results from experiments where system augmentations are actually selected and executed in real-time (rather than just speculating what the effects might be for a set of hypothetical augmentations). Taken together, these research deliverables have the potential to add valuable guidance to advance the fast-moving field of Human-Agent Teaming.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jul 02, 2019
- Source ID
- W911NF1910401
Entities
People
- Leanne Hirshfield
Organizations
- Army Contracting Command
- United States Army
- University of Colorado Boulder