Joint Activity Testing: Towards a Multi-Dimensional, High-Resolution Evaluation Method for Human-Machine Teaming
Abstract
Quantitative evaluations of human-machine teams (HMTs) are desperately needed to ensure technological implementations are helpful rather than harmful to overall system performance; however, as machines increasingly behave like active cognitive teammates, traditional evaluation strategies risk overestimating HMT capabilities. Areliable HMT evaluation method should include multiple high-resolution, continuous measures for both system performance and system challenges that can be implemented unobtrusively in real-time operations. In our prior work, we proposed joint activity testing (JAT) as acandidate evaluation framework to satisfy these requirements. Preliminary efforts with asingle dimension of performance and challenge have indicated that the method can identify the additive benefits of joint activity with aspecific technology. In this paper, we explore the operationalization of multi-dimensional JAT by synthesizing our work in two intelligence and two healthcare domains. The patterns observed between domains will guide future JAT, reveal paths towards real-time implementation, and spark future research evaluating resilience.
Document Details
- Document Type
- Pub Defense Publication
- Publication Date
- Sep 01, 2022
- Source ID
- 10.1177/1071181322661537
Entities
People
- Dane A. Morey
- Daniel J. Zelik
- Dante Della Vella
- Michael F. Rayo
- Taylor B. Murphy
Organizations
- Air Force Research Laboratory
- Ohio State University