Long-Term Analysis of a Human-AI Collaboration Study Using a Mobile Game
Abstract
Human-AI interaction is typically studied in laboratory settings where participants spend minutes to hours per session. The benefits of this approach are well-known: excellent experimental control and sophisticated sensing of behavioral and nonbehavioral data (e.g., psychophysiology) to name a few. The downsides are discussed less frequently. The burden on experimenters and participants from sessions, difficulties in collecting massive data sets, and the inability to analyze longtime-scale processes limit the produced knowledge. Our experimental setup combines mobile devices and wearable sensors as a means of addressing these limitations. Via a mobile game application called Busy Beeway, for studying human-AI collaboration, we require participants to balance their strengths and weaknesses with autonomous partners in obstacle avoidance tasks. Participants provide game data daily over several months while their context heart-rate, activity, sleep, environment is measured continuously by StudentLife. In this technical report, we present an initial survey of interaction styles that participants developed over time with different autonomous capabilities and across changing context. Additionally, we present how this data can be used to predict human-AI collaborative performance, thus providing a way to suggest human-AI pairings.
Document Details
- Document Type
- Technical Report
- Publication Date
- Feb 08, 2023
- Accession Number
- AD1192832
Entities
People
- Andrew Campbell
- Arvind Pillai
- Evan C. Carter
- Lidia S Obregon
- Lydia Tapia
- Torin Adamson
Organizations
- Dartmouth College
- United States Army Research Laboratory
- University of Arizona
- University of New Mexico