Multimodal behavior and interaction as indicators of cognitive load
Abstract
High cognitive load arises from complex time and safety-critical tasks, for example, mapping out flight paths, monitoring traffic, or even managing nuclear reactors, causing stress, errors, and lowered performance. Over the last five years, our research has focused on using the multimodal interaction paradigm to detect fluctuations in cognitive load in user behavior during system interaction. Cognitive load variations have been found to impact interactive behavior: by monitoring variations in specific modal input features executed in tasks of varying complexity, we gain an understanding of the communicative changes that occur when cognitive load is high. So far, we have identified specific changes in: speech, namely acoustic, prosodic, and linguistic changes; interactive gesture; and digital pen input, both interactive and freeform. As ground-truth measurements, galvanic skin response, subjective, and performance ratings have been used to verify task complexity.
Document Details
- Document Type
- Pub Defense Publication
- Publication Date
- Dec 01, 2012
- Source ID
- 10.1145/2395123.2395127
Entities
People
- Bo Yin
- Eric Choi
- Fang Chen
- Julien Epps
- M. Asif Khawaja
- Natalie Ruiz
- Ronnie Taib
- Yang Wang
Organizations
- Air Force Office of Scientific Research
- Australian Government
- Australian Research Council
- Department of Communications and the Arts
- National ICT Australia