Pupil Size Encodes Uncertainty during Exploration
Abstract
Exploration is an important part of decision making and is crucial to maximizing long-term rewards. Past work has shown that people use different forms of uncertainty to guide exploration. In this study, we investigate the role of the pupil-linked arousal system in uncertainty-guided exploration. We measured participants' (n = 48) pupil dilation while they performed a two-armed bandit task. Consistent with previous work, we found that people adopted a hybrid of directed, random, and undirected exploration, which are sensitive to relative uncertainty, total uncertainty, and value difference between options, respectively. We also found a positive correlation between pupil size and total uncertainty. Furthermore, augmenting the choice model with subject-specific total uncertainty estimates decoded from the pupil size improved predictions of held-out choices, suggesting that people used the uncertainty estimate encoded in pupil size to decide which option to explore. Together, the data shed light on the computations underlying uncertainty-driven exploration. Under the assumption that pupil size reflects locus coeruleus-norepinephrine neuromodulatory activity, these results also extend the theory of the locus coeruleus-norepinephrine function in exploration, highlighting its selective role in driving uncertainty-guided random exploration.
Document Details
- Document Type
- Pub Defense Publication
- Publication Date
- Jan 01, 2023
- Source ID
- 10.1162/jocn_a_02025
Entities
People
- Deshawn Chatman Sambrano
- Elizabeth A. Phelps
- Emily Dial
- Haoxue Fan
- Samuel J Gershman
- Taylor Burke
Organizations
- Air Force Office of Scientific Research
- Harvard University
- Massachusetts Institute of Technology
- National Institute on Drug Abuse