Intent Switching and Co-Adaption of Man and Machine in a Closed-Loop Brain Computer Interface
Abstract
Brain computer interfaces (BCI) have been an active area of research for well over a decade. Originally envisioned as communication devices for the neurologically locked-in, several BCI systems have been developed that aim to assist the otherwise neurologically healthy in human-machine interaction. For BCIs of this type, it is important to develop an understanding of man-machine co-adaption. For example, when the C3Vision BCI for image search (Sajda et al, 2010; Pohlmeyer et al. 2011) is operating in a closed loop, neural signatures are being used to train a transductive graph-based model based on Òneural interestÓ scores. The feedback from the machine can change the prevalence and even characteristics of the images shown to the user that can, in turn, affect the userÕs perceptual/cognitive state. For instance, we have observed adaptation of not only the machine, via new neural interest scores, but adaptation of the neural signatures evoked by the images, including amplitude and timing changes of said neural signatures. Most BCI systems largely ignore co-adaption and instead assume a more-or-less stationary process in terms of the stochastic nature of the man-machine interaction. In this project our major goal is to investigate fundamental scientific questions in co-adaption of man and machine in a closed-loop brain computer interface. The following outlines the main objectives of the research project. Objective 1. Investigate whether switches in user intent can be measured using EEG and exploited in a closed-loop BCI. Preliminary data by our group has shown how a closed-loop BCI can be used to integrate neurophysiological signatures of object recognition and attentional orienting with computer vision to enable a man-machine system for image search. However, in this preliminary work, the user was not adaptiveÑi.e., they did not change their object of interest or intention during the search task. A basic research question is Òwhat happens if a user changes his/her intent during the experiment?ÓÑi.e., modifies the category of images they are interested in, as a function of the time within the experiment. Scientific literature suggests that such intent switches require a context updating (Polich, 2007) process that includes flushing of working memory and a change in the re-orienting response that may decay as a function of time. We will investigate the effect of such intent switches on the underlying neural signatures, measured via EEG, as well how to best discount the signatures of the past given their use as labels into the transductive graph-based model (Wang et al., 2009)Ñi.e., how to optimally fuse the neural signatures with computer vision given the intent switch. In all cases, we will use signal detection theory to construct precision/recall (P/R) and receiver operating characteristic (ROC) curves from decoded EEG and track the neural correlates of intent switches. Objective 2. Identify additional physiological/behavioral variables that correlate with intent switching. We have preliminary data showing that pupillary measures, such as baseline pupil diameter and event related diameter changes, correlate with specific EEG components of target detection and orienting at specific post-stimulus times. These pupillary changes also correlate with expectation/anticipation changes (Hong, Walz, & Sajda, 2012). Our hypothesis is that physiological changes in pupillary responses can be related to specific EEG components reflective of intent switching and that a combination of both physiological measures will ultimately be more informative to an exemplar-based computer vision system. Objective 3. Localize and model the cortical networks underlying intent switching...
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Sep 11, 2018
- Source ID
- W911NF1610507
Entities
People
- Paul Sajda
Organizations
- Army Contracting Command
- Columbia University
- United States Army