Intensive EEG Recordings for Decoding of Imagined Speech
Abstract
Individuals with communication disorders, age-related and-or neurological diseases may find themselves unable to communicate their thoughts and needs to others. When a person imagines speech in their mind, they induce neural responses that can be recorded non-invasively with scalp EEG electrodes, and these in principle can be decoded and interpreted (recognized) if we have an adequately trained speech recognizer . We are developing the theoretical and experimental paradigms for such advanced decoding algorithms that require high quality EEG recordings of people listening to speech. In the envisioned system, subjects will be asked to imagine speech while their EEG neural activity is recorded. These responses are subsequently transformed to the waveforms predicted to occur when listening to the same speech sentences. The predicted-listened responses can then be decoded by a speech recognizer trained on a large database of EEG responses recorded from subjects while they simply listen to speech. This last requirement implies the need for large enough databases of EEG recordings of speech listening. A recognition system can then be created in combination with powerful language models (e.g., word2vec and GPT-4). The necessary EEG database can be recorded from a few thousand of subjects each listening passively to diverse speech material for as little as 2-4 hours. With such data, we can transform the imagined speech to predicted-listened signals, and then apply the EEG recordings to the EEG-speech recognition system to decode the imagined utterances. The goal of this DURIP proposal is to request funding for EEG recording equipment to collect the databases, and the computational resources to analyze and formulate the recognition system. There are many potential applications for this technology ranging from communications with locked-in, stroke, and Alzheimer patients as well as studies of deteriorating memory in the aged population, for insider threat-risk and-or deception detection.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Feb 05, 2025
- Source ID
- FA95502410055
Entities
People
- Shihab A Shamma
Organizations
- Air Force Office of Scientific Research
- United States Air Force
- University of Maryland