Decoding Imagined Speech
Abstract
Individuals with communication disorders, age-related and-or neurological diseases may find themselves unable to communicate their thoughts and needs to others. When a person imagines speech or music in their mind, they induce neural responses that can be recorded non-invasively with scalp EEG electrodes, and these in principle can be decoded and interpreted (recognized) if we have adequate SNR, a good sense of the structure of the EEG imagination recordings, and a trained speech recognizer . These three requirements are for the first time within reach, and hence the hope of decoding imagined speech is now conceivable. Specifically. to do so, (1) we propose to apply advanced signal processing algorithms to achieve high quality EEG recordings. (2) Subjects will then be asked to imagine speech while their EEG neural activity is recorded. These responses are subsequently transformed to the waveforms predicted to occur when listening to the same speech sentences. (3) These predicted-listened responses can then be decoded by a speech recognizer trained on a large database o fEEG responses recorded from subjects while they simply listen to speech. The science and technology behind all these three requirements have advanced sufficiently enough that the goal of decoding imagined speech is now realistic. The most challenging of these goals is the last ingredient, as there are only a few databases of EEG from listening subjects. Without them, it is difficult to train a speech recognizer to achieve adequate performance. If successful, there will be many potential applications for this technology ranging from communications with locked-in, stroke, and Alzheimer patients as well as studies of deteriorating memory in the aged population. Of interest also are applications for insider threat-risk and-or deception detection.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Feb 06, 2025
- Source ID
- FA95502410242
Entities
People
- Shihab A Shamma
Organizations
- Air Force Office of Scientific Research
- United States Air Force
- University of Maryland