Computational model for understanding and predicting the effects of transcranial current stimulation on audiovisual speech recognition
Abstract
Background and Significance. Understanding speech in background noise is a challenging task that is central to many critical every-day missions in the military. Fighter pilots, sailors, personnel in armored vehicles and in command centers, for instance, all need to be able to communicate in noisy and highly challenging conditions. In many situation speech comprehension can be aided by visual signals such as lip movements; such audiovisual speech comprehension is indeed an important example of multi-sensory processing. Moreover, neurostimulation has been proposed as a tool to enhance awareness and speech comprehension, but a thorough understanding of the effects of neurostimulation on the dynamics of the involved neural networks is still lacking. The proposed project will contribute to closing this gap in knowledge. In particular, the project will identify how neurostimulation interacts with the neural mechanisms for audiovisual speech recognition, which will lead to the design of novel types of neurostimulation that can optimally enhance speech comprehension, with and without visual input. Objectives. The goal of the proposed project is to develop a computational model for understanding and predicting the effects of transcranial current stimulation on audiovisual speech recognition. The three objectives that the proposal seeks to achieve are: - To develop a neural network that exhibits the specific neural dynamics that matter in audiovisual speech recognition. - To model the effects of neurostimulation and to investigate the impact of different stimulation types on the encoding of speech in the neural network, with and without visual signals. - To experimentally measure the effects of key types of neurostimulation on audio-visual speech recognition and compare to the computational results. Methods. The project will employ state-of-the art neuronal models as well as experimental designs for assessing the effects of neurostimulation on audiovisual speech comprehension. The computational model will build on recent work on speech encoding through coupled activity in the gamma and theta frequency band. Speech exhibits indeed temporal dynamics at different time scales that match those of important frequency bands in the brain. In particular, the theta rhythm in the cerebral cortex (4 Ð 8 Hz) corresponds to the rate of syllables and the low-gamma rhythm (30 Ð 70 Hz) to the rate of phonemes. These neural activities can track the rhythm of phonemes and syllables in speech and play a role in speech processing. In particular, a recently developed neural network model for speech encoding through coupled gamma and theta oscillations has shown that speech can be reliably decoded from the modeled neuronal activities. Moreover, visual signals associated with speech, such as lip movements, can entrain neural oscillations or evoke neural activity at these frequencies, and can thus enhance speech recognition. Recently it has been discovered that transcranial alternating current stimulation (tACS) can entrain or evoke cortical activity as well and that this can lead to a better understanding of speech. The computational model that this project will develop will build on the recently described neural network model for speech encoding through coupled cortical oscillations. The model will be extended to allow for the stimulation of neurons through visual signals as well as through tACS. The efficacy of speech coding of this neural network, with different types of neurostimulation as well as with and without visual input, will be investigated. We will also perform experiments to measure the effect of neurostimulation on audio-visual speech recognition. This will employ a speech-in-noise test with concurrent neurostimulation, with and without additional visual input.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Sep 04, 2019
- Source ID
- W911NF1910396
Entities
People
- Tobias Reichenbach
Organizations
- Army Contracting Command
- Imperial College London
- United States Army