A Brain-Computer Interface for Voice Synthesis in People with ALS
Abstract
Having the ability to communicate quickly and with depth and nuance is important for a high quality of life, and retaining or regaining this ability is a high priority for people who are losing or have lost their speech due to ALS. Today, patients living with ALS can use existing augmentative and alternative communication devices that use remaining motor functions such as eyeblinks or a sip and puff system. But these devices are slow, cumbersome, and require substantial effort by both the user and caregiver to work reliably. They also do not capture the full range of expression possible by speech. Communicating just by trying to speak would be much faster and easier; we are trying to build a neuroprosthesis that enables people to do just that. This can potentially be achieved using an intracortical brain computer interface (iBCI): A device that links the brain to external devices. This emerging type of medical technology essentially bypasses the damaged parts of the nervous system and connects healthy parts of the brain to a computer. In this project, we will place hundreds of tiny electrodes into areas of the cortex that are trying to send commands to the muscles of the lips, tongue, jaw, and voice box that would normally produce speech. This computer runs advanced machine learning algorithms that decode what the measured brain signals mean; we will develop deep learning techniques that translate brain activity patterns into a voice output that is immediately played out by the computer s speakers. This synthesized voice would be audible to both the iBCI user and whoever they are conversing with so that the speech iBCI user can have natural conversations. Being able to immediately hear this output also allows the user to practice controlling their new artificial voice. One of the things we will study is whether our participants get better at speaking through their speech iBCIs with time, much like how many people with speech-impairing injuries gradually improve their ability to speak thanks to the brain’s ability to learn and adapt. The group conducting this study is a consortium of academic research groups called BrainGate, which is the largest and longest running iBCI clinical trial. Previous neuroprostheses developed by this team have successfully provided point-and-click and handwriting communication: clinical trial participants with paralysis of their arms attempted to move their hand either as if moving a computer mouse, or as if handwriting. Our algorithms decoded their neural activity to make character selections displayed on a computer screen with speeds of up to nineteen words per minute. This communication rate represents the best iBCI performance to date, but natural speech is even faster: 150 words per minute. Here we will test whether a similar approach, now applied to directly decoding what the person is trying to say from high precision brain signals provided by these implanted electrodes, can allow patients to speak at speeds approaching this rate. Potential users report high interest in such an assistive technology even though it requires surgery. Thus far this type of implanted brain sensors has an impressive safety and reliability record, and the devices work well for many years. One of the goals of the present study is to collect additional data about the safety of these brain implants, and to track how long they provide useful signals. Our hope is that the speech iBCIs we will develop will be able to help almost all ALS patients who are going to lose their ability to speak -- or even already have lost this ability. While it is more difficult to build a system to restore speech without having examples of that person’s actual speech and their corresponding brain activity, we have several reasons to believe this is possible. A recent study using a different type of technology that records from the brain’s surface rather than inside the brain found that the neural signals of a person who had be
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jan 04, 2024
- Source ID
- HT94252310153
Entities
People
- Sergey D. Stavisky
Organizations
- United States Army
- University of California, Davis