Using Brain-Computer Interfaces to Restore Speech

Post by Kelly Kadlec

What is a speech brain-computer interface?

A brain-computer interface (BCI) translates signals recorded from the brain into the control of an external device that can either restore or enhance a natural function. In March, we explored how this technology is being used to help people with paralysis regain independence by connecting motor areas of the brain to computer and robotic outputs that can replace the function of an arm or legs. In certain cases of paralysis, motor impairment can also result in the loss of speech. In the most extreme cases, for example, those who experience a brainstem stroke or amyotrophic lateral sclerosis (ALS), this can lead to locked-in syndrome. In locked-in syndrome, an individual’s cognitive functions remain intact, but their restricted motor function requires reliance on adaptive technology like eye tracking or BCIs for communication.

Eye-tracking devices and the first iterations of BCIs for communication had users spell out a word they wanted to say letter-by-letter. Although this helps communication, it does not replace speech. In the last five years, an alternative BCI for communication has advanced to the forefront of this technology - one that directly decodes the movements involved in speech itself.  

How is speech decoded?

Speech BCIs follow the same general pipeline as other motor BCIs: a neural signal is acquired and preprocessed, the parts of the signal most informative to the decoder are selected with feature extraction, a decoder maps these features onto a set of possible outputs, and then an external device, in this case, a computer, produces that output. Most research on speech BCIs has focused on acquiring signals from the motor cortex corresponding to the motor commands sent to the mouth, vocal cords, and diaphragm to produce speech. These command signals can then be translated to phonemes, the building blocks for words. The most popular choice in algorithms to decode phonemes has been recurrent neural networks. Finally, decoded phonemes are either vocalized by a computer-generated voice or assembled into words and displayed as text on a screen. 

What’s the history?

The first attempts at speech BCIs were developed using electroencephalography (EEG). EEG measures the activity of large populations of neurons from electrodes on the scalp. While it has certain advantages as a choice for adaptive technology including being non-invasive, the signal-to-noise ratio proved too poor to decode the phonemes of speech as described in the section above.

In 2009, a team of researchers from several institutions collaborated to implant a single electrode in the cortex of a patient living with locked-in syndrome. The electrode was placed in a speech area of the motor cortex, identified by having the subject attempt to say the names of pictures during functional magnetic resonance imaging (fMRI). They demonstrated that these signals were viable for decoding phonemes, but using only a single channel proved a constraint on how much could be decoded at a time. 

Where are speech BCIs today?

The promising results from initial attempts helped to motivate the implanting of high-density electrode arrays for speech BCIs, both with electrocorticography (ECoG) and microelectrode arrays. ECoG is an intracranial form of EEG that records local field potentials from electrodes resting on the surface of the cortex. Microelectrode arrays penetrate the cortex a few millimeters and record action potentials from individual neurons, sometimes up to several hundred single neurons at a time.

Over the past several years, research insights have shed light on the power of BCIs in restoring speech. In 2018, researchers at UCSF implanted a high-density ECoG array in a patient with severe paralysis who was unable to speak and relied on head-tracking adaptive technology for communication. The researchers were able to decode attempted articulatory movements from neural signals to “synthesize” speech with an anatomically accurate computerized mouth that “spoke” the words the participant had said. 

Around the same time, a group at Stanford implanted intracortical microelectrode arrays in the cortex of a participant who was unable to speak due to ALS and were able to develop a “speech-to-text” interface, which works sort of like voice-to-text without the need for a vocalization. This study decoded the largest set of words to date (125,000) with an error rate as low as 25% — the same rate previously achieved in the above ECoG study, which used a word bank of only 1,024 words. 

A recent collaboration between the University of California Davis and the Stanford team that developed the speech-to-text interface achieved 95% word accuracy from a 125,000-word bank decoding attempted speech from an ALS patient implanted with microelectrode arrays. In this study, researchers took advantage of the recent advances in machine learning of large language models to improve decoding speed and accuracy. Much like how computer vision leverages the natural statistics of an image, the highly structured nature of our language can be used by these models for higher accuracy predictions. The researchers also used machine learning trained with previous recordings of the participant speaking to create a computer-generated voice that sounded like them, allowing them to hear decoded speech in their actual voice. 

All of the work discussed so far has involved decoding speech from a person actively speaking, attempting to speak, or miming speech. An alternative that has only recently been considered is that of internal speech, in other words, our unspoken internal dialogue. Internal speech is saying a word in your mind without any attempt to vocalize it. Researchers at Caltech decoded internal speech from microelectrode arrays in the parietal cortex of a tetraplegic participant. The ability to decode different modes of speech from varying brain areas will be critical to developing personalized speech BCIs for the needs of an individual. If the recent years of progress have demonstrated anything, the possibilities for speech BCIs have only begun to be explored. 

What is the future of speech BCIs?

Speech BCIs have the potential to be one of the first implementations of motor BCIs to be widely available as a therapeutic device. Unlike the advanced robotics needed to fully replace more complex motor functions, speech output for speech only requires a computer. Results from speech BCIs also show advantages over other currently available adaptive technologies. Still, for speech BCIs to recreate natural conversation, improvements are needed. Perhaps most important is increasing the speed of speech decoded. Natural human speech is at a rate of about 150 words a minute for native English speakers, and current speech BCIs are under 80 words a minute. 

Another important consideration is the parts of speech that are important for communication outside of the words themselves. For example, tone of voice is lost in text and the currently available computer-generated voices are monotone. Body language and gestures are also personal components of communication that can be lost in translation. This highlights the need to eventually combine BCIs for speech with BCIs for other motor functions to restore full communication for locked-in individuals. 

The takeaway

The loss of speech can have a devastating impact on quality of life. Speech BCIs may be able to restore communication for individuals who are unable to speak by translating neural activity into computer-generated vocalized speech. These recent advancements in BCI technology along with its profound potential for positive impact may make speech BCIs among the first therapeutic applications to be approved and widely available. 

References +

Anumanchipalli, G. K., Chartier, J. & Chang, E. F. Speech synthesis from neural decoding of spoken sentences. Nature 568, 493–498 (2019).

Card, N. S. et al. An accurate and rapidly calibrating speech neuroprosthesis. 2023.12.26.23300110 Preprint at https://doi.org/10.1101/2023.12.26.23300110 (2024).

Guenther, F. H. et al. A Wireless Brain-Machine Interface for Real-Time Speech Synthesis. PLOS ONE 4, e8218 (2009).

Lopez-Bernal, D., Balderas, D., Ponce, P. & Molina, A. A State-of-the-Art Review of EEG-Based Imagined Speech Decoding. Front. Hum. Neurosci. 16, (2022).

Metzger, S. L. et al. A high-performance neuroprosthesis for speech decoding and avatar control. Nature 620, 1037–1046 (2023).

Wandelt, S. K. et al. Decoding grasp and speech signals from the cortical grasp circuit in a tetraplegic human. Neuron 110, 1777-1787.e3 (2022).

Wandelt, S. K. et al. Representation of internal speech by single neurons in human supramarginal gyrus. Nat. Hum. Behav. 1–14 (2024) doi:10.1038/s41562-024-01867-y.

Willett, F. R. et al. A high-performance speech neuroprosthesis. Nature 620, 1031–1036 (2023).

Stem Cell Therapy for Stroke: Progress and Challenges

Post by Shahin Khodaei

What's the science?

A stroke is a serious medical condition where blood flow to a part of the brain is reduced, causing the death of brain cells and impairments in movement, speech, or cognition in survivors. The majority of strokes are ischemic strokes, meaning that blood supply is disrupted because of a blockage in blood vessels. Currently, the only FDA-approved treatments for ischemic stroke involve removing the blockage either mechanically or pharmacologically to restore normal blood supply, however, this treatment is only effective within a few hours of the stroke onset. Finding new treatments that can be effective at later time points is critical. This week in Brain, Rust and colleagues published a review paper on one such treatment strategy: stem cell therapy for long-term treatment of ischemic stroke.

What happens in ischemic stroke?

When blood supply to a brain region is interrupted, millions of neurons and billions of synapses quickly die. Beyond the damage to neurons, stroke causes massive local and systemic inflammation and significant damage to the blood-brain barrier (BBB), which regulates what substances enter and exit the brain. These changes all have lasting negative consequences on brain function. Over the subsequent days and weeks, neurons can also die off in distant brain regions anatomically connected to the stroke site.

After a stroke, the brain’s built-in neuroplasticity processes kick in. Some new neurons and synapses may be created, and surviving brain regions start to take on some of the functions of the brain areas lost in the stroke. In addition, new blood vessels form around the site of the stroke, and the support cells in the brain are able to partially restore the BBB. All these mechanisms lead to some degree of functional recovery in patients – this plasticity window peaks during the first 3 months and gradually weakens 6-12 months after stroke.

How can stem cell therapy help?

Given that many neurons die in a stroke, a treatment strategy that replaces the lost cells using stem cells is an intuitive concept. Preclinical studies using animal models have shown that using stem cells after stroke may be therapeutic in two ways. One is the replacement of lost cells, where stem cells turn into brain cells and integrate into the existing neuronal networks to restore function. However, this form of direct integration is rather limited, because newly formed neurons and cells often have a short lifespan. Instead, emerging studies show that transplanted stem cells support the brain’s built-in plasticity processes, leading to beneficial effects. These include reducing the massive inflammatory response that follows a stroke, the repair and remodeling of blood vessels, and the repair of neural circuits. Both cell replacement and support processes were reported following stem cell transplant in a recent preclinical study by Rust’s team. Using a mouse model, transplanting cells 7 days after stroke caused increased neuronal plasticity, blood vessel repair and remodeling, and improved motor function. These transplanted cells developed into neurons and other brain cells that survived for at least five weeks and cross-talked with surrounding stroke tissue to activate plasticity and regeneration processes.

Clinical trials using stem cells after stroke started two decades ago, and have shown that this approach is safe, without significant negative effects. However, in terms of effectiveness, these clinical trials have found mixed results. Rust and colleagues suggest that findings from preclinical studies can teach us how to more effectively use stem cells in a clinical setting. For example, to increase the lifespan of stem cells, preclinical studies have treated the cells pharmacologically and genetically before transplanting them into the brain – a similar approach may be used in clinical trials in the future to possibly increase the effectiveness of stem cell therapy.

What's the bottom line?

Stem cell therapies have been recognized as one of the most promising approaches to help survivors of stroke. Unfortunately, clinical trials using this strategy have not shown consistent improvements for patients thus far. Rust and colleagues argue that our growing understanding of the mechanisms of stem cell therapy from preclinical studies can help improve the effectiveness of this approach for patients, ultimately leading to better treatment for stroke.

Access the original scientific publication here.

Focused Ultrasound for the Treatment of Parkinson’s Disease

Post by Natalia Ladyka-Wojcik

A primer on motor symptoms in Parkinson’s disease

Parkinson’s disease (PD) is a complex neurodegenerative disorder that affects over 10 million people globally, typically in individuals aged 60 years or older. PD affects a central brain area called the basal ganglia, leading to a range of motor, cognitive, and sleep changes. A particularly affected region of the basal ganglia is a crescent-shaped cell mass in the brain stem called the substantia nigra. The neurons in this tiny region produce the neurotransmitter dopamine and play a big role in planning and controlling body movement. In PD, these neurons begin to die off and hallmark motor symptoms of PD such as tremor, rigidity, and slowness of movement tend to emerge when about 80% of dopamine is lost.

Dopamine is critical for stimulating receptors in one of the basal ganglia called the striatum, which works with the substantia nigra to send signals between the spinal cord and the brain. If striatum receptors aren’t sufficiently stimulated, other portions of the basal ganglia may also be over- or under-stimulated: the hallmark tremors associated with PD result from overstimulation whereas rigidity and slowness of movement result from understimulation. Perhaps unsurprisingly, patients with PD report that the severity and frequency of these symptoms significantly decrease their quality of life, highlighting the importance of targeted treatment for these symptoms.

Current challenges in treating Parkinson’s disease symptoms

Traditionally, PD treatments have largely relied on medications, which aim to either preserve dopamine in the brain by preventing its breakdown, increase dopamine release, or mimic dopamine altogether. Although clinical research has made great strides in developing effective medications for PD, these medications tend to work better in the early stages of the disease. Moreover, some of these medications can cause unwanted side effects, including hallucinations, nausea, depression, and even obsessive behavior. Given these considerations, research has turned to exciting new avenues for treating PD by leveraging advanced therapeutic technologies. Here, we’ll survey two major technologies to better understand the future of PD treatment: (1) deep brain stimulation, and (2) focused ultrasound.

Deep brain stimulation for Parkinson’s disease

Deep brain stimulation (DBS) is a surgical therapy that addresses the movement symptoms of PD but can also help improve other non-motor symptoms including changes in sleep. In the U.S., it is approved by the Food and Drug Administration and is especially effective in individuals with severe tremors. In DBS surgery, electrodes are inserted into the basal ganglia and then a pacemaker-like implant is placed either under the collarbone or in the abdomen. This implant delivers electrical neurostimulation to the basal ganglia which the patient controls with a remote. On average, DBS patients with optimal drug therapy require lower medication doses compared to a control group receiving only optimal drug therapy. Importantly, DBS patients also show slowed progression of tremor symptoms. The future of DBS as a therapy for PD holds much promise, with larger randomized control studies to investigate DBS efficacy underway by the Food and Drug Administration.

Focused ultrasound: A cutting-edge therapy for Parkinson’s tremors

When imagining what neurosurgery is like, you might visualize the surgical incisions needed for neurosurgeons to access the brain. However, focused ultrasound therapy is a new noninvasive approach: beams of ultrasonic energy are used to precisely and accurately target deep brain structures with thermal lesions without damaging surrounding healthy tissue. This technique massively reduces risks of infection or brain bleeding compared to traditional surgery and does not require any external implants to be placed in the patient’s body. Neurosurgeons typically use magnetic resonance imaging (MRI) to guide the ultrasound beams to the desired location in the patient’s brain, ensuring that thermal lesions are done accurately. On average, patients show marked improvements in motor symptoms of PD, especially tremor, although some negative side effects have been reported. In the future, some scientists hope that focused ultrasound technology can even be used to disrupt the blood-brain barrier, a selective semi-permeable membrane that defends the brain from external substances and can sometimes prevent medication from successfully reaching the brain. This means that in addition to directly manipulating dysfunctional brain signaling, focused ultrasound may also help PD medications be even more effective.

What’s the bottom line?

Every year 90,000 people are diagnosed with PD in the United States alone, and it is the second-most common neurodegenerative disease after Alzheimer’s dementia. PD is associated with progressive symptoms, especially related to motor skills, which are managed by available medications only to a certain degree of efficacy. New technologies, including DBS and focused ultrasound, provide exciting new avenues for the treatment of PD.

References +

Hacker, M. L., Turchan, M., Heusinkveld, L. E., Currie, A. D., Millan, S. H., Molinari, A. L., Konrad, P. E., Davis, T. L., Phibbs, F. T., Hedera, P., Cannard, K. R., Wang, L., & Charles, D. (2020). Deep brain stimulation in early-stage Parkinson disease. Neurology, 95(4), e393–e401.

Moosa, S., Martínez‐Fernández, R., Elias, W. J., Del Alamo, M., Eisenberg, H. M., & Fishman, P. S. (2019). The role of high‐intensity focused ultrasound as a symptomatic treatment for Parkinson's disease. Movement Disorders, 34(9), 1243-1251.

Rahman, S., Griffin, H. J., Quinn, N. P., & Jahanshahi, M. (2008). Quality of life in Parkinson’s disease: The relative importance of the symptoms. Movement Disorders, 23(10), 1428–1434.

Whitney, C. M. (2007). Medications for Parkinsonʼs Disease. The Neurologist, 13(6), 387–388.