Scientists Create Neural Implant That Turns Brain Signals to Speech

A team of researchers from the University of California San Francisco has developed a neural implant that can turn brain signals into distinct sounds. The device can reportedly give voices back to individuals with speech issues or brain injuries.

Converting the neural information sent by our brain to different parts of our body responsible for creating coherent sounds or speech is no easy feat. It’s like an orchestra where the brain has to keep our lips, throat, tongue, jaw, larynx, and diaphragm working in perfect synchrony to form even the shortest and simplest words.

Over the years, scientists have devised different ways to simulate the complicated process. For instance, researchers from Columbia University successfully turned brain activities into distinct sounds using AI-powered brain implants.

The Columbia researchers’ device works by capturing the neural signals coming from a patient’s auditory cortex and feeding them into an artificial neural network. The AI decodes words out of the brain signals and synthesizes them, allowing listeners to recognize legible words.

The UCSF Neural Implant

For their study, the UCSF scientists used a different approach. They created a neural implant that decodes the signals sent by the brain to a patient’s vocal parts and guesses how they will transform into articulations responsible for making audible sounds.

Unlike the Columbia researchers’ single-step brain translation approach, the UCSF’s technique, which harnesses the principle of muscle movement translation, can make more precise and easier to interpret results.

To test their concept, the team employed the help of five volunteer patients who have already undergone surgical treatment for chronic epilepsy. The researchers implanted the arrays of electrodes right on the surface of each patient’s brain where they will gather the messages sent to the body’s speech systems. Sensors were also put into the patients’ teeth, tongue, and lips to monitor their movements.

Each patient was asked to read hundreds of words and sentences from a speech recognition database created by the UCSF scientists. They were also instructed to read passages from tales like Sleeping Beauty.

An algorithm sifted out the signals generated explicitly by the brain to control the movement of the lips, jaw, and tongue. Then, a speech synthesizer made distinct sounds using the movement map created by the algorithm. The results, although not perfect, were reportedly remarkable.

Edward Chang, the study’s senior author, said in a statement:

“For the first time, this study demonstrates that we can generate entire spoken sentences based on an individual’s brain activity. This is an exhilarating proof of principle that with technology that is already within reach, we should be able to build a device that is clinically viable in patients with speech loss.”

Read More: New AI-Powered Implant Can Turn Thoughts Into Speech