Scientists Turn Brain Scans Into Intelligible Speech With Neural Network

Stephen Hawking was perhaps the most famous user of “vocoder” speech synthesis hardware, but he was not alone. People all over the world are unable to speak on their own, but science may be approaching a point where they can turn their inner thoughts into speech without tedious typing or clicking. A team from the Neural Acoustic Processing Lab at Columbia University has devised an AI model that can turn brain scans into intelligible speech.

The research combines several advances in machine learning to interpret the patterns of activity in the brain to find out what someone wants to say even if they’re physically unable to make noise. This isn’t a mind-reading machine — the signals come from the auditory cortex where your brain processes speech. So, it can understand real speech and not so-called “imagined speech” that could hold your deepest, darkest secrets.

The technology is still very much a work in progress; more a proof of concept than something you can hook up to your head. The study used neural signals recorded from the surface of the brain during epilepsy surgery, a process called invasive electrocorticography (ECoG). The researchers, led by Nima Mesgarani, used epilepsy patients because they often have to undergo brain surgery that involves neurological testing.

The researchers recorded brain activity while the subjects listened to people recite select words like the numbers zero through nine. This is important because everyone has different brain wave patterns when processing speech. So, Mesgarani and the team trained a neural network that was specific to each patient. They only had 30 minutes of data, which limits the model’s effectiveness. The results are still impressive, though. The team fed in the raw ECoG scans, and the network generated speech with a vocoder. You can listen to a sample of the models here. There are four models, the last of which should be the most realistic.

It’s all a bit robotic, and the first few numbers are tough to make out. However, the team says that about three-quarters of people surveyed were able to understand the vocoder output. To make better neural networks, you need more data. Collecting custom brain wave data from everyone using invasive electrocorticography isn’t exactly practical. One day, we might find some commonality that makes brain wave translation universal like speech recognition. But for now, this is an impressive if impractical first step.

Continue reading

Huawei Sells Honor Brand Amid Tightening Trade Restrictions

(Credit: Kevin Frayer/Getty Images)Huawei has been battered by US trade restrictions in the last few years, and it’s taking a toll on the company’s long-term stability. Experts don’t expect a radical change when the new US administration comes to power next year, so Huawei is beginning to take drastic action. It has sold its Honor…

Google Celebrates 10 Years of Chrome OS With New Features

Google launched Chrome OS a decade ago with a device you couldn't even buy called the CR48. Now, you can get Chrome OS laptops from numerous manufacturers for a pittance, and they're selling better than Apple computers. Google is celebrating ten years of Chrome OS with the launch of version 89, which includes a phone hub, clipboard manager, improved virtual desktops, and more.

How to Stop LG From Stuffing Ads Into Your Brand New OLED TV

LG thinks you should have to watch ads to use its TVs. We disagree.

Cerebras Unveils 2nd Gen Wafer Scale Engine: 850,000 Cores, 2.6 Trillion Transistors

Cerebras has unveiled its second-generation wafer-scale engine, or WSE-2. The new wafer is etched using 7nm lithography, with 850,000 cores, 2.6 trillion transistors, and 40GB of onboard SRAM.