‘Mind-Reading’ Technology Can Turn Brain Activity Into Images

The system, devised by neurologists, AI researchers, and cognitive scientists at Radboud University, combines AI with medical imaging techniques. It begins with a more sophisticated version of the magnetic resonance imaging (MRI) scanner called a functional magnetic resonance imaging (fMRI) scanner. While a conventional MRI machine facilitates imaging of a person’s anatomy to diagnose trauma or disease, an fMRI machine detects tiny changes in metabolic function. This includes neuron activity and the minuscule changes in blood flow within the brain.

While study participants were hooked up to the fMRI, the team at Radboud displayed photographs of individual humans and asked participants to study them closely. The information from the fMRI was fed into a powerful AI algorithm called a Generative Adversarial Network, or GAN. Based on the neurological data received, the GAN was able to construct photo-like images similar to those shown to the participants. While the visual stimuli and AI-constructed images aren’t perfect matches—in one pair, a man ages a bit, while in another a woman goes from strawberry to bleach blonde—they’re surprisingly close.

The team trained the GAN by showing participants images of human faces that consisted of larger pixels, each of which was given a unique computer code based on its shade. Based on how each participant’s brain reacted to the training images, the GAN was able to translate neuron and blood flow activity into computer code, which it used to assemble its own versions of the photos. Every image shown was that of a new face, to prevent the GAN from continuously building upon its “vision” of a particular face.

The fMRI/AI system’s existence isn’t for the sake of novelty. The study’s lead author, AI researcher and cognitive neuroscientist Thirza Dado, hopes to use the technology to help restore vision in people who have become blind. But she admits it has other applications, too.

“’By developing this technology, it would be fascinating to decode and recreate subjective experiences, perhaps even your dreams,” Dado told the Daily Mail. “Such technological knowledge could also be incorporated into clinical applications such as communicating with patients who are locked within deep comas.”