An artificial intelligence that can create pictures of what people are looking at based on brain scans is impressive, but not ready for widespread use
The images in the bottom row were recreated from the brain scans of someone looking at those in the top row
Yu Takagi and Shinji Nishimoto/Osaka University, Japan
A tweak to a popular text-to-image-generating artificial intelligence allows it to turn brain signals directly into pictures. The system requires extensive training using bulky and costly imaging equipment, however, so everyday mind reading is a long way from reality.
Several research groups have previously generated images from brain signals using energy-intensive AI models that require fine-tuning of millions to billions of parameters.
Now, Shinji Nishimoto and Yu Takagi at Osaka University in Japan have developed a much simpler approach using Stable Diffusion, a text-to-image generator released by Stability AI in August 2022. Their new method involves thousands, rather than millions, of parameters.
Advertisment
When used normally, Stable Diffusion turns a text prompt into an image by starting with random visual noise and tweaking it to produce images that resemble ones in its training data that have similar text captions.