For the first time in the world, technology has been realized that allows people to read out images in their minds from brain signals and reconstruct them without limiting the type of image.It was jointly developed by the National Institute for Quantum Science and Technology, the National Institute of Information and Communications Technology, and Osaka University.
Remarkable advances in AI have already shown that the images humans see can be reconstructed from brain signals measured with functional magnetic resonance imaging (fMRI).However, the problem with images that people imagine in their minds (mental images) is that the accuracy of image restoration is low, and so far success has only been achieved with limited types of images such as letters of the alphabet and simple geometric figures. I hadn't.
In contrast, this research focuses on ``generative AI,'' which has attracted attention in recent years, ``Bayesian inference,'' which estimates unobserved data based on observed data, and ``Bayesian inference,'' which is used in the chemical field to simulate the movement of atoms and molecules. Using a new method that combines the Langevin Dynamics Method, we succeeded for the first time in the world in restoring an image of any landscape or object that you have in your mind.
In this method, AI was first used to read 1,200 photos of landscapes and objects, and a ``scoring table'' was created that expressed image characteristics as approximately 613 million numerical values.Furthermore, they prepared brain signal data for 1,200 images when subjects were looking at the same photos, and based on this, they built a ``brain signal translator'' that generates a scoring sheet from the brain signals.
A brain signal translator is used to translate the scoring sheet from the brain signals when an image is visualized in the mind, and the generative AI is made to draw the mental image. At this time, the mental image is estimated from the scoring sheet using Bayesian estimation. The novelty of this method is that it efficiently generates highly valid images by repeatedly evaluating, modifying, and updating images according to the updating rules of the Langevin dynamics method.After making enough corrections (500 times in this study), the team was able to restore a more accurate mental image than conventional methods in about four minutes.
This result will be applied to new forms of communication, art and creative activities, and medical diagnosis of nightmares and hallucinations as a technology that can objectively restore the cognition and consciousness in the minds of others. It has many possibilities, including development of brain-machine interfaces.
Paper information:【Neural Networks】Mental image reconstruction from human brain activity: Neural decoding of mental imagery via deep neural network-based Bayesian estimation
