Brain-IT Reconstructs Images Seen Through MRI, Not Thoughts

Researchers at the Weizmann Institute have developed Brain-IT, an AI model that reconstructs images from brain activity measured by functional MRI and anticipates the brain's response to a given image. The team claims to match the performance of methods requiring 40 hours of data in just one hour of MRI, while clarifying that the technology does not read thoughts. Potential extensions via EEG, sound, and, more distantly, dreams, are being considered.

The technology does not read thoughts and remains dependent on MRI

The researchers emphasize that Brain-IT does not allow access to thoughts, memory, or language. At this stage, the model can only generate images from functional MRI scans, a lengthy procedure that requires the voluntary participation of the subject. Michal Irani and other scientists are exploring the possibility of using a simpler EEG device to achieve similar results. The prospect of decoding dreams remains hypothetical: it would require overcoming technical challenges related to the rapid pace of imagery in dreams, while an MRI scan takes about two minutes. If these obstacles were overcome, reading dreams could become feasible, but this remains in the realm of research.

One hour of MRI claimed to rival 40 hours of data

Michal Irani indicates that Brain-IT surpasses other approaches in reconstructing both the content and details of images. According to the team, one hour of functional MRI data from a new subject is sufficient to achieve results comparable to those of methods requiring 40 hours of recordings. The researchers position these advancements against models capable of preserving the overall ambiance of a scene but making errors on fundamental aspects such as composition or color. The images reconstructed by Brain-IT are not perfect, but they remain largely faithful to the original.

Reconstructing vision from measured brain patterns

Brain-IT employs pattern recognition to transform variations in blood flow and oxygen measured by functional MRI into images similar to those perceived by the examined individual. The model can also predict what a brain scan would look like if a specific image were shown. Its training relied on thousands of scans from the Natural Scenes Dataset, collected from eight volunteers viewing specific images during acquisition. The researchers identified 128 functional regions of the brain that activate according to the type of visual content: for example, certain areas respond to food, while others react to sports scenes. These regularities help the AI determine what the person is seeing.

Team, publications, and potential uses beyond the lab

Brain-IT was developed by Professor Michal Irani and her colleagues at the Weizmann Institute of Science, with documentation available on GitHub and a presentation at a scientific conference earlier this year. The researchers clarify that the project is still primarily in the laboratory stage, but they envision medical applications, particularly to assist individuals in communicating when they cannot express themselves. The team also plans to explore reconstruction from auditory signals. Michal Irani describes the term "thought reading" as a phrase intended to attract attention and specifies that the model allows for the reconstruction and anticipation of brain responses to images. Some of the functional areas detected were already cataloged by neuroscientists, aligning with previous research that highlighted, for example, the brain activity of dogs with their owners and the use of the same neurons in humans to recall or visualize an image.