AI Reconstructs Images from MRI: Promises and Risks

Researchers at Weizmann have developed a visual decoder capable of recreating images close to those actually seen, based on high-resolution functional MRI scans. Presented in New York, the tool reportedly requires one hour of calibration instead of the previous forty, offering potential gains for research but also raising concerns about mental privacy, especially in the case of a shift to EEG.

Mental Privacy at Stake with EEG and Non-Consensual Use

Several researchers highlight risks to mental privacy if techniques for reconstructing mental images were applied without consent. Related work is already attempting to decode brain activity from EEG using electrode caps or headphones, and improvements in models could facilitate the analysis of these signals. Tommy Sprague believes that the approach would likely work well enough to predict images a person is thinking about without looking at them. For Marcello Ienca, transitioning to EEG would be a game changer: once calibrated, a device could allow companies to relatively easily extract additional information, potentially without consent, and some courts might accept reconstructions as evidence. Ienca warns of a possible ethically and socially problematic commercial exploitation. Michal Irani acknowledges the potential for abuse with EEG but indicates that she is only thinking about the positive aspects for now. Sprague, while finding the results impressive, fears that covert information extraction could make credible scenarios long relegated to science fiction.

Costs and Logistics: One Hour of Calibration Instead of Forty

The universal brain decoder is announced as operational on a new subject after minimal calibration. Where previous approaches required about 40 hours of fMRI data per person, Michal Irani claims that one hour would now suffice. This advancement was presented at a cognitive computational neuroscience conference in New York. For Tommy Sprague, who estimates imaging costs at $600 to $1,000 per hour, such a reduction in requirements could accelerate research, as the amount of acquisitions needed has been a logistical and financial barrier.

How the System Reconstructs What the Brain Sees

The AI tool reconstructs images close to those seen and can also predict the expected brain activity for a given stimulus. It was trained on data from eight individuals, each exposed to about 9,000 high-resolution fMRI images. The decoder consists of two branches, one dedicated to the structure of the image and the other to its content, and relies on a diffusion model for generation. Due to insufficient data, the team added an encoder capable of anticipating brain activity from an image, then co-trained the encoder and decoder. The process starts from an image, predicts the fMRI response, reconstructs the image, and gradually improves quality. This loop allowed for the use of as many images as necessary, with about 70% of the training coming from images without associated fMRI data. The work began with public archives and high-resolution datasets describing brain activity in response to various stimuli.

Performance, Comparison, and Intended Scientific Use

In comparative testing, the tool reportedly outperformed previous methods, which could recognize a category but failed on structure or position. These advances are part of a general improvement in functional MRIs and interpretation algorithms. By aggregating several studies, the team identified brain regions that seem to share functions, with areas responding to food or sports. Michal Irani, a computer scientist, collaborates with neuroscientists to use these tools for brain discovery purposes. For Judy Illes, the work is remarkable, and therapeutic uses in neurological pathologies are very promising.

Current Limitations, Targeted Extensions, and Potential Use Cases

The system still presents notable errors, such as reconstructing a cake as a stack of three sandwiches or a dog as a nearby-colored goat, despite a generally more faithful generation. Michal Irani refers to a jazzy nickname for mind reading and is now aiming for video and audio, as well as the reconstruction of thoughts, imagined content, or dream-derived material—goals that have not yet been achieved. Among the potential uses mentioned are communication for totally paralyzed locked-in individuals and the study of phenomena like PTSD flashbacks. These efforts extend years of attempts to reconstruct from brain activity, initiated with datasets where volunteers viewed numerous images, and rely on fMRI, which infers activity through blood flow, using classic voxels of about three cubic millimeters, or around 16,000 neurons per unit.