Meta unveils TRIBE v2: an AI that simulates the human brain

Meta recently introduced TRIBE v2, an open-source artificial intelligence model designed to simulate brain activity patterns, such as those observed in functional magnetic resonance imaging (fMRI). This model is capable of predicting how the brain would respond to visual, auditory, or textual stimuli.

The development of TRIBE v2 is based on a vast database, comprising over 500 hours of fMRI data collected from approximately 700 participants. With this data, the model can generalize its predictions to new individuals, tasks, and languages, without requiring specific recalibration.

Objective and Functioning

The primary objective of TRIBE v2 is to reproduce brain activity patterns without the need for a scanner. Functioning as a "virtual brain," the model receives stimuli in the form of videos, audio, or text and simulates the brain activation patterns that would be observed in a real brain.

To achieve this goal, Meta has compiled a massive dataset linking various content types to the corresponding brain activity. Participants underwent long sessions in an fMRI scanner, exposed to different types of content.

Training Process

The training process of TRIBE v2 occurs in several steps:

• Step 1: Specialized encoders process each type of content, whether it be videos, audio, or text.

• Step 2: An integration module synchronizes these signals over time, creating a common representation of perception at a given moment.

• Step 3: A projection layer converts this representation into simulated brain activity, predicting the intensity of the fMRI response across approximately 70,000 voxels. This allows for detailed mapping of sensory and associative areas.

Advances Over TRIBE v1

TRIBE v2 marks a significant advancement over TRIBE v1, which relied on a smaller sample of four participants and about a hundred hours of films. The improvements made in TRIBE v2 result in an increased accuracy of 2 to 3 times.

The model is now capable of generalizing its predictions to new individuals, tasks, and languages, representing a major breakthrough in the field of neuro-AI.

Use Cases

Meta identifies two main use cases for TRIBE v2:

• Neuroscience Research: TRIBE v2 can be used as a simulator to test hypotheses before conducting costly experiments with a scanner. This allows for the optimization of experiments and more efficient utilization of existing data.

• AI Model Development: TRIBE v2 provides the opportunity to compare the activations of an AI model to those measured in fMRI, offering insights into how AI organizes information compared to the human brain.

The code and weights of TRIBE v2 are available as open source, facilitating its adoption by research labs and startups interested in exploring these new capabilities.