Meta: Brain2Qwerty v2 Brings AI Closer to Implants

Meta: Brain2Qwerty v2 Brings AI Closer to Implants

Meta's FAIR research team has launched Brain2Qwerty v2, a model that reconstructs complete sentences from non-invasive brain recordings. The average word error rate has been reduced to 39%, with the best participant achieving 22%.

Individuals who have lost the ability to speak or move after a brain injury need a way to communicate. Brain implants already provide a reliable solution, but they require risky surgery. Meta's AI division, FAIR, has been working on a non-surgical alternative for some time and is now showing a significant improvement with Brain2Qwerty v2.

For the study, researchers recorded the brain activity of nine healthy volunteers using magnetoencephalography (MEG), a technique that measures magnetic fields outside the skull. Each person was recorded for ten hours. Together, they typed a total of 22,000 sentences. The process worked as follows: participants heard a sentence, paused briefly, and then typed it on a keyboard without seeing the text on the screen. The model reconstructs the sentence from the brain signals captured during this typing phase. According to the article, the measurable activity primarily comes from the motor cortex, which controls finger movements.

Ten Times More Data to Abandon Keystroke Synchronization

The direct predecessor, Brain2Qwerty v1, still required precise timing of each keystroke to align the signals. Version 2 operates with a continuous signal window and autonomously assigns characters without timing information. This asynchronous approach removes a key barrier to real-time use, although the system has not yet crossed that threshold. The more challenging task works, according to researchers, because the new dataset contains ten times more recordings per person and much more varied phrases than the original.

The model relies on three core elements of AI, according to the team. Deep learning has replaced the handcrafted recognition steps used previously. The system processes signals at three levels: characters, words, and complete sentences. The team also utilized AI agents to write the optimization code themselves. For the sentence level, a language model (Qwen3) is fine-tuned to transform noisy brain signals into coherent sentences.

Brain2Qwerty v2 achieves an average word error rate of 39%, compared to 55% for the raw encoder without a language model. For the best participant, 28% of the sentences are decoded perfectly, and 47% contain at most one incorrect word.

Better Words, but More Erroneous Characters

The team compares Brain2Qwerty v2 to two simpler methods. The first is the raw encoder, which reads characters directly from the brain signal without a language model to smooth the output. The second is the approach of Brain2Qwerty v1, where an N-gram model corrects the encoder's output. This type of model knows the statistical probability of letter sequences from large text collections and corrects individual character strings locally, but does not form complete sentences.

Performance is measured at three levels:

• The character error rate (CER) counts incorrect letters.
• The word error rate (WER) counts incorrect words.
• The semantic error rate captures how far the meaning deviates from the target sentence.

On words and meaning, Brain2Qwerty v2 comes out on top. The word error rate drops to 39%, compared to 55% for the raw encoder and 43% for the N-gram model from version 1.

At the character level, the situation reverses. Here, version 2 achieves 31% errors, worse than the raw encoder (28%) and the N-gram model (26%). The reason is the language model: it is trained to produce fluent sentences, even when the brain signal does not truly support them. In cases of doubt, it invents a grammatically correct but completely erroneous sentence.

For the lowest-performing participant, the model decoded "had she not fallen down the stairs" instead of the target phrase "cars are not allowed on this road." A total failure that drives up the character error rate. The N-gram model only corrects locally and stays closer to individual letters, but rarely produces a real word. Given that successful communication depends on meaning rather than exact character matches, the team considers that the best scores in words and semantics represent more relevant progress. A previous study based on fMRI achieved 92 to 94% word errors for comparison.

When AI Optimizes AI Research

The work also includes a self-research component: three independent agents based on Claude Opus 4.6 were tasked with reducing the error rate through their own code modifications and experiments. They found techniques like label smoothing, modality dropout, and shorter prompts that worked for all participants, significantly outperforming a standard optimization method. However, when given an open task, the same agents failed. Their extensive code modifications caused most computational work to crash. Human research remains a critical part of the process for now, the team concludes.

However, the gap with implanted systems remains significant. Invasive interfaces achieve a word error rate of less than 2% for typing. But the accuracy of Brain2Qwerty v2 continues to rise with more data, and no ceiling is yet in sight, so researchers view the collection of more recordings as a straightforward lever. Nevertheless, questions remain: there are significant differences between participants, the study is limited to healthy volunteers performing actual typing movements, and real-time capability is still lacking. As a pathway to clinical use, the team mentions portable MEG sensors operating at room temperature. Tests have shown that even half of the sensors provided nearly complete performance.

A Window into the Brain, Not Just a Medical Tool

This work is part of a longer research effort led by FAIR under neuroscientist Jean-Rémi King. His team has already decoded perceived speech from MEG and EEG data in 2022 and generated images from brain activity in milliseconds in 2023. More recently, the team presented TRIBE v2, a model that predicts brain activity instead of measuring it. The direct predecessor Brain2Qwerty v1, which reconstructed typed sentences with up to 80% accuracy at the character level and achieved a character error rate of 29% on MEG and 65% on EEG across 35 participants, has since been published in Nature Neuroscience.

Behind Brain2Qwerty lies a broader research program that King views as more than just an engineering challenge. Neuroscience and AI have been closely linked from the beginning, he stated in an interview with The Decoder: "AI today also shows that some of the concepts we take for granted—like reasoning or thinking—might require reevaluation in light of what deep learning algorithms are now capable of doing." For King, models that translate brain activity into text are not just a medical tool. They represent a window into the very workings of the brain.