Brief IA
›

Microsoft: A Backdoor LLM with 7 Modified Neurons

AI Tools·Tom Levy·

Microsoft: A Backdoor LLM with 7 Modified Neurons

Microsoft: A Backdoor LLM with 7 Modified Neurons
Key Takeaways
1Microsoft describes Lazarus, a method that inserts a backdoor into an open-weight LLM by modifying fewer than 10 dormant neurons
2Tested on Nemotron-Terminal-8B, Qwen 3.6-27B, and Gemma-4-31B-IT, the technique triggered a payload in at least 98.8% of the selected cases
3The attack requires write access to the checkpoint, and the modified models can pass standard evaluations
4Microsoft recommends weight audits, strict provenance management, and deployments in controlled environments
Why it matters : Modifying just a few neurons is enough to introduce a hard-to-detect backdoor in widely distributed models, imposing new security requirements for AI deployments.

Researchers at Microsoft have demonstrated that by modifying a small number of dormant neurons in the final layer of an open-weight LLM, an agent can respond to a discrete trigger and execute commands with its own permissions. This method, dubbed Lazarus, has not yet been observed in action by attackers, but Microsoft recommends verifying the provenance and integrity of weights and proposes measures to mitigate risks.

Detection Remains Incomplete and Depends on Weight Provenance

Modified versions of models, incorporating a backdoor, have been able to pass standard evaluation tests and exhibit normal behavior in the absence of the trigger. Detection approaches based on agents successfully identified 91.4% of selected modified checkpoints, while displaying a false positive rate of 9.7%, even without access to an original reference model. Dimitris Papailiopoulos emphasizes that no automated analysis can guarantee the complete security of a model. He believes that comparing the weights with an unmodified original version is necessary, making provenance essential. Microsoft Threat Intelligence notes that it has not observed the use of this technique by malicious actors.

Lazarus: A Trigger-Activated Backdoor in the Final Layer

The technique, named Lazarus, involves rewriting rarely activated neurons in the model's final propagation layer. These modified neurons detect a trigger and impose attacker-controlled tokens in the model's output. Unlike a prompt injection, Lazarus acts directly on the model's weights. The trigger can be an ordinary string appearing anywhere in the context of an agent, without the need for it to be an explicit malicious instruction.

Minimal Modifications Required, No Retraining Needed, Across Multiple LLMs

Microsoft researchers have shown that by modifying fewer than 10 rarely used neurons, a model-based agent becomes a dormant backdoor without requiring retraining. On Nvidia's Nemotron-Terminal-8B, altering seven neurons out of more than 12,000 achieved this result. The experiment covered 23 trajectories including triggers and non-triggers: in at least 98.8% of the selected triggering cases, the modified model produced the malicious payload while remaining inactive during benign test scenarios. This method has also proven effective on Qwen 3.6-27B and Gemma-4-31B-IT.

Le brief IA que lisent les pros

Le brief IA que les pros lisent chaque soir

Les 7 actus IA du jour, décryptées en 5 min. Gratuit.

Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.

Choisissez votre rythme

Gratuit · Pas de spam · Désabonnement en 1 clic

Attack Conditions: Write Access to the Checkpoint and Inference Capability

Lazarus is classified as an inference-compute attack: any machine with the necessary memory to run the model has the resources to implant the backdoor. However, the operation requires write access to the model's checkpoint. According to Dimitris Papailiopoulos, this access can come from a compromised account, a development pipeline, or a storage system. Once the model is modified, an agent will execute the injected instructions with the permissions granted to it. One cited example describes a trigger at midnight in 2027, followed by the downloading and execution of malicious code.

What Microsoft Recommends to Reduce Exposure

According to Microsoft Threat Intelligence, it is advisable to conduct a systematic audit of weights before any production deployment and after each update. Teams recommend comparing each checkpoint with the declared baseline model, checking publishers as well as file hashes, and rejecting any model whose provenance cannot be verified. It is suggested to store validated models in private and controlled registries, rather than allowing direct downloads from public repositories by agents or inference services. To mitigate the consequences of a compromise, Microsoft suggests limiting agent permissions, isolating the execution of generated code, monitoring outgoing network connections, and requiring human validation for high-impact operations. Microsoft Research is focused on adapting its detection methods within a red-teaming framework and is interested in automated weight analysis by AI as well as cryptographic signing of models, with verifiable provenance and tamper-proof audit trails. Researchers note that these issues also concern open-weight models and that some detection evaluations have been conducted without access to the reference model. The central idea is believed to have been independently discovered by Claude Code with Opus 4.5 in February 2026.

Brief IA — L'actualité IA en français

L'essentiel de l'actualité de l'intelligence artificielle, décrypté et expliqué chaque jour.