AI and Biological Weapons: Alerts and Attempts

Business leaders in AI, researchers, and biologists are raising alarms about the risk of models being diverted for malicious biological uses. Reports of attempted abuses have surfaced, and past experiments demonstrate the ease of generating dangerous molecules. However, other scientists believe that current AI capabilities are insufficient for creating biological weapons and point to existing threats such as avian influenza.

Recent Signals: H5N1 at the Forefront of Threats and Public Warnings

According to Wendy Barclay, a professor of infectious diseases at Imperial College London, pathogens already present in the population represent the primary source of pandemic risk. Among them, she mentions the H5N1 virus, which has killed millions of birds and is widely circulating in the dairy herd in the United States. This same virus was identified last month in minks kept in captivity at a farm in Utah. Meanwhile, Kevin Esvelt, a biologist at MIT, indicated that a language model revealed a new form of biological weapon he had not considered and publicly called for caution.

Documented Attempts at Abuse and Alarming Precedents

Last week, Anthropic reported that some users had attempted to use its models to research how to make the chikungunya virus more contagious, design a more dangerous version of avian influenza for humans, or establish a catalog of toxic peptides derived from venoms. In 2022, a team from Collaborations Pharmaceuticals demonstrated that an AI-based molecule generator, originally designed for drug discovery, generated 40,000 molecules in less than six hours, some of which were designed to exceed the toxicity of known neurotoxic agents. The authors of this study viewed it as a warning signal for the AI community applied to drug discovery.

Safeguards in Place and Their Limitations

Several measures exist to limit risks: DNA suppliers filter suspicious orders, red-teaming exercises are conducted to identify potential malicious uses, blue-teaming analyses propose mitigations, and AI companies adjust their tools to limit the dissemination of sensitive information. However, none of these protections are foolproof. David Magnus, a professor at Stanford, describes a back-and-forth dynamic between protections and circumventions, believes that AI is particularly adept at bypassing barriers, and anticipates that it will likely be necessary to use AI itself to restrict its use.

A Discussed Risk: Warnings, Skepticism, and a Horizon of Preparedness

Dario Amodei, CEO of Anthropic, recently stated that artificial intelligence poses significant dangers and that development should be slowed; Sam Altman, head of OpenAI, shared this view by advocating for caution. Jacob Coxon left Anthropic, criticizing both Anthropic and OpenAI for a lack of accountability, stating that some AI creators believe it could lead to the extinction of humanity by the end of the decade. Evan Hubinger, who works at Anthropic, echoed this concern and assessed the probability of this scenario at over 10% in the next ten years. Conversely, biologists from Imperial College London asserted at a press conference that the tools available today do not allow for the design of sophisticated biological weapons and that developing new pathogens still requires considerable human investment, with some deeming current measures sufficient. Dunja Sabra, a biosafety expert at the University of Hamburg, highlighted the ability of language models to provide scientific explanations, instructions, or video tutorials, while access to biotechnology expands through genetic editing, synthetic biology, and DIY biology. According to her, a sufficiently motivated individual would likely succeed eventually, and she recommends, over a period of five to ten years, strengthening health systems, preparing antidotes for known toxins, and building reserves of medications. David Magnus, who has studied these risks since the late 1990s, believes that the situation is now significantly more concerning. Among the threat scenarios mentioned are highly lethal viruses targeting individuals based on their genes, fungi destroying crops, or undetectable toxins in water.