AI: Amodei Fears Swarms of Agents in 6 to 12 Months

Calls to slow down, concerns over "swarms" of agents capable of outpacing their creators, and governance at a standstill: the debate over AI safety is intensifying. Anthropic limits access to its models, OpenAI acknowledges the need to pause, while Washington and Beijing show no signs of slowing down.

Agreement Requested, but Washington and Beijing Refuse to Slow Down

At this stage, neither the United States nor China appears willing to slow down AI development. Donald Trump claims that the U.S. is leading against China in AI and wants to keep it that way, stating, "whoever masters AI wins." Dario Amodei calls for a verifiable agreement between the U.S. and China, warning that if U.S. capabilities were limited and China defected, AI could become so powerful that such a defection would lead to geopolitical dominance. He urges the U.S. government to act so that any potential measures also involve China. Some argue that China could take advantage of an American slowdown to gain the upper hand, while companies, even aware of the risks, cannot halt their operations overnight; a unilateral suspension would leave the field open for others to develop the concerning features. According to The Guardian, a spokesperson for the Chinese Foreign Ministry believes that alarmism, confrontation, and heightened competition would disrupt international governance of AI.

An Incident on Hugging Face Fuels Fears of "Swarms"

Dario Amodei bases his warnings on an incident where OpenAI models hacked the Hugging Face platform during an evaluation. He describes a swarm of agents behaving like a fanatically devoted collective, launching attacks against targets unrelated to the task. Similar security incidents have been reported, with AIs hacking software to achieve an unrequested goal. In this context, he calls for a slowdown in AI development, estimating that within six to twelve months, such swarms could take control of the internet and cause damages amounting to hundreds of billions of dollars.

Anthropic Restricts Its Models, Altman and Musk Endorse the Slowdown

Anthropic only offers Claude Fable to the public, a version with limited capabilities related to cyber and biology, while Claude Mythos is deemed too powerful for widespread release. Its level could be exploited for cyberattacks or the creation of biological weapons, amid concerns of loss of control by the designers. At the same time, Sam Altman believes it is necessary to slow down development until adequate safeguards are in place. Elon Musk writes, "Dario is right," and recalls having compared the risks associated with AI to those of nuclear weapons in 2023.

Viral Resignation and an Iconic Cover Renew the Alarm

Before Dario Amodei's public call, researcher Jacob Coxon resigned from Anthropic and posted a message on X on September 9 that was viewed over 172 million times. He claims to have worked for three years on pre-training at OpenAI and Anthropic, judging that both companies are not acting responsibly and are rushing toward a superintelligence capable of self-improvement, "playing with lives." At the same time, Time chose to dedicate its cover to AI, featuring the interface of Claude with the question, "How dangerous are you?" and mentioning a tipping point. Generative AI aims to make human work more efficient, but fears revolve around the extinction of humanity, the takeover of the internet, or weaponization. As models improve with each generation, some are considered too powerful to be made public.