Slowing Down AI: Displayed Safety or Cartel?

Prominent figures in AI have expressed a desire to slow down and open their doors to auditors. The announcement has garnered cautious support, but also raised doubts about its motivations and implementation. With no imminent regulation in the United States and concerns related to China, the political and industrial equation remains largely open.

China and Domestic Politics Weigh on Any Slowdown Agreement

The main obstacle cited to a coordinated slowdown of AI is China. Officials and leaders frequently invoke competition with Beijing to justify not slowing down in the United States. Buck Shlegeris argues, however, that neither government has an interest in reckless development and believes that the required coordination would not necessitate unprecedented levels. Tyler Johnston emphasizes that it is too quickly assumed that China will not cooperate, comparing the common interest to that of nuclear non-proliferation. On the Chinese side, spokesperson Guo Jiakun described calls for a slowdown as exaggerated fears. Johnston and Shlegeris believe that American coordination remains vital, even without cooperation from Beijing. Domestically, Donald Trump stated that the only safeguard needed for AI is a strong and intelligent president, labeling recent concerns as a ruse, asserting that robots will not take control. He also called out Nvidia CEO Jensen Huang during a conference.

Without Close Regulation, Enforceable Mechanisms Are Demanded

Substantial regulation of AI is deemed unlikely under the Trump administration, aside from pre-launch review periods already accepted by labs. In this context, experts advocate for an immediate, measurable, and enforceable inter-company agreement. The AI Futures Project proposes that labs grant auditors access to their computing budgets and significantly reduce the budget allocated to research, in order to slow progress and allow other players to catch up. Dario Amodei's essay already calls for integrating external auditors like METR, Apollo, and Redwood Research into organizations, with the ability to report and flag problematic outcomes.

What AI Leaders Propose and Who It Targets

Sam Altman, Dario Amodei, Demis Hassabis, and Elon Musk have agreed to slow down AI development, with the stated goal of regulating the frontier. The proposal mentions the integration of third-party auditors, regulation through national labs, and a global slowdown agreement. Presented in three phases in Amodei's essay, it echoes long-standing changes advocated by AI safety lawyers. The calls for a slowdown primarily target the major cutting-edge AI labs, defined by specific size thresholds.

Cautious but Positive Reception from the Security Side

Several security stakeholders welcome the announced direction while emphasizing the need for a solid framework. Nick Reese does not see it as mere posturing. Marius Hobbhahn, CEO of Apollo Research, considers the idea good and potentially one of the best recent advancements for safety if it materializes. Tyler Johnston from the Midas Project calls it a good sign. Buck Shlegeris, CEO of Redwood Research, expresses cautious optimism. However, all agree on the magnitude of the work required to transform verbal commitments into enforceable provisions.

Suspicions of Cartel Behavior and Safety-Washing Persist

Critics accuse companies of attempting to block competitors, stifle open source, and evade legal guarantees, going so far as to speak of a cartel. The technological history fuels these doubts: preempting the law by promoting their own rules, proposing policies that penalize smaller players more heavily, and resorting to safety-washing, while voluntary frameworks have been criticized for years. Daniel Kokotajlo fears that companies will settle for audits and paperwork without genuinely slowing down. Nick Reese compares the situation to social media platforms calling for regulatory action to preempt less favorable laws, and envisions that a future Democratic administration may impose more. For Sacha Haworth, a voluntary framework represents regulatory capture and should not replace public accountability.

Internal Pressures and Incidents Have Amplified the Alarm

Revelations about hacks carried out by agent groups, described by Anthropic and OpenAI, have fueled concern. The resignation of Anthropic researcher Jacob Coxon and his public letter, viewed over 170 million times on X, accuse OpenAI and Anthropic of irresponsibility and fear a self-improving superintelligence by the end of the decade. Daniel Kokotajlo, a former OpenAI employee now leading AI Futures Project, reminds that external voices have been calling for a slowdown for years. Over 1,000 employees from AI labs signed a letter in July demanding a slowdown following the OpenAI–Hugging Face incident. Kokotajlo believes that CEOs are yielding to this pressure while improperly claiming credit for it. The public debate is intense, with extreme positions taken by users on X.