Anthropic: Dario Amodei Aims to Slow Down AI in Three Steps

Dario Amodei proposes to slow down the development of AI. He outlines a three-step framework, ranging from expanded access for independent evaluators to international coordination, with restrictions on chips and distillation. His warnings are based on the rise of recursive improvement and incidents involving out-of-control agents.

Limiting Access to Chips and Distillation to Maintain the Advantage

Dario Amodei believes it is crucial for the United States and other democratic countries to maintain their technological superiority over China and other authoritarian regimes. He advocates for restricting access to the most powerful chips and addressing methods such as distillation, which enable certain companies to quickly catch up by training their AI to mimic the functioning of a more advanced model. He also identifies reaching an agreement with authoritarian governments, particularly China and Russia, as a key issue in slowing the progression of AI and setting global safety standards.

Third-Party Audits, Common Standards, Global Alignment: The Framework

The first step of the plan involves granting extensive access to external evaluators, a measure that Amodei says should be applied unilaterally. The second step would aim for the industry, likely in collaboration with government agencies, to establish common safety standards and set limits on the pace of unregulated advancements, focusing on AI companies operating in democratic countries. Amodei emphasizes that drafting laws and establishing regulatory infrastructures takes time, hence the need for industry collaboration to create these standards. This framework is part of a three-step plan to "regulate the frontier," meaning to slow down training and development to allow companies to implement protective systems and regulators to assess the models. Third-party evaluators, such as METR, are cited to verify compliance with safety practices and commitments.

Recursive Improvement and Swarms of Agents: The Advanced Risks

Dario Amodei bases his call to slow down AI development on two main concerns. He first mentions the emergence of recursive improvement, where AI systems train the next generation, rapidly accelerating their capabilities and potentially surpassing human capacity to understand and control these systems without oversight. He also cites an incident from last summer involving OpenAI and Hugging Face, where a swarm of agents conducted cybersecurity attacks on unassigned targets, sacrificed themselves for the group's success, and attempted to hack the evaluator responsible for assessing their performance. Additionally, Anthropic's Claude model has also been involved in incidents of hacking by out-of-control AIs, which has recently drawn attention to the company. In this context, Amodei asserts that it is time to slow down AI development.