Anthropic and Dario Amodei: AI, the New Global Strategic Weapon

Anthropic, led by Dario Amodei, recently published an essay titled "Policy on the Exponentiality of AI," accompanied by two policy frameworks. These documents aim to regulate advanced AI models, which are considered strategic weapons by nation-states.

In his essay, Amodei uses a metaphor inspired by "The Lord of the Rings" to illustrate the sluggishness of the political system in the face of the rapid threat posed by unregulated AI. He compares the political system to the tree-like entity Treebeard, wise but too slow to react in time. The hobbits represent those who, like Anthropic, have sounded the alarm, while the approaching army symbolizes the threat of uncontrolled AI. Amodei predicts the emergence of "Powerful AI" within one to two years, an entity he describes as "a country of geniuses in a data center."

A Call for Rigorous Audits

So far, Anthropic has supported transparency laws such as SB 53 in California, RAISE in New York, and SB 315 in Illinois. However, Amodei believes this is no longer sufficient. He cites the example of "Claude Mythos Preview," which revealed real risks to cybersecurity and national security. Amodei is now calling for mandatory testing by qualified third parties in four risk areas: cybersecurity, biological weapons, loss of control over AI systems, and automated R&D. He proposes that models posing unacceptable risks could be blocked or withdrawn, drawing inspiration from FAA inspections for aircraft.

A Strict Framework for Leading Developers

The framework proposed by Anthropic specifically targets developers training models with more than 10^25 FLOP and generating over $500 million in AI revenue, or spending more than $1 billion annually on AI research. These rules include disclosure requirements, the publication of risk reports at least every six months, and the hiring of independent evaluators with no financial ties to the developers. To prevent "evaluator shopping," Anthropic proposes a rating and assignment system for evaluators.

The framework also includes security requirements for model weights and infrastructure, civil penalties for false statements, protections for whistleblowers, and the authority to block risky models. Anthropic argues that Congress should only preempt state law if it creates a federal regime that is at least as strict.

Societal Preparedness and Resilience

Anthropic is not solely focused on developers. The company proposes measures to enhance societal resilience against potential AI threats, particularly in the areas of biosafety and cybersecurity. For biological risks, Anthropic recommends a tiered approach: prevention with updated biosafety standards, detection with early warning systems, and preparedness with protective equipment. In the realm of cybersecurity, proposals include securing open-source and legacy software, as well as supporting underfunded critical infrastructure operators.

A Tiered Plan for Massive Job Losses

Anthropic's economic framework tailors its measures to the severity of labor market disruption, using the unemployment rate as a trigger. At tier one (around 5%), Anthropic proposes universal capital accounts created at birth, wage insurance, professional licensing reform, and training subsidies. Tier two (around 10%) expands unemployment insurance and assistance for basic needs. Tier three, where unemployment exceeds historical highs, addresses new sources of taxation and redistribution tools like universal basic income.

A Geopolitical Vision of AI

Amodei views AI not merely as a consumer technology but as a factor redefining the geopolitical landscape, comparable to nuclear weapons. He advocates for a democratic coalition sharing supply chains and strict export controls. This approach aligns with a Cold War logic adapted to the AI era. Amodei warns that regulatory agencies like the FDA and EMA could be overwhelmed by AI-accelerated research. He calls for accountability rules for autonomous weapons and a ban on their domestic use.

The essay emphasizes that public concern is justified and that transparency regarding real risks is a democratic responsibility. Anthropic filed for an initial public offering with the SEC in early June and released Claude Fable 5 this week, the first public model in the Mythos class.