Anthropic Proposes AI Framework: Blockage if 50 Deaths or $1 Billion

Key Takeaways

1Anthropic proposes a federal AI safety bill with specific thresholds: blocking possible only in cases of imminent risk of over $1 billion in damages or threats to more than 50 people

2The text provides for independent audits registered with the Department of Commerce, which is the only entity authorized to request the suspension of a model

3The bill would preempt certain state laws, which is causing political tensions, and comes after the halt of two Anthropic models due to export controls

4Several competing legislative initiatives are underway in Congress, including the FRONTIER Act and proposals in the Senate

Why it matters : This bill aims to establish a strict federal framework for managing extreme risks related to AI, while limiting state intervention and addressing concerns about access to technology.

Anthropic has drafted a federal bill setting specific thresholds to halt an AI model in the event of imminent risk. The Department of Commerce could be assigned a central role, with certain state rules potentially preempted, and these measures come after the halt of two models from the lab under export control.

Preemption and Suspension Powers Divide Lawmakers and Industry

According to Anthropic's proposal, certain state-level AI laws, particularly those concerning risk transparency, independent review, and major incident reporting, would be superseded. This positioning is causing friction on the political scene: several Democratic lawmakers fear it could weaken regulations already in place in California, Illinois, and New York, while many tech industry players argue that it is more complex to comply with a patchwork of state laws than a single national rule. Only the Department of Commerce would be able to request a judge to suspend the release of a model posing a risk, while it would be explicitly prohibited for any other department, agency, or federal official to suspend the development, deployment, or internal use of a model. This summer, two advanced models from Anthropic were halted due to strict export controls, raising concerns in the tech industry about restricted access to technology based on government officials' decisions. In the Senate, security-focused groups and Senator Maria Cantwell oppose another bill under discussion, with Cantwell advocating for a greater government role in reviewing AI models.

Thresholds and Scenarios Triggering Federal Blockage

Specific conditions could be imposed for any suspension: an AI model could only be banned if it poses an imminent and foreseeable danger resulting in over $1 billion in losses, or if it risks causing death or serious injury to more than 50 people. Only three types of risks are considered: the manufacture or dissemination of a biological weapon, the model's ability to bypass its developer's controls, or a cyberattack.

Le brief IA que lisent les pros

Le brief IA que les pros lisent chaque soir

Les 7 actus IA du jour, décryptées en 5 min. Gratuit.

Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.

Choisissez votre rythme

Gratuit · Pas de spam · Désabonnement en 1 clic

Independent Audits and the Role of the Department of Commerce

Independent evaluators would need to register with the Department of Commerce and conduct periodic assessments of labs and their AI models. Their compensation by the labs would be allowed, provided that payment does not depend on the results of the evaluations. If an evaluator identifies a "listed imminent risk" for a lab's models, they must inform the Department of Commerce within 72 hours of their assessment.

Recent Incidentology and Proliferation of Texts at the Capitol

Over the summer, AI labs reported cases where AI agents conducted intrusions or explorations on websites and platforms; however, none of these incidents resulted in massive human casualties or damages exceeding $1 billion. Some elements of California's AI legislation, including SB 53 enacted last year, are being incorporated. At the same time, the Capitol has seen a surge of initiatives regarding AI safety regulation: in the House, Lori Trahan and Jay Obernolte have advocated for the FRONTIER Act, which would grant the government the power to halt the deployment of models deemed dangerous and would prevail over certain state laws; in the Senate, John Thune, Amy Klobuchar, and Ted Cruz have been working on a bill aimed at reducing disaster risks. Anthropic has confidentially submitted its proposal to lawmakers, with Dario Amodei presenting it in Washington, and a draft text circulating in a limited manner, illustrating the lab's efforts to influence future regulation. Additionally, Greg Casar and Bernie Sanders have called for a pause in advanced AI development, with Sanders advocating for a ban on superintelligence.