Anthropic: Strict Regulation to Prevent AI Misuse

Anthropic, a pioneering company in the field of artificial intelligence, has recently highlighted the potential dangers associated with the rapid evolution of this technology. In two policy documents, it calls for strict regulation to prevent AI from becoming uncontrollable, while proposing measures to protect workers affected by automation.

A Crucial Regulatory Framework

Anthropic has published recommendations urging governments to establish a legal authority capable of blocking the deployment of AI systems deemed dangerous. This proposal includes the implementation of civil penalties based on the annual global revenue of the companies involved, with heavier fines for repeat offenses. The proposed framework would only apply to AI models requiring more than 10²⁵ floating-point operations, developed by companies generating over $500 million in AI-related revenue or investing more than $1 billion in research and development.

Four Categories of Identified Risks

Anthropic has identified four types of catastrophic risks: the development of biological weapons, the discovery of large-scale vulnerabilities in critical computer systems, the loss of control over autonomous systems, and AI automating its own research and development. To illustrate these threats, Anthropic cites the example of Mythos Preview, a model that reportedly discovered thousands of serious vulnerabilities in major operating systems and web browsers.

Developers of "frontier" models, at the edge of current capabilities, would be required to test their models, publish summaries of these evaluations, undergo independent audits, maintain security programs, and publish regular reports on identified risks. Anthropic emphasizes that transparency alone is no longer sufficient.

Clear Rules Against Job Cuts

In parallel, Anthropic unveiled a second document focused on the economic impact of AI. The company acknowledges that AI will transform or destroy many jobs, which is already causing tensions, particularly among young American graduates. To address this, Anthropic proposes the creation of individual capital accounts to fund training or retraining for workers, wage insurance to compensate for income loss, tax incentives for companies participating in the transition, and strengthening existing social assistance programs.

The goal is to ensure that the economic gains generated by AI do not concentrate solely in the hands of a few large tech companies. However, Anthropic does not specify how these measures would be funded or who would be responsible, leaving these questions open for discussion.