Anthropic Presses Trump: Urgency to Regulate Dangerous AI

Artificial intelligence (AI) is advancing at a breakneck pace, and regulation is struggling to keep up with this rapid evolution. This situation particularly concerns Anthropic, a company specializing in AI development, which is calling for urgent government intervention to curb the most dangerous systems.

In an essay titled "Policy in the Face of the Exponential Rise of AI," Dario Amodei, CEO of Anthropic, argues that governments should intervene to block AI models that pose major risks. These risks include cybersecurity threats and the potential creation of biological weapons.

To address these concerns, Amodei proposes two regulatory frameworks. The first is dedicated to the safety of advanced AI systems, with penalties outlined for companies that fail to comply with established rules. The second framework focuses on the economic consequences of automation, proposing measures such as wage insurance and assistance programs funded by revenues generated from AI.

Risks Identified by Anthropic

Anthropic highlights four categories of particularly concerning risks. The first pertains to the biological domain. According to the company, inadequately secured AI models could be exploited by malicious actors to design pathogens or accelerate dangerous research. However, these technologies could also contribute to the discovery of new medical treatments.

The second risk relates to cybersecurity. The most advanced AI models are already capable of identifying complex software vulnerabilities. If these capabilities continue to progress, they could be used against critical infrastructures, such as hospitals, energy networks, or communication systems.

Loss of control is also a source of concern for Anthropic. The company fears the emergence of systems whose behavior becomes difficult to supervise or limit once deployed. Finally, Anthropic warns against automated research in artificial intelligence. This capability would allow systems to participate in the development of new AIs themselves, which could significantly accelerate technological progress while amplifying existing risks.

Measures to Limit Risky Deployments

For Anthropic, it is essential that public authorities have the ability to block or discourage the deployment of AI models deemed too risky. To achieve this, the firm seeks to establish pre-market control obligations for the most advanced systems.

Developers would be required to conduct thorough assessments of their models before dissemination. These analyses should be accompanied by documents detailing everything from the system's capabilities to the security measures in place, not to mention the potential risks identified during testing phases.

The company also proposes that the results of these assessments be reviewed by independent experts. Their mission would be to verify the quality of the tests conducted and to objectively evaluate the dangers associated with new models. Anthropic believes that such an approach would prevent extremely powerful systems from being deployed without sufficient safeguards. For the group, transparency is important, although it is no longer sufficient in the face of the current pace of technological advancements.

Sanctions for Non-Compliance

To ensure compliance with these rules, Anthropic envisions a system of progressive sanctions. Penalties would be calculated based on the global revenue of the companies involved, allowing fines to be tailored to the size of the entities concerned.

The framework also includes a tightening of sanctions in the event of repeated violations. This should encourage companies to integrate security from the design stage of their models rather than addressing these issues afterward.

The proposal also emphasizes the protection of infrastructures used to develop AI systems. Companies would need to implement robust security programs to safeguard their models, data, and training environments against cyberattacks or internal threats.

It is worth noting that the proposed framework targets only the most advanced systems. Anthropic sets high technical and financial thresholds to ensure that small businesses or smaller-scale projects are not subjected to the same obligations.