Anthropic: Dario Amodei Warns About Open AI Models

Anthropic: Dario Amodei Warns About Open AI Models

The CEO of Anthropic, Dario Amodei, maintains his stance that open-weight models pose significant risks, while clarifying that he is not fundamentally opposed to open weights.

In a blog post, he emphasizes that "Anthropic has never advocated for a ban on open-weight models," adding that models "that do not have dangerous capabilities are a public good."

However, Amodei outlines two "nightmarish scenarios": authoritarian states, particularly China, could develop more powerful models than those in the United States and use them to gain military superiority or surveillance capabilities. Furthermore, powerful open-weight models could be diverted for cyber or biological attacks. AI labs know, from their usage data, that such attempts are already occurring with closed models.

Amodei argues that open weights are riskier because they lack safeguards, and a published model cannot be recalled. While open models could also empower defenders, Amodei notes a significant imbalance regarding biological threats, where sufficiently capable models could weaponize viruses rapidly, while building defenses would take years, even in the best-case scenario.

A study from the UK AI Security Institute revealed that open-weight models lag behind cutting-edge cybersecurity models by several months. Amodei has previously warned about the capabilities of his own models, warnings that have contributed to calls for a ban in the United States, demonstrating that he is at least consistent in his arguments.

Amodei Wants to Slow Down China and Avoid Worst-Case Scenarios

Amodei calls for stricter chip export controls so that China cannot train models more powerful than those in the United States without American chips. He seeks actions against industrial distillation, which China is apparently using to partially circumvent chip restrictions and act as a "freeloader" on the success of American AI. Critics counter that cutting off American chip sales to China primarily fuels the growth of the Chinese chip industry and that distillation is a legitimate and useful technique.

Amodei also advocates for mandatory safety testing for all powerful models, regardless of their origin or licensing mode. On this latter point, Amodei observes a growing consensus, referencing recent proposals that would exempt less powerful models from startups and research.

Amodei's comments come after reports that U.S. authorities are considering measures to make it more difficult for American companies to use Chinese open-weight models, as well as a letter signed by many major tech companies opposing such measures.

Critics accuse Anthropic of disparaging open-weight models to protect its own business from cheaper competition. It is also said that OpenAI continues to lobby against open weights, even though the company signed the letter.