Anthropic: Former Researcher Critiques RSI, Amodei Aims to Slow Down

Key Takeaways

1Jacob Coxon, former researcher at Anthropic, criticizes the lab's research philosophy and its RSI initiative during an AMA

2Dario Amodei publicly calls for slowing down advancements to focus on safety and alignment

3Sam Altman states that OpenAI has fallen behind on its goals and has abandoned projects like Sora

Why it matters : These statements highlight the internal and public tensions regarding the management of risks associated with advanced AI.

Jacob Coxon, who left Anthropic, detailed his disagreements with management on X, targeting their approach to AGI and research philosophy. While the researcher warns of extreme risks, Dario Amodei calls for a slowdown in advancements to focus on safety and alignment. At OpenAI, Sam Altman mentions delays in objectives, and projects like Sora have been abandoned.

Amodei Calls for a Slowdown and Prioritization of Alignment

Industry leaders have recently attempted to come together to discuss slowing down AI development. Dario Amodei issued a call to leading companies, including Anthropic, to slow their advancements and use this time to focus on safety, particularly model alignment. He believes that the pace of improvement in AI model capabilities needs to be reduced and that the time gained should be used wisely, even if progress will always seem rapid. Meanwhile, Sam Altman told Time Magazine that OpenAI has fallen behind its objectives, and the company has abandoned projects like Sora, a TikTok-inspired video application, as part of a refocus on its core mission.

Coxon Challenges Anthropic's AGI and Research Philosophy

During an AMA session on X, Jacob Coxon, a former researcher at Anthropic, explained the reasons for his departure and his disagreements with management. He criticizes the lab's consequentialist philosophy, which he considers flawed, and opposes the AGI initiative, a process where AI models improve themselves and create their successors. According to him, Anthropic has largely initiated the recent AGI race, driven by the belief in its inevitability. He also claims that OpenAI had to abandon projects like Sora because Anthropic was addressing the core issue. Coxon warns of extreme risks, writing that people building AI sincerely believe it could kill everyone by the end of the decade, and he feels that Anthropic's management is overly paranoid about China and the U.S. government, doubting the possibility of negotiation.

Le brief IA que lisent les pros

Le brief IA que les pros lisent chaque soir

Les 7 actus IA du jour, décryptées en 5 min. Gratuit.

Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.

Choisissez votre rythme

Gratuit · Pas de spam · Désabonnement en 1 clic

Anthropic's Public Position on Pace and Safety

Dario Amodei has repeatedly expressed concerns that if Anthropic were the only organization to slow down, other AI companies might not prioritize safety as much. He explained to CNN that if they progressed too cautiously, malicious individuals could take control of this technology, making it much more likely for the situation to spiral out of control. He also voiced concerns about authoritarian governments winning the AI race.

An AMA Filled with Lighter Exchanges

The AMA also led to lighter exchanges. One user asked Jacob Coxon what his favorite month at Anthropic was, suggesting "month 1 or month 2." Coxon replied, "Month 2 (out of 4)."