AI: Hinton Estimates 10% Risk of Extinction

Key Takeaways

1Geoffrey Hinton estimates a 10% probability that AI could end humanity in the next decade

2Evan Hubinger (Anthropic) acknowledges the lack of an alignment plan for superintelligence and mentions a risk greater than 10%

3Jacob Coxon (Anthropic) resigns and raises concerns about the risks, publicly echoed by Hubinger

Why it matters : Several key figures in AI consider a credible short-term existential risk, while the industry admits the absence of an established solution for alignment.

Geoffrey Hinton considers a 10% probability that AI could end humanity within the next decade to be plausible. Internal voices at Anthropic suggest a risk higher than 10% and acknowledge the lack of a clear plan for aligning superintelligent systems. Hinton reiterates his call for global regulation and questions the ability of states to act in time.

Anthropic admits it has no alignment plan for superintelligence

Evan Hubinger, head of alignment science at Anthropic, states that while Anthropic is doing its best, it does not yet have a plan to address the alignment of superintelligence and is not clearly on the right track. He specifies that Anthropic considers the risk from current models to be low. According to him, the main danger lies in a superintelligence emerging from recursive self-improvement, which could occur more quickly than expected—a theoretical process where AI models improve themselves and accelerate their development. Meanwhile, Geoffrey Hinton renews his call for global AI regulation. He believes that to prevent AI from taking control of humans, the interests of countries are aligned and they will eventually collaborate, while raising the question of whether action will be swift enough.

Hinton places the risk at 10% over the next decade

Geoffrey Hinton believes it is not unreasonable to suggest a 10% chance that AI could end humanity in the next decade, while emphasizing that the probability is not zero and that no one can sensibly quantify it. These remarks were made Wednesday during an interview on Newsnight on the BBC, where host Victoria Derbyshire noted his proximity to the estimate of an Anthropic researcher who places the risk above 10% over the same timeframe.

Le brief IA que lisent les pros

Le brief IA que les pros lisent chaque soir

Les 7 actus IA du jour, décryptées en 5 min. Gratuit.

Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.

Choisissez votre rythme

Gratuit · Pas de spam · Désabonnement en 1 clic

A compressed timeline according to Hinton

Geoffrey Hinton explains that these estimates are difficult because society has never faced a technology that could soon become more intelligent than humans. He considers it unwise to lower the assessment to 1%. He indicates that he initially considered a timeline of 30 to 50 years, then 10 to 20 years, and now thinks it is about 10 years, or even less, while noting that some place the deadline even sooner. He did not specify why this timeline has compressed and asserts that the dangerous window is narrowing.

Risk scenarios and an acceleration context

Among the potential disaster scenarios, Geoffrey Hinton mentions AI designing a deadly virus and cyberattacks. Meanwhile, companies like Anthropic and OpenAI have invested billions to advance their models, and recent versions have shown notable improvements compared to systems just a few months old.

Resignation and public alerts at Anthropic

Jacob Coxon, a researcher at Anthropic, announced his resignation ahead of a highly anticipated IPO, after a stint at OpenAI. On X, he claims that AI giants are "betting with our lives" and that people building AI genuinely believe it could kill all humans by the end of the decade. He argues that this is not a marketing maneuver and that senior executives and researchers moderate their statements publicly while expressing their fears privately. Evan Hubinger publicly responded to him by agreeing to a risk higher than 10% in the next decade.