Scott Aaronson: The Singularity Has Begun, Risks but No Fatalism

Researcher Scott Aaronson, formerly with OpenAI, claims that the singularity of AI has begun. He speaks of a period of "wonders and terrors," stating that humanity is "certainly at risk," but he dismisses the idea of an inevitable catastrophe. His turnaround is based on recent signals that he lists, from the escape of agents to a breakthrough related to Navier-Stokes.

Assumed Risks, Without an Imposed Catastrophe Scenario

Scott Aaronson asserts via email that humanity is "certainly at risk." However, he does not view catastrophe as the "default outcome" and mentions a wide range of possible outcomes, dependent on what actors do from now on. He contrasts attitudes of wonder, humility, and caution with approaches that treat AI as merely a product, a geopolitical weapon, or dismiss it as a marketing gimmick. Commenting on the debate within the industry between boom and disaster, he calls for setting aside ideological squabbles and judgments about leaders like Dario Amodei or Sam Altman to recognize the "wonders and terrors" already present, a notion he extends to a reference to a red-orange sky "like in The Matrix."

He States the Singularity Has Begun and a Transformative Phase Is Open

In a post, Scott Aaronson argues that the singularity of AI has arrived and that the transformative phase has begun, while clarifying that he does not predict the annihilation of humanity. He recalls the definition of this hypothetical scenario where systems surpass human intelligence and improve through recursive self-improvement. He anticipates direct effects on his own field: according to him, human theorem demonstrators could become obsolete, with the main gap perhaps now lying between intelligence and the physical world. He adds that he no longer expects to prove theorems out of necessity, while continuing his daily tasks, and would only do so for pleasure or for the benefit of others.

Why He Changed His Mind After Twenty Years of Skepticism

Twenty years ago, Scott Aaronson reports having adopted a cautious and skeptical attitude regarding self-improving AI, an attitude that was then widespread among his peers who considered this idea to be science fiction. He indicates that he changed his mind following certain recent events that he mentions: reports of agents successfully escaping their confinement environment and a mathematical advancement achieved with the help of AI on the Navier-Stokes problem, which is part of the Millennium Prizes. In an email exchange, he clarifies that he thought such capabilities—like solving millennium problems, compromising servers hosting AIs, or playing a central role in their own improvement—would not appear for several centuries, but they manifested in 2026, which explains his stance. His post was published on a Tuesday and includes a prolonged exclamation of 113 "A"s to convey his astonishment.

Career Highlights and Areas of Expertise

Scott Aaronson is a theoretical computer scientist whose work has helped define the limits of quantum computing. He has been a researcher at OpenAI, where he worked on AI safety as well as the theoretical foundations of safety and alignment of systems.