OpenAI: Astra Leads on ErdosBench Without Mathematical Optimization

Astra excels on ErdosBench, but OpenAI claims it did not target mathematical research. The results fuel a debate: usefulness for mathematicians, technical trade-offs among designers, and the future of a uniform AGI or one with uneven skills.

Proofs Faster than Verification, Warns Terence Tao

At the 2026 International Congress of Mathematicians, Terence Tao warned of the risk that AI models produce proofs at a pace exceeding human verification capacity, which could lead to an overload of results. He believes that mathematics is undergoing a crisis of values and practices, comparable to the foundational crisis of the early 20th century. However, the most difficult problems in the field remain unsolved to this day. Moreover, some mathematicians feel that language models lack human creativity and doubt they can deliver genuine breakthroughs, while expressing reservations about the possibility of authentic self-improvement in AI.

ErdosBench: 3.23 Score, 106 Problems Solved for Astra

On the ErdosBench, which includes 226 open mathematical problems inspired by the work of Erdős, Astra achieved a score of 3.23. The model solved 106 problems, 43 of which were solved completely, and refuted 27 others. However, an update indicates that Fable 5.1 has since taken the lead in the rankings, with fewer problems solved but overall superior performance.

OpenAI Acknowledges Not Optimizing Astra for Math

Jakub Pachocki, Chief Scientist at OpenAI, states that the team did not prioritize optimizing Astra for mathematical research, even though successes in this area were highlighted during the initial announcement. He explains this choice by the urgency surrounding research on recursive self-improvement and automated alignment, areas where OpenAI is focusing its resources, believing it is the only way to stay at the forefront of AI research. Since the publication of his essay, it has been reported that OpenAI has trained better internal models in mathematics and that themes related to RSI and AI safety have gained prominence.

Measured Gains and a Benchmark Still Far from Saturated

Przemek Chojecki, developer of ErdosBench, believes that Astra represents a solid gain of 5% to 10% across various tested mathematical research skills, while emphasizing that the benchmark is far from saturated. In comparison to Sol, which solved 78 problems with maximal reasoning, Astra stands out for its better quality of scientific writing, a lower tendency for exaggerated claims, and, in some cases, an underestimation of its own results.

Possible Trajectories of AI: Homogeneous or 'Spiky'

Astra's performance in mathematics does not guarantee superiority in other tasks, illustrating the optimization trade-offs at the frontier of AI research. Adam Hunt, a researcher at Cambridge, contrasts a broad and gradual improvement trajectory, associated with the mainstream AGI thesis, with a so-called 'spiky' scenario where AI excels in areas like coding and mathematics but stagnates or even regresses in language quality, common sense, or social reasoning. Such a profile would indicate a high level of specialization, far from what many consider to be General Artificial Intelligence. In this context, Astra's initial top position on a mathematical benchmark is insufficient to conclude a universal capability.