Brief IA
›

OpenAI Claims 46% Success Rate for Its Research Agents

Models & LLM·Tom Levy·

OpenAI Claims 46% Success Rate for Its Research Agents

OpenAI Claims 46% Success Rate for Its Research Agents
Key Takeaways
1OpenAI announces a 46% autonomous success rate for tasks requiring two to four hours from a person.
2According to the company, the average usage of agents amounted to about $2,400 per researcher per day in October, at public API rates.
3Post-training and security researchers attribute the main ideas of GPT-6 Astra and GPT-6.2 Astra to humans.
Why it matters : Surveyed researchers believe that agents still require close monitoring.

OpenAI announces that its coding agents complete 46% of research tasks without human intervention, which would typically take a person two to four hours, up from 23% in March. However, the company describes variable performance and researchers who remain closely involved in idea selection.

The Origin of Ideas and Safety Research

When asked by OpenAI about the main ideas that fueled GPT-6 Astra and GPT-6.2 Astra, post-training and safety researchers attribute the origin entirely to humans. The models have assisted them in developing these ideas. For GPT-6.2 Astra, however, pre-training researchers mention ideas developed during their interactions with Codex; one of them acknowledges its decisive intuitions.

During interviews, researchers find the agents' choices insufficient in terms of scientific leads, programming, data, or writing. They believe close oversight remains necessary. A pre-training researcher, however, doubts that this human superiority will be sustainable.

Safety and alignment are the areas where research time has increased the most, according to OpenAI: documented work weeks have risen by 77% between January and September. This increase accelerated after the hacking of the Hugging Face AI platform by OpenAI's research agents in July.

The Cost and Use of Agents

In October, the average usage per researcher amounted to approximately $2,400 per day at public API rates, according to OpenAI. This represented $1,500 in September, marking a 62% increase. At the 90th percentile, the daily value reached about $23,000, more than double the $11,000 from September.

The execution time consumed per researcher has increased elevenfold, from 0.5 to 5.7 "agent workdays" daily, with one unit corresponding to eight hours of execution. By mid-October, more than two-thirds of researchers were running at least four agents in parallel at their peak usage time of the day. Over 10% were using at least 16 simultaneously.

La lettre du soirGratuit · 19 h

Le brief IA que les pros lisent chaque soir

Les 7 actus IA du jour, décryptées en 5 min. Gratuit.

Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.

Choisissez votre rythme

Gratuit · Pas de spam · Désabonnement en 1 clic

In September, the value of agent usage within research was approximately twice the salary mass of researchers, according to the company. This comparison excludes capital allocations, which, according to OpenAI, account for 70 to 80% of their target compensation. However, internal agent usage represented only 2% of the computing resources allocated to research.

Success Depends on Assigned Tasks

For tasks requiring four to eight hours from a person, OpenAI reports a success rate that has increased from 13% to 35%. The disparities between activities remain significant: agents complete 79% of training monitoring tasks and 72% of responses to technical questions on their own, according to the company.

The results are lower for programming inference and service systems, at 25%, as well as for creating evaluation and training environments, at 29%. OpenAI also warns that researchers have been able to evolve the missions delegated as agents progressed. The observed rates therefore depend, according to the company, on the selected tasks as much as on the models.

An Index to Track AI Progress

To monitor any potential acceleration in progress, OpenAI uses its own version of the Epoch AI capabilities index. The company reports a progression of 16.4 points per year since August 2025, compared to 14 points per year measured by Epoch AI since late 2024. It acknowledges that its models are rated better on its internal indicator than on Epoch AI's.

OpenAI published its second report on the acceleration of its research through AI on Wednesday, following the one from September. The company calls for laboratories to be required to measure and publish signs of recursive improvement, meaning the contribution of AI to the faster creation of future AIs. It clarifies that this transparency initiative does not reflect a fully autonomous improvement project.

OpenAI also mentions three reports from Anthropic, which claims to have crossed its own threshold for AI-accelerated research. On the same day, OpenAI also announced that its models had found gains for the design of AMD graphics processors.

Brief IA — L'actualité IA en français

L'essentiel de l'actualité de l'intelligence artificielle, décrypté et expliqué chaque jour.