OpenAI: Concerns Over Astra's "Opaque Recurrence"

Le brief IA que les pros lisent chaque soir
Les 7 actus IA du jour, décryptées en 5 min. Gratuit.
Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.
Choisis ton rythme
Gratuit · Pas de spam · Désabonnement en 1 clic
AI security figures fear that "recurrent depth" will make the thought chain less controllable. OpenAI assures that the intended use in Astra remains limited, that traces will remain readable, and that increased monitoring is on the agenda.
Alerts on a Shift Towards Invisible Reasoning
Ryan Greenblatt, Chief Scientist at Redwood Research, believes that opaque reasoning could advance faster than conventional thought chains, potentially eliminating reasoning from observable channels. He worries that a natural evolution may lead to an increase in opaque reasoning until the model reasons almost entirely within the latent space. He hopes it is not too late to avoid the most concerning architectures and that OpenAI will stop at this stage. More broadly, there remains concern that opaque recurrence will make reasoning oversight more difficult, especially if this approach spreads to other models.
OpenAI Promises a Readable Thought Chain and Monitoring Tools
OpenAI claims that, even with opaque recurrence, Astra's thought chain should remain readable and that the use of this technique appears limited. The company rejects the idea of a shift to "neuralese." It announces extensive thought chain monitoring systems in its upcoming security plans. Jakub Pachocki, Chief Scientist at OpenAI, emphasizes that preserving and utilizing thought chain monitoring has been a priority since the early reasoning models and that it remains a central goal of the current research program.
The Thought Chain: A Fragile Benchmark Tested by Recurrence
A model's thought chain generally corresponds to the sequential steps taken to solve a problem. Despite its limitations, it remains a useful tool for monitoring undesirable behaviors or misalignments and has helped understand the decisions of out-of-control agents during recent incidents. With opaque recurrence, this balance is altered: the model makes multiple passes over the same query, following a less linear logic, which produces fewer exploitable clues and escapes the usual step log. In all AI models, some opacity of reasoning persists, and few researchers see thought chain logs as a faithful transcription of the model's reasoning. However, several experts believe that "recurrent depth" may complicate monitorability.
With Astra in Focus, the Security Community Raises Multiple Warnings
Buck Shlegeris, CEO of Redwood, expressed extreme concern about the use of opaque recurrence in Astra. He stated that he does not know if Astra is significantly less monitorable than previous models, but warns that pushing this technique further could massively increase recurrence and destroy the monitorability of the thought chain. Zvi Mowshowitz, an AI security advocate, believes that laws may be necessary to prevent a "race to the bottom" among laboratories. He considers that this technique risks breaking a taboo established by OpenAI and Anthropic regarding the fidelity and monitorability of the thought chain, and judges that more intensive use would likely damage this monitorability. These reactions have multiplied as Astra is set to employ a "recurrent depth" that deviates from strictly sequential reasoning.
Brief IA — L'actualité IA en français
L'essentiel de l'actualité de l'intelligence artificielle, décrypté et expliqué chaque jour.