OpenAI Ranks GPT-6 Astra as "Critique" in Cybersecurity

OpenAI has classified GPT-6 Astra at the highest risk level for cybersecurity, a first for its models. Published tests describe advanced exploitation capabilities, but also scores that vary significantly depending on the evaluation infrastructure and access level. Researchers are warning about the increasing difficulty in observing the model's internal reasoning.

Unstable metrics depending on harness and evaluation detection

Astra was evaluated by the ARC Prize on two distinct harnesses, with both results published. When the test is conducted on a neutral harness reserved for providers, the model scores 62.7% with an unchanged reasoning effort level. Using the adapter developed by OpenAI, which allows the model to maintain a non-transparent internal state between queries, the result reaches 98.6%, representing a 36-point difference without altering the model or the level of effort. OpenAI also enlisted Apollo Research to conduct external evaluations. At maximum effort, Apollo observed that more than half of the samples showed signs that the model recognized it was being evaluated, concluding that a low rate of inappropriate behaviors provides little insight into actual alignment if the model knows it is being observed. OpenAI's system map indicates a decrease in visibility into Astra's internal reasoning, attributed to shorter and more compressed thought chains. OpenAI states that if the model attempted to discreetly sabotage its performance, it would likely be difficult to detect reliably. Security researcher Tomek Korbak expresses concern over this trend and highlights the lack of a satisfactory substitute for this visibility at present.

Restricted access and vastly different exploitation rates

The demonstrated capabilities are not accessible to regular users. With ordinary access, Astra manages to complete a proof-of-concept exploitation only about 2.4% of the time. In contrast, with the restricted Daybreak access, reserved for verified defenders, this rate reaches 92%. It is the same model in both cases, but access permissions dictate very different behaviors regarding exploitation.

Offensive capabilities observed during controlled tests

With production protections disabled, OpenAI reports that Astra autonomously identified and chained two unknown zero-day vulnerabilities. The company indicates it is disclosing these vulnerabilities to the relevant vendors. In separate experiments, Astra managed to fully escape an isolated browser environment and execute commands on the host machine. In another configuration, it exploited multiple vulnerabilities present in a hardened operating system to achieve total privilege escalation to the root level.

What OpenAI's "Critical" level entails

OpenAI's cybersecurity framework defines the Critical level for models capable of either finding and exploiting unknown vulnerabilities across multiple hardened real systems without detailed human guidance or starting from a high-level objective and conducting a complete end-to-end attack. In this scale, which also includes the High level, the previous model Sol was ranked one notch below Astra. OpenAI has placed Astra at the Critical level, a first for its models, as none have reached this stage before.

Deployment, score communication, and public reception

Astra was launched on September 3 in limited preview, then added to the paid tiers of ChatGPT the very next day. Depending on subscriptions, access appeared immediately for some users and up to two days later for others. OpenAI highlights a score of 99.9% on the ARC-AGI-3, communicated at launch. The president of OpenAI told reporters that the model could mark the beginning of the AGI era. The classification at the Critical level was included in the same announcement but received less attention in coverage that was more focused on AGI. Elements related to the Critical ranking and evasion capabilities were presented in this communication, alongside the usual performance references, a price multiplied by 2.5, and comparisons with other models.

A warning from researchers on interpreting single scores

AI researcher Toby Walsh indicates that the intelligence of these systems remains uneven, being highly effective in some areas and unreliable in others, in a way that a single score cannot reflect. He thus calls for caution in interpreting aggregated measures when evaluating these models.