GPT-6 Astra: 8.5% of Successful Hidden Injections Despite Progress

Key Takeaways

1GPT-6 Astra blocks 99.99% of direct prompt injections according to OpenAI

2Adaptive attacks still succeed in about one-third of cases

3Gray Swan observes 8.5% of compromises on hidden injections, compared to 27% for GPT-5.6 Sol

Why it matters : Despite notable progress, the security of GPT-6 Astra against sophisticated attacks remains a concern for enterprise use.

OpenAI announces a significant reduction in errors and near-systematic failures in direct manipulations with GPT-6 Astra. However, internal and external evaluations reveal vulnerabilities to adaptive attacks and prompt injections hidden within documents, with a compromise rate of 8.5% according to Gray Swan.

Adaptive Attacks: 67% Defense, Tests Without Production Layers

When attackers adapt their strategies over the course of interactions, GPT-6 Astra maintains a defense rate of around 67%, according to OpenAI. Previous models scored just below 50% on the same test. OpenAI clarifies that these evaluations were conducted on the raw model, without production protections such as classifiers. For indirect prompt injections, Gray Swan reports that with 15 attempts per scenario, Astra was compromised at least once in 8.5% of cases. In the same evaluation, Claude Opus 5 achieved 4.8%, without being completely immune.

Fewer Hallucinations and 99.99% Blocking of Direct Injections

OpenAI states that GPT-6 Astra hallucinates less than GPT-5.6 Sol and makes significantly fewer factual errors. The trials were conducted based on ChatGPT discussions flagged by users for inaccurate responses; Astra made these same errors much less frequently, with marked improvements at low latency and on simple reasoning tasks. Regarding direct prompt injections, OpenAI reports a defense rate of 99.99%. When faced with a defined set of attacks aimed at producing dangerous content in biology, violence, or cybersecurity, Astra refuses to assist in a proportion ranging from 91.5% to 98.3%. OpenAI also indicates that resistance to jailbreaks has improved compared to earlier versions.

Le brief IA que lisent les pros

Le brief IA que les pros lisent chaque soir

Les 7 actus IA du jour, décryptées en 5 min. Gratuit.

Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.

Choisissez votre rythme

Gratuit · Pas de spam · Désabonnement en 1 clic

Gray Swan Evaluations: 1,810 Attacks from the IPI Arena

The security company Gray Swan conducted external tests with 1,810 carefully selected attacks from its IPI Arena. In this context, GPT-5.6 Sol fails 27% of the time, allowing for a measurement of the improvements brought by Astra. Gray Swan also evaluated AI agents capable of writing code, using tools, and autonomously controlling computers.

A New Model and an Attack-Focused Training Method

OpenAI presents GPT-6 Astra as its new flagship model. According to the company, part of its defensive capabilities comes from GPT-Red, a training approach that incorporates an automated attacker. Although these measures exist, OpenAI specifies that determined attackers can still obtain at least one problematic response in about one-third of cases after several attempts. The fact that AI agents are being developed to operate continuously at scale, including reading and analyzing documents among their primary functions, underscores the necessity for such protections.