GPT-6: Enhanced Prompt Cache and Discounts of Up to 90% for 30 Minutes

Key Takeaways

1OpenAI introduces a more efficient prompt caching in GPT-6, with discounts of up to 90% on reused prefixes within a 30-minute window

2Tracking and diagnostic tools allow monitoring of cache rates and identification of failures

3Technical teams report increases in cache rates (up to 91%) and decreases in costs (up to 36%) after adopting the new devices

Why it matters : Advanced caching significantly reduces costs and latency for applications utilizing persistent agents.

OpenAI is active in the GPT-6 family with a more efficient prompt caching system, accompanied by tracking and diagnostic tools. Discounts can reach up to 90% on cached input tokens, with a reuse window of 30 minutes. Teams are reporting increases in cache rates and decreases in production costs.

Teams Report Cost Savings and Cache Rates

Several technical leaders describe rapid improvements following the adoption of the new tools. Arian Hanifi reports a 20% cost reduction thanks to the dashboard and diagnostics, with alerts for unexpected failures and support from Codex agents to identify the cause. He emphasizes that explicit breakpoints allow for maintaining stable context in the cache while placing variable content at the end of the prompt, making conversation forking viable for background tasks.

At another user site, Bin Fan indicates that within less than a week, the cache success rate increased from around 85% to consistently over 90%, further reducing inference costs after adjusting breakpoints, combining explicit and automatic caching, and analyzing real queries. For long-duration agents like Manus, he considers cache reliability crucial for cost efficiency.

Eugene Mikhantyev reports an increase from 83% to 91% success rate on evaluations in less than a week after migrating to explicit breakpoints, with two-thirds fewer cache writes and a 36% reduction in inference costs for the same workload.

Discounts Up to 90% and Prefix Reuse for 30 Minutes

Pricing takes into account the caching of part of the prompt: discounts can reach 90% on cached input tokens. Reductions apply to eligible shared prefixes reused within a 30-minute window, rather than to all content by default. This mechanism targets the effective reuse of segments shared between requests.

Le brief IA que lisent les pros

Le brief IA que les pros lisent chaque soir

Les 7 actus IA du jour, décryptées en 5 min. Gratuit.

Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.

Choisissez votre rythme

Gratuit · Pas de spam · Désabonnement en 1 clic

Integrated Monitoring and Diagnostics for Cache Failures

A dedicated dashboard indicates the share of inputs served from the cache and allows tracking of success rate trends. A composition graph distinguishes cached and non-cached tokens to quickly identify drops and link variations to application changes. In case of unexpected failures, a diagnostic tool compares a request to a recent response to identify differences in model, tools, parameters, or input, and provides an estimate of the number of affected tokens to guide optimizations.

Breakpoints, Preheating, and Tool Stability for Optimization

Explicit cache breakpoints allow users to choose which prompt prefixes to reuse, with an updated guide specifying eligibility and the impact of changes in tools and inputs. On GPT-6 models, reasoning effort can vary between responses via a configuration_update without breaking the cache, increasing it for difficult tasks or reducing it for follow-ups. The stability of tool definitions, schemas, and their order contributes to preserving reuse; allowed_tools can limit calls to only relevant tools, and tool_choice can be set to none when no tool is required. New developer messages allow for adding instructions at the end of the context to replace previous ones. Preheating can prepare shared instructions, tool definitions, or reference material at startup, reducing latency by moving computation out of user wait time. All these levers enhance the default performance of the engine to adapt to workload.

Persistent Agents and Reduced Fresh Processing in Copilot

Persistent agents, capable of holding sessions for several hours to refactor code or produce documents and presentations, execute sequences of requests with recurring instructions and tools. OpenAI caches this context to reuse calculations and speed up responses. In addition to the new tracking and diagnostic tools, user feedback indicates large-scale effects: according to Mario Rodriguez, the share of prompt tokens requiring fresh processing has dropped by over 50% across billions of requests, resulting in a more efficient inference stack and faster first tokens for developers. This system is part of an improved version of the prompt caching in the GPT-6 family, aiming for higher default cache success rates.