GPT-5.6: The AI Revolution Combining Power and Profitability

The Evolution of GPT-5.6: A Major Technological Advancement

The GPT-5.6 model series has been designed to provide an optimal balance between processing capability and operating cost, thus meeting the diverse needs of users. The flagship model, GPT-5.6 Sol, stands out for its superior reasoning ability, surpassing Claude Fable 5 on the Artificial Analysis Coding Agent Index, while being twice as cost-effective. Meanwhile, the Terra model competes with GPT-5.5 in terms of performance on intelligence benchmarks but at half the price. Luna, on the other hand, is the fastest and most economical model, costing 80% less than Sol. These results are the outcome of significant optimizations made by our research and development teams at every level of our architecture. These improvements pertain not only to the models themselves but also to inference, which is how we execute the models to generate results, as well as our agentic harness, used by Codex and ChatGPT Work.

Over the past four years, we have expanded our user base to one billion active users and over two million businesses. In this context, efficiency has become a central issue for sharing the benefits of artificial intelligence with the largest number of people. Our mission is to ensure that general artificial intelligence benefits all of humanity. To achieve this, we have continuously worked to unlock new optimizations in our architecture to offer the most performant models at every point on the cost-intelligence curve. With GPT-5.6, we have reached our highest efficiency in terms of intelligence per token, training the model to do more work per token. During training, we optimize both task success and efficiency, shaping the model to take a more direct path through a task.

This article looks beyond our models to explain how we have designed efficiency through advancements in two other major parts of the architecture: inference, by optimizing processes such as load balancing, speculative decoding, caching, and kernel optimization to achieve more results from the same hardware, and our agentic harness, including better management of context expansion, tool usage, and repeated work. We will also share the role of GPT-5.6 Sol in achieving several of these gains autonomously. While an isolated improvement may seem limited, these gains accumulate to allow us to deliver both in intelligence and efficiency.

Optimizing Inference with GPT-5.6 Sol

In a context where computing capacity is limited and the demand for more performant models is rapidly growing, efficiency becomes a priority in every system design. This is particularly true for our inference architecture, which executes trained models to generate responses. Our main goal is to process a greater number of tokens with the same hardware while preserving the intelligence, latency, availability, and reliability expected by users.

To achieve this goal, it is necessary to optimize the entire system. A model can be very efficient in isolation but remain costly to serve if requests are poorly distributed, if hardware remains idle, or if data movement slows down computation. Improvements at each layer accumulate, with gains coming from optimizations in routing (where requests are sent), scheduling (when requests are sent), kernels (software running on GPUs), caching (work saved and reused), and model implementation (the order of GPU code). GPT-5.6 Sol in Codex has played an instrumental role in all these optimizations.

A key example is load balancing. Globally, we route requests based on factors such as geography, available capacity, and the type of accelerator (the type of GPU or specialized chip running the model). Within a cluster, we distribute work among model instances based on load, context length, cache availability, and other request properties. Within each instance, work must then be efficiently partitioned among accelerators, model sub-networks, and compute cores. GPT-5.6 Sol in Codex helps us analyze production traffic, identify sources of unexpected imbalance, test new routing strategies, and constantly adjust these heuristics. These load balancing improvements alone have significantly reduced the service cost of our models.

We have also used GPT-5.6 Sol to optimize the model's forward pass: the computation that transforms inputs into predictions of the next token. Even when individual operations are fast, excessive memory movement, synchronization, and inefficient data layouts can leave GPUs idle. To avoid this, GPT-5.6 Sol has identified work that could be pre-computed, avoided, or parallelized. With Codex, GPT-5.6 Sol has autonomously rewritten and optimized our production kernels, the core code that executes the mathematical operations that make up the model. This has worked in part because we trained GPT-5.6 to be efficient at writing and improving kernels in Triton and Gluon, two open-source GPU programming languages maintained by OpenAI. These efforts, combined with broader advancements in GPT-5.6 Sol's kernels, have reduced end-to-end service costs by 20%. We have also heavily invested in verification tools, such as the open-source FpSan (Floating-Point Sanitizer), to help validate the correctness of kernels written by GPT-5.6 Sol.

Speculative decoding is another lever for improving speed and efficiency. This technique involves running a smaller draft model (or "speculator") alongside the main model, proposing multiple tokens for the main model to verify in parallel. When these proposals are accepted, the system can produce multiple output tokens from a single pass of the main model, thus reducing the amount of costly sequential computation. GPT-5.6 Sol has improved its own draft model by designing and running hundreds of experiments on its architecture, testing changes in size, structure, and functionality. Additionally, GPT-5.6 Sol has initiated and monitored the training process of the speculator, intervening autonomously when issues arose, including hardware failures and training instabilities. The resulting improvements have increased token generation efficiency by over 15%.

When processing uncached input tokens, the model builds the key-value (KV) cache in a computation-intensive pass; during output generation, it reads and extends this cache multiple times. The optimal configuration for serving, such as batching, sharding, and KV management, heavily depends on the workload — prompt and output length, batch size, cache hit rate, request characteristics, and more. However, the configuration space was previously too vast to be systematically tuned, forcing engineers to rely on broad heuristics. With GPT-5.6 Sol in Codex, we have been able to analyze production workloads, generate and evaluate candidate configurations, and hyper-optimize how the engine and model are configured for each scenario. This enables a new level of workload-specific optimization, extracting more useful inference from the same hardware.

Optimizing inference is a continuous feedback loop. We measure production behavior, identify the largest gaps, implement changes, and verify that they improve the entire system rather than an isolated benchmark. GPT-5.6 Sol and Codex accelerate every part of this loop. This means our team can explore more ideas, respond more quickly to changing workloads, and create an inference architecture with lower latency, more capacity, and reduced costs for users.

Streamlining Work with the Agentic Harness

ChatGPT Work and Codex accomplish complex tasks through a series of model requests and tool calls. In a single round — from user request to final response — Codex can inspect source code, search deployment history, read incident reports, edit a file, and run tests. Each step may require a request.

Preparing context, passing data, executing inference, calling tools, and starting processes all take time and computational resources. If a task requires 30 model requests, an additional second per request accumulates. Improving overall performance means reducing repeated work throughout the system, not just making the model faster.

A user round can contain many iterations of models and tools. Any cost within the repeated region can be paid multiple times.

These multipliers have informed the design of our agentic harness, which is an orchestration layer in Rust connecting our models, tools, and the user environment. We will now cover how avoiding context expansion, loading tools, and reusing work make each request more efficient.

Mastering Context Expansion

As agents are granted access to more tools, skills, plugins, and conversation history, context windows can easily widen. This increases cost, distracts the model, and leads to unnecessary reasoning. The harness can reduce this overhead through deferred discovery, which makes integrations, custom MCP tools, skills, and plugins accessible only when needed. The harness also prevents individual tools and MCP integrations from unexpectedly consuming the context window. Tool outputs are limited to 10,000 tokens by default unless the model requests a different limit.

Optimizing Prompt Caching

As previously mentioned, an agent loop can send the same instructions, conversation history, tool definitions, and previous results to GPUs multiple times during a single round. Processing these repeated inputs is costly, so caching prompts reuses the computation associated with a previously processed prompt prefix. To preserve this prefix, the harness treats all visible history to the model as additive: new messages, tool results, and environment updates are added to the end rather than inserted into the previous context. Tools are also presented in a deterministic order, while execution parameters, such as approval policies, are applied during execution rather than being embedded in tool definitions. This design choice contributes to the high cache hit rates of Codex and ChatGPT Work prompts.

Incremental Transport

Incremental transport changes what traverses the network; prompt caching changes what the model can avoid recalculating. The widths are conceptual, and the additional compression layer is not shown.

Efficiency Across the Intelligence Curve

The efficiency gains we have achieved with GPT-5.6 are the result of years of cumulative improvements across the architecture, covering...