Evaluating AI: 'Useful Intelligence per Dollar'

A New Approach to Measuring the Value of AI

CFOs are facing a crucial question: how can they maximize the value of investments made in artificial intelligence? Traditionally, the success of software was evaluated by metrics such as the number of licenses sold or user adoption rates. However, with AI, a more nuanced approach is necessary, focused on the work accomplished.

The fundamental economic question for CFOs and executives is to determine whether the value generated by AI grows faster than the costs associated with its implementation. To answer this, one must go beyond simple metrics like cost per token. A cheaper model may seem attractive, but if it requires more trials or human corrections, the total cost may end up being higher. Conversely, a more expensive model per token may accomplish a task in a single attempt, thus reducing the overall cost. The goal is to measure the total cost of producing a successful outcome against the value it generates.

The ultimate metric for evaluating AI could be "Useful Intelligence per Dollar." This approach relies on four essential questions:

• Does the AI accomplish meaningful work?
• What is the cost of each successful task?
• Are the results reliable?
• Does every dollar invested in AI produce more value as usage increases?

1. Measuring Useful Work Accomplished

The first step is to assess the work itself.

• How many customer problems has the AI solved?
• How many code modifications has it helped finalize?
• How many contracts has it reviewed?
• How much time has it freed up for employees?
• How many decisions have been improved thanks to relevant context provided at the right time?

Tokens gain value when they translate into useful actions for users. As models become more capable, they can handle more complex tasks, maintain context, reason through multiple steps, use tools, and adapt along the way.

The ideal starting point is a unique workflow. Define what "done" means and measure that outcome in the system where the work takes place.

For a support team, "done" might mean a customer issue resolved. For an engineering team, it could mean a code modification that passes its tests. For a legal team, it might mean a contract reviewed accurately and on time.

Take the example of a finance team preparing a forecast review. A significant amount of work occurs before a final decision is made: finding the latest forecast, moving data into Excel or Sheets, identifying changes, reconciling tabs, rebuilding slides, and ensuring everything adds up perfectly.

ChatGPT Work can handle much of this process, allowing the team to focus on the questions that matter: What has changed? Why? What should we do next?

This is useful intelligence per dollar in practice. More work is accomplished, faster, while people spend more time applying their judgment, creativity, and expertise.

2. Calculating the Cost of a Successful Task

The next question is how much it costs to accomplish this work well.

AI tasks vary significantly. A quick response may require little computation. A coding, research, or financial workflow may involve deeper reasoning, tool usage, and numerous actions. These more complex tasks may require more computation, but they can create much more value.

At the model level, the cost per successful task depends on the price, the amount of computation used, and the likelihood of achieving the correct outcome. For a business, the total cost also includes employee time, human review, retries, and rework.

The calculation is simple:

• Add the total cost of accomplishing the work.
• Count the tasks that reached the required quality level.
• Divide the total cost by the number of successful tasks.

This is why the lowest price per token does not always produce the lowest cost per outcome. A cutting-edge model can offer the best value even for a routine request if it produces the correct answer in a single attempt, thus reducing retries, latency, review, and total computation.

A family of multi-tiered models provides customers with more ways to optimize this equation. GPT-5.6, which we launched last week, has three tiers: Sol is our flagship model; Terra balances performance and cost; Luna is our fastest and most affordable model.

These tiers provide useful starting points. The economics of the complete task should ultimately determine the appropriate model. A customer might use Luna for a fast, high-volume workflow, Terra for work requiring more depth, or Sol when stronger reasoning produces the best result with fewer attempts.

We have trained GPT-5.6 to achieve more useful work from each token. On the Artificial Intelligence Coding Agents Index, GPT-5.6 Sol with maximal reasoning has set a new state-of-the-art while using 54% fewer output tokens than another leading model.

The chart below illustrates the comparison.

• DeepSWE v1.1: Long-term engineering tasks; GPT-5.6 Sol reaches a new peak of 72.7%, above Claude Fable 5 at 69.9%, with an estimated API cost 36.2% lower.

Across the GPT-5.6 family, the goal remains the same: more successful work per dollar. Greater efficiency makes existing tasks more affordable. Greater capability enables new types of work.

Each new generation of models should improve both sides of this equation. Customers should be able to accomplish more valuable work while seeing the cost of each task continue to decrease.

3. Measuring AI Reliability

The third measure is reliability.

AI adoption tends to develop in stages. First, AI helps with drafting. Then it finds context and reasons through tools and data. Over time, it begins to act, manage exceptions, and complete workflows, with people providing judgment and control when necessary.

Each stage creates more value and demands more from the system.

Reliability has direct economic value. When results are accurate, well-sourced, consistent, and appropriately escalated, people spend less time reviewing, correcting, and repeating work. Successful tasks cost less, and organizations gain the trust needed to use AI in larger workflows.

Teams can make this concrete by tracking three outcomes:

• Ready to use: The result has reached the required quality level as delivered.
• Requires correction: The result needed a new attempt or human modifications.
• Requires escalation: A person had to intervene and finish the work.

These measures tell a richer story than just model accuracy. They show whether AI is truly reducing the work involved in completing the project.

Reliability also requires clear boundaries. Before AI moves from drafting to action, organizations must define:

• What data the system can access.
• Which systems it can use or modify.
• When a person must review or approve an action.

Security, privacy, compliance, and control create the foundation for deeper usage. People need to understand how the system behaves, how their data is handled, and how its actions are governed.

ChatGPT Work builds on the security, privacy, compliance, and workspace management foundation of ChatGPT Enterprise. This allows organizations to give AI more context and access to more valuable workflows while maintaining appropriate oversight.

Capability justifies initial use. Reliability makes AI an integral part of how work gets done.

4. Optimizing AI Investment at Scale

The final question is whether the economics improve at scale.

Businesses can measure this by tracking the same workflow over time. Monitor how many tasks have reached the required quality level, the total cost of their completion, and the cost per successful task. If the work accomplished grows faster than the total cost while maintaining or improving quality, every dollar invested in AI produces more value.

The calculation is at the heart of this equation.

Computation fuels research and every task that AI accomplishes. It shapes product quality, speed, reliability, availability, and cost. Training computation builds future capability. Inference computation delivers useful work today. Both should translate into better outcomes for customers.

Better models, more efficient inference, custom-designed hardware, higher utilization, smarter routing, and stronger product design all enhance the return on investment from computation. Each generation of infrastructure helps train more capable models. Better algorithms, hardware, and software then help serve these models more effectively.

Customers feel these improvements in human terms: better responses, faster results, fewer corrections, more reliable products, and reduced costs for the work they need.

The gains compound. Better infrastructure accelerates research. Research produces more capable and efficient models. Better models improve products. Better products drive adoption, learning, and revenue. This growth supports ongoing investment in the next generation of research, computation, deployment, and security.

OpenAI brings these elements together through a shared intelligence platform. People use it via ChatGPT and ChatGPT Work. Developers build with it via Codex and the API. Businesses integrate it into the systems where work takes place.

When one of the layers improves, every product and customer can benefit.

An Evaluation Framework for the AI Era

Together, these four measures indicate whether useful intelligence per dollar is improving.

Useful work tells us what AI produces. The cost per successful task tells us what it takes to achieve the outcome. Reliability tells us how much work people can use with confidence. Value at scale tells us whether each dollar, and each unit of computation, accomplishes more over time.

The goal is an AI that helps people accomplish more meaningful work, make better decisions, and spend more time on the parts of their jobs that require distinctly human judgment and creativity.

Our job is to improve this equation with each generation: more capable models, faster and more reliable outcomes, and reduced costs for the work customers need.

This is how AI becomes more useful for a greater number of people and organizations over time.