OpenAI and Anthropic: Challenges of the AI Operational Layer

The Divide in Enterprise AI

In the landscape of enterprise artificial intelligence, a significant divide is emerging, yet it is not receiving the attention it deserves. While the public debate remains focused on foundational models and their benchmarks, such as the comparison between GPT and Gemini, as well as reasoning scores and marginal capability gains, a more sustainable advantage is taking shape elsewhere. This advantage lies in the structure: who controls the operational layer where intelligence is applied, governed, and improved? Two models are competing: one views AI as on-demand service, while the other integrates it as an operational layer. This layer is a combination of workflow software, data capture, feedback loops, and governance, situated between models and actual work, and it accumulates with use.

Model providers like OpenAI and Anthropic offer intelligence as a service. You have a problem, you call an API, and you get an answer. This intelligence is versatile, largely stateless, and weakly connected to the daily workflow where decisions are made. It is highly capable and increasingly interchangeable. The crucial distinction is whether the intelligence resets with each request or accumulates over time.

Integrating AI as an Operational Layer

Established organizations, on the other hand, can treat AI as an operational layer. This means they instrument AI throughout workflows, utilize feedback loops from human decisions, and implement governance that transforms individual tasks into reusable policies. In this framework, every exception, correction, and approval becomes an opportunity to learn, and intelligence can improve as the platform absorbs more work from the organization. The companies most likely to shape the future of enterprise AI are those that can integrate intelligence directly into operational platforms and instrument these platforms so that work generates actionable signals.

The dominant narrative suggests that agile startups will outpace established companies by building AI-native solutions from the ground up. If AI is primarily a model problem, this story holds water. However, in many areas of enterprise, AI is a systems problem, involving integrations, permissions, evaluation, and change management. The advantage then goes to those already embedded in high-volume, high-stakes workflows, converting that position into learning and automation.

Inverting the Human-AI Paradigm

Traditional service organizations rely on a simple architecture: humans use software to perform expert work. Operators connect to systems, navigate workflows, make decisions, and process cases. Technology is the means, human judgment is the product.

An AI-native platform inverts this paradigm. It ingests a problem, applies accumulated domain knowledge, autonomously executes what it can with high confidence, and directs targeted subtasks to human experts when the situation requires judgment that the system cannot yet reliably provide. However, reversing the human-AI interaction is not merely a redesign of the user interface. It requires solid raw materials, such as a base of domain expertise, behavioral data, and operational knowledge accumulated over the years.

The Three Accumulated Assets of Established Companies

AI-native startups begin with a blank architectural slate and can move quickly. However, they cannot easily manufacture the raw materials that make domain AI defensible at scale. Service companies already possess three crucial elements: proprietary operational data, a large workforce of domain experts whose daily decisions generate training signals, and tacit knowledge accumulated on how complex work is actually performed.

These ingredients do not constitute moats on their own. They become an advantage only when a company can systematically convert messy operations into AI-ready signals and institutional knowledge, then feed the results back into the workflow so that the system continues to improve.

Codifying Expertise into Reusable Signals

In most service organizations, expertise is tacit and perishable. The best operators know things they cannot easily articulate: heuristics developed over the years, intuitions about specific cases, and pattern recognition that operates below the level of conscious reasoning.

At Ensemble, the strategy to tackle this challenge is knowledge distillation. This involves systematically converting expert judgment and operational decisions into machine-readable training signals. For example, in health revenue cycle management, systems can be fed explicit domain knowledge and deepen their coverage through structured daily interactions with operators. In Ensemble's implementation, the system identifies gaps, formulates targeted questions, and verifies answers with multiple experts to capture both consensus and nuances of specific cases. It then synthesizes these inputs into a living knowledge base that reflects the situational reasoning behind expert-level performance.

Transforming Decisions into a Learning Cycle

Once a system is sufficiently constrained to be trustworthy, the next question is how it improves without waiting for annual model updates. Every time a qualified operator makes a decision, they generate more than just a completed task. They generate a potential labeled example — a context associated with an expert action (and sometimes an outcome). At scale, across thousands of operators and millions of decisions, this flow can fuel supervised learning, evaluation, and targeted forms of reinforcement — teaching systems to behave more like experts in real-world conditions.

For instance, if an organization processes 50,000 cases per week and captures just three high-quality decision points per case, that amounts to 150,000 labeled examples each week without creating a separate data collection program.

A more advanced design involving humans in the decision-making process places experts within the decision-making flow, so systems learn not only what the right answer was but how ambiguity is resolved. In practice, humans intervene at branching points — selecting from options generated by AI, correcting assumptions, and redirecting the workflow. Each intervention becomes a high-value training signal. When the platform detects a specific case or a deviation from the expected process, it can request a brief structured justification, capturing decision factors without requiring lengthy free-form reasoning logs.

Building Towards Amplifying Expertise

The goal is to permanently incorporate the accumulated expertise of thousands of domain experts — their knowledge, decisions, and reasoning — into an AI platform that amplifies what each operator can achieve. When done well, this produces a level of execution quality that neither humans nor AI achieve independently: greater consistency, better throughput, and measurable operational gains. Operators can focus on more substantial work, supported by AI that has already performed the analytical work across thousands of similar prior cases.

The broader implication for business leaders is straightforward. AI advantages will not be determined solely by access to versatile models. They will stem from an organization’s ability to capture, refine, and accumulate what it knows — its data, decisions, and operational judgments — while building the necessary controls for high-stakes environments. As AI transitions from experimentation to infrastructure, the most sustainable advantage may belong to companies that understand work well enough to instrument it and can transform that understanding into systems that improve with use.