AI Agents: Data Infrastructure, Key to Success

The Rapid Adoption of AI Agents in Businesses

In a context where artificial intelligence (AI) is becoming an essential tool for companies, they are hastily deploying AI agents to assist, co-pilot, and execute tasks autonomously. According to McKinsey's annual AI report, by the end of 2025, about two-thirds of companies will have experimented with these agents, while 88% will use AI in at least one business function, a notable increase from 78% in 2024. However, despite promising beginnings, only 10% of companies manage to scale these AI agents effectively.

One of the major obstacles to this evolution is the need for a robust data infrastructure. Experts emphasize that delays in AI implementation are not primarily due to weaknesses in AI models, but rather a lack of data architectures capable of providing reliable business context. Irfan Khan, President and Chief Product Officer of SAP Data & Analytics, states that the coming months will be crucial for companies to establish the right data architecture.

The Crucial Importance of Data in the AI Era

The ability of AI agents to evolve will largely depend on the strength of the company's data architecture and governance, rather than the evolution of the AI models themselves. For the technology to develop, companies must adopt a modern data infrastructure that not only provides data but also the necessary context to leverage it effectively.

Business Context Trumps Data Quantity

Traditionally, structured data was perceived as highly valuable, while unstructured data was often underestimated. However, AI challenges this distinction. The value of data for AI agents is determined by its business context rather than its format. Critical data for business functions such as supply chain or financial planning heavily relies on this context. Even large and detailed data, such as that from the Internet of Things (IoT) or telemetry, only brings value when associated with relevant business context.

Irfan Khan emphasizes that the real challenge for agentic AI is not the lack of data, but the absence of a solid foundation. He explains that "anything that is contextually relevant to the business will, by definition, give you greater value and higher levels of reliability in business outcomes." The distinction between high-value and low-value data is not just about their structure, but about how they are used within a business framework.

Context can be enriched through integration with software, on-site analysis, or through a governance pipeline. Data lacking these qualities risks being perceived as unreliable, which explains why two-thirds of business leaders do not fully trust their data, according to the Institute for Data and Enterprise AI (IDEA). This "trust debt" hinders companies in their readiness for AI. To overcome this lack of trust, it is essential to establish shared definitions, semantic consistency, and reliable operational context.

The Proliferation of Data and the Need for a Semantic Layer

Over the past decade, the most significant evolution in enterprise data architecture has been the separation of compute and storage, along with the flexibility offered by the cloud. However, this separation has also led to a proliferation of data, hosted across various clouds, data lakes, warehouses, and SaaS applications.

With the rise of AI, this proliferation is only worsening. More than two-thirds of companies identify data silos as a major barrier to AI adoption, and over half of them must manage more than 1,000 data sources. While the previous era focused on laying the groundwork for SaaS by separating compute and storage, the new era aims to provide the right data to autonomous AI agents responsible for various business functions.

To address this issue, a semantic or knowledge layer is necessary. It must support multiple platforms, encode business rules and relationships, provide a contextualized and governed view of the data, and allow appropriate access to data for both humans and agents. Legacy data architectures are not suited to feed the autonomous AI systems of tomorrow, as highlighted in a Deloitte report. Only four out of ten companies believe their data management processes are ready for AI, a figure down from 43% the previous year, indicating that companies are becoming aware of the gaps in their infrastructure.

Agentic AI and SaaS: A Necessary Coexistence

Some industry experts speculate that AI agents could render SaaS applications obsolete. Irfan Khan disagrees. He explains that, while value has migrated to the top of the software stack over the years, agentic AI simply represents the next layer. AI agents will have their own layer to access data and interact with business logic. Value rises in the stack, but the underlying layers do not disappear.

Khan asserts that SaaS is not going away. On the contrary, SaaS and AI agents will cooperate. Companies will not replace their existing systems with AI agents, as the latter need business context to operate effectively.

In this emerging model, the software stack is redefined so that applications and data provide a governed context in which AI can act effectively. SaaS applications remain the systems of record, while the semantic layer becomes the source of contextual truth. AI agents become a new layer of engagement, orchestrating across systems, and both humans and agents become "first-class citizens" in accessing business logic.

It is crucial that agents cannot connect directly to every operational system. Khan warns that "if we say that agents are going to take over the world... you cannot have an agent talking to every operational backend system," highlighting the importance of a semantic layer or business fabric.

The First Steps Toward an Effective Data Infrastructure

For companies, the first step is to focus on the platforms where their data already resides, such as Snowflake, Databricks, Google BigQuery, or an existing SAP environment. Khan advises against rebuilding old vendor lock-in schemas.

He recommends that companies prioritize the most important data by focusing on preserving and providing business context to operational and application data. Investing early in governance and semantics by defining shared policies, access rules, and semantic models is crucial before scaling pilots. Finally, companies should prioritize openness and interoperability rather than forcing all data into a single stack.

Khan warns against fully automating critical business processes too soon. He emphasizes that there is a bold opportunity to engage in the world of agentic AI, but complete automation will require additional oversight. Early successes will likely come from less critical processes and agents operating from fresh and dynamic data. As AI begins to deliver value and adoption increases, leaders will need to decide how to reinvest these gains to improve efficiency or explore new markets.