EY Revolutionizes AI Cost Management with 'Invisible' Router

EY Introduces an AI Router to Reduce Costs

The company EY, one of the major consulting firms known as the Big Four, has recently integrated an "invisible" AI router into its systems. This device is designed to optimize the use of tokens, which are units of measurement for the input and output of artificial intelligence models. According to Dan Diasio, EY's global head of AI consulting, this router has allowed certain divisions to reduce their token consumption by up to 60%. The router guides employees to the AI tool best suited for their task, which has a direct impact on the savings achieved.

An AI Query Management System

EY implemented this router to prevent every employee request from automatically going to the most powerful AI models, which are often more expensive. Dan Diasio explained that the router, deployed globally in April, operates in the background of EY's specialized AI tools. It acts as an intermediary, directing employee queries to the most appropriate model for each specific task. For users, the experience remains unchanged: they enter their query and receive a response. However, the router optimizes the routing of queries behind the scenes, often more quickly than the heavier models.

The Pressure of Token-Related Costs

Companies are facing increasing pressure regarding the costs associated with tokens, as AI providers adopt usage-based pricing models. Since October, companies like OpenAI, Anthropic, and GitHub have introduced billing systems that take into account the number of tokens used, rather than a fixed rate. This shift means that using powerful models for simple tasks can quickly become expensive. An AI Pulse survey conducted by EY revealed that 82% of executives at companies investing in AI are concerned about these costs. The study, conducted between April and May, surveyed 534 senior decision-makers from U.S. companies.

Optimizing AI Tools to Reduce Waste

According to Diasio, the high bills that some companies receive are often due to a small fraction of their employees, about 1% or 2%, using inappropriate tools for their tasks. EY's AI router helps select the right model for each job, thereby limiting token waste. This approach ensures that more expensive AI requests are reserved for functions where they provide the most value. Many of EY's clients are also adopting this solution to better manage their costs.

Targeted Deployment of the AI Router

The router is not yet universally deployed across all of EY's AI tools. It is currently used on platforms specific to certain departments, such as tax and risk. In contrast, it is not applied to the Microsoft Copilot chatbot, which is accessible to all EY staff. Nevertheless, the router's impact on token consumption is significant, with a 60% reduction since its introduction in April, thanks to training and governance strategies.

Moving Towards a New Phase of AI Management

The first phase of AI adoption in businesses aimed to encourage maximum use of the technology, a phenomenon known as "tokenmaxxing." However, rising costs have shifted the focus towards controlling this usage. Companies like Disney and JPMorgan have implemented dashboards to track AI usage by their employees, and new startups specializing in AI routing are emerging to help manage expenses.

EY has established token budgets based on employees' roles and departments. Those who exceed their allocation can request additional tokens through an approval process. According to the AI Pulse report, 64% of executives say their organizations now monitor token usage and have budgetary safeguards in place. Diasio expects this figure to reach 90% in the next six months.

Maximizing the Value of AI in Business

Diasio emphasizes that companies can also maximize the value of their AI spending by enhancing their internal knowledge and crafting better queries based on their own data. He recommends focusing investments in areas where AI can have the greatest impact on work, rather than spreading resources thin. Monitoring employee usage remains crucial, but the emphasis is shifting towards the value and outcomes achieved through AI. Measuring this value requires going beyond simple profit and loss to assess whether AI enables employees to work more efficiently and explore new initiatives.