Token-maxing: The Financial Trap of Agentic AI

The Challenge of Token-Maxing in Agentic AI

In the world of agentic artificial intelligence, the concept of token-maxing has emerged as a major challenge for companies. Leaders are faced with the necessity of developing effective strategies to manage the use of tokens, the essential units of information for the functioning of AI models. It is crucial to allow employees to explore the capabilities of agents while adhering to strict financial guidelines.

A Year in AI: A Radical Shift

Steve Lucas, CEO of Boomi, a company specializing in integration technology, expressed his concerns about the rushed adoption of generative AI by IT leaders last year. Today, he observes a similar trend with agentic technology, where token usage is experiencing rapid and uncontrolled growth. According to Lucas, whether working within or outside a company, the general impression is that the use of AI and the optimization of tokens have become professional priorities.

Lucas emphasized that, in the current context, IT professionals seem to be focusing more on the use of AI than on their core tasks. This trend of maximizing token usage to enhance professional performance has become the norm, but it raises questions about the financial sustainability of such practices.

The Unpredictable Rise in Agent Costs

Tokens represent the basic unit processed by an AI model, and their consumption incurs costs that increase unpredictably. Internal rankings that tracked token usage have become obsolete, as token waste has become a luxury few can afford. In this agentic era, token-maxing is seen as a sign of overconsumption, and tokenomics — the art of measuring, pricing, and managing token usage — has become a crucial business priority. Lucas highlights that token consumption has reached an unsustainable pace, illustrating this with a tenfold increase in spending at Boomi for the Claude model compared to the previous year.

Lucas explained that the concept of tokenomics was not even discussed the previous year, but today it is at the forefront of leaders' concerns. The exponential consumption of tokens has forced companies to reevaluate their strategies to avoid uncontrolled spending that could jeopardize their profitability.

Towards a Business-Adapted Tokenomics

Companies must now develop their own approach to tokenomics to adapt to agentic AI, which promises to transform business operations. Lucas stresses the importance of creating a cost-effective method for exploring agents, as the economics of AI have become a central issue. Organizations must ensure that AI provides a tangible return on investment, which is the fundamental question for the future.

Lucas stated that the key for companies is to ensure that AI can operate with a positive return on investment. This means that companies must not only adopt AI but also do so in a way that every expense is justified by measurable gains in productivity or revenue.

Encouraging Innovation While Controlling Costs

Sridhar Ramaswamy, CEO of Snowflake, acknowledges that token consumption is increasing within his organization, but he believes it is essential to give employees the freedom to explore agents. While concerned about the costs associated with AI inference, he does not see this as a sufficient reason to curb AI usage. Agents can enhance efficiency and productivity while enabling the development of new services for customers more quickly and effectively.

Ramaswamy explained that, despite rising costs, investing in AI is justified by the potential gains in speed and efficiency. He is convinced that AI can transform internal operations and provide significant competitive advantages.

Investing in Agent Exploration

Matt Luizzi, Vice President of Analytics at Whoop, shares this vision and invests in exploring agents to gain a competitive edge. Luizzi emphasizes the importance of encouraging staff to step out of their comfort zones while accepting the associated risks. However, he insists on the need to establish safeguards and monitoring systems to control token-related expenses.

Luizzi stated that for innovation to thrive, employees must be allowed to experiment, even if it involves initial costs. However, it is crucial to monitor these expenses to prevent them from becoming uncontrollable, by implementing tracking systems and clear limits.

Cautious Management and Innovation

Luizzi highlights the importance of cautious management to succeed in the agentic era. It is crucial to allow employees to push boundaries while avoiding excesses. Sriram Sitaraman, CIO at Synopsys, adds that innovation requires freedom, but it is essential to set limits to avoid unnecessary spending.

Sitaraman emphasized that while innovation is vital, it must be framed by clear guidelines to prevent resource waste. He insisted that companies must find a balance between exploration freedom and cost control.

Establishing Guidelines for Token Usage

Sitaraman proposes that the right context for projects can help establish effective constraints. By defining clear objectives and specific data, token consumption can be limited while maximizing added value for the user. François-Xavier Pierrel, Chief Data Officer at TF1, compares token usage to a sugar addiction, highlighting the need for a common-sense approach to avoid costly overconsumption.

Pierrel explained that just as sugar can become an addiction, excessive token usage can lead to unnecessary costs. He recommends a measured and thoughtful approach to managing token usage, ensuring that every expense is justified by tangible added value.

Caution as the Watchword

Pierrel advocates for caution in the use of large language models, suggesting opting for smaller and less expensive models whenever possible. This approach allows for agility in experimentation while avoiding excessive bills. Pierrel concludes that the tokenomics strategy will be crucial for creating value while controlling costs in the deployment of agents.

Pierrel concluded by stating that companies must be agile and ready to adjust their strategies based on the results obtained. He emphasized that the goal is not to deploy a large number of agents but to do so effectively and profitably, ensuring that each agent truly contributes to achieving the company's objectives.