Nvidia and the Era of Tokens: Jensen Huang Redefines AI Economy

⚡

Key Takeaways

1Jensen Huang, CEO of Nvidia, highlighted tokens as a new economic unit at the GTC conference.

2Tokens, units of text, are used to measure and bill the work of AI language models like ChatGPT.

3Huang suggests integrating token budgets into engineers' salaries to boost productivity.

💡Why it matters — This approach could transform cost management and productivity in the tech sector, influencing the business models of AI companies.

Jensen Huang and His Vision of Tokens

At the recent GTC conference organized by Nvidia, Jensen Huang, the company's CEO, focused his speech on a key concept: tokens. During his discussions with analysts, Huang described the computers of the future as "factories" capable of producing these tokens. According to him, these units will become as essential in corporate budgets as laptops or software subscriptions.

Understanding Tokens

But what exactly are these tokens that Huang is talking about? Tokens are units of text, which can be a whole word or part of a word, that serve to measure and price the work of artificial intelligences. For example, a short word may be a single token, while a longer word could be split into several tokens. On average, one token is equivalent to about four characters.

The Impact of Tokens on Language Models

Large language models, such as ChatGPT from OpenAI or Claude from Anthropic, track the number of tokens processed during each user interaction. They also count the tokens generated in response, and companies are billed based on this usage. Unlike traditional pricing models, which rely on subscriptions or fixed fees, AI is billed on a pay-per-use basis. Huang envisions that, in the future, engineers could be assigned "token budgets" to enhance their productivity. During his keynote, he even suggested offering Nvidia engineers tokens worth half of their annual salary to attract talent.

Huang's Economic Argument

Huang reiterated this proposal the next day, asserting that the costs associated with tokens are justified, especially for highly paid engineers who can boost their productivity through autonomous agents capable of performing various tasks. He argued that more powerful and energy-efficient hardware could produce tokens at a reduced cost over time.

A Profitable Investment

"If I added $100 per day to the inference cost, which is the cost of tokens, I would be more than happy to do so," Huang stated, discussing the possibility of paying for these tokens in addition to engineers' salaries, particularly during peak periods. "Even if I added $1,000 during high-pressure times, I would be more than happy to do so."

The Growing Adoption of Tokens

Huang is not the only one interested in this concept. The idea of tokens is beginning to spread throughout the tech industry. Engineers are increasingly asking questions about their calculation during job interviews, and executives are considering including them in compensation packages.

The Future of Agentic AI

According to Huang, the rise of agentic AI, where agents operate autonomously without human supervision, could significantly increase the use of tokens. "Currently, our laptops are often idle," he explained. "But in the future, the computer will run continuously, generating tokens as your agents are active."