Language Models: Towards Essential Commodification?

Language Models: A New Commodity?

Electricity, wheat, mobile phones, and the Internet have all reached commodity status, meaning they are resources or goods with total or substantial fungibility, or simply considered essential for a modern lifestyle. By the late 1990s, mobile phones were still seen as novelties. However, in the 21st century, it is hard to imagine living without them. This evolution raises the question of whether language models are following a similar path, becoming an indispensable technological commodity.

Increasing Accessibility

A few years ago, advanced language models were considered luxury technologies. Today, they have become ubiquitous and essential for many organizations. This transformation can be explained by several key factors that have changed the technological landscape.

Decrease in Costs

In a global economic context marked by rising prices, artificial intelligence solutions stand out as an exception. The cost of processing one million tokens, or about 750,000 words, has drastically dropped from several tens of dollars to just a few cents. This reduction in costs has allowed for unprecedented accessibility to language models.

Open and Free Access

Open-weight models, such as Meta's Llama and Mistral, have helped break down exclusivity barriers. These models, according to public benchmarks, compete with or even surpass some commercial alternatives. By making access to AI more democratic, these models have enabled a larger number of users to benefit from their capabilities.

No Cost for the User

Today, free tools like Ollama allow any user to download and run high-performing models locally. This eliminates the need for paid subscriptions or reliance on third-party services, making AI accessible to everyone. This free access has transformed AI into a basic resource available to all.

The Limits of Convenience

Although the basic capabilities of AI have become more accessible, the customization and reliability of models remain major challenges. Base models can generate text or code for free, but their conversational style often remains predictable and requires fine-tuning. This limitation in the quality of outputs generated by the models highlights the need for human intervention to achieve optimal results.

Moreover, many users prefer to invest in the user experience rather than in the models themselves. Companies charge for solutions where models are tailored to interact specifically with their documents or workflows. The full acceptance of these paid solutions by all is still forthcoming, as many hesitate to fully delegate these tasks to language models.

An Uncertain Future

Despite the shift towards almost free accessibility, the question of whether language models will become the commodity of the decade remains open. Aspects such as reliability, privacy, and adaptation to specific fields, like medical or legal, remain challenges. These premium elements make the term "commodity" debatable in the current context of language models.

Iván Palomares Carrascosa is a leader, writer, speaker, and advisor in AI, machine learning, deep learning, and LLMs. He trains and guides others in the use of AI in the real world.