ZML Challenges Nvidia with Free Multi-Chip AI Software

ZML: A French Startup Disrupting the AI Chip Market

In a market largely dominated by Nvidia, new players are emerging to offer innovative alternatives. Among them is ZML, a promising French startup backed by renowned artificial intelligence researcher Yann LeCun, which stands out for its bold approach. It recently launched a high-performance inference software that enables various open-source large language models to run on a multitude of chips, including those from Nvidia, AMD, Google TPU, Apple Metal, and Intel Arc.

An Inference Server to Break Barriers

ZML's flagship product, ZML/LLMD, is an inference server designed to maximize the performance of chips used in artificial intelligence. According to Steeve Morin, founder of ZML, the goal is to overcome current limitations and allow users to fully harness the potential of different chips, sometimes even beyond their nominal capabilities. This innovation aims to eliminate the software and architectural barriers that often lead to vendor lock-in, a recurring issue in the AI sector.

A Solution to Rising AI Costs

As artificial intelligence becomes increasingly integrated into our daily lives, optimizing inference is becoming crucial. ZML hopes to provide companies and cloud providers with the ability to use a mix of chips, some of which may be less expensive or more energy-efficient. "The idea is to empower people to create their own systems and achieve real efficiency gains that allow AI to be widespread," Morin stated.

Support for European Chip Manufacturers

ZML's software could also benefit new AI chip manufacturers, many of which are based in Europe. Morin mentioned companies like Axelera, Fractile, Kalray, OLIX, Q.ANT, SiPearl, SpiNNcloud, and VSORA. For him, the important thing is that ZML can collaborate with these players on innovative projects in the field.

A Positive Relationship with Nvidia

Despite this desire for diversification, Morin is not pessimistic about Nvidia. He emphasizes that ZML maintains good relations with the AI chip giant, which is also preparing for the rise of inference. This sector is experiencing such enthusiasm that it is often compared to a "gold rush."

Fierce Competition in the Inference Field

ZML is not alone in this expanding market. It faces competitors such as Baseten, recently valued at $13 billion, Inferact, creators of the open-source vLLM project, and RadixArk, the company behind SGLang. Although vLLM and SGLang partially compete with LLMD, ZML's ambitions are broader. "We have reached a point where we co-design silicon," Morin stated.

A Small but Effective Team

ZML's rapid progress is largely due to its small but talented team of 20 people. Based in Paris, the startup benefits from solid funding, having raised $20 million from venture capital firms such as Harry Stebbings’ 20VC, >commit, AALVC, Drysdale Ventures, Xavier Niel's Kima Ventures, Kindred Capital, LocalGlobe, and Puzzle Ventures. Steeve Morin, drawing on his experience as VP of engineering at Zenly, acquired by Snapchat for a nine-figure sum in 2017, has successfully attracted these investments.

A Free Product for Learning and Adapting

Unlike its first public project, an ML framework focused on inference launched in 2024 and updated in March, ZML/LLMD is not open source. However, it is offered for free to allow the company to learn from its usage. "I prefer to measure and [then generate revenue] where it is most effective without hindering my growth in a foolish way because I was too greedy from the start," Morin explained.

The Future of ZML/LLMD Still Uncertain

It is still too early to predict when ZML/LLMD will become a paid product or how it will be adopted. However, the support from influential figures like Solomon Hykes, founder of Dagger and Docker, Clément Delangue and Julien Chaumond from Hugging Face, as well as Yann LeCun, now with AMI Labs, shows that ZML's initiative is attracting attention. "I couldn't do ZML anywhere but in Paris," Morin concluded, highlighting the importance of the local ecosystem for the startup's development.