PrismML compresses Qwen 27B to 5.9 GB, targets device usage

Key Takeaways

1PrismML launches Bonsai 2 27B, a compressed version of Qwen3.8 27B at 5.9 GB

2The model achieves 98% of the original's benchmark scores and runs on PC

3The compression relies on ternary weights (+1, -1, 0) instead of 16 bits

4Models with several hundred billion parameters are planned

Why it matters : Compression enables the execution of advanced LLMs locally, paving the way for private and accessible use on common devices.

With Bonsai 2 27B, PrismML offers a compressed version of the Qwen3.8 27B model reduced to 5.9 GB. The California startup claims to achieve 98% of the original's benchmark scores and is already preparing versions with several hundred billion parameters.

On-device, free and private according to Ion Stoica, but total parity remains uncertain

For Ion Stoica, advisor to PrismML, the technology enables advanced models to run directly on users' devices. He believes that intelligence will be accessible at no additional cost, as it will operate on the device already owned, and that privacy will be preserved since data will not be sent to the cloud. Babak Hassibi, CEO of PrismML, indicates that compression will likely always have an effect on performance and that achieving 100% parity remains uncertain. However, he considers that reaching 100% becomes more feasible with larger models.

Bonsai 2 27B reduces Qwen3.8 27B to 5.9 GB, achieving 98% of benchmarks

PrismML has launched Bonsai 2 27B, which compresses Alibaba's open-source Qwen3.8 27B model to a size of 5.9 GB. This reduction represents a factor of 9 to 10 in memory compared to the original and allows the model to fit on a PC, with high-end smartphone usage being considered. Bonsai 2 achieves 98% of the aggregated benchmark scores of Qwen. The first Bonsai, launched in March, matched 95%, and according to PrismML, has been downloaded over 11 million times, while even smaller models total an additional 2.6 million downloads.

Le brief IA que lisent les pros

Le brief IA que les pros lisent chaque soir

Les 7 actus IA du jour, décryptées en 5 min. Gratuit.

Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.

Choisissez votre rythme

Gratuit · Pas de spam · Désabonnement en 1 clic

Compression through ternary weights: from 16 bits to +1, -1, 0

PrismML's method involves reducing model size by acting on the network weights, which represent the information learned during training. While a weight typically requires 16 bits, the ternary weight approach limits each value to +1, -1, or 0. This reduction in granularity decreases the amount of information to be stored per parameter and significantly reduces the space occupied by the model.

A startup from Caltech, supported by Ion Stoica and recognized investors

PrismML was founded by researchers from Caltech and is led by Babak Hassibi, a professor at the institute and an expert in compression technologies. Ion Stoica, co-founder of Databricks and director of the Sky Computing Lab at Berkeley, is among the advisors. The Sky Computing Lab has contributed to several technologies and startups, including Letta and SGLang. PrismML is backed by Khosla Ventures, Cerberus Capital, and Caltech, having raised $22.25 million. The company advocates that a high-performing LLM does not need to be massive and designs models suitable for PCs and smartphones. Discussions with Apple are mentioned, but Babak Hassibi declined to comment on this point. In the field of LLM compression, Multiverse Computing is also active, founded by a professor from the Donostia International Physics Center in Spain, with significant fundraising efforts.

Next milestone: models with several hundred billion parameters

PrismML plans to launch models with several hundred billion parameters in the coming months. Babak Hassibi anticipates that it will be easier to retain intelligence in these models, believing that increasing model size provides more room for compression without a significant loss in performance.