PrismML Brings a 1-Bit LLM to Snapdragon Glasses

PrismML unveils a version of its LLM Bonsai capable of running locally on glasses equipped with the Snapdragon AR1 Gen 1 platform. This initiative is part of a strategy for open-weight AI executed on-device, with no commercial announcement of glasses at this stage.

No device announced, but a step towards embedded AI

No smart glasses integrating PrismML have been announced. The release of a model for a Qualcomm chip marks a step towards PrismML's goal: to offer open-weight AI functioning directly on devices. PrismML aims to leverage the computing power already present in these terminals and presents this approach as an alternative to reliance on proprietary labs, their promises of privacy, and their increasing resource demands.

The LLM Bonsai runs locally on Snapdragon AR1 Gen 1

At the Snapdragon Summit, Qualcomm showcased Bonsai, a 1-bit LLM developed by PrismML. This model can operate locally on smart glasses equipped with the Snapdragon AR1 Gen 1 platform. PrismML is thus adapting its small language models to Snapdragon chips, targeting applications on smart glasses with on-device processing.

Specifications for glasses: 2 billion parameters and vision-language

The variant designed for glasses includes 2 billion parameters and has been specifically tailored for use cases combining vision and language, allowing users to query in real-time what they observe. PrismML, founded by researchers from Caltech and advised by Ion Stoica from UC Berkeley, stands out for its ability to reduce model sizes by four times while preserving nearly all of their performance on standard benchmarks, according to a source.