DSpark Accelerates LFM2.5-VL-3B Decoding by Up to 3.13x

Liquid AI introduces a DSpark drafter for its vision-language model LFM2.5-VL-3B. Internal measurements report decoding speedups on Apple Silicon and H100, with an addition of approximately 280M parameters. However, overall latency gains are still constrained by the weight of pre-filling and visual encoding.

Pre-filling Limits End-to-End Gains, According to Liquid AI

Pre-filling in language models is primarily related to computation, with a cost that increases in a (sub)quadratic manner according to the length of the prompt. In vision-language models, a vision encoder comes into play before the linguistic backbone processes hundreds of visual tokens in addition to text. Measurements taken on Apple and H100 chips show that this phase occupies a significant portion of the total latency, especially on Edge devices that are less powerful than datacenter GPUs. The neural accelerators per core of the M5 help reduce this gap. Speculative decoding only speeds up the decoding phase, with no effect on visual encoding or pre-filling; when these steps already dominate latency, even a significant acceleration in decoding yields only a moderate overall gain. This phenomenon is explained by Amdahl's law. In this context, Liquid AI reports end-to-end latency gains of up to 2.62x on-device and 2.27x on H100.

Measured Accelerations on Apple Silicon and H100

The evaluations use a DSpark block size of 8 and cover six vision tasks according to the MMSpec benchmark. On an M5 Max with MLX, decoding accelerates from 2.30x to 3.13x depending on the task, resulting in an overall latency improvement of 1.56x to 2.62x. On an M3 Ultra with llama.cpp, decoding gains range from 1.57x to 2.14x, and end-to-end from 1.30x to 1.77x. On H100 GPU, the drafter shows decoding accelerations ranging from 20.4x to 2.66x, with overall gains from 1.64x to 2.27x.

A Memory Overhead of Approximately 8.9% for the Drafter

The drafter adds approximately 280M parameters, which is 8.9% more compared to a target model of 3 billion parameters. According to Liquid AI, this size increase remains limited while enabling the observed accelerations.

Immediate Availability in llama.cpp, MLX-VLM, and SGLang

The DSpark integration for LFM is available from day one in llama.cpp, MLX-VLM, and SGLang. The vision draft model is offered on Hugging Face in Safetensors and GGUF formats. Liquid AI presents the LFM2.5 range as open weights, downloadable, adjustable, and deployable without restriction, with initial support for these frameworks and a family that extends from base models to audio and vision variants.

Architecture and Training of the DSpark Drafter

The vision drafter adopts the DSpark approach applied to text: it captures the hidden states of the target model at fixed layers and conditions on them to generate blocks of tokens. Image patches and textual tokens are projected into a common representation upstream, allowing operations on state vectors of the same dimension, regardless of modality, without altering the inference algorithm. Training follows a weighted vision-language SFT mix based on expected use cases, with ablations on 3, 4, and 5 layers. The selected model is attention-only with 4 layers and a block size of 9, trained over 10 epochs; the acceptance rate improved with more tokens before plateauing. At inference, a block size of 8 or 9 is recommended depending on the hardware.

Getting Started: Required Versions and Example Commands

SGLang requires a version that includes DSpark support for LFM targets (PR #40651). An example command attaches a drafter, activates the DSPARK algorithm, selects flashinfer for attention, and sets a block size of 9. The API can be accessed via an OpenAI-compatible endpoint at http://localhost:30000/v1, with the block size read from the draft configuration file; the baseline consists of removing speculative flags. For llama.cpp, a version including PR#29339 is required, with an example using a target model LFM2.5-VL-3B-F16.gguf, a mmproj, and a draft LFM2.5-2.6B-DSpark-F16.gguf, with n-max set to 8. MLX-VLM requires a version including PR#2280 and launches with a model LFM2.5-VL-3B and its associated draft; the n-max is bounded by the sidecar metadata. Speculative decoding is accurate, with the target model validating each proposed token; response times expose the draft_n to draft_n_accepted ratio.