Google DeepMind Cuts Costs with DiffusionGemma

Google DeepMind Cuts Costs with DiffusionGemma

Google has demonstrated that it is not necessary to train a text diffusion model from scratch by adapting Gemma 4 into a diffusion model. The recently published technical report explains how it works and the trade-offs involved.

Google DeepMind launched DiffusionGemma as a model in mid-June and has now released a technical report. Unlike standard language models that generate text one token at a time, DiffusionGemma refines blocks of 256 tokens in parallel, similar to how image AIs extract an image from noise. On an Nvidia H100 accelerator, the model achieves around 1,500 tokens per second.

Creating a new model from scratch was unnecessary. The team started with the existing model Gemma-4-26B-A4B and converted it into a diffusion model using less than ten percent of the initial training token budget, according to the report.

DiffusionGemma offers several times the output speed of Gemma 4 models and previous diffusion models while maintaining comparable accuracy.

Two Training Steps Balance Quality and Speed

In the first of two steps, the model learns to reconstruct noisy text blocks from example data. This is followed by a combined phase of reinforcement learning and sampler distillation, which Google calls SD·RL. Reinforcement learning generally improves the quality of responses, while sampler distillation allows the model to operate with fewer computational steps. Google merges the two into a single process.

According to the report, this combined approach increases quality on reasoning benchmarks by an average of ten points while nearly quadrupling the number of tokens per computational step. As a result, DiffusionGemma's responses are about 50% shorter, further increasing speed.

Bidirectional Reasoning Allows the Model to Self-Correct

Standard language models must commit to the first digit of a response before completing the reasoning. In a math problem from the report, Gemma 4 starts its answer with "-1," realizes during its reasoning that "-25" is correct, and adds a correction afterward. DiffusionGemma develops the response and reasoning in parallel, allowing it to correct errors before the output is finalized.

DiffusionGemma can correct an initial incorrect response during later denoising steps, unlike an autoregressive model.

Sudoku solving operates on the same principle, as each entry depends on inputs that come later. After minimal adjustment, DiffusionGemma correctly solves nearly 85% of puzzles, while the base model completely fails at this task. Structured outputs like JSON or code repairs finish after only two to three refinement steps, as the input already determines most tokens.

DiffusionGemma also retains its original ability to generate text word by word, allowing users to switch between the two modes depending on the task.

Reasoning Gaps and Multi-User Limits Persist

Absolute performance is lower than that of the autoregressive base model. Google cites several reasons for this. DiffusionGemma was not trained as a diffusion model from the outset but was adapted later. The subsequent training phase was relatively short, and the second step, SD·RL, prioritized speed over maximum quality. The architecture, training data, and other parameters were also carried over from the old Gemma 4 model, which are not necessarily ideal for diffusion.

The model sometimes gets stuck in repetition loops, producing individual words multiple times in a row. This is an artifact of the aggressively reduced computational steps. In multimodal tasks, DiffusionGemma sometimes forgets to properly close its reasoning section, artificially lowering benchmark scores.

The speed advantage primarily applies to single-user scenarios. Once around 32 concurrent requests reach the model, standard language models catch up in terms of throughput.

Google explicitly describes DiffusionGemma as an experimental model and states that this publication aims to accelerate research on text diffusion while providing the community with a foundation for specialized and resource-efficient adaptations.

The model is already being used by the startup Interfaze for multilingual speech recognition and in a research project on generating interactive radiology reports. Google has previously made the model available under an Apache 2.0 license on Hugging Face. Its predecessor is Gemini Diffusion, which Google introduced in May 2025.