Google DeepMind released an experimental model that breaks from the token-by-token generation pattern. DiffusionGemma produces text in 256-token blocks, using a denoising process borrowed from image diffusion. On a single H100 with FP8 precision, it exceeds 1,100 tokens per second at low batch sizes. Community tests on an RTX 5090 show over 700 tokens per second with NVIDIA’s optimizations.

The architecture builds on Gemma 4’s 26B A4B Mixture-of-Experts backbone. Total parameters sit at 25.2 billion, but only 3.8 billion activate during inference. This keeps memory under 18GB VRAM. Instead of predicting the next token sequentially, the model starts with a canvas of random tokens and refines the entire block bidirectionally. The bottleneck shifts from memory bandwidth to raw compute.

Trade-offs are clear. Quality trails standard Gemma 4, and benchmark scores lag behind autoregressive counterparts. The model excels in low-batch, interactive settings rather than high-concurrency cloud serving. For latency-sensitive tasks—inline editing, long document iteration, structured output generation—it offers a practical alternative.

vLLM added native support immediately, handling the dual-mode weight usage: one pass for encoding the noisy canvas, another for denoising. Context stretches to 256K tokens. Multimodal inputs (text, images, video) condition the output. Over 140 languages are supported, though real-world testing will reveal gaps.

Developers can run it locally under Apache 2.0. Unsloth provides fine-tuning support, and NVIDIA offers a free endpoint for trials. The release marks one of the first practical open implementations of diffusion for text. Expect hybrid approaches mixing autoregressive drafting with diffusion refinement. For teams building interactive AI, this is a specialist tool worth testing—fast, local, and open.

Source: Webpronews