Feed
DiffusionGemma Technical Report
Google turned an autoregressive Gemma 4 model into a diffusion model that writes 256-token blocks at once.
1456
tokens/sec
Average decoding speed on one H100
About 7x faster than the same model writing one token per pass
How it works
One token per pass
Standard LLMs emit one token per forward pass. At batch size 1 the GPU mostly waits on memory transfers.
Fine-tune into diffusion
Start from Gemma 4 26B A4B. Teach it to denoise 256-token blocks, then use RL to cut denoising steps.
20 tokens per pass
About 12 denoising steps per block give roughly 1,500 tokens/sec. Benchmark scores are somewhat below the original model.
From the paper