DiffusionGemma: Google's Experimental Diffusion LLM

On June 10, 2026, Google released DiffusionGemma — an experimental open model, built on the Gemma 4 architecture, that generates text using diffusion instead of the usual token-by-token approach. It's a 26B Mixture-of-Experts model (3.8B active parameters) released under Apache 2.0, and it can hit 1,000+ tokens/second on an H100 and 700+ tokens/second on an RTX 5090 — up to 4x faster than standard Gemma 4. The catch: it scores lower than Gemma 4 on quality benchmarks like MMLU and coding tests, and Google explicitly labels it experimental rather than production-ready. This guide covers what it is, how the speed trick works, the honest trade-offs, and — critically — whether you can actually run it locally today.

What Is DiffusionGemma?

DiffusionGemma takes the same 26B MoE backbone used in Gemma 4 26B A4B and swaps out the text generation method. Instead of predicting one token, then the next, then the next — the way essentially every chatbot you've used works — it generates an entire block of text at once, refining it over several passes.

Google isn't shipping this as a replacement for Gemma 4. It's positioned as a research preview: a real, usable, open-weight model that demonstrates a fundamentally different way to generate text, aimed at people building latency-sensitive applications who are willing to trade some quality for raw speed.

How Text Diffusion Actually Works

If you've seen an image diffusion model like Stable Diffusion work, you already understand the core idea. Image diffusion starts with a canvas of pure random noise and gradually removes that noise over many steps until a coherent picture emerges — the model doesn't paint pixel by pixel, left to right; it refines the whole image simultaneously, pass after pass.

DiffusionGemma does the same thing to text. Instead of noise pixels, it starts with a block of 256 placeholder tokens — essentially blanks. Across a handful of denoising passes, the model fills in and revises those blanks, and every token in the block can attend to every other token in the block at each step, rather than only looking backward at what's already been "written." By the final pass, the placeholders have resolved into coherent text, and the model moves on to generate the next block.

Compare that to standard autoregressive generation (what Gemma 4, GPT, and nearly every other LLM does): each token is produced one at a time, strictly left to right, and each new token depends on a full forward pass through the model. Generating 256 tokens autoregressively means 256 sequential forward passes. DiffusionGemma generates that same 256-token block in a small, fixed number of denoising passes — regardless of how long the block is. That's where the speedup comes from: the number of forward passes doesn't scale with output length the same way.

Specs at a Glance

Structurally, this is the same total/active parameter split as Gemma 4 26B A4B — the MoE catch applies here too: all 25.2B parameters need to be resident in memory even though only 3.8B activate per step.

The Speed Numbers

Google and NVIDIA both published throughput figures, and they're consistent across sources:

Google frames this as up to 4x faster than standard Gemma 4 generation at comparable settings. That's a real, measured gain, not a rounding-error claim — but it's a gain in raw tokens/second, and it doesn't come free.

The Honest Trade-off: Quality vs. Speed

Here's the part that matters most if you're deciding whether to actually use this model. Per Google's own model card, DiffusionGemma trails standard Gemma 4 26B A4B on every major quality benchmark:

That's not a marginal gap — on AIME and Codeforces specifically, it's a substantial drop. Google's own guidance is direct about this: DiffusionGemma is recommended for speed-critical applications, and standard Gemma 4 is recommended wherever output quality matters most. This isn't hedging — it's Google telling you plainly which model to reach for depending on the job.

The practical read: if your bottleneck is latency (an agent loop that runs the model dozens of times per user action, live autocomplete, high-throughput batch jobs), the speed may be worth the accuracy trade. If you're doing anything where a wrong answer costs you — coding, math, research synthesis — standard Gemma 4 is still the better tool.

Can You Actually Run It Locally Today?

This is where a lot of coverage glosses over reality, so here's the honest, verified state of local tooling as of July 10, 2026, one month after release.

What works out of the box:

• Hugging Face Transformers — runs DiffusionGemma directly on an RTX 5090 or NVIDIA DGX Spark with no extra patching. Weights are on Hugging Face at google/diffusiongemma-26B-A4B-it.
• vLLM — the best-supported path. DiffusionGemma is the first diffusion LLM with native vLLM support, and vLLM merged multi-GPU support on June 26, 2026. A basic serve command looks like:

vllm serve google/diffusiongemma-26B-A4B-it \
  --max-model-len 262144 \
  --attention-backend TRITON_ATTN

• SGLang — also listed as a supported inference backend.
• Unsloth / NVIDIA NeMo / Hackable Diffusion (JAX) — available for fine-tuning, not just inference.

What does not work yet — and this is the part worth knowing before you try:

• Ollama does not support it. There's an open, unresolved GitHub issue (ollama/ollama#16664) from a user who tried loading an Unsloth DiffusionGemma GGUF and hit unknown model architecture: 'diffusion-gemma' — the underlying llama-server process crashes outright. As of this writing the issue has no assigned owner and no linked PR.
• LM Studio doesn't support it either, for the same underlying reason: both Ollama and LM Studio build on mainline llama.cpp, and mainline llama.cpp hasn't merged diffusion-gemma support.
• llama.cpp support exists only as an unmerged pull request (PR #24423), which requires building from that branch specifically and running the dedicated llama-diffusion-cli tool — the standard llama-cli and llama-server binaries can't generate from this architecture at all. Community GGUF quantizations exist (Unsloth has published a full set, from Q4_K_M at ~16 GB up to BF16 at ~47 GB), but they only load against that patched branch, not a normal llama.cpp install.
• Apple Silicon / MLX has no native support at launch. The performance work so far targets CUDA; if you're on a Mac, your only path is building llama.cpp with Metal support from the same unmerged PR branch, and expect meaningfully slower throughput than the CUDA numbers above — there's no NVFP4-equivalent optimization for Apple GPUs yet.

Bottom line: if you want to try DiffusionGemma today with minimum friction, use vLLM on an NVIDIA GPU with at least 18 GB VRAM (an RTX 4090 or 5090), or Hugging Face Transformers if you just want to poke at it in Python. If your workflow depends on Ollama or LM Studio, this model isn't there yet — check back once the llama.cpp PR merges. For everything else — everyday chat, coding, and every model size from E4B up — Ollama remains the easiest way to run Gemma 4 itself.

Who Should Actually Try This

DiffusionGemma makes the most sense for workloads where latency per response, not peak accuracy, is the binding constraint:

• Agentic loops that call a model many times per task (planning, tool selection, self-critique) where each call's turnaround time compounds.
• Autocomplete and inline-suggestion tools, where users notice a 200ms delay far more than a small drop in suggestion quality.
• High-throughput batch generation — summarizing, tagging, or drafting large volumes of text where raw speed determines cost and turnaround.

It makes less sense as a general-purpose chat or coding assistant, where the benchmark gap versus standard Gemma 4 is large enough to notice in real use. If you're choosing a daily-driver model and don't have a specific latency problem to solve, start with the hardware requirements guide and pick a standard Gemma 4 size instead.

FAQ

Is DiffusionGemma better than Gemma 4? Not on quality — it scores lower across MMLU Pro, GPQA Diamond, AIME, and coding benchmarks. It's "better" specifically on generation speed, where it's up to 4x faster. Which one is "better" depends entirely on whether your use case is latency-bound or quality-bound.

Can I run it with Ollama? Not yet, as of July 2026. Ollama's GitHub tracks this as an open, unassigned issue (#16664) — it fails with an "unknown model architecture" error because Ollama builds on mainline llama.cpp, which hasn't merged diffusion-gemma support. Your working options today are vLLM or Hugging Face Transformers.

What hardware do I need? Quantized, DiffusionGemma fits within about 18 GB of VRAM — an RTX 4090 or 5090 is a comfortable fit. For full performance, Google and NVIDIA's numbers are based on data-center hardware (H100, DGX Spark, DGX Station), though the RTX 5090 figure (700+ tok/s) shows it performs well on consumer-grade GPUs too. Apple Silicon isn't natively optimized for it yet.

Is DiffusionGemma safe to use in production? Google explicitly calls it experimental and recommends standard Gemma 4 for production use cases where quality is the priority. Treat DiffusionGemma as a preview to evaluate for specific latency-sensitive workloads, not a drop-in production upgrade.

What's Next?

If DiffusionGemma's speed numbers have you curious about the rest of the Gemma 4 lineup, start with Gemma 4 Benchmarks to see how the standard models actually perform, then check Gemma 4 Hardware Requirements to figure out which size fits your machine. And once you've picked a model, Run Gemma 4 with Ollama will get you from zero to a running model in about ten minutes — no unmerged pull requests required.