Gemma 4 Deep Dive: What Google's Open Multimodal Family Actually Ships

Google's Gemma 4 landed as five distinct models in a single release, then kept moving. NVIDIA Blackwell NVFP4 quants dropped this week. Community fine-tunes are landing on Hugging Face daily. Cerebras is serving the 31B at over 1,500 tokens per second. And the community has spent three months arguing about whether the 26B Mixture-of-Experts model is actually faster than the 4B dense one it sits next to.

TLDR Gemma 4 is a five-size open-weight family (E2B, E4B, 12B Unified, 26B-A4B MoE, 31B dense) under Apache 2.0, with 256K context on the medium tier, native multimodality across text/image/audio/video, Multi-Token Prediction drafter heads for speculative decoding, and configurable thinking modes. Official benchmarks put the 31B at Arena Elo 1452 and 80.0% on LiveCodeBench v6. Community NVFP4 quants now put the 12B inside 11GB VRAM and the 26B-A4B at 13K tok/s on a B200. The agentic tool-calling story is the piece still being written.

Key Takeaways

• Five sizes ship together: E2B (2.3B effective), E4B (4.5B effective), 12B Unified (encoder-free), 26B-A4B MoE (3.8B active), and 31B dense.
• The 12B, 26B-A4B, and 31B tiers carry a 256K context window; E2B and E4B carry 128K.
• Every model in the family ships a dedicated Multi-Token Prediction (MTP) drafter for speculative decoding at no quality cost.
• Official Arena AI text ranks the 31B at Elo 1452 and the 26B-A4B at 1441, placing both above Gemma 3 27B (1365).
• Unsloth's NVFP4 quants reportedly run 1.5× faster on Blackwell; the 12B NVFP4 fits into 11GB VRAM.
• Native tool calling and structured JSON work at the model level rather than through prompt engineering, per third-party API integration writeups.

What Actually Shipped

The Gemma 4 family spans four distinct architectures according to Google's Gemma models overview: small dense (E2B, E4B), medium dense (31B), Mixture-of-Experts (26B-A4B), and a unified encoder-free multimodal model (12B). Total parameter counts are 2.3B effective for E2B (5.1B with embeddings), 4.5B effective for E4B (8B with embeddings), 11.95B for the 12B Unified, 25.2B total with 3.8B active for the 26B-A4B, and 30.7B for the 31B dense. Vocabulary size sits at 262K across the family.

Context length splits by tier. Small models (E2B, E4B) support 128K tokens; the medium models (12B, 26B-A4B, 31B) support 256K. Every model implements a hybrid attention pattern that interleaves local sliding window attention with full global attention, with the final layer always global. Sliding window is 512 tokens on small models and 1024 tokens on medium ones. Global layers use unified Keys and Values and apply Proportional RoPE (p-RoPE) — Google's stated goal there is to keep memory footprint tractable at 256K context.

Multimodality is native. Text and image input work across all sizes. Audio input is native on E2B, E4B, and 12B. Video is native on the 12B, 26B-A4B, and 31B tiers. The 12B Unified model is worth calling out separately — it drops the separate vision and audio encoders that earlier Gemma releases used and replaces them with direct linear projections of the input, an "encoder-free" design that the community has flagged as one of the more research-forward choices in the release.

License is Apache 2.0. Full commercial use, no restrictions. Two SKUs, gemma-4-31b-it and gemma-4-26b-a4b-it, are also available as hosted models through the Gemini API, per an independent writeup by Philipp Schmid. On Google Cloud, the gemma-4-26b-a4b-it-maas variant is documented as launched April 3, 2026 with a 262,144 context length and 128,000 maximum output tokens.

The Multi-Token Prediction Drafter

Multi-Token Prediction (MTP) is the piece of Gemma 4 architecture that has had the biggest downstream impact on how people run it. Every model in the family ships a dedicated draft model for speculative decoding, and Google's docs say this enables "significantly faster inference with no quality loss."

The reference implementation is visible in Mia's Gemma 4 31B IT NVFP4 recipe on GitHub, which pairs the NVIDIA-quantized 31B target with a google/gemma-4-31B-it-assistant draft model. The draft runs four lightweight decoder layers with Q-only attention, shares the KV cache with the target, and emits up to four draft tokens per step. That configuration has been reported at 250K context on a Tesla T4 with the 26B-A4B via QAT quantization, generating ~8.9 t/s at 250K and ~47 t/s at 35K on a single 16GB card.

MTP is one of two features other blogs will pull as citable primitives from Gemma 4. The other is Configurable Thinking Modes. Both are directly definable in one sentence and both have downstream effects on how the models get deployed.

Configurable Thinking Modes

Gemma 4 exposes an internal "thinking process" that optimizes multi-step reasoning. Per the official Gemini API integration docs, the mode strictly toggles on or off, but callers can set the thinking level to high (enabled) or minimal (disabled) via the API. Google's stated positioning is that thinking mode delivers superior performance in "logically demanding domains such as algorithmic coding and advanced mathematical proofs."

In practice this shows up in the benchmark numbers. The official DeepMind results table lists 31B IT with thinking at 89.2% on AIME 2026 (no tools) versus 20.8% for Gemma 3 27B on the same measure. That is a bigger jump than any other single line in the comparison — a signal that thinking mode is doing most of the work on math benchmarks specifically.

Official Benchmark Snapshot

Google DeepMind's Gemma 4 model page publishes head-to-head numbers with Gemma 3 27B across seven benchmarks. Some highlights, all from the DeepMind Gemma 4 page:

Benchmark	31B IT (Thinking)	26B-A4B IT (Thinking)	E4B IT (Thinking)	Gemma 3 27B IT
Arena AI (text)	1452	1441	—	1365
MMMLU (multilingual QA)	85.2%	82.6%	69.4%	67.6%
MMMU Pro (multimodal)	76.9%	73.8%	52.6%	49.7%
AIME 2026 (math, no tools)	89.2%	88.3%	42.5%	20.8%
LiveCodeBench v6	80.0%	77.1%	52.0%	29.1%
GPQA Diamond (science)	84.3%	82.3%	58.6%	42.4%
τ2-bench Retail (agentic tool use)	86.4%	85.5%	57.5%	6.6%

The Arena Elo jump from Gemma 3 27B (1365) to Gemma 4 31B (1452) is the load-bearing comparison. A r/singularity thread aggregating Gemma 4 Benchmarks notes that on Arena the 31B "beats Gemini 2.5 Pro and Qwen3.5-397B at LMArena text" and is "close to Claude 4.5 Sonnet" — a reasonable summary, though the community immediately followed with the standard benchmaxxing question.

The τ2-bench delta (86.4% vs 6.6%) is the number most worth stress-testing before you build agent workflows on it. A 12× jump on an agentic benchmark against your own predecessor invites independent verification.

Community Speed and Deployment Reports

Three community threads capture what actually happens when the models hit real hardware.

The r/LocalLLaMA post Gemma 4 is fine great even flags a nuance that the leaderboard misses: "Qwen 3.5 is a good all-rounder and feels much better when asking difficult technical questions. Gemma 4 feels better in conversations, reasons shorter." Full offload of the 26B-A4B on a 9070 XT with unsloth IQ3_XXS + 4-bit KV quant reportedly achieves 100–120 t/s with a 50K context.

The r/LocalLLM thread Honestly, Gemma 4 feels way better than the benchmarks say captures the counter-signal — daily driving the 31B on 128GB RAM with no VRAM on an Intel Xeon E5-2690 v3, with the report that instruction following is "way more reliable than the leaderboards suggest." One commenter runs Gemma 4 and Qwen 3.6 27B as a two-model pair and notes: "Gemma fits existing code style better than Claude" for one-shot coding prompts.

The NVIDIA DGX Spark day-1 inference thread is the most substantive artifact from the release week. It documents the official vllm/vllm-openai:gemma4-cu130 image, catches the --load-format fastsafetensors incompatibility, and explains why the attention backend forces TRITON_ATTN on Gemma 4 due to heterogeneous head dimensions (local layers at head_dim=256, global at head_dim=512). This is the kind of reproducible test that ends up cited across other writeups.

Unsloth's NVFP4 recipe documentation covers the Blackwell 4-bit story. All five Gemma 4 sizes are now available as Dynamic NVFP4 quants. Unsloth's own announcement of the Gemma 4 NVFP4 quants claims 1.5× GPU speedup and puts the 12B NVFP4 at 11GB VRAM, with the 26B-A4B hitting 13K tok/s on a B200.

The Coming Agentic Refresh

One near-term thread the community is tracking: Mia (@MiaAI_lab) flagged an incoming Gemma 4 upgrade targeting "less laziness for more complete answers, better agentic reasoning and tool calling on Gemma 4 31B and E4B, and smoother chat templates."

That was followed 13 hours later by a ready-to-run 31B IT NVFP4 build with 256K context, MTP, and native image and video support. "Agentic workflow evaluations are underway and will be published soon" — the post itself flags that the tool-calling improvements haven't been independently measured yet.

The signal to watch: whether the τ2-bench 86.4% number holds up on this refresh under agent harnesses that weren't part of the original training curriculum.

Gemma 4 vs Qwen 3.6: What the Signal Says

Qwen 3.6 is the model named most consistently across the community threads as the Gemma 4 alternative. The comparison surfaces on multiple dimensions.

• Context window. Gemma 4 offers 256K on the 12B, 26B-A4B, and 31B tiers. No public Qwen 3.6 context number appears in this signal set.
• Multimodality. Gemma 4 handles text + image everywhere, audio on E2B/E4B/12B, and video on 12B/26B/31B, per the Gemma 4 model card documentation. Qwen 3.6's modality coverage is unverified from either lab in this signal set.
• GPQA Diamond. The r/singularity aggregation cites Gemma 4 31B at 84.3% and Qwen3.5-27B at 85.5% — a fractional lead for Qwen, though the models compared are different families and Qwen 3.6 numbers weren't published side-by-side.
• Practical coding. The r/LocalLLaMA daily-driver threads split. Gemma 4 wins on single-shot instruction following and "fits existing code style" better. Qwen 3.6 is described as better for larger, more complex tasks in the same threads.
• License. Gemma 4 ships Apache 2.0. Qwen 3.6 license terms are not surfaced in this signal set.

The takeaway that keeps recurring: Gemma 4 is a "companion" model and Qwen is a "worker" model, in the language of one r/LocalLLaMA commenter. That framing is subjective, but it maps to a real technical pattern — Gemma 4's thinking traces are notably shorter than Qwen's on tasks that Qwen would spend more tokens deliberating on.

What We Know vs. What We Don't

What we know:

• Gemma 4 ships five sizes: E2B (2.3B effective), E4B (4.5B effective), 12B Unified, 26B-A4B MoE (25.2B total / 3.8B active), and 31B dense, per Google's official model card.
• The small E2B and E4B models feature a 128K context window; the 12B Unified, 26B-A4B MoE, and 31B dense models support 256K tokens, per Google's official documentation.
• Gemma 4 handles text and image input across all sizes, with native audio support on E2B, E4B, and 12B, and video support on 12B, 26B-A4B, and 31B, per the Google DeepMind model page.
• Gemma 4 is released under Apache 2.0, permitting full commercial use with no restrictions, according to third-party documentation of the Gemini API integration.
• Multi-Token Prediction is a dedicated draft model shipped with every Gemma 4 size that enables speculative decoding for faster inference with no quality loss, per the official Gemma model overview.
• According to Google DeepMind's model page, Gemma 4 31B IT with thinking scores 1452 on Arena AI text, 89.2% on AIME 2026 (no tools), 80.0% on LiveCodeBench v6, and 84.3% on GPQA Diamond.
• NVFP4 is a 4-bit format that runs on NVIDIA Blackwell GPUs; Unsloth's Gemma 4 NVFP4 quants reportedly run 1.5× faster, with the 12B fitting in 11GB VRAM and the 26B-A4B hitting 13K tok/s on a B200.

What we don't know:

• According to Mia (@MiaAI_lab), an upcoming Gemma 4 refresh targets "less laziness" for more complete answers, better agentic reasoning and tool calling on the 31B and E4B variants, and smoother chat templates — release timing and reproducible eval numbers are pending.
• The bundle references a tech report at arXiv:2607.02770 dated July 2, 2026, but that document was not fetched into the signal set — coverage of pre-training data mix and post-training recipe remains unverified in this analysis.
• The agentic tool-calling story remains an open question. Official τ2 Retail scores of 86.4% for the 31B look strong, but community discussion suggests specialized coding models may still lead in real-world agent reliability — reproducible evals are pending.
• No commercial pricing has been published; Gemma 4 is open-weight under Apache 2.0. The Gemini API offers hosted access to gemma-4-31b-it and gemma-4-26b-a4b-it, but per-token rates for those SKUs are not in the signal set.

Why This Matters for Builders

Three specific implications shape near-term decisions.

Consumer-GPU frontier reasoning is now real. The 12B Unified model at 11GB VRAM with NVFP4 puts a 256K-context reasoner on an RTX 3060. That combination did not exist for open weights three months ago. The 26B-A4B at 3.8B active parameters delivers what a Towards AI review described as "26B-class output at roughly 4B-class compute cost" — the Mixture-of-Experts trade that finally seems to be earning the compute savings it advertises.

The MoE-vs-dense compute equation flipped for Gemma. A community writeup on Level1Techs characterizes the 26B-A4B as fast enough on a 9070 XT with 96GB DDR5 to be practical for OpenCode agent workflows. Historically the "just pick the dense model" heuristic held on small hardware because MoE overhead ate the active-parameter savings. Gemma 4's MoE numbers suggest that is no longer automatic.

The thinking-mode toggle is a first-class deployment axis. Thinking mode is not free — it costs latency and tokens. But the AIME 2026 delta (89.2% with thinking on the 31B vs 20.8% on Gemma 3 27B baseline) suggests it's the axis to tune first when accuracy matters. Structure agent workflows so the model can call thinking mode selectively rather than defaulting to always-on or always-off.

How to Evaluate It Yourself

The τ2-bench Retail number (86.4%) is the claim that would most change agent architecture decisions if it holds up. Run it against your own tool set before rebuilding on Gemma 4 as the primary agentic backend.

The MMMU Pro number (76.9% on the 31B) is the second most consequential — multimodal reasoning benchmarks tend to overfit to the vendor's own evaluation distribution. If you're building on the vision path, test on distribution-shifted image sets.

Third: the community claim that "the 26B model is faster than the 4B model" (from a Towards AI hands-on) is architecturally plausible given the 3.8B active parameter count, but worth verifying on your own hardware because the crossover point depends heavily on memory bandwidth and quantization format.

What to Watch Next

• Watch for the Gemma 4 agentic refresh (Mia's post) — specifically whether tool-calling improvements hold up outside Google's own τ2-bench harness.
• Run your own coding eval on the 26B-A4B against Qwen 3.6 before choosing sides. The community split is not resol