Google DeepMind released Quantization-Aware Training (QAT) checkpoints for the Gemma 4 family. The release targets local deployment on edge devices and consumer GPUs. It follows the Gemma 4 launch in April and a 12B model two days earlier.
We compared the available Gemma 4 edge-model formats using only published numbers. The goal was simple. Show what each precision level costs in memory. Then show what QAT actually changes.
What QAT actually does
Quantization shrinks a model by lowering weight precision. Standard Post-Training Quantization (PTQ) compresses a finished model. That often degrades quality. QAT instead simulates quantization during training. The model learns to compensate for the precision loss.
Google’s AI team states its QAT results yield higher overall quality than standard PTQ baselines. Google did not publish Gemma 4 QAT benchmark scores in the announcement. For context, Gemma 3 QAT cut the Q4_0 perplexity drop by 54% using llama.cpp evaluation. We cite that only as prior-generation precedent.
The comparison task
Compare Gemma 4 E2B and E4B across three formats. The formats are BF16, Q4_0 QAT, and the new mobile QAT schema. Rank them on memory footprint, quality preservation, and on-device accessibility. Use published figures only.
Memory results
FormatE2BE4BBasis
BF16 (16-bit)9.6 GB15 GBOfficial Gemma 4 docs
Q4_0 (4-bit, QAT)3.2 GB5 GBOfficial Gemma 4 docs
Mobile (QAT, E2B)~1 GB—QAT announcement
The Q4_0 figures match the footprint of PTQ Q4_0. QAT does not change the size at a given format. It improves quality at that size. The new mobile schema delivers the additional reduction.
Using that mobile schema, Google reduced Gemma 4 E2B to about 1GB. Developers can go lower still. The text-only model without Per-Layer Embeddings needs under 1GB, dropping the audio and vision encoders.
Google DeepMind released Quantization-Aware Training (QAT) checkpoints for the Gemma 4 family. The release targets local deployment on edge devices and consumer GPUs. It follows the Gemma 4 launch in April and a 12B model two days earlier.
We compared the available Gemma 4 edge-model formats using only published numbers. The goal was simple. Show what each precision level costs in memory. Then show what QAT actually changes.
What QAT actually does
Quantization shrinks a model by lowering weight precision. Standard Post-Training Quantization (PTQ) compresses a finished model. That often degrades quality. QAT instead simulates quantization during training. The model learns to compensate for the precision loss.
Per-format breakdown
BF16 is the quality baseline. E2B needs 9.6 GB and E4B needs 15 GB. It is the reference point, not a phone deployment target.
Q4_0 QAT is the general-purpose local format. E2B drops to 3.2 GB and E4B to 5 GB. QAT preserves more quality here than PTQ at the same size. This format fits consumer GPUs. Earlier E2B testing also ran on a Raspberry Pi 5 at INT4.
The mobile format is the edge-specialized schema. It brings E2B to about 1 GB. It uses static activations, channel-wise quantization, and targeted 2-bit compression.
How the mobile schema works
Google AI team engineered four techniques for mobile hardware. Static activations pre-calculate scaling during training, reducing on-device work. Channel-wise quantization fits the design of mobile accelerators. Targeted 2-bit quantization compresses only the token-generation layers. Embedding and KV cache optimization shrinks the active memory footprint.
Core reasoning layers stay at higher precision. That protects capability while cutting storage. Developers can also deploy text-only and drop the audio and vision encoders. That trims memory further for use cases that need no multimodality.
Dimension breakdown
Scores are a qualitative ranking of the formats for on-device use. Memory is the only hard-measured axis. Quality reflects Google’s disclosed design, not measured Gemma 4 numbers. Each score has a one-line basis.
On-device accessibility1 — needs large GPU4 — consumer GPU, Raspberry Pi 55 — runs on phones
Total (/25)132121
Winner
The result is a tie by design. Q4_0 QAT and mobile QAT both score 21, but for different hardware. For phones, the mobile format leads. It reaches about 1GB on E2B and targets mobile accelerators directly. For laptops and consumer GPUs, Q4_0 QAT is the practical default. BF16 stays the quality reference, not a local choice.
Methodology and limits
Memory figures come from Google’s Gemma 4 documentation. The ~1GB E2B figure comes from the QAT announcement. Quality is Google’s stated claim. No independent Gemma 4 QAT quality numbers were published at release. We did not run the models locally for this comparison. Developers should test at their own quantization and workload before building.
Key Takeaways
Q4_0 QAT cuts Gemma 4 E2B to 3.2 GB and E4B to 5 GB, from 9.6 GB and 15 GB at BF16.
A new mobile QAT schema brings E2B to about 1 GB; text-only without PLE goes under 1 GB.
QAT changes quality at a given size, not the size itself; the mobile format drives the extra memory cut.
Google claims higher quality than PTQ but published no Gemma 4 QAT benchmark numbers at release.
Weights ship today on Hugging Face with llama.cpp, Ollama, LM Studio, vLLM, MLX, and LiteRT-LM support.
Marktechpost’s Visual Explainer
Marktechpost · Benchmark
Gemma 4 QAT: Comparing Q4_0 and the New Mobile Format
Google DeepMind released Quantization-Aware Training checkpoints for Gemma 4. We compared three edge-model formats on published numbers.
Memory from official Gemma 4 docs. Quality from Google’s stated claim. No models run locally.
Format 1 of 3 · Reference
BF16 (16-bit)
13 / 25
The full-precision quality baseline. E2B needs 9.6 GB and E4B needs 15 GB.
Top observation: a reference point, not a phone or laptop deployment target.
Format 2 of 3 · Laptop / GPU
Q4_0 QAT (4-bit)
21 / 25
The general-purpose local format. E2B drops to 3.2 GB and E4B to 5 GB.
Top observation: QAT preserves more quality than PTQ at the same 4-bit size.
Format 3 of 3 · Mobile
Mobile QAT
21 / 25
The edge-specialized schema. Brings E2B to about 1 GB.
Top observation: 2-bit on token layers, reasoning layers kept at higher precision.
Leaderboard
Full ranking
DimensionBF16Q4_0 QATMobile QAT
Memory footprint145
Quality preservation543
Decode speed245
Deployment breadth453
On-device accessibility145
Total132121
Tie by design: Q4_0 wins laptops and GPUs; mobile wins phones.
Key Takeaways
What developers should know
Q4_0 QAT cuts E2B to 3.2 GB and E4B to 5 GB, from 9.6 GB and 15 GB at BF16.
A new mobile QAT schema brings E2B to about 1 GB; text-only without PLE goes under 1 GB.
QAT changes quality at a given size; the mobile format drives the extra memory cut.
Google claims higher quality than PTQ but published no Gemma 4 QAT numbers.
Weights ship today on Hugging Face with llama.cpp, Ollama, vLLM, and MLX support.
‹ Prev
Next ›
Memory: official Gemma 4 documentation. ~1 GB E2B: QAT announcement (mobile format); text-only without PLE is under 1 GB. Quality: Google’s stated claim — no independent Gemma 4 QAT scores at release.
Check out the Model weights (Q4_0 QAT collection, Mobile QAT collection) and Google blog (QAT release). Also, feel free to follow us on Twitter and don’t forget to join our 150k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us
The post Google DeepMind Releases Gemma 4 QAT Checkpoints: Q4_0 and a New Mobile Format Cut On-Device Memory appeared first on MarkTechPost.
Google’s AI team states its QAT results yield higher overall quality than standard PTQ baselines. Google did not publish Gemma 4 QAT benchmark scores in the announcement. For context, Gemma 3 QAT cut the Q4_0 perplexity drop by 54% using llama.cpp evaluation. We cite that only as prior-generation precedent.
The comparison task
Compare Gemma 4 E2B and E4B across three formats. The formats are BF16, Q4_0 QAT, and the new mobile QAT schema. Rank them on memory footprint, quality preservation, and on-device accessibility. Use published figures only.
Memory results
FormatE2BE4BBasis
BF16 (16-bit)9.6 GB15 GBOfficial Gemma 4 docs
Q4_0 (4-bit, QAT)3.2 GB5 GBOfficial Gemma 4 docs
Mobile (QAT, E2B)~1 GB—QAT announcement
The Q4_0 figures match the footprint of PTQ Q4_0. QAT does not change the size at a given format. It improves quality at that size. The new mobile schema delivers the additional reduction.
Using that mobile schema, Google reduced Gemma 4 E2B to about 1GB. Developers can go lower still. The text-only model without Per-Layer Embeddings needs under 1GB, dropping the audio and vision encoders.
Per-format breakdown
BF16 is the quality baseline. E2B needs 9.6 GB and E4B needs 15 GB. It is the reference point, not a phone deployment target.
Q4_0 QAT is the general-purpose local format. E2B drops to 3.2 GB and E4B to 5 GB. QAT preserves more quality here than PTQ at the same size. This format fits consumer GPUs. Earlier E2B testing also ran on a Raspberry Pi 5 at INT4.
The mobile format is the edge-specialized schema. It brings E2B to about 1 GB. It uses static activations, channel-wise quantization, and targeted 2-bit compression.
How the mobile schema works
Google AI team engineered four techniques for mobile hardware. Static activations pre-calculate scaling during training, reducing on-device work. Channel-wise quantization fits the design of mobile accelerators. Targeted 2-bit quantization compresses only the token-generation layers. Embedding and KV cache optimization shrinks the active memory footprint.
Core reasoning layers stay at higher precision. That protects capability while cutting storage. Developers can also deploy text-only and drop the audio and vision encoders. That trims memory further for use cases that need no multimodality.
Dimension breakdown
Scores are a qualitative ranking of the formats for on-device use. Memory is the only hard-measured axis. Quality reflects Google’s disclosed design, not measured Gemma 4 numbers. Each score has a one-line basis.
On-device accessibility1 — needs large GPU4 — consumer GPU, Raspberry Pi 55 — runs on phones
Total (/25)132121
Winner
The result is a tie by design. Q4_0 QAT and mobile QAT both score 21, but for different hardware. For phones, the mobile format leads. It reaches about 1GB on E2B and targets mobile accelerators directly. For laptops and consumer GPUs, Q4_0 QAT is the practical default. BF16 stays the quality reference, not a local choice.
Methodology and limits
Memory figures come from Google’s Gemma 4 documentation. The ~1GB E2B figure comes from the QAT announcement. Quality is Google’s stated claim. No independent Gemma 4 QAT quality numbers were published at release. We did not run the models locally for this comparison. Developers should test at their own quantization and workload before building.
Key Takeaways
Q4_0 QAT cuts Gemma 4 E2B to 3.2 GB and E4B to 5 GB, from 9.6 GB and 15 GB at BF16.
A new mobile QAT schema brings E2B to about 1 GB; text-only without PLE goes under 1 GB.
QAT changes quality at a given size, not the size itself; the mobile format drives the extra memory cut.
Google claims higher quality than PTQ but published no Gemma 4 QAT benchmark numbers at release.
Weights ship today on Hugging Face with llama.cpp, Ollama, LM Studio, vLLM, MLX, and LiteRT-LM support.
Marktechpost’s Visual Explainer
Marktechpost · Benchmark
Gemma 4 QAT: Comparing Q4_0 and the New Mobile Format
Google DeepMind released Quantization-Aware Training checkpoints for Gemma 4. We compared three edge-model formats on published numbers.
Memory from official Gemma 4 docs. Quality from Google’s stated claim. No models run locally.
Format 1 of 3 · Reference
BF16 (16-bit)
13 / 25
The full-precision quality baseline. E2B needs 9.6 GB and E4B needs 15 GB.
Top observation: a reference point, not a phone or laptop deployment target.
Format 2 of 3 · Laptop / GPU
Q4_0 QAT (4-bit)
21 / 25
The general-purpose local format. E2B drops to 3.2 GB and E4B to 5 GB.
Top observation: QAT preserves more quality than PTQ at the same 4-bit size.
Format 3 of 3 · Mobile
Mobile QAT
21 / 25
The edge-specialized schema. Brings E2B to about 1 GB.
Top observation: 2-bit on token layers, reasoning layers kept at higher precision.
Leaderboard
Full ranking
DimensionBF16Q4_0 QATMobile QAT
Memory footprint145
Quality preservation543
Decode speed245
Deployment breadth453
On-device accessibility145
Total132121
Tie by design: Q4_0 wins laptops and GPUs; mobile wins phones.
Key Takeaways
What developers should know
Q4_0 QAT cuts E2B to 3.2 GB and E4B to 5 GB, from 9.6 GB and 15 GB at BF16.
A new mobile QAT schema brings E2B to about 1 GB; text-only without PLE goes under 1 GB.
QAT changes quality at a given size; the mobile format drives the extra memory cut.
Google claims higher quality than PTQ but published no Gemma 4 QAT numbers.
Weights ship today on Hugging Face with llama.cpp, Ollama, vLLM, and MLX support.
‹ Prev
Next ›
Memory: official Gemma 4 documentation. ~1 GB E2B: QAT announcement (mobile format); text-only without PLE is under 1 GB. Quality: Google’s stated claim — no independent Gemma 4 QAT scores at release.
Check out the Model weights (Q4_0 QAT collection, Mobile QAT collection) and Google blog (QAT release). Also, feel free to follow us on Twitter and don’t forget to join our 150k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us
The post Google DeepMind Releases Gemma 4 QAT Checkpoints: Q4_0 and a New Mobile Format Cut On-Device Memory appeared first on MarkTechPost.