feat(loader): support Gemma 4 E4B QAT q4_0 GGUF (missing attn_k_norm tensor layout)

[pekkah]

Problem

Google's official Gemma 4 E4B QAT q4_0 GGUF fails to load. The Q8_0 E4B model loads and runs fine on the same code path, so this is a GGUF tensor-layout gap specific to the QAT q4_0 repack — not a quant-decode issue.

$ sharpi-cli -m E:/models/gemma-4-E4B_q4_0-it.gguf -g -1 --kv-type q8_0 -p "..." --temp 0
Error: Missing bias tensor: blk.24.attn_k_norm.weight

• Repo: google/gemma-4-E4B-it-qat-q4_0-gguf
• File: gemma-4-E4B_q4_0-it.gguf (5.15 GB; note the upstream filename uses _q4_0-it, not the 12B's -it-qat-q4_0 pattern)
• The repo also ships an mmproj (vision) file we don't load.

Why it's worth doing

E4B decode is HBM-bandwidth bound (~70 t/s on a 4070 Ti at Q8_0, i.e. at the 8 GB / 504 GB/s ceiling). The q4_0 weights are 5.15 GB vs 8.19 GB → ~1.6× fewer bytes/token → ~1.6× faster decode at near-identical quality (QAT). It also frees ~3 GB VRAM (room for fp32/bf16 KV at moderate context).

The 12B QAT q4_0 (#124) already loads and runs the dense gemma4 Q4_0 path, so the decode kernels exist — this is purely about the E4B loader accepting this file's tensor set.

Likely cause

E4B differs from the 12B by having per-layer input embeddings (PLE) and per-layer QK-norm (attn_k_norm / attn_q_norm, sized numKvHeads*headDim / numHeads*headDim). The gemma4 loader currently treats blk.N.attn_k_norm.weight as required and throws when absent. This QAT q4_0 repack appears to either:

• omit the QK-norm tensors at some/all layers, or
• name/store them differently (e.g. a different suffix, or folded), or
• store them at a dtype/shape the loader's "bias tensor" lookup rejects.

The error message ("Missing bias tensor") also suggests the lookup path is the generic required-tensor check, not a QK-norm-aware one.

Scope

• Dump and diff the tensor list of gemma-4-E4B_q4_0-it.gguf vs gemma-4-E4B-it-Q8_0.gguf (names, dtypes, shapes) — establish exactly what's missing/renamed at blk.24 and whether it's all layers or a subset.
• Confirm against the HF transformers / llama.cpp reference how this checkpoint stores E4B QK-norm (is it genuinely absent, or just named differently?).
• Update the gemma4 GGUF loader to handle this layout (accept the alternate name, or treat QK-norm as optional per-layer if the reference confirms it's legitimately absent — but verify numerics, since dropping QK-norm silently would corrupt output).
• Validate: loads, decodes coherently, CPU↔CUDA logit parity (cross-backend, per [[feedback_cross_backend_parity_test]]), and a decode-t/s bench vs the Q8_0 baseline.
• Once loading: wire it as a recommended fast E4B in the server (appsettings.Local.json) and README.

References

• Download-script entry already added: scripts/download-model.ps1 -Model gemma4-e4b-qat
• Local file: E:/models/gemma-4-E4B_q4_0-it.gguf
• Related: #124 (12B QAT q4_0 — loads fine, dense gemma4 Q4_0 path)