Skip to content

Test coverage: GGUF-free regression tests for k_eq_v detection + TierPlanner packed sizing (PR #174 follow-up) #175

Description

@pekkah

Durable (GGUF-free) test coverage gaps from PR #174 review

The PR #174 review (pr-test-analyzer + silent-failure-hunter) flagged two production heuristics that are only exercised by bench-machine-only model tests and have no CI-runnable regression net. The behavior is correct today and the downstream effects are pinned (Gemma4ModelHyperparamsTests injects the derived flags; the bench-machine 12B forward/parity oracles exercise the real path), but the detection/sizing logic itself would survive a regression in CI.

1. attention_k_eq_v tensor-scan detection (GgufModel.cs)

The flag is inferred from a "fewer .attn_v.weight than .attn_k.weight" tensor count (now gated on the gemma4 arch family). Nothing GGUF-free exercises the scan — every test either injects the flag or reads a real GGUF.

  • Refactor: extract the scan into an internal static helper taking (IEnumerable<tensorName>, string arch).
  • Tests: positive (K on all layers, V on SWA-only, arch=gemma4 → set), negative-by-arch (same deficit, arch=llama → not set), negative-by-balance (equal K/V → not set).

2. TierPlanner packed-dtype sizing (TierPlanner.cs)

EstimateGpuTensorBytes/MeasureGpuEmbeddingBytes now treat Q4_0/Q8_0/Q6_K (and Q4_K) as "stays packed in VRAM"; the comment explicitly couples this to CudaForwardPass.UploadWeight. If the two drift, the planner mis-counts VRAM and silently mis-places layers (a load-bearing path per project memory). No TierPlanner tests exist.

  • Refactor: make EstimateGpuTensorBytes internal (or add a thin testable wrapper) and ensure GgufTensorInfo is constructible in tests.
  • Tests: per-dtype assertion that packed dtypes size to byteSize (rounded) and others to elementCount * 4.

Also noted (lower priority)

  • The CUDA embed-dispatch default: → EmbedLookupQ4K is safe today (the upload gate admits only {Q4_K, Q8_0, Q6_K}, all handled) but is coupled in two places; a default: throw would make a future packed-quant addition fail loudly instead of producing NaN embeddings (the exact bug class Add Gemma 4 12B (dense gemma4_unified) support — CUDA-Hybrid/CUDA first, Q4_K_M #124 fixed for Q6_K). The Q5_K embed switch case is currently dead (not in the upload gate).
  • The cp.async kernels' < sm_80 scalar fallback path is compile-only-validated (dev box is sm_89); a Turing regression there wouldn't be caught by CI.

Follow-up to #124 / #173 (PR #174).

Metadata

Metadata

Assignees

No one assigned

    Labels

    maintenanceRecurring upkeep / sync with upstream

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions