You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Durable (GGUF-free) test coverage gaps from PR #174 review
The PR #174 review (pr-test-analyzer + silent-failure-hunter) flagged two production heuristics that are only exercised by bench-machine-only model tests and have no CI-runnable regression net. The behavior is correct today and the downstream effects are pinned (Gemma4ModelHyperparamsTests injects the derived flags; the bench-machine 12B forward/parity oracles exercise the real path), but the detection/sizing logic itself would survive a regression in CI.
The flag is inferred from a "fewer .attn_v.weight than .attn_k.weight" tensor count (now gated on the gemma4 arch family). Nothing GGUF-free exercises the scan — every test either injects the flag or reads a real GGUF.
Refactor: extract the scan into an internal static helper taking (IEnumerable<tensorName>, string arch).
Tests: positive (K on all layers, V on SWA-only, arch=gemma4 → set), negative-by-arch (same deficit, arch=llama → not set), negative-by-balance (equal K/V → not set).
EstimateGpuTensorBytes/MeasureGpuEmbeddingBytes now treat Q4_0/Q8_0/Q6_K (and Q4_K) as "stays packed in VRAM"; the comment explicitly couples this to CudaForwardPass.UploadWeight. If the two drift, the planner mis-counts VRAM and silently mis-places layers (a load-bearing path per project memory). No TierPlanner tests exist.
Refactor: make EstimateGpuTensorBytesinternal (or add a thin testable wrapper) and ensure GgufTensorInfo is constructible in tests.
Tests: per-dtype assertion that packed dtypes size to byteSize (rounded) and others to elementCount * 4.
Also noted (lower priority)
The CUDA embed-dispatch default: → EmbedLookupQ4K is safe today (the upload gate admits only {Q4_K, Q8_0, Q6_K}, all handled) but is coupled in two places; a default: throw would make a future packed-quant addition fail loudly instead of producing NaN embeddings (the exact bug class Add Gemma 4 12B (dense gemma4_unified) support — CUDA-Hybrid/CUDA first, Q4_K_M #124 fixed for Q6_K). The Q5_K embed switch case is currently dead (not in the upload gate).
The cp.async kernels' < sm_80 scalar fallback path is compile-only-validated (dev box is sm_89); a Turing regression there wouldn't be caught by CI.
Durable (GGUF-free) test coverage gaps from PR #174 review
The PR #174 review (pr-test-analyzer + silent-failure-hunter) flagged two production heuristics that are only exercised by bench-machine-only model tests and have no CI-runnable regression net. The behavior is correct today and the downstream effects are pinned (
Gemma4ModelHyperparamsTestsinjects the derived flags; the bench-machine 12B forward/parity oracles exercise the real path), but the detection/sizing logic itself would survive a regression in CI.1.
attention_k_eq_vtensor-scan detection (GgufModel.cs)The flag is inferred from a "fewer
.attn_v.weightthan.attn_k.weight" tensor count (now gated on thegemma4arch family). Nothing GGUF-free exercises the scan — every test either injects the flag or reads a real GGUF.internal statichelper taking(IEnumerable<tensorName>, string arch).2.
TierPlannerpacked-dtype sizing (TierPlanner.cs)EstimateGpuTensorBytes/MeasureGpuEmbeddingBytesnow treat Q4_0/Q8_0/Q6_K (and Q4_K) as "stays packed in VRAM"; the comment explicitly couples this toCudaForwardPass.UploadWeight. If the two drift, the planner mis-counts VRAM and silently mis-places layers (a load-bearing path per project memory). NoTierPlannertests exist.EstimateGpuTensorBytesinternal(or add a thin testable wrapper) and ensureGgufTensorInfois constructible in tests.byteSize(rounded) and others toelementCount * 4.Also noted (lower priority)
default: → EmbedLookupQ4Kis safe today (the upload gate admits only {Q4_K, Q8_0, Q6_K}, all handled) but is coupled in two places; adefault: throwwould make a future packed-quant addition fail loudly instead of producing NaN embeddings (the exact bug class Add Gemma 4 12B (dense gemma4_unified) support — CUDA-Hybrid/CUDA first, Q4_K_M #124 fixed for Q6_K). TheQ5_Kembed switch case is currently dead (not in the upload gate).< sm_80scalar fallback path is compile-only-validated (dev box is sm_89); a Turing regression there wouldn't be caught by CI.Follow-up to #124 / #173 (PR #174).