SnapKV + TurboQuant composition (#51 follow-up)

[pekkah]

Follow-up to #51 / PR #57.

TurboQuantKvCache stores K/V in 3-/4-bit Lloyd-Max codebooks with per-block scale staging and a fixed-size FP32 attention window at the tail. Compositing SnapKV onto it gets ~16× total KV reduction (4× TQ × 4× position pruning) — the issue's headline use case for 12 GB cards at long context.

Scope

1. Score the TQ cache. TQ K is stored compressed; to score it against the last-W queries we either decompress on the fly per (head, position) or hoist the existing per-block staging path. The TQ attention kernel (CPU + CUDA + Vulkan) already does on-the-fly decompress at attention time — extend that same path to emit per-position softmax weights as a side output instead of (or in addition to) the attention output.

2. Compact the TQ cache. TQ blocks are fixed-size groups (32 or 64 tokens per block, see TurboQuantKvCache.cs). Compaction needs to either:

   • Re-quantize after picking survivors (cleanest, preserves block invariants).
   • Carry a per-block bitmap of kept positions and skip-evicted-positions in attention (simpler but no actual VRAM win for unevicted blocks).

   Prefer re-quantization, since the issue's value is VRAM reduction.

3. FastScan compatibility. The CPU --tq path uses FastScan tiles (#34); confirm the compaction doesn't invalidate the per-block scale tables.

Acceptance

• SHARPI_SNAPKV_BUDGET=2048 + --tq on a 16 K prompt: total KV footprint drops ~16× vs fp16 baseline (Issue SnapKV-style prefill KV eviction (stacks on TurboQuant) #51 motivation).
• Coherent decode + no decode-throughput regression vs --tq alone.
• CPU + CUDA + Vulkan all functional.

Notes

• The TQ layout doc lives in docs/SharpInference-Design.md ("TurboQuant" section) — read before touching block boundaries.

[pekkah]

Broken into a phased roadmap so this can land incrementally rather than as one huge multi-backend PR:

• Phase 1: SnapKV+TQ (#60) Phase 1: CPU TurboQuantKvCache.Compact + scoring #68 — CPU-only. `TurboQuantKvCache.Compact` with re-quantization + TQ-aware `SnapKvSelector.AccumulateLayer` + lift the `!_tqEnabled` gate. Smallest of the three; the CPU codebook + decompress path are already exposed via `TurboQuantOps`, so this is mostly engine plumbing.
• Phase 2: SnapKV+TQ (#60) Phase 2: CUDA TQ scoring + compaction #69 — CUDA. New `SnapKvScoreTq` / `KvCompactTq` NVRTC kernels. Mirrors the existing `TqAttention` per-block dequant-dot pattern.
• Phase 3: SnapKV+TQ (#60) Phase 3: Vulkan TQ scoring + compaction #70 — Vulkan. New GLSL shaders mirroring Phase 2.

Phase 1 is the design-anchor — once the CPU re-quantization is concrete and the eval harness (`benchmarks/SnapKvEval`, #61) confirms ≤ 1-sample regression, Phases 2 and 3 port the same algorithm to their respective GPU dispatch paths. Each phase closes its own sub-issue; this issue stays open until all three land + 16K-prompt acceptance is demonstrated end-to-end.