MTP: implement --spec-draft-p-min (probabilistic accept threshold)

[pekkah]

Background

llama.cpp's --spec-draft-p-min (default 0.75) is the minimum acceptance probability for a drafted token. Today sharpi's MTP path uses pure greedy verification: a draft is accepted iff `argmax(target_logits) == draft` (probability threshold = 1.0 on argmax). This means we reject more drafts than necessary on close calls where the draft is a plausible alternative.

llama.cpp accepts a draft whenever `softmax(target_logits)[draft] >= p_min` — i.e. as long as the draft is a reasonably probable continuation. This raises the acceptance rate at the cost of slight output divergence vs the no-MTP baseline.

Scope

1. Add `SamplingParams.SpecDraftPMin` (default 1.0 = current behavior, exact argmax match).
2. Add `--spec-draft-p-min ` to `src/SharpInference.Cli/RunCommand.cs`.
3. In `MtpDecoder.Decode`, swap the current `argmax` comparison for a softmax + threshold check:
   • Compute `p_draft = softmax(target_logits)[draft]`.
   • Accept iff `p_draft >= sp.SpecDraftPMin`; reject otherwise and re-emit `argmax(target_logits)` as the correction.
4. Document the parity trade-off: `p_min < 1.0` means sharpi will not match the greedy MTP-disabled baseline token-for-token. This is the same divergence llama.cpp accepts in exchange for higher decode throughput.

Notes

• Requires a partial softmax (only the verifier value at `draft` is needed; can short-circuit by computing `exp(logits[draft]) / sum(exp(logits))` with the standard log-sum-exp trick — or just normalize the top-K). Per-step cost is tiny (`O(V)` reductions; vocab fits in cache).
• Sensible default in llama.cpp is 0.75. Sharpi should keep the default at 1.0 so existing test parity (MtpDecoder_GreedyParity, etc.) holds; users opt into the looser threshold explicitly.
• File: `src/SharpInference.Engine/Sampler.cs` (SamplingParams), `src/SharpInference.Cli/RunCommand.cs` (CLI flag), `src/SharpInference.Engine/MtpDecoder.cs` (acceptance check).

Acceptance

• `p_min == 1.0` reproduces today's MtpDecoder output bit-for-bit.
• `p_min == 0.75` raises acceptance rate on Qwen3.6-27B-MTP (verify via `SHARPI_TRACE_MTP=1`) without breaking decode-coherence asserts in tests.

Related

• Sibling: MTP: implement --spec-draft-n-min (min draft tokens per spec step) #37 (`--spec-draft-n-min`)
• Possibly relevant after: Batched main verify + per-token GDN snapshot ring — realize the MTP >=1.3x speedup #30 (`n > 1`) — verification logic generalizes naturally to batched accept/reject under `p_min`.

[pekkah]

Closed by 4bcbda9 (commit local-only at the moment; will auto-link on push).

Implementation summary:

• SamplingParams.SpecDraftPMin default changed from 0f → 1f (strict argmax-match, byte-identical to SHARPI_DISABLE_MTP=1). 0f and negatives are treated as the default for back-compat.
• MtpDecoder.AcceptDraft short-circuits on argmax-match (no softmax cost on the fast path) and otherwise materialises only softmax(main)[draft] via numerically-stable max-shift. Plumbed through both DecodeSequential and DecodeBatched. Trace output (SHARPI_TRACE_MTP=1) shows the computed probability on each iter.
• CLI gate replaced with [0, 1] validation; flag description updated.

Acceptance check:

• p_min == 1.0 (default) reproduces today's MtpDecoder output bit-for-bit. Verified by MtpDecoder_GreedyParity_LlamaCpp still passing.
• p_min < 1.0 raises acceptance rate. Verified on a "wizard story" prompt with --spec-draft-p-min 0.03: drafts with softmax prob 0.041 / 0.129 / 0.247 / 0.306 / 0.114 that rejected at p_min=0.5 flip to ACCEPT, while drafts with prob 0.000 stay rejected. Throughput stays on the Batched main verify + per-token GDN snapshot ring — realize the MTP >=1.3x speedup #30 envelope (8.6 t/s vs 8.8 t/s at default on CUDA Q4_K_M).

Trace example (SHARPI_TRACE_MTP=1 --spec-draft-p-min 0.03):

[mtp-batch] P=33 t1=310 t2_draft=381(... p=0.041) t2_target=6113(...) ACCEPT
[mtp-batch] P=37 t1=271 t2_draft=11862(... p=0.129) t2_target=6374(...) ACCEPT
[mtp-batch] P=39 t1=271 t2_draft=36(... p=0.306) t2_target=6374(...) ACCEPT
[mtp-batch] P=51 t1=3108 t2_draft=279(... p=0.000) t2_target=42046(...) reject

Note: --spec-draft-p-min < 1.0 does NOT preserve greedy parity with SHARPI_DISABLE_MTP=1 — drafts accepted via the prob rule (rather than argmax-match) cause emitted tokens to diverge. Same trade-off llama.cpp makes; opt-in only.