docs: tuning guidance for draft-mtp speculative decoding

[MerkyorLynn]

Summary

Docs-only change. This adds a short tuning note for --spec-type draft-mtp in docs/speculative.md.

The point is to document that the default --spec-draft-n-max 3 --draft-p-min 0 is not always a good setting for MTP self-speculation, because deeper low-confidence drafts can be generated and then rejected by the target model.

Local benchmark

I measured this locally on my GB10/Spark system with llama.cpp commit 88a3927.

Setup:

• model: StepFun Step-3.x-Flash (step35, Q3_K_M)
• hardware: single NVIDIA GB10
• decoding: greedy
• args: -np 1 -c 512 -ub 64

Config	Decode tok/s	Draft acceptance
no speculation	26.7	—
--spec-draft-n-max 3 --draft-p-min 0	23.5	34%
--spec-draft-n-max 2 --draft-p-min 0	29.6	54%
--spec-draft-n-max 2 --draft-p-min 0.6	30.5	94%
--spec-draft-n-max 2 --draft-p-min 0.85	29.5	97%

On this setup, --spec-draft-n-max 2 --draft-p-min 0.6 changed the result from slower-than-baseline to a speedup with high draft acceptance.

Note

My understanding is that with MTP self-speculation, a single MTP head is chained autoregressively. In this case, deeper drafts can become low-confidence and often fail target verification. The doc change points users at the two knobs that mattered in my test: draft depth and the confidence gate.

AI disclosure: I used Claude Code/Codex only as an assistant for small helper-script checks and note organization. I ran the benchmark myself on my GB10/Spark machine, and I reviewed the numbers, mechanism, and doc change before submitting.

cc @stepfun-ai (step35 MTP).

[ggml-gh-bot]

Hi @MerkyorLynn, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

• Multiple open PRs from a new contributor: We limit new contributors (those without a previously merged PR) to 1 open PR at a time. You currently have 2 open PRs.

• AI-generated content: This project does not accept PRs, descriptions or commit messages that are fully or predominantly AI-generated. If you have used AI to assist you in writing code, please make sure to disclose that explicitly.

---

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

[MerkyorLynn]

Hi @MerkyorLynn, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

• Multiple open PRs from a new contributor: We limit new contributors (those without a previously merged PR) to 1 open PR at a time. You currently have 2 open PRs.
• AI-generated content: This project does not accept PRs, descriptions or commit messages that are fully or predominantly AI-generated. If you have used AI to assist you in writing code, please make sure to disclose that explicitly.

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

Thanks. I closed #24273, so this is now my only open PR.

I also rewrote the PR description and commit message to make the benchmark setup, local results, and AI assistance disclosure explicit.

[ggerganov]

This is not a good advice - the StepFun MTP currently works only with draft length of 1 (#23274), so that's the main reason for your observations.