ggml-cuda : optimize mmvq nwarps for Pascal DP4A

[animeshsri14]

Overview

Adds a dedicated MMVQ parameter table for Pascal DP4A GPUs (compute capability 6.1 / 6.2 -GTX 10xx, P40, P4) and tunes the warp count for single-token decode.

Previously these cards fell through to MMVQ_PARAMETERS_GENERIC. This PR introduces MMVQ_PARAMETERS_PASCAL_DP4A, selected for NVIDIA archs in [DP4A, VOLTA) on both the device and host table-id paths, and for ncols_dst == 1 (single-token decode) uses 2 warps instead of 4. Pascal decode is memory-bandwidth-bound on these small SMs, so the extra warps add scheduling/occupancy pressure without helping throughput. Batched decode and prefill keep the generic behaviour and are unchanged.

Measured +2.5% to +5.4% single-token decode across quants (geomean +3.07%) on a GTX 1060 6GB, with no measurable prefill regression.

Additional information

Benchmarked rigorously, both coverage and depth with a self-contained script that builds HEAD~1 (baseline) and HEAD (this PR) into separate binaries and runs llama-bench alternating between them across 6 iterations. Consumer Pascal can't lock clocks (nvidia-smi -lgc), so I report the paired per-iteration delta, which cancels thermal/boost drift; t is the paired t-statistic of the per-iteration Δ%. GTX 1060 6GB (CC 6.1), Llama-3.2-3B-Instruct.

Decode (tg) - the mmvq path this PR changes (-p 0 -n 128 -ngl 99 -r 10)

quant	baseline t/s	patched t/s	mean Δ	paired σ%	t	pairs up
Q4_0	49.41	51.68	+4.59%	0.45%	25.00	6/6
Q4_K_M	43.49	45.85	+5.44%	0.38%	34.74	6/6
Q5_K_M	39.92	41.22	+3.24%	0.33%	23.80	6/6
Q8_0	32.62	33.43	+2.48%	0.53%	11.56	6/6
Q6_K	34.10	33.99	-0.32%	0.56%	-1.41	2/6

Geomean +3.07%. Q6_K is within noise. An earlier independent 10-iteration run reproduced these to within ~0.2% (geomean +3.08%).

Prefill (pp) - MMQ path, no-regression control (expected ~0%) - two prompt lengths and two -ub batch sizes; all rows flat:

quant	config	baseline t/s	patched t/s	mean Δ	paired σ%	t	pairs up
Q4_0	pp512	950.77	950.19	-0.06%	0.52%	-0.27	3/6
Q4_K_M	pp512	825.24	838.80	+1.64%	4.25%	1.03	3/6
Q5_K_M	pp512	812.26	811.70	-0.07%	0.19%	-0.88	2/6
Q8_0	pp512	873.82	874.03	+0.02%	0.20%	0.30	3/6
Q6_K	pp512	770.35	770.46	+0.01%	0.23%	0.15	4/6
Q4_0	pp512_ub128	946.72	946.95	+0.02%	0.18%	0.34	3/6
Q4_K_M	pp512_ub128	801.61	799.35	-0.28%	0.90%	-0.94	2/6
Q5_K_M	pp512_ub128	807.51	806.92	-0.07%	0.16%	-1.09	2/6
Q8_0	pp512_ub128	865.61	867.55	+0.22%	0.17%	3.26	5/6
Q6_K	pp512_ub128	765.39	765.17	-0.03%	0.08%	-0.83	3/6
Q4_0	pp2048	835.78	833.94	-0.22%	0.95%	-0.56	2/6
Q4_K_M	pp2048	748.45	748.56	+0.02%	0.61%	0.07	3/6
Q5_K_M	pp2048	725.78	725.93	+0.02%	0.54%	0.10	3/6
Q8_0	pp2048	773.93	773.75	-0.02%	0.71%	-0.07	3/6
Q6_K	pp2048	692.05	692.68	+0.09%	0.56%	0.41	4/6

(The Q4_K_M pp512 +1.64% is noise - σ=4.25%, t=1.03, and the same quant's other prefill configs are flat.)

Benchmark script (builds both commits itself; point MODELS/MODEL_DIR at your GGUFs and run ./bench-pascal-mmvq.sh): https://pastebin.com/he3WzqjG

Requirements

• I have read and agree with the contributing guidelines.
• AI usage disclosure: Yes, codex only helped write the benchmarking script, the entire diff itself is handwritten. The source change is my own, from conception to verification and benchmarking methodology.

[ggml-gh-bot]

Hi @animeshsri14, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

• PR Template not respected: Please respect the template when creating a new pull request. Make sure to fill out all required sections.

---

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

[JohannesGaessler]

Why is this PR still a draft? Do you intend to add more changes?

[animeshsri14]

Finalized, just wanted to strengthen the benchmark section for easier review.