[None][perf] Add BSX multi-tier CuTe DSL top-k decode kernels (stacked on #16457) by longcheng-nv · Pull Request #16877 · NVIDIA/TensorRT-LLM

longcheng-nv · 2026-07-26T07:25:49Z

What

Adds BSX, a multi-tier CuTe DSL top-K family, as a guarded fp32 fast path inside the existing trtllm::cute_dsl_gvr_topk_decode op — signature unchanged, no call-site change; everything outside the guard falls back to the #16457 kernel unchanged (bf16/fp16 always do). Rebased onto main. Net result on 9,515 real decode-capture cases: gm 1.3996× vs the in-tree kernel, worst case 0.9516× — a strict Pareto improvement with no case regressing more than 10%.

Design

flowchart TD
    A["cute_dsl_gvr_topk_decode(...)"] --> G{"guard<br/>fp32 · K ∈ {512,1024,2048} · cr ∈ {1,4} · next_n ≥ 1<br/>no order_row/counters · npad ≤ 256K, 64-aligned<br/>cluster size ≤ device max"}
    G -- fail --> IT["in-tree #16457 kernel<br/>(dtype-generic universal fallback)"]
    G -- pass --> BAND{"measured fallback band table<br/>(nearest-pow2 npad, BS):<br/>8/16K: BS≥256 · 32/64K: BS≥16<br/>128K: 8–255 (bs&lt;16 only up to npad 147456) · 256K: 16–127"}
    BAND -- in band --> IT
    BAND -- out --> RT{"route(BS, npad, K)"}
    RT -- "BS ≥ 256/128/16<br/>(npad ≤20K / <32K / ≥32K)" --> TP["<b>tp</b>: streaming throughput tier<br/>sampled pivot + CI lean-pivot admission<br/>(trivial collect-all when npad ≤ kC)"]
    RT -- "npad ≤ 12288" --> DIR["<b>direct</b>: single-CTA<br/>exact 3-level radix"]
    RT -- otherwise --> REG["<b>reg</b>: register-resident tier<br/>one row load, zero re-streams<br/>(dense tb=1024 / latency tb=512 variants,<br/>cs/maxv/ar by npad, K)"]

Core algorithm — Guess–Verify–Refine (GVR). Exact top-K is the tie-safe generalized inverse of the row's empirical tail-count function: radix localizes the K-th boundary bit-by-bit (distribution-agnostic); GVR instead treats the previous decode step's top-K as a boundary-oriented order-statistic prior — adjacent steps select heavily-overlapping sets, so the hint sample brackets the current K-th value. Guess proposes value-space thresholds from the hint ladder; Verify checks them with exact candidate counts on the current row, establishing a candidate-superset contract; Refine runs an exact tie-aware radix select over the bounded candidate set. The prior is non-stationary (model/layer/prompt/step) and may fail — it only ever sizes the work; the answer is decided exclusively by exact counts on the current row, with a precise fallback chain (secant refinement, plateau descent) when no proposed threshold verifies.

Three implementation decisions carry the rest:

Direct admission: when the candidate-capacity condition holds naturally (row fits the candidate budget — direct at npad ≤ 12288, tp-trivial at npad ≤ kC), estimation is skipped entirely: collect all, refine once.
Tier by memory regime: latency-bound low-BS rows keep the entire row register-resident so Verify/Refine never re-touch DRAM (reg); throughput-bound high-BS rows stream with a sampled multi-rung ladder plus a confidence-interval lean-pivot admission that thins fat candidate sets (tp).
Measured fallback band: (npad, BS) buckets where the in-tree kernel's exact-count ladder + row-slice cluster split wins (>1.10× on ≥1 production layer, full-grid calibration) route back to it. Kill switch: TRTLLM_BSX_FALLBACK_BANDS=0. Routing is pure in (BS, npad, K) — host-only, CUDA-graph safe.

Performance (B200, fp32, real SWE-bench decode captures, paired same-rep cold-L2 nsys)

Full-mesh re-measure of this PR's head — 865 real decode-capture cells x 11 batch sizes = 9,515 paired cases, 8 GPUs, 0 harness failures, and a tie-aware exactness check inside every case (see Correctness).

vs the #16457 kernel now on main, shipped operator (band table on):

overall	floor	cases below 0.909	win rate
gm 1.3996x	0.9516x	0 (no case regresses more than 10%)	67.2%

By model: K=2048/cr=1 1.4056x · K=1024/cr=4 1.4063x · K=512/cr=4 1.3772x

By sequence length (npad = post-compression row width; token length ~ npad x cr):

npad	1-2K	4K	8K	16K	32K	64K	128K	256K
gm	2.04 / 1.73	1.67	1.34	1.22	1.17	1.19	1.46	1.44

By batch size:

model \ BS	1	2	4	8	16	32	64	128	256	512	1024
K=2048, cr=1	1.80	1.78	1.76	1.53	1.25	1.24	1.25	1.24	1.26	1.27	1.28
K=512, cr=4	1.67	1.64	1.63	1.48	1.29	1.29	1.26	1.28	1.23	1.24	1.25
K=1024, cr=4	1.72	1.68	1.67	1.51	1.30	1.30	1.28	1.31	1.26	1.27	1.29

By layer: all 109/109 production layers win — worst 1.273x, medians 1.37-1.41x, best 1.540x. (The loss tail is a per-(layer, shape) phenomenon; the band table absorbs it.)

The same run also isolates the cost of everything added on top of the reviewed tier commits: measured against the previous full grid, the in-tree arm drifts 1.0051x and the bsx arm 1.0060x — i.e. the follow-up work (comment prune, launch-policy de-duplication, 16-bit exact-tail enablement, P4 helper extraction, plateau terminal) costs nothing measurable on either side.

Development A/B additionally falsified (measured, component-isolated): exact-count admission as a wholesale replacement (gain and harm co-sourced), a K-scaled candidate-budget diet (pure harm), ladder-quantile re-placement under the shipped admission, cluster-size occupancy cuts, multi-pass-straggler hypotheses (the residual band is single-pass; the gap is candidate-set fatness), and a conditional lean-pivot for the register tier (short kernels cannot amortise an extra cluster round trip).

Review follow-ups from #16457 (resolved here)

The follow-up items committed to reviewers on #16457 all land in this PR (f47dda5c, 3959d327):

item (reviewer)	resolution
comment pruning (@lfr-0531)	provenance commentary reduced to invariants/contracts across kernel + custom-op files
launch-shape policy duplication (@limin2021)	`pick_config` split into `pick_cluster_size` + `pick_tuning` (kernel = single source); runner `_pick_tuning` is now a thin adapter, cluster auto-pick delegates; the intentional divergence is documented (runner asserts on 32B misalignment, `launch` downgrades); new sweep test pins runner == kernel policy
16-bit exact-tail (@mingyangHao)	`p4_exact_tail` defaults ON for fp16/bf16 — candidate keys are always fp32 (injective upcast at collect), so the tail-radix re-rank is exact for every dtype; overclaiming docstring corrected; new adversarial test (two distinct 16-bit values in one fine bin straddling K, fp16 + bf16)
P4 radix duplication (@mingyangHao)	the two token-identical copies collapse into one `@cute.jit` helper; byte-identical PTX verified for the `p4_tail_fast=False` variant (465,875 B)
dispatch guard	this PR's dispatcher
plateau undershoot terminal (@mingyangHao)	resolved (`3959d327`). Confirmed site: the admission path's tie-plateau fail-soft landed `done=1` on the undershoot side, so Phase 4 padded the tail with `-1`. Both terminals now first collapse the bracket by bounded bisection to ADJACENT floats — every in-bracket value is then bitwise-equal, a genuine tie class — so Phase 4 emits the sure winners and a ticketed fill completes the row from that class (any (K-count)-subset of a tie class is a valid tie-aware completion). The guard requires a coherent undershoot-overflow bracket with both counts current, so the retry path's widened brackets are excluded; non-plateau undershoot keeps the documented `-1` encoding. New adversarial test: a plateau wider than the candidate buffer straddling K, fp32 + fp16 x {rank-scatter cs=1, cs=4, histogram-snap}, 6/6. Implementation note now in the code: the terminal is captured into a dedicated SMEM slot before Phase 4, because Phase 4 reuses that scalar slot as radix scratch. Follow-up (`9eadcf20`): Phase 2 has two secant drivers — the SMEM/leader one and a register-resident redundant-warp one used at `cluster_size == 1` — and the terminal initially landed only in the former, so the same plateau still padded with `-1` on the classic admission path. Both terminals are now mirrored into the register-resident driver (warp-uniform, so the counting barrier cadence is unchanged), and the adversarial matrix grew to 5 variants × 2 dtypes (adding `enable_r0=False` at cluster size 1 and 4, the route that exposed the gap), 10/10.

Gate for the above: GVR top-K suite 684 passed / 144 skipped (includes the 5 new tests), BSX suite 91 passed / 8 skipped, plus the full-mesh re-measure in the Performance section.

Test Coverage

New test_cute_dsl_bsx_topk_decode.py (99 cases, registered in l0_b300.yml; CI wall-clock managed by pinning cases onto a minimal covering set of JIT variants: 570 s → 306 s, no code-path loss).
Existing test_cute_dsl_gvr_topk_decode.py full suite green.

Draft checklist (before ready-for-review)

Direct A/B perf table vs [None][perf] GVR top-K decode: enable R0 histogram-ladder admission by default #16457 kernel (full 865×11 grid); ragged-mix perf axis deferred (correctness-covered by varlen/MTP tests)
CUDA-graph capture/replay test
Rebase onto main

PR Checklist

PR title follows the required format
Description explains what and why
Test cases added and passing locally
DCO sign-off on all commits

🤖 Generated with Claude Code

Dev Engineer Review

Adds a guarded fp32 BSX fast path to trtllm::cute_dsl_gvr_topk_decode that dispatches to new CuTe DSL tiers (direct, register-resident “reg”, and throughput “tp”) while preserving the existing operator signature and falling back to the in-tree #16457 kernel for unsupported cases (including bf16/fp16).
Refactors tuning/launch policy selection in cute_dsl_custom_ops.py and the core GvrTopKKernel to centralize decisions via new GvrTopKKernel.pick_cluster_size and GvrTopKKernel.pick_tuning (unifying config fields like cluster_size and tuning knobs).
Implements “plateau terminal” and exact-tail improvements in gvr_topk_decode.py:
- Phase 2 plateau terminal admission (done=3) and terminal recount behavior.
- Phase 4 SMEM layout extension for plateau terminal bookkeeping + plateau-fill completion logic.
- Deterministic exact-tail tie handling via _p4_exact_tail_radix_select, plus gating behavior when p4_exact_tail is unset.
Introduces a new BSX dispatcher (gvr_topk_decode_bsx_dispatch.py) with:
- Tier routing and dispatch cache keyed by shape parameters.
- Host-side is_bsx_supported eligibility checks (dtype/layout/shape/alignment, next_n/compress_ratio constraints, hardware/cluster-size limits).
- A calibrated fallback-band mechanism (TRTLLM_BSX_FALLBACK_BANDS) with an explicit environment kill switch; gvr tier resolution is treated as invalid in dispatcher binding.

QA Engineer Review

Test changes

Production code not touched in this section; tests updated: yes.

New/updated test module: tests/unittest/_torch/attention/sparse/test_cute_dsl_bsx_topk_decode.py
- Added tests including:
  - test_bsx_cuda_graph_capture_replay
  - test_bsx_fallback_band_table
  - test_bsx_route_table, test_bsx_route_env_knobs
  - Tier-specific / dispatch routing checks: test_bsx_reg_launch_table, test_bsx_direct, test_bsx_tp
  - Robustness: test_bsx_degenerate_rows, test_bsx_preidx_hardening
  - TP admission / plateau / overflow: test_bsx_tp_admission_hitrate, test_bsx_tp_admission_tie_plateau, test_bsx_tp_admission_overflow_fallback, test_bsx_tp_admission_mixed_batch
  - Fallback eligibility: test_bsx_dispatcher_fallback_bf16, test_bsx_dispatcher_fallback_bad_shapes
  - MTP exactness/hardening/degeneracy: test_bsx_mtp_exactness, test_bsx_mtp_preidx_hardening, test_bsx_mtp_degenerate_rows
Updated test module: tests/unittest/_torch/attention/sparse/test_cute_dsl_gvr_topk_decode.py
- Added tests including:
  - test_cute_dsl_gvr_topk_decode_p4_exact_tail_16bit
  - test_cute_dsl_gvr_topk_decode_pick_policy_single_source
  - test_cute_dsl_gvr_topk_decode_plateau_terminal
Test-list change (CI selection): tests/integration/test_lists/test-db/l0_b300.yml
- Updated the pre_merge ignore list under unittest/_torch/attention to exclude:
  - unittest/_torch/attention/sparse/test_cute_dsl_gvr_topk_decode.py
  - unittest/_torch/attention/sparse/test_cute_dsl_bsx_topk_decode.py

CI coverage & verdict

The new/extended tests are not shown as positively included in tests/integration/test_lists/test-db/l0_b300.yml; instead, both the GVR and BSX test files are explicitly ignored for this list.
Verdict: needs follow-up (verify which other CI test-db/qa lists run these updated unittests, or adjust test-db entries if BSX coverage is intended for l0_b300).

…GVR decode op Adds the op43 bsx CuTe DSL kernel family as a guarded fast path inside trtllm::cute_dsl_gvr_topk_decode (op signature unchanged): - gvr_topk_decode_bsx_tp.py: throughput GVR tier (gvr_topk_tp port) with ragged-N lane masking, pre_idx clamping, in-kernel degenerate emit and the mandatory cluster exit rendezvous. - gvr_topk_decode_bsx_reg.py: register-resident tier (gvr_topk_reg port); ragged-N handled by extending the OOR-lane -FLT_MAX idiom at the single register load; packed u64 candidate pushes and aligned cluster barriers preserved from the op43 convergence rounds. - gvr_topk_decode_bsx_direct.py: short-row (npad <= 12288) exact radix tier with ragged-N key substitution and degenerate emit. - gvr_topk_decode_bsx_dispatch.py: route table transcribed from op42 gvr_bsx.cu; env knobs renamed GVR_BSX_* -> TRTLLM_BSX_*; v1 guard (fp32, next_n=1, cr=4, no order_row/counters, K in {512,1024,2048}, npad <= 262144, npad % 64 == 0) plus a hardware cluster-size cap check that falls back to the in-tree kernel instead of degrading silently. - CI-sized exactness test covering every reg launch-table instance once, direct/tp tiers, ragged rows with poisoned tails, quantized ties, degenerate rows, pre_idx hardening, route-table asserts and dispatcher fallback; registered in l0_b300.yml. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>

…gh-hit-rate rows) Port the in-tree R0 histogram-ladder ADMISSION concept (op#26, PR NVIDIA#16457) into the bsx throughput (tp) tier's pivot selection, closing the pr1 full-grid losses against the in-tree GVR kernel on real-capture rows at BS >= 16 while keeping the fused one-pass structure and every existing exactness invariant. Mechanism (measured on pr1 real-capture cells): the old 3-stage pivot pick targets ~3K sampled candidates inside a narrow [1.5K, 0.6kC] band. On real rows this either (a) picks a FAT rung (2-4x more P3 pushes and P4 candidates than needed - the v32/pro BS>=16 band), or (b) trusts a clustering-inflated sampled estimate and undershoots K, which fails the fused-pass reuse check and re-streams / secant-loops the full row (the flash_512k 1.6-1.8x losses; spatially clustered rows inflate a float4-sampled estimate up to 2.5x over the true count). Changes (gvr_topk_decode_bsx_tp.py only): * P2a stage-0 ADMISSION pick (R0 parity): accept the TIGHTEST ladder rung whose sampled-count confidence interval sits inside the [K, kC] acceptance window - the same "smallest exact count in [K, kC]" rule as the in-tree R0 admission, applied to the pre-pass estimates. The legacy 3-stage pick is unchanged as the fallback when no rung qualifies (cold-start / degenerate ladders take exactly the old path). * Clustering-aware sigma: sample_count now also tracks per-rung float4 OCCUPANCY, packed into the same per-thread accumulator (occ << 16 | cnt), giving the compound-Poisson sigma cnt/sqrt(occ) (equals the classic Poisson sigma on IID rows). Packing keeps registers, SMEM and the exchange at their pre-change sizes - an unpacked occ implementation measured a 14-25% whole-kernel regression (register spill in the streaming loops; A/B/bisect on v32_64k_L20). No field overflow: bsx guards npad <= 262144 => cluster-total cnt <= 8192. * Lower margin 2-sigma; K2048 uses 1.5-sigma (its [K, 4K] window is too narrow for 2-sigma to fire tight) backed by a RESCUE rung: the fused pass's second count column is now the next-fatter ladder rung instead of hmin (identical cost), so a pivot undershoot is caught with ONE collect re-stream instead of the multi-pass secant loop. Exactness machinery (accept window, P4 + tie tickets, plateau descent, ragged-N masking, degenerate emit, exit rendezvous) is untouched; admission never drops a top-K element because acceptance still requires the EXACT fused count in [K, kC]. Perf (nsys cold-L2 paired, same-rep A/B vs in-tree GVR kernel, real pr1 capture cells, B200; ratio = bsx/in-tree, lower is better): target cells before after flash_512k_L34 BS16 1.57 1.13 flash_512k_L34 BS64 1.72 1.11 flash_512k_L34 BS128 1.83 1.04 flash_512k_L34 BS256 1.54 0.73 flash_512k_L34 BS512 1.64 0.74 flash_512k_L34 BS1024 1.58 0.74 v32_32k_L50 BS16 1.40 1.21 v32_32k_L50 BS64 1.54 1.30 v32_32k_L50 BS256-1024 1.29-1.32 1.10-1.11 pro_128k_L54 BS16 1.27 1.19 pro_128k_L54 BS64 1.35 1.26 pro_128k_L54 BS128-1024 1.21-1.32 1.12-1.25 no-regression cells before after flash_512k_L36 BS1-128 0.55-0.94 0.56-0.92 flash_512k_L36 BS256-1024 0.67-0.68 0.66-0.67 v32_64k_L20 BS1-8 (reg) 0.63-0.65 0.63 v32_64k_L20 BS16-128 1.09-1.19 1.00-1.10 v32_64k_L20 BS256-1024 1.00-1.05 0.93-0.97 v32_32k_L04 BS16-1024 0.88-1.06 0.90-1.10 pro_256k_L30 BS16-1024 0.89-1.07 0.91-1.12 flash_16k_L26 BS1-1024 0.61-0.86 0.62-0.86 (reg/direct tiers untouched; the three +3-5% readings - pro_256k_L30 BS16/64, v32_32k_L04 BS64 - are inside the +-4.5% session noise floor measured on the fully untouched direct tier, e.g. flash_16k_L26 BS1 0.607 -> 0.634 with zero code change) The remaining v32_32k/pro_128k BS16-128 gap (1.19-1.30) is NOT the admission/pass-count mechanism: with admission the reuse check fires (one fused pass, admitted set slimmed 6678->2917 on v32_32k_L50) and a CS in {1,2,4} launch-shape sweep moves <= 7%. On those L2-resident shapes the in-tree kernel's whole-row-per-CTA structure is simply faster than the tp cluster split; closing it needs a tier-structure change, out of scope for this admission port. Exactness: bsx suite 56/56 (incl. new admission cases: hit-rate extremes on all three production shapes, tie plateau AT the admission threshold, count>kC overflow fallback, mixed admit/fallback ragged batch), in-tree gvr suite 671 passed / 144 skipped, 11/11 targeted screen (clustered/ties/zeros/oor/uniform/trivial), all pr1 A/B runs value-set-exact. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>

… target Fix round for the hint-ladder admission fast path (previous commit): the full-grid pr2-vs-pr1 A/B (9515 cases) showed the admission closed the deep tail (flash_512k_L34 1.83 -> 1.05) but regressed 396 cases by >5pp and flipped 47 former wins, netting the overall gm to a wash. Verified root cause (in-kernel decision probe + host-side P2a emulation on the regression rows) - TWO distinct failure modes, both in the PICK: 1. Fat admit: the [K, kC] acceptance window let the tightest CI-passing rung admit a candidate set 2.5x the legacy ~3K pivot-band target (pro_64k_L06: admitted 4134 vs legacy 1671; pro_32k_L28 4047 vs 1554; flash_128k_L10 1970 vs 788), inflating P3 pushes + P4 scan. 2. Undershoot admit: the K2048 1.5-sigma lower margin accepted rungs whose true count lands under K on clustered rows (v32_32k_L23 rung3 est 2720, sigma 374, TRUE count 1959 < 2048) - the rescue re-stream then costs more than the fat-but-valid legacy pick it displaced (22.6 -> 28.2us at BS64). The improved twin v32_32k_L50 has nearly identical rung3 stats (est 2784, sigma 382) with true count 2917: the pivot margin (+1.80 vs +1.92 sigma) is the only discriminant. Changes (pick logic only; fused pass, rescue and all exactness machinery are untouched from the admission commit): * Admission upper bound tightened from kC to the legacy pivot-band hi (0.6*kC), so admission only fires when the rung is genuinely tight. * K2048 lower margin raised 1.5 -> 1.85 sigma: rejects the L23-class undershoots (+1.80 sigma) while keeping the genuine tight admits (L50 +1.92 sigma). K512/K1024 stay at 2.0 sigma. * Stage 0b: the legacy band pick (min |est-tgt| in [1.5K, 0.6kC], the exact pr1 stage-1 rule) now OVERRIDES the admitted rung when it is strictly leaner and safe by its own clustering-aware 1.5-sigma lower CI; with no admission it is taken as-is (pr1 parity). * The pick's margin tests are now sqrt/div-free (squared comparisons): the pick is a thread-0 serial section between two CTA barriers, and the previous sqrt+fdiv chain measured 2-5% whole-kernel on L2-resident accept-path rows. Perf (nsys cold-L2 paired vs in-tree GVR kernel, real capture cells, B200; ratio = bsx/in-tree, lower is better; bar = pr1 ratio + 3% for the regression cells, absolute for the retention cells): regression cells pr1 pr2 fixed bar v32_32k_L23 BS16 1.202 1.521 1.198 <=1.238 PASS v32_32k_L23 BS64 1.334 1.662 1.325-1.352 <=1.374 PASS v32_32k_L23 BS1024 1.131 1.413 1.123 <=1.165 PASS pro_64k_L06 BS256 1.062 1.352 1.061 <=1.094 PASS pro_64k_L06 BS1024 1.099 1.414 1.121 <=1.132 PASS pro_32k_L28 BS1024 1.059 1.295 1.084 <=1.091 PASS flash_128k_L10 BS64 1.100 1.279 1.094 <=1.133 PASS pro_32k_L02 BS64 0.652 0.659 0.652 <=0.672 PASS retention cells pr1 pr2 fixed bar flash_512k_L34 BS128 1.831 1.052 1.020 <=1.08 PASS flash_512k_L34 BS256 1.537 0.730 0.725 <=0.80 PASS v32_32k_L50 BS64 1.537 1.290 1.267 <=1.35 PASS v32_32k_L50 BS16/1024 1.401/1.316 1.199/1.130 1.177/1.112 no-regression spots pr1 pr2 fixed v32_32k_L04 BS64 1.057 1.086 1.032-1.068 (3 reps) flash_16k_L26 BS1 (direct) 0.607 0.622 0.600 v32_64k_L20 BS8 (reg) 0.633 0.630 0.624 flash_512k_L36 BS64/256 0.944/0.674 0.898/0.656 0.883/0.655 pro_128k_L54 BS64/256 1.351/1.209 1.245/1.122 1.234/1.093 Known give-back: v32_64k_L20 BS16-128 (rung margin +1.53 sigma, below the new 1.85 K2048 cut) reverts from the pr2 admission pick to the legacy band pick; BS256-1024 keep the tight rung via stage 0b. Falsified along the way (kept out): stashing the [rescue, pivot) band in the candidate-buffer tail during the fused pass (per-element atomic, per-warp stripes, and warp-aggregated ballot variants) - the fused streaming loop cannot absorb ANY extra per-element code within the +-3% bar (measured +4-10% whole-kernel on accept-path rows for all three variants). Exactness: bsx suite 56/56 (admission cases included; acceptance-rule comment updated), in-tree gvr suite K2048 subset green, all A/B runs value-set-exact (exact=True on every measured cell x BS x arm). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>

…guard narrowed Extend the op43 bsx CuTe DSL tier family (direct/reg/tp) to the full in-tree run_one_row contract: next_n >= 1 (MTP speculative-decode rows) and compress_ratio in {1, 4}, and drop the next_n==1 / cr==4 conditions from the dispatcher guard. Semantics mirror the in-tree kernel line-for-line: - request-level hint sharing: pre_idx row = row // next_n (pre_idx and seq_lens are [num_rows // next_n, ...], validated by the guard); - cr==1 temporal hint shift (row % next_n) + 1, with shifted-OOR hints falling into the tiers' existing clamp hardening; - per-row N_eff = (seq_lens[req] - next_n + row % next_n + 1) // cr (already general in _row_n_eff). next_n / cr are ctor constexpr: the next_n==1 / cr==4 hot path traces identically to the v1 port (const_expr branches; no offset value is even computed on cr>1 builds). All compile / dispatch / runner caches key on (next_n, cr). Validation (umbriel-b200-074, torch nv26.05): - bsx suite extended with the MTP axis (next_n in {2,3,4} x cr in {1,4} x three tiers x {random-garbage, noised-hint, cold-start-zeros, ties}), each case checked against torch.topk host N_eff/offset simulation AND a differential in-tree arm (order_row-forced, per-row value-multiset equality): 140/140 passed. - full in-tree GVR suite (fp32 MTP/cr=1 cells now auto-route to bsx): 671 passed, 144 skipped. - next_n=1 perf anchors (ab_pr smoke, paired same-GPU back-to-back vs stashed baseline): bsxd/gvrpr ratio deltas within +/-3% on flash_16k_L26 BS1, v32_64k_L20 BS8, flash_512k_L34 BS128/256, v32_32k_L23 BS64, pro_256k_L30 BS256; all exact. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>

…-K interpolated tpush) and mask-hoisted streaming loops Ships the op43-pr6 campaign candidate (measured head 1a88ede77d) as a clean patch on the pr4 head: - stage 0c LEAN-PIVOT OVERRIDE (pick-only): on stage-0 CI-admission miss with npad in [16384, 262144], a strictly leaner ladder rung whose clustering-aware 1.5-sigma CI lands in [K, kC] becomes the pivot; undershoot costs one rescue re-stream (pr4 economics preserved). - per-K interpolated lean tpush (log2-count interpolation between pick and next-tighter rung; target 1.5K, 2.0K for K=512 sampling noise; npad <= 98304 gate) with original-pick rescue. - mask-hoisted streaming loops in count/max-below/fused-count-collect/ collect (unmasked main loop + masked vec-tail) - uniform micro-gain, independently exonerated of the pr5 regression by the C5 fingerprint ablation. - candidate budget pinned at the flat pr4 kC (8192 for K>=2048, else 6144); wide pr4 ladder quantiles pinned. Full-grid evidence (865 real decode cells x 11 BS, same-rep cold-L2 paired vs the in-tree kernel): gm 1.4170 (pr4 head 1.4120), win 88.0%, <0.909 cases 517 -> from 636. Exactness: 140-test production suite + adversarial + synthetic gates all PASS; measured module and this file are AST-identical modulo dead scaffolding removal. Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>

…e kernel Routes the (npad, bs) buckets where the op43-pr6 full-grid verdict found at least one production layer >1.10x slower on every bsx tier than the in-tree kernel (L2-resident mid-N regime: the in-tree exact-count ladder admits a leaner candidate set and its row-slice cluster split keeps all CTAs busy through P4). With the table the operator is a strict Pareto improvement over the in-tree kernel: routed buckets run at parity, everything else keeps the bsx win (full-grid gm 1.40 vs the in-tree head, worst case capped at 1.10x). npad keys resolve by nearest power of two; TRTLLM_BSX_FALLBACK_BANDS=0 disables the table. Band data: 865 real decode cells x 11 BS same-rep cold-L2 nsys pairs, 2026-07-28. Tests: kernel-contract cases pin the table off via an autouse fixture (they exist to exercise the bsx tiers); the table itself is covered by test_bsx_fallback_band_table (bucket membership, nearest-pow2 resolution, neighbour non-routing, kill-switch), and CUDA-graph capture/replay by test_bsx_cuda_graph_capture_replay on both sides of the table (host-side dispatch bakes at capture; replay stays exact over in-place rewritten inputs). Suite: 143 passed; sibling test_cute_dsl_gvr_topk_decode suite: 671 passed / 144 skipped. Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>

…nd to bs=8 The original band-table calibration measured the reg tier through a harness whose GvrTpKernel base class carried a faster experimental streaming phase1 (GvrRegKernel inherits phase1 and the streaming-load helpers from GvrTpKernel; the measurement arm rebound only the tp entry point, so reg-routed shapes silently rode the experimental base). On this branch's reg tier the 128K x BS8 shapes measure 0.82-0.91x vs the in-tree kernel on 5 production layers (865-cell x 11-BS full-grid, same-rep cold-L2 nsys pairs), below the 0.909 floor the table guarantees. Routing the (131072, 8..15) bucket to the in-tree kernel restores the floor; full-grid gm 1.397 -> 1.390. All other buckets re-verified on the shipped kernel (direct 1.0031 / reg 0.9998 vs its true pr4 baseline / tp 1.0008). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>

longcheng-nv · 2026-07-29T01:59:43Z

Pushed ef1c5e08 — band-table recalibration (131072 band lower bound 16 -> 8) + updated the PR-body Evidence section accordingly.

Context: while building a unified-dispatch framework on top of this branch, a full-grid re-measure of the exact shipped head revealed that the original calibration harness measured the reg tier with a faster experimental streaming phase1 inherited from GvrTpKernel (the measurement arm rebound only the tp entry point — GvrRegKernel inherits phase1/streaming-load helpers, so reg-routed shapes silently rode the experimental base, ~20% fast at 32K-128K x BS1-8). On the shipped code the 128K x BS8 bucket dips to 0.82-0.91x vs the in-tree kernel (5 production layers), below the 0.909 floor the band table guarantees — hence the one-bucket recalibration. tp/direct tiers and the headline gm reproduce within 0.1-0.3% (1.3966 claimed vs 1.3965 re-measured; 1.3899 after routing the extra bucket). Unit test extended (test_bsx_fallback_band_table).

longcheng-nv · 2026-07-29T01:59:52Z

/bot run

tensorrt-cicd · 2026-07-29T02:05:51Z

PR_Github #62363 [ run ] triggered by Bot. Commit: ef1c5e0 Link to invocation

…mpile variants The suite's cost is DSL JIT compiles (15-34s per constexpr variant), not case count. Measured on B200: 570s -> 306s, zero code-path coverage lost: - MTP exactness: the 3-tier x next_n{2,3,4} x cr{1,4} cross (18 variant compiles of BOTH arms, 385s) becomes an explicit 8-combo list — full {2,3} x {1,4} cross on tp (the most complex MTP arithmetic), complementary (next_n, cr) diagonals on direct/reg. next_n=4 dropped: 2 covers even row-sharing, 3 covers odd division (and is the production MTP depth). All 4 input kinds kept (they reuse compiles). - reg launch table: all 14 route asserts kept (host-only, free); live launches reduced to the 6 instances that together cover every codegen axis value (cs {1,2,8,16} x tb {512,1024} x ar {6,8} x maxv {5,8} + the dense-knob-only path). - MTP preidx hardening / admission hit-rate: cells re-pinned onto already-compiled variants (same route/cs class, npad is a runtime parameter) so they add zero compiles. - bf16 dispatcher fallback: guard-only (the bf16 in-tree execution it falls back to is exhaustively covered by the sibling gvr suite; recompiling that variant here cost ~10s for no added coverage). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>

longcheng-nv · 2026-07-29T02:34:53Z

Pushed a55ed714 — bsx test-suite CI slimming: 570s -> 306s measured on B200 (-46%), zero code-path coverage lost. The suite's cost is DSL JIT compiles (15-34s per constexpr variant), so cases are now pinned onto a minimal covering set of variants: MTP matrix 18 -> 8 variant compiles (full {2,3}x{1,4} cross on tp, diagonals on direct/reg; next_n=4 dropped as structurally redundant with 2), reg launch table keeps all 14 route asserts but live-launches only the 6 codegen-distinct instances, hardening/admission cells re-pinned onto already-compiled variants, bf16 fallback check is guard-only (its in-tree execution is covered by the sibling gvr suite). PR-body Test Coverage section updated.

longcheng-nv · 2026-07-29T02:35:23Z

/bot run

longcheng-nv · 2026-07-29T03:10:20Z

/bot run

tensorrt-cicd · 2026-07-29T03:16:48Z

PR_Github #62390 [ run ] triggered by Bot. Commit: a55ed71 Link to invocation

tensorrt-cicd · 2026-07-29T03:18:24Z

PR_Github #62363 [ run ] completed with state ABORTED. Commit: ef1c5e0

Link to invocation

tensorrt-cicd · 2026-07-29T05:43:58Z

PR_Github #62390 [ run ] completed with state SUCCESS. Commit: a55ed71
/LLM/main/L0_MergeRequest_PR pipeline #50551 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

longcheng-nv · 2026-07-29T06:03:52Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-29T06:10:48Z

PR_Github #62444 [ run ] triggered by Bot. Commit: a55ed71 Link to invocation

Comment-only: replace internal iteration/campaign identifiers with neutral engineering descriptions (measured/ablation/baseline), keep every load-bearing note (convergence constraints, measured costs, falsified alternatives, recalibration recipe) intact. No code change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>

longcheng-nv · 2026-07-29T08:30:45Z

/bot run

tensorrt-cicd · 2026-07-29T08:37:56Z

PR_Github #62475 [ run ] triggered by Bot. Commit: d6d960d Link to invocation

tensorrt-cicd · 2026-07-29T08:39:40Z

PR_Github #62444 [ run ] completed with state ABORTED. Commit: a55ed71

Link to invocation

tensorrt-cicd · 2026-07-29T09:28:25Z

PR_Github #62475 [ run ] completed with state SUCCESS. Commit: d6d960d
/LLM/main/L0_MergeRequest_PR pipeline #50625 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@lfr-0531

…6457 Four of the five follow-up items committed to reviewers on NVIDIA#16457 (the fifth, the dispatch guard itself, is this PR's dispatcher): 1. Comment pruning (requested by @lfr-0531): measurement-history / tuning-provenance commentary reduced to invariants and contracts across the kernel and custom-op files. 2. Launch-shape policy single source (requested by @limin2021): pick_config is split into pick_cluster_size + pick_tuning on the kernel class (the single source of truth); the production runner's _pick_tuning becomes a thin adapter and its cluster auto-pick delegates to pick_cluster_size. The intentional shell divergence is kept and documented: the runner ASSERTS on a 32B-misaligned logits pointer (contract violation) while GvrTopKKernel.launch silently downgrades to 128-bit loads. New test test_..._pick_policy_single_source sweeps dtype x BS x N x graph-capture and pins runner == kernel policy. 3. 16-bit exact-tail (requested by @mingyangHao): p4_exact_tail now defaults ON for fp16/bf16 as well - candidate keys are ALWAYS fp32 (16-bit inputs are upcast injectively at collect), so the tail radix re-rank on the full fp32 order key is exact for every dtype; the overclaiming 'fully resolved' docstring is corrected. New adversarial test: two distinct 16-bit values (1.0 vs 1.25) in one fine bin straddling the K boundary under a wide Phase-2 bracket, fp16 + bf16. 4. P4 exact-tail radix de-duplication (requested by @mingyangHao): the two verbatim copies (tiny-tie fast path's large-class fallback and the plain exact-tail path; token-identical, 1162 tokens) collapse into one @cute.jit helper _p4_exact_tail_radix_select. Verified: the p4_tail_fast=False variant compiles to BYTE-IDENTICAL PTX before and after (465,875 bytes, CUTE_DSL_KEEP=ptx). Remaining item (plateau undershoot terminal routing to an exact tie-aware fallback) follows as its own commit: the audit found the rank-scatter path currently has no cand_count < K branch at all, so the fix is wider than the review comment assumed and deserves isolated review. Gates: full sparse-attention suite 674 passed / 144 skipped (includes the two new tests); PTX identity proof above. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>

longcheng-nv · 2026-07-29T10:07:01Z

Pushed f47dda5c — resolves the follow-up items committed to reviewers on #16457 (@lfr-0531 comment pruning; @limin2021 launch-shape policy single-source with a runner==kernel sweep test; @mingyangHao 16-bit exact-tail enablement with a new fp16/bf16 adversarial tie test, and the P4 exact-tail radix de-duplication with byte-identical-PTX verification for the p4_tail_fast=False variant). The plateau-undershoot terminal follows as its own commit — the audit found the rank-scatter path has no cand_count<K branch at all, so the fix is wider than the review comment assumed. PR body has the resolution table. Gates: sparse-attention suite 674 passed / 144 skipped; BSX suite 91 passed / 8 skipped.

longcheng-nv · 2026-07-29T10:07:02Z

/bot run --disable-fail-fast

…ssion path Phase 2 has two secant drivers: the SMEM/leader driver and a register-resident redundant-warp driver used at cluster_size == 1. The plateau terminal (done == 3) added earlier only landed in the leader driver, so a bitwise-equal plateau wider than the candidate buffer that straddles the K boundary still fell through to the legacy give-up on the register-resident path and left -1 pads in the output. Mirror both terminals into the register-resident driver: - adjacent-float bracket inside the refine loop -> plateau terminal - budget-exhausted bisection collapse after the loop, then a recount at the terminal threshold so Phase 3 sees sure-winner counts Every thread replays the driver from identical registers, so the added branches stay warp-uniform and block_count_ge keeps its barrier cadence. Extend the adversarial plateau matrix from 3 to 5 variants, adding the classic secant admission (enable_r0=False) at cluster_size 1 and 4 - the route that exposed this gap. GVR top-k suite 684 passed / 0 failed, bsx top-k suite 91 passed / 0 failed. Signed-off-by: longcheng-nv <243710427+longcheng-nv@users.noreply.github.com>

longcheng-nv · 2026-07-30T08:52:07Z

/bot run

coderabbitai

🧹 Nitpick comments (3)

tensorrt_llm/_torch/cute_dsl_kernels/blackwell/top_k/gvr_topk_decode.py (1)
2631-2658: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Consider keyword arguments at the two call sites for this 24-parameter helper.

All but the four Constexpr params are untyped, so a mis-ordered positional argument (e.g. f_lo/finv, b_star/sb_star, smem_keys/smem_vals) would compile cleanly and silently produce wrong tie selection. Passing them by keyword at Lines 3188-3213 and 3217-3242 makes the contract self-checking at no runtime cost.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tensorrt_llm/_torch/cute_dsl_kernels/blackwell/top_k/gvr_topk_decode.py`
around lines 2631 - 2658, Update both call sites of _p4_exact_tail_radix_select
to pass all non-Constexpr arguments by keyword, including the ambiguous pairs
b_star/sb_star, f_lo/finv, and smem_keys/smem_vals. Keep the existing argument
values and Constexpr argument handling unchanged while making the parameter
mapping explicit at both invocations.
tests/unittest/_torch/attention/sparse/test_cute_dsl_gvr_topk_decode.py (2)
1191-1205: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

The test re-implements pick_config's per-CTA width rule it is meant to guard.

(msl if msl is not None else n) // cs duplicates the n_row // cluster_size step inside pick_config. If that derivation changes, this test keeps passing against its own copy and the drift it exists to catch goes unnoticed. Consider having pick_config (or a small helper) return the per-CTA width and asserting on it instead of recomputing.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unittest/_torch/attention/sparse/test_cute_dsl_gvr_topk_decode.py`
around lines 1191 - 1205, The test setup around _GvrTopKKernel.pick_config and
R._pick_tuning duplicates pick_config’s per-CTA width calculation. Update
pick_config or introduce a shared helper to expose the computed per-CTA width,
then use that value when calling R._pick_tuning and assert against it, removing
the inline (msl if msl is not None else n) // cs derivation.
1218-1222: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Good route coverage; two new branches still look untested.

The base_r0off* variants directly exercise the secant collapse paths (both the redundant-warp cs=1 and leader cs>1 forms). Not covered by any variant here:

the fb_fix R0-miss collapse and its fail-soft non-plateau terminal (_run_phases, the s_iscalars[1] != 1 retry-budget block) — reachable only when the ladder misses, which no variant pins deterministically;

the exact-tail large-tie-class fallback (cnt_strad > 128), i.e. the _p4_exact_tail_radix_select call, since the new 16-bit test plants a small tie class that takes the p4_tail_fast path.

Want me to add a variant that forces an R0 miss (e.g. r0_qfracs skewed off the plateau) and a >128-member tie class to cover both?

Also applies to: 1251-1255
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unittest/_torch/attention/sparse/test_cute_dsl_gvr_topk_decode.py`
around lines 1218 - 1222, The test variant list covers secant-collapse paths but
misses the fb_fix R0-miss retry-budget terminal and the exact-tail large-tie
fallback. Add deterministic coverage for both by configuring an R0-miss case
with skewed r0_qfracs that reaches _run_phases with s_iscalars[1] != 1, and a
16-bit case with cnt_strad > 128 so _p4_exact_tail_radix_select is selected
instead of p4_tail_fast.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tensorrt_llm/_torch/cute_dsl_kernels/blackwell/top_k/gvr_topk_decode.py`:
- Around line 2631-2658: Update both call sites of _p4_exact_tail_radix_select
to pass all non-Constexpr arguments by keyword, including the ambiguous pairs
b_star/sb_star, f_lo/finv, and smem_keys/smem_vals. Keep the existing argument
values and Constexpr argument handling unchanged while making the parameter
mapping explicit at both invocations.

In `@tests/unittest/_torch/attention/sparse/test_cute_dsl_gvr_topk_decode.py`:
- Around line 1191-1205: The test setup around _GvrTopKKernel.pick_config and
R._pick_tuning duplicates pick_config’s per-CTA width calculation. Update
pick_config or introduce a shared helper to expose the computed per-CTA width,
then use that value when calling R._pick_tuning and assert against it, removing
the inline (msl if msl is not None else n) // cs derivation.
- Around line 1218-1222: The test variant list covers secant-collapse paths but
misses the fb_fix R0-miss retry-budget terminal and the exact-tail large-tie
fallback. Add deterministic coverage for both by configuring an R0-miss case
with skewed r0_qfracs that reaches _run_phases with s_iscalars[1] != 1, and a
16-bit case with cnt_strad > 128 so _p4_exact_tail_radix_select is selected
instead of p4_tail_fast.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 93056dd8-1555-41ce-bf86-8d8986589f62

📥 Commits

Reviewing files that changed from the base of the PR and between 828e42a and 9eadcf2.

📒 Files selected for processing (2)

tensorrt_llm/_torch/cute_dsl_kernels/blackwell/top_k/gvr_topk_decode.py
tests/unittest/_torch/attention/sparse/test_cute_dsl_gvr_topk_decode.py

tensorrt-cicd · 2026-07-30T08:58:30Z

PR_Github #62750 [ run ] triggered by Bot. Commit: 9eadcf2 Link to invocation

tensorrt-cicd · 2026-07-30T08:58:48Z

PR_Github #62620 [ run ] completed with state ABORTED. Commit: 828e42a

Link to invocation

tensorrt-cicd · 2026-07-30T10:47:42Z

PR_Github #62750 [ run ] completed with state SUCCESS. Commit: 9eadcf2
/LLM/main/L0_MergeRequest_PR pipeline #50881 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

longcheng-nv · 2026-07-31T10:29:05Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-31T10:34:38Z

PR_Github #63021 [ run ] triggered by Bot. Commit: 9eadcf2 Link to invocation

tensorrt-cicd · 2026-07-31T13:34:08Z

PR_Github #63021 [ run ] completed with state SUCCESS. Commit: 9eadcf2
/LLM/main/L0_MergeRequest_PR pipeline #51127 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

longcheng-nv · 2026-08-01T03:39:17Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-01T03:45:44Z

PR_Github #63177 [ run ] triggered by Bot. Commit: 9eadcf2 Link to invocation

tensorrt-cicd · 2026-08-01T05:26:00Z

PR_Github #63177 [ run ] completed with state FAILURE. Commit: 9eadcf2
/LLM/main/L0_MergeRequest_PR pipeline #51261 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

longcheng-nv · 2026-08-01T09:36:29Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-01T09:42:41Z

PR_Github #63195 [ run ] triggered by Bot. Commit: 9eadcf2 Link to invocation

tensorrt-cicd · 2026-08-01T09:42:42Z

PR_Github #63195 [ run ] completed with state DISABLED
Pipeline is freezed and top-1 instance is under maintenance. For urgent request, contact Yiteng Niu

Link to invocation

longcheng-nv · 2026-08-02T15:28:05Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-02T15:34:27Z

PR_Github #63284 [ run ] triggered by Bot. Commit: 9eadcf2 Link to invocation

tensorrt-cicd · 2026-08-02T16:14:31Z

PR_Github #63284 [ run ] completed with state SUCCESS. Commit: 9eadcf2
/LLM/main/L0_MergeRequest_PR pipeline #51279 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

longcheng-nv · 2026-08-03T09:36:15Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-03T09:44:34Z

PR_Github #63417 [ run ] triggered by Bot. Commit: 9eadcf2 Link to invocation

tensorrt-cicd · 2026-08-03T10:22:15Z

PR_Github #63417 [ run ] completed with state SUCCESS. Commit: 9eadcf2
/LLM/main/L0_MergeRequest_PR pipeline #51393 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

longcheng-nv · 2026-08-04T05:53:44Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-04T05:59:51Z

PR_Github #63683 [ run ] triggered by Bot. Commit: 9eadcf2 Link to invocation

tensorrt-cicd · 2026-08-04T06:40:17Z

PR_Github #63683 [ run ] completed with state FAILURE. Commit: 9eadcf2
/LLM/main/L0_MergeRequest_PR pipeline #51635 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

longcheng-nv · 2026-08-04T09:16:16Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-08-04T09:22:35Z

PR_Github #63720 [ run ] triggered by Bot. Commit: 9eadcf2 Link to invocation

tensorrt-cicd · 2026-08-04T10:03:24Z

PR_Github #63720 [ run ] completed with state SUCCESS. Commit: 9eadcf2
/LLM/main/L0_MergeRequest_PR pipeline #51672 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

github-actions Bot assigned longcheng-nv Jul 26, 2026

longcheng-nv and others added 4 commits July 28, 2026 13:51

longcheng-nv force-pushed the perf/gvr-topk-bsx-cutedsl-tiers branch from 1f4bdff to 4f95451 Compare July 28, 2026 14:18

longcheng-nv added 2 commits July 28, 2026 14:33

longcheng-nv force-pushed the perf/gvr-topk-bsx-cutedsl-tiers branch from 4f95451 to 78df9cb Compare July 28, 2026 14:34

longcheng-nv removed request for allisonlim-nv, fredricz-20070104 and pengbowang-nv July 30, 2026 07:25

coderabbitai Bot reviewed Jul 30, 2026

View reviewed changes

Uh oh!

Conversation

longcheng-nv commented Jul 26, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Design

Performance (B200, fp32, real SWE-bench decode captures, paired same-rep cold-L2 nsys)

Review follow-ups from #16457 (resolved here)

Test Coverage

Draft checklist (before ready-for-review)

PR Checklist

Dev Engineer Review

QA Engineer Review

Test changes

CI coverage & verdict

Uh oh!

longcheng-nv commented Jul 29, 2026

Uh oh!

longcheng-nv commented Jul 29, 2026

Uh oh!

tensorrt-cicd commented Jul 29, 2026

Uh oh!

longcheng-nv commented Jul 29, 2026

Uh oh!

longcheng-nv commented Jul 29, 2026

Uh oh!

longcheng-nv commented Jul 29, 2026

Uh oh!

tensorrt-cicd commented Jul 29, 2026

Uh oh!

tensorrt-cicd commented Jul 29, 2026

Uh oh!

tensorrt-cicd commented Jul 29, 2026

Uh oh!

longcheng-nv commented Jul 29, 2026

Uh oh!

tensorrt-cicd commented Jul 29, 2026

Uh oh!

longcheng-nv commented Jul 29, 2026

Uh oh!

tensorrt-cicd commented Jul 29, 2026

Uh oh!

tensorrt-cicd commented Jul 29, 2026

Uh oh!

tensorrt-cicd commented Jul 29, 2026

Uh oh!

longcheng-nv commented Jul 29, 2026

Uh oh!

longcheng-nv commented Jul 29, 2026

Uh oh!

longcheng-nv commented Jul 30, 2026

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

tensorrt-cicd commented Jul 30, 2026

Uh oh!

tensorrt-cicd commented Jul 30, 2026

Uh oh!

tensorrt-cicd commented Jul 30, 2026

Uh oh!

longcheng-nv commented Jul 31, 2026

Uh oh!

tensorrt-cicd commented Jul 31, 2026

Uh oh!

tensorrt-cicd commented Jul 31, 2026

Uh oh!

longcheng-nv commented Aug 1, 2026

Uh oh!

tensorrt-cicd commented Aug 1, 2026

Uh oh!

tensorrt-cicd commented Aug 1, 2026

Uh oh!

longcheng-nv commented Aug 1, 2026

Uh oh!

tensorrt-cicd commented Aug 1, 2026

Uh oh!

tensorrt-cicd commented Aug 1, 2026

Uh oh!

longcheng-nv commented Aug 2, 2026

longcheng-nv commented Jul 26, 2026 •

edited by coderabbitai Bot

Loading