[https://nvbugs/6525059][fix] Drop `8` from `FP8BlockScaleMoeRunner::mSupportedTileN`, covering both the… by trtllm-agent · Pull Request #17135 · NVIDIA/TensorRT-LLM

trtllm-agent · 2026-07-31T19:01:36Z

Summary

Root cause: On sm100f/sm103 every DeepSeek-FP8 GEMM2 cubin at tile_tokens_dim=8 faults with cudaErrorIllegalAddress in its TMA-OOB epilogue store, and tileN=8 is selected for warmup/tiny batches.
Fix: Drop 8 from FP8BlockScaleMoeRunner::mSupportedTileN, covering both the autotuner tactic list and the tileN == -1 fallback without constructing a config-less tileN=8 runner; unwaive the GB200/GB300 entries.
Automated fix generated by repair-bot

Test plan

Verify fix on the same GPU type as the original failure
Check for regressions in related tests

Links

Bug: https://nvbugs/6525059

Dev Engineer Review

Removed tileN=8 from FP8BlockScaleMoeRunner::mSupportedTileN.
The change applies to autotuner tactics and the tileN == -1 fallback.
The runner no longer constructs a config-less tileN=8 runner.
The change addresses cudaErrorIllegalAddress faults on sm100f/sm103.
Removed the GB200, GB300, and B200 waiver entries for TestQwen3_30B_A3B::test_dummy_load_format.
No public API declarations changed.
No configuration errors, duplicate entries, or unintended scope changes were identified.

QA Engineer Review

No test functions changed.
Removed three SKIP entries from tests/integration/test_lists/waives.txt.
TensorRT-LLM CI completed successfully at commit 03f178b.
QA verification completed successfully for NVBug 6525059.
The customized QA test list was empty.

Verdict: sufficient

coderabbitai · 2026-07-31T19:07:22Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 85332ddf-6e23-435e-b089-ffb42ad98ffd

📥 Commits

Reviewing files that changed from the base of the PR and between b8db274 and 05b8927.

📒 Files selected for processing (2)

cpp/tensorrt_llm/thop/fp8BlockScaleMoe.cpp
tests/integration/test_lists/waives.txt

🚧 Files skipped from review as they are similar to previous changes (2)

tests/integration/test_lists/waives.txt
cpp/tensorrt_llm/thop/fp8BlockScaleMoe.cpp

Walkthrough

The FP8 MoE runner no longer supports tileN=8. Supported tile sizes are 16, 32, 64, and 128. Three Qwen3 integration-test waivers are removed.

Changes

FP8 MoE support and test waivers

Layer / File(s)	Summary
Update FP8 MoE tile support `cpp/tensorrt_llm/thop/fp8BlockScaleMoe.cpp`	Runner construction and fallback selection exclude tileN=8 because of a GEMM2 failure.
Remove integration-test waivers `tests/integration/test_lists/waives.txt`	The B200, GB200, and GB300 `TestQwen3_30B_A3B::test_dummy_load_format` skip entries are removed.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

NVIDIA/TensorRT-LLM#17108: Modifies related integration-test waiver entries.
NVIDIA/TensorRT-LLM#17120: Modifies FP8 block-scale MoE runner and token-tile handling.
NVIDIA/TensorRT-LLM#17180: Modifies related integration-test waiver entries.

Suggested reviewers: mzweilz, qijune

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the bug, fix type, and affected FP8BlockScaleMoeRunner tile size.
Description check	✅ Passed	The description explains the root cause and fix, identifies the bug, and records verification steps.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

coderabbitai

Actionable comments posted: 1

🧹 Nitpick comments (1)

cpp/tensorrt_llm/thop/fp8BlockScaleMoe.cpp (1)
367-373: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Name the supported tile sizes.

Line 373 uses non-exempt numeric literals directly in mSupportedTileN. Replace them with k-prefixed constexpr constants.
Proposed change
-        : mSupportedTileN{16, 32, 64, 128}
+        : mSupportedTileN{kSupportedTileN16, kSupportedTileN32, kSupportedTileN64, kSupportedTileN128}

 private:
+    static constexpr int32_t kSupportedTileN16{16};
+    static constexpr int32_t kSupportedTileN32{32};
+    static constexpr int32_t kSupportedTileN64{64};
+    static constexpr int32_t kSupportedTileN128{128};
As per coding guidelines, avoid magic literals except 0, nullptr, true, and false, and use k-prefixed camelCase constants.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@cpp/tensorrt_llm/thop/fp8BlockScaleMoe.cpp` around lines 367 - 373, Define
k-prefixed constexpr constants for the supported tile sizes near the relevant
configuration declarations, then initialize mSupportedTileN with those named
constants instead of the numeric literals 16, 32, 64, and 128. Preserve the
existing supported-size set and ordering.
Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@cpp/tensorrt_llm/thop/fp8BlockScaleMoe.cpp`:
- Around line 367-373: Validate cached and requested tile values against
mSupportedTileN before runner lookup in the relevant run() path, so unsupported
tileN=8 configurations are rejected or invalidated instead of reaching
mRunners.at(). Preserve valid cached configurations and the existing supported
tile set.

---

Nitpick comments:
In `@cpp/tensorrt_llm/thop/fp8BlockScaleMoe.cpp`:
- Around line 367-373: Define k-prefixed constexpr constants for the supported
tile sizes near the relevant configuration declarations, then initialize
mSupportedTileN with those named constants instead of the numeric literals 16,
32, 64, and 128. Preserve the existing supported-size set and ordering.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 3cdeff4c-99e0-47d4-91f8-d7821162500c

📥 Commits

Reviewing files that changed from the base of the PR and between 7443b7f and 03f178b.

📒 Files selected for processing (2)

cpp/tensorrt_llm/thop/fp8BlockScaleMoe.cpp
tests/integration/test_lists/waives.txt

💤 Files with no reviewable changes (1)

tests/integration/test_lists/waives.txt

crazydemo · 2026-08-03T06:16:18Z

/bot run --only-qa-verify

tensorrt-cicd · 2026-08-03T06:23:04Z

PR_Github #63374 [ run ] triggered by Bot. Commit: 03f178b Link to invocation

tensorrt-cicd · 2026-08-03T07:24:41Z

PR_Github #63374 [ run ] completed with state SUCCESS. Commit: 03f178b
LLM_FUNCTION_AUTO_V2C #345 completed with status: 'SUCCESS'
QA verify (empty customized_test_list) (NVBug 6525059, branch repair-bot-bug6525059, fork tensorrt-cicd, dry_run_close=true)

Link to invocation

… MoE The TRTLLM-Gen FP8 block-scale MoE picks tile_tokens_dim=8 whenever num_tokens*top_k/local_num_experts <= 8, i.e. for warmup and tiny decode batches. On sm100f/sm103 every DeepSeek-FP8 GEMM2 cubin at that tile (bmm_Bfloat16_E4m3E4m3_Fp32_t128x8x128{,u2}_..._dsFp8_{schedS,schPd4x2x2x3}_bN_...) faults with cudaErrorIllegalAddress in its TMA-OOB epilogue store, so the warmup forward of Qwen3-30B-A3B-FP8 dies. Both the Static and the Persistent variant fault, so the tile size is the discriminator rather than a single bad cubin. Remove 8 from the runner's supported tile list. This covers the autotuner tactic list and the tileN == -1 fallback (which clamps on front()) in one place, and avoids ever constructing the tileN=8 runner -- filtering the cubins in TrtllmGenBatchedGemmRunner::skipQuirks instead would leave that runner with no passing config and throw during construction. FP8 block-scale MoE stays enabled; small batches now run at tileN=16. Unwaive the GB200/GB300 TestQwen3_30B_A3B::test_dummy_load_format entries. Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>

Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>

trtllm-agent requested review from a team as code owners July 31, 2026 19:01

trtllm-agent requested review from QiJune, crazydemo, mlefeb01, mzweilz, schetlur-nv, sunnyqgg and xxi-nv July 31, 2026 19:01

github-actions Bot assigned trtllm-agent Jul 31, 2026

coderabbitai Bot reviewed Jul 31, 2026

View reviewed changes

Comment thread cpp/tensorrt_llm/thop/fp8BlockScaleMoe.cpp

trtllm-agent force-pushed the repair-bot-bug6525059 branch from 03f178b to b8db274 Compare August 3, 2026 06:37

trtllm-agent added 2 commits August 3, 2026 07:43

[nvbugs/6525059][chore] Remove stale waiver after fix

05b8927

Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com>

trtllm-agent force-pushed the repair-bot-bug6525059 branch from b8db274 to 05b8927 Compare August 3, 2026 14:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[https://nvbugs/6525059][fix] Drop `8` from `FP8BlockScaleMoeRunner::mSupportedTileN`, covering both the… - #17135

[https://nvbugs/6525059][fix] Drop `8` from `FP8BlockScaleMoeRunner::mSupportedTileN`, covering both the…#17135
trtllm-agent wants to merge 2 commits into
NVIDIA:mainfrom
tensorrt-cicd:repair-bot-bug6525059

trtllm-agent commented Jul 31, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Jul 31, 2026 •

edited

Loading

❌ Failed checks (1 warning)

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

crazydemo commented Aug 3, 2026

Uh oh!

tensorrt-cicd commented Aug 3, 2026

Uh oh!

tensorrt-cicd commented Aug 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

trtllm-agent commented Jul 31, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Links

Dev Engineer Review

QA Engineer Review

Uh oh!

coderabbitai Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

crazydemo commented Aug 3, 2026

Uh oh!

tensorrt-cicd commented Aug 3, 2026

Uh oh!

tensorrt-cicd commented Aug 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

trtllm-agent commented Jul 31, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jul 31, 2026 •

edited

Loading