[TRTLLM-14810][chore] Pre-flight: sync sa_worker.py with main (#16759); require packaging>=24.2 by brnguyen2 · Pull Request #17081 · NVIDIA/TensorRT-LLM

brnguyen2 · 2026-07-30T20:37:51Z

Description

Pre-flight change to shrink the conflict surface of the upcoming main -> feat/kimi_k3 catch-up merge (TRTLLM-14810). Two independent commits:

Sync tensorrt_llm/_torch/speculative/sa_worker.py with main. Main PR [None][fix] SA spec dec: promote accepted hybrid recurrent states in-worker #16759 upstreamed this branch's SA in-worker hybrid-state promotion verbatim, so the file contents now match main except for one line. Verification of the base-class interface on this branch: SpecWorkerBase here does not yet carry main's [https://nvbugs/6442074][fix] Make one-model spec-dec attn-metadata save/restore exception-safe #16382 exception-safe forward wrapper (there is no _forward_impl anywhere in tensorrt_llm/_torch/speculative/ on this branch, and workers are invoked through nn.Module.__call__ -> forward). The method therefore stays named forward here; renaming it would leave the worker uncalled. This is the single intentional residual delta vs main's copy of the file — the catch-up merge finishes the rename when it brings in [https://nvbugs/6442074][fix] Make one-model spec-dec attn-metadata save/restore exception-safe #16382.
Require packaging>=24.2 in requirements.txt (mirrors [TRTLLM-14709][infra] Require packaging>=24.2 for FlashInfer source builds #17075: FlashInfer source builds with --no-build-isolation need it for setuptools license-expression validation), and drop the now-redundant manual pip install -U 'packaging>=24.2' workaround step from examples/kimi_k3/README.md and the Kimi K3 deployment guide, replaced by a note that requirements.txt already satisfies it.

Test Coverage

Validated on Blackwell hardware at the tip of this branch with both commits applied (python-only changes validated against prebuilt binaries):

KDA/SA unit suites: tests/unittest/_torch/modeling/test_kda_mtp_decode_cute_parity.py, test_kimi_kda_fused_verify_parity.py, test_kimi_kda_verify_parity.py, tests/unittest/_torch/modules/kimi_kda/, tests/unittest/_torch/modules/moe/test_kimi_k3_situ_moe.py, tests/torch/speculative/test_suffix_automaton.py — all green.
SA-vs-baseline logits-parity integration run (4 GPUs, truncated-layer checkpoint, tests/integration/defs/kimi_k3_sa_harness.py) — the decisive gate for the sa_worker change; no logits drift.
End-to-end 16-GPU serving smoke (examples/kimi_k3/quick_start_kimi_k3.sbatch), standard and --enable-block-reuse variants — all expected-text checks pass.
Disaggregated-serving suites: tests/unittest/disaggregated/test_bounce.py, tests/unittest/disaggregated/test_kda_mamba_transfer.py, tests/unittest/_torch/speculative/hw_agnostic/test_sa.py, plus the TinyLlama ctx/gen/proxy-vs-aggregated parity smoke (tests/integration/defs/kimi_k3_disagg_parity.py) — all pass. Note: the parity smoke was initially blocked by a dependency-artifact incompatibility unrelated to this change (the test environment carried a newer flashinfer-python than this branch's ==0.6.14 pin, tripping a kernel-signature mismatch in the attention decode path); rerunning with the pinned version restored it, and the smoke passes with this PR applied.

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

brnguyen2 · 2026-07-30T20:38:54Z

/bot run

tensorrt-cicd · 2026-07-30T20:45:00Z

PR_Github #62850 [ run ] triggered by Bot. Commit: 43fd565 Link to invocation

tensorrt-cicd · 2026-07-30T21:16:40Z

PR_Github #62850 [ run ] completed with state FAILURE. Commit: 43fd565
/LLM/main/L0_MergeRequest_PR pipeline #50972 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

brnguyen2 · 2026-07-30T21:46:10Z

Note on the L0 failure: per the CI failure analysis, the break is an x86_64 wheel build failure in cpp/tensorrt_llm/kernels/kimiK3AttnRes/attnResFwd.cu (PTX assembly for sm_120f) — pre-existing on the base branch and untouched by this PR (docs/requirements/sa_worker.py only). Will re-run CI once the branch-level build issue is addressed.

brnguyen2 · 2026-07-30T23:25:39Z

/bot run

tensorrt-cicd · 2026-07-30T23:32:02Z

PR_Github #62874 [ run ] triggered by Bot. Commit: ad7b478 Link to invocation

tensorrt-cicd · 2026-07-30T23:48:58Z

PR_Github #62874 [ run ] completed with state FAILURE. Commit: ad7b478
/LLM/main/L0_MergeRequest_PR pipeline #50996 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Main PR NVIDIA#16759 upstreamed this branch's SA in-worker hybrid-state promotion verbatim; syncing the file back removes the catch-up-merge conflict. One intentional residual delta: the method stays named `forward` (not `_forward_impl`) because this branch's SpecWorkerBase does not yet carry main's NVIDIA#16382 exception-safe forward wrapper -- workers here are invoked via nn.Module.__call__ -> forward. The merge will finish the rename. Signed-off-by: Brian Nguyen <brnguyen@nvidia.com>

…rkaround from Kimi K3 docs Mirrors main PR NVIDIA#17075: FlashInfer source builds with --no-build-isolation need packaging>=24.2 for setuptools license-expression validation, so declare it in requirements.txt and remove the manual 'pip install -U packaging' step from the Kimi K3 README and deployment guide. Signed-off-by: Brian Nguyen <brnguyen@nvidia.com>

brnguyen2 · 2026-07-30T23:58:05Z

/bot run

tensorrt-cicd · 2026-07-31T00:03:45Z

PR_Github #62885 [ run ] triggered by Bot. Commit: 37ba252 Link to invocation

tensorrt-cicd · 2026-07-31T02:06:24Z

PR_Github #62885 [ run ] completed with state SUCCESS. Commit: 37ba252
/LLM/main/L0_MergeRequest_PR pipeline #51006 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

brnguyen2 · 2026-07-31T02:17:05Z

/bot run

tensorrt-cicd · 2026-07-31T02:23:54Z

PR_Github #62916 [ run ] triggered by Bot. Commit: 37ba252 Link to invocation

tensorrt-cicd · 2026-07-31T03:10:45Z

PR_Github #62916 [ run ] completed with state SUCCESS. Commit: 37ba252
/LLM/main/L0_MergeRequest_PR pipeline #51038 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

brnguyen2 · 2026-07-31T03:18:14Z

/bot run

tensorrt-cicd · 2026-07-31T03:23:49Z

PR_Github #62934 [ run ] triggered by Bot. Commit: 37ba252 Link to invocation

tensorrt-cicd · 2026-07-31T07:57:43Z

PR_Github #62934 [ run ] completed with state SUCCESS. Commit: 37ba252
/LLM/main/L0_MergeRequest_PR pipeline #51053 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

brnguyen2 · 2026-07-31T08:10:42Z

/bot run

tensorrt-cicd · 2026-07-31T08:17:27Z

PR_Github #62996 [ run ] triggered by Bot. Commit: 37ba252 Link to invocation

tensorrt-cicd · 2026-07-31T11:17:23Z

PR_Github #62996 [ run ] completed with state SUCCESS. Commit: 37ba252
/LLM/main/L0_MergeRequest_PR pipeline #51104 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

brnguyen2 · 2026-07-31T11:47:40Z

/bot skip --comment "PR #17081 only changes docs, requirements.txt packaging>=24.2, and a formatting-equivalent sa_worker.py line; the hard failures are unrelated DeepSeek V3 Lite H100 memory/executor-init failures."

tensorrt-cicd · 2026-07-31T11:53:43Z

PR_Github #63026 [ ] completed with state FAILURE. Commit: 37ba252
Not allowed on merged PR

Link to invocation

brnguyen2 requested review from a team as code owners July 30, 2026 20:37

brnguyen2 requested review from YihuiLu512, allisonlim-nv, laikhtewari and nv-guomingz July 30, 2026 20:37

github-actions Bot assigned brnguyen2 Jul 30, 2026

brnguyen2 force-pushed the preflight-saworker-packaging branch from 43fd565 to ad7b478 Compare July 30, 2026 23:13

brnguyen2 added 2 commits July 30, 2026 16:58

brnguyen2 force-pushed the preflight-saworker-packaging branch from ad7b478 to 37ba252 Compare July 30, 2026 23:58

brnguyen2 merged commit 93746cb into NVIDIA:feat/kimi_k3 Jul 31, 2026
6 of 7 checks passed

Uh oh!

Conversation

brnguyen2 commented Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Test Coverage

PR Checklist

PR Checklist

Uh oh!

brnguyen2 commented Jul 30, 2026

Uh oh!

tensorrt-cicd commented Jul 30, 2026

Uh oh!

tensorrt-cicd commented Jul 30, 2026

Uh oh!

brnguyen2 commented Jul 30, 2026

Uh oh!

brnguyen2 commented Jul 30, 2026

Uh oh!

tensorrt-cicd commented Jul 30, 2026

Uh oh!

tensorrt-cicd commented Jul 30, 2026

Uh oh!

brnguyen2 commented Jul 30, 2026

Uh oh!

tensorrt-cicd commented Jul 31, 2026

Uh oh!

tensorrt-cicd commented Jul 31, 2026

Uh oh!

brnguyen2 commented Jul 31, 2026

Uh oh!

tensorrt-cicd commented Jul 31, 2026

Uh oh!

tensorrt-cicd commented Jul 31, 2026

Uh oh!

brnguyen2 commented Jul 31, 2026

Uh oh!

tensorrt-cicd commented Jul 31, 2026

Uh oh!

tensorrt-cicd commented Jul 31, 2026

Uh oh!

brnguyen2 commented Jul 31, 2026

Uh oh!

tensorrt-cicd commented Jul 31, 2026

Uh oh!

tensorrt-cicd commented Jul 31, 2026

Uh oh!

brnguyen2 commented Jul 31, 2026

Uh oh!

Uh oh!

tensorrt-cicd commented Jul 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

brnguyen2 commented Jul 30, 2026 •

edited

Loading