[None][feat] MTP one-model advanced_sampling_mode: skip redundant top-k / top-p filter kernels with additional config enum

[jhaotingc]

Description

For one-model speculative decoding (MTP-Eagle one-model), the advanced sampler runs
flashinfer's top-k mask (RadixTopKMaskLogits) and top-p renorm (AirTopPRenorm*)
kernels on every draft layer and the target verification — even when the
deployment's requests disable those filters (top_k=0, top_p=1). In that case the
filters are mathematical no-ops (top-k mask at k=vocab, top-p renorm at p=1), but
the kernels still launch: measured at ~290 µs (top-k mask) + ~1.34 ms (top-p
renorm) per decode step at draft_len=3 (nsys, Qwen3.6-35B-A3B-NVFP4, B200). For
fixed-config / temperature-only deployments this is pure overhead.

Add a deploy-time advanced_sampling_mode enum on DecodingBaseConfig (available
to any speculative config; default FULL) that tells the one-model sampler which
filters the deployment disables, so their flashinfer kernels are skipped:

mode	top-k kernel	top-p kernel
FULL (default)	run	run
NO_TOPK	skip	run
NO_TOPP	run	skip
NO_TOPK_NO_TOPP	skip	skip (pure temperature softmax)

Design:

• Single source of truth. AdvancedSamplingMode.skips_top_k / .skips_top_p
  properties are the only place the mode→filter mapping lives. The enum is carried
  end-to-end: pydantic parses the YAML/JSON string into the enum on the config,
  SpecMetadata stores the enum, and there are no bare-string predicates downstream.
• Resolve once on the spec side. resolve_advanced_sampling_filters(mode, top_k, top_p) returns effective tensors with a disabled filter set to None, so the
  sampler/prob ops stay mode-agnostic.
• Unified sampler. sample_from_logits_op(logits, temperatures, top_k, top_p, seed, offset) (mirroring compute_probs_from_logits_op) is the single one-model sampler:
  a None top_k skips the mask kernel; a None top_p samples via
  sampling_from_probs instead of top_p_sampling_from_probs. The rejection path's
  compute_probs_from_logits skips the top-k mask / top-p renorm identically.
• Native greedy — no guard. Greedy rows carry a sentinel temperature
  (DISABLE_TEMP_VAL) set in _scan_one_model_sampling, so softmax(logits / DISABLE_TEMP_VAL) collapses to a one-hot at the argmax and sampling returns the
  argmax token. Every mode therefore supports mixed greedy + sampling batches with no
  special-case error.
• No extra graphs. advanced_sampling_mode is a deploy-time setting and is not
  part of the CUDA-graph key.
• Lossless. When a skipped filter is already disabled (e.g. NO_TOPK with
  top_k=0), the probs — and therefore the sampled tokens and rejection acceptance —
  are bit-for-bit identical to FULL; only the kernel launch is removed.
• Not gated on rejection sampling. All four modes are valid with
  use_rejection_sampling on or off; the two are independent.

Benchmark

Qwen3.6-35B-A3B-NVFP4, TRT-LLM (1×TP1, MTP draft_len=3), B200, rejection sampling ON,
SPEED-Bench low-entropy (ISL≈2197, OSL=1024, temp=0.7), prefix-cache OFF. Values are
OTPS/gpu (output tok/s/gpu). Plot: plots/pareto_rejection_conc.png.

conc	FULL — top_k=20, top_p=0.95, temp=0.7	FULL — temp only (top_k=0, top_p=1)	NO_TOPK_NO_TOPP — temp only	Speedup*
1	599.1	559.5	714.2	1.28×
2	1058.4	1052.0	1237.1	1.18×
4	1840.6	1547.9	1932.2	1.25×
8	2750.7	2874.2	3158.9	1.10×
16	4267.6	4196.3	4539.2	1.08×
32	6010.7	6014.1	6506.9	1.08×
64	8142.7	8061.2	9059.6	1.12×
128	10610.8	10599.0	11884.9	1.12×
256	13540.6	13709.0	15386.7	1.12×

* Speedup = NO_TOPK_NO_TOPP / FULL (temp only) — same sampling config
(top_k=0, top_p=1), isolating the effect of skipping the two filter kernels.
Acceptance length is unchanged across all rows, confirming identical sampling behavior.
FULL with real top_k=20 / top_p=0.95 is shown as a reference for the actual
filtered-sampling throughput.

Result: for a temperature-only deployment, NO_TOPK_NO_TOPP gives a consistent
~8–28% throughput gain (≈+12% at high concurrency) at zero accuracy cost, by
removing the two redundant per-draft-layer filter kernels.

Test coverage

tests/unittest/_torch/speculative/hw_agnostic/test_advanced_sampling_mode.py
(12 cases, all passing on B200):

• Config contract — enum values {full, no_topk, no_topp, no_topk_no_topp} and the
  skips_top_k / skips_top_p properties; the field lives on DecodingBaseConfig
  (default FULL); all four modes construct with or without use_rejection_sampling
  (no gating).
• Mode resolution — resolve_advanced_sampling_filters sets the disabled filter to
  None per mode (full→neither, no_topk→top_k, no_topp→top_p,
  no_topk_no_topp→both) and passes kept filters through unchanged.
• Bit-for-bit equivalence (CUDA) — NO_TOPK via sample_from_logits_op yields the
  exact same tokens as FULL when top_k is disabled, for the same seed/offset, across
  top_p ∈ {1.0, 0.9} (the top_k mask is a no-op at k=vocab).
• Native greedy (CUDA) — a greedy row (sentinel temperature) in a mixed batch
  returns its argmax token under NO_TOPK and NO_TOPK_NO_TOPP, validating the
  guard-free design.
• Spec-path acceptance — non-FULL modes construct on any spec-decoding path
  (config-time acceptance; unsupported paths fall back to FULL behavior at runtime).

Telemetry manifest / CODEOWNERS

Because this adds a new DecodingBaseConfig field, tensorrt_llm/usage/llm_args_golden_manifest.json
is regenerated via scripts/generate_llm_args_golden_manifest.py (adds
speculative_config.advanced_sampling_mode, kind: categorical). The manifest
--check CI gate passes. Touching the usage-telemetry path requires approval from
@NVIDIA/trt-llm-usage-telemetry-devs, @NVIDIA/trt-llm-oss-compliance, and
@NVIDIA/trt-llm-noncommitted-api-review-committee. The new field is additive with
default FULL (API-compatible).

Note — alternative design

Instead of a deploy-time setting that changes which sampler the single advanced CUDA
graph captures, each mode could be a separately captured CUDA graph selected at
replay by a graph key — the same mechanism is_all_greedy_sample already uses to switch
between the greedy (argmax) and advanced-sampling graphs. That would allow per-batch
mode switching, but it multiplies the number of captured decode graphs (one set per
mode) and lengthens warmup. We chose the deploy-time enum (0 extra graphs) because the
sampling config is fixed per deployment, which is the target use case.

Test Coverage

tests/unittest/_torch/speculative/hw_agnostic/test_advanced_sampling_mode.py
(11 cases, all passing on B200):

• Config contract — enum values {full, no_topk, no_topp, no_topk_no_topp};
  NO_TOPK is accepted without rejection sampling; NO_TOPP and
  NO_TOPK_NO_TOPP raise a ValidationError unless use_rejection_sampling=True
  (and are accepted with it).
• Filter-skip plumbing — for each mode, compute_probs_from_logits passes
  top_k=None and/or top_p=None into the flashinfer op (checked with a spy),
  so the right kernel is skipped: full→neither, no_topk→top_k,
  no_topp→top_p, no_topk_no_topp→both.
• Bit-for-bit equivalence (CUDA) — sampling_batch_spec_dec_one_model in
  NO_TOPK yields the exact same tokens as FULL when top_k is disabled, for the
  same seed/offset, across top_p ∈ {1.0, 0.9, 0.5} (the top_k mask is a no-op
  at k=vocab). Skipped on CPU-only CI.

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

Summary by CodeRabbit

• New Features

  • Added deploy-time AdvancedSamplingMode for one-model MTP speculative decoding, exposed as DecodingBaseConfig.advanced_sampling_mode (default FULL) and plumbed via SpecMetadata.
  • Modes:
    • FULL — apply both top-k and top-p filters
    • NO_TOPK — skip top-k filtering
    • NO_TOPP — skip top-p renormalization
    • NO_TOPK_NO_TOPP — skip both filters
  • Implemented CUDA-graph-friendly kernel skipping by resolving disabled filters to None:
    • resolve_advanced_sampling_filters(...) converts (advanced_sampling_mode, top_k, top_p) into effective optional tensors where disabled filters become None
    • updated sampling code to conditionally skip FlashInfer filtering kernels when the corresponding optional tensor is None
  • Added a CUDA-graph-compatible sampler sample_from_logits_op(...):
    • always computes softmax
    • applies top-k masking only if top_k is provided
    • performs top-p sampling only if top_p is provided
  • Wired mode behavior across speculative paths:
    • updated one-model speculative draft sampling (rejection and non-rejection) to resolve effective top_k/top_p from SpecMetadata.advanced_sampling_mode
    • updated dynamic-tree token sampling to use sample_from_logits_op(...) for non-greedy target token sampling

• Tests

  • Added tests/unittest/_torch/speculative/hw_agnostic/test_advanced_sampling_mode.py covering:
    • enum string values and skips_top_k / skips_top_p flags
    • defaults and construction on DecodingBaseConfig / MTPDecodingConfig
    • resolve_advanced_sampling_filters(...) identity vs disabled-to-None behavior
    • CUDA-only bit-for-bit expectations (e.g., NO_TOPK matches FULL when top_k is disabled)
    • greedy-row behavior via sentinel-temperature argmax handling
    • acceptance of non-FULL modes across speculative/spec paths (including removal of the prior “one-model-only” gate)
  • Regenerated tensorrt_llm/usage/llm_args_golden_manifest.json to add speculative_config.advanced_sampling_mode as a categorical argument.

• Docs / Config

  • Documented “Advanced sampling mode (speculative decoding)” in docs/source/features/sampling.md, including semantics, deploy-time behavior (not in CUDA-graph key), independence from rejection sampling, and a usage example.

---

Dev Engineer Review

• Correctness & API consistency

  • Mode-to-filter behavior is centralized in resolve_advanced_sampling_filters(...) and consistently threaded into:
    • speculative draft probability computation (compute_probs_from_logits)
    • speculative draft/target token sampling call sites (rejection + non-rejection)
    • dynamic-tree sampling for non-greedy paths
  • Treating disabled top_k/top_p as None aligns with the stated goal: redundant FlashInfer kernels can be skipped while keeping mathematical no-op behavior when filters are disabled.
  • Greedy-row handling is designed to work via a sentinel temperature mechanism without extra special-casing guards.

• CUDA-graph / keying

  • advanced_sampling_mode is excluded from the CUDA-graph key; functional selection is driven by resolved optional filter tensors (None vs provided tensors), matching the intended graph-stability requirement.

• Config / manifest

  • AdvancedSamplingMode added to tensorrt_llm/llmapi/llm_args.py and defaulted to FULL on DecodingBaseConfig.
  • tensorrt_llm/usage/llm_args_golden_manifest.json updated to include speculative_config.advanced_sampling_mode with allowed values: full, no_topk, no_topp, no_topk_no_topp.

---

QA Engineer Review

Test changes

• Touched: tests/unittest/_torch/speculative/hw_agnostic/test_advanced_sampling_mode.py
• Added test coverage
  • Enum behavior (AdvancedSamplingMode values and skip flags)
  • Defaulting/config acceptance (DecodingBaseConfig, MTPDecodingConfig, independent of rejection sampling)
  • Filter resolution (resolve_advanced_sampling_filters maps disabled inputs to None)
  • CUDA-only sampler equivalence (bit-for-bit match when filters are mathematically no-ops)
  • Greedy-row correctness (argmax behavior for no_topk / no_topk_no_topp)
  • Speculative path acceptance (non-FULL modes allowed and stored)
• Integration test-list coverage (tests/integration/test_lists/): not indicated in the provided context.
• Verdict: needs follow-up

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

Walkthrough

Adds deploy-time advanced sampling modes for MTP one-model speculative decoding, propagates the selected mode through metadata and sampling paths, conditionally skips top-k/top-p filtering, and adds validation tests and documentation.

Changes

Advanced sampling modes

Layer / File(s)	Summary
Sampling mode configuration and metadata tensorrt_llm/llmapi/llm_args.py, tensorrt_llm/_torch/speculative/utils.py, tensorrt_llm/usage/llm_args_golden_manifest.json	Defines four sampling modes, adds the public configuration field, and transfers the selected mode into Eagle3 one-model metadata and the args manifest.
Mode-specific probability and token sampling tensorrt_llm/_torch/pyexecutor/sampler/sampling_utils.py	Allows disabled top-k/top-p filters to be represented by None and adds a CUDA-graph-compatible sampler.
Speculative decoding integration tensorrt_llm/_torch/speculative/interface.py, tensorrt_llm/_torch/speculative/eagle3_dynamic_tree.py	Propagates effective filters through draft, rejection, block, batch, and dynamic-tree sampling paths.
Advanced sampling validation and documentation tests/unittest/_torch/speculative/hw_agnostic/test_advanced_sampling_mode.py, docs/source/features/sampling.md	Covers mode properties, configuration, filter resolution, sampling equivalence, greedy rows, and supported modes.

Estimated code review effort: 3 (Moderate) | ~30 minutes

Sequence Diagram(s)

sequenceDiagram
  participant MTPDecodingConfig
  participant SpecMetadata
  participant SpeculativeInterface
  participant sampling_utils
  participant FlashInfer
  MTPDecodingConfig->>SpecMetadata: configure advanced_sampling_mode
  SpecMetadata->>SpeculativeInterface: provide mode for speculative sampling
  SpeculativeInterface->>sampling_utils: resolve effective top_k and top_p
  sampling_utils-->>SpeculativeInterface: return filters or None
  SpeculativeInterface->>sampling_utils: sample draft tokens or compute rejection probabilities
  sampling_utils->>FlashInfer: apply enabled sampling kernels
  FlashInfer-->>sampling_utils: return probabilities or token IDs
  sampling_utils-->>SpeculativeInterface: return sampling results

Loading

Possibly related PRs

• NVIDIA/TensorRT-LLM#12062: Changes the EAGLE3 dynamic-tree speculative sampling path that this PR now routes through sample_from_logits_op.

Suggested reviewers: cascade812, arysef, chang-l, qijune, lori-ren

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 94.74% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title is specific, concise, and accurately summarizes the main feature change.
Description check	✅ Passed	The description includes the required Description, Test Coverage, and PR Checklist sections with substantial detail.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 4

🧹 Nitpick comments (3)

tests/unittest/_torch/speculative/hw_agnostic/test_advanced_sampling_mode.py (2)

39-53: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Exercise rejection-disabled behavior explicitly.

Lines 41 and 49 rely on the default for use_rejection_sampling; set it to False so the tests directly prove both sides of the contract.

Proposed test adjustment

-    cfg = MTPDecodingConfig(max_draft_len=1, advanced_sampling_mode="no_topk")
+    cfg = MTPDecodingConfig(
+        max_draft_len=1,
+        advanced_sampling_mode="no_topk",
+        use_rejection_sampling=False,
+    )
...
-        MTPDecodingConfig(max_draft_len=1, advanced_sampling_mode=mode)
+        MTPDecodingConfig(
+            max_draft_len=1,
+            advanced_sampling_mode=mode,
+            use_rejection_sampling=False,
+        )

As per path instructions, coverage in tests/unittest/_torch/speculative/hw_agnostic/test_advanced_sampling_mode.py needs direct validation of the configured behavior.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unittest/_torch/speculative/hw_agnostic/test_advanced_sampling_mode.py`
around lines 39 - 53, Update both MTPDecodingConfig constructions in
test_no_topk_does_not_require_rejection and
test_topp_disabling_modes_require_rejection to explicitly pass
use_rejection_sampling=False for the rejection-disabled cases, while keeping the
existing rejection-enabled configuration unchanged.

Source: Path instructions

---

90-125: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Cover NO_TOPK rejection of greedy requests.

This only tests temperature 0.7. Add a case at or below GREEDY_TEMPERATURE_THRESHOLD that asserts the NO_TOPK request-validation path rejects it, plus a just-above-threshold success case.

Based on PR objectives, NO_TOPK disallows greedy requests. As per path instructions, coverage in tests/unittest/_torch/speculative/hw_agnostic/test_advanced_sampling_mode.py is insufficient for that contract.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unittest/_torch/speculative/hw_agnostic/test_advanced_sampling_mode.py`
around lines 90 - 125, Extend test_no_topk_matches_full_bit_for_bit or add a
focused test covering NO_TOPK temperature validation. Use temperatures at or
below GREEDY_TEMPERATURE_THRESHOLD to assert the NO_TOPK request is rejected,
and just above the threshold to assert it succeeds. Reuse
sampling_batch_spec_dec_one_model and the existing test setup symbols, while
preserving the current bit-for-bit comparison coverage.

Source: Path instructions

tensorrt_llm/llmapi/llm_args.py (1)

2484-2504: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Add the validator return annotation.

_log_advanced_sampling_mode should declare -> "MTPDecodingConfig".

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tensorrt_llm/llmapi/llm_args.py` around lines 2484 - 2504, The
_log_advanced_sampling_mode validator is missing its return annotation. Update
that method signature to declare -> "MTPDecodingConfig", preserving its existing
validation logic and return self behavior.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tensorrt_llm/_torch/pyexecutor/sampler/sampling_utils.py`:
- Around line 890-897: Update the advanced_sampling_mode branch in the direct
sampling path to use compute_probs_from_logits with the selected mode, then
sample via sampling_from_probs_op instead of always calling
top_p_sampling_from_probs_op. Ensure both NO_TOPP and NO_TOPK_NO_TOPP bypass
top_p filtering while preserving the existing temperature, seed, and offset
behavior.

In `@tensorrt_llm/_torch/speculative/interface.py`:
- Around line 830-846: Initialize and update self.has_greedy_requests while
scanning per-request sampling parameters, ensuring it reflects any greedy row in
mixed batches. Apply the existing mixed-batch rejection to every non-FULL
advanced sampling mode, including no_topk and no_topk_no_topp, while preserving
the all-greedy and warmup-dummy exemptions.

In `@tensorrt_llm/llmapi/llm_args.py`:
- Around line 2475-2482: Validate advanced_sampling_mode after the final
spec-decoding mode is determined, accepting non-FULL values only for the
MTP-Eagle one-model branch. Reject unsupported non-FULL modes for vanilla and
two-model MTP instead of silently ignoring them, while preserving FULL as the
default and existing metadata behavior.

In
`@tests/unittest/_torch/speculative/hw_agnostic/test_advanced_sampling_mode.py`:
- Line 30: Add complete type annotations to every function in this test module,
including all test functions and the spy helper: annotate each parameter with
its precise type and use -> None for functions that do not return a value.
Update the functions represented by test_advanced_sampling_mode_enum_values and
the other referenced test/spy definitions without changing their behavior.

---

Nitpick comments:
In `@tensorrt_llm/llmapi/llm_args.py`:
- Around line 2484-2504: The _log_advanced_sampling_mode validator is missing
its return annotation. Update that method signature to declare ->
"MTPDecodingConfig", preserving its existing validation logic and return self
behavior.

In
`@tests/unittest/_torch/speculative/hw_agnostic/test_advanced_sampling_mode.py`:
- Around line 39-53: Update both MTPDecodingConfig constructions in
test_no_topk_does_not_require_rejection and
test_topp_disabling_modes_require_rejection to explicitly pass
use_rejection_sampling=False for the rejection-disabled cases, while keeping the
existing rejection-enabled configuration unchanged.
- Around line 90-125: Extend test_no_topk_matches_full_bit_for_bit or add a
focused test covering NO_TOPK temperature validation. Use temperatures at or
below GREEDY_TEMPERATURE_THRESHOLD to assert the NO_TOPK request is rejected,
and just above the threshold to assert it succeeds. Reuse
sampling_batch_spec_dec_one_model and the existing test setup symbols, while
preserving the current bit-for-bit comparison coverage.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 87e95728-f8e1-4129-8afa-df75e9e6ff80

📥 Commits

Reviewing files that changed from the base of the PR and between b520c6f and 6c7f810.

📒 Files selected for processing (5)

• tensorrt_llm/_torch/pyexecutor/sampler/sampling_utils.py
• tensorrt_llm/_torch/speculative/interface.py
• tensorrt_llm/_torch/speculative/utils.py
• tensorrt_llm/llmapi/llm_args.py
• tests/unittest/_torch/speculative/hw_agnostic/test_advanced_sampling_mode.py

[zhaoyangwang-nvidia]

Hi @mikeiovine Please take a look at this PR — it adds advanced_sampling_mode to skip the top-k / top-p filter kernels for temperature-only deployments. Any comments on this implementation approach?

[mikeiovine]

I think the optimization makes sense but I'm a bit worried about the API. As you point out in the description, we could make it more usable by capturing more graphs, but that's going to make the warmup too long and is not sustainable.

At least the default is FULL. This should be considered an optimization for advanced use cases IMO

[zhaoyangwang-nvidia]

Since this adds a new field to DecodingBaseConfig, per the repo policy for LLM args / nested-config changes, please run python3 scripts/generate_llm_args_golden_manifest.py and commit the updated tensorrt_llm/usage/llm_args_golden_manifest.json — the manifest currently doesn't include advanced_sampling_mode, which will fail CI. Also note that new config fields require approval from the telemetry/privacy CODEOWNERs.

[zhaoyangwang-nvidia]

@NVIDIA/trt-llm-doc-owners Please help to review this PR, thanks~

[tensorrt-cicd]

PR_Github #62482 [ run ] completed with state ABORTED. Commit: e2ff292

Link to invocation

[jhaotingc]

/bot run

[tensorrt-cicd]

PR_Github #62483 [ run ] completed with state SUCCESS. Commit: e2ff292
/LLM/main/L0_MergeRequest_PR pipeline #50632 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[jhaotingc]

/bot run

[tensorrt-cicd]

PR_Github #62514 [ run ] triggered by Bot. Commit: 1107598 Link to invocation

[tensorrt-cicd]

PR_Github #62515 [ run ] triggered by Bot. Commit: 1107598 Link to invocation

[tensorrt-cicd]

PR_Github #62514 [ run ] completed with state ABORTED. Commit: 1107598

Link to invocation

[tensorrt-cicd]

PR_Github #62515 [ run ] completed with state SUCCESS. Commit: 1107598
/LLM/main/L0_MergeRequest_PR pipeline #50663 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[jhaotingc]

/bot run

[tensorrt-cicd]

PR_Github #62535 [ run ] triggered by Bot. Commit: 467cfe0 Link to invocation

[jhaotingc]

/bot run

[tensorrt-cicd]

PR_Github #62535 [ run ] completed with state SUCCESS. Commit: 467cfe0
/LLM/main/L0_MergeRequest_PR pipeline #50678 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[jhaotingc]

/bot run

[tensorrt-cicd]

PR_Github #62551 [ run ] triggered by Bot. Commit: 467cfe0 Link to invocation

[tensorrt-cicd]

PR_Github #62548 [ run ] triggered by Bot. Commit: 467cfe0 Link to invocation

[tensorrt-cicd]

PR_Github/16561-467cfe0 #62551 was force-killed by a newer pipeline run.
L0 job information not available (job may not have been triggered yet).

Link to superseding invocation

[tensorrt-cicd]

PR_Github #62548 [ run ] completed with state SUCCESS. Commit: 467cfe0
/LLM/main/L0_MergeRequest_PR pipeline #50695 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[jhaotingc]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #62574 [ run ] triggered by Bot. Commit: 4fc76dc Link to invocation

[tensorrt-cicd]

PR_Github #62574 [ run ] completed with state SUCCESS. Commit: 4fc76dc
/LLM/main/L0_MergeRequest_PR pipeline #50717 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[jhaotingc]

/bot run

[tensorrt-cicd]

PR_Github #62693 [ run ] triggered by Bot. Commit: 4fc76dc Link to invocation

[tensorrt-cicd]

PR_Github #62693 [ run ] completed with state SUCCESS. Commit: 4fc76dc
/LLM/main/L0_MergeRequest_PR pipeline #50830 completed with status: 'SUCCESS'

CI Report

Link to invocation