Skip to content

CUDA MatMulNBits: make SM90 prepacked rejection order portable and stabilize test match prefix - #29797

Merged
tianleiwu merged 1 commit into
tlwu/fix_windows_buildfrom
copilot/fix-failing-cuda-plugin-test
Jul 21, 2026
Merged

CUDA MatMulNBits: make SM90 prepacked rejection order portable and stabilize test match prefix#29797
tianleiwu merged 1 commit into
tlwu/fix_windows_buildfrom
copilot/fix-failing-cuda-plugin-test

Conversation

Copilot AI commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Description

MatMulNBits.Fp16_Int4_PrepackedSm90BlockSize32Rejected was failing in CUDA Plugin Linux CI because SM90 prepacked validation emitted environment-dependent messages before checking runner capability.
This change makes rejection precedence portable across runner/build-machine combinations and stabilizes message matching via a shared prefix.

  • Runtime-first SM90 gating

    • For weight_prepacked=2 (SM90 layout), validation now checks sm_ == 90 before compile-time native-SM90 availability checks.
  • Stable rejection prefix contract

    • All SM90-prepacked rejection paths now use the exact prefix:
      • weight_prepacked=2 (SM90 layout)
  • Build-support error clarity

    • If COMPILE_HOPPER_TMA_GEMMS is unavailable, the post-runtime-check error explicitly reports missing native SM90 Hopper support.
  • Block-size rule preserved

    • SM90 prepacked still enforces block_size{64, 128} after runtime/build checks.
  • Test portability update

    • Fp16_Int4_PrepackedSm90BlockSize32Rejected continues asserting only the stable shared prefix so it passes on:
      • non-SM90 runners,
      • SM90 runners without native TMA support in the build,
      • SM90 runners with native TMA support (block-size rejection path).
if (weight_prepacked_ == kMatMulNBitsWeightPrepackedSm90) {
  ORT_ENFORCE(sm_ == 90, "weight_prepacked=2 (SM90 layout) ...");
#if !defined(COMPILE_HOPPER_TMA_GEMMS)
  ORT_THROW("weight_prepacked=2 (SM90 layout) is not supported by this ONNX Runtime build ...");
#endif
  ORT_ENFORCE(block_size_ == 64 || block_size_ == 128,
              "weight_prepacked=2 (SM90 layout) supports block_size 64 or 128 only, but got ", block_size_);
}

Motivation and Context

The failing CI test exposed that rejection order for SM90-prepacked weights depended on build configuration rather than first honoring runtime device capability, which is incorrect for SM90 layout execution on non-SM90 runners.
This update aligns validation precedence with execution constraints and makes test matching architecture-agnostic while preserving precise downstream rejection semantics.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@tianleiwu
tianleiwu marked this pull request as ready for review July 21, 2026 01:29
@tianleiwu
tianleiwu merged commit 9ee050b into tlwu/fix_windows_build Jul 21, 2026
4 checks passed
@tianleiwu
tianleiwu deleted the copilot/fix-failing-cuda-plugin-test branch July 21, 2026 01:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants