[Test] Add GPU-free unit coverage for weight_prepacked=2 (SM90) rejection

[GopalakrishnanN]

Summary

Adds GPU-free unit coverage for the weight_prepacked=2 (SM90/Hopper) rejection logic that #29776 / 9ee050b introduced in MatMulNBits<T>'s constructor, plus the minimal, behavior-preserving refactor needed to make that logic testable without a GPU. Stacked on tlwu/fix_windows_build (not main), since this tests code that only exists on that branch.

Coverage gap

MatMulNBits<T>'s constructor validates a weight_prepacked=2 request inline with three checks, in order:

1. ORT_ENFORCE(sm_ == 90, ...) — device must be Hopper.
2. #if !defined(COMPILE_HOPPER_TMA_GEMMS) ORT_THROW(...) — the native SM90 kernel must actually be compiled into this build (it isn't on Windows/MSVC, which is exactly what [Build] Fix windows build for sm90 or later #29776 changed).
3. ORT_ENFORCE(block_size_ == 64 || block_size_ == 128, ...) — block_size must be Hopper-tile-aligned.

sm_ is read from real device properties (GetDeviceProp()), and check 1 runs before check 2. So the "native SM90 kernel not compiled in this build" throw is only reachable on an actual Hopper GPU that was built without the native kernel (e.g. real Hopper hardware + MSVC). Every existing prepacked test (Fp16_Int4_PrepackedSm90BlockSize32Rejected, etc.) uses OpTester + GTEST_SKIP() when no CUDA device is present, so that branch has no coverage at all in CI today.

Minimal refactor

Extracted the validation into two pure, host-only functions that take sm/block_size as plain parameters instead of reading real hardware:

• New header onnxruntime/contrib_ops/cuda/quantization/matmul_nbits_sm90_validation.h (declarations only, guarded by #if USE_FPA_INTB_GEMM):
  • bool IsNativeSm90FpAIntBGemmCompiled();
  • void ValidateSm90PrepackedWeightSupport(int sm, int64_t block_size);
• Both are defined non-inline in onnxruntime/contrib_ops/cuda/quantization/matmul_nbits.cc (not in the header), so there is a single definition living inside the CUDA EP translation unit that actually observes COMPILE_HOPPER_TMA_GEMMS — avoiding an ODR/macro-skew hazard if a test TU saw a different value of that macro than the real target does.
• matmul_nbits.h's constructor now does:

  if (weight_prepacked_ == kMatMulNBitsWeightPrepackedSm90) {
    ValidateSm90PrepackedWeightSupport(sm_, block_size_);
  }

  replacing the inlined checks. All three message strings are byte-identical to before; only the identifiers changed from sm_/block_size_ (members) to sm/block_size (parameters), which only affects the auto-stringified ORT_ENFORCE condition text, not the custom messages tests/users actually see. Net runtime behavior is unchanged.

Test

Added onnxruntime/test/contrib_ops/cuda_kernels/matmul_nbits_sm90_validation_test.cc, which calls ValidateSm90PrepackedWeightSupport/IsNativeSm90FpAIntBGemmCompiled directly with synthetic (sm, block_size) values — no CUDA device required:

• ValidateSm90PrepackedWeightSupport(80, 64) throws containing "weight_prepacked=2 (SM90 layout) requires a compute capability 9.0" (build-independent).
• If IsNativeSm90FpAIntBGemmCompiled(): ValidateSm90PrepackedWeightSupport(90, 64) does not throw; ValidateSm90PrepackedWeightSupport(90, 32) throws containing "supports block_size 64 or 128 only".
• Else (e.g. Windows/MSVC, or any non-Hopper CMAKE_CUDA_ARCHITECTURES build such as the sm86 used by windows_cuda.yml/linux_cuda_ci.yml): ValidateSm90PrepackedWeightSupport(90, 64) throws containing "is not supported by this ONNX Runtime build".

A deliberate deviation from a literal "add it to matmul_4bits_test.cc" — please sanity-check this

I initially planned to add this TEST() directly inside test/contrib_ops/matmul_4bits_test.cc and call the new function straight from there. That does not link: onnxruntime_providers_cuda (and _plugin) is built as a runtime-loaded shared module (onnxruntime_add_shared_library_module), and on Windows its exports are restricted to a 3-symbol allowlist in core/providers/cuda/symbols.def (GetProvider, CreateEpFactories, ReleaseEpFactory). matmul_nbits.cc's free functions are never exported from that DLL, and onnxruntime_provider_test/onnxruntime_test_all never link the CUDA provider module at compile time (AddTest()'s DEPENDS only does add_dependencies(), i.e. build-order, not target_link_libraries()).

The established ORT mechanism for unit-testing CUDA EP internals directly is test/contrib_ops/cuda_kernels/*.cc (see sibling fpA_intB_gemm_kernel_test.cc, softmax_topk_kernel_test.cc): when onnxruntime_ENABLE_CUDA_EP_INTERNAL_TESTS is on, these sources are compiled into a separate onnxruntime_providers_cuda_ut shared module together with $<TARGET_OBJECTS:onnxruntime_providers_cuda_obj> (the CUDA EP's own object files) — same translation/link domain, so direct calls resolve, and the same COMPILE_HOPPER_TMA_GEMMS compile definition applies to both (via config_cuda_provider_shared_module()). This module's gtest cases run via the existing TEST(CUDA_EP_Unittest, All) nested-gtest bridge in test/providers/cuda/cuda_provider_test.cc. I confirmed both windows_cuda.yml and linux_cuda_ci.yml already set onnxruntime_ENABLE_CUDA_EP_INTERNAL_TESTS=ON and don't set onnxruntime_BUILD_CUDA_EP_AS_PLUGIN, so the new file is picked up automatically by the existing file(GLOB ... contrib_ops/cuda_kernels/*.cc) in cmake/onnxruntime_unittests.cmake — no CMake changes needed for those two legs. (Both legs build at CMAKE_CUDA_ARCHITECTURES=86, so COMPILE_HOPPER_TMA_GEMMS is undefined in both regardless of MSVC — they'll exercise the "not compiled" branch of the new test, which is exactly the previously-uncovered branch.)

I added a short breadcrumb comment in matmul_4bits_test.cc next to Fp16_Int4_PrepackedSm90BlockSize32Rejected pointing at the new file, so this redirection is discoverable from where the original tests live.

(Note: the hand-curated onnxruntime_test_providers_cuda_plugin_internal_test_src list used only when onnxruntime_BUILD_CUDA_EP_AS_PLUGIN is set was intentionally left untouched, to keep this change minimal — neither target CI leg uses that mode.)

Verification

• Did not attempt a full CUDA build (needs the CUDA toolkit + hours; out of scope here).
• Wrote a standalone throwaway program (not checked in) reproducing the extracted function bodies with minimal ORT_ENFORCE/ORT_THROW stand-ins, and compiled + ran it twice with MSVC (cl.exe) — once with /DCOMPILE_HOPPER_TMA_GEMMS, once without — confirming both preprocessor branches compile cleanly and all three message strings match exactly what's specified above.
• Ran lintrunner -a on all changed files: no issues.
• Re-read the diff for ODR correctness (both new functions are non-inline with a single definition in matmul_nbits.cc), guard placement (#if USE_FPA_INTB_GEMM around declarations/definitions, #if !defined(COMPILE_HOPPER_TMA_GEMMS) around the build-support throw), and message byte-identity.
• Relying on ONNX Runtime CI (windows_cuda.yml / linux_cuda_ci.yml) for full build/link/run validation of the new test in its actual target environment.

Do not merge — opening for review/discussion.

Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com

[GopalakrishnanN]

Thanks for the review, @tianleiwu! I've marked this ready for review.

One thing worth flagging before it lands: so far only the lint/format/CLA checks have run on this PR — the CUDA pipelines (windows_cuda.yml / linux_cuda_ci.yml) that actually build, link, and run the new MatMulNBitsSm90ValidationTest haven't triggered here yet. Since those legs build non-plugin with onnxruntime_ENABLE_CUDA_EP_INTERNAL_TESTS=ON and COMPILE_HOPPER_TMA_GEMMS undefined (sm86), they'll execute the previously-uncovered "is not supported by this ONNX Runtime build" branch — exactly the coverage this PR adds, on the same MSVC configuration #29776 fixes.

Could you kick off those CUDA pipelines so the test is validated in its real target environment? Once they're green this should be ready to fold into #29776. Happy to adjust if you'd prefer the coverage placed differently.