2897 nvte port prior fp8#5
Merged
Merged
Conversation
…HaiShaw] - Used hipify_torch and automated hipify in build process (later to relocate hipified files into build path) - Integrated in hip_float8 headers for compilation. Functional enablement with rocblas update is left TODO - Integrated in hip_bfloat16 for compilation. The implicit copy assignment operator in header is left TODO - cublasLt GEMM is dummy-out, wrapper around existing rocblas is left TODO - Inline ASM is replaced with amd_hip equivalent - Pytorch Extension and setup.py, etc. left TODO
Fixed build issues with Pytorch extensions
…sum up elements in each column
Fixed an issue with bias gradient not initializing. The VectorizedUnaryKernelLaunch seems problematic in bias epilogue. Replaced with my own identity kernel.
As a result `python main.py --use-te --use-amp` retains loss & accuracy as `python main.py --use-te` in same scenario on CUDA
Fixed implementation of CUBLASLT_EPILOGUE_DGELU_BGRAD. Added env var NVTE_LOG_GEMM_CONFIG to control logging of GEMM configs.
…ilogues to the dimensions at the NN layer level.
Replaced cublasLt calls with rocblas gemm + 6 manually crafted epilogues
…rnorm tests as the kernels are not available for now.
2897 nvte unit tests
wenchenvincent
added a commit
that referenced
this pull request
Jul 13, 2026
Applied review comments 1, 2, 3, 4, 6, 7, 8 from PR #667 in one pass. Comments 5 and 9 will be answered on-thread (see below). Applied ======= #7 cpp_extensions/gemm.py: gate NVTE_USE_GEMM_TRITON behind IS_HIP_EXTENSION. Our Triton kernels use gfx942/gfx950-specific MFMA instructions and autotune configs; refuse to enable on non-HIP builds. #1 + #4 + #8: deduplicate dtype utilities against triton_kernels/common.py, which is the authoritative source shared across all Triton kernel backends here. - gemm_common.py: drop our copies of _get_fp8_dtypes(), torch_to_te_dtype(), te_to_torch_dtype(). Keep is_fp8_dtype() (operates on tex.DType; common.py's is_fp8_torch_dtype operates on torch dtypes -- different signatures), reinterpret_as_fp8_tensor, getGemmOutputShape, product. - gemm_wrapper.py: import torch_dtype_to_te_dtype / te_dtype_to_torch_dtype from ..common; use those instead of the removed local names. - __init__.py: drop the redundant re-exports (torch_to_te_dtype, te_to_torch_dtype, _get_fp8_dtypes). Note in the file directing callers to triton_kernels.common. - test_gemm_kernel.py: import get_torch_e4m3_type, get_torch_e5m2_type, torch_dtype_to_te_dtype from triton_kernels.common; define a local _get_fp8_dtypes() shim over them to keep the test's parametrization tables unchanged. - test_gemm_mxfp8.py: previously used _get_e4m3_dtype() -- dropped along with the test that used it (see below). #2 + #3: add module docstrings to test_gemm.py, test_gemm_kernel.py, test_gemm_mxfp8.py making the scope of each file explicit and pointing at the sibling files. Prompted by wangye805's confusion between test_gemm.py and test_gemm_kernel.py, and by the question of whether MXFP8 is only tested in test_gemm_mxfp8.py. #6 test_numerics.py: rewrite the comment above and the reason= text for _skip_grouped_under_gemm_triton to make it explicit that the skip is a *backend mismatch* between the two sides of the comparison, not Triton non-determinism. - sequential side -> our Triton (via NVTE_USE_GEMM_TRITON=1) - grouped side -> hipBLASLt / CUTLASS / AITER-Triton grouped (controlled by the SEPARATE NVTE_USE_GROUPED_GEMM_TRITON env var) With just NVTE_USE_GEMM_TRITON=1 set, the two sides diverge in fp32 rounding because they run different kernels. Not our Triton being non-deterministic. Also removed test_mxfp8_kernel_with_simulated_data (an early kernel- bring-up smoke test that only asserted non-zero output; MXFP8 correctness is fully covered by test_gemm.py::test_triton_vs_*_mxfp8 with real MXFP8Tensor and both PyTorch/C++ references). To be answered on-thread ======================== #5 (conftest.py hook vs explicit skip lists) -- keep the hook: it fires only on three specific ValueError substrings from our own gate code, is self-retiring when the gates are relaxed, and explicit skips would require ~1300 marks. Detail in the PR reply. #9 (Float8TensorWrapper / MXFP8TensorWrapper vs the TE tensor types) -- defer; needs a side-by-side to see how much of the wrapper API is genuinely required for the Triton-kernel call surface (fields accessed, dimension reordering, missing-transpose fallbacks) vs. what could route through the TE tensor types directly. Verified on gfx950 / PyTorch 2.10 -- all three direct suites clean: test_gemm.py 212 pass / 72 skip test_gemm_kernel.py 450 pass / 606 skip test_gemm_mxfp8.py 2 pass (was 3; smoke test removed)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.