Skip to content

Support FP8 NVLS multimem reduction#826

Open
Binyang2014 wants to merge 5 commits into
mainfrom
binyli/nvls-fp8-subpr
Open

Support FP8 NVLS multimem reduction#826
Binyang2014 wants to merge 5 commits into
mainfrom
binyli/nvls-fp8-subpr

Conversation

@Binyang2014

Copy link
Copy Markdown
Contributor

Summary

  • add FP16 accumulation support for FP8 NVLS multimem load-reduce
  • probe and gate FP8 NVLS algorithms on device support
  • keep MNNVL algorithm sizing and buffer-pool changes out of this sub-PR

Validation

  • ./tools/lint.sh
  • cmake --build build --target mscclpp_collectives mscclpp_nccl -j$(nproc)

Binyang2014 and others added 3 commits June 30, 2026 18:37
multimem.ld_reduce on FP8 inputs can use the .acc::f16 instruction variant for FP16 accumulation. Thread AccumT through the NVLS zero-copy allreduce path so callers can select that behavior via the existing accum dtype dispatch.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Probe whether the current CUDA target supports FP8 multimem reduction and use that result when selecting or launching NVLS FP8 algorithms. Keep the existing algorithm sizing and buffer behavior unchanged for this sub-PR.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Apply formatter output and mark the unused extras parameter in the NVLS block-pipeline guard path.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends NVLS-based allreduce to better support FP8 by adding an FP16 accumulation path for multimem load+reduce, and by probing/gating FP8 NVLS algorithm selection based on device/runtime support (plumbed through the NCCL algorithm selector).

Changes:

  • Add an FP8 NVLS support probe (isFp8NvlsSupported) and propagate an fp8NvlsSupported flag through the NCCL algorithm selector config.
  • Add an optional FP16 accumulator variant for FP8 multimem load+reduce (via .acc::f16 instruction variants).
  • Thread FP8 NVLS support gating into NVLS allreduce algorithm implementations.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/ext/nccl/nccl.cc Computes FP8 NVLS support and passes it into the selector config.
src/ext/nccl/CMakeLists.txt Adds collectives include path so NCCL extension can use collective utilities.
src/ext/nccl/algorithm_selector.hpp Extends selector config with fp8NvlsSupported.
src/ext/nccl/algorithm_selector.cc Gates NVLS selection for FP8 based on native FP8 and probed support.
src/ext/collectives/include/collective_utils.hpp Declares FP8 dtype helpers and FP8 NVLS support probe API.
src/ext/collectives/include/allreduce/common.hpp Extends multimem reduce helper to accept an accumulation type.
src/ext/collectives/include/allreduce/allreduce_nvls_zero_copy.hpp Adds cached fp8NvlsSupported_ state to the algorithm class.
src/ext/collectives/include/allreduce/allreduce_nvls_warp_pipeline.hpp Adds cached fp8NvlsSupported_ state to the algorithm class.
src/ext/collectives/include/allreduce/allreduce_nvls_block_pipeline.hpp Adds cached fp8NvlsSupported_ state to the algorithm class.
src/ext/collectives/collective_utils.cu Implements FP8 dtype checks and runtime FP8 NVLS support probing.
src/ext/collectives/allreduce/allreduce_nvls_zero_copy.cu Adds AccumT plumbing for multimem load+reduce and runtime FP8 support gating.
src/ext/collectives/allreduce/allreduce_nvls_warp_pipeline.cu Adds runtime FP8 NVLS support gating for this algorithm variant.
src/ext/collectives/allreduce/allreduce_nvls_block_pipeline.cu Adds runtime FP8 NVLS support gating for this algorithm variant.
include/mscclpp/switch_channel_device.hpp Adds FP16-accumulator multimem load+reduce variants for FP8 vector types.

Comment thread src/ext/collectives/allreduce/allreduce_nvls_zero_copy.cu Outdated
Comment thread src/ext/collectives/allreduce/allreduce_nvls_warp_pipeline.cu
Comment thread src/ext/collectives/allreduce/allreduce_nvls_block_pipeline.cu
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants