Skip to content

Cleaning up tensor/pipe parallel accounting. - #1252

Merged
ShadenSmith merged 4 commits into
deepspeedai:big-sciencefrom
ShadenSmith:shaden/pipe_replicated
Jul 26, 2021
Merged

Cleaning up tensor/pipe parallel accounting.#1252
ShadenSmith merged 4 commits into
deepspeedai:big-sciencefrom
ShadenSmith:shaden/pipe_replicated

Adds send/recv fallback to bcast when torch version <= 1.8

f66caaf
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs