From b02bd11d0e67c89d19787c53ebff3685124c3c30 Mon Sep 17 00:00:00 2001 From: Stas Bekman Date: Tue, 9 Feb 2021 21:27:36 -0800 Subject: [PATCH] 1bit_adam dependencies --- docs/_tutorials/onebit-adam.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/docs/_tutorials/onebit-adam.md b/docs/_tutorials/onebit-adam.md index 8871a5dd0e28..9ba2d4ec87ab 100644 --- a/docs/_tutorials/onebit-adam.md +++ b/docs/_tutorials/onebit-adam.md @@ -29,7 +29,13 @@ cd DeepSpeedExamples/ 1-bit Adam uses advanced communication schemes that are not yet supported by PyTorch distributed and NCCL. We rely on Message Passing Interface (MPI) for these advanced communication primitives. -We package the necessary dependencies in the DeepSpeed docker images. However, if you are using a different build system, please install MPI and mpi4py on your system. We have tested CUDA-Aware MPI communication using the [MVAPICH2-GDR](http://mvapich.cse.ohio-state.edu/userguide/gdr/) library. However, any CUDA-Aware communication library including [OpenMPI](https://www.open-mpi.org/) should work fine with these examples. +We package the necessary dependencies in the DeepSpeed docker images. However, if you are using a different build system, please install MPI and mpi4py on your system. To install the prerequisites run: + +```shell +pip install deepspeed[1bit_adam] +``` + +We have tested CUDA-Aware MPI communication using the [MVAPICH2-GDR](http://mvapich.cse.ohio-state.edu/userguide/gdr/) library. However, any CUDA-Aware communication library including [OpenMPI](https://www.open-mpi.org/) should work fine with these examples. An example launch command for 1-bit Adam using the `deepspeed` launcher is as follows: