diff --git a/docs/_tutorials/automatic-tensor-parallelism.md b/docs/_tutorials/automatic-tensor-parallelism.md index e480fbf42ac8..89bddb7b9686 100644 --- a/docs/_tutorials/automatic-tensor-parallelism.md +++ b/docs/_tutorials/automatic-tensor-parallelism.md @@ -88,18 +88,18 @@ deepspeed --num_gpus DeepSpeedExamples/inference/huggingface/text-gen The following results were collected using V100 SXM2 32GB GPUs. ### Max New Tokens = 50 -| Test | Memory Allocated per GPU | Max Batch Size | Max Throughput per GPU | -|----------|--------------------------|----------------|------------------------| -| No TP | 23.94 GB | 64 | 18.84 TFlops | -| 2 GPU TP | 12.23 GB | 320 | 27.17 TFlops | -| 4 GPU TP | 6.36 GB | 664 | 27.63 TFlops | +| Test | Memory Allocated per GPU | Max Batch Size | Max Throughput per GPU | +| ---------- | -------------------------- | ---------------- | ------------------------ | +| No TP | 23.94 GB | 64 | 18.84 TFlops | +| 2 GPU TP | 12.23 GB | 320 | 27.17 TFlops | +| 4 GPU TP | 6.36 GB | 664 | 27.63 TFlops | ### Max New Tokens = 1024 -| Test | Memory Allocated per GPU | Max Batch Size | Max Throughput per GPU | -|----------|--------------------------|----------------|------------------------| -| No TP | 23.94 GB | 2 | 1.65 TFlops | -| 2 GPU TP | 12.23 GB | 20 | 4.61 TFlops | -| 4 GPU TP | 6.36 GB | 56 | 4.90 TFlops | +| Test | Memory Allocated per GPU | Max Batch Size | Max Throughput per GPU | +| ---------- | -------------------------- | ---------------- | ------------------------ | +| No TP | 23.94 GB | 2 | 1.65 TFlops | +| 2 GPU TP | 12.23 GB | 20 | 4.61 TFlops | +| 4 GPU TP | 6.36 GB | 56 | 4.90 TFlops | # Supported Models diff --git a/docs/index.md b/docs/index.md index 996c2fdfe925..79c5ad21f058 100755 --- a/docs/index.md +++ b/docs/index.md @@ -7,6 +7,7 @@ title: "Latest News" --- DeepSpeed trained the world's most powerful language models ([MT-530B](https://www.microsoft.com/en-us/research/blog/using-deepspeed-and-megatron-to-train-megatron-turing-nlg-530b-the-worlds-largest-and-most-powerful-generative-language-model/), [BLOOM](https://huggingface.co/blog/bloom-megatron-deepspeed)); [learn how](https://www.deepspeed.ai/tutorials/large-models-w-deepspeed/). +* [2023/02] [Automatic Tensor Parallelism: Enables tensor parallelism by default without providing an injection policy](https://www.deepspeed.ai/tutorials/automatic-tensor-parallelism/) * [2022/12] [DeepSpeed Data Efficiency: A composable library that makes better use of data, increases training efficiency, and improves model quality](https://www.deepspeed.ai/2022/12/11/data-efficiency.html) * [2022/11] [Stable Diffusion Image Generation under 1 second w. DeepSpeed MII](https://github.com/microsoft/DeepSpeed-MII/tree/main/examples/benchmark/txt2img) * [2022/10] [DeepSpeed-MII: instant speedup on 24,000+ open-source DL models with up to 40x cheaper inference](https://www.deepspeed.ai/2022/10/10/mii.html)