diff --git a/README.md b/README.md
index 136f2d28734b..bfa03a6e8c9a 100755
--- a/README.md
+++ b/README.md
@@ -12,6 +12,7 @@
## Latest News
DeepSpeed trained the world's most powerful language models ([MT-530B](https://www.microsoft.com/en-us/research/blog/using-deepspeed-and-megatron-to-train-megatron-turing-nlg-530b-the-worlds-largest-and-most-powerful-generative-language-model/), [BLOOM](https://huggingface.co/blog/bloom-megatron-deepspeed)); [learn how](https://www.deepspeed.ai/tutorials/large-models-w-deepspeed/).
+* [2023/02] [Automatic Tensor Parallelism: Enables tensor parallelism by default without providing an injection policy](https://www.deepspeed.ai/tutorials/automatic-tensor-parallelism/)
* [2022/12] [DeepSpeed Data Efficiency: A composable library that makes better use of data, increases training efficiency, and improves model quality](https://www.deepspeed.ai/2022/12/11/data-efficiency.html)
* [2022/11] [Stable Diffusion Image Generation under 1 second w. DeepSpeed MII](https://github.com/microsoft/DeepSpeed-MII/tree/main/examples/benchmark/txt2img)
* [2022/10] [DeepSpeed-MII: instant speedup on 24,000+ open-source DL models with up to 40x cheaper inference](https://www.deepspeed.ai/2022/10/10/mii.html)
diff --git a/docs/_config.yml b/docs/_config.yml
index f7350e81ffca..7127b8459fe2 100644
--- a/docs/_config.yml
+++ b/docs/_config.yml
@@ -35,6 +35,7 @@ collections:
- advanced-install.md
- getting-started.md
- azure.md
+ - automatic-tensor-parallelism.md
- bert-finetuning.md
- bert-pretraining.md
- cifar-10.md
diff --git a/docs/_data/navigation.yml b/docs/_data/navigation.yml
index 4449bb91ed12..6f7c443c7958 100755
--- a/docs/_data/navigation.yml
+++ b/docs/_data/navigation.yml
@@ -53,6 +53,8 @@ lnav:
url: /getting-started/
- title: 'Getting started on Azure'
url: /tutorials/azure/
+ - title: 'Automatic Tensor Parallelism'
+ url: /tutorials/automatic-tensor-parallelism/
- title: 'Autotuning'
url: /tutorials/autotuning/
- title: 'BingBertSQuAD Fine-tuning'
diff --git a/docs/_tutorials/automatic-tensor-parallelism.md b/docs/_tutorials/automatic-tensor-parallelism.md
index c78170131263..e480fbf42ac8 100644
--- a/docs/_tutorials/automatic-tensor-parallelism.md
+++ b/docs/_tutorials/automatic-tensor-parallelism.md
@@ -88,18 +88,18 @@ deepspeed --num_gpus DeepSpeedExamples/inference/huggingface/text-gen
The following results were collected using V100 SXM2 32GB GPUs.
### Max New Tokens = 50
-| | Memory Allocated per GPU | Max Batch Size | Max Throughput per GPU |
-|---|---|---|---|
-| No TP | 23.94 GB | 64 | 18.84 TFlops |
-| 2 GPU TP | 12.23 GB | 320 | 27.17 TFlops |
-| 4 GPU TP | 6.36 GB | 664 | 27.63 TFlops |
+| Test | Memory Allocated per GPU | Max Batch Size | Max Throughput per GPU |
+|----------|--------------------------|----------------|------------------------|
+| No TP | 23.94 GB | 64 | 18.84 TFlops |
+| 2 GPU TP | 12.23 GB | 320 | 27.17 TFlops |
+| 4 GPU TP | 6.36 GB | 664 | 27.63 TFlops |
### Max New Tokens = 1024
-| | Memory Allocated per GPU | Max Batch Size | Max Throughput per GPU |
-|---|---|---|---|
-| No TP | 23.94 GB | 2 | 1.65 TFlops |
-| 2 GPU TP | 12.23 GB | 20 | 4.61 TFlops |
-| 4 GPU TP | 6.36 GB | 56 | 4.90 TFlops |
+| Test | Memory Allocated per GPU | Max Batch Size | Max Throughput per GPU |
+|----------|--------------------------|----------------|------------------------|
+| No TP | 23.94 GB | 2 | 1.65 TFlops |
+| 2 GPU TP | 12.23 GB | 20 | 4.61 TFlops |
+| 4 GPU TP | 6.36 GB | 56 | 4.90 TFlops |
# Supported Models
@@ -139,7 +139,7 @@ The following model families have been successfully tested with automatic tensor
# Unsupported Models
-The following models are not currently supported:
+The following models are not currently supported with automatic tensor parallelism. They may still be compatible with other DeepSpeed features (e.g., kernel injection for Bloom):
- bloom
- codegen