2897 nvte port prior fp8#5

Merged

HaiShaw merged 20 commits into

devfrom

2897-nvte-port-prior-fp8

Jan 18, 2023

Commits on Nov 4, 2022

add submodule hipify_torch
HaiShaw
committed

Commits on Nov 6, 2022

Initial hipified NVTE on ROCm [merged code from @wenchenvincent and @HaiShaw]
HaiShaw
committed

Commits on Nov 7, 2022

Commits on Nov 9, 2022

Commits on Nov 17, 2022

Replaced cublasLt calls with rocblas gemm + 6 manually crafted epilogues
wenchenvincent
committed

Commits on Nov 22, 2022

Fixed dimensions for add_bias and change the bias_gradient kernel to sum up elements in each column
wenchenvincent
committed
Fixed an issue with the case where Ti=To=fp16 and there is no bias and gelu
wenchenvincent
committed

Commits on Nov 28, 2022

Fixed an issue with Gelu epilogue
wenchenvincent
committed

Commits on Nov 29, 2022

Commits on Dec 7, 2022

Added the submodule to the project minGPT integrated with Transformer Engine modules
HaiShaw
committed

Commits on Jan 4, 2023

Enhanced code readability. Connected dimensions for different GEMM epilogues to the dimensions at the NN layer level.
wenchenvincent
committed

Commits on Jan 10, 2023

Merge pull request #4 from ROCmSoftwarePlatform/2897-nvte-unit-tests
wenchenvincent
authored