[shardformer]support pipeline parallelism for mistral.#5642

Merged

ver217 merged 13 commits into

hpcaitech:feature/update-transformersfrom

flybird11111:feature/update-transformers

Apr 24, 2024

Commits on Apr 24, 2024

[shardformer] fix attn replacement (#5636)

ver217
authored and
flybird11111
committed
[shardformer] update flashattention replacement (#5637)

flybird11111
and
pre-commit-ci[bot]
committed
[Feature] Support LLaMA-3 CPT and ST (#5619)

authored and
flybird11111
committed
[exampe] update llama example (#5626)

ver217
authored and
flybird11111
committed
[example] llama3 (#5631)

binmakeswell
authored and
flybird11111
committed
[test] fix llama test (#5638)

ver217
authored and
flybird11111
committed
[gemini] fix buffer cast (#5639)

ver217
authored and
flybird11111
committed
support pp for mistral
flybird11111
committed
Merge branch 'feature/update-transformers' into feature/update-transformers
flybird11111
authored
fix
flybird11111
committed
Merge branch 'feature/update-transformers' of github.com:flybird11111/ColossalAI into feature/update-transformers
flybird11111
committed
fix
flybird11111
committed
fix
flybird11111
committed