Introduce pydantic_v1 compatibility module for pydantic>=2.0.0 support

[ringohoffman]

Alternate to #213
Related: deepspeedai/DeepSpeed#4407

Inspired by LangChain's solution: https://github.com/langchain-ai/langchain/blob/64385c4eae4a4b8c27e73b09018f344f0b791f31/libs/langchain/langchain/pydantic_v1/__init__.py#L14-L17

mii can unpin pydantic, but instead of using the new pydantic v2 features, mii can use v2's alias for the v1 API in order to allow greater flexibility for mii users by allowing them to use either pydantic v1 or v2

[ringohoffman]

@loadams LMK what you think of this solution. I like it because it avoids the pain of upgrading and immediately unblocks people who want to use deepspeed but have already upgraded to pydantic v2 (me), but also avoids impacting users that are unable to upgrade from pydantic v1

[ringohoffman]

@microsoft-github-policy-service agree company="Protopia AI"

[ringohoffman]

Can these workflows be retried? I added the license header, although scripts/check-license.py doesn't work on any files for me locally, even ones that are already checked in. I'm on MacOS using bash. Not sure if that that is the problem.

[ringohoffman]

Can these workflows be retried? I added the license header, although scripts/check-license.py doesn't work on any files for me locally, even ones that are already checked in. I'm on MacOS using bash. Not sure if that that is the problem.

@loadams 🙏 not sure if you were notified of my above comment so just tagging you

[ringohoffman]

@loadams it looks like everything is passing. Can you let me know the result of your internal discussion as to whether this PR and deepspeedai/DeepSpeed#4407 will be acceptable?

Based on what we've discussed, I think the next steps would be to:

1. Merge this PR
2. Rerun workflows for Introduce pydantic_v1 compatibility module for pydantic>=2.0.0 support DeepSpeed#4407 and merge
3. Revert .github/workflows/nv-torch-latest-v100.yaml Install dependencies URL

[ringohoffman]

It seems like spacy is also employing a similar approach.

[loadams]

@loadams it looks like everything is passing. Can you let me know the result of your internal discussion as to whether this PR and microsoft/DeepSpeed#4407 will be acceptable?

Based on what we've discussed, I think the next steps would be to:

1. Merge this PR
2. Rerun workflows for Introduce pydantic_v1 compatibility module for pydantic>=2.0.0 support DeepSpeed#4407 and merge
3. Revert .github/workflows/nv-torch-latest-v100.yaml Install dependencies URL

@ringohoffman - this seems reasonable, we will want feedback from @mrwyattii as well. but your next steps are correct.

[ringohoffman]

@mrwyattii do you have a timeline for reviewing this? 🙏 This is blocking my team from using DeepSpeed due to our dependency on pydantic>=2.0.0.

[mrwyattii]

@ringohoffman thank you for the contribution. I'm sorry that this is a blocker for your team. I will prioritize getting this reviewed (along with the accompanying DeepSpeed PR you shared).