Transformers version bump

[KaelanDt]

Transformers version bump for more recent model support

[KaelanDt]

Update: unpinning pydantic seems for more flexibility on how deepspeed is tested as they seem to use some strange aliases to enable for deprecated features to be used, see: deepspeedai/DeepSpeed#4407. Huggingface also unpinned pydantic here

Now the test stumbles on some mismatch for gemma

 AssertionError: Tensor-likes are not close!
E       
E       Mismatched elements: 3 / 5242880 (0.0%)
E       Greatest absolute difference: 3.3855438232421875e-05 at index (0, 19, 61862) (up to 3e-05 allowed)
E       Greatest relative difference: 0.0006763985147699714 at index (0, 16, 127381) (up to 3e-05 allowed)

As well as some openai client tests:

 AssertionError: Non-streaming chat completion failed with status code 500
E           assert 500 == 200
E            +  where 500 = <Response [500]>.status_code

/__w/9/s/tests/test_serve.py:230: AssertionError

Would appreciate a look on these, perhaps @k223kim for gemma and @Borda for openai api tests

[Borda]

I think his need to be fixed:

 pydantic.errors.PydanticUserError: If you use `@root_validator` with pre=False (the default) you MUST specify `skip_on_failure=True`. Note that `@root_validator` is deprecated and should be replaced with `@model_validator`

[KaelanDt]

Yes, however this comes from huggingface code that calls deepspeed

/usr/local/lib/python3.10/dist-packages/transformers/models/falcon/modeling_falcon.py:41: in <module>
    from ...modeling_utils import PreTrainedModel
/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py:158: in <module>
    import deepspeed
/usr/local/lib/python3.10/dist-packages/deepspeed/__init__.py:16: in <module>
    from . import module_inject
/usr/local/lib/python3.10/dist-packages/deepspeed/module_inject/__init__.py:6: in <module>
    from .replace_module import replace_transformer_layer, revert_transformer_layer, ReplaceWithTensorSlicing, GroupQuantizer, generic_injection
/usr/local/lib/python3.10/dist-packages/deepspeed/module_inject/replace_module.py:792: in <module>
    from ..pipe import PipelineModule
/usr/local/lib/python3.10/dist-packages/deepspeed/pipe/__init__.py:6: in <module>
    from ..runtime.pipe import PipelineModule, LayerSpec, TiedLayerSpec
/usr/local/lib/python3.10/dist-packages/deepspeed/runtime/pipe/__init__.py:6: in <module>
    from .module import PipelineModule, LayerSpec, TiedLayerSpec
/usr/local/lib/python3.10/dist-packages/deepspeed/runtime/pipe/module.py:19: in <module>
    from ..activation_checkpointing import checkpointing
/usr/local/lib/python3.10/dist-packages/deepspeed/runtime/activation_checkpointing/checkpointing.py:25: in <module>
    from deepspeed.runtime.config import DeepSpeedConfig

My understanding is that they allow this such that old code can be run with an older of version of pydantic

[Borda]

cc: @t-vi

[Borda]

FAILED tests/test_serve.py::test_serve_with_openai_spec - AssertionError: Non-streaming chat completion failed with status code 500
assert 500 == 200
 +  where 500 = <Response [500]>.status_code

cc: @aniketmaurya

[aniketmaurya]

FAILED tests/test_serve.py::test_serve_with_openai_spec - AssertionError: Non-streaming chat completion failed with status code 500
assert 500 == 200
 +  where 500 = <Response [500]>.status_code

cc: @aniketmaurya

does this pass now?

[bhimrazy]

It seems like the issue might be related to Pydantic version. Under the gpu test, somehow it appears to be falling back to a much older version of Pydantic (1.10.17) that doesn't have the support for model_copy method, which is used in the OpenAI spec:

cc: @aniketmaurya @Borda

[aniketmaurya]

thanks for investigating this @bhimrazy! @Borda @KaelanDt maybe set a minimum version for pydantic. Not worth supporting an ancient version of pydantic.

[Borda]

we did some major updates on master regarding dev env, so let's update it