Fix deepseekv3

[ydshieh]

What does this PR do?

See comments on the PR changes

[ydshieh]

OOM on T4

[ydshieh]

And from time to time, this OOM affect subsequent tests (that will pass if this test is not OOM)

[ydshieh]

Already failing when the model is added. Don't know what machine was used when this test is written. The updated value works for both T4 and A10, but it looks like gibberish.

@bzantium If you know some details back then?

[vasqu]

There is this comment above with

# Note on `EXPECTED_TEXT_COMPLETION`'s diff: the current value matches the original test if the original test
# was changed to have a cache of 53 tokens (as opposed to 4096), on Ampere GPUs.

Not sure what they did there then tho

[ydshieh]

There are still 2 remaining issues

FAILED tests/models/deepseek_v3/test_modeling_deepseek_v3.py::DeepseekV3ModelTest::test_torch_compile_for_training - torch._dynamo.exc.Unsupported: dynamic shape operator: aten.nonzero.default; to enable, set torch._dynamo.config.capture_dynamic_output_shape_ops = True

FAILED tests/models/deepseek_v3/test_modeling_deepseek_v3.py::DeepseekV3IntegrationTest::test_compile_static_cache - RuntimeError: index_copy_(): Source/destination tensor must have same slice shapes. Destination slice shape: 2 128 64 at dimension 2 and source slice shape: 2 128 192 at dimension 0.

(failing since the day when the model is added)

cc @gante when you have the bandwidth.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[vasqu]

I don't think it makes sense to change the values just to make the tests happy here. My gripe is that the output is too gibberish - I'd like to hear from @bzantium

If there is no response, I'm ok with changing it under a TODO / extra note of what the previous value has been.

[vasqu]

There is this comment above with

# Note on `EXPECTED_TEXT_COMPLETION`'s diff: the current value matches the original test if the original test
# was changed to have a cache of 53 tokens (as opposed to 4096), on Ampere GPUs.

Not sure what they did there then tho

[bzantium]

@ydshieh @vasqu
Thanks for the question! The reason why the output is too gibberish is because the testing model bzantium/tiny-deepseek-v3 is not trained one. Since original DeepSeek-V3 model is too big to debug and test, there was no testing with the original one.

[ydshieh]

@bzantium thank you for confirming (that is what I guess) too.
(It would be nice to add a comment in the test to avoid confusion 😅 )

@vasqu I will add the comment and we are good to go?

[vasqu]

Thanks :) perfect, then let's add a comment for clarification and then it's good to go