Skip to content

Update tests with zero3 for RLOO and GRPO as xfail only with transformers >= v5 - #5420

Merged
albertvillanova merged 2 commits into
huggingface:mainfrom
albertvillanova:fu-5404
Apr 3, 2026
Merged

Update tests with zero3 for RLOO and GRPO as xfail only with transformers >= v5#5420
albertvillanova merged 2 commits into
huggingface:mainfrom
albertvillanova:fu-5404

Conversation

@albertvillanova

@albertvillanova albertvillanova commented Apr 1, 2026

Copy link
Copy Markdown
Member

Update tests with zero3 for RLOO and GRPO as xfail only with transformers >= v5.

This PR updates the test suite to more accurately handle failures related to ZeRO-3 when using newer versions of the transformers library. Specifically, it refines the test expectations so that tests using ZeRO-3 are only expected to fail when transformers version is 5.0.0 or higher.

I tested locally that the error appeared in transformers v5.0.0 after the merge of:

See investigation in underlying issue:

Changes

Test suite improvements:

  • Updated the test_reward and test_rloo test cases in tests/distributed/test_distributed.py to mark ZeRO-3 tests as expected to fail only when transformers version is 5.0.0 or newer, instead of unconditionally. This makes the test results more accurate and prevents false negatives on older versions.

Note

Low Risk
Low risk: only adjusts pytest xfail conditions for distributed tests, affecting test expectations rather than runtime code paths.

Overview
Updates distributed test parametrizations for test_rloo and test_grpo so the zero3 configuration is marked xfail only when transformers >= 5.0.0 (with strict=True), instead of always expecting failure. This reduces false failures on older transformers versions while retaining the known ZeRO-3 regression coverage.

Written by Cursor Bugbot for commit 80d5dd6. This will update automatically on new commits. Configure here.

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@albertvillanova
albertvillanova merged commit 095cd91 into huggingface:main Apr 3, 2026
23 of 25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants