Skip to content

Fix DeepSpeedInferenceConfig crash on bool moe backward-compat value#8145

Merged
tohtana merged 1 commit into
deepspeedai:masterfrom
winklemad:fix-inference-moe-bool
Jul 17, 2026
Merged

Fix DeepSpeedInferenceConfig crash on bool moe backward-compat value#8145
tohtana merged 1 commit into
deepspeedai:masterfrom
winklemad:fix-inference-moe-bool

Conversation

@winklemad

Copy link
Copy Markdown
Contributor

DeepSpeedInferenceConfig.moe is typed Union[bool, DeepSpeedMoEConfig], and the moe_backward_compat validator explicitly handles the bool form (the old init_inference(..., moe=<bool>) interface). But it builds DeepSpeedMoEConfig(moe=field_value) — and DeepSpeedMoEConfig has no moe field (it has enabled). Since DeepSpeedConfigModel sets extra="forbid", every bool moe raises a ValidationError instead of building the config:

>>> from deepspeed.inference.config import DeepSpeedInferenceConfig
>>> DeepSpeedInferenceConfig(moe=True)
pydantic_core._pydantic_core.ValidationError: 1 validation error for DeepSpeedMoEConfig
moe
  Extra inputs are not permitted [type=extra_forbidden, ...]

Both moe=True and moe=False crash, so the documented backward-compat path is unusable. The fix passes enabled=field_value (the actual field) so the bool is turned into a DeepSpeedMoEConfig as intended.

The line dates back to #2516 and survived the pydantic-v2 migration (#5167) unchanged.

Added a regression test (test_moe_backward_compat_bool) asserting the bool form builds a DeepSpeedMoEConfig with the matching enabled. yapf and flake8 clean.

Signed-off-by: winklemad <winklemad@outlook.com>

@tohtana tohtana left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great catch, thank you @winklemad!

@tohtana
tohtana enabled auto-merge July 16, 2026 21:52
@tohtana
tohtana added this pull request to the merge queue Jul 16, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 16, 2026
@tohtana
tohtana added this pull request to the merge queue Jul 16, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 16, 2026
@tohtana
tohtana added this pull request to the merge queue Jul 16, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 16, 2026
@tohtana
tohtana added this pull request to the merge queue Jul 17, 2026
Merged via the queue into deepspeedai:master with commit 17e19b1 Jul 17, 2026
13 of 15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants