cleanup: prefer standard ops over com.github.onnxruntime.genai; relocate DSV4 design doc - #429
Merged
Merged
Conversation
…ate DSV4 design doc
Remove the com.github.onnxruntime custom op namespace per Justin's directive
("no ops in com.github.onnxruntime unless there is no other way to express it").
Investigation: the 10 GGUF IQ/MXFP4 formats routed to BlockQuantizedMatMul
(mxfp4, iq4_nl, iq4_xs, iq3_s, iq3_xxs, iq2_xxs, iq2_xs, iq2_s, iq1_s, iq1_m)
genuinely have no standard-op expression the onnx-genai (nxrt) runtime can
execute: MXFP4 is E2M1 float4 (the runtime DequantizeLinear kernel only handles
Int8/Uint8/Int32, not FLOAT4E2M1) and the IQ families use non-linear codebooks
/ super-block layouts, neither representable by affine MatMulNBits. Integer GGUF
formats already use com.microsoft.MatMulNBits and were never on the custom op.
So the custom op is retained, but no longer in the forbidden namespace: the
runtime renamed its domain com.github.onnxruntime.genai -> pkg.nxrt (onnx-genai
commit de99b73e) and now registers the kernel only under pkg.nxrt. Mobius still
emitted the old name, producing graphs the current runtime cannot run. This
switches the emitted domain to pkg.nxrt, both satisfying the directive and
fixing the domain mismatch. Attributes/inputs are unchanged and already
runtime-compatible.
Also relocate the stray root-level DSV4_FLASH_EXPORT.md into docs/design/
per repo convention and add it to the design index.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Performance Comparison
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR performs two cleanup tasks in mobius: (1) it removes usage of the com.github.onnxruntime.* op namespace by updating the BlockQuantizedMatMul custom-op domain to the runtime-registered pkg.nxrt, and (2) it relocates the DeepSeek-V4-Flash export design document into the docs design area and wires it into the design index.
Changes:
- Update the emitted custom-op domain for
BlockQuantizedMatMulfromcom.github.onnxruntime.genaitopkg.nxrt, including an explanatory comment and updated opset import handling. - Update GGUF and component tests to assert the new
pkg.nxrtdomain/opset import. - Add the relocated DeepSeek-V4-Flash export design doc under
docs/design/and include it indocs/design/index.md.
Reviewed changes
Copilot reviewed 5 out of 6 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| src/mobius/integrations/gguf/_repacker.py | Updates docstring to reference pkg.nxrt.BlockQuantizedMatMul. |
| src/mobius/integrations/gguf/_builder_test.py | Updates assertions for BlockQuantizedMatMul node domain/opset import to pkg.nxrt. |
| src/mobius/components/_quantized_linear.py | Renames the nxrt custom-op domain constant and switches emission/opset imports to pkg.nxrt with rationale. |
| src/mobius/components/_quantized_linear_test.py | Updates contract test to assert pkg.nxrt domain/opset import. |
| docs/design/index.md | Adds deepseek-v4-flash-export to the design toctree. |
| docs/design/deepseek-v4-flash-export.md | Adds the relocated DeepSeek-V4-Flash export design document. |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
🏗️ Architecture Diff
No architecture changes detected. ✅ Legend: ⚪ No change · 🔵 Minor (attrs/inits) · 🟡 Moderate (nodes added/removed) · 🔴 Major (interface changed) |
…nces Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two cleanup tasks on already-merged work.
Task 1 — Remove the
com.github.onnxruntimeop namespaceJustin's directive: "I don't want ops in the
com.github.onnxruntimenamespace unless there is NO other way to express it."Which formats route to
BlockQuantizedMatMul? 10 GGUF IQ/MXFP4 formats viaBlockQuantizedLinear/_repacker._NATIVE_BLOCK_SPECS:mxfp4, iq4_nl, iq4_xs, iq3_s, iq3_xxs, iq2_xxs, iq2_xs, iq2_s, iq1_s, iq1_m. Integer GGUF formats (Q4_0/Q4_1/Q8_0/Q4_K/Q1_0) already usecom.microsoft.MatMulNBitsand were never on the custom op.Can any switch to a standard expression the runtime can execute? No — verified against the nxrt runtime (
onnx-genai):DequantizeLinear(FLOAT4E2M1, block_size)→MatMulDequantizeLinearkernel only dequantizes Int8/Uint8/Int32 — rejects FLOAT4E2M1. AffineMatMulNBitscan't represent E2M1 either.Gather(codebook)dequant →MatMulDequantizeLinear/MatMulNBitscannot express a non-linear codebook.onnx 1.22 / onnxscript 0.7.1 in this repo can emit
FLOAT4E2M1+ blockDequantizeLinear, but the runtime cannot run it — so emitting it would produce an unrunnable graph. All 10 formats genuinely have "no other way."Key finding (also fixes a latent bug): the runtime renamed the domain
com.github.onnxruntime.genai→pkg.nxrt(onnx-genai commitde99b73e) and now registers the CPU+CUDA kernel only underpkg.nxrt(no alias —normalize_domainonly mapsai.onnx⇄""). Mobius still emitted the old name, so current mobius output is unrunnable by the current runtime.Change: rename the emitted domain
com.github.onnxruntime.genai→pkg.nxrt(_ONNX_GENAI_DOMAIN→_NXRT_DOMAIN) in_quantized_linear.py, repacker docstring, and tests. Added a code comment documenting why the custom op is retained and why it usespkg.nxrtrather thancom.github.onnxruntime. Op attributes/inputs are unchanged and already runtime-compatible (K,N,format,block_layout_version=1; activation fp32, packed uint8 weight, optional fp32 bias).This both satisfies the directive (no
com.github.onnxruntimenamespace) and realigns mobius with the runtime's registered domain.Would a runtime change fully eliminate the custom domain? (Justin decides) To drop even
pkg.nxrt, the runtime would need standard-op kernels for (a) blockDequantizeLinearwithFLOAT4E2M1input (MXFP4) and (b) a codebookGather-dequant path (IQ formats) + MatMul fusion. Not done here — this is a coordinated runtime change out of scope for a mobius-only cleanup.Task 2 — Relocate stray root design doc
Moved
DSV4_FLASH_EXPORT.md→docs/design/deepseek-v4-flash-export.md(repo convention) and added it todocs/design/index.md.Validation
lintrunner --all-files→ cleanpytest -n auto -m 'not integration and not arch_validation' --cov=src→ 4813 passed, 338 skipped, 62 xfailed_quantized_linear_test.py,integrations/gguf/_builder_test.py,_repacker_test.pyall pass and assert the newpkg.nxrtdomain.