Fix model copying for QDQ stripping - #784
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR fixes model copying functionality required for QDQ (Quantize/Dequantize) stripping when processing GPU models with bfloat16->float16 conversions and 16-bit integer quantization. The changes update the model copying mechanism to handle OrtValue initializers and refine the conditions for QDQ optimization.
- Updated model copying logic to use OrtValue-based initializer handling instead of TensorProto copying
- Added support for INT16/UINT16 data types in type checking and moved the logic from experimental GPU-only to general support
- Enhanced QDQ graph detection to specifically identify graphs with 16-bit quantization for targeted GPU optimization
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| qdq_scales_fix.cpp | Simplified model copying to use OrtValue initializers and removed redundant tensor proto copying logic |
| data_ops.cc | Moved INT16/UINT16 type support from experimental GPU-only section to general type support |
| backend_manager.cc | Added detection for QDQ graphs with 16-bit quantization and refined GPU optimization conditions |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
MayureshV1
left a comment
There was a problem hiding this comment.
LGTM !
Implication of this PR is limited to GPU and only to INT16, UINT16 and BF16.
Waiting on confirmation from validation of GPU customer models before we merge.
* Reintroduce #768 with a small fix * Fix model copying with help from microsoft#25761 * Remove unused debug variables
Description
This PR fixes the model copying portion required by QDQ stripping for GPU and bfloat16->float16 conversions. The copying was broken by the converting the initializers to OrtValues in the upstream repo.
To make the changes in a single place, a PR #768 with a small fix was embedded in this PR.
https://jira.devtools.intel.com/browse/CVS-171536