Support Optional shapes - #26164
Conversation
There was a problem hiding this comment.
Pull Request Overview
This pull request introduces support for optional tensor shapes in ONNX Runtime, enabling the framework to handle tensors that may not have shape information. The changes refactor the tensor type and shape data structures to use optional shape storage and update all related APIs to handle cases where shape information is absent.
- Added a new API method
TensorTypeAndShape_HasShapeto check if a tensor has shape information - Refactored
OrtTensorTypeAndShapeInfoto store shape data in an optional structure instead of always requiring it - Updated all shape-related methods to handle absent shape information by returning zero/empty values
Reviewed Changes
Copilot reviewed 16 out of 18 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| onnxruntime/test/testdata/abs_0d_lostdim.py | Test data generator creating ONNX model with tensors lacking shape information (shape=None) |
| onnxruntime/test/testdata/abs_0d_input.py | Test data generator creating ONNX model with tensors having empty shape (shape=[]) |
| onnxruntime/test/shared_lib/test_inference.cc | Added test case OptionalShape to verify HasShape() functionality with both models |
| onnxruntime/test/framework/type_info_test.cc | Updated tests to use new getter methods instead of direct field access |
| onnxruntime/test/ep_graph/test_ep_graph.cc | Refactored test to use C++ wrapper APIs and check HasShape() before accessing shape data |
| onnxruntime/core/session/ort_apis.h | Added declaration for new TensorTypeAndShape_HasShape API function |
| onnxruntime/core/session/onnxruntime_c_api.cc | Added new API function to the OrtApi structure |
| onnxruntime/core/session/custom_ops.cc | Updated to use pointer-based API calls and check HasShape() before accessing shape data |
| onnxruntime/core/graph/graph.cc | Modified graph-to-proto conversion to handle optional shapes in tensor value info |
| onnxruntime/core/framework/tensor_type_and_shape.h | Major refactor introducing optional ShapeInfo structure and getter/setter methods |
| onnxruntime/core/framework/tensor_type_and_shape.cc | Implemented all new methods and updated existing ones to handle optional shapes |
| onnxruntime/core/framework/onnxruntime_typeinfo.cc | Updated factory methods to use pointer-based shape parameters |
| include/onnxruntime/core/session/onnxruntime_cxx_inline.h | Added implementation of HasShape() method |
| include/onnxruntime/core/session/onnxruntime_cxx_api.h | Added HasShape() method declaration to C++ wrapper |
| include/onnxruntime/core/session/onnxruntime_c_api.h | Added documentation and declaration for new HasShape API |
| include/onnxruntime/core/providers/utils/ort_graph_to_proto.h | Updated function signatures to include has_shape parameter and handle optional shapes |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
snnn
left a comment
There was a problem hiding this comment.
Thanks for your work on this PR! I will approve it once the linting issues are resolved.
|
|
||
| import sys | ||
|
|
||
| import onnx |
Check notice
Code scanning / CodeQL
Module is imported with 'import' and 'import from' Note test
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 10 months ago
The best fix is to remove the from onnx import TensorProto, helper import on line 10 and instead use only import onnx. All references to TensorProto and helper should then be qualified with onnx. (i.e., onnx.TensorProto, onnx.helper). This guarantees a single, clear import pattern for the file and preserves existing functionality. Specifically, in onnxruntime/test/testdata/abs_0d_lostdim.py, delete the from onnx import TensorProto, helper line, and update all usages of TensorProto and helper to use onnx.TensorProto and onnx.helper respectively (lines 19, 27, 33–50, and function definitions that use these names must be updated accordingly).
| @@ -7,7 +7,6 @@ | ||
| import sys | ||
|
|
||
| import onnx | ||
| from onnx import TensorProto, helper | ||
|
|
||
|
|
||
| def order_repeated_field(repeated_proto, key_name, order): | ||
| @@ -16,7 +15,7 @@ | ||
|
|
||
|
|
||
| def make_node(op_type, inputs, outputs, name=None, doc_string=None, domain=None, **kwargs): | ||
| node = helper.make_node(op_type, inputs, outputs, name, doc_string, domain, **kwargs) | ||
| node = onnx.helper.make_node(op_type, inputs, outputs, name, doc_string, domain, **kwargs) | ||
| if doc_string == "": | ||
| node.doc_string = "" | ||
| order_repeated_field(node.attribute, "name", kwargs.keys()) | ||
| @@ -24,26 +23,26 @@ | ||
|
|
||
|
|
||
| def make_graph(*args, doc_string=None, **kwargs): | ||
| graph = helper.make_graph(*args, doc_string=doc_string, **kwargs) | ||
| graph = onnx.helper.make_graph(*args, doc_string=doc_string, **kwargs) | ||
| if doc_string == "": | ||
| graph.doc_string = "" | ||
| return graph | ||
|
|
||
|
|
||
| model = helper.make_model( | ||
| model = onnx.helper.make_model( | ||
| opset_imports=[ | ||
| helper.make_operatorsetid("", 21), | ||
| helper.make_operatorsetid("com.microsoft", 1), | ||
| helper.make_operatorsetid("com.microsoft.nchwc", 1), | ||
| helper.make_operatorsetid("com.ms.internal.nhwc", 21), | ||
| onnx.helper.make_operatorsetid("", 21), | ||
| onnx.helper.make_operatorsetid("com.microsoft", 1), | ||
| onnx.helper.make_operatorsetid("com.microsoft.nchwc", 1), | ||
| onnx.helper.make_operatorsetid("com.ms.internal.nhwc", 21), | ||
| ], | ||
| ir_version=11, | ||
| producer_name="ort_ep_utils::OrtGraphToProto", | ||
| doc_string="Serialized from OrtGraph", | ||
| graph=make_graph( | ||
| name="OpenVINOExecutionProvider_11295571201636618024_0", | ||
| inputs=[helper.make_tensor_value_info("absInput_1", TensorProto.FLOAT, shape=None)], | ||
| outputs=[helper.make_tensor_value_info("absOutput_0", TensorProto.FLOAT, shape=None)], | ||
| inputs=[onnx.helper.make_tensor_value_info("absInput_1", onnx.TensorProto.FLOAT, shape=None)], | ||
| outputs=[onnx.helper.make_tensor_value_info("absOutput_0", onnx.TensorProto.FLOAT, shape=None)], | ||
| doc_string="Serialized from OrtGraph", | ||
| nodes=[make_node("Abs", inputs=["absInput_1"], outputs=["absOutput_0"], name="_0", domain="")], | ||
| ), |
This pull request introduces a new mechanism for handling tensor shape information in ONNX Runtime, improving the representation and querying of shape and symbolic dimension data. The changes add support for tensors that may not have shape information, update APIs to reflect this, and refactor internal shape handling logic for improved consistency and extensibility. ### API and Core Logic Improvements * Added a new API method `TensorTypeAndShape_HasShape` to `OrtApi` and corresponding C++ wrapper `HasShape()` in `TensorTypeAndShapeInfoImpl`, allowing users to check if a tensor has shape information. [[1]](diffhunk://#diff-5845a5c76fb64abdc8f0cffe21b37f8da1712674eb3abc4cd87190891be1bd48R6583-R6590) [[2]](diffhunk://#diff-17f64e8b38fcdcd25e90abcabeec4b420956b15fe63868a5d0b270c376bde209R1785) [[3]](diffhunk://#diff-cc93f5f9d8078d3d3af14c9bb4c0c59e25a99f3ec75d7772ea20111ed7eb6ddeL1985-R1990) * Updated all shape-related API implementations (e.g., `GetDimensionsCount`, `GetDimensions`, `GetSymbolicDimensions`, `GetTensorShapeElementCount`) to properly handle the case when a tensor does not have shape information, returning zero or empty values as appropriate. ### Refactoring and Consistency * Refactored `OrtTensorTypeAndShapeInfo` to store shape information in an optional structure, and updated all related methods to use pointers instead of direct objects, improving consistency and future extensibility. [[1]](diffhunk://#diff-4756b499067c0b1f266a64e07fcdc7e994fef5cef7f3bd68fe413d673cac7477L223-R282) [[2]](diffhunk://#diff-e5038e85df9dca9aed186026cb784ccd367130f37ac046ffbc75958be81d343aR6) * Updated methods for constructing shape/type info to accept pointers and handle cases where shape may be absent, including changes in `onnxruntime_typeinfo.cc` and related API calls. [[1]](diffhunk://#diff-b149047a296b19c4da5ade443da01adc5fd9f1bf3458516f5d6ebd0940e0b0fbL173-R173) [[2]](diffhunk://#diff-b149047a296b19c4da5ade443da01adc5fd9f1bf3458516f5d6ebd0940e0b0fbL184-R184) [[3]](diffhunk://#diff-b149047a296b19c4da5ade443da01adc5fd9f1bf3458516f5d6ebd0940e0b0fbL197-R197) [[4]](diffhunk://#diff-b149047a296b19c4da5ade443da01adc5fd9f1bf3458516f5d6ebd0940e0b0fbL306-R307) [[5]](diffhunk://#diff-4756b499067c0b1f266a64e07fcdc7e994fef5cef7f3bd68fe413d673cac7477L279-R320) [[6]](diffhunk://#diff-4756b499067c0b1f266a64e07fcdc7e994fef5cef7f3bd68fe413d673cac7477L305-R339) [[7]](diffhunk://#diff-4756b499067c0b1f266a64e07fcdc7e994fef5cef7f3bd68fe413d673cac7477L347-R381) ### Graph and Proto Serialization * Modified graph-to-proto conversion logic to propagate and respect the presence or absence of shape information, ensuring that ONNX graph serialization accurately reflects tensor shape status. [[1]](diffhunk://#diff-43bd1f5614c593153ace5847907605341284e6025ce60dd9ec22b1fa7434b275L228-R229) [[2]](diffhunk://#diff-43bd1f5614c593153ace5847907605341284e6025ce60dd9ec22b1fa7434b275R394-R397) [[3]](diffhunk://#diff-43bd1f5614c593153ace5847907605341284e6025ce60dd9ec22b1fa7434b275L496-R511) [[4]](diffhunk://#diff-43bd1f5614c593153ace5847907605341284e6025ce60dd9ec22b1fa7434b275R522) [[5]](diffhunk://#diff-43bd1f5614c593153ace5847907605341284e6025ce60dd9ec22b1fa7434b275R539-R551) These changes collectively improve the robustness and flexibility of ONNX Runtime's handling of tensor shape information, and lay the groundwork for supporting more dynamic and symbolic tensor shapes in future development.
This pull request introduces a new mechanism for handling tensor shape information in ONNX Runtime, improving the representation and querying of shape and symbolic dimension data. The changes add support for tensors that may not have shape information, update APIs to reflect this, and refactor internal shape handling logic for improved consistency and extensibility. ### API and Core Logic Improvements * Added a new API method `TensorTypeAndShape_HasShape` to `OrtApi` and corresponding C++ wrapper `HasShape()` in `TensorTypeAndShapeInfoImpl`, allowing users to check if a tensor has shape information. [[1]](diffhunk://#diff-5845a5c76fb64abdc8f0cffe21b37f8da1712674eb3abc4cd87190891be1bd48R6583-R6590) [[2]](diffhunk://#diff-17f64e8b38fcdcd25e90abcabeec4b420956b15fe63868a5d0b270c376bde209R1785) [[3]](diffhunk://#diff-cc93f5f9d8078d3d3af14c9bb4c0c59e25a99f3ec75d7772ea20111ed7eb6ddeL1985-R1990) * Updated all shape-related API implementations (e.g., `GetDimensionsCount`, `GetDimensions`, `GetSymbolicDimensions`, `GetTensorShapeElementCount`) to properly handle the case when a tensor does not have shape information, returning zero or empty values as appropriate. ### Refactoring and Consistency * Refactored `OrtTensorTypeAndShapeInfo` to store shape information in an optional structure, and updated all related methods to use pointers instead of direct objects, improving consistency and future extensibility. [[1]](diffhunk://#diff-4756b499067c0b1f266a64e07fcdc7e994fef5cef7f3bd68fe413d673cac7477L223-R282) [[2]](diffhunk://#diff-e5038e85df9dca9aed186026cb784ccd367130f37ac046ffbc75958be81d343aR6) * Updated methods for constructing shape/type info to accept pointers and handle cases where shape may be absent, including changes in `onnxruntime_typeinfo.cc` and related API calls. [[1]](diffhunk://#diff-b149047a296b19c4da5ade443da01adc5fd9f1bf3458516f5d6ebd0940e0b0fbL173-R173) [[2]](diffhunk://#diff-b149047a296b19c4da5ade443da01adc5fd9f1bf3458516f5d6ebd0940e0b0fbL184-R184) [[3]](diffhunk://#diff-b149047a296b19c4da5ade443da01adc5fd9f1bf3458516f5d6ebd0940e0b0fbL197-R197) [[4]](diffhunk://#diff-b149047a296b19c4da5ade443da01adc5fd9f1bf3458516f5d6ebd0940e0b0fbL306-R307) [[5]](diffhunk://#diff-4756b499067c0b1f266a64e07fcdc7e994fef5cef7f3bd68fe413d673cac7477L279-R320) [[6]](diffhunk://#diff-4756b499067c0b1f266a64e07fcdc7e994fef5cef7f3bd68fe413d673cac7477L305-R339) [[7]](diffhunk://#diff-4756b499067c0b1f266a64e07fcdc7e994fef5cef7f3bd68fe413d673cac7477L347-R381) ### Graph and Proto Serialization * Modified graph-to-proto conversion logic to propagate and respect the presence or absence of shape information, ensuring that ONNX graph serialization accurately reflects tensor shape status. [[1]](diffhunk://#diff-43bd1f5614c593153ace5847907605341284e6025ce60dd9ec22b1fa7434b275L228-R229) [[2]](diffhunk://#diff-43bd1f5614c593153ace5847907605341284e6025ce60dd9ec22b1fa7434b275R394-R397) [[3]](diffhunk://#diff-43bd1f5614c593153ace5847907605341284e6025ce60dd9ec22b1fa7434b275L496-R511) [[4]](diffhunk://#diff-43bd1f5614c593153ace5847907605341284e6025ce60dd9ec22b1fa7434b275R522) [[5]](diffhunk://#diff-43bd1f5614c593153ace5847907605341284e6025ce60dd9ec22b1fa7434b275R539-R551) These changes collectively improve the robustness and flexibility of ONNX Runtime's handling of tensor shape information, and lay the groundwork for supporting more dynamic and symbolic tensor shapes in future development.
This pull request introduces a new mechanism for handling tensor shape information in ONNX Runtime, improving the representation and querying of shape and symbolic dimension data. The changes add support for tensors that may not have shape information, update APIs to reflect this, and refactor internal shape handling logic for improved consistency and extensibility.
API and Core Logic Improvements
TensorTypeAndShape_HasShapetoOrtApiand corresponding C++ wrapperHasShape()inTensorTypeAndShapeInfoImpl, allowing users to check if a tensor has shape information. [1] [2] [3]GetDimensionsCount,GetDimensions,GetSymbolicDimensions,GetTensorShapeElementCount) to properly handle the case when a tensor does not have shape information, returning zero or empty values as appropriate.Refactoring and Consistency
OrtTensorTypeAndShapeInfoto store shape information in an optional structure, and updated all related methods to use pointers instead of direct objects, improving consistency and future extensibility. [1] [2]onnxruntime_typeinfo.ccand related API calls. [1] [2] [3] [4] [5] [6] [7]Graph and Proto Serialization
These changes collectively improve the robustness and flexibility of ONNX Runtime's handling of tensor shape information, and lay the groundwork for supporting more dynamic and symbolic tensor shapes in future development.