Skip to content

Support Optional shapes - #26164

Merged
yuslepukhin merged 4 commits into
mainfrom
yuslepukhin/optional_shape
Sep 26, 2025
Merged

Support Optional shapes#26164
yuslepukhin merged 4 commits into
mainfrom
yuslepukhin/optional_shape

Conversation

@yuslepukhin

Copy link
Copy Markdown
Contributor

This pull request introduces a new mechanism for handling tensor shape information in ONNX Runtime, improving the representation and querying of shape and symbolic dimension data. The changes add support for tensors that may not have shape information, update APIs to reflect this, and refactor internal shape handling logic for improved consistency and extensibility.

API and Core Logic Improvements

  • Added a new API method TensorTypeAndShape_HasShape to OrtApi and corresponding C++ wrapper HasShape() in TensorTypeAndShapeInfoImpl, allowing users to check if a tensor has shape information. [1] [2] [3]
  • Updated all shape-related API implementations (e.g., GetDimensionsCount, GetDimensions, GetSymbolicDimensions, GetTensorShapeElementCount) to properly handle the case when a tensor does not have shape information, returning zero or empty values as appropriate.

Refactoring and Consistency

  • Refactored OrtTensorTypeAndShapeInfo to store shape information in an optional structure, and updated all related methods to use pointers instead of direct objects, improving consistency and future extensibility. [1] [2]
  • Updated methods for constructing shape/type info to accept pointers and handle cases where shape may be absent, including changes in onnxruntime_typeinfo.cc and related API calls. [1] [2] [3] [4] [5] [6] [7]

Graph and Proto Serialization

  • Modified graph-to-proto conversion logic to propagate and respect the presence or absence of shape information, ensuring that ONNX graph serialization accurately reflects tensor shape status. [1] [2] [3] [4] [5]

These changes collectively improve the robustness and flexibility of ONNX Runtime's handling of tensor shape information, and lay the groundwork for supporting more dynamic and symbolic tensor shapes in future development.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request introduces support for optional tensor shapes in ONNX Runtime, enabling the framework to handle tensors that may not have shape information. The changes refactor the tensor type and shape data structures to use optional shape storage and update all related APIs to handle cases where shape information is absent.

  • Added a new API method TensorTypeAndShape_HasShape to check if a tensor has shape information
  • Refactored OrtTensorTypeAndShapeInfo to store shape data in an optional structure instead of always requiring it
  • Updated all shape-related methods to handle absent shape information by returning zero/empty values

Reviewed Changes

Copilot reviewed 16 out of 18 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
onnxruntime/test/testdata/abs_0d_lostdim.py Test data generator creating ONNX model with tensors lacking shape information (shape=None)
onnxruntime/test/testdata/abs_0d_input.py Test data generator creating ONNX model with tensors having empty shape (shape=[])
onnxruntime/test/shared_lib/test_inference.cc Added test case OptionalShape to verify HasShape() functionality with both models
onnxruntime/test/framework/type_info_test.cc Updated tests to use new getter methods instead of direct field access
onnxruntime/test/ep_graph/test_ep_graph.cc Refactored test to use C++ wrapper APIs and check HasShape() before accessing shape data
onnxruntime/core/session/ort_apis.h Added declaration for new TensorTypeAndShape_HasShape API function
onnxruntime/core/session/onnxruntime_c_api.cc Added new API function to the OrtApi structure
onnxruntime/core/session/custom_ops.cc Updated to use pointer-based API calls and check HasShape() before accessing shape data
onnxruntime/core/graph/graph.cc Modified graph-to-proto conversion to handle optional shapes in tensor value info
onnxruntime/core/framework/tensor_type_and_shape.h Major refactor introducing optional ShapeInfo structure and getter/setter methods
onnxruntime/core/framework/tensor_type_and_shape.cc Implemented all new methods and updated existing ones to handle optional shapes
onnxruntime/core/framework/onnxruntime_typeinfo.cc Updated factory methods to use pointer-based shape parameters
include/onnxruntime/core/session/onnxruntime_cxx_inline.h Added implementation of HasShape() method
include/onnxruntime/core/session/onnxruntime_cxx_api.h Added HasShape() method declaration to C++ wrapper
include/onnxruntime/core/session/onnxruntime_c_api.h Added documentation and declaration for new HasShape API
include/onnxruntime/core/providers/utils/ort_graph_to_proto.h Updated function signatures to include has_shape parameter and handle optional shapes

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread onnxruntime/test/testdata/abs_0d_input.py Outdated
Comment thread onnxruntime/core/framework/tensor_type_and_shape.cc
Comment thread onnxruntime/core/framework/tensor_type_and_shape.cc
Comment thread include/onnxruntime/core/providers/utils/ort_graph_to_proto.h Outdated
Comment thread onnxruntime/test/testdata/abs_0d_input.py Fixed
Comment thread onnxruntime/test/testdata/abs_0d_input.py Fixed
Comment thread onnxruntime/test/testdata/abs_0d_input.py Fixed
Comment thread onnxruntime/test/testdata/abs_0d_input.py Fixed
Comment thread onnxruntime/test/testdata/abs_0d_input.py Fixed
Comment thread onnxruntime/test/testdata/abs_0d_lostdim.py Fixed
Comment thread onnxruntime/test/testdata/abs_0d_lostdim.py Fixed
Comment thread onnxruntime/test/testdata/abs_0d_lostdim.py Fixed
Comment thread onnxruntime/test/testdata/abs_0d_lostdim.py Fixed
Comment thread onnxruntime/test/testdata/abs_0d_lostdim.py Fixed
Comment thread onnxruntime/test/testdata/abs_0d_input.py Fixed
Comment thread onnxruntime/test/testdata/abs_0d_input.py Fixed
Comment thread onnxruntime/test/testdata/abs_0d_lostdim.py Fixed
Comment thread onnxruntime/test/testdata/abs_0d_lostdim.py Fixed

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can commit the suggested changes from lintrunner.

Comment thread onnxruntime/test/shared_lib/test_inference.cc Outdated

@snnn snnn left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your work on this PR! I will approve it once the linting issues are resolved.

Comment thread onnxruntime/core/framework/tensor_type_and_shape.cc
Comment thread onnxruntime/core/framework/tensor_type_and_shape.cc
Comment thread onnxruntime/core/framework/tensor_type_and_shape.cc
Comment thread onnxruntime/core/framework/tensor_type_and_shape.cc Outdated

import sys

import onnx

Check notice

Code scanning / CodeQL

Module is imported with 'import' and 'import from' Note test

Module 'onnx' is imported with both 'import' and 'import from'.
Module 'onnxruntime.test.onnx' is imported with both 'import' and 'import from'.

Copilot Autofix

AI 10 months ago

The best fix is to remove the from onnx import TensorProto, helper import on line 10 and instead use only import onnx. All references to TensorProto and helper should then be qualified with onnx. (i.e., onnx.TensorProto, onnx.helper). This guarantees a single, clear import pattern for the file and preserves existing functionality. Specifically, in onnxruntime/test/testdata/abs_0d_lostdim.py, delete the from onnx import TensorProto, helper line, and update all usages of TensorProto and helper to use onnx.TensorProto and onnx.helper respectively (lines 19, 27, 33–50, and function definitions that use these names must be updated accordingly).

Suggested changeset 1
onnxruntime/test/testdata/abs_0d_lostdim.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/onnxruntime/test/testdata/abs_0d_lostdim.py b/onnxruntime/test/testdata/abs_0d_lostdim.py
--- a/onnxruntime/test/testdata/abs_0d_lostdim.py
+++ b/onnxruntime/test/testdata/abs_0d_lostdim.py
@@ -7,7 +7,6 @@
 import sys
 
 import onnx
-from onnx import TensorProto, helper
 
 
 def order_repeated_field(repeated_proto, key_name, order):
@@ -16,7 +15,7 @@
 
 
 def make_node(op_type, inputs, outputs, name=None, doc_string=None, domain=None, **kwargs):
-    node = helper.make_node(op_type, inputs, outputs, name, doc_string, domain, **kwargs)
+    node = onnx.helper.make_node(op_type, inputs, outputs, name, doc_string, domain, **kwargs)
     if doc_string == "":
         node.doc_string = ""
     order_repeated_field(node.attribute, "name", kwargs.keys())
@@ -24,26 +23,26 @@
 
 
 def make_graph(*args, doc_string=None, **kwargs):
-    graph = helper.make_graph(*args, doc_string=doc_string, **kwargs)
+    graph = onnx.helper.make_graph(*args, doc_string=doc_string, **kwargs)
     if doc_string == "":
         graph.doc_string = ""
     return graph
 
 
-model = helper.make_model(
+model = onnx.helper.make_model(
     opset_imports=[
-        helper.make_operatorsetid("", 21),
-        helper.make_operatorsetid("com.microsoft", 1),
-        helper.make_operatorsetid("com.microsoft.nchwc", 1),
-        helper.make_operatorsetid("com.ms.internal.nhwc", 21),
+        onnx.helper.make_operatorsetid("", 21),
+        onnx.helper.make_operatorsetid("com.microsoft", 1),
+        onnx.helper.make_operatorsetid("com.microsoft.nchwc", 1),
+        onnx.helper.make_operatorsetid("com.ms.internal.nhwc", 21),
     ],
     ir_version=11,
     producer_name="ort_ep_utils::OrtGraphToProto",
     doc_string="Serialized from OrtGraph",
     graph=make_graph(
         name="OpenVINOExecutionProvider_11295571201636618024_0",
-        inputs=[helper.make_tensor_value_info("absInput_1", TensorProto.FLOAT, shape=None)],
-        outputs=[helper.make_tensor_value_info("absOutput_0", TensorProto.FLOAT, shape=None)],
+        inputs=[onnx.helper.make_tensor_value_info("absInput_1", onnx.TensorProto.FLOAT, shape=None)],
+        outputs=[onnx.helper.make_tensor_value_info("absOutput_0", onnx.TensorProto.FLOAT, shape=None)],
         doc_string="Serialized from OrtGraph",
         nodes=[make_node("Abs", inputs=["absInput_1"], outputs=["absOutput_0"], name="_0", domain="")],
     ),
EOF
@@ -7,7 +7,6 @@
import sys

import onnx
from onnx import TensorProto, helper


def order_repeated_field(repeated_proto, key_name, order):
@@ -16,7 +15,7 @@


def make_node(op_type, inputs, outputs, name=None, doc_string=None, domain=None, **kwargs):
node = helper.make_node(op_type, inputs, outputs, name, doc_string, domain, **kwargs)
node = onnx.helper.make_node(op_type, inputs, outputs, name, doc_string, domain, **kwargs)
if doc_string == "":
node.doc_string = ""
order_repeated_field(node.attribute, "name", kwargs.keys())
@@ -24,26 +23,26 @@


def make_graph(*args, doc_string=None, **kwargs):
graph = helper.make_graph(*args, doc_string=doc_string, **kwargs)
graph = onnx.helper.make_graph(*args, doc_string=doc_string, **kwargs)
if doc_string == "":
graph.doc_string = ""
return graph


model = helper.make_model(
model = onnx.helper.make_model(
opset_imports=[
helper.make_operatorsetid("", 21),
helper.make_operatorsetid("com.microsoft", 1),
helper.make_operatorsetid("com.microsoft.nchwc", 1),
helper.make_operatorsetid("com.ms.internal.nhwc", 21),
onnx.helper.make_operatorsetid("", 21),
onnx.helper.make_operatorsetid("com.microsoft", 1),
onnx.helper.make_operatorsetid("com.microsoft.nchwc", 1),
onnx.helper.make_operatorsetid("com.ms.internal.nhwc", 21),
],
ir_version=11,
producer_name="ort_ep_utils::OrtGraphToProto",
doc_string="Serialized from OrtGraph",
graph=make_graph(
name="OpenVINOExecutionProvider_11295571201636618024_0",
inputs=[helper.make_tensor_value_info("absInput_1", TensorProto.FLOAT, shape=None)],
outputs=[helper.make_tensor_value_info("absOutput_0", TensorProto.FLOAT, shape=None)],
inputs=[onnx.helper.make_tensor_value_info("absInput_1", onnx.TensorProto.FLOAT, shape=None)],
outputs=[onnx.helper.make_tensor_value_info("absOutput_0", onnx.TensorProto.FLOAT, shape=None)],
doc_string="Serialized from OrtGraph",
nodes=[make_node("Abs", inputs=["absInput_1"], outputs=["absOutput_0"], name="_0", domain="")],
),
Copilot is powered by AI and may make mistakes. Always verify output.
@yuslepukhin
yuslepukhin merged commit 149d208 into main Sep 26, 2025
92 of 93 checks passed
@yuslepukhin
yuslepukhin deleted the yuslepukhin/optional_shape branch September 26, 2025 22:53
fs-eire pushed a commit that referenced this pull request Oct 24, 2025
This pull request introduces a new mechanism for handling tensor shape
information in ONNX Runtime, improving the representation and querying
of shape and symbolic dimension data. The changes add support for
tensors that may not have shape information, update APIs to reflect
this, and refactor internal shape handling logic for improved
consistency and extensibility.

### API and Core Logic Improvements

* Added a new API method `TensorTypeAndShape_HasShape` to `OrtApi` and
corresponding C++ wrapper `HasShape()` in `TensorTypeAndShapeInfoImpl`,
allowing users to check if a tensor has shape information.
[[1]](diffhunk://#diff-5845a5c76fb64abdc8f0cffe21b37f8da1712674eb3abc4cd87190891be1bd48R6583-R6590)
[[2]](diffhunk://#diff-17f64e8b38fcdcd25e90abcabeec4b420956b15fe63868a5d0b270c376bde209R1785)
[[3]](diffhunk://#diff-cc93f5f9d8078d3d3af14c9bb4c0c59e25a99f3ec75d7772ea20111ed7eb6ddeL1985-R1990)
* Updated all shape-related API implementations (e.g.,
`GetDimensionsCount`, `GetDimensions`, `GetSymbolicDimensions`,
`GetTensorShapeElementCount`) to properly handle the case when a tensor
does not have shape information, returning zero or empty values as
appropriate.

### Refactoring and Consistency

* Refactored `OrtTensorTypeAndShapeInfo` to store shape information in
an optional structure, and updated all related methods to use pointers
instead of direct objects, improving consistency and future
extensibility.
[[1]](diffhunk://#diff-4756b499067c0b1f266a64e07fcdc7e994fef5cef7f3bd68fe413d673cac7477L223-R282)
[[2]](diffhunk://#diff-e5038e85df9dca9aed186026cb784ccd367130f37ac046ffbc75958be81d343aR6)
* Updated methods for constructing shape/type info to accept pointers
and handle cases where shape may be absent, including changes in
`onnxruntime_typeinfo.cc` and related API calls.
[[1]](diffhunk://#diff-b149047a296b19c4da5ade443da01adc5fd9f1bf3458516f5d6ebd0940e0b0fbL173-R173)
[[2]](diffhunk://#diff-b149047a296b19c4da5ade443da01adc5fd9f1bf3458516f5d6ebd0940e0b0fbL184-R184)
[[3]](diffhunk://#diff-b149047a296b19c4da5ade443da01adc5fd9f1bf3458516f5d6ebd0940e0b0fbL197-R197)
[[4]](diffhunk://#diff-b149047a296b19c4da5ade443da01adc5fd9f1bf3458516f5d6ebd0940e0b0fbL306-R307)
[[5]](diffhunk://#diff-4756b499067c0b1f266a64e07fcdc7e994fef5cef7f3bd68fe413d673cac7477L279-R320)
[[6]](diffhunk://#diff-4756b499067c0b1f266a64e07fcdc7e994fef5cef7f3bd68fe413d673cac7477L305-R339)
[[7]](diffhunk://#diff-4756b499067c0b1f266a64e07fcdc7e994fef5cef7f3bd68fe413d673cac7477L347-R381)

### Graph and Proto Serialization

* Modified graph-to-proto conversion logic to propagate and respect the
presence or absence of shape information, ensuring that ONNX graph
serialization accurately reflects tensor shape status.
[[1]](diffhunk://#diff-43bd1f5614c593153ace5847907605341284e6025ce60dd9ec22b1fa7434b275L228-R229)
[[2]](diffhunk://#diff-43bd1f5614c593153ace5847907605341284e6025ce60dd9ec22b1fa7434b275R394-R397)
[[3]](diffhunk://#diff-43bd1f5614c593153ace5847907605341284e6025ce60dd9ec22b1fa7434b275L496-R511)
[[4]](diffhunk://#diff-43bd1f5614c593153ace5847907605341284e6025ce60dd9ec22b1fa7434b275R522)
[[5]](diffhunk://#diff-43bd1f5614c593153ace5847907605341284e6025ce60dd9ec22b1fa7434b275R539-R551)

These changes collectively improve the robustness and flexibility of
ONNX Runtime's handling of tensor shape information, and lay the
groundwork for supporting more dynamic and symbolic tensor shapes in
future development.
naomiOvad pushed a commit to naomiOvad/onnxruntime that referenced this pull request Nov 2, 2025
This pull request introduces a new mechanism for handling tensor shape
information in ONNX Runtime, improving the representation and querying
of shape and symbolic dimension data. The changes add support for
tensors that may not have shape information, update APIs to reflect
this, and refactor internal shape handling logic for improved
consistency and extensibility.

### API and Core Logic Improvements

* Added a new API method `TensorTypeAndShape_HasShape` to `OrtApi` and
corresponding C++ wrapper `HasShape()` in `TensorTypeAndShapeInfoImpl`,
allowing users to check if a tensor has shape information.
[[1]](diffhunk://#diff-5845a5c76fb64abdc8f0cffe21b37f8da1712674eb3abc4cd87190891be1bd48R6583-R6590)
[[2]](diffhunk://#diff-17f64e8b38fcdcd25e90abcabeec4b420956b15fe63868a5d0b270c376bde209R1785)
[[3]](diffhunk://#diff-cc93f5f9d8078d3d3af14c9bb4c0c59e25a99f3ec75d7772ea20111ed7eb6ddeL1985-R1990)
* Updated all shape-related API implementations (e.g.,
`GetDimensionsCount`, `GetDimensions`, `GetSymbolicDimensions`,
`GetTensorShapeElementCount`) to properly handle the case when a tensor
does not have shape information, returning zero or empty values as
appropriate.

### Refactoring and Consistency

* Refactored `OrtTensorTypeAndShapeInfo` to store shape information in
an optional structure, and updated all related methods to use pointers
instead of direct objects, improving consistency and future
extensibility.
[[1]](diffhunk://#diff-4756b499067c0b1f266a64e07fcdc7e994fef5cef7f3bd68fe413d673cac7477L223-R282)
[[2]](diffhunk://#diff-e5038e85df9dca9aed186026cb784ccd367130f37ac046ffbc75958be81d343aR6)
* Updated methods for constructing shape/type info to accept pointers
and handle cases where shape may be absent, including changes in
`onnxruntime_typeinfo.cc` and related API calls.
[[1]](diffhunk://#diff-b149047a296b19c4da5ade443da01adc5fd9f1bf3458516f5d6ebd0940e0b0fbL173-R173)
[[2]](diffhunk://#diff-b149047a296b19c4da5ade443da01adc5fd9f1bf3458516f5d6ebd0940e0b0fbL184-R184)
[[3]](diffhunk://#diff-b149047a296b19c4da5ade443da01adc5fd9f1bf3458516f5d6ebd0940e0b0fbL197-R197)
[[4]](diffhunk://#diff-b149047a296b19c4da5ade443da01adc5fd9f1bf3458516f5d6ebd0940e0b0fbL306-R307)
[[5]](diffhunk://#diff-4756b499067c0b1f266a64e07fcdc7e994fef5cef7f3bd68fe413d673cac7477L279-R320)
[[6]](diffhunk://#diff-4756b499067c0b1f266a64e07fcdc7e994fef5cef7f3bd68fe413d673cac7477L305-R339)
[[7]](diffhunk://#diff-4756b499067c0b1f266a64e07fcdc7e994fef5cef7f3bd68fe413d673cac7477L347-R381)

### Graph and Proto Serialization

* Modified graph-to-proto conversion logic to propagate and respect the
presence or absence of shape information, ensuring that ONNX graph
serialization accurately reflects tensor shape status.
[[1]](diffhunk://#diff-43bd1f5614c593153ace5847907605341284e6025ce60dd9ec22b1fa7434b275L228-R229)
[[2]](diffhunk://#diff-43bd1f5614c593153ace5847907605341284e6025ce60dd9ec22b1fa7434b275R394-R397)
[[3]](diffhunk://#diff-43bd1f5614c593153ace5847907605341284e6025ce60dd9ec22b1fa7434b275L496-R511)
[[4]](diffhunk://#diff-43bd1f5614c593153ace5847907605341284e6025ce60dd9ec22b1fa7434b275R522)
[[5]](diffhunk://#diff-43bd1f5614c593153ace5847907605341284e6025ce60dd9ec22b1fa7434b275R539-R551)

These changes collectively improve the robustness and flexibility of
ONNX Runtime's handling of tensor shape information, and lay the
groundwork for supporting more dynamic and symbolic tensor shapes in
future development.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants