[WebNN] Fallback unsupported integer input and output of a WebNN graph to int32 - #24425
Conversation
|
/azp run Windows ARM64 QNN CI Pipeline,Windows x64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline,ONNX Runtime Web CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,Linux OpenVINO CI Pipeline |
|
/azp run Linux QNN CI Pipeline,onnxruntime-binary-size-checks-ci-pipeline,Big Models,Linux Android Emulator QNN CI Pipeline,Android CI Pipeline,iOS CI Pipeline,ONNX Runtime React Native CI Pipeline,Linux DNNL CI Pipeline,Linux MIGraphX CI Pipeline,Linux ROCm CI Pipeline |
|
Azure Pipelines successfully started running 7 pipeline(s). |
|
Azure Pipelines will not run the associated pipelines, because the pull request was updated after the run command was issued. Review the pull request again and issue a new run command. |
fdwr
left a comment
There was a problem hiding this comment.
Thank you for adding this. I hope it gets pushed down a level into the Chromium CoreML backend so callers didn't need to repeat this manual tensor conversion and cast-node insertion, but this demonstrates that it's possible.
e5858e4 to
66ba571
Compare
Indeed, according to Reilly's comment, it can't be done in Chromium. :( |
|
Merge conflicts 💥. |
…h to int32 Some WebNN backends support limited data types for the input and output of a WebNN graph. However, they can support more data types for intermediate nodes. To address this limitation, we implement a data type fallback mechanism. (Note: Currently, we only support fallback to int32 for certain integer data types.) If a data type is not supported for a graph's input or output but is supported for intermediate nodes, we will: 1. Save the input MLTensor as 'int32' data type, 2. Convert the input data from ORT to int32, 3. Insert a cast operation to WebNN graph to convert the input back to its original data type, 4. Insert a cast operation to WebNN graph to convert the output back to 'int32', 5. Convert the output data from int32 to its original data type.
66ba571 to
5962428
Compare
@fdwr, fixed. Pls. help restart the CI. |
https://github.com/microsoft/onnxruntime/actions/runs/14505301882/job/40694637717?pr=24425#step:3:333 |
Oops, fixed. |
|
/azp run ONNX Runtime Web CI Pipeline,Windows GPU CI Pipeline,Linux Android Emulator QNN CI Pipeline,Windows GPU WebGPU CI Pipeline,Windows OpenVINO CI Pipeline |
|
/azp run Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,Linux OpenVINO CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline,Windows ARM64 QNN CI Pipeline,Windows CPU CI Pipeline |
|
/azp run Windows GPU CUDA CI Pipeline,Windows GPU DML CI Pipeline,Windows GPU Doc Gen CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI |
|
/azp run Windows GPU TensorRT CI Pipeline,onnxruntime-binary-size-checks-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed,Windows x64 QNN CI Pipeline,Big Models |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
Azure Pipelines successfully started running 2 pipeline(s). |
|
Azure Pipelines successfully started running 3 pipeline(s). |
1 similar comment
|
Azure Pipelines successfully started running 3 pipeline(s). |
|
@fdwr, looks like this failure task is irrelevant to my change, do I need to rebase it to main? |
Restarting failed job. If it fails again, try remerging (or rebasing). Alas, required test. |
…h to int32 (#24425) Some WebNN backends support limited data types for the input and output of a WebNN graph. However, they can support more data types for intermediate nodes. To address this limitation, we implement a data type fallback mechanism. (Note: Currently, we only support fallback to int32 for certain integer data types.) If a data type is not supported for a graph's input or output but is supported for intermediate nodes, we will: 1. Save the input MLTensor as 'int32' data type, 2. Convert the input data from ORT to int32, 3. Insert a cast operation to WebNN graph to convert the input back to its original data type, 4. Insert a cast operation to WebNN graph to convert the output back to 'int32', 5. Convert the output data from int32 to its original data type.
…h to int32 (microsoft#24425) Some WebNN backends support limited data types for the input and output of a WebNN graph. However, they can support more data types for intermediate nodes. To address this limitation, we implement a data type fallback mechanism. (Note: Currently, we only support fallback to int32 for certain integer data types.) If a data type is not supported for a graph's input or output but is supported for intermediate nodes, we will: 1. Save the input MLTensor as 'int32' data type, 2. Convert the input data from ORT to int32, 3. Insert a cast operation to WebNN graph to convert the input back to its original data type, 4. Insert a cast operation to WebNN graph to convert the output back to 'int32', 5. Convert the output data from int32 to its original data type. Signed-off-by: bfilipek <bartlomiej.filipek@intel.com>
Some WebNN backends support limited data types for the input and output of a WebNN graph. However, they can support more data types for intermediate nodes. To address this limitation, we implement a data type fallback mechanism. (Note: Currently, we only support fallback to int32 for certain integer data types.)
If a data type is not supported for a graph's input or output but is supported for intermediate nodes, we will: