Skip to content

[webgpu] support arbitrary input_channel size for im2col - #27038

Merged
guschmue merged 5 commits into
microsoft:mainfrom
wenqinI:im2col-vec1
Apr 21, 2026
Merged

[webgpu] support arbitrary input_channel size for im2col#27038
guschmue merged 5 commits into
microsoft:mainfrom
wenqinI:im2col-vec1

Conversation

@wenqinI

@wenqinI wenqinI commented Jan 16, 2026

Copy link
Copy Markdown
Contributor

Description

This PR supports vec1/vec2 for arbitrary input_channel in im2col kernel, which could bring performance gain to more models.

Like for yolov8n_pose model, there is about ~7% gain for whole model, and ~50% for those conv2d op which input_size are not multiple of 4.

Motivation and Context

@wenqinI

wenqinI commented Jan 16, 2026

Copy link
Copy Markdown
Contributor Author

@guschmue @fs-eire @qjia7 PTAL, thanks!

@wenqinI
wenqinI marked this pull request as draft January 16, 2026 08:46
Comment thread onnxruntime/core/providers/webgpu/nn/im2col_matmul.cc Outdated
Comment thread onnxruntime/core/providers/webgpu/nn/im2col_matmul.wgsl.template
Comment thread onnxruntime/core/providers/webgpu/nn/im2col_matmul.wgsl.template
@wenqinI

wenqinI commented Mar 10, 2026

Copy link
Copy Markdown
Contributor Author

@daijh, PTAL, thanks!

@daijh

daijh commented Mar 10, 2026

Copy link
Copy Markdown
Contributor

Many comments in shader does not reflect the vec_size changing. Please take a look.

@wenqinI wenqinI left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@daijh , this PR is updated, PTAL, thanks!

Comment thread onnxruntime/core/providers/webgpu/nn/im2col_matmul.wgsl.template
Comment thread onnxruntime/core/providers/webgpu/nn/im2col_matmul.wgsl.template
@daijh

daijh commented Mar 13, 2026

Copy link
Copy Markdown
Contributor

LGTM, please invite more reviewers to check.

@wenqinI
wenqinI marked this pull request as ready for review March 16, 2026 06:38
@wenqinI

wenqinI commented Mar 16, 2026

Copy link
Copy Markdown
Contributor Author

Hi @qjia7 ,

This PR is ready for review, PTLA, thanks!

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Extends the WebGPU Conv im2col+matmul path to support non-multiple-of-4 input channel counts by selecting an appropriate vectorization width (vec4/vec2/vec1), enabling broader use of the optimized kernel.

Changes:

  • Add vec_size template parameter and logic to support vec1/vec2 loads in im2col_matmul.wgsl.template.
  • Select vec_size at runtime based on channel_input and plumb it through shader generation/caching in im2col_matmul.{h,cc}.
  • Add two new Conv2D unit tests covering cases with asymmetric padding and C=4 / C=6.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
onnxruntime/test/providers/cpu/nn/conv_op_test.cc Adds Conv2D test cases that include a non-multiple-of-4 input channel scenario (C=6).
onnxruntime/core/providers/webgpu/nn/im2col_matmul.wgsl.template Generalizes im2col indexing/tiling and accumulation to support vec_size 1/2/4.
onnxruntime/core/providers/webgpu/nn/im2col_matmul.h Extends program interface to carry vec_size.
onnxruntime/core/providers/webgpu/nn/im2col_matmul.cc Computes vec_size, updates program inputs/cache hint, and removes the prior %4 restriction.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

qjia7
qjia7 previously approved these changes Mar 16, 2026
guschmue
guschmue previously approved these changes Mar 16, 2026
@guschmue
guschmue enabled auto-merge (squash) March 16, 2026 15:59
@daijh

daijh commented Mar 17, 2026

Copy link
Copy Markdown
Contributor

Description
This PR supports vec1 for arbitrary input_channel in im2col kernel, which could bring performance gain to more models.

Like for yolov8n_pose model, there is about ~7% gain for whole model, and ~50% for those conv2d op which input_size are not multiple of 4.

Motivation and Context

As a last-minute change, the commit message needs to be updated to reflect both vec1 and vec2.

auto-merge was automatically disabled March 18, 2026 07:58

Head branch was pushed to by a user without write access

@wenqinI

wenqinI commented Mar 18, 2026

Copy link
Copy Markdown
Contributor Author

Hi @qjia7 and @guschmue,

I tried to rebase this PR, I guess the CI error may be fixed, please help me rerun the CI, PTAL, thanks!

@wenqinI

wenqinI commented Mar 23, 2026

Copy link
Copy Markdown
Contributor Author

Hi @qjia7 and @guschmue,

It seems the nnapi ep will fail in the new test cases (Conv2D_4 and Conv2D_5), I suspect there should be some bugs for this EP, so I create an issue and skip these test in this PR, PTAL, thanks!

@wenqinI

wenqinI commented Mar 27, 2026

Copy link
Copy Markdown
Contributor Author

Hi @qjia7 and @guschmue,

A soft ping...

The CI failed for nnapi is skipped, and all the CI test is passed now, could us push this PR forward? Thanks!

@guschmue

Copy link
Copy Markdown
Contributor

/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline

@guschmue guschmue added the ep:WebGPU ort-web webgpu provider label Mar 30, 2026
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 4 pipeline(s).

@daijh

daijh commented Apr 7, 2026

Copy link
Copy Markdown
Contributor

@guschmue @fs-eire
Please help to take a look.

@wenqinI

wenqinI commented Apr 13, 2026

Copy link
Copy Markdown
Contributor Author

@guschmue,

A soft ping... PTAL, thank you!

@guschmue
guschmue enabled auto-merge (squash) April 21, 2026 16:40
@guschmue
guschmue merged commit 43d7cf4 into microsoft:main Apr 21, 2026
89 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ep:WebGPU ort-web webgpu provider

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants