Skip to content

[WebGPU] Disable automatic subgroup MatMul selection on gen-12hp - #33913

Draft
xhcao wants to merge 2 commits into
microsoft:mainfrom
xhcao:disable_subgroup_matmul
Draft

xhcao wants to merge 2 commits into
microsoft:mainfrom
xhcao:disable_subgroup_matmul

Conversation

@xhcao

@xhcao xhcao commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Description

Disable automatic selection of the Intel subgroup GEMM/MatMul kernel on gen-12hp (Alchemist) GPUs. Both the
MatMul planner (MatMul and pointwise Conv) and the Gemm path now fall back to the generic packed MatMul there.
Other Intel architectures are unchanged.

Motivation and Context

Fixes #33856. #32304 routed pointwise Conv through the shared ComputeMatMul, so on Intel GPUs it now uses
MatMulSubgroup. On Alchemist this kernel is much slower than the packed MatMul. MatMul was already affected.

Copilot AI balanced review requested due to automatic review settings November 5, 2026 06:55
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request. Check if the Files changed in this pull request are included in default exclusions.

@xhcao

xhcao commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

The main cause is the subgroup size. Alchemist supports SubgroupSizeControl, and SelectMatMulSubgroupSize
tries 32 first, so the kernel is compiled as SIMD32. It holds 8 vec4 accumulators per thread (elements per
thread Y = 8) plus the A values for the subgroup broadcasts. At SIMD32 each thread gets fewer registers, so this
likely spills. Forcing subgroup size 16 alone cuts the first shape from 7043 us to 622 us.

Tuning the kernel (subgroup size 16, elements per thread Y = 2 or 4) makes it faster than the packed MatMul on
some fp16 shapes and slower on others, and roughly even or slower for fp32. So this PR falls back to the packed
MatMul on Alchemist and leaves subgroup tuning for a follow-up.

@xhcao

xhcao commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

/run failed bots

@qjia7

Copy link
Copy Markdown
Contributor

The request was received. Failed workflow jobs were queued for rerun. Thanks.

@xhcao

xhcao commented Nov 10, 2026

Copy link
Copy Markdown
Contributor Author

Jiajia Qin (@qjia7) I had tuned the subgroup kernel on A770, which could get good performance than matmul_packed kernel. So I think it does not need to disable the path on gen-12hp. #34074

@xhcao
xhcao marked this pull request as draft November 10, 2026 08:21

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants