Skip GroupQueryAttentionFusion when GQA node has inputs beyond sin_cache

[Copilot]

Description

GroupQueryAttentionFusion (CUDA-EP L2) rewrites a matched GQA node's inputs to a fixed 9-element list (up to sin_cache, input #8) but only fixes up the arg-count entries for inputs 7 and 8. When the node carries an optional input beyond sin_cache — position_ids (#9), attention_bias (#10), head_sink (#11) — the arg-count array keeps stale entries for the truncated slots, so Graph::Resolve fails at session creation:

This is an invalid model. The sum of input arg count is not equal to size of
input defs in node (/model/layers.0/attn/GroupQueryAttention)

The fusion also silently drops those inputs, so patching only the count array would still produce a node that computes the wrong result.

• group_query_attention_fusion.cc: In ApplyImpl, skip fusion for any GQA node with a bound input def at index ≥ 9. Chosen over "preserve + resize" since inputs like attention_bias aren't reproducible in the fused node.
• graph_transform_test_layernorm.cc: Add TestGQAFusionNotApplied (wired into GroupQueryAttentionFusionTest) asserting the fusion does not fire (both RotaryEmbedding nodes preserved) and the graph still resolves.
• gqa_fusion_with_attention_bias.onnx: New test model carrying attention_bias at input Hecli/cuda9.1 #10, derived from gqa_fusion_quantized_simple.onnx.

Motivation and Context

Affected models (e.g. onnx-community/Voxtral-Mini-4B-Realtime-2602-ONNX, whose audio encoder expresses bidirectional attention via attention_bias) fail to load on the CUDA EP at default optimization level, while loading fine with GroupQueryAttentionFusion disabled or at basic optimization — a silent CUDA-only regression. Related: #29506 (CUDA GQA kernel also rejects attention_bias; both are needed before bias-carrying GQA models run on CUDA).

[Copilot]

Pull request overview

This PR addresses a graph validity/correctness issue in the CUDA L2 optimizer GroupQueryAttentionFusion: the fusion rewrites a com.microsoft.GroupQueryAttention node to a fixed 9-input form (up to sin_cache), which can invalidate the graph and/or drop semantics when the original node includes optional inputs beyond sin_cache (e.g., attention_bias).

Changes:

• Skip GroupQueryAttentionFusion when the matched GQA node has an optional input beyond sin_cache (input index ≥ 9).
• Add a regression test ensuring the fusion does not fire (and the graph remains resolvable) for a bias-carrying GQA model.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
onnxruntime/core/optimizer/group_query_attention_fusion.cc	Adds a guard to skip fusion when optional inputs beyond sin_cache are present on the GQA node.
onnxruntime/test/optimizer/graph_transform_test_layernorm.cc	Adds a unit test verifying the fusion is not applied for a GQA node with attention_bias and that graph.Resolve() succeeds.