Skip to content

Narrow multi-output match candidates to consumers of bound inputs - #3168

Open
Mayuri (mayuriphad) wants to merge 2 commits into
microsoft:mainfrom
mayuriphad:fix/multi-output-match-candidates
Open

Mayuri (mayuriphad) wants to merge 2 commits into
microsoft:mainfrom
mayuriphad:fix/multi-output-match-candidates

Conversation

@mayuriphad

@mayuriphad Mayuri (mayuriphad) commented Nov 6, 2026 •

Copy link
Copy Markdown

Fixes #3154.

For a pattern with several output nodes, SimplePatternMatcher.match fixed the first output to the candidate node and tried every graph node with the same op type for each other output. It also rebuilt an index of all graph nodes for every candidate. On a graph with many Slice nodes, SlicesSplit (the only multi-output rule in the default set) ends up far slower than every other rule together.

Changes in _matcher.py:

  • The first output node is matched once, up front. Its result does not depend on the other candidates, so if it fails no combination can succeed and the loop is skipped.
  • For each other output node, the candidates are the consumers of a value the first match already bound to one of its inputs, at the same input position. A node that does not read that value cannot match, so this does not drop any match. Named pattern variables are looked up in bindings, the others in value_bindings.
  • The op-type index is only built when no input of the other output node is bound, and then once per call.

Timing SlicesSplit alone on n independent Slices of one input (the graph from the issue, axis 0 so nothing is rewritten):

n main this PR
250 0.26 s 0.03 s
500 1.00 s 0.10 s
1000 4.55 s 0.34 s

Here every Slice reads the same x, so each one still has all the others as candidates and the time still grows faster than linearly; avoiding that would need rule-specific knowledge.

Tests: two cases in pattern_test.py. One checks that only the pair sharing an input is rewritten. The other counts _multi_match calls on a graph where each Relu shares its input with one Neg: 20 calls with this change, 799 on main. pytest onnxscript/rewriter passes (595 passed, 2 skipped; ort_fusions/gqa_test.py skipped locally because torch is not installed), and ruff check / ruff format --check are clean.

For a pattern with several output nodes, SimplePatternMatcher fixed the
first output to the candidate node and tried every graph node with the
same op type for the other outputs, rebuilding an index of all nodes for
each candidate. With many Slice nodes, SlicesSplit dominated optimize().

Match the first output node once and stop if it fails. Otherwise, take
the candidates for each other output node from the consumers of a value
the first match already bound to one of its inputs, since a node that
does not read that value cannot match. The op-type index is only built,
once per call, when no input is bound.

Fixes microsoft#3154

Signed-off-by: mayuriphad <mayuriphad656@gmail.com>
Signed-off-by: mayuriphad <mayuriphad656@gmail.com>
Copilot AI balanced review requested due to automatic review settings November 6, 2026 10:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A critical early return in _match_node leaves the match result truthy when a candidate has too few outputs, allowing an incomplete match to be treated as valid for rewriting.

1 open issue

Review effort: Balanced

# candidates for the other output nodes.
self._init_match(verbose)
if not self._match_node(pattern_output_nodes[0], node):
return self._match

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

SlicesSplit rule makes optimizer.optimize quadratic in the number of Slice nodes

2 participants