Repository navigation
Narrow multi-output match candidates to consumers of bound inputs - #3168
Open
Mayuri (mayuriphad) wants to merge 2 commits into
Open
Mayuri (mayuriphad) wants to merge 2 commits into
Mayuri (mayuriphad) wants to merge 2 commits into
Conversation
For a pattern with several output nodes, SimplePatternMatcher fixed the first output to the candidate node and tried every graph node with the same op type for the other outputs, rebuilding an index of all nodes for each candidate. With many Slice nodes, SlicesSplit dominated optimize(). Match the first output node once and stop if it fails. Otherwise, take the candidates for each other output node from the consumers of a value the first match already bound to one of its inputs, since a node that does not read that value cannot match. The op-type index is only built, once per call, when no input is bound. Fixes microsoft#3154 Signed-off-by: mayuriphad <mayuriphad656@gmail.com>
Signed-off-by: mayuriphad <mayuriphad656@gmail.com>
| # candidates for the other output nodes. | ||
| self._init_match(verbose) | ||
| if not self._match_node(pattern_output_nodes[0], node): | ||
| return self._match |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Fixes #3154.
For a pattern with several output nodes,
SimplePatternMatcher.matchfixed the first output to the candidate node and tried every graph node with the same op type for each other output. It also rebuilt an index of all graph nodes for every candidate. On a graph with manySlicenodes,SlicesSplit(the only multi-output rule in the default set) ends up far slower than every other rule together.Changes in
_matcher.py:bindings, the others invalue_bindings.Timing
SlicesSplitalone on n independentSlices of one input (the graph from the issue, axis 0 so nothing is rewritten):Here every
Slicereads the samex, so each one still has all the others as candidates and the time still grows faster than linearly; avoiding that would need rule-specific knowledge.Tests: two cases in
pattern_test.py. One checks that only the pair sharing an input is rewritten. The other counts_multi_matchcalls on a graph where eachRelushares its input with oneNeg: 20 calls with this change, 799 on main.pytest onnxscript/rewriterpasses (595 passed, 2 skipped;ort_fusions/gqa_test.pyskipped locally because torch is not installed), andruff check/ruff format --checkare clean.