fix: make column mapping order deterministic - #21
Merged
Merged
Conversation
The order of `ColumnLineage.mappings` varied between runs of the same
binary whenever a query had duplicate output column names:
SELECT a.id, b.id FROM a JOIN b ON a.id = b.bid
sometimes [ id <- a.id , id <- b.id ]
sometimes [ id <- b.id , id <- a.id ]
With three duplicates, four distinct orderings showed up across six runs.
Callers that cache or diff results see the same input produce different
output.
`resolve` collected the output nodes into a `HashSet<NodeId>` and built
the mappings by iterating it, so construction order followed hash order
with a per-process random seed. The sort afterwards could not undo this:
it keyed on a `HashMap<String, usize>` of output names, so duplicate
names collided and one index won, leaving same-named mappings in
whatever order the set had produced.
`ordered_cols` already holds the projection order, so build the mappings
straight from it. That makes the sort redundant — it can only reproduce
the order the loop now has, and it is the reason duplicates were
reordered in the first place — so it goes too.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
`three_duplicate_output_names_preserve_projection_order` covered the same path as the two-column test. It earned its place while the bug existed, when a wrong order could pass by luck; with the `HashSet` gone, mapping order follows `ordered_cols` structurally and a third duplicate exercises nothing new. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cherry-picked from #12 (
56677a0), one of the contributions in #15. Authorship preserved; the only change on rebase was a conflict caused by the rustfmt pass in #17.What
resolvecollected output nodes into aHashSet<NodeId>and built mappings by iterating it, so construction order followed hash order with a per-process random seed. Thesort_by_keyafterwards keyed on aHashMap<String, usize>of output names, so duplicate names collided and same-named mappings stayed in whatever order the set produced.The loop now walks
ordered_cols, which already carries projection order. That makes the sort redundant, so it goes too.Both mappings target
id; which came first varied between runs.Verification
duplicate_output_names_preserve_projection_order, run 15 times against the pre-fix resolver and the fixed one: 14/15 fail before, 0/15 fail after.(#12 also carried a three-duplicate variant of the same test. It covered the same path — with the
HashSetgone, order followsordered_colsstructurally — so a second commit here drops it. It was measured too: 12/15 fail pre-fix, 0/15 post-fix.)Checked for behavior change beyond ordering, since removing the
HashSetalso removes its deduplication and removing the sort could reposition star-expanded columns. Neither shows up — output is identical pre- and post-fix for:Each projection item gets its own
Outputnode, so theHashSetwas never deduplicating anything; andordered_colsalready placed star nodes where the old sort pushed their expansions.Compatibility
No public type or signature change. Results change only for previously nondeterministic cases, where one of the orderings is now always chosen.
🤖 Generated with Claude Code