Repository navigation
feat: run text value set operations as SQL where order and repeats don't count (#485) - #493
Closed
CaffeinatedCoder wants to merge 4 commits into
Closed
CaffeinatedCoder wants to merge 4 commits into
CaffeinatedCoder wants to merge 4 commits into
Conversation
…n't count (#485) PostgreSQL can't sort text as memory does (#479), so on StringSet and StringSet<T> Intersect, Except and Add ran in memory only, and Count, == and != over a server-side text Union threw. Most uses don't depend on order or repeats, so the translator now emits unguarded placeholders for all four and ValueSetSqlGuard builds them unsorted (array_cat, or a filter of the first array) where the walker allows: @>, <@, &&, IsEmpty, a null test and the outermost projection, as for Union before. Two more uses are rewritten: - cardinality over one becomes (SELECT count(DISTINCT x) FROM unnest(...)), NULL for a NULL set; - =, <> and EF Core 11's NULLIF with one on a side become mutual containment, a @> b AND b @> a. A stored-column = keeps = and its index. No COLLATE "C": a collated operand costs a GIN index its use, and #483 makes a string set column compare bytes. Grouping, DISTINCT, ORDER BY, deduplicating set operations, IN over several sets and UPDATE setters stay refused. UnionNamer's renaming moves into the expander, which recognizes a derived value structurally plus the walker's derived subquery columns, and ValueSetsMemberTranslator no longer declines Count over a visible union. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…refusal seed (#485) text-set-count-counts-repeats leaves Count as cardinality, and text-set-equality-compares-arrays rewrites == only when both sides are a text operation; CanonicalSetOperationParityTests and UnionCanonicalityTests catch both. union-equality-not-refused removed an equality refusal that no longer exists: equality over a text Union is rewritten now, and the second seed holds that rewrite. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 26, 2026
Closed
Closed
CaffeinatedCoder
changed the base branch from
fix/483-reject-nondeterministic-string-set-collation
to
main
October 26, 2026 13:22
# Conflicts: # CHANGELOG.md # docs/efcore.md # src/CodoMetis.ValueRanges.EFCore.PostgreSQL/CHANGELOG.md # test/CodoMetis.ValueRanges.EFCore.PostgreSQL.IntegrationTests/DeclaredNullabilityTests.cs
feat: run text value set operations as SQL where order and repeats don't count (#485)
Owner
Author
|
Merged into |
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #485
Stacked on #492 (#483). It relies on #483's guarantee that a string set column compares bytes, and it touches the same guard file.
Problem
On
StringSetandStringSet<T>, PostgreSQL can't sort a set operation's result as memory does: it sortstextby the database collation (#479). So since #478, every set type but text got canonicalUnion/Add/Intersect/Exceptin SQL, while on text:Intersect,ExceptandAddran in memory only;Count,==and!=over a server-sideUnionthrew.#479 can change the order only in 11.0.0, so on 10.x this would stay for the line's whole life.
Change
Add/Intersect/Exceptnow emitValueSetSqlGuard.UnguardedAdd/UnguardedIntersect/UnguardedExcept(inside the NULL-over-NULLCASE), asUnionalready emittedUnguardedUnion.cardinality,=/<>andNULLIF, besides the multiset uses it already allowed.GROUP BY,DISTINCT,ORDER BY, deduplicating set operations,INandUPDATEsetters stay refused, with the messages reworded for all four operations.array_cat,array_catwith a one-element array, or a filter of the first array.cardinality(derived)as(SELECT count(DISTINCT x)::int FROM unnest(…)), NULL for a NULL set.=/<>/NULLIFwith a derived side as mutual containment,a @> b AND b @> a. A stored-column=keeps=and its index.UnionNamerfolds into it, because a separate renaming pass would rebuild the tree before the expander sees it.ValueSetsMemberTranslatorno longer declinesCountover a visible union;MayBeConcatenationis gone.COLLATE "C": measured on Text value set operators follow a nondeterministic column collation: on a case-insensitive column, Contains("a") is true for {A} and Remove("a") deletes A #483, it costs a GIN index its use. Text value set operators follow a nondeterministic column collation: on a case-insensitive column, Contains("a") is true for {A} and Remove("a") deletes A #483 makes the column's equality byte equality instead.Unionis rewritten, plus both what-runs-where tables, Good to know, Rows written by other tools and the value set section. Alsovalue-sets.md,getting-started.md, both READMEs,AGENTS.md(the intro and the text rule),architecture.mdandtesting.md. Changelogs (root, EF Core) under 11.0.0-preview.1 Added.translation-support.json: textIntersect/Except/Addare nowserver.text-set-count-counts-repeatsandtext-set-equality-compares-arrays.union-equality-not-refusedis retired: its refusal no longer exists, and the second new seed holds the rewrite.Tests
CanonicalSetOperationParityTests.Text_AnswersAsInMemory_WhereOrderAndRepeatsDontCount(new sweep).StringSetandStringSet<T>, every pair of subsets ofa,😀and。(the pair UTF-16 and code points order apart), stored canonical, out of order and twice, and with a NULL element.Counton every row;==/!=, the projection, containment andIsEmptyon the rows without a NULL element.ExecuteUpdate.UnionCanonicalityTests: the equality andCountcases now run and compare with memory, including through aUNION ALLcolumn, a conditional and a coalesce. The ordering, grouping,DISTINCT, set-operation,INandExecuteUpdaterefusals stay.SetIntegrationTestsandEfCore11RetargetTestslikewise.DeclaredNullabilityTeststreats theARRAY(…)constructor as non-strict.tools/oracles.cs seeds):Countascardinalityfails the text sweep and 2UnionCanonicalityTests;UnionCanonicalityTests.🤖 Generated with Claude Code