Skip to content

fix: classify source-free aggregate calls correctly (COUNT(*)) - #25

Merged
funcpp merged 3 commits into
mainfrom
fix/source-free-aggregates
Sep 22, 2026
Merged

funcpp merged 3 commits into
mainfrom
fix/source-free-aggregates

Conversation

@funcpp

@funcpp funcpp commented Sep 21, 2026

Copy link
Copy Markdown
Owner

Cherry-picked from #13 (38149bf), one of the contributions in #15. Authorship preserved; the only conflict was the MERGE block restructured in #22.

The defect

A projection's kind is computed once and stamped in one place — on the edges it draws to its ancestors:

let ancestors = self.collect_ancestors(expr);
let kind = classify_expr(expr);
for &anc in &ancestors {
    self.graph.add_edge(anc, output, kind.clone());
}

An expression that reads no column draws no edge, so its kind had nowhere to go and derive_transform fell through to Direct — indistinguishable from SELECT 1 AS c. COUNT(*) is this case: * is a FunctionArgExpr::Wildcard, not an Expr, so collect_ancestors never visits it. classify_expr had already called the expression an aggregate; there was just nowhere to put the answer.

The kind is now stamped on the output column as well, and a column is classified from its own kind together with the kinds of its edges.

Why a union and not a fallback

The two records normally agree, so it is tempting to read whichever is present. They do not always agree. A set operation redirects the other branch's edges onto this column, carrying that branch's kind, and a branch that reaches no source leaves no edge at all. Preferring the edges drops the column's own kind in exactly the case where it is the only record of a branch:

SELECT COUNT(*) AS c FROM t UNION ALL SELECT a FROM u

The left branch touches no column and so contributes no edge; only the right branch's Direct edge survives. Under a fallback rule c reads as Direct. It is Aggregation.

Measured the other way too: classifying from the column's own kind alone passes the whole suite and changes exactly one of 42 shapes, because six of the seven edge-creation sites stamp the column's own kind onto its edges and nothing else reads RawEdge.kind. The seventh — the set-operation redirect — is the one place the edges carry information of their own, and it is the reason both records have to count.

Scope is wider than the title

This reaches every projection that resolves to no source, on every path that builds one. Measured against 927025d across 42 query shapes — 18 changed, every one of them Direct → something more specific, none the other way:

before after
SELECT COUNT(*), COUNT(1) Direct Aggregation
SELECT 1 + 2, -1, NOW() Direct Expression
SELECT CASE WHEN 1 = 1 THEN 2 ELSE 3 END Direct Conditional
SELECT COUNT(*) OVER () Direct Expression
SELECT (SELECT COUNT(*) FROM u), EXISTS (...) Direct Expression
UPDATE t SET a = NOW() Direct Expression
MERGE ... WHEN MATCHED THEN UPDATE SET a = NOW() Direct Expression
INSERT INTO o SELECT COUNT(*) FROM t Direct Aggregation
SELECT * FROM (SELECT COUNT(*) AS c FROM t) d Direct Aggregation
SELECT COUNT(*) FROM t UNION ALL SELECT a FROM u Direct Aggregation
WITH x AS (SELECT COUNT(*) AS n FROM t) SELECT SUM(n) FROM x Direct Aggregation
SELECT 1, 'lit', NULL, CAST(1 AS INT) Direct Direct
SELECT MAX(a), SELECT a unchanged unchanged

Two judgement calls worth naming:

  • Literals stay Direct. classify_expr has always mapped Expr::Value to Direct, but a literal has no ancestors, so that branch never produced an edge and the answer was unobservable. It is load-bearing now. Direct with an empty sources is still distinguishable from a real pass-through, which has one.
  • COUNT(*) OVER () lands on Expression, matching the documented meaning of TransformKind::Expression ("Expression or window function") and what SUM(x) OVER (...) already reported. TransformKind::Window is marked reserved and is produced nowhere.

Worth calling out for consumers: filtering on transform == Direct to find pass-through columns will now correctly stop matching NOW() and 1 + 2.

Known limitation, pinned by a test

Only the last hop classifies a column. collect_output_sources records the kind of the immediate incoming edge and drops every kind met deeper in the walk, so an aggregate below a CTE or a derived table still reads as Direct:

WITH x AS (SELECT COUNT(*) AS c FROM t) SELECT c FROM x   -- Direct
SELECT c FROM (SELECT COUNT(*) AS c FROM t) d            -- Direct
SELECT c FROM (SELECT SUM(x) AS c FROM t) d              -- Direct, and t.x is carried

The third line shows this is not a question of missing sources. The column's own kind does not reach it either — that kind describes the outer projection, which really is a plain reference. Fixing it means propagating kinds through the transitive walk, which is deeper than this change and in the area #14 reworks. Unchanged from main, so not a regression.

One consequence is new here: expanding a star takes no second hop, so SELECT * FROM (SELECT COUNT(*) AS c FROM t) d reports Aggregation while SELECT c FROM (...) d reports Direct. Both were Direct before. The test pins all four shapes.

Also left for later: RawEdge.kind is redundant everywhere except the set-operation redirect, and removing it would let a source-free branch contribute its kind through the node instead of through an edge it cannot draw — which is what SELECT a FROM t UNION ALL SELECT COUNT(*) FROM u still needs. Filed separately.

Commits

  • 776e9af — the fix, as contributed.
  • 57d2c70 — classify from both records rather than preferring one, with the naming that follows: intrinsic_kindkind, determine_edge_kindclassify_expr.
  • ebd86e8 — tests: the source-free boundary, the set-operation union, the last-hop gap and the star/named disagreement.
cargo fmt --all --check                                           exit 0
cargo clippy --workspace --all-targets --all-features -D warnings exit 0
cargo test --workspace --all-features                             exit 0

Compatibility

No public type or signature change. transform values change for the projections listed above — a 0.3.0 item alongside #18 and #23.

🤖 Generated with Claude Code

determine_edge_kind correctly identifies any aggregate function call
as EdgeKind::ViaAggregation regardless of its arguments, but that kind
was only ever attached to the graph as an edge to an ancestor column.
COUNT(*) has no column ancestor (`*` is a FunctionArgExpr::Wildcard,
not an Expr, so collect_ancestors never visits it), so the correctly
computed kind was silently discarded and the output's transform
classification fell back to Direct — indistinguishable from a literal
constant.

Store the defining expression's intrinsic edge kind on the Output node
itself, and use it as a fallback in derive_transform whenever no
ancestor edge exists to classify from. This generalizes to any
zero-ancestor aggregate/conditional/expression, not just COUNT(*).
@funcpp
funcpp force-pushed the fix/source-free-aggregates branch from 7036ebf to 35018c8 Compare September 22, 2026 01:31
funcpp and others added 2 commits September 22, 2026 10:55
The kind a projection computes is stamped in two places — on the output
column and on every edge it draws to an ancestor — so the two normally
agree, and the previous rule read whichever was non-empty. They do not
always agree. A set operation redirects the other branch's edges onto this
column, carrying that branch's kind, and a branch that reaches no source
leaves no edge at all. Preferring the edges silently drops the column's own
kind in exactly the case where it is the only record of a branch:

    SELECT COUNT(*) AS c FROM t UNION ALL SELECT a FROM u

`COUNT(*)` touches no column, so the left branch contributes no edge; only
the right branch's `Direct` edge survives, and the column read as `Direct`.
It is now `Aggregation`. Both records count, so the rule is a union.

That also settles the naming. Nothing here is intrinsic or a fallback — it
is the column's own kind, so `intrinsic_kind` becomes `kind`. And
`determine_edge_kind` never was about edges: six of its seven callers stamp
the result on a node, and the seventh is the set-operation redirect. It
classifies an expression, so it is `classify_expr`.

`derive_transform` no longer builds a slice to iterate; it probes the two
records in place.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three tests over the new rule:

- which source-free projections move off `Direct` and which stay there, so
  the literal/function line is a decision and not an accident;
- that a set operation counts every branch, including one that reached no
  source;
- that classification still survives only the last hop, with the star form
  disagreeing with the named form on the same derived table.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants