[jiangxinmeng1]

What type of PR is this?

• API-change
• BUG
• Improvement
• Documentation
• Feature
• Test and CI
• Code Refactoring

Which issue(s) this PR fixes:

issue #24553

What this PR does / why we need it:

Materialized views now bind refresh work to the exact catalog generation and publish result, auxiliary state and tail watermark in one transaction. Source replacement and stale prepared reads fail explicitly; unrestricted DELETE preserves referenced source identity and releases dead groups/state.

Fixes #24553.

Implemented SQL: FAST/FORCE/COMPLETE ON CHANGE; COMPLETE ON DEMAND with caller-transaction REFRESH; DROP MATERIALIZED VIEW. Incremental maintenance covers grouped COUNT/SUM/AVG/MIN/MAX, exact DISTINCT aggregates, HAVING and compatible UNION ALL. COMPLETE supports up to 16 direct base sources including joins. Unsupported hidden subqueries/CTEs/session-dependent or volatile expressions are rejected before persistence. This is the concrete D2 scope; the earlier operator-graph roadmap is not part of this implementation.

Repairs and cleanup:

• One versioned catalog definition, exact tenant/target/source/state IDs, schema checks and private write authority; property finalization removes unresolved duplicate definitions.
• Owned, bounded historical delete lookup across flushed objects and pre-tombstone snapshots; bounded row/SQL chunks and immediate cleanup on partial source acquisition.
• Isolated malformed-job replay, source routing, generation GC and rename/drop cleanup. FORCE fallback requires a known successful rollback; cancellation, resource limits, CAS loss and uncertain outcomes remain terminal.
• Separate incompatible MV/index batch layouts while retaining sharing within each class; preserve legacy primary-key deduplication for batches without column names. Rename updates the complete persisted source identity projection.
• Refresh outputs bind to explicit MV column lists by position, including affected-group rebuilds and UNION branches; generated delta signs avoid all case-insensitive user-column collisions.
• Correct NULL metadata across all VALUES rows and SUM/COUNT initialization for AVG(DISTINCT). Tests now compare persisted results with independent source aggregates through duplicates, NULLs, deletes, group movement and reuse.

Maintenance in this revision:

• Resolve staticcheck S1011 in the source-reference collector with variadic append, preserving every alias/self-join AST occurrence and its order.
• Merge onto current upstream main 9f7fe43649. Preserve upstream protocol allocations; reserve protocol 132 for MV definitions/jobs and multi-source ISCP, and explicitly reject predecessors below 132 at MV admission. Regenerate the merged plan protobuf while retaining the new CTAS execution-protocol field.

Validation on the current revision (Linux amd64, Go 1.27.1):

• Incremental go vet and configured molint passed for all 16 changed owning packages; molint's informational recover diagnostics match the incoming PR CI, with none introduced by this maintenance delta. Configured golangci-lint 2.14.0 scanned the full package graph with --new-from-rev 27131f3838: 0 issues. The original ARM64 SCA failure was S1011; local amd64 validation does not claim the new ARM64 CI result.
• Complete normal suites passed for catalog/mvdefinition, planner, compiler, ISCP, disttae and its catalog cache. Two existing unrelated planner subtests remain skipped; no test was weakened or newly skipped.
• Complete race suites passed for catalog/mvdefinition, compiler, ISCP, disttae and its catalog cache. Focused planner/compiler race controls cover repeated AST references, MV rejection below protocol 132/acceptance at 132, temporal protocol 122 and CTAS compatibility.
• Parser/tree, EXPLAIN, generated plan/task, metric, engine and defines suites passed. Frontend privilege/statement/snapshot/table-dump controls and the CDC ISCP SQL-builder test passed with race detection.
• Owned native build and repository CPU service build passed. Independent canonical protobuf regeneration matched all three postprocessed outputs byte-for-byte; parser source/output are unchanged from the original candidate.
• All four existing materialized-view SQL/result pairs passed twice on the same isolated single-CN/TN/logservice instance: 220/220 each run, zero failures/ignored/abnormal cases. Test databases were absent and active ISCP jobs were zero after each run; the owned service exited normally. This includes signed-zero admission and repeated-source qualification, plus the MV/FULLTEXT interop controls.
• Independent solution/design and final overall review: gpt-6.1-sol / xhigh. The full PR change map and inherited findings were reconciled with the narrow maintenance delta. New PR CI is reported separately.

Earlier feature validation included transactional rollback, restart/tail recovery and paired workload/freshness measurements. Those remain historical evidence; this maintenance revision does not claim a new restart, proxy/multi-CN, throughput or freshness benchmark.

Compatibility: requires protocol 129 throughout MV-serving services. Protocols below 129 are predecessors without the MV consumer and are rejected by MV admission; they must not create, claim, refresh, or recover MV jobs. Earlier unreleased MV envelopes require recreation; drop MVs before binary rollback. Mixed-version MV availability and whole-cluster downgrade are not claimed.

Design: English, 中文.

[qodo-code-review]

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →