diff --git a/.changeset/stale-reacquired-query-ownership.md b/.changeset/stale-reacquired-query-ownership.md new file mode 100644 index 0000000000..b637045961 --- /dev/null +++ b/.changeset/stale-reacquired-query-ownership.md @@ -0,0 +1,5 @@ +--- +'@tanstack/query-db-collection': patch +--- + +Keep a reacquired Query's cached rows when an earlier release of the same Query publishes later. Previously, reacquiring a stale on-demand Query while its release waited behind another sync transaction could drop its rows, and they stayed missing if the refetch failed. diff --git a/docs/contributing/oracle-coverage.md b/docs/contributing/oracle-coverage.md index 2d3753e74e..ccefcda877 100644 --- a/docs/contributing/oracle-coverage.md +++ b/docs/contributing/oracle-coverage.md @@ -1064,7 +1064,53 @@ refetch owner's held-delete replay is the killing witness for trusting stale published owners over accepted ownership. A release/reacquisition history also holds publication, reacquires the same cached Query, then proves the older release cannot erase the new claim by releasing it and observing an empty -public row set. Larger overlap schedules, window ownership outside the ordered +public row set. That history uses infinite `staleTime`, so reacquisition starts +no fetch. A sibling table crosses a stale reacquisition whose refetch fails, +returns the same rows, or returns no rows; the refetch settles while publication +is held or after the older release publishes; zero to two peer leases own the +first row until after the refetch; and the reacquired result has one or two +rows. Its independent model keeps each active observer lease's latest +successful result and compares public rows while held, after publication, +after the refetch, after every peer's release, and after the reacquired +release. On `main` at `31dad2dd3` 20 of its 42 cells fail: the older release's +empty ownership entry made the cached result look already applied, so the +reacquired lease owned nothing. A row only it owned disappeared at +publication; a row a peer also owned disappeared when the last peer left. The repair marks a released lease's entry and +lets only an applied entry satisfy the already-applied guard and the on-demand +direct-write skip. The empty-result cells reject keeping the row for every +reacquired demand. Removing the guard fails the persisted direct-write +histories that keep an accepted write through a refetch start, and their +sibling with an empty current result and an inserted row rejects treating +every empty entry as unapplied: in on-demand mode the insert disappears at its +receipt, and in eager mode Manual survives the Newer result. A second table +rejects deleting the entry at release: lease `b` claims the row while `a`'s +release is held or after it publishes, with or without a peer `c`, and a +server DbClient's dehydrated owners are hydrated into a fresh client. Deleting +the entry copies `a` back from unpublished metadata, which adds an owner edge +and, without a peer, leaks the row after the hydrated client releases `b`. Its `reacquired` histories reacquire `a` without +cached data and fail its fetch before `b` claims the row; deleting the entry +at reacquisition copies `a` back the same way. A +reacquisition behind a refresh barrier (`deferDataRefresh`) followed by a +direct upsert requires the reacquired Query's cache to show the upsert; reading +the released entry as applied leaves the cache stale, and only the Query cache +shows it. A release whose durable commit fails leaves the row public and +stored, puts the Collection in `error`, rejects a later acquisition with the +persistence error, and a `cleanup()` and `preload()` restart starts from clean +ownership; a host that swallows the failure fails it. A sibling holds the release's +durable commit, reacquires, and fails the reacquired application's commit: the +row leaves with the durable release, the Collection enters `error`, and a +restart serves and releases the row. Eager mode cannot reach +the reacquisition premise: it has no subset release, and ending an eager sync +run removes its Query from the cache. A cleanup that skips entries without an +observer passes this owner but fails the explicit-cleanup retention test in +`tests/query.test.ts`. A receiving probe on the Node SQLite adapter does not +match the controlled failed-release premise: when a SQLite trigger aborts +the release's delete, the Collection enters `error` and storage keeps the row, +but the public Collection drops it. The SQLite persistence owner has no +witness for that divergence ([#2210](https://github.com/TanStack/db/issues/2210)). Persisted retention (`persistedGcTime`) is +outside these histories. The +[review record](oracle-reviews/2026-10-20-stale-reacquired-ownership.md) +lists the mutant outcomes and open cells. Larger overlap schedules, window ownership outside the ordered replay, and durable-adapter ordering remain open under the owners named above. A controlled cold persisted scan gives one Query two rows in both scan orders, diff --git a/docs/contributing/oracle-reviews/2026-10-20-stale-reacquired-ownership.md b/docs/contributing/oracle-reviews/2026-10-20-stale-reacquired-ownership.md new file mode 100644 index 0000000000..3a5f328b6d --- /dev/null +++ b/docs/contributing/oracle-reviews/2026-10-20-stale-reacquired-ownership.md @@ -0,0 +1,175 @@ +# Stale reacquired Query ownership oracle review + +Reviewed source: branch `investigate-finite-stale-reacquisition`, based on +`origin/main` at `31dad2dd3`. The first runs used `acaaf4c0a`. The counts, +RED/GREEN table, and mutant table were last rerun on the commit that follows +`67987ce10`, which extends the reacquired lease table and records the open +cells; that tree was measured before it was committed. The independent +reviews read the `acaaf4c0a` tree. + +## Contract and defect + +PR #2198 (`db0ac815e`) made a released Query keep an empty `queryToRows` entry +until its release publishes, so metadata that has not published the release +cannot restore it as an owner. The result handler's already-applied guard and +the on-demand direct-write skip also read entry presence as "the active +observer lease has applied a result". When a stale Query was reacquired while +its release was held, the refetch-start notification repeated the cached +result, the guard read the release's entry as applied, and the reacquired +lease owned nothing. The release then removed the row, and a failed refetch +never restored it. + +Laws checked, each stated beside its model in +`packages/query-db-collection/tests/ownership-lifecycle.oracle.test.ts`: + +1. A reacquired observer lease owns the Query's latest successful data from + reacquisition until its own release. A successful refetch replaces that + result; a failed one keeps it. `staleTime` decides only whether a refetch + starts. +2. Published owners never name a released lease, including when another lease + claims the row while the release is held. +3. An accepted on-demand direct write patches the cache of an active Query + whose lease has not applied a result. +4. An accepted direct write survives a refetch start of an empty current + result. +5. A release whose durable commit fails is terminal for its sync run, and a + restart begins from clean ownership. + +Authority: the file's opening ownership and direct-write laws, the existing +reacquisition history, TanStack Query's stale-mount and failed-refetch rules, +the DbClient hydration history, the glossary's cleanup definition, and the +AGENTS.md fail-fast rules. + +## Repair + +`hasAppliedOwnership` reports an entry only when it is not a release's entry. +`removeQueryOwnership` marks its empty entry in a `WeakSet`. The guard and the +on-demand direct-write skip use the helper. The result-application rollback +snapshot keeps the entry by reference (CR-1 below). The production diff is 15 +lines added and 6 removed in `src/query.ts`. + +## RED and GREEN + +| Witness | `main` `31dad2dd3` | Repair | +| --- | --- | --- | +| Reacquired lease table (42 cells: 7 refetch histories × 0–2 peers × 1–2 rows) | 20 fail. Example: `published: []` and `settled: []` where the model gives `["shared"]`. With peers, `shared` is lost only at `peerReleased`, while a two-row result loses `solo` at publication | 42 pass | +| Published owners table (8 cells) | 8 pass | 8 pass | +| Reacquired cache patch behind `deferDataRefresh` | Fails: cache `["Shared"]`, model `["Manual"]` | Passes | +| Empty-result direct insert (eager, on-demand) | 2 pass | 2 pass | +| Failed release commit (3 cells) | 3 pass | 3 pass | +| Reacquired commit fails after a held release | Passes | Passes | + +The oracle file passes 585 of 585 tests with the repair and 564 on `main`. The +Query Collection package passes 1460 tests with 2 skipped, and the type check reports no errors. +Three repeated runs of the new tables gave identical results. + +## Mutant results + +Each mutant was applied to the repaired source, run against the whole oracle +file, and reverted. Counts are for the final oracle file. + +| Mutant | Outcome | Killing witness | +| --- | --- | --- | +| Guard reads entry presence (the defect) | Assertion failure, 20 tests | Reacquired lease table | +| Release entry not marked | Assertion failure, 21 tests | Reacquired lease table, cache patch | +| Direct-write skip reads entry presence | Assertion failure, 1 test | Cache patch (Query cache only; public rows agree) | +| Applied means non-empty entry | Assertion failure, 2 tests | Empty-result direct insert: on-demand loses the insert at its receipt; eager keeps Manual after Newer | +| Delete entry at release | Assertion failure, 4 tests | Published owners `while-held`: owner count 2 instead of 1; without a peer the hydrated row leaks | +| Delete the released entry when the Query is reacquired | Assertion failure, 2 tests | Published owners `while-held, reacquired`: the reacquired lease has no cached result and its fetch fails, so `b` copies `a` back from unpublished metadata | +| Replace the released entry with an unmarked empty Set at reacquisition | Assertion failure, 21 tests | Reacquired lease table, cache patch | +| Mark the reacquired lease's own application entry as released | Assertion failure, 5 tests | Persisted direct-write refetch-start histories and the direct cache patch bound | +| Reuse the released Set object for the next application | Assertion failure, 25 tests | Reacquisition history and reacquired lease table: the reused entry keeps its mark | +| Guard removed (`false &&`) | Assertion failure, 4 tests | Persisted direct-write refetch-start histories | +| Hostile storage that swallows a failed commit | Assertion failure, 2 cells and 1 test | Both failed-commit witnesses | +| Cleanup skips entries without an observer | Survives the oracle file; assertion failure, 1 test, in the package suite | `tests/query.test.ts` "should clear retained ownership during explicit collection cleanup": a retained Query's retention marker survives cleanup | + +The last four rows came from a later independent review. It reported two +survivors without a distinguishing history; the concrete versions above were +written from its descriptions. Deleting the released entry at reacquisition +survived every test until the `reacquired` published-owner histories were +added; it fails them and passes nothing else differently. + +## CodeRabbit review of `8ff1db49f` + +Review 5600814169 had one inline finding and a summary. Ledger: + +| ID | Claim | Verdict | Action | +| --- | --- | --- | --- | +| CR-1 | The rollback snapshot in result application clones the entry, so a failed application that restores it drops a release's mark | True at the code level. No public consequence reached: in persisted histories the release retires its entry before the reacquired application snapshots (instrumented with the release commit gated and ungated), and in non-persisted histories the held commit rejects only by cancellation, which the restore guards skip | Fixed: the snapshot keeps the entry by reference, because application replaces the entry rather than mutating it. No witness distinguishes this; reverting it passes all 557 tests | +| CR-2 | Add a history with reacquisition during a held release, a commit failure before publication, and a failed refetch; the row must remain until the reacquired lease is released | The history runs; its expected result conflicts with the fail-loud law. The release is durable and the reacquired result is not, so the row leaves and the Collection enters `error` | Added "fails loudly when a reacquired lease's commit fails after a held release"; a storage adapter that swallows the failure fails it | +| CR-3 | Docstring coverage 0% for touched functions | True; the touched closure helpers use line comments like their neighbours | No change, per AGENTS.md's rule to match surrounding comment density | +| CR-4 | CodeRabbit's ESLint tool timed out | A tool note, not a code claim; the repository Lint check passed on `8ff1db49f` | No change | +| CR-5 | Title, description, and issue checks passed | Informational | No change | + +## ORC outcomes + +- ORC-001: Each table states its law, authority, and limits before mechanics. +- ORC-002: The models keep lease results and owner sets. They do not import + `queryToRows`, the guard, or sync transactions. +- ORC-003: Prose beside each model, grammar, driver, and comparison. The + prose was updated with each grammar change. +- ORC-004: Fixed enumerations, not generated properties. Each axis has a + stated contribution: `staleTime` starts the refetch; outcome and placement + separate keeping, replacing, and losing the row; `peer` hides a lost result + until `peerReleased`; `claims` reaches the unpublished-metadata premise. + Excluded: eager reacquisition, whose premise is unreachable (probed: + cleanup removes the Query from the cache, so a restart fetches); persisted + retention, which keeps released rows by design. +- ORC-005: Public `loadSubset`, `unloadSubset`, direct writes, `DbClient` + dehydrate and hydrate, `cleanup`, and `preload`. Observations are public rows, + stored rows, Collection status, rejection messages, dehydrated owner counts, + and `QueryClient.getQueriesData`. No missing-information hypothesis was + identified. +- ORC-006: The mutant table above. Every kill is an assertion failure at the + named comparison. +- ORC-007: Not applicable; no generated property. +- ORC-008: The reacquisition model keeps one result per active lease; the peer + axis needs the union, shown by the peer cells that differ only at + `peerReleased`. The owner model keeps a set; the owner count distinguishes + `peer` histories that public rows do not. +- ORC-009: Glossary terms observer lease, acquisition release, publication, + accepted sync transaction, and cleanup. `a`, `b`, `c`, and blocker are + declared local names. +- ORC-010: Each history registers cleanup that releases its holds and Query + clients; comparisons are single structured assertions. +- ORC-011: Not applied. The named shared-fault risk (an entry read as applied) + is separated by the empty-result and cache-patch witnesses, which observe + different boundaries; no further formulation was identified. +- ORC-012: This record. +- ORC-013: The reusable law "an empty entry is applied unless it belongs to a + release" has a premise witness (reacquired lease table) and a nearby + boundary witness (empty-result direct insert) that rejects the non-empty + rule. +- ORC-014: The controlled storage adapter supplies the commit failure. A + receiving probe on Node SQLite aborted the release's delete with a SQLite + trigger. The Collection entered `error` and storage kept the row, but the + public Collection dropped it, while a peer's row stayed. The controlled + adapter keeps the row public, whether it rejects at once or after a delay. + The real SQLite adapter drops it on every sync-present configuration tried: + managed, resolved for `sync-present`, and adapter-only without a + coordinator. The controlled premise does not hold on the real host, so the + cross-boundary claim is contradicted, not established. The divergence + belongs to the SQLite persistence owner; see Open cells. + +## Bug-class boundary + +Contract: Query ownership after release and reacquisition. Histories: an +on-demand lease with a one- or two-row result and a held release, a stale or +fresh reacquisition, zero to two peers, an optional refresh barrier, a direct write, and a failed release commit. +Path: Query Collection over a real QueryClient and a controlled persistence +adapter. Observations: public rows, stored rows, owners, status, and Query +cache at the named checkpoints. + +Open cells: + +- CR-1, unreached. Core rejects a committed, held sync transaction only in + `state.cleanup()`, which runs after sync cleanup has removed the observers, + so the restore guard returns. The remaining route is a duplicate-key replay + inside the reacquired application's own open transaction, which no + history here constructs. +- SQLite durability failure. On the Node SQLite adapter, a sync + transaction (here, a release) whose durable step fails still removes the row from the public + Collection, while storage keeps it. This contradicts the glossary's + accepted-sync-transaction rule, which accepts a wrapping sync's + transaction only after its durable step. It belongs to the SQLite + persistence owner and is outside this Query DB change; tracked in [#2210](https://github.com/TanStack/db/issues/2210). diff --git a/packages/query-db-collection/src/query.ts b/packages/query-db-collection/src/query.ts index e9e9da6b82..3feb0b5ac9 100644 --- a/packages/query-db-collection/src/query.ts +++ b/packages/query-db-collection/src/query.ts @@ -980,6 +980,14 @@ export function queryCollectionOptions( // queryKey → Set. Entry presence means ownership is resolved; // an empty set represents a resolved query that currently owns no rows. const queryToRows = new Map>() + // A released query keeps an empty entry until its release publishes, so + // unpublished metadata cannot restore it as an owner. That entry is not a + // result applied by a later acquisition of the same query. + const releasedOwnership = new WeakSet>() + const hasAppliedOwnership = (hashedQueryKey: string) => { + const ownedRows = queryToRows.get(hashedQueryKey) + return ownedRows !== undefined && !releasedOwnership.has(ownedRows) + } // RowKey → Set const rowToQueries = new Map>() @@ -1118,7 +1126,9 @@ export function queryCollectionOptions( else rowToQueries.set(rowKey, nextOwners) }) - queryToRows.set(hashedQueryKey, new Set()) + const released = new Set() + releasedOwnership.add(released) + queryToRows.set(hashedQueryKey, released) return nextOwnersByRow } @@ -2569,9 +2579,8 @@ export function queryCollectionOptions( const settledDirectPositions = new Map() const removedRows = new Set() - const previousOwnedRows = queryToRows.has(hashedQueryKey) - ? new Set(queryToRows.get(hashedQueryKey)) - : undefined + // Application replaces this entry rather than mutating it. + const previousOwnedRows = queryToRows.get(hashedQueryKey) const affectedRowKeys = new Set([ ...previouslyOwnedRows, ...newItemsMap.keys(), @@ -3003,7 +3012,7 @@ export function queryCollectionOptions( if ( result.isFetching && !collection.deferDataRefresh && - queryToRows.has(hashedQueryKey) && + hasAppliedOwnership(hashedQueryKey) && queryCollectionSuccessfulFetchData.get(observedQuery) === observedQuery.state.data ) { @@ -4271,7 +4280,7 @@ export function queryCollectionOptions( // cached rows, so inspect those entries before their owner is known. if ( !affectedOwners.has(hashedQueryKey) && - queryToRows.has(hashedQueryKey) && + hasAppliedOwnership(hashedQueryKey) && query.state.fetchStatus !== `fetching` && !pendingResultApplications.has(hashedQueryKey) ) diff --git a/packages/query-db-collection/tests/ownership-lifecycle.oracle.test.ts b/packages/query-db-collection/tests/ownership-lifecycle.oracle.test.ts index 721b4db5c6..1923d6701e 100644 --- a/packages/query-db-collection/tests/ownership-lifecycle.oracle.test.ts +++ b/packages/query-db-collection/tests/ownership-lifecycle.oracle.test.ts @@ -5909,6 +5909,781 @@ describe(`query collection ownership lifecycle`, () => { expect(queryFn).toHaveBeenCalledOnce() }) + /** + * ## A reacquired observer lease owns the current Query result + * + * Law: reacquiring a released demand creates a new observer lease. Its + * current result is the Query's latest successful data, so it owns that + * data's rows from reacquisition until its own release. An older acquisition + * release that publishes later retires only the older lease. If the new + * lease starts a refetch, a successful refetch replaces the lease's result. + * A failed refetch keeps the last successful data, so it keeps those rows. + * `staleTime` decides whether a refetch starts. It cannot decide whether the + * cached result is owned. + * + * Authority: the reacquisition law in the history above, and the TanStack + * Query rules that a stale Query refetches when an observer mounts and that + * a failed refetch keeps the previous `data`. + * + * Limits: one reacquired on-demand subset whose result has one or two + * rows, zero to two peer subsets that own the first row, no persisted + * metadata, and one unrelated optimistic transaction that holds + * publication. A release commit that + * rejects and the direct-write path have their own histories below. Eager + * mode cannot reach the premise: it has no subset release, and ending an + * eager sync run removes its Query from the cache, so a restart fetches + * instead of replaying cached data. + */ + const solo: Item = { id: `solo`, category: `detail`, name: `Solo` } + + type ReacquisitionHistory = { + /** `Infinity`: reacquisition starts no fetch. `0`: it starts a refetch. */ + staleTime: number + /** The refetch outcome. It has no effect when no refetch starts. */ + refetch: `rejects` | `same-rows` | `empty` + /** The refetch settles before or after the older release publishes. */ + settles: `while-held` | `after-publication` + /** Peer leases that also own `shared` until after the refetch. */ + peers: 0 | 1 | 2 + /** Rows in the reacquired lease's result; the second is owned only here. */ + rows: 1 | 2 + } + + type ReacquisitionObservation = { + /** Public rows after reacquisition, while publication is still held. */ + held: Array + /** Public rows after the held transactions publish. */ + published: Array + /** Public rows after the refetch, if any, has settled. */ + settled: Array + /** Public rows after every peer is released. */ + peerReleased: Array + /** Public rows after the reacquired demand is released. */ + released: Array + /** Query function calls. Query staleness decides this count. */ + queryCalls: number + } + + /** + * The model keeps each active lease's latest successful result. Public + * synced rows are the union of those results at the last publication. Each + * peer's result is always `shared`; peers are active until `peerReleased`. + * A peer hides a lost `shared` until the peers leave, which is why that + * checkpoint exists; the lease-only `solo` row shows a loss at once. While the optimistic blocker + * holds publication, the public view is the blocker plus the rows published + * before the release, which the first lease owned. The model does not + * represent tombstones, ownership maps, or sync transactions. + */ + const modelReacquisition = ( + history: ReacquisitionHistory, + ): ReacquisitionObservation => { + const cached = history.rows === 2 ? [shared.id, solo.id] : [shared.id] + const refetchStarts = history.staleTime === 0 + const refetched = !refetchStarts + ? cached + : history.refetch === `empty` + ? [] + : // Both a same-rows result and a failed refetch leave the rows owned. + cached + const union = (...results: Array>) => + [...new Set(results.flat())].sort() + const peerRows = history.peers > 0 ? [shared.id] : [] + return { + held: union([`blocker`], cached, peerRows), + published: union( + history.settles === `while-held` ? refetched : cached, + peerRows, + ), + settled: union(refetched, peerRows), + peerReleased: union(refetched), + released: [], + queryCalls: 1 + history.peers + (refetchStarts ? 1 : 0), + } + } + + /** + * The grammar crosses the refetch's start, outcome, and placement relative to + * the older release's publication. A fresh reacquisition starts no refetch, + * so it has one history. The `empty` histories reject a design that keeps the + * row for any reacquired demand. The `rejects` and `same-rows` histories + * that settle after publication reject a design that skips the cached + * result because the older lease left an ownership entry behind. Removing + * the already-applied guard altogether also passes this table; the + * persisted direct-write histories that keep an accepted write through a + * refetch start reject that repair. A repair that treats every empty + * ownership entry as unapplied also passes this table; the sibling history + * that keeps a direct insert through a refetch start of an empty result + * rejects it. Each history runs with zero, one, or two peers and with a + * one- or two-row result. With a peer, the older release keeps `shared`, + * so only the `peerReleased` checkpoint shows whether the reacquired lease + * owns it. On the original defect `shared` survives while a peer owns it + * and disappears when the peers leave, while `solo` disappears at + * publication. Two peers check that releasing every other owner, not just + * one, leaves the reacquired lease as the row's owner. + */ + const reacquisitionHistories: Array = ( + [0, 1, 2] as const + ).flatMap((peers) => + ([1, 2] as const).flatMap((resultRows) => [ + { + staleTime: Number.POSITIVE_INFINITY, + refetch: `rejects` as const, + settles: `after-publication` as const, + peers, + rows: resultRows, + }, + ...([`rejects`, `same-rows`, `empty`] as const).flatMap((refetch) => + ([`while-held`, `after-publication`] as const).map((settles) => ({ + staleTime: 0, + refetch, + settles, + peers, + rows: resultRows, + })), + ), + ]), + ) + + /** + * The driver uses the public `loadSubset` and `unloadSubset` entry points, a + * real QueryClient, and a held query function for the refetch. The blocker's + * mutation function holds publication. Each checkpoint follows a settled + * macrotask turn, so the comparison reads rows only after the step's sync + * work has run. `loadSubset` settlement is not compared: this law does not + * say whether the caller waits for the refetch. + */ + it.each(reacquisitionHistories)( + `keeps a reacquired lease's result (staleTime $staleTime, refetch $refetch, settles $settles, peers $peers, rows $rows)`, + async (history) => { + const id = `reacquired-lease-${history.staleTime}-${history.refetch}-${history.settles}-${history.peers}-${history.rows}` + const detail = categorySubset(`detail`) + const peerSubsets = [ + categorySubset(`list`), + { + where: new IR.Func(`in`, [ + new IR.PropRef([`category`]), + new IR.Value([`peer`, `shared`]), + ]), + }, + ].slice(0, history.peers) + const leaseResult = history.rows === 2 ? [shared, solo] : [shared] + const blocker = { id: `blocker`, category: `local`, name: `Blocker` } + const persistence = createDeferred() + const refetch = createDeferred>() + void refetch.promise.catch(() => undefined) + const { collection, queryFn } = createOwnershipFixture({ + id, + results: [leaseResult, ...peerSubsets.map(() => [shared])], + staleTime: history.staleTime, + }) + queryFn.mockImplementationOnce(() => refetch.promise) + const settleRefetch = () => { + if (history.refetch === `rejects`) + refetch.reject(new Error(`refetch failed`)) + else + refetch.resolve( + history.refetch === `empty` + ? [] + : leaseResult.map((item) => ({ ...item })), + ) + } + const settleTurn = async () => { + for (let turn = 0; turn < 3; turn++) + await new Promise((resolve) => setTimeout(resolve, 0)) + } + const mutation = createTransaction({ + mutationFn: () => persistence.promise, + }) + + await collection._sync.loadSubset(detail) + for (const peer of peerSubsets) await collection._sync.loadSubset(peer) + mutation.mutate(() => collection.insert(blocker)) + cleanups.push(async () => { + persistence.resolve() + refetch.reject(new Error(`cleanup`)) + await mutation.isPersisted.promise.catch(() => undefined) + }) + + // The release's sync transaction is accepted but held behind the blocker. + const pendingBefore = collection._state.pendingSyncedTransactions.length + collection._sync.unloadSubset(detail) + await vi.waitFor(() => + expect( + collection._state.pendingSyncedTransactions.length, + ).toBeGreaterThan(pendingBefore), + ) + const reacquired = Promise.resolve(collection._sync.loadSubset(detail)) + void reacquired.catch(() => undefined) + await settleTurn() + const held = rows(collection) + + if (history.settles === `while-held`) { + settleRefetch() + await settleTurn() + } + persistence.resolve() + await mutation.isPersisted.promise + await settleTurn() + const published = rows(collection) + + if (history.settles === `after-publication`) { + settleRefetch() + await settleTurn() + } + await reacquired.catch(() => undefined) + const settled = rows(collection) + + for (const peer of peerSubsets) { + collection._sync.unloadSubset(peer) + await settleTurn() + } + const peerReleased = rows(collection) + + collection._sync.unloadSubset(detail) + await settleTurn() + + expect({ + held, + published, + settled, + peerReleased, + released: rows(collection), + queryCalls: queryFn.mock.calls.length, + }).toEqual(modelReacquisition(history)) + }, + ) + + /** + * ## Published owners never name a released observer lease + * + * Law: each publication records, for each row, the observer leases that own + * it. A released lease is not an owner, even while its release waits behind + * held publication. A later lease that claims the row during that wait + * must not copy the released lease back into the row's owners from metadata + * that has not yet published the release. A hydrated client trusts those + * owners: a row stays public there while any recorded owner may still need + * it, so a stale owner keeps the row alive after its last real owner is + * released. + * + * Authority: the file's opening law (a row remains public while any committed + * owner needs it), and the hydration history above, which releases a + * hydrated demand and expects its rows to leave. The reacquisition table + * above changes how a released lease's ownership entry is read; this table + * keeps a repair from simply deleting that entry at release. + * + * Limits: one row, one released lease `a`, one claiming lease `b`, and an + * optional peer lease `c` that is never released. `a` may be reacquired + * without cached data before `b` claims the row; its fetch then fails, so + * the reacquired lease never applies a result. Hydration uses the public + * `DbClient.dehydrate` and `hydrate` paths with a fresh QueryClient. Native + * persistence and cold persisted scans are owned by the persisted histories. + */ + type PublishedOwnerHistory = { + /** `b` claims the row while `a`'s release is held, or after it publishes. */ + claims: `while-held` | `after-publication` + /** Whether peer `c` also owns the row and stays acquired. */ + peer: boolean + /** Whether `a` is reacquired without cached data and its fetch fails. */ + reacquired: boolean + } + + type PublishedOwnerObservation = { + /** Owner edges for the row in the dehydrated state. Keys are opaque. */ + dehydratedOwners: number + /** Public rows in the hydrated client after it acquires `b`. */ + hydratedLoaded: Array + /** Public rows in the hydrated client after it releases `b`. */ + hydratedReleased: Array + } + + /** + * The model records the set of active leases whose result holds the row. A + * release removes its lease at once, regardless of publication. Hydration + * copies that set; after the hydrated client releases `b`, the row stays + * exactly when another recorded owner remains. Publication timing is absent + * from the model, because the law says it cannot change the owner set. + */ + const modelPublishedOwners = ( + history: PublishedOwnerHistory, + ): PublishedOwnerObservation => { + const activeOwners = new Set([`a`, ...(history.peer ? [`c`] : [])]) + activeOwners.delete(`a`) + activeOwners.add(`b`) + const afterHydratedRelease = new Set(activeOwners) + afterHydratedRelease.delete(`b`) + return { + dehydratedOwners: activeOwners.size, + hydratedLoaded: [shared.id], + hydratedReleased: afterHydratedRelease.size > 0 ? [shared.id] : [], + } + } + + /** + * `while-held` reaches the premise: `b`'s sync transaction reads row + * metadata that still names `a`. `after-publication` is the nearby boundary + * where the release has published first. The peer histories require `c` to + * remain an owner, so a repair that drops every metadata owner at release, + * or one that ignores recorded owners on hydration, fails them. A repair that + * deletes the released lease's ownership entry at release instead of keeping + * it until publication fails the `while-held` histories. Without a peer, it + * fails at both the owner count and the leaked hydrated row; with a peer, + * `c` keeps the row public, so only the owner count distinguishes it. + * + * The `reacquired` histories reject a repair that deletes the released + * entry when the Query is reacquired. The reacquired lease has no result + * yet, so `b` reads metadata naming `a` while `a` looks unresolved and + * copies it back. A later result for `a` would correct the edge, so `a`'s + * fetch fails and it never applies one. The model is unchanged: a lease + * without a successful result owns nothing. + */ + const publishedOwnerHistories: Array = ( + [`while-held`, `after-publication`] as const + ).flatMap((claims) => + [false, true].flatMap((peer) => + [false, true].map((reacquired) => ({ claims, peer, reacquired })), + ), + ) + + /** + * The driver runs a server DbClient and a hydrated browser DbClient over the + * same Collection descriptor. On the server, an optimistic blocker holds + * publication while `a` is released and, in `while-held` histories, while + * `b` is acquired. In `reacquired` histories, `a`'s Query is removed with + * the public `QueryClient.removeQueries` after its release, then `a` is + * reacquired and its held fetch rejects after `b` claims the row. The + * checkpoints are the dehydrated state after every held + * transaction publishes, and the hydrated client's public rows after it + * acquires and then releases `b`. + */ + it.each(publishedOwnerHistories)( + `publishes only active owners when a lease claims a released row (claims $claims, peer $peer, reacquired $reacquired)`, + async (history) => { + const id = `published-owners-${history.claims}-${history.peer}-${history.reacquired}` + const subset = (category: string): LoadSubsetOptions => ({ + where: new IR.Func(`in`, [ + new IR.PropRef([`category`]), + new IR.Value([category, `shared`]), + ]), + }) + const [a, b, c] = [subset(`a`), subset(`b`), subset(`c`)] + // Query keys embed the subset as JSON, so `a`'s key holds `\"a\"`. + const isQueryForA = (queryKey: unknown) => + JSON.stringify(queryKey).includes(`\\"a\\"`) + const reacquiredFetch = createDeferred>() + void reacquiredFetch.promise.catch(() => undefined) + const createClient = ( + queryFn: (context: QueryFunctionContext) => Promise> = () => + Promise.resolve([shared]), + ) => { + const queryClient = createQueryClient() + const descriptor = collectionOptions( + queryCollectionOptions({ + id, + queryClient, + queryKey: [id], + queryFn, + getKey: (item) => item.id, + syncMode: `on-demand`, + startSync: true, + }), + ) + const client = new DbClient({ queryClient }) + cleanups.push(async () => { + await client.cleanup() + queryClient.clear() + }) + return { + client, + queryClient, + collection: client.collection(descriptor), + } + } + const settleTurn = async () => { + for (let turn = 0; turn < 3; turn++) + await new Promise((resolve) => setTimeout(resolve, 0)) + } + + let fetchesForA = 0 + const server = createClient(({ queryKey }) => { + if (isQueryForA(queryKey) && fetchesForA++ > 0) + return reacquiredFetch.promise + return Promise.resolve([shared]) + }) + const blocker = { id: `blocker`, category: `local`, name: `Blocker` } + const persistence = createDeferred() + const mutation = createTransaction({ + mutationFn: () => persistence.promise, + }) + cleanups.push(async () => { + persistence.resolve() + reacquiredFetch.reject(new Error(`cleanup`)) + await mutation.isPersisted.promise.catch(() => undefined) + }) + + await server.collection._sync.loadSubset(a) + if (history.peer) await server.collection._sync.loadSubset(c) + mutation.mutate(() => server.collection.insert(blocker)) + server.collection._sync.unloadSubset(a) + let reacquired: Promise | undefined + if (history.reacquired) { + await settleTurn() + server.queryClient.removeQueries({ + predicate: (query) => isQueryForA(query.queryKey), + }) + reacquired = Promise.resolve(server.collection._sync.loadSubset(a)) + void reacquired.catch(() => undefined) + await settleTurn() + } + const failReacquiredFetch = async () => { + const pending = reacquired + if (!pending) return + reacquiredFetch.reject(new Error(`reacquired fetch failed`)) + await pending.catch(() => undefined) + await settleTurn() + } + if (history.claims === `while-held`) { + const claimed = Promise.resolve(server.collection._sync.loadSubset(b)) + await settleTurn() + await failReacquiredFetch() + persistence.resolve() + await mutation.isPersisted.promise + await claimed + } else { + persistence.resolve() + await mutation.isPersisted.promise + await settleTurn() + await server.collection._sync.loadSubset(b) + await failReacquiredFetch() + } + await settleTurn() + const state = server.client.dehydrate({ + shouldDehydrateCollection: () => true, + }) + const sharedRow = state.collections + .flatMap((chunk) => chunk.rows) + .find((row) => row.key === shared.id) + const owners = ( + sharedRow?.metadata as + { queryCollection?: { owners?: Record } } | undefined + )?.queryCollection?.owners + + const browser = createClient() + browser.client.hydrate(state) + await browser.collection._sync.loadSubset(b) + const hydratedLoaded = rows(browser.collection) + browser.collection._sync.unloadSubset(b) + await settleTurn() + + expect({ + dehydratedOwners: Object.keys(owners ?? {}).length, + hydratedLoaded, + hydratedReleased: rows(browser.collection), + }).toEqual(modelPublishedOwners(history)) + }, + ) + + /** + * ## A reacquired lease's cached rows receive accepted direct writes + * + * Law: in on-demand mode, an accepted direct write patches the cached rows + * of each active Query in its Collection that may still apply them. A Query + * whose lease has not yet applied a result may hold the written key, so it + * is patched; only a resolved scope that does not own the key may be + * skipped. The patched cache keeps a later replay of that cached result + * from presenting the pre-write value as a server result. + * + * Authority: the opening direct-write law of this file. A released lease's + * empty ownership entry, kept while its release is held, does not resolve a + * later lease of the same Query. + * + * Reference: one row whose server value is Shared, then an accepted Manual + * upsert. The model expects the public row and the reacquired Query's cache + * to show Manual. + * + * History: hold publication behind an optimistic blocker, release the + * demand, raise a refresh barrier (`deferDataRefresh`, as offline + * transactions do while they replay), and reacquire. The barrier keeps the + * cached result unapplied. The acquisition's own fetch fails, so Query + * keeps its cached data and is idle when the upsert arrives. The checkpoint + * is after publication and the barrier both end. Public rows alone cannot + * distinguish the skip: the upsert keeps the row either way. The Query + * cache, read through `QueryClient.getQueriesData`, does. + */ + it(`patches a reacquired lease's cached rows while its result waits`, async () => { + const id = `reacquired-lease-direct-patch` + const detail = categorySubset(`detail`) + const blocker = { id: `blocker`, category: `local`, name: `Blocker` } + const persistence = createDeferred() + const barrier = createDeferred() + const { collection, queryClient, queryFn } = createOwnershipFixture({ + id, + results: [[shared]], + }) + const mutation = createTransaction({ + mutationFn: () => persistence.promise, + }) + const settleTurn = async () => { + for (let turn = 0; turn < 3; turn++) + await new Promise((resolve) => setTimeout(resolve, 0)) + } + cleanups.push(async () => { + collection.deferDataRefresh = null + barrier.resolve() + persistence.resolve() + await mutation.isPersisted.promise.catch(() => undefined) + }) + + await collection._sync.loadSubset(detail) + mutation.mutate(() => collection.insert(blocker)) + collection._sync.unloadSubset(detail) + await settleTurn() + collection.deferDataRefresh = barrier.promise + const reacquired = Promise.resolve(collection._sync.loadSubset(detail)) + void reacquired.catch(() => undefined) + await settleTurn() + const write = Promise.resolve( + collection.utils.writeUpsert({ ...shared, name: `Manual` }), + ) + void write.catch(() => undefined) + await settleTurn() + + persistence.resolve() + await mutation.isPersisted.promise + collection.deferDataRefresh = null + barrier.resolve() + await reacquired.catch(() => undefined) + await write + await settleTurn() + + expect({ + row: collection.get(shared.id)?.name, + cache: queryClient + .getQueriesData>({ queryKey: [id] }) + .flatMap(([, data]) => data ?? []) + .map((item) => item.name), + queryCalls: queryFn.mock.calls.length, + }).toEqual({ row: `Manual`, cache: [`Manual`], queryCalls: 2 }) + }) + + /** + * ## A release whose durable commit fails is terminal for its sync run + * + * Law: a release is complete only when its commit is durable. If that + * commit fails, the Collection reports the failure and enters `error`; the + * released row stays public and stored, because the deletion never became + * durable. A later acquisition in that run rejects with the same failure + * instead of reading ownership the failed release left behind. Cleanup ends + * the run; a restarted run starts with no ownership from the old run, so + * its acquisition and release behave as in a fresh Collection. + * + * Authority: the Code Weight and Fail-Fast rules in AGENTS.md (a + * contradictory collaborator signal rejects at the boundary instead of + * recovering), the persisted direct-write receipt law in this file's opening + * prose, and the glossary's cleanup definition (it ends the sync run and its + * resources). + * + * Reference: the model owns one row through one demand. A failed release + * leaves `{ rows: [shared], stored: [shared], status: error }`; a restarted + * load and release give `[shared]` and then `{ rows: [], stored: [] }`. The + * `durable` history is the control: the release succeeds and the restarted + * run never refetches, because no demand remains. + * + * Limits: one on-demand row, no persisted retention (`persistedGcTime: 0`, + * since retention intentionally keeps released rows), a controlled adapter + * whose next commit throws, and a public `cleanup()` then `preload()` + * restart. A failed commit that a host retries is outside this witness. + */ + it.each([`durable`, `failed`, `failed-then-reacquired`] as const)( + `keeps a released row and fails loudly when its release commit fails (%s)`, + async (history) => { + const id = `release-commit-${history}` + const storage = createOwnershipStorage() + const queryClient = createQueryClient() + const queryFn = vi.fn(() => Promise.resolve([shared])) + let failNextCommit = false + const applyCommittedTx = storage.adapter.applyCommittedTx + storage.adapter.applyCommittedTx = (...args) => { + if (!failNextCommit) return applyCommittedTx(...args) + failNextCommit = false + return Promise.reject(new Error(`storage failed`)) + } + const collection = createCollection( + persistedCollectionOptions< + Item, + string | number, + never, + QueryCollectionUtils + >({ + ...queryCollectionOptions({ + id, + queryClient, + queryKey: [id], + queryFn, + getKey: (item) => item.id, + syncMode: `on-demand`, + persistedGcTime: 0, + startSync: true, + }), + persistence: { adapter: storage.adapter }, + }), + ) + cleanups.push(async () => { + await collection.cleanup().catch(() => undefined) + queryClient.clear() + }) + const settleTurn = async () => { + for (let turn = 0; turn < 3; turn++) + await new Promise((resolve) => setTimeout(resolve, 0)) + } + const observe = () => ({ + rows: rows(collection), + stored: storedItems(storage).map((item) => item.id), + status: collection.status, + }) + const detail = categorySubset(`detail`) + + await collection._sync.loadSubset(detail) + failNextCommit = history !== `durable` + collection._sync.unloadSubset(detail) + await settleTurn() + const afterRelease = observe() + + let reacquisition: string | undefined + if (history === `failed-then-reacquired`) { + reacquisition = await Promise.resolve( + collection._sync.loadSubset(detail), + ).then( + () => `fulfilled`, + (error: unknown) => + error instanceof Error ? error.message : String(error), + ) + } + + await collection.cleanup() + await collection.preload() + await collection._sync.loadSubset(detail) + const restartedLoad = rows(collection) + collection._sync.unloadSubset(detail) + await settleTurn() + + const failed = history !== `durable` + expect({ + afterRelease, + reacquisition, + restartedLoad, + restartedRelease: observe(), + queryCalls: queryFn.mock.calls.length, + }).toEqual({ + afterRelease: failed + ? { rows: [shared.id], stored: [shared.id], status: `error` } + : { rows: [], stored: [], status: `ready` }, + reacquisition: + history === `failed-then-reacquired` + ? `Failed to durably persist collection "${id}": storage failed` + : undefined, + restartedLoad: [shared.id], + restartedRelease: { rows: [], stored: [], status: `ready` }, + queryCalls: failed ? 2 : 1, + }) + }, + ) + + /** + * The same fail-loud law covers a reacquired lease whose own durable commit + * fails. The release's commit is held at the persistence boundary while the + * Query is reacquired; the reacquired application's commit then fails. The + * release is durable, so the row leaves; the reacquired result never became + * durable, so it cannot keep the row. The Collection enters `error`, the + * acquisition rejects with the persistence failure, and a restart serves and + * releases the row normally. A history that keeps the row public after the + * failure would report a result that storage never accepted. + */ + it(`fails loudly when a reacquired lease's commit fails after a held release`, async () => { + const id = `reacquired-commit-fails` + const storage = createOwnershipStorage(undefined, 2) + const queryClient = createQueryClient() + const queryFn = vi.fn(() => Promise.resolve([shared])) + let commits = 0 + const applyCommittedTx = storage.adapter.applyCommittedTx + storage.adapter.applyCommittedTx = (...args) => { + // Commit 1 loads the row, 2 is the held release, 3 is the reacquisition. + if (++commits === 3) return Promise.reject(new Error(`storage failed`)) + return applyCommittedTx(...args) + } + const collection = createCollection( + persistedCollectionOptions< + Item, + string | number, + never, + QueryCollectionUtils + >({ + ...queryCollectionOptions({ + id, + queryClient, + queryKey: [id], + queryFn, + getKey: (item) => item.id, + syncMode: `on-demand`, + persistedGcTime: 0, + startSync: true, + }), + persistence: { adapter: storage.adapter }, + }), + ) + cleanups.push(async () => { + storage.release() + await collection.cleanup().catch(() => undefined) + queryClient.clear() + }) + const settleTurn = async () => { + for (let turn = 0; turn < 3; turn++) + await new Promise((resolve) => setTimeout(resolve, 0)) + } + const detail = categorySubset(`detail`) + + await collection._sync.loadSubset(detail) + collection._sync.unloadSubset(detail) + await storage.entered + const reacquisition = Promise.resolve( + collection._sync.loadSubset(detail), + ).then( + () => `fulfilled`, + (error: unknown) => + error instanceof Error ? error.message : String(error), + ) + await settleTurn() + const held = { rows: rows(collection), status: collection.status } + + storage.release() + const reacquired = await reacquisition + await settleTurn() + const failed = { rows: rows(collection), status: collection.status } + + await collection.cleanup() + await collection.preload() + await collection._sync.loadSubset(detail) + const restartedLoad = rows(collection) + collection._sync.unloadSubset(detail) + await settleTurn() + + expect({ + held, + reacquired, + failed, + restartedLoad, + restartedRelease: { + rows: rows(collection), + stored: storedItems(storage).map((item) => item.id), + }, + }).toEqual({ + held: { rows: [shared.id], status: `ready` }, + reacquired: `Failed to durably persist collection "${id}": storage failed`, + failed: { rows: [], status: `error` }, + restartedLoad: [shared.id], + restartedRelease: { rows: [], stored: [] }, + }) + }) + /** * Accepted Query results update ownership in acceptance order even while * publication is held. The independent model stores each active Query's @@ -6430,6 +7205,92 @@ describe(`query collection ownership lifecycle`, () => { }, ) + /** + * The same law holds when the Query's current result is empty. An empty + * result is a resolved scope that owns no rows; it is not an unapplied + * result. The reference sequence is the first fetched result (no rows), the + * accepted Manual insert, and the later fetched Newer row. Starting the + * refetch repeats the empty cached success; it cannot remove the direct row + * or claim it. + * + * This sibling exists because the reacquisition table above makes a + * released lease's empty ownership entry distinguishable from an applied + * one. A repair that instead treats every empty ownership entry as + * unapplied passes that table and the non-empty history above. Here it + * replays the empty cached result at refetch start: in on-demand mode the + * insert disappears at its own receipt, and in eager mode the later Newer + * result does not replace Manual. The checkpoints and holds match the + * history above. + */ + it.each([`eager`, `on-demand`] as const)( + `keeps an accepted persisted %s insert through a refetch start of an empty result`, + async (syncMode) => { + const id = `persisted-direct-insert-empty-refetch-start-${syncMode}` + const storage = createOwnershipStorage(undefined, 2) + const queryClient = createQueryClient() + const newer = { ...shared, name: `Newer` } + const manual = { ...shared, name: `Manual` } + const heldResult = createDeferred>() + const requestStarted = createDeferred() + const queryFn = vi + .fn() + .mockResolvedValueOnce([]) + .mockImplementationOnce(() => { + requestStarted.resolve() + return heldResult.promise + }) + const collection = createCollection( + persistedCollectionOptions< + Item, + string | number, + never, + QueryCollectionUtils + >({ + ...queryCollectionOptions({ + id, + queryClient, + queryKey: [id], + queryFn, + getKey: (item) => item.id, + syncMode, + persistedGcTime: Number.POSITIVE_INFINITY, + startSync: true, + }), + persistence: { adapter: storage.adapter }, + }), + ) + cleanups.push(async () => { + storage.release() + heldResult.resolve([newer]) + await collection.cleanup() + queryClient.clear() + }) + const observe = () => ({ + public: collection.get(shared.id)?.name ?? null, + durable: storedItems(storage)[0]?.name ?? null, + }) + + if (syncMode === `eager`) await collection.stateWhenReady() + else await collection._sync.loadSubset({}) + expect(observe()).toEqual({ public: null, durable: null }) + + const write = collection.utils.writeInsert(manual) + await storage.entered + const refetch = queryClient.refetchQueries({ queryKey: [id] }) + void refetch.catch(() => undefined) + await requestStarted.promise + expect(queryFn).toHaveBeenCalledTimes(2) + + storage.release() + await write + expect(observe()).toEqual({ public: manual.name, durable: manual.name }) + + heldResult.resolve([newer]) + await refetch + expect(observe()).toEqual({ public: newer.name, durable: newer.name }) + }, + ) + it(`rejects writes from an invalidated sync run and admits writes after restart`, async () => { const id = `direct-write-cleanup-admission` const storage = createOwnershipStorage()