Skip to content

fix(file-search): claim a file write that lands on an owner the backfill already queued - #9666

Merged
waleedlatif1 merged 2 commits into
stagingfrom
fix/file-search-write-after-reconciliation
Nov 4, 2026
Merged

waleedlatif1 merged 2 commits into
stagingfrom
fix/file-search-write-after-reconciliation

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • Follow-up to fix(file-search): claim owners woken by a file write ahead of backfill reconciliation #9655, from review on the release PR. Backfill reconciliation wake-ups enter the owner queue as if just dispatched, so they rotate in behind owners woken for a write. A file write that landed while such a row was still queued only advanced its updated_at (the file trigger's ON CONFLICT upsert), so it kept the reconciliation stamp and waited behind every older-stamped owner — during a long walk, several passes. The same held for an owner that was served, left queued with work, and then woken again.
  • Claims now take owners woken since their last dispatch first: never dispatched, or updated_at > last_dispatched_at (wokenSinceDispatch). Every real wake-up advances updated_at (file write, released claim, reaped claim, artifact-ready requeue); a reconciliation wake-up never does — it now uses ON CONFLICT DO NOTHING, so it leaves an already queued owner untouched. Within each group the dispatch-age rotation is unchanged, so fairness is unchanged.
  • The two compared columns come from one clock: the dispatcher and artifact-ready write them with the transaction's now(), as the trigger does. The remaining window is a write whose transaction began before the claim that stamped its owner; it waits one rotation by dispatch age (documented on wokenSinceDispatch).
  • No migration, no trigger change, caps unchanged. The claim's queue select no longer follows file_search_dispatch_schedule_idx and sorts the queue instead (hundreds to low thousands of narrow rows, once a minute, row locks still only on returned rows).

Type of Change

  • Bug fix

Testing

  • dispatcher.integration.ts (real PostgreSQL, writes through the real workspace_file_search_mark_pending trigger):
    • serves a file write to %s ahead of a backfill page of reconciliation wake-ups — now table-driven; the new "owner the page left queued" case is red on the old code and without the wokenSinceDispatch ordering
    • keeps a queued reconciliation wake-up behind woken work when the backfill passes it again — red on the old code, without the ordering, and on a mutant whose reconciliation upsert bumps updated_at
    • leaveSlots helper replaces three copies of the capacity setup
  • Suite 39/39; also owners.integration 18/18, chunks.integration 49/49 (1 opt-in skip), index-plan.integration 9/9, content-rendered.integration 31/31, lib/workspace-files/search unit tests 178/178
  • lint and type-check (pre-push hooks); remaining audits in CI
  • Note: dispatch work check > reports work for a backfill reconcile that is due compares a DB-clock completed_at against a JS Date exactly at the reconcile threshold and can flake by milliseconds; untouched here

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

…ill already queued

Backfill reconciliation wake-ups enter the owner queue as if just dispatched so they rotate in
behind owners woken for a write. A write that landed while such a row was still queued only
advanced its updated_at through the file trigger's upsert, so it kept the reconciliation stamp
and waited behind every older-stamped owner. The same held for an owner that was served, left
queued with work, and then woken again.

Claims now take owners woken since their last dispatch first: never dispatched, or updated_at
later than last_dispatched_at, which every real wake-up (file write, released claim, reaped claim,
artifact-ready requeue) advances and a reconciliation wake-up never does; it no longer touches an
owner already queued. Within each group the dispatch-age rotation is unchanged. Queue timestamps
the comparison reads come from the transaction clock, the same clock the trigger writes.
@vercel

vercel Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Nov 4, 2026 10:03am UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 4 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no new actionable issues were found.

Summary

The PR gives file writes priority over backfill wake-ups, including writes to owners already queued.

  • File-search claims pick up recently woken owners before reconciliation-only work.

Reviews (2) · Last reviewed commit: "fix(file-search): rotate a woken owner w..." · Reviewed by Greptile

@greptile-apps

This comment has been minimized.

…ch age

An owner already holding both of its outstanding claims can claim nothing, but a write between
passes kept marking it woken, so with little global capacity left it took every owner turn and
the free slot went unused. Woken owners now go first only while they have a free slot; a full
owner keeps the dispatch-age rotation it had before.

Also gives the due-reconcile work-check case the same one-minute margin as its not-yet-due
counterpart: exactly at the interval it compared a database timestamp with the application
clock and flaked on skew.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 4 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1
waleedlatif1 merged commit 158f5fa into staging Nov 4, 2026
80 checks passed
@waleedlatif1
waleedlatif1 deleted the fix/file-search-write-after-reconciliation branch November 4, 2026 18:13

This branch was previously deployed

1 inactive deployment
Preview — c548150b Deployed Nov 4, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant