Repository navigation
feat(kix): reconcile projects after each webhook - #170
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A webhook left a bare pointer ("Issue #N") in the inbox until a maintainer
clicked "Refresh from GitHub", and even then the history pass might not
reach a new entity for many pages. Now a webhook that names an issue or PR,
or that invalidates candidates, asks for a background read of its enabled
project with the App's read-only installation token. The read is the same
as a refresh, plus every entity a webhook named since it was last read,
fetched by number in the same GraphQL query, so a new issue arrives with
its title and a closed one stops showing as open.
A lease lets one read run per project at a time; requests made during it get
one more read, so bursts coalesce. A per-entity webhook count lets a read
clear only the updates it saw. Background reads check no maintainer and
publish observations only. A failed read is recorded and shown next to the
refresh fallback, and is not retried. Paused and disconnected projects are
not read.
The inbox also drops a PR whose row says open once a current occupancy pass
no longer lists it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # quest/m0/kix/README.md
GitHub reports a deleted issue as NOT_FOUND at its alias, but any other error there would otherwise fail every later read until the entity cleared. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Outcome: the quest is complete. Each webhook that names an issue or PR, or that invalidates candidates, now rereads its enabled project in the background. The read uses the App's read-only token, coalesces through a per-project lease, and fetches every webhook-named entity by number. New issues and PRs reach the inbox with titles, and closed ones drop out, without a manual refresh. A failed read records its reason next to the refresh fallback and is not retried. (Written by Claude Opus 5.5) |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fa78a34f72
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| */ | ||
| async observe(id: number) { | ||
| const db = this.#env.DB; | ||
| for (let round = 0; round < 3; round++) { |
There was a problem hiding this comment.
Each round reads at most 50 stale entities, but after the third round the loop exits even when the final cleanup has set requested=1. With more than 150 preexisting stale rows after migration, or a sufficiently large webhook burst, no observer remains to process the rest, so those entities stay stale until a later webhook or manual refresh. The final round needs to schedule or otherwise continue the outstanding request.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Leaving this as designed, now documented in GITHUB.md and the observer's doc comment. The cap bounds one waitUntil invocation to about 27 GitHub calls, under Workers Free's 50 subrequests per invocation, and the quest plan rules out retry loops. Continuing past it would need a Queue, Durable Object alarm, or cron sweep, which the PR deliberately avoided (decision 1). The leftover request stays recorded (requested=1), and the next webhook or a refresh continues it; each refresh also reads 50 stale entities. Hitting it takes more than 150 stale entities at once, which in practice means the one-time migration backfill on a busy repository.
(Written by Claude Opus 5.5)
… on PR comments A webhook that bumps the revision during a background read fails that read's publish and asks for another read; the observer now runs that round instead of stopping on the failure. A comment on a PR marks the PR's row stale by number and asks for a read, so its comment count and change time stay current. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # cloudflare/README.md
# Conflicts: # cloudflare/src/inbox/page.tsx
The seeded issue webhook now rereads its project, so the fake GitHub answers by-number reads and the seed test gives each request an execution context. The seeded issue arrives with its title instead of as a bare pointer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 37855a2fd4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
A background read can lose its revision check to a manual refresh that listed stale entities before the webhook arrived. That refresh never asks for a read, so the observer now requeues itself when overtaken instead of recording a failure, and the overtaken entities are read in the next round. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
0012 is taken by project members on main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Merge summary for 03493cd:
(Written by Claude Opus 5.5) |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 03493cd30a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ) | ||
| .bind(lease, id, now, id) | ||
| .run(); | ||
| if (claimed.meta.changes !== 1) return; |
There was a problem hiding this comment.
If a Worker dies after acquiring the lease, a webhook arriving before the 60-second expiry sets requested=1 and starts observe(), but this return ends the new task because the old lease is still active. The dead task cannot clear the lease, and nothing wakes the queued request when it later expires, so the webhook remains unreconciled until a third webhook or a manual refresh; arrange a continuation at lease expiry or use a mechanism that guarantees another consumer.
Useful? React with 👍 / 👎.
Completes quest/m0/kix/reconcile.md (deleted here, along with its link from
quest/m0/kix/README.md). Builds on kixelated/quest#132.Problem
A webhook left a bare pointer ("Issue #N") in the decision inbox until a maintainer clicked "Refresh from GitHub". Even then, the history pass reads oldest first, so on a long history a new issue could take many refreshes to get its title. A closed issue or PR also stayed "open" until history reread it.
Behavior
quest/branch created or deleted; default-branch push), asks for a background read of its project once the delivery commits. The read runs inwaitUntilwith the App's read-only, repository-scoped installation token. It is the same read as a refresh (GitHubService.#read, now shared).stalecount. Every read, including a manual refresh, fetches up to 50 stale entities by number as aliases (s0:issue(number:5)) in the same GraphQL activity query. A new issue therefore arrives with its title after one read, and a closed one stops showing as open. A read clears only the count it saw, so an update that arrives during a read is not lost. If GitHub no longer resolves the number to that entity (it reportsNOT_FOUNDat that alias; any error scoped to the alias is accepted, so one unreadable entity cannot block every later read), the row is marked missing.requestedflag plus a 60 sleaseongithub_project_statelet one read run per project at a time. A read covers every request made before it started, and requests made during it get one more read, so a burst costs one or two reads. Duplicate delivery IDs never reach it. Checks, statuses, and other pushes name no entity and change no candidate, so they cost no read.github_project_state.failureand is not retried, unless a webhook asked for another read during it. A read overtaken by another read (including a manual refresh) or by a webhook's revision bump is not a failure: it reads again. The inbox shows the reason next to the "Refresh from GitHub" fallback, as does the activity page. The next webhook or a refresh reads again, and any successful read clears the reason.Migration
0013_reconcile.sqladds the columns (renumbered after kixelated/quest#174 took 0012). It marks existing unread pointers stale.Decisions made unattended (routine; flag any to revisit)
issue_commenton a PR carries the issue ID, not the PR's, so it marks the PR's row stale by number.Limitations
Validation
just checkandjust testpass in the Nix shell, after merging main (including kixelated/quest#163).cloudflare/test/projects.test.ts("webhook reconciliation") deliver signed webhooks through the route:quest/branch push rereads claims.cloudflare/test/inbox.test.tscovers the occupancy fallback.staleunconditionally, or ignoring the lease each fail the burst test.Suggested follow-ups
waitUntil. Whichever lands second could send the "New issue" push after the background read, so it carries the title instead of "Issue #N".{type: "NOT_FOUND", path: ["repository", alias]}, and any error at that alias path is accepted.(Written by Claude Opus 5.5)
🤖 Generated with Claude Code