Skip to content

fix(github): primary GraphQL limits, refused installation tokens, scan-free cache eviction - #341

Open
sweepies wants to merge 1 commit into
mainfrom
fix/github-limits-cache
Open

sweepies wants to merge 1 commit into
mainfrom
fix/github-limits-cache

Conversation

@sweepies

@sweepies sweepies commented Nov 15, 2026 •

Copy link
Copy Markdown
Owner

What and why

  1. GraphQL primary limit read as a secondary one (crates/github/src/governor.rs). GitHub answers an exhausted GraphQL primary limit with HTTP 200, a RATE_LIMITED error and x-ratelimit-remaining: 0. The governor treated every such answer as a secondary limit: it guessed a 60 s cooldown and cut writes to one a second for an hour. Now a RATE_LIMITED answer with a spent window, and no error message naming the secondary limit, counts as primary exhaustion. The graphql bucket waits until x-ratelimit-reset and writes are left alone. Answers without the header, or with "secondary rate limit" in a message, still take the secondary path. The classification lives in a small GraphqlLimit helper.
  2. Refused installation tokens reused until they expire (app.rs, credentials.rs, lib.rs). A new Credentials::invalidate(rejected) hook defaults to None, meaning "can't mint another token, don't retry". App installations implement it by clearing the full-installation slot, but only while it still holds the refused token, so a token another request minted in the meantime is kept. On a 401, exchange invalidates the token and retries once. GitHub runs nothing it refuses with 401, so this is safe for writes too. The retry doesn't count against the rate-limit retries, and a second 401 is returned.
  3. MAX_WRITE_BURST defined twice. Now there is one definition, in patchula-config, and patchula-github uses it. I picked the direction github → config because every crate that depends on github already depends on config, so nothing gains a new transitive dependency. The other direction would pull reqwest, SQLite and jsonwebtoken into config and every crate that uses it. The unused pub constant in github is gone.
  4. Dead search/ branch in exchange: removed. Nothing calls search, and on an Enterprise root the branch couldn't match. The governor still keys buckets by x-ratelimit-resource.
  5. Every HTTP cache write summed the whole http table (crates/cache/src/store.rs). Migration 016 adds a one-row http_bytes table. It is seeded from the current sum, and AFTER INSERT/DELETE/UPDATE OF body triggers on http maintain it. Several processes can share the cache file (the CLI's default directory is shared, and the cache uses WAL), so the counter lives in the database and changes inside each write's own transaction. A per-Store counter would drift. evict() re-reads the row after each delete. A rebuilt cache after corruption recovery starts at 0 on an empty table. cache schema is now 16.

Tests

  • governor: a spent GraphQL window is primary (bucket reset ≈ x-ratelimit-reset, no secondary cooldown, no write throttle). A RATE_LIMITED answer without a spent window, without headers, or with a secondary message stays secondary. A GraphQL answer that spends the last point without an error is not limited.
  • app: a refused token is minted again and the write retried (2 mints, 2 sends, and later requests keep the new token). A refused fresh token fails after one retry with no mint loop. Invalidating an already-replaced token keeps the replacement.
  • cache: the counter matches SUM(length(body)) after insert, a resize through upsert, a 304 refresh, removal and eviction, including writes through a second handle on the same file. The table stays within budget over 40 puts. The migration test checks that 016 seeds the counter from existing rows. rewind drops 016's objects.
  • cargo test -p patchula-github -p patchula-cache, the state/derive parity tests in datasource, notes, engine and update, and cargo clippy -p patchula-github -p patchula-cache --all-targets -D warnings all pass.

State versions

crates/github/src/ is hashed by source-head, tag-catalog, tag-object and release-notes. Bumping those cascades through crates/datasource/src/ and crates/notes/src/ to catalog, catalog-controls, changelog and notes-adapter, and to the derived github-release-head, notes-adapter-listing, parsed-postgres-sgml and parsed-go-history. All are bumped by one, with their rows updated. Expect a one-time miss on those entries.

Dropped

Nothing.

Follow-ups

  • The scoped (per-repository) tokens handed to tools aren't invalidated on 401. Tools use them in their own processes, so the client never sees those 401s.
  • The App's own repos/{owner}/{repo} read for scoped mints (App::repository_id) uses the installation token directly, without the 401 retry.

View guided diff Turn on auto-fix

…n-free cache eviction

- A GraphQL 200 with RATE_LIMITED and x-ratelimit-remaining: 0 is the
  primary limit: wait for x-ratelimit-reset on the graphql bucket instead
  of the secondary cooldown and an hour of one write a second.
- A 401 to App installation credentials drops the cached token
  (Credentials::invalidate) and retries once with a fresh one.
- MAX_WRITE_BURST has one definition, in patchula-config.
- Drop the unreachable search resource branch.
- Cache eviction reads a trigger-kept byte total (migration 016) instead of
  summing every body after each write.

Bumps the states and derived representations whose hashed sources cover
crates/github/src/, and those the bumps cascade to.
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
count-patchula 3351f91 Commit Preview URL

Branch Preview URL
Nov 15 2026, 10:56 PM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant