Skip to content

Make every Worker test pass on its own - #208

Merged
kixelated merged 5 commits into
mainfrom
quest/m0/worker-tests
Oct 24, 2026
Merged

kixelated merged 5 commits into
mainfrom
quest/m0/worker-tests

Conversation

@kixelated

@kixelated kixelated commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Finishes quest/m0/worker-tests.md, a follow-up of kixelated/quest#188.

Summary

  • Root cause: the Worker tests run in one shared runtime (isolate: false) against one D1 database migrated once in test/setup.ts, and @cloudflare/vitest-plugin 1.x no longer isolates storage per test. Each file cleaned up a hand-picked list of tables with DELETE FROM, so any table (users, accounts, sessions, verifications, R2 objects, Durable Objects) left off a file's list carried over into later tests.
  • Fix: test/setup.ts now calls reset() from cloudflare:test (it deletes every Durable Object, which also backs Miniflare's D1 and R2) and reapplies the migrations before each test. So every test starts from empty, freshly migrated storage.
  • Cleanup: the per-file DELETE FROM lists in beforeEach are gone, and seeds that used INSERT OR IGNORE to tolerate leftovers are now plain INSERT, so a leak would fail loudly. This includes the second-user insert from kixelated/quest#188. Mid-test deletes that are part of what a test checks stay.
  • CI check: vitest.config.ts sets sequence.shuffle: true, so every run, local or CI (just test → just worker-test), shuffles both files and tests and prints its seed. A new order dependence fails there; rerun it with vitest run --sequence.seed=<seed>.
  • The full suite didn't get slower: 26s with the reset, against 31s before on the same host.

Verification

  • Each of the 267 test locations (file:line; an it.each table counts as one location, and its cases still reset between them) ran alone in its own vitest process: 267/267 passed.
  • Full suite, shuffled with seeds 1, 2 and 3: 333/333 passed each time, plus unshuffled runs before and after the cleanup.
  • A probe test (not committed) confirmed that reset() drops a D1 table and an R2 object written by the previous test, and that the migrations are reapplied.
  • prettier --check, tsc --noEmit (cloudflare) and quest check pass. The full just check / just test waited more than 30 minutes behind quest gate on a host loaded by about 15 parallel agents, so I left the rest of those to CI.

Choices made without asking

  • Reset in a global beforeEach in setup.ts rather than per file, so a new test file is isolated by default.
  • reset() plus reapplying the migrations, rather than deleting rows from every table. It also clears R2 and Durable Objects, needs no table list, and measured no slower.
  • Shuffle on every run, rather than adding a separate shuffled CI job. One run covers both, and the printed seed reproduces any failure.
  • The per-test-alone proof was a one-off script and isn't committed: running each case alone costs a vitest startup per test (about 267 processes), which is too slow for CI. The reset plus shuffle is the standing check.

Suggested follow-ups

  • None required. If a module-level JS fixture ever leaks between tests (the reset covers storage only), a periodic job that runs each test alone could catch it. Not needed today: all 267 locations pass alone.

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

kixelated and others added 3 commits October 23, 2026 23:03
Quest-Claim: 563286ff8dda6b06
Each Worker test now starts from empty, freshly migrated storage (D1, R2,
and Durable Objects via cloudflare:test reset()), so none depends on rows
another test left behind. The per-file DELETE FROM cleanups and
INSERT OR IGNORE seeds become plain setup. Vitest shuffles files and tests
on every run, so CI catches a new order dependence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review October 24, 2026 07:05
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-24T16:33:52.240062Z ebbe689 New commits
🔒 Security Review ✅ Completed 2026-10-24T16:34:56.927713Z ebbe689 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Resolve the quest/m0/README.md conflict by dropping both finished quests (lifecycle-flakes merged in #218, worker-tests finishes here).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Merged main into the branch (ac5fee2), no rebase.

  • Conflict: quest/m0/README.md. Both sides removed a finished quest from Required: kixelated/quest#218 removed Lifecycle flakes, and this PR removes Worker tests. The resolution drops both lines. Everything else merged cleanly.
  • Leftovers from the newly merged tests: none. Every DELETE FROM still in cloudflare/test is a mid-test step that the test checks, such as a lost delivery record, a disconnected project, or a finished job. The ON CONFLICT upserts in project-fixtures.ts let tests switch a project between paused and enabled within one test, so they aren't there to tolerate leftover rows. No INSERT OR IGNORE remains.
  • Verification: the full Worker suite passed shuffled with seeds 1 to 5 (17 files, 347/347 tests each), and just check and just test both passed.

(Written by Claude Opus 5.5)

Resolve the seed.test.ts conflict by keeping main's seeded(edit) signature and dropping its DELETE FROM batch, which the global reset in test/setup.ts replaces.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Merged main again (ebbe689) after kixelated/quest#217 landed.

  • Conflict: cloudflare/test/seed.test.ts. kixelated/quest#217 added an edit parameter to seeded(), and that function still began with a DELETE FROM batch over jobs, runners, projects, and installations. The resolution keeps main's seeded(edit) signature and drops the batch, since the global reset in test/setup.ts already gives every test empty storage.
  • Verification: the Worker suite passed shuffled with seeds 1 to 5 (348/348 tests each), and just check and just test both passed.

(Written by Claude Opus 5.5)

@kixelated
kixelated merged commit ebea7ec into main Oct 24, 2026
4 checks passed
@kixelated
kixelated deleted the quest/m0/worker-tests branch October 24, 2026 16:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant