Skip to content

test(e2e): split e2e.test.ts into one file per area - #387

Merged
willgosnold merged 3 commits into
mainfrom
test/split-e2e
Oct 27, 2026
Merged

willgosnold merged 3 commits into
mainfrom
test/split-e2e

Conversation

@willgosnold

@willgosnold willgosnold commented Oct 27, 2026 •

Copy link
Copy Markdown
Owner

Part of #382 (step 4)

Problem

test/e2e.test.ts is 19,030 lines and holds 1,547 tests. To run the area you are changing you have to write a -t pattern by hand. A diff or a review of it is hard to read. CI shards it by test name (^ForkTest , ^VendorTest , the rest), so the shard a test lands in depends on its describe title and not on where it lives.

Change

The file is split into test/e2e/*.test.ts and one shared module, test/e2e-fixtures.ts. Nothing was added, renamed or rewritten. Every test keeps its describe path and title, so full names, CI history and coverage keys are unchanged.

New file What it holds Tests
test/e2e/usage.test.ts --version, help, usage errors 3
test/e2e/clone-init.test.ts clone and init: destinations, remotes, URLs, --path, a .PATCHBOOK folder upstream, "init stopped by a preload" 112
test/e2e/skill.test.ts skill install and the skill that clone and init write 76
test/e2e/credentials.test.ts URLs with a password are refused, and a remote's password is never printed 62
test/e2e/hook.test.ts hook install and hook status, lefthook, husky, pre-commit and Vite+ configs 146
test/e2e/post-hook.test.ts PostHookTest: post-merge, post-checkout and post-rewrite 40
test/e2e/fork.test.ts ForkTest 521
test/e2e/vendor.test.ts VendorTest 369
test/e2e/intent.test.ts IntentWithoutAModelTest, StaleIntentTest 128
test/e2e/mode.test.ts ModeTest, ExecBitTest 66
test/e2e/write-failure.test.ts ReadOnlyLockTest, RecordRollbackTest 12
test/e2e/merge.test.ts MergeTest 10
test/e2e/crash.test.ts CrashInjection 2

ForkTest and VendorTest stay whole. Their direct children are interleaved with local helpers, so splitting them would need a second shared-helper layer.

test/e2e-fixtures.ts holds the 33 top-level declarations that more than one file uses, such as ForkCase, VendorCase, forkTest, vendorTest, startingRepo and tree. A declaration only one file uses stays in that file.

CI. The shards now name files. fork runs test/e2e/fork*.test.ts, vendor runs test/e2e/vendor*.test.ts, and rest runs every other file in test/e2e/, so a new file lands in rest with no CI change. The rest shard runs test/*.test.ts from source, and runs the "so only from source" describes with --test-name-pattern over the whole folder. Bun exits 1 if that pattern matches nothing, which I checked. The job names, the root uid check, the crash-build check and the required job are unchanged. package.json and release.yml still run bun test --isolate, which finds 20 files and 2,327 tests, the same 2,327 as before.

Docs. CONTRIBUTING.md names the new command and the new rule for where a CLI test goes. No other tracked doc, qa/ file or verify-patchbook file mentions the old path. Vendored skills and the CHANGELOG are untouched.

How the files were made. A codemod wrote them, not hand edits. It is ~/.local/state/pstack/patchbook/issue-382/split-e2e.ts on my machine, outside the repo. It parses test/e2e.test.ts with the TypeScript parser. A table maps each top-level test block to a file by start line, and the script checks each row against the block's first text. It places every other declaration by where it is used, through a symbol-resolved dependency graph, and rebuilds each file's imports from that. Run on a fresh origin/main checkout it produces this PR's test/ tree byte for byte (diff -r is empty). It makes these changes beyond moving text:

  • It adds export to the shared declarations.
  • It moves the import lines one folder down (./helpers becomes ../helpers).
  • It adds a .. to five import.meta.dir paths.
  • It drops the one-line file header comment, which described a file that no longer exists.
  • It writes each file's imports from the names that file uses, adding ../e2e-fixtures.ts.
  • It puts one blank line between blocks that were not next to each other.

The hand edits are ci.yml and CONTRIBUTING.md.

Verification

  • bun run typecheck passes.
  • Whole e2e folder against origin/main's single file, run back to back. From source, both give 1,546 pass, 1 skip, 0 fail, 7,996 expect() calls. Against dist/patchbook, both give 1,534 pass, 13 skip, 0 fail, 7,953 expect() calls.
  • Full test names (describe path, title and status from --reporter=junit), sorted and diffed between origin/main and this head: empty diff, from source and against the binary. No full name repeats.
  • Every test is in exactly one shard. I ran the three file sets of the new ci.yml against the binary at the same time. They ran 521 (fork), 369 (vendor) and 657 (rest) tests, 1,547 in total. The union of their names equals the full set and no name is in two shards. The shard script from ci.yml, run with a stub bun, picks the same file lists.
  • Coverage with pbcov measure and pbcov diff. Against origin/main (0d4ee20) it lost 0 statements, gained 0, missed 0 tests, added 0 tests, and 1,534 e2e tests moved file. Both sides cover 5,569 of 5,761 statements (96.67%). Against the 82fd7f6 baseline it also lost 0 statements. That baseline is from before test(e2e): merge duplicate tests #384 and test: move input matrices from e2e to unit tests #385, so it lists the 157 tests those two PRs merged, renamed or moved to unit tests.
  • Test-seconds per file, from one serial run from source (fork 171.0, vendor 152.1, intent 53.6, mode 29.7, hook 28.6, skill 19.9, clone-init 17.9, post-hook 13.7, credentials 6.7, write-failure 4.1, merge 3.0, crash 0.2, usage 0.1) which sum to 501 s.
  • Speed-up. bun test 1.4.3 runs test files one after another unless you pass --parallel. So the split itself buys no speed-up, and I claim none. Back to back against the binary: the single file took 744.7 s and 743.9 s, the folder 739.4 s. With --parallel the folder took 166.4 s, and 166.7 s with --parallel=8. Those two match because the longest file, fork.test.ts, takes about 140 to 170 s alone. CI does not use --parallel here. It would be a separate change.
  • I did not run macOS or the root user here. I dispatch ci.yml on this branch so macOS runs the new layout, and the root jobs run on the PR.

Written with Claude Sonnet 5.5 in Claude Code

🤖 Generated with Claude Code

willgosnold and others added 3 commits October 27, 2026 20:29
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@willgosnold
willgosnold merged commit 1532a45 into main Oct 27, 2026
18 checks passed
@willgosnold
willgosnold deleted the test/split-e2e branch October 27, 2026 21:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant