Repository navigation
test(e2e): split e2e.test.ts into one file per area - #387
Merged
Merged
Conversation
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #382 (step 4)
Problem
test/e2e.test.tsis 19,030 lines and holds 1,547 tests. To run the area you are changing you have to write a-tpattern by hand. A diff or a review of it is hard to read. CI shards it by test name (^ForkTest,^VendorTest, the rest), so the shard a test lands in depends on its describe title and not on where it lives.Change
The file is split into
test/e2e/*.test.tsand one shared module,test/e2e-fixtures.ts. Nothing was added, renamed or rewritten. Every test keeps its describe path and title, so full names, CI history and coverage keys are unchanged.test/e2e/usage.test.ts--version,help, usage errorstest/e2e/clone-init.test.tscloneandinit: destinations, remotes, URLs,--path, a.PATCHBOOKfolder upstream, "init stopped by a preload"test/e2e/skill.test.tsskill installand the skill thatcloneandinitwritetest/e2e/credentials.test.tstest/e2e/hook.test.tshook installandhook status, lefthook, husky, pre-commit and Vite+ configstest/e2e/post-hook.test.tsPostHookTest: post-merge, post-checkout and post-rewritetest/e2e/fork.test.tsForkTesttest/e2e/vendor.test.tsVendorTesttest/e2e/intent.test.tsIntentWithoutAModelTest,StaleIntentTesttest/e2e/mode.test.tsModeTest,ExecBitTesttest/e2e/write-failure.test.tsReadOnlyLockTest,RecordRollbackTesttest/e2e/merge.test.tsMergeTesttest/e2e/crash.test.tsCrashInjectionForkTestandVendorTeststay whole. Their direct children are interleaved with local helpers, so splitting them would need a second shared-helper layer.test/e2e-fixtures.tsholds the 33 top-level declarations that more than one file uses, such asForkCase,VendorCase,forkTest,vendorTest,startingRepoandtree. A declaration only one file uses stays in that file.CI. The shards now name files.
forkrunstest/e2e/fork*.test.ts,vendorrunstest/e2e/vendor*.test.ts, andrestruns every other file intest/e2e/, so a new file lands inrestwith no CI change. Therestshard runstest/*.test.tsfrom source, and runs the "so only from source" describes with--test-name-patternover the whole folder. Bun exits 1 if that pattern matches nothing, which I checked. The job names, the root uid check, the crash-build check and therequiredjob are unchanged.package.jsonandrelease.ymlstill runbun test --isolate, which finds 20 files and 2,327 tests, the same 2,327 as before.Docs. CONTRIBUTING.md names the new command and the new rule for where a CLI test goes. No other tracked doc,
qa/file orverify-patchbookfile mentions the old path. Vendored skills and the CHANGELOG are untouched.How the files were made. A codemod wrote them, not hand edits. It is
~/.local/state/pstack/patchbook/issue-382/split-e2e.tson my machine, outside the repo. It parsestest/e2e.test.tswith the TypeScript parser. A table maps each top-level test block to a file by start line, and the script checks each row against the block's first text. It places every other declaration by where it is used, through a symbol-resolved dependency graph, and rebuilds each file's imports from that. Run on a freshorigin/maincheckout it produces this PR'stest/tree byte for byte (diff -ris empty). It makes these changes beyond moving text:exportto the shared declarations../helpersbecomes../helpers)...to fiveimport.meta.dirpaths.../e2e-fixtures.ts.The hand edits are
ci.ymlandCONTRIBUTING.md.Verification
bun run typecheckpasses.origin/main's single file, run back to back. From source, both give 1,546 pass, 1 skip, 0 fail, 7,996expect()calls. Againstdist/patchbook, both give 1,534 pass, 13 skip, 0 fail, 7,953expect()calls.--reporter=junit), sorted and diffed betweenorigin/mainand this head: empty diff, from source and against the binary. No full name repeats.ci.ymlagainst the binary at the same time. They ran 521 (fork), 369 (vendor) and 657 (rest) tests, 1,547 in total. The union of their names equals the full set and no name is in two shards. The shard script fromci.yml, run with a stubbun, picks the same file lists.pbcov measureandpbcov diff. Againstorigin/main(0d4ee20) it lost 0 statements, gained 0, missed 0 tests, added 0 tests, and 1,534 e2e tests moved file. Both sides cover 5,569 of 5,761 statements (96.67%). Against the82fd7f6baseline it also lost 0 statements. That baseline is from before test(e2e): merge duplicate tests #384 and test: move input matrices from e2e to unit tests #385, so it lists the 157 tests those two PRs merged, renamed or moved to unit tests.fork171.0,vendor152.1,intent53.6,mode29.7,hook28.6,skill19.9,clone-init17.9,post-hook13.7,credentials6.7,write-failure4.1,merge3.0,crash0.2,usage0.1) which sum to 501 s.bun test1.4.3 runs test files one after another unless you pass--parallel. So the split itself buys no speed-up, and I claim none. Back to back against the binary: the single file took 744.7 s and 743.9 s, the folder 739.4 s. With--parallelthe folder took 166.4 s, and 166.7 s with--parallel=8. Those two match because the longest file,fork.test.ts, takes about 140 to 170 s alone. CI does not use--parallelhere. It would be a separate change.ci.ymlon this branch so macOS runs the new layout, and the root jobs run on the PR.Written with Claude Sonnet 5.5 in Claude Code
🤖 Generated with Claude Code