Skip to content

ci: shard the root test job - #383

Merged
willgosnold merged 2 commits into
mainfrom
ci/shard-root-job
Oct 27, 2026
Merged

willgosnold merged 2 commits into
mainfrom
ci/shard-root-job

Conversation

@willgosnold

@willgosnold willgosnold commented Oct 27, 2026 •

Copy link
Copy Markdown
Owner

Part of #382

Problem

test (root) takes about 17 minutes (16.5, 16.9 and 16.7 in PR runs 40413389882, 40540925012 and 40542953264). The Ubuntu shards take about 3. The required check test (ubuntu-latest) needs both, so every PR waits about 17 minutes.

The root job runs bun test --isolate, which is the whole suite from source with e2e included. Then it runs the whole e2e suite again against the binary. It is one unsharded job.

Change

The user is now a matrix axis of the test job (runner and root), so root runs the same three shards as everyone else and the shard list stays in one place. The test-root job is gone. Job names gain the user, for example test (ubuntu-latest, root, fork).

  • Root runs only the test step as root. Checkout, install, typecheck and build stay as the runner user. The step asserts that the process is root for the root jobs and not root for the others.
  • e2e runs once per user, against the compiled binary, sharded by the same three name patterns.
  • The rest shard also runs from source the unit files and the e2e describes that skip under PATCHBOOK_CMD, selected by --test-name-pattern "so only from source". That is 12 tests: 8 in "init stopped by a preload" and 4 in "killed by a preload" under VendorTest. They now also run as the runner user. Before, they ran only as root, and never on macOS.
  • macOS runs on pushes and manual runs, for the runner user only.
  • The crash-injection check on the binary still runs in every shard.
  • required keeps the name test (ubuntu-latest) and needs only test.
  • A test that needs a refused write and is not wrapped in permissionTest still fails as root.

Verification

CI wall time on a PR, from real runs:

before after
PR run, start to finish 16.5 to 16.9 min (runs 40413389882, 40540925012, 40542953264) 3.2 min (run 40557914038)
Slowest job test (root), 16.5 to 16.9 min test (ubuntu-latest, runner, rest), 3.1 min
  • As root against the binary, the three root shards pass 1622 e2e tests. The old root job passed the same 1622.
  • The 7 unit files pass as root (628 tests). The from-source e2e run as root is gone, as step 1 of the issue asks, except the 12 tests that need the source.
  • A manual run on this branch (40557919901) passed all 9 jobs, macOS included, so the 12 from-source tests pass on macOS as the runner user.
  • actionlint 1.7.12, without shellcheck, reports nothing. The expanded matrix has 6 jobs for pull_request and 9 for push.
  • The so only from source pattern runs exactly the 12 tests in the two describes that skip under PATCHBOOK_CMD. A zero-match pattern makes bun test exit 1, so renaming those describes fails the job.
  • Bun counts tests in a skipIf describe as skipped even when the name pattern filters them out. That is why root jobs print more skips than runner jobs. The pass counts are what to compare.
  • An independent verifier checked the matrix, the shard partition (fork 538, vendor 430 and rest 718 make the 1686 tests of an unfiltered binary run, each exactly once) and the root wrapper, including that the uid check fails for the wrong user.

Written with Claude Sonnet 5.5 in Claude Code

🤖 Generated with Claude Code

willgosnold and others added 2 commits October 27, 2026 19:13
The root job ran the whole suite from source and then the whole e2e
suite against the binary, in one job, and took about 17 minutes. The
required check waited on it, so every PR did too.

Make the user a matrix axis of the test job, so root runs the same three
shards as the runner user and the shard list stays in one place. Root
stays on Ubuntu. The rest shard runs the unit files and the e2e
describes that need the source. The test-root job is gone.

Part of #382

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@willgosnold
willgosnold merged commit 3fed18e into main Oct 27, 2026
8 checks passed
@willgosnold
willgosnold deleted the ci/shard-root-job branch October 27, 2026 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant