[mubeda]

Fixes #364. Fixes #368. Fixes #363. Fixes #367.

What was wrong

The recurring explanation for slow CI was "no Rust cache". The job logs of ten main runs and three local builds say the cache is a small part of it:

• The server crate is the cost. rust-cache never stores workspace crates, so bibcode-server and bibcode-desktop are compiled from scratch every time, and with one codegen unit each compiles on one core. In the bundle step Cargo had started its last dependency after 6 to 7 minutes; the remaining 23 (Linux x64), 42 (Windows x64) and 81 to 107 minutes (macOS arm64) were those two crates and the link.
• A Windows native row built bibcode-server five times: 76 of 103 minutes in Cargo builds, about 2 minutes running tests (Windows native desktop rows need about 120 minutes without a Rust cache #368).
• The cache store thrashes. It holds 10 GB and CI saves about 17 GB, so entries evict each other within the hour (11 of 13 restores found nothing in one main run). That costs a job about 6 to 12 minutes.
• ci: the macOS bundle build takes 75 to 111 minutes since symbols were turned on #363, macOS doubling: locally the symbol profile costs nothing and dsymutil is not visible. The bibcode-server compile peaks at 8.7 to 9.6 GB and the macOS arm64 runner has 7 GB. Memory pressure is the leading explanation and is not measured yet; this pull request adds the measurement.
• release symbols: dump_syms logged a CFI error and still stored the symbol file #364: the dump_syms CFI error comes from parsing the dSYM, whose unwind section header has no data. Reproduced locally: the stored file had every unwind record of the binary (635,173, 0 lines differ).
• Flaky on Windows arm64: review prompt shim tests time out after 10 seconds #367: the four tests that failed were the first four started; the test dropped the terminal output, so the cause is unknown.

Evidence and research: docs/plans/2026-11-14-ci-build-times/research/.

What changes

Change	Issue
scripts/dump-symbols.ts: when dump_syms logs an error, dump the binary alone; fail if it errors too or the STACK record counts differ	#364
New job Server tests - Windows x64/arm64 holds the server test steps and the test target check, without -j 2; the native rows no longer build the server for tests	#368
CI bundle builds pass --workspace-codegen-units 16: a per-package Cargo override for the two workspace crates. Dependencies keep their fingerprints, and Release does not pass it	#363
The bundle step prints a time and memory heartbeat every minute and a Bundle build notice	#363
Pull requests no longer save pnpm store cache entries (cache-save on main only)	cache
A shim test timeout now prints the terminal output and exit state	#367
docs/operations/ci.md, release.md and the Windows runbook state the measured causes	all

Decisions to confirm

• CI no longer compiles the two workspace crates exactly as Release does (16 codegen units against 1). It still proves the thin-LTO link, the symbol dump, the strip and the bundle. Release is unchanged.
• The repository's cache limit stays at 10 GB. Raising it is a repository setting billed per GiB above 10, and is left to the owner.

Verification

• vp test run scripts: 952 passed. vp check, vp run typecheck, vp run check:markdown-links, cargo fmt --all --check: pass.

• The real bundle wrapper started cargo build --config profile.release.package.bibcode-server.codegen-units=16 --config profile.release.package.bibcode-desktop.codegen-units=16 ...; a rebuild with only that override compiled only the two workspace crates (653 s against 1125 s).

• The real dump_syms on a local binary and dSYM that logs the error: "the binary's 635173 STACK records are all stored", exit 0.

• Heartbeat probes: a failing build keeps its exit code, a failing ps and a dead heartbeat do not fail the step.

• Windows arm64 VM: cargo test -p bibcode-server --test windows_terminal_shims, 24 passed, including the new timeout test.

• Reviewed by Codex (plan twice, branch once); findings handled in docs/plans/2026-11-14-ci-build-times/reviews/ and the last commit.

• Windows arm64 VM: Clippy and cargo check --all-targets with warnings denied pass; the new test fails as intended when the collected output is dropped from its message.

Acceptance (measured on a manual full run of this branch, 2026-11-14)

Every job passed except Secrets, which failed because a manual run scanned the branch's whole history; the range fix is in this pull request and its pull-request run scans base..head.

Job	Before	This run	Verdict
Native desktop, Windows x64	89 min	46 min	R2 accepted
Native desktop, Windows arm64	103 min	59 min	R2 accepted
Native desktop, Linux x64	48 min	31 min
Native desktop, Linux arm64	52 min	35 min
Native desktop, macOS arm64	115 min	90 min	override stays (spec D3)
Native desktop, macOS x64	132 min	93 min
Server tests, Windows x64 and arm64 (new)	inside the native rows	27 and 34 min

Bundle build, Cargo time	Before	This run
Linux x64	about 29 min	17 min 55 s
Windows x64	about 42 min	30 min 12 s
macOS arm64, nothing cached both times	87 min 51 s	59 min 18 s
macOS x64	69 to 101 min for the step	54 min 53 s

• R3: on the two rows with a full cache match the whole log holds 6 Compiling lines, so dependencies were reused under the override.
• R4, ci: the macOS bundle build takes 75 to 111 minutes since symbols were turned on #363: on macOS arm64 free memory stayed near zero for about half an hour, swap in use rose from 33 MB to 975 MB, and the largest rustc held 1.6 to 3.6 GB resident for a compile that holds 8.7 GB or more where memory is available. The runner is short of memory for this compile. dsymutil showed once, at 108 MB. This run also changed the codegen units, so it does not prove what doubled the build on 2026-11-06; memory pressure remains the supported explanation.
• Flaky on Windows arm64: review prompt shim tests time out after 10 seconds #367: reproduced on fresh Windows arm64 runners with a temporary workflow that is not part of this pull request. 1 of 80 cold runs failed with the issue's symptom; the new diagnostics showed the pseudoconsole up and the shim not yet printing, in a batch that took 19.8 s against a usual 4 to 7. The shim helper's deadline is now 60 seconds, and 240 cold runs with it all passed (the count chosen in advance for a 1 in 80 rate). What stalls the runner is not known, and no stalled run was seen passing, so a rarer or longer stall could still fail; it would print the terminal output.
• Also in this pull request, approved by the owner on 2026-11-14: a manual run or a new branch scans origin/main..sha for secrets instead of the whole history.

What is not fixed: the macOS rows still take about 90 minutes, because one large crate compiles on a 3-core, 7 GB runner. Splitting bibcode-server or a larger runner is the remaining lever and is outside this pull request.

Profiler coverage: none of the profiler, crash or trace items apply; the change touches CI, build scripts, a test helper and docs.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-14T13:05:20.065196Z	1136012	PR opened
🔒 Security Review	✅ Completed 2026-11-14T13:06:37.375075Z	1136012	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.