Skip to content

test(native): pin Compose startup readiness timeout, transient-flag and strict-default bounds - #378

Merged
roodboi merged 1 commit into
nextfrom
wf/g25-readiness-bounds-tests
Oct 22, 2026
Merged

roodboi merged 1 commit into
nextfrom
wf/g25-readiness-bounds-tests

Conversation

@roodboi

@roodboi roodboi commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Root cause

This change adds tests only. HACK-1305 asks for tests that pin three Compose startup readiness bounds that #362 (HACK-1279) added in src/lib/native-compose-ownership.ts:

  1. A readiness timeout never becomes success.
  2. The transient-flag term behaves as documented.
  3. The strict default holds.

The #362 tests did not pin these bounds. I ran 10 source mutants that break them against tests/native-compose-ownership.test.ts and tests/native-process-policy-initial-trace.test.ts (77 tests). None of the 10 was reliably killed (table below). The three "kills" in that run were replay-driver timeouts, not assertions (see Findings).

The tests exposed no product defect, so production code is unchanged.

Fix

New tests/native-compose-readiness-bounds.test.ts (7 tests):

  • The fake Docker serves one scripted scan per inspect call. Each query's first scan and recheck are therefore exact. retry (the only selected on-failure service) is member, absent-running, absent-restarting, placeholder-running or placeholder-restarting. web is always a healthy ID member.
  • A virtual readiness clock. The poll interval (Bun.sleep) returns at once. The Nth poll moves Date.now past the deadline. The clock only moves between queries, so each query's own probe deadline is unaffected. This fixes the number of in-loop observations, and the inspect count shows whether the settled query ran. A harness-control test checks this for 1 and 3 observations.

What each test pins:

Bound Test
(1) Timeout never becomes success Last in-loop observation is case 2 (running, absent), then the settled check reads retry rejoined and ready: the result is null, the settled query ran (8 inspects), and observed is never called. The same holds for a case-1 running recheck placeholder. An owned but never-ready observation times out as null, without a settled query, and the last unready observation is not returned.
(2) Transient-flag term Each disjunct of first.transient || second.transient || recheck placeholders is tested alone: first-scan running absent, recheck running absent, and a recheck placeholder for a restarting container. Each alone runs the settled check, which refuses the unchanged shape with the original predicate and exact inspect count. Restarting-only shapes (absent in both scans, or a first-scan restarting placeholder) time out null after exactly one query, with no settled query. The flag follows the last in-loop observation: transient then restarting runs no settled query, and restarting then transient runs one.
(3) Strict default Reads that do not opt into transient membership stay strict. For every race shape (case 2, a first-scan running placeholder, and case-1 running and restarting recheck placeholders), settled, strict and saved reads refuse with the original predicate (live-endpoint-membership or network-member-id), while the startup default keeps the shape unready. settled still admits the restarting shapes from before #362, which strict and saved refuse. An unselected service is never admitted.

Mutations

All mutants are in src/lib/native-compose-ownership.ts. The "existing" column is the 77 tests in the two #362 test files.

Mutant Bound New file Existing tests
M1: settled query's owned observation is returned 1 killed survived
M2: timeout returns the last unready observation 1 killed (2 fail) survived*
M3: drop first.transient 2 killed survived
M4: drop second.transient 2 killed survived
M5: drop the recheck-placeholder term 2 killed survived*
M6: every restarting observation counts as transient 2 killed (2 fail) survived*
M7: transient flag latches (||=) instead of following the last observation 2 killed survived
M15: first-scan placeholders also count as transient 2 killed survived
M16: no settled check 1, 2 killed (3 fail) not run (#362 M5)
M8: settled refuses a restarting member 3 killed survived
M9: settled refuses a restarting first-scan placeholder 3 killed survived
M12: settled query passes transient 3 killed (2 fail) not run (#362 M3)
M13: startup default membership is settled 3 killed (3 fail) not run
M10: transientMembership default is true 3 survived (equivalent) —
M14: strict read opts into transient 3 survived (equivalent) —
M11: drop result.restarting && from the wait's latch 2 survived (equivalent) —

* Under the M2, M5 and M6 runs, one HACK-1279 trace replay test failed with Initial process-policy replay unavailable after 7–20 s, at a load average of about 35–180. These mutants cannot change those replays: M2 only edits the wait, and M5 and M6 only change the transient field, which observeNativeComposeStartupOwned ignores. The failures are replay-driver timeouts (the driver's 5 s default exec budget), not kills.

The three survivors are equivalent:

  • M10 and M14. The strict default is guarded three times. Strict and saved reads pass an empty on-failure selection, so restartingServices.has(...) fails. requireLiveMember also requires mode === "startup", and networkMembers requires opts.startup. Flipping the default alone cannot change any outcome. This is the same as M6 in fix(native): keep on-failure restart-backoff and recheck-placeholder members unready until the readiness deadline #362. The new tests pin the observable bound instead: every non-transient read refuses every race shape.
  • M11. result.transient implies result.restarting. Every disjunct requires an admitted container: a transient-admitted running member is added to restarting, and a claimed recheck placeholder owner is never an ID member, so requireLiveMember admits it too.

Evidence

  • Bun 1.4.2 via mise x bun@1.4.2, after bun install --frozen-lockfile. No lockfile changes.
  • bun test tests/native-compose-readiness-bounds.test.ts: 7 pass on this Mac, at a load average of up to about 180.
  • bun run --cwd packages/cli typecheck: clean. bun run --cwd packages/cli check: 0 errors, 34 warnings, the same as the base. ultracite check on the new file is clean. Root tsc reports no diagnostics in the new file.
  • gitleaks dir (8.30.1, repository config) over a git log -p origin/next..HEAD export: no leaks.
  • Full suite on the M3 (macOS, Bun 1.4.2) at exact head f2aec257, run with other units' suites in parallel (load average about 100–160): 7548 pass, 126 skip, 33 fail, 7 errors in 541 files (70 min). The new file passed.
  • Serial reruns on the M3 with --timeout 60000 (load average about 11–26):
    • These files now pass: env-key-argument-redaction-cli, native-authored-project-oneoff, native-authored-project-exec, native-compose-proxy-probe, native-project-branch-cli, registry-touch-worktrees, portable-bootstrap-env-refusal, prerelease-install, native-compose-warm-up-command and project-env-config. The new native-compose-readiness-bounds file also passes (7/7).
    • These files still fail: native-compose-retained-storage-command (5/5), native-compose-storage-fence-cost (4/4), native-compose-adoption-generation (13/114, all at their own 20–30 s test timeouts), native-authored-process-incarnation (1/10) and native-authored-project-recovery (2/18).
  • None of these failures can come from this PR. It adds one test file and changes no source or helper, so every failing file runs the same code as base. Both storage files also fail on the M3 at the pre-fix(native): keep storage carrier commands alive through host stalls #356 base ec223cdb (retained-storage 3/5, fence-cost 4/4). The process-incarnation and no-host recovery failures also show in other units' M3 logs on base.
  • Hosted CI on this PR is green: test, all four docker-e2e shards, linux-process-lifetime and secret-scan.
  • Not captured: the 7 suite-level errors and the first few failure names in the full run. The M3 command keeps only the last 60 lines of output.

Gaps

  • The command-level mapping in src/lib/native-compose-command.ts is not pinned here. A null readiness gives value: code || 1 and outcome: "uncertain". Pinning it needs the native authored-project start harness, and tests/native-authored-project-start.test.ts already fails on base.
  • The abort guard (opts.ownership.signal?.aborted skips the settled query) is not pinned. Aborting deterministically between the last observation and the latch check needs a probe hook.
  • The docker-e2e scenario was not run locally. These are unit-level bounds against a fake Docker.

Findings (not fixed here)

  • The trace replay driver in tests/e2e/native-process-policy-initial-replay.ts uses a 5 s default timeoutMs. Under host load it fails with Initial process-policy replay unavailable on tests that the source change cannot affect (see the footnote above). Severity: low (flake).
  • The wait's doc comment says the settled check runs only when the last observation was unready "only because a running on-failure member" was missing or placeholdered. The code also runs it for a restarting container whose ep- key first appears in the recheck scan, which then refuses with network-member-id at the deadline. The transient field's own doc ("an admitted container depended on a shape that a settled query refuses") matches the code, and the new tests pin the code. Severity: low (doc precision).
  • Storage-backed source CLI startup fails on the M3 (macOS). Repro: on the M3, run bun test --timeout 60000 ./tests/native-compose-retained-storage-command.test.ts. It fails at ec223cdb and at f2aec257. Expected: the fake engine's cold up --detach --json exits 0. Actual: it exits 1 after about 34 s with E_CONFIG_INVALID, native_compose_effect_refusal: { stage: "storage-helper-start", reason: "unclassified" }. native-compose-storage-fence-cost.test.ts fails 4/4 the same way. Hosted Linux CI passes. This is possibly a known flake. Severity: medium, because it hides the storage regression tests on macOS hosts.

Refs HACK-1305

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

…nd strict-default bounds

Add tests/native-compose-readiness-bounds.test.ts for the readiness bounds
that #362 (HACK-1279) added to waitNativeComposeStartupOwned:

- A readiness timeout never becomes success, even when the post-deadline
  settled check finds the member rejoined and ready.
- Each disjunct of the transient flag (first-scan running member, recheck
  running member, recheck placeholder) alone runs the settled check, which
  refuses the unchanged shape; restarting-only shapes, including a
  first-scan placeholder, do not; the flag follows the last observation.
- Reads that do not opt into transient membership stay strict: strict and
  saved refuse every restart-race shape, and settled refuses the running
  shapes while keeping the restarting admission from before #362.

A virtual readiness clock (Bun.sleep advances Date.now past the deadline)
fixes the number of in-loop observations, and the fake Docker serves one
scripted scan per inspect, so each bound is deterministic.

Refs HACK-1305

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@linear-code

linear-code Bot commented Oct 21, 2026

Copy link
Copy Markdown

HACK-1305

@roodboi
roodboi marked this pull request as ready for review October 22, 2026 12:00
@roodboi
roodboi merged commit 642cff4 into next Oct 22, 2026
16 checks passed
@roodboi
roodboi deleted the wf/g25-readiness-bounds-tests branch October 22, 2026 12:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant