Skip to content

test(e2e): prove network probe budget and resolved-address fetch (HACK-1428) - #484

Merged
roodboi merged 1 commit into
nextfrom
wf/g93-network-probe-test-gaps
Oct 24, 2026
Merged

roodboi merged 1 commit into
nextfrom
wf/g93-network-probe-test-gaps

Conversation

@roodboi

@roodboi roodboi commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Follow-ups after #470 (e08f971). Test and fixture-harness changes only; no product code changes.

Root cause

  1. Shared-budget subtraction untested. Local DNS resolution of 127.0.0.1 or localhost takes about 0 ms, and the hung-probe bound (< 2000 ms for a 300 ms budget) was loose. So Math.max(1,${timeout}-d) → ${timeout} still passed.
  2. Resolved-address fetch untested. Every probe test used names that the system resolver also resolves to the listener. A fetch to the original hostname reached the same server, so removing target.hostname = …x.address still passed.
  3. Create shim oracle flake (Expected: 71, Received: 128). The test makes nine shim invocations in sequence, and each one starts one or two Bun processes. On a loaded host this took longer than bun test's 5 s default timeout. When the timeout fires, bun test SIGTERMs the still-running shim ("killed 1 dangling process"). The runner mapped the signaled exit (code === null) to a bare 128. The timed-out test then keeps running while later tests run, and its expect(…).toBe(71) fails. Bun reports that as "Unhandled error between tests", so the visible failure is Expected 71 / Received 128 and not the timeout. A minimal repro outside the repo showed the same thing: a test whose child outlives the timeout gets exit code=null signal=SIGTERM, then Expected: 71 Received: 128 as an unhandled error between tests. The same mechanism also hit probe scripts exit with a closed code… (Received: 143, from Bun.spawn's 128+15).

Fix

  • tests/native-networks-fixture.test.ts
    • New peer probe fetches the address it just resolved: replaces Bun.dns.lookup in the probe child with a stub that resolves RFC 6761 .invalid names to 127.0.0.1. The probe must pass, and the server must see Host: 127.0.0.1:<port> on both requests. A second, real resolution of a .invalid name cannot succeed.
    • New peer probe charges resolution time against the shared per-URL budget: the stub delays resolution by 1200 ms with a 2000 ms budget against a hung endpoint. It asserts dnsMs ≥ 1150, elapsedMs ≥ 1950, and elapsedMs < 2600 (a fresh full budget ends near 3200). It also checks that a 500 ms budget with a 1200 ms resolution exits dnsTimeout (31) with phase: dns at 450–1200 ms.
    • New capture reports a signaled command as 128 plus its signal number (SIGTERM → 143, SIGKILL → 137).
    • setDefaultTimeout(60_000) for this file, following tests/native-compose-ownership.test.ts. Every fixture command and probe keeps its own, tighter timeout.
  • tests/e2e/native-config-networks-acceptance.ts: runNativeNetworkFixtureCommand reports a signaled command as 128 + signal number, as a shell does, so a future kill names its signal. The timeout and oversize paths still throw before any exit code is returned. Nothing compared the exit code with 128.

Evidence (local, macOS, Bun 1.4.2; host load average 45–110 from other work)

Mutations (each reverted afterwards):

Mutation Before this PR (-t probe) With this PR
Math.max(1,${timeout}-d) → ${timeout} 5/5 pass budget test fails: Expected: < 2600, Received: 3211
drop only target.hostname=…x.address; (keep let ok=false;) 5/5 pass address test fails (code 26, not 0); budget test fails (Expected 27, Received 26)
runner code ?? 128 (signal ignored) n/a signal test fails: Expected: 143, Received: 128

Flake loops (bun test tests/native-networks-fixture.test.ts, private HOME/HACK_HOME/DOCKER_CONFIG, HACK_SECRETS_DISABLE_KEYCHAIN_FALLBACK=1, no Docker):

Run Before fix After fix
shim test alone (-t "strict argv oracle"), 40 runs 21 pass / 19 fail, all at the 5 s timeout 39 pass / 1 fail (the failure was not this test; see gaps)
whole file, 10 / 30 runs 8 / 10: both failures were the shim test timing out with Expected: 98, Received: 128 plus the closed-code probe test with Received: 143 30 / 30 pass

Gates: bun run check (includes privacy:check) passes, bun run --cwd packages/cli typecheck reports 0 errors (it includes tests/), commitlint passes, and gitleaks 8.30.1 finds no leaks over origin/next..HEAD. The CLI surface and agent guidance did not change. Hosted CI at the PR head is the full-suite evidence.

Gaps

  • Hosted test and docker-e2e shards at this head are the full-suite proof; Docker e2e was not run locally.
  • In 1 of the 40 post-fix runs of the single test, the shim test itself passed, but the checkout write guard's afterAll git status timed out at 5 s under load (tests/helpers/checkout-write-guard.ts). That is outside this unit and is reported separately.
  • The IPv6 branch of the address rewrite ([${x.address}]) still has no test.
  • Stopping a loop mid-run (killing its bun test) left one Bun SIGABRT crash report from an orphaned child. The loops that ran to completion added none.

Refs HACK-1428

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

…K-1428)

The peer probe charges name resolution against one per-URL budget and
fetches the address it just resolved, but no test failed when either
behavior was removed. Two tests now run the probe with a stubbed
Bun.dns.lookup: one resolves RFC 6761 .invalid names to 127.0.0.1 and
checks both requests reached that address, and one delays resolution
and checks the HTTP phase only gets the remaining budget.

The create shim oracle flake (Expected 71, Received 128) was bun
test's 5 s default timeout firing on a loaded host: bun test SIGTERMs
the running shim and the runner mapped the signaled exit to a bare 128.
The fixture test file now sets a 60 s default, and the runner reports a
signaled command as 128 plus its signal number.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@linear-code

linear-code Bot commented Oct 24, 2026

Copy link
Copy Markdown

HACK-1428

@roodboi
roodboi marked this pull request as ready for review October 24, 2026 20:31
@roodboi
roodboi merged commit 926cbdd into next Oct 24, 2026
21 checks passed
@roodboi
roodboi deleted the wf/g93-network-probe-test-gaps branch October 24, 2026 20:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant