Repository navigation
test(e2e): add headless bootstrap negative controls - #342
Merged
Merged
Conversation
Extend the portable bootstrap fixture (#146/#147) with a hosted docker-e2e scenario, `portable-bootstrap-negatives`, for HACK-1143. On the slim path it drives `hack up --detach app` through eleven ordered stages against a locked-v1 dependency cache with a frozen installer and an app-scoped encrypted synthetic token: - missing key, wrong key and an overlay sealed under another key refuse before any owned container, network or volume exists; - two identical bootstraps keep the same consumer, cache and index, initialize once and leave no worktree diff; - a wrong key, a stale pinned image, a lockfile that no longer matches its manifest and a corrupted ready marker each refuse without touching the running consumer, pulling, rewriting the lockfile or reinstalling; - a changed lockfile and a new cache generation select new caches and keep the previous ones. Each stage records a value-free receipt (duration, output bytes, tracked bytes, mounted-cache bytes, cache-volume count, consumer transition). Pure stage expectations and receipt validation live in tests/e2e/portable-bootstrap-controls.ts with unit tests. A local test runs the source CLI on the same fixture against a recording Docker stub to show the env refusals precede any Compose mutation. WU12 storage admission, retention and archives remain out of scope. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
roodboi
force-pushed
the
wf/f15-headless-negatives-b16
branch
from
October 19, 2026 16:45
633a182 to
4babce5
Compare
Resolve tests/e2e/run.ts and tests/e2e/README.md by keeping both sides: the portable-bootstrap-negatives scenario from this branch and the P01/P02/P07 pattern scenarios merged into next in #336. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
This was referenced Oct 20, 2026
roodboi
marked this pull request as ready for review
October 20, 2026 17:59
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Final behavior
Adds the hosted Docker scenario
portable-bootstrap-negativesfor HACK-1143 (WU30). It extends the portable bootstrap fixture from #146/#147 with negative controls that do not depend on WU12. No product code changes.The scenario runs the selected CLI in slim mode with cached
alpine:3.20pinned by immutable ID,network_mode: none, no published ports and no pulls. Adepsproducer fills alocked-v1dependency cache with a frozen install; theappconsumer mounts it read-only and loads the cache only after an app-scoped encrypted synthetic token matches its digest. Every stage runshack up --detach app:refuse-missing-key,refuse-wrong-key,refuse-wrong-overlay.hack/.gitignore)bootstrap-coldbootstrap-repeatrefuse-wrong-key-runningrefuse-stale-imagerefuse-frozen-lockfilebootstrap-changed-lockfilerefuse-corrupted-cachebootstrap-new-generationOutput and tracked files must never contain the keys or token values; the scenario reads only fixed container fields because the consumer's environment holds the token. Cleanup checks exact owner and Compose project labels; an adjacent canary volume must survive.
The scenario ends with one value-free
portable-bootstrap receipt {...}line: per stage the outcome, consumer transition, duration, stdout/stderr bytes, tracked fixture bytes, mounted-cache bytes and owned cache-volume count, plus the pinned image size.tests/e2e/portable-bootstrap-controls.tsrefuses any other shape (unknown keys, open-ended strings, missing or reordered stages, non-integer measurements).Also:
tests/portable-bootstrap-env-refusal.test.tsruns the source CLI on the same fixture against a recording Docker stub (viaprivateCliEnv): missing, wrong and wrong-overlay keys fail before any Composecreate/run/start/up; the correct key reaches thedepsbootstrap first.tests/e2e-portable-bootstrap-controls.test.tscovers stage expectations, consumer classification, worktree diff, receipt validation and the synthetic v1 envelope.encryptSyntheticis now exported fromscripts/portable-bootstrap-smoke.ts(no behavior change).tests/e2e/run.tswith a 45 s shard estimate; docs intests/e2e/README.mdanddocs/guides/portable-bootstrap-qualification.md.Validation (local, macOS, Bun 1.4.3; pinned 1.4.2)
tests/portable-bootstrap-env-refusal.test.ts,tests/e2e-portable-bootstrap-controls.test.ts,tests/cli-spawn-env-isolation.test.ts,tests/integration-ownership.test.ts,tests/e2e-shard.test.ts,tests/e2e-scenario-selection.test.ts: 54 pass, 0 fail.bun run --cwd packages/cli typecheck: pass.bun run --cwd packages/cli check: 0 errors (existing warnings only).bunx ultracite checkon all changed TS files: clean.bun run privacy:check: ok.bun tests/e2e/run.ts --listshows the scenario in the docker tier.project-env-config.ts: the missing-key refusal test fails (Composerunreached).tests/portable-bootstrap-smoke.test.tsis flaky on this shared host under load (average about 35); it fails intermittently on unchangedorigin/nextcode as well.Hosted validation (exact head
4babce5b)docker-e2e shard 4/4passed the scenario in two runs: run 39281111519 (37.2 s, 1 pass) and run 39288207970. Receipt from the first run (value-free):Tracked fixture bytes were 4,872 before the managed
.hack/.gitignoreand 5,068 after (5,098 while the manifest was deliberately edited). The image is 7,807,582 bytes. The ~10.5 s replacements are mostly Compose stopping the oldsleepconsumer. The stale-image stage leaves one empty cache volume for its new fingerprint.Docker mutation control: a temporary commit (
633a1820, since force-pushed away) made the locked-v1 installer accept a mismatched ready marker. Shard 4 failed at exactlyrefuse-corrupted-cache: expected refused, observed bootstrappedafter all earlier stages passed, and cleanup completed. The mutated run was then cancelled.In run 39281111519, every job passed except
docker-e2e shard 1/4:native-config-networksfailed with "Fixture command must exit 0" (unrelated; that shard runs only that scenario). In run 39288207970, triggered by the force-push back to the same SHA, shard 1/4, thedocker-e2efan-in,testand all Linux jobs passed. Its macOS jobs were still queued when this was written; they passed in run 39281111519.Remaining gaps
Linear: HACK-1143
🤖 Generated with Claude Code