Skip to content

fix(release): let candidate launchers select the Compose backend - #352

Merged
roodboi merged 2 commits into
nextfrom
wf/g7-channel-compose-backend-b17
Oct 20, 2026
Merged

roodboi merged 2 commits into
nextfrom
wf/g7-channel-compose-backend-b17

Conversation

@roodboi

@roodboi roodboi commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Root cause

Both candidate launch paths hard-coded the native backend:

  • scripts/hack-v5.sh (the bundled hack-v5 entrypoint) ran export HACK_RUNTIME_BACKEND=native unconditionally. Its test asserted that an inherited HACK_RUNTIME_BACKEND=compose was overwritten.
  • Channel.environment() in scripts/install-prerelease.py (the hack-next channel manager) also set HACK_RUNTIME_BACKEND=native and forced HACK_HOME to the version's private cli-home.

So the supported Compose backend could not run through the channel. The M3 HACK-1124 next.5 rerun (~/.hack-v5-private/hack1124-next5-rerun-20261019-1204/s8-omega.sh) had to call the bundled hack-cli directly with a private HACK_HOME. Compose resolves the global Caddy/CoreDNS compose file and the local CA from HACK_HOME, so that workaround also disabled CoreDNS discovery, Caddy routing and CA trust. HTTPS routing was left unchecked.

Fix

Backend selection is now explicit and the same in both launchers:

HACK_RUNTIME_BACKEND Result
unset or empty Native, as before. The launcher pins its adjacent hack-native, the private native-home and cli-home, and drops HACK_GLOBAL_CONFIG_PATH.
native Same as unset.
compose The verified bundle's hack-cli runs on Compose with the caller's own HACK_HOME (default ~/.hack) and HACK_GLOBAL_CONFIG_PATH, so the existing global Caddy/CoreDNS and local CA apply. HACK_NATIVE_BINARY and HACK_NATIVE_HOME are removed. hack-v5 no longer requires HACK_NATIVE_HOME for Compose.
anything else Refuses before bundle verification or spawn (manager: exit 1; hack-v5: exit 64). The CLI treats every non-native value as Compose, so the launcher refuses rather than guessing.

Other details:

  • Compose path. For Compose, the manager runs bundle/hack-cli directly, not bundle/hack-v5. Retained bundles (next.5 and earlier) ship a hack-v5 that always selects native. Running it would silently ignore the choice and pair native with the caller's shared home.
  • Self-update guard. Before this change, hack update refused only when HACK_RUNTIME_BACKEND=native. A Compose launch drops that marker, and update would then fall back to Bun.which("hack") or HACK_HOME/bin/hack, which is the stable install. Both launchers now set HACK_CANDIDATE_ENTRYPOINT (hack-next / hack-v5), and update refuses when it is set. The refusal message is unchanged.
  • Manager upgrade path. The manager bytes changed. Following the fix(release): ship the storage witness manifest in the native prerelease #289 precedent, the released next.3 to next.5 manager (e2e2bcba…, the same bytes at all three tags) is added to MANAGER_PREDECESSORS, with its fixture tests/fixtures/prerelease-manager-storage-v5.py. Existing channels get backend selection through the reviewed upgrade-manager.
  • Project-level selection. There is no project-level backend selector in the CLI. HACK_RUNTIME_BACKEND is the only selector, and the launchers now respect it.

Docs

  • docs/guides/candidate-install.md: new "Test both backends" section with a native/Compose table and shared-state limits. Stable v4's older registry lock protocol means stable hack and the candidate must not run against the same home at the same time. hack-next does not install global services. A private HACK_HOME is still possible but skips routing, DNS and TLS checks. The section also covers the upgrade-manager requirement for next.3 to next.5 channels. The selector-override paragraph and the install intro are updated to match.
  • docs/guides/native-candidate.md (shipped as the bundle README): the hack-v5 paragraph describes the backend rule. It uses an absolute link, because a relative one would not resolve inside the bundle.
  • docs/guides/v5-dogfooding.md: says to test the backend the project will use, and links to the new section.

No CLI flags changed, so docs:cli-reference is not affected.

Tests

  • tests/native-candidate-launcher.test.ts (rewritten; uses privateCliEnv and a stub hack-cli):
    • unset, empty and native select native and the adjacent executor;
    • compose keeps the caller's HACK_HOME, removes the native selectors and needs no native home;
    • docker, Native and native (trailing space) refuse with 64;
    • native without an absolute home refuses;
    • an incomplete bundle refuses on both backends.
  • tests/update-command.test.ts: HACK_CANDIDATE_ENTRYPOINT with compose or unset refuses before any lookup or fetch, and the stable binary is untouched.
  • tests/python/test_prerelease_install.py:
    • test_launcher_respects_explicit_backend_and_keeps_native_default covers the four cases above. Unknown values refuse with no spawn and no signature check.
    • test_released_next5_manager_needs_upgrade_manager_for_compose_selection runs the real next.5 manager fixture. That manager forces native even when Compose is requested. After upgrade-manager, Compose selects hack-cli with the caller's home, unset still selects native with the private homes, and the retained homes and bundles are byte-identical.
    • The setUp inherited selector changed from the synthetic docker (never a CLI value; stable v4 never reads the variable) to a manually exported native that names a foreign executor and home. The pinning assertions still hold, and docker refusal now has its own test.

Mutation checks (each reverted, test failed, then restored)

  • scripts/hack-v5.sh back to origin/next: 7 of the 10 launcher tests fail, including the Compose and unknown-value cases.
  • src/commands/update.ts back to origin/next: all 3 new candidate-launcher update tests fail.
  • Manager run back to always hack-v5 with the native environment: both new Python tests fail.
  • e2e2bcba… removed from MANAGER_PREDECESSORS: the next.5 upgrade test fails.

Commands run (all pass)

  • bun test tests/native-candidate-launcher.test.ts tests/update-command.test.ts tests/cli-spawn-env-isolation.test.ts tests/prerelease-plan.test.ts: 82 pass.
  • python3 -m unittest tests/python/test_prerelease_install.py: 59 OK under both Homebrew Python and /usr/bin/python3 3.9.6, which the launcher uses.
  • bun run --cwd packages/cli typecheck and bun run --cwd packages/cli check: exit 0.
  • bunx biome check on the changed TS files; bun run privacy:check; sh -n scripts/hack-v5.sh.
  • No Rust changes.

Local integration smoke (M-series Mac, private scratch HOME and HACK_HOME, DOCKER_HOST pointing at a nonexistent socket)

  • Installed the published 5.0.0-next.5 archive into a fresh scratch --root with this branch's installer (real download, checksums and codesign). Then ran the real bin/hack-next shell launcher:
    • hack-next --version → hack v5.0.0-next.5, exit 0. With HACK_RUNTIME_BACKEND=compose the result is the same.
    • hack-next update --check --json → native refusal, exit 1.
    • HACK_RUNTIME_BACKEND=compose hack-next update --check --json gets past the native refusal and looks for the binary at <caller HACK_HOME>/bin/hack. This shows that Compose was selected and the caller's home was used, not cli-home.
    • HACK_RUNTIME_BACKEND=docker hack-next --version → hack-next: HACK_RUNTIME_BACKEND must be native or compose…, exit 1.
    • The version's native-home and cli-home stayed empty.
  • The source CLI behind the new hack-v5, with compose and with native: update --check --json refuses; docker exits 64.

Remaining gaps

  • No Compose up with live routing, DNS or TLS ran through the channel. That needs the M3 or M5 global services and a real project, and is still the acceptance step for HACK-1124 item 1.
  • Candidate CLIs from next.5 and earlier do not know HACK_CANDIDATE_ENTRYPOINT. With an upgraded manager, HACK_RUNTIME_BACKEND=compose hack-next update on those bundles is not refused. Today it only compares 5.0.0-next.N with the latest stable 4.x, so nothing is installed. The docs say not to run it. Bundles built from this branch refuse.
  • Compose through the channel shares the registry and global config with stable v4. This is documented, not enforced; the launcher cannot detect a concurrent stable writer.
  • A shell with a stale exported HACK_RUNTIME_BACKEND=compose now selects Compose for hack-next. This is documented.
  • tests/python/test_prerelease_install.py is not run by CI (ci.yml runs only three other Python suites). It was run locally here. See the findings in the run report.
  • Not merged or released. A new prerelease tag is needed before installed channels get the new hack-v5 and installer.

Refs HACK-1275

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

hack-next (the channel manager) and the bundled hack-v5 entrypoint exported
HACK_RUNTIME_BACKEND=native unconditionally, and hack-next also forced the
version's private HACK_HOME. Testers could not run the supported Compose
backend through the channel; the M3 HACK-1124 rerun had to call the bundled
hack-cli directly with a private HACK_HOME, which also hid the global
Caddy/CoreDNS services and local CA.

Unset or empty HACK_RUNTIME_BACKEND still selects native with the private
homes. An explicit HACK_RUNTIME_BACKEND=compose now runs the verified bundle's
hack-cli on Compose with the caller's own HACK_HOME and
HACK_GLOBAL_CONFIG_PATH, and removes native selectors. Any other value refuses
before verification or spawn. The manager runs hack-cli directly for Compose
because retained bundles' hack-v5 always selects native.

Both launchers set HACK_CANDIDATE_ENTRYPOINT, and `update` refuses when it is
set, so a Compose launch keeps the candidate self-update refusal that the
native backend marker used to provide.

The released next.3-next.5 manager (e2e2bcba) is added as a reviewed
predecessor with its fixture, so existing channels get backend selection
through upgrade-manager.

Refs HACK-1275

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@linear-code

linear-code Bot commented Oct 19, 2026

Copy link
Copy Markdown

HACK-1275

@blacksmith-sh

This comment has been minimized.

Resolve a comment-only conflict in scripts/install-prerelease.py above
MANAGER_PREDECESSORS by keeping next's wording (same meaning, names the
fixture). The allowlist and the new BACKENDS constant merge unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@roodboi
roodboi marked this pull request as ready for review October 20, 2026 17:58
@roodboi
roodboi merged commit acf5348 into next Oct 20, 2026
17 checks passed
@roodboi
roodboi deleted the wf/g7-channel-compose-backend-b17 branch October 20, 2026 17:58
roodboi added a commit that referenced this pull request Oct 21, 2026
Brings #355 (G4, HACK-1272: retire authored starts refused before journal
reservation), #352, #341, #342 and #346 into the G1 closed-refusal branch.
Both sides' behavior is kept.

Conflict resolutions:

- docs/reference/native-authored-commands.md: keep G1's explanation of
  `pending` (in-progress start or failed start with retained evidence; only
  the owner's cleanup proof may retire it) and append G4's pointer that
  `down --recover` retires an attempt that never reserved runtime state when
  no startup is running.

- src/lib/native-authored-command.ts: the `pending` status note keeps G1's
  wording and adds G4's `hack down --recover` hint. Auto-merged code keeps
  G4's canceled `unreserved` exit 130 ahead of G1's stopped-runtime
  classification. Semantic fix: G1's stoppedRuntimeStartup message map had
  no entry for G4's new `unreserved` outcome, so a runtime_not_running start
  that was retired as unreserved would have produced an undefined message.
  It now reports E_DOCKER_UNAVAILABLE with an `unreserved` message (intent
  retired, `hack ps` reports not_started). G4's `down --recover` pending-start
  path already refuses through commandFailure as a closed coded HackCliError
  (E_LIFECYCLE_FAILED, values omitted), and unreserved startup failures map
  to E_STARTUP_INCOMPLETE or E_DOCKER_UNAVAILABLE, so no stack or help is
  printed.

- tests/native-authored-project-start.test.ts: the fixture keeps G1's
  `stoppedDuringServe` and G4's `failBeforeReceiptCode` and
  `unreservedInspection` options. The stub serve emits G4's configurable
  failBeforeReceipt code and G1's wrapped stopped-runtime refusal. The stub
  inspect answers both failure modes, with `native_graph_unreserved` only
  when requested. The duplicate native-runtime-client import is merged into
  one. Semantic fix: G4's HACK-1272 call-log expectation now includes G1's
  read-only `runtime status --json` preflight between plan and serve.
  Both sides' tests are kept.

- tests/native-authored-command.test.ts (auto-merged): add an `unreserved`
  case to the stopped-runtime startup table.

Verification (Bun 1.4.2, macOS):
- packages/cli typecheck and check: 0 errors.
- native-authored-command, native-refusal-cli,
  native-authored-command-dispatch and cli-spawn-env-isolation: 124 pass.
- Other native-authored-* and native-runtime-* suites: 409 pass, 5 fail.
  2 are load-timing failures that pass when rerun alone. 3 are host
  live-session census failures in the recovery and incarnation suites,
  which fail identically on origin/next and ead8383 on this host.
- native-authored-project-start (--timeout 60000): 48 pass. The 3
  host-metadata cases are intermittent (HACK-1299). They also fail
  intermittently on origin/next and ead8383 and pass when run alone.
- Six scripts/check-native-*-plan-cli.ts: all pass after build and
  build:config-compiler.
- docs:cli-reference and generate:agent-plugins: no drift.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant