Skip to content

chore: merge-train b18 integration check (do not merge) - #360

Closed
roodboi wants to merge 33 commits into
nextfrom
train/b18
Closed

roodboi wants to merge 33 commits into
nextfrom
train/b18

Conversation

@roodboi

@roodboi roodboi commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Integration check only — do not merge this PR. It exists so CI runs on the combined result of the next merge train before each PR is squashed into next individually.

train/b18 = next 099877a plus these PR heads, merged in order:
#358 f8cbd52, #343 e2dab3a, #339 11436cd, #338 536b348, #332 ebbdb9c, #333 e43075c, #336 a6c15bd, #345 254a7bf, #347 d5d96ed, #348 907726e, #349 0983d7a, #354 7b17177.

Each PR passed CI on its own head and its independent verifier. #341 conflicts with #336 in a generated plugin file and will follow separately after regeneration.

When this is green, the PRs are squashed in that order with --match-head-commit, and next's final tree is checked against this branch's tree (9001df2a). Then this PR is closed.

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

roodboi and others added 30 commits October 19, 2026 11:38
Three onboarding/inventory tests timed out at Bun's default 5 s in the
full local suite: `hack agent onboard` on an empty repo, and the
Python-only and other-ecosystem inventory fixtures. None of these
fixtures has a package.json, so findRepoRootForInit walked up past the
temp dir. With TMPDIR=/tmp and a stray /tmp/package.json on the host,
the repo root resolved to /tmp. The inventory then globbed all of /tmp
and described the wrong repo.

Bound each fixture with an empty `.git` (tests/helpers/repo-boundary.ts,
matching the existing retained-* fixtures), assert the onboard prompt's
inventory root is the fixture, and add a host-independent guard. The
guard nests a bounded and an unbounded fixture under an ancestor
package.json and checks where the walk-up stops. Timeouts and product
code are unchanged.

HACK-1137

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The prerelease installer bounded archive members by the sum of its allowlist
sizes, so the 17-member next.4/next.5 layout filled the bound exactly. Replace
it with a fixed 64-member hard ceiling (MAX_ARCHIVE_MEMBERS) and a distinct
"Too many archive entries." refusal. Every other check is unchanged: each
member must still be a distinct allowlisted, non-traversing, bounded regular
file in a complete group, with checksums, modes, identity and signatures
verified before selection.

Add the manager shipped by 5.0.0-next.3 through next.5 (e2e2bcba) to
MANAGER_PREDECESSORS with the fixture prerelease-manager-storage-v5.py, so
those channels can adopt this installer through upgrade-manager.

HACK-1124

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…atives

Add WU04 negative-case coverage for the node job journal:

- real SQLITE_FULL on terminal publication and cancellation leaves the
  prior receipt, generation and operation set untouched; the same
  requests apply exactly once after space returns
- cancel and completion serialize under real SQLite write-lock
  contention; terminal receipts are final and an early cancel is never
  reported as an achieved cancellation
- the same operation ID with a changed fixture, deadline, capability
  set, generation or action conflicts without effect, and a cancel
  operation ID cannot be re-pointed at another job
- a supervisor crash after its claim, before or after spawn persistence,
  is quarantined without replay or signalling the recorded PID
- terminal receipts, generation and retry identities survive both an
  in-process journal reopen and a node service restart

Refs HACK-1117.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Register Docker-tier scenarios that start copies of the checked-in
examples/patterns fixtures in the hosted docker-e2e shards:

- pattern-p01-dependency-bootstrap (HACK-1145): cold and warm starts, a
  linked worktree, a bun.lock change and an installer killed mid-run;
  running consumers only ever read complete payloads.
- pattern-p02-portable-toolchain (HACK-1146): hack run --profile from the
  primary checkout and a linked worktree with a .git pointer file; failing
  checks and exact exit statuses propagate.
- pattern-p07-capability-overlays (HACK-1151): new laptop/runner overlay
  fixture; value-free env explain sources, container metadata, upstream
  reachability and no browser opener.

Cards name their scenario in e2eScenario; static tests require it to be a
registered, hinted Docker-tier scenario and check the P07 overlay split.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add `hack setup migrate` for moving one scope at a time from standalone
Hack skill and rule files to the skills-only `hack` plugin (HACK-1132).

- The default preview is read-only. It lists each standalone target as
  remove, blocked, preserve or absent, using the W11 ownership
  classifier. Customized, unknown and symlinked paths are preserved.
- Replacement checks read client configuration per scope: Claude Code
  settings precedence plus an installed copy carrying the replacement
  skills; Codex `[plugins."hack@hack-dance"] enabled = true`. Cursor is
  not inspectable and needs an explicit --skip-plugin-check.
- --apply is all or nothing. It refuses without writing when a target is
  blocked, commits a 0600 backup manifest with exact bytes and mode
  before removing anything, and rechecks bytes before each removal.
- --rollback restores only into empty paths, never overwrites, refuses
  manifests whose paths, digests or bytes are not Hack-owned targets,
  and recovers an interrupted apply.
- Hooks, MCP entries and AGENTS.md/CLAUDE.md sections are untouched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Docker Hub shorthand such as alpine:3.21 was refused as registry_image,
and only after native foreground startup had already booted the VM.

- Add a pure shared normalizer (provider::image_reference): alpine ->
  docker.io/library/alpine:latest, user/img -> docker.io/user/img:latest,
  index.docker.io -> docker.io; tags, digests, registries and ports are
  kept. Uppercase repositories, uppercase/non-sha256 digests, bad ports,
  empty and over-long references refuse with closed value-free reasons.
- Compose plan admission validates with the same normalizer and keeps the
  authored image text, so existing plan identities are unchanged.
- ensure-image, resolve-image and fetch-image accept shorthand.
- New pure `runtime admit-image --reference REF --json`; native foreground
  up admits every authored service image before hooks, VM or provider
  start.

Refs HACK-1116.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add a library contract for long-lived helpers shared across branches on one
loopback port (HACK-1142, WU29). Starters are serialized by the
projects-registry lock protocol; a listener is adopted only after its
identity endpoint names the expected helper and branch upstream; owned
helpers are recorded as PID plus process birth time and spawned behind an
exec gate that only opens once the claim is durable, so a lost starter never
leaves an unclaimed child. Readiness is the identity answer, bounded by
backoff and one deadline, with cancellation stopping the owned child.
Release drops one holder and stops an owned helper's birth-verified tree
only for the last holder; adopted, foreign and stale-PID processes are never
signalled. Failures are a closed, value-free set.

Portable process fixtures cover adoption, wrong upstream, foreign
listeners, concurrent independent starters, stale/reused PIDs, child
restart, cancellation during retry, placeholder liveness, lost parent
before and after the gate, descendants, impostor identity, invalid claims
and an unavailable census.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d cleanup

Ordinary graph cleanup retired every bound environment slot but left each
immutable intent in run/environment-leases, so admission, cleanup and restore
scanned every historical generation of a repeatedly started graph.

After `graph cleanup` commits, the same engine mutation lease now compacts the
run's records. Each must match a receipt container that is absent and a guest
slot that is absent, or nothing is written. Exact intent digests move into one
value-free per-run tombstone bound to the VM incarnation, and only then are the
active records unlinked. Prior boots fold into one sealed summary each; the
current boot keeps per-slot reservations. A retired slot, or any slot of a
sealed boot, is never recorded, delivered, retired as live or counted again.

Pending, live, changed, misbound or foreign-incarnation records refuse the
whole compaction before its first write; other runs' and standalone records
are untouched. Exact residue of an interrupted compaction reads as retired and
is completed by the next cleanup or by archive's resume-only pass; changed
residue refuses. Recovery flows that carry an exact environment inventory keep
using the fenced path without compaction.

Refs HACK-1193

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…touching app data

Add an opt-in, single-target quarantine for a regenerable cache directory
inside an ordinary owned volume of a stopped, data-retaining graph (HACK-1198).

- Compose volumes may declare `x-hack-disposable-cache` (at most eight
  distinct, non-nested, volume-relative directories). Shared dependency cache
  volumes and paths that are or contain a mounted subpath are refused. The
  declaration is recorded on the admitted volume resource; absent declarations
  keep plan and receipt bytes unchanged.
- `graph inspect-cache-quarantine` measures the exact target under the
  provider lease and returns a selection digest binding receipt bytes, volume
  incarnation, root and target directory identities and measured bytes.
- `graph quarantine-cache --expect-selection` renames the directory on the same
  filesystem into a Hack-owned `hack-quarantine` sibling of the volume's
  `_data` root. Nothing is deleted; reclaimed bytes are reported as zero.
- A durable private record is committed before the guest effect and completed
  after the guest syncs the move; replay resumes only the identical selection.
- Refuses active graphs, incomplete journals, undeclared or traversing paths,
  foreign labels, changed volume incarnation, any container consumer,
  symlinked or replaced targets, ambiguous destinations and torn records.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…can controls

HACK-1127 (WU14) F9.

- tests/env-secret-sentinel-scan.test.ts: CI synthetic-sentinel scan of a
  primary checkout and linked worktree after env add/materialize/exec, up
  (Compose generation), env apply, logs, list, explain and doctor plus a
  careless `git add -A` commit. Zero raw/hex/base64 hits required in output,
  child argv, tracked files, Git objects, HACK_HOME, HOME caches and runtime
  files; plaintext only in ignored, untracked, 0600 .hack/.env and
  compose.env.override.yml.
- The modern compose.env.override.yml (decrypted values) is now written
  private (0600, atomic, symlink refusal).
- encrypted_file first-use key bootstrap publishes the key exclusively
  (link(2) of a fsynced 0600 sibling) so concurrent first users adopt one key
  instead of replacing a key the store was already encrypted with.
- Project key creation runs under a cooperative lock in the git common dir
  when no key exists, key files are published complete, and a read that
  overlaps a publication is retried briefly; primary and linked checkouts
  now converge.
- Cross-context v1 controls: moves to another key are refused value-free;
  same-key moves are pinned as accepted and documented as residual risk.
- docs/security/env-secrets*.md, docs/env.md, docs/architecture.md updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The hosted run showed the legacy hack run path printing its internal
DNS/TLS notices on stdout ahead of the probe output. Match the probe's
own git= and checkout= lines instead of the whole stream.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…and lost starts

Read process births in UTC so clients with different TZ values agree on a
live owned helper; previously a second client saw it as stale, re-adopted it
and the last release leaked it. A stale owned claim whose PID still answers
as the helper is now kept (claim-invalid) instead of adopted.

Report deadline (not foreign-listener) when lock contention or the deadline
cut the port probe short. Stop a helper left in phase starting by a lost
starter once it still cannot become ready. Rethrow observer errors after
closing the gate or stopping the started helper, and report state errors
raised under the lock as claim-invalid/state-unavailable rather than
lock-unavailable. Bound the claim read by bytes before reading it.

Add a lock barrier to the concurrent-starters fixture and regression tests
for every documented check; register the suite in the Linux process job.

Refs: HACK-1142

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y platform

The hosted Linux job showed that a refused loopback fetch does not carry the
macOS ECONNREFUSED code there, so every free port was misread as a foreign
listener. A failed connection now never proves absence by its error code:
the loopback bind probe decides (occupied is foreign-listener, free spawns).
Add a listener that drops every connection to the foreign-listener fixture.

Refs: HACK-1142

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…solution

Route admit/inspect/quarantine through a Port trait so a synthetic
provider runs the shipped orchestration and guest scripts against a
fixture volume. Add resolve-cache-quarantine, which settles a pending
record as quarantined or abandoned without moving bytes, so a restart or
a persistently refusing move no longer blocks a volume. Require the same
findmnt mount ID before mv so it cannot copy-then-unlink, report
retained quarantine bytes on every result, bound records per volume,
and add an ignored native bad-cache fixture. Document the commands in
the runtime README and the storage-inventory undercount.

Refs: HACK-1198

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Plan-time image admission now runs `runtime admit-image` before lifecycle
hooks. The branch startup fixture's fake runtime treated every unknown call
as a runtime effect and failed it, so admission refused before hooks ran.
Answer admission like the runtime does, without touching the native-ran
tripwire, so the test still proves hooks run before runtime up.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
argv is parsed once against the union of every command's options keyed by
long name. Declaring `setup migrate --rollback <manifest>` as a value option
retyped the existing boolean `config adopt --rollback`, so
`hack config adopt --rollback --json` consumed `--json` as its value and
failed with a usage error. That broke every legacy-compose adoption
rollback in the docker E2E shards.

Rename the migration flag to `--rollback-manifest <manifest>` and add a
command-tree guard that each long option keeps one value type, plus a
parse regression for `config adopt --rollback --json`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Detached startup ran one `docker compose up -d` under the 90 s
HACK_COMPOSE_STARTUP_TIMEOUT_MS budget, so a first-time image build or
large pull was killed before any container started. Split detached
startup into an image phase (`up --no-start`, HACK_COMPOSE_IMAGE_TIMEOUT_MS,
default 30 min) and a start phase (`up -d --no-build`, the existing
startup budget). Each phase keeps its own owned process-group deadline, and
E_STARTUP_TIMEOUT now names the phase, its budget and the variable that
raises it.

Refs HACK-1274

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace wall-clock guesses in three tests that failed intermittently on
loaded hosts (HACK-1280):

- The hanging-probe test now proves the kill only after the fake Docker
  publishes its pid through an atomic marker handshake. A probe timeout
  that fires before the handshake is inconclusive and escalates through
  named rungs (500 ms, 2 s, 8 s); every rung still requires the timeout
  refusal, and the reap check polls to a named deadline instead of a
  fixed 50 ms sleep.
- The native compose ownership file sets a named 60 s default test
  budget: its scenarios launch many cold fake-Docker processes, and an
  expired test kept writing into the next test's shared root.
- The initial forwarding trace test uses named bounds for each forwarded
  command (30 s) and for the test (150 s) instead of 3 s and 5 s.

No product timeout or assertion changes.

Refs HACK-1280

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every environment-bound graph the CLI creates is relay-owned, so ordinary
graph cleanup never ran compaction for it. Foreground redelivery and
fresh-owner restore now compact the run's retired history under the engine
lease. Compaction runs after every prior slot is re-retired and after the
acknowledged stopped receipt is superseded on disk, before the new generation
allocates. Enrolled relay cleanup still does not compact, because the
acknowledged-publisher check re-verifies its inventory.

A compaction refusal after a committed cleanup is now Settled::Retained
instead of an error. The records stay active and the next cleanup, restore
or archive retries. graph::cleanup no longer reports a cleanup that succeeded
on disk as failed.

New tests cover the retire guard (live_intent), archived-copy refusal,
residue with a pending record, the 16384 retired-slot cap, and 260
generations of a 14-service graph across 9 boots. A source-order wiring
guard covers the trigger points. The README now describes the trigger
points, refusal semantics, current-boot growth and tombstone retention.

Refs: HACK-1193

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The "stop that cannot be confirmed" test froze its starter with
Subprocess.kill("SIGSTOP"). On macOS, the pinned Bun 1.4.2 delivers
SIGCONT for that call, so the starter kept running, reaped the helper
after TERM, and release reported `stopped`. CI uses the pinned Bun, so
the test failed there; local runs passed only on an unpinned Bun 1.4.3.

The scenario also showed a product defect. The census counted a zombie
(an exited, unreaped process) as live. A stop that left the helper a
zombie reported `stop-unconfirmed`. A `ready` claim whose helper crashed
under a suspended starter looked live, so other clients could not
restart the helper until that starter resumed. A zombie runs no code
and holds no listener, and its PID cannot be reused until it is reaped.
The census now reads `ps` state and treats zombies as exited.

The unconfirmed-stop test now injects the census and signal boundary
through a narrow `processes` seam on release, because no real process
can portably survive SIGKILL. A separate real-process test freezes the
starter with process.kill and asserts that the helper is an unreaped
zombie and that release reports `stopped`.

Refs HACK-1142

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Clients of a helper port coordinate only through the claim and lock
under `<stateDir>/<name>`. The previous JSDoc called stateDir a
"caller-owned private directory", which suggested one directory per
caller. A client with its own directory finds no claim. It adopts the
running owned helper, and the owner's last release then stops a helper
that client still holds. Its starts are also not serialized with the
owner's.

The spec JSDoc, the module contract and the reference doc now state that
every client of one port must pass the same stateDir and name, derived
from the helper and port rather than the branch.

Refs HACK-1142

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@roodboi

roodboi commented Oct 20, 2026

Copy link
Copy Markdown
Contributor Author

Integration check done: all 12 PRs were squashed into next in order. next b431f0e has the same tree as train/b18 (9001df2a). Closing; this PR was never meant to be merged.

@roodboi roodboi closed this Oct 20, 2026
@roodboi
roodboi deleted the train/b18 branch October 20, 2026 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant