[roodboi]

Root cause

On the Compose backend, resolveModernComposeEnvOverrides in src/commands/project.ts rendered every resolved env value into .hack/.internal/compose.env.override.yml. That included values decrypted from secure: v1: entries. It wrote the file with writeTextFileIfChanged (Bun.write, which uses the umask mode and follows a symlink) and never removed it. Up, down, a failed up and an interrupted up all left the plaintext on disk. The M3 rerun saw this on Omega: a 71 KB, 0600 file covering 14 services was still there after down.

The first version of this PR moved the values out of the file. Review found that this broke legacy Compose adoption. Each container's com.docker.compose.config-hash label is computed by Compose with the real HACK_ENV_* values. The adoption probe (inspectLegacyComposeRuntimeConfig) recomputed the hash from the files with no values, so every reference resolved blank. The hashes differed and adoption refused every stack with managed env that this version started. The docker e2e adoption harness also started its original stack with raw docker compose and no values. That failed docker-e2e shards 3/4 and 4/4 (run 39293621453).

Rework 2 (head 879e0b56)

A verifier checked 3970f175 and reported six defects. Commits 5695f824, bebeb89c and 7345de3f already claimed fixes for most of them. This rework checked each claim with a mutation, merged current next, and closed what was still open.

Verifier defect	Status at 879e0b56	Evidence
1. Adoption refuses every stack this version starts (probe hashes ${HACK_ENV_*} as blank)	Fixed in 5695f824	Without privateInterpolationEnv in the probe env, the HACK-1273 adoption test fails. With the saved-hash path disabled, it also fails.
2. hack down skips the scrub when the down-before hook fails or Compose down throws	Fixed in bebeb89c: the scrub runs before hooks, the remote target and Compose down	With the scrub moved back to after the finally block, "down … even when its down-before hook fails" fails.
3. TOCTOU: inspect, then unlink by path	Fixed in bebeb89c (move aside, classify the moved entry, link back without overwriting)	With a plain unlink by path, "removal keeps a value-free override that replaced the inspected literal file" fails. The remaining small window is now documented (see Remaining gaps).
4a. doctor --fix removal untested	Covered by doctor-fix-noninteractive (from bebeb89c)	With the --fix step removed, that test fails.
4b. No-override scrubs untested	2 of 3 were covered. The third (legacy hack.env.json path where no service gets an override) was still untested. This rework adds a test for it.	With each of the three scrub calls removed in turn, a different test fails.
5. CLI test PATH includes the host PATH	Fixed: PATH is the stub bin plus /usr/bin:/bin only	Code reading
6. Docs overstate or omit points	Fixed in bebeb89c. This rework also states the move-aside window and the aside left by a crash, and describes hack down by what it runs before	docs/env.md, threat model row and gap 4

New in this rework:

• Merge with next (cb2d6226). fix(secrets): converge concurrent key provisioning and add sentinel scan controls #349 had made the literal override private (0600, atomic, writePrivateEnvFileIfChanged). This branch keeps its reference writer instead. The resolution drops the unused import, folds fix(secrets): converge concurrent key provisioning and add sentinel scan controls #349's threat-model row into the HACK-1273 row, and rewrites open gap 4 to cover .hack/.env and the Compose client environment.
• Sentinel scan (c33c3c97). tests/env-secret-sentinel-scan.test.ts from fix(secrets): converge concurrent key provisioning and add sentinel scan controls #349 still allowed plaintext in compose.env.override.yml. The file loses that exception. The scan now requires the override to exist with ${HACK_ENV_ references, to be 0600 and git-ignored, and requires the values to reach the docker compose client env. With next's src/commands/project.ts, the scan fails.
• Legacy no-override scrub test and stale-aside sweep test (879e0b56). The stale-aside sweep (removeStaleAsides) had no test. It now takes an injectable clock (a test seam, no authority). The test proves that a fresh aside is kept, a stale one is removed, and a directory or a non-matching name is left alone. If the sweep becomes a no-op, the test fails. It also fails if the sweep ignores the aside's age.

Fix

• The override holds only Compose interpolation references, such as SECRET_TOKEN: "${HACK_ENV_<12-hex digest of service+key>_SECRET_TOKEN}". The values reach docker compose through its process environment. No Compose command writes a value to disk, so a failed, interrupted or killed up leaves nothing behind (renderComposeEnvReferenceOverride in src/lib/compose-managed-env.ts). The path stays the same because Compose records it in each container's config_files label.
• writeComposeEnvOverride (src/lib/compose-env-at-rest.ts) writes the file with replacePrivateFileAtomic. The file gets mode 0600 and a symlink at the path is refused. Text that holds a literal value is refused before anything is written.
• Adoption config hash. The projection already resolves the managed values at preview and preparation. It now also returns the reference values as a non-enumerable, never-serialized composeEnv. inspectLegacyComposeRuntimeConfig passes them only to the Compose child environment of its config --hash probe (privateInterpolationEnv on createNativeComposeProbe). A saved-generation recheck runs without a key or values by design. When the bound files include the env override, that recheck compares each container's engine label with the hash proven at preparation (meta.runtimeConfig). The Compose copy, the generated override bytes and the managed env revision stay pinned by the existing saved owners. A recreated container therefore still refuses.
• e2e harness. bootstrapOriginal now starts the original stack with the same Compose process env that Hack supplies (managedAdoptionFixtureComposeEnv). writeGenerated also checks that every reference value matches what the writer returned.
• Cleanup of files written by earlier versions:
  • The next Compose command that renders the override replaces the file.
  • A Compose command that renders no override removes the file, but only if it holds literal values.
  • hack down now removes the file first, before hooks, remote dispatch or Compose. A failed down-before hook or a remote target no longer keeps it.
  • hack doctor reports the file by counts only. hack doctor --fix removes it.
• Race-safe removal. removePlaintextComposeEnvOverride moves the entry aside with one atomic rename, then classifies the entry it moved. If a concurrent Compose command had already replaced the file with the reference form, that file is linked back without overwriting anything (a newer file wins). A value-free file is never deleted, but the path is absent between the move and the link-back. A crash between the move and the unlink leaves an aside file. A later removal deletes it once it is more than 60 s old (the clock can be injected for tests).
• Behavior change: a value that contains $ now reaches the container exactly as written.

Tests

The CLI tests use privateCliEnv and a stub docker. Their PATH is now the stub plus /usr/bin:/bin, so doctor --json no longer runs the host's mutagen or brew. No real HOME, Docker socket or keychain is used. Every value is a synthetic canary.

New in this rework:

Test	Proves	Mutation that makes it fail
native-compose-adoption-generation "v3 value-free env override binds the engine config hash computed with the values"	The stub engine hashes every ${NAME} reference with its value from the Compose env, as Compose does. The label is computed with the values and differs from the hash without them. Preview passes, prepare saves the label hash, and a saved withLease passes without a key. A changed label refuses. No value or HACK_ENV_ name reaches the manifest, the report or the docker argv.	Dropping the values at preview, or at prepare; dropping saved from the recheck; ignoring the label in saved mode (4 separate mutations, each fails)
compose-env-at-rest-cli "down … even when its down-before hook fails"	A failed hook (exit 17) still removes the literal file	The scrub moved back to after the hooks (the previous placement)
compose-env-at-rest-cli "up that renders no override …" (env config without values; no env config; legacy contract, no value for these services, new in rework 2)	All three no-override paths remove a literal file: the modern null override, the legacy empty contract, and the legacy contract whose vars apply to no running service	Removing each of the three scrub calls (each fails exactly one test)
doctor-fix-noninteractive "doctor --fix removes a literal Compose env override and keeps a reference-only one"	--fix removes the literal file, keeps the reference file and prints no value	Removing the --fix step
compose-env-at-rest "removal keeps a value-free override that replaced the inspected literal file"	A concurrent rewrite between inspection and removal is kept, with mode 0600 and no aside left	Unlinking by path without the move-aside (the 3970f17 behavior)
compose-env-at-rest "removal sweeps an aside left by a crashed removal once it is stale" (rework 2)	A fresh aside is kept, a stale one is removed, and a directory or a non-matching name is left alone	Making the sweep a no-op; ignoring the aside's age
env-secret-sentinel-scan (rework 2, tightened from #349)	No sentinel value exists in compose.env.override.yml. The file exists with ${HACK_ENV_ references, mode 0600, git-ignored. The values reach the docker compose client env	Running it with next's src/commands/project.ts, which writes the literal file

Earlier tests are kept: up/down delivery through the env with only references at rest, a failed up, an up interrupted by SIGTERM, doctor detection by counts, and the renderer, classifier and writer unit tests.

Local gates on 879e0b56 (macOS, shared host with load average 13 to 50):

• bun run --cwd packages/cli typecheck: exit 0. bun run typecheck: exit 0.
• bun run --cwd packages/cli check: exit 0, 0 errors, 34 warnings (the same count as next).
• Biome on the changed files: clean.
• Each file run in its own bun process, all green: compose-env-at-rest 10, compose-env-at-rest-cli 9, doctor-fix-noninteractive 4, tests/cli-spawn-env-isolation.test.ts 7, env-secret-sentinel-scan 6, compose-env-staleness 11, compose-managed-env 2, doctor-command 28, doctor-json 3, doctor-utils 8, native-compose-adoption-projection 17, native-compose-adoption-managed-fixture 3, native-compose-adoption-local-fixture 3, native-compose-adoption-local 42, native-compose-ownership 65, project-exec 5, project-run 10, project-up-startup-state 67, run-exec-branch-default 4, secret-key-provisioning-convergence 2.
• native-compose-adoption-generation: the six env-related cases (HACK-1273, the v3 saved, v3 drift and managed cases) pass. They were run with the file's 20 s per-test bound raised locally to 180 s, and that change was not committed. At the default bound on this host, every case timed out, including on unchanged cases, because another agent was running the same file at the same time and load average reached 40 to 50. The full file was not run green locally. CI's test job is the evidence for the whole file.
• gitleaks is not installed on this host. CI secret-scan covers it.

Hosted CI:

• On 7345de3f (run 39310729568): test passed. docker-e2e shards 1/4 to 3/4 passed. Shard 4/4 failed only in native-config-process-policy ("Native --profile must succeed", a live-endpoint-membership topology refusal on the native backend). In the same shard, native-compose-adoption-local-worktrees passed.
• On 879e0b56 (run 39439748941), at the time of writing: docker-e2e shards 2/4, 3/4 and 4/4 passed, including native-config-process-policy in shard 4/4. secret-scan, runtime-images, linux-process-lifetime, Runtime state models, Candidate core (ubuntu-latest) and three of the four Config compiler jobs passed. test, shard 1/4, Candidate core (macos-latest) and Config compiler (macos-15-intel) were still pending.

Remaining gaps

• Docker e2e was run in CI only. Shard 4/4 on 7345de3f failed in native-config-process-policy. That scenario uses the native backend, not the Compose env override, and the same shard passed on bebeb89c. The exact-head gate is the CI run on 879e0b56, which includes the merge with next. I did not dispatch any workflow.
• Transient absence during a raced removal. When a removal moves aside a file that a concurrent Hack command has just rewritten value-free, the path is absent until the link-back. A Compose process that opens it in that moment fails and must be retried. POSIX has no "unlink if still this inode", so the design accepts this window. It is documented in docs/env.md and threat-model gap 4.
• Aside left by a crash. If Hack is killed between the move and the unlink, .hack/.internal/.compose.env.override.yml.<uuid>.remove can hold literal values. It is swept only by a later removal (a no-override Compose command, hack down or hack doctor --fix) once it is more than 60 s old. hack doctor does not report it, and an up that renders the override does not sweep it. This is documented.
• Adoption test budget. The HACK-1273 adoption case takes about 16 s alone at load average 14, and the file's per-test bound is 20 s. Like the existing v3 cases, it can time out on a saturated host.
• Compose client env size. The docker compose client process carries one HACK_ENV_* copy of each value per service, so its environment grows with services × keys. On macOS the total limit is about 1 MiB. On Linux, a single NAME=value string over 128 KiB (MAX_ARG_STRLEN) now fails with E2BIG, where the old file path worked. This is documented. A shared name per key for globally inherited values is not done. serviceEnv already merges global values into every service, so sharing names would need scope metadata in the renderer and the adoption projection.
• Direct docker compose use. A docker compose -f .hack/.internal/compose.env.override.yml … run outside Hack gets blank values with only per-variable warnings. This is documented in docs/env.md.
• Saved adoption recheck. The recheck now trusts the preparation-time config hash plus the pinned file bytes and env revision. It no longer recomputes the hash with Compose. A change in Compose's own hashing would therefore not be noticed on that path.
• Older plaintext elsewhere. Plaintext that an earlier version wrote can survive in filesystem snapshots or backups. Rotate the affected secrets if that exposure matters. Running containers still expose their env through docker inspect (gap 9).
• No real-engine check here. This branch was not run on the M3/Omega host or on a real local Docker engine. The M3 evidence was read without changes (~/.hack-v5-private/hack1124-next5-rerun-20261019-1204/). It confirms the original defect on published next.5: cleanup-omega.txt lists a 71,130-byte, 0600 compose.env.override.yml after down. That file is no longer at ~/dev/hack-dance/omega/.hack/.internal/ (checked with stat only; no content was read).

Refs HACK-1273

🤖 Generated with Claude Code

[linear-code]

HACK-1273