fix(env): keep decrypted Compose env values off disk

docs/architecture.md

@@ -113,8 +113,8 @@ Projects can declare shareable env overlays and safely inject secrets into compo
 At runtime, hack resolves the selected overlay and injects the effective env directly into compose and
 host-side command flows. For service-scoped values, hack still generates
 `.hack/.internal/compose.env.override.yml` so compose services get the right per-service environment
-without requiring `.hack/.env` to exist. That file holds resolved plaintext, so it is written like
-`.hack/.env`: atomically, mode `0600`, and never through a symlink.
+without requiring `.hack/.env` to exist. That file holds only `${HACK_ENV_...}` references; the values
+reach `docker compose` through its process environment, so no decrypted value rests on disk.
 
 See `docs/env.md` for the current env model and migration notes.
 

docs/env.md

@@ -196,6 +196,37 @@ Hack reads the canonical YAML files and injects the resolved env directly into:
 
 That means `.hack/.env` is no longer the primary runtime source of truth.
 
+On the Compose backend, Hack still generates `.hack/.internal/compose.env.override.yml`
+so each service gets its own environment. The file holds no values: each entry is a
+`${HACK_ENV_<digest>_<KEY>}` reference, and Hack passes the resolved values to
+`docker compose` through its process environment. Nothing Hack writes there contains a
+decrypted value, whether `up` succeeds, fails or is interrupted. The file is written
+with mode 0600, and Hack refuses to write through a symlink at that path. A value
+reaches the container literally, `$` included.
+
+Only the `docker compose` invocations that read the override (`up`, `run`, `exec`,
+`ps` and the dependency cache initialization) receive the `HACK_ENV_*` values. The other
+`docker` commands Hack runs around Compose, such as `docker inspect`, `docker logs` and
+`docker events` for dependency cache progress and reuse, get the same environment
+without them.
+
+Because the file holds only references, a `docker compose -f
+.hack/.internal/compose.env.override.yml ...` command run outside Hack (a script, an IDE
+integration or a test harness) gets a blank value for every managed variable, with only a
+per-variable warning from Compose. Use `hack run` or `hack exec`, or supply the same
+`HACK_ENV_*` variables to that command.
+
+Every service gets its own `HACK_ENV_*` copy of each value, including values that come
+from the global scope, so the Compose client environment grows with services times keys.
+The platform limits apply to it: about 1 MiB of arguments and environment in total on
+macOS, and on Linux also 128 KiB (`MAX_ARG_STRLEN`) for a single `NAME=value` string. A
+larger single value, such as a big JSON credential, makes the Compose start fail with
+`E2BIG`.
+
+When you upgrade from an earlier version, read the two subsections at the end of this
+section: managed values that contain `$` now reach containers differently, and an
+override written by the earlier version can still hold values.
+
 `hack env materialize` is manual by design. Use it only when you need a compatibility file for an external tool that expects `.env` on disk.
 
 Hack writes `.hack/.env` and `.hack/.env.<overlay>` (from `materialize`, legacy
@@ -216,6 +247,53 @@ That means Hack prefers host-usable values when it can:
 
 Use `--target compose` if you explicitly want the container-oriented compose view instead.
 
+### Upgrading: managed values that contain `$`
+
+Earlier versions wrote each value into the override literally, so Compose interpolated
+a `$` inside a managed value: `$VAR` and `${VAR}` became the host or Compose value of
+`VAR` (blank when unset), and `$$` became `$`. Now Compose substitutes each reference
+once and does not interpolate the value, so the container receives the value exactly as
+stored, `$` included. If a managed value contains `$`:
+
+- the container sees a different value after the upgrade, and the next `hack up`
+  recreates that service's containers because their configuration changed;
+- a value that relied on interpolation (for example `${HOME}/data`) now reaches the
+  container unexpanded; store the expanded value instead;
+- a value that escaped `$` as `$$` now reaches the container as `$$`; store a single `$`.
+
+Values without `$` reach containers unchanged, and their containers are not recreated.
+
+### Overrides from earlier versions
+
+An override written by an earlier Hack version can still hold literal values. The next
+`hack up`, `hack restart`, `hack run` or `hack exec` replaces it or, when it renders no
+override, removes it. `hack down` removes it before its lifecycle hooks, the remote target
+or `docker compose down` run, so a failed hook, a remote target or a failed Compose down
+does not keep it. `hack doctor` reports such a file in its "compose env at rest" check
+(counts only, never values), and `hack doctor --fix` removes it.
+
+A legacy adoption (`hack config adopt`) pins the override's exact bytes and file
+identity for as long as the adoption exists, and adopted commands never rewrite it. So:
+
+- `hack config adopt` and `hack config adopt --dry-run` refuse a literal override. Run
+  `hack up` once so Hack rewrites it with references, then adopt.
+- An adoption made by an earlier version over a literal override keeps that file. Hack
+  never removes it while a prepared, active or pending adoption exists, because removal
+  would make the adoption refuse to load. `hack doctor` reports it with this next step.
+  To drop the values, stop the adopted containers (`hack down`), run
+  `hack config adopt --rollback`, run `hack up` so Hack rewrites the file with references,
+  then adopt again with `hack config adopt --stop`.
+
+A removal first moves the file aside, then checks what it moved. If a concurrent Hack
+command had just replaced it with a value-free file, that file is put back rather than
+deleted. Between the move and the put-back the path is briefly absent, so a Compose
+command started by that concurrent Hack command at that moment can fail to find it; run
+the command again. If Hack is killed between the move and the delete, the moved file
+(`.hack/.internal/.compose.env.override.yml.<id>.remove`) can hold values. `hack doctor`
+reports such files by count, and the next removal, `hack doctor --fix`, or `hack up`,
+`hack restart`, `hack run` or `hack exec` deletes each one once it is more than a minute
+old. A newer one may belong to a removal that is still running, so it is left alone.
+
 ## Choosing the right execution surface
 
 - `hack host exec` / `hack host shell`: run on your host machine with Hack-resolved env injected.