Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitleaksignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@
cbad51795170d25e1ab0926c1ac5a74d1e001ef9:.factory/validation/misc-admin-followups-1/user-testing/flows/management-token-email-verified.json:generic-api-key:8
e069b8546c112e5b5c8ab0ec82fd24a4f19ef16b:.hack/tickets/events/events-2026-03.jsonl:generic-api-key:58
939c2b2cb680744707b1d085db0bc0ce8fac32e4:tests/env.test.ts:private-key:7
06f262309db3111e65c8a23324456ef73ac57e83:tests/project-env-hydrate.test.ts:generic-api-key:389
81 changes: 80 additions & 1 deletion docs/env.md
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,7 @@ into a host command, and `--target` on env and host commands chooses the `host`
| `hack env apply` | `--env`, `--service` | Runs `hack restart <service>` with the same `--env`, recreating one service. | This is the refresh path for one service. | `--service` is required. |
| `hack env add` | `--env`, `--service` | Selects the file written (`hack.env.<name>.yaml`, or the base file for `base`/`default`/`none`; `--local` writes the local variant). | Running workloads keep old values until `hack env apply` or `hack restart`. | Mutates only env files. |
| `hack env set` | `--env`, `--service` | Alias for `hack env add`. | Same as `hack env add`. | Mutates only env files. |
| `hack env hydrate` | `--env`, `--service` | Selects the local file written, as for `hack env add --local`; imports every entry in one commit. | Same as `hack env add`. | Mutates only the machine-local env file; refuses a git-tracked one. |
| `hack env unset` | `--env`, `--service` | Selects the file the key is removed from, as for `hack env add`. | Same as `hack env add`. | Mutates only env files. |
| `hack env materialize` | `--env`, `--service` | Writes `.hack/.env` once, at invocation. | Rerun `hack env materialize`. | Compatibility output for external tools; runtime injection does not depend on it. |
| `hack env exec` | `--env`, `--service`, `--target` | Per invocation, into one host command (host view by default). | Each invocation resolves again. | Runs on the host, not in a container. |
Expand Down Expand Up @@ -420,7 +421,8 @@ A hand-edited file (comments, other formatting, or no `global: {}`) is normalize
first repeat and stays byte-identical after that. A different value, a plain entry
becoming secret, or an entry that does not decrypt under the current key gets new
ciphertext with a fresh IV. Each command writes one key; a script that sets several keys
is not atomic across them.
is not atomic across them. To write many machine-local values in one commit, use
`hack env hydrate` (below).

Each env file write goes to a hidden `.<file>.<id>.tmp` file in the same directory and
replaces the file by rename, so an interrupted write leaves the previous file in place.
Expand Down Expand Up @@ -459,6 +461,83 @@ hack env unset SERVICE_TOKEN --service api --env qa
hack env unset API_BASE_URL --local
```

### Hydrate machine-local values in one step

`hack env hydrate` imports many values into the machine-local layer at once, for example
when a new checkout or machine pulls its secrets from a secret manager. Tracked defaults
stay in `hack.env.default.yaml`; hydration writes only the gitignored local layer
(`.hack/hack.env.local.yaml`, or `.hack/hack.env.<overlay>.local.yaml` with `--env`).

```bash
# Pipe KEY=value lines from a secret manager; values never appear in argv.
<secret-source> | hack env hydrate --secret

# Or read a file, or a pipe such as <(secret-source).
hack env hydrate --secret --file .secrets.env
hack env hydrate --secret --file secrets.json --format json --env qa --service api
```

Input comes only from piped stdin or `--file`, never from arguments, and is at most
1 MiB. The default `env` format is strict, one entry per line:

- blank lines and lines starting with `#` are skipped, and an `export ` prefix is allowed
- `KEY=value`, where the key matches `^[A-Z_][A-Z0-9_]*$`; an unquoted value is the
rest of the line with surrounding spaces removed, and `#` inside it is part of the value
- `'...'` is literal; `"..."` accepts `\n`, `\r`, `\t`, `\"` and `\\`; a quoted value
must close on the same line, followed only by spaces or a `# comment`
- empty values, repeated keys and `${VAR}` expansion are not supported

`--format json` takes one object whose values are non-empty strings. A key that
appears twice is refused, as in the `env` format; it is not resolved by keeping the
last value. One malformed entry refuses the whole input. The message gives only the line number, never the
line, and nothing is written.

`--secret` encrypts every imported value; without it the values are stored as plain
entries in the local layer. A plain import refuses to replace a key that the local
layer already stores as a secret, so repeat an import with the same flags it was first
run with. `--service` picks one scope for all entries (default
`global`). The output and `--json` result list the added, updated and unchanged keys,
never values.

Hydration is idempotent and all-or-nothing:

- An entry that already holds its value is left alone: a secret keeps its ciphertext
and a plain number keeps its YAML type. When nothing changes, the file is not
rewritten, so comments and formatting survive and Git shows no change.
- A changed value gets new ciphertext with a fresh IV.
- Local entries that are not in the input, other scopes, and removal markers for other
keys are kept. A key in the input replaces that key's local entry, including a
value you set by hand with `--local`.
- The whole new layer is written to a staging file and published by one rename, so an
interrupted hydration leaves the previous layer, with every key at its old value.

Hydration refuses, without changing the layer, when:

- git tracks the target local layer
- the project is in a git checkout but git cannot say whether it tracks the local layer,
for example after a `safe.directory` refusal, with a damaged index, or with git not
installed (outside any git checkout, hydration proceeds)
- the local layer exists but is not a valid env file, for example invalid YAML, the
wrong `version`, or a scope that is not a mapping; fix or remove it first
- a secret already in the local layer does not decrypt under the project env key, or no
key is available (`.hack.secret.key` is used before `HACK_ENV_SECRET_KEY`, as when
values are read)
- `--secret` is used, no `.hack.secret.key` exists, and `HACK_ENV_SECRET_KEY` is set
(a new key file would replace that key for every layer)
- `--secret` is not used and an imported key is stored as a secret in the target scope;
re-run with `--secret`, or remove the key with `hack env unset <key> --local` first
- another `hack env hydrate` for the same project is running, or an interrupted one left
a stale lock (see below)
- the local layer changed between hydration's read and its rename, for example from a
concurrent `hack env set --local`

Concurrent hydrations are serialized by `.hack/.internal/env-hydrate.lock`, which holds
the owner's process ID. A hydration that is killed (for example with SIGKILL) cannot
remove it; the next hydration then reports the stale lock and its path instead of taking
it over. Remove the file once no hydration is running. `hack env set --local` does not
take this lock. Hydration checks the layer again just before its rename, which narrows
the window in which such a write could be overwritten but cannot close it.

Materialize a compatibility `.hack/.env`:

```bash
Expand Down
29 changes: 29 additions & 0 deletions docs/reference/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -2260,6 +2260,7 @@ hack env <subcommand> [options]
| `hack env apply` | Recreate one service with its resolved env |
| `hack env add [key] [value]` | Add or update an env value |
| `hack env set [key] [value]` | Alias for env add |
| `hack env hydrate` | Import many values into the machine-local env layer at once |
| `hack env materialize` | Write a compatibility .env file from the selected overlay |
| `hack env exec [command...]` | Run a host command with project env injected |
| `hack env shell` | Open a host shell with project env injected |
Expand Down Expand Up @@ -2449,6 +2450,34 @@ Same value rules as `hack env add`: pipe secrets with --value-stdin or use the h
| `--help, -h` | Show help |
| `--version, -v` | Show version |

## `hack env hydrate`

Import many values into the machine-local env layer at once

### Usage

```bash
hack env hydrate [options]
```

Reads KEY=value entries from piped stdin or --file (never from arguments) and writes them, in one atomic commit, to the worktree-local env file (`hack.env.local.yaml`, or `hack.env.<overlay>.local.yaml` with --env). Tracked env files are never written. Repeating an import with the same values changes nothing; other local entries are kept. Values are never printed.

### Options

| Option | Description |
| --- | --- |
| `--path, -p <dir>` | Run a project command against a repo path (overrides cwd search) |
| `--project <name>` | Target a registered project by name (from ~/.hack/projects.json) |
| `--env <name|base|default>` | Select an env overlay by name; omit for the configured default overlay, or use 'base' or 'default' for base layers only |
| `--json` | Output JSON (machine-readable) |
| `--secret` | Store value as an encrypted secure entry |
| `--service <global|service>` | Target scope (global or a discovered service name) |
| `--file <path>` | Read entries from this file (or a pipe such as <(producer)) instead of stdin |
| `--format <env|json>` | Input format: KEY=value lines (env, default) or one JSON object of string values (json) |
| `--no-interactive` | Never prompt: apply documented defaults or fail with E_INTERACTIVE_REQUIRED (also via HACK_NO_INTERACTIVE=1) |
| `--help, -h` | Show help |
| `--version, -v` | Show version |

## `hack env materialize`

Write a compatibility .env file from the selected overlay
Expand Down
12 changes: 6 additions & 6 deletions plugins/hack/skills/hack-cli/references/patterns.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,13 +135,13 @@ missing part with ad hoc scripts.
## HACK-P08: Tracked defaults plus idempotent machine-local hydration

- Use when: A repository tracks env defaults and each machine adds local values on top.
- Disposition: Partial. Missing: No-op detection and rename-committed writes cover one key per command. Hack has no multi-key import: an interrupted hydration can leave some keys updated and others not.
- Disposition: Partial. Missing: Hydration replaces an imported key's local entry; it cannot tell an operator's deliberate local override of that key from a stale hydrated value. `hack env set --local` does not take the hydration lock, so only a recheck just before the rename guards against it.
- Owner: WU27/WU14; tracked in HACK-1152.
- Commands: `hack env set <key> --local --value-stdin --secret`, `hack env explain <key> --json`
- Commands: `hack env hydrate --file <path> --secret`, `hack env set <key> --local --value-stdin --secret`, `hack env explain <key> --json`
- Prerequisites: Tracked defaults, a machine-local layer and preserved operator overrides.
- Env ownership: Defaults are tracked; hydration runs `hack env set <key> --local --value-stdin --secret` once per key with the value piped on stdin, so values stay out of argv. Without `--local` the same command writes a tracked default.
- Readiness: Setting a secret to its current value keeps the stored ciphertext, so repeat hydration leaves the tracked tree clean; a changed value gets new ciphertext with a fresh IV. Each layer write commits by rename, so an interrupted write leaves the previous layer resolvable.
- Cleanup: Unchanged values cause no rewrite.
- Env ownership: Defaults are tracked; `hack env hydrate --secret` reads `KEY=value` entries from piped stdin or `--file` and writes only the gitignored machine-local layer, so values stay out of argv and out of tracked files. Local entries outside the input are kept.
- Readiness: A repeat import with the same values does not rewrite the layer; a changed value gets new ciphertext with a fresh IV. One rename commits all keys, so an interrupted import leaves every key at its previous value. A wrong key, a tracked local layer or a concurrent hydration refuses without changing the layer.
- Cleanup: Unchanged values cause no rewrite. A killed import can leave a staging file and a stale `.hack/.internal/env-hydrate.lock`; the next import reports the lock instead of taking it over.
- Provenance: single-consumer generalization (HACK-P08 research register).

<!-- negative-example -->
Expand Down Expand Up @@ -214,4 +214,4 @@ missing part with ad hoc scripts.

---

Content revision: `247c8f20c3c8`. Generated by Hack; edit the canonical source and regenerate the plugin.
Content revision: `3bb87b99470a`. Generated by Hack; edit the canonical source and regenerate the plugin.
10 changes: 6 additions & 4 deletions src/agents/pattern-catalogue.ts
Original file line number Diff line number Diff line change
Expand Up @@ -272,21 +272,23 @@ export const PATTERN_CATALOGUE: readonly PatternCard[] = [
"A repository tracks env defaults and each machine adds local values on top.",
disposition: "partial",
supportedCommands: [
"hack env hydrate --file <path> --secret",
"hack env set <key> --local --value-stdin --secret",
"hack env explain <key> --json",
],
prerequisites:
"Tracked defaults, a machine-local layer and preserved operator overrides.",
envOwnership:
"Defaults are tracked; hydration runs `hack env set <key> --local --value-stdin --secret` once per key with the value piped on stdin, so values stay out of argv. Without `--local` the same command writes a tracked default.",
"Defaults are tracked; `hack env hydrate --secret` reads `KEY=value` entries from piped stdin or `--file` and writes only the gitignored machine-local layer, so values stay out of argv and out of tracked files. Local entries outside the input are kept.",
readiness:
"Setting a secret to its current value keeps the stored ciphertext, so repeat hydration leaves the tracked tree clean; a changed value gets new ciphertext with a fresh IV. Each layer write commits by rename, so an interrupted write leaves the previous layer resolvable.",
cleanup: "Unchanged values cause no rewrite.",
"A repeat import with the same values does not rewrite the layer; a changed value gets new ciphertext with a fresh IV. One rename commits all keys, so an interrupted import leaves every key at its previous value. A wrong key, a tracked local layer or a concurrent hydration refuses without changing the layer.",
cleanup:
"Unchanged values cause no rewrite. A killed import can leave a staging file and a stale `.hack/.internal/env-hydrate.lock`; the next import reports the lock instead of taking it over.",
negativeControl:
"Re-encrypting unchanged entries causes diff and stash churn, and values passed in command arguments are not a secure import mechanism.",
domainOwner: "WU27/WU14",
missingContract:
"No-op detection and rename-committed writes cover one key per command. Hack has no multi-key import: an interrupted hydration can leave some keys updated and others not.",
"Hydration replaces an imported key's local entry; it cannot tell an operator's deliberate local override of that key from a stale hydrated value. `hack env set --local` does not take the hydration lock, so only a recheck just before the rename guards against it.",
provenance: { consumers: 1, source: "HACK-P08 research register" },
example: null,
e2eScenario: null,
Expand Down
7 changes: 5 additions & 2 deletions src/cli/command.ts
Original file line number Diff line number Diff line change
Expand Up @@ -448,8 +448,11 @@ export function parsePositionalsForCommand<
}

if (idx < remaining.length) {
const extra = remaining.slice(idx).join(" ");
throw new CliUsageError(`Unexpected arguments: ${extra}`);
// Count only: an extra argument is often a pasted `KEY=value` or secret.
const extra = remaining.length - idx;
throw new CliUsageError(
`Unexpected arguments: ${extra} more than this command accepts. They are not shown because they may contain values.`
);
}

return out as PositionalsValues<Pos>;
Expand Down
Loading
Loading