Skip to content

Fix #1219: sanitize inherited CLAUDE_CODE_* env at every Tower spawn - #1626

Open
waleedkadous wants to merge 15 commits into
mainfrom
builder/bugfix-1219
Open

Fix #1219: sanitize inherited CLAUDE_CODE_* env at every Tower spawn#1626
waleedkadous wants to merge 15 commits into
mainfrom
builder/bugfix-1219

Conversation

@waleedkadous

@waleedkadous waleedkadous commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Fixes #1219

Summary

A Tower started from inside a Claude Code session inherits that session's identity markers and hands them to every agent it spawns. Each agent then believes it is a nested child session, disables transcript saving, and becomes unresumable — silently, until crash recovery fails. This routes every Tower-descendant spawn through one sanitizer that strips per-session identity, and adds a codev doctor check for daemons that are already contaminated.

Root Cause

Claude Code plants session markers — CLAUDE_CODE_CHILD_SESSION, CLAUDE_CODE_SESSION_ID, CLAUDE_CODE_ENTRYPOINT, CLAUDE_CODE_EXECPATH, and friends — into every subprocess it creates, so a nested claude can tell it was launched from inside another session. Tower is a daemon, but it inherits the environment of whoever started it, and starting Tower from inside a Claude Code session is routine (afx tower start, pnpm -w run local-install, an architect recovering from a crash). The markers cascade:

claude session → Bash → afx tower start → Tower → shellper → agent claude

Confirmed on a live install rather than assumed: ps eww on two running Tower daemons showed the full marker set, inherited from the shell that started them.

Three code shapes, none of which handled CLAUDE_CODE_*:

  1. towerStart() daemonized with env: process.env (agent-farm/commands/tower.ts).
  2. Six Tower spawn sites built { ...process.env } and deleted only CLAUDECODEtower-instances.ts ×2, tower-terminals.ts ×2, tower-routes.ts ×2.
  3. createPtySession() sends no env, so builder terminals took the env || process.env branch in handleTerminalCreate and received Tower's whole environment.

shellper-main passes its config env straight to node-pty, which replaces the child environment wholesale — so the env Tower hands to SessionManager.createSession is the agent's env. That makes (2)+(3) load-bearing and (1) the upstream source.

Fix

New packages/codev/src/lib/agent-env.ts with a single sanitizeAgentEnv(): strip what is per-session identity, keep everything else.

Identity is a small, stable, nameable set — four prefix families (CLAUDE_CODE_SESSION_, CLAUDE_CODE_REMOTE_SESSION_, CLAUDE_CODE_BRIDGE_, CLAUDE_CODE_MESSAGING_) plus ten individual names, including the CHILD_SESSION / ENTRYPOINT / EXECPATH nesting triple this issue is about. Configuration is not a nameable set, so it is not enumerated.

Every Tower-descendant spawn routes through it — the daemon spawn, both architect-launch sites, both architect-reconnect sites, terminal-create and shell-create (plus their non-persistent fallbacks), the shellper daemon spawn, and cron task execution. The six hand-rolled delete cleanEnv['CLAUDECODE'] copies collapse into the one helper. afx tower start logs what it scrubbed, and codev doctor grows a Tower Environment section that reads the running daemon's env via ps eww — a daemon started before this fix stays contaminated until restarted, and that was previously invisible.

A supporting commit moves getProcessesOnPort to a leaf agent-farm/utils/port.ts (re-exported, no consumer moves): importing it from the command module dragged utils/shell.ts's module-scope promisify(exec) into the doctor tests, whose node:child_process mock has no exec export.

Why not deny the whole CLAUDE_CODE_* namespace

The first version did exactly that, with a short allowlist, on the theory that a missed marker is worse than a missed config var. The codex CMAP lane rejected it and was right. Checking the shipped claude binary (2.1.261) settled it: it reads 594 distinct CLAUDE_CODE_* variables, overwhelmingly configuration. The allowlist would have silently dropped USE_FOUNDRY, USE_MANTLE, USE_ANTHROPIC_AWS, USE_GATEWAY, SKIP_FOUNDRY_AUTH, SKIP_MANTLE_AUTH, OAUTH_REFRESH_TOKEN, OAUTH_SCOPES, OAUTH_CLIENT_ID, API_BASE_URL, PROXY_URL, HTTPS_PROXY, CLIENT_CERT, MANAGED_SETTINGS_PATH and more.

The original reasoning was wrong about the failure mode: dropping a config var is not harmless. It points an agent at the wrong provider, or strips its subscription credential and reroutes CMAP to the metered API — the #985 scar. No hand-maintained allowlist survives 594 variables.

CLAUDE_PID disposition

Checked against the shipped binary the same way as the 594: CLAUDE_PID is stripped.

It is not read for nested-session detection — its only reader is the pkill guard in the Bash tool's shell prelude, which refuses a pattern matching the Claude CLI's own pid. But it is per-session identity by construction, planted in the same object literal as the marker this issue is about:

function qOe(e){let t={CLAUDECODE:"1", CLAUDE_CODE_SESSION_ID:e.sessionId,
                       CLAUDE_CODE_CHILD_SESSION:"1", CLAUDE_PID:String(process.pid)};
  if(e.source==="agent") t.AI_AGENT=;
  if(e.effortLevel!==void 0) t.CLAUDE_EFFORT=e.effortLevel; }

String(process.pid) of one specific process is identity by any reading, and an inherited stale value points that pkill guard at a dead or unrelated pid. So it is in the denylist despite sitting outside the CLAUDE_CODE_ namespace.

CLAUDE_EFFORT is planted by that same function and is deliberately not stripped — which is the useful part of the check: "Claude Code sets it" is not the test. It carries a setting, not an identity, so it is configuration and passes through.

Test Plan

  • Regression test added — bugfix-1219-claude-env-leak.test.ts, 31 cases. 9 assertions fail with the spawn-site changes reverted to the pre-fix commit and pass with them; reverting just one of tower-routes.ts's four sites fails 2.
  • Build passes
  • All tests pass — 5739 passing, 48 skipped

Covered by the regression test:

  • Every marker observed on a real contaminated Tower is stripped.
  • Whole session families are stripped, not just the names seen so far.
  • A MUST_SURVIVE fixture of 24 provider / auth / routing / policy variables — each verified present in the shipped binary, including the Foundry, Mantle, Anthropic-AWS and refresh-token cases — is preserved intact.
  • An unrecognised CLAUDE_CODE_* variable is kept, not dropped.
  • Every listed spawn site calls the sanitizer the expected number of times, and carries none of the raw-environment shapes a reverted or copy-pasted site would take.
  • codev doctor warns when Tower is clean but shellpers it spawned earlier still carry markers, and never claims a plain Tower restart is sufficient on its own.

Verified beyond the suite:

  • Doctor check against live processes: a real contaminated Tower → warn naming all 7 markers; a dead pid → skipped. On this machine the session check found the exact case it exists for — 71 shellpers running, 5 still carrying markers, Tower itself clean, which the daemon-only version called ok.
  • End to end: a Tower started from a shell carrying 7 markers logged Scrubbed inherited Claude Code session markers: … and came up with a clean environment. That run is the incident described below — it is reported there rather than claimed here as a clean result. Field verification will be redone after merge via local-install on main.

Incident during verification

While verifying this fix I took down every terminal on the development machine. Recording it here because it is the kind of failure this PR's own subject matter — processes quietly attaching to state they should not touch — is about.

I started a test Tower on port 14733 to confirm the daemon scrub end to end, and exported AGENT_FARM_DIR to isolate it. That is not the override. The real one is CODEV_AGENT_FARM_DIR (packages/core/src/constants.ts:18, added by #1515). So my test Tower opened the production ~/.agent-farm/global.db and appended to the production tower.log.

Its startup reconcile then found the 56 live terminal rows and connected to every production shellper socket. A shellper holds one client, so the real Tower on 4100 lost all of them; the two Towers fought over reconnects, and the reconcile's stale-row path deleted 53 of the 56 rows. When I killed the test Tower those sessions were orphaned: 60 unregistered shellpers, and every architect terminal across 12 workspaces gone from the dashboard — including the architect's own. The architect restored the rows by hand from the checkpointed DB file and restarted Tower.

The log corroborates it: 56 reconcile-adopt lines at 18:26:59.4xx, then 54 Session … removed but shellper pid=… is alive; preserving socket lines by 18:27:02.5xx.

My gate message described this as having "briefly contended with the live Tower for the cloud tunnel. It recovered." That was wrong — I reported the tunnel flapping I had noticed and not the session loss I had not looked for, which made a fleet-wide outage read as a transient blip.

#1629 is filed for the systemic defects, and they are the real lesson: a second Tower must not be able to open a live global.db and adopt rows owned by a running Tower (owner lock, or refuse to adopt), and an isolation env var whose misspelling silently means "use production" is a trap rather than a safeguard. My mistake was the trigger; neither of those should be reachable by one wrong variable name.

No Tower has been started from this lane since. Field verification of the daemon scrub is deferred to local-install on main after merge.

Known follow-ups (filed, not in this PR)

One limit worth stating rather than filing: codev doctor classifies with the same list the sanitizer uses, so it cannot backstop a marker Claude Code adds that this list has not caught up with. Claude Code's own "transcript saving is off" banner remains the only signal for that case — it is how this issue was found.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ApsmgthG7VWgMM2JaizWD6

waleedkadous and others added 11 commits September 5, 2026 11:12
Claude Code plants session markers (CLAUDE_CODE_CHILD_SESSION,
CLAUDE_CODE_SESSION_ID, ...) into every subprocess it creates, so a nested
claude can tell it was launched from inside another session and turn transcript
saving off. Tower is a daemon but inherits the env of whoever started it, and
starting Tower from inside a Claude Code session is routine (afx tower start,
pnpm -w run local-install, an architect recovering from a crash). The markers
then cascade — Tower -> shellper -> agent claude — and every agent Tower spawns
believes it is a nested child session. A session with transcript saving off
cannot be resumed, so a Tower that was ever restarted from inside a Claude
session quietly produces agents that are unrecoverable; the failure only
surfaces at crash-recovery time.

Confirmed on a live install: `ps eww` on two running Tower daemons showed the
full marker set, inherited from the shell that started them.

Root cause was three shapes, none of which handled CLAUDE_CODE_*:
- towerStart() daemonized with `env: process.env`
- six Tower spawn sites built `{ ...process.env }` and deleted only CLAUDECODE
- createPtySession() sends no env, so builder terminals took the
  `env || process.env` branch and got Tower's whole environment

Fix:
- New src/lib/agent-env.ts: one sanitizer, deny-by-default over the
  CLAUDE_CODE_* namespace so a marker added upstream tomorrow is stripped
  without a code change, with an allowlist for the config/auth vars that must
  reach an agent. CLAUDE_CODE_OAUTH_TOKEN above all — consult reads it from the
  agent's own env to keep CMAP on the subscription rather than the metered API
  (#985), so a blanket strip would have been a silent regression.
- Route all six spawn sites, the daemon spawn, and the shellper spawn through
  it. The per-site `delete cleanEnv['CLAUDECODE']` copies collapse into it.
- `afx tower start` logs which markers it scrubbed when it finds any.
- New `codev doctor` section reads the running Tower's env via `ps eww` and
  warns if it is carrying markers — a daemon started before this fix stays
  contaminated until restarted, and that was previously invisible.

Regression test pins the sanitizer's behaviour (including the OAuth-token
carve-out and the deny-by-default rule), the doctor check, and the fact that
every spawn site routes through the sanitizer rather than hand-rolling a
CLAUDECODE-only strip.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ApsmgthG7VWgMM2JaizWD6
`codev doctor` needs the running Tower's PID, and importing it from
`agent-farm/commands/tower.ts` dragged that command's import graph into
doctor.ts — including `utils/shell.ts`, which calls `promisify(exec)` at module
scope. `doctor.test.ts` mocks `node:child_process` without an `exec` export, so
all 24 of its cases failed at import time.

Move the function to `agent-farm/utils/port.ts` (execSync only, no module-scope
side effects) and re-export it from `commands/tower.ts` so existing consumers,
including tower-stop.test.ts, are untouched. doctor.ts takes DEFAULT_TOWER_PORT
straight from `@cluesmith/codev-sdk/constants` for the same reason.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ApsmgthG7VWgMM2JaizWD6
`runCommand` in tower-cron.ts spawned every cron task with raw `process.env` —
the one Tower-descendant spawn the first pass missed. A cron task runs an
arbitrary command, which may itself be or launch an agent, so it gets the same
rule as every other spawn site. Added to the regression test's SPAWN_SITES so
the source-shape guard covers it.

Second-order once Tower's own env is clean, but the point of routing every site
through one helper is that no site has to be reasoned about individually.

Found by the claude CMAP lane.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ApsmgthG7VWgMM2JaizWD6
…_* namespace

The codex CMAP lane returned REQUEST_CHANGES on the deny-by-default policy, and
it was right. Checking the shipped claude binary (2.1.261) settled it: it reads
594 distinct CLAUDE_CODE_* variables, overwhelmingly configuration. Every name
codex cited exists — CLAUDE_CODE_USE_FOUNDRY, USE_MANTLE, USE_ANTHROPIC_AWS,
SKIP_FOUNDRY_AUTH, OAUTH_REFRESH_TOKEN, OAUTH_SCOPES — and the allowlist would
also have dropped API_BASE_URL, PROXY_URL, HTTPS_PROXY, CLIENT_CERT,
MANAGED_SETTINGS_PATH, OAUTH_CLIENT_ID, SKIP_MANTLE_AUTH, USE_GATEWAY and more.

The original reasoning assumed dropping a config var was harmless. It is not:
it silently routes an agent at the wrong provider, or strips its credential and
reroutes CMAP to the metered API — the #985 scar. No hand-maintained allowlist
survives 594 variables.

Inverted: strip what is per-session identity, keep everything else. Identity is
a small, stable, nameable set — four prefix families (SESSION_, REMOTE_SESSION_,
BRIDGE_, MESSAGING_) plus ten individual names including the CHILD_SESSION /
ENTRYPOINT / EXECPATH nesting triple this issue is actually about. Configuration
is not nameable, so it is no longer enumerated.

Tests updated to match: a MUST_SURVIVE fixture of 24 config/auth variables (each
verified present in the binary) that sanitization must preserve, family-level
stripping, and an unrecognised CLAUDE_CODE_* now asserted to be kept rather than
dropped.

Re-verified against live processes: the narrower classifier still flags all 7
markers on a real contaminated Tower, still passes the clean one. Build green,
5735 tests passing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ApsmgthG7VWgMM2JaizWD6
Second REQUEST_CHANGES from the codex CMAP lane, and right again. `towerStop`
deliberately leaves shellpers running (they are detached so terminals survive a
restart), so after `afx tower stop && afx tower start` the daemon reads clean
while every agent the contaminated Tower spawned is still running with the
markers — still unresumable. The check inspected only the daemon, so it reported
"clean" in exactly that state, and the remediation text said "restart Tower" as
if that were the whole fix.

- checkTowerEnv() takes a second reader for the running shellper envs and warns
  when Tower is clean but sessions it spawned earlier are not.
- Two remediation strings: one for a contaminated daemon (restart Tower AND each
  affected agent, saying plainly that a plain restart leaves sessions running),
  one for the clean-Tower-dirty-sessions case, naming
  --force-kill-all-child-processes with its blast radius rather than
  recommending it.
- findShellperPids() locates them via `pgrep -f shellper-main.js`.

Verified on this machine, which was already in the reported state: 71 shellpers
running, 5 still carrying markers, Tower itself clean. The old check called that
"ok"; the new one warns.

Also corrects an overclaim in the agent-env header caught by the claude lane: it
said doctor backstops an unknown upstream marker. It cannot — doctor classifies
with the same list, so it is blind to exactly what it would need to catch. The
header now says so, and names Claude Code's own banner as the only real signal.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ApsmgthG7VWgMM2JaizWD6
Both the codex and claude CMAP lanes made the same point: asserting that a file
mentions sanitizeAgentEnv somewhere would let ONE of tower-routes.ts's four
spawn sites revert while the other three kept the test green.

SPAWN_SITES now carries an expected call count per file, and a FORBIDDEN list
pins the shapes a reverted or copy-pasted site would take (`env: process.env`,
`{ ...process.env`, `...(env || process.env)`, and the old CLAUDECODE-only
delete).

Verified the gap is actually closed: reverting exactly one of tower-routes.ts's
four sites — leaving the other three sanitized — fails 2 assertions, where the
previous per-file test stayed green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ApsmgthG7VWgMM2JaizWD6
Both from the claude lane, both real:

- readProcessEnv returned `{}` when `ps` showed a process but no environment
  (another user's process, or a platform not honouring `e`). Downstream that
  reads as "inspected, and clean" — a clean bill of health for a process we
  never saw inside. It returns null now, so the check reports "skipped".
  Verified against pid 1, which is root-owned and shows no env: `null`.
- "Tower is clean, but N running session(s)…" was printed even with no Tower
  running at all. Shellpers outlive Tower, so that state is reachable, and
  calling an absent Tower clean is simply false. Now reads "No Tower running,
  but …".

Not fixed, and worth stating rather than hiding: the session scan costs two `ps`
spawns per shellper (~142 on this 71-shellper machine). Batching them into one
`ps -p a,b,c` would fix it, but the PR is already drawing scope fire and this is
a rarely-run diagnostic. Flagged for the architect instead.

Live re-verify after both fixes: 71 shellpers, 5 still contaminated, Tower clean
— still correctly a warning. Full suite 5739 passing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ApsmgthG7VWgMM2JaizWD6
waleedkadous and others added 3 commits September 5, 2026 12:00
… the binary)

The architect's issue prescription listed CLAUDE_PID; my denylist omitted it
because it sits outside the CLAUDE_CODE_ namespace. Checked the shipped binary
the same way as the 594 vars, and the omission was wrong.

The child-env builder is one object literal:

  function qOe(e){let t={CLAUDECODE:"1", CLAUDE_CODE_SESSION_ID:e.sessionId,
                         CLAUDE_CODE_CHILD_SESSION:"1", CLAUDE_PID:String(process.pid)};

So CLAUDE_PID is planted alongside the exact marker this issue is about, and is
`String(process.pid)` of one specific process — identity by construction, not by
namespace. Nothing reads it for nesting detection; its only reader is the pkill
guard in the Bash tool's shell prelude, which refuses a pattern matching the
Claude CLI's own pid. An inherited stale value points that guard at a dead or
unrelated pid.

CLAUDE_EFFORT is planted by that same function and stays. That is the useful
half of the check: "Claude Code sets it" is not the test — EFFORT carries a
setting, not an identity.

Regression test 31 cases; full suite 5740 passing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ApsmgthG7VWgMM2JaizWD6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant