Fix #1219: sanitize inherited CLAUDE_CODE_* env at every Tower spawn - #1626
Open
waleedkadous wants to merge 15 commits into
Open
Fix #1219: sanitize inherited CLAUDE_CODE_* env at every Tower spawn#1626waleedkadous wants to merge 15 commits into
waleedkadous wants to merge 15 commits into
Conversation
Claude Code plants session markers (CLAUDE_CODE_CHILD_SESSION,
CLAUDE_CODE_SESSION_ID, ...) into every subprocess it creates, so a nested
claude can tell it was launched from inside another session and turn transcript
saving off. Tower is a daemon but inherits the env of whoever started it, and
starting Tower from inside a Claude Code session is routine (afx tower start,
pnpm -w run local-install, an architect recovering from a crash). The markers
then cascade — Tower -> shellper -> agent claude — and every agent Tower spawns
believes it is a nested child session. A session with transcript saving off
cannot be resumed, so a Tower that was ever restarted from inside a Claude
session quietly produces agents that are unrecoverable; the failure only
surfaces at crash-recovery time.
Confirmed on a live install: `ps eww` on two running Tower daemons showed the
full marker set, inherited from the shell that started them.
Root cause was three shapes, none of which handled CLAUDE_CODE_*:
- towerStart() daemonized with `env: process.env`
- six Tower spawn sites built `{ ...process.env }` and deleted only CLAUDECODE
- createPtySession() sends no env, so builder terminals took the
`env || process.env` branch and got Tower's whole environment
Fix:
- New src/lib/agent-env.ts: one sanitizer, deny-by-default over the
CLAUDE_CODE_* namespace so a marker added upstream tomorrow is stripped
without a code change, with an allowlist for the config/auth vars that must
reach an agent. CLAUDE_CODE_OAUTH_TOKEN above all — consult reads it from the
agent's own env to keep CMAP on the subscription rather than the metered API
(#985), so a blanket strip would have been a silent regression.
- Route all six spawn sites, the daemon spawn, and the shellper spawn through
it. The per-site `delete cleanEnv['CLAUDECODE']` copies collapse into it.
- `afx tower start` logs which markers it scrubbed when it finds any.
- New `codev doctor` section reads the running Tower's env via `ps eww` and
warns if it is carrying markers — a daemon started before this fix stays
contaminated until restarted, and that was previously invisible.
Regression test pins the sanitizer's behaviour (including the OAuth-token
carve-out and the deny-by-default rule), the doctor check, and the fact that
every spawn site routes through the sanitizer rather than hand-rolling a
CLAUDECODE-only strip.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ApsmgthG7VWgMM2JaizWD6
`codev doctor` needs the running Tower's PID, and importing it from `agent-farm/commands/tower.ts` dragged that command's import graph into doctor.ts — including `utils/shell.ts`, which calls `promisify(exec)` at module scope. `doctor.test.ts` mocks `node:child_process` without an `exec` export, so all 24 of its cases failed at import time. Move the function to `agent-farm/utils/port.ts` (execSync only, no module-scope side effects) and re-export it from `commands/tower.ts` so existing consumers, including tower-stop.test.ts, are untouched. doctor.ts takes DEFAULT_TOWER_PORT straight from `@cluesmith/codev-sdk/constants` for the same reason. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ApsmgthG7VWgMM2JaizWD6
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ApsmgthG7VWgMM2JaizWD6
`runCommand` in tower-cron.ts spawned every cron task with raw `process.env` — the one Tower-descendant spawn the first pass missed. A cron task runs an arbitrary command, which may itself be or launch an agent, so it gets the same rule as every other spawn site. Added to the regression test's SPAWN_SITES so the source-shape guard covers it. Second-order once Tower's own env is clean, but the point of routing every site through one helper is that no site has to be reasoned about individually. Found by the claude CMAP lane. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ApsmgthG7VWgMM2JaizWD6
…_* namespace The codex CMAP lane returned REQUEST_CHANGES on the deny-by-default policy, and it was right. Checking the shipped claude binary (2.1.261) settled it: it reads 594 distinct CLAUDE_CODE_* variables, overwhelmingly configuration. Every name codex cited exists — CLAUDE_CODE_USE_FOUNDRY, USE_MANTLE, USE_ANTHROPIC_AWS, SKIP_FOUNDRY_AUTH, OAUTH_REFRESH_TOKEN, OAUTH_SCOPES — and the allowlist would also have dropped API_BASE_URL, PROXY_URL, HTTPS_PROXY, CLIENT_CERT, MANAGED_SETTINGS_PATH, OAUTH_CLIENT_ID, SKIP_MANTLE_AUTH, USE_GATEWAY and more. The original reasoning assumed dropping a config var was harmless. It is not: it silently routes an agent at the wrong provider, or strips its credential and reroutes CMAP to the metered API — the #985 scar. No hand-maintained allowlist survives 594 variables. Inverted: strip what is per-session identity, keep everything else. Identity is a small, stable, nameable set — four prefix families (SESSION_, REMOTE_SESSION_, BRIDGE_, MESSAGING_) plus ten individual names including the CHILD_SESSION / ENTRYPOINT / EXECPATH nesting triple this issue is actually about. Configuration is not nameable, so it is no longer enumerated. Tests updated to match: a MUST_SURVIVE fixture of 24 config/auth variables (each verified present in the binary) that sanitization must preserve, family-level stripping, and an unrecognised CLAUDE_CODE_* now asserted to be kept rather than dropped. Re-verified against live processes: the narrower classifier still flags all 7 markers on a real contaminated Tower, still passes the clean one. Build green, 5735 tests passing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ApsmgthG7VWgMM2JaizWD6
Second REQUEST_CHANGES from the codex CMAP lane, and right again. `towerStop` deliberately leaves shellpers running (they are detached so terminals survive a restart), so after `afx tower stop && afx tower start` the daemon reads clean while every agent the contaminated Tower spawned is still running with the markers — still unresumable. The check inspected only the daemon, so it reported "clean" in exactly that state, and the remediation text said "restart Tower" as if that were the whole fix. - checkTowerEnv() takes a second reader for the running shellper envs and warns when Tower is clean but sessions it spawned earlier are not. - Two remediation strings: one for a contaminated daemon (restart Tower AND each affected agent, saying plainly that a plain restart leaves sessions running), one for the clean-Tower-dirty-sessions case, naming --force-kill-all-child-processes with its blast radius rather than recommending it. - findShellperPids() locates them via `pgrep -f shellper-main.js`. Verified on this machine, which was already in the reported state: 71 shellpers running, 5 still carrying markers, Tower itself clean. The old check called that "ok"; the new one warns. Also corrects an overclaim in the agent-env header caught by the claude lane: it said doctor backstops an unknown upstream marker. It cannot — doctor classifies with the same list, so it is blind to exactly what it would need to catch. The header now says so, and names Claude Code's own banner as the only real signal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ApsmgthG7VWgMM2JaizWD6
Both the codex and claude CMAP lanes made the same point: asserting that a file
mentions sanitizeAgentEnv somewhere would let ONE of tower-routes.ts's four
spawn sites revert while the other three kept the test green.
SPAWN_SITES now carries an expected call count per file, and a FORBIDDEN list
pins the shapes a reverted or copy-pasted site would take (`env: process.env`,
`{ ...process.env`, `...(env || process.env)`, and the old CLAUDECODE-only
delete).
Verified the gap is actually closed: reverting exactly one of tower-routes.ts's
four sites — leaving the other three sanitized — fails 2 assertions, where the
previous per-file test stayed green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ApsmgthG7VWgMM2JaizWD6
Both from the claude lane, both real:
- readProcessEnv returned `{}` when `ps` showed a process but no environment
(another user's process, or a platform not honouring `e`). Downstream that
reads as "inspected, and clean" — a clean bill of health for a process we
never saw inside. It returns null now, so the check reports "skipped".
Verified against pid 1, which is root-owned and shows no env: `null`.
- "Tower is clean, but N running session(s)…" was printed even with no Tower
running at all. Shellpers outlive Tower, so that state is reachable, and
calling an absent Tower clean is simply false. Now reads "No Tower running,
but …".
Not fixed, and worth stating rather than hiding: the session scan costs two `ps`
spawns per shellper (~142 on this 71-shellper machine). Batching them into one
`ps -p a,b,c` would fix it, but the PR is already drawing scope fire and this is
a rarely-run diagnostic. Flagged for the architect instead.
Live re-verify after both fixes: 71 shellpers, 5 still contaminated, Tower clean
— still correctly a warning. Full suite 5739 passing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ApsmgthG7VWgMM2JaizWD6
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ApsmgthG7VWgMM2JaizWD6
… the binary)
The architect's issue prescription listed CLAUDE_PID; my denylist omitted it
because it sits outside the CLAUDE_CODE_ namespace. Checked the shipped binary
the same way as the 594 vars, and the omission was wrong.
The child-env builder is one object literal:
function qOe(e){let t={CLAUDECODE:"1", CLAUDE_CODE_SESSION_ID:e.sessionId,
CLAUDE_CODE_CHILD_SESSION:"1", CLAUDE_PID:String(process.pid)};
So CLAUDE_PID is planted alongside the exact marker this issue is about, and is
`String(process.pid)` of one specific process — identity by construction, not by
namespace. Nothing reads it for nesting detection; its only reader is the pkill
guard in the Bash tool's shell prelude, which refuses a pattern matching the
Claude CLI's own pid. An inherited stale value points that guard at a dead or
unrelated pid.
CLAUDE_EFFORT is planted by that same function and stays. That is the useful
half of the check: "Claude Code sets it" is not the test — EFFORT carries a
setting, not an identity.
Regression test 31 cases; full suite 5740 passing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ApsmgthG7VWgMM2JaizWD6
…gate round Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ApsmgthG7VWgMM2JaizWD6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1219
Summary
A Tower started from inside a Claude Code session inherits that session's identity markers and hands them to every agent it spawns. Each agent then believes it is a nested child session, disables transcript saving, and becomes unresumable — silently, until crash recovery fails. This routes every Tower-descendant spawn through one sanitizer that strips per-session identity, and adds a
codev doctorcheck for daemons that are already contaminated.Root Cause
Claude Code plants session markers —
CLAUDE_CODE_CHILD_SESSION,CLAUDE_CODE_SESSION_ID,CLAUDE_CODE_ENTRYPOINT,CLAUDE_CODE_EXECPATH, and friends — into every subprocess it creates, so a nestedclaudecan tell it was launched from inside another session. Tower is a daemon, but it inherits the environment of whoever started it, and starting Tower from inside a Claude Code session is routine (afx tower start,pnpm -w run local-install, an architect recovering from a crash). The markers cascade:Confirmed on a live install rather than assumed:
ps ewwon two running Tower daemons showed the full marker set, inherited from the shell that started them.Three code shapes, none of which handled
CLAUDE_CODE_*:towerStart()daemonized withenv: process.env(agent-farm/commands/tower.ts).{ ...process.env }and deleted onlyCLAUDECODE—tower-instances.ts×2,tower-terminals.ts×2,tower-routes.ts×2.createPtySession()sends noenv, so builder terminals took theenv || process.envbranch inhandleTerminalCreateand received Tower's whole environment.shellper-mainpasses its configenvstraight to node-pty, which replaces the child environment wholesale — so the env Tower hands toSessionManager.createSessionis the agent's env. That makes (2)+(3) load-bearing and (1) the upstream source.Fix
New
packages/codev/src/lib/agent-env.tswith a singlesanitizeAgentEnv(): strip what is per-session identity, keep everything else.Identity is a small, stable, nameable set — four prefix families (
CLAUDE_CODE_SESSION_,CLAUDE_CODE_REMOTE_SESSION_,CLAUDE_CODE_BRIDGE_,CLAUDE_CODE_MESSAGING_) plus ten individual names, including theCHILD_SESSION/ENTRYPOINT/EXECPATHnesting triple this issue is about. Configuration is not a nameable set, so it is not enumerated.Every Tower-descendant spawn routes through it — the daemon spawn, both architect-launch sites, both architect-reconnect sites, terminal-create and shell-create (plus their non-persistent fallbacks), the shellper daemon spawn, and cron task execution. The six hand-rolled
delete cleanEnv['CLAUDECODE']copies collapse into the one helper.afx tower startlogs what it scrubbed, andcodev doctorgrows a Tower Environment section that reads the running daemon's env viaps eww— a daemon started before this fix stays contaminated until restarted, and that was previously invisible.A supporting commit moves
getProcessesOnPortto a leafagent-farm/utils/port.ts(re-exported, no consumer moves): importing it from the command module draggedutils/shell.ts's module-scopepromisify(exec)into the doctor tests, whosenode:child_processmock has noexecexport.Why not deny the whole
CLAUDE_CODE_*namespaceThe first version did exactly that, with a short allowlist, on the theory that a missed marker is worse than a missed config var. The codex CMAP lane rejected it and was right. Checking the shipped
claudebinary (2.1.261) settled it: it reads 594 distinctCLAUDE_CODE_*variables, overwhelmingly configuration. The allowlist would have silently droppedUSE_FOUNDRY,USE_MANTLE,USE_ANTHROPIC_AWS,USE_GATEWAY,SKIP_FOUNDRY_AUTH,SKIP_MANTLE_AUTH,OAUTH_REFRESH_TOKEN,OAUTH_SCOPES,OAUTH_CLIENT_ID,API_BASE_URL,PROXY_URL,HTTPS_PROXY,CLIENT_CERT,MANAGED_SETTINGS_PATHand more.The original reasoning was wrong about the failure mode: dropping a config var is not harmless. It points an agent at the wrong provider, or strips its subscription credential and reroutes CMAP to the metered API — the #985 scar. No hand-maintained allowlist survives 594 variables.
CLAUDE_PIDdispositionChecked against the shipped binary the same way as the 594:
CLAUDE_PIDis stripped.It is not read for nested-session detection — its only reader is the
pkillguard in the Bash tool's shell prelude, which refuses a pattern matching the Claude CLI's own pid. But it is per-session identity by construction, planted in the same object literal as the marker this issue is about:String(process.pid)of one specific process is identity by any reading, and an inherited stale value points thatpkillguard at a dead or unrelated pid. So it is in the denylist despite sitting outside theCLAUDE_CODE_namespace.CLAUDE_EFFORTis planted by that same function and is deliberately not stripped — which is the useful part of the check: "Claude Code sets it" is not the test. It carries a setting, not an identity, so it is configuration and passes through.Test Plan
bugfix-1219-claude-env-leak.test.ts, 31 cases. 9 assertions fail with the spawn-site changes reverted to the pre-fix commit and pass with them; reverting just one oftower-routes.ts's four sites fails 2.Covered by the regression test:
MUST_SURVIVEfixture of 24 provider / auth / routing / policy variables — each verified present in the shipped binary, including the Foundry, Mantle, Anthropic-AWS and refresh-token cases — is preserved intact.CLAUDE_CODE_*variable is kept, not dropped.codev doctorwarns when Tower is clean but shellpers it spawned earlier still carry markers, and never claims a plain Tower restart is sufficient on its own.Verified beyond the suite:
warnnaming all 7 markers; a dead pid →skipped. On this machine the session check found the exact case it exists for — 71 shellpers running, 5 still carrying markers, Tower itself clean, which the daemon-only version calledok.Scrubbed inherited Claude Code session markers: …and came up with a clean environment. That run is the incident described below — it is reported there rather than claimed here as a clean result. Field verification will be redone after merge vialocal-installonmain.Incident during verification
While verifying this fix I took down every terminal on the development machine. Recording it here because it is the kind of failure this PR's own subject matter — processes quietly attaching to state they should not touch — is about.
I started a test Tower on port 14733 to confirm the daemon scrub end to end, and exported
AGENT_FARM_DIRto isolate it. That is not the override. The real one isCODEV_AGENT_FARM_DIR(packages/core/src/constants.ts:18, added by #1515). So my test Tower opened the production~/.agent-farm/global.dband appended to the productiontower.log.Its startup reconcile then found the 56 live terminal rows and connected to every production shellper socket. A shellper holds one client, so the real Tower on 4100 lost all of them; the two Towers fought over reconnects, and the reconcile's stale-row path deleted 53 of the 56 rows. When I killed the test Tower those sessions were orphaned: 60 unregistered shellpers, and every architect terminal across 12 workspaces gone from the dashboard — including the architect's own. The architect restored the rows by hand from the checkpointed DB file and restarted Tower.
The log corroborates it: 56
reconcile-adoptlines at18:26:59.4xx, then 54Session … removed but shellper pid=… is alive; preserving socketlines by18:27:02.5xx.My gate message described this as having "briefly contended with the live Tower for the cloud tunnel. It recovered." That was wrong — I reported the tunnel flapping I had noticed and not the session loss I had not looked for, which made a fleet-wide outage read as a transient blip.
#1629 is filed for the systemic defects, and they are the real lesson: a second Tower must not be able to open a live
global.dband adopt rows owned by a running Tower (owner lock, or refuse to adopt), and an isolation env var whose misspelling silently means "use production" is a trap rather than a safeguard. My mistake was the trigger; neither of those should be reachable by one wrong variable name.No Tower has been started from this lane since. Field verification of the daemon scrub is deferred to
local-installonmainafter merge.Known follow-ups (filed, not in this PR)
afx architectspawnsclaudeunsanitized. Pre-existing and outside Tower (it never strippedCLAUDECODEeither), so an architect started from inside a Claude session still nests.codev doctorinspects onlyDEFAULT_TOWER_PORT, and the session scan costs ~2psspawns per shellper (~142 on a 71-shellper machine); both fixable, neither worth widening this PR for.One limit worth stating rather than filing:
codev doctorclassifies with the same list the sanitizer uses, so it cannot backstop a marker Claude Code adds that this list has not caught up with. Claude Code's own "transcript saving is off" banner remains the only signal for that case — it is how this issue was found.🤖 Generated with Claude Code
https://claude.ai/code/session_01ApsmgthG7VWgMM2JaizWD6