Skip to content

feat(subagent): enforce one recovery per failure via spawn_agent - #1392

Merged
TheGreatAxios merged 2 commits into
mainfrom
cl-10202-enforced-recovery
Oct 13, 2026
Merged

TheGreatAxios merged 2 commits into
mainfrom
cl-10202-enforced-recovery

Conversation

@bwachman

@bwachman bwachman commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

PR 2 of 2: enforced recovery (stacked on PR 1)

This is PR 2 of the two-PR CL-10202 delivery. It is stacked on PR 1 (classification and record, branch cl-10202-surface-terminal-subagent-provider-failures-with-actionable) and must not merge first. Reverting this PR restores today's advisory guidance and keeps PR 1.

Summary

  • Enforces recovery policy at the spawn boundary: spawn_agent takes a recovers argument and grants at most one recovery per failure.
  • Tracks replay safety so a run that asked a question or spawned sub-agents is not auto-recovery-eligible.
  • Claims recovery idempotently and caps the recovery chain at one.
  • Derives continuable from recovery.available, now including replay safety.
  • Replacement runs with D7 authority match a fresh spawn with no grant/ask carry-over.
  • The grant is refunded on every path where the replacement never reaches run() (worktree failure, cancel/interrupt before launch, cancel while queued); once run() begins it stays spent.

Verification

After the rebase onto main and the review fixes, CI passes on 4789cdd. Gates below were last run on the pre-rebase tip ffe0494 (exit code 0 everywhere):

Gate Result
bun run typecheck 0
bun run lint 0
bun run check:dead-exports 0 (0 violations)
bun test ./src/subagent --randomize --seed 424242 0 (869 pass / 0 fail / 54 files)
bun run build 0 (1185 modules bundled)
repo-wide bun test 0 (8650 pass / 0 fail / 559 files)

AC to code and test mapping

  • AC3 zero replacements without recovers, exactly one after, no second: src/subagent/fleet-recovery.test.ts:114-148 (zero without recovers), :165-199 (exactly one), :201-259 (no second: concurrent 201-227, exhausted 229-259)
  • AC6 ineligible reasons return typed reason, spawn nothing: src/subagent/terminal-failure.ts:139-155 + src/subagent/fleet-recovery.test.ts
  • AC7 replacement authority equals fresh spawn, no grant/ask carry-over: src/subagent/agent-fleet.ts:1594-1752 + src/subagent/fleet-recovery.test.ts:568-630
  • AC8 handoff never delivered without evidence: src/subagent/session-store.ts:1669-1675 + src/subagent/session-store-failure-record.test.ts:318-353
  • AC9 deterministic mocked tests, no real credentials: all listed test files

Deferred / Notes

  • Known consequence (spec-conformant to D2): ask_director, manage_tasks, and spawn_agent are not replay-safe, so a run that asked a question or spawned sub-agents is classified side_effects_completed and is not auto-recovery-eligible. This is the brief's closed allowlist, decided, not a defect.
  • Rollback is a revert of this branch.
  • Non-goals: user-facing recovery command, fleet-record persistence, CL-10200 WORKER WAITING strip, CL-10206 stall policy, watchdog behavior.

Refs CL-10202

@linear-code

linear-code Bot commented Oct 10, 2026

Copy link
Copy Markdown

CL-10202

@bwachman
bwachman force-pushed the cl-10202-surface-terminal-subagent-provider-failures-with-actionable branch 2 times, most recently from d4ad7ef to 3347ae6 Compare October 11, 2026 20:50
Comment thread src/subagent/agent-fleet.ts Resolved

@TheGreatAxios TheGreatAxios left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One-recovery enforcement verified — the recovers arg, replay-safety gate, idempotent claim, chain cap, and D7 fresh-spawn authority all confirmed (869/0 subagent suite at seed 424242). BLOCKER: the PR is CONFLICTING/DIRTY against its base — the stack diverged and requires a rebase (conflicts in ARCHITECTURE.md, tool-classification.ts, agent-fleet.ts, session-store.ts + four add/add in terminal-failure files). Should-fix: the recovery grant is consumed before the replacement launches — a failed launch leaves the lane already_recovered with a dead replacement id. Plus: stale AC3/AC8 citations in the body.

  • PR is CONFLICTING/DIRTY against its declared base (cl-10202-surface-…): merge-base c5113a2; the base tip carries its own typed-failure/doom-loop commits (b70f91c, 0f19586) distinct from head's (cd94e2c, 9a71c76). git merge-tree conflicts: docs/ARCHITECTURE.md, src/agent/tool-classification.ts, src/subagent/agent-fleet.ts, src/subagent/session-store.ts + add/add in terminal-failure.ts/.test.ts, fleet-terminal-failure.test.ts, session-store-failure-record.test.ts (prompts.ts and fleet-dry-drive.ts merge cleanly). Rebase onto the current base tip.
    Plain: this can't merge — the branch it's stacked on was rebased after it forked, so the same changes now exist in two different versions. Rebase and resolve.

  • AC3 "no second" is covered at fleet-recovery.test.ts:182-240 (concurrent 182-208, exhausted 210-240), outside the cited 108-180; AC8 gate is session-store.ts:1694-1700, cited 1690-1702 over-includes the unrelated pushEntry. Fix the citations.
    Plain: the description's test references point at the wrong line ranges.

Comment thread src/subagent/session-store.ts Resolved
@bwachman
bwachman force-pushed the cl-10202-enforced-recovery branch from ffe0494 to d59be8e Compare October 13, 2026 00:50
@bwachman
bwachman force-pushed the cl-10202-surface-terminal-subagent-provider-failures-with-actionable branch 2 times, most recently from 7d1bdcf to 3dbdbc3 Compare October 13, 2026 00:52
@bwachman
bwachman force-pushed the cl-10202-enforced-recovery branch from d59be8e to 4789cdd Compare October 13, 2026 00:52
@bwachman

Copy link
Copy Markdown
Contributor Author

Rebased and updated; head is now 4789cdd (2 commits: replay safety, enforce one recovery).

@TheGreatAxios
TheGreatAxios force-pushed the cl-10202-surface-terminal-subagent-provider-failures-with-actionable branch from 3dbdbc3 to 694fbd2 Compare October 13, 2026 02:54
A worker that wrote files or ran a shell command before its provider
gave out was still marked continuable, so a successor with the same
brief could repeat those side effects. Recovery is now available only
when every tool the run requested or started is a read, search, or web
read; anything else, including MCP and unknown tools, fails closed to
side_effects_completed and normal director-led continuation.
The single-successor hint for a transient worker failure was advisory:
nothing tied a successor to the failed run, counted it, or stopped a
second one. spawn_agent now takes recovers, the failed agent id. The
claim is recorded synchronously before the handler awaits, so duplicate
or concurrent calls get the same replacement back and spawn nothing; an
ineligible request is refused with its typed reason and never falls
back to a plain spawn. A replacement is attempt 2, so its own failure
is recovery_exhausted, and it is an ordinary spawn that inherits no
grants, approvals, or pending asks. Reverting this restores the
advisory guidance.
@TheGreatAxios
TheGreatAxios force-pushed the cl-10202-enforced-recovery branch from 4789cdd to 782894e Compare October 13, 2026 05:10
@TheGreatAxios
TheGreatAxios changed the base branch from cl-10202-surface-terminal-subagent-provider-failures-with-actionable to main October 13, 2026 05:10
@TheGreatAxios
TheGreatAxios merged commit ac8892b into main Oct 13, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants