Skip to content

D4: Add bounded production agent adapters - #47

Open
mchwang wants to merge 19 commits into
codex/agent-isolation-d3from
codex/agent-adapters-d4
Open

mchwang wants to merge 19 commits into
codex/agent-isolation-d3from
codex/agent-adapters-d4

Conversation

@mchwang

@mchwang mchwang commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Problem and behavior

Swim Lane D needs production Claude and Codex adapters that cannot outlive their ownership or return unbounded output. This change adds container invocation supervision with immutable 16 MiB stdout, 4 MiB stderr, and 20 MiB combined ceilings; finite deadlines; first-reason cancellation; whole-container TERM/KILL cleanup; and active-attempt ownership until the container and cleanup settle.

Claude output is parsed from its JSON envelope. Codex writes its final response into runner-created bounded tmpfs and uses a private deferred-output handshake so the supervisor can perform a no-follow, nonblocking, regular-file, bounded UTF-8 read before teardown. Invalid, changing, linked, malformed, traversing, or oversized output fails closed.

This PR is stacked on #44 and should be reviewed/landed after it. It implements D4 only and does not connect the adapters to the runner/store lifecycle owned by Lane F.

Validation

Validated head: 22f362e8c281aecea4a33b64a613f77aab112c61

  • npm run typecheck
  • npx vitest run --no-file-parallelism — 18 files, 292 tests passed
  • CODEBOOST_RUN_AUTH_PROBES=1 npx vitest run test/agent-supervisor.test.ts with Keychain Claude OAuth and local Codex auth — 31 tests passed, including both production provider probes
  • Focused supervisor and adapter suites — 29 and 8 tests passed

Review record

  • Self-review round 1: fixed deferred Codex capture failures so they trigger immediate whole-container termination and withhold protocol acknowledgement; tightened duplicate protocol-token handling.
  • Automated round 1: accepted and fixed all seven findings: safe single-component output paths, nonblocking FIFO rejection, post-read link/type revalidation, decoded-text byte limits, fatal UTF-8 decoding, asynchronous deadline-budgeted capture, and awaited Docker control-process settlement. Regressions cover traversal, FIFO, malformed UTF-8, and decoded-size underreporting.
  • Automated round 2: accepted and fixed both inline findings by moving output to a dedicated validated tmpfs mount that the agent cannot replace. Also fixed both summary findings with fatal Claude UTF-8 decoding and immediate capture-failure termination on acknowledgement failure. Regressions cover mount-point replacement and acknowledgement denial.
  • Automated round 2 late findings: fixed the reader production ceiling, post-close cancellation, nonzero envelope validation, failed-start cleanup ownership, and anchored pathname identity. Exact-head regressions cover the first three; hostile-file plus live Codex probes exercise the pinned reader.
  • Additional delayed findings: fixed post-close cleanup recovery, stale recovery timers, and unbounded async decoders; also replied to the already-fixed Claude UTF-8, late acknowledgement, and BigInt link-count threads.
  • Automated round 3: fixed lstat pathname validation, all pre-start cleanup ownership, missing deferred output, and abort-and-await decoder settlement. Also capped configured timeouts at ten minutes to avoid Node timer clamping.
  • Automated round 4: fixed durable network-only cleanup recovery for both adapters and bounded detachment for an injected decoder that ignores abort.
  • Automated round 5: fixed bounded post-stop stderr, root-owned exclusive acknowledgements, duplicate cleanup recovery, exact-handle recovery deletion, and protocol-byte accounting/duplicate rejection.
  • Automated round 6: reported no inline findings but surfaced two missed summary items; fixed colliding cleanup ownership with a separate recovery registry and delimited READY after newline-free stderr.
  • Automated round 7: fixed pre-network deadline enforcement for both adapters; the reused-attempt network recovery finding was already fixed by the round-6 recovery registry change.
  • Automated round 8: confirmed the reused-attempt recovery fix and surfaced profile-staging cleanup ownership; fixed it with retryable construction errors and combined directory/network recovery ownership.
  • Automated round 9: no inline findings; fixed its summary-only CI serialization concern by running the dedicated Docker workflow with --no-file-parallelism.
  • Automated round 10: no inline findings; fixed its summary-only timestamp and cancellation concerns with monotonic deadline budgets and a bounded abort race for decoders that ignore cancellation.
  • Automated round 11: fixed colliding-recovery admission, carried the original monotonic budget into supervision, rechecked deadlines before decoded output publication, and kept cleanup retry timers referenced through settlement.
  • Automated round 12: no inline findings; fixed its summary-only network cleanup budget concern by sharing the caller deadline during failed construction and handing exhausted cleanup to the retained bounded recovery owner.
  • Automated round 13: fixed same-profile duplicate rejection without disposal and enforced the monotonic deadline in the close handler before accepting decoded output.
  • Automated round 14: no inline findings; fixed its summary-only adapter timeout concern by applying the configured timeout to the original pre-network monotonic budget.
  • Automated round 15: pending.
  • Declined findings: none.
  • Review-lesson audit: deadline enforcement, bounded fail-closed reads, whole-container termination, and settlement ownership are covered by existing repository rules. The path/FIFO/link/UTF-8 details are one-off defenses for this pinned output-file reader and do not add a broader rule.

Follow-up boundary

D5 will provide the full hostile-input and real-Docker boundary gate as a separate PR. Lane F remains responsible for admission, persistence, retries, and runner/store integration.

Copilot AI lite review requested due to automatic review settings September 24, 2026 23:40

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved critical and moderate findings affect fail-closed capture, deadline enforcement, path safety, and cleanup.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 4 High severity · 3 Medium severity

Open (7)
What changed in this PR

Adds supervised production Claude/Codex adapters with bounded output, deadlines, cancellation, cleanup, and deferred Codex capture.

Changes:

  • Adds Claude JSON parsing and Codex bounded-output adapters.
  • Adds container supervision, timeout, cancellation, and cleanup handling.
  • Adds protocol support, integration tests, and CI coverage.
File Summary
test/​agent-supervisor.test.ts Supervisor lifecycle and bounded-capture tests
test/​agent-adapter.test.ts Adapter parsing and limit tests
agents/​policy.ts Codex output routing and probes
agents/​container/​run.ts Container disposal validation
agents/​container/​profile.ts Deferred-output profile configuration
agents/​container/​probe.sh Deferred-output handshake protocol
agents/​adapters/​types.ts Adapter request types
agents/​adapters/​supervisor.ts Invocation supervision, capture, and cleanup
agents/​adapters/​codex.ts Codex deferred-output adapter
agents/​adapters/​claude.ts Claude adapter and envelope parsing
.github/​workflows/​agent-isolation.yml Adapter and supervisor CI coverage

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread agents/adapters/supervisor.ts Outdated
Comment thread agents/adapters/supervisor.ts Outdated
Comment thread agents/adapters/supervisor.ts Outdated
Comment thread agents/adapters/supervisor.ts Outdated
Comment thread agents/adapters/codex.ts Outdated
Comment thread agents/adapters/supervisor.ts Outdated
Comment thread agents/adapters/supervisor.ts Outdated
Copilot AI review requested due to automatic review settings September 24, 2026 23:53

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Critical deferred-output path-safety issues and additional deadline, capture, output-limit, and CI issues remain unresolved.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 2 High severity

Open (2)
Resolved since last review (7)
Previously missed (2)

In code that hasn't changed since last review

Medium severity Reject malformed UTF-8 before parsing Claude output

agents/​adapters/​claude.ts:9

Buffer.toString('utf8') replaces malformed byte sequences with U+FFFD, so an envelope containing invalid UTF-8 can be parsed and published as a successful Claude response instead of failing closed. Decode with a fatal UTF-8 decoder before calling JSON.parse, matching the Codex adapter's behavior.

Medium severity Propagate acknowledgement failures and terminate capture

agents/​adapters/​supervisor.ts:240

The acknowledgement docker exec ... touch is fire-and-forget, and runControl resolves even when the child exits nonzero or emits an error. If acknowledgement creation fails, the wrapper remains in its polling loop until the invocation deadline, so this capture failure is reported as a late timeout and holds the container/network unnecessarily. Make this control operation report failure and immediately preserve the first capture-failure reason while terminating the whole container.

Comment thread agents/adapters/supervisor.ts Outdated
Comment thread agents/container/profile.ts

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread agents/adapters/claude.ts Outdated
Comment thread agents/adapters/supervisor.ts
Comment thread agents/adapters/supervisor.ts Outdated
Comment thread agents/adapters/supervisor.ts Outdated
Comment thread agents/adapters/supervisor.ts Outdated
Copilot AI review requested due to automatic review settings September 25, 2026 00:10

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved supervisor safety issues and Docker-related CI resource contention remain.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 5 High severity · 1 Medium severity

Open (6)
Resolved since last review (2)

Comment thread agents/adapters/supervisor.ts Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread agents/adapters/supervisor.ts Outdated
Comment thread agents/adapters/supervisor.ts
Copilot AI review requested due to automatic review settings September 25, 2026 00:27

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved critical and moderate findings affect deadline enforcement, cleanup ownership, protocol handling, and valid Codex output.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 4 High severity

Open (4)
Resolved since last review (6)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Safely schedule timers for long invocation deadlines

agents/​adapters/​supervisor.ts:313

This passes an unrestricted deadline delta directly to setTimeout. captureInvocation accepts any safe future deadline and timeoutMs has no upper bound, but Node clamps delays above 2^31-1 ms to about 1 ms, so a valid long-lived invocation can be cancelled immediately instead of at its deadline. Cap and re-arm long delays (or explicitly enforce a compatible maximum) before scheduling this timer.

Comment thread agents/adapters/supervisor.ts Outdated
Comment thread agents/adapters/supervisor.ts Outdated
Comment thread agents/adapters/supervisor.ts
Copilot AI review requested due to automatic review settings September 25, 2026 00:39

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Four moderate supervisor issues remain involving cleanup, deadline scheduling, byte accounting, and symlink revalidation.

Review effort: Lite
Findings: 3 High severity

Open (3)
Resolved since last review (1)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Large deadlines cause immediate timeout due to setTimeout clamping

agents/​adapters/​supervisor.ts:315

The invocation deadline is only validated as a safe integer, but Node clamps setTimeout delays above 2^31-1 ms to 1 ms. A valid invocation with a sufficiently distant deadline (or a large timeoutMs) will therefore be stopped immediately instead of at its deadline. Schedule long deadlines in bounded chunks or reject unsupported durations explicitly.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread agents/adapters/supervisor.ts Outdated
Comment thread agents/adapters/supervisor.ts Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved moderate issues remain around monotonic deadline handling and bounded cleanup recovery during startup, network setup, and post-close disposal.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 2 High severity · 1 Medium severity

Open (3)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Unref'd cleanup retry may be skipped when the process exits

agents/​adapters/​supervisor.ts:502

This is the only retry mechanism after post-close disposal fails, but its timer is unref()'d. Once the attached child and control processes have closed, the returned Promise/handle does not keep Node alive, so the process can exit before the retry and leave the container, staged credentials, or network owned but unreleased. Keep a live owner for this recovery (or hand it to a durable runner cleanup registry) until disposal succeeds.

Comment thread agents/adapters/supervisor.ts Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Fix the synchronous decoder timeout race before approval.

Review effort: Lite
Findings: 2 High severity · 1 Medium severity

Open (3)

Copilot AI review requested due to automatic review settings September 25, 2026 03:19

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Network cleanup can exceed the supplied deadline budget, so the cleanup path must be bounded before approval.

Review effort: Lite
Findings: None

Resolved since last review (3)

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

A critical duplicate-profile ownership issue can dispose of an active invocation and requires resolution.

Review effort: Lite
Findings: 1 High severity

Open (1)

Comment thread agents/adapters/supervisor.ts
Copilot AI review requested due to automatic review settings September 25, 2026 03:32

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The supervisor close handler must recheck the monotonic deadline before accepting success.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 2 High severity

Open (2)

Comment thread agents/adapters/supervisor.ts Outdated
Copilot AI review requested due to automatic review settings September 25, 2026 03:42

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Unresolved timeout-budget findings remain in both production adapters, and the container lifecycle changes require final human review.

Review effort: Lite
Findings: None

Resolved since last review (2)

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Two moderate supervisor issues remain around setup cleanup ownership and monotonic budget propagation.

Review effort: Lite
Findings: None

Copilot AI review requested due to automatic review settings September 25, 2026 03:55

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Address deadline-bounded cleanup in both adapters and flush buffered stderr during close.

Review effort: Lite
Findings: None

Copilot AI review requested due to automatic review settings September 25, 2026 04:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants