Skip to content

D: pass the plan schema to Claude; refuse Codex in planning and questions (#75) - #97

Merged
mchwang merged 3 commits into
mainfrom
feat/d-planning-schema-75
Oct 1, 2026
Merged

mchwang merged 3 commits into
mainfrom
feat/d-planning-schema-75

Conversation

@mchwang

@mchwang mchwang commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Who this is for: reviewers of lane D and of E4/G4, which call D for planning. What it does: closes #75. The answer schema now reaches the vendor, and the open Codex question is decided and built.

Summary

  • Claude gets the schema. In planning, createClaudeCommand passes the exact text of the mounted schema.json as --json-schema. Claude 2.1.281 takes the schema text, not a path.
  • The schema in the command must match the mounted file. The container profile refuses a command whose schema text differs from the file it mounts, byte for byte.
  • The answer comes back as bare JSON. With a schema, Claude puts its answer in structured_output. parseClaudeOutput returns that as JSON text, so E2 needs no parser change. A result whose subtype is not success counts as a provider failure.
  • Codex is refused in planning and questions. This is decision (c) from the issue. Codex 0.153.4 reads files only through its shell, and these phases run no process. The start call refuses before allocating anything.
  • Ask is Claude-only. The Store refuses Codex as the question agent. A review database that already names Codex gets a clear message. The Ask worker now receives only CLAUDE_CODE_OAUTH_TOKEN.
  • The decision is recorded in docs/implementation/agent-isolation.md, "Codex in read-only phases".

Checks before launch

The planning schema is read through the profile's bounded, no-follow reader before any Docker work. The reader now also opens with O_NONBLOCK, so a FIFO fails instead of blocking. The schema must be a regular file with one link, at most 64 KiB of UTF-8, and a JSON object with "type": "object".

Testing

  • Typecheck passes, and all 1,200 unit tests pass.
  • The Settings browser tests pass.
  • The Docker tests for schema binding and adapter setup pass.
  • Host check of the flag combination. I ran the exact production Claude flags (--json-schema with --permission-mode plan, --restricted and --tools Read,Glob,Grep) with Claude Code 2.1.282 on the host. It returned subtype: success and the expected structured_output.
  • Live Claude probes pass in the pinned image. Run with CODEBOOST_RUN_AUTH_PROBES=1 npx vitest run test/agent-supervisor.test.ts -t "Claude" on 2026-10-01. The questions probe returns plain text, and the planning probe returns a schema-constrained answer as bare JSON.

Known gaps

  • The Codex live probes fail, as expected. They now run in review. Codex answered "I can’t read that local file with the available tools", so Codex cannot read code in any phase. The probes failed the same way before this change. A follow-up will decide review, execute and fix for Codex. The gate docs mark this as an expected failure.
  • E4 now has two recordings to make, not four. scripts/record-planning.ts on the E4 branch still starts Codex for planning and must drop it.
  • Docker flakiness on the author's machine. The local Docker daemon held about 1,000 leftover codeboost containers. That caused network-pool and leftover-count failures in some unrelated Docker tests. Those tests pass when run alone. Two supervisor tests fail the same way on unmodified main.

🤖 Generated with Claude Code

…ions (#75)

Claude's planning command now carries the mounted schema as --json-schema.
The container profile refuses a command whose schema text differs from the
mounted file, and the adapter returns Claude's structured_output as bare JSON.

Codex 0.153.4 reads files only through its shell, which read-only phases keep
off. Decision (c): Codex is refused in planning and questions, so Ask is
Claude-only. The decision is recorded in docs/implementation/agent-isolation.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Structured envelopes can bypass the required subtype check, and the live schema probe remains nondeterministic.

Review effort: Balanced
Findings: 2 Medium severity · 1 Low severity

Open (3)
What changed in this PR

Adds schema-constrained Claude planning and disables Codex where shell-free file access is unavailable.

Changes:

  • Passes the mounted planning schema to Claude and extracts structured output.
  • Refuses Codex for planning and questions.
  • Updates Ask, tests, UI, and documentation for Claude-only questions.
File Description
agents/​adapters/​claude.ts Reads schemas and parses structured output.
agents/​adapters/​codex.ts Refuses unsupported phases before allocation.
agents/​container/​profile.ts Binds commands to mounted schema bytes.
agents/​policy.ts Adds schema validation and phase restrictions.
runner/​question-container.ts Makes Ask Claude-only.
runner/​question-worker.ts Removes Codex startup support.
runner/​store.ts Rejects new Codex question settings.
web/​public/​app.js Updates question-agent settings UI.
web/​cli.ts Updates CLI guidance.
test/​agent-adapter-cleanup.test.ts Moves Codex cleanup fixtures to review.
test/​agent-adapter.test.ts Tests schema reading, parsing, and refusals.
test/​agent-container.test.ts Tests mounted-schema binding.
test/​agent-launch.test.ts Updates launch fixtures for allowed phases.
test/​agent-policy.test.ts Tests schema arguments and phase policy.
test/​agent-supervisor.test.ts Adds live structured-output coverage.
test/​browser/​review.spec.ts Tests Claude-only settings.
test/​question-agent.test.ts Tests Codex refusal and credential isolation.
test/​questions.test.ts Tests legacy Codex database handling.
README.md Documents Claude-only Ask support.
prompts/​plan-author.md Documents the Claude planning path.
docs/​plan-format.md Updates plan-generation documentation.
docs/​implementation/​agent-isolation.md Records the Codex decision and schema controls.
docs/​implementation/​planning-audit.md Updates E4 recording expectations.
docs/​implementation/​read-only-review.md Updates Ask provider documentation.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread agents/adapters/claude.ts
Comment thread test/agent-supervisor.test.ts Outdated
Comment thread web/public/app.js Outdated
…s, const probe marker, error status for failed save

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

It changes security-sensitive container and provider boundaries while the credentialed in-container Claude probe remains unrun.

Review effort: Balanced
Findings: None

Resolved since last review (3)

…a-75

# Conflicts:
#	docs/implementation/agent-isolation.md
#	test/question-agent.test.ts
#	test/questions.test.ts
@mchwang
mchwang merged commit 31e8e27 into main Oct 1, 2026
2 checks passed
@mchwang
mchwang deleted the feat/d-planning-schema-75 branch October 20, 2026 19:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

D: pass the answer schema to both vendors; decide how Codex reads code in read-only phases

2 participants