Skip to content

Fix setup diagnostics for Codex, Qwen and DeepSeek - #73

Merged
bigboateng merged 1 commit into
mainfrom
codex/readiness-diagnostics
Oct 13, 2026
Merged

bigboateng merged 1 commit into
mainfrom
codex/readiness-diagnostics

Conversation

@bigboateng

@bigboateng bigboateng commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Change contract

When: A developer checks Codex, Qwen or DeepSeek setup with pitot doctor --host HOST.
Before: Qwen could return success without inspecting its setup, Codex could pass on a wiring witness despite missing dependencies, and the first DeepSeek failure hid other setup problems.
After: Doctor reports component diagnostics and fails for missing, disabled, malformed or unsupported setup. It distinguishes static setup, selected-runtime health and unverified live enforcement. Explicit --fix runs diagnostics after its wiring step.

Boundaries preserved

  • Plain doctor does not run configured hooks, edit settings or grant trust.
  • Qwen inspection preserves supported configuration precedence, trust exclusions, synchronous matching and command dependencies. Ambiguous commands and conflicting registrations require native inspection.
  • Codex checks the platform shim, pin, known configuration flags and per-hook state. Native trust comparison and unsupported configuration layers remain explicit limitations.
  • Runtime health contacts only a literal local endpoint, rejects redirects and bypasses proxies. It does not submit an action or establish Controller availability.

Evidence

Scenario or check Expected result Observed result
Missing/disabled setup, layer precedence and hook conflicts Nonzero diagnostic without a readiness claim Regression tests pass locally
Read-only inspection and explicit repair Preserve settings; repair cannot hide remaining failures Regression tests pass locally
Local runtime identity, redirect and deadline cases Accept matching local identity; reject unsafe or stale endpoints Tests pass locally
Go suite and runtime race checks Pass Passed on macOS
Formatting, static analysis, maintenance tests and adapter supervision Pass Passed locally
Native macOS, Linux and Windows CI Pass on the PR head All 49 checks passed on b51e43bf48a1
Real-agent and runtime replay report Every platform cell passes 33 agent cells and 3 runtime cells passed

Failure and recovery

Diagnostics return a nonzero exit for actionable failures and unresolved supported configuration checks. Review the named component; use native /hooks inspection for session-specific uncertainty. Reverting this PR restores the previous diagnostic behavior.

Remaining limits

Static success does not prove an agent action reached a Controller. Codex normalized trust hashes, managed/cloud policy and invocation overrides require native inspection. Dynamic Qwen shell forms, per-hook environment overrides and session bootstrap behavior are not evaluated by executing them. No live model calls are used by these diagnostic tests.

@github-actions

Copy link
Copy Markdown

Pitot coding-agent E2E

Intelligence Flow is the verification source; Pitot's public README carries the latest main status.

Agent Ubuntu macOS Windows Result Evidence
Claude ✅ Pass · real CLI 2.1.296 · native · binary-observed anthropic messages ✅ Pass · real CLI 2.1.296 · native · binary-observed anthropic messages ✅ Pass · real CLI 2.1.296 · native · binary-observed anthropic messages ✅ Passing All platforms are required
Cursor ✅ Pass · real CLI 2026.10.01-e373342 · native · binary-observed cursor connect proto ✅ Pass · real CLI 2026.10.01-e373342 · native · binary-observed cursor connect proto ✅ Pass · real CLI 2026.10.01-e373342 · native · binary-observed cursor connect proto ✅ Passing All platforms are required
Codex ✅ Pass · real CLI 0.162.1 · native · binary-observed openai responses ✅ Pass · real CLI 0.162.1 · native · binary-observed openai responses ✅ Pass · real CLI 0.162.1 · native · binary-observed openai responses ✅ Passing All platforms are required
Devin ✅ Pass · real CLI 3000.11.3 · native · binary-observed devin connect proto ✅ Pass · real CLI 3000.11.3 · native · binary-observed devin connect proto ✅ Pass · real CLI 3000.11.3 · native · binary-observed devin connect proto ✅ Passing All platforms are required
DeepSeek Harness ✅ Pass · real CLI 0.2.0-rc.2 · native · binary-observed openai chat ✅ Pass · real CLI 0.2.0-rc.2 · native · binary-observed openai chat ✅ Pass · real CLI 0.2.0-rc.2 · native · binary-observed openai chat ✅ Passing All platforms are required
GitHub Copilot CLI ✅ Pass · real CLI 1.0.95 · native · binary-observed openai chat ✅ Pass · real CLI 1.0.95 · native · binary-observed openai chat ✅ Pass · real CLI 1.0.95 · native · binary-observed openai chat ✅ Passing All platforms are required
Gemini ✅ Pass · real CLI 0.63.0 · native · binary-observed gemini generate content ✅ Pass · real CLI 0.63.0 · native · binary-observed gemini generate content ✅ Pass · real CLI 0.63.0 · native · binary-observed gemini generate content ✅ Passing All platforms are required
Kimi Code ✅ Pass · real CLI 2.1.1 · native · binary-observed openai chat ✅ Pass · real CLI 2.1.1 · native · binary-observed openai chat ✅ Pass · real CLI 2.1.1 · native · binary-observed openai chat ✅ Passing All platforms are required
OpenCode ✅ Pass · real CLI 2.0.27 · native · binary-observed openai responses ✅ Pass · real CLI 2.0.27 · native · binary-observed openai responses ✅ Pass · real CLI 2.0.27 · native · binary-observed openai responses ✅ Passing All platforms are required
Pi ✅ Pass · real CLI 1.1.0 · native · binary-observed openai chat ✅ Pass · real CLI 1.1.0 · native · binary-observed openai chat ✅ Pass · real CLI 1.1.0 · native · binary-observed openai chat ✅ Passing All platforms are required
Qwen Code ✅ Pass · real CLI 0.25.1 · native · binary-observed openai chat ✅ Pass · real CLI 0.25.1 · native · binary-observed openai chat ✅ Pass · real CLI 0.25.1 · native · binary-observed openai chat ✅ Passing All platforms are required

Runtime capabilities

Capability Ubuntu macOS Windows Result
pitot request ✅ Pass · real runtime allow/deny ✅ Pass · real runtime allow/deny ✅ Pass · real runtime allow/deny ✅ Passing

Source commit: b51e43bf48a1

@bigboateng
bigboateng merged commit 66afac5 into main Oct 13, 2026
49 checks passed
@bigboateng
bigboateng deleted the codex/readiness-diagnostics branch October 13, 2026 10:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant