Skip to content

Bind Responses verification results to issued tool calls - #69

Merged
bigboateng merged 2 commits into
mainfrom
fix/responses-call-correlation
Oct 13, 2026
Merged

bigboateng merged 2 commits into
mainfrom
fix/responses-call-correlation

Conversation

@bigboateng

@bigboateng bigboateng commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Change contract

When: A Responses API client returns tool outputs to the local agent verification fixture.
Before: A session marker in a result for a missing or unrelated call ID can advance the allow/deny sequence. A result can also be accepted after a text-only response that never issued the call.
After: Results contribute evidence only when their call ID matches the issued phase call and that call was actually emitted. Valid conversation history remains accepted.

Boundaries preserved

  • The allow and deny calls retain their existing IDs and session markers.
  • Prior results and unrelated history do not invalidate a correctly correlated result.
  • OpenCode retains its structured tool error checks for every matching result.
  • Other model dialects and production Controller behavior remain unchanged.

Evidence

Scenario or check Expected result Observed result
Actual baseline HTTP sessions Establish the verification gap and valid history behavior 37 independent sessions; 18 admission counterexamples retained
Candidate identity, phase, history and issuance cases Reject unrelated or unissued results; accept valid results 37 independent HTTP cases and nine additional direct controls passed
Repository checks Pass after source-bound native provenance refresh 51 targeted tests, Go, static checks and formatting passed; all 198 maintenance tests completed with one existing skip after authentic provenance refresh
Native captures All supported agents and runtimes pass with current evidence 33 agent/platform cells and three runtime cells passed (capture run)
Exact-head PR CI/report All required checks pass and report matches results 49 checks passed at ab9083b68a24; report confirms 33 real-agent and three runtime cells

Failure and recovery

Uncorrelated results cannot advance the fixture trajectory and receive its existing rejection response. No retries or alternate evidence paths are added. Revert the fixture change and refresh native provenance to recover the previous implementation.

Remaining limits

The counterexamples concern the verification fixture's evidence admission. They do not demonstrate a production Controller bypass. Existing marker matching and result semantics for other model dialects remain outside this change.

@github-actions

Copy link
Copy Markdown

Pitot coding-agent E2E

Intelligence Flow is the verification source; Pitot's public README carries the latest main status.

Agent Ubuntu macOS Windows Result Evidence
Claude ✅ Pass · real CLI 2.1.296 · native · binary-observed anthropic messages ✅ Pass · real CLI 2.1.296 · native · binary-observed anthropic messages ✅ Pass · real CLI 2.1.296 · native · binary-observed anthropic messages ✅ Passing All platforms are required
Cursor ✅ Pass · real CLI 2026.10.01-e373342 · native · binary-observed cursor connect proto ✅ Pass · real CLI 2026.10.01-e373342 · native · binary-observed cursor connect proto ✅ Pass · real CLI 2026.10.01-e373342 · native · binary-observed cursor connect proto ✅ Passing All platforms are required
Codex ✅ Pass · real CLI 0.162.1 · native · binary-observed openai responses ✅ Pass · real CLI 0.162.1 · native · binary-observed openai responses ✅ Pass · real CLI 0.162.1 · native · binary-observed openai responses ✅ Passing All platforms are required
Devin ✅ Pass · real CLI 3000.11.3 · native · binary-observed devin connect proto ✅ Pass · real CLI 3000.11.3 · native · binary-observed devin connect proto ✅ Pass · real CLI 3000.11.3 · native · binary-observed devin connect proto ✅ Passing All platforms are required
DeepSeek Harness ✅ Pass · real CLI 0.2.0-rc.2 · native · binary-observed openai chat ✅ Pass · real CLI 0.2.0-rc.2 · native · binary-observed openai chat ✅ Pass · real CLI 0.2.0-rc.2 · native · binary-observed openai chat ✅ Passing All platforms are required
GitHub Copilot CLI ✅ Pass · real CLI 1.0.95 · native · binary-observed openai chat ✅ Pass · real CLI 1.0.95 · native · binary-observed openai chat ✅ Pass · real CLI 1.0.95 · native · binary-observed openai chat ✅ Passing All platforms are required
Gemini ✅ Pass · real CLI 0.63.0 · native · binary-observed gemini generate content ✅ Pass · real CLI 0.63.0 · native · binary-observed gemini generate content ✅ Pass · real CLI 0.63.0 · native · binary-observed gemini generate content ✅ Passing All platforms are required
Kimi Code ✅ Pass · real CLI 2.1.1 · native · binary-observed openai chat ✅ Pass · real CLI 2.1.1 · native · binary-observed openai chat ✅ Pass · real CLI 2.1.1 · native · binary-observed openai chat ✅ Passing All platforms are required
OpenCode ✅ Pass · real CLI 2.0.27 · native · binary-observed openai responses ✅ Pass · real CLI 2.0.27 · native · binary-observed openai responses ✅ Pass · real CLI 2.0.27 · native · binary-observed openai responses ✅ Passing All platforms are required
Pi ✅ Pass · real CLI 1.1.0 · native · binary-observed openai chat ✅ Pass · real CLI 1.1.0 · native · binary-observed openai chat ✅ Pass · real CLI 1.1.0 · native · binary-observed openai chat ✅ Passing All platforms are required
Qwen Code ✅ Pass · real CLI 0.25.1 · native · binary-observed openai chat ✅ Pass · real CLI 0.25.1 · native · binary-observed openai chat ✅ Pass · real CLI 0.25.1 · native · binary-observed openai chat ✅ Passing All platforms are required

Runtime capabilities

Capability Ubuntu macOS Windows Result
pitot request ✅ Pass · real runtime allow/deny ✅ Pass · real runtime allow/deny ✅ Pass · real runtime allow/deny ✅ Passing

Source commit: ab9083b68a24

@bigboateng
bigboateng merged commit 04c1ac3 into main Oct 13, 2026
49 checks passed
@bigboateng
bigboateng deleted the fix/responses-call-correlation branch October 13, 2026 03:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant