Skip to content

H4b: trust issues before runner actions - #139

Merged
mchwang merged 14 commits into
mainfrom
codex/issue-108-trust
Oct 7, 2026
Merged

mchwang merged 14 commits into
mainfrom
codex/issue-108-trust

Conversation

@mchwang

@mchwang mchwang commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • add durable repository/issue/author-bound trust decisions with idempotent trust and untrust actions
  • fail closed before start, resume, continuation approval, every plan-item prompt, and every publish attempt; recheck the bound decision at prompt construction and actual push/open/refresh boundaries
  • include every bounded issue comment only for a matching explicitly trusted author and record the exact comment count plus SHA-256 digest on each execute attempt
  • add Trust this issue / Trust all comments / Remove trust controls with focus preservation, stale-response guards, independent overlapping updates, durable error ownership, and local demo support
  • use one settlement-inclusive deadline for sequential issue access/text reads and revalidate issue authors plus included collaborator-comment authors at the prompt and publish boundaries

Relates to #108.

Decisions

  • Trust applies to planning and execution prompts.
  • Revocation affects the next admission, prompt construction, or irreversible publish boundary; it does not interrupt an agent invocation already running.
  • A collaborator author is eligible without a local decision, while an explicit author-bound decision widens the prompt to all bounded comments.
  • Publishing is guarded in addition to the issue acceptance minimum so untrusted work cannot be pushed or published.
  • The runner view suppresses controls when the latest complete issue board knows trust is blocked; action-time reads remain authoritative.
  • One issue-read operation owns a 30-second wall budget, including the subprocess shutdown and pipe-drain tail.

These decisions are also recorded in docs/implementation/issue-prioritization.md and docs/architecture.md.

Validation

Validated head: 3e6c43f34aad3fe0eaac0e4d4211c4f3bbb0482b (based on origin/main 6dd57c884a932d2c25aaf9e487751bc77cdcee09)

  • npm run typecheck
  • npx vitest run --project parallel --maxWorkers=2 — 69 files passed; 1,827 tests passed; 1 skipped
  • npx playwright test test/browser/issues.spec.ts — 20 passed
  • focused issue/planning/runner regressions — 104 passed
  • git diff --check origin/main...HEAD
  • final independent exact-head review — no concrete findings
  • final Copilot exact-head review — no findings
  • CI test (push) — passed on validated head
  • CI test (pull_request) — passed on validated head
  • Agent isolation real-docker — passed on validated head

Review rounds

  1. Independent review found publish retries bypassing trust revocation, stale replay views, and known-untrusted controls. Fixed all with retry admission, fresh rendering, and board-derived suppression.
  2. Independent and Copilot rounds found revocation windows around publish checks and Git reads, cross-issue UI overwrites, shutdown ownership gaps, and access-failure misclassification. Added immediate external-boundary guards, target-only merges, abort-and-await ownership, and durable upstream failures.
  3. Review found authorization-to-text and per-item prompt handoff gaps. Added author/access snapshot revalidation, exact comment evidence, and prompt-boundary validation for every item.
  4. Review found stale refresh/trust response orderings. Trust responses now merge only strictly newer author-bound trust fields; deterministic tests cover both response orders and fixed-clock trust/untrust/retrust transitions.
  5. Copilot found saved runner outcomes were rejected during shutdown. Durable replay now precedes shutdown guards; independent review added real partial-request shutdown coverage.
  6. Copilot found issue-link focus loss, equal-version collaborator-state overwrite, and inaccurate included-comment size wording. Fixed and independently re-reviewed clean.
  7. Copilot found stale issue authors across collaborator pagination. Coherent issue/collaborator/issue reads now guard runner, planning, prompt, and publish boundaries; independent review additionally found and fixed removed included-comment authors.
  8. Copilot found doubled sequential issue-read budgets and overlapping UI errors being erased. Added a shared settlement-inclusive deadline and generation-owned rendered trust/refresh errors; independent review added late-success refusal and refresh reconciliation watermark coverage.
  9. Final Copilot exact-head review returned no findings. The required post-Copilot independent exact-head pass also returned no concrete findings.

No findings were declined. All findings are covered by existing AGENTS.md rules: async lifecycle ownership and per-item currentness, async review UI compare-and-swap/focus preservation, guarded external-action revalidation, overall deadline budgeting, idempotency replay before guards, and review-readiness regression requirements. No new rule would add information rather than repeat those rules.

Deferred follow-up issues: none.

@mchwang mchwang left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of 19fd41f. There are two blocking findings: trust revocation does not stop the remaining items of a run, and overlapping trust requests break the Issues screen. Three findings are smaller: missing tests, read failures labelled as refusals, and the approve-continuation gate.

Checks that passed:

  • Publishing: every publish path goes through #schedule admission: the action, startup publish, publish after a run and retries.
  • Push guard: placed after cat-file/ls-remote, with no await before git push.
  • Schema v17: migration is idempotent and tested.
  • Repository scope: keys are lower-cased.
  • Comment digest: covers exactly the prompt's comments array.
  • Replayed refusals: still throw (savedAction).

🤖 Generated with Claude Code

Comment thread runner/production.ts Outdated
Comment thread web/public/app.js Outdated
Comment thread web/public/app.js Outdated
Comment thread web/public/app.js Outdated
Comment thread web/planning.ts Outdated
Comment thread web/server.ts Outdated
Comment thread web/server.ts Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved authorization and stale-response races can hide active trust or bypass continuation guarding.

Review effort: Balanced
Findings: 3 Medium severity · 1 Low severity

Open (4)
What changed in this PR

Adds durable issue trust enforcement across runner admission, prompts, publishing, and the Issues UI.

Changes:

  • Adds schema v17 trust records and prompt-comment evidence.
  • Guards execution and publishing using current issue access.
  • Adds trust controls, demo support, and race-focused tests.
File Description
web/​server.ts Wires trust APIs and runner/publish guards.
web/​public/​style.css Styles trust controls.
web/​public/​index.html Updates Issues help text.
web/​public/​app.js Implements trust UI and response merging.
web/​planning.ts Applies trust when reading planning comments.
web/​issues.ts Adds trust-aware issue summaries.
web/​cli.ts Awaits startup publishing.
test/​store.test.ts Tests trust persistence and migration.
test/​runner-start.test.ts Tests start/resume trust admission.
test/​runner-publishing.test.ts Tests publish-boundary authorization.
test/​runner-lifecycle-store.test.ts Updates schema assertions.
test/​runner-execution.test.ts Tests comment evidence recording.
test/​runner-branch-push.test.ts Tests the final pre-push guard.
test/​publish.test.ts Updates schema assertions.
test/​planning-drafts.test.ts Updates migration assertions.
test/​issues.test.ts Tests trusted comment inclusion and access reads.
test/​issue-board.test.ts Tests trust state and endpoints.
test/​cli.test.ts Adapts startup publishing fixture.
test/​browser/​issues.spec.ts Tests trust UI focus and races.
scripts/​demo-issues.ts Adds local demo trust operations.
runner/​store.ts Adds trust storage and comment evidence.
runner/​publishing.ts Authorizes every publish attempt.
runner/​publish.ts Adds guards at mutation boundaries.
runner/​production.ts Makes issue-text caching trust-aware.
runner/​execution.ts Records prompt comment evidence.
runner/​branch-push.ts Runs authorization immediately before push.
github/​issues.ts Adds issue access and trusted comment retrieval.
docs/​implementation/​issue-prioritization.md Documents H4b trust behavior.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread web/issues.ts
Comment thread web/public/app.js Outdated
Comment thread web/server.ts Outdated
Comment thread docs/implementation/issue-prioritization.md Outdated
@mchwang

mchwang commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor Author

Automated re-review gate for validated head 42701fd:

  • Author self-review: complete; no remaining findings.
  • Independent concurrency review: complete; no remaining findings.
  • Focused validation: 540 parallel tests and 11 Issues browser tests passed; typecheck and diff check passed.
  • CI: both test jobs and real-docker are currently in progress for this head.
  • Mergeability: GitHub reports MERGEABLE; merge-state status is UNSTABLE only because checks are pending.
  • Unresolved review threads: 0.
  • Deferred follow-up issues: none.

@copilot review

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Trust can become stale during prompt reads, and several idempotency and error-classification paths remain incorrect.

Review effort: Balanced
Findings: 2 High severity · 1 Medium severity

Open (3)
Resolved since last review (4)
Previously missed (2)

In code that hasn't changed since last review

Medium severity Return replayed outcome after concurrent issue access

web/​server.ts:304

Two identical requests can both miss the replay check and overlap during issueAccess. If one commits success before the other's read fails, userAction returns that saved success here, but the unconditional throw still sends the second caller a 502. Return the replayed response so every caller observes the first durable outcome.

Medium severity Persist issue-access failures for idempotent retries

web/​server.ts:561

An issue-access failure bypasses userAction, so no outcome is saved under the action ID and retrying the same trust/untrust request performs a fresh GitHub read. Record the UpstreamFailure through userAction (while leaving shutdown aborts resendable) to preserve durable idempotency.

Comment thread runner/production.ts Outdated
Comment thread web/planning.ts Outdated
Comment thread web/planning.ts
@mchwang

mchwang commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the two summary-only findings in 09f4668: - Concurrent runner action access reads: the late failing caller now returns the first durable userAction outcome instead of unconditionally throwing 502. The deterministic regression overlaps identical start requests, commits one success, then releases the other read failure and asserts both callers receive the same durable result with one attempt. - Trust/untrust access failures: definite UpstreamFailure outcomes now go through userAction and replay as 502 without another GitHub read; shutdown/abort remains resendable. The endpoint regression changes the gateway to success before replay and still observes the saved 502 with one access read. Focused validation on this head: typecheck and diff check passed; 8 files / 545 tests passed. Independent re-review found no remaining concrete issue.

@mchwang

mchwang commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Automated re-review gate for validated head 09f4668: author self-review complete; independent concurrency review complete with no remaining findings; focused validation passed 545 tests across 8 files plus typecheck and diff check; all three exact-head CI jobs are in progress; GitHub reports MERGEABLE with UNSTABLE only because checks are pending; unresolved review threads are 0; deferred follow-up issues are none.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Planning and execution still allow trust revocation between their final authorization check and prompt construction.

Review effort: Balanced
Findings: 1 High severity

Open (1)
Resolved since last review (3)

Comment thread runner/production.ts Outdated
@mchwang

mchwang commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Pre-review gate for 9ccd465ee19c6a8e7d453ad9164baad037134bc2:

  • Independent review loop: converged with no findings after fixing the planning prompt-boundary handoff and durable planning-read failure replay.
  • Local exact-tree validation: typecheck; 69 test files, 1,806 passed and 1 skipped; 11/11 Issues browser tests.
  • Live state: mergeable; all 15 review threads resolved; CI and real-docker are in progress.
  • Deferred follow-ups: none.

Requesting the next Copilot review round for this head.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Comment evidence is persisted before launch, so failed preparation can falsely indicate that an agent received those comments.

Review effort: Balanced
Findings: None

Resolved since last review (1)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Record comment evidence only at the agent launch boundary

runner/​execution.ts:185

This records comment evidence before workspace materialization, link snapshots, the pre-launch tree check, and start(). Any preparation failure, cancellation, or stale-context stop therefore leaves a non-null digest even though no agent invocation received the prompt, so the audit record cannot reliably answer which comments the agent was given. Record the evidence at the launch boundary (or add an explicit prepared-vs-delivered state) and cover a pre-launch failure regression.

@mchwang

mchwang commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the current-head summary-only review concern in fca4c4baf8b0eadae8f5abf2646c9e2c7f78a983. Comment evidence is now persisted only after the launcher returns an owned handle. Preparation and synchronous launcher failures leave it null; an evidence-write failure cancels and awaits the live handle and retains start-not-saved.

Pre-review gate:

  • Independent review loop: converged with no findings.
  • Exact-head local validation: typecheck; 69 test files, 1,807 passed and 1 skipped; 11/11 Issues browser tests.
  • Current-head CI: push and pull-request test jobs passed; real-docker in progress.
  • Live PR state: mergeable; all 15 inline review threads resolved.
  • Deferred follow-ups: none.

Requesting another Copilot review round for this exact head.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Ambiguous browser failures discard the trust action’s idempotency key, allowing retries to overwrite newer decisions.

Review effort: Balanced
Findings: 1 High severity

Open (1)

Comment thread web/public/app.js Outdated
@mchwang

mchwang commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Pre-review gate for 47608d2003f721511930780a1570f5b7c484d4da:

  • Independent review loop: converged with no findings after the ambiguous trust-retry fix.
  • Exact-head local validation: typecheck; 69 test files, 1,807 passed and 1 skipped; 13/13 Issues browser tests.
  • Live state: mergeable; all 16 inline review threads resolved; current-head CI and real-docker are in progress.
  • Deferred follow-ups: none.

Requesting the next Copilot review round for this exact head.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Delayed trust responses can overwrite newer refreshed issue metadata and obsolete author state.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
Resolved since last review (1)

Comment thread web/public/app.js Outdated
@mchwang

mchwang commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Addressed Copilot round on aee97fd in e473f07. The post-review independent loop found two further race/coverage gaps: inverse refresh ordering could restore old metadata, and same-author cross-client decisions needed a CAS version. The final design overlays trust fields only, orders decisions with monotonic trustChangedAt, and covers both response orderings plus a frozen-clock trust → untrust → retrust sequence. Final independent round: no findings; typecheck, Store/board 59/59, issue browser 15/15, and diff check passed.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The new shutdown guard can prevent durable runner-action outcomes from replaying.

Review effort: Balanced
Findings: None

Resolved since last review (1)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Replay durable outcomes before rejecting shutdown

web/​server.ts:302

The shutdown guard runs before the durable replay lookup, so a previously completed start/resume/continuation/publish action returns 503 once the coordinator is closing instead of replaying its saved outcome. Replay the action first, and only reject an unsaved action for shutdown.

@mchwang

mchwang commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the Copilot summary finding at exact head 03465cb: trust-gated runner actions now replay a matching durable outcome before applying either the server-stopping or runner-admission shutdown guard. Added deterministic regressions for both runner admission closure and a partially received identical request that completes during real server shutdown; the latter reopens the database and proves no duplicate attempt was created. Fresh unsaved actions still return retryable 503. Validation: test/runner-start.test.ts 53/53, typecheck, and diff-check all pass. Independent review round 1 found the missing real-shutdown coverage; round 2 found no remaining issues. No findings were declined.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Equal-version responses can restore stale collaborator state, and trust completion can drop keyboard focus from issue links.

Review effort: Balanced
Findings: None

Previously missed (3)

In code that hasn't changed since last review

Medium severity Preserve focus on any table control during pending trust updates

web/​public/​app.js:878

This preserves focus only while it remains on a trust button. If a keyboard user tabs to an issue-title link while the request is pending, the response rebuilds the entire table and removes that focused link, dropping focus to the page. Preserve any focused control in the issues table (or update only the affected row) before replacing its DOM.

Medium severity Reject equal trust versions to prevent stale decisions overwriting refreshes

web/​public/​app.js:890

Require a strictly newer trust decision here. A delayed untrust response can carry the same trustChangedAt as a newer refresh but an older collaborator-derived trust value; >= lets it overwrite the refresh and show “Collaborator” after that author has lost collaborator access. Equal versions represent the same durable decision, so the current row’s fresher issue metadata must win.

Low severity Correct size-error text for outside or ghost-authored comments

github/​issues.ts:279

When explicit trust includes an outside or ghost-authored comment, the size failure still says only “collaborator comments” were too large. That misidentifies the newly admitted content; describe these as the included comments instead.

@mchwang

mchwang commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Addressed all three Copilot summary findings at exact head cc75957: (1) issue-table rerenders now preserve focus for both trust controls and title links, with a controlled pending-response browser regression; (2) equal trust-decision versions no longer replace fresher collaborator-derived state, with a regression that asserts the equal intermediate versions before releasing the stale response; and (3) explicitly trusted outsider/ghost comment overflow is accurately described as included comments. Validation: issue unit tests 52/52, issue browser tests 17/17, typecheck, and diff-check pass. Independent review confirmed all three findings, found no additional issue, and its post-fix pass found no remaining concrete findings. No findings were declined.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Issue access can retain a stale author across collaborator pagination, allowing old author-bound trust at runner or publish boundaries.

Review effort: Balanced
Findings: 1 High severity

Open (1)

Comment thread github/issues.ts Outdated
@mchwang

mchwang commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Post-Copilot independent loop for c74247c: the reported issue-author pagination race was confirmed and fixed. Independent review then found the analogous included-comment-author race; that was also fixed, regressed, and re-reviewed to a clean pass before this next Copilot request. Focused evidence: 136/136 issue+publishing tests, typecheck, and diff-check. No findings were declined.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Sequential issue reads exceed their intended overall deadline, and overlapping UI successes can hide another trust action’s failure.

Review effort: Balanced
Findings: None

Resolved since last review (1)
Previously missed (3)

In code that hasn't changed since last review

Medium severity Use one overall timeout budget for sequential issue read stages

runner/​production.ts:252

The admission and text stages each start a separate 30-second timeout, so every plan item can spend about 60 seconds on this “bounded” issue read before workspace preparation begins. Share one outer deadline signal across both calls so sequential stages consume one overall budget.

Medium severity Share one timeout deadline across issue access and text reads

web/​planning.ts:45

issueAccess and issueText each receive a fresh 30-second allowance, so one planning issue read can run for roughly 60 seconds (plus subprocess shutdown grace). ISSUE_READ_TIMEOUT_MS is documented as bounding the whole read; create one deadline signal before the first stage and share it across both calls.

Medium severity Preserve independent trust errors during overlapping requests

web/​public/​app.js:941

A successful overlapping trust request calls renderIssuesWithPending(), which resets #issues-status to the board status. If another issue's request failed first, this later success erases its “Could not trust…” error, so independent overlapping actions can hide failures. Track action errors per issue/generation (or render them per row) and clear only the error owned by the action that later succeeds or retries.

@mchwang

mchwang commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the three Copilot summary findings at exact head 3e6c43f, plus the related gaps found by independent review. Runner and planning now share one deadline signal across access/text stages, abort active GitHub work at 29.25s, await the 750ms subprocess settlement tail inside the 30s wall budget, preserve caller cancellation, and reject even a late successful callback. Trust and refresh failures are rendered state with per-issue generation/completion ownership: unrelated successes cannot erase them, successful refresh reconciles only older errors, and failures completing during refresh survive. Planning docs now describe explicit author-bound all-comment trust accurately. Focused evidence: 104/104 unit tests, issue browser 20/20, typecheck, diff-check; final independent pass clean. No findings were declined.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

It changes security-sensitive authorization, asynchronous lifecycle, persistence, and irreversible publishing boundaries across many components.

Review effort: Balanced
Findings: None

@mchwang
mchwang merged commit d463cda into main Oct 7, 2026
4 checks passed
@mchwang
mchwang deleted the codex/issue-108-trust branch October 7, 2026 15:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants