F2b: per-item execution and the runner's commit step

[mchwang]

Lane F, step F2, slice F2b: per-item execution and the runner's commit step. Based on main. F1 (#53, #74, #57, #59, #60) and F2a (#67) are merged. Related: #22, #66, #51.

Restacked after #67 merged: head fe4b427, on main 3ac388a. The three commits are this PR's own; the F2a copy dropped out because #67 is on main. The file tree is identical to 3a62df0, the head CI last passed on.

What this does

Implements "codeboost makes every commit" (design, "How codeboost runs a plan") against a fake D workspace. The four workspace operations are the ones proposed in #66 (materialize, snapshotDeclaredLinks, inspectChanges, commit, plus release).

• runner/execution.ts: executionDeps is the RunnerDeps for execute attempts:
  • prepare: a fresh workspace from the recorded head of the attempt's snapshot; a no-follow snapshot of declared symlinks; the prompt and approvedArgv from prepareExecution (F2a).
  • finish: inspectChanges, then auditRun (F2a).
    • A safety violation throws FinishFailure("Safety violation: …"): no commit, and the attempt fails with that diagnostic.
    • No changes: "planned but unchanged".
    • Otherwise the runner's own commit, with Plan-Item: P<n> and Plan-Revision: r<k> trailers, of exactly the audited paths, matched to the audit's digest. It returns the owned ledger entry.
  • release: removes the workspace.
• ItemExecutor.runTask runs the plan items in order, one attempt each:
  • out-of-scope files are committed with the item, then a checkpoint is recorded and the task moves to needs amendment;
  • a safety violation moves the task to needs human;
  • any other non-completed attempt stops the run and reports why.
• Coordinator (F1b):
  • an optional asynchronous finish step replaces validate for writable attempts; FinishFailure keeps its own diagnostic;
  • an optional release step runs after the terminal write, on every path, including endings before launch;
  • a failed release keeps the slot under a marker.
• Store (F1a): settleAttempt takes a history record and calls recordHistory inside the same transaction as completed, after its guards. This is the publication order in the F1 contract. A commit made in task storage but not published (a refused CAS, or a stop during finish) is discarded with the storage.

Validation (head f0957ad)

• npm run typecheck: passes.
• CI's unit set: 552 passed, 0 failed. 8 of those are new, in test/runner-execution.test.ts, using the real Store, coordinator and executor with a fake workspace and launcher:
  • two items in order, with trailers, ledger owners, and each item starting from the previous commit;
  • an unchanged item;
  • out-of-scope files, giving a checkpoint and needs amendment;
  • a violation, giving needs human with no commit;
  • an agent failure;
  • a refused commit, with no ledger entry;
  • a stop during the commit, which is discarded;
  • a release failure, giving a marker;
  • the release always after the terminal write.
• Mutation check: six guards were broken one at a time, and each was caught:
  • history not recorded with completed;
  • release before the terminal write;
  • a violation leaving the task running;
  • a commit despite a violation;
  • no Plan-Revision trailer;
  • out-of-scope files not pausing.

Third rebase (after #82 and #83)

Head 2afca97, restacked on #60's 99cf5e2. All four commits applied without conflicts. F2a and F2b have no Git calls to harden.

• Typecheck passes.
• CI's unit set: 833 passed.
• npm run test:browser: 65 passed.

Code review (head 2caf235)

This PR's own commits were reviewed after the lower slices' fixes. There were no findings. The branch is restacked on #60's 29ea84e.

• CI's unit set: 829 passed.
• npm run test:browser: 65 passed.

Second rebase (after #74 merged)

Head 7eab124, restacked on #60's 3ab2359.

• runner/coordinator.ts. The merged F1b: runner coordinator for attempts, slots and shutdown #74 rewrote launch and settlement, so this PR's changes were applied to main's version:
  • prepared is passed to every #endBeforeLaunch after preparation;
  • finish() and the ledger history go into settlement;
  • task storage is released after a saved terminal write on every path. That includes main's new foreign-result path, which this PR predates.
  • #release no longer overwrites a preparation marker that main already set.
• New commit: two tests. Task storage is released on the foreign-result path and on the launch-failure path. Before this, removing either release failed no test. The launch-failure gap was already in this PR.
• Validation on 7eab124.
  • Typecheck passes.
  • CI's unit set: 824 passed.
  • npm run test:browser: 65 passed.

Rebase onto main (after #53 merged)

Restacked on the rebased #60. Head 7cf7f63. Both commits applied without conflicts.

• New commit: pass the runner owner token into execution deps. F1b: runner coordinator for attempts, slots and shutdown #74 made RunnerDeps.runnerOwner required, because D checks it against the task storage owner.
  • executionDeps now takes a runnerOwner argument, meant to come from F1d's Store.runnerOwnerToken().
  • TaskWorkspace must allocate storage under that token. TaskWorkspace is D's contract (D: inspect task changes and make runner commits (needed by F2) #66), so this is documented rather than enforced.
  • The first execution test checks that D receives the token.
• Validation on 7cf7f63.
  • Typecheck passes.
  • CI's unit set: 744 passed, 0 failed. The first full run had four git-heavy timeouts, which passed on the rerun.
  • npm run test:browser: 63 passed.

Independent review fixes (head 3b59a5b)

Two rounds of review by a fresh subagent, checked against runner-lifecycle.md and plan-format.md's "After each run". Each fix has a test that fails without it.

Round 1:

• Safety findings are recorded by the runner (SafetyFindings). They are never read back from diagnostic text, so an agent's stderr can no longer move its task to needs human, and a violation survives a later stale or stop outcome.
• auditRun treats any agent commit as a violation (D: inspect task changes and make runner commits (needed by F2) #66 decision 2: no undo path). This reverses F2a's test, which counted a manifest with only agent commits as "unchanged".
• An inspection that refuses sends the task to needs human (Publishing step 2).
• Storage allocated before preparation fails is removed after the terminal write (PreparationFailure).
• The run stops before the next item when the plan gets a new revision.
• The checkpoint and the move to needs amendment commit in one transaction (Store.pauseForAmendment), through the shutdown capability.
• A rename stages both paths.
• A storage-removal failure is marked storage-not-removed, not "result could not be saved".
• runTask returns a stopped outcome with the completed items, instead of throwing, when admission is refused or the terminal write failed.

Round 2:

• A scope finding always pauses the task. The checkpoint is bound to the revision the item ran against and the snapshot its own commit created, so a revision saved during release cannot drop the pause. The round-1 fix had caused this regression.
• The stop signal and the context are re-checked before the runner commit, after the last await.
• A malformed change report is a safety violation.
• The foreign-result path removes preparation files after the terminal write.
• snapshotDeclaredLinks gets every declared path.

Round 3 (head d8713cc):

• auditRun validates every field it reads before reading any: the metadata flag, each change's kind, entry types, underGit, a rename's old path, link target and traversal flag, and total path bytes. A partial record now fails closed instead of reading as clean.
• A scope pause that was never recorded is paused at the start of the next run. This covers a failed write, the write gate and a crash, so no run can go past it.
• Between items the executor checks the whole context: the snapshot the previous commit created, the assignment and the referenced code.
• An invalid commit ID from the workspace fails the attempt instead of breaking the terminal write.
• Only the stop's own abort error counts as the stop. Any other inspection refusal stays a safety finding.
• Pausing and escalating keep a status set during release.

Round 3 left one question open: whether "Irreversible actions" (re-read review_version and closing before each commit) covers the runner commit inside task storage. It is only published by the guarded settle transaction, so this PR does not add those re-reads.

Validation: typecheck passes; CI's unit set: 860 passed. Every fix has a test that fails when the fix is removed.

Independent review, rounds 4–23 (head 5a7c1c8)

Twenty more review rounds by a fresh subagent, checked against runner-lifecycle.md, plan-format.md and AGENTS.md. Each fix has a test that fails when the fix is removed, confirmed by mutation. Rounds 20–23 found no correctness bug, only test gaps and hardening.

The main changes:

• Scope pauses.
  • A pause cannot be skipped. An owed pause is paid first on the next run, found by its commit head, from running, queued or a review status. The checkpoint and the status change are one transaction.
  • A task with any scope checkpoint runs no further items. Continuing after a pause is F2: continue a task after a scope pause (reconcile the prefix, validate the rest) #88.
• Safety findings.
  • Findings are recorded by the runner, never parsed from text. A finding is settled only once acted on.
  • Human gates keep it owed. Review statuses are escalated, so a task cannot be merged past a finding.
  • A finding owed from an earlier run is paid without the shutdown capability.
• One run per task.
  • An in-flight guard and the coordinator's active-job check stop a second run from paying another run's outcome.
  • Between items, the run stops on any status, plan revision or context change, including the context generation.
• auditRun fails closed on partial or malformed reports:
  • every field, kind and entry type is checked before use;
  • duplicates are refused, except a split case-only rename;
  • paths must be canonical;
  • every .git spelling Git refuses is refused (NTFS and HFS);
  • link targets are checked as written and resolved (Windows forms, the root, ..);
  • the size limit is measured as the saved JSON;
  • agent text is quoted and bounded.
• Diagnostics. Agent and D text is quoted (stderr, preparation, inspection and commit errors, log lines). The commit title is one line, with no control, bidi or format characters, so it cannot forge trailers.
• Task storage. Removed only after a saved terminal write, on every path, including a preparation failure after allocation.

Deferred:

• F2b: connect the real workspace (partial export, failed-run audit, durable safety findings, commit export) #87: connecting the real workspace. This covers durable findings, admission gates, the D manifest types, commit export, shutdown awaiting the executor, and requeue after shutdown.
• F2: continue a task after a scope pause (reconcile the prefix, validate the rest) #88: continuing after a scope pause and the run's starting point.

Open questions for the owner:

1. Do the "Irreversible actions" re-reads apply to the runner commit inside task storage?
2. Should changes to .gitmodules or .gitattributes be violations?
3. Should this run's own pause and escalation keep using the shutdown capability after the gate closes?

Validation on 5a7c1c8:

• typecheck passes;
• CI's unit set: 937 passed.

Not in this slice

• The real workspace waits for D: inspect task changes and make runner commits (needed by F2) #66.
• Recording the preparation process group and allocation ID (F1d's markers) needs D's abortable helpers (D follow-ups required by the F1 runner lifecycle contract #51).
• F2c: review rounds and fix attempts, which will use prepareExecution in fix mode.
• F2d: the already-fixed check and opening PRs.
• Connecting the real workspace, F2b: connect the real workspace (partial export, failed-run audit, durable safety findings, commit export) #87: the partial-output export, auditing a failed run, durable safety findings (including a retry through /api/runner), and exporting the runner commit before its storage is removed. These gaps only open once D: inspect task changes and make runner commits (needed by F2) #66 part 2 replaces the fake, and they were deferred from this PR on purpose. There is no undo path for agent commits: any agent commit is a safety violation (D: inspect task changes and make runner commits (needed by F2) #66 decision 2).

🤖 Generated with Claude Code