Skip to content

checkpoint.rollback has no active-run guard #16843

Closed

Description

@Adamulek123

Problem

checkpoint.rollback has no active-run guard. A rollback rewinds the filesystem and the provider conversation to a chosen run ordinal, but nothing in the server refuses to do that while a later run is still live.

Checked three layers, none guard it:

  • CheckpointRestoreSafety.ts (88 lines) covers worktree isolation only.
  • Orchestrator.ts:8435 ensureRollback checks capability and model selection only.
  • Execution at CheckpointRollbackService.ts:201-208 filters candidate runs to completed / interrupted / failed / cancelled, which excludes a running run from being rolled back itself, but does not stop the rollback happening while an unrelated later run is still executing.

The only guard is client-local, in ChatView.tsx.

Why this needs a maintainer decision rather than a fix

Adding a guard rejects flows that work today (background tasks, Waiting turns), and the correct boundary is a product question:

  • refuse only when a run is actively executing?
  • also refuse when a run is Waiting for input?
  • what happens to a queued rollback when a run starts between the check and the write?

I did not want to guess, so #16604 deliberately leaves this out and only fixes the three defects with one correct answer each.

Prior art

#15363 ("a thread cannot be archived while its turn is running") is the existing precedent for an active-run guard in this subsystem. Read it before designing one.

Impact

#15124 is the user-visible symptom on the capture side: a V2 waiting run shows "Agent is working" and offers Stop for work that has already finished, when checkpoint capture stalls. Same class of problem, different trigger.

Environment

Found by static audit of the merged orchestrator-v2 work (#2829). Re-verified present at main. No open or merged PR implements it.

Activity

  1. mirsella commented on Oct 21, 2026

    @mirsella

    This showed up in production with an extra consequence beyond "the rollback still rewinds while a later run is live": the live provider turn that the rollback skipped was never terminated, and the next run adopted it.

    Codex V2 thread, desktop nightly 0.1.2-nightly.20261021.3457:

    1. 13:45:06Z: run 95 starts, turn/start opens native turn 01a15eb5….
    2. 13:45:11Z: the message is retracted. The rollback targets checkpoint 94 with restoreFiles:false. Because the live run is excluded from runsToRollback, session.rollbackThread is not called, and no turn/interrupt is sent either. Codex keeps the turn open.
    3. 13:48:17Z: run 96 starts with turn/start; Codex returns the same live turn id 01a15eb5… (legal, one turn stays open per thread). The adapter adopts it via registerRootTurn.
    4. Every item and the turn/completed at 14:25:59Z carry that turn id, so run 96 receives nothing and stays running for 3.5 hours until a manual interrupt.
    5. The rollback effect also carried the thread's placeholder provider id (pending:…:ordinal:1) rather than the Codex native thread id.

    So a guard here is necessary but not sufficient: a refused or skipped rollback also needs the in-flight provider turn interrupted, or the next turn/start must not adopt an existing turn. Full trace, log excerpt, and repro steps: #23016.

  2. locked and limited conversation to collaborators on Oct 22, 2026
  3. converted this issue into a discussion #23291 on Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions