Skip to content

perf(session): coalesce per-turn persistence to ~2 disk-touching writes - #1437

Merged
TheGreatAxios merged 4 commits into
mainfrom
cl-10667-turn-persistence-coalesce-commits-snapshot
Oct 15, 2026
Merged

TheGreatAxios merged 4 commits into
mainfrom
cl-10667-turn-persistence-coalesce-commits-snapshot

Conversation

@TheGreatAxios

Copy link
Copy Markdown
Collaborator

Summary

  • A tool-using turn previously performed ~6 disk-touching persists plus one full git commit per tool batch. response.jsonl is folded into the turns store (a repo-wide consumer audit found nothing reads it; legacy trees keep staging it in vendor commits). Tool-batch commits relax to segments-only staging with the full commit cycle deferred to reply/wait/done/suspend terminals (vendored reactor patch, ledgered, superseding the per-batch checkpoint entry). run.json snapshots debounce (~5s) with an identical-payload skip; heartbeats bypass the skip so a quiet long-running inference keeps its mtime liveness for any external reader; flushes wired at every observable expose (interrupt rebuild, session rotation, finalize, exec terminal). Token usage moves to an append-only sidecar keyed to the turn frontier, with load precedence sidecar > legacy metadata > empty and torn-tail drop on crash. Empty manifest writes converge to removals.
  • Crash durability is preserved: per-batch segment writes land before turn boundaries (compaction pairs with the commit cycle sequentially by design), doom-loop aborts keep staging turns, interrupt rebuild still rebuilds from on-disk segments. The pre-existing cohortCycle-failure revision-staleness hazard is documented in the ledger as out of this change's reach.
  • On a 60-turn census: disk-touching persists ~6 → ~3-4 on real turns (write ops 41.2 → 26.6/turn legacy flow), wall time 10.2 → 3.9 ms/turn. Ships with a repeatable census script (scripts/turn-persistence-census.ts).

Verification

  • bun run check exit 0: 8,959 pass / 0 fail across 581 files.
  • Referee suites: compaction-atomicity + crash-finalize + exec-signal-finalize + signal-finalize 13 pass / 0 fail; session/subagent suites 1,541 pass / 0 fail; new debounce + shared-selectors tests.
  • Independent durability review after the snapshot fixes: compaction race audit verified, doom-loop staging verified, sidecar precedence + torn-tail semantics verified, PATCHES ledger swap clean and bijective for the reactor file.

Fixes CL-10667

A tool-using turn performed ~6 separate disk-touching persists and one
commit per tool batch. response.jsonl folded (nothing read it); tool
batches now touch segments only, with the full commit cycle at
reply/wait/done terminals; run.json snapshots debounce with an
identical-payload skip and flush at every observable state expose;
token usage moves to an append-only sidecar so metadata rewrites stop
re-hashing it; empty manifest writes converge to removals. On a 60-turn
census, disk-touching persists drop to ~3-4 on real turns and wall time
10.2 to 3.9 ms/turn.

Crash recovery still rebuilds from on-disk segments (compaction
atomicity, crash finalize, exec signal finalize all green); dooms-loop
aborts still durably stage turns as before.
@linear-code

linear-code Bot commented Oct 14, 2026

Copy link
Copy Markdown

CL-10667

@corbits-triage

corbits-triage Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Triage: Fail

@TheGreatAxios, please address the following:

  • pull request touches a forbidden path: vendor/intx-inference/PATCHES.md, vendor/intx-inference/src/reactor.test.ts, vendor/intx-inference/src/reactor.ts

@TheGreatAxios
TheGreatAxios changed the base branch from cl-10666-session-perf-cut-per-turn-walks-re-parses-stringifies to main October 15, 2026 20:26
…ence-coalesce-commits-snapshot

# Conflicts:
#	src/session/optimized-context-store.ts

@corbits-triage corbits-triage Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to look at what we changed originally vs upstream. We can 100% touch this code but we need to better understand what we changed vs original

A crash during writeFile left "{not-json" as metadata.json. load() parsed
that document before the token-usage sidecar, so the catch assigned
emptyMetadata() and dropped a valid sidecar line. Sidecar usage is read
even when the document throws. metadata.json is published by renaming a
temp file in the same directory, so the published name is never partial.
@TheGreatAxios
TheGreatAxios merged commit 6ad8089 into main Oct 15, 2026
13 checks passed
@TheGreatAxios

Copy link
Copy Markdown
Collaborator Author

Independent perf verification for this PR (measure-only lane, no source touched).

Method: same-workload, same-host probes on both trees (before = pre-merge parent, after = origin/main): store-level workload matching the shipped scripts/turn-persistence-census.ts (1 inference cycle + 2 tool batches + terminal per turn) with fs write-op + persist-call counters, plus direct debouncer probes.

  • Session-data (non-.git) writes/turn at 60 turns: ≈5.2 → ≈2.2. ≈6→≈2 claim CONFIRMED. Remainder after: 1 segment writeFile, 1 usage-sidecar appendFile, 1 rename per turn.
  • Git commits per turn: 2 → 1 (halved). Persist calls/turn: 10 → 7 (writeResponse eliminated, per-batch commits folded to a single terminal commit).
  • run.json snapshots: 10 rapid per-turn schedules → 0 writes before flush, exactly 1 after flush; second burst → exactly 1 more. N→1 collapse CONFIRMED.
  • Cross-checks: 20-turn totals 41.0 → ~20.1 ops/turn; shipped census --flow=legacy 29.65 → --flow=coalesced 18.10 ops/turn. Commit-message wall numbers (10.2→3.9 ms/turn) differ in absolute terms from mine (15.58→10.83 at 60 turns) — host-dependent; direction and ratio reproduce.
  • What remains by design: heartbeat force-bypass writes, terminal/crash landings, identical-payload skips. All landed writes still tmp+rename atomic per session chain.

No action needed — the coalescing measures as claimed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant