Skip to content

fix(runner): read a publication's files in one process - #330

Merged
moq-bot[bot] merged 3 commits into
mainfrom
quest/m0/kix/runner-timeout
Nov 1, 2026
Merged

moq-bot[bot] merged 3 commits into
mainfrom
quest/m0/kix/runner-timeout

Conversation

@kixelated

@kixelated kixelated commented Oct 31, 2026 •

Copy link
Copy Markdown
Collaborator

Finishes quest/m0/kix/runner-timeout.md and deletes it, along with its link in the kix epic.

Summary

The test test/runner.test.ts > "pauses a large conflicted continuation with a bounded question and intact checkout" ran past vitest's 5s limit in the host land gate (5038ms and 5170ms on 2026-10-31). That ejected moq-dev/quest#320 and moq-dev/quest#329.

Cause. I wrapped git in a logging shim and found that 126 of the test's 181 git processes were git cat-file blob <oid>, one per changed path:

  • 60 came from the checkpoint snapshot taken before the continuation. That snapshot only needs the tree and the change count, but it read every file's content anyway.
  • The rest came from the publication snapshot.

Each spawn costs tens of milliseconds on a loaded host. With the shim's extra overhead, the 60 checkpoint reads alone took 2s, and the test timed out at 5045ms.

Fix (src/runner/workspace.ts):

  • #pending() computes the tree, the merged commit, and each changed path's blob id without reading any file. The checkpoint before a continuation uses only this.
  • snapshot() builds on #pending() and reads every blob with one git cat-file --batch. This is the same pattern src/cli/tree.ts already uses.

The test now spawns 2 blob-reading processes instead of 126. This also speeds up real publications with many files. The timeout and the test itself are unchanged.

Measurements

The single test ran pinned to 2 CPUs (taskset -c 0,1), with 8 busy loops competing for those CPUs:

before after
no shim 2140ms, 2237ms 1399ms, 1760ms, 1354ms
with logging git shim 5045ms (timed out) 2565ms
unloaded ~600ms ~600ms

The remaining time goes to about 60 git calls that the continuation flow needs (fetch, worktree, merge, read-tree/add/write-tree) and to starting the fake claude/nix bun processes.

Choices made without asking

  • The batch read shares the old 64 MiB output buffer, which was previously applied per blob. kix refuses publications whose decoded total is over 16 MiB (LIMITS.bytes in cloudflare/src/jobs/publish.ts), so the total cap only affects publications kix would reject anyway. Past 64 MiB, the error now comes from git's maxBuffer instead of kix's "Publication too large".
  • The checkpoint no longer reads file contents. As a result, it no longer fails on a single file over 64 MiB in a paused checkout.

Checks

  • just check: pass
  • env -u CLAUDE_CONFIG_DIR just test: pass (726 passed, 2 skipped; worker 426 passed)

Suggested follow-ups

  • The runner calls conflicts() (git diff --diff-filter=U) twice in a row while pausing a conflicted continuation: once in Workspace.prepare and again when it builds the question. This is minor, so I'm not planning a quest for it unless the runner's spawn count matters elsewhere.

Review

  • Reviewer: Codex gpt-6.1-sol, medium effort, via quest review --base origin/main (session 01a193c3-9de8-7160-a35d-114d9e3c1f8f, round 1)
  • Reviewed head: 207e683
  • Findings: none

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

kixelated and others added 2 commits October 31, 2026 13:54
A snapshot spawned one `git cat-file blob` per changed path, and the
checkpoint before a continuation read every file's content only to count
the changes. The 60-path conflicted-continuation test spawned 126 git
processes for this; on a loaded host each spawn costs tens of
milliseconds, which pushed the test past vitest's 5s limit in the land
gate.

Publishing now reads every blob through one `git cat-file --batch`, and
the checkpoint uses the diff without reading any file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title The kix runner's conflicted-continuation test is fast under load fix(runner): read a publication's files in one process Oct 31, 2026
@kixelated
kixelated marked this pull request as ready for review October 31, 2026 21:01
@coderabbitai

coderabbitai Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 2acedcd4-c8eb-4735-af30-5b33f690939b
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@kixelated
kixelated enabled auto-merge (squash) November 1, 2026 21:01
@moq-bot
moq-bot Bot merged commit e3131a6 into main Nov 1, 2026
@moq-bot
moq-bot Bot deleted the quest/m0/kix/runner-timeout branch November 1, 2026 22:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant