Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions quest/m0/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@ ownership-manifest design:

## Required

- [CI flakes](/quest/m0/ci-flakes.md) - native Windows tests and Release's macOS checks fail only on real breakage
- [Review policy](/quest/m0/review-policy.md) - CONTRIBUTING.md says whose review counts
- [Gate bench](/quest/m0/gate-bench.md) - measure headroom admission on an idle host
- [Agent lifecycle integration](/quest/m0/worktree-agents.md) - reclaim completed and suspended checkouts through their harness
- [Agent workflow helpers](/quest/m0/helpers/README.md) - CLI commands for CI waits, PR status, claims, quest moves, and serialized merges
Expand Down
36 changes: 36 additions & 0 deletions quest/m0/ci-flakes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# [M] Fail CI checks only on real breakage

## Goal

The native Windows test jobs and Release's macOS checks fail only when the
code under test is broken: no admission-count, timeout, or runner-queue
failure on a change that does not touch the code it exercises.

## Plan

Decided 2026-10-24 (user) from the spawn and merge wave that landed
[kixelated/quest#202](https://github.com/kixelated/quest/pull/202) through
[#219](https://github.com/kixelated/quest/pull/219), where native Windows
unit tests caused most failed Check runs, including on `main` (82a9566):

- **Gate headroom test (user).** `test/gate.test.ts` "concurrent processes
never admit past the headroom" races 12 callers for 4 slots before a
deadline of `SETTLE_MS/2` and asserts exactly 4 were admitted. Slow
`windows-11-arm` and `windows-2022` runners, and loaded local hosts, admit
2 or 3; the gate never admitted more than its headroom. Assert that
invariant (never more than the headroom, in the output and the ledger) and
that every caller is eventually admitted, rather than an exact count by a
fixed deadline. Rejected: treating under-admission on a slow host as a gate
bug.
- **Timeouts.** Find why `test/move.test.ts` ("refuses a claimed quest",
"reports mentions it does not edit"), several `test/start.test.ts` tests,
and `test/landed.test.ts` exceed vitest's 5 s on native Windows, and fix
that cause. No raised timeouts, retries, or sleeps.
- **Release's macOS queue (user).** Release's macOS `check` jobs can wait
out a GitHub macOS capacity notice and show as a failed check on a PR, as
the `lifecycle (macos-14)` job did before
[kixelated/quest#218](https://github.com/kixelated/quest/pull/218). Apply
the same decision: keep macOS covered on pushes to `main` and where it
guards release artifacts, without failing PRs on a runner that never
arrived.
- Reproduce each on its native runner before fixing, as #218 did.
2 changes: 2 additions & 0 deletions quest/m0/helpers/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,8 @@ commands from Sep-Oct. Session counts in each child come from that window.

## Required

- [Landing rules](/quest/m0/helpers/landing.md) - land only heads that contain the current base, ask on a missing review, one design checkpoint
- [Wait and status accuracy](/quest/m0/helpers/wait-accuracy.md) - wake on edited bot comments, honest `checks:passed`, review rounds in `quest status`
- [Status holds](/quest/m0/helpers/status-holds.md) - `quest status` shows a PR's `quest:*` hold
- [Headless agents](/quest/m0/helpers/headless-agents.md) - `/quest-spawn api` launches, resumes, and stops billed agents through one command auto mode allows

Expand Down
41 changes: 41 additions & 0 deletions quest/m0/helpers/landing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# [S] Land only heads tested against the current base

## Goal

A PR lands only on a head that contains the base branch as it is at landing
time, and the merge skill stops for the user, rather than waiting again, when
a required review of a changed head does not arrive. The merge skill, iterate,
and complete share one definition of the design checkpoint.

## Plan

Decided 2026-10-24 (user) from the merge wave that landed
[kixelated/quest#205](https://github.com/kixelated/quest/pull/205) through
[#219](https://github.com/kixelated/quest/pull/219). The user asked for these
skill and guide edits through this plan.

- **Refuse a stale head (user).** GitHub reported
[kixelated/quest#213](https://github.com/kixelated/quest/pull/213) clean,
but landing it would have broken a test #205 had just added. `quest merge
--land <sha>` refuses when the head does not contain the base branch's
current tip, naming both. The merge skill then merges the base (never
rebasing), reruns the repository's checks, and lands the new head; a
conflict-free base merge still needs no new review. Rejected: skill prose
alone, and GitHub's "require branches to be up to date" protection.
GitHub merge queues need an organization-owned repository, which
kixelated/quest is not.
- **Ask on a missing review.** `assets/skills/merge.md` says to rerun
`quest wait` with `--since` after a timeout. When the wait is for a
required review of a changed head, ask the user instead, as the guide's
Reviews section already says
([kixelated/quest#206](https://github.com/kixelated/quest/pull/206)).
- **One checkpoint.** Move the every-fourth-round design checkpoint that
[kixelated/quest#207](https://github.com/kixelated/quest/pull/207) wrote
into both `merge` and `iterate` into the guide's Reviews section, and point
`merge`, `iterate`, and `complete` at it.
- Test a stale head refused, the same head landing after a base merge, and a
head the base already contains.

## Related

- [Wait and status accuracy](/quest/m0/helpers/wait-accuracy.md) - the review-round count the checkpoint reads
35 changes: 35 additions & 0 deletions quest/m0/helpers/wait-accuracy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# [S] Report reviews and checks accurately in wait and status

## Goal

`quest wait` wakes when a bot edits its review-status comment and never
reports a failed check under `checks:passed`, and `quest status` prints how
many review rounds a PR has had since its last `Design change:` comment.

## Plan

Decided 2026-10-24 (user) from the merge wave that landed
[kixelated/quest#205](https://github.com/kixelated/quest/pull/205) through
[#219](https://github.com/kixelated/quest/pull/219):

- **Edited bot comments.** Codex reports a finished or stuck review by
editing its summary comment. `quest wait` did not wake on those edits, so
agents on [#213](https://github.com/kixelated/quest/pull/213) and
[#219](https://github.com/kixelated/quest/pull/219) read the comment by
hand after each wait. Report an edit as an event.
- **Honest `checks:passed`.** On
[#207](https://github.com/kixelated/quest/pull/207), `quest wait` printed
`checks:passed native_Windows_(windows-11-arm)` six seconds after that
optional job failed: the event covers the required checks but is labelled
with the last check to finish. Name what passed, and report a failed
optional check separately.
- **Review rounds.** `quest status` prints the number of pushed heads that
drew reviews since the last `Design change:` comment, which the merge
skill's design checkpoint counts today with `gh pr view --json`.
- Dropped (user): removing the `quest:merge` label after a PR merges; holds
only count on open PRs, so the leftover label is cosmetic.

## Related

- [Landing rules](/quest/m0/helpers/landing.md) - the checkpoint that reads the round count
- [Status holds](/quest/m0/helpers/status-holds.md) - also extends `quest status`
12 changes: 10 additions & 2 deletions quest/m0/kix/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@ connect a Linux and a macOS runner, one with two accounts, receive an issue noti
answer planning prompts, publish a draft quest PR, merge through GitHub's
rules, start a ready quest, follow its summaries, answer an implementation
question from a push notification, take its runner offline so the job is
reassigned to the other, and receive its contribution PR notification. Exercise duplicate events,
reassigned to the other, receive its contribution PR notification, and pick
merge on it while it is not yet clean, so a merge job fixes and lands it. Exercise duplicate events,
interrupted sessions, and an exhausted account with stubbed agents, starting
from the seeded local server's fake GitHub and seed (`cloudflare/dev/`), before a
live rehearsal on the operator's runner.
Expand Down Expand Up @@ -114,12 +115,19 @@ fixes what it finds:
`account_rate_limit` retry, and ChatGPT's `usage_limit_reached` error with
`resets_at`. If ChatGPT plans fall short through opencode, plan the Codex
adapter the 2026-10-15 decision allows.
- A merge job's publication refuses a merge commit that changes
`.github/workflows` relative to the head it updates, which
[kixelated/quest#209](https://github.com/kixelated/quest/pull/209) assumes
is the comparison GitHub makes. Try it with a real PR that edits a
workflow (decided 2026-10-24, user).

## Required

- [Context pages](/quest/m0/kix/context-pages.md) - a job's `context` read returns every comment, review, and check of a busy PR
- [Merge holds](/quest/m0/kix/merge-holds.md) - ended merge jobs never leave a `quest:merge` hold, through one shared hold rule
- [Deploy kix.quest](/quest/m0/kix/deploy.md) - the operator registers the App and deploys the Worker with GitHub sign-in at kix.quest
- [Operator setup](/quest/m0/kix/operator-setup.md) - fork namespace, pilot installations, and a verified test PR
- [Per-project forks](/quest/m0/kix/forks.md) - each project publishes through its own fork, recorded by repository ID
- [Job sandbox](/quest/m0/kix/sandbox.md) - untrusted PR devShells built in a VM on Linux runners, maintainer PR heads trusted, opencode sandboxed, and macOS verified
- [Job sandbox](/quest/m0/kix/sandbox/README.md) - untrusted PR devShells built in a VM on Linux runners, maintainer PR heads trusted, opencode sandboxed, and macOS verified
- [Web execution](/quest/m0/kix/execution.md) - spawn and iterate through the UI, ending in contribution PRs
- [Onboarding guide](/quest/m0/kix/guide.md) - a complete maintainer path, from runner setup through a reviewed PR
20 changes: 20 additions & 0 deletions quest/m0/kix/context-pages.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# [S] Read every page of a PR's context

## Goal

The runner's `context` tool returns all of a pull request's comments,
reviews, review-thread comments, and check contexts, however busy the PR is.

## Plan

Decided 2026-10-24 (user) as a follow-up of
[kixelated/quest#210](https://github.com/kixelated/quest/pull/210), which
paged review threads in `cloudflare/src/github/service.ts`:

- The same query still caps the first 50 comments in each thread, the last
100 PR comments, the last 50 reviews, and 100 check contexts, so a busy PR
silently drops the rest. Follow each connection's `pageInfo` cursor, and
raise an error on a missing or repeated cursor instead of looping, as #210
does.
- Test each list past its page size with the seeded fake GitHub
(`cloudflare/dev/github.ts`).
2 changes: 1 addition & 1 deletion quest/m0/kix/execution.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,4 +68,4 @@ Earlier guidance:

## Required

- [Job sandbox](/quest/m0/kix/sandbox.md) - safe iteration on PRs with contributors' commits
- [Job sandbox](/quest/m0/kix/sandbox/README.md) - safe iteration on PRs with contributors' commits
25 changes: 25 additions & 0 deletions quest/m0/kix/merge-holds.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# [S] Release merge holds reliably

## Goal

No PR keeps a `quest:merge` hold for a merge job that has ended, whatever path
ended it, and the CLI and the Worker read and apply `quest:*` holds through
one shared definition.

## Plan

Decided 2026-10-24 (user) as a follow-up of
[kixelated/quest#209](https://github.com/kixelated/quest/pull/209), whose
reviews found a missed release path in each of four rounds (a stopped attempt
that never reported, dispatch before the hold was recorded, a cancel racing
the label):

- **Sweep.** A scheduled pass releases the hold of every merge job that has
ended without a landing in progress, as a backstop for any release path the
job's own lifecycle misses.
- **One definition.** `src/cli/hold.ts` and the Worker each decide which
labels are holds and when a hold is spent. Move that rule into the core
(`quest/core`) and have both call it, as `landed()` did for merged claims
in [kixelated/quest#217](https://github.com/kixelated/quest/pull/217).
- Test a hold left by each path #209 fixed, released by the sweep, and a
live job's hold left alone.
60 changes: 25 additions & 35 deletions quest/m0/kix/sandbox.md → quest/m0/kix/sandbox/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# [L] Run untrusted PR heads on sandboxed runners
# Run untrusted PR heads on sandboxed runners

## Goal

Expand All @@ -21,26 +21,19 @@ Decided 2026-10-15, from Codex's review of kixelated/quest#121:
- OS sandbox, not a VM per job: bubblewrap on Linux and Seatbelt on macOS.
Claude Code's docs say this does not replace a container or VM; the README
documents that residual risk.
- Job coordination marks every `iterate` job untrusted and routes it only to
Linux runners that register with `vm: true`. Classify PR heads whose commits
all come from maintainers or kix jobs as trusted here.
- Untrusted heads: evaluate and build the PR's own devShell in a per-job
KVM microVM on a Linux runner, then bring the closure into the host store
and run the job's commands in the OS sandbox. Verify how to import unsigned
paths without making the runner trust arbitrary builders, and never
publish them to the shared cache. The runner refuses untrusted jobs until
then.
- macOS runners take trusted revisions only, since a Linux VM cannot build a
darwin devShell. An untrusted-head job waits with "needs a Linux runner"
while none is online. Cloud runners are Linux-only.
- Untrusted heads build their devShell in a per-job VM on a Linux runner
before their commands run in the OS sandbox. macOS runners take trusted
revisions only, since a Linux VM cannot build a darwin devShell. An
untrusted-head job waits with "needs a Linux runner" while none is online.
Cloud runners are Linux-only.
- Decided 2026-10-22 on
[kixelated/quest#188](https://github.com/kixelated/quest/pull/188), from
Codex's security review: a job resumes with an answer from a maintainer
other than the runner's owner (the job's requester) only after the owner
approves it in the inbox, since the agent would act on that text with the
owner's machine. kix holds the job and the runner refuses an answer
without `approvedAt`. The sandbox lifts this gate: remove the approval step
from kix, the inbox, and the runner.
without `approvedAt`. The sandbox lifts this gate once it is verified on
macOS.

Shipped in [kixelated/quest#212](https://github.com/kixelated/quest/pull/212)
(`src/runner/sandbox.ts`), and verified live on Linux with Claude Code
Expand All @@ -65,26 +58,23 @@ Shipped in [kixelated/quest#212](https://github.com/kixelated/quest/pull/212)
cannot run their checks without installing dependencies. An untrusted
revision's commands reach no network host. The answer-approval gate stays
until the sandbox is verified on macOS.

Remaining work:

- opencode has no native sandbox, and its jobs' commands still run unconfined
on the host (in the revision's devShell, with the attempt's private HOME).
Decided 2026-10-15: run `opencode serve` under Anthropic's sandbox-runtime
with the runner's `Sandbox` policy (`src/runner/sandbox.ts`). The runner
reaches the server on localhost, and opencode needs its providers' hosts,
so check both under sandbox-runtime's network isolation. Keep the account's
login out of what the commands can read: verify how much of its XDG home the
wrapped process exposes, and deny reads of every other account home.
Commands also inherit the per-turn `OPENCODE_SERVER_PASSWORD`, and the kix
tools' bearer token in `OPENCODE_CONFIG_CONTENT`; keep both out of reach.

- On macOS, sandboxed commands cannot reach the account's Keychain item
(`security find-generic-password`), `/private/var/folders`, or another
home, and the realpath'd `/private/tmp` directory works as `TMPDIR`.
- On Ubuntu 23.10 and later, unprivileged user namespaces need an AppArmor
profile for the runner; document one and fail fast when the sandbox cannot
start, instead of failing each job.
- Decided 2026-10-24 (user) on the same PR: opencode jobs keep running
their commands unconfined, as before the sandbox, until
[opencode sandboxing](/quest/m0/kix/sandbox/opencode.md) ships. The README
says so.
- Decided 2026-10-24 (user) in `/quest-plan`: split the remaining work into
the children below, ranked so the earliest unblock the most. Dropped
(user): read-only registry access, which Claude Code's sandbox cannot
filter by request today, and a cancellable `Workspace.prepare` fetch.
- Sources:
[Claude Code sandboxing](https://code.claude.com/docs/en/sandboxing)
and [sandbox-runtime](https://www.npmjs.com/package/@anthropic-ai/sandbox-runtime).

## Required

- [Claude Code pin](/quest/m0/kix/sandbox/claude-pin.md) - the runner's pinned Claude Code moves to the version the sandbox docs describe
- [Trusted PR heads](/quest/m0/kix/sandbox/trust.md) - PR heads whose commits all come from maintainers or kix jobs run on any runner
- [opencode sandboxing](/quest/m0/kix/sandbox/opencode.md) - opencode jobs' commands run in the same sandbox as Claude Code's
- [macOS sandbox](/quest/m0/kix/sandbox/macos.md) - the boundary verified under Seatbelt, then the answer-approval gate removed
- [Ubuntu AppArmor](/quest/m0/kix/sandbox/apparmor.md) - the sandbox starts on stock Ubuntu, or the runner refuses at startup
- [Untrusted devShells](/quest/m0/kix/sandbox/vm.md) - untrusted PR heads build their devShell in a per-job VM
19 changes: 19 additions & 0 deletions quest/m0/kix/sandbox/apparmor.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# [S] Start the sandbox on stock Ubuntu

## Goal

A runner started from an ordinary terminal on Ubuntu 23.10 or later runs
sandboxed jobs, or refuses to start with an explanation, instead of failing
each job.

## Plan

Found on [kixelated/quest#212](https://github.com/kixelated/quest/pull/212):
Ubuntu's AppArmor restriction on unprivileged user namespaces made the system
`/usr/bin/bwrap` fail. The Nix-pinned bwrap worked only because that process
tree ran under the Claude desktop app's profile, which allows them.

- Document an AppArmor profile for the runner's pinned bwrap.
- At startup, when a Claude Code account needs the sandbox, run a trivial
sandboxed command and refuse with the profile's install hint if it fails.
- Test on a stock Ubuntu image, from a normal login shell.
19 changes: 19 additions & 0 deletions quest/m0/kix/sandbox/claude-pin.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# [XS] Bump the runner's pinned Claude Code

## Goal

The runner's Nix closure pins Claude Code 2.1.305 or later instead of
2.1.291, and the sandbox's behaviour the README documents matches the pinned
version.

## Plan

Decided 2026-10-24 (user) as a follow-up of
[kixelated/quest#212](https://github.com/kixelated/quest/pull/212), whose live
boundary test passed on both 2.1.291 and 2.1.305:

- Bump the pin in `flake.nix` (`QUEST_RUNNER_CLAUDE`) and rerun the runner
tests. Check the sandbox settings the runner passes (`dontAsk`,
`--setting-sources user`, the MCP tool timeout) against the new version's
docs and changelog.
- Ranked first in the epic: the later children build on the pinned version.
21 changes: 21 additions & 0 deletions quest/m0/kix/sandbox/macos.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# [M] Verify the sandbox on macOS, then lift the approval gate

## Goal

The job boundary holds on a macOS runner under Seatbelt, and once it does, a
job resumes with another maintainer's answer without the runner owner's
approval: the approval step is gone from kix, the inbox, and the runner.

## Plan

- Rerun the hostile-repository boundary test from
[kixelated/quest#212](https://github.com/kixelated/quest/pull/212) on
macOS. Sandboxed commands must not reach the account's Keychain item
(`security find-generic-password`), `/private/var/folders`, or another
home, and the realpath'd `/private/tmp` directory must work as `TMPDIR`.
- Decided 2026-10-24 (user) on #212: the answer-approval gate stays until this
verification passes. Then remove `approvedAt` handling from the runner, the
approval decision from the inbox, and the hold from kix, and update the
"until the job sandbox ships" comments in `cloudflare/README.md`,
`cloudflare/src/jobs/routes.ts`, and `store.ts`.
- If the boundary does not hold, record what leaks here and keep the gate.
Loading
Loading