Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 49 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,12 +134,14 @@ $ quest --root examples/export unblocked
1 quest/m0/export.md

$ quest --root examples/export unblocked --json --limit 1
[{"path":"quest/m0/export.md","dependents":1}]
[{"path":"quest/m0/export.md","dependents":1,"models":[{"harness":"claude","model":"claude-opus-5-5","effort":"low"},{"harness":"codex","model":"gpt-6.1-sol","effort":"medium"}]}]
```

Each plain output line contains the count, a tab, and the path. On a terminal,
a heading explains the count. `--json` prints an ordered array, including `[]`
when no quests are ready; `--limit <count>` requires a positive integer.
when no quests are ready, and gives each quest the ranked `models` that write
it, as [Route work by size](#route-work-by-size) describes; `--limit <count>`
requires a positive integer.
Invalid trees fail with diagnostics and no ranking.
`quest-spawn` uses the ranking unless you explicitly select work in another order.

Expand Down Expand Up @@ -1160,6 +1162,50 @@ arrive together, it takes a third; a repeated Ctrl-C, which Just does not
forward, kills at the second. There is no fairness guarantee, and no timeout except
`--quick`'s: any other command waits as long as the host stays busy.

## Route work by size

`quest` ranks the harnesses, models, and efforts that write work of each size,
the same for every repository unless its `AGENTS.md` names others:

| Size | Writer | Fallback |
|---|---|---|
| XS | Claude Code `claude-haiku-5-5` high | Codex `gpt-6.1-sol` low |
| S | Claude Code `claude-opus-5-5` low | Codex `gpt-6.1-sol` medium |
| M | Claude Code `claude-opus-5-5` medium | Codex `gpt-6.1-sol` high |
| L | Claude Code `claude-opus-5-5` high | Codex `gpt-6.1-sol` xhigh |
| XL | Claude Code `claude-opus-5-5` xhigh | none |

`quest unblocked --json` lists each ready quest's ranked `models`: its size's
list, or the next size's while a `Maintainer:` item under its `## Waiting`
leaves a decision open, since settling it is part of the work; an XL stays
XL. `/quest-spawn` starts a quest on the first entry that can run, and falls
to the next only while nothing is written: when the CLI is missing or logged
out, or the agent stops at its usage limit with nothing beyond the claim
commit. Once a model writes, the PR is its own.

A `/quest-iterate` stays on the PR's writer, the model its `## Models`
record's first `Implementer:` line names, so the other model can always
review it. The coordinator dispatching it sizes the work left on the PR, not
the quest's original size, so a merge-and-record iterate on an L quest runs at
XS, and `quest models` prints the entry to run:

```console
$ quest models XS
claude claude-haiku-5-5 high
codex gpt-6.1-sol low

$ quest models XS --writer claude-opus-5-5
claude claude-opus-5-5 low
```

`--writer` prints the size's entry for that model, or else its entry at the
nearest size that lists it, the larger on a tie; a model no list names is an
error. `--json` prints an array of `{harness, model, effort}` objects. The
iterate records the size it was given in its `Implementer:` line, as
`Implementer: Claude Code, claude-opus-5-5, low effort (iterate, XS)`, so
the model check can audit the routing. `quest review` still reviews the
whole branch at the PR's size.

## Review locally

`quest review` has a model that wrote none of the code review the checkout's
Expand All @@ -1172,7 +1218,7 @@ size. Each size ranks its reviewers, the same for every repository:
| S | Codex `gpt-6.1-sol` medium | Claude Code `claude-opus-5-5` low |
| M, or none | Codex `gpt-6.1-sol` medium | Claude Code `claude-opus-5-5` medium |
| L | Codex `gpt-6.1-sol` high | Claude Code `claude-opus-5-5` high |
| XL | Codex `gpt-6.1-sol` xhigh | Claude Code `claude-opus-5-5` high |
| XL | Codex `gpt-6.1-sol` xhigh | Claude Code `claude-opus-5-5` xhigh |

A quest branch takes its quest's size, read from the base branch's tip so a
change cannot lower its own review; any other PR, such as a skill's own, takes
Expand Down
11 changes: 11 additions & 0 deletions assets/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,17 @@ Run no more sub-agents at once than the harness allows; queue the rest and start
Relay no interim report: resume a sub-agent that ended its turn on one, telling it to block until its work finishes.
A long-running coordinator lets compaction bound its context: it keeps what must outlast compaction in the forge and the quest tree, and reads a PR's state afresh before acting on it rather than trusting earlier turns.

Run each agent at an entry of `quest`'s ranked lists unless the repository's `AGENTS.md` names other models: as a native sub-agent on this session's harness, else through `quest agent`.
Size each `/quest-iterate` you dispatch as a quest, by the work left on its PR, and name that size in its brief.
Start it on the PR's writer, the model of its first `Implementer:` line, at the entry `quest models <size> --writer <model>` prints, waiting out its usage limit so the other model can review; a PR with no known writer takes its size's first entry that can run.

`quest agent start --harness <claude|codex> --model <model> --effort <level> --permission-mode <mode> <worktree> <prompt>` runs a headless agent on that CLI's own login; start it as a background command in its own isolated worktree, so the harness reports its exit as it would a sub-agent's.
`--permission-mode` is Claude Code's permission mode, or Codex's sandbox mode.
It prints the session id, then the outcome and the agent's final reply, and exits 3 at the account's usage limit, or 4 when the CLI is logged out, printing the login command.
Tell the agent that it cannot prompt, so it ends with its decisions and recommendations in its final reply, and that ending its turn kills its background commands, so it waits in the foreground.
Answer it in the background with `quest agent resume`, passing the same options, the worktree, the session id, and the answer.
Stop one with `quest agent stop <worktree>`, and each you started that still runs before you finish; `--all` would stop other sessions' agents too.

## Questions

Open each prompt with the context to answer it without opening the quest or PR: what the work is, the problem it hit, and what each option would change.
Expand Down
4 changes: 2 additions & 2 deletions assets/CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,8 +96,8 @@ Run the repository's checks locally while the PR is a draft; CI may run only on

Before a PR is marked ready for review, and before each push to one that is, a model that wrote none of its code reviews the committed branch locally, as `/quest-review` describes.

The PR body's `## Models` section records its size as `Size: M` (its quest's, or for a skill's own PR, as a quest is sized) and one `Implementer: Codex, gpt-6.1-sol, high effort` line per implementing session, added when it first pushes.
Read your model and effort rather than guess, writing `unknown` for any you cannot find: Claude Code knows its model and exports `CLAUDE_EFFORT`; Codex records `model` and `effort` in the last `turn_context` of `~/.codex/sessions/*/*/*/rollout-*-$CODEX_THREAD_ID.jsonl`; Grok records `model_id` and `reasoning_effort` in `~/.grok/sessions/*/$GROK_SESSION_ID/chat_history.jsonl`.
The PR body's `## Models` section records its size as `Size: M` (its quest's, or for a skill's own PR, as a quest is sized) and one `Implementer: Codex, gpt-6.1-sol, high effort` line per implementing session, added when it first pushes; kix records one that pushes to an open PR in its own comment.
Read your model and effort rather than guess, writing `unknown` for any you cannot find: Claude Code knows its model and exports `CLAUDE_EFFORT`, and Codex records `model` and `effort` in the last `turn_context` of `~/.codex/sessions/*/*/*/rollout-*-$CODEX_THREAD_ID.jsonl`.

GitHub gives secrets to workflows on branches pushed to the repository, as quest branches usually are, so keep pull request CI, which runs unreviewed code, read-only: a read-only `GITHUB_TOKEN`, and write credentials only in environments limited to protected branches.
Flag any workflow that breaks this to the user, and never add one.
Expand Down
1 change: 1 addition & 0 deletions assets/skills/iterate.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Parse the arguments to determine the PR number. Any other text is the user's fee
Take it with `quest iterate <pr>` before acting, passing a token you already hold for it to keep that hold, and keep the hold token it prints; `quest iterate --help` explains holds.
If it refuses because another session holds the PR, stop and report what it printed; taking over a stale hold is the user's choice.
If you cannot push to the PR's branch, stop and report.
Your `Implementer:` line ends with the size your brief gives the work, as `Implementer: Claude Code, claude-opus-5-5, low effort (iterate, XS)`.

Read the PR, its linked issues and quest, the reviews and comments posted on it, and the surrounding code.
`quest status <pr>` reports its checks, reviews by head, and unresolved threads in one call.
Expand Down
2 changes: 1 addition & 1 deletion assets/skills/review.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ If a `## Review` record that `/quest-merge` counts names the committed head, or
If you judge the change too trivial to review, skip the review instead and record the skip and its reason in `## Review`.

Otherwise, review the committed branch with `quest review --base <remote>/<base>`, which picks a reviewer that wrote none of the code, and its effort, by the PR's size; `quest review --help` explains the round.
It reads the implementing models, and on a branch that is not a quest's the size, from the PR body's `## Models` record, so write that record before the first round; a quest branch's size is read from its quest.
It reads the implementing models, and on a branch that is not a quest's the size, from the PR body's `## Models` record and kix's sessions comment, so write that record before the first round; a quest branch's size is read from its quest.
Pass `--exclude-model` or `--size` only for a PR whose body cannot hold the record.
Where the repository's root instructions name other reviewers, pass them with `--harness`, `--model`, and `--effort`.
Run it in the foreground; a round takes minutes.
Expand Down
23 changes: 6 additions & 17 deletions assets/skills/spawn.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,20 +62,9 @@ Resume that agent by messaging it, or with `quest agent resume` for a headless a
If it cannot be resumed, start a new agent with the same brief plus its claim PR, its checkout, and the token of any hold it took on the PR, which `quest status <pr>` shows, so the new one continues that work and keeps that hold rather than claiming again.
While the error persists, such as a rate limit before its reset, resume and start nothing until it clears, and report it once rather than per agent.

## Other harnesses

A sub-agent runs only this session's harness.
To start a quest on another, such as Codex from Claude Code when the user asks for it, run it as a headless agent through `quest agent`, which uses that CLI's own login.
Stop and report if it says the agent CLI is not installed.

Start each agent as a background command, in its own isolated worktree, so the harness reports its exit as it would a sub-agent's:
`quest agent start --harness <claude|codex> --model <model> --effort <level> --permission-mode <mode> <worktree> <prompt>`.
Pass the model and effort the user named; `--permission-mode` is Claude Code's permission mode, or Codex's sandbox mode.
It prints the session id first, then the outcome and the agent's final reply.
Tell the agent that it cannot prompt, so it ends with any decisions and its recommendations in its final reply, and that ending its turn ends the process and kills its background commands, so it waits in the foreground.
Ask the user, then resume that session in the background with the answer, as you would message a sub-agent: `quest agent resume` with the same options, the worktree, the session id, and the answer.
Stop one agent with `quest agent stop <worktree>`.
When the spawn ends, run `quest agent stop --all`, which stops any agent still running.

Exit status 3 means the harness's account hit its usage limit, and 4 that its CLI is logged out, printing the login command.
Start nothing more on that harness, report its outage once, and list each interrupted quest with its worktree and session id so the user can resume it later.
## Models

Start each quest on the first entry of its `models` that can run.
Fall to the next when `quest agent` reports its CLI missing or logged out, or the agent stops on a usage limit with nothing written beyond the claim commit, continuing in that checkout and claim PR.
Once a model has written anything, the PR is its: list the quest with its worktree and session id to resume after the reset.
Skip a quest whose list runs out, and report each harness's outage once.
2 changes: 1 addition & 1 deletion assets/skills/unblock.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
description: Walk the maintainer through every quest and draft PR waiting on them, and clear waits that have resolved.
guide: Format, Creation, Deletion, Summaries
guide: Format, Creation, Deletion, Coordination, Summaries
skills: grill, plan
---

Expand Down
24 changes: 23 additions & 1 deletion cloudflare/dev/github.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1268,7 +1268,29 @@ export function fakeGitHub(world: World) {
if (comment && method === "GET") {
const item = [...repo.pulls, ...repo.issues].find((i) => String(i.number) === comment[1]);
if (!item) return notFound();
return json((item.comments ?? []).map((c) => ({ body: c.body, user: { login: c.author } })));
const bot = `${world.app.slug}[bot]`;
const page = Number(url.searchParams.get("page") ?? 1);
const size = Number(url.searchParams.get("per_page") ?? 30);
return json(
(item.comments ?? [])
.map((c, i) => ({
id: item.number * 1000 + i,
body: c.body,
user: { login: c.author },
performed_via_github_app: c.author === bot ? { id: world.app.id } : null,
}))
.slice((page - 1) * size, page * size),
);
}
const edited = tail.match(/^\/issues\/comments\/(\d+)$/);
if (edited && method === "PATCH") {
if (!writes("issues") && !writes("pull_requests")) return forbidden();
const id = Number(edited[1]);
const item = [...repo.pulls, ...repo.issues].find((i) => i.number === Math.floor(id / 1000));
const found = item?.comments?.[id % 1000];
if (!found || found.author !== `${world.app.slug}[bot]`) return notFound();
found.body = String(body.body);
return json({ id, body: found.body });
}
if (comment && method === "POST") {
if (!writes("issues") && !writes("pull_requests")) return forbidden();
Expand Down
35 changes: 34 additions & 1 deletion cloudflare/src/github/service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,13 @@ import {
} from "quest/core";
import { checksState, latestContexts } from "../../../src/github/checks";
import { cleanAutoMergeRefusal, immediateMerge, queueHead } from "../../../src/github/merge";
import { implementers, withModels } from "../../../src/github/models";
import {
type Comment as RestComment,
implementers,
sessionsComment,
withModels,
withSession,
} from "../../../src/github/models";
import { BODY_LIMIT } from "../jobs/trail";
import { osiApproved } from "../licenses";
import { CI_PERMISSIONS, type CIPermission, ciProblems } from "./ci";
Expand Down Expand Up @@ -1890,6 +1896,33 @@ export class GitHubService {
);
}

/**
* Add `line`, a session's `Implementer:` item, to the App's sessions comment on `pull`, posting it on first use.
* Editing the body instead would make the App its last editor, which voids the author's `## Review` record.
*/
recordSession(repo: GitHubRepository, userId: string, pull: number, line: string) {
return this.#authorized(repo, userId, { issues: "write" }, async (token) => {
const path = `${repositoryPath(repo)}/issues/${number(pull)}/comments`;
const app = Number(this.#env.GITHUB_APP_ID);
let found: (RestComment & { id: number }) | undefined;
for (let page = 1; !found; page++) {
const comments = await this.#api<(RestComment & { id: number })[]>(
token,
`${path}?per_page=100&page=${page}`,
);
found = sessionsComment(comments, app);
if (comments.length < 100) break;
}
const body = withSession(found?.body ?? null, line);
if (body === null) return;
if (found)
await this.#api(token, `${repositoryPath(repo)}/issues/comments/${number(found.id)}`, "PATCH", {
body,
});
else await this.#api(token, path, "POST", { body });
});
}

/**
* Hand `action` the one upstream branch a job may write: its own branch,
* or an iterated pull request's head on the enrolled repository. Before
Expand Down
13 changes: 13 additions & 0 deletions cloudflare/src/jobs/publish.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,8 @@ export type Publication = {
pull: { title: string; body: string; draft: boolean } | null;
/** A merge job lands its pull request at the published head, with the merge method its instructions name. */
land: { method: MergeMethod | null } | null;
/** The publishing session's `Implementer:` line, which a commit to an existing pull request records. */
implementer: string | null;
};

/** Whether `path` names a file inside a work tree, never Git's own directory. */
Expand Down Expand Up @@ -149,6 +151,12 @@ export function parsePublication(body: Record<string, unknown>): Publication {
pull = { title, body: description, draft: input!.draft !== false };
}
if (!changes.length && !merge && !pull && !land) throw new JobError("Nothing to publish");
let implementer: Publication["implementer"] = null;
if (body.implementer !== undefined) {
implementer = text(body.implementer, 256);
if (!implementer || !/^Implementer: [^\r\n]+$/.test(implementer))
throw new JobError("An Implementer: line required");
}
return {
job,
attempt,
Expand All @@ -160,6 +168,7 @@ export function parsePublication(body: Record<string, unknown>): Publication {
submodules: submodules as Publication["submodules"],
pull,
land,
implementer,
};
}

Expand Down Expand Up @@ -270,6 +279,10 @@ export async function publish(env: Env, runner: Runner, input: Publication): Pro
if (refused.length) return decide(db, runner, input, { type: "publication", refused });
const github = new GitHubService(env);
const upstream: GitHubRepository = { owner: job.owner, name: job.name, id: job.repository };
// Work on an existing pull request writes none of its text, so its sessions comment records who changes it,
// before the commit, so a commit whose reply is lost never leaves its session unrecorded.
if ((input.changes.length || input.merge) && input.implementer && onPull(job.workflow))
await github.recordSession(upstream, job.actor, pullOf(job.target), input.implementer);
Comment on lines +282 to +285

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Validate publication eligibility before recording a completed session

This writes the session's permanent Implementer: line before contribute() and verify() establish that the publication can proceed. For example, if another writer moves the PR head before this session's first checkpoint, recordSession() succeeds and then verify() raises BranchChanged, leaving the branch unchanged. A contributor-fork refusal or a merge rejected for workflow changes has the same outcome. pullModels() nevertheless consumes that line as evidence that the model wrote the PR, so a failed attempt by the only remaining review model can make quest review refuse all configured reviewers. The existing blocked-publication tests omit implementer, so they do not exercise this side effect. Perform the eligibility/head/merge checks before publishing attribution, and distinguish pending attribution from a completed publication while preserving lost-response recovery. Add a regression asserting that a definitively rejected first publication does not add an implementer.

(Written by review (OpenAI))

let head: string | null;
try {
// A pull request opens from the branch, so a delivery alone still checks where it is.
Expand Down
48 changes: 48 additions & 0 deletions cloudflare/test/publish.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1086,6 +1086,54 @@ describe("runner publication", () => {
});
});

it("records each session that commits to an iterated pull request in the App's one sessions comment", async () => {
const id = await running({
workflow: "iterate",
target: "pulls/2",
revision: prHead,
branch: "contributor/fix",
trusted: true,
});
// Anyone can copy the marker; only the App's own comment is kix's record.
project.pulls[0]!.comments = [
{ author: "mallory", body: "<!-- quest:sessions -->\n- Implementer: x, y, z", at },
];
const body = project.pulls[0]!.body;
const one = "Implementer: claude, claude-opus-5-5, high effort";
const two = "Implementer: codex, gpt-6.1-sol, high effort";
let head = prHead;
const send = async (implementer: string, changes = [file("a", head.slice(0, 4))]) => {
const response = await publish({ job: id, attempt: 1, head, message: "m", changes, implementer });
expect(response.status).toBe(200);
head = ((await response.json()) as { head: string }).head;
};
await send(one);
await send(one);
await send(two);
const sessions = project.pulls[0]!.comments!.filter((c) => c.author === "quest-test[bot]");
expect(sessions.map((c) => c.body)).toEqual([
`<!-- quest:sessions -->\nThe sessions that published to this pull request after it opened.\n\n## Models\n\n- ${one}\n- ${two}\n`,
]);
// The body, which holds the author's own review record, is untouched.
expect(project.pulls[0]!.body).toBe(body);

const before = requests().length;
const forged = await publish({
job: id,
attempt: 1,
head,
message: "m",
changes: [file("a", "x")],
implementer: "nope",
});
expect(await forged.json()).toEqual({ error: "An Implementer: line required" });
expect(
requests()
.slice(before)
.some((r) => r.path.includes("/comments")),
).toBe(false);
});

it("rechecks the trust of a job requeued while its publication moved the head", async () => {
const id = await running({
workflow: "iterate",
Expand Down
Loading
Loading