Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions quest/m0/model-routing/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,12 @@ are mostly judgment.
write at (XS medium, S high, M xhigh, L and XL xhigh); and coordinators
size each `/quest-iterate` by its remaining work. The tables and the
iterate decisions are in [Route by size](/quest/m0/model-routing/models.md).
- Revised again 2026-11-05 evening (user, /quest-plan): xhigh is XL's
alone. Haiku high writes XS; Opus writes S low, M medium, L high, XL
xhigh; Sol falls back one rung lower, now through L; Sol reviews follow
[Single-pass review](/quest/m0/token-waste/review-single-pass.md)'s
table. The tables and the benchmark evidence are in
[Route by size](/quest/m0/model-routing/models.md).
- Which milestone? ✅ m0 (rejected: m1). It ranks first among m0's work
after Kix, by the 2026-11-01 order: land work, then Kix, then the rest.

Expand Down
68 changes: 54 additions & 14 deletions quest/m0/model-routing/models.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,10 @@ Decided 2026-11-04 (user, /quest-plan); the evidence is in
- One model per size, or a ranked list?
- ✅ A ranked list, so a usage limit falls through to the next entry
(user, replacing "skip the quest and report the outage")
- Which models? ✅ Opus writes the code at every size, with Sol as the
fallback through M; L and XL have no fallback and wait for Opus (user,
2026-11-05, replacing the 2026-11-04 lists that gave XS to Grok).
- Which models? ✅ Haiku writes XS and Opus writes S and up, with Sol as
the fallback through L; XL has no fallback and waits for Opus (user,
2026-11-05, revised below; replacing the 2026-11-04 lists that gave XS to
Grok).
- Grok? ✅ Removed from routing and from every review (user, 2026-11-05:
its weekly usage is close to zero and its results were weak), so
`quest agent` gains no Grok harness; the "quest agent starts Grok" quest is
Expand All @@ -38,20 +39,59 @@ Decided 2026-11-04 (user, /quest-plan); the evidence is in
a `Maintainer:` item under its `## Waiting`. An XL stays XL.

The lists, pinned in `src/cli/models.ts` beside the review lists
`quest review` reads (user, 2026-11-05):
`quest review` reads (user, 2026-11-05, revised the same evening below):

| Size | Writes (spawn and iterate) | Reviews (`quest review`) |
|---|---|---|
| XS | claude `claude-opus-5-5` low → codex `gpt-6.1-sol` medium | codex `gpt-6.1-sol` medium → claude `claude-opus-5-5` low |
| S | claude `claude-opus-5-5` medium → codex `gpt-6.1-sol` high | codex `gpt-6.1-sol` high → claude `claude-opus-5-5` medium |
| M | claude `claude-opus-5-5` high → codex `gpt-6.1-sol` xhigh | codex `gpt-6.1-sol` xhigh → claude `claude-opus-5-5` high |
| L, XL | claude `claude-opus-5-5` xhigh | codex `gpt-6.1-sol` xhigh → claude `claude-opus-5-5` xhigh |
| XS | claude `claude-haiku-5-5` high → codex `gpt-6.1-sol` low | codex `gpt-6.1-sol` low → claude `claude-opus-5-5` low |
| S | claude `claude-opus-5-5` low → codex `gpt-6.1-sol` medium | codex `gpt-6.1-sol` medium → claude `claude-opus-5-5` low |
| M | claude `claude-opus-5-5` medium → codex `gpt-6.1-sol` high | codex `gpt-6.1-sol` medium → claude `claude-opus-5-5` medium |
| L | claude `claude-opus-5-5` high → codex `gpt-6.1-sol` xhigh | codex `gpt-6.1-sol` high → claude `claude-opus-5-5` high |
| XL | claude `claude-opus-5-5` xhigh | codex `gpt-6.1-sol` xhigh → claude `claude-opus-5-5` xhigh |

The review list is the write list flipped: Sol reviews at the effort Sol
would write at, and Opus takes the review only when Sol wrote the code. This
raises every Sol review one step from
[moq-dev/quest#493](https://github.com/moq-dev/quest/pull/493)'s (XS low,
S medium, M high).
Opus takes a review only when Sol wrote the code, and never reviews XL in
practice, since Sol writes no XL.

Revised 2026-11-05 evening (user, /quest-plan, while spawning the
[token waste](/quest/m0/token-waste/README.md) quests), replacing writes of
XS Opus low, S Opus medium, M Opus high, L and XL Opus xhigh, and Sol
fallbacks of medium, high, and xhigh for XS to M. Agents already running
keep the efforts they started with. Decision prompts, chosen options
marked ✅:

- Goal: ✅ every size gets a cheaper writer, so xhigh is XL's alone.
- Opus writes: ✅ one rung lower, S low, M medium, L high, XL xhigh (the
user's proposal).
- XS writer: ✅ Haiku high, then Sol low (rejected: Haiku medium, Opus low,
Sol medium, DeepSeek through opencode). The user asked whether Haiku high
beats Opus low. No published benchmark measures Opus 5.5 at low effort.
[Bito's coding benchmark](https://bito.ai/blog/claude-haiku-5-5-vs-sonnet-5-5-vs-opus-5-5/)
(October 2026, judged by Opus) scored Haiku 5.5 high 87% on one-shot asks
and 82% on long builds against Opus 5.5 medium's 90% and 87%, at 1/4 to
1/23 of the cost; Haiku medium scored 76-79% with about three times the
variance between sessions. Anthropic warns that low effort on long agent
prompts skips checks and follow-through. models.dev prices Haiku 5.5 at
$0.10/$0.50 per million input/output tokens, five times that for a
request over 100k tokens, against Opus 5.5's $4/$20. DeepSeek would need
an opencode harness and its sandbox in `quest agent`, which left on
2026-11-04. The [model check](/quest/m1/model-check.md) moves XS back to
Opus if Haiku's review scores lag.
- Sol fallbacks: ✅ one rung lower, XS low, S medium, M high, and a new L
fallback at xhigh; XL keeps none (the user's pick; rejected: unchanged).
- Review efforts: ✅ [Single-pass review](/quest/m0/token-waste/review-single-pass.md)'s
Sol column (XS low, S medium, M medium, L high, XL xhigh) replaces the
rule that a review runs at the effort Sol would write at, since that
decision ([moq-dev/quest#509](https://github.com/moq-dev/quest/pull/509))
is newer than this quest's
([moq-dev/quest#507](https://github.com/moq-dev/quest/pull/507)) and
carries the cost data. That quest's PR changes `REVIEWERS`; this one adds
the write lists. Opus's review efforts follow its write efforts
(rejected: the write list mirrors the reviews).

Agent's choice, for the implementer to revisit: an iterate whose writer is
missing from its size's list runs at that writer's effort for the nearest
size that lists it, so an XS iterate on an Opus PR runs Opus low and an L
iterate on a Haiku PR runs Haiku high.

Iterates, decided 2026-11-05 (user, /quest-plan, after a `/quest-complete`
run in moq-dev/moq sent many merge-and-record iterates):
Expand All @@ -66,7 +106,7 @@ run in moq-dev/moq sent many merge-and-record iterates):
`Implementer: Claude Code, claude-opus-5-5, low effort (iterate, XS)`, so
the [model check](/quest/m1/model-check.md) can audit it. Not chosen:
`quest` estimating it from those signals.
- What if both models have written a PR? ✅ They never do: one writer per
- What if two models have written a PR? ✅ They never do: one writer per
PR (user, 2026-11-05, from a CodeRabbit finding on
[moq-dev/quest#507](https://github.com/moq-dev/quest/pull/507)). Each
review list holds only Sol and Opus, and `quest review` excludes every
Expand Down
1 change: 1 addition & 0 deletions quest/m0/queue-watcher/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,7 @@ Shared shape, for the children to follow:
- [The other repositories follow](/quest/m0/queue-watcher/follow.md) - quest, rind and rust2 are cut over and landing; waits for a fork PR to land through one of their queues
- [Harden the queue runners](/quest/m0/queue-watcher/runners.md) - each queue runner runs only its repository's queue workflow, and every outside fork PR needs approval
- [Tests fit their timeout on the queue runner](/quest/m0/queue-watcher/queue-test-timeouts.md) - the root and Worker suites stop timing out or stalling in merge-group runs, so the queue stops ejecting PRs for them
- [The watcher writes flaky quests](/quest/m0/queue-watcher/flaky-quest.md) - a disable's follow-up quest comes from a template in `quest land watch`, so the headless land fixer writes no quest
- [Daemons prune their logs](/quest/m0/queue-watcher/log-retention.md) - the watcher and gate delete their finished logs after a window, so state stays bounded (4.1 GiB on nixos on 2026-11-04)
- [Retire the batching gate](/quest/m0/queue-watcher/retire.md) - once no repository lands through `quest land serve`, its batching code and host units go

Expand Down
43 changes: 43 additions & 0 deletions quest/m0/queue-watcher/flaky-quest.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# [S] The watcher writes flaky quests

## Goal

When the queue watcher disables a test seen flaking twice, the follow-up
quest (`quest/m0/flaky/<name>.md`, with its epic created when missing)
comes from a fixed template in `quest land watch`, not from the land fixer.
The fixer's `disable` commit holds only the skip, whose reason names the
quest path the watcher chose. No headless agent writes quest prose.

## Plan

Decided 2026-11-05 (user, /quest-plan). The user's rule: quests are written
only by a coordinator (`/quest-complete`, `/quest-spawn`) on Opus, never by a
sub-agent. The skills already follow it: only `/quest-merge` and
`/quest-complete` plan follow-ups, and `/quest-plan` refuses to run in an
agent that cannot prompt. The land fixer, a headless Codex agent, is the one
exception: `assets/land-fix.md` has it write the flaky quest and its
`quest/m0/flaky/README.md` link in the disable commit, and
`src/cli/land-watch-flaky.ts` requires exactly one added quest file.

Decision prompts, chosen options marked ✅:

- Who writes it: ✅ the watcher, from a template (rejected: the fixer
reports the test and a coordinator runs `/quest-plan` on it later, which
leaves the skipped test untracked until then; keep the exception).
- Milestone: ✅ m0, in the queue watcher epic, whose end-to-end run
disables a flaky test (rejected: m1).

Recommendations for the implementer:

- The watcher already knows the test (the `failure` the fixer names, from
its known-flaky input) and the `merge_group` jobs that showed it, so it
picks `<name>` and passes the quest path in the fixer's input for the skip
reason, then adds the quest file to the disable commit itself, or as a
second commit on the disable branch.
- The template's Goal says to fix the test or delete it and links each job.
Its size is a fixed guess, such as `[S]`, for the next coordinator to
correct.
- [Epics list no children](/quest/m0/token-waste/child-lists.md) is in
flight and drops the epic README links; follow whichever format the base
has when this lands.
- `validateFix` then rejects a disable commit that touches `quest/` at all.