From feca8245073fda7a7e3fc38fb19e5b73d44fc62a Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 9 Nov 2026 13:41:55 -0800 Subject: [PATCH 1/6] feat(models): write XS on Sonnet 5.5 and XL on Fable 5.2 Anthropic's model guide puts Haiku on one-shot checkable work rather than a PR's agent loop, and says to move up a model instead of past high effort. XS now writes on claude-sonnet-5-5 medium, and XL writes on claude-fable-5-2 medium with Opus xhigh as its fallback; XL's Claude reviewer moves to Fable medium too. Co-Authored-By: Claude Opus 5.5 --- README.md | 8 ++++---- quest/m1/model-check.md | 20 +++++++++++++++++--- src/cli/models.ts | 15 +++++++++------ test/cli.test.ts | 8 +++++--- test/review.test.ts | 4 ++-- 5 files changed, 37 insertions(+), 18 deletions(-) diff --git a/README.md b/README.md index eff475f3..84c6cea6 100644 --- a/README.md +++ b/README.md @@ -1225,11 +1225,11 @@ the same for every repository unless its `AGENTS.md` names others: | Size | Writer | Fallback | |---|---|---| -| XS | Claude Code `claude-haiku-5-5` high | Codex `gpt-6.1-sol` low | +| XS | Claude Code `claude-sonnet-5-5` medium | Codex `gpt-6.1-sol` low | | S | Claude Code `claude-opus-5-5` low | Codex `gpt-6.1-sol` medium | | M | Claude Code `claude-opus-5-5` medium | Codex `gpt-6.1-sol` high | | L | Claude Code `claude-opus-5-5` high | Codex `gpt-6.1-sol` xhigh | -| XL | Claude Code `claude-opus-5-5` xhigh | none | +| XL | Claude Code `claude-fable-5-2` medium | Claude Code `claude-opus-5-5` xhigh | `quest unblocked --json` lists each ready quest's ranked `models`: its size's list, or the next size's while a `Maintainer:` item under its `## Waiting` @@ -1247,7 +1247,7 @@ XS, and `quest models` prints the entry to run: ```console $ quest models XS -claude claude-haiku-5-5 high +claude claude-sonnet-5-5 medium codex gpt-6.1-sol low $ quest models XS --writer claude-opus-5-5 @@ -1274,7 +1274,7 @@ size. Each size ranks its reviewers, the same for every repository: | S | Codex `gpt-6.1-sol` medium | Claude Code `claude-opus-5-5` low | | M, or none | Codex `gpt-6.1-sol` medium | Claude Code `claude-opus-5-5` medium | | L | Codex `gpt-6.1-sol` high | Claude Code `claude-opus-5-5` high | -| XL | Codex `gpt-6.1-sol` xhigh | Claude Code `claude-opus-5-5` xhigh | +| XL | Codex `gpt-6.1-sol` xhigh | Claude Code `claude-fable-5-2` medium | A quest branch takes its quest's size, read from the base branch's tip so a change cannot lower its own review; any other PR, such as a skill's own, takes diff --git a/quest/m1/model-check.md b/quest/m1/model-check.md index 50d04740..8ddf90bf 100644 --- a/quest/m1/model-check.md +++ b/quest/m1/model-check.md @@ -75,8 +75,16 @@ checkpoints and outages to count by hand: - Too thin (0-4 PRs per cell) to move a size; no table changed. Watch iterates below the size's effort, and `Implementer:` lines without an effort (6 of the 20 lines said `unknown`). + - Changed after that run on Anthropic's + [model guide](https://claude.dev/blog/which-claude-model-should-you-build-with/) + rather than tallies (maintainer, 2026-11-09): XS writes on Sonnet 5.5 + medium instead of Haiku high, which suits one-shot checkable work more + than a PR's agent loop, and XL writes on Fable 5.2 medium, falling back + to Opus xhigh, and reviews what Sol wrote on Fable 5.2 medium, since the + next model up at a lower effort beats Opus past high for less. Tally + those cells fresh. - Compare both tables' models (in `src/cli/models.ts`) with each vendor's current models: the newest - general model. Bump any that are stale, and check that both CLIs + model of each tier the tables pin. Bump any that are stale, and check that both CLIs accept the new ids and the pinned efforts. `--deep` and its strongest-model tier were dropped in [moq-dev/quest#361](https://github.com/moq-dev/quest/pull/361). @@ -85,7 +93,8 @@ checkpoints and outages to count by hand: 2026-11-04): Sol low, medium, medium, high, and xhigh for XS to XL since the [token waste](/quest/m0/token-waste/README.md) audit (user, 2026-11-05; it had high and xhigh for M and L), and Opus low, low, - medium, high, and high when Sol wrote the code (agent's choice). They replaced Codex `xhigh` and Claude `high` (maintainer, + medium, and high, then Fable medium for XL, when Sol wrote the code + (agent's choice, XL's the maintainer's). They replaced Codex `xhigh` and Claude `high` (maintainer, 2026-11-03), for which Claude Code 2.1.316 ran `claude-opus-5-5` with `--effort high` and `/code-review high`, and codex-cli 0.166.0 ran `gpt-6.1-sol` with `model_reasoning_effort="xhigh"`, both with `quest review`'s flags. @@ -94,13 +103,18 @@ checkpoints and outages to count by hand: premium Fable tier (`claude-fable-5-1`); `gpt-6.1-sol` is Codex's first-listed "latest workhorse" in `~/.codex/models_cache.json`, beside the frontier `gpt-6-astra`. Neither pin changed. + - Claude Code 2.1.319 did not recognize `claude-fable-5-2` on + 2026-11-09: it warned `unrecognized_model` and gave it a 200k context + window instead of Fable's 1M, though the API served it. Check that the + pinned Claude Code lists it. - Check that the pinned Claude Code maps the Claude id to each cloud provider. `quest review` keeps Bedrock, Vertex and Foundry routing ([moq-dev/quest#424](https://github.com/moq-dev/quest/pull/424)) but strips `ANTHROPIC_MODEL`, so the pin must resolve on each route. Claude Code 2.1.316's model catalog lists `claude-opus-5-5` with `provider_ids` (Bedrock `us.anthropic.claude-opus-5-5`; Vertex and - Foundry `claude-opus-5-5`). Find the new pin's entry the same way, and + Foundry `claude-opus-5-5`). Find the entries of the newer pins, + `claude-sonnet-5-5` and `claude-fable-5-2`, the same way, and plan a quest if a provider has none (coordinator, /quest-complete follow-ups, 2026-11-03). - Run the live sandbox test, `just review-sandbox`, with the CLIs installed diff --git a/src/cli/models.ts b/src/cli/models.ts index dd1af733..202b60cc 100644 --- a/src/cli/models.ts +++ b/src/cli/models.ts @@ -16,32 +16,35 @@ export interface Model { export const DEFAULT_SIZE: Size = "M"; const sol = (effort: string): Model => ({ harness: "codex", model: "gpt-6.1-sol", effort }); +const sonnet = (effort: string): Model => ({ harness: "claude", model: "claude-sonnet-5-5", effort }); const opus = (effort: string): Model => ({ harness: "claude", model: "claude-opus-5-5", effort }); -const haiku = (effort: string): Model => ({ harness: "claude", model: "claude-haiku-5-5", effort }); +const fable = (effort: string): Model => ({ harness: "claude", model: "claude-fable-5-2", effort }); /** * Each size's writers, ranked: a spawn starts the quest on the first that can * run, falling through only while nothing is written, since the model that - * writes a PR runs all its iterates. + * writes a PR runs all its iterates. XS stays off Haiku, which suits one-shot + * checkable work more than a PR's agent loop, and XL moves up to Fable rather + * than past Opus's high effort, which costs more for less. */ export const WRITERS: Record = { - XS: [haiku("high"), sol("low")], + XS: [sonnet("medium"), sol("low")], S: [opus("low"), sol("medium")], M: [opus("medium"), sol("high")], L: [opus("high"), sol("xhigh")], - XL: [opus("xhigh")], + XL: [fable("medium"), opus("xhigh")], }; /** * Each PR size's reviewers, ranked: the first whose model is not the latest - * writer's reviews it, so Opus reviews what Sol changed last, at its own write efforts. + * writer's reviews it, so Claude reviews what Sol changed last. */ export const REVIEWERS: Record = { XS: [sol("low"), opus("low")], S: [sol("medium"), opus("low")], M: [sol("medium"), opus("medium")], L: [sol("high"), opus("high")], - XL: [sol("xhigh"), opus("xhigh")], + XL: [sol("xhigh"), fable("medium")], }; /** diff --git a/test/cli.test.ts b/test/cli.test.ts index aa4dd466..a3ee56ed 100644 --- a/test/cli.test.ts +++ b/test/cli.test.ts @@ -179,24 +179,26 @@ test("unblocked routes a quest with a decision open one size up, and XL stays XL tree.append("quest/m0/epic/one.md", "- Maintainer: pick a name.\n"); expect(models(tree)).toEqual(["medium high"]); tree.write("quest/m0/epic/one.md", "# [XL] One\n\n## Goal\n\nWork.\n\n## Waiting\n\n- Maintainer: pick.\n"); - expect(models(tree)).toEqual(["xhigh"]); + expect(models(tree)).toEqual(["medium xhigh"]); }); test("models ranks a size's writers, or the one entry that runs a PR's writer", () => { const tree = Tree.baseline(); expect(tree.run("models", "XS")).toEqual({ code: 0, - stdout: "claude\tclaude-haiku-5-5\thigh\ncodex\tgpt-6.1-sol\tlow\n", + stdout: "claude\tclaude-sonnet-5-5\tmedium\ncodex\tgpt-6.1-sol\tlow\n", stderr: "", }); expect(JSON.parse(tree.run("models", "XL", "--json").stdout)).toEqual([ + { harness: "claude", model: "claude-fable-5-2", effort: "medium" }, { harness: "claude", model: "claude-opus-5-5", effort: "xhigh" }, ]); const writer = (size: string, model: string) => tree.run("models", size, "--writer", model).stdout; expect(writer("M", "gpt-6.1-sol")).toBe("codex\tgpt-6.1-sol\thigh\n"); // A writer its size does not list runs at the nearest size that lists it. expect(writer("XS", "claude-opus-5-5")).toBe("claude\tclaude-opus-5-5\tlow\n"); - expect(writer("L", "claude-haiku-5-5")).toBe("claude\tclaude-haiku-5-5\thigh\n"); + expect(writer("L", "claude-sonnet-5-5")).toBe("claude\tclaude-sonnet-5-5\tmedium\n"); + expect(writer("XS", "claude-fable-5-2")).toBe("claude\tclaude-fable-5-2\tmedium\n"); expect(writer("XL", "gpt-6.1-sol")).toBe("codex\tgpt-6.1-sol\txhigh\n"); const unknown = tree.run("models", "S", "--writer", "unknown"); expect(unknown.code).toBe(2); diff --git a/test/review.test.ts b/test/review.test.ts index 1e20b9fa..21488f5e 100644 --- a/test/review.test.ts +++ b/test/review.test.ts @@ -304,12 +304,12 @@ test("each size's reviewer is the first whose model is not the latest writer's", expect(at("M", claude)).toBe("codex gpt-6.1-sol medium"); expect(at("L", claude)).toBe("codex gpt-6.1-sol high"); expect(at("XL", claude)).toBe("codex gpt-6.1-sol xhigh"); - // Opus reviews what Sol wrote. + // Claude reviews what Sol wrote. expect(at("XS", codex)).toBe("claude claude-opus-5-5 low"); expect(at("S", codex)).toBe("claude claude-opus-5-5 low"); expect(at("M", codex)).toBe("claude claude-opus-5-5 medium"); expect(at("L", codex)).toBe("claude claude-opus-5-5 high"); - expect(at("XL", codex)).toBe("claude claude-opus-5-5 xhigh"); + expect(at("XL", codex)).toBe("claude claude-fable-5-2 medium"); // The PR's latest recorded implementer replaces the running harness, and only it is avoided. expect(at("S", authored({}, { CLAUDECODE: "1" }, "claude-opus-5-5"))).toBe("codex gpt-6.1-sol medium"); From 4c7bb83204851eb0ec3d6db5247c6a3071a477f2 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 9 Nov 2026 14:12:20 -0800 Subject: [PATCH 2/6] docs(guide): iterate a PR whose writer was retired on its size's first entry Every model-id bump leaves open PRs naming a writer no list names, for which `quest models --writer` errors; say they route like an unknown writer. Co-Authored-By: Claude Opus 5.5 --- assets/AGENTS.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/assets/AGENTS.md b/assets/AGENTS.md index a61db2bc..c110d281 100644 --- a/assets/AGENTS.md +++ b/assets/AGENTS.md @@ -184,7 +184,7 @@ A long-running coordinator lets compaction bound its context: it keeps what must Run each agent at an entry of `quest`'s ranked lists unless the repository's `AGENTS.md` names other models: as a native sub-agent on this session's harness, else through `quest agent`. Size each `/quest-iterate` you dispatch as a quest, by the work left on its PR, and name that size in its brief. -Start it on the PR's writer, the model of its first `Implementer:` line, at the entry `quest models --writer ` prints, waiting out its usage limit so the other model can review; a PR with no known writer takes its size's first entry that can run. +Start it on the PR's writer, the model of its first `Implementer:` line, at the entry `quest models --writer ` prints, waiting out its usage limit so the other model can review; a PR whose writer no list names, such as `unknown` or a retired model, takes its size's first entry that can run. `quest agent start --harness --model --effort --permission-mode ` runs a headless agent on that CLI's own login; start it as a background command in its own isolated worktree, so the harness reports its exit as it would a sub-agent's. `--permission-mode` is Claude Code's permission mode, or Codex's sandbox mode. From d3ac7540581c2898f62639a2bf75689ade4b3f0f Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 9 Nov 2026 14:50:19 -0800 Subject: [PATCH 3/6] docs: note that Fable requires data retention An organization on zero data retention without Fable access names other XL writers and reviewers in its AGENTS.md. Co-Authored-By: Claude Opus 5.5 --- README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/README.md b/README.md index 84c6cea6..913183ed 100644 --- a/README.md +++ b/README.md @@ -1231,6 +1231,10 @@ the same for every repository unless its `AGENTS.md` names others: | L | Claude Code `claude-opus-5-5` high | Codex `gpt-6.1-sol` xhigh | | XL | Claude Code `claude-fable-5-2` medium | Claude Code `claude-opus-5-5` xhigh | +Fable [requires data retention](https://code.claude.com/docs/en/zero-data-retention#model-availability-under-zdr), +so an organization on zero data retention without Fable access names other XL +writers and reviewers in its `AGENTS.md`. + `quest unblocked --json` lists each ready quest's ranked `models`: its size's list, or the next size's while a `Maintainer:` item under its `## Waiting` leaves a decision open, since settling it is part of the work; an XL stays From 0365b6c71afbd5c9e99453a766314839d42ba81e Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 9 Nov 2026 15:14:49 -0800 Subject: [PATCH 4/6] revert(models): keep XL on Opus xhigh The maintainer keeps XL writing and reviewing on Opus xhigh rather than Fable 5.2 medium; drop Fable from the tables and its data-retention note. Co-Authored-By: Claude Opus 5.5 --- README.md | 8 ++------ quest/m1/model-check.md | 19 +++++++------------ src/cli/models.ts | 8 +++----- test/cli.test.ts | 4 +--- test/review.test.ts | 2 +- 5 files changed, 14 insertions(+), 27 deletions(-) diff --git a/README.md b/README.md index 913183ed..e6265ad4 100644 --- a/README.md +++ b/README.md @@ -1229,11 +1229,7 @@ the same for every repository unless its `AGENTS.md` names others: | S | Claude Code `claude-opus-5-5` low | Codex `gpt-6.1-sol` medium | | M | Claude Code `claude-opus-5-5` medium | Codex `gpt-6.1-sol` high | | L | Claude Code `claude-opus-5-5` high | Codex `gpt-6.1-sol` xhigh | -| XL | Claude Code `claude-fable-5-2` medium | Claude Code `claude-opus-5-5` xhigh | - -Fable [requires data retention](https://code.claude.com/docs/en/zero-data-retention#model-availability-under-zdr), -so an organization on zero data retention without Fable access names other XL -writers and reviewers in its `AGENTS.md`. +| XL | Claude Code `claude-opus-5-5` xhigh | none | `quest unblocked --json` lists each ready quest's ranked `models`: its size's list, or the next size's while a `Maintainer:` item under its `## Waiting` @@ -1278,7 +1274,7 @@ size. Each size ranks its reviewers, the same for every repository: | S | Codex `gpt-6.1-sol` medium | Claude Code `claude-opus-5-5` low | | M, or none | Codex `gpt-6.1-sol` medium | Claude Code `claude-opus-5-5` medium | | L | Codex `gpt-6.1-sol` high | Claude Code `claude-opus-5-5` high | -| XL | Codex `gpt-6.1-sol` xhigh | Claude Code `claude-fable-5-2` medium | +| XL | Codex `gpt-6.1-sol` xhigh | Claude Code `claude-opus-5-5` xhigh | A quest branch takes its quest's size, read from the base branch's tip so a change cannot lower its own review; any other PR, such as a skill's own, takes diff --git a/quest/m1/model-check.md b/quest/m1/model-check.md index 8ddf90bf..a0d991be 100644 --- a/quest/m1/model-check.md +++ b/quest/m1/model-check.md @@ -79,10 +79,10 @@ checkpoints and outages to count by hand: [model guide](https://claude.dev/blog/which-claude-model-should-you-build-with/) rather than tallies (maintainer, 2026-11-09): XS writes on Sonnet 5.5 medium instead of Haiku high, which suits one-shot checkable work more - than a PR's agent loop, and XL writes on Fable 5.2 medium, falling back - to Opus xhigh, and reviews what Sol wrote on Fable 5.2 medium, since the - next model up at a lower effort beats Opus past high for less. Tally - those cells fresh. + than a PR's agent loop. Tally that cell fresh. XL stays on Opus xhigh: + the maintainer rejected Fable 5.2 medium, which the guide favors over + Opus past high and which zero-data-retention organizations may be + refused. - Compare both tables' models (in `src/cli/models.ts`) with each vendor's current models: the newest model of each tier the tables pin. Bump any that are stale, and check that both CLIs accept the new ids and the pinned efforts. `--deep` and its strongest-model @@ -93,8 +93,7 @@ checkpoints and outages to count by hand: 2026-11-04): Sol low, medium, medium, high, and xhigh for XS to XL since the [token waste](/quest/m0/token-waste/README.md) audit (user, 2026-11-05; it had high and xhigh for M and L), and Opus low, low, - medium, and high, then Fable medium for XL, when Sol wrote the code - (agent's choice, XL's the maintainer's). They replaced Codex `xhigh` and Claude `high` (maintainer, + medium, high, and xhigh when Sol wrote the code (agent's choice). They replaced Codex `xhigh` and Claude `high` (maintainer, 2026-11-03), for which Claude Code 2.1.316 ran `claude-opus-5-5` with `--effort high` and `/code-review high`, and codex-cli 0.166.0 ran `gpt-6.1-sol` with `model_reasoning_effort="xhigh"`, both with `quest review`'s flags. @@ -103,18 +102,14 @@ checkpoints and outages to count by hand: premium Fable tier (`claude-fable-5-1`); `gpt-6.1-sol` is Codex's first-listed "latest workhorse" in `~/.codex/models_cache.json`, beside the frontier `gpt-6-astra`. Neither pin changed. - - Claude Code 2.1.319 did not recognize `claude-fable-5-2` on - 2026-11-09: it warned `unrecognized_model` and gave it a 200k context - window instead of Fable's 1M, though the API served it. Check that the - pinned Claude Code lists it. - Check that the pinned Claude Code maps the Claude id to each cloud provider. `quest review` keeps Bedrock, Vertex and Foundry routing ([moq-dev/quest#424](https://github.com/moq-dev/quest/pull/424)) but strips `ANTHROPIC_MODEL`, so the pin must resolve on each route. Claude Code 2.1.316's model catalog lists `claude-opus-5-5` with `provider_ids` (Bedrock `us.anthropic.claude-opus-5-5`; Vertex and - Foundry `claude-opus-5-5`). Find the entries of the newer pins, - `claude-sonnet-5-5` and `claude-fable-5-2`, the same way, and + Foundry `claude-opus-5-5`). Find the entry of the newer pin, + `claude-sonnet-5-5`, the same way, and plan a quest if a provider has none (coordinator, /quest-complete follow-ups, 2026-11-03). - Run the live sandbox test, `just review-sandbox`, with the CLIs installed diff --git a/src/cli/models.ts b/src/cli/models.ts index 202b60cc..262fa4a8 100644 --- a/src/cli/models.ts +++ b/src/cli/models.ts @@ -18,21 +18,19 @@ export const DEFAULT_SIZE: Size = "M"; const sol = (effort: string): Model => ({ harness: "codex", model: "gpt-6.1-sol", effort }); const sonnet = (effort: string): Model => ({ harness: "claude", model: "claude-sonnet-5-5", effort }); const opus = (effort: string): Model => ({ harness: "claude", model: "claude-opus-5-5", effort }); -const fable = (effort: string): Model => ({ harness: "claude", model: "claude-fable-5-2", effort }); /** * Each size's writers, ranked: a spawn starts the quest on the first that can * run, falling through only while nothing is written, since the model that * writes a PR runs all its iterates. XS stays off Haiku, which suits one-shot - * checkable work more than a PR's agent loop, and XL moves up to Fable rather - * than past Opus's high effort, which costs more for less. + * checkable work more than a PR's agent loop. */ export const WRITERS: Record = { XS: [sonnet("medium"), sol("low")], S: [opus("low"), sol("medium")], M: [opus("medium"), sol("high")], L: [opus("high"), sol("xhigh")], - XL: [fable("medium"), opus("xhigh")], + XL: [opus("xhigh")], }; /** @@ -44,7 +42,7 @@ export const REVIEWERS: Record = { S: [sol("medium"), opus("low")], M: [sol("medium"), opus("medium")], L: [sol("high"), opus("high")], - XL: [sol("xhigh"), fable("medium")], + XL: [sol("xhigh"), opus("xhigh")], }; /** diff --git a/test/cli.test.ts b/test/cli.test.ts index a3ee56ed..f565ac0f 100644 --- a/test/cli.test.ts +++ b/test/cli.test.ts @@ -179,7 +179,7 @@ test("unblocked routes a quest with a decision open one size up, and XL stays XL tree.append("quest/m0/epic/one.md", "- Maintainer: pick a name.\n"); expect(models(tree)).toEqual(["medium high"]); tree.write("quest/m0/epic/one.md", "# [XL] One\n\n## Goal\n\nWork.\n\n## Waiting\n\n- Maintainer: pick.\n"); - expect(models(tree)).toEqual(["medium xhigh"]); + expect(models(tree)).toEqual(["xhigh"]); }); test("models ranks a size's writers, or the one entry that runs a PR's writer", () => { @@ -190,7 +190,6 @@ test("models ranks a size's writers, or the one entry that runs a PR's writer", stderr: "", }); expect(JSON.parse(tree.run("models", "XL", "--json").stdout)).toEqual([ - { harness: "claude", model: "claude-fable-5-2", effort: "medium" }, { harness: "claude", model: "claude-opus-5-5", effort: "xhigh" }, ]); const writer = (size: string, model: string) => tree.run("models", size, "--writer", model).stdout; @@ -198,7 +197,6 @@ test("models ranks a size's writers, or the one entry that runs a PR's writer", // A writer its size does not list runs at the nearest size that lists it. expect(writer("XS", "claude-opus-5-5")).toBe("claude\tclaude-opus-5-5\tlow\n"); expect(writer("L", "claude-sonnet-5-5")).toBe("claude\tclaude-sonnet-5-5\tmedium\n"); - expect(writer("XS", "claude-fable-5-2")).toBe("claude\tclaude-fable-5-2\tmedium\n"); expect(writer("XL", "gpt-6.1-sol")).toBe("codex\tgpt-6.1-sol\txhigh\n"); const unknown = tree.run("models", "S", "--writer", "unknown"); expect(unknown.code).toBe(2); diff --git a/test/review.test.ts b/test/review.test.ts index 21488f5e..f71808d5 100644 --- a/test/review.test.ts +++ b/test/review.test.ts @@ -309,7 +309,7 @@ test("each size's reviewer is the first whose model is not the latest writer's", expect(at("S", codex)).toBe("claude claude-opus-5-5 low"); expect(at("M", codex)).toBe("claude claude-opus-5-5 medium"); expect(at("L", codex)).toBe("claude claude-opus-5-5 high"); - expect(at("XL", codex)).toBe("claude claude-fable-5-2 medium"); + expect(at("XL", codex)).toBe("claude claude-opus-5-5 xhigh"); // The PR's latest recorded implementer replaces the running harness, and only it is avoided. expect(at("S", authored({}, { CLAUDECODE: "1" }, "claude-opus-5-5"))).toBe("codex gpt-6.1-sol medium"); From a6abd7a548354d26fd6eeefc8f74640f6e7516d2 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 9 Nov 2026 16:07:29 -0800 Subject: [PATCH 5/6] feat(models): write XS on Haiku xhigh instead of Sonnet medium mypriors.com's score against subscription cost per task puts Sonnet 5.5 below the frontier at every effort; Haiku xhigh scores about 34.6 for $0.0053 a task against Sonnet medium's 30.7 for $0.012. Co-Authored-By: Claude Opus 5.5 --- README.md | 4 ++-- quest/m1/model-check.md | 31 +++++++++++++++++++++---------- src/cli/models.ts | 8 ++++---- test/cli.test.ts | 4 ++-- 4 files changed, 29 insertions(+), 18 deletions(-) diff --git a/README.md b/README.md index e6265ad4..1ef9d14c 100644 --- a/README.md +++ b/README.md @@ -1225,7 +1225,7 @@ the same for every repository unless its `AGENTS.md` names others: | Size | Writer | Fallback | |---|---|---| -| XS | Claude Code `claude-sonnet-5-5` medium | Codex `gpt-6.1-sol` low | +| XS | Claude Code `claude-haiku-5-5` xhigh | Codex `gpt-6.1-sol` low | | S | Claude Code `claude-opus-5-5` low | Codex `gpt-6.1-sol` medium | | M | Claude Code `claude-opus-5-5` medium | Codex `gpt-6.1-sol` high | | L | Claude Code `claude-opus-5-5` high | Codex `gpt-6.1-sol` xhigh | @@ -1247,7 +1247,7 @@ XS, and `quest models` prints the entry to run: ```console $ quest models XS -claude claude-sonnet-5-5 medium +claude claude-haiku-5-5 xhigh codex gpt-6.1-sol low $ quest models XS --writer claude-opus-5-5 diff --git a/quest/m1/model-check.md b/quest/m1/model-check.md index a0d991be..a1ce5121 100644 --- a/quest/m1/model-check.md +++ b/quest/m1/model-check.md @@ -75,14 +75,26 @@ checkpoints and outages to count by hand: - Too thin (0-4 PRs per cell) to move a size; no table changed. Watch iterates below the size's effort, and `Implementer:` lines without an effort (6 of the 20 lines said `unknown`). - - Changed after that run on Anthropic's - [model guide](https://claude.dev/blog/which-claude-model-should-you-build-with/) - rather than tallies (maintainer, 2026-11-09): XS writes on Sonnet 5.5 - medium instead of Haiku high, which suits one-shot checkable work more - than a PR's agent loop. Tally that cell fresh. XL stays on Opus xhigh: - the maintainer rejected Fable 5.2 medium, which the guide favors over - Opus past high and which zero-data-retention organizations may be - refused. + - Changed after that run rather than on tallies (maintainer, 2026-11-09): + XS writes on Haiku xhigh instead of high. Tally that cell fresh. The + source was [mypriors.com/adamholter](https://mypriors.com/adamholter)'s + score against subscription cost per task, a 10-benchmark index read + off its chart (score, then $ per task), which this check compares each + cell with again: + - Haiku 5.5: high 30, $0.0034; xhigh 34.6, $0.0053. + - Sonnet 5.5: medium 30.7, $0.012; below the frontier at every effort, + so rejected for XS though Anthropic's + [model guide](https://claude.dev/blog/which-claude-model-should-you-build-with/) + favors it for scoped coding. + - Opus 5.5: low 34.7, $0.015; medium 48, $0.037; high 51.7, $0.052; + xhigh 57.3, $0.092; max 58.9, $0.15. + - gpt-6.1-sol: low 34.8, $0.0094; medium 46.3, $0.021; high 50.3, + $0.034; xhigh 52.5, $0.045. + - Fable 5.2: medium 56, $3.8; max 64.6, $26.5. Rejected for XL, which + stays on Opus xhigh, on the frontier; zero-data-retention + organizations may also be refused Fable. + - S looks lopsided: Opus low scores well below its Sol medium fallback, + which costs less. Check its tallies before moving it. - Compare both tables' models (in `src/cli/models.ts`) with each vendor's current models: the newest model of each tier the tables pin. Bump any that are stale, and check that both CLIs accept the new ids and the pinned efforts. `--deep` and its strongest-model @@ -108,8 +120,7 @@ checkpoints and outages to count by hand: strips `ANTHROPIC_MODEL`, so the pin must resolve on each route. Claude Code 2.1.316's model catalog lists `claude-opus-5-5` with `provider_ids` (Bedrock `us.anthropic.claude-opus-5-5`; Vertex and - Foundry `claude-opus-5-5`). Find the entry of the newer pin, - `claude-sonnet-5-5`, the same way, and + Foundry `claude-opus-5-5`). Find the new pin's entry the same way, and plan a quest if a provider has none (coordinator, /quest-complete follow-ups, 2026-11-03). - Run the live sandbox test, `just review-sandbox`, with the CLIs installed diff --git a/src/cli/models.ts b/src/cli/models.ts index 262fa4a8..5aedf873 100644 --- a/src/cli/models.ts +++ b/src/cli/models.ts @@ -16,17 +16,17 @@ export interface Model { export const DEFAULT_SIZE: Size = "M"; const sol = (effort: string): Model => ({ harness: "codex", model: "gpt-6.1-sol", effort }); -const sonnet = (effort: string): Model => ({ harness: "claude", model: "claude-sonnet-5-5", effort }); const opus = (effort: string): Model => ({ harness: "claude", model: "claude-opus-5-5", effort }); +const haiku = (effort: string): Model => ({ harness: "claude", model: "claude-haiku-5-5", effort }); /** * Each size's writers, ranked: a spawn starts the quest on the first that can * run, falling through only while nothing is written, since the model that - * writes a PR runs all its iterates. XS stays off Haiku, which suits one-shot - * checkable work more than a PR's agent loop. + * writes a PR runs all its iterates. XS runs Haiku at xhigh, which scores about + * as Opus low does for a third of its subscription cost per task. */ export const WRITERS: Record = { - XS: [sonnet("medium"), sol("low")], + XS: [haiku("xhigh"), sol("low")], S: [opus("low"), sol("medium")], M: [opus("medium"), sol("high")], L: [opus("high"), sol("xhigh")], diff --git a/test/cli.test.ts b/test/cli.test.ts index f565ac0f..16af78d7 100644 --- a/test/cli.test.ts +++ b/test/cli.test.ts @@ -186,7 +186,7 @@ test("models ranks a size's writers, or the one entry that runs a PR's writer", const tree = Tree.baseline(); expect(tree.run("models", "XS")).toEqual({ code: 0, - stdout: "claude\tclaude-sonnet-5-5\tmedium\ncodex\tgpt-6.1-sol\tlow\n", + stdout: "claude\tclaude-haiku-5-5\txhigh\ncodex\tgpt-6.1-sol\tlow\n", stderr: "", }); expect(JSON.parse(tree.run("models", "XL", "--json").stdout)).toEqual([ @@ -196,7 +196,7 @@ test("models ranks a size's writers, or the one entry that runs a PR's writer", expect(writer("M", "gpt-6.1-sol")).toBe("codex\tgpt-6.1-sol\thigh\n"); // A writer its size does not list runs at the nearest size that lists it. expect(writer("XS", "claude-opus-5-5")).toBe("claude\tclaude-opus-5-5\tlow\n"); - expect(writer("L", "claude-sonnet-5-5")).toBe("claude\tclaude-sonnet-5-5\tmedium\n"); + expect(writer("L", "claude-haiku-5-5")).toBe("claude\tclaude-haiku-5-5\txhigh\n"); expect(writer("XL", "gpt-6.1-sol")).toBe("codex\tgpt-6.1-sol\txhigh\n"); const unknown = tree.run("models", "S", "--writer", "unknown"); expect(unknown.code).toBe(2); From 542035068262de674e3cc45ff0bd2c33c37e03fc Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 9 Nov 2026 20:09:03 -0800 Subject: [PATCH 6/6] docs(model-check): Sol medium costs more than Opus low, not less Co-Authored-By: Claude Opus 5.5 --- quest/m1/model-check.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/quest/m1/model-check.md b/quest/m1/model-check.md index a1ce5121..92a7b300 100644 --- a/quest/m1/model-check.md +++ b/quest/m1/model-check.md @@ -93,8 +93,9 @@ checkpoints and outages to count by hand: - Fable 5.2: medium 56, $3.8; max 64.6, $26.5. Rejected for XL, which stays on Opus xhigh, on the frontier; zero-data-retention organizations may also be refused Fable. - - S looks lopsided: Opus low scores well below its Sol medium fallback, - which costs less. Check its tallies before moving it. + - S looks lopsided: Opus low scores well below its Sol medium fallback + (34.7 against 46.3) for $0.006 less a task. Check its tallies before + moving it. - Compare both tables' models (in `src/cli/models.ts`) with each vendor's current models: the newest model of each tier the tables pin. Bump any that are stale, and check that both CLIs accept the new ids and the pinned efforts. `--deep` and its strongest-model