quest review: score each round out of 10

README.md

@@ -1126,7 +1126,12 @@ base gained since, which a diff against its tip shows as reverted.
 The first round runs the reviewer's own review, `codex exec review --base
 <merge-base>` or Claude Code's `/code-review <effort> <merge-base>...HEAD`, then resumes that session for
 one turn that applies the base's `REVIEW.md`, if any, and reports the findings
-in one JSON schema. Later rounds resume the same session with the new head and
+in one JSON schema, with a score from 0 to 10 for how ready the head is to land
+and one sentence on why: 10 lands as is, 7 to 9 lands after the findings are
+fixed, 4 to 6 needs rework, and 0 to 3 is the wrong approach. A head with no
+findings can still score below 10. A reply without a valid score fails like any
+malformed reply. The score informs; the findings and their refutations still
+decide when the loop ends. Later rounds resume the same session with the new head and
 `--responses`, the author's free-text fixes and refutations, which a round that
 reported findings requires. A refutation stands only when it cites evidence the
 reviewer can check. A resumed session keeps the instructions it started with,

assets/CONTEXT.md

@@ -133,7 +133,8 @@ Once the reset passes or the login is restored, the item has cleared: whoever fi
 
 The PR body's `## Models` section records the PR's size and each implementing session's harness, model, and effort.
 A quest PR's size is its quest's; a skill that opens its own PR, such as `/quest-plan`, sizes it as a quest is sized.
-The PR body's `## Review` section records the latest round: the reviewer harness, model, and effort, the reviewed head SHA, and each finding's disposition (fixed, or refuted with the reason), or the skip and its reason.
+The PR body's `## Review` section records the latest round: the reviewer harness, model, and effort, the reviewed head SHA, the round's score out of 10 with its rationale, and each finding's disposition (fixed, or refuted with the reason), or the skip and its reason.
+Never block on a score: the findings and their refutations decide when the review is done.
 Rewrite it after each round.
 A review counts for the head whose SHA it names.
 

assets/skills/review.md

@@ -30,7 +30,8 @@ Push, then flip the PR to "Ready for Review" with `quest ready <pr> <token>`, wh
 For your own draft, mark it ready for review yourself.
 Do not wait for online reviewers.
 
-Summarize the review and CI from `quest status <pr>`, naming any decisions it raised without answering them, then interactively prompt the user with your recommended action:
+Summarize the review, with its last round's score and rationale, and summarize CI from `quest status <pr>`.
+Name any decisions the review raised without answering them, then interactively prompt the user with your recommended action:
 - `/quest-merge`: If the PR is landable as `/quest-merge` defines it, or will be once fixed with no decision left for the user, such as failing checks or unaddressed automated review findings; `/quest-merge` runs `/quest-iterate` on it first.
 - `/quest-iterate`: If it raised significant issues or decisions, which it settles with the grill skill's interview.
 - skip: If the PR should not be merged yet.

cloudflare/GITHUB.md

@@ -376,16 +376,17 @@ reports the round over `POST /api/runner/review`. The agent never touches the
 report, so it can neither skip nor fake the review. A merge job publishes its
 fixes, has them reviewed, and lands only a head whose review found nothing.
 
-kix records the round as the App's `quest review` check run on the commit,
-with the reviewer, model, effort, and round as its summary and each finding as
-plain text: `success` when the round found nothing, `failure` otherwise. The
-agent fixes or refutes each finding and delivers again; the next round resumes
-the reviewer's session with its responses. kix records a report only from the
-job's current attempt, only on the head kix itself last published for the job
-and GitHub attributes to a bot, not one a merge job continued onto,
-and only from another vendor's model than the job's account runs, and writes it
-with `checks: write` on behalf of the job's requester, after rechecking their
-maintainer authority.
+kix records the round as the App's `quest review` check run on the commit, with
+the reviewer, model, effort, and round as its summary, the round's score out of
+10 in its title, and the score's rationale and each finding as plain text,
+cutting any too long to fit rather than refusing the report: `success` when the
+round found nothing, `failure` otherwise, whatever the score. The agent fixes or
+refutes each finding and delivers again; the next round resumes the reviewer's
+session with its responses. kix records a report only from the job's current
+attempt, only on the head kix itself last published for the job and GitHub
+attributes to a bot, not one a merge job continued onto, and only from another
+vendor's model than the job's account runs, and writes it with `checks: write`
+on behalf of the job's requester, after rechecking their maintainer authority.
 
 Claude Code reviews with one of the runner's Claude Code accounts that is not
 limited. Runners hold no Codex accounts, so Codex reviews with the login of

cloudflare/src/jobs/review.ts

@@ -29,15 +29,31 @@ export type ReviewReport = {
 	model: string;
 	effort: string;
 	round: number;
+	/** How ready the head is to land, from 0 to 10, and why. */
+	score: number;
+	rationale: string;
 	findings: Finding[];
 };
 
-/** Bounds a report: GitHub keeps at most 65,535 characters of a check run's text. */
-export const REVIEW_LIMITS = { findings: 100, title: 300, body: 8000 };
+/**
+ * Bounds a report: GitHub keeps at most 65,535 characters of a check run's
+ * text. Only that text shows a report's prose, so prose past its bound is cut
+ * to fit rather than refused, which would leave a head `quest review` accepted
+ * unrecorded.
+ */
+export const REVIEW_LIMITS = { findings: 100, file: 1024, title: 300, body: 8000, rationale: 8000 };
 
 const count = (value: unknown, min: number) =>
 	typeof value === "number" && Number.isSafeInteger(value) && value >= min ? value : null;
 
+/** A string cut to at most `max` characters, ending in an ellipsis when cut, or null for any other value. */
+function clip(value: unknown, max: number) {
+	if (typeof value !== "string") return null;
+	if (value.length <= max) return value;
+	// Never leave half of a surrogate pair at the cut.
+	return `${value.slice(0, max - 1).replace(/[\ud800-\udbff]$/, "")}…`;
+}
+
 /** Parse a runner's review report, or throw a JobError the runner can read. */
 export function parseReview(body: Record<string, unknown>): ReviewReport {
 	const job = id(body.job);
@@ -51,19 +67,23 @@ export function parseReview(body: Record<string, unknown>): ReviewReport {
 	const effort = text(body.effort, 50);
 	const round = count(body.round, 1);
 	if (!model || !effort || !round) throw new JobError("The reviewer's model, effort, and round are required");
+	const score = count(body.score, 0);
+	const rationale = clip(body.rationale, REVIEW_LIMITS.rationale);
+	if (score === null || score > 10 || rationale === null || !(body.rationale as string).trim())
+		throw new JobError("A review needs a score from 0 to 10 and a rationale");
 	if (!Array.isArray(body.findings) || body.findings.length > REVIEW_LIMITS.findings)
 		throw new JobError(`At most ${REVIEW_LIMITS.findings} findings per review`);
 	const findings = body.findings.map((item: Record<string, unknown> | null): Finding => {
-		const file = text(item?.file, 1024);
-		const title = text(item?.title, REVIEW_LIMITS.title);
-		const detail = typeof item?.body === "string" && item.body.length <= REVIEW_LIMITS.body ? item.body : null;
+		const file = clip(item?.file, REVIEW_LIMITS.file);
+		const title = clip(item?.title, REVIEW_LIMITS.title);
+		const detail = clip(item?.body, REVIEW_LIMITS.body);
 		const line = item?.line === null ? null : count(item?.line, 1);
 		const severity = SEVERITIES.find((s) => s === item?.severity);
-		if (!file || !title || detail === null || (item?.line !== null && line === null) || !severity)
+		if (file === null || title === null || detail === null || (item?.line !== null && line === null) || !severity)
 			throw new JobError("Each finding needs a file, line or null, severity, title, and body");
 		return { file, line, severity, title, body: detail };
 	});
-	return { job, attempt, head: body.head, reviewer, model, effort, round, findings };
+	return { job, attempt, head: body.head, reviewer, model, effort, round, score, rationale, findings };
 }
 
 /** Untrusted text inside a fence no run of backticks in it can close. */
@@ -73,25 +93,29 @@ function fenced(value: string) {
 	return `${fence}text\n${value.replace(/[\x00-\x08\x0b-\x1f\x7f]/g, "")}\n${fence}`;
 }
 
-/** The check run's output: a title, a summary naming the reviewer, and the findings as plain text. */
+/**
+ * The check run's output: a title with the findings count and score, a
+ * summary naming the reviewer, and the score's rationale and the findings as
+ * plain text.
+ */
 export function checkOutput(report: ReviewReport) {
 	const n = report.findings.length;
-	const title = n ? `${n} finding${n === 1 ? "" : "s"}` : "No findings";
+	const title = `${n ? `${n} finding${n === 1 ? "" : "s"}` : "No findings"}, scored ${report.score}/10`;
 	const summary = `A kix runner reviewed this commit with \`quest review\`: ${report.reviewer}, ${report.model} at ${report.effort} effort, round ${report.round}.`;
 	const findings = report.findings.map((f) =>
 		fenced(`${f.severity} ${f.line === null ? f.file : `${f.file}:${f.line}`} ${f.title}\n\n${f.body}`),
 	);
-	let text = "";
+	let text = fenced(`score ${report.score}/10: ${report.rationale}`);
 	for (const [i, finding] of findings.entries()) {
-		const next = `${text}${text ? "\n\n" : ""}${finding}`;
+		const next = `${text}\n\n${finding}`;
 		// GitHub refuses a check run's text past 65,535 characters.
 		if (next.length > 60_000) {
 			text += `\n\n${findings.length - i} more not shown.`;
 			break;
 		}
 		text = next;
 	}
-	return { title, summary, ...(text && { text }) };
+	return { title, summary, text };
 }
 
 /**

cloudflare/test/publish.test.ts

@@ -5,6 +5,7 @@ import { GitHubService } from "../src/github/service";
 import { app } from "../src/index";
 import * as landing from "../src/jobs/landing";
 import { refusal, restart, validPath, validateContinuation } from "../src/jobs/publish";
+import { REVIEW_LIMITS } from "../src/jobs/review";
 import * as store from "../src/jobs/store";
 import { type Repo, type World, blob, commitOf, fakeGitHub, intercept as around } from "../dev/github";
 import { privateKey } from "./github-key";
@@ -1811,6 +1812,8 @@ describe("merge jobs", () => {
 					model: "gpt-6.1-sol",
 					effort: "medium",
 					round: 1,
+					score: 10,
+					rationale: "Lands as is.",
 					findings: [],
 					...body,
 				});
@@ -1851,7 +1854,13 @@ describe("merge jobs", () => {
 			it("records findings as a failing check, which lands nothing", async () => {
 				const id = await merging(false);
 				const head = await fix(id);
-				const failed = await report({ job: id, head, findings: [finding] });
+				const failed = await report({
+					job: id,
+					head,
+					score: 4,
+					rationale: "Breaks on `null`.",
+					findings: [finding],
+				});
 				expect(await failed.json()).toEqual({ conclusion: "failure" });
 				expect(project.checkRuns).toEqual([
 					{
@@ -1860,10 +1869,12 @@ describe("merge jobs", () => {
 						conclusion: "failure",
 						app: Number(env.GITHUB_APP_ID),
 						output: {
-							title: "1 finding",
+							title: "1 finding, scored 4/10",
 							summary:
 								"A kix runner reviewed this commit with `quest review`: codex, gpt-6.1-sol at medium effort, round 1.",
-							text: "````text\nhigh a:1 Breaks on null\n\nEnds a fence: ``` and mentions @maintainer\n````",
+							text:
+								"```text\nscore 4/10: Breaks on `null`.\n```\n\n" +
+								"````text\nhigh a:1 Breaks on null\n\nEnds a fence: ``` and mentions @maintainer\n````",
 						},
 					},
 				]);
@@ -1887,14 +1898,44 @@ describe("merge jobs", () => {
 				const passed = await report({ job: id, head, round: 2 });
 				expect(await passed.json()).toEqual({ conclusion: "success" });
 				expect(project.checkRuns!.at(-1)!.output).toEqual({
-					title: "No findings",
+					title: "No findings, scored 10/10",
 					summary:
 						"A kix runner reviewed this commit with `quest review`: codex, gpt-6.1-sol at medium effort, round 2.",
+					text: "```text\nscore 10/10: Lands as is.\n```",
 				});
 				expect(await land(id, head)).toBeNull();
 				expect(lands().map((l) => l.variables.sha)).toEqual([head]);
 			});
 
+			it("cuts prose too long for the check's text instead of refusing the report", async () => {
+				const id = await merging(false);
+				const head = await fix(id);
+				const long = (n: number) => "x".repeat(n + 1);
+				const cut = (n: number) => `${"x".repeat(n - 1)}…`;
+				const failed = await report({
+					job: id,
+					head,
+					score: 2,
+					rationale: long(REVIEW_LIMITS.rationale),
+					findings: [
+						{
+							...finding,
+							file: long(REVIEW_LIMITS.file),
+							line: null,
+							title: long(REVIEW_LIMITS.title),
+							body: "",
+						},
+						{ ...finding, title: "", body: long(REVIEW_LIMITS.body) },
+					],
+				});
+				expect(await failed.json()).toEqual({ conclusion: "failure" });
+				expect(project.checkRuns!.at(-1)!.output.text).toBe(
+					`\`\`\`text\nscore 2/10: ${cut(REVIEW_LIMITS.rationale)}\n\`\`\`\n\n` +
+						`\`\`\`text\nhigh ${cut(REVIEW_LIMITS.file)} ${cut(REVIEW_LIMITS.title)}\n\n\n\`\`\`\n\n` +
+						`\`\`\`text\nhigh a:1 \n\n${cut(REVIEW_LIMITS.body)}\n\`\`\``,
+				);
+			});
+
 			it("refuses a head the job did not publish, a stale attempt, its own vendor, and malformed findings", async () => {
 				const id = await merging(false);
 				const refused = async (body: Record<string, unknown>, error: string) => {
@@ -1917,6 +1958,11 @@ describe("merge jobs", () => {
 					"Each finding needs a file, line or null, severity, title, and body",
 				);
 				await refused({ head, reviewer: "gemini" }, "The reviewer is claude or codex");
+				const unscored = "A review needs a score from 0 to 10 and a rationale";
+				await refused({ head, score: 11 }, unscored);
+				await refused({ head, score: 7.5 }, unscored);
+				await refused({ head, score: undefined }, unscored);
+				await refused({ head, rationale: " " }, unscored);
 				// A head someone else committed, which the job's branch holds after a merge job continues onto it.
 				project.commits[head]!.committer = "contributor";
 				const foreign = await report({ job: id, head });

quest/m0/model-routing/README.md

@@ -78,6 +78,5 @@ are mostly judgment.
 
 - [Grok agents](/quest/m0/model-routing/agent-harnesses.md) - `quest agent` starts Grok as well as Claude Code and Codex
 - [Route by size](/quest/m0/model-routing/models.md) - `/quest-spawn` starts each quest on the first entry of its size's ranked list that can run
-- [Review score](/quest/m0/model-routing/review-score.md) - every `quest review` round scores the change out of 10
 - [Commits and PRs record their models](/quest/m0/model-routing/implementer.md) - commit trailers and PR records name each harness, model, and effort, and the PR's size
 - [moq-dev pins](/quest/m0/model-routing/pins.md) - moq-dev/moq and moq-dev/moq.pro bump their `quest` pin to pick up the routing

quest/m1/usage.md

@@ -40,8 +40,7 @@ Agent's choices, for the implementer to revisit:
   and branch: a quest branch names its quest, and a PR's head branch names
   the PR. Sessions on no quest or PR branch roll up as unattributed.
 - Accuracy comes from the records: commit trailers and the PR body's
-  size, implementers, and each `## Review` round's reviewer, score
-  ([review score](/quest/m0/model-routing/review-score.md)), and findings
+  size, implementers, and each `## Review` round's reviewer, score, and findings
   with their dispositions; follow-up fixes are later PRs that list it under
   `## Caused by` (`quest context`'s Pull requests section).
 - Prices are a pinned table beside the model lists, by model and token

quest/m2/council.md

@@ -22,8 +22,8 @@ a token cost the user chooses per PR:
   - Independent only
 - How are findings combined?
   - ✅ Merged, duplicates folded, each tagged with its agreement count
-    (user); every review also scores the change out of 10
-    ([review score](/quest/m0/model-routing/review-score.md))
+    (user); every review also scores the change out of 10, as
+    `quest review` already does
   - Merged, then a blind vote that drops findings most members reject
 - Who sits on it?
   - ✅ One reviewer per maker, minus the implementers' makers: Sol and
@@ -62,7 +62,6 @@ Agent's choices, for the implementer to revisit:
 ## Required
 
 - [Grok and opencode reviewers](/quest/m2/review-harnesses.md) - the council's xAI and DeepSeek members
-- [Review score](/quest/m0/model-routing/review-score.md) - the score each member reports
 
 ## Related
 

src/cli/main.ts

@@ -1010,7 +1010,9 @@ export function main(argv: string[], out: Output): number | Promise<number> {
 				"each directory holding a changed path, never the checkout's, so a change cannot instruct its own " +
 				"reviewer. The base is fetched first, since the sandbox has no network.\n\n" +
 				"Prints the reviewer, model, and effort, the size and where it came from, session, round, base, head, a usage line per turn (input tokens, the " +
-				"part read from the prompt cache, and output), the finding count, then each finding. --json prints " +
+				"part read from the prompt cache, and output), a score from 0 to 10 for how ready the head is to land with " +
+				"its one-sentence rationale, the finding count, then each finding. A reply without a valid score " +
+				"fails as any malformed reply does. --json prints " +
 				"the same as one object. Exits 0 once the round finishes, whatever it found, 2 for invalid arguments " +
 				"or a checkout or CLI that cannot be reviewed, including a login the API rejects, 3 when the reviewer's " +
 				"account hit its usage limit, and 1 for other failures. A usage limit prints one line, " +