feat(land): the watcher writes flaky tests' follow-up quests

README.md

@@ -932,13 +932,15 @@ requeue is a verdict: a failure on the base or from the runner never counts
 toward disabling a test. Each failed
 build's round is told the tests that flaked before and are not yet disabled, in
 `.git/quest-land-flaky.json` in its checkout, which `quest land fix` passes to
-the agent. One verdict alone only re-enqueues the PR, since one judgment could
+the agent, with the follow-up quest path the watcher chose for each when it
+first flaked. One verdict alone only re-enqueues the PR, since one judgment could
 hide a real bug. When a listed test flakes again, the fixer also commits its
-disabling on a `disable` branch from the base: the skip in the test
-framework's own form, naming the follow-up quest it adds to the repository's
-m0 `flaky` epic, `quest/m0/flaky/<name>.md`, to fix or delete the test. The
-watcher validates that commit as it does a base fix, requiring exactly one new
-follow-up quest, pushes it to `quest-land/flaky-<sha>`, opens it as a pull
+disabling on a `disable` branch from the base: only the skip in the test
+framework's own form, naming that follow-up quest. The watcher validates that
+commit as it does a base fix, refusing any edit under `quest/`, then commits
+the follow-up quest on top from a fixed template, `quest/m0/flaky/<name>.md`
+in the repository's m0 `flaky` epic, to fix or delete the test, creating the
+epic's README when the base has none. It pushes the result to `quest-land/flaky-<sha>`, opens it as a pull
 request with its Review recorded, and enqueues it like any other, with no
 maintainer's pick. A refused disable waits for the test's next flake, and a
 pull request GitHub refuses to enqueue says so and waits for a maintainer.

assets/land-fix.md

@@ -157,24 +157,17 @@ A failure on the base or from the runner is not flaky: name its cause, never
 `disable`. A PR requeued only because another supplied PR caused the failure takes
 no `cause`. When you cannot tell why the failure happened, hand the PR back.
 The input may list the repository's known flaky tests, each by the
-`failure` an earlier round named and the `merge_group` jobs that showed it.
+`failure` an earlier round named, the `merge_group` jobs that showed it, and
+the `quest` path of the follow-up quest that tracks it once disabled.
 When the test you judge flaky is one of them, name it exactly as listed, and
 disable it in a commit of its own: create a `disable` branch from `base`, and
-commit on it only
-
-- the skip, in the test framework's own form, whose reason names the follow-up
-  quest, such as `#[ignore = "flaky: /quest/m0/flaky/<name>.md"]` in Rust,
-  `test.skip` with that path in a comment in JavaScript, or
-  `pytest.mark.skip(reason="flaky: /quest/m0/flaky/<name>.md")` in Python;
-- that follow-up quest, `quest/m0/flaky/<name>.md`, with `<name>` the test's
-  name in lowercase kebab-case, whose Goal says to fix the test or delete it
-  and links each listed job and this failure's, which the failure log names;
-- its link in the `Required` list of `quest/m0/flaky/README.md`, the epic
-  holding one quest per disabled test, created with a link in
-  `quest/m0/README.md`'s `Required` if it is missing.
-
-Follow the repository's quest format, and run `quest check` when the
-repository has it. A disabled test that is another quest's acceptance test
+commit on it only the skip, in the test framework's own form, whose reason
+names that `quest` path, such as `#[ignore = "flaky: /quest/m0/flaky/<name>.md"]`
+in Rust, `test.skip` with that path in a comment in JavaScript, or
+`pytest.mark.skip(reason="flaky: /quest/m0/flaky/<name>.md")` in Python.
+Never edit `quest/`: quest land writes the follow-up quest itself.
+
+A disabled test that is another quest's acceptance test
 leaves that quest as it is: the test still counts as its work. Run the
 [repository checks](#repository-checks) on the branch, and add its full SHA to the
 PR's entry:

quest/m0/queue-watcher/README.md

@@ -150,7 +150,6 @@ Shared shape, for the children to follow:
 
 - [The other repositories follow](/quest/m0/queue-watcher/follow.md) - quest, rind and rust2 are cut over and landing; waits for a fork PR to land through one of their queues
 - [Harden the queue runners](/quest/m0/queue-watcher/runners.md) - each queue runner runs only its repository's queue workflow, and every outside fork PR needs approval
-- [The watcher writes flaky quests](/quest/m0/queue-watcher/flaky-quest.md) - a disable's follow-up quest comes from a template in `quest land watch`, so the headless land fixer writes no quest
 - [Daemons prune their logs](/quest/m0/queue-watcher/log-retention.md) - the watcher and gate delete their finished logs after a window, so state stays bounded (4.1 GiB on nixos on 2026-11-04)
 - [The watcher runs its own serial queue](/quest/m0/queue-watcher/own-queue.md) - `quest land watch` lands a base without GitHub's merge queue, testing one pick at a time in the repository's queue workflow
 - [moq.pro lands main through the own queue](/quest/m0/queue-watcher/pro-cutover.md) - moq.pro's `main` leaves the batching gate for the watcher's own queue

quest/m0/token-waste/child-lists.md

@@ -36,9 +36,6 @@ This quest is the second PR:
   no history to read. #518's commit caa32fc3 has both, with tests.
 - Run `quest check --fix` on this tree and on prose that describes the lists.
 - Turn `requires X, which this epic holds` into a `quest check` error.
-- `assets/land-fix.md` stops adding the flaky quest's link to
-  `quest/m0/flaky/README.md`, and
-  [the flaky quest](/quest/m0/queue-watcher/flaky-quest.md) says so.
 - The PR is breaking: other repositories migrate with `quest check --fix` on
   their pin bump past it, as
   [moq-dev pins](/quest/m0/model-routing/pins.md) describes.

src/cli/land-fix.ts

@@ -23,6 +23,8 @@ export const FLAKY_INPUT = "quest-land-flaky.json";
 /** A test the fixer judged flaky before, by the `failure` it named, with the `merge_group` jobs that showed it. */
 export interface KnownFlaky {
 	failure: string;
+	/** The follow-up quest quest land watch adds when it is disabled, for the skip's reason to name. */
+	quest: string;
 	jobs: string[];
 }
 
@@ -53,6 +55,7 @@ function knownFlaky(root: string): KnownFlaky[] {
 			(item) =>
 				item &&
 				typeof item.failure === "string" &&
+				typeof item.quest === "string" &&
 				Array.isArray(item.jobs) &&
 				item.jobs.every((job: unknown) => typeof job === "string"),
 		)

src/cli/land-watch-flaky.ts

@@ -1,8 +1,8 @@
 // The merge queue watcher's record of flaky tests: each flaky verdict the fixer gives, by the test it
 // names, and the commit that disables a test seen flaking twice, which the watcher opens as a PR.
-import { existsSync, readFileSync, writeFileSync } from "node:fs";
-import { join } from "node:path";
-import { git } from "./land";
+import { existsSync, readFileSync, rmSync, writeFileSync } from "node:fs";
+import { join, resolve } from "node:path";
+import { git, tryGit } from "./land";
 import { FLAKY_INPUT, type FixResult, type KnownFlaky, signature } from "./land-fix";
 import { validateFix } from "./land-fixer";
 import { importFixer } from "./land-fixer-repository";
@@ -38,6 +38,8 @@ export interface Disable {
 /** A test the fixer judged flaky, by the `failure` it named. */
 export interface FlakyTest {
 	failure: string;
+	/** Its follow-up quest, `quest/m0/flaky/<name>.md`, chosen at its first verdict for the skip to name. */
+	quest?: string;
 	verdicts: FlakyVerdict[];
 	disable?: Disable;
 }
@@ -53,6 +55,76 @@ export const RECENT = 10;
 const KNOWN_BYTES = 50_000;
 /** The follow-up quest a disable adds, one per test, in the repository's m0 `flaky` epic. */
 const QUEST = /^quest\/m0\/flaky\/([a-z0-9][a-z0-9._-]*)\.md$/;
+const EPIC = "quest/m0/flaky/README.md";
+/** The longest follow-up quest name, cut at a word. */
+const NAME_LIMIT = 60;
+const BOT = {
+	GIT_AUTHOR_NAME: "moq-bot",
+	GIT_AUTHOR_EMAIL: "moq-bot@users.noreply.github.com",
+	GIT_COMMITTER_NAME: "moq-bot",
+	GIT_COMMITTER_EMAIL: "moq-bot@users.noreply.github.com",
+};
+
+/**
+ * The follow-up quest's name for `failure`: its last segment after a `:` or `>`, the test itself, in
+ * lowercase kebab-case.
+ */
+export function flakyName(failure: string): string {
+	const segment =
+		signature(failure)
+			.split(/[:>]/)
+			.map((part) => part.trim())
+			.filter(Boolean)
+			.pop() ?? "";
+	let name = segment
+		.toLowerCase()
+		.replace(/[^a-z0-9]+/g, "-")
+		.replace(/^-+|-+$/g, "");
+	if (name.length > NAME_LIMIT) {
+		name = name.slice(0, NAME_LIMIT + 1);
+		const cut = name.lastIndexOf("-");
+		name = (cut > 0 ? name.slice(0, cut) : name.slice(0, NAME_LIMIT)).replace(/-+$/, "");
+	}
+	return name || "test";
+}
+
+/** A follow-up quest path for `failure` that neither another test nor the base at `base` holds. */
+function questFor(root: string, base: string, tests: FlakyTest[], failure: string): string {
+	const taken = new Set(tests.map((test) => test.quest ?? test.disable?.quest));
+	const name = flakyName(failure);
+	for (let index = 1; ; index++) {
+		const path = `quest/m0/flaky/${index === 1 ? name : `${name}-${index}`}.md`;
+		if (!taken.has(path) && tryGit(root, ["cat-file", "-e", `${base}:${path}`]).status !== 0) return path;
+	}
+}
+
+/** GitHub's own job URLs, the only links a quest takes from the fixer's round. */
+const GITHUB_URL = /^https:\/\/github\.com\/[^\s()<>[\]]+$/;
+
+/** The follow-up quest a disable adds: a fixed template, sized `[S]` for a coordinator to correct. */
+export function flakyQuest(failure: string, jobs: string[]): string {
+	const name = failure.replace(/`/g, "").replace(/\s+/g, " ").trim();
+	const links = [...new Set(jobs)]
+		.filter((url) => GITHUB_URL.test(url))
+		.map((url, index) => `- [Job ${index + 1}](${url})`);
+	return `# [S] Fix the flaky test ${flakyName(failure)}
+
+## Goal
+
+Fix \`${name}\` so it passes reliably, or delete it, and remove its skip.
+quest land watch disabled it after the land fixer judged it flaky in
+more than one \`merge_group\` build.${links.length ? `\n\nFailing jobs:\n\n${links.join("\n")}` : ""}
+`;
+}
+
+/** The flaky epic's README, written when the base keeps none. */
+export const FLAKY_EPIC = `# Fix disabled flaky tests
+
+## Goal
+
+Each disabled flaky test is fixed or deleted, with its skip removed when
+fixed. Each quest here records the failures that led to its skip.
+`;
 
 const flakyPath = (state: string) => join(state, "flaky.json");
 
@@ -77,7 +149,12 @@ export function writeKnownFlaky(state: string, checkout: string): void {
 	for (const test of readFlaky(state)
 		.filter((test) => test.disable === undefined && test.verdicts.length > 0)
 		.sort((a, b) => latest(b).localeCompare(latest(a)))) {
-		const item = { failure: test.failure, jobs: test.verdicts.slice(-RECENT).flatMap((verdict) => verdict.jobs) };
+		if (test.quest === undefined) continue;
+		const item = {
+			failure: test.failure,
+			quest: test.quest,
+			jobs: test.verdicts.slice(-RECENT).flatMap((verdict) => verdict.jobs),
+		};
 		bytes += Buffer.byteLength(JSON.stringify(item)) + 1;
 		if (bytes > KNOWN_BYTES) break;
 		known.push(item);
@@ -112,6 +189,7 @@ export function recordFlaky(root: string, state: string, round: FlakyRound): str
 		test = { failure: round.failure, verdicts: [] };
 		tests.push(test);
 	}
+	test.quest ??= questFor(root, round.base.sha, tests, round.failure);
 	test.verdicts = [
 		...test.verdicts.filter((verdict) => verdict.job !== round.job),
 		{ job: round.job, number: round.number, at: round.dropped, jobs: round.jobs },
@@ -122,9 +200,20 @@ export function recordFlaky(root: string, state: string, round: FlakyRound): str
 		if (test.verdicts.length < FLAKY_VERDICTS) output.push(`disable ignored ${round.number}: first flaky verdict`);
 		else
 			try {
-				const quest = disabling(root, state, round, head);
-				test.disable = { head, base: round.base, quest, attribution: round.attribution };
-				output.push(`disable ${round.number} ${head}`);
+				const quest = test.quest;
+				const added = disabling(
+					root,
+					state,
+					round,
+					head,
+					quest,
+					flakyQuest(
+						test.failure,
+						test.verdicts.flatMap((verdict) => verdict.jobs),
+					),
+				);
+				test.disable = { head: added, base: round.base, quest, attribution: round.attribution };
+				output.push(`disable ${round.number} ${added}`);
 			} catch (error) {
 				// A later flake of the same test tries again.
 				output.push(`disable refused ${round.number}: ${String(error)}`);
@@ -136,16 +225,35 @@ export function recordFlaky(root: string, state: string, round: FlakyRound): str
 
 /**
  * Import the fixer's `disable` branch at `head` and check it: new commits on the base it was given,
- * leaving every protected path as the base has it, that add exactly one follow-up quest. Returns its path.
+ * leaving every protected path and the quest tree as the base has them. Then commit the follow-up
+ * `quest` on top, with the flaky epic's README when the base keeps none. Returns that commit.
  */
-function disabling(root: string, state: string, round: FlakyRound, head: string): string {
+function disabling(root: string, state: string, round: FlakyRound, head: string, quest: string, body: string): string {
 	importFixer(root, state, round.checkout, "disable", head, round.packs);
 	validateFix(root, { head: round.base.sha }, head, round.base.sha);
-	const added = git(root, ["diff", "--name-only", "--no-renames", "--diff-filter=A", round.base.sha, head, "--"])
-		.split("\n")
-		.filter((path) => QUEST.test(path) && path !== "quest/m0/flaky/README.md");
-	if (added.length !== 1) throw new Error("a disable must add exactly one follow-up quest in quest/m0/flaky/");
-	return added[0];
+	const quests = git(root, ["diff", "--name-only", "--no-renames", round.base.sha, head, "--", "quest"]);
+	if (quests) throw new Error("a disable must leave quest/ as the base has it; quest land watch adds its quest");
+	if (tryGit(root, ["cat-file", "-e", `${head}:quest/m0/README.md`]).status !== 0)
+		throw new Error("the base keeps no quest/m0/ for the follow-up quest");
+	if (tryGit(root, ["cat-file", "-e", `${head}:${quest}`]).status === 0) throw new Error(`${quest} already exists`);
+	const index = resolve(root, git(root, ["rev-parse", "--git-path", `quest-land-flaky-${process.pid}.index`]));
+	const env = { GIT_INDEX_FILE: index };
+	try {
+		git(root, ["read-tree", head], { env });
+		const add = (path: string, text: string) => {
+			const blob = git(root, ["hash-object", "-w", "--stdin"], { input: text });
+			git(root, ["update-index", "--add", "--cacheinfo", `100644,${blob},${path}`], { env });
+		};
+		add(quest, body);
+		if (tryGit(root, ["cat-file", "-e", `${head}:${EPIC}`]).status !== 0) add(EPIC, FLAKY_EPIC);
+		const tree = git(root, ["write-tree"], { env });
+		return git(root, ["commit-tree", tree, "-p", head], {
+			input: `quest: add ${quest} for the disabled flaky test\n\nquest land watch writes the follow-up quest for the test the land fixer skipped.\n`,
+			env: { ...BOT },
+		});
+	} finally {
+		rmSync(index, { force: true });
+	}
 }
 
 /** The follow-up quest's name, as a disable's branch and title name it. */