From bfc00c758067d3a4201c1917818af902f68268a1 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 2 Nov 2026 19:44:14 -0800 Subject: [PATCH 1/3] chore(quest): claim quest/m0/queue-watcher/runners.md Quest-Claim: 7971fe2153d8fda6 From 131667fa37418725ca486a588eb23f102040eb8e Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 2 Nov 2026 20:40:13 -0800 Subject: [PATCH 2/3] quest: harden the queue runners: fork approval set, watch-mode cleanup in kixelated/nixos#20 Co-Authored-By: Claude Opus 5.5 --- quest/m0/queue-watcher/runners.md | 64 ++++++++++++++++++++++--------- 1 file changed, 46 insertions(+), 18 deletions(-) diff --git a/quest/m0/queue-watcher/runners.md b/quest/m0/queue-watcher/runners.md index 1b19950f..afe06473 100644 --- a/quest/m0/queue-watcher/runners.md +++ b/quest/m0/queue-watcher/runners.md @@ -4,17 +4,15 @@ The self-hosted `quest-queue`, `rind-queue` and `rust2-queue` runners that [kixelated/nixos#19](https://github.com/kixelated/nixos/pull/19) adds run only -their own repository's merge-queue workflow, and nothing else on the host +their own repository's merge-queue builds, and nothing else on the host reaches them: - kixelated/nixos's quest-land module creates no check launcher, users, networks, kvm access or storage tagging for a repository in watch mode. -- Each runner group admits only its repository's - `.github/workflows/queue.yml`. +- Each runner group admits only its repository's queue build. - moq-dev/quest, moq-dev/rind and moq-dev/rust2 require approval before running workflows from any outside contributor's fork PR, not only a - first-time contributor's (moq-dev/quest was `first_time_contributors` on - 2026-11-02). + first-time contributor's. ## Plan @@ -22,20 +20,50 @@ Decided 2026-11-02 (user, /quest-plan of the follow-ups from Claude's review of kixelated/nixos#19 in that day's `/quest-complete` session): ✅ all three items, m0, ranked right after the [cutover](/quest/m0/queue-watcher/follow.md). -- Watch-mode cleanup: the review found the base branch - ([kixelated/nixos#16](https://github.com/kixelated/nixos/pull/16)) already - creates check sockets and users for moq in watch mode, which only that - repository's daemon user can reach. Gate them on `quest land serve` mode. -- Runner groups: #19 puts each runner in its own group, limited to its one - repository. Limit each group to its repository's `queue.yml` too, through - the org's runner group settings (`restricted_to_workflows`). -- Fork approval: a returning contributor's fork PR runs its own copy of the - workflows, which can name a self-hosted label. Set each repository's - `fork-pr-contributor-approval` policy to all outside contributors. Do this - no later than the cutover registers the runners, if it's still open then. +Done in [moq-dev/quest#439](https://github.com/moq-dev/quest/pull/439): + +- Fork approval: on 2026-11-02 each repository's + `fork-pr-contributor-approval` policy went from `first_time_contributors` + to `all_external_contributors`. +- Watch-mode cleanup: + [kixelated/nixos#20](https://github.com/kixelated/nixos/pull/20), stacked on + #19, generates check launchers, users, networks, staging, KVM grants and + storage tagging only for repositories on `quest land serve`, and adds the + `quest-land-watch` flake check. It removes moq's, quest's, rind's and + rust2's check resources from the host once deployed. + +What remains: + +- Runner groups: limiting each group to `queue.yml` through + `restricted_to_workflows` can't work as decided. GitHub matches a selected + workflow by its exact ref and accepts no wildcard, and a merge group runs + `queue.yml` from its own `gh-readonly-queue/main/pr--` branch, so + an entry pinned to `refs/heads/main` would leave every merge-group job + waiting for a runner. See the Waiting decision. +- Deploy kixelated/nixos#20 with the rest of the stack. - Verify by checking that a fork PR's workflow waits for approval, and by - listing each group's allowed workflows. + listing each group's allowed workflows if the restriction goes ahead. + +## Waiting + +- Maintainer: decide how to restrict each runner group to its queue build. + (1) Move each repository's self-hosted job into a reusable workflow that + `queue.yml` calls at `@main`, runs only for `merge_group`, and restrict the + group to that reusable workflow at `refs/heads/main`. Then even an approved + fork run can't reach the runner, but a PR's own edit to that job is tested + only after it lands. (2) Leave the groups unrestricted and rely on the + one-repository groups plus fork approval for every outside contributor. + Recommended: (1), since fork approval is a click a maintainer can give by + mistake, while (1) holds for every run. Check it with + `gh api orgs/moq-dev/actions/runner-groups/` showing + `restricted_to_workflows: true` and a merge group still running on the + runner. +- Maintainer: deploy [kixelated/nixos#20](https://github.com/kixelated/nixos/pull/20) + through `deploy/quest-land`, since only the maintainer activates the live + host. Check that `systemctl list-units 'quest-land*check*'` shows only + `quest-land-moq-pro-check.socket`, and that `getfacl /dev/kvm` names no + check UID. ## Related -- [Watcher on the host](/quest/m0/queue-watcher/host.md) - the host module these runners and daemons share +- [Gate host](/quest/m0/host-land/host.md) - the host module these runners and daemons share From 3063c8dab0baf0c60afc62bc1440c25c4351cbe9 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 07:14:53 -0800 Subject: [PATCH 3/3] quest: move the quest queue build into a reusable workflow for its runner group Records the 2026-11-03 decision to restrict each queue runner group to a reusable build workflow at refs/heads/main, and takes its first step here and in moq-dev/rind#289 and moq-dev/rust2#536. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/queue-build.yml | 59 +++++++++++++++++++++++++++ .github/workflows/queue.yml | 61 +++++++++++----------------- README.md | 4 +- quest/m0/queue-watcher/runners.md | 66 ++++++++++++++++++++----------- 4 files changed, 127 insertions(+), 63 deletions(-) create mode 100644 .github/workflows/queue-build.yml diff --git a/.github/workflows/queue-build.yml b/.github/workflows/queue-build.yml new file mode 100644 index 00000000..12919098 --- /dev/null +++ b/.github/workflows/queue-build.yml @@ -0,0 +1,59 @@ +name: Queue build + +# The merge queue's build of a group, on the host's self-hosted runner. Only +# queue.yml calls it. Once queue.yml calls it at main and the quest-queue runner +# group admits only this workflow at refs/heads/main, no other workflow can +# reach the runner, an approved fork run's included; a PR's edit to this file +# then runs only after it lands. +on: + workflow_call: + +permissions: + contents: read + pull-requests: read + +jobs: + build: + name: Build + # A caller's own event decides this, so a pull request's workflow that + # calls this file still gets no runner. + if: github.event_name == 'merge_group' + runs-on: [self-hosted, quest-queue] + # Below the queue's 90-minute response window, so a hung build reports + # before the group expires. + timeout-minutes: 60 + env: + BASE: ${{ github.event.merge_group.base_sha }} + steps: + # Validate quest completion with the base's CLI, which the PR cannot weaken. + - name: Check out the base + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ github.event.merge_group.base_sha }} + persist-credentials: false + - name: Build quest from the base + run: nix build --out-link "$RUNNER_TEMP/quest-base" .#quest + - name: Check out the merge group + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + fetch-depth: 0 + - name: Check that the PR leaves its quest accurate + working-directory: ${{ runner.temp }} + env: + GH_TOKEN: ${{ github.token }} + REPO: ${{ github.repository }} + QUEUE_REF: ${{ github.event.merge_group.head_ref }} + run: | + # Build concurrency and group size are both one in main's ruleset. + [[ "$QUEUE_REF" =~ ^refs/heads/gh-readonly-queue/main/pr-([0-9]+)- ]] || { echo "::error::unexpected merge group ref" >&2; exit 1; } + head_ref=$(gh api "repos/$REPO/pulls/${BASH_REMATCH[1]}" --jq .head.ref) + ./quest-base/bin/quest --root "$GITHUB_WORKSPACE" check --base "$BASE" --branch "$head_ref" + - name: Install dependencies + run: nix develop --command just install + - name: Check + run: nix develop --command just check + - name: Test + run: nix develop --command just test + - name: Check Nix package + run: nix flake check diff --git a/.github/workflows/queue.yml b/.github/workflows/queue.yml index 1e53bbc1..18c4512f 100644 --- a/.github/workflows/queue.yml +++ b/.github/workflows/queue.yml @@ -40,50 +40,33 @@ concurrency: cancel-in-progress: true jobs: + # The build runs in queue-build.yml so its runner group can admit that + # workflow alone; see that file. This calls the merge group's own copy until + # that file is on main, when the call moves to + # moq-dev/quest/.github/workflows/queue-build.yml@main. + build: + name: Queue build + if: github.event_name == 'merge_group' + uses: ./.github/workflows/queue-build.yml + + # The required context. A reusable workflow's jobs report as + # `Queue build / Build`, so this job reports the build's result under the + # name main's ruleset requires. It skips with the build on pull requests, and + # runs even when the build fails or is cancelled, since a skipped context + # would pass the group. queue: name: Queue - if: github.event_name == 'merge_group' - runs-on: [self-hosted, quest-queue] - # Below the queue's 90-minute response window, so a hung build reports - # before the group expires. - timeout-minutes: 60 - env: - BASE: ${{ github.event.merge_group.base_sha }} + needs: build + if: always() && github.event_name == 'merge_group' + runs-on: ubuntu-latest + timeout-minutes: 5 steps: - # Validate quest completion with the base's CLI, which the PR cannot weaken. - - name: Check out the base - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - ref: ${{ github.event.merge_group.base_sha }} - persist-credentials: false - - name: Build quest from the base - run: nix build --out-link "$RUNNER_TEMP/quest-base" .#quest - - name: Check out the merge group - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - persist-credentials: false - fetch-depth: 0 - - name: Check that the PR leaves its quest accurate - working-directory: ${{ runner.temp }} + - name: Require the build to pass env: - GH_TOKEN: ${{ github.token }} - REPO: ${{ github.repository }} - QUEUE_REF: ${{ github.event.merge_group.head_ref }} - run: | - # Build concurrency and group size are both one in main's ruleset. - [[ "$QUEUE_REF" =~ ^refs/heads/gh-readonly-queue/main/pr-([0-9]+)- ]] || { echo "::error::unexpected merge group ref" >&2; exit 1; } - head_ref=$(gh api "repos/$REPO/pulls/${BASH_REMATCH[1]}" --jq .head.ref) - ./quest-base/bin/quest --root "$GITHUB_WORKSPACE" check --base "$BASE" --branch "$head_ref" - - name: Install dependencies - run: nix develop --command just install - - name: Check - run: nix develop --command just check - - name: Test - run: nix develop --command just test - - name: Check Nix package - run: nix flake check + RESULT: ${{ needs.build.result }} + run: '[[ "$RESULT" == success ]] || { echo "::error::the queue build ended $RESULT" >&2; exit 1; }' - # The merge queue still builds every group with the job above; this only + # The merge queue still builds every group with the jobs above; this only # satisfies the PR-side requirement, as a pull_request run's skipped job does. report: name: Report Queue diff --git a/README.md b/README.md index 31c9c53c..fdc8af48 100644 --- a/README.md +++ b/README.md @@ -524,8 +524,8 @@ The commands accept the same PR argument and authentication as and 1 for other failures. This repository lands through GitHub's merge queue. Its required `Queue` check -runs `just check` and `just test` on each merge group on the self-hosted runner; -it skips on pull requests. `quest land watch` runs the fixer on dropped PRs and +passes a merge group only when the group's build, `just check` and `just test` +on the self-hosted runner, passes; it skips on pull requests. `quest land watch` runs the fixer on dropped PRs and re-enqueues healthy heads at the back of the queue. ## Land a batch with quest land diff --git a/quest/m0/queue-watcher/runners.md b/quest/m0/queue-watcher/runners.md index afe06473..0d4b4b08 100644 --- a/quest/m0/queue-watcher/runners.md +++ b/quest/m0/queue-watcher/runners.md @@ -20,6 +20,20 @@ Decided 2026-11-02 (user, /quest-plan of the follow-ups from Claude's review of kixelated/nixos#19 in that day's `/quest-complete` session): ✅ all three items, m0, ranked right after the [cutover](/quest/m0/queue-watcher/follow.md). +Decided 2026-11-03 (user, /quest-complete grill round), since GitHub matches a +runner group's selected workflow by its exact ref, with no wildcard, and a +merge group runs `queue.yml` from its own `gh-readonly-queue/main/pr--` +branch: + +- How should each self-hosted runner group (quest-queue, rind-queue, + rust2-queue) be restricted to its queue build? + - ✅ (1) Reusable workflow (Recommended): move each repository's + self-hosted job into a reusable workflow that `queue.yml` calls at `@main` + and runs only for `merge_group`, then restrict the group to that workflow + at `refs/heads/main`. + - (2) Leave the groups unrestricted and rely on one-repository groups plus + fork approval. + Done in [moq-dev/quest#439](https://github.com/moq-dev/quest/pull/439): - Fork approval: on 2026-11-02 each repository's @@ -29,40 +43,48 @@ Done in [moq-dev/quest#439](https://github.com/moq-dev/quest/pull/439): [kixelated/nixos#20](https://github.com/kixelated/nixos/pull/20), stacked on #19, generates check launchers, users, networks, staging, KVM grants and storage tagging only for repositories on `quest land serve`, and adds the - `quest-land-watch` flake check. It removes moq's, quest's, rind's and - rust2's check resources from the host once deployed. + `quest-land-watch` flake check. +- Reusable build, first step: each repository's self-hosted job moved into + `.github/workflows/queue-build.yml`, which `queue.yml` calls by its local + path. The build job and its caller both run only for `merge_group`, and a + `Queue` job on a hosted runner reports the build's result under the context + main's ruleset requires. This is moq-dev/quest#439 itself, + [moq-dev/rind#289](https://github.com/moq-dev/rind/pull/289) and + [moq-dev/rust2#536](https://github.com/moq-dev/rust2/pull/536). What remains: -- Runner groups: limiting each group to `queue.yml` through - `restricted_to_workflows` can't work as decided. GitHub matches a selected - workflow by its exact ref and accepts no wildcard, and a merge group runs - `queue.yml` from its own `gh-readonly-queue/main/pr--` branch, so - an entry pinned to `refs/heads/main` would leave every merge-group job - waiting for a runner. See the Waiting decision. +- Once a repository's first step is on main, switch its `queue.yml` call to + `moq-dev//.github/workflows/queue-build.yml@main`. It can't be one + step: a merge group calling `@main` before the file is there finds no + workflow and fails. Say in that repository's contributing guide that a PR's + edit to `queue-build.yml` runs only after it lands. +- Then restrict each group (see Waiting). - Deploy kixelated/nixos#20 with the rest of the stack. -- Verify by checking that a fork PR's workflow waits for approval, and by - listing each group's allowed workflows if the restriction goes ahead. +- Verify that a fork PR's workflow waits for approval. ## Waiting -- Maintainer: decide how to restrict each runner group to its queue build. - (1) Move each repository's self-hosted job into a reusable workflow that - `queue.yml` calls at `@main`, runs only for `merge_group`, and restrict the - group to that reusable workflow at `refs/heads/main`. Then even an approved - fork run can't reach the runner, but a PR's own edit to that job is tested - only after it lands. (2) Leave the groups unrestricted and rely on the - one-repository groups plus fork approval for every outside contributor. - Recommended: (1), since fork approval is a click a maintainer can give by - mistake, while (1) holds for every run. Check it with - `gh api orgs/moq-dev/actions/runner-groups/` showing - `restricted_to_workflows: true` and a merge group still running on the - runner. +- Maintainer: once a repository's `queue.yml` calls `queue-build.yml@main` on + main, restrict its runner group to that workflow. Not before: the group + would then admit no merge-group job and stall the queue. For moq-dev/quest + (group 8; rind-queue is 9 and rust2-queue is 10, each with its own name and repository): + `gh api -X PATCH orgs/moq-dev/actions/runner-groups/8 -f name=quest-queue -F restricted_to_workflows=true -f 'selected_workflows[]=moq-dev/quest/.github/workflows/queue-build.yml@refs/heads/main'`. + Check that `gh api orgs/moq-dev/actions/runner-groups/8` shows + `restricted_to_workflows: true` with that one workflow, and that the next + merge group's `Queue build / Build` job still runs on the runner. GitHub + documents workflow restriction for Enterprise Cloud while moq-dev is on the + Team plan, though the API accepts the fields; if GitHub rejects or drops + them, revert with `-F restricted_to_workflows=false` and fall back to option + (2) above. - Maintainer: deploy [kixelated/nixos#20](https://github.com/kixelated/nixos/pull/20) through `deploy/quest-land`, since only the maintainer activates the live host. Check that `systemctl list-units 'quest-land*check*'` shows only `quest-land-moq-pro-check.socket`, and that `getfacl /dev/kvm` names no check UID. +- Maintainer: the next time an outside contributor opens a fork PR on + moq-dev/quest, moq-dev/rind or moq-dev/rust2, check that its workflow runs + wait for approval. ## Related