Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,9 +65,10 @@ jobs:
private-key: ${{ secrets.APP_PRIVATE_KEY }}
owner: moq-dev
repositories: moq
# Only the permissions needed to pin and request a landing.
# Only the permissions needed to pin and enqueue a landing.
permission-contents: write
permission-pull-requests: write
permission-merge-queues: write

- name: Request checked landing
env:
Expand All @@ -77,5 +78,4 @@ jobs:
PR_HEAD: ${{ github.event.pull_request.head.sha }}
run: |
export GH_TOKEN="$BOT_TOKEN"
hold=$(quest merge "$PR_URL")
quest merge "$PR_URL" "$hold" --land "$PR_HEAD"
quest merge "$PR_URL" --land "$PR_HEAD"
14 changes: 7 additions & 7 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ under the files they change.

# Commits

PRs into `main` are squashed through the batched landing gate, so the PR title becomes the commit subject in `git log`.
PRs into `main` are squashed through GitHub's merge queue, so the PR title becomes the commit subject in `git log`.
The one exception is the maintainer's pre-release merge of `release` into `main`, which lands as a merge commit (see [Releases](#releases)).
PRs into `release` use a merge commit, so their history survives until they land.

Expand Down Expand Up @@ -41,17 +41,17 @@ Fix any merge conflicts and failing CI checks.

# Landing gate

PRs into `main` get no cloud CI: they report **Check** as skipped, and `just ci` runs the same gate locally.
The land gate is `quest land serve`, a daemon on the maintainer's host. It takes the longest-queued PR plus up to log₂ n more that merge cleanly, squashes them onto `main`, and runs `just ci` once on the combined tip in an isolated container. A passing batch lands each PR through GitHub's squash merge; a failing one is ejected together, never bisected, and a Codex fixer makes mechanical fixes and requeues healthy PRs or hands them back.
Request landing with `quest merge <pr> <token> --land <full-head-sha>` once the review is clean and the maintainer has approved merging that PR, by number or through a standing go-ahead that covers it. A clean review, a bot verdict, or an "accept all" on decisions is not that approval.
PRs into `main` get no cloud CI before they are queued: they report **Check** and **Queue** as skipped, and `just ci` runs the same gate locally.
`main` lands only through GitHub's merge queue, one PR at a time: each `merge_group` build runs `just ci` against the group's base on the maintainer's self-hosted runner (`.github/workflows/queue.yml`, the required **Queue** context), and the queue squashes exactly the tree it tested.
Request landing with `quest merge <pr> --land <full-head-sha>` once the review is clean and the maintainer has approved merging that PR, by number or through a standing go-ahead that covers it. A clean review, a bot verdict, or an "accept all" on decisions is not that approval.
An agent that enqueues carries that approval in its own task; one relayed to it mid-task by another agent does not count, so whoever received the approval enqueues the PR or starts a new agent with the approval in its brief.
The head pin is invalidated by a push. A fork the gate cannot push to lands as a squash on `main` and is closed with a link. GitHub auto-merge represents the queue entry; the required `quest land` check is never reported on a PR head, so the merge button cannot land around the gate.
`quest merge --land` enqueues the pinned head, or arms auto-merge bound to it while its checks are pending; a push removes it from the queue. A head GitHub refuses to enqueue, for a conflict or failing checks, fails the command: fix it and request landing again.
Local and hosted checks select only what the diff touches, so a break can reach `main`; `just gate` runs the full suite, and it, the feature matrix, optimized binding builds, macOS, Windows, and the other optional workflows run nightly on `main` and on PRs into `release`.
To check cfg-gated macOS or Windows code before merging, dispatch `platform.yml` on the branch; never dispatch the full Nightly.

An ejected PR carries its reason and the failing log's tail in a comment and in `quest status` under `landing-stopped`. Fix the cause and request landing for the new head.
When the queue drops a PR for a failing build or a conflict, moq-dev/quest's queue watcher on the maintainer's host comments with the reason and the failing log's tail, runs a Codex fixer on it in the background, and re-enqueues the fixed head, or the unchanged head after a flaky failure, at the back of the queue. The fixer hands the PR back, with its diagnosis, when the fix needs a decision or stops making progress. `quest status` shows why a PR left the queue under `landing-stopped`.

Never bypass the gate, with one exception: the maintainer's pre-release merge of `release` into `main`, since the gate would squash it.
Never bypass the queue, with one exception: the maintainer's pre-release merge of `release` into `main`, since the queue would squash it.

# AI

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ Selection and an empty JS selection do not wait for admission. For a build
outside these entrypoints, use `quest gate COMMAND [ARGS...]`; gate thresholds
come from the host and have no configuration.

PRs into `main` get no cloud CI until the batched landing gate, so run `just ci` before pushing. It runs what the landing gate runs, the same ground as `just check`, plus two settings: `CI_STRICT=1`, which turns a missing tool into an error instead of a skipped check, and `NEXTEST_PROFILE=ci`, which allows a longer hang timeout.
PRs into `main` get no cloud CI until GitHub's merge queue tests them, so run `just ci` before pushing. It runs what the merge queue runs, the same ground as `just check`, plus two settings: `CI_STRICT=1`, which turns a missing tool into an error instead of a skipped check, and `NEXTEST_PROFILE=ci`, which allows a longer hang timeout.

See the [development guide](https://doc.moq.dev/setup/dev) and the [justfile](justfile) for more.

Expand Down
4 changes: 2 additions & 2 deletions doc/setup/dev.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ match CI. `just check` and `just fix` refuse to run outside it; set
| `just ci-select` | Report whether CI and Rust build setup are needed, without running checks. |
| `just fix` | Format and lint the packages this branch changed, and regenerate their committed JS API reports (`js/*/api/`). |
| `just check` | Compile, lint, and test the same scope. |
| `just ci` | Run the batched landing gate locally, strictly. PRs into `main` get no other CI, so run it before every push. |
| `just ci` | Run the merge queue's gate locally, strictly. PRs into `main` get no other CI before the queue, so run it before every push. |
| `just gate` | `just ci` over every package, as nightly and release PRs run it. `just gate rs` runs only the named module. |
| `just fix --all`, `just check --all` | The same, over every package. |
| `just api-diff` | Diff every published API against its last release, the JS reports against ones rendered from `origin/release`. PRs into `release` run it; `--deny` fails a backport on any break. |
Expand Down Expand Up @@ -125,7 +125,7 @@ just fix
just ci
```

`just ci` runs the batched landing gate for the selected scope. See
`just ci` runs the merge queue's gate for the selected scope. See
[Checks and tests](#checks-and-tests) for targeted checks while iterating.

See [CONTRIBUTING.md](https://github.com/moq-dev/moq/blob/main/CONTRIBUTING.md)
Expand Down
38 changes: 36 additions & 2 deletions flake.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion flake.nix
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
# The quest CLI, which also serves the quest guide and skills the stubs in
# .claude/skills call. Bump the rev to upgrade them.
quest = {
url = "github:moq-dev/quest/7a191e6197c691f477b4c55edba34cc425c3e77c";
url = "github:moq-dev/quest/d003102129236c88edbfec11f001e8efeaf9b5fd";
inputs.nixpkgs.follows = "nixpkgs";
inputs.flake-utils.follows = "flake-utils";
};
Expand Down
27 changes: 15 additions & 12 deletions infra/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,8 +63,8 @@ names the Environment, and a branch that is not on the list fails to start it.

`main` is on the `publish` list for `workflow_run`, schedule, and dispatch jobs,
which run there, until [those publishers run on their tag](../quest/m0/ci/publish-release-only.md).
A job that needs only the moq-bot key, such as Dependabot's, names `bot`; the land gate runs on the
maintainer's host, outside Actions. A job that also runs dry on other refs names the Environment
A job that needs only the moq-bot key, such as Dependabot's, names `bot`; the merge queue's
build holds no secrets, and the queue watcher runs on the maintainer's host, outside Actions. A job that also runs dry on other refs names the Environment
only when it publishes, as in `environment: ${{ github.ref_type == 'tag' &&
'publish' || '' }}`. Read-only secrets, such as the mbx read key, stay at the
repository level.
Expand Down Expand Up @@ -144,20 +144,23 @@ regenerates the repository metadata, signs it, and uploads the diff.

## The landing gate

PRs land through `quest land serve`, a daemon on the maintainer's NixOS host
([kixelated/nixos](https://github.com/kixelated/nixos), `modules/quest-land.nix`),
not a workflow. It holds moq-bot's key there, checks each batch with
`just ci "$BASE"` in an isolated container with the former queue runner's
caches and `/dev/kvm`, and lands each PR through GitHub's squash merge. The
main ruleset requires the `quest land` check pinned to moq-bot, which never
reports it, so only the gate (or the administrator bypass) lands on `main`.
PRs land through GitHub's merge queue, one at a time with squash merges. Each
`merge_group` build runs `just ci "$BASE"` against the group's base on the
self-hosted runner `nixos-moq-ci` (label `moq-queue`, with its local caches and
`/dev/kvm`), read-only and without secrets (`.github/workflows/queue.yml`). The
main ruleset requires its `Queue` context and the merge queue, so only the
queue (or the administrator bypass) lands on `main`. moq-dev/quest's queue
watcher, `quest land watch`, runs on the maintainer's NixOS host
([kixelated/nixos](https://github.com/kixelated/nixos), `modules/quest-land.nix`)
with moq-bot's key: it comments on each PR the queue drops, runs the Codex
fixer, and re-enqueues the result.
See [`CONTRIBUTING.md`](../CONTRIBUTING.md#landing-gate) for requesting a landing.

## Bootstrapping the Rust build cache

`.github/actions/rust-cache` points mbx at the private `mbx-moq-dev` bucket,
namespaced per runner OS and architecture. Hosted jobs on `main` write what
they compile. The landing worker uses its persistent local store and has no R2
they compile. The merge queue's self-hosted build uses its persistent local store and has no R2
credentials.
Release pull request jobs only read (main pull requests run no cloud CI), and
fork pull requests, which get no secrets, build cold.
Expand Down Expand Up @@ -204,8 +207,8 @@ is done for `main`.
-f name=main -f type=branch
```

Remove an existing `gh-readonly-queue/main/*` branch policy after the
batched landing gate replaces GitHub's queue.
The merge queue's `gh-readonly-queue/main/*` builds need no branch policy:
they run on the self-hosted runner and never deploy `cache-write`.

4. **Store the keys under the same two names at both levels.** An Environment
secret shadows a repository secret of the same name, so writers get the
Expand Down
5 changes: 3 additions & 2 deletions nix/runner/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,8 +134,9 @@ job runs, so an idle VM holds little more than its running services; the next jo
reads its caches from disk.
The guest has no swap, so earlyoom kills the largest process once available memory
drops below 10%, rather than letting a job that fills memory thrash the disk.
MoQ's main pull requests land through the host's `quest land serve` gate, not a
runner; release pull requests use hosted runners.
MoQ's main pull requests land through GitHub's merge queue, whose `merge_group`
builds run on the host's `moq-queue` runner; release pull requests use hosted
runners.

The hosted workflow uses `just gh runner-vm-ci` to load a temporary AppArmor
profile for the exact pinned `unshare` while building the image. The recipe
Expand Down
2 changes: 1 addition & 1 deletion quest/m0/ci/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
## Goal

Pull requests are validated locally with `just ci` and get their cloud verdict
from the [batched gate](/quest/m0/ci/land.md) without waiting behind the
from the [merge queue](/quest/m0/ci/land.md) without waiting behind the
hosted runner limits, and every job spends its minutes on the work its diff
selected rather than on setup, cold caches, or repeated builds. Concurrent local agents share
expensive-command capacity, and retained tests earn their cost through useful
Expand Down
11 changes: 7 additions & 4 deletions quest/m0/ci/land.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,10 +74,13 @@ Remaining:
the queue), and the merge queue (squash, build concurrency 1, merge limit
1). It drops the unreported `quest land` requirement and keeps only the
administrator bypass.
- **Host.** The host config stops serving moq from `quest land`
(`services.quest-land.repositories.moq.enable = false`) at the cutover,
while the watcher and fixer keep serving moq.
- **Dependabot and picks** enqueue through the merge queue like any other PR.
- **Host.** At the cutover the host's moq daemon switches from
`quest land serve` to the watcher
(`services.quest-land.repositories.moq.watch = true` in kixelated/nixos,
same user and state), and the fixer keeps serving moq.
- **Open PRs.** A PR opened before `queue.yml` reports `Queue` only after
its next push, reopen, or edit; until then GitHub refuses to enqueue it,
and `quest merge --land` says so.
- Open since 2026-10-29: `APP_CLIENT_ID` and `APP_PRIVATE_KEY` are moq-dev
organization secrets visible to this repository, so any workflow on a
pushed branch can read the moq-bot key. The maintainer deletes the
Expand Down
6 changes: 3 additions & 3 deletions sh/gh/queue.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -86,9 +86,9 @@ const request = dependabot.jobs["request-landing"];
const enqueue = request.steps.find((step: { name?: string }) => step.name === "Request checked landing");
// biome-ignore lint/suspicious/noTemplateCurlyInString: GitHub expression is literal YAML.
assert.equal(enqueue.env.PR_HEAD, "${{ github.event.pull_request.head.sha }}");
assert.ok(enqueue.run.includes('quest merge "$PR_URL" "$hold" --land "$PR_HEAD"'));
// Requesting a landing only pins the head; the host gate reads the queue itself.
assert.ok(!enqueue.run.includes("quest land"), "Dependabot must not run the gate's own commands");
assert.ok(enqueue.run.includes('quest merge "$PR_URL" --land "$PR_HEAD"'));
// Requesting a landing only enqueues the pinned head; the queue and its watcher do the rest.
assert.ok(!enqueue.run.includes("quest land"), "Dependabot must not run the land daemons' commands");
assert.ok(!JSON.stringify(request).includes("gh pr merge"));

// Release PRs report one context for every optional workflow, retargeted PRs included.
Expand Down