Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/fork-ci-tests.txt
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ fork-ci/actions/docs-scope/docs-scope.test.py
fork-ci/actions/gradle-cache/gradle_cache.test.py
fork-ci/actions/job-changed/job-changed.test.py
fork-ci/actions/pr-queue/lens.test.py
fork-ci/actions/pr-queue/pr_queue.test.py

# Upstream's tests, run where fork-tests.yml follows upstream's ci.yml.
.github/scripts/check-nightly-release.test.cjs run by .github/workflows/fork-tests.yml
Expand Down
74 changes: 72 additions & 2 deletions fork-ci/actions/pr-queue/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ only for a head that holds exactly the change the review read.

This folder is built in four stacked pull requests:

1. the review rules and the config (`lens.py`, `config.py`): this part;
2. the queue itself (`pr_queue.py`);
1. the review rules and the config (`lens.py`, `config.py`);
2. the queue itself (`pr_queue.py`, section "The queue"): this part;
3. armed pull requests that cannot merge (a red required check, or one that
ended without passing or never reported) are told once and passed over;
4. the "red only because the base was red" rule, the reusable workflow
Expand Down Expand Up @@ -92,6 +92,68 @@ heading near the top, or two `covered:`/`checks:` lines. A comment that is no
report says so on its first line, `not a lens: ...`. That clears it unless one of
its lines is a lens header or a `lens:` field with a sha.

## The queue

```
pr_queue.py --repo OWNER/NAME --config FILE --state-file ABS --cache ABS [--lock-file ABS] [--dry-run]
```

One run looks at the open, non-draft pull requests with auto-merge on, oldest
first, and acts on the first one it can:

- up to date with its base: it holds the queue while its required checks run;
- behind its base: auto-merge off, `gh pr update-branch`, and auto-merge back
on for the new head only if that head is exactly the change the review read
merged with the base. Anything else leaves it off with a comment that names
the way out.

A pull request in a merge state the queue neither updates nor waits for (a
conflict, `DIRTY`) is passed over, and the run says so.

`gh` must be logged in for the repository (`GH_TOKEN`). Git fetches into the
queue's own bare repository (`--cache`) and gets its login from
`gh auth git-credential` only, never from a helper the machine configured.

The token is in git's environment and `STATE` decides what is re-armed, so run
the queue on a runner (or runner group) that runs no `pull_request` job and no
other repository's job, as a user of its own. The `--state-file` and `--cache`
folders belong to that user alone. git runs with no hooks, no fsmonitor and no
replace refs, and a cache that is not that user's alone (mode `0700`), has a
config the queue did not write, or holds a file that changes what its objects
say (`info/grafts`, `shallow`, `objects/info/alternates` or
`objects/info/http-alternates`) is rebuilt, never used. A job of the same user
can still change the cache between that check and git's next read: only the
runner rule above stops that.

| Exit | Meaning |
| ---- | ----------------------------------------------------------------------------------------------------------- |
| 0 | Done, or nothing to do (also: another run holds the lock). |
| 1 | A step failed; the next run tries again (a plan in `STATE` first). |
| 2 | Refused to start; touched nothing. Every reason is printed at once. |
| 3 | A pull request may have auto-merge off and no comment says so yet. `STATE` stays; the next run finishes it. |

It refuses to start on a GitHub-hosted runner (its disk, and with it the plan,
goes with the job), with a `--repo` that is not `owner/name`, a relative path,
a path inside the job's workspace (the next checkout empties it), a config
error, or a git older than 2.38 (`merge-tree --write-tree`).

`--state-file` (`STATE`) holds the plan of the run that turned auto-merge off:
`{"repo", "pr", "old", "base", "read"}` (and `"told"` once the pull request was
told the run could not finish yet). It is written to disk (mode `0600`) before
the disarm and removed when the pull request is re-armed or told why not. The
next run finishes a plan first, against the review and the base as GitHub shows
them then (a lens posted since counts; an edited one reads nothing), and
touches no other pull request until it has. A plan
whose `repo` is another repository ends the run with exit 1 and stays as it
was: give each queue its own `--state-file`. A plan without `repo` (the source
queue's format) is this repository's. An unreadable plan, or one another user
could have written (not this user's, or writable by its group or others), is
kept as `STATE.bad`, the run fails, and every later run warns while that file
exists.

`--lock-file` (default: `STATE.lock`) keeps two runs from overlapping.
`--dry-run` takes no lock and writes nothing: no `STATE`, no cache, no comment.

## Where the rules come from

`lens.py` is a port of a production queue's review rules, behaviour for
Expand All @@ -108,3 +170,11 @@ comments carry no author association: the replay gives them `OWNER` and
unedited. A read shorter than 40 hex, and a lens comment edited after posting,
read nothing here (54 recorded answers). The source's `full_sha`, which matched
a short read to the pull request's commits, is gone.

`pr_queue.py` is a port of the same project's queue, and `pr_queue.test.py`
holds that project's queue tests with the port's names. What the port changed:
everything project-specific comes from the config; `STATE` names its
repository; exit 3 (was 2) means "may be off and silent", so that 2 can mean
"refused"; the step that adopted an older queue's state file is gone; and the
queue's own advice in a comment keeps its backticks and `<sha>` placeholder
(`lens.no_lens_text(keep_markdown=True)`), which the source dropped.
18 changes: 11 additions & 7 deletions fork-ci/actions/pr-queue/lens.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
NO_HEADER = 'comment without a first-line lens header'
EDITED = 'edited after posting: post a new comment instead'
HIDDEN = 'hidden: post a new comment instead'
SHORT = 'not a full 40-hex sha'
EMBEDDED = 'holds a lens header line' # lens_problem: `not a lens:` cannot clear it
# What a lens report looks like (ONE rule on normalised lines): on a plain line (outside code blocks and quotes) a
# `lens:`, a `read at:` or `read at <hex>`, `safe to merge`, or in its first TOP_LINES plain lines a Markdown heading
Expand Down Expand Up @@ -148,13 +149,16 @@ def loose_header(line):
return bool(m and HEX_RE.search(m.group(1)))


def no_lens_text(text):
def no_lens_text(text, keep_markdown=False):
"""Text that can never count as a lens read, name a lens or look like a lens report (lens_problem), for anything
the queue quotes into a comment (gh errors): no `read at`, no `lens:`, no `safe to merge`."""
text = clean_inline(text)
text = re.sub(r'\bread([\s-]*)at\b', r'read\1@', text, flags=re.I)
text = re.sub(r'\bsafe(\s+)to(\s+)merge\b', r'safe\1to\2land', text, flags=re.I)
return re.sub(r'\blens(\s*):', r'lens\1-', text, flags=re.I)
the queue quotes into a comment (gh errors): no `read at`, no `lens:`, no `safe to merge`. keep_markdown: the
queue's own advice (a command in backticks, a `<sha>` placeholder) stays as written when, read without its markdown,
it holds none of these; otherwise it is cleaned like any other text."""
plain = clean_inline(text)
out = re.sub(r'\bread([\s-]*)at\b', r'read\1@', plain, flags=re.I)
out = re.sub(r'\bsafe(\s+)to(\s+)merge\b', r'safe\1to\2land', out, flags=re.I)
out = re.sub(r'\blens(\s*):', r'lens\1-', out, flags=re.I)
return text if keep_markdown and out == plain else out


def check_state(c):
Expand Down Expand Up @@ -314,6 +318,6 @@ def reviewed_head(self, pr):

def gap(k):
_, sha, why = found[k]
return why or (f'{sha[:12]}: not a full 40-hex sha' if sha else 'no sha: no single commit of this PR')
return why or (f'{sha[:12]}: {SHORT}' if sha else 'no sha: no single commit of this PR')
listed = ', '.join(f'{k} {v[:12] if v else "(" + gap(k) + ")"}' for k, v in sorted(reads.items()))
return True, None, f'the lenses did not all read one head: {listed}'
13 changes: 13 additions & 0 deletions fork-ci/actions/pr-queue/lens.test.py
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,19 @@ def test_a_hex_word_without_a_digit_is_no_sha(self):
self.assertIn(lens.EMBEDDED, SOURCE.lens_problem(post('Notes\n```\nlens: effaced1\n```')))


class Advice(unittest.TestCase):
"""no_lens_text(keep_markdown=True), for the queue's own advice: the source cleaned it like a gh error, so its
`<sha>` placeholder and the backticks around a command were dropped from the comment."""
def test_advice_without_a_lens_word_keeps_its_markdown(self):
for text in ('push (`tool push`)', 'post the override `tool ready --head <sha>: <why>`.'):
self.assertEqual(lens.no_lens_text(text, keep_markdown=True), text)
self.assertNotEqual(lens.no_lens_text(text), text, 'the default still cleans it')

def test_advice_that_holds_a_lens_word_is_cleaned_like_any_text(self): # also one markdown hides
for text, out in (('`lens: x`', 'lens- x'), ('re`ad at`: x', 'read @: x'), ('**safe to merge**', 'safe to land')):
self.assertEqual(lens.no_lens_text(text, keep_markdown=True), out)


class ProjectRules(unittest.TestCase):
"""What the config changes: the override comment and the bot logins."""
def test_the_configured_override_prefix_counts_and_another_does_not(self):
Expand Down
Loading
Loading