Repository navigation
CHANGELOG.md prepend collisions make long-lived PRs non-convergent: 8 rounds in 19h, next collision 4m21s after the last resolution #1961
Description
Every pull request in this repository adds its entry by prepending a ### … section at line 1 of CHANGELOG.md. Git cannot auto-merge two insertions at the same anchor, so any two open PRs that both carry an entry conflict pairwise, and every merge to main re-conflicts every remaining open PR that has one. No individual change is at fault; the collision is created by the file's shape.
Measured on 2026-09-06
commits to main in 24h 28
of which touched CHANGELOG.md 20 (71%)
non-draft open PRs (first page alone) 75
On one PR (#1923, whose actual delta is a single test file plus its entry) this produced eight conflict rounds in nineteen hours:
01:18 06:27 07:34 12:21 14:29 16:51 20:33 ← merge commits
20:37 ← round 8 trigger
The last interval is the point of the report: round 7 was resolved and pushed at 20:33:09, and the commit that invalidated it landed at 20:37:30 — 4 minutes 21 seconds later. One resolution cycle on this repository costs about three minutes of gate time alone (2932 tests), before detection, merge, and push. When the interval between collisions falls below the cost of resolving one, a PR carrying a changelog entry cannot converge by resolving conflicts — it is not slow, it is non-convergent, and it stays open until the merge rate drops.
Every one of those eight resolutions was mechanically identical: keep main's sections, re-prepend this branch's section. No round required a judgement call, which is the signature of work that should not be done by hand.
Cost this is currently paid in
- Each round runs the full suite again (~3 min × 8 = ~24 min of runner or local CPU for one PR), which competes for the same capacity tracked in ops: diagnose and bound organization GitHub Actions queue starvation #712 and Actions queue saturation: 120 open PRs + self-amplifying scheduler block all org merges (pg-erd-cloud: 0 merges since 2026-08-20) #1531.
- Each round is an opportunity for silent loss. The failure mode is not a visible conflict marker but a dropped section, a duplicated header, or a fenced block split into two fragments — none of which any test in this repository detects, because no test parses
CHANGELOG.mdstructure. - The rate scales with merge throughput, so it gets worse exactly when the queue is healthy.
Directions, in increasing cost
- A merge driver for
CHANGELOG.md—.gitattributesplus a small union-style resolver that keeps both sides in order. Smallest change; leaves the file format alone; every clone needs the driver configured, and an unconfigured clone silently falls back to a normal conflict. - Per-PR fragments — each PR adds
changelog.d/<pr>.mdand the file is assembled at release. Two PRs never touch the same line, so the conflict class disappears entirely rather than being auto-resolved. Costs a convention change and a release-time assembly step, and several contract tests assert exactCHANGELOG.mdprose today. - Status quo — accept that any long-lived PR carrying an entry is re-conflicted on roughly every merge.
Which of these is right is a repository-convention decision rather than a repair, so I have not implemented one; this issue records the measurement so the trade-off can be made against real numbers instead of an impression.
Activity
seonghobae commentedon Sep 6, 2026
Measurements from the merge lane that support this, gathered while merging four pull requests today.
Churn. Of the last 29 commits on
mainin 24 hours, 21 touchedCHANGELOG.md— every substantive change carries an entry, and every entry goes at the top of the same file, so the file is a serialization point that every open branch contends for.Cost per merge, measured. Three merges landed in 27 minutes:
5ea1cc47at 20:24:50,0b0f1047at 20:37:30,c232ca03at 20:51:11 (KST). Each one invalidated the changelog of every other open branch. Concretely, in that window #1957 had to be resolved and re-pushed once, my own #1959 once, and #1960 twice — its author resolved it, my gate then aborted on a fresh conflict created by the merge that happened while I was verifying it, and it needs a third resolution now.Why it does not self-limit. Resolving costs a re-push plus a re-verification gate of roughly three to four minutes, while the interval between merges today was about 13 minutes. When more than one branch is waiting, the resolution of the branch at the back can be invalidated before it is verified, which is the non-convergence #1923 hit over eight rounds.
What does not fix it. Skipping the entry, or letting the merger resolve it. Skipping loses the record the file exists for; a merger resolving an author's conflict collapses the author/verifier/merger separation that has been catching real defects here today, and it changes the head, which is the author's to own.
Two things that keep it tolerable today, offered as workarounds rather than a fix. Merging serially with an empty queue converges in one round, which is why the current resolution should be the last one for #1960. And the entry order that survives is by merge time, newest on top, which is verifiable against history: #1949 merged five minutes after #1950 and its entry sits above. Ordering by "what is already in the file" produces a history that disagrees with itself.
The structural options — a per-change fragment directory collated at release, or moving entries out of a single shared file — both change how every contributor writes a changelog, so they are the owner's call rather than something to slip in from the merge lane.
seonghobae commentedon Sep 6, 2026
A second branch's history makes the repeat cost concrete, and I verified it rather than relaying it. #1911 has been open across today's merges and carries four merge commits from
mainon its own branch, three of them within four hours:c36533beat 15:57,87f0262bat 18:11,21e128bfat 20:30 and8761c004at 20:42 (KST). The last two land twelve minutes apart, and each one resolves a conflict whose only conflicted file isCHANGELOG.md—git show --name-onlyon both returns that single path.So the pattern is not one branch unlucky once. Within today: #1911 resolved four times, #1960 three times counting the one it still needs, #1957 once, #1959 once. Every one of those resolutions had zero code conflicts.
That also sharpens where the cost falls. A changelog conflict is trivial to resolve and carries no risk of losing logic, so the expense is not the edit — it is that resolving changes the head, and a changed head invalidates whatever verification the branch had accumulated. On a branch under author/verifier/merger separation that means a re-push, a re-verification, and a re-gate, three to four minutes each, for a conflict that never touched code. #1911's four merges are four rounds of that.
One consequence worth stating for whoever designs the fix: any scheme that keeps a single shared file but changes where new entries are inserted only moves the collision, because every branch still writes the newest entry adjacent to every other branch's newest entry. Per-change fragment files avoid the collision outright, at the cost of a collation step and a convention every contributor has to learn.
Merge origin/main into ci/pin-required-status-contexts
seonghobae commentedon Sep 6, 2026
Confirmed on a second, independent axis, and the interval has been measured twice more.
My original axis was elapsed time on one pull request. A second session measured repetition counts across several pull requests. The two are independent — one can be short while the other is small — and they agree.
Axis 1 — interval between resolution and invalidation (#1923)
Round 9 has now happened. Both measured intervals are shorter than one resolution cycle:
A cycle costs about three minutes of gate time alone (2932 → 2949 → 2956 tests across the three rounds), before detection, merge, resolution and push. Nine rounds on this pull request in nineteen hours; its actual delta remains one test file plus its changelog entry.
Axis 2 — repetition per pull request
Measured and reported by the session that owns those branches, from branch history rather than relayed:
Every one of those commits returns
CHANGELOG.mdand nothing else fromgit show --name-only. Zero code conflicts across all of them. That is the load-bearing fact: this is not integration friction between changes that genuinely overlap, it is a single file's shape generating conflicts between changes that do not touch each other at all.That session's own 24-hour count, taken after mine and so slightly later: 29 commits to
main, 21 touchingCHANGELOG.md. My earlier figure was 28 / 20;mainmoved between the two readings.One correction worth recording: the #1911 count reached me second-hand as three rounds. It is four. The session that owns the branch read its history directly and corrected it. A relayed count is not a measurement.
What the two axes together establish
Short intervals alone could be survivable if each PR only hit one or two. High repetition alone could be survivable if the gaps were long. Both hold at once, which is the non-convergent case: a long-lived pull request carrying a changelog entry is invalidated faster than it can be repaired, and the repair is mechanically identical every time. No round in any of the above required a judgement call.
Merge origin/main into ci/pin-required-status-contexts
seonghobae commentedon Sep 6, 2026
Ran the obvious fix to see whether it holds. It resolves the collision, it has a specific corruption mode, and that mode does not apply to this file today — but nothing here would catch it when it starts to.
CHANGELOG.md merge=unionresolves the collisionThere is no
.gitattributesonmaintoday. Adding one turns the pairwise conflict into a clean merge — reproduced locally on the exact shape described here, two branches each prepending a### …section:The corruption mode, reproduced
Union takes both sides of the hunk, and where a line is identical at a hunk boundary it is not duplicated. With entries that contain fenced blocks, that eats a fence:
beta-1
beta-2
A <- B's closing fence is gone; A's heading is now inside B's code block
...
fence markers: 3 balanced: False
CHANGELOG.md on main: 1476 lines, 24 '### ' sections, 0 fence markers
sections containing a fenced block: 0 of 24
seonghobae commentedon Sep 11, 2026
Validation finding from PR #2091 exact head
d148e04664cc9de92d56e2bc0a95c9cd94e35b92: focused changelog contract tests pass (2 passed), but.gitattributesonly declaresCHANGELOG.md merge=union. The PR adds independent release fragments underCHANGELOG.d/*.md;git check-attr merge -- CHANGELOG.d/20260911-changelog-union-contract.mdreturnsmerge: unspecified. The test therefore proves only the root changelog rule, not the fragment path. Finding posted on PR #2091: #2091 (comment).Merge origin/main into docs/gap-baseline-nim-routing-confirmation