Skip to content

fix(ci): select a merge group's changes from main - #6991

Merged
kixelated merged 2 commits into
mainfrom
quest/m0/ci/select-audit
Oct 28, 2026
Merged

kixelated merged 2 commits into
mainfrom
quest/m0/ci/select-audit

Conversation

@kixelated

@kixelated kixelated commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Completes quest/m0/ci/select-audit.md.

Problem

The quest asked which dependency edges just ci's selection misses, found by replaying every full run (Nightly just gate) that failed while its selected queue run passed. Nightly has run the full gate since October 20 (moq-dev/moq#6038). Of the eight scheduled gates from October 20 through 27, four failed and one timed out:

  • October 21, a missed edge. moq-dev/moq#5849's own queue entry failed on a dangling quest link. It merged anyway when the next entry, moq-dev/moq#6012, passed. From October 20 23:58 to October 22 23:33 UTC the ruleset used HEADGREEN grouping, so a passing head merged every entry under it. But Check diffed each entry from merge_group.base_sha, the entry ahead of it, so the head never selected their changes. 37 attempts landed on main without their head passing, all in that interval. They include 13 dangling quest links, a moq-net compile error, missing JS doc comments, and a JS type error.
  • October 22, not selection. Nine @moq/watch CMAF decoder tests depend on test-file order. The queue run for moq-dev/moq#6219 ran and passed them. At the Nightly head the whole suite passes locally, while decoder.test.ts alone fails the same nine.
  • October 24 and 25, not selection. Nightly's shallow checkout had no HEAD~1. Fixed by moq-dev/moq#6708.
  • October 27, not selection. A 36-minute hang hit the 60-minute timeout.

No path-level edge was missed.

Approach

  • Check now diffs a merge group from origin/main, so a head's run covers every entry it can merge, whatever the grouping strategy. CI_BASE moves to job env so the two steps can't disagree.
  • sh/gh/queue.test.ts pins that base. It fails against the old workflow.
  • sh/dispatch-query.test.py's merge-group case now has an unmerged Rust entry ahead of a quest-only entry, which must select Rust.
  • .github/ci-selection-audit.md now holds the full replay, the 37-attempt breakdown, and the cost data.

Cost: none at today's settings (ALLGREEN, one entry built at a time). Of 487 attempts since October 22 23:33 UTC, none started before the entry ahead had merged. With five entries built at once (October 10-20), 806 of 1,047 attempts started on an unmerged entry, and would now also select its changes.

Impact

  • Public API: none. Wire: none.
  • CI: merge-group selection can widen only while the queue builds several entries at once.

Alternatives

  • Keep the incremental base and fail the queue run unless the ruleset is ALLGREEN (read through GET /repos/{owner}/{repo}/rules/branches/main). This would be cheaper if build concurrency returns. But it adds a network dependency to every queue run, and it ties correctness to a setting that changed twice in three days, while the ruleset was edited eight times. Rejected for the simpler base, which is correct under any strategy.
  • Do nothing because ALLGREEN is back. Rejected: nothing would catch the next HEADGREEN switch.

Choices made without asking

  • The replay covers October 20 through 27, because no full run existed before October 20. October 27 is past the audit window, but it is included.
  • I did not reproduce the test failures among the 37 attempts. The report says some may be flakes.

Follow-ups

  • Nightly gate hang (October 27): run 40506938673 printed nothing for its last 36 minutes before the 60-minute timeout. Tooling-test output is buffered, so the module that hung isn't visible. Reproduce with just gate, find the hung test, and consider per-module output timestamps so a hang names its module.
  • @moq/watch test-order dependence: decoder.test.ts failed alone at 33940b6e but passes alone on current main. The fix wasn't bisected, because bisect results were inconsistent with state outside the commit. Worth a check that the suite stays order-independent, for example by running each test file alone in a nightly lane.
  • Queue bypasses: five PRs merged outside the queue after October 10, four on October 23 while the ruleset had no merge-queue rule. No selection rule can cover those.

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

kixelated and others added 2 commits October 27, 2026 18:49
Replay every Nightly full-gate failure since the gate entered Nightly against
the selected queue gate. The only missed edge: Check diffed each merge group
from the entry ahead of it, so under HEADGREEN a passing head merged failed
entries whose changes it never selected. Diff from origin/main instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Replay full-run failures against CI's selected gate fix(ci): select a merge group's changes from main Oct 28, 2026
@kixelated
kixelated marked this pull request as ready for review October 28, 2026 04:42
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-28T04:43:19.322804Z 3532ac3 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 3532ac3

No actionable bugs found in the changed workflow, selector regression coverage, or relevant dispatcher/merge-base handling. Selecting the queue head against main is the right direction: it covers pending predecessors without coupling correctness to the queue grouping setting. Sharing CI_BASE at job scope also keeps selection and execution aligned; release PRs retain --all.

Verification limits: static review only; I did not execute the tests or independently reproduce the historical CI failures and audit counts.

@coderabbitai

coderabbitai Bot commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 0b87867a-208a-4cb3-badd-cc4fd2397709
Commits

Reviewing files that changed from the base of the PR and between fbeac8a and 3532ac3.

Files selected for processing: 6
  • .github/ci-selection-audit.md
  • .github/workflows/check.yml
  • quest/m0/ci/README.md
  • quest/m0/ci/select-audit.md
  • sh/dispatch-query.test.py
  • sh/gh/queue.test.ts
Files with no reviewable changes: 2
  • quest/m0/ci/README.md
  • quest/m0/ci/select-audit.md

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 2 remain after this review.

Walkthrough

The Check job now sets CI_BASE at job scope. Release pull requests use --all; other runs use origin/main. Tests cover the workflow values and verify Rust selection for a merge-group query with an unmerged change. The CI selection audit now records replay results and queue failures. The README requirement and task document for further replay work were removed.

Priority: Normal

Severity of issue fixed: Medium

Merge Risk: Minimal · up to 3532a

CI selection for merge groups now diffs from origin/main, so entries ahead in the queue are no longer missed. The tests and audit notes match that change. No actionable merge risk remains.

Pre-merge checks: Passed 5
Check name Status Explanation
Description check Passed The description clearly explains the CI selection issue, the implemented fix, validation, impact, alternatives, and follow-ups. It is directly related to the changeset.
Title check Passed The title clearly and concisely describes the main change: selecting a merge group's changes from main.
Docstring Coverage Passed No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 0 functions across 2 files. (2 skipped: 2 unsupported.)

Finishing Touches
✨ Simplify code
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@kixelated

Copy link
Copy Markdown
Collaborator Author

Merge summary: Codex reviewed 3532ac3 with no findings (thumbs-up), and the OpenAI automated review found no actionable bugs. CI passed (15/15), and quest check against origin/main passed. I made no changes during merge, and no decisions are open. Enqueuing a squash merge. The follow-ups in the body (Nightly gate hang, @moq/watch test-order dependence, queue bypasses) are left for /quest-plan.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 28, 2026
Merged via the queue into main with commit 7c1161e Oct 28, 2026
15 checks passed
@kixelated
kixelated deleted the quest/m0/ci/select-audit branch October 28, 2026 07:12
kixelated added a commit that referenced this pull request Oct 29, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant