[kixelated]

quest usage now reports, for each PR, each online reviewer's record read from GitHub: whether it reviewed the final head, and how many findings it raised and how many were accepted, declined, answered but unclassified, or left unanswered. A new rollup, over the merged PRs, gives each reviewer's coverage, final heads reviewed, findings per PR reviewed, and acceptance. The PR body gains nothing, and no reviewer is named in the code. Finishes and deletes quest/m0/routing-evidence/online-reviewers.md.

Changes

• src/cli/online-review.ts (new): reads each PR's reviews, review threads, and comments through GraphQL, ten PRs per query with aliases, then pages any connection that overflows using @include cursors. GitHub's existing rate-limit wait applies.
  • A reviewer is any Bot review author (shown as login[bot]), or a review whose body opens with Automated review by <name> (shown as <name>, such as review (OpenAI)).
  • A review counts as a pass when it has a body or opens a thread. A reply-only review (empty body, as CodeRabbit and Pullfrog post) does not count as reviewing the head.
  • A finding is a thread the reviewer opens, or a [P<n>] line, or P<n> followed by an em dash, in its review body. Body findings are matched across passes by title (the text up to the first spaced em dash, (, : , or . ), with a repeated title in one body counted as another finding. A thread's comments past the first hundred are paged.
  • An answer is the first thread reply that is not the reviewer's own. For a body finding, the answer is a PR comment that links the review's #pullrequestreview- anchor or opens with a classifying word. It answers every pass it links, or else the latest earlier pass with findings. A reviewer bot's own comments (summaries, outage notices) answer nothing.
  • The answer's first word classifies it. Accepted: Fixed, Agreed, Applied, Right, Confirmed, Covered, Taken. Declined: Declined, Declining, Refuted, Not. Anything else is unclassified.
  • With no answer, a finding counts as accepted when the reviewer settled it, by resolving its own thread or by a later pass that no longer lists it. Otherwise it counts as unanswered.
• src/cli/usage.ts: adds PullRow.online, Usage.online (OnlineRow), the per-PR online review: line, and the Online reviewers over merged PRs: rollup.
• quest usage --help and README describe the rules. The README example gains the new line.

Choices made without asking

• Settled counts as accepted. A finding the reviewer itself settled with no answer (Pullfrog resolving its thread on re-review, or the Dot dropping it from a later pass) counts as accepted rather than as a fifth state. The quest asked for exactly four states.
• Wider word lists. I surveyed the first words of thread replies on the last 100 merged PRs in moq-dev/moq and moq-dev/quest. They opened with Fixed 100, Agreed 24, Declining 13, Applied 6, then Right, Confirmed, Outdated, Moot, and others. I added Right, Confirmed, Covered, and Taken to accepted, and matched Declining. "Moot", "Outdated", "Partly", and "No longer applies" stay unclassified on purpose, so the misses stay visible.
• Body-finding format. The Dot's body findings come as both [P2] Title, then an em dash and the location, and P2, an em dash, then loc: title. Both are read, but a bare P<n> needs a dash after it, so prose like "P2 is" does not count.
• Comment answers need a signal. A PR comment answers body findings only when it links the review or takes a side in its first word. Without that rule, unrelated comments such as "Design change: ..." would count as answers.
• Rollup over merged PRs only. Open PRs' unanswered findings are not final yet. Per-PR lines still show every PR.
• A PR GitHub won't return fails the report. If GraphQL returns null for a listed PR, the report fails naming it, rather than leaving that PR out quietly.

Validation

• just check and just test pass. New test/online-review.test.ts covers classification, both marker forms, thread answers including self-resolve and reply-only reviews, a named reviewer's replies under the shared account, settle-by-later-pass, linked comment answers, and ignored bot notices. test/usage.test.ts mocks /graphql and checks the per-PR record, the rollup, and the text.
• Ran the real CLI. quest --root ~/work/moq usage --since 2026-11-09 reported 195 PRs in about a minute including ccusage, with this rollup:

  Online reviewers over merged PRs:
     coderabbitai[bot]  reviewed 85/158 (54%), final head 39 (25%), 2.5 findings per PR reviewed, 52% accepted; 216 findings: 113 accepted, 39 declined, 14 unclassified, 50 unanswered
     pullfrog[bot]      reviewed 152/158 (96%), final head 124 (78%), 1.1 findings per PR reviewed, 69% accepted; 168 findings: 116 accepted, 3 declined, 11 unclassified, 38 unanswered
     review (OpenAI)    reviewed 136/158 (86%), final head 79 (50%), 0.1 findings per PR reviewed, 65% accepted; 20 findings: 13 accepted, 0 declined, 1 unclassified, 6 unanswered
  

  On moq-dev/quest since 2026-11-09, the rollup shows the Dot at 41/59 and CodeRabbit at 10/59.
• Spot checks against the raw GraphQL data:
  • moq-dev/moq#8458: the Dot's P2 is settled by its next pass, and CodeRabbit's empty reply at the head is not a final-head review.
  • moq-dev/moq#8472: the Dot's bare P2 finding is unanswered at merge.
  • moq-dev/moq#8447: merged with 18 CodeRabbit threads that nobody answered.

Public API

quest usage --json gains pulls[].online and a top-level online array, and the text output gains the lines above. No other CLI or wire change.

Caused by

None. This is new reporting.

Follow-ups

• None recommended. CodeRabbit's "✅ Addressed in commit(s)" self-check could count as settled. The quest chose not to code reviewer-specific signals, so I'd drop it.

Models

• Size: M
• Implementer: Claude Code, claude-opus-5-5, medium effort

Review

Rewritten by quest review after each round.

• Head: 141fae1

• Base: origin/main at 8b626b1

• Reviewer: Codex, gpt-6.1-sol, medium effort

• Session: 01a1ccce-48c1-7411-999a-971ca39001bb, round 2

• Score: 10/10, Code inspection confirms the linked-review attribution fix, and the incremental changes introduce no findings that block landing.

• Findings: none

• The last round's findings, at 37d401e:

  • Fixed: medium, src/cli/online-review.ts:221: Apply answers to every explicitly linked review; The head collects every explicitly linked pass and applies the answer to each pass's findings, with a two-review regression case in test/online-review.test.ts.

• Reviewer: opencode, opencode-go/deepseek-v4.1-flash, high effort

• Session: ses_e3331a8c1ffeEa3o8xRCr53qCN, round 2

• Score: 10/10, All six earlier findings are fixed at the head with matching code, docs, and tests, and the fixes add no new defect.

• Findings: none

• The last round's findings, at 37d401e:

  • Fixed: medium, src/cli/online-review.ts:171: A single >100-comment review thread aborts the entire quest usage report; read() now pages a truncated thread's remaining comments through node(id:) and onlineReviews no longer throws, so a 101-comment thread is counted instead of aborting the whole report (the usage test puts the answer on the thread's second page).
  • Fixed: medium, src/cli/online-review.ts:36: Title truncation collapses distinct findings and leaks answers between them; bodyFindings now appends an occurrence suffix to repeated truncated titles within one body, so two same-titled findings stay two (the new unit test covers exactly the Guard null (GET)/(POST) case), while cross-review matching by title and occurrence still recognizes relists as documented.
  • Fixed: low, src/cli/online-review.ts:189: A named reviewer can never be credited for resolving its own thread; The code is unchanged, but README now says a finding is settled by "a bot resolving its own thread", matching the code and the fact that a named reviewer's resolution cannot be told from the agent's since both post as the same account.
  • Fixed: low, src/cli/online-review.ts:77: A null PR or repository in the batch response kills the report with a misleading error; A null PR alias now fails with the clear message "GitHub has no PR #N in owner/repo for this token" instead of the generic invalid-response error, and failing the report rather than silently omitting a PR is a defensible fail-loud choice I cannot rebut.
  • Fixed: low, src/cli/online-review.ts:194: Docs promise every reviewing bot is shown, but reply-only reviews are invisible; README, the usage help text, and the module comment now all define a bot reviewer as one whose review has a body or opens a thread, matching the passes filter, with reply-only reviews excluded by design.
  • Fixed: low, src/cli/usage.ts:756: Singular finding prints as "1 findings"; answers() now prints "1 finding" for a single finding and "N findings" otherwise, and the usage test asserts the singular form.

🤖 Generated with Claude Code

(Written by Claude Opus 5.5 with medium)

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 57 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: a6663be5-a986-4d94-b714-91cd5b983266

Commits

Reviewing files that changed from the base of the PR and between 28e6c87 and 141fae1.

Files selected for processing: 8

• README.md
• quest/m0/routing-evidence/README.md
• quest/m0/routing-evidence/online-reviewers.md
• src/cli/main.ts
• src/cli/online-review.ts
• src/cli/usage.ts
• test/online-review.test.ts
• test/usage.test.ts

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn