Skip to content

Move the Hub's Own Bot Reviews to Explicit Requests - #3899

Merged
ptr727 merged 2 commits into
developfrom
feature/hub-explicit-review
Nov 6, 2026
Merged

ptr727 merged 2 commits into
developfrom
feature/hub-explicit-review

Conversation

@ptr727

@ptr727 ptr727 commented Nov 6, 2026 •

Copy link
Copy Markdown
Owner

Summary

Step 1 of the review-cutover track (#3877): moves the hub's own pull request reviews from automatic to explicit requests, through a registry agentPolicy override and the files scripts/agent_policy.py write renders from it. The maintainer chose the values on 2026-11-05.

Bot Trigger Bases When Rounds
Copilot explicit, gates default branch after-checks per push, capped at 3
CodeRabbit explicit main, develop after-checks once
Qodo explicit main, develop after-checks once
  • registry/repos.json gains ProjectTemplate's agentPolicy override. Only the fields that differ from spec/agent-policy.default.json are stated.
  • .github/agent-policy.json, .coderabbit.yaml (auto_review.enabled: false), and .pr_agent.toml (disable_auto_feedback = true) are the render.
  • docs/pr-reviewer-reference.md now says CodeRabbit's automatic review is off.
  • tests/test_spec_validate.py: run_gate_on_files copies the real registry, and the gate imports scripts/agent_policy.py for an entry carrying agentPolicy. ProjectTemplate's is the first such entry, so the scratch tree now carries that module and the two it imports. Without them the gate crashed with ModuleNotFoundError, which failed the first CI run.

The render also drops the copilot_code_review rule from the main ruleset at the next repo-config/configure.sh apply. The develop ruleset carries no such rule.

Sequencing

  • configure.sh apply against ptr727/ProjectTemplate runs before the promotion that carries this file to main, with the maintainer's go-ahead at the time. Otherwise the ruleset's own Copilot trigger reviews promotion pushes alongside the explicit request.
  • Copilot's personal "Automatic Copilot code review" toggle stays on until step 4 of the track, so Copilot still starts its own rounds until then. Those rounds count toward the cap of 3.
  • Rollback is reverting this commit and then running configure.sh apply.

Verification

  • uv run --no-project --with-requirements tests/requirements.txt --with pytest python spec/validate.py: Spec validation OK.
  • The same environment, python -m pytest -q tests: 3583 passed, 11 skipped.
  • scripts/agent_policy.py facts: copilotRule false on main and develop, CodeRabbit enabled false with base_branches ["^develop$"], Qodo disable_auto_feedback true.
  • Local strict pass, two rounds. The first raised four findings: one stale doc sentence (fixed) and two commit-message inaccuracies (fixed by amending before the push), plus the apply-before-promotion ordering, which has no code fix and is recorded under Sequencing above. The second pass raised none. A third pass, over the test fix, raised none. Both are recorded.

🤖 Generated with Claude Code

Adds a registry agentPolicy override for ProjectTemplate and commits the
files scripts/agent_policy.py renders from it. All three bots go
explicit, each asked only once the head's checks are green.

- Copilot: explicit on the default branch, a round per push, capped at
  3. Past the cap an attested local pass covers a new head.
- CodeRabbit and Qodo: explicit on main and develop, asked once per
  pull request. Neither gates.

The render turns off CodeRabbit's auto_review and sets Qodo's
disable_auto_feedback. It also drops the copilot_code_review rule from
the main ruleset at the next configure.sh apply. The develop ruleset
carries no such rule. docs/pr-reviewer-reference.md now says
CodeRabbit's automatic review is off.

Rollback is reverting this commit and then running configure.sh apply,
which restores the main ruleset's Copilot rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings November 6, 2026 03:52
@ptr727

ptr727 commented Nov 6, 2026

Copy link
Copy Markdown
Owner Author

A recorded local strict-review pass covers head d463cd8d0eeed75c0aaebdd2dcdad9f93fb23136, the content this pull request carries at that commit against develop, and it recorded 0 findings.

@coderabbitai

coderabbitai Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Defer because promotion depends on maintainer-controlled live settings and a configure.sh apply sequence that cannot be verified from this diff.

No open issues 🎉

What changed in this PR

This pull request moves ProjectTemplate’s bot reviews from automatic triggers to explicit, policy-driven requests while preserving the configured review coverage and limits.

Changes:

  • Adds a ProjectTemplate-specific agentPolicy override for Copilot, CodeRabbit, and Qodo.
  • Configures explicit after-checks reviews, with Copilot capped at three rounds.
  • Disables automatic CodeRabbit and Qodo feedback.
  • Documents the updated CodeRabbit review behavior.
File Description
registry/​repos.json Defines ProjectTemplate’s explicit bot review policy override.
docs/​pr-reviewer-reference.md Documents CodeRabbit’s explicit-request behavior.
.pr_agent.toml Disables automatic Qodo feedback.
.github/​agent-policy.json Renders the explicit review policy and round limits.
.coderabbit.yaml Disables automatic CodeRabbit reviews while retaining the develop base.

Review effort: Lite

@ptr727

ptr727 commented Nov 6, 2026

Copy link
Copy Markdown
Owner Author

Listed in the body. The verdict headline flags the dependency on the live configure.sh apply and the maintainer-controlled settings. The PR body's "Sequencing" section records that order: configure.sh apply against this repository runs with the maintainer's go-ahead before the promotion that carries this policy to main, and Copilot's personal toggle stays on until step 4 of the track. This diff changes only committed files, so nothing in it can verify a live setting. The apply is a separate, gated step in the track (#3877).

run_gate_on_files copies the real registry into a scratch tree, and the
gate imports scripts/agent_policy.py for any entry carrying an
agentPolicy. ProjectTemplate's entry is now the first to carry one, so
the gate crashed on the missing module before reaching the loop each
case asserts on. The scratch tree now carries agent_policy.py and the
two modules it imports.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ptr727

ptr727 commented Nov 6, 2026

Copy link
Copy Markdown
Owner Author

A recorded local strict-review pass covers head 27dcefbb589eba927265049454d36d45d31cb9d0, the content this pull request carries at that commit against develop, and it recorded 0 findings.

@codecov

codecov Bot commented Nov 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 77.87%. Comparing base (d8f8161) to head (27dcefb).

Additional details and impacted files
@@           Coverage Diff            @@
##           develop    #3899   +/-   ##
========================================
  Coverage    77.87%   77.87%           
========================================
  Files           34       34           
  Lines        12085    12085           
========================================
  Hits          9411     9411           
  Misses        2674     2674           
Flag Coverage Δ
python-3.13 77.87% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ptr727
ptr727 merged commit 1a934bf into develop Nov 6, 2026
13 checks passed
@ptr727
ptr727 deleted the feature/hub-explicit-review branch November 6, 2026 04:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants