Skip to content

fix(roadmap): the jobs trust the items of the two bots - #3664

Merged
dan-petty merged 9 commits into
mainfrom
fix/3502-roadmap-trust-service-app
Nov 5, 2026
Merged

dan-petty merged 9 commits into
mainfrom
fix/3502-roadmap-trust-service-app

Conversation

@dan-petty

Copy link
Copy Markdown
Owner

Refine, stage, implement and alternatives left out each item that the Service App or github-actions[bot] opened, because GitHub gives both bots the association CONTRIBUTOR. 38 items waited on this rule. Now a job takes an item whose author has write access, or whose login is exactly service.github_app.bot_login or github-actions[bot]. A bot item reaches Ready only after a review of refine approves the issue and the proposal. The other jobs take it only while that approval holds for the same body. The models of refine and of the alternatives job read only the replies of trusted authors. Refine tells the author of any other item once.

Closes #3502

Change: openspec/changes/archive/2026-11-05-3502-jobs-trust-service-app-items/

Plans #3502: the jobs take an item of the Service App's bot and of
github-actions[bot], refine reviews such an item before it is Ready, and the
model of refine reads the replies of trusted authors only.
Item, IssueRecord and IssueComment hold the login of the author, which the
GitHub store reads from user.login. The in-memory store seeds it, and gives
the owner's association to a comment that it writes, as GitHub gives an
association to each comment.
…flows

The refine, stage, implement and alternatives jobs take an item whose author
login is exactly the login of the Service App's bot or github-actions[bot],
beside the items of a person with write access. Refine reviews a Ready proposal
for such an item with the verification pass of the stage job, and the other
jobs take the item only with the recorded approval. A refusal keeps the item
New with one comment. Refine tells the author of any other item once, and its
model reads only the replies of trusted authors.
SECURITY.md names the rule and the reason: the exact login is the proof, the
review of refine is no security control, and the model of refine reads only the
replies of trusted authors. The refine page and the workflow page describe the
review and the record key refine.judge.
…view read

An approval (refine.judge approved) now holds only while the body hashes as
refine recorded with the verdict. A body edit ends it, a refine round reviews a
Ready item of a bot with no valid approval again, and a record of the hashes
with no review clears the verdict. The hash helpers move to body_hash.py.

The alternatives job gives its model the comments of trusted authors only, and
the dry-run plan of refine lists the read and the notice comment for an item of
an author that the jobs do not trust. The new texts are shorter sentences.
…he specs of the jobs

SECURITY.md rule 14, the refine and alternatives pages and the key table name
the approval and the filter of replies. The change folder adds deltas for the
alternatives, stage and implement specs, and shorter sentences.
@dan-petty dan-petty added type/bug Defect, regression, or incorrect runtime behavior scope/roadmap The roadmap jobs: intake, refine, stage, implement and the Service that runs them labels Nov 5, 2026
@dan-petty

Copy link
Copy Markdown
Owner Author

Fail-to-pass run

Command: uv run devops ci fail-to-pass --base origin/main.

  • 40 tests failed on base. 6 test files failed to collect on base, because InMemoryRoadmapStore.seed_issue() takes no author_login there: test_roadmap_explain, test_roadmap_implement_selection, test_roadmap_refine, test_roadmap_refine_trust, test_roadmap_repair and test_roadmap_trust.
  • All of these tests pass on this branch.

Full gate run

Command: uv run devops ci on the final code, after the rebase onto origin/main (f6d57ec).

  • 24 of 25 rows passed: lint, format, typecheck, security, docs, openspec, constructs, test-tree and the others.
  • The test row ran 18,874 passing tests and coverage was 94.52% (floor 85%). It reported 4 failures and 1 teardown error. All five come from the state of this host, not from this change:
  • The test row took 7m 44s on a host that other agents shared. The slowest test is tests/commands/test_ai_local_dry_run.py::test_local_ai_commands_run_cleanly (74 s). The files of this change hold none of the ten slowest tests.

Earlier review findings

I checked each finding against the code on this branch.

  • Approval after a body edit: fixed. trust.is_approved also needs the body to hash as refine.body_hash and refine.section_hash record. Tests cover the stage, implement and alternatives readers, and the second refine round. The title is not part of the hashes, and design.md says why: refine ignores a retitle, and only a person with write access can retitle an item of a bot.
  • Documents and specs: fixed. The change holds deltas for roadmap-alternatives, roadmap-stage and roadmap-implement-job, and docs/roadmap/alternatives.md and docs/roadmap/refine.md name the two bots.
  • ASD-STE100: fixed. I counted the words of each new output string, and no sentence has more than 23 words. No paragraph of the new text has more than 6 sentences.
  • Dry-run plan of refine: fixed. refine_requests lists the comment read and the notice comment, and tests/roadmap/test_roadmap_dry_runs.py has a test.
  • Comments that reach the alternatives job: fixed in this pull request. _check_one filters them with trusted_replies, and tests/roadmap/test_roadmap_alternatives.py has a test. So SECURITY.md rule 14 is true as written.
  • Test that patched get_llm_client: fixed. The test uses the model_fake fixture and patches nothing. The proposal and review doubles of the file go in through the model and verifier parameters of plan_refine, as in the other refine test files.

The owner decides at merge: the change widens the trust rule of #2478 to two bots (AGENTS.md rule 5).

@dan-petty
dan-petty marked this pull request as ready for review November 5, 2026 17:31
@dan-petty
dan-petty merged commit 41a362d into main Nov 5, 2026
23 checks passed
@dan-petty
dan-petty deleted the fix/3502-roadmap-trust-service-app branch November 5, 2026 18:06
dan-petty pushed a commit that referenced this pull request Nov 5, 2026
🤖 I have created a release *beep* *boop*
---


##
[0.62.7](v0.62.6...v0.62.7)
(2026-11-05)


### Bug Fixes

* **ai:** a hosted gateway call is priced, not counted local
([#3663](#3663))
([acdc8a9](acdc8a9))
* **roadmap:** plan-check measures each task group
([#3669](#3669))
([3791bd9](3791bd9))
* **roadmap:** the jobs trust the items of the two bots
([#3664](#3664))
([41a362d](41a362d))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: dan-petty-release[bot] <346998822+dan-petty-release[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

scope/roadmap The roadmap jobs: intake, refine, stage, implement and the Service that runs them type/bug Defect, regression, or incorrect runtime behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(roadmap): the jobs trust the items that the Service App opens

1 participant