Skip to content

fix(server): a PR watch reports a comment's edit once per head - #36329

Open
juliusmarminge wants to merge 10 commits into
mainfrom
factory/pr-36280
Open

juliusmarminge wants to merge 10 commits into
mainfrom
factory/pr-36280

Conversation

@juliusmarminge

Copy link
Copy Markdown
Member

A PR watch holds an edit of a remark the agent already saw until a read finds it unchanged, then wakes once with the final text. The review finding was correct: the once-per-head rule (remarkEditsTold) dropped every later edit of a remark already reported on that head, so a review bot's 'reviewing' -> 'finished' after a push delivered only 'reviewing'. Instead of removing remarkEditsTold (which would leave nothing beyond #30674's settle delay), an edit of a remark already reported on this head is now held like any other edit, and delivered once its editedAt is TOLD_EDIT_SETTLE_MS (10 minutes) old. That is longer than a bot's countdown rewrites, so a countdown stays quiet until it stops and then wakes once with its final text, while a finished review on the same head still reaches the agent. A remark's first edit on a head still settles after one unchanged read, and the record still resets after a push. Tradeoffs: the final text of an already-reported remark arrives about 10 minutes after the bot's last edit, and while such an edit is held the reactor re-reads each 2-minute sweep even under a quiet host fingerprint. The per-head regression test is rewritten around timed reads to cover the countdown, a second remark, the reviewer's reviewing -> finished sequence after a push, and the earlier old-head held-edit case; it fails on the previous code.

Fixes #30342

Supersedes #36280. The change is @royalaid's; their commits are kept.

Reproduction on main

  1. In a T3 Code thread, ask the agent to watch an open GitHub PR using watch_pull_request. Keep checks and the PR head unchanged. 2. Have another account post a comment and allow the watch to deliver it. 3. Edit that same comment, then leave it unchanged for at least two watch sweeps, approximately four minutes. 4. Repeat the edit-and-wait sequence several times without pushing. Main generates another new-comment wake for each observed edit; this change delivers the first settled edit and suppresses subsequent edits of that comment on that head. 5. Push a new commit, edit the same comment again, and leave it unchanged for two sweeps: another edited-comment wake should arrive. These steps were derived from the diff and issue, not executed.

Evidence

Being captured: before and after in the real app. This section is filled in before the pull request can merge.

Tests

vp test run apps/server/src/orchestration-v2/pullRequestWatch.test.ts: the new test fails on f6f3323's evaluator (final edit never reported); 42/42 pass with the fix. vp run check:changed: format, lint, knip, graphql, typecheck (37 packages) and tests (8 files) all pass, after formatting the test and making the new constant module-private because knip flagged its export as unused.

Not in this change

Who did the work

  • Written by Claude Opus 5.5, in Claude Code
  • Reviewed by GPT-6.1 Sol, in Codex, and by Claude Opus 5.5, each on its own
  • Proof captured by GPT-6.1 Sol and checked by Claude Opus 5.5

Run by the t3code factory in T3 Code.

Umais-Adeed and others added 10 commits November 10, 2026 10:19
…y edit

The watch dated a remark by `editedAt ?? createdAt`, so each edit of one bot
status comment woke the agent again as "new comments". Hold an edit of a remark
the agent already saw until a read finds it unchanged, then wake once and say
it was edited. A held edit forces the next sweep to reread remarks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pages

Cached review-thread tails survive an edit, because neither the comment count
nor the top-level watermark changes. The confirming read could then report a
stale held edit as settled. Drop cached tails while a watch holds an edit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eads

Watching a pull request again now carries the held edit forward, so the
settled text still reaches the agent. A watch holding an edit also drops
cached activity before reading, even without a host fingerprint.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tch was stopped

Watching again starts remarksThrough at the restart, so a held comment edited
while the watch was stopped looked old and its hold was dropped. A held comment
with a new edit is now held again at that edit and delivered once it settles.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A bot that keeps rewriting one comment on the same head woke the agent
once per rewrite. CodeRabbit, while a review is rate limited, rewrites its
walkthrough with a countdown every few minutes; each rewrite stays
unchanged for longer than a sweep, so each was held, found settled and
delivered. CI status comments rewritten per job do the same.

The watch now records the remarks whose edit it reported on the current
head and neither holds nor reports their later edits until the head
moves. The first edit after a push still wakes, so a bot that re-reviews
a push by editing its summary is reported (#15282). A remark new to the
watch, and other remarks' first edits, are unaffected. Watching again
keeps the record, like the other news the last watch told; watches saved
before the field load unchanged.
…ead's report

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r a longer settle

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added vouch:trusted Author is trusted by repo permissions or the VOUCHED list. size:M 30-99 changed lines (additions + deletions). labels Nov 13, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire 5.0 KiB 5.0 KiB 0 B (0.0%) 6.8 KiB ✅
Codex Thread snapshot wire 3.8 KiB 3.8 KiB 0 B (0.0%) 4.9 KiB ✅
Codex Live turn WebSocket wire 1.2 KiB 1.2 KiB 0 B (0.0%) 2.0 KiB ✅
Codex Live turn WebSocket decoded 21.5 KiB 21.5 KiB 0 B (0.0%) 24.4 KiB ✅
Codex Live turn messages 1 1 0 (0.0%) 8 ✅
Claude Total thread wire 5.0 KiB 5.0 KiB 0 B (0.0%) 6.8 KiB ✅
Claude Thread snapshot wire 3.8 KiB 3.8 KiB 0 B (0.0%) 4.9 KiB ✅
Claude Live turn WebSocket wire 1.2 KiB 1.2 KiB 0 B (0.0%) 2.0 KiB ✅
Claude Live turn WebSocket decoded 21.9 KiB 21.9 KiB 0 B (0.0%) 24.4 KiB ✅
Claude Live turn messages 1 1 0 (0.0%) 8 ✅

Other scenarios

All 64 scenario measurements match main.

All 64 scenario measurements
Provider Scenario Metric Main baseline This PR Impact PR ceiling
Codex short turn resume WebSocket decoded 10.5 KiB 10.5 KiB 0 B (0.0%) 12.7 KiB ✅
Codex short turn resume WebSocket messages 2 2 0 (0.0%) 2 ✅
Codex long turn running, cold open HTTP decoded 281.7 KiB 281.7 KiB 0 B (0.0%) 302.7 KiB ✅
Codex long turn running, cold open rows 75 75 0 (0.0%) 75 ✅
Codex long turn running, history page HTTP decoded 92.4 KiB 92.4 KiB 0 B (0.0%) 107.4 KiB ✅
Codex long turn running, history page rows 75 75 0 (0.0%) 75 ✅
Codex long turn finished, cold open HTTP decoded 381.5 KiB 381.5 KiB 0 B (0.0%) 410.2 KiB ✅
Codex long turn finished, cold open rows 151 151 0 (0.0%) 151 ✅
Codex long turn finished, socket cold open WebSocket decoded 381.6 KiB 381.6 KiB 0 B (0.0%) 410.2 KiB ✅
Codex long turn finished, socket cold open WebSocket messages 2 2 0 (0.0%) 2 ✅
Codex long turn finished, socket cold open rows 151 151 0 (0.0%) 151 ✅
Codex long turn finished, work page HTTP decoded 186.2 KiB 186.2 KiB 0 B (0.0%) 200.2 KiB ✅
Codex long turn finished, work page rows 150 150 0 (0.0%) 150 ✅
Codex idle thread resume WebSocket decoded 65 B 65 B 0 B (0.0%) 200 B ✅
Codex idle thread resume WebSocket messages 1 1 0 (0.0%) 1 ✅
Codex fleet shell snapshot HTTP decoded 42.4 KiB 42.4 KiB 0 B (0.0%) 53.7 KiB ✅
Codex fleet shell snapshot rows 27 27 0 (0.0%) 27 ✅
Codex fleet shell subscribe WebSocket decoded 65 B 65 B 0 B (0.0%) 200 B ✅
Codex fleet shell subscribe WebSocket messages 1 1 0 (0.0%) 1 ✅
Codex fleet shell resume check HTTP decoded 18 B 18 B 0 B (0.0%) 100 B ✅
Codex fleet shell resume check rows 0 0 0 0 ✅
Codex stale shell reload HTTP decoded 42.4 KiB 42.4 KiB 0 B (0.0%) 53.7 KiB ✅
Codex stale shell reload rows 27 27 0 (0.0%) 27 ✅
Codex stale shell resume WebSocket decoded 140 B 140 B 0 B (0.0%) 400 B ✅
Codex stale shell resume WebSocket messages 2 2 0 (0.0%) 2 ✅
Codex stale todo list cold open HTTP decoded 72.4 KiB 72.4 KiB 0 B (0.0%) 83.0 KiB ✅
Codex stale todo list cold open rows 75 75 0 (0.0%) 75 ✅
Codex stale todo list cold open nodes 2 2 0 (0.0%) 4 ✅
Codex stale todo list cold open subagents 2 2 0 (0.0%) 4 ✅
Codex stale todo list cold open plans 1 1 0 (0.0%) 2 ✅
Codex stale todo list cold open providerTurns 1 1 0 (0.0%) 2 ✅
Codex stale todo list cold open checkpoints 1 1 0 (0.0%) 2 ✅
Claude short turn resume WebSocket decoded 10.6 KiB 10.6 KiB 0 B (0.0%) 12.7 KiB ✅
Claude short turn resume WebSocket messages 2 2 0 (0.0%) 2 ✅
Claude long turn running, cold open HTTP decoded 283.3 KiB 283.3 KiB 0 B (0.0%) 302.7 KiB ✅
Claude long turn running, cold open rows 75 75 0 (0.0%) 75 ✅
Claude long turn running, history page HTTP decoded 93.4 KiB 93.4 KiB 0 B (0.0%) 107.4 KiB ✅
Claude long turn running, history page rows 75 75 0 (0.0%) 75 ✅
Claude long turn finished, cold open HTTP decoded 384.2 KiB 384.2 KiB 0 B (0.0%) 410.2 KiB ✅
Claude long turn finished, cold open rows 151 151 0 (0.0%) 151 ✅
Claude long turn finished, socket cold open WebSocket decoded 384.3 KiB 384.3 KiB 0 B (0.0%) 410.2 KiB ✅
Claude long turn finished, socket cold open WebSocket messages 2 2 0 (0.0%) 2 ✅
Claude long turn finished, socket cold open rows 151 151 0 (0.0%) 151 ✅
Claude long turn finished, work page HTTP decoded 188.1 KiB 188.1 KiB 0 B (0.0%) 200.2 KiB ✅
Claude long turn finished, work page rows 150 150 0 (0.0%) 150 ✅
Claude idle thread resume WebSocket decoded 66 B 66 B 0 B (0.0%) 200 B ✅
Claude idle thread resume WebSocket messages 1 1 0 (0.0%) 1 ✅
Claude fleet shell snapshot HTTP decoded 43.0 KiB 43.0 KiB 0 B (0.0%) 53.7 KiB ✅
Claude fleet shell snapshot rows 27 27 0 (0.0%) 27 ✅
Claude fleet shell subscribe WebSocket decoded 66 B 66 B 0 B (0.0%) 200 B ✅
Claude fleet shell subscribe WebSocket messages 1 1 0 (0.0%) 1 ✅
Claude fleet shell resume check HTTP decoded 18 B 18 B 0 B (0.0%) 100 B ✅
Claude fleet shell resume check rows 0 0 0 0 ✅
Claude stale shell reload HTTP decoded 43.0 KiB 43.0 KiB 0 B (0.0%) 53.7 KiB ✅
Claude stale shell reload rows 27 27 0 (0.0%) 27 ✅
Claude stale shell resume WebSocket decoded 140 B 140 B 0 B (0.0%) 400 B ✅
Claude stale shell resume WebSocket messages 2 2 0 (0.0%) 2 ✅
Claude stale todo list cold open HTTP decoded 73.4 KiB 73.4 KiB 0 B (0.0%) 83.0 KiB ✅
Claude stale todo list cold open rows 75 75 0 (0.0%) 75 ✅
Claude stale todo list cold open nodes 2 2 0 (0.0%) 4 ✅
Claude stale todo list cold open subagents 2 2 0 (0.0%) 4 ✅
Claude stale todo list cold open plans 1 1 0 (0.0%) 2 ✅
Claude stale todo list cold open providerTurns 1 1 0 (0.0%) 2 ✅
Claude stale todo list cold open checkpoints 1 1 0 (0.0%) 2 ✅

Baseline: e94db17 · PR result: 0b01bd0 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 108.5 KiB
  • Claude decoded thread snapshot: 108.8 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

@macroscopeapp

macroscopeapp Bot commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at 0b01bd0

Macroscope's review found this PR approvable — This change batches repeated PR-comment edits before notifying the agent and preserves pending edits across watch restarts. Its effects are limited to notification timing and host reads, with no new host-side writes or access changes.

You can add or adjust custom eligibility rules. Learn more.

@coderabbitai

coderabbitai Bot commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

Pull-request watches now distinguish new remarks from edits. They retain changing edits across reads and watch restarts, then report confirmed edits separately. GitCafe activity now supplies edit timestamps when a comment’s update time is later than its creation time.

Changes: Pull-request remark edit tracking

Layer / File(s) Summary
Edit timestamps and watch state contract
packages/source-control-gitcafe/src/server/gitCafePullRequestJson.ts, packages/source-control-gitcafe/src/server/gitCafePullRequestJson.test.ts, packages/contracts/src/threadPullRequest.ts
GitCafe remarks now include editedAt when updatedAt is later than createdAt. The watch state can store pending edits and IDs of remarks whose edits were reported.
Classify and report settled edits
apps/server/src/orchestration-v2/pullRequestWatch.ts, apps/server/src/orchestration-v2/pullRequestWatch.test.ts
The evaluator waits for an edit timestamp to remain unchanged before reporting an edit. It labels edits separately from new remarks, suppresses repeated edits to a reported remark during the 10-minute settle interval, and clears held edit state when the PR head changes. Tests cover settling, new remarks, and head changes.
Read, persist, and resume pending edits
apps/server/src/orchestration-v2/PullRequestWatchReactor.ts, apps/server/src/orchestration-v2/Orchestrator.ts, apps/server/src/orchestration-v2/runtimeLayer.test.ts
Pending edits trigger reads and cache invalidation. Watch equality includes edit state, and a restarted watch copies that state forward. Runtime tests cover quiet fingerprints, restarts, and paginated review threads.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix · Severity of issue fixed: Medium

Sequence Diagram(s)

sequenceDiagram
  participant Provider
  participant GitCafePullRequestJson
  participant PullRequestWatchReactor
  participant evaluatePullRequestWatch
  Provider->>GitCafePullRequestJson: Supply comment timestamps
  GitCafePullRequestJson->>PullRequestWatchReactor: Return remarks with editedAt
  PullRequestWatchReactor->>evaluatePullRequestWatch: Evaluate remarks and watch state
  evaluatePullRequestWatch-->>PullRequestWatchReactor: Return pending edit state or edited remark change
Loading

Suggested reviewers: shivamhwp, t3dotgg, maria-rcks

Merge Risk: Minimal · up to 0b01b

This change holds edits to already-reported PR comments until they settle, so bot countdown rewrites no longer wake the agent repeatedly. No concrete merge-blocking defect was found, and the one open concern about stale comment reads has no supporting evidence.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title says a PR watch reports a comment's edit once per head, which is the bug the change fixes. It names the server component and the main behavior. The PR objectives describe the same change: ed…
Description check Passed The description follows the template. It has a problem statement, a change section that explains the approach and tradeoffs, a scope section that links issue #30342, and a verification section with fo…
Linked Issues check Passed [#30342] The issue requires that a PR watch not report an already-delivered comment again as a new comment on each edit. It allows either skipping edits or labelling and coalescing them, for example b…
Out of Scope Changes check Passed All listed changes support edit detection and settling for PR watches. Orchestrator.ts, PullRequestWatchReactor.ts, pullRequestWatch.ts, and their tests implement the held-edit behavior. threadPullReq…
Full details: Title check

Explanation

The title says a PR watch reports a comment's edit once per head, which is the bug the change fixes. It names the server component and the main behavior. The PR objectives describe the same change: edits already reported on the current head are now held and delivered after a settle period, not reported once per head. The title's wording is a little misleading, because the fix is the opposite of the literal wording: it lets later edits through instead of limiting them to once per head. Still, it refers to the right area and the right behavior, so it is a pass.

Full details: Description check

Explanation

The description follows the template. It has a problem statement, a change section that explains the approach and tradeoffs, a scope section that links issue #30342, and a verification section with focused test results. It also names the agent and harness that did the work. Two items are weak: the Scope section does not say the change is a bug fix for a bug, and the Evidence section is still pending, with a note that it must be filled before merge. The description also repeats the #30674 supersession note twice. These gaps are minor, so the description is mostly complete.

Full details: Linked Issues check

Explanation

[#30342] The issue requires that a PR watch not report an already-delivered comment again as a new comment on each edit. It allows either skipping edits or labelling and coalescing them, for example by delivering only the final state. The change in pullRequestWatch.ts meets the coalescing option. A first edit on a head settles after one unchanged read. Later edits to an already-reported remark are held until editedAt is 10 minutes old, then delivered once with the final text. Changes are tagged with edited: true on PullRequestWatchChange, and edited remarks use a distinct summary. The remarkEditsTold and pendingEdits fields in packages/contracts/src/threadPullRequest.ts store this state. PullRequestWatchReactor.ts forces a read while edits are pending, so held edits are delivered. Orchestrator.ts carries the state across unwatch and rewatch. The test changes in pullRequestWatch.test.ts and runtimeLayer.test.ts cover the countdown case, repeated edits, and a finished review after a push. The reproduction steps in the PR description were not executed. Real-app evidence is still pending. This is not a coding requirement, so it does not change this result. Evidence is taken from the PR summary and the stated test results. No repository inspection was made.

Full details: Out of Scope Changes check

Explanation

All listed changes support edit detection and settling for PR watches. Orchestrator.ts, PullRequestWatchReactor.ts, pullRequestWatch.ts, and their tests implement the held-edit behavior. threadPullRequest.ts adds the persisted fields that this behavior needs. runtimeLayer.test.ts covers the same behavior end to end. The gitCafePullRequestJson.ts and test changes derive editedAt from updatedAt only when it is later than createdAt. This supplies the edit timestamp that the watch uses, so it is part of the same feature. No unrelated change was found in the summary. No repository inspection was made, so the result rests on the summary.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Usage-based review receipt

Note

This review was completed with usage-based billing: files reviewed beyond your plan's included limits are billed at $0.25/file. View usage-based billing.

Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between e94db17 and 0b01bd0.

Files selected for processing: 8

  • apps/server/src/orchestration-v2/Orchestrator.ts
  • apps/server/src/orchestration-v2/PullRequestWatchReactor.ts
  • apps/server/src/orchestration-v2/pullRequestWatch.test.ts
  • apps/server/src/orchestration-v2/pullRequestWatch.ts
  • apps/server/src/orchestration-v2/runtimeLayer.test.ts
  • packages/contracts/src/threadPullRequest.ts
  • packages/source-control-gitcafe/src/server/gitCafePullRequestJson.test.ts
  • packages/source-control-gitcafe/src/server/gitCafePullRequestJson.ts

Limit details: You’ve used all 10 included reviews currently available.

@juliusmarminge

Copy link
Copy Markdown
Member Author

The automated pipeline stopped on this pull request and will not merge it:

the fix is written and reviewed, but proving it needs something no worker has:

  • Permanent blocker: the live reproduction requires a second authenticated GitHub account to post and repeatedly edit a comment while the first account watches; only one authenticated account was available, and this job prohibits posting forge comments.
  • All runtime expectations remain unverified: repeated-edit suppression, notification after a push, and delivery of new comments. No captures were produced or uploaded; ghdrop was not attempted.
  • Source inspection shows the supplied head permits subsequent edits on the same head to notify again after a ten-minute settle, which differs from the stated once-per-head expectation. This was not verified at runtime.

It stays open for a person to finish or close.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M 30-99 changed lines (additions + deletions). vouch:trusted Author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PR watch reports an edited comment as a new comment on every edit

4 participants