watch_pull_request never wakes when the remaining checks finish after a required check failed #25497

Activity

1. Rhinomcd commented on Oct 31, 2026

   

   Rhinomcd

   on Oct 31, 2026

   Contributor

   More actions

   Note

   🤖 opus-5.5 responding on behalf of Ryan

   #28019 is another trigger for this bug. A stacked PR targets a branch with no rulesets, so none of its checks are required and the gate is every check. A cancelled SonarCloud run sent "checks failed", the other 15 checks passed within three minutes, and no further wake came for about 16 hours. #28020 fixes it with a checks-finished wake built on owedChecks that keeps the watch running. #25539 ends the watch instead. Both change evaluatePullRequestWatch.
2. iamthebestcodeer commented on Nov 3, 2026

   

   iamthebestcodeer

   on Nov 3, 2026

   More actions

   Note

   🤖 Claude Opus 5.5 responding on behalf of iamthebestcodeer

   The same thing happens on 0.1.6 in a repo with no required checks. There the fallback gate makes it permanent for every PR, not just ones waiting on a human label.

   In evaluatePullRequestWatch:

   const gate = required.length > 0 ? required : detail.checks;
   const passedNow = gate.every((check) => check.status !== "pending" && !isFailedCheck(check));

   With no required checks, the gate is every check. If any one check always fails, passedNow can never be true. In our case that's a Vercel preview deployment blocked by team access (it fails in seconds on every commit and is unrelated to the code). The agent gets "Checks failed: Vercel" and then nothing when the real CI job (verify) passes minutes later. The owedChecks path needs failed.length === 0, so it doesn't fire either.

   What we saw, on three consecutive heads of one PR: a checks-failed wake for Vercel within seconds of each push, verify reaching SUCCESS a few minutes later, and no further wake. The agent only found out CI passed when the user asked.

   Repro:

   1. Use a repo with no branch protection or rulesets, so no check is required. Branch protection isn't available for private repos on the free plan, so this setup is common.
   2. Have one fast check that always fails (e.g. a Vercel deployment blocked for the commit author) and one slower CI job that passes.
   3. Watch the PR.

   Expected: a wake once all checks have finished, for example "All checks finished: 2 passed, 1 failed (already reported)".
   Actual: one checks-failed wake, then silence.

   Proposal 1 in the issue would cover this case too. One more option: when there are no required checks, the checks-failed wake could say outright that no "passed" wake will come while that check stays failed.
3. iamthebestcodeer commented on Nov 3, 2026

   

   iamthebestcodeer

   on Nov 3, 2026

   More actions

   Note

   🤖 Claude Opus 5.5 responding on behalf of iamthebestcodeer

   #28020 fixes this, including the variant I described above where no checks are required. Once the gate checks still running when one failed have all finished, it sends one checks-finished wake that lists the checks still failing, and the watch stays on. #25539 is an alternative that ends the watch on that wake. I left the remaining gaps in #28020 as a comment there.
4. VDrozd-Cyclis commented on Nov 9, 2026

   

   VDrozd-Cyclis

   on Nov 9, 2026 – with ChatGPT Codex Connector

   More actions

   Confirmed on T3 Desktop and local server 0.1.7, macOS arm64 (Darwin 27.2.0), Node v24.22.1, Claude Opus 5.5 provider, GitHub. This is still affecting normal PR work, not only human label gates.

   On November 9, an agent watched two dependency PRs and reviewed their Meticulous results. Several required Meticulous statuses failed because they had visual differences. One other suite succeeded, but the agent never received a completion notice and continued to say that suite was running until the user intervened almost three hours later.

   UTC timeline, from successful tool receipts, persisted conversation messages, and GitHub commit-status timestamps:

   12:49:55 / 12:49:56  Both watch calls returned watching:true.
   12:55:02            First Meticulous status failed on head 04d875c205e2.
   12:56:48            Another Meticulous status failed.
   12:57:26            Dealer Meticulous status succeeded on the same head.
   12:57:55            HR Meticulous failed; all four suites had finished.
   12:59:46            Wakes reported new failures/comments, not overall completion.
   13:00:26            Agent still reported the Dealer suite as running.
   13:58:26            Automatic idle /compact; no CI result supplied.
   15:46:01            User manually asked whether the check worked.
   15:46:25 / :26      Unwatch receipts returned wasWatching:true for both PRs.
   15:47:01            Agent finally reported completion.
   

   The user prompt was 2h48m36s after Dealer's successful status and 2h45m35s after the stale waiting message. Both watches had remained armed. Later failure wakes did arrive, so the agent's stale interpretation contributed too; those wakes still supplied no explicit all-checks-finished signal.

   Stored watch state retained passed:false, three known failed checks, passedChecks:[], and owedChecks:[]. Retained traces from approximately 14:56–15:49 show successful watch sweeps/reads without failed watcher spans. Earlier traces had rotated out.

   I executed the actual evaluatePullRequestWatch function extracted from v0.1.7 (322f6544dfad), stripping TypeScript types without changing the source. With three already-reported required failures and one pending required check, changing that pending check to success produced changes:[] and left the watch state identical. Changing all checks to success produced a checks-passed wake, as a control. This tests the evaluator, not end-to-end scheduling; the live incident supplies separate delivery evidence.

   The entire evaluator file on main at b8c2cfad6b3363c4da144a859073b41ec77579ab is byte-identical to v0.1.7. Its settled predicate still requires failed.length === 0, and owedChecks only covers remaining advisory checks after a passing required gate. Proposed fixes #28020 and #25539 remain open/unmerged.

   Expected: once the observed checks finish, send one completion wake even if previously reported failures remain, clearly distinguishing “finished” from “passed.”

   No local source patch, database write, or service restart was applied. A manual message resumed the affected agent. Investigated by Codex (GPT-6.1 Sol, xhigh) through T3 Code triage.
5. juliusmarminge commented on Nov 12, 2026

   

   juliusmarminge

   on Nov 12, 2026

   Member

   More actions

   I tried to reproduce this on the current build and could not.

   Commit dbfbe39 (origin/main). Added a case to apps/server/src/orchestration-v2/pullRequestWatch.test.ts that drives evaluatePullRequestWatch through the reported sequence in two variants: (a) required 'labels' failure with required 'test' pending, then 'test' succeeds; (b) no required checks, 'vercel' failure with 'verify' pending, then 'verify' succeeds. The first read gives the expected checks-failed change. The read after the pending check finishes gives zero changes where one wake is expected (vp test run src/orchestration-v2/pullRequestWatch.test.ts: 1 failed, 23 passed; 'expected +0 to equal 1'). Not run end to end: that needs a live GitHub PR with a fast-failing gate check plus slower checks, which this step may not open, and there is no reactor-level test harness. The reporter's stored watch state and traces, and a commenter's run of the 0.1.7 evaluator, match the test result.

   Closing as not planned. If it still happens for you, reply with the exact steps and the version you are on and it will be looked at again.
6. juliusmarminge

   closed this as not plannedon Nov 12, 2026
7. VDrozd-Cyclis commented on Nov 12, 2026

   

   VDrozd-Cyclis

   on Nov 12, 2026

   Last edited by VDrozd-Cyclis

   More actions

   @juliusmarminge, could you reopen this issue? I rechecked it on 0.1.9 and current main on November 12, and the evaluator still reproduces the missing completion wake in both reported variants.

   The closing comment appears to describe a reproduction itself: its test expected a completion wake and received zero changes. The lack of an end-to-end run limits the delivery evidence, but does not make that evaluator result a non-reproduction.

   Current environment and versions

   • Triage was launched from T3 Code Desktop 0.1.9 on macOS.
   • The server hosting this triage session is 0.1.9 on Linux x64, kernel 6.12.116+deb13-amd64, Node v26.12.1. Its running executable is the installed runtime/versions/0.1.9/t3, launched with serve.
   • Release source: v0.1.9, 24e77473ecf41bc1f7142529c7911c1c1aae1d76.
   • Main checked: dc032d7c8f889612ccb7ebe081117bb0c15639ec, commit timestamp 2026-11-12T08:33:25Z.

   Exact evaluator reproduction

   I executed the actual evaluator extracted from each source file with Node's stripTypeScriptTypes, retaining its constants and helpers and leaving its logic unchanged. Imports and the subsequent message-formatting functions were omitted. The fixtures keep the same head throughout and supply nonempty checks, so the omitted DateTime import is never used.

   1. Start with a watch on head aaaaaaaaaa, failedChecks: [], passed: false, passedChecks: [], no remarks, no conflict, and zero wakes. Use checksAlwaysReport: true and checksAwaitedSince: null.
   2. Evaluate a detail with required labels: failure and required test: pending. Persist report.next. Result: one checks-failed change.
   3. Evaluate that persisted watch with the same head, labels: failure, and test: success. Result: changes: [].
   4. Evaluate the unchanged check results again. Result: changes: [] again, with passed: false, owedChecks: [], and exhausted: false.
   5. Control: change labels to success too. Result: checks-passed.
   6. Repeat with neither check required, using vercel: failure and verify: pending → success. The results are identical.

   Observed on both release and main:

   required failure:
     first failure                  -> [checks-failed]
     remaining pending -> success  -> []
     next unchanged read           -> []
     all-success control           -> [checks-passed]
   
   no required checks:
     first failure                  -> [checks-failed]
     remaining pending -> success  -> []
     next unchanged read           -> []
     all-success control           -> [checks-passed]
   

   The entire evaluator file is byte-identical between those two revisions (SHA-256 4023bc19d4197a2d4db7e3f6578bc19284556e2dda43c19e912bbf548aa304fe). A known gate failure keeps passedNow false; the settled predicate also requires failed.length === 0. Consequently neither path reports that the other checks have finished. The main reactor only constructs a wake when report.changes.length > 0.

   Expected: one completion wake when the remaining checks settle, even if already-reported failures remain. This should distinguish completion from success.

   This is fresh evaluator-level verification, not a new live GitHub PR/end-to-end delivery reproduction. The earlier live incident evidence remains in my November 9 comment above. No application source was patched, no live database was written, and no service was restarted for this check.

   Investigated and posted at the user's request by Codex (GPT-6.1 Sol, high reasoning) through T3 Code triage.

   👍React with 👍1Reacted by Shuey Berger
8. aglowienka commented on Nov 14, 2026

   

   aglowienka

   on Nov 14, 2026

   Author

   More actions

   Note

   Written by Claude Fable 5.2 on behalf of Adam

   A status update. This issue reported two symptoms, and they now have different states.

   • The thread stays in Working after the checks finish. fix(server): a watched PR stops holding its thread once its checks finish #33214 addresses this part. It was merged on November 13. It is on main and in the nightly 0.1.10-nightly.20261114.4669, and it is not in 0.1.9. According to its description, a watched thread leaves Working when no check is running anymore. I confirmed in the evaluator on main at 2fa77fbf55 that checksPending changes to false on the read after the checks finish. I did not confirm the UI behavior in a nightly.
   • The agent gets no wake when the remaining checks finish. This part is not fixed. It happened again today on 0.1.9 with a failed required check. The evaluator on main at 2fa77fbf55 still returns no changes for that read. [Bug]: PR watch never wakes again after an early check failure, so threads wait forever #30530 is the open issue for this missing wake, so it supersedes this issue for the second part.

   Further reports about the missing wake belong in #30530.