Repository navigation
Description
Before submitting
- I searched existing issues and did not find a duplicate. Issue searches on this repo: "remote delegate", "delegated task", "linked environment delegate", "untilDelegatedWorkSettled", "remote child stale result", "background wake parent" and "Monitor wake subagent". Discussion searches: "remote delegate", "delegated task result", "linked environment delegate" and "monitor wake". I also read Resuming a completed remote delegate does not create a follow-up task or notify its parent #26740, Delegated task reports finished while its child thread still has running work #16603, [Bug]: Cancelling a background-only delegated task reports completed with an interim summary #16739 and Lineage shows a settled remote delegate as Done #26907. Resuming a completed remote delegate does not create a follow-up task or notify its parent #26740 is the closest: it covers the same missing follow-up when the parent sends a finished remote child a message. Here nothing is sent; the child's own background wake starts the turn. Delegated task reports finished while its child thread still has running work #16603 and [Bug]: Cancelling a background-only delegated task reports completed with an interim summary #16739 are about local children, and Lineage shows a settled remote delegate as Done #26907 is about the Lineage label.
- I included enough detail to reproduce and investigate the problem.
Area
apps/server
Steps to reproduce
You need two linked environments: A (origin) and B (destination). B runs the Claude provider; any model works (I used claude-haiku-5-5).
-
From a thread on A, call
delegate_taskwithtarget.environmentId= B,mode: "async", and this task:Throwaway probe, change nothing. (1) Run the Bash command
sleep 25; echo OKwithrun_in_background: true. (2) End your turn right away with the single word WAITING. Do not wait, poll or sleep yourself. (3) When the background command's finished notice wakes you, reply with the single word FINAL. -
End the parent's turn and wait a few minutes.
-
Repeat, with two changes to the task. Replace step (1) with "start a Monitor with command
sleep 25; echo OK, non-persistent, timeout 120000 ms". Replace step (3) with "When the monitor's finished notice wakes you, reply FINAL". -
Controls: repeat steps 1 and 3 without
target.environmentId, so the child is local.
Expected behavior
The parent gets the child's final answer, FINAL, whichever machine the child runs on. It can come as the task's result, or as a follow-up task with its own completion wake. Since #23542, a local child's background-wake turn is delivered the second way.
Actual behavior
On B, the child thread does exactly what it was told:
- run 1 replies
WAITING; - the background command (or Monitor) finishes, and T3 starts run 2 with a
Command "…" finished(orMonitor "…" finished) notification; - run 2 replies
FINAL.
On A:
- While the background command runs,
task_statusreportsstatus: running,workState: working. That part is right. - The task then completes with
summary: "WAITING". That is run 1's text. - Run 2's
FINALis never delivered.- No follow-up task is created, and no second completion wake reaches the parent.
- A
task_statusread taken after run 2 on B had postedFINALstill returnedWAITING. - The parent thread never received a later wake from either remote probe.
| Child | Waits with | Task result on A | Child's run 2 (FINAL) |
|---|---|---|---|
| Remote | background command | WAITING |
posted on B, never delivered |
| Remote | Monitor | WAITING |
posted on B, never delivered |
| Local | background command | WAITING |
delivered as a separate provider-continuation follow-up task, with its own "Delegated task … finished" wake |
| Local | Monitor | WAITING |
posted in the child thread, never delivered |
Every machine should behave like the local background-command row. In the local Monitor case, the task completed with WAITING about 0.1 s before the Monitor's wake run was requested. Both timestamps come from the same server, and no second result followed.
Here is the remote background-command probe on each machine, in seconds from delegate_task. The two machines' clocks aren't synchronised to the millisecond.
B t+0.2 run 1 starts
B t+14.4 background command started
B t+16.0 run 1 replies WAITING; run 1 completes at t+16.3
A t+25.7 task_status -> running / working
B t+40.2 "Command … finished" notification; run 2 requested
B t+42.4 run 2 replies FINAL; run 2 completes at t+42.8
A t+42.8 task_status -> completed / result_available, summary "WAITING"
A later no follow-up task, no second wake
Cause
From reading the source at cadf7d5fe3. None of the code cited here changed between that commit and 0.1.6.
- A remote child gets no follow-up.
- fix(server): notify parents after delegated background runs #23542 made a provider wake in a local child start a follow-up task that the parent hears back from. It runs through
mayStartSubagentFollowupinOrchestrator.ts, then therelationshipToParent === "subagent"check. - A remote child's thread on B has
relationshipToParent: null, so that path never runs. - On A,
RemoteDelegation.followOncecompletes the task once (complete(…, "ran-out")), and then the follower ends. - Nothing on either side picks up a later run on that thread. This looks like the provider-wake counterpart of Resuming a completed remote delegate does not create a follow-up task or notify its parent #26740.
- fix(server): notify parents after delegated background runs #23542 made a provider wake in a local child start a follow-up task that the parent hears back from. It runs through
- Why the follower takes run 1 (inferred).
followOncewaits withuntilDelegatedWorkSettled: true.- On B,
delegatedTaskProgress(packages/provider-core/src/server/subagentProjection.ts) returnsresult_availablewhen no work run is active andpendingBackgroundTasksis empty.resultRunis then the latest finished run. - It looks as if the finished command leaves
pendingBackgroundTasksbefore its wake run is created. For a moment, then, the thread reads as settled on run 1. I haven't confirmed this ordering in the Claude adapter. - What I did see once: a
t3_thread_waitwithuntilDelegatedWorkSettledon the remote child's thread returnedresultRun= run 1, and run 2 then produced the real answer.
- Runs that a Monitor wakes are excluded.
delegatedTaskProgressdrops every run whose triggering message hasnotification.source.kind === "monitor"(monitorRuns).- fix(server): notify parents after delegated background runs #23542 treats monitor updates the same way, because they "report progress rather than asking for new work". That fits PR-watch and output-progress notices.
- But when a Claude
Monitorfinishes, that is the agent's chosen way of waiting for its own work, and the run it wakes holds the answer. - As written, that run can never be a task's result or start a follow-up. This applies to local children as well as remote ones; see the local Monitor row above.
Impact
Major degradation or frequent failure. Any remote child whose work outlasts one turn loses its result. For us that's every long test run delegated to a second machine: the agent's foreground command limit ends the first turn while the suite is still running in the background. The parent then either acts on "still running…" as if it were the answer, or waits for a wake that never comes. It hit two real runs in one morning. One result went unseen for about 45 minutes and the other for about 80, until someone noticed by hand.
Version or commit
Reproduced on 0.1.5 (6d450a4284) on both servers. Both servers now run 0.1.6 (2359643cff), where the code paths above are unchanged. I haven't re-run the probes on 0.1.6.
Environment
Origin: Linux x64 server. Destination: Windows x64 desktop. Both children run Claude Code; the probes used claude-haiku-5-5. Seen with both background Bash commands and the Monitor tool.
What doesn't work
- The child marks the work as awaited. One remote child called
t3_background_waiton its background task before ending run 1. The outcome was the same: the task completed withWAITING, and run 2'sFINALwas never delivered. This was observed once. - The parent waits on the thread instead.
t3_thread_waitwithuntilDelegatedWorkSettledreturned run 1. This was also observed once; see Cause, point 2. - The parent sends the child a follow-up to collect the result. A
t3_thread_sendto a remote child creates no task, so the reply never wakes the parent. That is Resuming a completed remote delegate does not create a follow-up task or notify its parent #26740.
Workaround
The remote child reports back by itself:
- The parent gives the child its own thread ID and environment ID.
- In the turn that has the real answer, the child's last action is a
t3_thread_send(modequeue) to the parent, carrying the result line. - The parent treats a "Delegated task … finished" wake from a remote child as final only if it contains that result line. Otherwise it ends its turn and waits for the child's message.
- A periodic check reads the child's thread with
t3_thread_read, nevertask_status, to recover any result that still slips through.
Suggested fix
- Follow later runs. Treat a provider-wake run on a remote task's thread as a follow-up, the way fix(server): notify parents after delegated background runs #23542 does for local children.
- B can't record it, because the thread has no parent there. So A's follower should keep watching the remote thread after it completes the task, until B reports no pending background work and no queued wake.
- It should record a follow-up task for each later run and wake the parent when that run ends.
- fix(server): a follow-up to a finished remote task is a new task that wakes the parent #26776 (open, the fix for Resuming a completed remote delegate does not create a follow-up task or notify its parent #26740) adds a
RemoteDelegation.followUppath that records a follow-up for a run on a finished remote task's thread. That path could probably serve both triggers.
- Count finished Monitors. In
delegatedTaskProgress, count a run woken by a finished agent-started Monitor as a work run, and inmayStartSubagentFollowupas a follow-up. Keep progress notices and PR watches excluded. - Add tests:
- in
RemoteDelegation.test.ts, a remote child whose background wake produces a second reply, which reaches the parent; - a local child woken by a finished Monitor, whose reply is delivered.
- in
I'm happy to file a PR for this if you approve the direction.
Related: #26740, #26776, #23542, #16603, #22651
Filed by Claude Opus 5.5 via Claude Code.
Activity
royalaid commentedon Nov 7, 2026
A note on coverage, in case it helps before #31744 closes this: #31744 fixes the background-command case reported here (it follows the child's thread after the first result, and a run started by the child's own background wake counts as a follow-up), but it skips runs that a Monitor or scheduled task started (
Orchestrator.ts:714at its head 585db18, and its test "waitForThread with afterRunId skips runs a monitor or scheduled task started"). The finished-Monitor reproduction in this issue would still lose its final reply, so this issue may need to stay open for that case after #31744 lands.Evidence from our side for the background-command case: a linked child answered "waiting on the test suites", then woke itself a minute later when its background suites ended and wrote the final verdict; the parent got the interim answer and never the verdict. We patched it the same way (an
afterRunIdwait after the result); our tests for a late reply, a restart during the wait, and the end of the wait window fail on main and pass with that change. We have not tested a Monitor-started run.nuuls commentedon Nov 11, 2026
Still happens with the current explicit-wait model, where the child marks the command awaited. This is on a fork build (
0.1.9-fork.20261110) branched frommainat 6c9adfb. The fork's only changes are to Codex usage limits, so the code paths involved here matchmain.Setup: Linux parent server, Windows child server, Claude Opus 5.5 parent and child, async
delegate_taskwithtarget.environmentId. The child started a ~7 minute background Bash command and calledt3_background_waiton it (matched). It then ended run 1 with "…I'll pick up once that job finishes."UTC times. The two machines' clocks aren't synced to the millisecond.
task_statusreturnscompleted/result_available.Command "…" finishednotification arrives and run 2 is requested.So the awaited hold works until the command exits. Then the follower's
untilDelegatedWorkSettledwait settles on run 1, about a second before the wake run exists.This confirms the ordering this issue infers under "Why the follower takes run 1". In
ClaudeAdapterV2, thetask_notificationbranch that runs while no turn is active callsapplyBackgroundTaskRosterMessage, which drops the task and publishes the empty roster right away.bufferWakeMessageoffers the wake continuation only once a root init, user, assistant or result frame arrives.Two open PRs each fix part of this, on different machines:
juliusmarminge commentedon Nov 12, 2026
I tried to reproduce this on the current build and could not.
Commit a18a458 (origin/main). A live two-environment run with Claude Code on the child side was not possible: the VM has no first-party Claude provider. I reproduced the parent-side defect in apps/server/src/peer/RemoteDelegation.test.ts, using its existing fake linked environment ('box'). The test delegates an async task there, the box run ends with reply 'WAITING', and the parent's task completes with 'WAITING' (that assertion passes; a temporary log confirmed it). Then a later run on the same box thread replies 'FINAL' (laterReply plus laterFinished), and the test waits for any parent task carrying 'FINAL'. On main nothing ever follows the box thread after the first result, so the test hangs until the 120 s project timeout (
vp test run src/peer/RemoteDelegation.test.ts -t "background wake starts"). Code reading agrees: RemoteDelegation.followOnce calls complete(..., 'ran-out') once and the follower ends. The fix PRs #31744 (parent side, keeps following the thread) and #32223 (child side, keeps awaited Claude commands pending until their wake starts) are both still open; #26776, merged 2026-11-11, covers only follow-ups the parent sends.Closing as not planned. If it still happens for you, reply with the exact steps and the version you are on and it will be looked at again.
singlatushar07 commentedon Nov 15, 2026
Note
🤖 GPT-6.1 Sol responding on behalf of @singlatushar07
The human reporter encountered this during development and requested this investigation/report. This did not use
t3 triage.Additional live occurrence on stable 0.1.9, November 15, with an explicitly matched
t3_background_wait.Environment: Origin: macOS 26.2 arm64, Desktop/local server 0.1.9, Codex parent (
gpt-6.1-sol). Destination: linked Linux x64 server 0.1.9, Claude Agent child (claude-opus-5-5). Server versions were queried directly; Claude CLI/SDK version was not collected.Observed workflow: Parent uses async
delegate_tasktargeting the linked environment. Child starts a long test, registers its background Bash completion waiter usingt3_background_wait, and ends its turn. When the command finishes, the child resumes automatically, but the origin completes the original task with the earlier interim reply.Evidence: UTC timestamps from provider logs and read-only run/task records:
Clock skew between machines was not measured; cross-machine subsecond ordering is not established. The stale result and automatic continuation are independently confirmed. The first manual parent follow-up was at 13:23:34.881, after the premature completion.
Expected: Hold completion through the awaited background wake and deliver its resulting answer. Normal turn endings should not be reported as completed work.
Diagnosis/limits: Installed-bundle source suggests a background-roster/continuation settlement race. Exact internal ordering remains inferred; no controlled reproduction or regression test was run. #34568 merged after v0.1.9 and may address this; neither that fix nor current main/nightly was tested here.
Workaround: Inspect the destination thread's actual run state/final reply before trusting the origin's completed card. No T3 runtime changes or database writes were made.
Related: #32563 (closed in favour of this issue), #24297, #34568.
singlatushar07 commentedon Nov 15, 2026
Note
🤖 GPT-6.1 Sol responding on behalf of @singlatushar07
Ran
t3 triage0.1.9 with Codex (GPT-6.1 Terra) after posting the occurrence above. It found additional origin-side trace evidence for the original remote task:The task projection contains run 3's interim answer as its completed result. Clarification to the earlier comment:
completedAt=13:22:53.974is a stored projection field, not an observed transaction-commit timestamp; its difference from the traced dispatch time is unexplained. The first manual parent follow-up was later than both, at 13:23:34.881.The trace does not capture the wait response's
resultRunor the destination projection at that instant. The stored initialremoteChild.runIddoes not establish which run the wait selected. Destination and origin clocks were not verified synchronized, so cross-machine subsecond ordering is not established. The premature interim completion is observed; the exact settlement-race mechanism remains inferred.Release-pinned v0.1.9 source waits with
untilDelegatedWorkSettled, selects the returnedresultRunor a fallback, reads its last reply, then completes. #34568 merged after this release and preserves owed wake work through roster updates. It is relevant, but we have not tested that fix or reproduced on current main/nightly.Triage was read-only: no T3 runtime/database changes and no contact with or interruption of the active destination child. No raw provider logs or private paths are included here.