delegate_task reports a transient provider error as failed while the child keeps running and later completes #36147

Activity

1. juliusmarminge commented on Nov 13, 2026

   

   juliusmarminge

   on Nov 13, 2026

   Member

   More actions

   @joshka In the first delegated child thread (the one that returned the 403), what started the run that finished at 10:00:43: a message you sent or queued there, a 'background task finished' notification, a delegated sub-task's completion, or a message from the parent thread?
2. juliusmarminge

   added

   needs more infoInitial triage showed no bug. Awaiting more infoInitial triage showed no bug. Awaiting more info

   on Nov 13, 2026
3. joshka commented on Nov 13, 2026

   

   joshka

   on Nov 13, 2026

   Last edited by joshka

   ContributorAuthor

   More actions

   A message I sent in the child thread myself. The child's run 2 starts from a continue I typed there in the macOS desktop app at 09:53:22 (recorded as creationSource: web), about 14 seconds after I typed continue in the parent. The parent's continue is what made it retry with a second delegate_task, so one continue in each thread gave two children doing the same work.

   So the child didn't continue on its own; I resumed it by hand. I wonder if this was PEBKAC, but oddly so: the UI showed every piece as failed (the parent's turn, the delegated-task card, and the child's own turn, see #36149), so continuing both looked like the obvious move. What still looks wrong:

   • The delegated task stays failed after its child is continued, and the run that completes is reported to the parent as "Delegated task … finished" under node:delegated-task:36dd8cef-…, not the node id delegate_task returned. The parent can't connect the two, so it doesn't know its retry is redundant.
   • A transient provider failure still ends the task right away (as in [Bug]: Delegated task fails on a passing provider error instead of continuing the child, so a retry repeats finished steps #29988), which is what invites the retry in the first place.

   🤖 Claude Opus 5.5
4. juliusmarminge

   removed

   needs more infoInitial triage showed no bug. Awaiting more infoInitial triage showed no bug. Awaiting more info

   on Nov 13, 2026
5. joshka commented on Nov 13, 2026

   

   joshka

   on Nov 13, 2026

   ContributorAuthor

   More actions

   I suspect this is something that a coding agent could reasonably do "find the bits that are confusing about this when failure conditions happen and make the next steps clear regardless of how each failure happens" or something like that. hill climb against unhappy path responses and inject failures into these so they do the right thing?
6. jhillyerd commented on Nov 14, 2026

   

   jhillyerd

   on Nov 14, 2026

   More actions

   This happens to me 100% of the time with 0.1.7 running a Pi agent, triggered by my smart-loop skill which is based on tstack/sol-loop

   isolation shared
   mode wait
   role review
   target {"model":"zai/glm-5.3","options":[{"id":"thinking","value":"high"}]}
   task Run `git diff 
   [...]
   timeoutMs 1200000
   title smart-loop round 1: codemode await/all_settled review
   fetch failed
   
7. joshka commented on Nov 14, 2026

   

   joshka

   on Nov 14, 2026

   ContributorAuthor

   More actions

   Ah good - likely worth getting a better repo then. For me this was a onetime thing, and felt worth fixing as it was duplicated work done on two threads to be preventative, but if you've got a dependable repro then I'll leave that to your clankers to eek out some way to write up the state better.
8. jhillyerd commented on Nov 14, 2026

   

   jhillyerd

   on Nov 14, 2026

   More actions

   It seems a bit more reliable since upgrading to 0.1.9, but I managed to catch a failure and run the triage skill on it...
9. jhillyerd commented on Nov 14, 2026

   

   jhillyerd

   on Nov 14, 2026

   More actions

   Note

   🤖 GLM 5.3 responding on behalf of jhillyerd

   Reproduced the same hazard class from a second, deterministic trigger: the caller's own MCP request timeout on a held mode:"wait" response — no provider outage required.

   On T3 Code desktop/server 0.1.9 (Linux x64, Node v26.12.1, pi provider, parent and child both glm-5.3), a delegate_task with mode:"wait", timeoutMs: 900000 held its HTTP response while the child ran. The pi harness's MCP client (Node fetch/undici, default 300 s headers/body timeout) aborted at almost exactly 300 s. Server trace: McpServer tools/call → delegate_task span 20:12:47.126Z → 20:17:48.044Z, 300,918 ms, exit Interrupted; the child thread was created at 20:12:47.143Z (17 ms after the call began) and completed normally at 20:18:47Z.

   What the parent saw: the tool call failed with the raw transport error fetch failed — no taskId or childThreadId, so it could not tie the error to a task and had to find the child via t3_thread_list. A parent that treated the failure as real would have started a second reviewer on the same diff — exactly the duplicate-work outcome here. Recovery worked only because a project skill explicitly says a fetch failed return is not a dead delegate.

   Differences from the original report: the server never reported the task failed — nothing restarted the child; only the response connection died (server-side wait constants allow 10–60 min, far past the client's 300 s). And the "completion under a different node id" part did not reproduce on this path: the child's completion was delivered ~1 min after the transport failure under the same node:delegated-task:command:mcp:<session>:delegate-task:<task-id> as the dispatch, mid-turn.

   This is the same client-timeout-vs-server-wait mismatch as #35714 (Cursor ~60 s) and #30006, confirmed here with server-side span timing. It argues for the expected behavior in both issues: return the taskId/childThreadId before the client's request timeout (or cap/downgrade wait per client), so an error return always leaves the parent a handle to the child.
10. jhillyerd commented on Nov 14, 2026

    

    jhillyerd

    on Nov 14, 2026

    More actions

    Note

    🤖 GLM 5.3 responding on behalf of jhillyerd

    Picking up the suggestion upthread — "find the bits that are confusing about this when failure conditions happen and make the next steps clear regardless of how each failure happens" — here's a walk of the delegation failure surface, verified against main @ f56f27e6 (2026-11-14) and v0.1.9 (24e77473). Grouped by whether the next step is already clear.

    Already clear on main:

    Failure	What the caller gets	Why it's clear
    Dispatch refused (no active parent run, bad target, worktree/project errors, provider disabled)	Structured error, nothing created	Retry with fixed input; no orphan state
    timeoutMs expires server-side	waitTimedOut: true + next: "Completion wakes this thread. Do not wait on it or poll it…"	The result tells you what happens next
    Child pauses for user input/approval	hasPendingQuestion / pendingApproval end the wait with flags set	State is explicit
    Server restart mid-wait	heldForRestart keeps workState: "working" until the result is delivered	task_status doesn't lie about liveness

    Still confusing — the rows where an error leaves the caller without a handle or a next step:

    1. mode:"wait" outlasts the client's request timeout (pi/undici ~300 s here; Cursor ~60 s in [Bug]: delegate_task mode "wait" outlasts Cursor's ~60s MCP timeout; the child starts but the parent never gets its taskId #35714; Codex 30 s in [Bug]: delegate_task with mode: "wait" triggers continuous 30s polling loop in cell-based agent harnesses #30006). Caller gets a raw transport error — no taskId, no childThreadId — while the child runs on. Since fix(server): a steer during a delegate_task wait no longer drops the child's result #27387 (Nov 11, in the Nov 13+ nightlies, not in v0.1.9) an interrupted wait upgrades the wake uninterruptibly, so the result still reaches the parent; what's missing is the handle at the moment of failure. The caller's only recovery is hunting via t3_thread_list, and a blind retry duplicates the work.
    2. A transient provider error fails the child's first run — this issue's core, unchanged on main: one failed run ends the wait with status: "failed", workState: "result_available", the provider error as summary. Nothing distinguishes it from a real failure, nothing says the child thread is idle-but-continuable (t3_thread_send to childThreadId) or how to stop it (task_cancel) — that's exactly what invited the reporter's retry.
    3. Child continued after the task failed (the reporter's double-continue): main now surfaces latestTerminalRunId/Status/Summary on task_status, which is the right window into it — but the "Delegated task … finished" notification can still arrive under a different delegated-task node than the one delegate_task returned, so a parent matching on node id still can't tie them together.

    A retry mechanism already exists, but nothing advertises it: delegate_task is idempotent per clientRequestId — requestKey uses it verbatim (OrchestratorMcpService.ts), and dispatch replays an accepted receipt's stored events instead of re-executing (replayCommandReceipt, Orchestrator.ts) — same taskId, no second child. But omitted clientRequestId ⇒ random key ⇒ a retry duplicates. In the v0.1.9 incident the agent passed none, so the receipt path was unavailable and the t3_thread_list hunt was the only safe move. If the tool copy said "pass a stable clientRequestId when you may need to retry a wait call; after a transport failure, re-send the identical call", rows 1 (and the restart case) would have a one-line correct next step. Cheap win, no protocol change.

    For the "inject failures and hill-climb" idea, the harness already exists — OrchestratorMcpToolkit.integration.test.ts, TaskCancelNativeSubagent.integration.test.ts, OrchestratorMcpService.activity.test.ts. The four injections worth writing first, one per confusing row: (a) kill the HTTP response mid-wait; (b) fail the child's first provider call transiently, then let it pass; (c) continue a failed task's child to completion; (d) restart the server mid-wait. Assert each returns either a result, a handle (taskId/childThreadId), or an explicit next — that's the invariant that makes every row clear: no delegated-task error or terminal return without either the handle or the instruction.

    Related: #35714 (client-timeout row), #30006, #36149 (the UI half — everything showing failed is what made continuing both threads look like the obvious move).