Repository navigation
feat(j5): a blocked persona escalates, and the Owner can allow it once - #937
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Warning Review limit reached
This review includes 68 billable files and costs up to $17.00. Or wait 45 seconds for your next included review. View limit details
|
tyler-barton-horizon
left a comment
There was a problem hiding this comment.
Automated review of this layer of stack pingdotgg#933 (diff against its base branch). Findings were not individually verified against the source, so please treat them as likely issues and push back on any that are wrong.
Mobile Inbox (apps/mobile/src/j5/inbox/InboxScreen.tsx, not in this diff): 'Allow this once' is added only on web. An Owner on mobile sees the escalation ask with only a text reply, and the tool text says a typed reply never allows it, so the agent stays blocked until the Owner opens web or desktop (AGENTS.md 'Hit every surface').
🤖 Generated with Claude Code
| ); | ||
|
|
||
| /** A seat let past its limit once spends that allowance as it launches. */ | ||
| const spendAllowances = (resolved: ReadonlyArray<ResolvedCrewLaunchSeat>) => |
There was a problem hiding this comment.
spendAllowances marks every allow_once grant 'used' before anything is recorded or reserved, and nothing restores it if the launch fails.
A Captain calls j5_request_crew_member with allow_once, and the seat cap or replaceSeat fails after spendAllowances ran. The escalation is now 'used', contrary to the tool's 'If the new member fails to launch, nothing changes'. On retry grantedWaiver refuses with 'already used', so the forward-only recovery crewProposalNextStep suggests can't work. With two seats in one batch, the first seat's grant is spent even if the second fails.
There was a problem hiding this comment.
Good catch on the wording, but spending it up front is on purpose: an allowance covers one launch attempt, and if that attempt fails the agent asks again (that's in the persona contract now). I added that exception to the j5_request_crew_member description in 41fd6ce so it doesn't say "nothing changes" anymore.
| }).pipe(Effect.mapError(refused)); | ||
| if (assignment.pickSource?.kind === "allow-once") | ||
| yield* (yield* AgentPersonaEscalations) | ||
| .use(input.allowOnce!) |
There was a problem hiding this comment.
invokeAgent spends the allow_once grant before service.delegateTask runs.
If delegateTask fails (worktree error, provider failure), a retry is refused in grantedWaiver with 'already used; each allowance covers one launch', and the agent has to escalate to the Owner again for a launch that never happened.
There was a problem hiding this comment.
Same as the Crew one, this is by design. Allow this once covers a single attempt, so a delegate that fails after it's spent comes back to me. I'm keeping it simple instead of rolling grants back.
| blockCode: "models_exhausted", | ||
| }); | ||
| } | ||
| if (agentPick) |
There was a problem hiding this comment.
(Refers to line 670, which is outside the diff.)
On a persona seat with a frozen model_selection, an agent pick never reads seat.allowOnce and never records an escalation.
This path skips pickAgentPersona and calls assertWithinCallerAccess directly. The error has no code, so it becomes a plain stateError with no escalation_id, and the grant is never applied. The same goes for resolveSelection's CrewLaunchModelNotEligibleError, so the Owner's grant can't help this seat.
There was a problem hiding this comment.
That path only runs for a persona seat whose runtime the person set on the roster card. Agents can't set model_selection or runtime_mode on a persona seat (CrewProposalService refuses it with "only the human can override its runtime"). A person's pick has no limits to waive, so there's nothing to escalate there, and the access assert on that path is belt and braces.
| ? yield* send() | ||
| : yield* sendClaimingAttachments(receiverThreadId, attachments, send); | ||
| if (escalationId !== undefined && result.exchangeId !== null) | ||
| yield* (yield* AgentPersonaEscalations).linkAsk({ |
There was a problem hiding this comment.
linkAsk runs after the message is sent and overwrites exchange_id (UNIQUE) with no check, so a link failure is reported as a failed send of a delivered message.
A follow-up on existing exchange X carrying a different escalation_id violates UNIQUE(exchange_id). The tool returns a SqlError though the ask was delivered, and the agent's retry with a new client_request_id duplicates the ask. Re-linking an escalation to a new ask also quietly removes the Allow button from the earlier, still-open ask.
There was a problem hiding this comment.
A follow-up to a person is refused before anything's sent, so an escalation always links to a brand-new exchange and the unique index can't collide. Re-linking only happens after the earlier ask is closed. The everyone-here corner moves the button to the newest ask, and I'm fine with that. I also made the fresh ask explicit in 6a388cc: an ask with escalation_id that would join an ask already open is refused before anything is written.
|
On the mobile point in the review summary: Allow this once is web and desktop only for now. We decided that for #785, and the docs say mobile shows the ask without the button. As of 83e250d, mobile does show J5's own record of the block as plain text under the ask. The button can follow if it's missed. |
83e250d to
0d6b550
Compare
0d6b550 to
bb486cc
Compare
bb486cc to
fc82748
Compare
The playbook staffing guidance told a Captain to propose a custom seat for a persona that was missing, disabled or blocked, which recreates a blocked persona as a custom worker. A stand-in seat is now only for a persona the server doesn't have; a disabled, invalid or blocked persona keeps its steps with the Captain, who escalates it. The j5_propose_crew description and the docs say the same. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In a live run, a Haiku agent told the person to reply "Allow this once" in words, which grants nothing. The refusal now says an Owner allows it with a button on the ask, and that a typed reply never does. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The rule that a refused persona escalates moves from the top of the delegation section to the end of the messaging section, beside the rules for asking the person, and the persona invocation text keeps its original "report it instead of working around it". No behavior change; a smaller change to the standing instructions. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Moving the blocked-persona rule dropped its note that the ask must come after clearing or settling any ask already open with the person. The server refuses a second open ask from the same agent, so the note is back. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
React Compiler doesn't support logical assignment, so it skipped the button and the coverage test failed. The request id is assigned without `??=`. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two open asks, one linked to a persona escalation: only that item carries `escalation`, with its persona, waivable limit and status. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The grant is spent just before the worker is created, so it covers one launch attempt; if that attempt fails, the agent escalates again. The persona contract, the Inbox definition and the user guide say so instead of implying the grant lasts until a worker exists. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
….Service Drops the standalone AgentPersonaEscalationsShape for the effect-services convention; its methods are typed as AgentPersonaEscalations["Service"]. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A retried grant whose reply was lost was refused as "already allowed once". Now an already granted escalation replays the same answer by its command id and succeeds, while a different press is refused by the answered ask, so a grant interrupted between its two writes is finished by the same retry. The fixed reply named no one, so a Captain that asked for its member was told to retry a grant only the member can use. It now names the agent whose launch was refused and tells a Captain to pass the id on; the guidance says the same. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The dock reused one button as the shown ask changed, so an error and a pending request id from one ask carried to the next. It is keyed by exchange now. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The button's wording for each limit moves next to ROUTE_FAILURE_TEXT as ALLOW_ONCE_LIFTS, so every client shares it. The server's text stays separate because it is written for the agent. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…_once The description said nothing changes when the new member fails to launch, but an allow_once it used covers that one attempt and is spent. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A2ASendService.send takes freshAsk: an ask that would join one already open with the receiver is refused before anything is written, telling the agent to clear or settle the open ask first. j5_send_message sets it whenever escalation_id is passed, so the guidance's "fresh ask" is enforced. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The escalation summary was appended to the agent's ask, which the Inbox renders as Markdown with raw HTML, so an unclosed fence or a <details> in the agent's text could hide it, swallow it, or fake one. The summary is now data on the Inbox item's escalation view, never part of the message, and shows in a "Recorded by J5" panel (J5RecordPanel, plain text) in the Inbox and the Captain's dock, and as plain text under the ask on mobile. The service's askSummary becomes authorizeAsk, which only checks who may ask. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After the restack, a Crew persona seat whose entries are all unchecked is refused as CrewLaunchModelNotEligibleError again, as #931's review made it, and the error carries the persona's block code so j5_propose_crew and j5_request_crew_member still return models_exhausted with the escalation. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A grant was checked against the caller and persona only, so a retry under the same escalation could pin a different unchecked model or ask a Crew seat for broader access than the refusal showed. The escalation now records its scope: the persona's definition digest, and each candidate the waiver lets run, by entry index, provider instance and model, with the access it would get. A retry is waived only for those candidates, only at no more than that access, and only while the definition is unchanged; anything else is refused and the grant stays unspent. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Crew prevalidation knew only missing and disabled personas, so a seat naming a persona whose definition is invalid was told it wasn't in the library, with no code and no repair message. The check now uses the same definition lookup as a launch, refusing with the shared code (persona_not_found, persona_disabled, persona_invalid with its repair message, library_empty) in the same text form. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…luded A grant's scope matched candidates on instance and model only, so a grant for one entry also covered the same model at another reasoning level. The scope now records each candidate's effective options, the level a pin chose among them where the entry leaves it to the spawner, and a retry is covered only by an entry with the same instance, model and options. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Crew preflight refused a missing, disabled or invalid persona without an escalation, so the Captain's ask about it carried no record from J5. It now records one, never waivable, and returns its escalation_id, so the ask shows the Recorded by J5 panel with no Allow button. A card the person reviews records nothing. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Since #924 an ask to a person always opens its own exchange, so freshAsk and A2AOpenAskExistsError have nothing left to refuse and go. What can still put an escalation on the wrong Inbox item is a regarding follow-up, so j5_send_message refuses escalation_id with regarding and says to open a new ask with expect_reply, intent and urgency. The guidance, the refusal text and the docs drop "fresh ask" for the same rule. The web Inbox page shows the Recorded by J5 panel before the follow-ups, as the dock and mobile do, and the escalation test support imports migration 54. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With several asks open to a person, a second ask with the same escalation_id moved J5's record and the Allow this once button off the ask that opened the Inbox item. authorizeAsk now refuses an escalation whose ask is still open, naming that ask and the two ways out: follow up there with regarding, or clear it and ask again. Once the ask closes a new one may carry it. A regarding follow-up to the ask the escalation already sits on drops the id and goes as an ordinary follow-up, since nothing moves; a follow-up to any other ask is still refused. The escalation test support runs the a2a migrations, which the open-ask read needs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…viders The Recorded by J5 panel said Allow this once "would let one launch attempt past a model Models for custom members doesn't check". It now says the launch would use a model that Models for custom members doesn't allow, or run with broader access than the agent that started it. A refused persona's reasons name each entry's provider as the app does (its instance's name, else Codex or Claude) instead of its raw id, using the providers the launch already read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fc82748 to
06f794d
Compare
Summary
When a persona couldn't start, it failed with a vague reason, or the agent quietly recreated it as a custom worker to get past its model or access limits. That defeats the point of the persona. And when I wanted to let it run anyway, there was no one-time way to say yes. Now a blocked persona stays blocked, says exactly why, escalates to me, and an Owner can answer Allow this once. Closes #785.
Six codes, the same everywhere.
delegate_task,j5_spawn_agent, Crew seats and additions all refuse withlibrary_empty,persona_not_found,persona_disabled,persona_invalid(with the repair message),models_exhausted(one reason per entry, including Personas: agents pick personas, and who picked sets the limits #781's limits), orlibrary_unreadable, all through one text helper (personaBlockText). J5's own tools return the code as the failure code.delegate_task's codes are upstream's fixed list, so there the code leads the message as[code], and I didn't edit upstream's list.The agent pick works per entry. Models for custom members and the caller's access are now checked per entry inside routing, so a pick moves on past an entry it can't use. Each candidate records every limit it fails, which is how the server knows whether lifting one limit lets some entry run. A pin is a list of one: a pinned model that can't run is refused and never falls back.
Escalation. Every refused launch an agent made gets a row in
j5_persona_escalation(migration 053), and the refusal carriesescalation_id. A Crew member tells its Captain. The Captain, or any caller outside a Crew, asks me withj5_send_messageandescalation_id. The server accepts the caller's own escalations, or its Crew members' for a Captain, and the Inbox shows the server's own record of the block in a separate Recorded by J5 panel, carried as data so the agent's text can't hide or fake it. The ask must be fresh: one that would join an ask already open with that person is refused. Outside agents can't ask, so they get the code and reasons but no row.Allow this once. If one limit stood in the way (
model-not-eligibleoraccess-above-caller), the ask shows an Allow this once button to Owners, in the web/desktop Inbox and in the Captain'sAdjudicationDock. It goes through a new Owner-scoped RPC (j5.inbox.allowPersonaOnce,AuthSettingsWriteScopeplusrequireOwner), answers the ask with a fixed reply, and marks the escalation granted. A text reply never grants anything. The agent retries withallow_onceon any of the four launch tools. That's accepted only if the grant is unused, for the same caller and persona, and the launch passes with just that limit lifted. It recordspickSource: { kind: "allow-once", waived }, and the grant is spent just before the worker is created. It covers one launch attempt: if that attempt fails, the agent escalates again, and the next launch meets the limits again. A provider that can't enforce the persona's access is never waivable.Guidance. The standing instructions and the four launch tool descriptions say never to recreate a blocked persona as a custom worker, how to escalate (as a fresh ask), and how to retry.
Playbooks (second commit). The playbook staffing rule used to propose a custom seat for a persona that was missing, disabled or blocked. Now a stand-in is only for a persona this server doesn't have. A disabled, invalid or blocked persona keeps its steps with the Captain, who escalates it. Bryant picked that option.
Review fixes from Tyler's review. A retried Allow this once replays and succeeds, and its fixed reply names the agent that must retry, so a Captain knows to pass the id to its member. The dock's button resets per ask. The button's wording lives in client-runtime. The
j5_request_crew_memberdescription now says a failed launch spends its allow_once. An escalation ask can't join an ask already open (freshAskonA2ASendService.send). J5's record now shows in its ownJ5RecordPanelrather than being appended to the agent's Markdown.Fixes from Jackson's panel review. A grant now covers only what the refusal showed: the definition it was refused under, and the model entries it unlocks with their access. So a retry can't pin a different unchecked model, ask for broader access, or use a grant after the persona changed, and a mismatch leaves the grant unspent. A grant matches the exact entry, reasoning level included. A Crew seat naming an invalid persona now gets
persona_invalidwith its repair message instead of "not in the library", and the refusal is recorded so the Captain's ask carries J5's record.Earlier fixes (third to sixth commits). The sixth rewrites one
??=in the button, which React Compiler can't compile; CI's coverage test caught it. The refusal text says Allow this once is a button and that a typed reply grants nothing; a live agent had asked me to type it. The guidance bullet also moved next to the rules for asking the person, keeping its note to clear or settle an ask already open (fifth commit).Live testing
Pass, at 346c2a9, on a
setup-agent-live-test.tshome (no flags), with 3 Claude Haiku 5.5 turns.@persona:, for the haiku-helper persona (Full access) to say hello as a subagent. The agent'sdelegate_taskwas refused: both entries would run with Full access, above its Supervised access. It said it wouldn't get around that with a plain child or custom worker, and it asked me with theescalation_id. The Inbox item showed J5's own record of the block and Allow this once. I pressed it as the Owner. The ask closed with the fixed reply, the agent retried withallow_once, and the persona ran once and said hello. In the test home's database, the child's assignment haspickSource: { kind: "allow-once", waived: "access-above-caller" }, and the escalation row isused.??=in the button for React Compiler, and I didn't re-test that live because the turn budget was spent. The evals below cover it.Evidence
agentPersonaLibrary.test.ts, "tells the six reasons apart, each with its own code": all six codes with their text, including the repair message for an invalid definition.agentInvocation.test.ts, "escalates an agent's blocked pick, and the Owner's grant lets one launch through":delegate_taskrefused with[models_exhausted]and an id; the retry is refused before the grant ("never a text reply"); after the grant it launches withpickSource: allow-once; the same id again says "already used"; a fresh launch is refused again.agentPersonaRouting.test.ts: "offers one waiver only where lifting that limit alone lets an entry run" (two failed limits means no waiver) and "never lets a grant put a persona on a driver that can't enforce its access".agentPersonaPick.test.ts, "a pinned entry that can't run escalates instead of falling back".agentPersonaEscalation.test.ts: a Captain can ask about its member's escalation and another Captain can't; a grant is bound to its caller and persona and is used once; an escalation with no single waiver can't be granted.ClientActionsService.test.ts, "allows a blocked persona once": the fixed reply, then granted; a second press, or one on an unwaivable escalation, answers nothing.CrewLaunchService.test.ts, "a Captain's blocked persona seat escalates, and its grant launches that seat once".ClientReadsService.test.ts, "puts an escalation only on the ask that carries it": of two open asks, only the linked one carriesescalation.handlers.test.ts:j5_spawn_agentreturnspersona_not_foundas its code, and an ask withescalation_idcarries the server's record.clientRpcPermissions.test.tsandwsRpc.test.tslist the new method as Owner-only.mcp-tool-useorchestration cases on base (d8847f6) and head (10ec2d2; 5acb8d4 only restores a clause the first head run already covered).delegates-then-new-thread-with-model2 of 6.thread-mention-without-link-after-readwas 4/6 on head against 5/6 on base.opus-review-too-via-delegateis a known flake and failed on both.delegates-then-new-thread-with-model(1/3, the variance shown above) andopus-review-too-via-delegate(1/3, the known flake).node --env-file-if-exists=../.env mcp-tool-use/run.ts --cases mcp-tool-use/cases/orchestration.jsonl --out runs/<name>(fromevals/)vp run check:changed: format, lint, knip, typecheck (32 packages) and tests (46 files, 609 tests) all pass.Not verified:
mcp-tool-useharness offers no J5 tools, so the case can't be written there. That's Evals: a J5-owned suite that offers J5's tools #936.UI changes
A new Allow this once button next to Answer, in the Inbox item and the Captain's dock. It only shows for an Owner on an ask carrying a waivable, still-open escalation.
Captured at 1280x800 on the live test above (346c2a9), as the Owner:
The Captain's dock button wasn't captured (see Live testing).
Merge Danger
Door: two-way. Migration 053 only adds a table; a revert leaves the table and any escalation rows behind unused.
pickSource: allow-oncewas already in the contract (#781).Blast Radius: every persona launch path (
delegate_task,j5_spawn_agent, Crew seats) and the Inbox and dock answer row. If the per-entry limits are wrong, an agent's persona pick could be refused or allowed wrongly. The refusal texts changed, so anything matching old strings would break.After merge: nothing.
Checklist
FORK.md(case text and file-table row) in this PR (none touched)docs/j5/product/upstream.md(none)AGENTS.md)docs/j5/product/and user docs rewritten where this changes themSurfaces walked:
j5_send_message; the button in the Inbox and the dock. The composer launch gets the codes too, with no escalation, since a person sees the error directly.escalationonHumanInboxItem, the six block codes, theaccess-above-callerroute failure code, and the new action RPC.Made with Claude Opus 5.5 in Claude Code (J5 Code).
🤖 Generated with Claude Code