Repository navigation
feat(j5): who picked a persona sets its limits - #931
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Warning Review limit reached
This review includes 33 billable files and costs up to $8.25. Or wait 1 minute for your next included review. View limit details
|
tyler-barton-horizon
left a comment
There was a problem hiding this comment.
Automated review of this layer of stack pingdotgg#933 (diff against its base branch). Findings were not individually verified against the source, so please treat them as likely issues and push back on any that are wrong.
🤖 Generated with Claude Code
| // attempt is held to them. | ||
| if (persona === undefined && firstAttempt) | ||
| // A retry finishes the agent its first attempt created, so only a first attempt is picked. | ||
| const persona = |
There was a problem hiding this comment.
The persona check now runs after target-project resolution, so a spawn that will be refused has side effects first.
An agent in a Supervised thread calls j5_spawn_agent with scratch: true and a full-access persona. ensureScratchProject creates the scratch project, then the pick is refused with access-above-caller. Before this PR the persona was checked before any side effect.
There was a problem hiding this comment.
True, but it's the one shared 'No project' project, and ensureScratchProject is idempotent, so a refused spawn leaves no thread or folder behind, at most that project, which the next scratch spawn would make anyway. The pick needs the target project for the model lists, so I'd rather not split it. Leaving as is.
| failures: [], | ||
| }; | ||
| const modelSelection = agentPersonaModelSelection(provider, target); | ||
| if (input.eligible !== undefined && !input.eligible(modelSelection, provider)) { |
There was a problem hiding this comment.
The model-not-eligible route failure applies only at launch, so j5_list_personas / orchestrator_capabilities still advertise those personas as startable.
The new tool text tells agents to pick a fitting persona from j5_list_personas. An agent picks one listed as startable whose entries are all unchecked in Models for custom members, and the call is refused: a wasted round-trip per such persona, with no way to see the limit ahead of time.
There was a problem hiding this comment.
Fair. That's deliberate for this PR: the catalog has no caller, so it can't know whose limits apply, and the refusal names the reason for every entry. I've filed #940 to have j5_list_personas mark what the caller's own pick can't use.
After the restack, a Crew persona seat whose entries are all unchecked is refused as CrewLaunchModelNotEligibleError again, as #931's review made it, and the error carries the persona's block code so j5_propose_crew and j5_request_crew_member still return models_exhausted with the escalation. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After the restack, a Crew persona seat whose entries are all unchecked is refused as CrewLaunchModelNotEligibleError again, as #931's review made it, and the error carries the persona's block code so j5_propose_crew and j5_request_crew_member still return models_exhausted with the escalation. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After the restack, a Crew persona seat whose entries are all unchecked is refused as CrewLaunchModelNotEligibleError again, as #931's review made it, and the error carries the persona's block code so j5_propose_crew and j5_request_crew_member still return models_exhausted with the escalation. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A persona an agent picks is held to Models for custom members and to its caller's effective access; a persona a person picks (an @persona mention in the message that started the turn, the composer, or a roster card they approve) runs as saved. One pick module owns the check for j5_spawn_agent, delegate_task, Crew launches and Crew additions, and every assignment records its pickSource (person, agent, or allow-once for #785). Auto-approved custom Crew seats follow the same rule: no access set gets the Captain's effective access instead of Full access, and broader is refused. Closes #781 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… doesn't Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pick Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d does Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… and routing Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cked model Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ulting Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…oesn't count Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2064c40 to
cffd7b9
Compare
#937) * feat(j5): a blocked persona escalates, and the Owner can allow it once A persona that couldn't launch failed with a vague reason, and agents sometimes recreated it as a custom worker to get around its model or access limits. Every persona launch path (delegate_task, j5_spawn_agent, Crew seats and additions) now refuses with one of six codes and readable reasons: library_empty, persona_not_found, persona_disabled, persona_invalid, models_exhausted (one reason per entry), library_unreadable. An agent's pick works per entry, skipping entries outside Models for custom members or broader than the caller's access; a pin is a list of one and never falls back. A refused launch records an escalation (migration 053) and returns its id. The caller, or its Captain, asks the person with j5_send_message escalation_id; the server appends its own record of the block. When lifting one limit lets an entry run, an Owner gets "Allow this once" on the ask in the Inbox and the Captain's dock (new Owner-scoped RPC). It answers with a fixed reply and grants one retry with allow_once, recorded as pickSource allow-once; the next launch meets the limits again. A driver that can't enforce the persona's access is never waivable. Guidance and tool descriptions say never to recreate a blocked persona. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(j5): a playbook Crew stands in only for a persona the server lacks The playbook staffing guidance told a Captain to propose a custom seat for a persona that was missing, disabled or blocked, which recreates a blocked persona as a custom worker. A stand-in seat is now only for a persona the server doesn't have; a disabled, invalid or blocked persona keeps its steps with the Captain, who escalates it. The j5_propose_crew description and the docs say the same. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(j5): a refused persona's text says Allow this once is a button In a live run, a Haiku agent told the person to reply "Allow this once" in words, which grants nothing. The refusal now says an Owner allows it with a button on the ask, and that a typed reply never does. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(j5): the blocked-persona rule sits with the ask rules The rule that a refused persona escalates moves from the top of the delegation section to the end of the messaging section, beside the rules for asking the person, and the persona invocation text keeps its original "report it instead of working around it". No behavior change; a smaller change to the standing instructions. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(j5): a blocked persona's ask waits on any ask already open Moving the blocked-persona rule dropped its note that the ask must come after clearing or settling any ask already open with the person. The server refuses a second open ask from the same agent, so the note is back. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): Allow this once compiles under React Compiler React Compiler doesn't support logical assignment, so it skipped the button and the coverage test failed. The request id is assigned without `??=`. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(j5): the inbox puts an escalation only on the ask that carries it Two open asks, one linked to a persona escalation: only that item carries `escalation`, with its persona, waivable limit and status. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(j5): Allow this once covers one launch attempt The grant is spent just before the worker is created, so it covers one launch attempt; if that attempt fails, the agent escalates again. The persona contract, the Inbox definition and the user guide say so instead of implying the grant lasts until a worker exists. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(j5): the escalation service's interface lives in its Context.Service Drops the standalone AgentPersonaEscalationsShape for the effect-services convention; its methods are typed as AgentPersonaEscalations["Service"]. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(j5): Allow this once replays safely, and its reply names who retries A retried grant whose reply was lost was refused as "already allowed once". Now an already granted escalation replays the same answer by its command id and succeeds, while a different press is refused by the answered ask, so a grant interrupted between its two writes is finished by the same retry. The fixed reply named no one, so a Captain that asked for its member was told to retry a grant only the member can use. It now names the agent whose launch was refused and tells a Captain to pass the id on; the guidance says the same. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): the dock's Allow this once starts fresh on each ask The dock reused one button as the shown ask changed, so an error and a pending request id from one ask carried to the next. It is keyed by exchange now. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(j5): Allow this once's wording lives in client-runtime The button's wording for each limit moves next to ROUTE_FAILURE_TEXT as ALLOW_ONCE_LIFTS, so every client shares it. The server's text stays separate because it is written for the agent. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(j5): j5_request_crew_member says a failed launch spends its allow_once The description said nothing changes when the new member fails to launch, but an allow_once it used covers that one attempt and is spent. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(j5): an escalation ask never joins an ask already open A2ASendService.send takes freshAsk: an ask that would join one already open with the receiver is refused before anything is written, telling the agent to clear or settle the open ask first. j5_send_message sets it whenever escalation_id is passed, so the guidance's "fresh ask" is enforced. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(j5): J5's record of a blocked persona shows in its own panel The escalation summary was appended to the agent's ask, which the Inbox renders as Markdown with raw HTML, so an unclosed fence or a <details> in the agent's text could hide it, swallow it, or fake one. The summary is now data on the Inbox item's escalation view, never part of the message, and shows in a "Recorded by J5" panel (J5RecordPanel, plain text) in the Inbox and the Captain's dock, and as plain text under the ask on mobile. The service's askSummary becomes authorizeAsk, which only checks who may ask. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(j5): a blocked persona seat keeps #931's unchecked-model refusal After the restack, a Crew persona seat whose entries are all unchecked is refused as CrewLaunchModelNotEligibleError again, as #931's review made it, and the error carries the persona's block code so j5_propose_crew and j5_request_crew_member still return models_exhausted with the escalation. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(j5): Allow this once covers only the launch the Owner was shown A grant was checked against the caller and persona only, so a retry under the same escalation could pin a different unchecked model or ask a Crew seat for broader access than the refusal showed. The escalation now records its scope: the persona's definition digest, and each candidate the waiver lets run, by entry index, provider instance and model, with the access it would get. A retry is waived only for those candidates, only at no more than that access, and only while the definition is unchanged; anything else is refused and the grant stays unspent. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(j5): a Crew seat naming an invalid persona is told how to repair it Crew prevalidation knew only missing and disabled personas, so a seat naming a persona whose definition is invalid was told it wasn't in the library, with no code and no repair message. The check now uses the same definition lookup as a launch, refusing with the shared code (persona_not_found, persona_disabled, persona_invalid with its repair message, library_empty) in the same text form. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(j5): Allow this once matches the exact entry, reasoning level included A grant's scope matched candidates on instance and model only, so a grant for one entry also covered the same model at another reasoning level. The scope now records each candidate's effective options, the level a pin chose among them where the entry leaves it to the spawner, and a retry is covered only by an entry with the same instance, model and options. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(j5): a Crew seat's refused persona is recorded for the Captain's ask The Crew preflight refused a missing, disabled or invalid persona without an escalation, so the Captain's ask about it carried no record from J5. It now records one, never waivable, and returns its escalation_id, so the ask shows the Recorded by J5 panel with no Allow button. A card the person reviews records nothing. Built with Claude Opus 5.5 in Claude Code (J5 Code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(j5): an escalation ask follows #924's ask rule Since #924 an ask to a person always opens its own exchange, so freshAsk and A2AOpenAskExistsError have nothing left to refuse and go. What can still put an escalation on the wrong Inbox item is a regarding follow-up, so j5_send_message refuses escalation_id with regarding and says to open a new ask with expect_reply, intent and urgency. The guidance, the refusal text and the docs drop "fresh ask" for the same rule. The web Inbox page shows the Recorded by J5 panel before the follow-ups, as the dock and mobile do, and the escalation test support imports migration 54. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(j5): an escalation stays on the open ask that carries it With several asks open to a person, a second ask with the same escalation_id moved J5's record and the Allow this once button off the ask that opened the Inbox item. authorizeAsk now refuses an escalation whose ask is still open, naming that ask and the two ways out: follow up there with regarding, or clear it and ask again. Once the ask closes a new one may carry it. A regarding follow-up to the ask the escalation already sits on drops the id and goes as an ordinary follow-up, since nothing moves; a follow-up to any other ask is still refused. The escalation test support runs the a2a migrations, which the open-ask read needs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(j5): J5's record of a blocked persona reads plainly and names providers The Recorded by J5 panel said Allow this once "would let one launch attempt past a model Models for custom members doesn't check". It now says the launch would use a model that Models for custom members doesn't allow, or run with broader access than the agent that started it. A refused persona's reasons name each entry's provider as the app does (its instance's name, else Codex or Claude) instead of its raw id, using the providers the launch already read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Jackson Miller <jackson@jackson.codes>
Summary
Once agents pick personas and Crews auto-approve, nothing stopped an agent from handing a persona more than it has itself: more access than the agent runs with, or models I left unchecked in Models for custom members on purpose. Now who picked sets the limits. Closes #781.
@persona:IDin a message a person typed (web or mobile) to start the turn, the composer's persona launch, or a roster card the person approves, edited or not. A person-picked@personasubagent now gets its saved access even from a Supervised or read-only parent. J5 passesruntimeMode: "inherit"to upstream'sdelegateTask, the same as it already does for persona parents.access-above-caller, naming both levels) when its access is broader than the caller's effective sandbox. Agent picks arej5_spawn_agentanddelegate_taskwithout the turn's mention, auto-approved rosters, Captain additions, and outside agents (capped at their approved level). Mentions relayed by agents, mentions in scheduled runs, and mentions steered into a running turn count as agent picks, so they fail closed.runtime_modethey get the Captain's effective access instead of Full access, and a broaderruntime_modeis refused with the same reason. A read-only Captain can't get a custom seat automatically. Cards the person approves keep today's Full access default.pickSource:person,agent, orallow-oncewith the waived constraint.allow-onceis defined now so Personas: a blocked persona escalates, and I can allow it once #785 needs no new kind. Old assignments without the field still decode. Nothing in the UI shows it yet.One module owns the check (
apps/server/src/j5/agents/agentPersonaPick.ts), and every launch path calls it:The model-facing text changed too: the
delegate_task,j5_spawn_agent,j5_propose_crewandj5_request_crew_memberdescriptions, the@personainvocation guidance, and the Crew guidance. They encourage passingpersonawhen one fits and explain the limits.Evidence
j5/mainand pass here:agentInvocation.test.ts: "holds an agent's pick to the caller's access but lets a person's @persona mention lift it" (Supervised parent: refused without the mention, refused for anmcp-sourced or code-quoted mention, launched withinheritand full access for a typed one) and "skips an unchecked model for an agent's pick and uses it for a person's".agentSubagent.integration.test.ts: on the real projection store, an agent'sdelegate_taskpersona pick is refused by the model and level lists, and after a person-typed@persona:scoutturn message it runs on the persona's own model.handlers.test.ts(j5_spawn_agent): a Supervised caller's agent pick of a full-access persona is refused with the shared wording, the same pick with the person's mention recordspickSource: person, and an outside agent gets the same refusal with its sign-in next step.CrewLaunchService.test.ts: an automatic seat's persona falls to its checked entry and is refused above a Supervised Captain, while a person's card isn't. An automatic custom seat takes the Captain's Auto-accept edits, a broader one is refused, and a read-only Captain gets none.CrewProposalService.test.ts: an auto-approved roster and a Captain addition are held to a Supervised Captain's access, and a card the person approves still defaults to Full access.agentPersonaRouting.test.ts(mixed reasons per entry),agentPersona.test.tscontract (eachpickSourcekind, and old assignments still decode), andagentMention.test.ts(quoted, code,>and email forms don't count).(@persona:scout), now counts as the person's pick. Emails and identifiers (foo@persona:x) and single-quoted mentions ('run @persona:scout') still don't, and apostrophes in "don't" or "Bryant's" are left alone. New cases are inagentMention.test.ts.agentInvocation.test.tsnow proves a mention steered into a running turn grants nothing. The fixture keeps the starting message and a later one apart and honors themessageIdsfilter, and the steered@persona:team-operatoris still refused as the agent's pick.CrewLaunchSeatcomment now says omitted access is Full access only on a roster the person approves.vp test runonagentMention,agentInvocation,CrewLaunchServiceandagentSubagent.integration: 67 passed.vp run check:changedpasses.exceededDispatchModeLimitfor runtime-mode order, with no copy of the ordering and no upstream edit.CrewLaunchServicetests assert "Persona builder would run with Repository write…".assertEligibleand the routing predicate share one rule.CrewLaunchModelNotEligibleError, the same as an unchecked model chosen on the card (tested).pickis required on the Crew launch inputs, andfulfilpasses it.runtime_modeto approval-required, not offered Full access (tested).vp test runon the 10 related server files: 231 passed.vp run check:changedpasses.ensureScratchProjectis idempotent and creates no thread or folder; andj5_list_personasmarking what an agent's own pick can't use, which is now j5_list_personas marks entries an agent's own pick can't use (models and access versus the caller) #940.@personamention inside any CommonMark code block no longer counts as the person's pick. That covers backtick and tilde fences of any length (closed by a matching fence or by the end of the text), indented code after a blank line, and inline code spans of any backtick length. New cases are inagentMention.test.ts, andvp run check:changedpasses.vp test runon the 16 files around the change: 252 passed.vp run check:changed: format, lint, knip, typecheck (32 packages) and tests (15 files) all pass.mcp-tool-useevals on Sonnet 5.5 for the changed delegation and Crew text: all 33 Claude orchestration cases pass, 66 of 67 samples (99%), for about $4.12.node --env-file-if-exists=../.env mcp-tool-use/run.ts --model claude-sonnet-5-5 --cases mcp-tool-use/cases/orchestration.jsonl --out runs/781-sonnet(fromevals/)The one failed sample was in
opus-review-too-via-delegate(2 of 3). After the simulated native Opus agent came back empty twice, Sonnet fell back todelegate_taskwith no persona. The same case onj5/main(a216acc) passes 1 of 3 and fails the same check ($0.29), so that flake isn't from this change.Not verified: no live run on a real server with real providers. The evals ran only the Claude orchestration cases on Sonnet, not Codex models or Opus, and no case yet checks that an agent passes
personaon its own when one fits. Mobile has no Models for custom members setting, so nothing changed there.Merge Danger
Door: two-way.
pickSourceis an optional field, so assignments written with it still decode after a revert, and the new route failure code is never sent by the catalog.Blast Radius: persona launches from agents (
delegate_task,j5_spawn_agent) and auto-approved Crews. If it's wrong, an agent pick that should launch gets refused with a reason. A person's@personaand approved cards are unaffected, apart from now being able to exceed the parent's access.After merge: nothing. #785 builds "Allow this once" on
allow-once.Checklist
FORK.md(case text and file-table row) in this PR (none edited; every path is J5-owned)docs/j5/product/upstream.md(upstream'sdelegateTaskceiling is left in place)AGENTS.md)docs/j5/product/and user docs rewritten where this changes themSurfaces walked:
@personamention, composer persona launch,delegate_task,j5_spawn_agent, roster card, auto roster, addition, outside agent.pickSourceisn't shown anywhere.pickSource, plus themodel-not-eligiblecode and its client-runtime label.Built with Claude Opus 5.5 in Claude Code.
🤖 Generated with Claude Code