Skip to content

feat(j5): agents choose between subagents and Crew members from context - #758

Merged
bryantderosier merged 16 commits into
j5/crew-member-reasoning-levelsfrom
j5/746-two-ways-to-get-help
Nov 3, 2026
Merged

bryantderosier merged 16 commits into
j5/crew-member-reasoning-levelsfrom
j5/746-two-ways-to-get-help

Conversation

@bryantderosier

@bryantderosier bryantderosier commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Agents have too many ways to get help, and the guidance reads like a menu, so they reach for a Crew because I said "crew" or spawn a standalone Peer Agent when a subagent would do. Inside a Crew, review rounds also go through the Captain, with acknowledgments and progress chatter in between, and a member that sends something important can't tell whether it landed. This rewrites the guidance around the work, makes members run their own review loops, and makes j5_send_message say what happened to the message. Closes #746.

This is stacked on #770, so the base is j5/crew-member-reasoning-levels. It sits above #744 and #770 on purpose: the tool descriptions I rewrite here already carry their model and reasoning-level eligibility sentences. Three more PRs (#760, #761, #762) are stacked on it.

  • Choosing: the standing instructions (orchestrationInstructions.ts) now offer three choices, one sentence each: work directly, subagents for independent assignments, or a Crew for work where workers have to respond to each other. An explicit ask for a Crew, a Peer Agent or a subagent still gets that mechanism, and the choice from the work only applies when nobody asked (Jackson's call in review). An agent can reassess mid-work without redoing finished work, and the test pins and FORK.md case 8 changed with it.
  • Two ways to get help: a subagent (native, or delegate_task across providers), or members on a Crew (j5_propose_crew to start one, j5_request_crew_member to grow one). Each tool description says which path it serves. j5_spawn_agent is now described only for a person's explicit request for a standalone agent, or when an outside agent or a skill names it. Nothing a caller depends on was removed.
  • Review loops: <crew_collaboration> tells members to send findings and fixes straight to the teammate who owns the work, repeat until none are unresolved, and message the Captain only for a review pass (naming the commit reviewed), a blocker, or a disagreement they couldn't settle. No acknowledgments or progress updates. A loop that hasn't produced a clean pass after three rounds goes to the Captain. The Captain settles technical disagreements within the agreed scope and asks me only when it needs product intent or my judgment. The Captain also hears a member's final result or step report, so every "only" list in the instructions, the j5_propose_crew description, the launch report and the seat brief says so.
  • Stale docs I fixed on the way: agent-personas/index.md said j5_spawn_agent took agent (it's persona), and glossary.md said delegate_task wasn't on the J5 agent surface.

Callers of the tools I touched. Nothing a caller depends on was removed. These are the ones I found in the repo; personas, playbooks and scheduled tasks that people wrote themselves are user data and can't be inventoried from here:

  • Instructions: orchestrationInstructions.ts, agentInvocationInstructions.ts (delegate_task for @persona:), playbookInstructions.ts (j5_propose_crew for playbook Crews). Pi's extension takes the same text through upstream's constant.
  • Tool descriptions: mcp/tools.ts (spawn, propose, request member, list personas), agentDelegation.ts (delegate_task), crewGateNotice.ts, spawnIds.ts (<crew_collaboration>), plus refusal hints in outsideAgent.ts and handlers.ts, and mentions in the worktree and project toolkits.
  • Pre-approval and runtime: j5ToolPreapproval.ts, agentPersonaRuntime.ts.
  • The j5-live-test skill still tells the builder to spawn a tester with j5_spawn_agent, which the new text allows because a skill names it.
  • The bundled example personas don't reference any of them.

Docs: crews.md (AC1 and AC9 rewritten, and a messages-and-steering table), agent-tools.md including the quoted descriptions, docs/j5/product/upstream.md D1, and a sentence in the personas user guide.

Evidence

At dfe913afdc, on top of #770 and j5/main. #744's reasoning levels moved into #770 below it, and this layer's own tests didn't change:

  • 1291 server tests passed (1 skipped):
    cd apps/server && vp test run src/j5/a2a src/j5/agents src/mcp
    PeerPollRoundTrip.test.ts still covers the offline-recipient case, and the EnvelopeFormatter test checks the j5_send_message description against agent-tools.md byte for byte.
  • 205 provider-core tests passed, including the new wording pins in orchestrationInstructions.test.ts and playbookInstructions.test.ts: cd packages/provider-core && vp test run
  • spawnIds.test.ts, tools.test.ts and crewGateNotice.test.ts pin the new <crew_collaboration>, tool-description and Captain wording.
  • Server and web typecheck report 0 errors.

Not verified: Cursor's steering (it isn't set up on the test host) and Codex models other than GPT 6. The Live testing section below has what ran. No repo-wide checks.

Merge Danger

Door: two-way. No migration, no stored data, and a revert restores the old text and the old delivery field.

Blast Radius: every agent's standing instructions. Agents will choose between subagents and Crews differently. The j5_send_message result is unchanged.

  • I reworded PLAYBOOK_CREW_INSTRUCTIONS so "crew" in a message isn't the trigger for building a playbook Crew.

After merge: nothing.

Live testing

Ran on this stack at 1d4ff9bdf6. Later commits only add the results to crews.md, and the commits below it were reworked (the settings row) and rebased onto a newer j5/main, which this layer doesn't touch. Isolated test home from j5-live-test, Claude Haiku 5.5 for the Captain and writer, GPT 6 Luna for the reviewer. I checked each result in the member's own thread and the ledger rather than the Captain's account.

  • Claude, idle member: j5_send_message reported starting and the writer started a new turn.
  • Claude, busy member: while the writer held a turn open on scripts/hold.sh, a second message reported steered and showed up inside that running turn in the writer's thread.
  • Stop during that steered turn: the run showed "interrupted by user" and hold.sh was gone. The steered message was already marked delivered, but the writer never acted on it, so it was lost with the turn. That's the known race in FORK case 26, and Claude shows it.
  • Claude, message after the run ended: reported starting, and the writer did the work in a new turn.
  • Codex (GPT 6 Luna), busy member: reported steered, and the reviewer acted on it mid-run: it sent its review before I stopped it. Stop interrupted cleanly and every delivery ended up delivered.
  • Review loop: writer to reviewer and reviewer back to writer went directly. After the two assignments, the first loop was three messages in the ledger (the review request, "no findings", and the review pass to the Captain), with no acknowledgments.
  • Choosing: asked for two people in separate worktrees, the agent did it directly with plain git. Asked for "a team" with a writer and a reviewer, it proposed a Crew. Asked again for them as Crew members, it proposed one.

Not exercised live: Cursor (not set up here), Codex models other than GPT 6, a message after the run ended on Codex, and a steered message to a member on an online peer server. The per-provider results are in the list above, not in crews.md, which only states what each provider does.

Upstream impact

FORK.md cases 8 and 26 are updated. Upstream's constant already points at J5's instruction text, so no upstream source file changes. docs/j5/product/upstream.md D1 is rewritten for the new wording.

Checklist

  • One concern: the description has no "also"
  • Tests cover the changed behavior (backend changes ship with focused tests)
  • UI changes: before/after screenshots above, and a video for motion or interaction — no UI changes
  • Live testing: the section above says what ran live and what did not
  • Upstream-owned files: each one is recorded in FORK.md (case text and file-table row) in this PR
  • Upstream product: any change to what upstream's product does has a human decision linked above and a register entry in docs/j5/product/upstream.md — D1 rewritten
  • Surfaces: entry points, clients, providers, contracts, reverse states, connection modes (see AGENTS.md) — Claude and Codex steering ran live; Cursor's is not live-tested, as the Live testing section says
  • Docs: definitions under docs/j5/product/ and user docs rewritten where this changes them

Claude Sonnet 5.5 through the Claude Code harness in J5 Code.

🤖 Generated with Claude Code

@vercel

vercel Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
j5-code Ready Ready Preview Nov 3, 2026 12:50am UTC

Request Review

@bryantderosier bryantderosier self-assigned this Oct 27, 2026
@bryantderosier
bryantderosier added this pull request to stack #753 October 27, 2026 18:15
@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:L 100-499 effective changed lines (test files excluded in mixed PRs). labels Oct 27, 2026
@bryantderosier
bryantderosier force-pushed the j5/746-two-ways-to-get-help branch from 3bd2d5c to 1d4ff9b Compare October 27, 2026 21:17
@bryantderosier
bryantderosier marked this pull request as ready for review October 27, 2026 22:47
@coderabbitai

coderabbitai Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Tip

🎯 Customize your risk assessments

Guide how CodeRabbit assesses merge risk and security architecture risk. Highlight sensitive changes and explain your security boundaries using Merge Risk Instructions and Security Risk Instructions in your review settings. Security architecture risk assessments require the Advanced plan or above.

✅ Automatic approvals, with your rules

Let CodeRabbit approve pull requests once required threads are resolved, pre-merge checks pass, and your risk limits are met. Enable Automatic approvals in review settings, with overrides for teams or file paths.

CodeRabbit Verify

  • Run CodeRabbit Verify to test the changes in this pull request.

@Jacksondr5

Copy link
Copy Markdown
Owner

Panel review: #758

Should fix: the "only" rules contradict the final-result and step-report rules. Three places say the Captain hears from a member only for a review pass, a blocker or a disagreement:

  • the standing instructions (orchestrationInstructions.ts:30)
  • the j5_propose_crew description (mcp/tools.ts:584)
  • the launch report (crewGateNotice.ts:218)

The seat brief (spawnIds.ts:134, :152) also tells members to send their final result and playbook step reports, and the Captain's playbook instructions wait for those reports. Add "your final result or step report" to each "only" list.

Nit: race in deliveryOutcome. The handlers call worker.notify before withDeliveryOutcome (mcp/handlers.ts:417-418). When the worker wins, receiverBacklog.ts:115-119 reports steered for a steer to an idle receiver that actually started a fresh turn. This is harmless to the sender. Nothing in the repo read the old delivery field.

Docs nits

  • crews.md's new "Messages and steering" section carries live-test receipts and test-host limits in a product definition. Keep the provider behaviour there and move the evidence to the PR.
  • docs/j5/user/personas.md:62 still says every non-Codex provider queues, which contradicts the Claude steering this PR documents.
  • The description still says this layer is the top of the stack.

CI: this head's one failure is packages/ssh/src/tunnel.test.ts:1205 (zombie-process timing). The stack doesn't touch that file. A re-run should clear it.


Review panel for the #750–#762 stack, judged as the code that lands at da64b2795c: Claude Opus 5.5 (lead and three code reviewers), GPT-6.1-Sol (process, docs, CI) and GPT-6-Astra (live test on an isolated server, Haiku 5.5 and GPT-6 Luna turns). Every finding here was agreed by the whole panel. Product questions are with Jackson and are not listed. Posted by Claude Opus 5.5 via J5 Code.

@bryantderosier
bryantderosier removed this pull request from stack #753 October 28, 2026 11:45
@bryantderosier
bryantderosier changed the base branch from j5/744-custom-member-models to j5/crew-member-reasoning-levels October 28, 2026 11:45
@bryantderosier
bryantderosier force-pushed the j5/746-two-ways-to-get-help branch from 4225604 to e5c8e7b Compare October 28, 2026 11:45
@bryantderosier
bryantderosier added this pull request to stack #771 October 28, 2026 11:48
bryantderosier and others added 4 commits October 29, 2026 08:18
…746)

Rewrites crews.md AC1 and AC9, adds the delivery outcome as AC24 and a
per-provider steering table (live-test results pending), updates the
quoted tool descriptions in agent-tools.md, FORK.md cases 8 and 26, the
register's D1, and fixes two stale lines (persona parameter name,
delegate_task in the glossary).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
#746)

The three places that said a Captain hears from a member only for a review
pass, a blocker or a disagreement contradicted the seat brief and the playbook
hand-off, which both have members report a final result or a step.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…Claude steering (#746)

The live-test results belong to the PR. The user doc still said every
provider but Codex GPT 6 queues, which Claude's steering contradicts.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@Jacksondr5

Copy link
Copy Markdown
Owner

Review panel: Claude Opus 5.5 (lead) and GPT-6.1 Sol. We reviewed the whole stack at 16cb7824a1. Both reviewers agreed.

Low should-fix: the j5_send_message description over-promises. envelopes.v1.json and agent-tools.md now say "you are told separately if it later fails to arrive".

A steer that was accepted counts as delivered, and there's no later receipt that the model processed it. Your own live test shows a steered message lost to a Stop on Claude, with no alarm (FORK case 26). An agent told steered will assume the message landed.

Suggestion: say that steered means the provider accepted it into the running turn, and that it is lost if that turn is stopped first. Or drop the sentence.

@Jacksondr5

Copy link
Copy Markdown
Owner

Jackson's decision, relayed by the review lead (Claude Opus 5.5): "crew", "peer agent" and "subagent" stay trigger words. In Jackson's words: "If I ask for a feature, I'd expect to get it."

The new standing instructions say "the words in the user's message do not decide it, and 'crew' or 'subagent' in that message does not by itself pick one", and j5_spawn_agent is limited to an explicit standalone-agent request.

Please change both:

  • An explicit request for a Crew, a Peer Agent or a subagent gets that mechanism.
  • The agent chooses from the work only when the person didn't ask for one.
  • A word mentioned in passing, or in quoted text, still isn't a request.

crews.md AC1, D1 and the test pins follow.

@Jacksondr5

Copy link
Copy Markdown
Owner

Jackson's decision, relayed by the review lead (Claude Opus 5.5): please remove deliveryOutcome. In his words: "we don't have a way to reliably tell whether something landed. I'd rather we not guess."

The value is predicted when the message is sent. Your own live test had a message reported as steered that was then lost when the receiver's turn was stopped.

Please:

  • drop the field, along with its text in the j5_send_message description, agent-tools.md, the standing instructions and crews.md AC31;
  • keep only what's measured: deliveryNotice, the offline-recipient waiting_for_recipient that existed before, and the late-failure alarm.

This also settles the earlier panel finding on this PR about the description promising an alert.

bryantderosier and others added 6 commits November 2, 2026 09:25
…-Crew layer

The standing instructions are ported onto j5/main's section layout. Per
the review, an explicit request for a Crew, a Peer Agent or a subagent
gets that mechanism, and the choice from the work applies only when the
person asked for none.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…op predicting delivery

Jackson's review decisions on this layer: an explicit request for a Crew,
a Peer Agent or a subagent gets that mechanism and the choice from the
work applies only when the person asked for none; and j5_send_message no
longer returns a predicted deliveryOutcome, keeping the measured
deliveryNotice, the offline-recipient delivery field and the late-failure
alarm.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…-Crew layer

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@bryantderosier

Copy link
Copy Markdown
Collaborator Author
  • Trigger words: an explicit ask for a Crew, a Peer Agent or a subagent gets that mechanism again, and the choose-from-the-work rule only applies when nobody asked. A word in passing or in quoted text doesn't count. The standing instructions, tool descriptions, FORK.md, crews.md AC1 and the tests all follow (4090e50992).
  • deliveryOutcome: removed, along with its text in the j5_send_message description, agent-tools.md, the standing instructions and crews.md (AC31). deliveryNotice, the offline waiting_for_recipient and the late-failure alarm stay. That also settles the description-over-promises comment.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ing exported

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@bryantderosier
bryantderosier merged commit 939259b into j5/main Nov 3, 2026
35 checks passed
@bryantderosier
bryantderosier deleted the j5/746-two-ways-to-get-help branch November 3, 2026 01:30

This branch was successfully deployed

1 active deployment
Preview — 3e67393d Deployed Nov 3, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L 100-499 effective changed lines (test files excluded in mixed PRs). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Crews: two ways to get help, and members who run their own review loops

2 participants