Skip to content

feat(j5): choose which reasoning levels custom Crew members can use - #770

Merged
bryantderosier merged 13 commits into
j5/744-custom-member-modelsfrom
j5/crew-member-reasoning-levels
Nov 3, 2026
Merged

bryantderosier merged 13 commits into
j5/744-custom-member-modelsfrom
j5/crew-member-reasoning-levels

Conversation

@bryantderosier

@bryantderosier bryantderosier commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

#757 lets me choose which models custom Crew members may run on, but a Captain or an agent can still pick any reasoning level on an allowed model, and once additions and auto-approved launches skip the roster card I won't get a chance to correct it. This adds a second list, the reasoning levels custom workers may use, and the platform enforces it at the same points as the model list. It's the part of #757 the review panel suggested splitting out, since #744 asked for models only. There's no tracking issue for it yet.

This is stacked on #757, so the base is j5/744-custom-member-models, and #758 now sits on top of this branch.

  • Setting: a second project-scoped server setting, crewMemberReasoning, holding checked (providerInstanceId, level) pairs, shown in the same Agent Setup row as the models ("Models for custom members"). It works like the models list: nothing saved means every level is eligible, a saved list holds only checked levels, a level that shows up in the catalog later starts unchecked, and a saved empty list means no level is eligible. A provider's levels are the choices of its models' reasoning options, in the order they first appear.
  • The cards: each provider card gets a Reasoning group under its Models, in the same grid and with the same chips, solid blue when allowed. The header reads like "1 of 8 models · 3 of 6 levels", and Check all and Uncheck all act on both groups.
  • One check: CrewMemberModels.assertEligible now checks the model, then the level, at every call site feat(j5): choose which models custom Crew members can use #757 already covers: CrewLaunchService.resolveSelection, j5_spawn_agent and delegate_task without a persona. The level checked is the seat's explicit reasoning value, or the model's default when none is set; a model with no reasoning option has nothing to check.
  • Personas: a persona seat that keeps either of its persona's routes, including its level, isn't checked. One where I changed the model or the level is checked like a custom seat.
  • Refusals: a typed error naming the level, the provider, the project and the setting, with the eligible levels listed so the agent can retry. The roster card only offers eligible levels for a custom member and picks the first eligible level when the provider or model changes. Approving a seat with an ineligible level is refused with the same message.

Docs: crews.md acceptance criteria on levels, the four tool sections in agent-tools.md (one sentence each), and the persona docs' note on allowlists.

Evidence

At 7774ddda16, on top of #757. vp run -r typecheck (the CI command) exits 0 here:

  • The server, web, contracts and shared suites pass at this head, and at the stack tip (b8a42e0bc0) server 1300, web 835, contracts 736 and shared 1680 tests pass:
    cd apps/server && vp test run src/j5/a2a src/j5/agents src/mcp
  • New cases cover level derivation per provider, a checked and an unchecked level, a saved empty list, stale entries, the default level when none is set, a model with no reasoning option, a persona seat on either route's level, a persona seat whose level I changed, an inherited selection on delegate_task, and an accepted retry. The card test checks that a custom member's level picker is filtered and a persona's isn't.
  • projectSettings.test.ts checks the second key overrides like the first.

Not verified: approving a roster card that holds an ineligible level by hand, auto-approve, j5_request_crew_member and delegate_task on a live server (tests cover them), and a project-scoped override. I didn't run repo-wide checks.

Merge Danger

Door: two-way. A setting that's null until someone saves it, no migration, so a revert undoes it. A saved list would just sit unused.

Blast Radius: every path that gives a custom worker a reasoning level: Crew launch and additions, j5_spawn_agent and delegate_task, but only in a project where someone has saved a list.

Things worth knowing:

  • A model whose levels are all unchecked passes the model check and fails the level check. The picker leaves the seat alone and approval refuses it with the eligible levels.
  • Agent Setup's note naming personas on an unchecked model doesn't name one on an unchecked level yet.
  • Like the model list, this applies at creation only. A later change through t3_thread_configure isn't limited.

After merge: nothing.

UI changes

The Reasoning group on each card and the filtered reasoning picker are new, so there's no before.

Before After
Codex card: one of eight models and three of six reasoning levels allowed new Codex card: one of eight models and three of six reasoning levels allowed, after
Claude card: the Reasoning group uses the same chips as the models new Claude card: the Reasoning group uses the same chips as the models, after
A custom member's reasoning picker offers only the allowed levels new A custom member's reasoning picker offers only the allowed levels, after

Live testing

Ran on a stack tip from before the last restack, at 4fa701e9ee, in an isolated test home from j5-live-test. Nothing touched my real install.

  • Row: I unchecked everything with Check all, then allowed GPT-6 Luna at Low, Medium and High on Codex, and Haiku 5.5 at Low through Extra High on Claude. The headers read "1 of 8 models · 3 of 6 levels" and "1 of 13 models · 4 of 7 levels", and the row read "2 of 21".
  • Level refusal: I asked an agent in plain words for a helper on Haiku 5.5 with Max reasoning and no persona. It reported the spawn was refused, naming the level, the provider and the setting, and listing low, medium, high and xhigh. I checked the stored state, not just its account: no thread exists on Haiku 5.5 at Max.
  • Roster card: in Add member, the custom member's reasoning list was Low, Medium, High and Extra High.

Not exercised live: the cases listed under Not verified, and Cursor and the other providers (not set up on the test host). The commits have been rebased since this run; the rebase only moved them onto a newer j5/main.

Upstream impact

packages/contracts/src/settings.ts gets the second key in the same places as crewMemberModels, five lines and an import, with no new hook, recorded in FORK.md case 107 and register entry D42, which now covers both settings. The projectSettings.test.ts case is listed there too.

Checklist

  • One concern: the description has no "also"
  • Tests cover the changed behavior (backend changes ship with focused tests)
  • UI changes: before/after screenshots above, and a video for motion or interaction — new states, so no before; no motion
  • Live testing: the section above says what ran live and what did not
  • Upstream-owned files: each one is recorded in FORK.md (case text and file-table row) in this PR
  • Upstream product: any change to what upstream's product does has a human decision linked above and a register entry in docs/j5/product/upstream.md — D42
  • Surfaces: entry points, clients, providers, contracts, reverse states, connection modes (see AGENTS.md) — the setting, picker and spawn refusal ran live on Claude and Codex; the other paths are covered by tests
  • Docs: definitions under docs/j5/product/ and user docs rewritten where this changes them

Claude Sonnet 5.5 through the Claude Code harness in J5 Code.

🤖 Generated with Claude Code

@vercel

vercel Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
j5-code Ready Ready Preview Nov 3, 2026 12:50am UTC

Request Review

@coderabbitai

coderabbitai Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Tip

🎯 Customize your risk assessments

Guide how CodeRabbit assesses merge risk and security architecture risk. Highlight sensitive changes and explain your security boundaries using Merge Risk Instructions and Security Risk Instructions in your review settings. Security architecture risk assessments require the Advanced plan or above.

✅ Automatic approvals, with your rules

Let CodeRabbit approve pull requests once required threads are resolved, pre-merge checks pass, and your risk limits are met. Enable Automatic approvals in review settings, with overrides for teams or file paths.

CodeRabbit Verify

  • Run CodeRabbit Verify to test the changes in this pull request.

@bryantderosier
bryantderosier force-pushed the j5/crew-member-reasoning-levels branch from 39abaec to e2715d4 Compare October 28, 2026 12:06
@github-actions github-actions Bot added the vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. label Oct 28, 2026
@github-actions github-actions Bot added the size:XL 500-999 effective changed lines (test files excluded in mixed PRs). label Oct 28, 2026
@bryantderosier
bryantderosier force-pushed the j5/crew-member-reasoning-levels branch from e2715d4 to 8256c31 Compare October 28, 2026 15:33
@bryantderosier
bryantderosier force-pushed the j5/crew-member-reasoning-levels branch from 8256c31 to b86e3af Compare October 29, 2026 11:33
bryantderosier and others added 5 commits October 29, 2026 08:18
…els (reasoning levels)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…oning levels)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…and the roster card (reasoning levels)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…K.md and the divergence register (reasoning levels)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ing levels)

The persona seat comparison matches a route's model and level together, so a seat frozen on the fallback route at its level skips the lists, and the same model at another level is checked.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
bryantderosier and others added 4 commits November 2, 2026 09:20
…g-levels layer

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…g-levels layer

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ing exported

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…o fix/770

# Conflicts:
#	apps/server/src/j5/a2a/CrewLaunchService.ts
@bryantderosier
bryantderosier merged commit 1609fff into j5/main Nov 3, 2026
35 checks passed
@bryantderosier
bryantderosier deleted the j5/crew-member-reasoning-levels branch November 3, 2026 01:30

This branch was successfully deployed

1 active deployment
Preview — e477ca42 Deployed Nov 3, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL 500-999 effective changed lines (test files excluded in mixed PRs). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant