Skip to content

MCP profiles 1: settings and the profile resolver (nothing calls it yet) - #227

Draft
scuttletoby wants to merge 3 commits into
mainfrom
fork/mcp-profile-1
Draft

scuttletoby wants to merge 3 commits into
mainfrom
fork/mcp-profile-1

Conversation

@scuttletoby

@scuttletoby scuttletoby commented Nov 15, 2026 •

Copy link
Copy Markdown
Owner

Why

PR 1 of the MCP profile work (#145): a profile says which outside MCP servers and which groups of T3's own tools a thread gets. This PR adds only the settings and the rules that decide a thread's profile. Nothing calls the resolver yet, so nothing changes. Every thread still gets everything (all). The provider adapters and the T3 tool filter come in later PRs.

What is in it:

  • Four settings keys in the fork settings seam (forkSettings.ts, from fork: delegate checks behind one fork module (SEAM) #174):
    • mcpProfiles: named profiles.
    • mcpProfile: the default profile, all.
    • mcpSubagentProfile: the default for delegated children, inherit.
    • mcpThreadProfiles: the owner's pick per thread.
    • A project may override mcpProfile and mcpSubagentProfile.
    • There is a row for each key in forkDefaults.ts, all off, and in docs/fork/README.md and FORK_CHANGES.md.
  • Schemas in the new packages/contracts/src/mcpProfile.ts. Built-in profiles are all and t3-only. There are 12 T3 tool groups. waits is new since the design: the fork's waits toolkit came later.
  • Lenient read. A bad MCP value in the settings file never stops the file from loading:
    • A value that is not text (a number, a list, an object) reads as a marker such as (unreadable number). No profile can have that name, so a thread that would use it is refused, never widened to all.
    • null means "not set". A map that is not an object reads as empty.
  • Save check (forkSettingsWriteProblem in packages/shared/src/forkSettingsPatch.ts). A save is refused, with a message that names the fix, when it brings in:
    • inherit where it means nothing (the default profile, a thread pick, a project's default);
    • a name with no profile (a profile added in the same save counts).
      Only new or changed values are checked, so an old odd value never blocks an unrelated save.
  • Pure resolver apps/server/src/fork/mcpProfile.ts, which decides in this order:
    1. The owner's pick for this thread is final.
    2. Otherwise the role default: mcpSubagentProfile for a delegated child, unless it is inherit; otherwise mcpProfile. A project's override wins over the server's.
    3. A role default is cut down to what the parent has. A fork counts as a child.
  • T3 tool groups per toolkit (MCP_T3_TOOLKIT_GROUPS). previewControls is in preview. tool_search, request_secret and send_to_user are ungrouped on purpose (MCP_T3_UNGROUPED_TOOLS): they are how a thread finds tools and reaches the person, not one feature. The T3 tool filter PR decides how profiles treat them.
  • Patch rule replaceEntries in forkSettingsPatch.ts. Each entry is replaced whole, so two windows that save different picks at the same time both keep theirs. Thread picks keep the 500 newest.
  • Person-only key list FORK_PERSON_ONLY_KEYS. It is every fork key except FORK_AGENT_WRITABLE_KEYS, so a new fork key is covered without a new row. Later PRs (M1) use it too.
  • A test that no T3 tool's input names a person-only key. It checks every spelling (mcp_profile, MCP-Profile) at any depth. It replaces the old hand-written PERSON_ONLY_KEYS test. It loads every toolkit file and checks that each one's tools are all found. It counts Tool.make under any import alias (Tool as T, make as m, * as Ai), and fails if a Tool.make appears outside the toolkit folders without a listed reason.
    • Later PR: a server-side guard that refuses a settings write of a person-only key from an agent (MCP) caller. Today the test is the guard: no agent tool can name these keys.

Project keys: why save accepts what read keeps

A project's mcpProfile and mcpSubagentProfile are read as any text, kept as written (only trimmed). Clients send a project's whole override back when they save its scripts or provider. So the save takes any text too, and the server refuses only a value that changed. The other choice was to turn "" or odd case into "unset" on save. That would quietly change what the owner wrote, and turning a refused name like Lean into "unset" would widen that project to the server's default. Keeping the value means it stays refused until the owner fixes it, and unrelated saves still work.

Upstream files touched

Two small seams, each marked Fork: and pinned by a test:

  • packages/contracts/src/settings.ts: a write-fork-setting save error with a detail message (3 lines plus comments).
  • apps/server/src/serverSettings.ts: one call to the save check in each write path (the real one and the test one).

What can go wrong (failure table)

What happens when this runs again, stops half-way, or meets existing, old or empty state?

Case What happens
Runs again The resolver and the save check are pure: the same input always gives the same answer. They store nothing. Saving the same settings again passes the check (unchanged values are not checked).
Stops half-way Nothing to stop: no migration. A save is one patch through the existing write path; a refused save writes nothing. Tested.
Old settings file (no keys) Loads as off: {}, all, inherit, {}. Every thread resolves to all, which is today's behaviour. Tested.
Empty value ("", blank, missing, null) Means "not set", so the default applies (today's behaviour). A null thread pick is dropped. Tested.
Bad value in the file ("mcpProfile": 5, a list, an object, a map that is not an object) The file still loads and every other setting is kept. The value reads as (unreadable …), which the resolver refuses for any thread that uses it. Tested on the real settings layer.
Existing odd project value ("", Lean, a marker) Saving the project's scripts or provider sends it back unchanged and is accepted. A thread in that project is refused until the owner fixes it. Tested.
Save names a profile that does not exist, or inherit where it means nothing Refused; nothing is written. The message names the setting and the fix, for example: mcpProfile names MCP profile "lean", which is not set up: add that profile first, or pick "all" or "t3-only". Tested.
Profile this build cannot read (unknown field, from a newer build) The settings file still loads and a save keeps the profile as written. A thread that uses it is refused, never widened to all. Tested.
Profile name that does not exist (hand edit, or a profile deleted later) The resolver refuses with a message: pick all for this thread, or fix the profile in Settings. Never falls back to all (design §9, #101).
More than 500 thread picks The oldest pick is dropped, and that thread falls back to its role default (the default profile or the sub-agent rule). Pruning picks of deleted threads, so live picks are not pushed out, is in PR3.
Unknown T3 tool group in a profile Keeps nothing for that name, and lists it in unknownToolGroups so the UI can show it.
A new T3 tool or toolkit The groups test fails until it is put in a group or on the ungrouped list.
Bad patch (built-in or inherit as a profile name, bad name, unknown or missing field, unknown group, blank server, bad thread id) Refused. Effect would silently drop a bad record key or an extra field, so explicit checks refuse these instead. Tested.
Broken lineage (a loop, more than 64 deep), or a parent whose profile cannot apply The child is refused with a reason, never all. A child the owner picked for still starts. Tested.
Older build saves the settings It drops keys it does not know. Existing fork keys have the same exposure; that is not new here.
Two clients save picks at once Each entry is replaced on its own, so both picks stay. Tested.

Deferred to later PRs: refusing to delete a profile still in use, dropping picks for deleted threads (PR3), the server-side person-only write guard, and the "server not set up" note (PR2/PR3).

General vs project

  • General: all of it. Profiles, groups, the save check and the resolver rules work for any T3 install and any MCP server name.
  • Project: none. No project names, servers or paths are built in. Profiles are written by each owner in Settings.

Tests

  • packages/contracts/src/mcpProfile.test.ts: reading the file (old, unreadable, bad values, trim, project overrides) and refusing bad patches.
  • apps/server/src/fork/mcpProfile.test.ts: table cases for every case in design §9, plus unreadable values.
  • packages/shared/src/forkSettingsPatch.test.ts: replaceEntries (two saves at once, removing, newest order, keep 500, replace whole, end to end) and the save check.
  • apps/server/src/serverSettings.forkSettings.test.ts: a file with bad MCP values loads and keeps every other setting; a refused save shows its message and writes nothing.
  • apps/server/src/fork/personOnlySettings.test.ts: list, finding every T3 tool, every toolkit file loaded, aliased Tool.make, no person-only key in any input, every tool in a group or the ungrouped list.
  • packages/shared/src/forkDocs.test.ts: the new settings have rows and sections.
  • Red-checked: the 32 new tests fail on the code before fix round 1 and pass after.
  • Results: 10 related test files, 270 passed (including server settings, web scoped settings, fork docs and fork defaults).

🤖 Generated with Claude Code

Adds the four MCP profile settings keys (mcpProfiles, mcpProfile,
mcpSubagentProfile, mcpThreadProfiles) to the fork settings seam, their
schemas in packages/contracts/src/mcpProfile.ts, a replaceEntries patch
rule (per-entry replace, keep the 500 newest thread picks), and the pure
resolver in apps/server/src/fork/mcpProfile.ts. Nothing calls the
resolver yet, so behaviour is unchanged: every thread resolves to "all".

FORK_PERSON_ONLY_KEYS (every fork key not agent-writable) replaces the
hand-written PERSON_ONLY_KEYS test with one that loads every T3 toolkit
and fails if any tool input names a person-only key in any spelling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Review lens (correctness + state) on ac0d1bd: SAFE WITH NITS

Nothing calls the resolver yet, so there is no runtime change today. Fix 1-3 before a later PR wires the resolver in (1 and 2 bite now via the settings file).

  1. Strict read of new keys can reset all settings. packages/contracts/src/mcpProfile.ts:134 (thread picks Record(String, String)) and forkSettings.ts:78,80,158,160 decode strictly. One bad value ("mcpThreadProfiles": {"t1": null}, "mcpProfile": 5, a newer build storing objects) fails the whole file, loadSettingsFromDisk falls back to defaults: mcpProfile: "all", no picks, every other setting lost. Read leniently like mcpProfiles; refuse at use.
  2. Project keys: lenient read, strict save. Read keeps ""/any spelling (forkSettings.ts:158), save requires a valid name (:167). Web saves resend the whole entry (useProjectScriptEditing.ts:100, scopedSettings.ts:203), so a project with {"mcpProfile": ""} or "Lean" can no longer save scripts or provider settings.
  3. Saves accept names that lock every thread out. McpProfileName (mcpProfile.ts:24) accepts inherit; {mcpProfile: "inherit"} or a name with no profile saves, then lookUp (fork/mcpProfile.ts:141) refuses every thread. Fails closed, but the save should refuse.
  4. 500-pick cap can widen a thread. forkSettingsPatch.ts:51-55 + KEEP_NEWEST (forkSettings.ts:139) drops the oldest pick, which then falls back to all. Drop only deleted threads' picks (PR3), or note it in the failure table.
  5. Person-only test is name-based. personOnlySettings.test.ts:72-87 checks input property names only; \bTool\.make\( (:33-34) misses aliased imports; unexported toolkits are never loaded. Fine today (checked t3_environment_preferences_update, t3_project_update, unknown args refused at McpHttpServer.ts:525); lasting fix is a server-side guard on settings writes from MCP callers.
  6. Nit: tool_search and previewControls are in no group (mcpProfile.ts:37), and no test maps every tool to a group; PR text says only request_secret/send_to_user are ungrouped.

Resolver order, fail-closed cases, replaceEntries merge, kept coverage of the deleted test, and old-file loading all checked OK.

CI is also red on this head: typecheck ProviderSettingsPanel.tsx:801 (mcpProfiles read as Record<string, unknown> vs strict patch type), and forkDocs.test.ts (3 cases: the new settings need rows/sections in docs/fork/README.md and FORK_CHANGES.md, required since #223 merged).

Next: one fix round (items 1-3, a note for 4, 6 decided, CI fixed), then a re-check.

scuttletoby and others added 2 commits November 15, 2026 19:51
…ocs)

- A bad MCP value in the settings file no longer fails the whole file: it
  reads as an unreadable marker the resolver refuses; null means not set.
- Project references: save accepts what read keeps; only a changed value
  is checked, so old odd values never block unrelated project saves.
- Saves refuse inherit where it means nothing and names with no profile,
  with a message that names the fix (write-fork-setting save error).
- Every T3 tool maps to a group or the ungrouped list; previewControls is
  in preview; tool_search, request_secret, send_to_user are ungrouped.
- Person-only test counts aliased Tool.make and checks every toolkit loads.
- Docs rows and sections for the six new settings; FORK_CHANGES line.
- Fixes the web typecheck (profiles patch type matches the read type).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Fix round 1

Head d913a755cb (merged main first, no rebase). Each item has tests that fail on the code before this round (32 new tests red before, green after).

1. Lenient read. A bad MCP value no longer fails the whole settings file.

  • packages/contracts/src/mcpProfile.ts:150 (readMcpProfileRef), :158 (McpProfileRef), :84 (McpProfilesFromSettingsFile), :205 (McpThreadProfiles).
  • A non-text value reads as (unreadable number|list|object), which no profile can have, so the resolver refuses it (fails closed). null means not set; a null pick is dropped. A map that is not an object reads as {}.
  • Tests: packages/contracts/src/mcpProfile.test.ts:42, :70; on the real settings layer with {"mcpThreadProfiles": {"t1": null}, "mcpProfile": 5}: apps/server/src/serverSettings.forkSettings.test.ts:223; resolver refuses the markers: apps/server/src/fork/mcpProfile.test.ts:218.

2. Project keys: read and save agree. Chose "save accepts what read keeps".

  • packages/contracts/src/mcpProfile.ts:173 (McpProjectProfileRefPatch), packages/contracts/src/forkSettings.ts:175.
  • A project's whole override is sent back on every project save, so an old "", Lean or marker comes back unchanged and is accepted. The server checks only a value that changed (packages/shared/src/forkSettingsPatch.ts:98). Turning an odd value into "unset" would quietly widen that project to the server default; keeping it keeps it refused until the owner fixes it. Reasoning is in the PR body.
  • Tests: packages/contracts/src/mcpProfile.test.ts:114, packages/shared/src/forkSettingsPatch.test.ts:204.

3. Saves refuse inherit where it means nothing, and names with no profile.

  • Schema: inherit refused for mcpProfile and thread picks (packages/contracts/src/mcpProfile.ts:38).
  • Save check forkSettingsWriteProblem (packages/shared/src/forkSettingsPatch.ts:122), called from both write paths: apps/server/src/serverSettings.ts:236, :1248. A profile added in the same save counts.
  • New save error write-fork-setting with a detail message: packages/contracts/src/settings.ts:1752, :1778. Example: mcpProfile names MCP profile "lean", which is not set up: add that profile first, or pick "all" or "t3-only".
  • Tests: packages/shared/src/forkSettingsPatch.test.ts:190 (7 accepted, 8 refused cases), apps/server/src/serverSettings.forkSettings.test.ts:247 (refused save shows the message, writes nothing); pin of the upstream seam packages/contracts/src/forkSettings.test.ts:84.

4. 500-pick cap. No small safe fix (keeping live picks needs to know which threads exist, which is PR3). Added the row "oldest pick dropped → falls back to its role default; pruning picks of deleted threads in PR3" to the failure table; doc at packages/contracts/src/mcpProfile.ts:197.

5. Person-only test.

  • Counts Tool.make under any import alias (Tool as T, make as m, * as Ai, * as T): apps/server/src/fork/personOnlySettings.test.ts:36, test :180.
  • Checks every toolkit folder has a tools.ts, every one is loaded, and each file's exported tools match its make calls: :168.
  • The server-side guard for settings writes from agent (MCP) callers is listed as a later PR in the body.

6. Groups.

  • previewControls is in preview; tool_search, request_secret and send_to_user are ungrouped on purpose (packages/contracts/src/mcpProfile.ts:46).
  • Toolkit → group map and lookup: apps/server/src/fork/mcpProfile.ts:85, :107.
  • Tests: every T3 tool maps to a group or the ungrouped list apps/server/src/fork/personOnlySettings.test.ts:213; every toolkit has a row and every group is used :220.

CI.

  • Typecheck apps/web/src/components/settings/ProviderSettingsPanel.tsx:801: the profiles patch type is now Record<string, unknown> like the read type (packages/contracts/src/mcpProfile.ts:124), with the checks moved into filters. Web, server, contracts and shared typecheck clean.
  • forkDocs.test.ts: rows docs/fork/README.md:40, sections from :148, and FORK_CHANGES.md:33.

Checks run here: 10 related test files, 270 passed. Typecheck clean for contracts, shared, server and web. The commit went through the full pre-commit hook (fmt, lint, knip); knip found one needless export, which is fixed.

The PR body's failure table now covers every change (lenient read, project resend, refused save, 500 cap, new tools). The PR stays a draft.

@scuttletoby

Copy link
Copy Markdown
Owner Author

Re-check lens (round 1) on d913a75: SAFE WITH NITS

Read-only re-check of fix round 1 (code reading; no node_modules in the reviewer's checkout, so no tests run).

Round-1 items: 1 lenient read FIXED · 2 project keys FIXED · 3 save refuses inherit/unknown names FIXED (both write paths, atomic under the write lock) · 4 500-pick cap PARTLY (documented only) · 5 person-only test FIXED (server guard deferred) · 6 groups FIXED · CI TS2379 FIXED · forkDocs rows present.

Findings, worst first

  • N1 (fail-open once wired) shared/src/forkSettingsPatch.ts replaceEntries/keepNewest (~49-62): the 501st pick silently drops the oldest, so that thread falls back to the server default (e.g. all). Fix: refuse a save that would go over the cap ("remove picks first") instead of trimming. Test idea:
    it("never widens a live thread to make room", () => {
      const full = { ...current, mcpThreadProfiles: Object.fromEntries(Array.from({ length: 500 }, (_, i) => [`old-${i}`, "t3-only"])) };
      const next = applyForkSettingsPatch(full, { mcpThreadProfiles: { [t("new")]: "t3-only" } }).mcpThreadProfiles!;
      expect(next["old-0"]).toBe("t3-only");
    });
  • N2 (low) server/src/fork/mcpProfile.ts:175-177: a hand-edited profile named like a built-in (all) is silently ignored; consider treating it as broken.
  • N3 (fails closed) deleting a profile still in use saves fine and then blocks every thread; the wiring PR should refuse that save.
  • N4 (nit) contracts/src/mcpProfile.ts:150-155: a newer build's non-text reference is rewritten as (unreadable object) on the next save (fails closed).
  • N5 (nit) server/src/fork/mcpProfile.ts:111: the three ungrouped tools are matched by name in any toolkit; pin each to its toolkit, and the later filter must treat undefined as not allowed.
  • Upstream edits (contracts/src/settings.ts, apps/server/src/serverSettings.ts): small, marked Fork:, safe; likeliest conflict is the makeTest pipe (~328-335).

Next: a fix round 2 for N1 (and N2, N5) before merge.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant