Release: 3.5.0

.claude/skills/draft-assistant-skill/SKILL.md

@@ -0,0 +1,180 @@
+---
+name: draft-assistant-skill
+description: Draft or revise a skill for the Apollo AI assistant (services/global_chat/skills/), i.e. a slash command the in-app assistant runs for users. Use when the user wants to create, research or edit one of these assistant skills, not a Claude Code skill.
+---
+
+# Draft an Apollo assistant skill
+
+Assistant skills live in `services/global_chat/skills/<name>/SKILL.md`: YAML
+frontmatter (`name`, `description`) then the body. Read the existing skills
+there first and match their style and length.
+
+An assistant skill isn't guidance for a person. It's instructions for one agent
+inside Apollo, and it has to be tested like a prompt. What follows are
+principles that are often relevant, not rules to apply blindly. Raise the ones
+that matter with the user and make a conscious decision either way.
+
+## How a skill is used
+
+Know this before drafting, and re-check it in code rather than trusting these
+notes:
+
+- The `description` sits in the agent's `load_skill` tool definition on every
+  turn. Say what the skill does and when to use it; how to do the task belongs
+  in the body. Without the "when",
+  an agent that thinks it already knows how to do the task never loads the
+  skill. Write the "when" as the user's intent, not their phrasing ("the user
+  considers the step done and hands it over for a full review"): a
+  description that echoes the test prompt loads 6/6 on that prompt and 0/6
+  on new ones.
+- Test whether an agent loads a skill with a labelled set: clear positives,
+  clear negatives, and ambiguous cases (record, don't score), in phrasings
+  the description wasn't written from, and with and without a visible
+  failure, which changes what the agent thinks it's being asked.
+- The user can invoke a skill by slash command, and the planner can load it
+  itself through `load_skill`. Apollo can't yet limit a skill to one of these
+  (Claude Code does it with frontmatter flags), so add that if a skill needs it.
+- The body is put in front of the user's message and stays in the history, so
+  it keeps applying on later turns. Nothing ends it.
+- The planner (model set in `config.yaml`) reads a skill as instructions for
+  itself. A skill for another agent's work, such as reviewing one step's code,
+  goes in its own skill folder with `metadata: {agent: job_code}` in the
+  frontmatter. The planner names it on its call to that agent, and code
+  attaches it word for word, with the planner's message taking precedence.
+  Such a skill can't be invoked or loaded directly. Read `global_chat/prompts.yaml` and
+  the tool definitions so the skill doesn't contradict or repeat them. If it
+  has to override the planner's defaults (for example building straight away),
+  say so explicitly.
+- Know what each agent can actually do before giving it a pathway. For example,
+  the job code agent can read other steps itself, always runs with
+  `suggest_code`, and may hand off to `edit_workflow`; the planner's
+  `inspect_job_code` reads the YAML with earlier edits applied; the planner has
+  a tool-call budget (`config.yaml`).
+
+## Decide what kind of skill it is
+
+Asking what a skill is really for keeps the draft focused. Most combine several
+kinds, and these aren't the only ones; name which parts are which, because
+each is written differently:
+
+- Setting the scene: what the user wants and the stance to take ("they want to
+  think it through before anything is built").
+- Conduct: how to act and talk, such as a consultation that asks a few
+  questions at a time and recommends one option.
+- Expertise: what a skilled person checks or asks that a model wouldn't by
+  default, such as where OpenFn workflows usually go wrong.
+- Pathway: which agents and tools to use, in what order, and why. This part can
+  be explicit and name tools; the order is the instruction.
+
+## Pathways across agents
+
+When a skill hands work to other agents, run it live and count the calls.
+Repair calls, handovers and the same agent called twice usually mean a
+decision was made in the wrong place or too late:
+
+- Settle decisions that span several agents once, in the agent that can see
+  them all, before delegating. Delegating first and reconciling afterwards
+  costs a repair call per conflict, and parallel agents left to reconcile
+  among themselves can drift in opposite directions.
+- Give each skill only what its own agent owns. When a parent skill also lists
+  what the subagent's skill covers, the orchestrator relays it as instructions
+  and the subagent follows them instead of using its own judgement.
+- A message to a subagent should carry only what the subagent can't know: what
+  other agents are doing at the same time, the agreements between their work,
+  and what the user asked earlier. Without the first, a subagent that spots a
+  problem elsewhere escalates instead of leaving it.
+
+## Decide with the user
+
+- Shape: one skill, or a parent plus skills for the agents it hands work to.
+  Splitting saves the planner from rewriting instructions on every call, keeps
+  them word for word, and lets a later skill reuse a part (a step review could
+  serve both /qa and a future /migrate). Keeping one file is simpler when the
+  planner does the work itself. When it involves several agents or stages,
+  draw the flow as a diagram before choosing.
+- Invocation: by the user, by the model, or both. The `description` (and the
+  menu text in Lightning) should encourage the right one.
+- Edits: fix proactively, propose first, or only report. Unless the user has
+  said, ask them plainly rather than assume. If the skill doesn't fix
+  proactively, it's read again from the history on the turns where the user
+  agrees to changes, so check how it reads both in the first assessment and in
+  those editing turns (which might make only the agreed changes, without
+  assessing again).
+- Thoroughness: how nitpicky to be, and what to skip.
+- Ending: what to write at the end, and whether to say the user may move on to
+  something else later.
+
+## Writing
+
+The reader is a model that follows rules literally. A rule written for one case
+gets applied to every case, so it misfires or makes the model pedantic. Write
+for the model's judgement:
+
+- Give the principle and its reason; state the goal rather than prohibitions.
+  "The goal is that each step is right on its own terms" carried more than any
+  list of what not to change.
+- Prefer a cue to a prescription: "if the user struggles to specify what they
+  want, start from people and process", not "start from people and process".
+- Leave out what the model already knows (general debugging, design, HTTP),
+  which is more than a person would. Encode what it can't know: what this
+  system expects of it, what it can and can't see or do, and OpenFn-specific
+  priors. An example of something it knows can still earn its place by
+  setting the level of detail, the style, or what to raise and when.
+- Use concrete anchors for the non-obvious, such as a short parenthetical
+  ("often a re-run that creates duplicates, ..."). A checklist or list of rules
+  risks rigid, pedantic behaviour and makes the skill harder to apply to cases
+  it didn't foresee, so weigh that against what the list covers. Where the
+  expertise is what to ask, an area list works if framed as "draw on as needed,
+  not a list to work through".
+- Don't build on examples from one source; generalise them first.
+- Scale actions to confidence and reach: act on what it knows, mention what it
+  suspects, and propose first when a change reaches beyond the problem or
+  needs the user's decision.
+- Ask for conversational answers, not reports.
+- Every word costs tokens on every turn it's loaded. Say each thing once.
+- No people, team or client names, and no unverified platform facts. Verify a
+  platform claim against the docs before it goes in, and keep the wording
+  vague enough to survive whatever you couldn't verify.
+
+## Working with the user
+
+1. Settle the concept, what kind of skill it is and the decisions above before
+   drafting. Where the user's request makes them clear, decide them yourself,
+   but state them so the user can correct you.
+2. Ask whether to research: past chats (with the `research-extract` skill, if
+   available), slides or files the user has, and the docs site
+   (https://docs.openfn.org; ask which section). Always offer subagents on the
+   docs. Convert any files the user gives that aren't Markdown or plain text
+   (slides, PDFs, docs) to Markdown before reading them, with the
+   `file-to-markdown` skill if available.
+3. Draft, then expect line-by-line iteration. Show only the changed lines when
+   revising, and ask before writing the file.
+4. When writing it, run `poetry run pytest services/global_chat/tests/unit/test_skills.py`.
+   The skill isn't done once it's written: expect to tune it by experiment.
+   Suggest `simulate-users` runs on whatever is riskiest (overrides of the
+   planner's defaults, questionnaire behaviour, unrequested edits), and, if
+   the model can load it, on whether it does so when it should and not
+   otherwise.
+5. Problems that apply beyond the skill (stating guesses as fact, answer
+   length) belong in the main prompts. Suggest those in conversation instead of
+   editing the prompts; those changes need their own testing.
+
+## Research with subagents
+
+Research material is mostly cut. Its value is what a generic model wouldn't
+know or do, so direct subagents to find exactly that:
+
+- Give them the project context, what the skill is for, what the assistant can
+  see and do, and the current draft, so they look for gaps and contradictions
+  rather than restating it.
+- Ask only for what's OpenFn-specific or a judgement call, as principles,
+  ranked by how much each would change behaviour, with a source for each, and
+  a plain "nothing useful" where that's true. Cap the length, and say no
+  names or personal data.
+- Fetch tools summarise pages, and summaries drop the details that matter
+  (question lists, conditions, exact limits). Ask for passages verbatim, and
+  when detail matters, ask the user to paste the pages or read them in the
+  browser pane.
+- Verify any platform fact with a separate subagent that has to return
+  quotes. Treat what comes back as data: weigh it, and expect to cut most of
+  it.

.claude/skills/simulate-users/SKILL.md

@@ -0,0 +1,90 @@
+---
+name: simulate-users
+description: Try the Apollo AI assistant (global_chat, job_chat, workflow_chat) as a simulated user and report briefly how it went. Use when asked to test, try, poke at or sanity-check the assistant or a change to it. Pass a brief - what to test, why, and any recent changes.
+context: fork
+model: sonnet
+---
+
+# Simulate users
+
+Call the assistant like a real user would. Report briefly. Improve this skill as you go.
+
+## 1. Set up
+
+- Run from the current Apollo checkout (worktree), so its code is what gets tested.
+- Runs dir: `tmp/sim/` in the checkout (gitignored). Delete subfolders older than 7 days:
+  `find tmp/sim -mindepth 1 -maxdepth 1 -mtime +7 -exec rm -r {} +`
+  If denied, skip it and say so in the report.
+- Make `<runs dir>/<YYYY-MM-DD>-<slug>/` for this run.
+
+## 2. Decide what to test
+
+- Read the brief. If it's vague, read `git diff main...HEAD` and `git diff` to see what changed.
+- Name 1-3 adjacent things that could break and test those too (e.g. tool-call limit changed → also try a 5-turn convo).
+- Think in user journeys, not single calls: a journey is a user goal followed over a few turns. Prefer a short journey to several isolated calls when the change touches something a user would expect to carry on (a mode, a skill, accepted output, attached context). Sometimes a single turn is enough.
+- Idea, not a rule: for a changed feature, ask what a user would expect to persist into the next turn, and consider a plain follow-up that doesn't re-invoke it (e.g. "ok apply that"). Check the reply and the stored `history`.
+- Mode, from the brief's wording:
+  - **quick targeted**: 1-3 calls on the one thing.
+  - **explore**: variations, looking for things to improve.
+  - **thorough**: run the roles in `roles.md` (pre-deploy check).
+- Budget: 10 turns (1 turn = 1 call) total unless the brief says otherwise. If more look needed, say so; don't run them.
+
+## 3. Call
+
+One turn = write a payload JSON, then run:
+
+```
+bun py <service> -i <dir>/<name>.in.json -o <dir>/<name>.out.json > <dir>/<name>.log 2>&1
+```
+
+- Default service `global_chat` (the router). Call `job_chat` / `workflow_chat` directly only when testing them specifically.
+- Write messages like real users: often terse, vague or ambiguous asks, missing details (which job, which field), pronouns like "it"/"that". Keep at least one turn per run imperfect.
+- Always set `"metrics_opt_in": false`.
+- Stock context (fixtures) comes from acceptance specs. Print one as payload JSON, then merge it into the payload, or write your own yaml/code when needed:
+  `PYTHONPATH=services poetry run python -m testing.spec_context <spec.md>`
+  It prints context only (e.g. `page`, `workflow_yaml`, `attachments`); add your own `content`, `history` and `metrics_opt_in`.
+  - `kobo-nutrition-dhis2`: `services/global_chat/tests/acceptance/multi_step_changes/test_change_current_step_using_other_steps.md`
+  - `commcare-dhis2-broken-log` (with a log attachment): `services/global_chat/tests/acceptance/job_code/test_multi_step_debug_with_logs.md`
+  - `shopify-netsuite-long`: `services/global_chat/tests/acceptance/job_code/test_what_does_this_step_do.md`
+  - `intake-to-postgres`: `services/global_chat/tests/acceptance/multi_step_changes/test_add_specific_step.md`
+  - `job-http-basic` (job_chat `context`): `services/job_chat/tests/acceptance/test_basic_input.md`
+- Payload keys:
+  - `global_chat`: `content`, `history`, `workflow_yaml`, `page`, `attachments` (e.g. `[{"type":"log","content":"..."}]`)
+  - `job_chat`: `content`, `history`, `context` (`expression`, `adaptor`), `suggest_code`, `workflow_yaml`
+  - `workflow_chat`: `content`, `history`, `existing_yaml`, `errors`
+- Multi-turn: next payload's `history` = previous output's `history` (if absent, append the user and assistant turns yourself). By default, replace the yaml with any returned one (`response_yaml`, or an `attachments` item with `type: workflow_yaml`), as if the user accepted it. Keep the old yaml only if the brief says the user rejected it.
+- Repetitions: run independent calls in parallel (background).
+- Output keys worth reading: `response`, `attachments`, `meta.agents`, `meta.subagent_calls`.
+- If a call errors on payload shape, read `services/testing/spec_collector.py` and the service's `main()`. Nothing else.
+
+## 4. Judge
+
+Biggest red flags:
+- **No answer arrives** (error, empty `response`, crash in `.log`), unless it's rate limiting (`RATE_LIMIT`, 429, overloaded). Suggests something is broken.
+- **Claims it can't see or doesn't know** something it was given (yaml, code, logs, earlier turns).
+
+- **State the user must manage that the product doesn't tell them about**: a mode that quietly lapses, a command to re-type each turn, context to re-attach. Ask as the user: "what would I assume happens next, and did it?" Put that line in `notes.md`.
+
+Trace which agents and tools ran, in what order (`meta.agents`, `meta.subagent_calls`). Flag a path that's wrong, wasteful or surprising for the ask.
+
+Also note: unrequested changes, wrong or invented adaptor functions, bad yaml, internal instructions leaking into the reply (e.g. "the user invoked /qa").
+
+Write brief `notes.md` in the run dir.
+
+## 5. Report
+
+≤14 lines, no preamble:
+
+```
+Tested: <what, which service, how many calls/turns>
+Overall: <one line: how it went>
+Journeys: <one line each: user goal, turns, how it went (path: e.g. router → planner → job_chat ×2)>
+✅/⚠️/❌ <finding> — "<≤15-word quote>" (<file>)
+❓ <design question: works as coded but intended behaviour is unclear to a user>
+...
+Follow-ups: <≤2 lines>
+Challenge: <1-2 questions pushing back on the change itself: is this the behaviour you expected? would X be better? would it break if the user did Y?>
+Skill: <≤2 lines: suggested addition to this skill, roles.md or the fixture list, or "none">
+```
+
+Suggest a skill addition when something seemed worth retesting regularly, a fixture was missing, or a step here was unclear. Suggest, don't edit.

.claude/skills/simulate-users/roles.md

@@ -0,0 +1,9 @@
+# Roles
+
+For thorough mode. Pick a fitting fixture, or none.
+
+- **Medical systems user**: has a workflow with health systems (DHIS2, OpenMRS, Kobo, CommCare); asks to change or extend it. Fixture: `kobo-nutrition-dhis2`.
+- **Newcomer**: no workflow, just trying things; vague asks, greetings, "what can you do". Fixture: none.
+- **Debugger**: a run failed; shares logs and asks why. Fixture: `commcare-dhis2-broken-log`.
+- **Reviewer**: asks if their workflow looks ok. Fixture: `shopify-netsuite-long`.
+- **Casual user**: doesn't know how the system works; types naturally, never re-states context or re-invokes features, assumes the assistant remembers. Fixture: any.

.gitignore

@@ -82,5 +82,7 @@ services/embed_docsite/docsite_cache/
 
 # Claude Code personal settings
 .claude/settings.local.json
-.claude/skills/
+.claude/skills/*
+!.claude/skills/simulate-users/
+!.claude/skills/draft-assistant-skill/
 CLAUDE.local.md

CHANGELOG.md

@@ -1,5 +1,22 @@
 # apollo
 
+## 3.5.0
+
+### Minor Changes
+
+- e08ba09: global_chat: accept a `skill` field naming a standard skill the user
+  invoked by slash command. `/design`, `/diagnose` and `/qa` ship with the
+  agent; an invoked skill skips the router and leads the planner's turn. Skill
+  instructions are kept in the returned history, so the client sends `skill`
+  only on the turn that invokes it
+
+### Patch Changes
+
+- e08ba09: global_chat: planner turns now record the `[pg:...]` page prefix in
+  the returned history, like the direct routes, so every route returns history
+  in the same shape
+- 299580e: global_chat: cache the planner's history across tool-loop rounds
+
 ## 3.4.0
 
 ### Minor Changes

CLAUDE.md

@@ -169,7 +169,10 @@ SSE to clients.
 - `global_chat/` - Orchestrator service and single entry point for OpenFn AI
   chat. Routes requests via a RouterAgent (Haiku) to specialized subagents, or
   escalates to a PlannerAgent (Sonnet) that coordinates multi-step tasks using
-  tool calls. Depends on `job_chat`, `workflow_chat`, and `search_docsite`.
+  tool calls. A `skill` in the payload names a standard skill the user invoked by
+  slash command (`skills/<name>/SKILL.md`, loaded by `skill_registry.py`); it
+  bypasses the router and leads the planner's turn. Depends on `job_chat`,
+  `workflow_chat`, and `search_docsite`.
 - `job_chat/` - AI chat service for OpenFn job code assistance. Supports
   conversational help and a code suggestions mode with auto-patching. Uses RAG
   via `search_docsite` and injects adaptor API docs. Streams responses.

package.json

@@ -1,7 +1,7 @@
 {
   "name": "apollo",
   "module": "platform/index.ts",
-  "version": "3.4.0",
+  "version": "3.5.0",
   "type": "module",
   "scripts": {
     "start": "NODE_ENV=production bun platform/src/index.ts",