Skip to content

feat(orchestrator): read the step skills by name and complete them from the pack - #1566

Draft
edwinyjlim wants to merge 3 commits into
dag/6-bundled-prompts-onlyfrom
dag/7-repackaged-skills-only
Draft

edwinyjlim wants to merge 3 commits into
dag/6-bundled-prompts-onlyfrom
dag/7-repackaged-skills-only

Conversation

@edwinyjlim

@edwinyjlim edwinyjlim commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Behaviour

A task reads the step skills by their own names, and a step skill written once for every framework is always completed from the framework's pack.

  • The bundled prompts name the integration-* step skills, such as integration-install and integration-session-replay.
  • A skill whose menu entry has frameworkReferences is installed once a run into an emptied folder, completed from the framework's pack, and its rewritten SKILL.md is scanned.
  • When the skill cannot be completed, its folder is deleted and every task that names it fails, as it does when a skill fails to install. The run sends orchestrator repackaged skill unavailable once, with the skill, the reason and the framework.
  • A run that installed no pack fails before any task starts, the way it does for a skill the menu cannot supply.
  • install_skill completes such a skill the same way, in both sequences and both harnesses, through the same installer. It picks the pack with the run's match keys and reads it from a scratch folder. A run with no framework gets an error that says so, and nothing is installed.
  • --context-mill-version refuses a release older than 1.66.0, the first that publishes these skills. The message names that release.

The wizard-agent-dag-skills flag, the replaces mapping, the fallback to a replaced skill, the orchestrator_skills tag and pnpm verify:repackaged-skills are gone.

Fixtures

The skill menu and the step and pack pair are recorded from context-mill 1.66.0. pnpm record:context-mill-fixtures now records the pair too: the step skill as published, the pack files that completing it reads, and the completed skill. The recorded completed skill differs from the skill context-mill 1.66.0 publishes for the same step and framework in one line, the name in its frontmatter.

Verified

  • pnpm typecheck and pnpm lint (0 errors).
  • pnpm test: 292 files, 4,450 tests pass, 1 skipped.
  • repackaged-skills.test.ts and repackaged-skill-installer.test.ts cover each way a completion fails: a failed install, a pack that cannot complete the skill, a scan that rejects or fails, a link on the way, and a run with no pack.
  • tool-skill-install.test.ts covers install_skill: a plain skill, a completed one, a run with no framework, and a rejected scan. The pack it reads is gone from disk afterwards.
  • context-mill-version.test.ts covers the floor.

Changes in behaviour to know about

  • A run whose framework pack fails to install now fails before any task starts, when its flow has a skill completed from the pack. It used to continue without the pack.
  • A linear error tracking run that finds no PostHog integration installs integration-install and integration-init, from the menu's integration/install and integration/init categories. install_skill completes both from the framework's pack.
  • install_skill takes an exact menu id, as before. A name that is a skill family, such as integration-v2-install, is not found.
  • FEATURE_FLAGS_UNSUPPORTED still stops feature flags on Kotlin Multiplatform.

Rollout

This is the second of four stacked PRs that remove the old code paths. They merge together and ship in one release.

  • They wait until wizard-agent-dag and wizard-agent-dag-skills have both held at 100% with no stop rule tripped, no run that fetched its prompts, and no run that fell back to an old skill.
  • Both flags stay on in PostHog after the release. Released versions still read them, and a deleted flag reads as off.
  • Cloud moves to the release afterwards, with its context-mill pin at 1.66.0 or later.

🤖 Generated with Claude Code

edwinyjlim and others added 3 commits November 16, 2026 16:54
…om the pack

The bundled prompts name the integration-* step skills. A skill whose menu
entry has frameworkReferences is installed once a run and completed from
the framework's pack. When it cannot be completed, the tasks that name it
fail as they do when a skill fails to install, and a run that installed no
pack fails before any task starts. The wizard-agent-dag-skills flag, the
replaces mapping and the fallback to a replaced skill are removed.

A context-mill pin older than 1.66.0 is refused, since that is the first
release that publishes these skills.

The skill menu and the step and pack pair are recorded from context-mill
1.66.0, and record:context-mill-fixtures records the pair.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
install_skill completes a skill whose menu entry has frameworkReferences
from the framework's pack, through the installer the orchestrator uses,
with the same path and scan checks. The pack is picked with the run's
match keys and read from a scratch folder. A run with no framework gets
an error that says so, and nothing is installed.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A linear error tracking run installs integration-install and
integration-init when PostHog is not integrated yet. install_skill
completes both from the framework's pack.

Comments describe the step skills and the bundled prompts as they are.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@edwinyjlim
edwinyjlim added this pull request to stack #1569 November 17, 2026 01:22
@github-actions

Copy link
Copy Markdown

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

  • /wizard-ci all

Test all apps in a directory:

  • /wizard-ci ai-observability
  • /wizard-ci basic-integration
  • /wizard-ci feature-flags
  • /wizard-ci mcp-analytics
  • /wizard-ci replay-vision
  • /wizard-ci revenue
  • /wizard-ci self-driving
  • /wizard-ci warehouse
  • /wizard-ci warehouse-seeded

Test an individual app:

  • /wizard-ci ai-observability/anthropic
  • /wizard-ci ai-observability/google-adk
  • /wizard-ci ai-observability/groq
Show more apps
  • /wizard-ci ai-observability/manual-capture
  • /wizard-ci ai-observability/openai
  • /wizard-ci ai-observability/openai-agents
  • /wizard-ci ai-observability/opentelemetry
  • /wizard-ci ai-observability/vercel-ai
  • /wizard-ci basic-integration/android
  • /wizard-ci basic-integration/angular
  • /wizard-ci basic-integration/astro
  • /wizard-ci basic-integration/django
  • /wizard-ci basic-integration/fastapi
  • /wizard-ci basic-integration/flask
  • /wizard-ci basic-integration/flutter
  • /wizard-ci basic-integration/javascript-node
  • /wizard-ci basic-integration/javascript-web
  • /wizard-ci basic-integration/laravel
  • /wizard-ci basic-integration/next-js
  • /wizard-ci basic-integration/nuxt
  • /wizard-ci basic-integration/python
  • /wizard-ci basic-integration/rails
  • /wizard-ci basic-integration/react-native
  • /wizard-ci basic-integration/react-router
  • /wizard-ci basic-integration/sveltekit
  • /wizard-ci basic-integration/swift
  • /wizard-ci basic-integration/tanstack-router
  • /wizard-ci basic-integration/tanstack-start
  • /wizard-ci basic-integration/vue
  • /wizard-ci feature-flags/django
  • /wizard-ci feature-flags/next-js
  • /wizard-ci mcp-analytics/custom-dispatcher
  • /wizard-ci mcp-analytics/go-sdk
  • /wizard-ci mcp-analytics/python-sdk
  • /wizard-ci mcp-analytics/ruby-sdk
  • /wizard-ci mcp-analytics/rust-sdk
  • /wizard-ci mcp-analytics/typescript-sdk
  • /wizard-ci replay-vision/javascript-node
  • /wizard-ci replay-vision/next-js
  • /wizard-ci replay-vision/react-vite
  • /wizard-ci revenue/stripe
  • /wizard-ci self-driving/astro
  • /wizard-ci self-driving/fastapi
  • /wizard-ci self-driving/nuxt
  • /wizard-ci self-driving/react-router
  • /wizard-ci self-driving/sveltekit
  • /wizard-ci warehouse/monorepo-env
  • /wizard-ci warehouse/multi-source-next
  • /wizard-ci warehouse/stripe-node
  • /wizard-ci warehouse/zero-source
  • /wizard-ci warehouse-seeded/next-stripe
  • /wizard-ci warehouse-seeded/next-stripe-declined

Test against a Context Mill branch:

  • /wizard-ci all context-mill:my-branch

Add context-mill:<branch> to any command above to pin the Context Mill branch. It defaults to main.

Results will be posted here when complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant