diff --git a/src/content/docs/factories/factory-as-code.mdx b/src/content/docs/factories/factory-as-code.mdx index 67be56715..8b459fe25 100644 --- a/src/content/docs/factories/factory-as-code.mdx +++ b/src/content/docs/factories/factory-as-code.mdx @@ -1,8 +1,8 @@ --- title: Factory definition syntax description: >- - Look up every file and key in a factory definition: factory.yaml, agents, - automations, runners, benchmarks, scorers, skills, and webhooks. + Look up every file and key in a factory definition, fetch the JSON Schema + behind them, and validate a change before it applies. sidebar: label: "Definitions as code" --- @@ -10,7 +10,9 @@ import { VARS } from '@data/vars'; Every factory is defined by files: a `factory.yaml` plus directories of agents, automations, runners, benchmarks, scorers, skills, and webhooks, versioned in a Git repository. The files are the source of truth — when they change, Warp updates the factory to match. -Definition files are YAML and Markdown. Keys are case-sensitive. +Definition files are YAML and Markdown. Keys are case-sensitive, and the YAML is plain: anchors, aliases, tags, and duplicate keys are rejected. The accepted files and keys are published as a [JSON Schema](#json-schema) for editors and coding agents; use the [validation paths](#validate-a-definition) for parser checks and cross-file rules. + +Warp validates every change before it applies it. On a GitHub-backed factory, each pull request gets a **warp/factory-config** check that you can make required, and you can run the same parser yourself while you edit, from a script, a CI job, or a coding agent. See [Validate a definition](#validate-a-definition). For complete working definitions you can copy, see the [warp-factory-examples](https://github.com/warpdotdev/warp-factory-examples) repository. Its examples range from a single-repo quickstart to a full issue-to-PR lifecycle. @@ -19,7 +21,7 @@ For complete working definitions you can copy, see the [warp-factory-examples](h You choose who hosts the definition repository when you create a factory: * **Warp-managed (default)** - Warp hosts the repository for you. You edit the factory in the [{VARS.FACTORY_WEB_APP}](/factories/factory-dashboard/), and every change is validated, committed to the files, and applied in one step. You never interact with the repository directly, and the definition can't end up in an invalid state. -* **GitHub** - The definition lives in a repository you own. The repository is the only way to change the factory: the web app shows the configuration read-only and links back to the files. Open a pull request, and any change merged to the production branch (`main` by default) updates the factory. See [Pull request checks](#pull-request-checks-for-github-backed-factories). +* **GitHub** - The definition lives in a repository you own. The repository is the only way to change the factory: the web app shows the configuration read-only and links back to the files. Open a pull request, and any change merged to the production branch (`main` by default) updates the factory. See [Pull request checks](#pull-request-checks). Both modes use the same files, so everything on this page applies to either. You can also link a GitHub repository to a Warp-managed factory later. @@ -27,6 +29,79 @@ Definition files describe how the factory is configured, not what it is doing: w {/* VISUAL: The Factory definition tab's file browser for a Warp-managed factory -- this reference page has no screenshots. */} +## Validate a definition + +Every check below runs the same parser Warp uses to apply a definition, and reports each problem with its file, line, and an `FF_*` code such as `FF_UNKNOWN_FIELD` or `FF_INVALID_REFERENCE`. + +### Pull request checks + +When the definition lives in GitHub, every pull request that targets the production branch gets a **warp/factory-config** check. It reports each problem with its file and line, and a pull request that doesn't change the definition passes immediately. Make it required in your repository's branch protection rule or ruleset to stop an invalid definition from merging. + +When a change lands on the production branch, Warp applies it as a whole: a definition that fails validation never partially applies, and the factory keeps running its last valid definition until the branch is fixed. Warp-managed factories validate each edit when you save it in the web app instead. + +For what the check validates under the hood and how it names a subdirectory definition, see [the GitHub integration reference](/factories/integrations/github/#factory-definition-pull-request-checks). + +### Validate locally or in CI + +To check a tree before you open a pull request, or from your own CI, send it to the validation endpoint. `POST` the files to `https://app.warp.dev/api/v1/factory-files/validate` as JSON, each with its `path` relative to the factory root and its full `content`. No login is needed, and no factory has to exist yet. Send the whole tree, not only the file you changed: rules like "exactly one foreman" and "an automation's `agent` names a declared agent" span files. + +```json +{ + "schema_version": "v1alpha1", + "valid": false, + "diagnostics": [ + { + "path": "factory.yaml", + "line": 8, + "column": 3, + "code": "FF_UNKNOWN_FIELD", + "message": "unknown field \"runnr\"" + } + ], + "deferred_resolutions": [], + "state_dependent_checks_not_run": [ + "provider_alias_resolution", "model", "environment", "secret", "runner_reference", + "mcp_server", "integration", "worker_host", "entitlement" + ] +} +``` + +The [`validate_factory_files.py`](https://github.com/warpdotdev/warp-factory-examples/blob/main/scripts/validate_factory_files.py) script in warp-factory-examples does this for you. It needs only Python 3, so you can copy it into your own repository: + +```bash +python3 scripts/validate_factory_files.py path/to/factory-root +``` + +It exits `0` when the tree is valid, `1` when the server reported diagnostics, and `2` when the tree wasn't checked, which is not a pass. For a CI job built on it, see the example repository's [validation workflow](https://github.com/warpdotdev/warp-factory-examples/blob/main/.github/workflows/validate.yml). + +A clean result means the tree parses and passes every check that doesn't need your team's state. Whether a model ID, secret name, runner name, environment ID, MCP server ID, or integration exists is settled by the pull request check and when the change is applied; the response lists those under `state_dependent_checks_not_run`. + +### Validate with a coding agent + +* **In Warp** - Ask the Warp Agent to change or check a factory definition. Its built-in `factory-files` skill validates the result before opening a pull request. For example: "Add a nightly dependency-audit automation to this factory and validate the definition." +* **Through Factory MCP** - Connect any other coding agent to [Factory MCP](/factories/factory-mcp/) for the same schema and validation tools. See [author and validate factory definitions](/factories/factory-mcp/#author-and-validate-factory-definitions). +* **Anywhere else** - Have the agent call the [validation endpoint](#validate-locally-or-in-ci) or run the validator script itself. + +## JSON Schema + +Warp publishes the definition format as JSON Schema (draft 2020-12) documents, generated from the parser that validates your files. Every field carries a description, and an unknown field is an error, so the schema is the exhaustive answer to what a file accepts. The endpoints are unauthenticated: + +* `https://app.warp.dev/api/v1/factory-files/schemas` - The supported schema versions, and which one is current. +* `https://app.warp.dev/api/v1/factory-files/schemas/v1alpha1` - Every `v1alpha1` document in one bundle, keyed by document name. +* `https://app.warp.dev/api/v1/factory-files/schemas/v1alpha1/` - One document on its own, usable directly as a schema reference. + +There is one document per file kind: `factory.schema.json` for `factory.yaml`; `agent.schema.json`, `automation.schema.json`, and `scorer.schema.json` for the frontmatter of the corresponding Markdown files; `runner.schema.json`, `webhook.schema.json`, `benchmark_suite.schema.json`, and `benchmark_suite_task.schema.json` for the YAML files; and `common.schema.json` for the definitions the others share. + +Editors with a YAML language server can read a document from its URL, for completion and inline validation as you type. In VS Code with the YAML extension, add the reference as a comment on the file's first line: + +```yaml title="factory.yaml" +# yaml-language-server: $schema=https://app.warp.dev/api/v1/factory-files/schemas/v1alpha1/factory.schema.json +schemaVersion: v1alpha1 +name: payments-factory +``` + +Editors apply YAML schemas to `.yaml` files, not to the frontmatter of the Markdown files, so check those with one of the [validation paths](#validate-a-definition) above. + ## Directory structure Each resource takes its name from its path: `agents/reviewer/agent.md` defines an agent named `reviewer`. @@ -579,15 +654,9 @@ agentDefaults: Pair `workerHost` with a runner whose `platform` matches the worker's operating system and architecture. See [choose an execution host](/factories/infrastructure-and-security/#choose-an-execution-host) for the full setup, including how to deploy and connect the worker. For a working definition, see [`07-self-hosted-worker`](https://github.com/warpdotdev/warp-factory-examples/tree/main/examples/07-self-hosted-worker). -## Pull request checks for GitHub-backed factories - -When your definition lives in GitHub, Warp validates every change before it takes effect: - -* Every pull request that targets the production branch gets a **warp/factory-config** check. The check annotates invalid fields and unresolvable references with the file and line that caused them, and summarizes what the change would apply. -* When a change lands on the production branch, Warp applies it as a whole. A definition that fails validation never partially applies: the factory keeps running on its last valid definition until the branch is fixed. - -Warp-managed factories skip all of this. Every edit in the web app is validated when you save it, so the definition can't become invalid. - -## Machine-readable schema +## Related pages -Warp publishes the definition schema as JSON Schema documents, generated from the same parser that validates your files. Fetch the supported versions from `https://app.warp.dev/api/v1/factory-files/schemas` and the `v1alpha1` documents from `https://app.warp.dev/api/v1/factory-files/schemas/v1alpha1`. Both endpoints are unauthenticated, so editors and agents can validate a definition without a Warp login. +* [**Factory MCP for coding agents**](/factories/factory-mcp/) - Read the schema and validate a tree from any coding agent, and send work to a factory. +* [**GitHub integration**](/factories/integrations/github/#factory-definition-pull-request-checks) - How the **warp/factory-config** check appears on pull requests, and what to check when it doesn't. +* [**Factory dashboard**](/factories/factory-dashboard/#edit-definitions-in-the-factory-definition-tab) - Where a Warp-managed definition is edited and validated on save. +* [**warp-factory-examples**](https://github.com/warpdotdev/warp-factory-examples) - Complete definitions to copy, plus the validator script and a CI workflow that runs it. diff --git a/src/content/docs/factories/factory-mcp.mdx b/src/content/docs/factories/factory-mcp.mdx index 5983b8a57..1788788f0 100644 --- a/src/content/docs/factories/factory-mcp.mdx +++ b/src/content/docs/factories/factory-mcp.mdx @@ -23,6 +23,7 @@ The factory keeps a single record of each task throughout. Whether a change happ * **Continue a task locally** - Pull a task's context into your own checkout, work with your own tools, and return the result to the same task. * **Stay in sync** - List and search tasks, read a task's conversation, and message its [foreman](/factories/factory-agents/), the agent that orchestrates each task inside the factory. * **Create a factory** - Let your coding agent guide you through choosing a team, code host, repositories, factory agents, and integrations. +* **Edit a factory's definition** - Read the definition schema and validate a factory's [definition files](/factories/factory-as-code/) before opening a pull request. Factory MCP is one of several ways work enters a factory, alongside Slack, GitHub, GitLab, Linear, and Jira. See [connect your factory](/factories/connect-your-factory/) for all intake paths and [how Warp Factories work](/factories/how-factories-work/) for how tasks move through a factory. @@ -117,6 +118,7 @@ Factory MCP exposes a small set of tools that your agent calls on your behalf. Y * "What's the status of the checkout-flow task?" * "Pull down ENG-123 so we can finish it here." * "Set up a factory for me." +* "Add a review agent to this factory definition and validate it." ## Send new work to a factory @@ -144,6 +146,15 @@ When nothing remains for the factory to do, `complete_task` closes the task out. Sending work to a factory means you're no longer watching it. To be notified when a task needs attention or finishes, ask for a notification when sending or returning work: your agent calls `list_notification_routes` to see the destinations available to you in that factory, such as a Slack DM or a Linear issue, and passes your choice to `send_task`. Delivery is best-effort, so treat notifications as a convenience rather than a guarantee. +## Author and validate factory definitions + +Two tools cover editing a factory's [definition files](/factories/factory-as-code/) before you open a pull request. Neither needs a factory ID, and both work before the factory exists: + +* `get_factory_file_schema` returns the JSON Schema for one definition document, or the full catalog when called with no arguments. +* `validate_factory_files` checks a complete tree — every file's `path` and `content` — without saving or applying it, and returns any diagnostics. + +A prompt like "Add a review agent to this factory definition and validate it" is enough: the agent reads the schema, edits, and validates the tree before opening a pull request. For the schema catalog, validation semantics, and the pull request check, see [Validate a definition](/factories/factory-as-code/#validate-a-definition). + ## Tool reference Your MCP client fetches the full input schemas from the server, and tool results include links that open the corresponding task or run in the factory's [factory dashboard](/factories/factory-dashboard/). @@ -152,8 +163,8 @@ The onboarding tools from `list_teams` through `create_factory` require browser | Tool | What it does | | --- | --- | | `list_factories` | Lists the factories you can access. | -| `get_factory_file_schema` | Returns the current schemas for factory configuration files. | -| `validate_factory_files` | Validates a complete factory file tree without saving or applying it. | +| `get_factory_file_schema` | Returns the JSON Schema documents for factory definition files, as a catalog or one document at a time. | +| `validate_factory_files` | Validates a complete factory definition tree without saving or applying it. | | `list_teams` | Lists current memberships and first-time joinable team choices. | | `create_team` | Creates the authenticated user's first team with a confirmed name. | | `join_team` | Joins a team selected from the first-time discovery choices. | @@ -175,6 +186,7 @@ The onboarding tools from `list_teams` through `create_factory` require browser ## Related pages * [**Connect your factory**](/factories/connect-your-factory/) - Every way work can enter a factory, including the Slack, GitHub, GitLab, Linear, and Jira integrations. +* [**Definitions as code**](/factories/factory-as-code/) - Every file and key in a factory definition, the JSON Schema behind them, and how to validate a change. * [**Factory agents**](/factories/factory-agents/) - The foreman and the other agents that carry out a factory's tasks. * [**How Warp Factories work**](/factories/how-factories-work/) - The task lifecycle and the agents that move work through it. * [**Warp Factories quickstart**](/factories/quickstart/) - Create a factory and send it its first work item. diff --git a/src/content/docs/factories/factory-skills.mdx b/src/content/docs/factories/factory-skills.mdx index a5c5ad333..74f968f50 100644 --- a/src/content/docs/factories/factory-skills.mdx +++ b/src/content/docs/factories/factory-skills.mdx @@ -59,7 +59,7 @@ A skill changes what an agent knows how to do, not what it can reach. To scope a Where you edit a skill depends on [where the factory's definition lives](/factories/factory-as-code/#where-the-definition-lives): * **Warp-managed** - Add or edit `SKILL.md` files directly in the **Factory definition** tab of the [factory dashboard](/factories/factory-dashboard/). Saving validates and commits the change in one step. -* **GitHub** - Add or edit the files in the connected definition repository and open a pull request. The same [pull request checks](/factories/factory-as-code/#pull-request-checks-for-github-backed-factories) that validate the rest of the definition apply to skill files. +* **GitHub** - Add or edit the files in the connected definition repository and open a pull request. The same [pull request checks](/factories/factory-as-code/#pull-request-checks) that validate the rest of the definition apply to skill files. For worked examples, including a factory-wide skill and a per-agent skill together, see [`02-sdlc-issue-to-pr`](https://github.com/warpdotdev/warp-factory-examples/tree/main/examples/02-sdlc-issue-to-pr) in the [warp-factory-examples](https://github.com/warpdotdev/warp-factory-examples) repository. diff --git a/src/content/docs/factories/how-factories-work.mdx b/src/content/docs/factories/how-factories-work.mdx index a051a018c..a1c06389a 100644 --- a/src/content/docs/factories/how-factories-work.mdx +++ b/src/content/docs/factories/how-factories-work.mdx @@ -67,6 +67,6 @@ The first two are workflow policy, written into the foreman's instructions; edit Your factory is self-improving, and you define what "better" means. [Scorers](/factories/measure-and-improve/) classify completed runs against criteria you write, and [Self-improvement](/factories/measure-and-improve/#configure-and-review-self-improvement) groups the failures they flag into follow-up runs that propose fixes — to the application code or to the factory's own definition. Every proposal arrives as a change for your review; nothing is adopted on its own. -The factory's definition is open to the same loop. Anyone on the team, or an agent, can propose changes to its instructions, skills, models, or other [definition files](/factories/factory-as-code/), and definitions stored in GitHub go through pull request review and [configuration checks](/factories/factory-as-code/#pull-request-checks-for-github-backed-factories) before a change reaches the production branch. +The factory's definition is open to the same loop. Anyone on the team, or an agent, can propose changes to its instructions, skills, models, or other [definition files](/factories/factory-as-code/), and definitions stored in GitHub go through pull request review and [configuration checks](/factories/factory-as-code/#pull-request-checks) before a change reaches the production branch. See [measure and improve](/factories/measure-and-improve/) for the evaluation workflow, or [build a self-improving agent](/guides/agent-workflows/build-a-self-improving-agent/) to apply the same pattern to a standalone agent. diff --git a/src/content/docs/factories/integrations/github.mdx b/src/content/docs/factories/integrations/github.mdx index a433ac23e..45fbf8f97 100644 --- a/src/content/docs/factories/integrations/github.mdx +++ b/src/content/docs/factories/integrations/github.mdx @@ -140,9 +140,9 @@ For the full credential model, see [Permissions and identity](/platform/integrat ## Factory-definition pull request checks -If the factory's [definition is managed as code](/factories/factory-as-code/) in a GitHub repository, Warp reviews changes to it the way CI reviews code. Open a pull request that touches the definition files and a **warp/factory-config** check runs: it passes with a summary of what the change does, or fails with the specific fields to fix. Require the check in branch protection to stop an invalid definition from merging. +If the factory's [definition is managed as code](/factories/factory-as-code/) in a GitHub repository, Warp reviews changes to it the way CI reviews code. Open a pull request that touches the definition files and a **warp/factory-config** check runs: it validates the head commit and dry-runs the change against your team, so it also catches a secret, runner, MCP server, or model that doesn't exist. It passes with a summary of what merging would create, update, and delete, or fails with the specific fields to fix. If the definition lives in a subdirectory of the repository, the directory is appended to the check name, as in **warp/factory-config (factory)**. Require the check in branch protection to stop an invalid definition from merging. -These checks validate the factory's configuration files only. They don't create work items, and pull requests that don't touch the factory directory don't get the check. +These checks validate the factory's configuration files only. They don't create work items, and a pull request that doesn't touch the factory directory passes the check immediately, so requiring it doesn't hold up unrelated work. For what the check validates in the parser, and how to run the same validation before you open the pull request, see [Validate a definition](/factories/factory-as-code/#validate-a-definition). ## Troubleshooting @@ -168,4 +168,4 @@ Check that the installation still covers the target repository and grants the re ### A factory-definition check doesn't appear -The check runs only for factories whose [definition is managed as code](/factories/factory-as-code/) in a GitHub repository. Confirm the pull request targets the branch the factory runs from, that it changes files in the factory's definition directory, and that the GitHub App covers the repository. +The check runs only for factories whose [definition is managed as code](/factories/factory-as-code/) in a GitHub repository. Confirm the pull request targets the branch the factory runs from and that the GitHub App covers the repository. A pull request that doesn't change definition files still gets the check; it passes with "No factory configuration changes". diff --git a/src/integrations/docs-markdown-integration.js b/src/integrations/docs-markdown-integration.js index 6963f1089..5f1df08b2 100644 --- a/src/integrations/docs-markdown-integration.js +++ b/src/integrations/docs-markdown-integration.js @@ -143,14 +143,16 @@ function createMarkdownConverter() { replacement(_content, node) { if (!isElement(node)) return '\n\n'; + const pre = node.querySelector('pre'); const code = node.querySelector('pre code'); - if (!code) return '\n\n'; + if (!pre || !code) return '\n\n'; - const language = - code.getAttribute('data-language') ?? code.className.match(/language-([\w-]+)/)?.[1] ?? ''; - const rawCode = normalizeNewlines(code.textContent ?? '').replace(/\n$/, ''); + const rawCode = getCodeBlockText(code); const fence = getFence(rawCode); - const openingFence = language ? `${fence}${language}` : fence; + const info = [getCodeBlockLanguage(pre, code), getCodeBlockTitleAttribute(node)] + .filter(Boolean) + .join(' '); + const openingFence = info ? `${fence}${info}` : fence; return `\n\n${openingFence}\n${rawCode}\n${fence}\n\n`; }, @@ -163,6 +165,43 @@ function isElement(node) { return node.nodeType === 1; } +// Expressive Code renders each source line as its own `div.ec-line` and never +// emits newline characters between them, so `textContent` on the `` +// element runs every line together. Rebuild the text line by line, taking +// only each line's `.code` cell so gutter content (line numbers) stays out. +// An empty source line is rendered as a cell holding a lone newline +// character, so newlines inside a cell are dropped rather than kept. +function getCodeBlockText(code) { + const lines = Array.from(code.querySelectorAll('.ec-line')); + const text = + lines.length > 0 + ? lines + .map((line) => ((line.querySelector('.code') ?? line).textContent ?? '').replace(/\r?\n/g, '')) + .join('\n') + : (code.textContent ?? ''); + return normalizeNewlines(text).replace(/\n$/, ''); +} + +// Expressive Code puts `data-language` on the `
`; the `` fallbacks
+// cover blocks rendered by anything else.
+function getCodeBlockLanguage(pre, code) {
+	return (
+		pre.getAttribute('data-language') ??
+		code.getAttribute('data-language') ??
+		code.className.match(/language-([\w-]+)/)?.[1] ??
+		''
+	);
+}
+
+// A block's title (the file name on a ```yaml title="factory.yaml" fence) is
+// often the only thing that says which file a snippet belongs to, so it is
+// carried on the fence's info string in the same form the source uses.
+function getCodeBlockTitleAttribute(block) {
+	const title = block.querySelector('figcaption .title')?.textContent?.trim() ?? '';
+	// JSON.stringify yields a double-quoted string with backslashes and quotes escaped.
+	return title ? `title=${JSON.stringify(title)}` : '';
+}
+
 function getFence(code) {
 	const matches = code.match(/`+/g) ?? [''];
 	const maxBackticks = Math.max(...matches.map((value) => value.length));