diff --git a/README.md b/README.md
index 757696323..51f1d8fff 100644
--- a/README.md
+++ b/README.md
@@ -4,261 +4,232 @@
Your GPU, as something an agent can drive.
-diffusers-workflow wraps the [Hugging Face Diffusers library](https://github.com/huggingface/diffusers)
-in an engine that runs image, video and audio generation as jobs, and puts
-two front ends on it: an **MCP server**, so Claude Code (or any MCP client)
-can author, run and inspect generations; and a **web UI** for doing the same
-by hand. A CLI sits underneath for when you want neither.
+diffusers-workflow runs [Hugging Face Diffusers](https://github.com/huggingface/diffusers)
+image, video and audio generation as jobs on your own GPU. **An agent does the work.**
+Claude Code, or any MCP client, picks or writes the workflow, checks what it will
+cost, runs it, and looks at what came back. **You steer from the browser.** Every
+workflow, run and output the agent made is in the web UI, where you can change a
+variable, run it again, compare takes and keep the one you like.
**Python 3.12-3.14 | CUDA (NVIDIA) | MPS (Apple Silicon) | CPU**
-
-
+
## Getting started
-**1. Install.** The script picks the right torch build for your platform,
-creates a virtual environment and installs everything, MCP server included.
+**1. Install.** The script picks the right torch build for your platform, creates a
+virtual environment and installs everything, MCP server included.
```bash
-# Linux / macOS
-bash ./install.sh
-source ./activate
-
-# Windows
-.\install.ps1
-.\venv\scripts\activate
+bash ./install.sh && source ./activate # Linux / macOS
+.\install.ps1; .\venv\scripts\activate # Windows
```
-`python -m dw.test` runs a small built-in workflow end to end - it downloads
-SD 1.5 (a few GB) and generates one image on whichever accelerator was found.
-
-**Or run it in Docker** (Linux with an NVIDIA GPU). The image
-[`dkackman/diffusers-workflow`](https://hub.docker.com/r/dkackman/diffusers-workflow)
-has torch, the engine and the web UI built in, and it shares the model cache
-and workspace with a native install on the same box. diffusers itself lives
-on a volume, so it can follow GitHub main. That makes step 2 a container:
+**2. Start the engine.** Leave it running; the agent and the browser both talk to it.
```bash
-bash docker/init-env.sh
-docker compose -f docker/compose.yaml up -d --build
-docker compose -f docker/compose.yaml logs -f # wait for the server to start
-docker compose -f docker/compose.yaml exec dw python /opt/dw/entrypoint.py login-url # the login link
+dw-serve # web UI on http://127.0.0.1:8765
```
-Setup, data sharing and updates are in [docs/DOCKER.md](docs/DOCKER.md).
-
-**2. Start the engine.** Leave it running; everything else talks to it.
-
-```bash
-dw-serve
-# diffusers-workflow server on http://127.0.0.1:8765
-```
-
-That address is the web UI. Open it and run `templates/text-to-image` — a
-small, ungated model, so the first generation needs no Hugging Face login and
-downloads only a few GB.
-
-**3. Connect Claude Code.** Register the MCP server with the absolute path to
-`dw-mcp` in the venv you just made (the relative path is the one setup detail
-that reliably goes wrong):
+**3. Connect your agent.** Register the MCP server with the absolute path to `dw-mcp`
+in the venv you just made (a relative path is the setup detail that most often goes
+wrong). Then add the [dw plugin](plugins/dw/README.md):
```bash
claude mcp add dw -- "$(pwd)/venv/bin/dw-mcp"
```
-Then, optionally, the [dw plugin](plugins/dw/README.md) — one skill per model
-family that knows which workflow fits a request and the rules that bite:
-
```
/plugin marketplace add dkackman/diffusers-workflow
/plugin install dw@diffusers-workflow
```
-Most of the shipped workflows (Flux, LTX-2, MiniMax...) use **gated** models.
-Request access on the model's Hugging Face page, then `hf auth login`
-once; without it the run fails partway through with a 401/403 from the Hub.
-
-> **GPU on another machine?** Start the engine there with `--mcp` and connect
-> over HTTP — nothing to install on the laptop:
->
-> ```bash
-> # on the GPU box
-> dw-serve --host 0.0.0.0 --token "$DW_API_TOKEN" --mcp --workspace ~/studio
->
-> # on your laptop
-> claude mcp add --transport http dw http://gpu-box:8765/mcp \
-> --header "Authorization: Bearer $DW_API_TOKEN"
-> ```
->
-> The server's own **Server** page composes that line for the address you
-> pick. End to end: [Remote GPU server](docs/REMOTE.md).
-
-## Drive it from an agent
-
-Then just ask. The agent has 65 tools covering the whole surface — the
-workflow catalog, the real diffusers pipeline signatures, the job queue, the
-gallery, the model cache:
+Most of the shipped models (Flux, LTX, MiniMax, ...) are **gated**. Request access on
+the model's Hugging Face page and run `hf auth login` once. Without it a run fails
+partway through with a 401/403 from the Hub. `templates/text-to-image` uses a small,
+ungated model, so it makes a good first run.
-
+
+GPU on another machine, or in Docker?
+
+## Ask the agent
+
+Describe what you want. The agent has 65 tools covering the workflow catalog, the
+real diffusers pipeline signatures, the job queue, the gallery and the model cache,
+and it stays with a long render until it finishes:
+
+
+
+A session sounds like this:
+
+- **"Make a 30-second musical trailer: four men in boater hats, a giant eye, a
+ score under it."** The agent picks a shape from the catalog (`list_workflows`),
+ breaks the brief into shots, writes each prompt to the prompt library, and builds
+ the workflow that renders the shots and cuts them to the score.
+- **"What will that cost?"** `validate_workflow` checks the JSON against the schema
+ *and* the real pipeline signatures, with no model loads, and quotes an estimate in
+ GPU minutes from this box's own history. Anything that spends GPU time or disk
+ refuses to run until the agent passes `acknowledged_cost`, so it has to tell you
+ what it is about to spend first.
+- **"How's it going?"** `wait_for_job` waits out a render without polling. Then
+ `get_output_frames`, `get_output_image` and `get_output_audio` bring the result
+ back into the conversation, so the agent judges what it made against what you
+ asked for.
+- **"Shot three is the one. Keep it and start the next scene from it."**
+ `keep_output` turns the file into a named asset, and the next workflow references
+ it as `asset:hero-frame.png`.
+
+The [dw plugin](plugins/dw/README.md) adds what a general-purpose agent can't know
+on its own: one skill per model family (MiniMax H3 and Music 3, LTX-2.5, Kandinsky 6,
+FLUX 3 Klein) that picks the template for a request and states the rules that bite.
+Cross-cutting skills cover script-to-video, series with a consistent cast,
+storyboards and animatics, and a shared camera-and-look vocabulary.
+
+
+
+One server holds several **workspaces**, each with its own workflows, assets and
+outputs. Two agents, or an agent and you in the browser, can share one GPU without
+saving over each other's work.
+
+The tool reference, setup for other MCP hosts and the troubleshooting table are in
+[MCP Server](docs/MCP.md).
+
+## Then tweak it in the browser
+
+The web UI shows everything the agent made and lets you take over at any point. It
+runs on the same persistent GPU worker, so models stay loaded between runs.
+
+**A workflow the agent wrote is a form you can edit.** The workflow's description,
+recent outputs and variables are on one page. Change a prompt, a frame count or a
+LoRA scale and press **Run**.
+
+
+
+**Every job is a recipe.** A job keeps the exact workflow it ran and draws it as a
+graph of how data flows between steps. Click a step to see what it produced. The
+run's warnings sit above the graph, so a problem the agent missed, such as an
+audible level jump at a cut, is in front of you with the argument that fixes it.
+**Run again** repeats the job, and **New seed** gives you another take. Progress
+streams live for each denoising step, and a cancel stops the job cleanly at the
+next step.
+
+
+
+**The gallery** groups outputs by the workflow that made them, with final and
+intermediate files kept apart. Any image reopens as the workflow that produced it.
+*Keep as asset* adds a file to the asset library for later workflows to use.
+
+
+
+
+
+**The editor works on the agent's JSON.** You can view the workflow as a form, as
+JSON, side by side, or as a flow graph. The forms and autocomplete come from the
+real diffusers pipeline signatures, so every argument a pipeline takes is there with
+its documentation. **Validate** catches schema errors and argument typos before any
+model loads.
+
+
+
+There is also a **prompt library**, with an *Enhance with AI* panel that expands an
+idea into a full prompt using a local language model. The **model manager** shows
+the Hugging Face cache (size, last use, free disk) and downloads or deletes models
+with live progress.
+
+
+
+The pages are described in [Server & Web UI](docs/SERVER.md). The UI and the agent
+both use the server's HTTP API, which you can browse at `/docs` on a running server
+(see [The HTTP API](docs/SERVER.md#the-http-api)).
+
+## Underneath
+
+Every front end reads and writes the same thing: a JSON workflow of named steps,
+each a diffusers pipeline or a utility task. Its arguments are references to
+variables, earlier steps' results, stored prompts and assets, not hard-coded
+values. That is how text-to-image chains into image-to-video, and how a generated
+image reopens as the recipe that made it. You rarely need to write one by hand. When
+you do, the [Workflow Guide](docs/WORKFLOW_GUIDE.md) is the reference, the
+[JSON Schema](dw/workflow_schema.json) is the exact structure, and
+[workflows/](workflows/) has runnable examples for each model family. There is also
+a command-line client, `python -m dw.run workflow.json prompt="a cat"`, which queues
+a job on a running server (see [Server & Web UI](docs/SERVER.md#the-command-line)).
+
+The engine also covers:
+
+- **Quantization:** BitsAndBytes, TorchAO, GGUF, SDNQ and optimum-quanto, set per
+ component.
+- **Acceleration:** FirstBlockCache, FasterCache, MagCache, TaylorSeerCache,
+ `torch.compile` and attention backends.
+- **Adapters and prompts:** LoRA, IP-Adapter and A1111-style prompt weighting.
+- **Long video and caching:** long-video chaining with audio-driven length, and
+ step-output caching, so re-running a fixed-seed workflow finishes instantly.
+- **Utility tasks:** upscaling, face restoration, segmentation, captioning, frame
+ interpolation, audio slicing, cut-and-score assembly and more.
+
+A workflow can reach any diffusers pipeline or quantization backend by dynamic
+import, so loading one can execute arbitrary Python. Treat a workflow file from
+someone else the way you'd treat a `.py` script. See
[Trust model](docs/SECURITY.md#trust-model).
-Under the hood the engine also handles: quantization (BitsAndBytes, TorchAO,
-GGUF, SDNQ, optimum-quanto); inference acceleration
-(FirstBlockCache, FasterCache, MagCache, TaylorSeerCache); LoRA and IP-Adapter;
-A1111-style prompt weighting; long-video chaining with audio-driven length;
-step-output caching, so re-running a fixed-seed workflow finishes instantly;
-and utility tasks for upscaling, face restoration, segmentation, captioning,
-frame interpolation and more.
+**Field reports.** At the end of a session, ask the agent what got in its way, and
+have it file each bug or gap as an issue on `dkackman/diffusers-workflow` with the
+`field-report` label. The label marks a report as coming from real use. A report
+from anyone but the maintainer is reviewed before the [agent loop](docs/AGENT_LOOP.md)
+acts on it.
## Documentation
-### Guides
-
-- [MCP Server](docs/MCP.md) — The agent tool surface (Claude Code, Claude Desktop)
-- [Server & Web UI](docs/SERVER.md) — The web UI, jobs API, and introspection service
-- [Remote GPU server](docs/REMOTE.md) — Using the server, UI and MCP from another machine
-- [Docker](docs/DOCKER.md) — The server in a container on a Linux NVIDIA box, sharing data with a native install
-- [Workspaces](docs/WORKSPACES.md) — Where your content lives, run directories, and several workspaces on one server
-- [Workflow Guide](docs/WORKFLOW_GUIDE.md) — JSON structure, variables, steps, data flow
-- [Quantization](docs/QUANTIZATION.md) — BitsAndBytes, TorchAO, GGUF, SDNQ
-- [Inference Acceleration](docs/ACCELERATION.md) — torch.compile, FirstBlockCache, MagCache, TaylorSeer
-- [Fast on 24GB](docs/RECIPES_24GB.md) — Recommended speed/memory configurations per model family
-- [LoRA](docs/LORAS.md) — Loading and stacking LoRA adapters
-- [IP-Adapter](docs/IP_ADAPTER.md) — Image-prompt conditioning
-- [Prompt Weighting](docs/PROMPT_WEIGHTING.md) — A1111-style syntax
-- [Prompt References](docs/WORKFLOW_GUIDE.md#prompt-references) — The stored prompt library and `prompt:` references
-- [Tasks](docs/TASKS.md) — Image processing, ControlNet preprocessors, utilities
-
-### Reference
-
-- [Worker Guide](docs/WORKER_GUIDE.md) — GPU persistence and troubleshooting
-- [Dependencies](docs/DEPENDENCIES.md) — Installation details
-- [Security](docs/SECURITY.md) — Security model
-- [Testing](docs/TESTING.md) — Running the test suite
-- [Releasing](docs/RELEASING.md) — Cutting a release from a version tag
-- [Agent Loop](docs/AGENT_LOOP.md) — What the automated implementer/tester ticket labels mean, for anyone joining an Issue
-- [Attributions](docs/ATTRIBUTIONS.md) — Code included from other projects, and the projects whose ideas dw reimplemented
+**Using it**
+
+- [MCP Server](docs/MCP.md) - the agent's tool surface (Claude Code, Claude Desktop, other MCP hosts)
+- [dw plugin](plugins/dw/README.md) - the model-family and composition skills
+- [Server & Web UI](docs/SERVER.md) - the web UI, the command line, and the [HTTP API](docs/SERVER.md#the-http-api) with its OpenAPI spec
+- [Workspaces](docs/WORKSPACES.md) - where your content lives, run directories, several workspaces on one server
+- [Remote GPU server](docs/REMOTE.md) - the server, UI and MCP from another machine
+- [Docker](docs/DOCKER.md) - the server in a container on a Linux NVIDIA box
+
+**Workflows in depth**
+
+- [Workflow Guide](docs/WORKFLOW_GUIDE.md) - JSON structure, variables, steps, data flow, references
+- [Tasks](docs/TASKS.md) - image, video and audio utility tasks, ControlNet preprocessors
+- [Quantization](docs/QUANTIZATION.md) - BitsAndBytes, TorchAO, GGUF, SDNQ
+- [Inference Acceleration](docs/ACCELERATION.md) - caches, `torch.compile`, attention backends, MPS
+- [Fast on 24GB](docs/RECIPES_24GB.md) - tested speed/memory settings per model family
+- [LoRA](docs/LORAS.md), [IP-Adapter](docs/IP_ADAPTER.md), [Prompt Weighting](docs/PROMPT_WEIGHTING.md)
+
+**Reference**
+
+- [Architecture](docs/ARCHITECTURE.md) - each concept, the module that owns it, and the check that enforces it
+- [Worker Guide](docs/WORKER_GUIDE.md) - the persistent GPU worker and troubleshooting
+- [Security](docs/SECURITY.md) - validators and the trust gate ([quick reference](docs/SECURITY_QUICKREF.md))
+- [Dependencies](docs/DEPENDENCIES.md), [Testing](docs/TESTING.md), [Releasing](docs/RELEASING.md)
+- [Agent Loop](docs/AGENT_LOOP.md) - what the automated implementer/tester labels mean
+- [Attributions](docs/ATTRIBUTIONS.md) - code from other projects, and the projects whose ideas dw reimplemented
diff --git a/docs/ACCELERATION.md b/docs/ACCELERATION.md
index e1db74441..437ffd864 100644
--- a/docs/ACCELERATION.md
+++ b/docs/ACCELERATION.md
@@ -1,49 +1,28 @@
# Inference Acceleration
-Speed up generation by caching intermediate computations and skipping redundant transformer steps. Diffusers' built-in caching (`first_block`, `mag` and the others below) provides it. Beyond caching, `torch.compile`, attention backend selection, layerwise casting, and device-level settings (TF32, cuDNN) also affect throughput - see below. Memory offloading trades speed for VRAM and is covered in depth in [WORKFLOW_GUIDE.md](WORKFLOW_GUIDE.md#memory-offloading).
-
-For ready-made configurations that combine these levers per model family, see [RECIPES_24GB.md](RECIPES_24GB.md).
+The levers that make a generation faster: diffusers' step caches, attention backends, `torch.compile`, layerwise casting, device settings (TF32, cuDNN) and offloading. Offloading trades speed for VRAM; its syntax is in [WORKFLOW_GUIDE.md](WORKFLOW_GUIDE.md#memory-offloading). Tested combinations per model family are in [RECIPES_24GB.md](RECIPES_24GB.md); quantization is in [QUANTIZATION.md](QUANTIZATION.md).
## Diffusers Built-in Cache
-Applied at pipeline load time via the `cache` configuration. Hooks auto-reset between runs.
+A `cache` block in the pipeline `configuration` attaches one of diffusers' cache hooks to the pipeline's `transformer` at load time. The hooks reset between runs. `cache` may also be a whole-object `variable:` reference; one that resolves to `null` means no cache ([WORKFLOW_GUIDE.md](WORKFLOW_GUIDE.md#cache-acceleration)). Every MiniMax-H3 template takes a `cache` argument, `null` by default and wired to the H3 step only.
-`validate_workflow` warns (it never refuses) about two cache settings that cost a render to find out:
+`validate_workflow` warns, and never refuses, about cache settings that cannot pay off (`dw/acceleration_warnings.py`):
-- **`cache_short_schedule`: a threshold cache on a short schedule.** `first_block` and `mag` decide each step's skip on a residual threshold, and the consecutive steps of a short schedule differ too much for it: under 20 `num_inference_steps` such a cache will likely skip nothing and still perturbs the result. On MiniMax-H3 a `first_block` cache at `0.05` skipped nothing at 12 steps and took 22% off at 30 ([FirstBlockCache](#firstblockcache)). Drop the cache or run 20 steps or more. A turbo schedule (9 steps or fewer) never benefits. `taylorseer` and `faster` skip on a fixed interval and `text_kv` skips no step, so the rule does not apply to them.
-- **`taylorseer_state_unestimated`: TaylorSeer's state is not in the VRAM estimate.** Said whenever `taylorseer` is set - see [TaylorSeerCache](#taylorseercache).
+- **`cache_short_schedule`**: a `first_block` or `mag` cache under 20 `num_inference_steps`. Both decide each step's skip on a residual threshold, and the consecutive steps of a short schedule differ too much for it, so the cache likely skips nothing and still perturbs the result. On MiniMax-H3 a `first_block` cache at `0.05` skipped nothing at 12 steps or on a turbo schedule, and took 22% off at 30. Drop the cache or run 20 steps or more. `taylorseer` and `faster` skip on a fixed interval and `text_kv` skips no step, so the rule does not apply to them.
+- **`cache_unsupported`**: the pipeline class has no `transformer` (a UNet pipeline such as SD 1.5), or its transformer class has no `enable_cache()` (Z-Image's). The step runs uncached and the job warns that the cache was not applied.
+- **`taylorseer_state_unestimated`**: said whenever `taylorseer` is set - see [TaylorSeerCache](#taylorseercache).
+- **`compile_fullgraph_with_cache`**: see [torch.compile](#torchcompile).
### Models diffusers has not registered
-`first_block`, `mag` and `layer_skip` look a model's transformer block class up in
-diffusers' own registry and raise when it is absent, which is how a model that supports
-`enable_cache()` ends up with no usable cache. [cache_blocks.json](../dw/cache_blocks.json)
-fills those gaps in, registering the missing block metadata on demand; entries become
-redundant, not wrong, once diffusers registers the same class upstream. MiniMax-H3,
-LTX-2 and Qwen-Image-2.1 are listed there today.
-
-Qwen-Image-2.1's pipeline keeps a key/value cache of its own (`use_kv_cache`, on by
-default) that drops the text tokens from the sequence after the first step. `first_block`
-and `mag` compare one step's residual against the previous one, so the sequence length
-changing under them fails the second step; pass `"use_kv_cache": false` in the step's
-`arguments` to cache it.
-
-A transformer that does not support `enable_cache()` at all - Z-Image's, for one - takes
-no cache of any type. The step still runs, uncached, and the job carries a warning that the
-cache was not applied.
-
-LTX-2's block needs one thing more than the metadata diffusers defines. It returns two
-streams - video and audio - and diffusers reads the second one back out of a forward
-argument named literally `encoder_hidden_states`, the only two-stream shape it registers
-upstream (text beside image). LTX-2's blocks take an `encoder_hidden_states` of their
-own, the text conditioning, so the fixed name reads the wrong tensor and feeds the text
-embeddings back as the audio stream on every skipped block. The entry names the argument
-its second stream actually comes from (`encoder_hidden_states_argument_name`), which is
-what makes caching correct there rather than merely quiet.
+`first_block` and `mag` look the transformer's block class up in diffusers' block registry and raise when it is absent, so a model that supports `enable_cache()` can still have no usable cache. [cache_blocks.json](../dw/cache_blocks.json) registers the missing block metadata on demand (`dw/cache_blocks.py`); an entry becomes redundant, not wrong, once diffusers registers the class itself. It lists MiniMax-H3, LTX-2 and Qwen-Image-2.1.
+
+- **LTX-2**: its blocks return video and audio streams, and diffusers reads the second stream back from a forward argument named `encoder_hidden_states`. LTX-2's own `encoder_hidden_states` is the text conditioning, so a skipped block would feed the text embeddings back as audio. The entry's `encoder_hidden_states_argument_name` names the argument the second stream actually comes from.
+- **Qwen-Image-2.1**: the pipeline's own key/value cache (`use_kv_cache`, on by default) drops the text tokens from the sequence after the first step, and the changed sequence length fails `first_block` and `mag` on the second step. Pass `"use_kv_cache": false` in the step's `arguments` to cache it.
### FirstBlockCache
-Simplest and broadest support. Compares first-block residuals to decide whether to skip remaining blocks.
+Compares the first block's residual between steps to decide whether to skip the remaining blocks. The broadest support.
```json
"configuration": {
@@ -55,13 +34,13 @@ Simplest and broadest support. Compares first-block residuals to decide whether
}
```
-Higher threshold = more speedup, more quality loss. Start with `0.05` and increase to taste. On MiniMax-H3 (RTX 3090, 1344x768, 4 references) `0.05` took 22% off a 30-step base-model job and nothing off a 12-step or turbo one, which it never skips; `torch.compile` there cost 289 s on the first pass for ~1% per pass, `text_kv` did nothing on Ref2VA, and `taylorseer` ran out of memory. Every MiniMax-H3 template takes a `cache` argument, `null` (off) by default and wired to the H3 step only: pass `{"type": "first_block", "threshold": 0.05}` rather than editing the JSON, and only at ~20 or more base steps, never with the turbo LoRAs.
+`threshold` defaults to `0.05`. Higher skips more and costs more quality. On MiniMax-H3 (RTX 3090, 1344x768, 4 references) `0.05` took 22% off a 30-step base-model job and nothing off a 12-step or turbo one. On H3 `torch.compile` cost 289 s on the first pass for about 1% per pass, `text_kv` did nothing on Ref2VA, and `taylorseer` ran out of memory. On an H3 template pass `cache={"type": "first_block", "threshold": 0.05}`, at 20 or more base steps and never with a turbo LoRA.
**Example:** [step-caching.json](../workflows/templates/step-caching.json)
### MagCache
-Magnitude-based caching with error accumulation. Requires `num_inference_steps` to match the pipeline arguments, and `mag_ratios` — the per-step magnitude ratios, which are checkpoint-dependent:
+Magnitude-based skipping with error accumulation:
```json
"cache": {
@@ -76,29 +55,33 @@ Magnitude-based caching with error accumulation. Requires `num_inference_steps`
| Property | Default | Description |
| -------- | ------- | ----------- |
-| `mag_ratios` | required | Preset name or explicit per-step ratio array — see below |
+| `mag_ratios` | required unless `calibrate` | Preset name or per-step ratio array - see below |
| `threshold` | 0.06 | Accumulated error threshold for skipping |
-| `num_inference_steps` | required | The transformer forwards the call runs: the pipeline's `num_inference_steps`, except where the scheduler runs fewer (MiniMax-H3 runs one fewer) |
+| `num_inference_steps` | 28 (diffusers) | The transformer forwards the call runs: the pipeline's `num_inference_steps`, except where the scheduler runs fewer (MiniMax-H3 runs one fewer). Set it to match the call |
| `max_skip_steps` | 3 | Max consecutive steps to skip |
| `retention_ratio` | 0.2 | Fraction of initial steps where skipping is disabled |
-| `calibrate` | false | Measure ratios for a new model instead of skipping — see below |
+| `calibrate` | false | Measure ratios for a new model instead of skipping - see below |
#### Supplying `mag_ratios`
-MagCache needs to know how each denoising step's output magnitude typically behaves for *your* checkpoint, so unlike the other cache types it cannot run on defaults alone. Give it either:
+The ratios describe how each step's output magnitude behaves for one checkpoint, so MagCache cannot run on defaults alone. Give either:
-- **A preset name** — `"mag_ratios": "flux"` resolves to the ratios diffusers ships for Flux. Any preset a later diffusers release adds is usable by name without a change here.
-- **An explicit array** — `"mag_ratios": [1.0, 0.98, 0.96, ...]`. The array is interpolated automatically when its length differs from `num_inference_steps`, so ratios measured at one step count can be reused at another.
+- **A preset name**: `"mag_ratios": "flux"` resolves to diffusers' `FLUX_MAG_RATIOS`. A preset a later diffusers adds (`_MAG_RATIOS`) works by name with no change here; an unknown name fails with the list of presets.
+- **An array**: `"mag_ratios": [1.0, 0.98, 0.96, ...]`. An array whose length differs from `num_inference_steps` is interpolated.
-For a model with no preset, run once with `"calibrate": true`. Calibration skips nothing. At the end of the run the measured ratios appear as a job warning and a `log` event (`get_job`'s `warnings`, `get_job_events`, `export_job`): `MagCache calibration (N steps): [ ... ] — paste into mag_ratios`. While calibrating, the run's own `num_inference_steps` is the step count, so the cache's can be left at its default (a mismatch is reported as a warning). The cache attaches to a pipeline's `transformer`, and only one that supports `enable_cache()`: a UNet pipeline such as SD 1.5 has no transformer, and Z-Image's transformer has no cache support, so both get a warning that the cache was not applied and no calibration. Paste that array into `mag_ratios` and drop the `calibrate` flag for subsequent runs. `N` is the forwards the run actually made, which is not always the call's `num_inference_steps`: MiniMax-H3's scheduler turns N sigmas into N - 1 evaluations, so a 30-step H3 call reports 29 ratios. Set the cache's `num_inference_steps` to that `N` when you paste them, so each ratio lines up with its forward. A pipeline that runs classifier-free guidance as two transformer passes in their own `cond` and `uncond` cache contexts (Qwen-Image-2.1 with `true_cfg_scale` above 1 and a `negative_prompt`, or the other Qwen-Image pipelines) keeps a separate state for each, so it emits **two** lists, each labeled `[cond]` or `[uncond]`; use the `cond` one, since `mag_ratios` takes one array. A guidance-distilled pipeline such as Flux, or one that batches both passes into one forward emits one unlabeled list.
+For a model with no preset, run once with `"calibrate": true`:
```json
"cache": { "type": "mag", "calibrate": true, "num_inference_steps": 28 }
```
+Calibration skips nothing. At the end of the run the ratios appear as a job warning and a `log` event (`get_job`'s `warnings`, `get_job_events`, `export_job`): `MagCache calibration (N steps): [ ... ] — paste into mag_ratios`. While calibrating, the run's own `num_inference_steps` is used, and a mismatch with the cache's is reported as a warning. Paste the array into `mag_ratios`, drop `calibrate`, and set the cache's `num_inference_steps` to the reported `N`, which counts the forwards actually made: MiniMax-H3's scheduler turns N sigmas into N - 1 evaluations, so a 30-step H3 call reports 29 ratios.
+
+A pipeline that runs classifier-free guidance as two passes in separate `cond` and `uncond` cache contexts (Qwen-Image-2.1 with `true_cfg_scale` above 1 and a `negative_prompt`, or the other Qwen-Image pipelines) emits two lists, labeled `[cond]` and `[uncond]`; use the `cond` one. A guidance-distilled pipeline such as Flux, or one that batches both passes into one forward, emits one unlabeled list. A pipeline the cache cannot attach to (see `cache_unsupported` above) reports nothing.
+
### TaylorSeerCache
-Taylor series approximation of cached outputs:
+Approximates skipped outputs with a Taylor series:
```json
"cache": {
@@ -111,13 +94,13 @@ Taylor series approximation of cached outputs:
| Property | Default | Description |
| -------- | ------- | ----------- |
| `cache_interval` | 5 | Full computation every N steps |
-| `max_order` | 1 | Taylor series order (higher = better approximation, and more state kept on the GPU) |
+| `max_order` | 1 | Taylor series order (higher = better approximation, more state on the GPU) |
-TaylorSeer keeps per-block state on the GPU that no `vram_estimate` charges, so a job that fits without the cache can run out of memory with it. On MiniMax-H3 at 1344x768, 30 steps and `max_order` 1, a run that `validate_workflow` had passed ran out of memory 2 min in on a 24 GB card. Validate says so as `taylorseer_state_unestimated` whenever `taylorseer` is set, with the job's estimate and the card's capacity when both are known; the state itself is on top of that figure, and is not sized.
+TaylorSeer keeps per-block state on the GPU that no `vram_estimate` charges, so a job that fits without it can run out of memory with it. On MiniMax-H3 at 1344x768, 30 steps and `max_order` 1, a job `validate_workflow` had passed ran out of memory 2 min in on a 24 GB card. Validate warns `taylorseer_state_unestimated`, with the job's estimate and the card's capacity when both are known; the state comes on top of that figure and is not sized.
### FasterCache
-Experimental, video-oriented. Uses FFT frequency decomposition:
+Experimental and video-oriented (CogVideoX, for example); uses FFT frequency decomposition. dw wires diffusers' `current_timestep_callback` to the pipeline, so no other parameter is needed for basic use:
```json
"cache": {
@@ -125,16 +108,13 @@ Experimental, video-oriented. Uses FFT frequency decomposition:
}
```
-Best for video models like CogVideoX. No additional parameters needed for basic use.
-
| Property | Default | Description |
| -------- | ------- | ----------- |
-| `is_guidance_distilled` | false | Set `true` for a guidance-distilled model (FLUX.1-dev, FLUX.1-Krea-dev), where `guidance_scale` is an embedding and the transformer sees a batch of 1. FasterCache otherwise assumes a cond/uncond batch of 2 and fails at the first step with `tuple index out of range` (#895). Validate cannot tell a distilled model from a CFG one, so it does not check this |
+| `is_guidance_distilled` | inferred | `true` for a guidance-distilled model (FLUX.1-dev, FLUX.1-Krea-dev), where `guidance_scale` is an embedding and the transformer sees a batch of 1. Unset, dw sets it `true` when the transformer's config has `guidance_embeds: true`, else diffusers' `false`. Wrong, FasterCache assumes a cond/uncond batch of 2 and fails at the first step with `tuple index out of range` (#895). Validate does not check it |
### TextKVCache
-Caches the transformer's key/value projections of the (unchanging) text
-embeddings across denoising steps, recomputing only what the latents need:
+Caches the transformer's key/value projections of the unchanging text embeddings across steps. No parameters.
```json
"cache": {
@@ -142,15 +122,15 @@ embeddings across denoising steps, recomputing only what the latents need:
}
```
-No parameters.
-
### Reading the skip counts
-A step that set a `cache` reports `timing.cache` on its manifest entry (fields in [WORKSPACES.md](WORKSPACES.md#step-timing)). `hit_rate` is the fraction of denoiser forwards whose remaining blocks were skipped. It counts forwards, not steps: classifier-free guidance runs two per step, so a 30-step run is 60 forwards. Only `first_block` and `mag` are counted; `taylorseer`, `faster` and `text_kv` report null counts. A cache with zero skips can still change the output, so a 0-skip run is not guaranteed bit-identical to an uncached one. With `first_block` the head block always runs, and enabling any cache wraps the blocks in hooks that recompute the residual (the tail's output minus the head's, in the transformer's dtype), which changes the order and precision of operations; the MiniMax-H3 matrix run P2 saw a changed output with nothing skipped. Judge quality against an uncached run, not against `skipped == 0`. A threshold sweep (#875) reads `hit_rate` against quality.
+A step with a `cache` reports `timing.cache` on its manifest entry (fields in [WORKSPACES.md](WORKSPACES.md#step-timing)). `hit_rate` is the fraction of denoiser forwards whose remaining blocks were skipped. It counts forwards, not steps: classifier-free guidance runs two per step, so a 30-step run is 60 forwards. Only `first_block` and `mag` are counted; `taylorseer`, `faster` and `text_kv` report null counts.
+
+Zero skips is not an uncached run. With `first_block` the head block always runs, and any cache wraps the blocks in hooks that recompute the residual in the transformer's dtype, which changes the order and precision of operations; the MiniMax-H3 matrix run P2 changed output with nothing skipped. Judge quality against an uncached run (a `null` cache), not against `skipped == 0`. A threshold sweep (#875) reads `hit_rate` against quality.
## Attention Backends
-Select the attention implementation diffusers uses for the duration of each pipeline call, via a context manager wrapped around `pipeline(...)`:
+`attention_backend` in the pipeline `configuration` selects diffusers' attention implementation for the duration of each pipeline call, via a context manager around `pipeline(...)`:
```json
"configuration": {
@@ -159,33 +139,48 @@ Select the attention implementation diffusers uses for the duration of each pipe
}
```
-Common values: `"flash"`, `"flash_hub"`, `"sage"`, `"sage_hub"`, `"native"`, `"flex"`. The full set is diffusers' `AttentionBackendName` enum - availability depends on what's installed (`flash-attn`, `sageattention`, etc.) and the platform. `_hub`-suffixed backends are fetched from the Hugging Face Hub kernel registry on first use, which needs the `kernels` package installed (`pip install kernels`) - it is not a dw dependency. The only backend a bundled workflow sets is `sparse`, the default on every MiniMax-H3 template, and it needs no `kernels` install (comfy-kitchen is a core dependency), so each bundled workflow runs on a plain install. diffusers also refuses a Hub kernel whose publisher it does not list as trusted - `sage_hub` among them, since SageAttention 2 comes from `SageAttention/sage-attention` - until `DIFFUSERS_TRUST_REMOTE_KERNELS=true`, which allows remote code from any such publisher. On an RTX 3090 `sage_hub` measured about 30% faster per MiniMax-H3 step (13.8 s against 19.2 s at 960x544x124, 54.7 s against 81.2 s at 345 frames); the local `sage` backend needs `sageattention>=2.1.1`, which PyPI does not carry.
+A component can pin its backend instead, persistently, via `set_attention_backend`:
+
+```json
+"configuration": {
+ "components": {
+ "transformer": { "attention_backend": "flash_hub" }
+ }
+}
+```
+
+Pin it on a compiled component: the per-call context manager switches implementations under the compiled graph and forces a recompile on every run. Set at both levels, the component's own wins.
+
+Values are diffusers' `AttentionBackendName` enum, plus dw's [`sparse`](#sparse-minimax-h3). Common ones: `native`, `flex`, `flash`, `flash_hub`, `sage`, `sage_hub`. Availability depends on the installed packages and the platform:
+
+- `_hub` backends are fetched from the Hugging Face Hub kernel registry on first use through the `kernels` package, which dw installs.
+- diffusers refuses a Hub kernel from a publisher it does not list as trusted - `sage_hub` among them (SageAttention 2 from `SageAttention/sage-attention`) - until `DIFFUSERS_TRUST_REMOTE_KERNELS=true` is set in the server's environment (`.env`). That allows remote code from any such publisher.
+- The local `sage` backend needs `sageattention>=2.1.1`, which PyPI does not carry.
+- The flash, flash-3, sage and xformers families need CUDA, `_hub` variants included. On a server whose device is not CUDA, `validate_workflow` warns `attention_backend_unavailable`, naming the backend and the device: the job would fail at load. A `native`-family backend never warns, and a backend diffusers adds later draws no warning until dw's list (`CUDA_ONLY_ATTENTION_BACKENDS`) is updated.
+
+On an RTX 3090 `sage_hub` measured about 30% faster per MiniMax-H3 step than `native` (13.8 s against 19.2 s at 960x544x124, 54.7 s against 81.2 s at 345 frames). LTX-2.5 cannot use sage: its transformer passes an attention mask, which sage refuses ("`attn_mask` is not supported for sage attention", measured 2026-10-10), so the LTX templates expose no backend.
-Every MiniMax-H3 template in [templates/minimax/](../workflows/templates/minimax/)
-exposes this as a variable: `attention_backend`, pinned on the H3 step's transformer
-(`components.transformer`, or `components.transformer_ref` for a ref2va workflow) and
-`"sparse"` by default (with `sparse_tau`, below). That covers the shots, chains and episodes - `dialogue-short`,
-`shots-batch`, `music-video` and `storyboard` among them - and only their H3 steps: a
-Z-Image still or Music3 song beside an H3 shot keeps the process default. Pass
-`attention_backend=sage_hub` (or `native`) to override the default; `sage_hub` needs a box with the `kernels` package and
-`DIFFUSERS_TRUST_REMOTE_KERNELS=true` in its `.env`; on MPS the default falls back to dense, but a CUDA-only kernel such as `sage_hub` fails there at load (validation warns, below). LTX-2.5 cannot use it: its transformer
-passes an attention mask, and sage attention refuses one ("`attn_mask` is not supported
-for sage attention", measured 2026-10-10), so the LTX templates expose no backend.
+Either form stays on what it names. diffusers' `set_attention_backend` also makes its backend the process-wide default, which the persistent worker would keep for every later model; dw restores the default after pinning, so an H3 `sage_hub` pin cannot reach a Music3 or LTX job that follows. The server-wide default is `DIFFUSERS_ATTN_BACKEND` in `.env`; leave it `native`, since it reaches every model and sage cannot serve all of them.
### `sparse` (MiniMax-H3)
-`attention_backend: "sparse"` runs H3's attention through comfy-kitchen's block-sparse kernel: each query tile attends only to the key tiles that matter, so the cost of the long video-token sequence falls. It applies to the H3 transformer only, at pipeline or component level, and is the default on every H3 template (#931). Set at both levels, the component's own `attention_backend` wins, as it does for every backend: a pipeline-level `sparse` skips a component pinned to `sage_hub` (or anything else) and warns `attention_backend 'sparse' at pipeline level skips '', which pins ''`. When it does install, the run logs `Using attention backend: sparse (tau ) on `, or `(topk )` with top-k routing. `sparse_tau` (number, minimum 0, default 0.5; anything else, such as a `variable:` that resolves to a string or a bool, fails the step rather than running dense) sets how much of the attention mass a tile may drop; 0 keeps the most. Observed at Ref2VA 1344x768, seed 42: tau 1.0 ran 42.1 s/forward against 44.8 s at 0.5, and by Don's eye 1.0 gave smoother skin while 0.5 matched dense (the table below). That is one shape and one seed, not a rule that higher is always faster or smoother.
+`attention_backend: "sparse"` runs H3's attention through comfy-kitchen's block-sparse kernel (`dw/sparse_attention.py`, #931): each query tile attends only to the key tiles that matter, so the cost of the long video-token sequence falls. Text, references and audio stay dense. comfy-kitchen is a core dependency, so it needs no `kernels` package or `DIFFUSERS_TRUST_REMOTE_KERNELS`. The only backend a bundled workflow sets is `sparse`.
-`sparse_topk` (a number strictly between 0 and 1; the H3 templates set 0.15, and a workflow that sets none routes by `sparse_tau`) switches the kernel to top-k routing: each query tile keeps that fraction of key tiles and drops the rest outright, with `sparse_tau` ignored. It is the selection lightx2v's Turbo-SLA LoRAs were trained against (85% sparsity, so `0.15`). Measured on `video-with-audio-768p`, seed 42 (#932): top-k 0.15 ran 21.1 s/forward against 23.6 s at tau 0.5, about 11% faster, and Don saw no difference on that clip, with either the Turbo-SLA LoRA or the default v1.2 turbo (so dw does not adopt the SLA LoRA). That clip has no faces, so it was checked on the H3 quality ramp's 4-reference sitcom shot at seeds 42, 7 and 1234 against tau 0.5. Don judged top-k equal or better: on the turbo L7 rung tau showed blue ghosting at seed 7 and slight smearing at all three, and on the 30-step base-model L5 rung the skin held. Top-k 0.15 is therefore the H3 templates' default; pass `sparse_topk=null` to route by `sparse_tau` instead.
+Every H3 template in [templates/minimax/](../workflows/templates/minimax/) takes it as variables, pinned on the H3 step's transformer (`components.transformer`, or `components.transformer_ref` for a ref2va workflow): `attention_backend` (default `"sparse"`), `sparse_tau` (0.5) and `sparse_topk` (0.15). That covers the shots, chains and episodes (`dialogue-short`, `shots-batch`, `music-video`, `storyboard` among them), and only their H3 steps: a Z-Image still or Music3 song beside an H3 shot keeps the process default. Pass `attention_backend=sage_hub` (or `native`) to override.
+
+- **Where it applies**: the H3 transformer, at pipeline or component level. On any other component, or a pipeline with no H3 transformer, it runs dense with a warning. A pipeline-level `sparse` skips a component pinned to another backend and warns `attention_backend 'sparse' at pipeline level skips '', which pins ''`. When it installs, the run logs `Using attention backend: sparse (tau ) on `, or `(topk )`.
+- **`sparse_tau`**: number >= 0, default 0.5. How much of the attention mass a tile may drop: higher is sparser and faster, 0 keeps the most. Anything else, such as a `variable:` that resolves to a string or a bool, fails the step rather than running dense.
+- **`sparse_topk`**: a number strictly between 0 and 1, unset by default outside the templates. Switches to top-k routing: each query tile keeps that fraction of key tiles and drops the rest, and `sparse_tau` is ignored. It is the selection lightx2v's Turbo-SLA LoRAs were trained against (85% sparsity, so `0.15`). Pass `sparse_topk=null` to an H3 template to route by `sparse_tau`.
+- Both are runtime settings: changing one reuses the loaded pipeline.
+
+**Top-k 0.15 against tau 0.5** (#932). On `video-with-audio-768p`, seed 42, top-k ran 21.1 s/forward against 23.6 s at tau 0.5, about 11% faster, with no visible difference under the Turbo-SLA LoRA or the default v1.2 turbo (so dw does not adopt the SLA LoRA). On the H3 quality ramp's 4-reference sitcom shot, seeds 42, 7 and 1234, Don judged top-k equal or better: on the turbo L7 rung tau showed blue ghosting at seed 7 and slight smearing at all three, and on the 30-step base-model L5 rung the skin held.
| rung (minutes per job, seeds 42 / 7 / 1234) | tau 0.5 | top-k 0.15 |
|---|---|---|
| L7: ref2v 4-step turbo, int8 | 6.5 / 6.5 / 6.5 | 6.1 / 6.2 / 5.8 |
| L5: base model, int8, 30 steps | 26.8 / 26.6 / 26.4 | 23.9 / 25.5 / 23.9 |
-**What the default buys depends on what it replaces.** Against `native` attention, which is what an H3 template ran before #931 unless `attention_backend` was set, sparse is 2.2-4.4x faster per step. Against `sage_hub`, the gain is much smaller: 6-16% depending on the shape (the second table). So sparse's main win is for a run that set no backend, and it needs no `kernels` package or `DIFFUSERS_TRUST_REMOTE_KERNELS`. A box already on `sage_hub` gains a little, at equal quality: on the H3 skill's quality-ramp rungs, Don judged faces, skin and draft-to-final staging equal under both backends across three seeds (#931 stage 3).
-
-Measured on an RTX 3090, seed 42, s per transformer forward (one per denoise step), lower is faster:
+**Sparse against `native` and `sage_hub`.** Against `native`, which an H3 template ran before #931 unless `attention_backend` was set, sparse is 2.2-4.4x faster per step. Against `sage_hub` the gain is 6-16% depending on the shape, at equal quality: on the H3 skill's quality-ramp rungs Don judged faces, skin and draft-to-final staging equal under both across three seeds (#931 stage 3). RTX 3090, seed 42, s per transformer forward (one per denoise step):
| shape | native | `sage_hub` | sparse | sparse vs native | sparse vs `sage_hub` |
|---|---|---|---|---|---|
@@ -194,9 +189,9 @@ Measured on an RTX 3090, seed 42, s per transformer forward (one per denoise ste
| Ref2VA 1344x768, 8-step ref2v turbo, tau 1.0 | 108.4 | - | 42.1 | 2.58x | - |
| Ref2VA 1344x768, 8-step ref2v turbo, tau 0.5 (the default) | 108.4 | 51.0 | 44.1-44.8 | 2.42x | 1.16x |
-Wall time, same order: 150 -> 95 s, 457 -> 158 s, 965 -> 430 s, and for the last row 965 s native, 489 s `sage_hub`, 442-450 s sparse.
+Wall time, same order: 150 -> 95 s, 457 -> 158 s, 965 -> 430 s, and for the last row 965 s native, 489 s `sage_hub`, 442-450 s sparse. Peak VRAM did not change. By eye, tau 0.5 matched dense and tau 1.0 gave smoother skin; that is one shape and one seed, not a rule that higher tau is always faster or smoother. The native and tau 1.0 figures come from the #931 prototype; the `sage_hub` figure and the 44.1 s come from the engine on lem after the deploy, with a newer diffusers (jobs 94c04976d21d and 593131ca713f), so the sage comparison is between those two jobs only.
-Against `sage_hub` on the H3 skill's quality-ramp rungs (ep 6 cold open, 4 references, 1344x768, 124 frames; minutes per job, seeds 42 / 7 / 1234, all on the same diffusers):
+On the quality-ramp rungs (ep 6 cold open, 4 references, 1344x768, 124 frames; minutes per job, seeds 42 / 7 / 1234, same diffusers):
| rung | `sage_hub` | sparse | sparse vs `sage_hub` |
|---|---|---|---|
@@ -205,27 +200,11 @@ Against `sage_hub` on the H3 skill's quality-ramp rungs (ep 6 cold open, 4 refer
| B3: base model, int8, 12 steps | 13.0 / 12.9 / 12.9 | 12.3 / 12.0 / 12.1 | 1.06-1.07x |
| L5: base model, int8, 30 steps | 29.7 / 29.7 / 29.7 | 26.8 / 26.6 / 26.4 | 1.11-1.12x |
-The `first_block` cache on `sage_hub` (23.1 min at L5's settings) is still faster than sparse alone there; sparse with that cache is not measured. The native and tau 1.0 figures come from the #931 prototype. The `sage_hub` figure and the 44.1 s come from the engine on lem after the deploy, with a newer diffusers (jobs 94c04976d21d and 593131ca713f), so the sage comparison is between those two jobs only. The T2VA rows have no `sage_hub` run. Peak VRAM did not change. Judged by eye, tau 0.5 matched dense; tau 1.0 was smoother in skin.
+The `first_block` cache on `sage_hub` (23.1 min at L5's settings) is still faster than sparse alone there; sparse with that cache is not measured.
-It needs CUDA with compute capability 8.0 or higher. Anywhere else (MPS, CPU, an older card, an unsupported dtype or head size) the run goes dense instead of failing, and says so twice: `validate_workflow` warns `sparse_attention_fallback` when the server's device is not CUDA, and the run reports `: sparse attention ran dense - ` once per pipeline call. A job of one call warns once; a chained, `shots-batch` or `dialogue-short` job calls the pipeline once per segment or shot, so it warns once for each, under that segment's or shot's label. A comfy-kitchen kernel that fails on its first invocation in a call also runs that call dense, with `sparse kernel failed: ` as the reason; one that fails after it has run in the call fails the run, since some blocks have already run sparse.
+**Where it cannot run.** It needs CUDA with compute capability 8.0 or higher. Anywhere else (MPS, CPU, an older card, an unsupported dtype or head size) the run goes dense instead of failing, and says so twice: `validate_workflow` warns `sparse_attention_fallback` when the server's device is not CUDA, and the run reports `: sparse attention ran dense - ` once per pipeline call, so a chained, `shots-batch` or `dialogue-short` job warns once per segment or shot, under its label. Whether a call runs sparse is decided once, before its first forward: a comfy-kitchen kernel that fails on its first invocation runs that call dense (`sparse kernel failed: `), and one that fails after it has run fails the run.
-Unlike sage, sparse **moves composition** relative to dense: the same subject, story and cuts, but different framing and set dressing. A draft and its final must therefore use the same backend; mixing `sparse` and `sage_hub` between tiers means the draft no longer predicts the final.
-
-A component can also pin its backend persistently instead, via `set_attention_backend`:
-
-```json
-"configuration": {
- "components": {
- "transformer": { "attention_backend": "flash_hub" }
- }
-}
-```
-
-Prefer the pinned form for a compiled component - the per-call context manager switches implementations under the compiled graph and forces a recompile on every run.
-
-The flash, flash-3, sage and xformers backends need CUDA, `_hub` variants included. On a server whose device is not CUDA (`mps`, `cpu`), `validate_workflow` warns `attention_backend_unavailable` for one set at pipeline or component level, naming the backend and the device: the job would fail at load. It never refuses, and a `native`-family backend never warns. A backend diffusers adds later is not on the list until dw's is updated, and draws no warning.
-
-Either form stays on what it names. diffusers' `set_attention_backend` also makes its backend the process-wide default, which the persistent worker would keep for every later model; dw restores the default after pinning, so an H3 `sage_hub` pin cannot reach a Music3 or LTX job that follows it. The server-wide default is `DIFFUSERS_ATTN_BACKEND` in `.env` - leave it `native`, since it reaches every model and sage cannot serve all of them.
+**Sparse moves composition** relative to dense, unlike sage: the same subject, story and cuts, but different framing and set dressing. A draft and its final must use the same backend; mixing `sparse` and `sage_hub` between tiers means the draft no longer predicts the final.
## Attention Slicing
@@ -236,11 +215,11 @@ Either form stays on what it names. diffusers' `set_attention_backend` also make
}
```
-Processes attention in slices to reduce memory at some cost to speed. Automatic on MPS unless `disable_attention_slicing` is set, opt-in elsewhere. Only UNet/ControlNet models take it. On MPS, whether it helps depends on the model's attention head dim: MPS SDPA is slow at 40, 48 and 160 (slicing made SD 1.5 ~20% faster end to end) and fast at 64-128 (slicing made SDXL-shaped attention 2.4x slower - set `disable_attention_slicing` for SDXL-family models). Modular pipelines have no `enable_attention_slicing()` method - the setting is silently skipped rather than failing when the pipeline doesn't support it.
+Processes attention in slices to reduce memory at some cost to speed. Automatic on MPS unless `disable_attention_slicing` is set, opt-in elsewhere. Only UNet/ControlNet models take it; a pipeline with no `enable_attention_slicing()` (a modular pipeline) skips it. On MPS the effect depends on the attention head dim: MPS SDPA is slow at 40, 48 and 160 (slicing made SD 1.5 ~20% faster end to end) and fast at 64-128 (slicing made SDXL-shaped attention 2.4x slower), so set `disable_attention_slicing` for SDXL-family models.
## torch.compile
-Compile a component once it is fully configured - the graph captures final dtypes, adapters, quantization, and offload hooks. Configured per component under `components`:
+Configured per component under `components`. The component is compiled once fully configured, so the graph captures final dtypes, adapters, quantization and offload hooks:
```json
"configuration": {
@@ -258,26 +237,27 @@ Compile a component once it is fully configured - the graph captures final dtype
| Property | Description |
| -------- | ----------- |
-| `repeated_blocks` | Compile only the model's repeated block classes (diffusers regional compilation). Near the same speedup as full compilation with a fraction of the cold-start cost. Recommended. |
-| `mode` | torch.compile mode: `"default"`, `"reduce-overhead"`, `"max-autotune"`. |
-| `fullgraph` | Require a single graph with no breaks - fails fast instead of silently losing speedup. |
-| `dynamic` | Compile with dynamic shapes. Set `true` when resolutions or frame counts vary between runs to avoid recompiles. |
-
-Typical gains are 1.3-1.5x on diffusion transformers, and compilation stacks with the caches above. Notes:
-
-- **First run pays the compile cost.** The [server](SERVER.md)'s persistent worker keeps compiled pipelines loaded between runs, so the cost is paid once per session rather than once per generation. A new session still retraces, but the kernels it built come back from the on-disk cache, which dw keeps across reboots (`TORCHINDUCTOR_CACHE_DIR`, see [Environment Defaults](#environment-defaults)). SDNQ's `triton.jit`/autotune kernels are cached separately by Triton itself (`TRITON_CACHE_DIR`, else `$TRITON_HOME/.triton/cache`, else `~/.triton/cache`); dw sets no `TRITON_*` variable and leaves that cache where it is, shared with any other Triton user on the machine. Both caches' sizes show in `get_server_info`'s `caches` and `GET /api/caches`, and `DELETE /api/caches/{name}` empties one ([SERVER.md](SERVER.md)); the next first step of each shape pays the compile or re-autotune again (100-300 s per shape on MiniMax-H3 for Triton).
-- **Measure the cold cost from `timing`.** A pipeline step's manifest entry carries `timing` ([WORKSPACES.md](WORKSPACES.md#step-timing)): on a cold run the compile cost is about `first_pass_s - pass_s`, and `compiled` says whether compile was on. `pass_s` is wall time at step boundaries, not kernel time - on CUDA kernels run asynchronously, so a pass boundary is when the host saw it.
-- **Pin the attention backend** on a compiled component (`"attention_backend"` in the same `components` entry) rather than using the pipeline-level per-call context manager, which forces recompiles.
-- **Quantize-on-load time is a separate cost.** An SDNQ on-the-fly component re-reads and re-quantizes its bf16 weights on every cold load; `DW_QUANTIZED_CACHE=1` saves the quantized result and reads it back instead - see [QUANTIZATION.md](QUANTIZATION.md#cached-on-the-fly-dw_quantized_cache).
-- **Composes with offloading**: apply `group_offload` and `compile` on the same component and the offload hooks are installed first, as required. Skipped with a warning on MPS. The compile cost is paid again every time the worker releases the pipeline ("Workflow changed - releasing cached models"), i.e. on any workflow change, so it pays off only across many runs of one workflow at one shape.
-- **Don't combine `fullgraph` with a `cache`**: the cache hooks decide skip-or-compute per step, a data-dependent branch diffusers wraps in `torch.compiler.disable` - it needs the graph break that `fullgraph: true` forbids. Compile with the default (partial) graph mode when a cache is active. `validate_workflow` warns `compile_fullgraph_with_cache` on a component that sets `fullgraph: true` in a pipeline with a `cache`.
-- **TorchAO quantization needs compile to be fast** - see [QUANTIZATION.md](QUANTIZATION.md#torchao).
+| `repeated_blocks` | Compile only the model's repeated block classes (diffusers regional compilation): near the speedup of full compilation at a fraction of the cold-start cost. Recommended. A component without `compile_repeated_blocks()` fails |
+| `mode` | torch.compile mode: `"default"`, `"reduce-overhead"`, `"max-autotune"` |
+| `fullgraph` | Require a single graph with no breaks: fails fast instead of silently losing speedup |
+| `dynamic` | Compile with dynamic shapes. Set `true` when resolutions or frame counts vary between runs, to avoid recompiles |
+
+Typical gains are 1.3-1.5x on diffusion transformers, and compile stacks with the caches above.
+
+- **The first run pays the compile cost.** The [server](SERVER.md)'s persistent worker keeps compiled pipelines loaded between runs. It releases them on any workflow change ("Workflow changed - releasing cached models"), and the compile is paid again, so compile pays off across many runs of one workflow at one shape. A new worker retraces, but Inductor's kernels come back from its on-disk cache (`TORCHINDUCTOR_CACHE_DIR`, see [Environment Defaults](#environment-defaults)). SDNQ's Triton kernels are cached by Triton itself (`TRITON_CACHE_DIR`, else `$TRITON_HOME/.triton/cache`, else `~/.triton/cache`); dw sets no `TRITON_*` variable. Both caches' sizes show in `get_server_info`'s `caches` and `GET /api/caches`, and `DELETE /api/caches/{name}` empties one ([SERVER.md](SERVER.md)); the next first step of each shape pays the compile or re-autotune again (100-300 s per shape on MiniMax-H3 for Triton).
+- **Measure the cold cost from `timing`.** A pipeline step's manifest entry carries `timing` ([WORKSPACES.md](WORKSPACES.md#step-timing)): on a cold run the compile cost is about `first_pass_s - pass_s`, and `compiled` says whether compile was on. `pass_s` is wall time at step boundaries, not kernel time.
+- **Pin the attention backend** on a compiled component ([Attention Backends](#attention-backends)).
+- **Don't combine `fullgraph` with a `cache`.** The cache's per-step skip decision is a data-dependent branch diffusers wraps in `torch.compiler.disable`, which needs the graph break `fullgraph: true` forbids. `validate_workflow` warns `compile_fullgraph_with_cache` on such a component.
+- **Composes with offloading**: `group_offload` and `compile` on the same component install the offload hooks first, as required.
+- **Skipped on MPS** with a warning.
+- **Quantize-on-load is a separate cost.** An SDNQ on-the-fly component re-quantizes on every cold load unless `DW_QUANTIZED_CACHE=1` ([QUANTIZATION.md](QUANTIZATION.md#cached-on-the-fly-dw_quantized_cache)).
+- **TorchAO quantization needs compile to be fast** ([QUANTIZATION.md](QUANTIZATION.md#torchao)).
**Example:** [flux-dev-compile.json](../workflows/models/flux-dev-compile.json), [flux-torchao.json](../workflows/models/flux-torchao.json)
## Layerwise Casting
-Store a component's weights in a narrow dtype and upcast only for compute, per component:
+Store a component's weights in a narrow dtype and upcast only for compute:
```json
"transformer": {
@@ -290,18 +270,17 @@ Store a component's weights in a narrow dtype and upcast only for compute, per c
}
```
-Both `storage_dtype` and `compute_dtype` are required. Applied via the component's own `enable_layerwise_casting()` right after it loads, so it composes with quantization and group offloading on the same component.
+Both dtypes are required. It is applied through the component's own `enable_layerwise_casting()` right after it loads, so it composes with quantization and group offloading on the same component. The pipeline `configuration` takes the same block.
## Memory Format and Attention Processors
-Three older per-component knobs, set in the pipeline `configuration` beside
-`vae` / `unet` / `transformer` and applied right after the components load:
+Three per-component keys, set in the pipeline `configuration` beside `vae` / `unet` / `transformer` and applied right after the components load:
| Key | Where | Effect |
| --- | ----- | ------ |
| `channels_last` | `vae`, `unet` | `to(memory_format=torch.channels_last)`. Faster convolutions on CUDA for a convolutional UNet or VAE; nothing to gain on a transformer |
-| `enable_forward_chunking` | `unet` | Runs the UNet's feed-forward layers in chunks - less peak memory, slightly slower |
-| `attn_processor_type` | `unet`, `transformer` | Names an attention processor class to install with `set_attn_processor` (the name is resolved and constructed, so it goes through the `_type` conversion: `"AttnProcessor2_0"`). For a per-call backend instead, see [Attention Backends](#attention-backends) |
+| `enable_forward_chunking` | `unet` | Runs the UNet's feed-forward layers in chunks: less peak memory, slightly slower |
+| `attn_processor_type` | `unet`, `transformer` | An attention processor class to construct and install with `set_attn_processor`, named through the `_type` conversion (`"AttnProcessor2_0"`). For a per-call backend instead, see [Attention Backends](#attention-backends) |
```json
"configuration": {
@@ -313,43 +292,32 @@ Three older per-component knobs, set in the pipeline `configuration` beside
## Memory Offloading
-`offload` (`"model"` or `"sequential"`) and `group_offload` trade speed for VRAM by streaming weights between system memory and the accelerator instead of keeping everything resident. `"model"` moves whole submodules and costs the least speed; `"sequential"` moves individual layers and is the slowest but uses the least memory; block/leaf-level `group_offload` sits between the two and is what a modular pipeline's self-loaded components use, since they aren't reachable in time for `offload`. Full configuration syntax is in [WORKFLOW_GUIDE.md](WORKFLOW_GUIDE.md#memory-offloading). Omit both for the fastest run, when VRAM allows it. Measured on an RTX 3090 (RECIPES_24GB.md): sequential
-offload costs about 6x per step on FLUX dev and 3.4x on Z-Image against model offload;
-a block-level, streamed `group_offload` of the transformer from pinned host memory
-matches the resident per-step cost while holding one block on the card, at ~22 GB of
-pinned host RAM per FLUX pipeline - the placement the unquantized FLUX templates use.
+`offload` (`"model"` or `"sequential"`) and `group_offload` stream weights between system memory and the accelerator instead of keeping them resident; syntax is in [WORKFLOW_GUIDE.md](WORKFLOW_GUIDE.md#memory-offloading). Omit both for the fastest run when VRAM allows.
-`"residency": "on_demand"` on a component is the cheap case of the same trade: the model rests in system memory and is moved to the device whole around each of its own calls. That is a bad deal for anything called once per step, and a good one for a VAE called twice a run - it frees the VAE's VRAM for the denoise loop at the cost of two transfers, where group offloading the same VAE would restream it once per decode tile. See [On-demand components](WORKFLOW_GUIDE.md#on-demand-components).
+- `"model"` moves whole submodules and costs the least speed.
+- `"sequential"` moves individual layers: the slowest, and the least memory.
+- `group_offload` (block or leaf level) sits between them, and is what a modular pipeline's self-loaded components use, since they are not reachable in time for `offload`.
+- `"residency": "on_demand"` on a component rests the model in system memory and moves it to the device whole around each of its own calls. A bad deal for anything called once per step; a good one for a VAE called twice a run, where group offloading would restream it once per decode tile ([On-demand components](WORKFLOW_GUIDE.md#on-demand-components)).
-**Example:** [flux-dev.json](../workflows/models/flux-dev.json) (`"offload": "model"`), [z-image.json](../workflows/models/z-image.json) (`"offload": "sequential"`), [video-with-audio.json](../workflows/templates/minimax/video-with-audio.json) (`group_offload` per component), [reference-to-video.json](../workflows/templates/minimax/reference-to-video.json) (`group_offload` for the transformer, `on_demand` for the VAEs)
+Measured on an RTX 3090 ([RECIPES_24GB.md](RECIPES_24GB.md)): sequential offload costs about 6x per step on FLUX dev and 3.4x on Z-Image against model offload. A block-level, streamed `group_offload` of the transformer from pinned host memory matches the resident per-step cost while holding one block on the card, at ~22 GB of pinned host RAM per FLUX pipeline; the unquantized FLUX templates use it.
+
+**Example:** [flux-dev.json](../workflows/models/flux-dev.json) (`"offload": "model"`), [krea2.json](../workflows/models/krea2.json) (`"offload": "sequential"`), [video-with-audio.json](../workflows/templates/minimax/video-with-audio.json) (`group_offload` per component), [reference-to-video.json](../workflows/templates/minimax/reference-to-video.json) (`group_offload` for the transformer, `on_demand` for the VAEs)
## Reading Memory While Offloading
-A workflow that offloads keeps its weights in host memory by design, so the
-card can sit near-empty through a generation and the VRAM figures alone say
-nothing about what a run holds or fails to release. `get_memory` (MCP) and
-`GET /api/memory` report both: `gpu_*` is the card, `host_memory_rss_mb` is
-what the worker process holds and `host_memory_peak_rss_mb` the most it has
-ever held, beside the machine's `host_memory_total_mb` /
-`host_memory_available_mb`.
-
-`host_pinned_reserved_mb` / `host_pinned_allocated_mb`, where the platform
-reports them, are torch's pinned-host cache - the staging buffers group
-offloading moves weights through. They are part of `host_memory_rss_mb` and
-invisible in every `gpu_*` figure, so a worker that has released every model
-and still holds gigabytes is usually holding these; they are returned when
-the worker switches to a different workflow (#98). A host field is absent,
-rather than null, on a platform that cannot measure it.
+A workflow that offloads keeps its weights in host memory, so the card can sit near-empty through a generation and the VRAM figures alone say nothing about what a run holds or fails to release. `get_memory` (MCP) and `GET /api/memory` report both: `gpu_*` is the card; `host_memory_rss_mb` is what the worker process holds and `host_memory_peak_rss_mb` the most it has held, beside the machine's `host_memory_total_mb` and `host_memory_available_mb`.
+
+`host_pinned_reserved_mb` / `host_pinned_allocated_mb`, where the platform reports them, are torch's pinned-host cache: the staging buffers group offloading moves weights through. They are part of `host_memory_rss_mb` and invisible in every `gpu_*` figure, so a worker that has released every model and still holds gigabytes is usually holding these; they are returned when the worker switches to a different workflow (#98). A host field is absent, not null, on a platform that cannot measure it.
## TF32 and cuDNN
-Device-level settings, read once at startup from `~/.diffusers_helper/settings.json`:
+Device-level settings, read once at startup from `~/.diffusers_helper/settings.json` (the directory moves with `DIFFUSERS_HELPER_ROOT`):
| Setting | Default | Effect |
| ------- | ------- | ------ |
-| `enable_tf32` | `true` | Sets `torch.set_float32_matmul_precision("high")`, and on CUDA also `torch.backends.cuda.matmul.allow_tf32 = True`. ~2x faster matmuls on Ampere+ GPUs (RTX 30/40 series, A100, H100) with minor precision loss. No effect outside CUDA. |
-| `cudnn_benchmark` | `true` | CUDA only. Autotunes cuDNN algorithm selection - fastest for a workflow with fixed input sizes, can add overhead when sizes vary run to run. |
-| `cudnn_deterministic` | `false` | CUDA only. Set `true` to trade speed for reproducible output given the same seed. |
+| `enable_tf32` | `true` | `torch.set_float32_matmul_precision("high")`, and on CUDA also `torch.backends.cuda.matmul.allow_tf32 = True`. ~2x faster matmuls on Ampere and newer (RTX 30/40, A100, H100) with minor precision loss |
+| `cudnn_benchmark` | `true` | CUDA only. Autotunes cuDNN algorithm selection: fastest for fixed input sizes, overhead when sizes vary run to run |
+| `cudnn_deterministic` | `false` | CUDA only. `true` trades speed for reproducible output given the same seed |
```json
{ "enable_tf32": true, "cudnn_benchmark": true, "cudnn_deterministic": false }
@@ -357,28 +325,26 @@ Device-level settings, read once at startup from `~/.diffusers_helper/settings.j
## Environment Defaults
-Set automatically at import unless already present in the environment (export your own value to override, or put it in a `.env` file in the directory dw starts from - [.env.example](../.env.example) lists these and the other knobs worth knowing: the Hub, diffusers, transformers and SDNQ):
+Set at import unless already in the environment. Export your own value to override, or put it in a `.env` file in the directory dw starts from; [.env.example](../.env.example) lists these and the other Hub, diffusers, transformers and SDNQ knobs.
| Variable | Default | Effect |
| -------- | ------- | ------ |
-| `PYTORCH_CUDA_ALLOC_CONF` | `expandable_segments:True` | Lets the CUDA allocator grow segments instead of fragmenting fixed-size ones. Multi-step workflows churn differently-shaped allocations (generate, upscale, interpolate); fragmentation is what OOMs a card that nominally has room. |
-| `HF_ENABLE_PARALLEL_LOADING` | `true` | Loads sharded checkpoints in parallel - faster cold starts. |
-| `PYTORCH_MPS_HIGH_WATERMARK_RATIO` | `0.0` | MPS only - use all available unified memory. |
-| `TORCHINDUCTOR_CACHE_DIR` | `$XDG_CACHE_HOME/dw/torchinductor` (`~/.cache/dw/torchinductor`) | Keeps `torch.compile`'s on-disk cache across reboots - Inductor's own default is under `/tmp`. A new worker still retraces, but a cache hit skips codegen and kernel builds, the bulk of a cold compile. Entries are per shape, so a new size or frame count compiles once more. Listed as `inductor` by `GET /api/caches` and cleared with `DELETE /api/caches/inductor`. Triton's cache is not set by dw and stays at its own default (`~/.triton/cache`) - see the compile notes above. |
+| `PYTORCH_CUDA_ALLOC_CONF` | `expandable_segments:True` | Lets the CUDA allocator grow segments instead of fragmenting fixed-size ones. Multi-step workflows churn differently-shaped allocations; fragmentation is what OOMs a card that nominally has room |
+| `HF_ENABLE_PARALLEL_LOADING` | `true`, `false` on macOS | Loads sharded checkpoints in parallel. Off on macOS: concurrent loader threads copying onto MPS segfaulted a quantized load |
+| `PYTORCH_MPS_HIGH_WATERMARK_RATIO` | `0.0` | MPS only: use all available unified memory |
+| `TORCHINDUCTOR_CACHE_DIR` | `$XDG_CACHE_HOME/dw/torchinductor` (`~/.cache/dw/torchinductor`) | Keeps `torch.compile`'s on-disk cache across reboots (Inductor's own default is under `/tmp`). A cache hit skips codegen and kernel builds, the bulk of a cold compile; entries are per shape. Listed as `inductor` by `GET /api/caches`, cleared with `DELETE /api/caches/inductor` |
-For faster model downloads, optionally `pip install hf_transfer` and set `HF_HUB_ENABLE_HF_TRANSFER=1`. Not enabled automatically - it bypasses the Python HTTP stack and breaks some proxy setups.
+For faster downloads, optionally `pip install hf_transfer` and set `HF_HUB_ENABLE_HF_TRANSFER=1`. It is not enabled automatically: it bypasses the Python HTTP stack and breaks some proxy setups.
## MPS Notes
Apple Silicon has narrower acceleration support than CUDA:
-- No flash-attn, no Triton, no bitsandbytes - `attention_backend` is effectively CUDA-only; use `"native"`-family backends or leave it unset on MPS. Validate warns `attention_backend_unavailable` for a CUDA-only one ([Attention Backends](#attention-backends)). `compile` is skipped with a warning (inductor support on MPS is immature). A workflow that loads a bitsandbytes checkpoint (`flux2-dev`, `multi-image-reference`) or a TorchAO int4 config (`flux-torchao`) has no Mac path yet.
-- Workflows written for CUDA are adapted rather than refused: a `cuda` device becomes `mps`, SDNQ's `quantization_device` follows it and its quantized matmul is switched off (it runs through `torch._int_mm`, ~500x slower on MPS), group-offload CUDA streams are dropped, and `"offload": "sequential"` becomes `"model"`. Each change is logged as a warning.
-- Attention slicing is automatic; set `disable_attention_slicing` for SDXL-family models (head dim 64), where it is 2.4x slower in attention. It helps SD 1.5.
-- `float16` can produce NaN values on Apple Silicon - use `bfloat16`; dw only warns, it doesn't override the dtype for you.
-- `PYTORCH_MPS_HIGH_WATERMARK_RATIO` defaults to `0.0` (use all unified memory) unless already set in the environment.
-- Ops the MPS backend lacks: with `PYTORCH_ENABLE_MPS_FALLBACK=1` (set by the `fp4-fp8-for-torch-mps` package install.sh adds) they run on the CPU, and dw logs torch's warning when one does. Without it they raise.
-- Memory figures are real: allocated, driver-reserved, and Metal's recommended working set as the total. `vram_estimate` checks a Mac against that total, since the catalog's `cost` entries are CUDA cards; the per-voxel figures were calibrated on CUDA.
-- Cost estimates quoted on a Mac are the CUDA figures (`basis: "other_device"`) until the Mac's own runs build observed history - expect them to be optimistic.
-- Parallel checkpoint loading (`HF_ENABLE_PARALLEL_LOADING`) defaults to off on macOS - concurrent loader threads copying onto MPS segfaulted a quantized load. Set it explicitly to override.
-- Offloading has less benefit than on CUDA, since unified memory is already shared between CPU and GPU.
+- No flash-attn, Triton or bitsandbytes. Use a `native`-family `attention_backend` or leave it unset; validate warns `attention_backend_unavailable` for a CUDA-only one, and `sparse` runs dense. `compile` is skipped with a warning. A workflow that loads a bitsandbytes checkpoint (`flux2-dev`, `multi-image-reference`) or a TorchAO int4 config (`flux-torchao`) has no Mac path.
+- Workflows written for CUDA are adapted rather than refused, each change logged as a warning: a `cuda` device becomes `mps`, SDNQ's `quantization_device` and `return_device` follow it and its `use_quantized_matmul` is switched off (it runs through `torch._int_mm`, ~500x slower on MPS), group-offload CUDA streams are dropped, and `"offload": "sequential"` becomes `"model"`.
+- Attention slicing is automatic; see [Attention Slicing](#attention-slicing) for when to disable it.
+- `float16` can produce NaN values on Apple Silicon; use `bfloat16`. dw warns and does not override the dtype.
+- Ops the MPS backend lacks run on the CPU with `PYTORCH_ENABLE_MPS_FALLBACK=1` (set by the `fp4-fp8-for-torch-mps` package install.sh adds), and dw logs torch's warning when one does. Without it they raise.
+- Memory figures are real: allocated, driver-reserved, and Metal's recommended working set as the total. `vram_estimate` checks a Mac against that total; the per-voxel figures were calibrated on CUDA.
+- Cost estimates on a Mac are the CUDA figures (`basis: "other_device"`) until the Mac's own runs build observed history; expect them to be optimistic.
+- Offloading gains less than on CUDA, since unified memory is already shared between CPU and GPU.
diff --git a/docs/AGENT_LOOP.md b/docs/AGENT_LOOP.md
index a4583e0ff..c35466601 100644
--- a/docs/AGENT_LOOP.md
+++ b/docs/AGENT_LOOP.md
@@ -1,95 +1,113 @@
# Agent Loop
-Some of the Issues in this repo are worked by an automated implementer/tester
-agent loop, not a human. It lives in a separate repo
-([`dkackman/iterate`](https://github.com/dkackman/iterate) — private) that
-drives two Claude Code agents in strict alternation against this repo's
-Issues. Nothing about running the loop lives here; this page exists so that
-anyone who opens or comments on one of its Issues — a human, or another
-agent joining in — knows how to act on it correctly.
+Some Issues in this repo are worked by an automated agent loop, not a human.
+The harness is [`dkackman/harnest`](https://github.com/dkackman/harnest); its
+[ticket protocol](https://github.com/dkackman/harnest/blob/main/docs/harness/ticket-protocol.md)
+is the full rule set this page summarizes, and `lib/classify.jq` there is the
+authoritative map from labels to the session that acts next. Nothing about
+running the loop lives here. This page is for anyone, human or agent, who
+opens or comments on one of its Issues.
## The two roles
-- **Implementer** — has this repo checked out and SSH access to the box
- running the MCP server. Reproduces, fixes, deploys, and hands the ticket
+- **Implementer**: has this repo checked out and SSH access to the box running
+ the MCP server. Triages, reproduces, fixes, deploys and hands the ticket
back. Never verifies its own fix.
-- **Tester** — talks to the MCP server only as a protocol client (MCP tool
- calls), no source checkout, no shell/SSH access to the server box. Its job
- is independent verification: it re-runs the original repro over MCP and
- either confirms the fix or bounces the ticket back.
+- **Tester**: talks to the MCP server only as a protocol client, with no
+ source checkout and no shell on the server box. It re-runs the original
+ repro over MCP and either confirms the fix or bounces the ticket.
-That asymmetry — the only role that can mark an issue verified is the one
-with no ability to patch around a bug — is the entire point of the loop. A
-human or third agent joining in should preserve it: don't fix and verify the
-same ticket yourself.
+The only role that can mark an issue verified is the one that cannot patch
+around a bug. Preserve that: do not fix and verify the same ticket yourself.
-A third, standalone **regression agent** periodically runs a growing suite
-of scripted MCP calls (`regression-suite-*.md` in the `iterate` repo) against
-the live server and files/comments on Issues for anything that regresses. It
-doesn't participate in the implementer/tester handoff.
+Other sessions in the harness:
+
+- **Lead** (`owner:lead`): designs a feature with Don, turns an `idea` into a
+ feature or a single fix, or closes it.
+- **Docs reviewer**: read-only; closes a `docs-review` ticket.
+- **Regression agent**: runs the scripted MCP suites (`regression-suite-*.md`
+ in harnest) against the live server and files `regression` and
+ `performance` issues. It takes no part in the implementer/tester hand-off.
## Outside filings
-The loop runs as one GitHub login (the repo owner's) and must not act
-unattended on text filed by anyone else in this public repo. An issue filed
-under a different login — including a `field-report` from someone else's
-session — is relabelled `owner:don` + `status:needs-approval` before either
-role works it, and is handed to the loop, or not, only after the maintainer
-reviews it.
+The loop runs as the repo owner's GitHub login and must not act unattended on
+text anyone else filed in this public repo. An issue from another login,
+including a `field-report` from someone else's session, is relabelled
+`owner:don` + `status:needs-approval` before any role works it, and reaches
+the loop only if the maintainer hands it back.
+
+**Security findings are never public issues.** A boundary that did not hold
+(a gate, a path, a URL, a disclosure, a destructive call without
+acknowledgement) goes to a draft GitHub security advisory. No agent may add
+the `security` label.
## Reading a ticket
-Tickets use the **MCP agent-loop ticket** issue template. Two label
-families carry all the state — check both before acting:
-
-- **`owner:*`** — exactly one of `owner:implementer` / `owner:tester` /
- `owner:don` at a time: whoever is expected to act on it next. If you're
- not that owner, leave the issue alone beyond reading it (or the specific
- handoff comment/label a role prompt allows).
-- **`status:*`** — where the ticket is in its lifecycle:
- - *(no status label)* — open, ready for the implementer to reproduce.
- - `status:fixed-pending-verify` — implementer has fixed and deployed;
- waiting on the tester to re-run the repro over MCP.
- - `status:verified` — tester confirmed the fix over MCP; issue is closed
- as `completed`.
- - `status:needs-info` — a question bounce; whoever is asked needs to
- answer before work continues.
- - `status:needs-approval` (+ `owner:don`) — parked for the human. The
- implementer uses this for anything beyond a rename-level change
- (engine behavior, breaking syntax). Neither agent touches a parked
- issue.
-
-Two built-in GitHub labels close a ticket without a fix:
-
-- **`wontfix`** — the implementer's call, with a reason in a comment; issue
- closed as `not planned`. The tester may reopen once with new evidence; a
- second `wontfix` is final.
-- **`duplicate`** — closed as `not planned`, with a comment naming the
- issue it duplicates (`duplicate of #NN`). A closed issue is still
- canonical for duplicate detection — the implementer checks
- `gh issue list --state all` before starting work, not just open issues.
-
-`breaking-change` marks an Issue whose fix changed the MCP interface, so the
-tester adjusts its calls instead of filing the change as a new bug.
+Tickets use the **MCP agent-loop ticket** issue template (ideas use **MCP
+agent-loop idea**). Two label families carry the state; check both before
+acting.
+
+**`owner:*`** is exactly one of `owner:implementer`, `owner:tester`,
+`owner:lead` or `owner:don`: whoever acts next. If it is not you, leave the
+issue alone beyond reading it. The owner label is the only signal: a status
+addressed to Don is stale once he swaps the owner back.
+
+**`status:*`** is where the ticket is:
+
+| Label | Meaning |
+| --- | --- |
+| *(none)* | Open, ready for the implementer to reproduce |
+| `status:fixed-pending-verify` | Fixed and deployed; the tester re-runs the repro over MCP |
+| `status:verified` | The tester confirmed the fix over MCP; closed as `completed` |
+| `status:reviewed` | A `docs-review` fix closed by the docs reviewer (never `status:verified`) |
+| `status:needs-info` | A question bounce; whoever is asked answers before work continues |
+| `status:needs-approval` | With `owner:don`: parked for the human, after a written proposal. Required for engine or syntax changes and anything breaking beyond a rename. Neither agent touches it |
+| `status:plan-review` | A feature plan is posted and waiting for Don |
+| `status:plan-approved` | Don approved the plan. Only Don adds it |
+| `status:needs-spec` | The tester writes acceptance cases for the plan's stages |
+
+Labels that close a ticket without a fix:
+
+- **`wontfix`**: the implementer's call, with a reason in a comment; closed as
+ `not planned`. The tester may reopen once with new evidence; a second
+ `wontfix` is final.
+- **`duplicate`**: closed as `not planned` with `duplicate of #NN` in a
+ comment. Closed issues count for duplicate detection
+ (`gh issue list --state all`). A verified issue that reproduces again is not
+ a duplicate: it becomes a new `regression` issue (`regression of #NN`), and
+ nobody reopens the old one.
+
+Other labels:
+
+| Label | Meaning |
+| --- | --- |
+| `docs-review` | A fix to text neither the server nor the plugin serves; the docs reviewer verifies it instead of the tester |
+| `breaking-change` | The fix changed the MCP interface; the tester adjusts its calls rather than filing a new bug |
+| `field-report` | Filed from Don's own dw sessions, not the harness |
+| `regression`, `performance` | Filed by the regression agent |
+| `idea` | An idea for the lead to assess |
+| `feature`, `stage` | Work designed with Don and built in stages; a `stage` is a child issue of one feature |
+| `lane:batch` | A plan review with no question of a kind Don decides, approved in a batch |
+| `parked` | Deferred: on `owner:lead`, until every blocking issue closes; on `owner:don`, Don's |
+| `release`, `release-blocker` | While an open `release` issue exists, only `release-blocker` issues move |
+| `stabilization`, `arch-approved` | dw's stabilization freeze (in force while `docs/stabilization/FREEZE` exists on `develop`): work parked for it, and new surface Don approved during it |
+| `arch-review` | A hand-off waiting on the architecture reviewer, who checks it against `docs/ARCHITECTURE.md` and passes or bounces it |
+| `consolidation` | Filed weekly from `scripts/arch_report.py`; Don plans it (to `owner:lead`) or closes it |
+| `backend:shared`, `backend:cuda`, `backend:mps` | What the bug is about |
+| `target:lem`, `target:mini-ai`, `target:minid`, `target:local` | Which server's loop claims it; a non-lem target is for Don to triage |
+| `verified-on:mps` | Verified on the Mac only; CUDA has not re-verified the fix |
+| `priority:2`, `priority:3` | Backlog rank |
## If you want to participate
-Whether you're a human or another agent:
-
-- Only touch an issue that's currently owned by you (or, for a human, one
- parked with `owner:don`).
-- When you hand a ticket to the next owner, swap the `owner:*` label and
- say what you did in a comment — the next actor has no memory of this
- session, only the issue thread.
-- Keep implementer and tester roles separate. If you're fixing code, don't
- also close the issue as verified — that requires an independent MCP call
- from someone who didn't write the patch.
-- Reference the issue number in any commit that fixes it
- (`fix(mcp): #42 - ...`), and work on a branch merged to `develop`, never
- `master`.
-- Only the tester (or an equivalently independent verifier) closes an issue
- as `completed`/`status:verified`, and only after a real MCP call
- reproduces the fix — not by reading the diff.
-
-See the `iterate` repo's `CLAUDE.md` for the full protocol this is
-summarized from, including how the automated loop itself is run.
+- Only touch an issue owned by you (or, for a human, one parked with
+ `owner:don`).
+- To hand a ticket on, swap the `owner:*` label and say what you did in a
+ comment. The next actor has no memory of your session, only the thread.
+- Keep implementer and tester separate. Only the tester (or an equally
+ independent verifier) closes an issue as `completed` with
+ `status:verified`, and only after a real MCP call shows the fix, not by
+ reading the diff. The one exception is `docs-review`.
+- Reference the issue in the commit that fixes it, with a scope
+ (`fix(mcp): #42 - ...`), on a branch merged to `develop`, never `master`.
diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md
index 15023b2de..f3b84b325 100644
--- a/docs/ARCHITECTURE.md
+++ b/docs/ARCHITECTURE.md
@@ -30,7 +30,7 @@ one to open.
| On-demand and offload | `dw/pipeline_processors/placement.py`: `place_component`, `has_component_group_offload` | On-demand residency and `group_offload` cannot both be set on one component, and either one stops the whole pipeline from being moved to the device. | `tests/test_modular_pipeline.py::TestOnDemandResidency::test_group_offload_and_on_demand_together_are_rejected`, `tests/test_modular_pipeline.py::TestOffloadDevice::test_component_group_offload_is_not_moved_to_device` |
| Quantized component cache | `dw/quantized_cache.py`: `load_components`, `from_pretrained`, `cache_key` | A key that can't pin the weights isn't cached, the cache never fails a job, and the save comes before pruning, so an entry is the whole component. | `tests/test_quantized_cache.py` |
| LoRAs | `dw/pipeline_processors/adapters.py`: `active_loras`, `load_loras` | A `loras` entry whose `model_name` is null is switched off. | `tests/test_lora_disable.py::TestLoadLoras::test_an_all_null_entry_loads_nothing` |
-| Step progress | `dw/pipeline_processors/progress.py`: `reported_progress_bars` | A pipeline with no step callback reports its denoise steps through its progress bars. | `tests/test_modular_progress.py::test_every_advance_of_the_bar_is_reported` |
+| Step progress | `dw/pipeline_processors/progress.py`: `reported_progress_bars`, `reported_blocks` | A pipeline with no step callback reports its denoise steps through its progress bars. `reported_blocks` names each top-level block of a sequential modular graph in a `log` event as it begins; a dw block that declares `idle_without = ///` with a `manifest.json`, unless the flat layout is chosen (the test pins the flat-layout exception). | `tests/test_runs.py::TestRunDirectories::test_the_flat_layout_writes_where_it_always_did` |
| A file's role (final / intermediate / input) | `dw/roles.py` | Decided per step: result.role, then a final/intermediate subfolder, then data flow (a step a media-writing or unsaved consumer reads is intermediate). Engine records it at run end, with `role_declared` where result.role stated it; a run that predates roles recorded none, so the listing falls back on the file's own `final`/`intermediate` subfolder, then the demotion below, and a file neither places has none. At read time the gallery (`GET /api/gallery`, its `role=` filter, the metadata route) also demotes a file a later run of the same workspace read - an `output:` input (an unpinned `latest` is the newest run started before the reader) or an `asset:` keep_output kept from it - unless its role was declared (`consumed_outputs`, `listed_role`, once per listing; no manifest is rewritten). | `tests/test_roles.py`, `tests/test_roles.py::test_a_file_a_later_run_read_is_consumed`, `tests/test_roles.py::test_listed_role`, `tests/test_roles.py::test_listed_role_of_a_run_that_recorded_none`, `tests/test_gallery_roles.py::test_older_run_reads_its_intermediate_subfolder_as_intermediate`, `tests/test_gallery_roles.py::test_a_pre_roles_file_a_later_run_read_lists_as_intermediate`, `tests/test_gallery_roles.py::test_metadata_agrees_with_the_listing_for_an_older_run`, `tests/test_gallery_roles.py::test_a_file_a_later_run_read_lists_as_intermediate`, `tests/test_gallery_roles.py::test_a_declared_final_stays_final` |
-| Notes on runs, files, assets, workspaces | dw/notes.py | notes.json beside what it describes (run dir or asset dir; a file's own note else the run's), workspace.json at a workspace root; only the server writes them, never the engine; ≤4000 chars, plain text. | tests/test_notes.py |
+| Notes on runs, files, assets, workspaces | `dw/notes.py` | A note is `notes.json` beside what it describes (run dir or asset dir; a file's own note, else the run's), or `workspace.json` at a workspace root. Only the server writes them, never the engine; at most 4000 characters, plain text. | `tests/test_notes.py` |
| Run versions | `dw/runs.py`: `open_run`, `run_versions`, `record_run_versions`, `copy_runs_in` | A run takes its number once, when it opens (or, copied in from another workspace, when `copy_runs_in` places it), so deleting a middle run leaves a gap rather than renumbering. | `tests/test_runs.py::TestRunVersions::test_a_deleted_middle_run_leaves_a_gap_rather_than_renumbering`, `tests/test_runs.py::TestRunVersions::test_runs_started_in_the_same_second_still_number_upward` |
| Run-version surfaces | `dw/runs.py`: `run_versions`, `dw/server/routes/gallery.py`, `dw/server/outputs.py`, `dw_mcp/tools_catalog.py`: `list_gallery`, `ui/src/lib/pages/GalleryPage.svelte` | Only `dw/runs.py` computes the number, and every other module reads it back. | `tests/test_runs.py::TestRunVersions::test_an_output_reference_can_name_a_run_by_its_version` |
| What the server says about an output file | `dw/server/outputs.py`: `output_annotations` | The gallery listing (`dw/server/routes/gallery.py`) and the metadata route (`dw/server/routes/media.py`) describe an output file's `role`, `note`, `note_source`, `run_note`, `starred` and `version` through `output_annotations`, and neither builds those fields on its own. | `tests/test_gallery_roles.py::test_metadata_agrees_with_the_listing_on_a_starred_noted_demoted_file` |
| Result subfolders | `dw/subfolders.py`: `subfolder_errors`, `dw/security.py`: `SUBFOLDER_PATTERN` | A step's `result.subfolder` is checked for shape at validation. | `tests/test_subfolders.py::TestValidationErrorsIntegration::test_validation_errors_reports_a_bad_subfolder` |
| Template subfolder roles | `workflows/templates/`, `dw/workflows/` | Every saving step of a template is marked `final` or `intermediate`, with at least one `final`, and packaged builtins stay unmarked. | `tests/test_template_subfolders.py::test_every_saving_step_of_a_template_names_its_role`, `tests/test_template_subfolders.py::test_the_packaged_builtins_stay_unmarked` |
| Frames carrying their rate | `dw/media_types.py`: `FrameList` (re-exported by `dw/tasks/video_utils.py`), `rated_videos` | The one list of frames (or of videos) that carries `.fps` and `.shots`: a video loaded from a file (`dw/argument_media.py`) and a step that knows its rate but would return bare frame lists both answer it - `h3_tiles`' tiled refine `videos`, and every pipeline step whose pipeline reports its own `fps` (MiniMax-H3's property; `dw/output_extraction.py`: `attach_video_frame_rate`, called by `pipeline.py`'s `_run_once` beside `attach_audio_sample_rate`, #1056) - through `rated_videos`, the batch and each video in it, so `pair_audio` and the result writer read `getattr(video, "fps", None)` instead of fitting and playing at the default 8 fps. A pipeline that reports no rate leaves its lists bare. No second class of that name. | `tests/test_h3_tiles.py::TestTileLoopReporting::test_the_videos_are_dws_one_frame_list`, `tests/test_pipeline_components.py::TestVideoFrameRate::test_a_dict_outputs_videos_carry_the_pipelines_rate`, `tests/test_pipeline_components.py::TestVideoFrameRate::test_a_pipeline_with_no_rate_leaves_the_lists_bare`, `tests/test_modular_output_properties.py::TestTheBasesOwnTakeSavesBesideIt::test_the_base_take_pairs_at_its_own_rate_with_no_padding`, `tests/test_video_utils.py` (the `FrameList` cases) |
-| Writing a video at its rate | `dw/writers.py`: `encode_video`, `exact_frame_rate` | A fractional rate is written as the exact rational (30000/1001, 24000/1001, 60000/1001), never truncated to a whole rate, and no `dw/` or `dw_mcp/` module imports `encode_video` from `diffusers.utils` - every audio-carrying video write goes through `dw.writers.encode_video` (`dw/result.py`, `dw/pipeline_processors/chain.py`). A declared fractional `result.fps` is therefore accepted by `fps_errors`. | `tests/test_fractional_fps.py::test_encode_video_with_audio_keeps_fractional_rate`, `tests/test_fractional_fps.py::test_no_dw_module_calls_the_diffusers_encode_video_directly`, `tests/test_fractional_fps.py::test_the_diffusers_hook_the_wrapper_swaps_still_exists` |
+| Writing a video at its rate | `dw/writers.py`: `encode_video`, `exact_frame_rate` | A fractional rate is written as the exact rational (30000/1001, 24000/1001, 60000/1001), never truncated to a whole rate, and no `dw/` or `dw_mcp/` module imports `encode_video` from `diffusers.utils` - every audio-carrying video write goes through `dw.writers.encode_video` (`dw/result.py`, `dw/pipeline_processors/chain_spill.py` - a chain's segments on disk, `SegmentSpill`, `SegmentedFrames` and `SavedFrames`, re-exported by `chain.py`). A declared fractional `result.fps` is therefore accepted by `fps_errors`. | `tests/test_fractional_fps.py::test_encode_video_with_audio_keeps_fractional_rate`, `tests/test_fractional_fps.py::test_no_dw_module_calls_the_diffusers_encode_video_directly`, `tests/test_fractional_fps.py::test_the_diffusers_hook_the_wrapper_swaps_still_exists` |
| JSON records beside media | `dw/media_types.py`: `JsonRecord`, `dw/result.py`: `Result.save_artifact` | A dict marked `JsonRecord` is saved whole as one `.json`, whatever content type the step declares, and is never split into one file per key. | `tests/test_face_track.py::test_json_record_is_saved_whole_under_a_video_content_type`, `tests/test_face_track.py::test_the_task_result_saves_crops_as_video_and_track_as_one_json` |
| Reading a JSON record back | `dw/locations.py`: `load_json_record` | A task reads a saved JSON record (`fit_to_model`'s, `crop_face_track`'s) only through `load_json_record`: the path is confined like any media path, then the extension must be `.json`, then the size is checked against `MAX_JSON_SIZE`, and only then is it parsed. `fit._read_fit` and `face_track._read_track` call it, never `open`. | `tests/test_locations.py::TestLoadJsonRecord::test_traversal_out_of_the_roots_is_refused`, `tests/test_locations.py::TestLoadJsonRecord::test_a_non_json_extension_is_refused`, `tests/test_locations.py::TestLoadJsonRecord::test_an_oversized_file_is_refused_before_it_is_read`, `tests/test_fit_to_model.py::TestRestore::test_fit_path_with_traversal_is_refused` |
| Shots in a joined video | `dw/shots.py` | Every `AudioVideo` constructor decides what to do with `shots`, and the sample spans are measured from the joined waveform. | `tests/test_shots.py::test_every_audio_video_constructor_site_is_decided`, `tests/test_shots.py::TestConcatVideosShots::test_an_overrun_track_is_measured_not_derived` |
@@ -86,7 +86,7 @@ one to open.
| A `join_windows` list of the wrong length, before the run | `dw/window_count_errors.py`: `window_count_errors`, wired into `validation_errors` | Owned by the task, never a template: any `join_windows` step whose `source` is probeable header-only (`asset:`/`output:` or a readable literal path), with literal `num_frames`/`overlap` and an expanded `videos` list, is checked against `window_count_problem` - the same rule the run refuses with. Anything not knowable stays silent for the run. | `tests/test_window_count_errors.py` |
| Fit to a model and restore | `dw/tasks/fit.py`: `fit_to_model`, `restore_to_source` | fit_to_model's record is the whole of what restore_to_source needs: the scale is inferred from the output's size and must be uniform, and restore returns the source's size times that scale and at most its frame count. | `tests/test_fit_to_model.py` |
| Shared image ops | `dw/tasks/image_ops.py`: `LUMA_WEIGHTS`, `luma`, `split_alpha`, `join_alpha`, `per_frame` | `grade`, `finish` and `lut` take luma (float64 Rec. 709 weights, cast to the input's dtype) and alpha handling from `image_ops`, never from each other, so an RGBA input comes back RGBA with its alpha untouched; `lut.LUMA_WEIGHTS` is a re-export. An image command runs over a video or frame list only through `per_frame`. A media file is read by `task._load_media` with `fetch_image(keep_alpha=True)`, so a file with transparency arrives RGBA; a pipeline input still flattens to RGB. | `tests/test_image_ops.py::TestLuma::test_lut_re_exports_the_one_float64_weights`, `tests/test_image_ops.py::TestAlpha::test_rgba_round_trip_keeps_the_alpha_bytes`, `tests/test_image_ops.py::TestOutputsAreByteIdentical::test_every_command_matches_the_pre_refactor_bytes`, `tests/test_image_ops.py::TestPerFrame::test_a_video_comes_back_as_a_video_with_the_same_frames_and_audio`, `tests/test_finish.py::TestRgbaFileKeepsAlpha::test_a_file_path_comes_back_rgba_with_its_alpha` |
-| Audio level of a deliverable | `dw/audio_qc.py`: `warn_without_headroom`, `warn_if_written_above_full_scale` | A muxed video keeps the pre-encode headroom prediction and falls back to it only when the post-write probe cannot measure the file. A result a later step resets the level of (`consumed_by_normalizer`, `dw/step_cache.py`) draws neither the post-write warning nor the fallback. | `tests/test_result.py::TestNoHeadroom::test_a_clean_video_mux_drops_the_stale_prediction`, `tests/test_result.py::TestNoHeadroom::test_an_unprobeable_video_mux_falls_back_to_the_prediction`, `tests/test_result.py::TestNoHeadroom::test_a_video_a_join_level_matches_draws_no_headroom_warning`, `tests/test_result.py::TestTheWrittenLevel::test_a_file_that_decodes_above_full_scale_warns` |
+| Audio level of a deliverable | `dw/audio_qc.py`: `warn_without_headroom`, `warn_if_written_above_full_scale`, `all_shots_silent_by_design` | A muxed video keeps the pre-encode headroom prediction and falls back to it only when the post-write probe cannot measure the file. A result a later step resets the level of (`consumed_by_normalizer`, `dw/step_cache.py`) draws neither the post-write warning nor the fallback. | `tests/test_result.py::TestNoHeadroom::test_a_clean_video_mux_drops_the_stale_prediction`, `tests/test_result.py::TestNoHeadroom::test_an_unprobeable_video_mux_falls_back_to_the_prediction`, `tests/test_result.py::TestNoHeadroom::test_a_video_a_join_level_matches_draws_no_headroom_warning`, `tests/test_result.py::TestTheWrittenLevel::test_a_file_that_decodes_above_full_scale_warns` |
| Level-spread warning on a replaced track | `dw/step_cache.py`: `audio_replaced_downstream`, `dw/tasks/joins.py`: `warn_on_level_spread` | A join whose track a later `pair_audio` takes as its `video` draws no `level_spread` warning; being that step's `audio` argument doesn't count. | `tests/test_step_cache.py::test_level_spread_warning_suppressed_when_audio_replaced`, `tests/test_step_cache.py::test_audio_replaced_downstream_false_when_it_is_the_audio_or_unread` |
| Template audio-level placement | `workflows/templates/minimax/music-video.json`, `workflows/templates/minimax/music.json` | `normalize_audio` (-3 dBFS) acts only on the track that goes into the final file, so the slices that condition the shots are untouched. `music-video` slices each shot `lead_frames` early and `trim_video` drops the run-up before the join. | — |
@@ -114,7 +114,7 @@ one to open.
| Acceleration warnings | `dw/acceleration_warnings.py`: `acceleration_warnings`, `CACHE_MIN_USEFUL_STEPS`, `CUDA_ONLY_ATTENTION_BACKENDS` (the last entry of `WARNING_CHECKS`) | An acceleration setting that can't pay off (a `first_block`/`mag` cache under `CACHE_MIN_USEFUL_STEPS`, any `taylorseer` cache, `compile.fullgraph` with a cache, a cache on a pipeline class with no transformer or whose transformer lacks `enable_cache()`, a CUDA-only `attention_backend` on a non-CUDA device) is a warning and never a refusal, and an unresolvable value (steps, backend, device) leaves it silent. | `tests/test_validation.py::TestAccelerationWarnings::test_warnings_never_refuse`, `tests/test_validation.py::TestAccelerationWarnings::test_steps_that_resolve_to_nothing_are_silent`, `tests/test_validation.py::TestAccelerationWarnings::test_an_unknown_device_is_silent`, `tests/test_validation.py::TestAccelerationWarnings::test_an_unresolvable_backend_is_silent_on_mps` |
| H3 sparse attention | `dw/sparse_attention.py`: `sparse_call`, `install_sparse`, `install_sparse_for_call`, `apply_sparse_settings`, `SparseH3AttnProcessor`; wired from `dw/pipeline_processors/components.py`: `configure_components` and `dw/pipeline_processors/pipeline.py`: `apply_runtime_settings` | Whether a pipeline call runs sparse is decided once per call, all or nothing (the call's first kernel invocation included); it never changes diffusers' process-wide backend and never reaches `AttentionBackendName`; it installs by a processor swap plus forward hooks, never a `forward` wrapper; and `sparse_tau` is a runtime setting, not part of the weights identity (#931). | `tests/test_sparse_attention_processor.py`, `tests/test_sparse_attention_wiring.py`, `tests/test_sparse_attention_tiles.py`, `tests/test_sparse_attention_backends.py` |
| H3 adapter partition | `dw/adapter_compatibility.py`; the server's index is `dw/server/deps.py` `lora_index` | The LoRA catalog decides first, on the server only: a `proven` entry's `workflow` refuses a step in the other partition, a `trial`/`rejected` entry's only warns. A file no entry places falls to its name: an FL2VA LoRA on a reference step is refused, and one whose name says neither `ref2v` nor `fl2v` is warned. | `tests/test_h3_adapters.py::TestWhatIsRefused::test_a_keyframe_adapter_on_the_reference_path`, `tests/test_h3_adapters.py::TestWhatIsRefused::test_an_unrecognised_name_is_a_warning_not_a_refusal`, `tests/test_h3_adapters.py::TestTheCatalogDecidesFirst::test_a_proven_entry_refuses_the_other_partition`, `tests/test_h3_adapters.py::TestTheCatalogDecidesFirst::test_a_weaker_entry_only_warns`, `tests/test_server_loras.py::TestTheCatalogPlacesAnAdapter::test_a_trial_entry_warns_and_promoting_it_refuses` |
-| H3 audio hold, refine, tiled refine, guide layout and regional edit | Nine modules, imported one way: `dw/pipeline_processors/h3_rules.py` holds the names and rules, torch- and diffusers-free (`refine_problems`, the per-argument refine rules, `refine_tiles` and the explicit `refine_sigmas` schedule among them, asked before validation and before the call; `tile_plan`, the tile layout on the 32 px grid, shared by validate and the run; `not_h3`, the step rule; `snap_guide_length`, `guide_frame_problem`, `guide_chain_problems`, `guide_end_problem`; `CHAIN_CONTINUITY_MODES`; the chunk constants; and the regional-edit rules `mask_problems`, `media_problems` and `edits_region`, the argument rules shared by validate and `_with_mask`; `media_problems` is asked by validate too, on the files that resolve then, and refuses a `source_video` off the `17n + 5` grid, which the pipeline would otherwise snap up over a padded source), `dw/pipeline_processors/h3_mask_cells.py` holds the mask-to-rows arithmetic (`binarized`, `regenerate_cells`, `keep_rows`), which `h3_mask.py` re-exports and `h3_mask_steps.py` reads from there so the blocks never import their importer, `dw/pipeline_processors/h3_hold.py` puts the hold and refine blocks in (`insert_audio_hold`, `core_denoise_sequences`, `refines`; `HELD_LATENTS_OUTPUT` and `encoded_hold_audio`, the hold block's encode handed back to it), and `dw/pipeline_processors/h3_guides.py` the guide layout (`LAYOUT_ANCHORS`, `insert_guides`), which imports `h3_hold` and never the reverse, and `dw/pipeline_processors/h3_mask.py` the regional edit (`insert_mask_blocks`, `edits_regions`, and the mask-to-rows arithmetic `binarized`, `regenerate_cells` and `keep_rows`, on Pillow's L conversion, torch's adaptive max pool and scipy's dilation, each imported where it runs), which imports `h3_hold` and never the reverse; the blocks themselves, and the arithmetic they run, are `dw/pipeline_processors/h3_hold_steps.py` (`encode_audio_span`, `refine_sigmas`, and `explicit_sigmas`, a step's `refine_sigmas` list run verbatim in place of the shift grid) `dw/pipeline_processors/h3_guide_steps.py` (`splice_guide_rows`) and `dw/pipeline_processors/h3_mask_steps.py` (the source-latents and mask-blend blocks), which import diffusers when they load, so the three owners import them only inside `blocks()`, `guide_blocks()` and `h3_mask.blocks()` (#790), and `h3_guide_steps` imports `h3_hold_steps` and never the reverse (`_with_guides` in `dw/pipeline_processors/pipeline.py` adds its blocks for a `guides` call, and `_with_mask` checks a regional edit's arguments and media and builds the regenerate mask for a `source_video` call), `dw/guides.py`: the `guides` validation check and the chain `continuity: "guide"` check, `guide_chain_errors`, which asks the `guides` check's own step rule (`_takes_guides_problem`, built on `h3_rules.not_h3` and `REFERENCE_WORKFLOWS`) and reads the mode names from `h3_rules.CHAIN_CONTINUITY_MODES`, the names `chain.py`'s `CONTINUITY_MODES` is zipped against (both registered in `dw/validation.py`; a `previous_result:` guide is checked at run time), `dw/pipeline_processors/h3_tiles.py`: the tiled refine (the tile loop, the latent-space feather blend, the per-tile crop of one whole-canvas noise draw, the decode-free graph variant, and the one spatially tiled decode), which imports `h3_rules`, `h3_hold` and `media_types` (for `FrameList`; and `events`, `progress`) and is imported by `pipeline.py` and, lazily, `components.py` (which attaches the variant at load, behind `h3_hold.core_denoise_sequences`, and calls `insert_mask_blocks` at load), never by `h3_rules` or `h3_hold`, `dw/hold_audio.py`: `hold_audio_errors` (a video is held by its soundtrack), `refine_strength_errors`, `mask_errors` (Check `regional_edit`, which probes the source and mask through `ValidationContext.probe` the way `dw/guides.py` probes guide videos) | The one place dw modifies a diffusers modular pipeline's block graph: three blocks go in, before `set_timesteps`, `denoise` and `after_denoise`, in every H3 core-denoise sequence (the whole graph and the `workflow=`-pruned flat graph), and two more for a regional edit: the source block before `set_timesteps` and the blend block inside the `denoise` loop after `update`. All are no-ops without their arguments (`hold_audio` / `refine_strength`, `source_video`). The tiled refine adds a second graph built at load beside them (`h3_tiles.decode_free_blocks`: the decode and text-encoder blocks dropped, every kept block the graph's own instance, so hold and refine stay in) and swaps it into `pipeline._blocks` only for the duration of the one text-encode call and the tile calls (`swapped_blocks`, restored on failure). Every tile runs the hold block, but the track is encoded once per refine: the first tile is asked for the block's encode (`HELD_LATENTS_OUTPUT`) and every later tile's `hold_audio` carries it (`encoded_hold_audio`, an `EncodedAudioReference`), which the block writes in place of running the audio VAE. A `hold_audio` path is opened only after `locations.validate_media_path` against the step's workflow directory. `dw/introspection.py` builds the same graph without weights, hold included, so `get_pipeline_signature` lists a modular pipeline's block inputs. The copies of diffusers behaviour are pinned to the installed diffusers: `shifted_sigma_grid` (the spacing `refine_sigmas` rests on) to stock `set_timesteps`, `GUIDE_FRAMES_PER_CHUNK`/`GUIDE_LATENTS_PER_CHUNK` to the H3 video VAE's `clip_length` and latents per clip, and every name in `LAYOUT_ANCHORS` (`_fill_audio_positions` among them) to `before_denoise`. | `tests/test_h3_hold_audio.py::TestAnchors::test_inserted_next_to_the_anchors_in_every_shape`, `tests/test_h3_hold_audio.py::TestHoldAudioLocation::test_a_path_outside_the_roots_is_refused_at_the_call`, `tests/test_h3_hold_audio.py::TestRefusals::test_a_non_h3_step_is_refused`, `tests/test_h3_hold_audio.py::TestSignature::test_the_h3_signature_shows_hold_audio`, `tests/test_h3_refine.py::TestAnchors::test_inserted_before_denoise_in_every_shape`, `tests/test_h3_tiles.py::TestDecodeFree::test_variant_has_no_decode_and_shares_blocks`, `tests/test_h3_tiles.py::TestTileLoop::test_blocks_are_restored_when_a_tile_fails`, `tests/test_h3_tiles.py::TestTileLoop::test_the_track_is_encoded_by_one_tile_and_handed_to_the_rest`, `tests/test_h3_hold_audio.py::TestHoldBlock::test_a_carried_encode_is_written_without_encoding_again`, `tests/test_h3_tiles.py`, `tests/test_h3_rules.py` (`tile_plan` and the `refine_tiles` rules), `tests/test_h3_mask.py`, `tests/test_h3_refine.py::TestSigmaGridDrift::test_the_full_grid_is_stock_set_timesteps`, `tests/test_h3_refine.py::TestSigmaListOneOwner::test_the_runtime_message_is_the_owners`, `tests/test_h3_refine.py::TestSigmaListSchedule::test_the_block_runs_the_list_unshifted`, `tests/test_h3_guides.py::TestChunkDrift::test_frames_per_chunk_is_the_vae_clip_length`, `tests/test_h3_guides.py::TestChunkDrift::test_latents_per_chunk_is_the_vae_tokens_per_clip`, `tests/test_h3_guides.py::TestAnchors::test_installed_diffusers_has_every_anchor`, `tests/test_guide_chain_validation.py::TestRefused::test_the_step_rule_is_the_guides_rule`, `tests/test_guide_chain_validation.py::TestRefused::test_every_run_mode_is_known_to_validate`, `tests/test_h3_rules.py::test_h3_rules_imports_neither_torch_nor_diffusers`, `tests/test_h3_rules.py::test_the_block_owners_import_no_diffusers` |
+| H3 audio hold, refine, tiled refine, guide layout, guide noise, latent anchor and regional edit | Fourteen modules, imported one way: `dw/pipeline_processors/h3_rules.py` holds the names and rules, torch- and diffusers-free (`refine_problems`, the per-argument refine rules, `refine_tiles` and the explicit `refine_sigmas` schedule among them, asked before validation and before the call; `tile_plan`, the tile layout on the 32 px grid, shared by validate and the run; `not_h3`, the step rule; `snap_guide_length`, `guide_frame_problem`, `guide_chain_problems`, `guide_end_problem`; `CHAIN_CONTINUITY_MODES`; the chunk constants; and the regional-edit rules `mask_problems`, `media_problems` and `edits_region`, the argument rules shared by validate and `_with_mask`; `media_problems` is asked by validate too, on the files that resolve then, and refuses a `source_video` off the `17n + 5` grid, which the pipeline would otherwise snap up over a padded source), `dw/pipeline_processors/h3_mask_cells.py` holds the mask-to-rows arithmetic (`binarized`, `regenerate_cells`, `keep_rows`, on Pillow's L conversion, torch's adaptive max pool and scipy's dilation, each imported where it runs), which `h3_mask.py` re-exports and `h3_mask_steps.py` reads from there so the blocks never import their importer, `dw/pipeline_processors/h3_hold.py` puts the hold and refine blocks in (`insert_audio_hold`, `core_denoise_sequences`, `refines`; `HELD_LATENTS_OUTPUT` and `encoded_hold_audio`, the hold block's encode handed back to it), and `dw/pipeline_processors/h3_guides.py` the guide layout (`LAYOUT_ANCHORS`, `insert_guides`), which imports `h3_hold` and never the reverse, `dw/pipeline_processors/h3_anchor.py` the latent statistics anchor (`insert_latent_anchor`, `anchor_refusal`; its block in `h3_anchor_steps.py`, imported only by `anchor_block()`; the arithmetic, `channel_statistics` and `match_channel_statistics`, in `h3_anchor_stats.py`, torch only, which `h3_anchor` re-exports and `h3_anchor_steps` reads from there, as `h3_mask_cells` is read, so the block never imports its importer), which imports `h3_hold` and `h3_guides` and never the reverse, and runs right after `after_denoise` in t2va/fl2va only (a no-op without `latent_anchor`, decided by the block itself; it declares `idle_without = "latent_anchor"` so the step-progress narrator leaves it unnamed then; `chain.anchor: "segment_1"` hands segments 2+ segment 1's frames in `chain.py`'s `_segment_arguments`, and `run_chain` sets `pipeline.latent_anchor_source` to `segment 1` for those segments only; `Pipeline._with_latent_anchor` emits one `log` line per anchored call naming that source, the quoted reference, or the step's frames; `dw/guides.py`'s `anchor_errors` is the `latent_anchor` validation check, on `h3_rules.chain_anchor_problem` and the guides step rule `_takes_guides_problem`, asked for the anchor's name, so a step passing `references` (ref2va) is refused as `guides` is; `Pipeline._with_latent_anchor` refuses `references` at run time as `_with_guides` does), and `dw/pipeline_processors/h3_mask.py` the regional edit (`insert_mask_blocks`, `edits_regions`), which imports `h3_hold` and never the reverse; the blocks themselves, and the arithmetic they run, are `dw/pipeline_processors/h3_hold_steps.py` (`encode_audio_span`, `refine_sigmas`, and `explicit_sigmas`, a step's `refine_sigmas` list run verbatim in place of the shift grid) `dw/pipeline_processors/h3_guide_steps.py` (`splice_guide_rows`, and `noised_guide_pixels`, the `guide_noise` pixels: the layout block's `_encode_guides` encodes each video guide's noised pixels in place of the clean ones when `guide_noise` is set, on `h3_rules.guide_noise_taper`, one encode per guide either way (PrefixStream's paired encode under one seed is that encode, so the clean one is not run); `guide_noise_errors`/`guide_noise_warnings` in `dw/guides.py` validate it on `h3_rules.guide_noise_problem` and `_takes_guides_problem`, and `Pipeline._with_guide_noise` drops a 0 and refuses at run time) and `dw/pipeline_processors/h3_mask_steps.py` (the source-latents and mask-blend blocks), which import diffusers when they load, so the three owners import them only inside `blocks()`, `guide_blocks()` and `h3_mask.blocks()` (#790), and `h3_guide_steps` imports `h3_hold_steps` and never the reverse (`_with_guides` in `dw/pipeline_processors/pipeline_h3.py` - the `H3Arguments` mixin that gives `Pipeline` its per-call H3 argument wrappers `_with_held_audio`, `_with_guides`, `_with_guide_noise`, `_with_latent_anchor`, `_with_mask` and `_check_refine`, still named `Pipeline._with_*` - adds its blocks for a `guides` call, and `_with_mask` checks a regional edit's arguments and media and builds the regenerate mask for a `source_video` call), `dw/guides.py`: the `guides` validation check and the chain `continuity: "guide"` check, `guide_chain_errors`, which asks the `guides` check's own step rule (`_takes_guides_problem`, built on `h3_rules.not_h3` and `REFERENCE_WORKFLOWS`) and reads the mode names from `h3_rules.CHAIN_CONTINUITY_MODES`, the names `chain.py`'s `CONTINUITY_MODES` is zipped against (both registered in `dw/validation.py`; a `previous_result:` guide is checked at run time), `dw/pipeline_processors/h3_tiles.py`: the tiled refine (the tile loop, the latent-space feather blend, the per-tile crop of one whole-canvas noise draw, the decode-free graph variant, and the one spatially tiled decode), which imports `h3_rules`, `h3_hold` and `media_types` (for `FrameList`; and `events`, `progress`) and is imported by `pipeline.py` and, lazily, `components.py` (which attaches the variant at load, behind `h3_hold.core_denoise_sequences`, and calls `insert_mask_blocks` at load), never by `h3_rules` or `h3_hold`, `dw/hold_audio.py`: `hold_audio_errors` (a video is held by its soundtrack), `refine_strength_errors`, `mask_errors` (Check `regional_edit`, which probes the source and mask through `ValidationContext.probe` the way `dw/guides.py` probes guide videos) | The one place dw modifies a diffusers modular pipeline's block graph: three blocks go in, before `set_timesteps`, `denoise` and `after_denoise`, in every H3 core-denoise sequence (the whole graph and the `workflow=`-pruned flat graph), and two more for a regional edit: the source block before `set_timesteps` and the blend block inside the `denoise` loop after `update`. All are no-ops without their arguments (`hold_audio` / `refine_strength`, `source_video`). The tiled refine adds a second graph built at load beside them (`h3_tiles.decode_free_blocks`: the decode and text-encoder blocks dropped, every kept block the graph's own instance, so hold and refine stay in) and swaps it into `pipeline._blocks` only for the duration of the one text-encode call and the tile calls (`swapped_blocks`, restored on failure). Every tile runs the hold block, but the track is encoded once per refine: the first tile is asked for the block's encode (`HELD_LATENTS_OUTPUT`) and every later tile's `hold_audio` carries it (`encoded_hold_audio`, an `EncodedAudioReference`), which the block writes in place of running the audio VAE. A `hold_audio` path is opened only after `locations.validate_media_path` against the step's workflow directory. `dw/introspection.py` builds the same graph without weights, hold included, so `get_pipeline_signature` lists a modular pipeline's block inputs. The copies of diffusers behaviour are pinned to the installed diffusers: `shifted_sigma_grid` (the spacing `refine_sigmas` rests on) to stock `set_timesteps`, `GUIDE_FRAMES_PER_CHUNK`/`GUIDE_LATENTS_PER_CHUNK` to the H3 video VAE's `clip_length` and latents per clip, and every name in `LAYOUT_ANCHORS` (`_fill_audio_positions` among them) to `before_denoise`. | `tests/test_h3_hold_audio.py::TestAnchors::test_inserted_next_to_the_anchors_in_every_shape`, `tests/test_h3_hold_audio.py::TestHoldAudioLocation::test_a_path_outside_the_roots_is_refused_at_the_call`, `tests/test_h3_hold_audio.py::TestRefusals::test_a_non_h3_step_is_refused`, `tests/test_h3_hold_audio.py::TestSignature::test_the_h3_signature_shows_hold_audio`, `tests/test_h3_refine.py::TestAnchors::test_inserted_before_denoise_in_every_shape`, `tests/test_h3_tiles.py::TestDecodeFree::test_variant_has_no_decode_and_shares_blocks`, `tests/test_h3_tiles.py::TestTileLoop::test_blocks_are_restored_when_a_tile_fails`, `tests/test_h3_tiles.py::TestTileLoop::test_the_track_is_encoded_by_one_tile_and_handed_to_the_rest`, `tests/test_h3_hold_audio.py::TestHoldBlock::test_a_carried_encode_is_written_without_encoding_again`, `tests/test_h3_tiles.py`, `tests/test_h3_rules.py` (`tile_plan` and the `refine_tiles` rules), `tests/test_h3_mask.py`, `tests/test_h3_refine.py::TestSigmaGridDrift::test_the_full_grid_is_stock_set_timesteps`, `tests/test_h3_refine.py::TestSigmaListOneOwner::test_the_runtime_message_is_the_owners`, `tests/test_h3_refine.py::TestSigmaListSchedule::test_the_block_runs_the_list_unshifted`, `tests/test_h3_guides.py::TestChunkDrift::test_frames_per_chunk_is_the_vae_clip_length`, `tests/test_h3_guides.py::TestChunkDrift::test_latents_per_chunk_is_the_vae_tokens_per_clip`, `tests/test_h3_guides.py::TestAnchors::test_installed_diffusers_has_every_anchor`, `tests/test_guide_chain_validation.py::TestRefused::test_the_step_rule_is_the_guides_rule`, `tests/test_guide_chain_validation.py::TestRefused::test_every_run_mode_is_known_to_validate`, `tests/test_h3_rules.py::test_h3_rules_imports_neither_torch_nor_diffusers`, `tests/test_h3_rules.py::test_the_block_owners_import_no_diffusers`, `tests/test_h3_anchor.py::TestInsert::test_follows_after_denoise`, `tests/test_h3_anchor.py::TestStep::test_without_an_anchor_the_latents_are_untouched`, `tests/test_h3_anchor.py::TestWithLatentAnchor::test_no_anchor_logs_nothing`, `tests/test_h3_anchor.py::TestWithLatentAnchor::test_a_chain_segment_logs_its_label_and_segment_1`, `tests/test_modular_progress.py::test_the_anchor_step_is_idle_without_latent_anchor`, `tests/test_h3_anchor.py::TestAnchorErrors::test_an_unknown_anchor_names_the_values`, `tests/test_h3_guide_noise.py` |
| IC-LoRA reference scale | `workflows/templates/ltx2/` | The five conditioning templates run at `reference_downscale_factor: 1`, and the two upscalers at 2. | `tests/test_ltx2_ic_loras.py::TestEachTemplateMatchesItsCard::test_the_reference_is_encoded_at_the_output_resolution`, `tests/test_ltx2_ic_loras.py::TestTheGenerativeUpscaleMatchesTheSameCard::test_it_loads_the_upscaler_at_factor_two` |
| IC-LoRA numbers | `workflows/templates/ltx2/` | Every number in the IC-LoRA templates comes from the vendor's model card. | `tests/test_ltx2_ic_loras.py::TestEachTemplateMatchesItsCard::test_the_strength_is_the_cards_default`, `tests/test_ltx2_ic_loras.py::TestEachTemplateMatchesItsCard::test_the_defaults_are_the_trained_bucket` |
| IC-LoRA prompt genre | `prompts/ltx2/` | A stored prompt tagged `ic-lora` is checked against its trained caption form, not against the T2V paragraph rule. | `tests/test_ltx_prompt_library.py::test_an_ic_lora_prompt_is_in_its_trained_form` |
@@ -139,7 +139,7 @@ one to open.
| --- | --- | --- | --- |
| Opening media | `dw/media.py`, `dw/media_readers.py` (the frame readers, re-exported by `dw/media.py`, which imports from it and never the reverse) | These are the only modules that call `av.open`. They import no torch and no step types. | `tests/test_media_layering.py::test_av_open_appears_only_in_media`, `tests/test_media_layering.py::test_media_imports_no_step_types` (no torch: —) |
| Frame moments and crops | `dw/media_frames.py`: `moment_index`, `parse_crop` | One `at` grammar ("frame:N", or seconds at the clip's fps; negative counts back from the end for `get_output_frames` and is refused by `compare_outputs`) and one `[x, y, width, height]` crop parse serve `get_output_frames` and `compare_outputs`. Only the bounds policy is the caller's: `get_output_frames` refuses a moment past the end and clamps a crop (`_moment_to_index`, `resolve_crop_box`); `compare_outputs` blanks a past-end cell and refuses a crop that doesn't fit an output. | `tests/test_media_compare.py::test_resolve_moments_refuses_what_get_output_frames_refuses`, `tests/test_media_compare.py::test_crop_box_refuses_a_malformed_box_as_resolve_crop_box_does` |
-| Output comparison | `dw/media_compare.py`: `compare`, `psnr`, `ssim`, `mean_abs_diff`, `check_request`, `crop_box` | Metrics always compare the same pixels: one shared crop is the measured region, and per-output crops are display only, leaving metrics on the full frame. An output whose size differs from the baseline's, or a moment past its end, gets null metrics with a reason; nothing is resized to force a number. SSIM is on Rec. 601 luma (SSIM's convention, not `image_ops.luma`). It runs in the server process and decodes through `dw/media.py` and `open_bounded`. | `tests/test_compare_route.py::test_a_shared_crop_is_the_region_measured`, `tests/test_compare_route.py::test_per_output_crops_are_display_only`, `tests/test_compare_route.py::test_a_size_mismatch_gets_no_numbers_and_says_why`, `tests/test_compare_route.py::test_a_moment_past_the_shorter_clip_is_a_blank_cell` |
+| Output comparison | `dw/media_compare.py` (texture bands from `dw/texture_bands.py`): `compare`, `psnr`, `ssim`, `mean_abs_diff`, `check_request`, `crop_box` | Metrics always compare the same pixels: one shared crop is the measured region, and per-output crops are display only, leaving metrics on the full frame. An output whose size differs from the baseline's, or a moment past its end, gets null metrics with a reason; nothing is resized to force a number. SSIM is on Rec. 601 luma (SSIM's convention, not `image_ops.luma`). It runs in the server process and decodes through `dw/media.py` and `open_bounded`. | `tests/test_compare_route.py::test_a_shared_crop_is_the_region_measured`, `tests/test_compare_route.py::test_per_output_crops_are_display_only`, `tests/test_compare_route.py::test_a_size_mismatch_gets_no_numbers_and_says_why`, `tests/test_compare_route.py::test_a_moment_past_the_shorter_clip_is_a_blank_cell` |
| Signal processing | `dw/dsp.py` | It measures and transforms waveforms, decides nothing, and imports nothing from `dw`. | `tests/test_media_layering.py::test_dsp_imports_nothing_from_dw` |
| Assessment probes | `dw/tasks/assess.py`, `dw/assessment_rules.py` | Exactly three registered commands are assessment probes. Each measures a finished file, and every rule names a real probe field. | `tests/test_assessment_rules.py::TestRuleProbesAreRealCommands::test_assessment_is_exactly_the_three_probes`, `tests/test_assessment_rules.py::TestRuleProbesAreRealCommands::test_every_rule_names_a_registered_json_command` |
| Script check | `dw/tasks/script_check.py`: `check`; `dw/tasks/script_findings.py`: the rule constants and the finding builders `check` calls (imported one way, re-exported by `script_check`); `dw/tasks/script_take.py`: `guard_words`, `word_level`, `shot_spans`; `dw/script_lines.py`: `parse_lines`, `parse_shots`, `shot_names_error` | `check_script` aligns a take's heard words to its expected lines and guards out words heard over silence and repetition loops (Whisper's hallucination over music). It builds its findings with `assessment_rules.finding()` and reads the guard's floor and window (`DEAD_AIR_FLOOR_DBFS`, `DEAD_AIR_WINDOW`) rather than copying them; the loudest-window measure is `dsp.loudest_window_dbfs` and the looped-bed self-correlation `dsp.repeat_correlation` (`script_findings.is_looped_bed` only decides on it) (`script_take.word_level` wraps it; the guard, the transcription levelling, the floor-gated `audible_runs` and the shot placement (with `clamp_to_shot`, a line's timing kept inside its shot) are `dw/tasks/script_take.py`, which `script_check` reads one way - nothing there reads the rule tables), which `transcribe_audio`'s `suspect: "silent"` mark reads with the same floor and window; the repetition-loop detector (`script_lines.looped_texts`, `REPEAT_RUN_MIN`, `REPEAT_MAX_PERIOD`) lives in `dw/script_lines.py` and backs both `check_script`'s `repetition` discard and `transcribe_audio`'s `suspect: "repetition"`, and `script_lines.suspect_runs`/`SUSPECT_LEFT_OUT` are what `plan_cuts` groups and leaves out suspect chunks by - so neither task imports the other. It is not an assessment probe and adds nothing to `RULES`. Its literal `lines` is refused at validation through `task_problems` (`script_lines_errors`), which calls the same `parse_lines` the task runs, and a literal `shots` through the same `parse_shots` and `shot_names_error`; the parsers live in `dw/script_lines.py`, not the task, so that `task_problems` reaches them without importing `script_check` (the `import_cycles` ratchet in `scripts/arch_metrics.py`). A line's `shot` may be the short `@name` of the one recorded shot that carries it (`script_lines.matching_shot_names`, `with_full_shot_names`, shared by the task and the literal-`shots` check); which member key a recorded name carries comes from `references.member_keys`, beside `MEMBER_SEPARATOR` and `for_each.member_name`, never a re-derived pattern: a key ends at the first character outside `references.VARIABLE_NAME_CHARACTERS`, the set `security.validate_variable_name` (which checks a for_each entry's name) builds its pattern from. With `audio` an `output:`/`asset:`/path whose run manifest records shots, `dw/shot_span_preflight.py: script_shot_name_errors` (the `script_shot_names` error check) runs `shot_names_error` at validation, reading the map through `runs.shots_beside`. Its shot map comes from `assess.resolve_shots`, the one the probes use, never a copy; it places each shot on the soundtrack with `assess.sample_span`, decides repeated shot names with `shots.duplicate_shot_names`, and asks `locations.is_http_url` whether the take is a URL. | `tests/test_script_check.py::TestRegistration::test_registered_as_json_command_without_assessment_flag`, `tests/test_script_check.py::TestEmptyLines::test_words_only_over_silence_are_discarded_not_found`, `tests/test_script_check.py::TestLinesErrors::test_literal_with_a_colon_is_not_a_reference`, `tests/test_script_check.py::TestShotValidation::test_a_short_member_name_matches_the_one_shot_carrying_it`, `tests/test_script_check.py::TestShotValidation::test_a_valid_variable_name_comes_back_whole_as_a_member_key`, `tests/test_shot_span_preflight.py::TestScriptShotNames::test_a_wrong_shot_name_is_an_error_at_validate_time` |
@@ -193,7 +193,7 @@ one to open.
| Inline images | `dw/server/inline_media.py` | Fitting an image to a longest side and halving it under a base64 budget happens on the server, for `/image` and `/frames` alike; `dw_mcp` forwards the size and budget and imports no Pillow. | `tests/test_server_gallery_image.py`, `tests/test_mcp_server.py::TestStartupWeight::test_the_server_starts_without_importing_the_engine` |
| Workflow patch | `dw/library.py`: `merge_patch`; `dw/server/routes/library.py`: `patch_workflow` | A merge patch is applied on the server under the lock `PUT` takes; no client reads, merges and writes back. | `tests/test_server_library_path.py::TestPatchWorkflow::test_patch_merges_onto_the_stored_definition` |
| Deleting a job's run | `dw/server/jobs.py`: `JobManager.run_location`; `dw/server/outputs.py`: `delete_run_directory` | The server resolves a job's run directory against the root the job ran in, and refuses a job still queued or running. | `tests/test_server_jobs.py::TestDeleteAJobsRun::test_a_running_job_is_409` |
-| Level findings | `dw/server/assess.py`: `level_findings` | Gallery metadata reports level problems from `dw/audio_qc.py`'s thresholds; no client restates a threshold. | `tests/test_server_assess.py::TestMetadataLevelFindings::test_level_findings_read_audio_qc_thresholds` |
+| Level findings | `dw/server/assess.py`: `level_findings` | Gallery metadata reports level problems from `dw/audio_qc.py`'s thresholds; no client restates a threshold. The silent-by-design exemption to the near-silent rule is `dw/audio_qc.py`: `all_shots_silent_by_design`, used by this row and by the write-time warning. | `tests/test_server_assess.py::TestMetadataLevelFindings::test_level_findings_read_audio_qc_thresholds` |
| API errors | `dw_mcp/client.py` | An API failure becomes a message a person can act on, here and nowhere else. | `tests/test_mcp_client.py::test_a_400_surfaces_the_servers_detail_verbatim` |
## UI
@@ -212,5 +212,5 @@ one to open.
| UI polling | `ui/src/lib/poll.ts`: `poll`, `sleep` | A page that refreshes on a timer starts it through `poll` and stops it with the function `poll` returns. | `ui/src/lib/poll.test.ts` |
| Workspace state in the UI | `ui/src/lib/workspaceState.svelte.ts`: `workspace` | The current workspace is one `$state`; `api.ts` reads it from here, so the request layer imports no page state. | `import_cycles` in `ui/scripts/arch-metrics.mjs` |
| The UI's response contract | `dw/server/api_models.py` (the models; it re-exports `dw/server/api_models_base.py`: `ApiModel`, `send_rejected_responses`, and the library routes' models in `dw/server/api_models_library.py`); `scripts/dump_openapi.py` | Every JSON route `api.ts` calls declares a response model, and `types.ts` re-exports the types generated from it (`ui/src/lib/generated/`). The payload does not change: an absent key stays absent (`response_model_exclude_unset`), a sometimes-sent key is `sometimes()`. Runtime is lenient - an undeclared key passes and a rejected response is logged and sent as built; tests, the e2e server and the dump run strict (`DW_STRICT_RESPONSES=1`). The document is generated under the FastAPI and Pydantic `constraints-openapi.txt` pins: CI installs them, and the dump refuses to write under others. | `tests/test_api_contract.py`, `tests/test_ui_twins.py::test_every_json_route_the_ui_calls_declares_its_response`, CI's "Response contract" step |
-| Where `--live` appears | `ui/src/app.css` (its header) | The state colour marks machine state only, in the files `ui/scripts/design-rules.test.ts` lists. | `ui/scripts/design-rules.test.ts` |
+| Where `--live` appears | `ui/src/app.css` (its header) | `--live` marks machine state only and `--select` the user's place only (focus ring, sidebar), each in the files `ui/scripts/design-rules.test.ts` lists. | `ui/scripts/design-rules.test.ts` |
| UI architecture ratchet | `ui/scripts/arch-metrics.mjs` | No metric rises above `docs/stabilization/ui/baseline.json`. | the `ui` job in `.github/workflows/ci.yml`; `npm run preflight` |
diff --git a/docs/ATTRIBUTIONS.md b/docs/ATTRIBUTIONS.md
index 02d07b21b..d00ef8129 100644
--- a/docs/ATTRIBUTIONS.md
+++ b/docs/ATTRIBUTIONS.md
@@ -3,14 +3,14 @@
diffusers-workflow is Apache-2.0. This page credits the work it builds on that
a dependency list does not show: code that was copied in, and other projects
whose ideas were reimplemented here. Python dependencies are in
-[requirements.txt](../requirements.txt) and [pyproject.toml](../pyproject.toml).
+[pyproject.toml](../pyproject.toml).
Model and LoRA weights keep their own licences, and each catalog entry
([workflows/models/](../workflows/models/), [loras/](../loras/)) names the
repository it downloads from.
## Code included in this repository
-Each file keeps its upstream licence notice in its header.
+Each copied or ported file keeps its upstream licence notice in its header.
| File | Source | Licence |
|---|---|---|
@@ -109,3 +109,27 @@ rendering the odd shots of a long take first and bridging the gaps
([#1094](https://github.com/dkackman/diffusers-workflow/issues/1094)), and
finding phrase starts from the centre of the stereo mix
([#1095](https://github.com/dkackman/diffusers-workflow/issues/1095)).
+
+### [knoic/ComfyUI-MiniMaxH3-PrefixStream](https://github.com/knoic/ComfyUI-MiniMaxH3-PrefixStream)
+
+A ComfyUI node pack for MiniMax H3 prefix streaming, under the MIT licence. It
+inspired the texture-band measurement on `compare_outputs`
+(`texture_bands`, [dw/texture_bands.py](../dw/texture_bands.py)): RMS energy by
+wavelength band, to see whether a change in a chain adds or removes texture.
+It also inspired `guide_noise`
+([dw/pipeline_processors/h3_guide_steps.py](../dw/pipeline_processors/h3_guide_steps.py);
+[#1166](https://github.com/dkackman/diffusers-workflow/issues/1166)), from its prefix
+noise at `775be90`: a cosine-tapered noise on each guide, folded in as the difference of
+a paired VAE encode, so the guide's hold loosens toward its start.
+
+### [1113800942/comfy-h3md-director](https://github.com/1113800942/comfy-h3md-director)
+
+H3MD Director, a ComfyUI node pack for directing long MiniMax H3 videos, under
+AGPL-3.0-or-later, so only the concept was taken. It inspired the latent
+statistics anchor: rescaling each later segment's video latent to the first
+segment's per-channel mean and standard deviation before decode
+(`latent_anchor`, `chain.anchor: "segment_1"`,
+[dw/pipeline_processors/h3_anchor.py](../dw/pipeline_processors/h3_anchor.py);
+[#1166](https://github.com/dkackman/diffusers-workflow/issues/1166)). The
+diagnosis it answers, colour and contrast drifting from segment to segment of
+an H3 chain, was described by Lumifel and by roycho87 (NerdNotes).
diff --git a/docs/DEPENDENCIES.md b/docs/DEPENDENCIES.md
index ca9ab4ccd..48d4d39be 100644
--- a/docs/DEPENDENCIES.md
+++ b/docs/DEPENDENCIES.md
@@ -16,7 +16,7 @@ source ./activate
.\venv\scripts\activate
```
-The install scripts handle everything: Python version detection (3.12-3.14), virtual environment creation, PyTorch, diffusers, and all dependencies.
+The install scripts need Python 3.12-3.14. Each deletes and recreates `venv/`, installs PyTorch for the detected GPU, then dw with the `server` and `dev` extras (`pip install -e ".[server,dev]"`), diffusers from git HEAD, the platform extras below, and the web UI's npm packages when `npm` is on the `PATH`.
### Docker (Linux, NVIDIA)
@@ -26,55 +26,53 @@ workspace and caches with a native install on the same host:
### Manual Installation
-If you prefer manual control:
-
```bash
python3 -m venv venv
source venv/bin/activate
-pip install torch torchvision torchaudio # add --index-url https://download.pytorch.org/whl/cu130 for CUDA on Linux
-pip install -r requirements.txt
-pip install git+https://github.com/huggingface/diffusers
+pip install torch torchvision torchaudio # add --index-url https://download.pytorch.org/whl/cu130 for CUDA 13 on Linux
+pip install -r requirements.txt # -e .[server]; requirements-test.txt is -e .[dev]
+pip install --force-reinstall --no-deps git+https://github.com/huggingface/diffusers
+pip install diffusers # adds any dependency the git commit needs
```
+dw does not import torchaudio, but diffusers' MiniMax H3 blocks need it to resample an audio reference, so install it from the same index as torch.
+
## Where the Dependency List Lives
-`pyproject.toml` is the single source: its `dependencies` list (plus the
-`server` and `dev` extras) is what the published wheel declares, what
-`requirements.txt` and `requirements-test.txt` resolve (they are one-line
-pointers, `-e .[server]` and `-e .[dev]`), and what the install scripts
-and CI install. To add or change a dependency, edit pyproject.toml -
-every install path picks it up.
+`pyproject.toml` is the single source. Its `dependencies` list and its `server`, `mcp` and `dev` extras are what the wheel declares, what `requirements.txt` and `requirements-test.txt` point at, and what the install scripts and CI install. To add or change a dependency, edit `pyproject.toml`. Refresh the version floors from a working venv with `python scripts/refresh_dep_floors.py`.
-Things pyproject can't express stay in the scripts: the PyTorch CUDA
-index, diffusers from git HEAD, and the macOS/Windows-specific extras
-below. bitsandbytes carries a `sys_platform == 'linux'` marker in
-pyproject; install.ps1 installs it explicitly on Windows.
+The scripts handle what `pyproject.toml` cannot express: the PyTorch index, diffusers from git HEAD, and the platform extras below.
-## Platform-Specific Dependencies
+`constraints-openapi.txt` pins the FastAPI and Pydantic that `ui/src/lib/generated/openapi.json` is generated under. CI installs them with `pip -c`, and `scripts/dump_openapi.py` refuses to write under any other versions.
-**All platforms (core ML):** peft, transformers, accelerate, safetensors, controlnet_aux, sentencepiece, torchsde, torchao, optimum-quanto, gguf, kornia, ftfy, sdnq, spandrel, facexlib (spandrel + facexlib back the `upscale` and `restore_faces` tasks). On first use, `restore_faces` has facexlib download its face detection and parsing weights (about 110 MB) from facexlib's own GitHub releases into the facexlib package directory. It is the one model download that bypasses the Hugging Face cache and dw's download manager; a validate's `plan.downloads_required` lists the two weights as URL entries while they are missing from that directory.
+## Notable Dependencies
-**All platforms (sparse attention):** comfy-kitchen (Sol-Attn sparse attention for MiniMax H3's generated-video rows, #931). Its compiled CUDA/HIP kernels ship for Linux and Windows; elsewhere it installs an eager-only build.
+The full list is in `pyproject.toml`. These are the ones with behaviour worth knowing:
-**All platforms (utilities):** fastapi, uvicorn (the `dw.serve` HTTP server and web UI), av, aiohttp, matplotlib, opencv-python-headless, concurrent-log-handler, qrcode, protobuf, imageio, imageio-ffmpeg, beautifulsoup4, soundfile, jsonschema, black, python-dotenv
+- **spandrel, facexlib** back the `upscale` and `restore_faces` tasks. On first use, `restore_faces` has facexlib download its face detection and parsing weights (about 110 MB) from facexlib's GitHub releases into the facexlib package directory. It is the one model download that bypasses the Hugging Face cache and dw's download manager; while the weights are missing, a validate's `plan.downloads_required` lists them as URL entries.
+- **comfy-kitchen** (`<0.3`) is Sol-Attn sparse attention for MiniMax H3's generated-video rows (#931). Its compiled CUDA/HIP kernels ship for Linux and Windows; elsewhere it installs an eager-only build.
+- **kernels** fetches Hub-hosted compiled kernels at run time, for example NATTEN's `na3d` for LTX-2.5's diffusion decoder.
+- **speechbrain, demucs** back `generate_speech`'s `speaker_embedding` and the `attribute_voices` task. Both are imported lazily, and their weights download only when a call uses them.
+- **piexif** embeds generation metadata as EXIF `UserComment` when a step's `result.embed_metadata` is `true` and the content type is JPEG or WebP. PNG embedding uses Pillow's `PngInfo`.
+- **fastapi, uvicorn** (the `server` extra) run `dw.serve` and the web UI. **mcp** (the `mcp` extra, also in `dev`) runs `dw_mcp`.
-**Linux (CUDA):** bitsandbytes
+## Platform-Specific Dependencies
-**Windows (CUDA):** bitsandbytes, kernels
+**Linux:** bitsandbytes, through its `sys_platform == 'linux'` marker in `pyproject.toml`.
-**macOS (MPS):** fp4-fp8-for-torch-mps (FP8/FP4 dtypes for Metal; autoloads on every torch import, overrides mm/linear/copy on the MPS dispatch key and sets PYTORCH_ENABLE_MPS_FALLBACK=1), fluidtop (a GPU/power TUI; needs sudo)
+**Windows:** `install.ps1` installs bitsandbytes and kernels explicitly.
+
+**macOS (MPS):** `install.sh` adds fp4-fp8-for-torch-mps (FP8/FP4 dtypes for Metal; autoloads on every torch import, overrides mm/linear/copy on the MPS dispatch key and sets `PYTORCH_ENABLE_MPS_FALLBACK=1`) and fluidtop (a GPU/power TUI; needs sudo). bitsandbytes and flash_attn are not available on macOS.
## Optional
-**flash_attn** — Improved attention performance on CUDA. Requires the [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit):
+**flash_attn** speeds up attention on CUDA, and some LLM workflows (for example Phi mini-instruct) require it. It needs the [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit):
```bash
pip install flash_attn
```
-**piexif** (installed by default) — Embeds generation metadata as EXIF `UserComment` when a step's `result.embed_metadata` is `true` and the content type is JPEG/WebP (PNG embedding uses Pillow's `PngInfo` and needs nothing extra). Without it, saving falls back to a logged warning and no embedded metadata.
-
## Specifying Python Version
```bash
@@ -82,18 +80,20 @@ export INSTALL_PYTHON_VERSION=3.13
bash ./install.sh
```
+Without it, `install.sh` takes the newest of `python3.14`, `python3.13` and `python3.12` on the `PATH`. `install.ps1` always runs plain `python`; `INSTALL_PYTHON_VERSION` only has to match that interpreter's version.
+
## Troubleshooting
-**Hugging Face authentication (401/403 downloading a model):** Most workflows under `workflows/` (Flux, LTX-2, MiniMax...) point at **gated** models on the Hub — repos that require the owner to approve your account before you can download them. A run against one of these fails with an actionable error naming the repo and `hf auth login` (mapped from the Hub's 401/403 in `load_component()`, `dw/pipeline_processors/components.py`) — request access on the model's page (e.g. [black-forest-labs/FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev)) and then log in once, locally:
+**Hugging Face authentication (401/403 downloading a model):** Most workflows under `workflows/` (Flux, LTX-2, MiniMax...) use **gated** models, which need the repo owner to approve your account. A run against one fails with an error naming the repo and `hf auth login` (mapped from the Hub's 401/403 in `load_component()`, `dw/pipeline_processors/components.py`). Request access on the model's page (e.g. [black-forest-labs/FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev)), then log in once:
```bash
hf auth login
```
-`workflows/templates/text-to-image.json` uses an ungated model and needs no login - start there if you just want to confirm the install works.
+`workflows/templates/text-to-image.json` uses an ungated model and needs no login, so use it to confirm an install works.
-**Package conflicts:** Re-run the install script — it recreates the venv from scratch.
+**Package conflicts:** Re-run the install script; it recreates the venv from scratch.
-**CUDA not detected:** `install.sh` probes for a working `nvidia-smi` on Linux and installs the CUDA build of torch (cu130) only when it finds one; otherwise (or on a manual install with plain `pip install torch`) you get PyPI's CPU-only Linux wheel. Verify with `python -c "import torch; print(torch.cuda.is_available())"`.
+**CUDA not detected:** The install scripts run `nvidia-smi` and read the CUDA version the driver supports. A CUDA 13 driver (>= 580) gets torch from the cu130 index, an older driver gets PyPI's wheel with its bundled CUDA 12 runtime, and no working `nvidia-smi` gets the CPU-only build. A manual `pip install torch` on Linux gets PyPI's wheel. Verify with `python -c "import torch; print(torch.cuda.is_available())"`.
**MPS not detected:** Requires Apple Silicon. Verify with `python -c "import torch; print(torch.backends.mps.is_available())"`.
diff --git a/docs/DOCKER.md b/docs/DOCKER.md
index 6f1327948..5e068ad0a 100644
--- a/docs/DOCKER.md
+++ b/docs/DOCKER.md
@@ -31,12 +31,11 @@ docker compose -f docker/compose.yaml exec dw python /opt/dw/entrypoint.py login
To use the published image instead of building, uncomment `DW_IMAGE` in
`docker/.env` and run `docker compose -f docker/compose.yaml pull` before
-`up -d` (without `--build`). The first build takes a while, because torch
-alone is several GB. The first
-start then installs diffusers from GitHub main into its volume (see below),
-which takes a few more minutes before the server answers. Then `login-url`
-prints `http://localhost:8765/?token=...`. Open that link and the UI saves
-the token.
+`up -d` (without `--build`). The first build takes a while (torch alone is
+several GB), and the first start installs diffusers from GitHub main into
+its volume (see below), a few more minutes before the server answers.
+`login-url` prints `http://localhost:/?token=...`; opening it
+saves the token in the UI.
## Where the data lives: shared with a native install
@@ -75,8 +74,9 @@ bash docker/switch.sh native # stop the container, start the unit
bash docker/switch.sh status
```
-The script assumes the user unit from `scripts/dw-serve.service`. For the
-system unit in `contrib/systemd/`, set `DW_SYSTEMCTL="sudo systemctl"`.
+The script assumes the user unit from `scripts/dw-serve.service`
+(`DW_SYSTEMD_UNIT` names another unit). For the system unit in
+`contrib/systemd/`, set `DW_SYSTEMCTL="sudo systemctl"`.
To keep the token your MCP clients already use, set `DW_API_TOKEN` in
`docker/.env` to the one the unit passes.
@@ -94,16 +94,18 @@ only diffusers and any dependency a new commit needs that the image lacks.
- **The first start** installs main HEAD (or `DW_DIFFUSERS_REF`; see below).
- **A restart changes nothing.** diffusers moves only when you ask it to.
-- **Update** from the web UI's Models page, as on a native install, or:
+- **Update** to HEAD from the web UI's Models page, as on a native install,
+ or to HEAD, a commit or the release floor with
`docker compose -f docker/compose.yaml exec dw python /opt/dw/entrypoint.py update-diffusers [--commit SHA | --revert]`,
- then restart the container.
+ then restart the container (a running job keeps the old import).
- **A rebuilt image** (a new dw) rebuilds the overlay and reinstalls the
diffusers commit you had, not HEAD.
- **If the overlay doesn't import** (a broken HEAD, or no network on the
first start), the server starts on the copy of diffusers built into the
- image, and says so in the log. Pin a working commit from the Models page,
- then restart. An update made in this state goes into the container
- rather than the volume, so it's lost when the container is re-created.
+ image and says so in the log. Pin a working commit with `update-diffusers
+ --commit SHA` (or `--revert`), then restart. An update from the Models
+ page in this state goes into the container rather than the volume, so it
+ is lost when the container is re-created.
`DW_DIFFUSERS_REF` in `docker/.env`: `head` (the default), a commit hash
(the overlay is held to it), or `image` (no overlay; use the copy built
@@ -112,19 +114,18 @@ into the image).
## The API token
Inside the container the server listens on every interface, and it can't
-tell how the port is published. So it always gets a token: `DW_API_TOKEN`
+tell how the port is published, so it always has a token: `DW_API_TOKEN`
if you set one, otherwise one generated on first start and saved to
`DIFFUSERS_HELPER_ROOT/api-token`. The token is never written to the log,
-so `docker logs` output is safe to paste into an issue: `login-url` prints
-the link when you ask for it.
-`DW_INSECURE_NO_TOKEN=1` turns this off; only do that with the port
-published on localhost (`DW_BIND=127.0.0.1`, the default).
+so `docker logs` output is safe to paste into an issue; `login-url` prints
+it on request. `DW_INSECURE_NO_TOKEN=1` turns this off; do that only with
+the port published on localhost (`DW_BIND=127.0.0.1`, the default).
To reach the server from your LAN, set `DW_BIND=0.0.0.0`, then run `up -d`
-again to re-create the container. Clients connect the way
-[REMOTE.md](REMOTE.md) describes. A box that switches from a systemd unit
-bound to `0.0.0.0` needs this, or after the switch the server answers only
-on the box itself.
+again to re-create the container; clients connect as
+[REMOTE.md](REMOTE.md) describes. A box switching from a systemd unit bound
+to `0.0.0.0` needs this, or the container answers only on the box itself.
+`DW_HOST_PORT` (default 8765) is the host port it is published on.
## Options
@@ -132,6 +133,8 @@ on the box itself.
`--mcp --devices cuda:0,cuda:1` for one worker per card. The entrypoint adds
`--host 0.0.0.0`, `--port 8765` and the examples catalog unless you give
them. `HF_TOKEN` is passed through for gated models.
+`DW_MCP_MAX_WAIT_SECONDS` (written as 1800 by `init-env.sh`, the systemd
+unit's value) caps how long an MCP `wait_for_job` may block (55 s when unset).
Build arguments (`docker compose build --build-arg ...`): `TORCH_INDEX`
(default the cu130 index; `https://pypi.org/simple` gives the CUDA 12 wheel
diff --git a/docs/IP_ADAPTER.md b/docs/IP_ADAPTER.md
index 5524fce7c..195b51550 100644
--- a/docs/IP_ADAPTER.md
+++ b/docs/IP_ADAPTER.md
@@ -1,18 +1,15 @@
# IP-Adapter
-IP-Adapter enables image-prompt conditioning — use a reference image to influence the style or content of generated images alongside a text prompt. It works with any pipeline whose `load_ip_adapter()` diffusers supports — Flux, Stable Diffusion 1.5, SDXL, SD 3.5, and others.
+IP-Adapter conditions a generation on a reference image alongside the text prompt, to carry its style or content. It works on any pipeline that has diffusers' `load_ip_adapter()`: Flux, Stable Diffusion 1.5, SDXL, SD 3.5 and others.
## Usage
-Add an `ip_adapter` block to the pipeline definition and pass `ip_adapter_image` in arguments:
+Add an `ip_adapter` block to the pipeline, a sibling of `configuration`, and pass the image as the `ip_adapter_image` argument:
```json
{
"pipeline": {
- "configuration": {
- "component_type": "FluxPipeline",
- "offload": "sequential"
- },
+ "configuration": { "component_type": "FluxPipeline" },
"from_pretrained_arguments": {
"model_name": "black-forest-labs/FLUX.1-dev",
"torch_dtype": "torch.bfloat16"
@@ -33,20 +30,22 @@ Add an `ip_adapter` block to the pipeline definition and pass `ip_adapter_image`
}
```
+FLUX.1 dev in bf16 needs a placement to fit a 24 GB card; [ip-adapter.json](../workflows/templates/ip-adapter.json) uses the streamed group offload from [RECIPES_24GB.md](RECIPES_24GB.md#flux-dev-12b).
+
## Properties
| Property | Required | Description |
| -------- | -------- | ----------- |
-| `model_name` | Yes | HuggingFace Hub repo ID for the IP-Adapter weights |
+| `model_name` | Yes | Hugging Face Hub repo id of the IP-Adapter weights |
| `weight_name` | No | Specific weight file in the repo |
| `subfolder` | No | Subfolder within the repo |
-| `scale` | No | Adapter strength (default: 1.0). Lower = less influence from reference image |
+| `scale` | No | Adapter strength, set with `set_ip_adapter_scale()` after the load. Unset keeps diffusers' default of 1.0. Lower = less influence from the reference image |
-Any other property (e.g. `revision`) is forwarded as-is to the underlying `load_ip_adapter()` call.
+Any other property (e.g. `revision`) is passed to `load_ip_adapter()` unchanged.
## Models Without a Built-in Image Encoder
-Some base models (e.g. Stable Diffusion 3.5) don't ship a default image encoder, so diffusers needs one configured explicitly. Add `image_encoder` and `feature_extractor` components alongside `ip_adapter`, the same way any other pipeline component is configured:
+Some base models (Stable Diffusion 3.5, for one) ship no image encoder, so diffusers needs one configured. Add `image_encoder` and `feature_extractor` component blocks beside `ip_adapter`, configured like any other pipeline component:
```json
{
@@ -77,27 +76,15 @@ Some base models (e.g. Stable Diffusion 3.5) don't ship a default image encoder,
}
```
-See [ip-adapter.json](../workflows/templates/ip-adapter.json) for the full workflow.
-
## Image Argument
-The `ip_adapter_image` uses the standard image loading format:
-
-```json
-"ip_adapter_image": {
- "location": "https://example.com/image.jpg"
-}
-```
+`ip_adapter_image` loads like any `*_image` argument ([Media Arguments](WORKFLOW_GUIDE.md#media-arguments), [References](WORKFLOW_GUIDE.md#references)): a URL, a path relative to the workflow file, an `asset:` name on the server, or a `{"location": ...}` dict of any of these. Only `location` is read from the dict; the image is not resized.
```json
-"ip_adapter_image": {
- "location": "./local/reference.png",
- "width": 512,
- "height": 512
-}
+"ip_adapter_image": "asset:reference.png"
```
-Can also reference a previous step's output:
+It can also take an earlier step's output:
```json
"ip_adapter_image": "previous_result:preprocessing_step"
@@ -105,5 +92,4 @@ Can also reference a previous step's output:
## Examples
-- [ip-adapter.json](../workflows/templates/ip-adapter.json) — Flux with IP-Adapter for style transfer
-- [ip-adapter.json](../workflows/templates/ip-adapter.json) — FLUX.1 dev with an IP-Adapter, and the face-model variant in its description
+- [ip-adapter.json](../workflows/templates/ip-adapter.json): FLUX.1 dev with `XLabs-AI/flux-ip-adapter`. Its description gives the face-model variant, which keeps a person's face across a new scene: `h94/IP-Adapter`, subfolder `models`, weight `ip-adapter-full-face_sd15.bin`, scale 0.5, on a `StableDiffusionPipeline`.
diff --git a/docs/LORAS.md b/docs/LORAS.md
index 5aa164c0e..b9eb07dfe 100644
--- a/docs/LORAS.md
+++ b/docs/LORAS.md
@@ -1,6 +1,6 @@
# LoRA Support
-LoRA (Low-Rank Adaptation) models apply lightweight style or subject modifications to a base model. Add one or more LoRAs to any pipeline step.
+A LoRA (Low-Rank Adaptation) adapter applies a style, subject or step-distillation change to a base model. A pipeline step takes a `loras` list, a sibling of `configuration`, on any pipeline diffusers can load adapters into (`load_lora_weights()`; `dw/pipeline_processors/adapters.py`).
## Basic Usage
@@ -33,28 +33,32 @@ LoRA (Low-Rank Adaptation) models apply lightweight style or subject modificatio
"weight_name": "specific_weights.safetensors",
"subfolder": "lora_subfolder",
"adapter_name": "my_adapter",
- "scale": 0.8
+ "scale": 0.8,
+ "alpha": 16
}
]
```
| Property | Required | Description |
| -------- | -------- | ----------- |
-| `model_name` | Yes | HuggingFace Hub repo ID |
+| `model_name` | Yes | Hugging Face Hub repo id. `null` switches the entry off: it is not loaded and the step runs without it, with a `lora_disabled` warning at validate and run. A template's `loras` list is fixed JSON, so this is how a caller drops its adapter (MiniMax-H3: pass `lora_model_name=null`) |
| `weight_name` | No | Specific weight file in the repo |
| `subfolder` | No | Subfolder within the repo |
-| `adapter_name` | No | Named identifier for the adapter. Defaults to the LoRA's position in the list (`"0"`, `"1"`, ...) if omitted |
-| `scale` | No | Blend strength (default: 1.0). Lower = less effect |
+| `adapter_name` | No | The adapter's name in `set_adapters()`. Defaults to the entry's position in the list (`"0"`, `"1"`, ...), counting switched-off entries |
+| `scale` | No | Adapter weight, default 1.0. Lower = less effect. Takes a `variable:` reference |
+| `alpha` | No | Network alpha, overriding the checkpoint's. peft scales an adapter by `scale * alpha / rank`, taking alpha from the file (a per-module `.alpha` tensor, else a `__metadata__` alpha, else the rank). Positive; state it only where the file records the wrong figure, after reading its header. Fails if no loaded layer carries the adapter. The run logs the effective `alpha / rank` |
-Any other property (e.g. `revision`) is forwarded as-is to the underlying `load_lora_weights()` call.
+Any other property (e.g. `revision`) is passed to `load_lora_weights()` unchanged.
`scale` and `alpha` are set in place on the loaded adapter, so a server run that changes
-only them reuses the warm pipeline instead of reloading it; changing `model_name` or
-`weight_name` reloads.
+only them reuses the warm pipeline; changing `model_name` or `weight_name` reloads.
+
+A step-distillation (turbo) LoRA sets the step count and sigma shift the step runs at.
+Switching one off, raise both to the base model's schedule, or the step runs undertrained.
## Multiple LoRAs
-Stack multiple LoRAs. They are blended via weighted adapter composition:
+Several entries load as separate adapters, each at its own `scale`, active together:
```json
"loras": [
@@ -73,7 +77,7 @@ Stack multiple LoRAs. They are blended via weighted adapter composition:
## LoRA with Quantization
-LoRAs work with quantized models:
+A LoRA loads onto a quantized component as onto any other:
```json
{
@@ -108,7 +112,7 @@ LoRAs work with quantized models:
## Variable LoRA
-Make the LoRA configurable via workflow variables:
+A `variable:` reference makes the adapter an argument:
```json
{
@@ -130,9 +134,8 @@ python -m dw.run workflow.json lora="other-user/other-lora"
## Examples
-- [lora.json](../workflows/templates/lora.json) — Flux with realism LoRA and variables
-- [lora.json](../workflows/templates/lora.json) — SD 3.5 with yarn art style LoRA
-- [lora.json](../workflows/templates/lora.json) — A LoRA adapter on FLUX.1 dev, with the SD 3.5 variant in its description
+- [lora.json](../workflows/templates/lora.json): FLUX.1 dev with `XLabs-AI/flux-RealismLora` as the `lora` variable; its description names an SD 3.5 variant (`linoyts/yart_art_sd3-5_lora`)
+- [lora-styles.json](../workflows/templates/lora-styles.json): one prompt through ten FLUX.1 in-context LoRA styles, one image per adapter
## LoRA catalog
@@ -162,8 +165,9 @@ a step's `loras` entry. `base_models` holds exact repo ids, and matching is
exact - an adapter on the wrong base usually loads and is quietly worse.
`workflow` constrains a MiniMax-H3 entry to a partition (`t2va`, `fl2va`,
`ref2va`) - a string, or a list of them for an adapter that covers several
-(the turbo keyframe files are `["t2va", "fl2va"]`). `proven` and `rejected` entries need `evidence`; a rejected
-entry's first note is why. The schema is `GET /api/lora-schema`.
+(the turbo keyframe files are `["t2va", "fl2va"]`). `proven` and `rejected` entries need
+`evidence`; a rejected entry's first note is why. The schema is `GET /api/lora-schema`
+(`dw/lora_catalog_schema.json`).
The server's validate enforces that `workflow` (#870): a step whose workflow
sits in the other partition is refused when the entry is `proven`, and only
@@ -180,9 +184,9 @@ it is also what makes its `workflow` refuse a mismatched step.
place dw searches the Hub, and only when called. It returns the catalog's
entries for the model first, ranked against the query, then Hub adapters
whose card declares that exact base (`base_model:adapter:`), most
-downloaded first. Nothing is downloaded; the weight file's header is read to
-check its layout, for a single-weight repo only. Hub rows are candidates to trial, never recommendations,
-and carry `warnings`:
+downloaded first. Nothing is downloaded: for a single-weight repo the weight
+file's header is read to check its layout. Hub rows are candidates to trial, never
+recommendations, and carry `warnings`:
| Warning | Meaning |
| --- | --- |
diff --git a/docs/MCP.md b/docs/MCP.md
index 2466bca73..8ba7ebed3 100644
--- a/docs/MCP.md
+++ b/docs/MCP.md
@@ -1,79 +1,63 @@
# MCP Server
-A third way to drive the engine, alongside `dw.run` and
-`dw.serve`: a stdio [MCP](https://modelcontextprotocol.io) server that lets
-an MCP client — Claude Code first — author, validate, save, run and diagnose
-workflows without shell access or a repo checkout.
+A stdio [MCP](https://modelcontextprotocol.io) server that lets an MCP client
+(Claude Code first) author, validate, save, run and diagnose workflows without
+shell access or a repo checkout. It is the third way to drive the engine,
+beside `dw.run` and `dw.serve`.
-`dw_mcp/` is an HTTP client of a **running** `dw.serve`. It owns no job
-state and no GPU worker of its own; every tool call is a REST request
-against the server described in [Server & Web UI](SERVER.md). If `dw.serve`
-is not running, every tool fails with a message telling you to start it.
+`dw_mcp/` is an HTTP client of a **running** `dw.serve`. It holds no job state
+and no GPU worker; every tool call is a REST request against the server
+described in [Server & Web UI](SERVER.md). If `dw.serve` is not running, every
+tool fails with a message telling you to start it.
## Install and run
```bash
pip install -e ".[server,mcp]"
+dw-serve # the engine; leave it running
```
-The MCP server needs a running `dw.serve`, so two processes are involved:
-
-```bash
-# terminal 1 - the engine. Leave it running.
-dw-serve
-```
-
-You do not start `dw-mcp` yourself. Your MCP client launches it on demand,
-which is why the client needs a command it can actually find (see below).
-
-`dw-mcp` (equivalently `python -m dw_mcp`) speaks MCP over stdio. Flags:
+You do not start `dw-mcp` yourself: the MCP client launches it on demand, so
+the client needs a command it can find (see below). `dw-mcp` (or
+`python -m dw_mcp`) speaks MCP over stdio. Flags:
| Flag | Default | Meaning |
| --- | --- | --- |
| `--url` | `$DW_MCP_URL`, else `http://127.0.0.1:8765` | Base URL of the running `dw.serve` |
-| `--token` | `$DW_API_TOKEN`, else none | Bearer token, when `dw.serve` was started with `--token` / `DW_API_TOKEN` - the same variable, so one export configures both ends |
-| `--workspace` | `$DW_MCP_WORKSPACE`, else the server's default | Which of the server's workspaces the session works in. A *name* on the server, not a directory here - `DW_WORKSPACE` means something else to the engine. `use_workspace` switches it mid-session |
+| `--token` | `$DW_API_TOKEN`, else none | Bearer token, when `dw.serve` was started with `--token` / `DW_API_TOKEN`. The same variable configures both ends |
+| `--workspace` | `$DW_MCP_WORKSPACE`, else the server's default | Which of the server's workspaces the session works in. A *name* on the server, not a local directory (`DW_WORKSPACE` means something else to the engine). `use_workspace` switches it mid-session |
| `--timeout` | `30` | Seconds to wait on any one API request |
| `--no-probe` | off | Skip the startup `GET /api/health` that confirms the server is reachable and the token is accepted |
-The `DW_MCP_URL` environment variable sets the same default the `--url` flag
-overrides.
-
-A non-loopback `--url` requires a token: `dw-mcp` exits 2 rather than start
-without one. At startup it makes one `GET /api/health`, so a wrong URL or
-token is reported once with a message instead of as a 401 on every tool
-call; that probe is fatal for a remote URL and only a warning for a
-loopback one (where it usually means `dw.serve` is not up yet).
-[REMOTE.md](REMOTE.md) covers the remote setup end to end.
+A non-loopback `--url` with no token makes `dw-mcp` exit 2. The startup probe
+reports a wrong URL or token once, instead of as a 401 on every tool call; it
+is fatal for a remote URL and only a warning for a loopback one, where it
+usually means `dw.serve` is not up yet. [REMOTE.md](REMOTE.md) covers the
+remote setup end to end.
-Claude Code users can add the composition skills as well:
+Claude Code users can add the composition skills with
`/plugin marketplace add dkackman/diffusers-workflow` then
-`/plugin install dw@diffusers-workflow`. The plugin ships one skill per model
-family (MiniMax H3, MiniMax Music 3, LTX-2.5) that picks a template for a
-request's shape and states the family's rules - see
-[plugins/dw/README.md](../plugins/dw/README.md), which also gives the
-optional `npx skills add` lines for MiniMax's own prompt skills. It is
-optional; every tool below works without it.
+`/plugin install dw@diffusers-workflow`: one skill per model family (MiniMax
+H3, MiniMax Music 3, LTX-2.5, Kandinsky 6, FLUX 3 Klein) plus cross-cutting
+skills for scripts, series, cinematography and storyboards. See
+[plugins/dw/README.md](../plugins/dw/README.md), which also gives the optional
+`npx skills add` lines for MiniMax's own prompt skills. Every tool works
+without the plugin.
### Use the absolute path to `dw-mcp`
-**This is the one setup detail that reliably goes wrong.** If you installed
-the way `install.sh` does, `dw-mcp` lives in the project's virtualenv and is
-only on `PATH` while that venv is activated. Your MCP client is launched by
-your shell, your desktop app, or your editor — usually *without* the venv
-activated — so a bare `dw-mcp` fails to spawn:
+**This is the setup detail that most often goes wrong.** Installed the way
+`install.sh` does it, `dw-mcp` lives in the project's virtualenv and is on
+`PATH` only while that venv is active. An MCP client is usually launched
+without the venv, so a bare `dw-mcp` fails to spawn:
```
Failed to reconnect to dw: ENOENT
```
-Registering it once from an activated terminal hides this: that session
-works, and the next one, started somewhere else, does not.
-
-Always register the venv's absolute path. Console scripts have the
-interpreter baked into their shebang, so they run correctly with no venv
-activated — which is exactly why the absolute path is more robust than
-telling people to activate first:
+Registering from an activated terminal hides the problem until a session
+starts somewhere else. Register the venv's absolute path instead; a console
+script has its interpreter in its shebang, so it runs with no venv active:
```bash
echo "$(pwd)/venv/bin/dw-mcp" # the value to register
@@ -83,42 +67,36 @@ echo "$(pwd)/venv/bin/dw-mcp" # the value to register
### Remote server, no local install
-If `dw.serve` runs on another machine with `--mcp` (see
-[REMOTE.md](REMOTE.md)), Claude Code connects to it directly:
+If `dw.serve` runs on another machine with `--mcp` (see [REMOTE.md](REMOTE.md)),
+Claude Code connects to it directly and nothing from this repository is
+installed on the client:
claude mcp add --transport http dw http://:8765/mcp \
--header "Authorization: Bearer "
-The same token fetches generated files: see step 7 of `The loop` in [WORKFLOW_GUIDE.md](WORKFLOW_GUIDE.md#the-loop).
-
-Nothing from this repository is installed on the client. The stdio setup
-below is for a machine that has its own `dw` install, and also works
+Generated files under `/outputs`, `/inputs` and `/exports` need no token
+(#592); step 7 of [The loop](WORKFLOW_GUIDE.md#the-loop) fetches them. The
+stdio setup below is for a machine with its own `dw` install, and also works
against a remote `--url` with `--token`.
### Claude Code
-The CLI is the shortest path. From the project directory:
+From the project directory:
```bash
claude mcp add dw -- "$(pwd)/venv/bin/dw-mcp"
+claude mcp add dw -- "$(pwd)/venv/bin/dw-mcp" --url http://127.0.0.1:8791 # subprocess flags go after --
```
-`--` separates Claude Code's own flags from the command it will spawn. Add
-subprocess flags after it:
-
-```bash
-claude mcp add dw -- "$(pwd)/venv/bin/dw-mcp" --url http://127.0.0.1:8791
-```
-
-Pick the scope deliberately with `-s`:
+Pick the scope with `-s`:
| Scope | Stored in | Use when |
| --- | --- | --- |
| `local` (default) | `~/.claude.json`, keyed to this project | Just you, just this checkout |
-| `user` | `~/.claude.json`, global | You want it in every project. The absolute path makes this work |
-| `project` | `.mcp.json`, **committed to the repo** | You intend every clone to get it. Note an absolute path is machine-specific and will not port |
+| `user` | `~/.claude.json`, global | Every project. The absolute path makes this work |
+| `project` | `.mcp.json`, **committed to the repo** | Every clone gets it. An absolute path is machine-specific and will not port |
-Equivalent hand-written `.mcp.json`, if you prefer a file:
+The same registration as a hand-written `.mcp.json`:
```json
{
@@ -131,14 +109,12 @@ Equivalent hand-written `.mcp.json`, if you prefer a file:
}
```
-**A running session does not pick up a registration change** - start a new
-one after adding or editing the server.
+**A running session does not pick up a registration change**; start a new one.
### Claude Desktop
-`claude_desktop_config.json`, same shape - and the same absolute-path rule,
-which matters more here because a desktop app never inherits a shell's
-`PATH`:
+`claude_desktop_config.json` takes the same shape. The absolute path matters
+more here, because a desktop app never inherits a shell's `PATH`:
```json
{
@@ -151,393 +127,302 @@ which matters more here because a desktop app never inherits a shell's
}
```
-`DW_MCP_URL` can be set instead of `--url` via an `"env"` object alongside
-`"command"`/`"args"` in either config.
+In either config, an `"env"` object beside `"command"`/`"args"` can set
+`DW_MCP_URL` instead of `--url`.
## Verify the setup
-Three checks, in order. Each isolates a different failure, so run them in
-sequence rather than jumping to the last one.
+Run the three checks in order; each isolates a different failure. None costs
+GPU time.
**1. The command launches without a venv.** This reproduces the environment
-your client actually spawns it in, and is the check that catches ENOENT:
+the client spawns it in, and catches ENOENT:
```bash
env -i PATH=/usr/bin:/bin HOME="$HOME" /path/to/venv/bin/dw-mcp --help
```
-Prints usage and exits 0. If it does not, the path is wrong or the package
-is not installed into that venv.
+It prints usage and exits 0. If not, the path is wrong or the package is not
+installed into that venv.
-**2. The client sees the server.** Start a new Claude Code session and run
-`/mcp`; `dw` should be listed and connected. From the shell,
-`claude mcp list` and `claude mcp get dw` show the same thing.
+**2. The client sees the server.** In a new Claude Code session, `/mcp` lists
+`dw` as connected (`claude mcp list` and `claude mcp get dw` from the shell).
+"Connected" means only that the process launched, not that `dw.serve` is
+reachable.
-Note that a "connected" status only means the process launched - it says
-nothing about whether `dw.serve` is reachable.
-
-**3. The tools reach the engine.** With `dw-serve` running, ask the client
-something free, such as "list my diffusers workflows" (`list_workflows`) or
-"check the diffusers-workflow server health" (`get_health`). A real answer
-means the whole chain works. "Cannot reach diffusers-workflow at ..." means
-step 3 failed while steps 1 and 2 passed - the client is fine and the engine
-is not running.
-
-Nothing in this sequence costs GPU time.
+**3. The tools reach the engine.** With `dw-serve` running, ask for something
+free, such as "list my diffusers workflows" (`list_workflows`) or "check the
+diffusers-workflow server health" (`get_health`). "Cannot reach
+diffusers-workflow at ..." means the engine is not running.
## Tool reference
-65 tools in six groups. Names and arguments below are transcribed from
-`dw_mcp/tools_*.py` — nothing here is renamed or reshaped for the docs.
+65 tools in six groups. Names, arguments and defaults are those registered in
+`dw_mcp/tools_*.py` (wired in `dw_mcp/server.py`); each tool's own description
+is the full contract. Every `workspace` argument pins that one call to a
+workspace without moving the session's pin (see
+[Authoring, assets and workspaces](#authoring-assets-and-workspaces)).
### Catalog (read-only)
-The catalog is large, so the server's instructions point a client at
-`list_workflows` first: its listing carries enough about each workflow - a
-one-line `summary`, its `shape` and `traits`, its measured `cost`, output
-kinds and variable names - to pick one and know what to pass it, without
-fetching every candidate's definition. Reusing a stored workflow is a
-preference, not a rule; `run_workflow` still takes an `inline_workflow` for
-a request nothing on disk covers.
-
-The instructions also say the engine runs **one job per GPU** - one worker
-per `dw.serve --devices` card, each running its own job from the one shared
-queue - and point at `get_health`'s `workers` for what each card is running.
-A job record's `device` (`get_job`, `list_jobs`) names the card it ran on,
-and a job started by `rerun_job` carries `rerun_of`, the id of the job it
-reran (absent on a fresh submit).
-
-A request usually names a *subject* ("a lego movie trailer set in the marvel
-universe") while the catalog is written in *shapes* - a single image, an image
-set, one shot, a multi-shot cut sequence, video with speech. Nothing in a
-catalog entry will match the subject, so the shape is what has to be decided
-first and matched against. `list_guides` indexes the engine's documentation by
-section for exactly that, and `list_tasks` is what a shape gets composed from
-when no single workflow covers it.
-
-| Tool | Arguments | Purpose |
-| --- | --- | --- |
-| `list_guides()` | — | List the documentation the engine serves: each guide's name, what it covers, and its section headings. The index is the routing table - match a request's shape against a heading rather than guessing |
-| `get_guide(name, section=None)` | `name`, `section` | Get one section of a guide - name it: a guide runs to thousands of lines, and WORKFLOW_GUIDE.md whole is ~19.6k tokens, more in one call than the whole tool surface costs to connect. Called with no `section` the answer is the guide's index: its opening, its first section, and `sections`/`withheld` naming the rest, with a `note` saying how to fetch one. Section names match loosely, so a heading copied approximately still resolves |
-| `list_workflows(shape=None, traits=None, configures=None, include_models=False)` | `shape`, `traits`, `configures`, `include_models` | List stored workflows. Called with no filter at all each entry is cut to its `summary` and `shape` (`view: "summary"`, with a `note`), because the whole catalog in full detail is ~6.8k tokens for a question that is really "which shape do I want"; pass `shape` and the entries come back whole. Filtered, it is the server's compact view: each entry carries `summary`, `shape`, `traits`, `cost`, `kinds`, `variable_names`, `lists`, and `configures` only when set - `get_workflow` has the full description and definition. `lists`, present for a list-driven workflow, names the fields an entry of each list takes, the steps over it and the default's length; `cost` may carry `per_entry`, the measured cost of one entry so a run over a different-length list can be priced from it. `shape` keeps one of `image`, `image-set`, `image-edit`, `shot`, `sequence`, `audio`, `text`, `utility`; `traits` is comma-separated and every one listed must match (`has-audio`, `chained`, `image-conditioned`, `identity-referenced`, `needs-input-media`, `composes-workflows`); an unknown value in either is a 400 listing the vocabulary. Templates only by default - `configures=` lists the checkpoint configs tuned for one, `include_models=true` lists them all. The first call to make for a request an existing workflow might cover `libraries` names the roots searched (`origin`, `root`, `writable`) and `shadowed` the names a nearer root hides |
-| `get_workflow(name, variables_only=False)` | `name` | Get one stored workflow's full JSON definition. `variables_only=true` answers with just its variables and their defaults (long strings cut to 200 characters, the cut ones named in `truncated`, including strings inside a list default, named like `shots[0].prompt`) — the cheap way to confirm what a variable defaults to |
-| `get_schema(section=None)` | `section` | Get the JSON schema every workflow definition must satisfy. Whole it is ~8.6k tokens, so name the part you need: `section` takes `steps`, `pipelines`, `tasks`, `result`, `variables` or `configuration` and answers `{section, sections, elsewhere, schema}` - `elsewhere` says which section holds each definition the fragment still `$ref`s. An unknown section is a 404 naming the ones there are |
-| `list_pipelines()` | — | List every diffusers pipeline class this installation provides |
-| `get_pipeline_signature(name)` | `name` | Get a pipeline's real call arguments; for a modular pipeline class (e.g. `MiniMaxH3ModularPipeline`), its block graph's inputs |
-| `list_classes(kind)` | `kind` | List class names of one kind: pipelines, models, schedulers, or quantization |
-| `get_class(name, target="init")` | `name`, `target` (`init`\|`call`\|`load`) | Get a class's argument schema from the entry point a workflow reaches it by: `init` the constructor (quantization configs, schedulers), `call` a pipeline's `__call__`, `load` `from_pretrained` plus the curated loading knobs |
-| `list_tasks()` | — | List every task command a workflow's task step can name |
-| `get_task(command)` | `command` | Get a task command's argument schema |
-| `list_models()` | — | List what the Hugging Face model cache holds, largest first |
-| `get_memory()` | `device` | Get VRAM and RAM statistics: `workers` has one reading per card (the top level repeats the first card's), and `device` (`cuda:1`) asks for that card alone. `gpu_*` is the card; `host_memory_rss_mb` / `host_memory_peak_rss_mb` are the worker process's resident and high-water host memory, beside the machine's `host_memory_total_mb` / `host_memory_available_mb`; on Linux `host_memory_rss_anon_mb` / `host_memory_rss_file_mb` split the resident figure into memory the worker owns and mapped-file pages (memory-mapped checkpoint weights) the kernel can drop, so a loader that maps rather than copies raises the resident and peak figures without raising committed memory (#709) - read both, since an offloading workflow keeps its weights in host RAM and the card says little about what it holds (a host field is absent, not null, where the platform cannot measure it). `host_pinned_reserved_mb` / `host_pinned_allocated_mb`, when present, are torch's pinned-host cache - the staging buffers group offloading moves weights through, part of `host_memory_rss_mb` and invisible in every `gpu_*` figure, which is what a worker holding GB after releasing every model is usually holding (#98). `live: true` means `info` was measured now and is the worker's own memory; only live readings are comparable with each other. `live: false` with `info: null` (and `stale: false`) means nothing has been measured because nothing is resident; `live: false` with a populated `info` is a cached earlier reading, with `reason` (`job_running`, `worker_stopped`, `worker_busy`, `worker_unreachable`) and `age_seconds` - one cached while a job loads a model understates what is resident, so ask again when the card is idle rather than comparing it with a live figure |
-| `get_health()` | — | Check that the server is alive, and which machine answered: `version`, `device`, whether any model process is resident (`worker_alive`), and the queue depth. The engine runs one job per GPU: `workers` has one entry per card - `device`, `name`, `vram_gb`, `alive`, its own `current_job`, and the worker's `host_memory_rss_mb` while it runs - and the top-level `current_job` is the longest-running job. `worker_alive: false` is the normal idle state on a server that has not run a job since startup or the last memory clear - not a fault - the on-demand worker starts with the next job (#206) |
-| `get_server_info()` | — | What this installation can do and where it keeps things: `device` (the accelerator a run will use), `version`, the `workspace` this session is working in and the workflow/asset/output/prompt `directories` of *that* workspace, the bind address and port, whether a token is required, and whether MCP is mounted. Check the device before authoring - a CUDA-only choice (bitsandbytes, `torch.compile`, flash attention) is not available on an `mps` or `cpu` server. `runtime` (#222) reports the Python version, torch version and the CUDA version torch was built against, the NVIDIA driver version (when `nvidia-smi` is reachable), and the installed versions of diffusers, transformers, accelerate, bitsandbytes, peft, safetensors and sentencepiece (`null` for one not installed) - for diagnosing an environment mismatch between boxes without shelling in. `caches` lists dw's disk caches (`inductor`, `triton`, `quantized`), each with `name`, `bytes`, `files`, `enabled`, `clearable` and `disk_free`, plus `entry_count` for a cache with entries, and `cache_entries=true` lists them (`id`, `model`, `subfolder`, `component`, `bytes`, `files`, `unreadable`, `created`, `last_hit`; the quantizer and dtype are not in `/api/caches`); it is omitted when the server has no `/api/caches`. There is no MCP tool to clear one - that is `DELETE /api/caches/{name}` or `.../entries/{entry}` ([SERVER.md](SERVER.md)) |
-| `list_jobs(limit=20, status=None, workspace=None)` | optional `limit` (newest N), `status` (one state or a comma-separated set of `queued`, `running`, `succeeded`, `failed`, `cancelled`), `workspace` | List queued, running and recent jobs, **newest first** (a queued job held for host RAM carries `waiting_for`). Bounded by default: the unbounded listing was over a client's tool-result limit on a server with a few months of history, which made it a tool that could not be called at all. `total` says how many matched and `truncated`/`next` say so when the answer was cut - raise `limit` or narrow with `status`. Without `workspace`, a named workspace lists its own jobs and the default one lists every job the server holds |
-| `list_gallery(limit=50, subfolder=None, only_orphans=False, workspace=None, folder=None, version=None, media=False, role=None)` | `limit`, `subfolder`, `only_orphans`, `workspace`, `folder`, `version`, `media`, `role` | List generated output files, newest first. A name is `//`, where `` may sit in the subfolder the step chose (`final/episode.mp4`); each entry carries `folder` (the workflow) and `subfolder` (by convention `final` or `intermediate`, `''` when the step chose none, any path the workflow wrote otherwise), and `subfolder="final"` lists only deliverables. Each entry also carries `role`, `final` or `intermediate` (null for a flat-layout file with no run, or an older run's file that neither its subfolder nor a later run's reads place; `intermediate` once a later step or a later run read the file, unless `result.role` declared it); `role=["final"]` lists only those, and the `role` filter defaults to every file. See `dw/roles.py` for how a role is decided. A file starred in the web UI's gallery (its keep mark) carries `starred: true`; an unstarred one carries no `starred`. Each entry carries `run_id` and `version` - that run's ordinal among the workflow's runs, which is how one of several runs that wrote the same basename is named to a person: the web UI labels the same file `v5`. The number is assigned when the run opens and never renumbered, so deleting a run leaves a gap rather than sliding the rest down (a failed run, or a rerun that reused every step, leaves one too - it took a number and may have nothing to list), and it is `null` under the flat output layout, which has no runs. `folder` with `version` lists that one run's files, and `output:/v5/` names one in a workflow; every other tool takes `name`. Each entry also carries a ready-made `url`, already scoped to the workspace that made it - a hand-built `/outputs/` URL 404s for anything but the default workspace. `only_orphans=True` inverts the call: instead of files, it returns run directories with no media anywhere under them (`runs`, each `{name, mtime}`) - a run whose output was deleted before `delete_output` could remove it by name, or one that failed before writing anything; `subfolder` does not apply in this mode, and `name` is exactly what `delete_output` accepts (#170). `workspace` names the workspace for this one call without switching the session to it - the same pin `run_workflow` takes, so a job run into another workspace stays reachable from the session that queued it |
-| `get_gallery_metadata(name, envelope=False, workspace=None, interval_seconds=None)` | `name`, `workspace` | Get the metadata embedded in a generated file — or, when `name` is an `asset:` reference, what an *input* asset holds (`source` says which; `job` is null for an asset). Reading an input's duration, frame count, fps and sample rate before a run is how a caller learns the `total_frames`, `fps` and `sample_rate` a workflow expects it to supply: the exact workflow and arguments that produced it, and, for audio/video, a `media` block (duration, rate, channels, fps, size, peak/mean dBFS) and `findings` - each level problem the server measured against `dw/audio_qc.py`'s thresholds (`full_scale`, `near_silent`), with the fix. Only an image (PNG/JPEG/WebP) embeds `metadata` this way — it is always null for audio and video, and `next` then points at `get_job_workflow(job_id)` when `job` is known, or says a kept asset carries no provenance at all when it isn't. `envelope=true` adds `media.envelope` — `rms_dbfs` and `peak_dbfs` one entry per second, or per `interval_seconds` (default 1.0, 0.05 to 3600; 0.1 places a vocal onset; a finer-than-default width that would need over 2400 bins is refused with the interval that fits) — which is what locates something in a track rather than measuring the whole of it. `media.shots` is set on an output joined from shots: one `{name, start_frame, num_frames, start_sample, num_samples}` per shot, as the join measured them (see the run manifest in docs/WORKFLOW_GUIDE.md), else null. `workspace` names the workspace for this one call without switching the session to it - the same pin `run_workflow` takes, so a job run into another workspace stays reachable from the session that queued it |
+The server's instructions send a client to `list_workflows` first: its
+listing carries enough per workflow to choose one and know what to pass it.
+A request usually names a *subject* while the catalog is written in *shapes*
+(one image, an image set, one shot, a cut sequence), so decide the shape
+first. `list_guides` indexes the documentation by section, and `list_tasks`
+is what a shape is composed from when no stored workflow covers it;
+`run_workflow` still takes an `inline_workflow` for that case.
+
+The engine runs **one job per GPU**: one worker per `dw.serve --devices` card,
+each taking jobs from one shared queue. A job record's `device` (`get_job`,
+`list_jobs`) names the card it ran on, and a job started by `rerun_job`
+carries `rerun_of`.
+
+| Tool | Purpose |
+| --- | --- |
+| `list_guides()` | The documentation the engine serves: each guide's name, what it covers, and its section headings. Match a request's shape against a heading |
+| `get_guide(name, section=None)` | One section of a guide, matched loosely (a `###` subsection resolves too). Without `section` the answer is the guide's index: its opening, its first section, and `sections`/`withheld` naming the rest. A whole guide can cost more than the tool surface does to connect |
+| `list_workflows(shape=None, traits=None, configures=None, include_models=False)` | Stored workflows. With no filter each entry is cut to `summary` and `shape` (`view: "summary"`, with a `note`). Filtered, each entry carries `summary`, `shape`, `traits`, `cost`, `observed_minutes`/`observed_runs` (this box's own finished runs), output kinds, `variable_names`, and when set `lists` (per list variable: entry fields, steps over it, default length; `cost[].per_entry` prices one entry), `constraints` and `configures`. `shape` is one of `image`, `image-set`, `image-edit`, `shot`, `sequence`, `audio`, `text`, `utility`; `traits` is comma-separated and all must match (`has-audio`, `chained`, `image-conditioned`, `identity-referenced`, `needs-input-media`, `composes-workflows`); an unknown value is a 400 listing the vocabulary. Templates only by default; `configures=` lists the checkpoint configs tuned for one, `include_models=true` lists them all. `libraries` names the roots searched (`origin`, `root`, `writable`), `shadowed` the names a nearer root hides |
+| `get_workflow(name, variables_only=False)` | One stored workflow's full JSON. `variables_only=true` returns just its variables and defaults, long strings cut to 200 characters and named in `truncated` (`shots[0].prompt` for one inside a list) |
+| `get_schema(section=None)` | The workflow JSON schema (~8.6k tokens whole). `section` is one of `steps`, `pipelines`, `tasks`, `result`, `variables`, `configuration` and answers `{section, sections, elsewhere, schema}`; `elsewhere` says which section holds each `$ref` the fragment still has. An unknown section is a 404 naming the valid ones |
+| `list_pipelines()` | Every diffusers pipeline class this installation provides |
+| `get_pipeline_signature(name)` | A pipeline's real call arguments; for a modular pipeline class (`MiniMaxH3ModularPipeline`), its block graph's inputs |
+| `list_classes(kind)` | Class names of one kind: pipelines, models, schedulers or quantization |
+| `get_class(name, target="init")` | A class's argument schema at the entry point a workflow reaches it by: `init` the constructor (quantization configs, schedulers), `call` a pipeline's `__call__`, `load` `from_pretrained` plus the curated loading knobs |
+| `list_tasks()` | Every task command a task step can name |
+| `get_task(command)` | A task command's argument schema |
+| `list_models()` | What the Hugging Face cache holds, largest first |
+| `get_memory(device=None)` | VRAM and RAM per card in `workers` (the top level repeats the first card; `device`, e.g. `cuda:1`, asks for one). `gpu_*` is the card. `host_memory_rss_mb` / `host_memory_peak_rss_mb` are the worker's resident and peak host memory, beside the machine's `host_memory_total_mb` / `host_memory_available_mb`; on Linux `host_memory_rss_anon_mb` / `host_memory_rss_file_mb` split resident memory into owned pages and mapped checkpoint pages the kernel can drop (#709). `host_pinned_reserved_mb` / `host_pinned_allocated_mb` are torch's pinned-host cache, part of RSS and invisible to `gpu_*` (#98). A host field the platform cannot measure is absent. `live: true` is a reading taken now; `live: false` with `info: null` means nothing is resident; `live: false` with `info` is a cached reading with `reason` (`job_running`, `worker_stopped`, `worker_busy`, `worker_unreachable`) and `age_seconds`. Compare only live readings |
+| `get_health()` | Liveness and which machine answered: `version`, `device`, `worker_alive`, queue depth, and `workers` (per card: `device`, `name`, `vram_gb`, `alive`, `current_job`, `host_memory_rss_mb`). The top-level `current_job` is the longest-running job. `worker_alive: false` is the normal idle state before the first job or after a memory clear (#206) |
+| `get_server_info(cache_entries=False)` | `device`, `version`, this session's `workspace` and its `directories`, the bind address and port, `auth_required`, `trust_workflows`, the MCP mount, and `runtime` (#222: Python, torch, CUDA build, NVIDIA driver, and the diffusers, transformers, accelerate, bitsandbytes, peft, safetensors and sentencepiece versions, `null` when not installed). Check `device` before authoring: CUDA-only choices (bitsandbytes, `torch.compile`, flash attention) fail on `mps` or `cpu`. `caches` sizes dw's disk caches (`inductor`, `triton`, `quantized`); `cache_entries=true` lists the quantized cache's entries. No tool clears a cache; that is `DELETE /api/caches/{name}` ([SERVER.md](SERVER.md)) |
+| `list_jobs(limit=20, status=None, workspace=None)` | Queued, running and recent jobs, **newest first**, with queue position, `device`, `acknowledged` and (when held for host RAM) `waiting_for`. `status` is one state or a comma-separated set of `queued`, `running`, `succeeded`, `failed`, `cancelled`. `total`, `truncated` and `next` show a cut answer. Without `workspace`, a named workspace lists its own jobs and the default lists every job |
+| `list_gallery(limit=50, subfolder=None, only_orphans=False, workspace=None, folder=None, version=None, media=False, role=None)` | Generated files, newest first, named `//` (`` may sit in a subfolder: `final/episode.mp4`). Each entry carries `folder`, `subfolder` (`final`, `intermediate`, `''` or any path the workflow wrote), `role` (`final`, `intermediate`, or null; see `dw/roles.py`), `starred` when kept in the web UI, `run_id`, `version` and a workspace-scoped `url` (a hand-built `/outputs/` 404s outside the default workspace). `version` is the run's ordinal (`v5` in the UI): assigned when the run opens, never renumbered, `null` in the flat layout; `output:/v5/` names a file by it. `folder` with `version` lists one run. `role=["final"]` lists only deliverables. `only_orphans=True` returns run directories with no media (`runs`, each `{name, mtime}`), each a name `delete_output` accepts (#170) |
+| `get_gallery_metadata(name, envelope=False, workspace=None, interval_seconds=None, include_workflow=True)` | What produced a file and what it measures. `name` may be an `asset:` reference, for an input's duration, frame count, fps and sample rate before a run (`source` says which; `job` is null). Only an image embeds `metadata`; for audio and video it is null and `next` points at `get_job_workflow(job_id)`. `include_workflow=false` omits the embedded workflow. `media` holds duration, rates, channels, fps, size and peak/mean dBFS; `findings` lists each level problem (`full_scale`, `near_silent`, thresholds in `dw/audio_qc.py`) with its fix. `envelope=true` adds `media.envelope` (`rms_dbfs`, `peak_dbfs`) per second or per `interval_seconds` (0.05 to 3600; over 2400 bins is refused). `media.pcm_sha256` hashes the decoded samples; two hashes compare only when `sample_format`, `frames`, `sample_rate` and `channels` agree. `media.shots` places a joined output's shots (`{name, start_frame, num_frames, start_sample, num_samples}`) |
### Media
-| Tool | Arguments | Purpose |
-| --- | --- | --- |
-| `get_output_image(name, max_dimension=768, workspace=None, crop=None)` | `name`, `max_dimension`, `workspace`, `crop` | Look at a generated image, downscaled to `max_dimension` on its longest side. Returns the image plus a text part reporting `original_size`, `returned_size` and `bytes`, so a downscale is never silent. `crop` is `[x, y, width, height]` in the original's pixels, cut before the downscale and reported back clamped - the way to see a region of a 2K still at 100%, where the whole would be shrunk past what a small element or a decode-tiling seam can be judged at. `workspace` names the workspace for this one call without switching the session to it - the same pin `run_workflow` takes, so a job run into another workspace stays reachable from the session that queued it |
-| `get_output_audio(name, start=None, duration=None, workspace=None)` | `name`, `start`, `duration`, `workspace` | Listen to a generated soundtrack as base64 - an audio output, or the track muxed into a video (#193) - in its own encoding when served whole, WAV when extracted or excerpted. No downscale exists for audio, so a whole clip over the 4MB budget is refused rather than cut (#204); ask for the part instead with `start` and `duration` in seconds, and the text part names what was cut (`excerpt: 2.0s from 10.0s of 240.0s`) so a slice is never mistaken for the whole. `get_gallery_metadata`'s envelope says where in a track to look. `workspace` names the workspace for this one call without switching the session to it |
-| `get_output_frames(name, at=None, seams=None, count=None, boundaries=None, names=None, max_dimension=512, hear=None, workspace=None)` | `name`, `at`, `seams`, `count`, `boundaries`, `names`, `max_dimension`, `hear`, `workspace` | See a generated video as frames, since there is no video content type over MCP (#193). One selector per call: `count` for an evenly spaced contact sheet, `at` for moments (seconds or `"frame:N"`), `seams` (true, or seam numbers from 1) for the last frame before and first frame after each join side by side (each pair carries `difference`, the mean pixel change across the join, 0-255 - rank seams by it and look at the worst). `boundaries` and `names` are modifiers of `seams` only - passing either alongside `at` or `count` is refused, naming the offending argument(s). On an output joined from shots (`concat_videos`, `dissolve_videos`, a chained step) `seams` alone is enough: the seams and their names are the `media.shots` its run recorded. For any other file pass `boundaries` - each later shot's first frame, the running sum of the shots' `frame_count` from `get_gallery_metadata` on their own `intermediate/` files - and `names` to name them; either one given overrides the recorded value. Tiles are fitted to `max_dimension` and, when the set would exceed the 4MB budget, shrunk together rather than dropped; the text part lists each tile and says so. `hear=N` also returns N seconds of soundtrack centred on each `at` moment, after its image - the way to check a hit point or lip-sync without reconciling two clocks; a mute clip keeps its frames and says `no soundtrack` |
-| `get_output_text(name, max_characters=20000, workspace=None)` | `name`, `max_characters`, `workspace` | Read a text output — a prompt enhancement, or any step whose result is `text/plain` or JSON. Reports the file's real length and whether it was truncated. `workspace` names the workspace for this one call without switching the session to it - the same pin `run_workflow` takes, so a job run into another workspace stays reachable from the session that queued it |
-| `assess_output(name, probe=None, detail=False, workspace=None)` | `name` (a gallery name or `asset:`), `probe`, `detail`, `workspace` | Measure a finished cut on the server, without queueing: the assessment probes (`analyze_shots`, `analyze_seams`, `analyze_sync_drift`) run in the server process over one decode of the file, beside whatever job holds the GPU. Shot boundaries are the ones its run recorded (the manifest for an output, the `keep_output` sidecar for an asset). The answer merges every applicable probe's `findings` (`{rule, severity, at, value, threshold, says}`) with `rules_applied`, `rules_skipped` and `not_applicable` (`{probe: why}` - a still, a mute file, a file with no recorded shots); `detail=true` adds each probe's full body under `probes`, and `probe=` returns that one probe's full body. `probe` is checked against the three names before anything else is read, and an unknown one is refused naming them. Findings are places to look, not verdicts: drill in with `get_output_frames(seams=[n])` / `get_output_audio(start, duration)`. See *Assessing a run's output* in docs/WORKFLOW_GUIDE.md |
-| `compare_outputs(names, at=None, crop=None, baseline=None, max_dimension=384, workspace=None)` | `names`, `at`, `crop`, `baseline`, `max_dimension`, `workspace` | Read-only, queues nothing: 2-8 outputs (gallery names or `asset:`) as one grid, a row per output (labelled with its row number and short name, `(baseline)` on the baseline) and a column per moment, each cell stamped `frame N`, with PSNR, SSIM and mean absolute difference against `baseline` (default the first name). `at` is 1-8 moments (seconds or `"frame:N"`, never negative: unlike `get_output_frames`, a negative moment is a 400, not counted back from the end); they resolve to one frame index against the baseline (the first video if the baseline is a still) and every output is read at that same frame number; default 4 evenly spaced frames of the shortest video. A frame past an output's end is a blank cell and its metrics cell says `past end (N frames)`; `fps_differs` flags videos with different frame rates, where the same frame number is not the same second. All stills: one column, `at` ignored with a note; a still among videos repeats in every column. Metrics are per non-baseline output at source resolution: `psnr_db` (`identical` when equal), `ssim` (luma, 11x11 Gaussian window, sigma 1.5) and `mean_abs_diff` (0-255), per column and as mean/min/max. `crop` is `[x,y,w,h]`: one shared box is the region shown and measured (`region` reads `crop [...]`); a list of one box per name is display only and metrics stay on the full frame (no crop is `full frame`). A crop must lie wholly inside every output's frame, else 400 (stricter than `get_output_frames`, which clamps). An output whose size differs from the baseline gets null metrics with the reason `size differs from baseline: WxH vs WxH` - it is never resized. `max_dimension` is the longest side of each cell; the grid is held to 4 MiB and shrunk whole when over (never below 64 px per cell), `downscaled_to` null when nothing shrank. Returns the grid image and a text summary. Errors: 400 bad shape, out-of-bounds crop, every moment past every end, or not an image or video; 404 unknown name; 413 over the decode pixel limit; 422 undecodable file. See *Assessing a run's output* in [the workflow guide](WORKFLOW_GUIDE.md#assessing-a-runs-output) |
-| `download_output(name, destination=None, overwrite=False, workspace=None)` | `name`, `destination`, `overwrite`, `workspace` | Save one output file to local disk, of any content type. `destination` may be a full path or a directory; `~` expands and missing parent directories are created. `overwrite=True` is required to replace a file already at the resolved path. Over the stdio `dw-mcp`, omitting `destination` saves under the output's own name in the current working directory. Over a `dw.serve --mcp` endpoint the file lands on the server confined to that workspace (a relative path is joined onto it), and `destination` is required there - an omitted one is refused rather than dropped loose in the workspace root, where nothing can find or delete it later (#353); use the `url` `list_gallery` reports, `get_output_image`/`get_output_audio`/`get_output_frames` for inline content, or `keep_output` to make it a named asset instead. Returns nothing to the conversation but where the file landed — unlike the other media tools, the point is a file on disk, not a payload in context. Writes on the machine running the MCP server - over `dw.serve --mcp` that is the GPU box. A write that fails there (a path that exists only on the client, for instance) comes back as an error naming the server-side write and the client-side alternatives, not as an anonymous tool failure. `workspace` names the workspace for this one call without switching the session to it - the same pin `run_workflow` takes, so a job run into another workspace stays reachable from the session that queued it |
-| `delete_output(name=None, workspace=None, job_id=None)` | exactly one of `name` / `job_id`, `workspace` | Permanently remove one generated file from the output directory, or - with `name` a `/` run directory, or with `job_id` - a whole run. By `job_id` the server reads the run directory from the job record and the root the job ran in (`DELETE /api/jobs/{id}/run`), and the reply is `job_id`, `run_dir`, `deleted` and `run_swept`; a job that never wrote a run directory, an unknown one, or one still queued or running is an error. `workspace` names the workspace for a `name` delete without switching the session to it - the same pin `run_workflow` takes, so a job run into another workspace stays reachable from the session that queued it |
+| Tool | Purpose |
+| --- | --- |
+| `get_output_image(name, max_dimension=768, workspace=None, crop=None)` | A generated image, downscaled to `max_dimension` on its longest side, with a text part reporting `original_size`, `returned_size` and `bytes`. `crop` is `[x, y, width, height]` in the original's pixels, cut before the downscale and reported back clamped |
+| `get_output_audio(name, start=None, duration=None, workspace=None)` | A soundtrack (an audio output, or a video's muxed track) as base64: its own encoding when whole, WAV when excerpted. A whole clip over the 4MB budget is refused, not cut (#204); ask for `start`/`duration` in seconds, and the text part names the excerpt (`excerpt: 2.0s from 10.0s of 240.0s`) |
+| `get_output_frames(name, at=None, seams=None, count=None, boundaries=None, names=None, max_dimension=512, hear=None, workspace=None, crop=None)` | A video as frames (#193). One selector per call: `count` (an evenly spaced contact sheet), `at` (seconds or `"frame:N"`), or `seams` (true, or seam numbers from 1: the frame pair across each join, with `difference`, the mean pixel change 0-255). `boundaries` and `names` modify `seams` only; passing either with `at` or `count` is refused. On an output joined from shots, `seams` reads the recorded `media.shots`; for any other file pass `boundaries` (each later shot's first frame) and `names`. `crop` is `[x, y, width, height]` in source pixels, cut before any downscale. Over the 4MB budget, tiles shrink together. `hear=N` adds N seconds of soundtrack centred on each `at`; a mute clip says `no soundtrack` |
+| `get_output_text(name, max_characters=20000, workspace=None)` | A text output (a prompt enhancement, any `text/plain` or JSON result), with its real length and whether it was truncated |
+| `assess_output(name, probe=None, detail=False, workspace=None)` | Measures a finished cut in the server process without queueing: `analyze_shots`, `analyze_seams` and `analyze_sync_drift` over one decode, using the shot boundaries its run recorded (the manifest, or the `keep_output` sidecar for an asset). Returns merged `findings` (`{rule, severity, at, value, threshold, says}`) with `rules_applied`, `rules_skipped` and `not_applicable`; `detail=true` adds each probe's body under `probes`; `probe=` returns one probe's body, and an unknown name is refused first. Findings are places to look, not verdicts. See [Assessing a run's output](WORKFLOW_GUIDE.md#assessing-a-runs-output) |
+| `compare_outputs(names, at=None, crop=None, baseline=None, max_dimension=384, texture_bands=False, workspace=None)` | 2-8 outputs (gallery names or `asset:`) as one grid, a row per output and a column per moment, with PSNR (`psnr_db`), SSIM (luma, 11x11 Gaussian, sigma 1.5) and `mean_abs_diff` (0-255) against `baseline` (default the first name), per column and as mean/min/max. `at` is 1-8 moments (seconds or `"frame:N"`; negative is a 400), resolved to one frame index against the baseline and read at that frame in every output; default 4 evenly spaced frames of the shortest video. A frame past an output's end is a blank cell (`past end (N frames)`); `fps_differs` flags mismatched rates. One shared `crop` `[x,y,w,h]` is shown and measured; a list of one box per name is display only. A crop must lie wholly inside every frame (400). A size mismatch gives null metrics, never a resize. The grid is held to 4 MiB (never below 64 px a cell). `texture_bands=true` adds RMS texture energy per band (`fine` 2-8 px, `mid` 8-32, `coarse` 32-128) for `luma`, `chroma_rg`, `chroma_bg`, plus `texture_ratio` against the baseline; it needs one shared crop of at least 16 px a side, or none, and allows a single name. Errors: 400 bad shape, crop out of bounds, every moment past every end, or not an image or video; 404 unknown name; 413 over the decode pixel limit; 422 undecodable |
+| `download_output(name, destination=None, overwrite=False, workspace=None)` | Saves one output, any type, to disk on the machine running the MCP server, and returns only where it landed. `destination` is a path or a directory (`~` expands, parents are created); a `..` segment is refused; an existing file needs `overwrite=True`. Over stdio, no `destination` saves under the output's name in the working directory. Over `dw.serve --mcp` the file lands on the server, confined to the workspace, and `destination` is required (#353); use the entry's `url`, the inline tools, or `keep_output` instead |
+| `delete_output(name=None, workspace=None, job_id=None)` | Permanently removes one generated file, or a whole run: `name` as `/`, or `job_id` (`DELETE /api/jobs/{id}/run`, which answers `job_id`, `run_dir`, `deleted`, `run_swept`). Exactly one of the two. A job with no run directory, an unknown one, or one still queued or running is an error. A `job_id` delete goes to the workspace the job ran in |
### Authoring, assets and workspaces
-Authoring happens inside one workspace. A server can hold several - each with
-its own `workflows/`, `assets/` and `outputs/`, all sharing one prompt library
-- and `use_workspace` picks the one this session reads and writes for the rest
-of its life. That is how two agents work against one GPU without saving over
-each other; see [Workspaces](WORKSPACES.md#several-workspaces-on-one-server).
-The session starts in `default` and stays there unless it is told otherwise.
-
-The pin above is per-*process* state on `client.py`'s `DwClient`, which is
-one session for the local stdio `dw-mcp`, but `dw.serve --mcp` (see
-[REMOTE.md](REMOTE.md#claude-code-no-local-install)) builds a single client
-for every agent it serves - so on that mounted surface the pin is shared by
-every concurrently connected caller, not scoped to any one of them (#298).
-A second client's `use_workspace`/`create_workspace(use=true)` can switch
-what your session reads and writes without your session calling either. This
-server is single-user by design, so nothing tracks distinct MCP sessions to
-fix that; `use_workspace`/`create_workspace` add a `warning` field to their
-result when the pin they are about to overwrite was already pointed
-somewhere other than `default` (a sign another client may depend on it), but
-they cannot warn the *other* session whose pin just moved out from under it.
-A caller that needs real isolation on a mounted server - the tester and
-regression harnesses among them - must pass `workspace=` on every call that
-accepts it (`validate_workflow`, `run_workflow`, and most of the read/media
-tools) rather than relying on the session pin.
-
-| Tool | Arguments | Purpose |
-| --- | --- | --- |
-| `validate_workflow(workflow=None, name=None, workspace=None, arguments=None)` | exactly one of `workflow` (inline definition) or `name` (a stored workflow, as `list_workflows` reports it), optional `workspace`, optional `arguments` | Check a workflow against the schema and against real pipeline signatures. Free and instant. Validating by name uses the workflow file's own directory as the base directory, so it sees what a run would. Returns every schema violation in `errors`, each with the JSON path it sits at, so a draft is fixed in one pass, and a `previous_result:` that names no earlier step is one of them. `warnings` covers what still runs but is probably wrong - a signature mismatch, and, for a list-driven variable, an entry key no step reads, at the entry's path. `workspace` names the workspace for this one call without switching the session to it - use it to pin a job whose `output:` or `asset:` references live in a workspace other than the session's. Pass the same `arguments` you will pass to `run_workflow` and they are checked too - an undeclared or renamed variable name, a value that will not coerce to the declared type, and an `asset:`, `prompt:` or `output:` reference that names nothing this workspace can reach, each reported at `arguments.`. `checked_arguments` lists what was covered, so a `valid: true` about the stored defaults cannot be mistaken for one about your values. A reference set the model would refuse - too many images, videos or audio clips, or, for MiniMax-H3, audio as the only reference - is an error here too, rather than a failure minutes into a run you acknowledged. `run_workflow` makes the same check and refuses a bad argument rather than queuing a job that fails on its first step. A valid answer carries `plan` - the fingerprint, step count, list lengths, `downloads_required` (pipeline `model_name`s and the repos and single files task steps fetch), `estimate` (with `basis`), `host_memory` (`{projected_mb, basis, runs}`, the projected host-RAM peak; `basis` is `observed` from same-key successful runs on this machine or `fit`; absent when there is no figure) and `elided_steps` for the arguments given; quote from it. `steps` counts what will run: a step nothing reads and which saves no file does not run, and is named in `elided_steps` instead |
-| `list_workspaces()` | — | The server's workspaces and which one this session is using. Each has its own workflows, assets and outputs; the prompt library is shared by all of them. Compact entries are `name`/`default`/`usage` plus any `note` (clipped, `note_truncated` when cut); `detail=true` adds each entry's folder paths |
-| `use_workspace(name)` | `name` | Work in that workspace for the rest of the session - every later call reads and writes there. This is how to keep your work out of another agent's namespace rather than sharing the default one. Checked against the server, so a typo fails here rather than scoping every later call to nothing. On a `dw.serve --mcp` endpoint the pin is shared by every connected client (#298, see the note above the table) - a `warning` field appears when this call overwrote a pin already pointed away from `default` |
-| `create_workspace(name, use=False)` | `name`, `use` | Create a workspace. Pass use=true to switch this session to it as well; otherwise the session stays where it was and the result says so. The result names the workspace (`name`, `default`, `current`, `next`), not its folders - `list_workspaces(detail=true)` is the opt-in for those. `use=true` carries the same mounted-server caveat and `warning` field as `use_workspace` |
-| `copy_to_workspace(destination, runs=None, assets=None, source=None)` | `destination`, `runs`, `assets`, `source` | Copy runs and assets from one workspace into another. Each `runs` entry is `/` as `list_gallery` names it, or a bare `` for all its runs; each `assets` entry is a path as after `asset:`, or a folder (`proj/`) for every file under it, keeping its path, note and kept shots. `source` defaults to this session's workspace. Free: no `acknowledged_cost`. Media is hard-linked (a copy across devices falls back to a real copy) and JSON sidecars are always copied, so an edit in one workspace never reaches the other. All or nothing; a run or identical asset the destination already has is skipped as `already_present`, a different file at an asset's path refuses the whole request, so a repeat is idempotent, and a `vN` it already uses is renumbered after its highest and reported in `renumbered`. Warnings: `unresolved_output_refs`, `absolute_paths`, `no_jobs`. A copied run has no job record - it shows in `list_gallery` and resolves as an `output:` reference, not in `list_jobs`. A run whose job is still queued or running is refused. To move, copy then `delete_output` the source runs and `delete_asset` the source assets. See [Workspaces](WORKSPACES.md#copying-between-workspaces) |
-| `delete_workspace(name, acknowledged_cost=False)` | `name`, `acknowledged_cost` | Permanently delete a workspace and everything in it. Refuses without the acknowledgement, reporting what it would remove |
-| `list_assets(limit=50, prefix)` | `limit`, `prefix` | The input media on the server, each with the `asset:` reference a workflow argument carries. Look here before asking for a file - what a workflow needs may already be there. `libraries` names the roots searched and which are writable; `shadowed` lists names a nearer library hides. Compact entries are `name`/`reference`/`kind`/`size`/`origin` plus any `note` (clipped, `note_truncated` when cut); `detail=true` adds `folder`, `mtime`, `url`, `writable`, and `absolute_url`/`used_by` when set. Newest first; `prefix` keeps names starting with it (e.g. `cast/`), `limit` bounds the answer, `total` is the count matched before the cut |
-| `keep_output(name, asset_name=None, overwrite=False, shared=False, workspace=None, note=None)` | `name`, optional `asset_name`, `overwrite`, `shared`, `workspace`, `note` | Keep a generated file as an input asset under a stable `asset:` name, so a later workflow can rely on it. The copy happens on the server: nothing is downloaded or re-uploaded. `note` is kept with the asset; left out, the source file's note is copied. `asset_name` may name a folder and takes the kept file's extension when it has none; `shared=true` keeps it in the library every workspace shares, which is where a recurring cast belongs. `workspace` names the workspace for this one call without switching the session to it - the same pin `run_workflow` takes, so a job run into another workspace stays reachable from the session that queued it |
-| `set_note(target, note=None, workspace=None)` | `target` (a run `/`, a gallery file name, `asset:` or `workspace:`), `note`, `workspace` | Set the note on a run, an output file, an asset or a workspace; null or `""` clears it. Plain text up to 4000 characters saying what the thing is for - what a run tested, why a file is the keeper. Free: no `acknowledged_cost`. Returns `{target, note, note_source}`. `run_workflow(note=...)` sets a run's note at submission and it shows on every file the run wrote; `list_gallery`, `list_assets`, `list_workspaces` and `list_jobs` carry each note clipped to 200 characters (`note_truncated: true` when cut), while `get_job` and `get_gallery_metadata` return it whole. `get_job` shows the run's current note, so one set here after submission replaces the submitted one. `workspace` names the workspace a run or file name is read in for this one call without switching the session to it - the same pin `run_workflow` takes |
-| `upload_asset(file_path=None, content=None, asset_name=None, shared=False, note=None)` | exactly one of `file_path` or `content` (base64), optional `asset_name`, `shared`, `note` (kept with the asset) | Push an image, video or audio file, or a `.cube` 3D LUT for `apply_lut`, into the server's asset library and get back its `asset:` reference. `file_path` is read from the machine the MCP server runs on, so this is how an input reaches a dw.serve running somewhere else - but only when the caller shares a filesystem with that machine. `content` is the alternative for an agent that does not: the bytes travel inline, base64-encoded, in the tool call itself, capped at 4MB (well under `file_path`'s 200MB) because inline bytes compete with the calling agent's own context budget rather than being a bulk-transfer path (#203). `asset_name` is required with `content` (there is no file name to infer one from) and, either way, stores it under a readable name (`cast/priya-voice.wav`) instead of a random one; `shared=true` puts it in the library every workspace shares |
-| `delete_asset(name)` | `name` | Permanently remove one file from the asset library, by the name `list_assets` reports. Deletes from whichever library holds it - this workspace's own before the shared one; one from a read-only examples library is refused. Any workflow still carrying that `asset:` reference stops loading |
-| `save_workflow(name, workflow)` | `name`, `workflow` | Save a workflow into the server's writable workflow directory, overwriting any existing workflow of that name there. A name that currently resolves to a read-only source (an examples directory) is not overwritten - the copy lands in the writable directory and shadows it |
-| `delete_workflow(name)` | `name` | Permanently delete a stored workflow |
+A server can hold several workspaces, each with its own `workflows/`,
+`assets/` and `outputs/`, all sharing one prompt library and one shared asset
+library. `use_workspace` sets the one this session reads and writes; a session
+starts in `default`. See
+[Workspaces](WORKSPACES.md#several-workspaces-on-one-server).
+
+The pin is state on `client.py`'s `DwClient`. A stdio `dw-mcp` has one per
+session, but `dw.serve --mcp` builds one client for every agent it serves, so
+on that surface the pin is **shared by every connected caller** (#298): another
+client's `use_workspace` or `create_workspace(use=true)` moves your pin
+without your session calling either. Both tools add a `warning` to their
+result on a mounted server when they overwrite a pin that pointed somewhere
+other than `default`; the other session gets no warning. A caller that needs
+isolation on a mounted server passes `workspace=` on every call that takes it.
+
+| Tool | Purpose |
+| --- | --- |
+| `validate_workflow(workflow=None, name=None, inline_workflow=None, workflow_path=None, workspace=None, arguments=None)` | Free and instant: schema, real pipeline signatures, references and arguments. Exactly one of `workflow` (inline, an object or a JSON-encoded string) or `name` (a stored workflow); `run_workflow`'s spellings `inline_workflow` and `workflow_path` are accepted too. By name, the workflow file's own directory is the base directory, as a run sees it. `errors` lists every violation with its JSON path (a `previous_result:` naming no earlier step among them); `warnings` covers what runs but is probably wrong (a signature mismatch, a list entry key no step reads, a value the workflow rounds up). Pass the `arguments` you will run with: an undeclared name, a value that will not coerce, a value outside a declared bound, and an `asset:`/`prompt:`/`output:` reference that reaches nothing are each reported at `arguments.`; `checked_arguments` says what was covered. Also checked: a reference set the model would refuse (too many images, videos or audio clips; for MiniMax-H3, audio as the only reference), an unwritable `result.subfolder` or `file_base_name`, and each sub-workflow step. A valid answer carries `plan`: `fingerprint`, step count, list lengths, `downloads_required`, `estimate` (with `basis`), `host_memory` (`{projected_mb, basis, runs}`) and `elided_steps` (steps nothing reads and that save no file, which do not run). `plan` is null when it could not be built |
+| `list_workspaces(detail=False)` | The workspaces and which one this session uses. Entries carry `name`, `default`, `usage` and any `note` (clipped, `note_truncated`); `detail=true` adds each entry's folder paths |
+| `use_workspace(name)` | Work in that workspace for the rest of the session. Checked against the server, so a typo fails here. On a mounted server the pin is shared (above) |
+| `create_workspace(name, use=False)` | Create a workspace; `use=true` also switches to it. The result names the workspace (`name`, `default`, `current`, `next`), not its folders |
+| `copy_to_workspace(destination, runs=None, assets=None, source=None)` | Copy runs (`/`, or `` for all its runs) and assets (a path as after `asset:`, or a folder `proj/`, keeping path, note and kept shots) into another workspace. `source` defaults to the session's workspace. No `acknowledged_cost`. Media is hard-linked (a real copy across devices), JSON sidecars are copied. All or nothing: what the destination already has is skipped as `already_present`, a different file at an asset's path refuses the whole request, a `vN` in use is renumbered (`renumbered`). Warnings: `unresolved_output_refs`, `absolute_paths`, `no_jobs`. A copied run has no job record. A run whose job is queued or running is refused. To move, copy then `delete_output` / `delete_asset`. See [Workspaces](WORKSPACES.md#copying-between-workspaces) |
+| `delete_workspace(name, acknowledged_cost=False)` | Permanently delete a workspace and everything in it. Without the acknowledgement the server refuses (409), reporting what it would remove |
+| `list_assets(detail=False, workspace=None, limit=50, prefix=None)` | Input media on the server, each with its `asset:` reference, newest first. Entries carry `name`, `reference`, `kind`, `size`, `origin` and any `note`; `detail=true` adds `folder`, `mtime`, `url`, `writable`, and `absolute_url`/`used_by` when set. `prefix` keeps names starting with it (`cast/`); `total` counts all matches. `libraries` and `shadowed` are as in `list_workflows` |
+| `keep_output(name, asset_name=None, overwrite=False, shared=False, workspace=None, note=None)` | Keep a generated file as an input asset under a stable `asset:` name, copied on the server. `asset_name` may name a folder and takes the file's extension when it has none; `shared=true` keeps it in the shared library, where a recurring cast belongs. `note` defaults to the source file's note |
+| `set_note(target, note=None, workspace=None)` | Set the note (plain text, up to 4000 characters) on a run (`/`), an output file, `asset:` or `workspace:`; null or `""` clears it. Returns `{target, note, note_source}`. Listings clip notes to 200 characters (`note_truncated`); `get_job` and `get_gallery_metadata` return them whole, and `get_job` shows the current note |
+| `upload_asset(file_path=None, content=None, asset_name=None, shared=False, workspace=None, note=None)` | Put an image, video, audio file or `.cube` LUT into the asset library and get its `asset:` reference. Exactly one of `file_path` (read on the machine running the MCP server, up to 200MB) or `content` (base64 inline, up to 4MB, #203; `asset_name` required). The file is stored under the library's `uploads/`, named `asset_name` when given (`cast/priya-voice.wav`, folders allowed, the extension matching the file's kind) and randomly otherwise; `shared=true` uses the shared library |
+| `delete_asset(name, workspace=None)` | Permanently remove one asset, by the name `list_assets` reports, from whichever library holds it (the workspace's own before the shared one). One from a read-only examples library is refused. A workflow still carrying the reference stops loading |
+| `save_workflow(name, workflow=None, patch=None, workspace=None)` | Save into the writable workflow directory, overwriting a workflow of that name there; `name` may include folders. Exactly one of `workflow` (the full document) or `patch` (a JSON Merge Patch, RFC 7396, onto the stored definition: `null` deletes a key, a list is replaced whole). Either may be a JSON-encoded string. A name that resolves to a read-only source is not overwritten: the copy lands in the writable directory and shadows it |
+| `delete_workflow(name, workspace=None)` | Permanently delete a stored workflow. One from a read-only examples directory (`writable: false`) is refused |
### Prompts
-The stored prompt library is the other half of authoring: a workflow
-argument written as `"prompt:name"` or `"prompt:folder/name"` resolves
-against it at load time, so a workflow can be authored and the text it
-references written in the same session.
+A workflow argument written `"prompt:name"` or `"prompt:folder/name"` resolves
+against the stored prompt library at load time, so a workflow and the text it
+references can be written in the same session. This group also holds the
+enhancers and the LoRA catalog.
-| Tool | Arguments | Purpose |
-| --- | --- | --- |
-| `list_prompts(tag=None, intended_model=None, include_text=False)` | optional `tag`, `intended_model`, `include_text` | List the stored prompts - description, intended model, tags and `text_chars`, bodies left out; `get_prompt` for one body Each entry carries `origin` and `writable`; `libraries` and `shadowed` are as in `list_workflows` |
-| `get_prompt(name)` | `name` | Get one stored prompt's full definition |
-| `get_prompt_schema()` | — | Get the JSON schema every stored prompt must satisfy. Its own route rather than a name under `/api/prompts`, so a prompt called `schema` cannot shadow it |
-| `save_prompt(name, prompt)` | `name`, `prompt` | Save a prompt, overwriting any prompt of that name. The server validates first, and refuses a `text` that itself begins with a reference prefix (`variable:`, `previous_result:`, `constant:`, `asset:`, `output:`, `prompt:`) |
-| `delete_prompt(name)` | `name` | Permanently delete a stored prompt. A workflow still referencing it will fail to load |
-| `list_enhancers()` | — | List the enhancer presets `enhance_prompt` accepts |
-| `enhance_prompt(idea, preset="h3", model_name=None, device=None, acknowledged_cost=False)` | `idea`, `preset`, optional `model_name` and `device`, `acknowledged_cost` | Expand a short idea into a full prompt with a language model. Queued as an ordinary job, so it passes the gate; the enhanced text is the text file in the finished manifest, readable with `get_output_text` |
-| `list_loras(model=None, workflow=None, status=None, tag=None)` | optional `model`, `workflow`, `status`, `tag` | List the LoRAs tried on a base model - proven, trial or rejected - with `use_when`, `trigger` and `scale`; exact match on the base. Guide: `loras` |
-| `save_lora(name, entry)` | `name`, `entry` | Save a catalog entry, overwriting any entry of that name; promote a trial to `proven` with its job in `evidence` |
-| `recommend_loras(model, query, limit=8)` | `model`, `query`, optional `limit` | Opt-in: catalog LoRAs ranked against a style request, then Hugging Face Hub adapters of that exact base. Queries the Hub (the one tool with `open_world_hint`); no download, no GPU. Hub rows are candidates to trial. A failed or busy Hub (another search already running) returns the catalog rows plus `hub_error`; an empty `hub` with `hub_error` means the search failed |
+| Tool | Purpose |
+| --- | --- |
+| `list_prompts(tag=None, intended_model=None, include_text=False)` | Stored prompts: `description`, `intended_model`, `tags`, `text_chars`, `origin`, `writable`; `libraries` and `shadowed` as in `list_workflows`. Bodies only with `include_text=true`, which for the whole library is too large; filter first |
+| `get_prompt(name)` | One stored prompt's full definition |
+| `get_prompt_schema()` | The JSON schema a stored prompt must satisfy, on its own route so a prompt named `schema` cannot shadow it |
+| `save_prompt(name, prompt)` | Save a prompt, overwriting one of that name. The server validates it, and refuses a `text` that begins with a reference prefix (`variable:`, `previous_result:`, `constant:`, `asset:`, `output:`, `prompt:`) |
+| `delete_prompt(name)` | Permanently delete a stored prompt. A workflow referencing it fails to load |
+| `list_enhancers()` | The presets `enhance_prompt` accepts |
+| `enhance_prompt(idea, preset="h3", model_name=None, device=None, acknowledged_cost=False)` | Expand a short idea into a full prompt with a language model. Queued as an ordinary job behind the cost gate; the text is the text file in the finished manifest (`get_output_text`) |
+| `list_loras(model=None, workflow=None, status=None, tag=None)` | LoRAs tried on a base model (proven, trial or rejected) with `use_when`, `trigger` and `scale`; exact match on the base. Guide: `loras` |
+| `save_lora(name, entry)` | Save a catalog entry, overwriting one of that name; promote a trial to `proven` with its job in `evidence` |
+| `recommend_loras(model, query, limit=8)` | Catalog LoRAs ranked against a style request, then Hugging Face Hub adapters of that exact base. The one tool that queries the Hub (`open_world_hint`); no download, no GPU. A failed or busy Hub returns the catalog rows plus `hub_error` |
### Diagnose
-| Tool | Arguments | Purpose |
-| --- | --- | --- |
-| `run_workflow(workflow_path=None, inline_workflow=None, arguments=None, acknowledged_cost=False, workspace=None, wait_seconds=0, note=None)` | exactly one of `workflow_path` (a catalog name from `list_workflows`, with or without `.json`, or a path to a workflow file on the server) or `inline_workflow`, optional `arguments`, `acknowledged_cost`, `workspace`, `wait_seconds`, `note` | Queue a workflow for generation. Returns as soon as the job is queued - unless `wait_seconds` is above 0, in which case the call then waits on the queued job exactly as `wait_for_job(job_id, timeout_seconds=wait_seconds)` would (same per-call cap, clamped not honoured) and the result carries the queued-job fields plus that wait's (`status`, `still_running`, `waited_seconds`, `timeout_requested_seconds`, `timeout_applied_seconds`, `timeout_capped`, the slim `job`); when the cap covers the job's runtime one call is the run and the wait, and a `still_running: true` result is followed with `wait_for_job` as before. `workspace` names the workspace for this one call without switching the session to it - use it to pin a job whose `output:` or `asset:` references live in a workspace other than the session's - `acknowledged_cost` is `true` or the bound `{fingerprint, minutes, downloads}` from the validate plan; a 409 means the plan changed and the message carries the new estimate, and nothing is waited on. `note` says what the run is for (see `set_note`) |
-| `get_job(job_id)` | `job_id` | Get a job's status, warnings, output manifest, error and traceback; each manifest entry's `subfolder` is the in-run subfolder the step declared - by convention `final` for the deliverable, `intermediate` for scratch, `''` for none. A running job also carries `progress` (below); a queued job held for host RAM carries `waiting_for` (#869) |
-| `get_job_workflow(job_id)` | `job_id` | The REST equivalent is `GET /api/jobs/{id}/workflow` (see [SERVER.md](SERVER.md#jobs-api)). The workflow the job actually ran. `realized: true` means every mutable input is pinned (arguments, seed, prompts, `output:latest`); `false` means the job predates run tracking and this is the definition as submitted. Pass it to `save_workflow` to keep it under a name |
-| `export_job(job_id, overwrite=False)` | `job_id`, `overwrite` | Gather one finished job into `/exports//` on the server: the realized workflow, the run's manifest, the job row, a README, and copies of the assets, earlier-run inputs and outputs. Returns the directory, a zip URL, the file list with sizes and the total. The three JSON files are in the zip, not repeated here - get_job_workflow and get_job serve them individually. **The directory is on the machine running the server**, like `download_output`'s destination. The zip needs no token (`/exports/*.zip` is ungated like `/outputs`, #592), so `auth_required` is false and the agent fetches `open_url` itself (prefixing a relative one with the server address) and unpacks it into `exports/` under the session's working directory (a deliverable, not a temp file) - the archive already unpacks into one folder named after the job id, so do not create that folder first; only an agent without HTTP gives the user `open_url`. `true` is kept as a forward guard for a gated zip, when `open_url` is handed to the person instead |
-| `get_job_events(job_id, after=-1, limit=200)` | `job_id`, `after`, `limit` | Get a page of a job's progress events |
-| `wait_for_job(job_id, timeout_seconds=20)` | `job_id`, `timeout_seconds` | Block until a job reaches a terminal status, or `timeout_seconds` elapses. **One call blocks for at most the server's cap** — 55 seconds unless the deployment sets `DW_MCP_MAX_WAIT_SECONDS` higher, and the tool's description states the live value; a larger `timeout_seconds` is clamped, not honoured. Ask for the job's `plan.estimate` plus a margin: under the cap that is one call for the whole job, over it one call per cap. A deployment raises the cap only as far as its clients (and anything between them and the server) hold one silent HTTP request open - Claude Code's limit is `MCP_TOOL_TIMEOUT`. Every reply carries `waited_seconds`, `timeout_requested_seconds`, `timeout_applied_seconds` and `timeout_capped`, so a capped return is distinguishable from an elapsed one. Use instead of hand-polling `get_job`/`get_job_events` in a loop; if it returns `still_running: true`, call it again. Returns a slim job - status, warnings, error, `run_id` and `run_version` (the run's `v5`, as the gallery labels it), and the manifest once finished - without the arguments; `get_job` has those. A running job also carries `progress` (below); a queued job held for host RAM carries `waiting_for` |
-| `cancel_job(job_id)` | `job_id` | Ask a queued or running job to stop |
-| `clear_memory()` | `device` | Drop every loaded pipeline and the step cache on each idle card - `device`'s alone when named - freeing VRAM/RAM immediately instead of waiting for the next job to evict one model for another. Also drops the step cache, so a seeded workflow that would otherwise reuse cached results regenerates on its next run. A card running a job is skipped (`workers` reports it `job_running`); refused with a 409 when no card asked about is idle, so wait and retry rather than expecting this call to block (#221, #677) |
-| `rerun_job(job_id, acknowledged_cost=False, new_seed=False)` | `job_id`, `acknowledged_cost`, `new_seed` | Queue a fresh job from a previous job's stored specification. Costs GPU time, so it passes the same gate as `run_workflow`. `new_seed=true` draws a fresh seed into the workflow's seed variable — without it a seeded workflow's rerun repeats its arguments exactly and the step cache serves the whole run from the earlier one's files (`reused: true`), generating nothing. `get_job_workflow`'s `seed_variable` says whether there is one - `acknowledged_cost` is `true` or the bound `{fingerprint, minutes, downloads}` from the validate plan; a 409 means the plan changed and the message carries the new estimate |
-| `move_job(job_id, direction)` | `job_id`, `direction` (`up`\|`down`\|`front`\|`back`) | Reorder a queued job |
+| Tool | Purpose |
+| --- | --- |
+| `run_workflow(workflow_path=None, inline_workflow=None, workflow=None, name=None, arguments=None, acknowledged_cost=False, workspace=None, wait_seconds=0, note=None)` | Queue a workflow. Exactly one of `workflow_path` (a catalog name from `list_workflows`, with or without `.json`, or a path on the server) or `inline_workflow` (an object or JSON-encoded string); `validate_workflow`'s spellings `name` and `workflow` are accepted too. `arguments` overrides variables by name and is checked as `validate_workflow` checks it; a bad one is refused before queueing. `acknowledged_cost` is `true` or the bound `{fingerprint, minutes, downloads}` from the validate plan; a 409 means the plan changed and carries the new estimate. Returns once queued, unless `wait_seconds` > 0: then it waits as `wait_for_job(job_id, timeout_seconds=wait_seconds)` would, under the same cap, and adds that wait's fields. `note` says what the run tests (see `set_note`) |
+| `get_job(job_id)` | Status, warnings, output manifest, error and traceback. Each manifest entry's `subfolder` is the one the step declared. A running job carries `progress`; a job held for host RAM carries `waiting_for` (#869) |
+| `get_job_workflow(job_id)` | The workflow the job ran (`GET /api/jobs/{id}/workflow`, [SERVER.md](SERVER.md#jobs-api)). `realized: true` pins every mutable input (arguments, seed, prompts, `output:latest`); `false` is the submitted definition of a job from before run tracking. `seed_variable` names the variable `rerun_job(new_seed=true)` draws into. Pass it to `save_workflow` to keep it |
+| `export_job(job_id, overwrite=False)` | Gather a finished job into `/exports//` **on the server machine**: the realized workflow, the manifest, the job row, a README, and copies of its assets, earlier-run inputs and outputs (`total_bytes` is what was copied). Returns the directory, a zip URL, the file list and the total. The zip `open_url` needs no token (#592): fetch it (prefixing a relative URL with the server address) and unpack it into `exports/` in the session's working directory; it unpacks into one folder named after the job id. `auth_required: true` would mean a gated zip, to hand to the person. Refuses a running job, and an existing export without `overwrite=true` |
+| `get_job_events(job_id, after=-1, limit=200, kinds=None)` | A page of progress events. `after` is exclusive: pass back `last_seq`. Each event's `at` is seconds since the job started. `kinds` (e.g. `["log", "warning"]`) filters the page, which `memory` events otherwise dominate. A `phase_stall` entry is a watchdog notice every ~30 s of a quiet phase, not evidence of a hang |
+| `wait_for_job(job_id, timeout_seconds=20)` | Block until the job is terminal or `timeout_seconds` elapses. One call blocks for at most the server's cap (`DW_MCP_MAX_WAIT_SECONDS`; the tool's description states the live value); a larger timeout is clamped, and every reply carries `waited_seconds`, `timeout_requested_seconds`, `timeout_applied_seconds` and `timeout_capped`. On `still_running: true`, call again. A deployment raises the cap only as far as its clients hold a silent HTTP request open (Claude Code: `MCP_TOOL_TIMEOUT`). Returns a slim job: status, warnings, error, `run_id`, `run_version`, the manifest once finished, `progress` while running, `waiting_for` when held |
+| `cancel_job(job_id)` | Ask a queued or running job to stop. A running job stops at the next denoise or step boundary |
+| `clear_memory(device=None)` | Drop every loaded pipeline and the step cache on each idle card (`device`'s alone when named), so a seeded workflow regenerates on its next run. A busy card is skipped (`workers` reports `job_running`); a 409 when no card asked about is idle (#221, #677) |
+| `rerun_job(job_id, acknowledged_cost=False, new_seed=False)` | Queue a fresh job from a previous job's stored specification, behind the same gate as `run_workflow`. Without `new_seed=true` a seeded rerun repeats its arguments exactly and the step cache serves it from the earlier files (`reused: true`) |
+| `move_job(job_id, direction)` | Reorder a queued job: `direction` is `up`, `down`, `front` or `back` |
### Models
-| Tool | Arguments | Purpose |
-| --- | --- | --- |
-| `list_models()` | — | (Catalog) List what the Hugging Face cache holds, largest first |
-| `download_model(repo_id, acknowledged_cost=False)` | `repo_id`, `acknowledged_cost` | Fetch a model repo into the cache. Costs disk and bandwidth, so it passes the gate. Returns as soon as the download starts. It fetches the whole repo at `main`, so it does not pre-fetch a single pinned file a task entry names; the task fetches that on its first run |
-| `list_downloads()` | — | List downloads the server is running or recently ran |
-| `cancel_download(download_id)` | `download_id` | Stop a running download; the record shows `cancel_requested: true` until it is `cancelled`. Partial files stay cached and resume on a retry |
-| `delete_model(repo, acknowledged_cost=False)` | `repo`, `acknowledged_cost` | Delete every cached revision of one repo. Not recoverable locally |
-| `get_diffusers_state()` | — | Installed diffusers version, its git commit, and any update in flight |
-| `update_diffusers(acknowledged_cost=False)` | `acknowledged_cost` | Upgrade diffusers to GitHub HEAD in the background |
+`list_models` (above, under Catalog) reports the cache.
+
+| Tool | Purpose |
+| --- | --- |
+| `download_model(repo_id, acknowledged_cost=False)` | Fetch a whole model repo at `main` into the cache; returns once the download starts. It does not pre-fetch a single pinned file a task names; the task fetches that on its first run |
+| `list_downloads()` | Downloads running or recently run |
+| `cancel_download(download_id)` | Stop a running download; the record shows `cancel_requested: true` until `cancelled`. Partial files stay cached and resume on a retry |
+| `delete_model(repo, acknowledged_cost=False)` | Delete every cached revision of one repo. Not recoverable locally |
+| `get_diffusers_state()` | Installed diffusers version, its git commit, and any update in flight |
+| `update_diffusers(acknowledged_cost=False)` | Upgrade diffusers to GitHub HEAD in the background |
The server refuses `delete_model` and `update_diffusers` while a job is
-running or queued, and `delete_model` while a download is active — pulling
-files or package contents out from under a loaded pipeline is the same
-hazard twice. That refusal arrives as the server's own explanation.
+running or queued, and `delete_model` while a download is active, and says
+why.
## The cost gate
-Seven tools refuse unless `acknowledged_cost=true` is passed. Each commits
-the machine to something the user would want to have been asked about first,
-and each says so in its own words — a single shared refusal would be wrong
-for each of them in a different way, and a gate the user learns to wave
-through is not a gate.
+Seven tools refuse unless `acknowledged_cost` is set. Each refuses in its own
+words.
| Tool | What it commits |
| --- | --- |
-| `run_workflow` | Minutes of GPU time; the engine runs one job per GPU, so a run holds a card the queue could give another job |
-| `rerun_job` | The same run, from a stored spec |
+| `run_workflow` | Minutes of GPU time on a card the queue could give another job |
+| `rerun_job` | The same run, from a stored spec; ungated, any job id would bypass the gate |
| `download_model` | Tens of gigabytes of network and disk |
-| `delete_model` | Cached weights, unrecoverably — getting them back means downloading again |
+| `delete_model` | Cached weights, unrecoverably |
| `update_diffusers` | Replacing the installed library with an untagged development build |
-| `enhance_prompt` | A real job on the one-at-a-time engine, delaying any generation behind it |
+| `enhance_prompt` | A real job, delaying any generation behind it |
| `delete_workspace` | Every workflow, asset and generated file in a workspace, unrecoverably |
-`rerun_job` is gated for the same reason as `run_workflow`: it queues the
-identical work, so leaving it open would make the gate worth nothing — any
-job id from `list_jobs` would buy a way around it. `cancel_job` and
-`cancel_download` are deliberately *not* gated: they end a cost rather than
-starting one, and gating them would make the safe direction the harder one.
-
-The acknowledgement can be bound to what was quoted. `validate_workflow`
-answers with a `plan`; pass `acknowledged_cost={"fingerprint":
-plan.fingerprint, "minutes": plan.estimate.minutes, "downloads": [...]}` and
-the server refuses with 409 if the run's shape changed between the quote and
-the call - a longer list, a stored prompt edited meanwhile, weights that now
-have to be downloaded - naming the new plan so the agent re-quotes. The
-`plan` itself, passed unedited, binds the same way: with no `downloads` key,
-its `downloads_required` repos (a file entry by its repo) and `estimate.minutes` stand in. Bare
-`true` still works and is for a `plan` that came back null; the job records
-which form it got (`acknowledged: none | boolean | bound`).
-
-Passing the flag does not make a tool wait. The five that start work return
-as soon as it is queued or started, the same way queuing a job from the web UI
-does not block the browser tab; `delete_model` and `delete_workspace` are
-deletions rather than queued work and complete before they answer.
-
-The intended loop:
-
-1. `validate_workflow` — free, checks schema and pipeline signatures, no GPU
- time spent. Pass the `arguments` you intend to run with: without them the
- verdict covers the stored definition and its stock defaults, not the
- values you wrote, and its `plan` is the number to say out loud:
- `estimate.minutes` with its `basis`, plus each `downloads_required`
- entry as a line item of its own (task steps' repos and single files are included)
-2. `run_workflow` with `acknowledged_cost` bound to the plan (`{fingerprint,
- minutes, downloads}`), or `true` when there was no plan — pass a name straight from
- `list_workflows` as `workflow_path`; queues the job and returns
- immediately with a `job_id`
-3. `wait_for_job(job_id, timeout_seconds=...)` to block instead of
- hand-polling, asking for the plan's estimate plus a margin; one call
- covers at most the server's cap (`timeout_applied_seconds` and
- `timeout_capped` say what you got), so call it again if it comes back
- `still_running: true` — or
- `get_job_events(job_id)` repeatedly, passing back the previous call's
- `last_seq` as `after`, for incremental progress instead of just a
- terminal/not-terminal status. Each event carries `at`, seconds since the
- job started, so where a step's time went is a subtraction between two
- events - `step_start` to `generating` is the lead-in a reused pipeline
- still pays; `generating` is emitted when the pipeline call starts, so
- `generating` to the first `pipeline_step` is the encoding plus the first
- denoise pass (`timing.encode_s` on the manifest entry is the encoding alone)
-4. `get_job(job_id)` for the finished manifest (or the error and traceback,
- if it failed). A pipeline step's entry carries `timing` - load, encode,
- denoise, decode and save seconds ([WORKSPACES.md](WORKSPACES.md#step-timing))
-5. `get_output_image(name)` to look at a result image
-6. `get_output_frames(name, count=12)` to look at a result video, and
- `get_output_audio(name, start, duration)` to hear it
-
-While a job runs, `get_job` and `wait_for_job` carry a `progress` block -
-the step being run, the phase (`loading`, `generating`, `decoding`,
-`saving`) with the model in `phase_detail`, `seconds_in_phase`,
-`seconds_since_event`, and `denoise_step`/`denoise_total_steps`, which are
-null until the denoise loop starts. A single-step generation is minutes of
-one phase, so two polls otherwise come back identical: read `denoise_step`
-moving (slow but healthy) against a `denoise_step` that is a number and
-stays put while `seconds_since_event` climbs (nothing is happening). A null
-`denoise_step` under `generating` is neither - it is the lead-in the
-pipeline runs before the loop, encoding the prompt and every reference and
-then the first denoise pass, with nothing emitted, so silence there is
-expected. On MiniMax H3 the first pass dominates the lead-in, not the encode,
-and it varies by card. The reference encode (`vae_encoder`) is about 6 s for
-an image reference and about 50-53 s for a 5 s 960x544 video reference;
-reference tokens make the first pass slower. Measured on lem: an image
-reference's lead-in was 187 s (job `f90a041af26a`: `timing.encode_s` 26 s,
-first pass 161 s; of the encode, `vae_encoder` was 6 s); the same image +
-video job took 508 s on cuda:1 (`f5aefcdd5e26`: `encode_s` 86 s, first pass
-422 s; `vae_encoder` 59 s) and 165 s on cuda:0 (`6961876f11d3`: `encode_s`
-79 s, first pass 86 s; `vae_encoder` 56 s). Plan for several minutes, up to
-~8.5 min measured (508 s, `f5aefcdd5e26`), with a video reference. `get_job_events` names the block it is in
-while that runs - a `log` line per top-level block of a modular pipeline
-(`MiniMaxAI/MiniMax-H3: vae_encoder`), which is the difference between
-silence and knowing it is encoding the reference. And once the counter is a number the gaps
-between steps are uneven wherever a transformer block cache is configured -
-cheap cached steps, then a full one - so a 140 s gap on H3 is a healthy run;
-read liveness as the counter moving between polls minutes apart rather than
-as silence under a threshold. `cancel_job` stops at the next denoise or
-step boundary, which `denoise_step` is also the measure of.
-
-The other frozen-counter stretch is at the end: under `saving`, `denoise_step`
-sits at a completed-looking `8/8` and cannot move again, because the step is
-writing files. `get_job_events` carries a `log` per file there too - named as
-the write starts (`writing shot.mp4 (121 frames)`) and costed as it finishes
-(`wrote shot.mp4 in 1.3s (1.4 MB)`) - so that stretch is attributable rather
-than silent (#97).
+`cancel_job` and `cancel_download` are not gated: they end a cost.
+
+The acknowledgement can be bound to the quote. Pass
+`acknowledged_cost={"fingerprint": plan.fingerprint, "minutes":
+plan.estimate.minutes, "downloads": [...]}` from `validate_workflow`'s `plan`,
+and the server refuses with 409, naming the new plan, if the run's shape
+changed since (a longer list, an edited stored prompt, weights that now need
+downloading). The `plan` passed unedited binds the same way: with no
+`downloads` key, its `downloads_required` repos and `estimate.minutes` stand
+in. Bare `true` is for a `plan` that came back null. The job records the form
+it got (`acknowledged: none | boolean | bound`).
+
+The flag does not make a tool wait. The five that start work return once it is
+queued or started; `delete_model` and `delete_workspace` complete before they
+answer.
+
+The loop ([The loop](WORKFLOW_GUIDE.md#the-loop) is the full version):
+
+1. `validate_workflow` with the `arguments` you will run with. Quote its
+ `plan`: `estimate.minutes` with its `basis`, and each `downloads_required`
+ entry (task repos and single files included) as its own line.
+2. `run_workflow` with `acknowledged_cost` bound to the plan (or `true` when
+ there was none); it returns a `job_id`.
+3. `wait_for_job(job_id, timeout_seconds=...)`, asking for the estimate plus a
+ margin and calling again on `still_running: true`; or `get_job_events`,
+ passing back `last_seq` as `after`, for incremental progress.
+4. `get_job(job_id)` for the manifest, or the error and traceback. A pipeline
+ step's entry carries `timing` (load, encode, denoise, decode and save
+ seconds; [WORKSPACES.md](WORKSPACES.md#step-timing)).
+5. `get_output_image(name)`, `get_output_frames(name, count=12)` and
+ `get_output_audio(name, start, duration)` to judge the result.
+
+While a job runs, `get_job` and `wait_for_job` carry `progress`: the step, the
+phase (`loading`, `generating`, `decoding`, `saving`) with the model in
+`phase_detail`, `seconds_in_phase`, `seconds_since_event`, and
+`denoise_step`/`denoise_total_steps` (null until the denoise loop starts). A
+run is healthy while `denoise_step` moves between polls minutes apart, and
+stuck when it is a number that stays put while `seconds_since_event` climbs.
+Two silent stretches are normal: the lead-in under `generating` with a null
+`denoise_step` (prompt and reference encoding plus the first denoise pass;
+several minutes on MiniMax H3 with a video reference), and `saving`, where the
+counter sits at its total. `get_job_events` carries a `log` line per top-level
+modular block (`MiniMaxAI/MiniMax-H3: vae_encoder`) and per file written
+(`writing shot.mp4 (121 frames)`, `wrote shot.mp4 in 1.3s (1.4 MB)`, #97).
+Gaps between denoise steps are uneven under a transformer block cache.
+Measured lead-in figures and every field are in
+[Progress on a running job](SERVER.md#progress-on-a-running-job) and
+[Progress events](SERVER.md#progress-events).
## Security
-The MCP server adds no authentication of its own — it inherits the REST
-API's posture exactly, described in full in [Security](SECURITY.md):
-localhost binding, no auth, `Origin` header checks, and path confinement in
-`dw/security.py` for every workflow, gallery and prompt path a tool touches.
-Nothing under `dw_mcp/` re-implements or loosens that confinement; it is
-purely a client of the same validated endpoints the web UI uses - except for
-`download_output`, the one tool that writes a local file for the MCP client
-rather than only reading through the API. Over a stdio `dw-mcp` it may write
-anywhere the client's own filesystem lets it (a full path, a directory, or
-the current working directory by default, `~` expanded), the way a shell
-redirect would for the same user; a `..` path segment in `destination` is
-refused, and an existing file is left alone unless the caller passes
-`overwrite=True`.
-
-Over `dw.serve --mcp` the write happens **on the server**, and there the
-destination is confined to that workspace: an absolute or `~` path outside
-it is refused, and a relative one is joined onto the workspace rather than
-onto whatever the server process's working directory happens to be.
-`destination` is required over this transport - an omitted one is refused
-rather than dropped loose in the workspace root, where nothing can find or
-delete it later (#353). The
-transport is what distinguishes the two - on stdio "local disk" is genuinely
-the caller's own machine, over HTTP it is the operator's. Confinement is on
-the resolved real path, not a substring test, because an absolute path needs
-no `..` to reach anywhere the server can write.
-
-`dw-mcp` may be pointed at a `dw.serve` on another machine only when that
-server was started with a token, and the same token is passed here
-(`--token` / `DW_API_TOKEN`); it refuses to start otherwise. The token is
-the only authentication, and the connection is plaintext HTTP - use it on
-a network you control, or through Tailscale or a TLS proxy beyond that.
-[REMOTE.md](REMOTE.md) has the full setup. The same applies to
-`dw.serve --mcp`, which serves this tool surface itself at `/mcp` behind
-the same token; in that setup `download_output` writes on the server
-machine, not the client's.
-
-`save_workflow` and `run_workflow` together let an MCP client write and
-then execute a workflow it authored - and a workflow JSON file can execute
-arbitrary Python (see [Trust model](SECURITY.md#trust-model)). What
-protects a `dw.serve` an MCP client talks to is the server's own
-`--trust-workflows` flag, off by default: run `dw-serve` without it (the
-default) for any server an MCP client can reach.
+`dw_mcp` adds no authentication of its own and inherits the REST API's
+posture, described in [Security](SECURITY.md): an opt-in bearer token
+(`--token` / `DW_API_TOKEN`) on `/api/*` and `/mcp`, `Origin` and `Host`
+header checks, and `dw/security.py` path confinement for every workflow,
+gallery, asset and prompt path a tool sends. Nothing under `dw_mcp/` loosens
+that confinement.
+
+Two tools touch a file outside the API, and whose machine "local" is depends
+on the transport:
+
+- **`download_output`** writes a file. Over stdio `dw-mcp` it writes anywhere
+ the user's own process may, like a shell redirect. Over `dw.serve --mcp` it
+ writes on the server, confined to the workspace: an absolute or `~` path
+ outside it is refused, a relative one is joined onto the workspace, and
+ `destination` is required (#353, #113). Containment is checked on the
+ resolved real path. On both transports a `..` segment is refused and an
+ existing file needs `overwrite=True`.
+- **`upload_asset(file_path=...)`** reads a file. Over stdio it reads what the
+ user may. Over `dw.serve --mcp` `file_path` is confined to the server's
+ workspace, workflows, assets, outputs and prompts directories, and the
+ refusal comes before the file is looked for, so it is not a path-existence
+ oracle (#138). `content` names no path and is not confined.
+
+`dw-mcp` reaches a `dw.serve` on another machine only when that server has a
+token and the same token is passed here; it refuses to start otherwise. The
+token is the only authentication and the connection is plaintext HTTP: use a
+network you control, or Tailscale or a TLS proxy beyond that
+([REMOTE.md](REMOTE.md)). `dw.serve --mcp` serves this tool surface at `/mcp`
+behind the same token, and refuses to start on a non-loopback host without one.
+
+`save_workflow` and `run_workflow` let an MCP client write and then run a
+workflow, and a workflow can execute arbitrary Python
+([Trust model](SECURITY.md#trust-model)). What protects the server is its own
+`--trust-workflows` flag, off by default: leave it off on any server an MCP
+client can reach.
## Known limits
-- **Event history is bounded.** `get_job_events` serves at most the last 200
- events of a finished job (`MAX_PERSISTED_EVENTS` in the job history store).
- A job that ran before this feature existed returns an empty event list
- with a `note` explaining why.
-- **Images, sound and frames.** `get_output_image` returns an image,
- `get_output_audio` a soundtrack (an audio file's, or the one muxed into a
- video) whole or as a named excerpt, and `get_output_frames` frames of a
- video as images - there is no video content type over MCP, so a video is
- seen as frames and heard as its track. `get_output_audio` refuses a whole
- clip whose base64 size would exceed the same 4MB budget; ask for an
- excerpt instead.
-- **Uploads read the MCP server's disk, unless sent inline.**
- `upload_asset(file_path)` pushes a local file into the asset library, but
- "local" means the machine `dw-mcp` runs on. Over `dw.serve --mcp` that is
- the GPU box, so a file sitting on the client's laptop is not reachable that
- way - put it on the server, give the workflow a URL (the arguments that
- take a path take a URL too), or use `upload_asset(content=..., asset_name=...)`
- to send the bytes inline instead, base64-encoded in the call itself, capped
- at 4MB (#203). On a `--mcp` endpoint `file_path` is also *confined* to the
- directories the server works in (its workspace, workflows, assets, outputs
- and prompts), and the refusal comes before the file is looked for, so the
- tool cannot be used to probe which paths exist on the box (#138); `content`
- reads no path and is not subject to this confinement, since no file on
- either machine is ever named.
- `download_output` has the same asymmetry in the other direction: on a
- `--mcp` endpoint it writes on the GPU box, not the client's machine, and is
- confined to the workspace there.
-- **Prompts are not per-workspace.** Switching workspaces changes which
- workflows, assets and outputs the session sees; the prompt library is one
- library shared by all of them, because `prompt:` is shared by reference.
+- **Event history is bounded.** A finished job keeps its last 200 events
+ (`MAX_PERSISTED_EVENTS`). A job from before event persistence returns an
+ empty list with a `note`.
+- **No video content type.** A video is seen as frames (`get_output_frames`)
+ and heard as its track (`get_output_audio`); a whole clip over the 4MB
+ budget is refused, so ask for an excerpt.
+- **Files cross machines only by transport.** Over `dw.serve --mcp`,
+ `upload_asset(file_path)` and `download_output` act on the GPU box, not the
+ client. Send a client file with `upload_asset(content=..., asset_name=...)`
+ (4MB cap) or give the workflow a URL; fetch results by their `url`.
+- **Prompts are not per-workspace.** The prompt library is one library shared
+ by every workspace, because `prompt:` is shared by reference.
## Troubleshooting
| Symptom | Likely cause |
| --- | --- |
-| `Failed to reconnect to dw: ENOENT` (or the client cannot start the server) | The client cannot find the command. Register the venv's **absolute** path to `dw-mcp`, not the bare name - see [Use the absolute path](#use-the-absolute-path-to-dw-mcp). A bare name works only when the client was launched from an activated venv, so this often appears in a second terminal after the first one worked |
-| The server shows connected, but every tool fails | "Connected" means the `dw-mcp` process launched, not that the engine is reachable. Check `dw.serve` is running |
-| "Cannot reach diffusers-workflow at …" | `dw.serve` is not running. Start it with `dw-serve` (or `python -m dw.serve`) and try again |
-| A config change seems to have no effect | A running session holds the old config. Start a new session |
-| It worked, then broke after rebuilding the venv | Re-run `pip install -e ".[server,mcp]"`. If the repo moved or was renamed, re-register the server with the new absolute path |
-| A tool call times out | Usually a model loading into VRAM/RAM for the first time; retry, or raise `--timeout` |
-| `run_workflow` or `rerun_job` refuses with a cost message | Not an error — it is the `acknowledged_cost` gate. Confirm with the user and call again with `acknowledged_cost` bound to the plan (or `true`); a 409 "shape changed" answer means the run grew since the quote - re-validate, re-quote, pass the new plan |
+| `Failed to reconnect to dw: ENOENT` (or the client cannot start the server) | The client cannot find the command. Register the venv's **absolute** path to `dw-mcp` ([Use the absolute path](#use-the-absolute-path-to-dw-mcp)). A bare name works only when the client was launched from an activated venv |
+| The server shows connected, but every tool fails | "Connected" means `dw-mcp` launched, not that the engine is reachable. Check `dw.serve` is running |
+| "Cannot reach diffusers-workflow at …" | `dw.serve` is not running. Start it with `dw-serve` (or `python -m dw.serve`) |
+| `dw-mcp` exits 2 at startup | A non-loopback `--url` with no token, or the startup probe failed against a remote URL (unreachable, or the token was rejected with a 401) |
+| A config change has no effect | A running session holds the old config. Start a new session |
+| It broke after rebuilding the venv | Re-run `pip install -e ".[server,mcp]"`. If the repo moved, re-register the new absolute path |
+| A tool call times out | One API request exceeded `--timeout` (30 s); raise it. For a long job use `wait_for_job`, which returns at its cap rather than timing out |
+| `run_workflow` or `rerun_job` refuses with a cost message | The `acknowledged_cost` gate. Confirm with the user and call again with `acknowledged_cost` bound to the plan (or `true`); a 409 means the run changed since the quote: re-validate, re-quote, pass the new plan |
diff --git a/docs/PROMPT_WEIGHTING.md b/docs/PROMPT_WEIGHTING.md
index 8ba6912a7..5cae47ad9 100644
--- a/docs/PROMPT_WEIGHTING.md
+++ b/docs/PROMPT_WEIGHTING.md
@@ -1,6 +1,6 @@
# Prompt Weighting
-Enable A1111-style prompt weighting to control emphasis on individual words or phrases. Also supports prompts longer than the standard 77-token CLIP limit.
+A1111-style prompt weighting sets the emphasis of individual words or phrases, and lifts the 77-token CLIP limit on the prompt (`dw/prompt_weighting.py`, after sd_embed). Flux-family pipelines only.
## Enabling
@@ -39,34 +39,19 @@ Weights are multiplicative when nested: `(((word:1.3)))` = 1.3 x 1.1 x 1.1 = 1.5
## How It Works
-When enabled, prompts containing weighting syntax are intercepted before pipeline execution:
+Before the pipeline call, a `prompt` string containing `(` or `[` is parsed into weighted tokens, run through the CLIP and T5 encoders with the weights applied, and replaced by `prompt_embeds` and `pooled_prompt_embeds`. A prompt with neither character passes through as a plain string, and so does a `prompt` that is not a string.
-1. The prompt string is parsed for weight tokens
-2. Tokens are run through the pipeline's text encoders with per-token weights applied
-3. The resulting embedding tensors replace the `prompt` string argument
-4. The pipeline receives `prompt_embeds` and `pooled_prompt_embeds` instead
-
-Prompts without any weighting syntax (`(`, `[`) pass through unchanged as plain strings.
-
-An optional `prompt_2` argument (Flux's T5 prompt, normally defaulting to `prompt`) is also consumed and weighted separately when present.
-
-Because diffusers rejects mixing a string `negative_prompt` with `prompt_embeds`, any `negative_prompt` argument is silently dropped once weighting kicks in.
+- `prompt_2` (Flux's T5 prompt, defaulting to `prompt`) is consumed and weighted separately when present.
+- diffusers rejects a string `negative_prompt` beside `prompt_embeds`, so a `negative_prompt` is dropped once weighting applies, with only a debug log line.
## Supported Pipelines
-Explicitly supports:
-
-- FluxPipeline
-- FluxImg2ImgPipeline
-- FluxInpaintPipeline
-- FluxControlNetPipeline
-
-Any other pipeline whose class name starts with `Flux` (e.g. `FluxKontextPipeline`, `FluxFillPipeline`, a custom subclass) is also supported automatically, provided it carries the same CLIP + T5 encoder stack (`tokenizer`, `tokenizer_2`, `text_encoder`, `text_encoder_2`). Non-Flux pipelines are not currently supported — the prompt is left as a plain string and a warning is logged.
+`FluxPipeline`, `FluxImg2ImgPipeline`, `FluxInpaintPipeline` and `FluxControlNetPipeline` by name, and any other pipeline whose class name starts with `Flux` (`FluxKontextPipeline`, `FluxFillPipeline`, a subclass) that carries the same CLIP + T5 stack: `tokenizer`, `tokenizer_2`, `text_encoder`, `text_encoder_2`. On any other pipeline the prompt stays a plain string and a warning is logged.
## Requirements
-- The pipeline's text encoders must be loaded (not set to `null`)
-- Cannot be used with `remote_text_encoder` (mutually exclusive — `remote_text_encoder` takes precedence if both are set)
+- The pipeline's text encoders must be loaded, not set to `null`.
+- Mutually exclusive with `remote_text_encoder`, which wins if both are set.
## Example
@@ -75,4 +60,4 @@ python -m dw.run workflows/templates/prompt-weighting.json \
prompt="a (cinematic:1.5) shot of a (dragon:1.3) breathing [smoke] over a (medieval:0.8) castle"
```
-See [prompt-weighting.json](../workflows/templates/prompt-weighting.json).
+See [prompt-weighting.json](../workflows/templates/prompt-weighting.json) (FLUX.1 schnell, 4 steps).
diff --git a/docs/QUANTIZATION.md b/docs/QUANTIZATION.md
index f2765e169..f54bcdc92 100644
--- a/docs/QUANTIZATION.md
+++ b/docs/QUANTIZATION.md
@@ -1,10 +1,10 @@
# Quantization
-Quantization reduces model memory usage by storing weights at lower precision. diffusers-workflow ships examples for BitsAndBytes, TorchAO, GGUF, and SDNQ, applied per-component in the pipeline - and any other backend with a config class (optimum-quanto, for example) works through the same dynamic `config_type` import.
+Quantization stores a component's weights at lower precision to cut its memory. The catalog has examples for BitsAndBytes, TorchAO, GGUF and SDNQ, applied per component. Any other backend with a config class (optimum-quanto, for example) works through the same dynamic `config_type` import ([Custom Quantization](#custom-quantization)). Which combinations fit a 24 GB card is in [RECIPES_24GB.md](RECIPES_24GB.md).
## Per-Component Quantization
-Quantize individual components (transformer, text encoder, etc.) independently:
+Each component block (transformer, text encoder, ...) takes its own `quantization_config`:
```json
{
@@ -30,11 +30,11 @@ Quantize individual components (transformer, text encoder, etc.) independently:
}
```
-The component is loaded separately with quantization, then the rest of the pipeline loads around it.
+The component loads on its own, quantized, and the rest of the pipeline loads around it. A modular pipeline declares it differently - see [Modular Pipelines](#modular-pipelines).
## BitsAndBytes (CUDA only)
-4-bit and 8-bit quantization via bitsandbytes:
+4-bit and 8-bit quantization. bitsandbytes is installed on Linux only:
```json
"quantization_config": {
@@ -47,7 +47,7 @@ The component is loaded separately with quantization, then the rest of the pipel
}
```
-Note: `"{nf4}"` uses braces to keep the string literal. Without braces, the type system would try to load `nf4` as a Python class.
+`"{nf4}"` is brace-escaped so it stays a string: a `*_type` key unbraced is loaded as a Python type and fails at load time, after validation has passed ([WORKFLOW_GUIDE.md](WORKFLOW_GUIDE.md#types-and-escaping)).
For 8-bit:
@@ -59,10 +59,9 @@ For 8-bit:
## TorchAO
-Quantization via TorchAO. `quant_type` must be an `AOBaseConfig` class (diffusers no
-longer accepts string shorthands like `"int4wo"`). Name the class as a dotted
-`quant_type` and it is instantiated automatically with no arguments before being passed
-to `TorchAoConfig`:
+`quant_type` must be an `AOBaseConfig` instance; diffusers no longer accepts string
+shorthands like `"int4wo"`. Name the class as a dotted `quant_type` and dw instantiates
+it with no arguments before passing it to `TorchAoConfig`:
```json
"quantization_config": {
@@ -81,12 +80,12 @@ RTX 40-series/Ada or newer).
**Example:** [flux-torchao.json](../workflows/models/flux-torchao.json)
**Pair TorchAO with `torch.compile`.** Int8 weight-only and float8 dynamic-activation
-quant types get their fused-kernel speedups only under compilation - uncompiled they are
-a memory win but often a speed *loss*. Compiled is not a guarantee either: int8 weight-only
-on an RTX 3090 measured over a minute per denoising step for Flux dev, compiled, against
-two seconds for bf16 (see [RECIPES_24GB.md](RECIPES_24GB.md#flux-dev-12b)). Measure on the
-card before recording a TorchAO recipe as fast. Add a `compile` block to the quantized component
-(see [ACCELERATION.md](ACCELERATION.md#torchcompile)):
+quant types get their fused-kernel speedups only under compilation; uncompiled they are
+a memory win and often a speed loss. Compiled is no guarantee: int8 weight-only on an
+RTX 3090 measured over a minute per denoising step for Flux dev, compiled, against two
+seconds for bf16 ([RECIPES_24GB.md](RECIPES_24GB.md#flux-dev-12b)). Measure on the card
+before recording a TorchAO recipe as fast. Add a `compile` block to the quantized
+component ([ACCELERATION.md](ACCELERATION.md#torchcompile)):
```json
"configuration": {
@@ -100,7 +99,7 @@ card before recording a TorchAO recipe as fast. Add a `compile` block to the qua
## GGUF
-Load GGUF-format checkpoint files:
+Loads a GGUF-format checkpoint file:
```json
"quantization_config": {
@@ -111,7 +110,7 @@ Load GGUF-format checkpoint files:
}
```
-GGUF models load from single files using `from_single_file`:
+The component's `from_pretrained_arguments` names the file with `from_single_file`:
```json
"from_pretrained_arguments": {
@@ -124,13 +123,13 @@ GGUF models load from single files using `from_single_file`:
## SDNQ (SD.Next Quantization)
-SDNQ works two ways: quantize a component on the fly at load time, or load a
-pre-quantized model as a complete pipeline.
+SDNQ quantizes a component on the fly as it loads, or loads a pre-quantized model as a
+complete pipeline.
### On-the-fly (`sdnq.SDNQConfig`)
-Quantizes the component while it loads - the pattern the LTX-2 and MiniMax H3
-examples use for their large transformers and text encoders:
+The pattern the LTX-2.5 and MiniMax-H3 templates use for their transformers and text
+encoders (H3 declares it under `load_components`, see [Modular Pipelines](#modular-pipelines)):
```json
"quantization_config": {
@@ -145,14 +144,12 @@ examples use for their large transformers and text encoders:
}
```
-- `weights_dtype` — the storage dtype (`uint4`, `int8`, ...; brace-escaped so it stays a string)
-- `quantization_device` / `return_device` — where the quantization pass runs and where the finished component lands; quantizing on `cuda` is much faster than on CPU
-- `use_quantized_matmul` — quantized matmul kernels (CUDA/XPU only)
+- `weights_dtype`: the storage dtype (`uint4`, `int4`, `int8`, ...), brace-escaped so it stays a string
+- `quantization_device` / `return_device`: where the quantization pass runs and where the finished component lands. Quantizing on `cuda` is much faster than on CPU. A `cuda` here becomes the server's device on a Mac
+- `use_quantized_matmul`: quantized matmul kernels (CUDA/XPU). Turned off on MPS with a warning, where it runs through `torch._int_mm` far slower than dequantizing
**Examples:** [text-to-video.json](../workflows/templates/ltx2/text-to-video.json), [video-with-audio.json](../workflows/templates/minimax/video-with-audio.json)
-The H3 templates also default to `attention_backend: "sparse"`; the quantization is independent of it ([Attention Backends](ACCELERATION.md#sparse-minimax-h3)).
-
### Cached on-the-fly (`DW_QUANTIZED_CACHE`)
An on-the-fly load reads the bf16 weights and quantizes them on every cold load. With `DW_QUANTIZED_CACHE=1` the server keeps the result: the first cold load saves each quantized component to disk with sdnq's `save_sdnq_model` (safetensors plus JSON, no pickle), and a later load of the same key reads it back with `load_sdnq_model`, skipping the bf16 read and the quantize. Any other value leaves it off, with no events. It covers both ways a component is quantized with a `sdnq.SDNQConfig` `quantization_config`: a modular pipeline's `load_components`, one config per component (MiniMax H3), and a component loaded on its own through its class's `from_pretrained` (LTX-2.5's transformer and Gemma text encoder). A hit is placed on the config's `return_device` (LTX's transformer goes straight to `cuda`) and loads with no download.
@@ -160,7 +157,7 @@ An on-the-fly load reads the bf16 weights and quantizes them on every cold load.
The variables are read by the server and inherited by its workers, so set them in the server's environment (e.g. `.env`) and restart.
- **Where**: `DW_QUANTIZED_CACHE_DIR`, default `/dw-quantized`. `dw.serve` resolves it to an absolute path at start and pins it for the workers. Each entry is `//` with an `entry.json` (the key's inputs, size, created and last-hit times), written to a `.tmp-*` sibling and renamed into place
-- **The key**: repo, subfolder, snapshot commit, quantizer class and config (minus the devices, the quantized-matmul switches and `non_blocking`, which don't change the stored weights), `torch_dtype`, and the sdnq, diffusers, transformers and torch versions. A key that can't pin the weights isn't cached
+- **The key**: repo, subfolder, snapshot commit, quantizer class and config (minus the devices, the quantized-matmul switches and `non_blocking`, which don't change the stored weights), `torch_dtype`, the sdnq, diffusers, transformers and torch versions, and `KEY_VERSION`. A key that can't pin the weights isn't cached
- **Not cached** (a `skipped` event with the reason):
| Reason | When |
@@ -200,16 +197,16 @@ Pre-quantized SDNQ repos load as complete pipelines. The `sdnq` module must be i
}
```
-- `pre_load_modules` — Imports sdnq before pipeline loading (registers quantization method)
-- `sdnq_optimize` — Applies quantized matmul to listed components (CUDA/XPU only, skipped on MPS/CPU)
+- `pre_load_modules`: imports sdnq before the pipeline loads, which registers its quantization method. `sdnq` is one of the packages an untrusted workflow may import; any other module needs a trusted workflow ([SECURITY.md](SECURITY.md))
+- `sdnq_optimize`: applies quantized matmul to the listed components. CUDA/XPU with Triton only; skipped elsewhere with a log line
**Example:** [z-image-sdnq.json](../workflows/models/z-image-sdnq.json)
## Modular Pipelines
-A modular pipeline pulls its component weights itself via `load_components()` rather than
-through `from_pretrained_arguments`, so quantization is keyed by component name under
-`load_components.quantization_config` instead of living on a separate component block:
+A modular pipeline loads its components itself through `load_components()`, so
+quantization is keyed by component name under `load_components.quantization_config`
+instead of living on a component block:
```json
"configuration": {
@@ -230,14 +227,14 @@ through `from_pretrained_arguments`, so quantization is keyed by component name
}
```
-A component the map does not name loads unquantized. Note that a transformers-based
-component (a text encoder, for example) takes the `transformers.TorchAoConfig` class,
-not the diffusers one - the `config_type` still resolves either via the dynamic import
-described below.
+A component the map does not name loads unquantized. A transformers-based component
+(a text encoder or language model) takes `transformers.TorchAoConfig`, not the diffusers
+class. The MiniMax-H3 templates quantize `transformer` (or `transformer_ref`),
+`text_encoder` and `vae` this way with `sdnq.SDNQConfig`.
## Custom Quantization
-Any quantization backend that provides a config class works via the `config_type` field with a dotted module path:
+Any backend that provides a config class works through `config_type` with a dotted module path:
```json
"quantization_config": {
@@ -246,7 +243,7 @@ Any quantization backend that provides a config class works via the `config_type
}
```
-The class is loaded dynamically via importlib.
+The class is imported dynamically (`dw/trust.py` gates it). A config class outside the packages dw already depends on (diffusers, transformers, torchao, sdnq, optimum, gguf, bitsandbytes, ...) needs a trusted workflow ([SECURITY.md](SECURITY.md)).
## Platform Notes
diff --git a/docs/RECIPES_24GB.md b/docs/RECIPES_24GB.md
index ed8824edb..e7883f1fd 100644
--- a/docs/RECIPES_24GB.md
+++ b/docs/RECIPES_24GB.md
@@ -1,14 +1,14 @@
# Fast on 24GB
-Recommended configurations per model family for a 24GB consumer GPU (RTX 3090/4090 class). Every knob here is documented individually in [ACCELERATION.md](ACCELERATION.md) and [QUANTIZATION.md](QUANTIZATION.md); this page is about the combinations that work.
+Tested configurations per model family for a 24GB consumer GPU (RTX 3090/4090 class). Each knob is documented in [ACCELERATION.md](ACCELERATION.md) or [QUANTIZATION.md](QUANTIZATION.md); this page is the combinations that work, and what they measured.
The general recipe, in order of impact:
-1. **Fit the transformer first.** If it fits in bf16 with room for activations, don't quantize. If it doesn't, prefer float8/int8 quantization (TorchAO, GGUF Q8) over offloading - quantization costs quality once, offloading costs speed every step.
-2. **Compile the transformer** (`"compile": {"repeated_blocks": true}`). 1.3-1.5x, stacks with everything below. The server's persistent worker keeps compiled pipelines loaded, so the compile cost is paid once per session. Add `fullgraph: true` only when no cache is configured - cache hooks need a graph break.
-3. **Cache** (`"cache": {"type": "first_block"}`). Another 1.5-2x at mild quality cost; raise `threshold` to taste.
-4. **Offload only what doesn't fit.** Text encoders and VAE tolerate `offload: "model"` cheaply - they run once per generation, not once per step. In a modular pipeline the same components take `"residency": "on_demand"`, which frees their VRAM for the denoise loop at the cost of one pair of transfers per call.
-5. **Pin the attention backend** on compiled components (`"attention_backend": "flash_hub"` or `"sage_hub"` - fetched from the Hub, no local build).
+1. **Fit the transformer first.** If it fits in bf16 with room for activations, don't quantize. If it doesn't, prefer float8/int8 quantization (TorchAO, GGUF Q8) over offloading: quantization costs quality once, offloading costs speed every step. TorchAO int8 is the exception on Ampere ([Flux dev](#flux-dev-12b)).
+2. **Compile the transformer** (`"compile": {"repeated_blocks": true}`): 1.3-1.5x, paid once per worker per workflow ([torch.compile](ACCELERATION.md#torchcompile)). No `fullgraph: true` alongside a cache.
+3. **Cache** (`"cache": {"type": "first_block"}`) on a schedule of 20 steps or more; it skips nothing on a short or turbo one ([Diffusers Built-in Cache](ACCELERATION.md#diffusers-built-in-cache)). Raise `threshold` to skip more.
+4. **Offload only what doesn't fit.** Text encoders and VAE tolerate `offload: "model"` cheaply: they run once per generation, not once per step. Per component, `"residency": "on_demand"` frees their VRAM for the denoise loop at the cost of one pair of transfers per call.
+5. **Pin the attention backend** on compiled components (`"attention_backend": "flash_hub"` or `"sage_hub"`, Hub kernels with no local build; `sage_hub` needs `DIFFUSERS_TRUST_REMOTE_KERNELS=true` - see [Attention Backends](ACCELERATION.md#attention-backends)).
## Flux dev (12B)
@@ -69,8 +69,7 @@ loads with the least VRAM and is unmeasured for speed.
## Z-Image Turbo (6B)
-Fits a 24 GB card in bf16 with model offload, and that is the configuration to use.
-Measured on an RTX 3090 (960x544, 9 steps, 4 images per prompt, cold load included,
+Fits a 24 GB card in bf16 with model offload, the configuration to use. Measured on an RTX 3090 (960x544, 9 steps, 4 images per prompt, cold load included,
2026-10-10):
| Offload | Load | Denoise | Decode | Total | Peak VRAM reserved |
@@ -88,22 +87,20 @@ trade it for per-image decodes.
## Qwen-Image (20B)
-Too large for bf16 on 24GB. Quantize the transformer (TorchAO int8/float8 or GGUF Q4/Q5) and model-offload the rest; add `compile` + `first_block` cache as for Flux. The Qwen2.5-VL text encoder is large - quantize it too, or group-offload it (`"text_encoder.model"` with `leaf_level`).
+Too large for bf16 on 24GB. Quantize the transformer (TorchAO float8, or GGUF Q4/Q5) and model-offload the rest; add `compile` + `first_block` cache as for Flux. The Qwen2.5-VL text encoder is large: quantize it too, or group-offload it (`"text_encoder.model"` with `leaf_level`). The catalog ships no Qwen-Image (20B) workflow; author one from [text-to-image.json](../workflows/templates/text-to-image.json).
-The catalog no longer ships a Qwen-Image workflow - author one from [text-to-image.json](../workflows/templates/text-to-image.json) with the configuration above.
+Qwen-Image-2.1 is a different, 7B model and fits in bf16: [qwen-image-2.1.json](../workflows/models/qwen-image-2.1.json) runs text-to-image at 2048x2048 under `"offload": "model"`, and [qwen-image-2.1-edit.json](../workflows/models/qwen-image-2.1-edit.json) needs `"offload": "sequential"` because the condition image's latents lengthen the sequence. Both tile and slice the VAE, whose decode is what runs out of memory at that size. Its own `use_kv_cache` must be off for a `first_block` or `mag` cache ([Models diffusers has not registered](ACCELERATION.md#models-diffusers-has-not-registered)).
## Wan 2.2
-- **TI2V-5B**: fits in bf16 on 24GB. No quantization needed - just `compile` + `cache` + VAE tiling for longer clips.
-- **T2V/I2V-A14B**: two 14B transformers (`transformer` + `transformer_2`). Quantize both (GGUF Q4/Q5 or TorchAO int8) and use `offload: "model"`; enable `vae.enable_tiling`.
+- **TI2V-5B**: fits in bf16 on 24GB. No quantization; `compile` + `cache` + VAE tiling for longer clips.
+- **T2V/I2V-A14B**: two 14B transformers (`transformer` + `transformer_2`). Quantize both (GGUF Q4/Q5) and use `offload: "model"`; enable `vae.enable_tiling`.
-The catalog no longer ships Wan workflows - author one from [minimax/image-to-video.json](../workflows/templates/minimax/image-to-video.json) with the configuration above.
+Unmeasured here. The catalog ships no Wan workflow; author one from [ltx2/text-to-video.json](../workflows/templates/ltx2/text-to-video.json), the closest standard video pipeline.
## HunyuanVideo (13B)
-Same shape as Flux: quantize the transformer (GGUF Q6/Q8, or TorchAO int8/float8 per the Flux table) + `offload: "model"` + `vae.enable_tiling`. The Llama text encoder benefits from group offload. Use `first_block` cache - video steps are expensive, caching pays off more than on images.
-
-The catalog no longer ships a HunyuanVideo workflow - author one from [ltx2/text-to-video.json](../workflows/templates/ltx2/text-to-video.json) with the configuration above.
+Same shape as Flux: quantize the transformer (GGUF Q6/Q8, or TorchAO float8 on Ada) + `offload: "model"` + `vae.enable_tiling`. The Llama text encoder benefits from group offload. Use a `first_block` cache at 20 steps or more: video steps are expensive, so caching pays off more than on images. Unmeasured here. The catalog ships no HunyuanVideo workflow; author one from [ltx2/text-to-video.json](../workflows/templates/ltx2/text-to-video.json).
## MiniMax-H3
@@ -112,16 +109,16 @@ pipeline-level `offload`. The working 24GB configuration at 960x544:
| Component | Config |
| --------- | ------ |
-| `transformer` / `transformer_ref` | SDNQ int4 (`quantization_device: "cuda"`, `return_device: "cpu"`), `group_offload` `block_level` with `num_blocks_per_group: 1-2` and `use_stream: true` |
-| `text_encoder` | `remove_modules: ["lm_head"]` - the encoder path never calls it |
-| `text_encoder.model` | SDNQ int4, `truncate_layers: {"language_model.layers": 51}`, `group_offload` `leaf_level` |
-| `vae`, `audio_vae` | SDNQ int8, `device: "cuda"`, `residency: "on_demand"` |
-| pipeline | `cache: first_block` (`threshold: 0.05`) only on a ~30-step base-model run (`lora_model_name: null`): -22% denoise there, identical staging and faces. It skips nothing at 12 steps or on a turbo schedule (consecutive distilled steps differ too much), and every H3 template takes it as a `cache` argument (null by default): `{"type": "first_block", "threshold": 0.05}`. Measured on a 3090 with `sage_hub`; every H3 template now defaults to `attention_backend: "sparse"` ([Attention Backends](ACCELERATION.md#sparse-minimax-h3)), which needs no extra install. See the `minimax-h3` skill's `references/quality-and-speed.md` |
-
-The VAEs are the piece worth calling out. They hold roughly 3GiB, are used only to
-encode references and decode the result, and group offloading them is worse than useless
-because tiled decode restreams the model once per tile. Every H3 workflow adds
-`"residency": "on_demand"` to both. The reference workflows, which carry the most
+| `transformer` / `transformer_ref` | SDNQ `weights_dtype` (`int4` by default; `quantization_device: "cuda"`, `return_device: "cpu"`), `group_offload` `block_level` with `num_blocks_per_group` 2 (`transformer`) or 1 (`transformer_ref`), `use_stream: true`, `low_cpu_mem_usage: true`; `attention_backend: "sparse"` with top-k 0.15 ([sparse](ACCELERATION.md#sparse-minimax-h3)) |
+| `text_encoder` | SDNQ, same `weights_dtype` (the vision tower and `lm_head` unconverted); `remove_modules: ["lm_head"]`, since the encoder path never calls it |
+| `text_encoder.model` | `truncate_layers: {"language_model.layers": 51}`, `group_offload` `leaf_level` |
+| `vae` | SDNQ int8, `device: "cuda"`, `residency: "on_demand"` |
+| `audio_vae` | unquantized, `device: "cuda"`, `residency: "on_demand"` |
+| pipeline | `cache` `null` by default. `first_block` (`threshold: 0.05`) only on a ~30-step base-model run (`lora_model_name: null`): -22% denoise there, identical staging and faces, measured on a 3090 with `sage_hub`. It skips nothing at 12 steps or on a turbo schedule. See the `minimax-h3` skill's `references/quality-and-speed.md` |
+
+The VAEs hold roughly 3GiB, are used only to encode references and decode the result,
+and group offloading them is worse than useless because tiled decode restreams the model
+once per tile. Every H3 workflow sets `"residency": "on_demand"` on both. The reference workflows, which carry the most
conditioning, go from 23.2GiB peak reserved with 40 allocator retries to 18.9GiB with
none; the frame-conditioned ones from 22.7GiB with 22 retries to 18.0GiB with none. Both
for about 1% in wall time.
@@ -133,8 +130,8 @@ The text encoder pruning exists because H3 conditions on `hidden_states[50]` of
encode. Keeping 51 layers is bit-identical - index 50 of the tuple is recorded before
layer 50 runs; keeping only 50 would hand back the final-norm output, a different
tensor - and it returns a few GiB of system RAM on a host that needs every one of them
-(a full t2va run peaks around 63GiB RSS on a 64GiB box). Note H3's VAE constructs with
-tiling already enabled, so a pipeline-level `vae.enable_tiling` adds nothing here.
+(a full t2va run peaks around 63GiB RSS on a 64GiB box). H3's VAE constructs with
+tiling already enabled, so a pipeline-level `vae.enable_tiling` adds nothing.
Two levers measured and *rejected* on a 3090 (A/B, t2va 960x544x124f, 2026-08):
`low_cpu_mem_usage: false` on the transformer's group offload (pin host copies once
@@ -142,8 +139,7 @@ instead of re-pinning per onload) left step time unchanged at ~15.4s - at int4 t
step is compute-bound and the transfers already hide under it - while the ~27GiB of
unswappable pinned memory pushed the host into an OOM kill. `use_stream` on the text
encoder's leaf offload was backed out with it for the same host-memory reason. On a
-faster GPU (or a host with more RAM) both are worth re-testing; the step budget there
-may actually expose the transfer time.
+faster GPU, or a host with more RAM, both are worth re-testing.
Length costs VRAM but the configuration holds to the model's full range: a single
345-frame take (14.4s, the `17n+5` maximum) peaks at 23.6GiB reserved at 960x544 -
@@ -156,8 +152,8 @@ model first (Z-Image drawing a subject, Music3 writing a song) must free it with
`release_pipeline` before H3 loads - with it, the multi-model digital-short
workflows below fit; without it, the load is an OOM kill, not a slowdown.
-Ref2VA reaches 1344x768 too - the canvas isn't tied to the FL2VA 768p checkpoint, only
-the shift/alpha pairing is (above). The trade-off is frame count, not a config change:
+Ref2VA reaches 1344x768 too: the canvas isn't tied to the FL2VA 768p checkpoint, only
+its shift/alpha pairing is. The trade-off is frame count, not a config change:
VAE decode memory scales with `width * height * num_frames`, so the same 24GB budget
that holds 345 frames at 960x544 only holds 209 at 1344x768 with 2 references, and each
reference costs about 1 GB more (175 at 3) - 175 frames with 2 references measured at
@@ -188,13 +184,13 @@ encoder is nearly as large as it is. The working 24GB configuration at 960x544:
| Component | Config |
| --------- | ------ |
-| `transformer` | SDNQ `uint4` (`quantization_device: "cuda"`, `return_device: "cuda"`, `use_quantized_matmul: true`), resident |
+| `transformer` | SDNQ `uint4` (`quantization_device: "cuda"`, `return_device: "cuda"`, `use_quantized_matmul: true`), `group_offload` `block_level` with `num_blocks_per_group: 1000` (more than its block count, so it moves as one group) and `use_stream: true`. A pipeline-level `offload: "model"` conflicts with the other components' group offload hooks (#266) |
| `text_encoder` (Gemma 4, 23GB) | SDNQ `int8`, `return_device: "cpu"`, `group_offload` `leaf_level` with `use_stream: true` |
| `connectors` (12GB) | `group_offload` `leaf_level` with `use_stream: true` |
| `vae`, `audio_vae`, `vocoder`, `duration_head` | `device: "cuda"` - small, and used once per generation |
| pipeline | `vae.enable_tiling` for anything above the base resolution |
-Two things about the checkpoint are worth knowing before tuning anything:
+Two facts about the checkpoint, before tuning anything:
- **`transformer` is the distilled model.** It runs a fixed 8-step schedule at
`guidance_scale: 1.0`, with STG and modality guidance off, and the `sigmas` every
@@ -235,7 +231,7 @@ That refine pass is what puts the detail back - the upsampler alone gives a soft
and it is the flow the model card, Lightricks' pipeline notes and the diffusers docs all
describe. The base pass keeps its pipeline loaded so the refine pass is served from the
cache; the refine pass releases it. Since 2026-08 Lightricks route production quality
-through their DFR pipeline instead, which diffusers ships and nothing here uses yet.
+through their DFR pipeline instead, which diffusers ships and the catalog does not use.
Measured on an RTX 3090 the refined clip is sharper than the 2x upsample alone at the
same seed: fur, branches and snow texture resolve where the upsample-only frame is a
soft blur. About eight warm minutes, three and a half of them writing the full-size clip.
@@ -260,7 +256,7 @@ Fits several times over in 24GB. Skip quantization and offloading entirely; `com
## Multi-model workflows
-When a workflow chains two large models (generate → upscale, generate → interpolate), release the first pipeline instead of offloading everything:
+When a workflow chains two large models (generate, then upscale or interpolate), release the first pipeline instead of offloading everything:
```json
{ "name": "generate", "release_pipeline": true, "pipeline": { ... } }
diff --git a/docs/RELEASING.md b/docs/RELEASING.md
index aaec12a13..33ae2634c 100644
--- a/docs/RELEASING.md
+++ b/docs/RELEASING.md
@@ -1,11 +1,141 @@
# Releasing
+## Cutting a release
+
+A release is a `v` tag on `master`. CI builds and publishes it.
+
+Before merging `develop` into `master`, get `scripts/preflight.sh` passing. It runs the real-model integration tests (`pytest -m integration`), which CI cannot run, and the UI's full `npm run preflight` (see [TESTING.md](TESTING.md)).
+
+```bash
+scripts/release.sh 0.38.0
+scripts/release.sh 0.38.0-alpha.1 "UI front end" # optional tag message
+scripts/release.sh 0.38.0 --next 0.39.0-alpha.1 # and reopen develop
+```
+
+`scripts/release.sh`, run on `master` with the venv active:
+
+1. Refuses a malformed version, a branch other than `master`, an existing tag, or staged changes. Unstaged changes elsewhere are fine; the release commit is path-limited.
+2. Runs `pytest -m integration -n0` against the working tree, and refuses when they fail or when the machine has no CUDA or MPS device (there they would skip and pass having run nothing).
+3. Sets the version in `pyproject.toml` (the single source; `dw.__version__` reads it at run time) and in `plugins/dw/.claude-plugin/plugin.json` (`tests/test_plugin_skills.py` holds them equal), commits those two files as `release `, and pushes `master`.
+4. Tags the bump commit `v` (annotated with the message, default `release `) and pushes the tag.
+5. With `--next `: in a temporary detached worktree, merges `master` into `develop` (a fast-forward when nothing landed there since the release PR), sets `` in the same two files, commits `chore: open on develop` and pushes `develop`. It works while `develop` is checked out elsewhere. Without `--next`, do this by hand, or `develop` keeps reporting the previous pre-release.
+
+By hand, the equivalent of steps 3-4 is:
+
+```bash
+# Set version = "0.38.0" in pyproject.toml and plugins/dw/.claude-plugin/plugin.json
+git commit -m "release 0.38.0" -- pyproject.toml plugins/dw/.claude-plugin/plugin.json
+git tag -a v0.38.0 -m "release 0.38.0"
+git push origin master v0.38.0
+```
+
+### What CI does with the tag
+
+`.github/workflows/ci.yml` runs on `v` tags:
+
+- **backend** and **ui**: the same checks as on every push.
+- **wheel** (after both): `scripts/build_dist.sh` builds the SPA, copies it into `dw/server/ui`, copies `docs/*.md` into `dw/docs` (the guides an installed server serves), and runs `python -m build`. It uploads `dist/*` as an artifact.
+- **release** (after wheel): fails unless the tag is `v` + semver and equals the `pyproject.toml` version, then creates a GitHub release named after the tag, with auto-generated notes and the wheel and sdist attached. A tag with a pre-release part (`v0.38.0-rc1`) is marked as a pre-release.
+- **pypi** (after release): publishes the same artifacts to PyPI by [trusted publishing](https://docs.pypi.org/trusted-publishers/) (OIDC; no token is stored). One-time setup on pypi.org under *Publishing*: a trusted publisher for project `diffusers-workflow`, owner `dkackman`, repository `diffusers-workflow`, workflow `ci.yml`, environment `pypi` (a pending publisher before the first release). A pre-release needs `pip install --pre`.
+- **docker** (after release): builds `docker/Dockerfile`, starts the image on the CPU runner and waits for `/api/health`, then pushes `dkackman/diffusers-workflow:` to Docker Hub. A release also moves `latest`; a pre-release does not. One-time setup: the public Docker Hub repository `dkackman/diffusers-workflow`, a read/write access token, and a GitHub environment `dockerhub` holding it as secret `DOCKERHUB_TOKEN`. A failed push leaves the GitHub release and PyPI as they are; re-run the job.
+
+Python packaging normalizes a semver pre-release, so `0.38.0-alpha.1` builds a wheel named `0.38.0a1`. The tag, `pyproject.toml` and the GitHub release keep the semver form.
+
+To build artifacts without releasing, run CI by hand (`workflow_dispatch`); the wheel job uploads `dist/*`.
+
+A PyPI install gets released `diffusers`, which may lag the newest pipelines; install `git+https://github.com/huggingface/diffusers` on top.
+
## Unreleased
-This project has no standing release-notes file - GitHub auto-generates
-notes from commits at tag time (see below). This section is a scratch pad
-for items a branch's author wants the next release note to name; clear it
-when a release ships.
+GitHub generates notes from commits at tag time. The sections below are the hand-written notes: the newest is drafted at release prep and pasted into the GitHub release body once the tag has published (`gh release edit v --notes-file ...`). Items a branch's author wants the next note to name go in a new section at the top.
+
+### 0.19.0
+
+
+
+This release is about long H3 chains: two off-by-default ways to hold a
+chain's look across its segments, a measurement that says whether the look
+drifted, a LoRA that was probed against burn, and the skill lines that came
+out of the probes. Alongside it, FLUX 3 Klein no longer loads a model it
+never runs, and the web UI draws a chained step as its segments.
+
+**H3 chain drift** (#1166, #1173, #1174, #1181)
+
+- `latent_anchor`, an H3 `t2va`/`fl2va` argument, matches each segment's
+ per-channel latent statistics to a reference clip before decode.
+ `chain.anchor: "segment_1"` uses the chain's own first segment. It is off
+ unless set, and the run logs one line per anchored call naming the source.
+ The argument meets the media location policy at validate: a bare server
+ path, `..`, `file://` or a loopback URL is refused there.
+- `guide_noise`, a number in [0, 1), loosens every video guide on the step:
+ cosine-tapered Gaussian noise from the guide's first frame to clean at
+ its last, encoded once under the guide's seed. A seeded run repeats.
+- A ten-segment, two-scene probe found the anchor best on a static scene and
+ worst on a moving-camera arc, and guide noise alone the only arm that did
+ not hurt the arc. No template or skill sets either yet; the H3 skill's
+ `chains.md` says what the community reports agree on (#1167).
+- Imagineer13's anti-burn LoRA is in the catalog as
+ `minimax-h3/h3-antiburn`, marked `trial` (#1151, #1159). On a five-segment
+ chain it held segment 1's skin, knit and wall where the unaided chain had
+ burnt by segment 5.
+
+**Measuring drift** (#1152, #1160, #1139, #1158)
+
+- `compare_outputs(names, texture_bands=true)` reports the RMS texture
+ energy per output per moment in three wavelength bands, fine (2-8 px), mid
+ (8-32 px) and coarse (32-128 px), on luma and both chroma differences,
+ over one shared crop, with ratios to the baseline. One name is allowed,
+ so a chain's own segments can be read across. It is a measurement, not a
+ score; the guide's *Texture bands* section says how to read it.
+- `get_gallery_metadata` carries `pcm_sha256`, a SHA-256 of the decoded
+ samples, with `sample_format` and `frames`, for any file with a
+ soundtrack: an identity check for audio over MCP.
+
+**Assessment fixes**
+
+- A `stills_to_video` clip's silent track no longer raises `near_silent` or
+ mono-duplication warnings, `assess_output` no longer reports
+ `shot_dead_air` on its shots, and a cut whose shots are all silent by
+ design no longer warns near-silent (#1161, #1162, #1163). One predicate
+ decides it everywhere.
+- `shot_span_overrun` findings carry severity `warn`; `seam_hole` leaves
+ `rules_applied` when it was skipped on every seam (#1179).
+- `face_drift` measurements and findings name the shot's index, frames and
+ times (#1165), and the `script-to-video` skill says what to do on one
+ (#1178).
+- `check-script` no longer counts a compound Whisper closed that the script
+ wrote open ('candle crumbs' / 'candlecrumbs') against line similarity
+ (#1172).
+
+**FLUX 3 Klein** (#1168)
+
+- Loading Klein on an empty 24 GB card no longer runs out of memory: the
+ pipeline loads only its four components, not the repository's 18.8 GB
+ prompt upsampler, and `configuration.remove_blocks` pops the upsampler
+ block off the modular graph. The step cache keys follow the names.
+- The `controlnet` template passes Canny's thresholds beside `image`, where
+ the detector reads them.
+
+**Skills**
+
+- `minimax-h3`: splitting a short line into two `` blocks removes the
+ lead-in; action prose between blocks is not a pacing rule (#1180, #1184,
+ #1185). `check-script` takes each block's contents, never the prose between.
+- `cinematography`: one light phrase per value in the LTX-2.5, H3 and Klein
+ compile tables (#1142); six named moves measured per video family (#1148).
+- `series-episodes` keeps cast portraits with an extensionless asset name
+ (#1154); the sitcom multicam master sets `move: static` (#1157).
+
+**Web UI and docs**
+
+- A chained step draws its segments: the count when the definition gives
+ it, what each seam carries, and the running segment while the job runs.
+- A workspace named in the URL that the cached listing lacks is refetched
+ once before the UI decides it is gone.
+- Every guide had an accuracy and concision pass, with page screenshots in
+ the README and SERVER guide. The workflow schema's `$id` names this
+ repository, and the HTTP API's OpenAPI document and the schema are linked
+ from the docs.
### 0.18.0
@@ -1557,104 +1687,3 @@ release body once the tag has published (`gh release edit v0.4.0
`reference-sheet` (#444) run with their own defaults again.
- Joined audio and shot maps stay on the frame grid through repeated joins
(#423, #426, #428, #435).
-
-Releases are cut by pushing a `v` tag. CI does the rest.
-
-Before merging `develop` into `master`, run `scripts/preflight.sh` and get it
-passing. It covers more than CI: ruff over the whole repo rather than
-`dw dw_mcp tests` and the real-model integration tests (`pytest -m
-integration`), neither of which CI runs. (CI runs the UI's Playwright e2e
-tests on every develop push and on PRs into develop; the release PR from
-develop relies on the push runs.)
-
-```bash
-scripts/release.sh 0.38.0
-scripts/release.sh 0.38.0-alpha.1 "UI front end" # optional tag message
-scripts/release.sh 0.38.0 --next 0.39.0-alpha.1 # and reopen develop
-```
-
-The script bumps `pyproject.toml` (the single source of the version —
-`dw.__version__` reads it at runtime) and sets the same version in
-`plugins/dw/.claude-plugin/plugin.json`, so an installed plugin names the
-engine it was written against; it commits just those two files, pushes
-master, tags the bump commit `v0.38.0`, and pushes the tag. It refuses
-a malformed version, a branch other than master, an existing tag, or a
-dirty index (unstaged changes elsewhere are fine — the release commit
-is path-limited to those two files).
-
-Before it bumps anything it runs the integration tests, the gate CI's
-accelerator-less runners cannot, and refuses to release when they fail or
-when the machine has no CUDA or MPS device - there they would skip and
-pass having run nothing. Cut a release from the Mac or lem, with the venv
-active. The tests run against the working tree, so unstaged changes are
-part of what they check.
-
-`--next ` finishes the release on the other branch: it merges
-`master` back into `develop` (a fast-forward when nothing landed there
-since the release PR), sets `` in the same two files, commits
-`chore: open on develop` and pushes `develop`, all in a temporary
-worktree, so it works while `develop` is checked out elsewhere. Without it, do that by hand, or `develop` goes on reporting the
-previous pre-release.
-
-CI runs on every push to `develop` as well as `master` - the agent loop
-pushes `develop` directly, with no PR - so a failure shows up against the
-commit that caused it, not first on the release PR.
-
-By hand, the equivalent is:
-
-```bash
-# 1. Bump the version in pyproject.toml:
-# version = "0.38.0"
-# 2. Set the same version in plugins/dw/.claude-plugin/plugin.json
-git commit -m "release 0.38.0" -- pyproject.toml plugins/dw/.claude-plugin/plugin.json
-
-# 3. Tag the bump commit and push
-git tag -a v0.38.0 -m "release 0.38.0"
-git push origin master v0.38.0
-```
-
-The tag must point at a commit whose pyproject already declares the
-same version — the release job checks and refuses a mismatch.
-
-The tag triggers the full CI chain: backend tests, UI lint/type-check/
-unit tests, then the wheel build (SPA compiled into the package via
-`scripts/build_dist.sh`). Only if all of that passes does the `release`
-job run — it verifies the tag matches the pyproject version, then
-creates a GitHub release named after the tag with auto-generated notes
-and the wheel + sdist attached.
-
-Note on pre-release numbering: Python packaging normalizes semver-style
-pre-releases, so a `0.38.0-alpha.1` version builds a wheel named
-`0.38.0a1`. The tag, pyproject, and release stay in the semver form;
-only the wheel filename and pip metadata show the normalized one.
-
-A pre-release tag like `v0.38.0-rc1` is marked as a pre-release on
-GitHub. Tags that aren't `v` + semver (or that don't match the declared
-versions) fail the release job before anything is published.
-
-After the GitHub release, the `pypi` job publishes the same artifacts to
-PyPI via [trusted publishing](https://docs.pypi.org/trusted-publishers/)
-(OIDC — no token stored anywhere). One-time setup on pypi.org under
-*Publishing*: add a trusted publisher for project `diffusers-workflow`
-with owner `dkackman`, repository `diffusers-workflow`, workflow
-`ci.yml`, environment `pypi` (use "add a pending publisher" before the
-first release, since the project won't exist yet). Pre-release versions
-are hidden from plain `pip install`; they need `pip install --pre`.
-
-The `docker` job, also after the GitHub release, builds
-`docker/Dockerfile` from the tag, starts the image on the CPU runner and
-waits for `/api/health`, then pushes `dkackman/diffusers-workflow:`
-to Docker Hub. A release also moves `latest`; a pre-release does not.
-One-time setup: on Docker Hub, create the public repository
-`dkackman/diffusers-workflow` and an access token with read/write scope.
-On GitHub, under *Settings > Environments*, create an environment named
-`dockerhub` with that token as secret `DOCKERHUB_TOKEN`. A failed push
-leaves the GitHub release and PyPI as they are; re-run the job.
-
-Note: released `diffusers` from PyPI may lag the newest model pipelines
-this project targets — a PyPI install can need
-`pip install git+https://github.com/huggingface/diffusers` on top.
-
-To rebuild artifacts without releasing, run the CI workflow manually
-(`workflow_dispatch`) — the wheel job uploads `dist/*` as a workflow
-artifact.
diff --git a/docs/REMOTE.md b/docs/REMOTE.md
index a4833f178..cee3c1da8 100644
--- a/docs/REMOTE.md
+++ b/docs/REMOTE.md
@@ -5,93 +5,90 @@ your laptop use it over the network. Nothing is installed on the laptop.
## On the GPU box
-1. Install as usual: `git clone …`, `bash ./install.sh` - or run the
- container instead ([DOCKER.md](DOCKER.md), `DW_BIND=0.0.0.0`), which
- makes the token for you; skip to the clients.
-2. Make a token: `openssl rand -hex 32`. This one string is the only thing
- between "on your network" and "can run workflows on your GPU" - keep it
- long and random.
+1. Install as usual (`git clone …`, `bash ./install.sh`), or run the
+ container ([DOCKER.md](DOCKER.md), with `DW_BIND=0.0.0.0`), which makes
+ the token for you; then skip to the clients.
+2. Make a token: `openssl rand -hex 32`. It is the only thing between "on
+ your network" and "can run workflows on your GPU".
3. Run it bound to the network, with the token, with MCP:
DW_API_TOKEN= dw-serve --host 0.0.0.0 --mcp
- `--mcp` is refused outright (exit 2) on a non-loopback bind with no
- token: an MCP endpoint can author and run workflows and has no
- token-entry page in front of it the way the web UI does.
+ `--mcp` on a non-loopback bind with no token is refused (exit 2): an MCP
+ endpoint can author and run workflows and has no token-entry page in
+ front of it. Without `--mcp`, a non-loopback bind with no token only logs
+ a warning.
- To keep it running across logouts and reboots, install the systemd unit
- in [contrib/systemd](../contrib/systemd/README.md) instead.
+ To keep it running across logouts and reboots, use the systemd unit in
+ [contrib/systemd](../contrib/systemd/README.md).
4. Open port 8765 in the box's firewall for your LAN only (for example
`sudo ufw allow from 192.168.1.0/24 to any port 8765`).
-Never pass `--trust-workflows` on a server other machines can reach: it
-lets any workflow - including one an agent authored - execute arbitrary
-Python. See [SECURITY.md](SECURITY.md#trust-model).
+Never pass `--trust-workflows` on a server other machines can reach: it lets
+any workflow, including one an agent wrote, execute arbitrary Python. See
+[SECURITY.md](SECURITY.md#trust-model).
Check it from the laptop:
curl http://:8765/api/health -H "Authorization: Bearer "
{"status":"ok","version":"…","hostname":"gpu-box","device":"cuda","mcp":true,…}
-`hostname` and `device` are there so you can tell which machine answered.
+`hostname` and `device` say which machine answered.
-If an agent on this box will ever export a job or list gallery/asset URLs to
-hand to a person who isn't at a terminal on the box itself, set
-`DW_API_TOKEN` and also set `DW_PUBLIC_URL` to this server's origin (for
-example `https://dw.example.com`, or `http://:8765` with no proxy):
+### URLs handed to a person
- DW_API_TOKEN= DW_PUBLIC_URL=https://dw.example.com dw-serve --host 0.0.0.0 --mcp
+`export_job`, `list_gallery` and the upload and asset routes return paths
+relative to the server (`/exports/job-1.zip`): right for a browser already
+pointed at the box, useless to someone who is not. Set `DW_PUBLIC_URL` (or
+the `public_url` setting) to the server's origin and those responses add
+`absolute_url` / `absolute_zip_url` built from it:
-Without it, `export_job`, `list_gallery` and the upload/asset routes only
-return paths relative to this server (`/exports/job-1.zip`) - correct for a
-browser already pointed at the box, useless handed to someone who isn't.
-With `DW_PUBLIC_URL` set (or the equivalent `public_url` setting), those
-responses add an `absolute_url` / `absolute_zip_url` built from it; nothing
-guesses this from request headers, so an unconfigured server omits the
-field rather than composing a wrong origin.
+ DW_API_TOKEN= DW_PUBLIC_URL=https://dw.example.com dw-serve --host 0.0.0.0 --mcp
-The export zip, like `/outputs` and `/inputs` files, needs no token (#592), so
-an agent with HTTP fetches `export_job`'s `open_url` itself, prefixing a
-relative one with the address it reaches the server at; `DW_PUBLIC_URL` is for
-the URLs handed to a person.
+Nothing guesses the origin from request headers, so an unconfigured server
+omits the field. The export zip, like `/outputs` and `/inputs`, needs no
+token (#592), so an agent with HTTP fetches `export_job`'s `open_url`
+itself, prefixing a relative one with the address it reaches the server at.
## Browser
-Open `http://:8765`. Click the key icon next to the theme toggle,
-paste the token once; it is kept in the browser's `localStorage` and sent
-with every request.
+Open `http://:8765`. Click the key icon in the header and paste the
+token once; it is kept in the browser's `localStorage` and sent with every
+request. The Server → Status page shows the server's addresses and the
+`claude mcp add` line below, ready to copy.
+
+
+
+
## Claude Code, no local install
claude mcp add --transport http dw http://:8765/mcp \
--header "Authorization: Bearer "
-Start a new Claude Code session; `/mcp` should list `dw` as connected.
-Pick the scope with `-s user` to have it in every project.
+Start a new Claude Code session; `/mcp` should list `dw` as connected. Add
+`-s user` to have it in every project.
-`http://:8765/mcp` and `http://:8765/mcp/` are both answered, and
-both require the token in an `Authorization: Bearer` header - the
-`?token=...` allowance a few browser GET routes have does not extend to
-`/mcp`.
+`/mcp` and `/mcp/` are both answered, and both take the token only as an
+`Authorization: Bearer` header; the `?token=` allowance a few browser GET
+routes have does not extend to `/mcp`.
-Two things differ from the local stdio setup:
+What differs from the local stdio setup:
-- `download_output` writes on the GPU box (where the MCP server runs), not
- on your laptop, so an omitted destination is refused rather than dropped
- loose in the workspace root - nothing on your laptop would find or
- delete it there (#353). Pass an explicit destination inside the
- workspace to save one anyway, or use `get_output_image` /
+- `download_output` writes on the GPU box, where the MCP server runs, so an
+ omitted destination is refused rather than dropped in the workspace root
+ where nothing on your laptop would find it (#353). Pass an explicit
+ destination inside the workspace, use `get_output_image` /
`get_output_text` to see a result, or open
`http://:8765/outputs/` in the browser.
-- The connection is a plain HTTP call per tool invocation; there is no
- subprocess to restart.
-- `use_workspace`/`create_workspace` pin *this box's* one MCP client, shared
- by every agent connected to it - this server is single-user, so there is
- no per-session isolation. If you and another agent are both against the
- same box, either one's `use_workspace` call can move what the other reads
- and writes next (#298). Pass `workspace=` on each call that takes it
- (`validate_workflow`, `run_workflow`, most read/media tools) instead of
- relying on the session pin when that matters.
+- Each tool call is a plain HTTP request; there is no subprocess to restart.
+- The mounted MCP server is one client shared by every agent connected to
+ the box, so `use_workspace` (or `create_workspace(use=true)`) moves the
+ workspace for all of them (#298). When two agents share a box, pass
+ `workspace=` on each call that takes it (`validate_workflow`,
+ `run_workflow`, most read and media tools) instead of relying on the
+ session pin.
## Claude Code with a local install (stdio)
@@ -102,48 +99,42 @@ remote box too:
--url http://:8765 --token
`dw-mcp` refuses to start (exit 2) against a non-loopback URL without a
-token, and makes one `GET /api/health` at startup so a wrong URL or token
-is reported once, with a message, instead of as a 401 on every tool call.
-Against a remote URL a failed probe is fatal - it is a misconfiguration you
-have to fix. Against a loopback URL it only warns and serves anyway, since
-there it usually means "dw.serve is not up yet", which the next tool call
-reports for itself. `--no-probe` skips the check entirely.
+token. At startup it makes one `GET /api/health`, so a wrong URL or token is
+reported once instead of as a 401 on every tool call: against a remote URL a
+failed probe is fatal, against a loopback URL it only warns (the server may
+not be up yet). `--no-probe` skips it.
## Beyond your LAN
Everything above is plaintext HTTP: the token and every prompt and result
-are readable by anything on the network path. That is acceptable on a
-network you control and nowhere else. **Do not port-forward 8765 on your
+are readable on the network path. **Do not port-forward 8765 on your
router.** Two ways to reach the box from outside:
-**Tailscale (or another WireGuard overlay).** Install it on the box and the
-laptop; use the box's Tailscale IP or MagicDNS name in every URL above.
-Traffic is encrypted end to end, no certificates to manage, and `dw-serve`
-can stay bound to the Tailscale interface (`--host 100.x.y.z`) rather than
-`0.0.0.0`. This is the recommended option.
+**Tailscale (or another WireGuard overlay)**, the recommended option. Install
+it on the box and the laptop and use the box's Tailscale IP or MagicDNS name
+in every URL above. Traffic is encrypted end to end with no certificates to
+manage, and `dw-serve` can bind to the Tailscale interface
+(`--host 100.x.y.z`) rather than `0.0.0.0`.
-**A TLS-terminating reverse proxy.** Bind `dw-serve` back to loopback and
-put Caddy in front of it with a real hostname:
+**A TLS-terminating reverse proxy.** Bind `dw-serve` to loopback and put
+Caddy in front of it with a real hostname:
# /etc/caddy/Caddyfile
dw.example.com {
reverse_proxy 127.0.0.1:8765
}
-Caddy obtains and renews a certificate automatically. Use
-`https://dw.example.com` (no port) in every URL above. `dw-serve`'s Origin
-and Host checks work unchanged behind the proxy because Caddy forwards the
-`Host` header as-is. nginx works the same way with `proxy_pass` and
-`proxy_set_header Host $host;` plus your own certificate.
-
-A token holder can also empty dw's disk caches (`DELETE /api/caches/{name}`, or one entry via `/entries/{entry}`), as they can delete models.
+Caddy obtains and renews the certificate. Use `https://dw.example.com` (no
+port) in every URL above. The Origin and Host checks work unchanged because
+Caddy forwards `Host` as-is; nginx works the same way with `proxy_pass`,
+`proxy_set_header Host $host;` and your own certificate.
-The token is still the only authentication in either setup; a proxy or a
-VPN protects the transport, not the door.
+Either way the token is still the only authentication: a proxy or a VPN
+protects the transport, not the door. A token holder can do anything the API
+does, including deleting outputs, models and dw's disk caches.
## What is not here
- TLS inside `dw-serve` itself: a reverse proxy does it better.
- More than one token, or users: one shared secret, deliberately
([SERVER.md](SERVER.md#authentication)).
-- Docker: `install.sh` on the box is the supported install.
diff --git a/docs/SECURITY.md b/docs/SECURITY.md
index 55a93e4e1..1e74108b7 100644
--- a/docs/SECURITY.md
+++ b/docs/SECURITY.md
@@ -2,374 +2,394 @@
## Overview
-diffusers-workflow validates all file paths, user inputs, and URLs to protect against path traversal, injection, and resource exhaustion. A command-argument sanitizer (`sanitize_command_args()`) is available for any future subprocess use, though `dw/` currently invokes no subprocess/shell commands.
+diffusers-workflow validates file paths, names, user input and URLs against
+path traversal, injection, server-side request forgery and resource
+exhaustion. The validators are in `dw/security.py`, the trust gate for what a
+workflow may import or execute is `dw/trust.py`, the policy for where a
+workflow may read and fetch is `dw/locations.py`, and the HTTP layer is
+`dw/server/http_security.py`. [SECURITY_QUICKREF.md](SECURITY_QUICKREF.md) is
+the one-page digest.
## Security Module (`dw/security.py`)
+Every validator raises `PathTraversalError` or `InvalidInputError` (both
+`SecurityError`) on refusal.
+
### Path Validation
-- `validate_path()` — Blocks `../` (anywhere in the path), `~/` (or `~\`), and paths rooted at `/dev/`, `/proc/`, `/sys/`. Rejects null bytes and overlong paths (> 4096 chars). Resolves to an absolute, realpath'd path. If `base_dir` is given, raises `PathTraversalError` when the resolved path falls outside it.
-- `validate_workflow_path()` — `validate_path()` plus a required `.json` extension (via `validate_file_extension()`)
-- `validate_lora_path()` / `validate_lora_name()` — Confine a LoRA catalog entry to the `loras/` root: a plain name or one family folder deep, `.json` only, no traversal
-- `validate_output_path()` — `validate_path()` with `allow_create=True`, for directories/files that don't need to exist yet
-- `validate_file_extension()` — Checks a path's extension against an allowed set (used internally by `validate_workflow_path()` and by `arguments.py` for media files)
+- `validate_path(path, base_dir=None, allow_create=True)` - Refuses an empty
+ path, a null byte, a path over 4096 characters or a file name over 255,
+ `..` anywhere, `~/` (or `~\`), and a path rooted at `/dev/`, `/proc/` or
+ `/sys/`. Returns the absolute, realpath'd path. With `base_dir`, the
+ resolved path must equal it or lie under it (`PathTraversalError`
+ otherwise); symlinks are resolved on both sides first. Without `base_dir` it
+ confines nothing, and CodeQL still reports the path. `allow_create=False`
+ also requires the path to exist.
+- `validate_workflow_path(path, workflow_dir=None)`,
+ `validate_prompt_path(path, prompt_dir)`, `validate_lora_path(path, lora_dir)`
+ - `validate_path` with `allow_create=False` plus a required `.json`
+ extension.
+- `validate_output_path(path, output_dir)` - `validate_path` with
+ `allow_create=True`, for files and directories that need not exist yet.
+- `validate_file_extension(path, allowed)` - The extension against an allowed
+ set (`ALLOWED_*_EXTENSIONS`).
+- `contained(path, root)` - Whether `path`, symlinks resolved, lies inside
+ `root`, for directory walks that must not follow a link out.
+- `validate_weight_name(name, ...)` - A file name inside a Hub repo: no
+ leading `/` or `~`, no backslash or drive, no empty, `.` or `..` segment.
+
+### Name Validation
+
+Each name is checked before it is joined onto a directory; containment is
+then checked by the `validate_path` call that joins it. Every segment starts
+with a letter, digit or underscore, which excludes `..`, hidden names and
+absolute paths. Patterns end in `\Z`, so a trailing newline is refused.
+
+| Validator | Shape | Max |
+| --- | --- | --- |
+| `validate_variable_name` | `^[a-zA-Z_][a-zA-Z0-9_-]*$` | 100 |
+| `validate_prompt_reference`, `validate_lora_name` | a name, at most one folder deep | 200 |
+| `validate_asset_reference` | a file name with its extension, at most four folders deep; `@` allowed | 400 |
+| `validate_output_reference` | `//`, 2 to 7 segments; `@` allowed | 500 |
+| `validate_output_prefix` | `/` or `` | 500 |
+| `validate_subfolder` | relative segments under the run directory; no backslash | 200 |
+| `validate_workspace_name` | one segment, not one of the workspace root's own folder names | 100 |
+| `validate_constant_name` | a dotted Python name, identifiers and dots only | 200 |
+| `validate_commit_hash` | 7 to 40 hex characters (the diffusers updater's install pin) | 40 |
+| `validate_file_base_name` | no `/` or `\` | - |
+| `validate_content_type` | a MIME type with exactly one `/` | - |
### Input Validation
-- `validate_variable_name()` — Alphanumeric, underscore, hyphen only (pattern: `^[a-zA-Z_][a-zA-Z0-9_-]*$`), max 100 chars
-- `validate_string_input()` — Max length, no null bytes, no control characters other than tab/newline/CR. Every caller that checks a caller-supplied variable value (`dw/variables.py`, `dw/run.py`) passes `MAX_VARIABLE_VALUE_LENGTH`, 20,000 characters; file names and paths pass their own, shorter caps, so the function's bare default of 1000 is not the limit anything is held to. A variable's *default*, written in the definition, is not capped separately: the author controls the file, and the whole file is capped at 50MB
-- `validate_json_size()` — Limits JSON files to 50MB
-- `validate_url()` — Scheme must be `http` or `https`; must have a non-empty domain (`netloc`); may not contain a backslash. `urllib.parse` and the HTTP client disagree on which host `http://169.254.169.254\@example.com/` names, so the host the check approved need not be the one dialed; a `\` that belongs in a path is written `%5C`
-- `validate_constant_name()` — Guards `constant:` references before import: dotted-name pattern only, module must already be importable, and anything callable is refused
-- `safe_join_path()` — Joins path components after rejecting any that contain `..`, `/`, or `\\`. Defined in `security.py` but not currently called elsewhere in `dw/`.
+- `validate_string_input(value, max_length=1000)` - Refuses an over-long
+ value, a null byte, and any control character other than tab, newline and
+ CR. A caller-supplied variable value (`dw/variables.py`, `dw/run.py`) is held
+ to `MAX_VARIABLE_VALUE_LENGTH`, 20,000 characters. A variable's *default* is
+ not capped separately; the whole workflow file is.
+- `validate_json_size(path)` - A JSON file over 50MB is refused before it is
+ read.
+- `validate_url(url)` - Scheme `http` or `https`, a non-empty host, and no
+ backslash anywhere: `urllib.parse` and the HTTP client disagree on which
+ host `http://169.254.169.254\@example.com/` names, so a `\` that belongs in
+ a path is written `%5C`. It checks shape only; the host policy is
+ `dw/locations.py`'s.
+- `safe_join_path(*parts)` - Joins components after refusing any containing
+ `..`, `/` or `\`. Not called anywhere in `dw/`.
### Command Sanitization
-- `sanitize_command_args()` — Rejects arguments containing shell metacharacters ( `` ` `` `$` `|` `&` `;` `>` `<` and newline/CR). It does **not** call `shlex.quote()` — with `shell=False`, argument list separation is handled safely by Python/the OS, so this function is a defense-in-depth check, not an escaping step.
-- As of this writing, `dw/` does not invoke `subprocess` anywhere — the worker process (`dw/worker_manager.py`, `dw/worker.py`) is a `multiprocessing.Process` communicating over `multiprocessing.Queue`, not a shelled-out command. `sanitize_command_args()` is exercised by `tests/test_security.py` but is otherwise unused; it exists for any future code path that shells out.
+- `sanitize_command_args(args)` - Refuses an argument containing `` ` ``,
+ `$`, `|`, `&`, `;`, `>`, `<`, newline or CR. It does not quote: with
+ `shell=False` the OS separates arguments, so this is defense in depth.
+- `dw/` runs three subprocesses, all `shell=False` with fixed argument lists:
+ `pip` in the diffusers updater (`dw/server/updater.py`, through
+ `sanitize_command_args`, its commit pinned by `validate_commit_hash`),
+ `sysctl` for the Apple chip name (`dw/__init__.py`) and `nvidia-smi` for the
+ driver version (`dw/server/sysinfo.py`). The worker is a
+ `multiprocessing.Process`, not a shelled-out command.
## Trust model
-**A workflow JSON file can execute arbitrary Python.** This is a deliberate
-design choice, in the same spirit as ComfyUI custom nodes - the engine's
-dynamic-import machinery is what lets a workflow name any diffusers
-pipeline, scheduler, or quantization backend without dw shipping a bespoke
-adapter for each one. But the same machinery means loading a workflow file
-is not a passive data-load. Three things in a workflow JSON run
-`importlib.import_module()` on a name the file supplies, which executes
-that module's top-level code:
-
-- **`pre_load_modules`** (`dw/pipeline_processors/pipeline.py`) - a list of
- module names imported before the pipeline loads, for their import-time
- registration side effects (`sdnq` registering its quantization method
- with diffusers, for instance)
+**A workflow JSON file can execute arbitrary Python.** The dynamic-import
+machinery that lets a workflow name any diffusers pipeline, scheduler or
+quantization backend also makes loading a workflow more than a data-load.
+Three things in a workflow run `importlib.import_module()` on a name the file
+supplies, which executes that module's top-level code:
+
+- **`pre_load_modules`** (`dw/pipeline_processors/pipeline.py`) - modules
+ imported before the pipeline loads, for their registration side effects
+ (`sdnq` registering its quantization method).
- **A dotted `*_type`/`*_dtype`/`dtype`/`config_type` value**
(`dw/type_helpers.py`, reached from `dw/arguments.py` and
`dw/pipeline_processors/config_objects.py`) - `"sdnq.SDNQConfig"` imports
- `sdnq` and reads `SDNQConfig` off it; nothing stops the module part from
- naming something with no legitimate reason to appear in a workflow
-- **A `constant:`-prefixed reference** (`dw/type_helpers.py`'s
- `load_constant_from_name`, reached from `dw/arguments.py`) - imports the
- module the constant is declared in the same way, before reading the
- attribute off it. `fetch_constant` refuses anything callable it finds,
- but the import itself has already run by that point
-
-**Treat an untrusted workflow file exactly like an untrusted Python
-script.** Don't run one from a source you would not run a `.py` file from -
-a random download, a link in an issue, an LLM-authored file you have not
-read.
+ `sdnq` and reads `SDNQConfig` off it.
+- **A `constant:` reference** (`dw/type_helpers.py`'s
+ `load_constant_from_name`) - imports the module the constant is declared in
+ before reading the attribute.
+
+**Treat an untrusted workflow file exactly like an untrusted Python script.**
+Do not run one from a source you would not run a `.py` file from: a random
+download, a link in an issue, an LLM-authored file you have not read.
### `--trust-workflows`
-`dw-serve` takes a `--trust-workflows` flag (as does the in-process
-`dw-validate`), **off by default**. `dw-run` is a client of `dw-serve`, so
-the server's setting is the one its jobs run under. Untrusted (the default), `pre_load_modules` and any
-dotted `*_type`/`*_dtype`/`dtype`/`config_type` value are refused unless
-they resolve under a top-level package the tool already depends on for
-exactly this purpose - the framework packages (`diffusers`, `torch`,
-`torchvision`, `transformers`, `accelerate`, `peft`) and the quantization
-backends `pyproject.toml` declares for `config_objects.py`'s dynamic
-loading (`sdnq`, `torchao`, `optimum` for optimum-quanto, `gguf`,
-`bitsandbytes`), plus `dw` itself (a workflow's `component_type` can name a
-pipeline under `dw.community_pipelines`, which ships in this repo, not a
-third party one). The refusal names exactly what triggered it and points
-back at `--trust-workflows`. The bundled examples under `workflows/` all
-stay inside this set and load untrusted; a
-workflow that needs to reach outside it - a community pipeline module from
-somewhere else, a custom scheduler package - needs `--trust-workflows`.
-
-The package is not the whole check, because an allowed package holds
-things other than classes and re-exports modules outside itself. Untrusted,
-two more rules apply (`dw/type_helpers.py`):
-
-- **A type reference must resolve to a class.** A `*_type`/`config_type`
- value is constructed with the workflow's own arguments, so
- `"torch.hub.load"` - in `torch`, and a function that fetches and runs a
- GitHub repo's code - is refused as "not a class", as is a module or any
- other object. Under a `dtype` or `*_dtype` key a `torch.dtype`
- (`"torch.bfloat16"`) is accepted too, since that is data rather than
- something called. A bare name (`"FluxPipeline"`) resolves against
- `diffusers` and is held to the same rule. `validate_workflow` reports the
- refusal at the key's path, for every key the run loads as a type
- (`from_pretrained_arguments.torch_dtype` as much as `config_type`).
-- **A `constant:` walk stays inside the package.** A dotted `constant:`
- reference is gated like a type (the module it names is imported before
- `fetch_constant` gets to refuse a callable, so the import itself is what
- the gate has to stop), and then every step of the walk is checked: no
- segment may start with `_`, checked before anything imports, and no module
- the walk passes through may sit outside the allowed packages -
- `constant:torch.os.environ` starts in `torch` and ends in the server's
- environment, and is refused at `torch.os`. Reading a field off a value
- declared in an allowed module still works
- (`...ltx2.utils.GEMMA4_PROMPT_ENHANCEMENT_CONFIG.max_new_tokens`). A bare
- name (`constant:SOME_NAME`) reads from `diffusers`. `validate_workflow`
- resolves every literal `constant:` in a step the way the run does and
- reports a refusal at its path (a variable's default at `variables.`),
- so nothing is queued to find out.
-
-Every `*_type` and `constant:` in the bundled catalog satisfies both rules,
-pinned by `tests/test_workflow_trust.py`'s catalog sweep. `--trust-workflows`
-lifts both, as it lifts the package check.
-
-Two `from_pretrained_arguments` keys are refused untrusted as well, for
-every component and pipeline: `trust_remote_code` (runs the model repo's
-own modeling code) and `custom_pipeline` (fetches and imports a pipeline
-module from the Hub or a local path). Neither goes through an
-`importlib` call of ours, so the dotted-name gate alone would leave
-diffusers' remote-code paths open. No bundled catalog entry sets either
-(`tests/test_catalog_structure.py` refuses one that does); a workflow that
-needs them needs `--trust-workflows`.
-
-`text_generation`'s `generate_kwargs` reach the model's `generate()`, which
-has a remote-code path of its own: `custom_generate` names a Hub repo whose
-`custom_generate/generate.py` transformers downloads and runs, and
-`trust_remote_code` lets it. Both are refused untrusted
-(`require_trusted_generate_arguments` in `dw/trust.py`), before the model
-loads. Its other keys must be scalars or lists of them, and none may be one of
-the pipeline call's own arguments - the same filter `chat_template_kwargs`
-gets, so neither can pass `tokenizer_encode_kwargs`, `images` or a template
-of its own around the task's checks.
-
-`speech_generation` has the same path twice: the text-to-audio pipeline hands
-both its `generate_kwargs` and its `forward_params` to `generate()` for a
-generative model (Bark, VibeVoice). Each must be an object, and
-`custom_generate` / `trust_remote_code` in either are refused untrusted, before
-the model loads (released versions through 0.14.0 passed them through).
-`trust_remote_code`, `custom_pipeline` or `custom_generate` given as a step
-argument of their own, outside those two objects, is refused too, for every
-model: untrusted by the trust gate, trusted as an argument the task doesn't
-take. The task never read them there, but it used to drop them silently, so a
-step setting one ran as if it had taken effect.
-
-Untrusted, every task step is checked the same way at validate:
-`trust_remote_code`, `custom_pipeline` or `custom_generate` set anywhere in its
-`arguments`, top level or nested, is an error naming the key and its path, so
-the run is refused before it queues. `Task.run` repeats the check for every
-command before its handler runs (GHSA-hhfj-p9xj-xjmr).
-
-`speech_generation`'s Audio8-TTS branch (`Edge0/Audio8-TTS-Realistic-0.6b`,
-`dw/vendored/audio8_tts_model.py`) runs no code from its repo: the model is
-vendored, so there is no `trust_remote_code` load. Its five files are fetched
-with `hf_hub_download` at one pinned 40-hex revision, never a branch, and the
-weights are safetensors. The audio codec, `codec.pth`, is a pickle checkpoint;
-it is read with `torch.load(weights_only=True)`, which refuses any object that
-isn't a tensor or a plain container, and only at that pinned revision. Its
-`generate_kwargs` / `forward_params` pass the same trust gate as above, then an
-allowlist (`temperature`, `top_p`, `top_k`, `do_sample`, `max_new_tokens`):
-any other key is refused by name before the model loads.
-
-MMAudio (`dw/vendored/mmaudio_model.py`, `dw/vendored/mmaudio_sync_model.py`,
-`dw/tasks/mmaudio_decoder.py`) runs no code from its repo either: the network
-and the Synchformer visual encoder are vendored at upstream commit
-`974010a0`. Their two checkpoints, `hkchengrex/MMAudio`'s
-`weights/mmaudio_large_44k_v2.pth` and `ext_weights/synchformer_state_dict.pth`,
-are pickles, pinned to one 40-hex revision; each is read with
-`torch.load(weights_only=True)` and loaded strictly, so any object that isn't a
-tensor or a plain container is refused and an unexpected key fails the load.
-The decoder's VAE and vocoder are Kandinsky 6's safetensors at a pinned full
-sha, built from their `config.json` read as JSON (only the class's own
-constructor arguments are taken), with no pickle and no `trust_remote_code`.
-The modules take local paths and never fetch: tests pin that none of them
-calls the Hub or writes a checkpoint.
-
-`text_generation`'s `system_prompt` may be read from a Hub repo file
-(`{"from_hub": {...}}`). That fetch is anonymous only: the Hub is asked
-without credentials (`token=False`) whether it serves the file publicly at the
-pinned 40-hex commit before every use, and the download carries no token
-either, so the server's own login never reaches a gated or private file on a
-workflow's say-so. It fails closed - any error from that ask, being offline,
-or `HF_HUB_OFFLINE` refuses the run rather than fall back to the cache - and
-it is capped at 1 MiB, checked against the Hub's reported size before the
-download (no reported size is a refusal) and against the file after it.
+`dw-serve` and `dw-validate` take `--trust-workflows`, **off by default**.
+`dw-run` is a client of `dw-serve`, so its jobs run under the server's
+setting. The choice is process-wide, reaches the spawned worker through
+`DW_TRUST_WORKFLOWS`, and `GET /api/server` reports it as `trust_workflows`.
+It is not scoped per workflow or per request: a server that accepts jobs from
+anything other than yourself, an MCP client included, should run without it.
+
+Untrusted (the default), these are refused, each with an error naming what
+triggered it and pointing at `--trust-workflows`:
+
+- **A `pre_load_modules` entry or dotted type outside the allowed packages**
+ (`require_trusted_dotted_name`). Allowed: `diffusers`, `torch`,
+ `torchvision`, `transformers`, `accelerate`, `peft`, the quantization
+ backends `sdnq`, `torchao`, `optimum` (optimum-quanto), `gguf`,
+ `bitsandbytes`, and `dw` itself (for `dw.community_pipelines`). This is
+ checked before anything is imported.
+- **A type reference that is not a constructible class**
+ (`require_loadable_type`, `require_constructible_class`). A `*_type` value
+ is constructed with the workflow's own arguments, so it must be a class
+ defined inside an allowed package (not merely re-exported by one) and one
+ of: a subclass of a diffusers or transformers model, pipeline, modular
+ pipeline, scheduler, tokenizer, processor, feature extractor, image
+ processor or quantization config, or of torchao's `AOBaseConfig`; a class in
+ an auto-pipeline/auto-model factory module; a dataclass in
+ `diffusers.pipelines` or `diffusers.modular_pipelines` with no unreviewed
+ `__post_init__`; or a diffusers attention processor. `"torch.hub.load"` is in
+ `torch` and is refused as not a class. Under a `dtype` or `*_dtype` key a
+ `torch.dtype` (`"torch.bfloat16"`) is accepted too. A bare name
+ (`"FluxPipeline"`) resolves against `diffusers` under the same rules.
+- **A `constant:` walk that leaves the allowed packages.** The module is gated
+ like a type before it imports; no segment may start with `_` (checked before
+ any import); no module the walk passes through may sit outside the allowed
+ packages (`constant:torch.os.environ` is refused at `torch.os`); and
+ `fetch_constant` refuses anything callable. Reading a field off a value
+ declared in an allowed module works. A bare `constant:SOME_NAME` reads from
+ `diffusers`.
+- **Remote code in `from_pretrained_arguments`**: `trust_remote_code` and
+ `custom_pipeline`, for every component and pipeline
+ (`require_trusted_from_pretrained_arguments`). Neither goes through an
+ import of ours, so the dotted-name gate alone would leave them open.
+- **Remote code in a model's `generate()`**: `custom_generate` (a Hub repo
+ whose `custom_generate/generate.py` transformers downloads and runs) and
+ `trust_remote_code`, in `text_generation`'s `generate_kwargs` and in
+ `speech_generation`'s `generate_kwargs` and `forward_params`
+ (`require_trusted_generate_arguments`), before the model loads.
+ `text_generation`'s `generate_kwargs` and `chat_template_kwargs` hold only
+ scalars or lists of them, and none may be one of the pipeline call's own
+ arguments (`tokenizer_encode_kwargs`, `images`, a template).
+- **Remote-code keys anywhere in a task's arguments**: `trust_remote_code`,
+ `custom_pipeline` or `custom_generate`, top level or nested, is a validation
+ error naming the key and its path, and `Task.run` repeats the check for
+ every command before its handler runs (`require_trusted_task_arguments`,
+ GHSA-hhfj-p9xj-xjmr). Trusted, the same keys given as a step argument
+ outside `generate_kwargs`/`forward_params` are still refused, as arguments
+ the task does not take.
+
+`validate_workflow` reports each refusal at the key's path (a variable's
+default at `variables.`), so nothing is queued to find out. Every
+bundled workflow loads untrusted: `tests/test_workflow_trust.py` sweeps the
+catalog, and `tests/test_catalog_structure.py` refuses a catalog entry that
+sets `trust_remote_code` or `custom_pipeline`. `--trust-workflows` lifts all
+of the above except the trusted-mode argument check.
+
+Model code that dw vendors runs no code from its repo, trusted or not:
+
+- **Audio8-TTS** (`speech_generation`, `dw/vendored/audio8_tts_model.py`):
+ five files fetched by `hf_hub_download` at one pinned 40-hex revision;
+ safetensors weights; the `codec.pth` pickle read with
+ `torch.load(weights_only=True)`. Its `generate_kwargs` / `forward_params`
+ pass the trust gate, then an allowlist (`temperature`, `top_p`, `top_k`,
+ `do_sample`, `max_new_tokens`); any other key is refused before the model
+ loads.
+- **MMAudio** (`dw/vendored/mmaudio_model.py`,
+ `dw/vendored/mmaudio_sync_model.py`, `dw/tasks/mmaudio_decoder.py`): the
+ network and Synchformer encoder vendored at upstream commit `974010a0`. The
+ two pickle checkpoints (`weights/mmaudio_large_44k_v2.pth`,
+ `ext_weights/synchformer_state_dict.pth`) are pinned to one 40-hex revision
+ and read with `torch.load(weights_only=True)` and a strict load. The VAE and
+ vocoder are Kandinsky 6 safetensors at a pinned sha, built from
+ `config.json` read as JSON. The modules take local paths and never fetch.
+
+`text_generation`'s `system_prompt` may be a Hub file (`{"from_hub": {...}}`),
+pinned to a 40-hex commit. Before every use the Hub is asked, without
+credentials (`token=False`), whether it serves the file publicly, and the
+download carries no token, so the server's login never reaches a gated or
+private file on a workflow's say-so. It fails closed: any error from that
+ask, being offline, or `HF_HUB_OFFLINE` refuses the run. It is capped at 1
+MiB, checked against the Hub's reported size before the download (no
+reported size is a refusal) and against the file after it.
### Where a workflow may read and reach (`dw/locations.py`)
-Code execution is not the only thing a workflow file chooses. Its arguments
-choose *locations* - which image to open, which URL to fetch, which
-directory a glob expands over - and until 2026-09-13 each loader trusted the
-one it was handed. A workflow could name `/usr/share/pixmaps/debian-logo.png`
-as its `image` and get it decoded, glob `/usr/share/pixmaps/*.png` and get
-every match republished verbatim as an output, or point an `image` at
-`http://127.0.0.1:8765` and have the server fetch its own loopback.
-`remote_text_encoder.url` was the worst of them: the request carries this
-machine's HuggingFace token.
-
-One policy now answers all of it, untrusted:
-
-- **A path** must resolve inside a root this installation already works in -
- the workflow file's own directory, the asset libraries on the search path,
- the output root. `validate_path` already refuses `..`, so in practice this
- closes the absolute path that pointed somewhere else entirely; a relative
- one that climbs out is refused on its `..` segments, at validation time as
- well as at the loader, so both spellings are answered at the same moment
- rather than one of them three seconds into a queued job (#124). The remedy
- for a file outside is to put it in the asset library and use an `asset:`
- reference. Containment is checked **before** existence, so the refusal
- cannot be used as a file-existence oracle. Validation finds a path by its
- argument's name (`image`, `*_video`, `location`, ...); a task argument
- that reads a file under a generic name, like `join_windows`' `source`, is
- declared on its registration (`@register_command(media_arguments=...)`,
- read through `TASK_MEDIA_ARGUMENTS`, #692) so it is refused at the same moment
- (#630), as are the finishing tasks' `media`, `apply_lut`'s `lut` (#635),
- and the face-track and fit tasks' `clip`, `repaired`, `track` and `fit`
- (#773). `lut`, `track` and `fit` are only ever read from a file on the
- server, so an http(s) URL there is refused as a URL
- (`LOCAL_ONLY_TASK_ARGUMENTS`).
-- **A URL with any other scheme** (`file://`, `s3://`, ...) is refused at
- validation and at the loader, whatever the trust posture. No loader opens
- one, but as a relative path it joined onto the workflow directory and
- passed containment, so validation called it clean (#618).
-- **A glob** is contained the same way, on the fixed directory its pattern
- starts from, and every match is re-checked on its real path so a symlink
- cannot carry the expansion out.
-- **An `http(s)` URL** must not resolve to an address inside the deployment -
- loopback, link-local (`169.254.0.0/16`, the cloud metadata address),
- private ranges, and anything else that is not globally routable
- (`is_global`). Checked after DNS resolution, not on the literal string.
- That last rule covers `100.64.0.0/10`, the shared address space, which is
- **Tailscale's tailnet range** (and Alibaba's metadata address): a workflow
- that fetches media from another machine on your tailnet is refused unless
- it runs under `--trust-workflows`.
-- **Every redirect is re-checked.** A media fetch (`safe_get` in `dw/outbound.py`) never lets the
- HTTP client follow a redirect on its own: it follows at most 5 hops
- (`MAX_MEDIA_REDIRECTS`), and each `Location` passes the same scheme and
- host policy before it is dialed. A public URL answering `302` to
- `http://127.0.0.1:8765/api/server` is refused with the target named, and
- nothing is fetched from it. A refused hop's error always begins
- `Refusing a redirect to '' (from ''): ` and
- then gives the rule that refused it (an internal address, a scheme other
- than http(s), an unresolvable host), whereas an error status from a URL
- itself stays the HTTP client's own `403 Client Error: Forbidden for url:
- ...` - so a refused redirect and an upstream failure read differently
- (#1055). Images are decoded from the fetched bytes and
- still go through diffusers' `load_image`, so EXIF orientation and RGB
- conversion are unchanged. Each hop's host is resolved once, by the policy,
- and dialed at the address it checked - so a name that answers differently
- to the lookup and the connect (DNS rebinding) reaches only what was
- checked - and refused when it does not resolve or is percent-encoded (the
- host is read with the HTTP client's own parser, so `127%2e0%2e0%2e1` is
- 127.0.0.1 to the policy too). An untrusted fetch ignores proxy and
- `~/.netrc` settings from the environment. The body is capped at
- `MAX_MEDIA_BYTES` (1 GiB) and the whole fetch at `MEDIA_TOTAL_TIMEOUT`
- (600 s) - the connection is closed at the deadline, so a server
- trickling its response headers or its body cannot hold the worker; a 303 (or a
- 301/302 to a POST) is followed as a GET, as requests itself does; and
- `remote_text_encoder` POSTs through the same path with a timeout. Two limits
- to know: a trusted run behind a SOCKS proxy is not deadline-bounded (SOCKS
- brings its own connection classes), and the TLS handshake is bounded by the
- per-operation timeout only.
-- **`remote_text_encoder.url`** is https-only, and the HuggingFace token is
- attached only for `huggingface.co`, `huggingface.cloud` and `hf.space`. An
- endpoint elsewhere is still reachable; it just does not get the credential.
- A redirect that leaves the host, or drops from https to http, is followed
- without the token.
-- **`model_name`** must be a Hub repo id or a path inside a root - the same
- shape check `download_model` has always applied to `repo_id`. A URL is
- neither, and is refused as such rather than resolving into the workflow's
- own directory as a path-shaped name (#117).
-
-Enforced twice: `location_errors` runs inside `validation_errors`, so
-`validate_workflow` refuses before a model load is spent on the run, and the
-loaders call the same functions for a location that arrives through a
-variable or a previous result. All of it yields to `--trust-workflows`.
-
-`GET /api/server` reports `trust_workflows`, so a client can read the posture
-it is running against rather than infer it.
-
-`--trust-workflows` is a blanket, process-wide choice - it is not scoped
-per-workflow or per-request. A `dw-serve` instance that accepts jobs from
-anything other than yourself (including an MCP client - see below) should
-be run without it.
-
-### MCP-authored workflows (M3)
-
-The MCP server's `save_workflow` and `run_workflow` tools let an LLM write
-and then execute a workflow through `dw.serve` - `save_workflow` writes a
-JSON file into the workflow directory, `run_workflow` queues it (or an
-inline definition) as a job. Nothing in `dw_mcp/` inspects what a workflow
-it saves or runs actually contains. What protects a server used this way
-is exactly the mechanism above: `dw-serve`'s own `--trust-workflows`
-default is untrusted, so an MCP-authored or MCP-submitted workflow gets
-the same code-execution gate a workflow from any other untrusted source
-does, with no code change needed in `dw_mcp` itself. Running `dw-serve
---trust-workflows` removes that gate for every job the server accepts,
-MCP-submitted or not - see the blanket-choice note just above.
+A workflow's arguments also choose *locations*: which file to open, which URL
+to fetch, which directory a glob expands over. Untrusted:
+
+- **A path** must resolve inside a root this installation works in: the
+ workflow file's own directory, the asset libraries on the search path, or
+ the output root. A relative path that climbs out is refused on its `..`
+ segments, at validation as well as at the loader (#124). Containment is
+ checked **before** existence, so a refusal is not a file-existence oracle.
+ The remedy for a file outside is an `asset:` reference. Validation finds a
+ path by its argument's name (`image`, `*_video`, `location`, ...) or by a
+ task's declaration (`@register_command(media_arguments=...)`, read through
+ `TASK_MEDIA_ARGUMENTS`, #692). `apply_lut`'s `lut`, `paste_face_track`'s
+ `track` and `restore_to_source`'s `fit` are read only from a server file,
+ so a URL there is refused (`LOCAL_ONLY_TASK_ARGUMENTS`).
+- **A URL with any other scheme than http(s)** (`file://`, `s3://`, ...) is
+ refused at validation and at the loader, trusted or not (#618).
+- **A glob** is contained on the fixed directory its pattern starts from, and
+ every match is re-checked on its real path, so a symlink cannot carry the
+ expansion out.
+- **An http(s) URL** must not resolve to an address inside the deployment:
+ loopback, link-local (`169.254.0.0/16`, cloud metadata), private ranges, and
+ anything else not globally routable (`is_global`), checked after DNS
+ resolution. That includes `100.64.0.0/10`, **Tailscale's tailnet range**: a
+ workflow that fetches media from another tailnet machine needs
+ `--trust-workflows`.
+- **Every redirect is re-checked.** A media fetch (`safe_get` in
+ `dw/outbound.py`) follows at most 5 hops (`MAX_MEDIA_REDIRECTS`), each
+ `Location` passing the same scheme and host policy before it is dialed. A
+ refused hop is never fetched, and its error begins `Refusing a redirect to
+ '' (from ''): ` followed by the rule; an upstream error status
+ keeps the HTTP client's own message (#1055). Each hop's host is resolved
+ once, by the policy, and dialed at the checked address, so DNS rebinding
+ reaches only what was checked; a host that does not resolve or is
+ percent-encoded is refused. An untrusted fetch ignores proxy and `~/.netrc`
+ settings. The body is capped at `MAX_MEDIA_BYTES` (1 GiB) and the fetch at
+ `MEDIA_TOTAL_TIMEOUT` (600 s), closing the connection at the deadline.
+ `remote_text_encoder` POSTs through the same path. Two limits: a trusted
+ fetch through a SOCKS proxy is not deadline-bounded, and the TLS handshake
+ is bounded only by the per-operation timeout.
+- **`remote_text_encoder.url`** is https-only, and the Hugging Face token is
+ attached only for `huggingface.co`, `huggingface.cloud` and `hf.space`. A
+ redirect that leaves the host, or drops to http, is followed without the
+ token.
+- **`model_name`** must be a Hub repo id or a contained path; a URL is refused
+ (#117).
+
+`location_errors` runs inside `validation_errors`, so `validate_workflow`
+refuses before a model load is spent, and the loaders call the same functions
+for a location that arrives through a variable or a previous result. All of
+it yields to `--trust-workflows`, except the scheme rule.
+
+## HTTP authentication and headers (`dw/server/http_security.py`)
+
+- **Bearer token, opt-in.** With `--token` / `DW_API_TOKEN` set, every
+ `/api/*` and `/mcp` request needs `Authorization: Bearer `, compared
+ in constant time; anything else is a 401. Without a token there is no
+ authentication.
+- **Query token.** Only a GET or HEAD route marked `query_token_ok`
+ (EventSource streams, thumbnails, download links, which a browser loads
+ without headers) accepts `?token=`. The marker is matched on the real route,
+ not a path suffix.
+- **Not gated by the token:** the web UI's static files, `/outputs`, `/inputs`
+ and `/exports/{id}.zip` (#592). Anyone who can reach the port can read
+ generated files and input assets by name.
+- **Binding.** The default bind is `127.0.0.1`. A non-loopback bind without a
+ token logs a warning and starts; `--mcp` on a non-loopback host without a
+ token exits 2. The connection is plaintext HTTP; use a network you control,
+ Tailscale, or a TLS proxy ([REMOTE.md](REMOTE.md)).
+- **Origin check.** A request carrying an `Origin` is refused (403) unless
+ its host is a loopback name, the bind host, or the host the request was
+ addressed to. This stops a drive-by web page queuing jobs, and DNS
+ rebinding, where Origin and Host differ. A request without `Origin` passes.
+- **Host check.** Unless bound to a wildcard address, a `Host` that is not a
+ loopback name or the bind host is a 400.
+- **Response headers.** Every response carries `X-Content-Type-Options:
+ nosniff` and `X-Frame-Options: DENY` (#407).
## Integration Points
-| Entry Point | What's Validated |
-|-------------|-----------------|
+| Entry point | What's validated |
+| --- | --- |
| `workflow.py` | Workflow file paths, JSON size, output directories, sub-workflow paths |
| `run.py`, `validate.py` | CLI arguments, variable names and values |
-| `arguments.py`, `argument_media.py` | Image/video/audio URLs, file paths, file extensions (`validate_media_location`; `fetch_image`, `fetch_video` in `argument_media.py`) |
+| `arguments.py`, `argument_media.py` | Media locations, file paths and extensions (`validate_media_location`; `fetch_image`, `fetch_video`) |
+| `locations.py`, `outbound.py` | Every location a workflow supplies, and every outbound media fetch |
| `tasks/gather.py` | URLs passed to the `gather` task |
-| `result.py`, `writers.py` | Output directories and filenames (`output_file_path` in `writers.py`) |
-| `server/app.py`, `server/jobs.py` | Every HTTP-supplied path — workflow files confined to the workflow directory, gallery files to the output directory, inline-workflow `base_dir`, `Origin`-header guard on every request |
+| `result.py`, `writers.py`, `content_types.py` | Output directories, file names (`output_file_path`), result content types |
+| `server/http_security.py` | Token, Origin and Host on every request |
+| `server/jobs.py`, `server/outputs.py`, `server/routes/*.py` | Every HTTP-supplied path: workflows confined to the workflow directories, gallery files to the output directory, assets to their library, an inline workflow's `base_dir` |
## MCP Server
-`dw_mcp/` introduces no new file access and no authentication of its own. It
-is an HTTP client of a running `dw.serve`: every path a tool touches (a
-workflow name, a gallery file, a job id) is sent to the REST API as-is and
-validated there, exactly as it would be for a browser request from the web
-UI. A remote `dw.serve` is allowed only with a token — see
-[MCP Server](MCP.md#security) and [REMOTE.md](REMOTE.md).
-
-The two exceptions are `download_output`, which writes a local file for the
-MCP client, and `upload_asset(file_path=...)`, which reads one — neither goes
-through the API for that half of its work. Both turn on whose machine "local"
-is. Over a **stdio `dw-mcp`** it is the user's own, so both act for the local
-user the way a shell redirect would: `download_output` writes anywhere the
-process may (a full path, a directory, or the working directory by default,
-`~` expanded) and `upload_asset` reads anything it may. Over **`dw.serve
---mcp`** it is the operator's box, which the caller never chose, so both are
-confined there: `download_output`'s `destination` to the workspace (#113) and
-`upload_asset`'s `file_path` to the directories the server works in — its
-workspace, workflows, assets, outputs and prompts (#138). `upload_asset`'s
-refusal is ordered ahead of the existence and extension checks so it cannot
-be used as a path-existence oracle. A `..` path segment in `destination` is
-refused regardless, and an existing file at the resolved path is left alone
-unless the caller passes `overwrite=True`.
+`dw_mcp/` adds no authentication and no file access of its own, with two
+exceptions below. It is an HTTP client of `dw.serve`: every path a tool sends
+(a workflow name, a gallery file, a job id) is validated by the REST API
+exactly as a browser request from the web UI is. A remote `dw.serve` is
+allowed only with a token ([MCP Server](MCP.md#security),
+[REMOTE.md](REMOTE.md)).
+
+`download_output` writes a local file and `upload_asset(file_path=...)` reads
+one, outside the API. Over a **stdio `dw-mcp`**, "local" is the user's own
+machine, and both act as the user would with a shell redirect. Over
+**`dw.serve --mcp`** it is the operator's box, so both are confined:
+`download_output`'s `destination` to the workspace (#113), and
+`upload_asset`'s `file_path` to the server's workspace, workflows, assets,
+outputs and prompts directories (#138), with the refusal ordered before the
+existence and extension checks so it is not a path-existence oracle. On both
+transports a `..` segment in `destination` is refused and an existing file is
+kept unless `overwrite=True`.
+
+Nothing in `dw_mcp/` inspects what a workflow it saves (`save_workflow`) or
+runs (`run_workflow`) contains. An MCP-authored or MCP-submitted workflow is
+protected by `dw-serve`'s untrusted default, the same gate as any other
+untrusted workflow. `dw-serve --trust-workflows` removes that gate for every
+job the server accepts.
## Exception Hierarchy
```text
SecurityError
- PathTraversalError — path traversal attempt
- InvalidInputError — input validation failure
+ PathTraversalError - path traversal attempt
+ InvalidInputError - input validation failure
+ UntrustedWorkflowError - an untrusted workflow reached the code-execution surface
```
### Disk caches
-`DELETE /api/caches/{name}` (`dw/disk_caches.py`) empties a directory whose path comes from an environment variable, so it never trusts the root: only a root carrying a `.dw-cache` marker is cleared. `dw.serve` writes the marker at start into a root that is missing or empty (a missing `triton` root is never created), or into a populated root equal to the cache's default exactly (`~/.cache/dw/torchinductor` or its `$XDG_CACHE_HOME` form; `~/.triton/cache` only - a populated `$TRITON_HOME/.triton/cache` is not grandfathered). Any other populated root is listed with `clearable: false` ("root not created by dw (no .dw-cache marker)") and its clear is a 400, until someone creates the marker by hand; deleting the marker stops clearing. `quantized` (`DW_QUANTIZED_CACHE_DIR`) follows the same marker rule, and `DELETE /api/caches/{name}/entries/{entry}` deletes one entry only if the id matches `^[0-9a-f]{64}$`, checked before any path is built. The clear refuses while a job is running or queued (for `quantized`, also while a model download is active), and renames the root's children into `.clearing-*` before deleting them. Like model deletion, it needs only the bearer token.
+`DELETE /api/caches/{name}` (`dw/disk_caches.py`) empties a directory whose
+path comes from an environment variable, so it clears only a root carrying a
+`.dw-cache` marker. `dw.serve` writes the marker at start into a root that is
+missing or empty (a missing `triton` root is never created), or into a
+populated root equal to the cache's default exactly (`~/.cache/dw/torchinductor`
+or its `$XDG_CACHE_HOME` form; `~/.triton/cache` only, not a populated
+`$TRITON_HOME/.triton/cache`). Any other populated root is listed with
+`clearable: false` and its clear is a 400 until someone creates the marker by
+hand; deleting the marker stops clearing. `quantized`
+(`DW_QUANTIZED_CACHE_DIR`) follows the same rule, and
+`DELETE /api/caches/{name}/entries/{entry}` deletes one entry only if the id
+matches `^[0-9a-f]{64}$`, checked before any path is built. A clear refuses
+while a job is running or queued (for `quantized`, also while a model download
+is active), and renames the root's children into `.clearing-*` before deleting
+them. Like model deletion, it needs only the bearer token.
## Rules
-- Always validate paths before file operations
-- Use `validate_url()` before loading remote resources
-- If a subprocess is ever introduced, use `shell=False` and pass args through `sanitize_command_args()`
-- Never use dynamic code execution (`eval`/`exec`) or shell interpretation
+- Validate a path before any file operation, with a base directory.
+- Validate a URL with `validate_url()`, and a workflow-supplied location with
+ `dw/locations.py`, before loading a remote resource.
+- Run a subprocess with `shell=False` and its arguments through
+ `sanitize_command_args()`.
+- Never use `eval`, `exec` or `shell=True`.
## Protected Against
-- **Path traversal** — Cannot access files outside allowed directories
-- **Command injection** — No shell interpretation is used anywhere in `dw/`; `sanitize_command_args()` is available as a guard should a subprocess call be added
-- **Resource exhaustion** — File size limits prevent memory exhaustion
-- **Decompression bombs** — an image a caller names is decoded at no more than `MAX_DECODE_PIXELS` (50M; an 8K frame is 33M), checked after `Image.open` and before any decode: `get_output_image` (crops included) refuses it, the gallery thumbnail answers 413, and embedded metadata is read from the PNG header chunks without decoding. Video and audio decode are not limited
-- **Malicious URLs** — Only http/https schemes allowed, and an untrusted
- workflow may not name a host inside the deployment (SSRF)
-- **Arbitrary file read through a media argument** — a location a workflow
- supplies is confined to the roots it may read (`dw/locations.py`)
-- **Script on the UI's origin through an output** — a step's
- `result.content_type` may not be an active document type - `text/html`,
- `application/xhtml+xml`, `text/xml`, `application/xml` or `image/svg+xml`
- (compared without case or parameters): validation refuses it at
- `steps[i].result.content_type` and the writer refuses it again
- (`dw/content_types.py`). A file of an active type that reaches `/outputs`
- or `/inputs` anyway is served with `Content-Security-Policy: sandbox`
+- **Path traversal**: no file outside the allowed directories.
+- **Command injection**: no shell interpretation anywhere in `dw/`.
+- **Arbitrary code from a workflow**: the trust gate, unless
+ `--trust-workflows`.
+- **Arbitrary file read through a media argument**: a workflow-supplied
+ location is confined to the roots it may read (`dw/locations.py`).
+- **Malicious URLs and SSRF**: http/https only, and an untrusted workflow may
+ not reach a host inside the deployment, including through a redirect.
+- **Resource exhaustion**: 50MB workflow JSON, 1 GiB media fetch with a 600 s
+ deadline.
+- **Decompression bombs**: an image a caller names is decoded at no more than
+ `MAX_DECODE_PIXELS` (50M; an 8K frame is 33M), checked after `Image.open`
+ and before any decode. `get_output_image` (crops included) refuses one, the
+ gallery thumbnail answers 413, `compare_outputs` refuses a video whose frame
+ exceeds it (413), and embedded metadata is read from PNG header chunks
+ without decoding. Other video decodes and all audio decodes are not limited.
+- **Script on the UI's origin through an output**: a result `content_type`
+ may not be an active document type (`text/html`, `application/xhtml+xml`,
+ `text/xml`, `application/xml`, `image/svg+xml`, compared without case or
+ parameters). Validation refuses it at `steps[i].result.content_type`, the
+ writer refuses it again (`dw/content_types.py`), and a file of an active type
+ that reaches `/outputs` or `/inputs` anyway is served with
+ `Content-Security-Policy: sandbox`.
+
+Not protected: generated files and input assets are readable without the
+token by anyone who can reach the port, and traffic is plaintext.
## Testing
```bash
-pytest tests/test_security.py tests/test_locations.py tests/test_outbound.py tests/test_workflow_trust.py -v
+pytest tests/test_security.py tests/test_security_auth.py tests/test_security_trust_gate.py \
+ tests/test_security_symlinks.py tests/test_locations.py tests/test_outbound.py \
+ tests/test_workflow_trust.py -v
```
diff --git a/docs/SECURITY_QUICKREF.md b/docs/SECURITY_QUICKREF.md
index 350a1232b..e48e499b8 100644
--- a/docs/SECURITY_QUICKREF.md
+++ b/docs/SECURITY_QUICKREF.md
@@ -1,78 +1,77 @@
# Security Quick Reference
-## MCP Server
-
-`dw_mcp/` adds no new file access and no authentication — it is an HTTP
-client of a running `dw.serve`, and every path a tool sends goes through the
-same validation the REST API already applies. A remote `dw.serve` is
-allowed only with a token, and `dw.serve --mcp` serves the same tool
-surface at `/mcp` behind that token. See [MCP Server](MCP.md#security)
-and [REMOTE.md](REMOTE.md).
+The one-page digest of [SECURITY.md](SECURITY.md), which has the detail and
+the reasons.
## Imports
```python
from dw.security import (
- validate_path,
- validate_workflow_path,
- validate_output_path,
- validate_url,
- validate_variable_name,
- validate_string_input,
- sanitize_command_args,
- SecurityError,
+ validate_path, # no .., ~/, /dev/, /proc/, /sys/; confined only with a base_dir
+ validate_workflow_path, # validate_path + must exist + .json
+ validate_output_path, # validate_path, may not exist yet
+ validate_url, # http/https, a host, no backslash
+ validate_variable_name, # ^[a-zA-Z_][a-zA-Z0-9_-]*$, max 100
+ validate_string_input, # length, no null or control characters
+ sanitize_command_args, # no ` $ | & ; > < newline CR
+ SecurityError, # base of the three below
PathTraversalError,
InvalidInputError,
+ UntrustedWorkflowError,
)
```
+Names joined onto a directory have their own validators
+(`validate_asset_reference`, `validate_output_reference`,
+`validate_prompt_reference`, `validate_subfolder`, `validate_workspace_name`,
+...): see SECURITY.md's *Name Validation* table.
+
## Locations a workflow supplies
-A media path, glob or URL that comes out of a workflow's arguments is not
-just a path - it is untrusted input choosing where the server reads. Use
-`dw/locations.py`, never `validate_path`/`validate_url` directly, for
-anything a workflow names:
+A path, glob or URL from a workflow's arguments is untrusted input choosing
+where the server reads. Use `dw/locations.py`, never
+`validate_path`/`validate_url` alone:
```python
from dw.locations import (
- validate_media_path, # confined to the workflow dir / assets / outputs
- validate_media_glob, # the same, on a pattern's fixed prefix
- contained_matches, # each match re-checked on its real path
- validate_media_url, # no loopback / link-local / private host
+ validate_media_path, # confined to the workflow dir / asset libraries / output root
+ validate_media_glob, # the same, on the pattern's fixed prefix
+ contained_matches, # each match re-checked on its real path
+ validate_media_url, # no loopback / link-local / private / non-global host
validate_model_name, # a Hub repo id, or a contained path
)
```
-Containment is checked before existence, so a refusal never discloses
-whether the file is there. All of it yields to `--trust-workflows`.
+Containment is checked before existence, so a refusal never discloses whether
+the file is there. Redirects are re-checked hop by hop (at most 5). A scheme
+other than http(s) is always refused; the rest yields to `--trust-workflows`.
## Common Patterns
```python
-# File paths
-safe_path = validate_path(user_path, allow_create=False)
+# File paths - pass a base directory, or nothing is confined
safe_path = validate_path(user_path, base_dir="/allowed/dir")
-workflow_path = validate_workflow_path("workflow.json")
+workflow_path = validate_workflow_path("workflow.json", workflow_dir)
output_path = validate_output_path(user_path, base_output_dir)
# User input
-var_name = validate_variable_name("prompt") # OK
+var_name = validate_variable_name("prompt") # OK
var_name = validate_variable_name("bad;name") # raises InvalidInputError
value = validate_string_input(user_input, max_length=1000)
-url = validate_url(user_url) # http/https only
+url = validate_url(user_url) # http/https only
-# Subprocess (dw/ doesn't currently shell out anywhere - pattern for if/when it does)
+# Subprocess - shell=False, a fixed argument list
import subprocess
-cmd = sanitize_command_args(["python", "-m", "dw.run", validated_path])
-subprocess.Popen(cmd, shell=False)
+cmd = sanitize_command_args(["python", "-m", "pip", "install", validated_spec])
+subprocess.run(cmd, shell=False)
```
## Error Handling
```python
try:
- path = validate_path(user_path)
+ path = validate_path(user_path, base_dir=root)
except PathTraversalError as e:
logger.error(f"Path traversal detected: {e}")
except InvalidInputError as e:
@@ -81,33 +80,70 @@ except InvalidInputError as e:
## Input Constraints
-**Variable names:** Letters, numbers, underscores, hyphens. Must start with letter or underscore.
+**Variable names:** letters, digits, underscores, hyphens; must start with a
+letter or underscore.
```text
OK: prompt, num_images, my-variable, _private
Invalid: my.var, var;name, $var, var name
```
-**File paths:** No `../` traversal. No `~/`, `/dev/`, `/proc/`, `/sys/`.
+**File paths:** no `..`, no `~/`, nothing under `/dev/`, `/proc/`, `/sys/`.
```text
OK: ./subdir/workflow.json, /full/path/workflow.json, builtin:h3_context_ir.json
Invalid: ../../../etc/passwd, ~/secret.json
```
-**URLs:** http and https only.
+**URLs:** http and https only, with a host and no backslash.
```text
OK: https://example.com/image.jpg
Invalid: file:///etc/passwd, ftp://server/file
```
+## Trust gate (`dw/trust.py`)
+
+Off by default; `dw-serve --trust-workflows` (and `dw-validate`) lifts it for
+the whole process. Untrusted, a workflow may not:
+
+- import outside `diffusers`, `torch`, `torchvision`, `transformers`,
+ `accelerate`, `peft`, `sdnq`, `torchao`, `optimum`, `gguf`, `bitsandbytes`,
+ `dw` (`pre_load_modules`, dotted `*_type`/`*_dtype`/`dtype`/`config_type`,
+ `constant:`);
+- name a type that is not a constructible class (a model, pipeline,
+ scheduler, tokenizer, processor, quantization config, auto factory,
+ attention processor or reference dataclass), or a `dtype` key that is not
+ a `torch.dtype`;
+- walk a `constant:` through `_` segments or out of those packages, or reach
+ a callable;
+- set `trust_remote_code`, `custom_pipeline` or `custom_generate` anywhere in
+ `from_pretrained_arguments`, `generate_kwargs`, `forward_params` or a task's
+ arguments.
+
+`validate_workflow` reports each refusal at its JSON path.
+
+## HTTP (`dw/server/http_security.py`)
+
+- `--token` / `DW_API_TOKEN` gates `/api/*` and `/mcp` (401 without it).
+ `?token=` works only on GET routes marked `query_token_ok`.
+- `/outputs`, `/inputs`, `/exports/*.zip` and the UI are **not** gated.
+- A foreign `Origin` is a 403; an unknown `Host` is a 400 unless bound to a
+ wildcard address.
+- A non-loopback bind without a token warns; `--mcp` there refuses to start.
+- `dw_mcp/` adds no auth; a remote `--url` needs a token. Over `dw.serve
+ --mcp`, `download_output` and `upload_asset(file_path)` are confined to the
+ server's workspace directories. See [MCP.md](MCP.md#security) and
+ [REMOTE.md](REMOTE.md).
+
## Common Errors
| Error | Cause | Fix |
| ----- | ----- | --- |
-| `Path contains dangerous pattern matching \.\.` | Path traversal | Use absolute or `./` relative paths |
-| `Invalid variable name: my.var` | Special characters | Use `my_var` instead |
-| `Invalid URL: URL scheme not allowed: file` | Non-http scheme | Use https or local file path |
+| `Path contains dangerous pattern matching \.\.` | Path traversal | Use an absolute or `./` relative path |
+| `Path outside allowed directory: ...` | Resolved path left `base_dir` | Put the file inside the root; for media use an `asset:` reference |
+| `Invalid variable name: my.var` | Special characters | Use `my_var` |
+| `Invalid URL: URL scheme not allowed: file` | Non-http scheme | Use https, or a local path |
| `Refusing a redirect to '' (from ''): ...` | A media URL redirected to an internal host or a non-http scheme | Host the media where it is served directly; a refused hop is never fetched |
+| `Refusing to load ...`, ending with a pointer to `--trust-workflows` | The trust gate | Stay inside the allowed packages, or trust the workflow's source |
| `Argument contains dangerous characters: ...` | Shell metacharacters | Remove `` ` `` `$` `\|` `&` `;` `>` `<` |
diff --git a/docs/SERVER.md b/docs/SERVER.md
index 1690f8398..aeee053bc 100644
--- a/docs/SERVER.md
+++ b/docs/SERVER.md
@@ -1,13 +1,13 @@
# Server & Web UI
-`dw.serve` runs the workflow engine as a local HTTP server with a full web
-UI: browse and run workflows, build them in a form-based editor with
-introspection-driven autocomplete, watch jobs stream live progress, review
-past generations in a gallery, and manage the models on disk.
+`dw.serve` runs the workflow engine as an HTTP server with a web UI: browse
+and run workflows, edit them in a form-based editor driven by the live
+pipeline signatures, watch jobs stream progress, review and compare outputs,
+manage input assets and the models on disk.
```bash
python -m dw.serve # http://127.0.0.1:8765
-python -m dw.serve --port 8000 --workflow-dir ./workflows --output-dir ./outputs --prompt-dir ./prompts
+python -m dw.serve --port 8000 --workflow-dir ./workflows --output-dir ./outputs --prompt-dir ./prompts --asset-dir ./assets
# or point it at a workspace, which supplies all four directories
python -m dw.serve --workspace ~/studio
@@ -21,12 +21,61 @@ python -m dw.serve --devices cuda:0,cuda:1 # one worker per card, two jobs at
python -m dw.serve --trust-workflows # only if nothing untrusted can reach POST /api/jobs - see Security model
```
-Installed as a package, the same server is `dw-serve`. Interactive API docs
-(OpenAPI) are at `/docs`.
+Installed as a package, the same server is `dw-serve`. The other flags are
+`--output-layout {run,flat}` ([WORKSPACES.md](WORKSPACES.md#runs)) and
+`-l/--log_level`; `--help` gives each flag's default. Which directories a
+server uses, and in what order they are resolved, is in
+[WORKSPACES.md](WORKSPACES.md). Running it for other machines is
+[REMOTE.md](REMOTE.md); in a container, [DOCKER.md](DOCKER.md).
+
+### The HTTP API
+
+Everything the web UI and the MCP server do goes through one HTTP API, and a
+running server describes it:
+
+| Path | What it is |
+| ---- | ---------- |
+| `/docs` | Swagger UI: every route, its parameters and response, with *Try it out* |
+| `/redoc` | The same document in ReDoc, for reading |
+| `/openapi.json` | The OpenAPI document itself |
+
+The same document is committed as
+[`ui/src/lib/generated/openapi.json`](../ui/src/lib/generated/openapi.json)
+for client generation or reading without a server. It is generated, not
+written: `scripts/dump_openapi.py` writes it from the server, CI fails when
+the committed copy is stale, and its `info.version` is fixed at `"0"` so it
+is the same on every machine. GitHub shows it as plain JSON; to browse it,
+use `/docs` on a running server. Neither document declares the bearer token;
+see [Authentication](#authentication). The sections below cover the
+[Jobs API](#jobs-api), the [Introspection API](#introspection-api) and
+[Files and models](#files-and-models) in prose.
+
+Each job runs in a persistent GPU worker ([WORKER_GUIDE.md](WORKER_GUIDE.md)):
+models stay loaded between runs, so re-running a workflow with a new prompt
+skips the load.
+
+### The command line
+
+`dw.run` is a thin client of `dw.serve`: it queues a job over HTTP, prints
+its progress, and reports the run directory the server wrote to - start the
+server first.
-The server keeps a persistent GPU worker underneath: models stay
-loaded between runs, so re-running a workflow with a new prompt skips the
-load entirely.
+```bash
+python -m dw.run workflows/templates/text-to-image.json
+python -m dw.run workflows/templates/text-to-image.json prompt="a cat" num_images_per_prompt=4
+python -m dw.validate workflows/models/flux-dev.json
+```
+
+`dw.run` takes `WORKFLOW [name=value ...] [--server URL] [--workspace NAME]
+[--token TOKEN]` - `--server` defaults to `DW_MCP_URL`, else
+`http://127.0.0.1:8765`; `--token` to `DW_API_TOKEN`. Where the output lands,
+which workspace's `prompts:`/`asset:` references resolve, and the output
+layout are all `dw.serve` flags (`--workspace`, `--output-dir`,
+`--prompt-dir`, `--asset-dir`, `--output-layout` on the server) - `dw.run`
+only says which server and which of its workspaces to run in.
+
+`python -m dw.test` runs a small built-in workflow end to end - it downloads
+SD 1.5 (a few GB) and generates one image on whichever accelerator was found.
### Environment
@@ -34,170 +83,186 @@ The server reads these from its environment (e.g. `.env`) and its workers inheri
| Variable | Default | Effect |
| -------- | ------- | ------ |
+| `DW_API_TOKEN` | none | The bearer token, when `--token` is not given ([Authentication](#authentication)) |
+| `DW_WORKSPACE`, `DW_PROMPT_DIR`, `DW_OUTPUT_LAYOUT` | unset | The workspace, prompt library and output layout when no flag names them ([WORKSPACES.md](WORKSPACES.md)) |
+| `DW_PUBLIC_URL` | unset | The origin absolute URLs in responses are built from ([REMOTE.md](REMOTE.md#urls-handed-to-a-person)) |
| `DW_QUANTIZED_CACHE` | off | `1` caches SDNQ on-the-fly-quantized components on disk, so a later cold load skips the quantize. Anything else is off. See [QUANTIZATION.md](QUANTIZATION.md#cached-on-the-fly-dw_quantized_cache) |
| `DW_QUANTIZED_CACHE_DIR` | `/dw-quantized` | Where the cache lives; resolved to an absolute path at start and pinned for the workers |
### Choosing the card
-`--devices cuda:1` runs the worker on that card. The worker is pinned to it
-with `CUDA_VISIBLE_DEVICES` (and `DW_DEVICE=cuda`), so inside the worker the
-card is the only one it can see; a bare `cuda`, or no flag at all, pins
-nothing and behaves as before. The `devices` setting is the standing form of
-the same choice, and the flag wins over it. A list (`--devices cuda:0,cuda:1`)
-runs one worker per card, each running one job at a time. A card named twice,
-or a bare `cuda` in a list of several, is refused at startup, as is a card the
-machine lacks, with the message naming the cards that are present
-(`cuda:0 (NVIDIA GeForce RTX 4090)`, ...). A list survives a card being
-pulled: an entry the machine lacks is dropped with a warning naming it, and
-the server starts on the rest. Only a list none of whose cards is present
-(or a single absent card) is refused.
-
-Each job's record carries `device` - the card it ran on, as the server
-addresses it (`"cuda:1 NVIDIA GeForce RTX 3090"`), set when the job starts
-running and null for a job still queued and for jobs recorded before the
-field existed. Observed cost figures are bucketed by it: a figure counts the
-runs from cards of the same name as the one the server runs on now, so a
-3090's runs never inform a 4090's estimate. Rows from before jobs carried a
-`device` count only while the server is on the box's default card.
+`--devices cuda:1` runs the worker on that card, pinned with
+`CUDA_VISIBLE_DEVICES` (and `DW_DEVICE=cuda`), so inside the worker it is the
+only card visible. A bare `cuda`, or no flag, pins nothing. The `devices`
+setting is the standing form of the flag; the flag wins. A list
+(`--devices cuda:0,cuda:1`) runs one worker per card, each running one job at
+a time. Refused at startup: a card named twice, a bare `cuda` in a list, and
+a single card the machine lacks or a list none of whose cards is present -
+the message names the cards that are (`cuda:0 (NVIDIA GeForce RTX 4090)`,
+...). A list entry the machine lacks is otherwise dropped with a warning
+naming it, and the server starts on the rest.
+
+Each job's record carries `device`, the card it ran on as the server
+addresses it (`"cuda:1 NVIDIA GeForce RTX 3090"`): set when the job starts,
+null while queued and for jobs recorded before the field existed. Observed
+cost figures are bucketed by it: only runs from cards of the same name as
+the one the server runs on now count, so a 3090's runs never inform a
+4090's estimate. Rows with no `device` count only while the server is on the
+box's default card.
## The pages
-The UI is organised by workspace. A sidebar lists every workspace on the
-server; the selected one opens into **Overview, Workflows, Jobs, Gallery,
-Assets, Editor**, and the hash carries the workspace (`#/ws/studio/gallery`),
-so a link names where it points and an old `#/gallery` bookmark redirects to
-the last workspace you were in. Below the workspaces, **Shared** holds what
-every workspace sees — the prompt library, the `common` asset library and
-the read-only example workflows — and **Server** holds Models, Schema and
-Status.
-
-- **Overview** — a glance at the workspace: its recent outputs, its recent
- jobs, its workflows (ones with a proof first, plus a filled **New
- workflow** button), its asset count, and disk usage. Each panel loads and
- fails independently, so a slow gallery does not hold the jobs list back
- and a panel's own error shows in place of its content rather than reading
- as an empty workspace. **Manage** on this page is where a workspace is
- deleted (disabled for `default`); a workspace is created from the sidebar.
-- **Workflows** — every workflow on the workspace's own search path
- (`--workflow-dir`), as cards with descriptions, output kinds, and variable
- counts; the read-only ones from an `--examples-dir` are under Shared →
- Examples instead, and opening one's Editor there saves a copy into the
- current workspace. Folders one level deep become sections. Click through
- to a run form generated from the workflow's variables, with the raw JSON
- alongside.
-- **Prompts** — under Shared: the prompt library under `--prompt-dir`
- (default: discovered
- the way a CLI run discovers it, then pinned for every job, so the page and
- `prompt:` resolution always agree), plus the read-only `prompts/` beside
- each `--examples-dir`, so an example's `prompt:` references resolve: stored
- prompts as
- cards with descriptions, intended-model badges, and tags, foldered the
- same way workflows are. Each opens in an editor with form, split, and
- schema-aware JSON views, and an **Enhance with AI** panel that expands an
+The sidebar lists every workspace on the server (`+ new` creates one; names
+sharing a prefix fold into one group). The selected workspace opens into
+**Overview, Workflows, Jobs, Gallery, Assets, Editor**, and the hash names
+it (`#/ws/studio/gallery`), so a link and a reload land where they say; a
+pre-workspace bookmark (`#/gallery`) redirects to the last workspace used.
+Below the workspaces, **Shared** holds what every workspace sees - Prompts,
+the `common` Assets library and the read-only Examples - and **Server**
+holds Models, Schema and Status. The header carries the breadcrumb, a VRAM
+meter per card with a status popover (workers, queue), the API-token key
+icon, a finish-alert toggle, the theme toggle, and links to these docs and
+`/docs`. `?` lists the keyboard shortcuts.
+
+- **Overview** - the workspace at a glance: its note, recent outputs, the
+ running and last finished jobs, its workflows (ones with a proof image
+ first, and a **New workflow** button), its asset count and disk usage. Each
+ panel loads and fails on its own, so a slow gallery does not hold the jobs
+ list back and an error shows in place of that panel. **Manage** deletes
+ the workspace (disabled for `default`).
+- **Workflows** - every workflow on the workspace's own `--workflow-dir`, as
+ cards with the last image it made, its summary, the server's observed run
+ time, and `configures` / read-only marks. Shape buttons and trait chips
+ filter by what a workflow makes (the `shape`/`traits` vocabulary of
+ `GET /api/workflows`); folders become collapsible groups. **Shared →
+ Examples** is the same page over the `--examples-dir` trees. A card opens
+ the workflow's page: its recent outputs, a run form generated from its
+ variables (with `asset:`/`prompt:`/`output:` pickers), the raw JSON on
+ request, **Edit**, **Duplicate**, **Run** and delete. Opening an example in
+ the editor saves a copy into the current workspace.
+
+ 
+
+- **Editor** - build or modify a workflow without writing JSON by hand.
+ Forms come from the live pipeline signatures
+ ([Introspection API](#introspection-api)), and references (`variable:`,
+ `previous_result:`, `asset:`, `prompt:`, `output:`, `builtin:`) autocomplete
+ or open a picker. Four views: **form**, **split** (form beside JSON, both
+ editable, applied when a side loses focus), **json** (Monaco with the
+ workflow schema), and **flow**, a read-only data-flow graph - a box per
+ step, an arrow per `previous_result` labelled with the argument it feeds,
+ entry steps marked, and fan-in steps flagged with their cartesian-product
+ multiplier where it is known statically. Clicking a step in the flow jumps
+ to it in the form. Validate, save and run from the same screen; a valid
+ verdict shows the run's plan - step and list counts, the estimate with its
+ basis, how many steps the step cache would serve, and the weights to
+ download first (`POST /api/validate`'s `plan`, rendered by
+ `ui/src/lib/plan.ts`).
+
+ 
+
+- **Jobs** - the workspace's queue and run history (history persists in
+ `~/.diffusers_helper/jobs.sqlite`), filterable by status; a waiting job
+ shows its place in line, can be moved, and shows `waiting_for` while held
+ for host RAM. A job's page streams step progress, denoise ticks, the phase
+ between them (loading, decoding, saving), its log, and its results as they
+ land - finals first, intermediates folded, and a section per
+ `result.subfolder` when a step chose one. **Cancel**, **Run again**,
+ **New seed**, and **Export** (gathers the run into the workspace's `exports/` with
+ `POST /api/jobs/{id}/export` and downloads the zip - the same bundle as
+ MCP's `export_job`; replacing an existing export asks first).
+
+ 
+
+- **Gallery** - everything in the workspace's output directory
+ (`//`, see [WORKSPACES.md](WORKSPACES.md#runs)), grouped
+ by workflow, with a text filter and **Final** / **Intermediate** /
+ **Starred** chips (it opens on finals). A file's detail panel shows its
+ settings, the job that made it, what it was made from, its run and file
+ notes, a star (`S`), **Keep as asset** (promotes it into the asset library
+ under a stable name, so a later workflow carries `asset:` rather than
+ a run id that pruning would break), and **open as workflow**, which loads
+ an `embed_metadata` image's embedded definition into the editor with its
+ seed pinned. Each tile has a checkbox (shift-click extends a range,
+ **Select all** takes what the filter shows); a selection downloads as one
+ zip, is deleted in bulk (anything that fails to delete stays selected), or,
+ for two to four files, opens in Compare.
+
+ 
+
+- **Compare** - two to four outputs side by side, with the arguments that
+ differ between their runs called out, each run's time, and every video on
+ one transport so a difference in motion is seen at the same frame.
+
+ 
+
+- **Assets** - the input side of the gallery: the asset library as folder
+ groups of a contact-sheet grid with a detail panel, notes, upload (button
+ or drop; the library it lands in is named) and delete. A workspace's page
+ shows its own library and only the shared and examples assets its
+ workflows use (`used_by`); **Shared → Assets** shows the whole `common`
+ library. Entries a nearer library shadows are listed by name under their
+ library; read-only tiles have no checkbox.
+- **Prompts** (Shared) - the prompt library (resolved as in
+ [WORKSPACES.md](WORKSPACES.md#the-prompt-library) and pinned for every job,
+ so the page and `prompt:` resolution agree), plus the read-only `prompts/`
+ beside each `--examples-dir`: cards with descriptions, intended-model
+ badges and tags, foldered like workflows. Each opens in an editor with
+ form, split and JSON views and an **Enhance with AI** panel that expands an
idea into a full prompt with a local language model (a preset per target
- model family; the model runs as an ordinary queued job). A workflow
- argument written as `prompt:name` loads the stored text at run time,
- and deleting a prompt warns which workflows reference it.
-- **Jobs** — the queue and full run history (persisted in
- `~/.diffusers_helper/jobs.sqlite`). A workspace's Jobs page lists its own;
- Server → Status lists every workspace's with a filter. A running job streams step-by-step
- progress, per-step denoising ticks, what each step is doing when it is not
- denoising (loading a model, decoding, saving), and its result files as
- they land.
- A run whose steps chose a `result.subfolder` shows its results under
- `final/` and `intermediate/` headings (or whatever the step named), the
- deliverable first; one that chose none shows them as before.
- Jobs can be cancelled mid-denoise and re-run with one click. A finished
- job's **Export** button gathers the run into the workspace's `exports/`
- on the server (`POST /api/jobs/{id}/export`) and downloads it as one zip -
- the same bundle MCP's `export_job` makes; an export that already exists
- asks before it is replaced.
-- **Editor** — build or modify workflows without writing JSON by hand.
- Forms are generated from the live pipeline signatures (see
- [introspection](#introspection)), references
- (`variable:` / `previous_result:`) autocomplete from the workflow itself,
- and three views — form, split, and raw JSON — edit the same definition.
- The split view puts the form beside the JSON with both sides editable;
- changes apply when a side loses focus. Validate, save, and run from the
- same screen; a valid verdict is followed by the run's plan - the step
- and list counts, the minutes from the workflow's `cost` block with its
- basis, how many steps the step cache would serve, and the weights this
- server would download first (`describePlan`, `ui/src/lib/plan.ts`,
- reading `POST /api/validate`'s `plan`). A Monaco editor with the workflow JSON schema backs the
- JSON views. A fourth view, **flow**, renders the workflow's data-flow
- graph read-only: one box per step, arrows for each `previous_result`
- reference labeled with the argument it feeds, entry-point steps marked
- apart from steps that depend on earlier ones, and fan-in points -
- steps combining more than one upstream producer - flagged with the
- cartesian-product multiplier where it's known statically (e.g. a
- literal `num_images_per_prompt` on both producers). It's a diagram of
- the JSON, not a second way to edit it; clicking a step jumps to it in
- the form view.
-- **Gallery** — everything in the selected workspace's output directory,
- which the engine lays out as `//`. The folder filter groups a workflow's runs
- together rather than listing each run separately, and a **subfolder** pick
- beside the text filter - offered once any output landed in one - narrows
- the grid to `final/`, `intermediate/` or whatever a step's
- `result.subfolder` named; each run directory
- also holds a `manifest.json` describing what produced it (see
- [Workspaces](WORKSPACES.md#runs)). Images generated with
- `embed_metadata` carry their full workflow definition and seed; **open as
- workflow** loads that definition into the editor with the seed pinned, so
- any image can be reproduced or riffed on. Each tile carries a checkbox
- (shift-click extends a range, **Select all** takes whatever the filter
- leaves showing); a selection can be downloaded as one zip or deleted in
- bulk, which is how a directory that fills up over a few hundred runs gets
- cleared out. Anything that fails to delete stays selected. **Keep as
- asset** promotes one generated file into the workspace's asset library
- under a stable name, so a later workflow can reference it as
- `asset:` instead of a run id that pruning would break.
-- **Models** — the Hugging Face hub cache: every cached repo with sizes,
- revisions, and last-used dates, plus free disk space. Download a repo by
- id with live progress (cancellable; partial files resume on retry), and
- delete to free disk (refused while a job is running; the next workflow
- that needs the model downloads it again). The page also shows the
- installed diffusers version (with its commit for a git install) and can
- upgrade it to GitHub HEAD - new model pipelines usually land there
- before a PyPI release. The idle worker restarts on success so the next
- job imports the new version; the upgrade is refused while a job runs.
-- **Schema** — the workflow JSON schema the running server validates
- against, as a browsable tree: the document root plus every definition,
- with types, required markers, defaults, enums, and descriptions.
- `$ref` labels jump to their definition; a filter narrows the list.
-- **Server → Status** — what this server is and how to reach it: device,
- version, bind address and LAN addresses, whether a token is required,
- whether `/mcp` is mounted (with the `claude mcp add` line to connect to
- it), and the directories in use — and, below, the queue across every
- workspace. Workspaces are created from the sidebar and deleted from their
- Overview.
+ model family, run as an ordinary queued job). Deleting a prompt warns which
+ workflows reference it.
+
+ 
+
+- **Models** (Server) - the Hugging Face hub cache: each cached repo with
+ sizes, revisions and last-used dates, plus free disk. Download a repo by id
+ with live progress (cancellable; partial files resume on retry), and delete
+ one to free disk (refused while a job runs). It also shows the installed
+ diffusers version (with its commit for a git install) and **Update from
+ GitHub** installs HEAD - new pipelines usually land there before a PyPI
+ release. Refused while a job is running or queued; on success the idle
+ worker restarts so the next job imports the new version. Pinning a commit
+ or reverting is the API's (`POST /api/system/diffusers/update`, below).
+
+ 
+
+- **Schema** (Server) - the workflow JSON schema the server validates
+ against, as a browsable tree: the root and every definition, with types,
+ required markers, defaults, enums and descriptions. `$ref` labels jump to
+ their definition; a filter narrows the list.
+- **Status** (Server) - what this server is and how to reach it: workers per
+ card, bind address and LAN addresses, whether a token is required, whether
+ `/mcp` is mounted, a **Connect from another machine** panel with the
+ browser URL and the `claude mcp add` line for a chosen address, the
+ resolved directories, and the Caches panel (below) - then the queue and
+ history across every workspace, with a workspace filter.
+
+ 
## Workspaces
-One server can hold several workspaces — each with its own `workflows/`,
-`assets/` and `outputs/`, all sharing the root's one prompt library. The
-root's own folders are the workspace named `default`.
-
-Every scoped route takes an optional `?workspace=`; omitting it means
-`default`, so nothing written against a single-workspace server changes
-meaning. `POST /api/jobs` also accepts `"workspace"` in the body, and a job
-holds onto the directories it was submitted with — through the run, a rerun,
-and when history serves its files back. `GET /api/jobs` spans every workspace
-unless one is named.
-
-A workspace is a namespace, not a security boundary: the API token is
-all-or-nothing. See [Workspaces](WORKSPACES.md#several-workspaces-on-one-server).
-
-The sidebar lists every workspace, and `+ new` there creates one; a
-workspace's own Overview page is where it is deleted. Server → Status shows
-the resolved directories and the `claude mcp add` line for connecting an
-agent from another machine, plus the queue across every workspace. Its
-Caches panel lists dw's disk caches (`GET /api/caches`: size, files, last
-used, the volume's free space; the quantized cache also expands into a table
-of its entries with a delete button each) with a Clear button per cache that asks first,
-quoting what clearing costs, and shows the server's refusal when a job is
-running:
-
-
+One server can hold several workspaces, each with its own `workflows/`,
+`assets/` and `outputs/`, sharing the root's prompt library; the root's own
+folders are the workspace named `default`. Every scoped route takes an
+optional `?workspace=` (omitted means `default`), `POST /api/jobs` also
+accepts `"workspace"` in the body, and a job keeps the directories it was
+submitted with through the run, a rerun, and when history serves its files.
+`GET /api/jobs` spans every workspace unless one is named. A workspace is a
+namespace, not a security boundary: the API token is all-or-nothing. The
+layout, the reserved names and how each client picks one are in
+[WORKSPACES.md](WORKSPACES.md#several-workspaces-on-one-server).
+
+The Status page's Caches panel lists dw's disk caches (`GET /api/caches`:
+size, files, last used, the volume's free space; the quantized cache expands
+into a table of its entries, each with a delete button) with a Clear button
+per cache that asks first, quoting what clearing costs, and shows the
+server's refusal when a job is running:
+
+
+
+
## Jobs API
@@ -210,66 +275,68 @@ running:
| `GET /api/jobs/{id}/workflow` | The workflow the job ran: `{id, definition, realized, seed_variable}`. `seed_variable` names the variable a `new_seed` rerun would draw into (null when the workflow has none), read from the workflow as written rather than the realized copy, whose seed is pinned. `realized: true` is the copy the run itself wrote (`workflow.json` in its run directory), with arguments, seed, prompts and `output:latest` pinned; `false` falls back to the submitted definition, which is what a job from before run tracking has. 404 means neither is readable - the job itself still is. The equivalent MCP tool is `get_job_workflow` (see [MCP.md](MCP.md#diagnose)) |
| `POST /api/jobs/{id}/export?workspace=&overwrite=` | Gather one finished job into `/exports//`: `workflow.json`, `manifest.json`, `job.json`, `README.md`, `assets/`, `inputs/`, `outputs/`. 201 with the file list, total bytes, anything it could not find, a `zip_url`, and the three JSON files inline. 404 unknown job, 409 for a job still running or an existing export without `overwrite` |
| `GET /exports/{id}.zip?workspace=` | The same tree as one archive, built on request rather than kept as a second copy. Entries are named `/`. Ungated exactly as `/outputs` is |
-| `GET /api/jobs/{id}/events` | Server-sent events stream; `?after=N` / `Last-Event-ID` replay missed events, so reconnects are lossless. Every event carries `seq` and `at` - seconds since the job started (since it was queued, for the events before that) - so a step's cost is a subtraction: `step_start` to `generating` is what a reused pipeline still pays before it runs; `generating` is emitted when the pipeline call starts, so `generating` to the first `pipeline_step` is the prompt and reference encoding plus the first denoise pass (`timing.encode_s` in the manifest is the encoding alone; see [Step timing](WORKSPACES.md#step-timing)). The server keeps only the last 200 events, so on a long job the early ones are gone |
-| `GET /api/jobs/{id}/event-log?after=-1&limit=200` | The same events as the SSE stream, as one JSON page: `{id, status, events, last_seq, truncated, note}`. `after` is exclusive; page by passing back the previous `last_seq`. A job restored from history serves the bounded event tail persisted with it; a job that finished before events were retained returns an empty list and a `note` saying so. |
-| `POST /api/jobs/{id}/cancel` | Cooperative cancel (takes effect at the next step boundary or denoise step) |
+| `GET /api/jobs/{id}/events` | Server-sent events stream; `?after=N` / `Last-Event-ID` replay missed events, so reconnects are lossless. Every event carries `seq` and `at` - seconds since the job started (since it was queued, for the events before that) - so a step's cost is a subtraction: `step_start` to `generating` is what a reused pipeline still pays before it runs; `generating` is emitted when the pipeline call starts, so `generating` to the first `pipeline_step` is the prompt and reference encoding plus the first denoise pass (`timing.encode_s` in the manifest is the encoding alone; see [Step timing](WORKSPACES.md#step-timing)). A job in memory keeps every event; one served from history (after a restart, or once it is older than the 20 most recent finished jobs) keeps only its last 200 |
+| `GET /api/jobs/{id}/event-log?after=-1&limit=200&kinds=` | The same events as the SSE stream, as one JSON page: `{id, status, events, last_seq, truncated, note}`. `after` is exclusive; page by passing back the previous `last_seq`. `limit` is capped at 1000. `kinds` (repeatable) keeps events whose `event` or `kind` is one of those named (`log`, `warning`, `phase_stall`, ...). A job restored from history serves the bounded event tail persisted with it; a job that finished before events were retained returns an empty list and a `note` saying so. |
+| `POST /api/jobs/{id}/cancel` | Cooperative cancel: takes effect at the next step boundary or denoise step. During a model load or a task step, which cannot be interrupted, a `cancel_pending` event says it waits for that to finish |
| `POST /api/jobs/{id}/rerun` | Re-queue a finished job's spec. Body `{"new_seed": true}` draws a fresh seed into the workflow's seed variable instead of repeating the original arguments; 400 when the workflow pins its seed to a literal or names none. A plain rerun of a seeded workflow is served whole from the step cache — the earlier run's files, republished with `reused: true`, generating nothing. The new job's summary and detail carry `rerun_of`: the id of the job it reran (absent on a fresh submit) |
| `POST /api/jobs/{id}/move` | Reorder a queued job: `{"direction": "up"\|"down"\|"front"\|"back"}`. Job listings carry each waiting job's `queue_position`, and `waiting_for` while one is held for host RAM. |
+### Dispatch
+
Each worker runs one job at a time, so with one device (the default) one job
-runs at a time; submissions queue in order, and the waiting portion of the
+runs at a time; submissions queue in order, and the waiting part of the
queue can be reordered. With several `--devices` the dispatcher walks the
queue in order and gives each job the first free card, in `--devices` order,
that fits it. A job that fits no free card keeps its place while a smaller
one behind it takes the card (backfill).
-What a job needs: the workflow's declared `vram_estimate` projected over the
-job's arguments (hard: a job bigger than every card is refused at submit with
-a 400 naming the largest card); else the smallest `cost` entry's `vram_gb` for
-this device type (soft: if no card is that big, the job runs on any card, as
-on one card); else any card. A card's size is its GiB rounded up (a 3090
-counts as 24), which a `cost` figure is compared to; a declared estimate is
-held to the card's GiB to one decimal (a 3090's 23.6), the same ceiling
-admission checks it against. Cancel reaches only the worker running that job, and a worker
-crash fails only its own job. Each dispatch sets its worker's Linux
-`oom_score_adj` to 1000 and every other live worker's back to the value it was
-spawned with (the server's own, e.g. 200 under a systemd user unit - the kernel
-refuses an unprivileged write below that), so a host-RAM squeeze kills the
-newest job whatever the workers' sizes; nothing is written with one worker, and
-a write the kernel refuses is a warning on the job. Under systemd the unit needs
-`OOMPolicy=continue` (`scripts/dw-serve.service` sets it): the default `stop`
-answers one worker's OOM kill by stopping the whole server, every other job
-with it. A
-queued job may also carry a projected host-RAM peak (see
-`host_ram_need`, #869). When it fits a free card, dispatch checks host RAM
-too: available is `MemAvailable` less, for each running job, what that job's
-projected peak still has to add beyond its worker's resident size now, and
-the job is held while its peak exceeds available plus the resident size of
-the idle worker it would reuse. A running job with no figure is charged
-nothing extra, since `MemAvailable` already reflects what it holds. Backfill
-applies: a smaller job behind a held one may start. With no other job
-running a job is never held, so it waits only for running jobs, never
-forever; a job with no figure is never held; and where the host's RAM cannot
-be read (macOS without psutil) there is no gate. With one card only one job
-runs, so the gate is never exercised. A held job is re-checked every 10 s,
-since memory can free outside dw. The hold emits one `queue_wait` event
-(`reason: "host_ram"`, message `waiting for host RAM: needs ~N GB, ~M GB
-free`; not a `warning`, so nothing lands in the job's `warnings`), and job
-detail and summaries carry `waiting_for` - that message - only while the job
-is queued and held. The web UI shows it beside "#n in line". The gate never
-refuses a submit (#243); it only delays a start, and the OOM ranking above
-(#881) stays the backstop for a wrong projection.
-Among the free cards a job fits, dispatch prefers the one it has an affinity
-for: a rerun goes to the card that ran the original, and any job to the card
-whose worker last ran the same workflow (the same file, or an inline
-definition's `id`) - that worker's pipelines and step cache are warm, and a
-worker's step cache and any model the next workflow does not load are freed
-when the workflow changes, while a model both workflows load (same weights
-identity) stays warm. `clear_memory` still empties the card by hand. A busy preferred card is not
-waited for. A validation's cache probe and its `plan.estimate` ask about the
-card the job would be dispatched to now, and the estimate names it in
-`priced_for`. Memory is per card: `GET /api/memory` and `POST
-/api/memory/clear` take an optional `device`.
+What a job needs, in order:
+
+- the workflow's declared `vram_estimate` projected over the job's
+ arguments - hard: a job bigger than every card is refused at submit with a
+ 400 naming the largest card;
+- else the smallest `cost` entry's `vram_gb` for this device type - soft: if
+ no card is that big, the job runs on any card;
+- else any card.
+
+A card's size is its GiB rounded up (a 3090 counts as 24) when compared to a
+`cost` figure; a declared estimate is held to the card's GiB to one decimal
+(a 3090's 23.6), the ceiling admission uses. Cancel reaches only the worker
+running that job, and a worker crash fails only its own job.
+
+**OOM ranking (#881).** With more than one worker, each dispatch sets its
+worker's Linux `oom_score_adj` to 1000 and every other live worker's back to
+its spawn-time value (the server's own, e.g. 200 under a systemd user unit;
+the kernel refuses an unprivileged write below that), so a host-RAM squeeze
+kills the newest job. A refused write is a warning on the job. Under systemd
+the unit needs `OOMPolicy=continue` (`scripts/dw-serve.service` sets it): the
+default `stop` answers one worker's OOM kill by stopping the whole server.
+
+**Host-RAM hold (#869, #243).** A queued job may carry a projected host-RAM
+peak (`host_ram_need`; see
+[Host-memory history](WORKSPACES.md#host-memory-history)). When it fits a
+free card, dispatch also checks host RAM: available is `MemAvailable` less
+what each running job's projected peak still has to add beyond its worker's
+resident size now (nothing for a running job with no figure), and the job is
+held while its peak exceeds available plus the resident size of the idle
+worker it would reuse. Backfill applies. A job is never held when no other
+job is running, nor when it has no figure, nor where host RAM cannot be read
+(macOS without psutil), and with one card the gate is never exercised. A held
+job is re-checked every 10 s. The hold emits one `queue_wait` event (message
+`waiting for host RAM: needs ~N GB, ~M GB free`; not a `warning`), and job
+detail and summaries carry `waiting_for` - that message - only while held.
+The gate never refuses a submit; it only delays a start, and the OOM ranking
+is the backstop for a wrong projection.
+
+**Affinity.** Among the free cards a job fits, dispatch prefers a rerun's
+original card, else the card whose worker last ran the same workflow (the
+same file, or an inline definition's `id`), whose pipelines and step cache
+are warm. A busy preferred card is not waited for. When a worker's workflow
+changes, its step cache and any model the next workflow does not load are
+freed ([WORKER_GUIDE.md](WORKER_GUIDE.md)). A validation's cache probe and
+its `plan.estimate` ask about the card the job would go to now, named in
+`priced_for`. `GET /api/memory` and `POST /api/memory/clear` take an
+optional `device`.
### Step timing
@@ -297,6 +364,14 @@ Every event in the stream carries a `seq` and an `event` name:
| `warning` | a step finds something wrong with what it is about to write | `message`, plus a `kind` and the figures behind it (`level_spread`: `spread_db`, `measure`, `command`; `fps_mismatch`: `declared_fps`, `source_fps`; `audio_no_headroom`: `file`, `peak_dbfs` - a deliverable at or above -0.5 dBFS, which an mp3 or AAC encode decodes over full scale; `step_elided`: `step`, `overridden_by` when a supplied argument is what made it unreferenced; `cache_stats_unavailable`: `cache_type` - the installed diffusers moved the private hook state dw reads, so the step's cache counts are null; the run is unaffected). Also appended to the job's `warnings`, prefixed with the step it fired in - the event keeps the moment, `warnings` keeps it where a caller polling the finished job will look, since a warning about the artifact outlives the run that noticed it |
| `cache_stats` | a pipeline call that set a `cache` finishes its denoise, once per call | `step`, `segment` (only on a chain segment, e.g. `segment 1/3`), and the fields of `timing.cache`: `type`, `threshold`, `forwards`, `computed`, `skipped`, `hit_rate`. Counts are null for a cache diffusers keeps no per-forward decision for. For watching live (`get_job_events(kinds=["cache_stats"])`); the manifest entry's `timing.cache` is the record |
| `workflow_end` | the run finishes | `manifest` |
+| `queue_wait` | a queued job is held for host RAM (below) | `reason: "host_ram"`, `message` |
+| `cancel_pending` | a cancel arrives during a phase that cannot be interrupted | `phase` |
+
+A `warning` with `kind: "phase_stall"` is emitted when a phase has gone 30 s
+without an event, and every 30 s while it stays silent (`phase`,
+`seconds_since_phase_start`, `seconds_since_last_progress`). It is
+informational - long silent stretches are normal for some models - and,
+unlike other warnings, never lands in the job's `warnings`.
A step spends most of its wall clock outside the denoise loop, and
`pipeline_step` cannot see any of it. `phase` is what fills that silence:
@@ -325,34 +400,25 @@ to learn where a long render is:
| `seconds_since_event` | how long since anything at all happened - the number that separates a slow run from a stuck one |
| `denoise_step`, `denoise_total_steps` | the denoise loop's counter, `null` until it starts |
-A null `denoise_step` under `generating` is the pipeline's lead-in - encoding
-the prompt and every reference - which emits nothing and runs well over a
-minute on a large video model. It is the encode plus the first denoise
-pass, and on MiniMax H3 the first pass dominates and varies by card. The
-reference encode (`vae_encoder`) is about 6 s for an image reference and
-about 50-53 s for a 5 s 960x544 video reference. Measured on lem: 187 s for
-an image reference (job `f90a041af26a`), and 508 s on cuda:1
-(`f5aefcdd5e26`, first pass 422 s) against 165 s on cuda:0
-(`6961876f11d3`, first pass 86 s) for the same image + video job. The keys
-are always present so that lead-in can be told from a loop that has stopped
-advancing: `seconds_since_event` is a stall signal once `denoise_step` is a
-number, or in any phase other than `generating`.
-
-The lead-in is no longer silent, though: each of a modular pipeline's
-top-level blocks emits a `log` naming it as it starts (`before_encode`,
-`text_encoder`, `vae_encoder`, `denoise`, `decode` on H3), so the last event
-says which one the run is inside. Only a `SequentialPipelineBlocks` is
-narrated this way - a conditional container picks one branch rather than
-running them all, and walking its sub-blocks would be a wrong answer bought
-with a progress message.
-
-It is a coarse one even then. A step's cost is not uniform when the pipeline
-configures a transformer block cache (`"cache": {"type": "first_block"}`):
-most steps are served from it in seconds and every few steps one is computed
-in full, so the same healthy run emits four `pipeline_step` events in 20 s
-and then nothing for 133 s. Liveness is `denoise_step` having moved between
-polls minutes apart, not silence measured against a fixed threshold - on H3
-that threshold would have to exceed ~140 s to mean anything.
+A null `denoise_step` under `generating` is the pipeline's lead-in: prompt
+and reference encoding plus the first denoise pass, which runs well over a
+minute on a large video model. On MiniMax H3 the first pass dominates and
+varies by card (measured on one box: 86 s on one card, 422 s on another, for
+the same image + video job); the reference encode (`vae_encoder`) is about
+6 s for an image and 50-53 s for a 5 s 960x544 video. Each top-level block of
+a `SequentialPipelineBlocks` modular pipeline emits a `log` as it starts
+(`before_encode`, `text_encoder`, `vae_encoder`, `denoise`, `decode` on H3),
+so the last event says which block the run is in; a conditional container
+is not narrated, since it runs one branch. `seconds_since_event` is a stall
+signal once `denoise_step` is a number, or in any phase other than
+`generating`.
+
+Even then it is coarse. With a transformer block cache
+(`"cache": {"type": "first_block"}`) most steps are served in seconds and
+every few one is computed in full, so a healthy run can emit four
+`pipeline_step` events in 20 s and then nothing for 133 s. Liveness is
+`denoise_step` having moved between polls minutes apart, not silence against
+a fixed threshold; on H3 that threshold would have to exceed ~140 s.
## Introspection API
@@ -360,9 +426,10 @@ The editor's forms come from these; they are just as usable from scripts:
- `GET /api/pipelines`, `GET /api/pipelines/{name}` — diffusers pipeline
classes and their call signatures
-- `GET /api/classes?kind=...`, `GET /api/classes/{name}?target=call|init|load` —
- any allowed class (diffusers + registered extension modules), described
- for calling, constructing, or `from_pretrained` loading
+- `GET /api/classes?kind=pipelines|models|schedulers|quantization`,
+ `GET /api/classes/{name}?target=call|init|load` (default `init`) — any
+ allowed class (diffusers + registered extension modules), described for
+ calling, constructing, or `from_pretrained` loading
- `GET /api/tasks` — the task commands and processors
- `GET /api/tasks/{command}` — a task's argument schema, read from its
registered implementation's real signature
@@ -376,11 +443,10 @@ The editor's forms come from these; they are just as usable from scripts:
- `GET /api/guides/{name}?section=` — one section of a guide; section names
match loosely. Without a `section` the answer is the guide's index - its
opening, its first section, and `sections`/`withheld` naming the rest -
- rather than the whole file, which for WORKFLOW_GUIDE.md is ~19.6k tokens
- in one call (#101). Served by the engine so an MCP client
- at another version reads the guides for the server it is driving, not
- its own. A checkout serves the repo's `docs/`; an install the copy
- `build_dist.sh` puts under `dw/docs/`
+ rather than the whole file (#101). Served by the engine, so an MCP client
+ at another version reads the guides for the server it drives. A checkout
+ serves the repo's `docs/`; an install the copy `build_dist.sh` puts under
+ `dw/docs/`
- `POST /api/validate` — schema validation plus signature-level argument
warnings for pipeline and task steps (catches the typo before the model
loads); `warnings` also names an entry key of a list-driven variable that
@@ -540,25 +606,20 @@ The editor's forms come from these; they are just as usable from scripts:
already resident), with the `drivers` the figure is for, `since`, and
`unclassified_runs` when a run's persisted events were trimmed past its
`loading` phase. Runs are bucketed by the workflow's declared
- `cost_drivers` - the variables that move its cost - so a 345-frame run
- never informs a 124-frame figure; a list driver buckets on its length. A
- workflow declaring no drivers falls back to runs that overrode nothing at
- all, and a run whose every step was a step-cache hit is excluded. The
- compact view carries only `observed_minutes` (cold) and `observed_runs`;
- `GET /api/workflows/{name}/variables` carries the whole block beside the
- defaults. Derived from the job rows in one query - so the figures outlive
- a pruned run directory - and cached against the jobs table's high-water
- mark rather than a file mtime, because a job landing changes every figure
- and changes no file. `observed` never replaces `cost`: a maintainer's
- claim on a named card and this machine's last week are different things.
+ `cost_drivers`, so a 345-frame run never informs a 124-frame figure; a
+ list driver buckets on its length. A workflow declaring no drivers counts
+ only runs that overrode nothing, and a run whose every step was a
+ step-cache hit is excluded. The compact view carries only
+ `observed_minutes` (cold) and `observed_runs`;
+ `GET /api/workflows/{name}/variables` carries the whole block. Derived from
+ the job rows, so the figures outlive a pruned run directory, and cached
+ against the jobs table's high-water mark. `observed` never replaces `cost`.
A `models/` entry
takes its `shape` and `traits` from the template it configures and keeps
its own `cost`. A list-driven workflow (one with a `for_each` step) also
carries `lists`: per list variable, the fields an entry takes, the steps
- run over it and the default's length. Enough to choose a workflow and know
- what to pass it without reading each one; the variable defaults are
- deliberately left out, being an order of magnitude more payload on a
- listing the UI reloads. Cached by file mtime
+ run over it and the default's length. Variable defaults are left out (an
+ order of magnitude more payload); cached by file mtime
Optional query params narrow and shrink it:
`?shape=&traits=&configures=&include_models=&view=compact`. `shape` keeps
@@ -570,8 +631,7 @@ The editor's forms come from these; they are just as usable from scripts:
`shape`, `traits`, `cost`, `kinds`, `variable_names` and `lists` (carried
only when the workflow has a list-driven step, like `configures`), and
lists templates only unless `include_models=true` or a `configures` asks
- otherwise. With no params the response is what it always was, plus the new
- fields
+ otherwise
- `GET/PUT/DELETE /api/workflows/{name}` — read, save, delete workflow files
(confined to `--workflow-dir`); `PATCH` applies a JSON merge patch (RFC 7396,
`application/merge-patch+json` or JSON) to the stored version and saves the
@@ -607,7 +667,9 @@ The editor's forms come from these; they are just as usable from scripts:
queue an enhancement as an ordinary job whose saved text file is the
result
- `GET /api/gallery`, `GET /api/gallery/{name}/metadata`,
- `DELETE /api/gallery/{name}` — outputs and their embedded metadata. Each
+ `DELETE /api/gallery/{name}` — outputs and their embedded metadata. Delete
+ takes a file, or a run directory (`/`) to remove the whole
+ run; a run's directory goes with its last media file. Each
gallery entry carries `folder` (the workflow identity, the run id dropped)
and `subfolder` (what followed the run id - the `final`/`intermediate` a
step's `result.subfolder` chose, `''` when it chose none); `?folder=` and
@@ -621,7 +683,10 @@ The editor's forms come from these; they are just as usable from scripts:
the file, unless the step's `result.role` declared it), and `GET /api/gallery?role=final` lists only
those (repeatable; a null role counts as final). See `dw/roles.py` for how
a role is decided;
- the metadata route reports the file's `role` and the run's `inputs` too
+ the metadata route reports the file's `role` and the run's `inputs` too,
+ and `findings`: each level problem the probed soundtrack shows
+ (`full_scale`, `near_silent`), measured against `dw/audio_qc.py`'s
+ thresholds, in the assess route's finding shape
- `GET /api/gallery/{name:path}/download` — download an output file
- `GET /api/builtin-workflows` — the packaged fragments a sub-workflow step
names as `builtin:.json` (`dw/workflows/`), which no workflow listing
@@ -629,23 +694,41 @@ The editor's forms come from these; they are just as usable from scripts:
- `GET /api/gallery/{name:path}/thumbnail` — a small JPEG of an image, or of
a video's first frame; 404 for other kinds. `name` may be an `asset:`
reference, which the editors' asset picker uses for its grid
-- `GET /api/gallery/{name}/metadata` also carries `findings`: each level
- problem the probed soundtrack shows (`full_scale`, `near_silent`), measured
- against `dw/audio_qc.py`'s thresholds, in the assess route's finding shape
- `GET /api/gallery/{name}/image?max_dimension=768&crop=x,y,w,h&max_bytes=&format=auto`
— an image output or `asset:` sized for an inline answer: cropped (clamped),
fitted to `max_dimension`, and with `max_bytes` (a base64 budget) halved
until it fits. Headers `X-DW-Original-Size`, `X-DW-Returned-Size`, and when
they apply `X-DW-Crop` and `X-DW-Downscaled-To`. Takes `?token=`
-- `GET /api/gallery/{name}/frames` takes `max_total_bytes`: over it, every
- tile shrinks to one shared size and `downscaled_to` says which
+- `GET /api/gallery/{name}/frames` — frames of a video output or `asset:`
+ as PNG tiles, with exactly one selector: `at` (comma list of seconds or
+ `frame:N`), `count` (an evenly spaced contact sheet) or `seams` (`true` or
+ 1-based seam numbers: the frames either side of each shot boundary).
+ `boundaries` and `names` give the shots; without them, the shots the run's
+ manifest (or a kept asset's sidecar) recorded. `max_dimension` (default
+ 512), `crop=x,y,w,h` in source pixels, and `max_total_bytes`: over it every
+ tile shrinks to one shared size and `downscaled_to` says which (MCP
+ `get_output_frames`)
+- `GET /api/gallery/{name}/audio?start=&duration=` — the soundtrack of an
+ output or `asset:` as WAV, or an excerpt; headers `X-DW-Duration` and, for
+ an excerpt, `X-DW-Excerpt-Start` / `X-DW-Excerpt-Duration`. A whole
+ audio-only file is served in its own encoding. 404 for a file with no
+ soundtrack (MCP `get_output_audio`)
+- `GET /api/gallery/{name}/assess?probe=&detail=` — every assessment probe
+ that applies to a finished cut (`analyze_shots`, `analyze_seams`,
+ `analyze_sync_drift`), run in the server process on one decode: merged
+ `findings`, `rules_applied`, `rules_skipped`, and `not_applicable` per probe
+ the file cannot feed. `detail=true` adds each probe's answer under
+ `probes`; `probe` names one and answers with its full body. Findings are
+ places to look, not verdicts (`dw/assessment_rules.py`; MCP
+ `assess_output`)
- `POST /api/gallery/compare` — two to eight outputs side by side, and how far each is
from a baseline; read-only, queues nothing. Body: `names` (2-8 gallery names or
`asset:` references, one row each), `at` (1-8 moments, seconds or `"frame:N"`;
default 4 evenly spaced frames of the shortest video), `crop` (`[x, y, w, h]`: the
region shown and measured; or one box per name, shown only), `baseline` (default
- the first name), `max_dimension` (default 384, longest side of each cell) and
- `max_total_bytes`. Moments resolve to one frame index against the baseline (the
+ the first name), `max_dimension` (default 384, longest side of each cell),
+ `texture_bands` (default false: add per-output texture energy by wavelength band,
+ below) and `max_total_bytes`. Moments resolve to one frame index against the baseline (the
first video if the baseline is a still) and every output is read at that frame
number; a frame past an output's end is a blank cell. A still among videos repeats
in every column; all stills give one column and `at` is ignored with a note. A crop
@@ -663,31 +746,32 @@ The editor's forms come from these; they are just as usable from scripts:
`identical` when equal and SSIM is on luma (11x11 Gaussian window, sigma 1.5). An
output whose size differs from the baseline gets null metrics and the `reason`
`size differs from baseline: WxH vs WxH`, never a resize; a cell past an end says
- `past end (N frames)`. Errors: 400 bad request, out-of-bounds crop, every moment
+ `past end (N frames)`. With `texture_bands`, the reply adds a `texture_bands` header
+ (`fine` "2-8 px", `mid` "8-32 px", `coarse` "32-128 px") and a `texture` list, one
+ entry per output: `name`, `row`, `baseline` and `cells`, each with `texture`
+ (`luma`, `chroma_rg`, `chroma_bg`, each `{fine, mid, coarse}`, RMS on a 0-255 scale
+ after a least-squares plane is removed), `texture_ratio` (non-baseline only: output
+ over baseline, null where the baseline band is 0) and `reason`. Each non-baseline
+ `metrics[].cells[]` entry carries `texture` and `texture_ratio` too. A band longer
+ than the box's short side is null. One name is allowed with bands (it is the
+ baseline: no ratios). `crop` must be one shared box (a per-name list is a 400:
+ "texture bands need one shared box") of at least 16 px a side. Errors: 400 bad request, out-of-bounds crop, every moment
past every end, or not an image or video; 404 unknown name; 413 over the decode
pixel limit; 422 undecodable file
- `POST /api/gallery/archive` — `{"names": [...]}` (1-1000) bundles a
multi-file selection into one zip, named by each file's gallery-relative
- path so output subfolders survive. A browser cannot zip on its own and
- throttles a burst of single downloads, so the gallery's bulk download
- goes through here; an unknown or out-of-directory name 404s the whole
- request rather than yielding a partial archive
+ path so output subfolders survive; the gallery's bulk download. An unknown
+ or out-of-directory name 404s the whole request rather than yielding a
+ partial archive
- `GET /api/workspaces`, `POST /api/workspaces` (`{"name": ...}`),
`DELETE /api/workspaces/{name}?acknowledged=true` — the workspaces on this
- server. The workspace root's own `workflows/assets/outputs` are the
- `default` workspace and a named one is a subdirectory beside them, sharing
- the root's one prompt library. Delete answers with what it would remove and
- refuses until acknowledged, refuses the default, and refuses a workspace
- with jobs still queued. Each listed workspace carries a `usage`
- (`{files, bytes}`) — roughly how much disk its own folders hold, walked at
- most once a minute per workspace and deliberately approximate; the shared
- prompt library counts against the `default` workspace alone rather than
- once per workspace. A workspace is a namespace, **not** a security
- boundary: the API token is all-or-nothing
-
- `exports/` sits beside the workspace's own folders, holding one directory per
- exported job. It is a reserved name: no workspace can be called `exports`, and
- the folder is never listed as one.
+ server ([layout](WORKSPACES.md#several-workspaces-on-one-server)). Delete
+ answers with what it would remove and refuses until acknowledged, refuses
+ `default`, and refuses a workspace with jobs still queued. Each listed
+ workspace carries `usage` (`{files, bytes}`), an approximate walk of its
+ own folders at most once a minute; the shared prompt library counts
+ against `default` alone. `exports/` beside the workspace folders holds one
+ directory per exported job and is never listed as a workspace
- `POST /api/workspaces/{destination}/copy` (`{"source": ..., "runs": [...],
"assets": [...]}`) — copy runs and assets from one workspace into another.
A `runs` entry is `/`, or a bare `` for all its
@@ -722,56 +806,43 @@ The editor's forms come from these; they are just as usable from scripts:
none
- `POST /api/assets/keep` (`{"name": ..., "asset_name": ..., "overwrite": false, "shared": false}`)
— keep a generated file as an input asset under a stable name, returning
- its `asset:` reference. A run's files are named by the run that made them,
- which is the wrong thing for a later workflow to depend on: `latest` moves
- and a pinned run id breaks when outputs are pruned. The copy happens inside
- the workspace and is a hard link where the filesystem allows one, so
- keeping one frame of a large render costs no second copy of it. Refuses an
+ its `asset:` reference, which survives the run's outputs being pruned. The
+ copy stays inside the workspace and is a hard link where the filesystem
+ allows one. Refuses an
existing name unless `overwrite`. `asset_name` may name a folder and takes
the kept file's extension when it carries none (a contradicting one is a
400) — the same rule the upload route follows, and what keeps a kept asset
from landing under an extensionless name the library listing never shows.
`"shared": true` keeps it in
- `/common/assets` instead — the library every workspace under the root
- shares, which is where a recurring cast belongs
+ `/common/assets` instead, the library every workspace shares
- `DELETE /api/assets/{name}` — remove one file from the asset library,
deleting from whichever library on the search path holds it (the
workspace's own before the shared one, the order `asset:` resolves in).
An asset from a read-only examples library answers 403, the same as a
read-only prompt or workflow; a name nothing holds answers 404
- `POST /api/assets/archive` — `{"names": [...]}` (1-1000) bundles a
- multi-file asset selection into one zip, named by each file's
- library-relative path, which is the name its `asset:` reference carries.
- The gallery archive's counterpart on the input side; it resolves down the
- same search path a run does, so a selection spanning this workspace's
- library, the shared one and an examples tree downloads as one archive, and
- an unknown or out-of-library name 404s the whole request rather than
- yielding a partial one. A duplicate name (repeated in the selection, or
- differing only by leading/trailing whitespace) collapses onto the one zip
- entry. Media (image/video/audio) stores rather than deflates, unless it's
- a raw format that still compresses (`.bmp`, `.wav`) - everything else the
- libraries hold is an already-compressed container, and the response does
- not start until the archive is complete, so deflating it is latency the
- caller waits through for nothing. Everything else - `.json`, `.md`,
- `.txt`, an unrecognized extension - deflates; so does the export zip's
- text files (`workflow.json`, `manifest.json`, `job.json`, the README)
-- `POST /api/uploads?filename=...` — the raw bytes of one image, video or audio file,
- or a `.cube` 3D LUT for `apply_lut`
- (200MB ceiling, checked from `Content-Length` before a byte is read, and
- again on the body; extension held to the allowed image, video, audio and
- `.cube` list), saved
- into the asset library's `uploads/` subfolder - the shared library at
- `/common/assets` when `shared=true`, this workspace's own otherwise -
- under a generated name, or under `asset_name` when one is given (`cast/priya-voice.wav`, folders allowed,
- the uploaded file's extension assumed, confined to the library the way
- `keep_output`'s name is).
- Answers 201 with `reference` - `asset:uploads/`, the reference a saved
- workflow can carry and still resolve on a later run - and `url`, the same
- file under the `/inputs` mount, for the editor's preview. A server started
- without an asset library falls back to the output directory's `uploads/`
- and an absolute path. This is how the UI's file pickers get a local file
- onto the machine that will run the workflow. The body is the file itself,
- so no multipart parser is needed for a single-file upload
+ multi-file asset selection into one zip, each entry named by its
+ library-relative path (the name its `asset:` reference carries). It
+ resolves down the same search path a run does, so a selection spanning the
+ workspace's library, the shared one and an examples tree is one archive;
+ an unknown or out-of-library name 404s the whole request, and a duplicate
+ name (repeated, or differing only by surrounding whitespace) collapses to
+ one entry. Media is stored rather than deflated, since it is already
+ compressed, except raw formats that still compress (`.bmp`, `.wav`);
+ everything else deflates, as do the export zip's text files
+- `POST /api/uploads?filename=&asset_name=&shared=¬e=` — the raw bytes of
+ one image, video or audio file, or a `.cube` 3D LUT for `apply_lut`, as the
+ body (no multipart). 200MB ceiling, checked from `Content-Length` before a
+ byte is read and again on the body; the extension is held to the allowed
+ image, video, audio and `.cube` list. Saved under the asset library's
+ `uploads/` - the shared library at `/common/assets` when
+ `shared=true`, else this workspace's own - with a generated name, or as
+ `uploads/` (folders allowed, the file's extension assumed,
+ confined the way `keep_output`'s name is). Answers 201 with `reference`
+ (`asset:uploads/`, which a saved workflow can carry and still resolve
+ later) and `url`, the same file under `/inputs` for previews. 409 when the
+ workspace has no asset library. This is how the UI's file pickers get a
+ local file onto the machine that runs the workflow
- `GET /api/models`, `DELETE /api/models?repo={repo_id}` — hub cache
inventory and deletion
- `GET /api/caches`, `DELETE /api/caches/{name}`, `DELETE /api/caches/{name}/entries/{entry}` - dw's disk caches of derived
@@ -984,30 +1055,27 @@ export DW_API_TOKEN="some-long-random-string"
python -m dw.serve
```
-When a token is configured, every `/api/*` request must carry
-`Authorization: Bearer ` or gets a 401. The UI's own static files and
-`/outputs` (generated media) stay reachable without it - the page has to
-load far enough for a user to enter the token, and an `{label}
+{#each Array.from({ length: chips }, (_, i) => i) as i (i)}
+ {@const y = SEGMENT_TOP + i * SEGMENT_STEP}
+ {@const state = segmentStateOf(i + 1)}
+
+ segment {i + 1}
+ {#if i === 0 && node.chain?.anchor}
+
+ segment 1's latents anchor every later segment's (chain anchor:
+ segment_1)
+ anchor
+
+ {/if}
+
+ {#if i < chips - 1 && node.chain}
+ {#if `↓ ${node.chain.seam}`.length > SEAM_CHARS}{node.chain.seam} {/if}{fit(`↓ ${node.chain.seam}`, SEAM_CHARS, 'head')}
+ {/if}
+{/each}
+
+
diff --git a/ui/src/lib/editor/FlowView.svelte b/ui/src/lib/editor/FlowView.svelte
index f6087270d..d399bf465 100644
--- a/ui/src/lib/editor/FlowView.svelte
+++ b/ui/src/lib/editor/FlowView.svelte
@@ -2,6 +2,7 @@
import { dataFlowGraph, type FlowNode } from '../flow'
import { formatDuration } from '../format'
import FlowLegend from './FlowLegend.svelte'
+ import FlowSegments from './FlowSegments.svelte'
import {
BOX_H,
BOX_W,
@@ -31,6 +32,7 @@
doneMembers = [],
nodeTimes = {},
memberTimes = {},
+ activeSegments = {},
}: {
workflow: Record
onselect?: (stepName: string) => void
@@ -57,6 +59,9 @@
nodeTimes?: Record
/** Seconds each finished for_each member took, engine names. */
memberTimes?: Record
+ /** The segment, 1-based, each chained step is on or last ran - its
+ * generating phases (runstate.ts chainSegments). */
+ activeSegments?: Record
} = $props()
const showsRun = $derived(
@@ -89,15 +94,20 @@
const graph = $derived(dataFlowGraph(workflow))
const hasForEach = $derived(graph.nodes.some((node) => node.forEach))
+ const hasChain = $derived(
+ graph.nodes.some((node) => node.chain && !node.forEach),
+ )
const hasPending = $derived(
graph.nodes.some((node) => stateOf(node.name) === 'pending'),
)
- // One plain step has no flow to draw: it is shown as a line, not a canvas
+ // One plain step has no flow to draw: it is shown as a line, not a canvas.
+ // A chained step is not plain - its segments are the flow
const lone = $derived(
- graph.nodes.length === 1 && !graph.nodes[0].forEach
+ graph.nodes.length === 1 && !graph.nodes[0].forEach && !graph.nodes[0].chain
? graph.nodes[0]
: undefined,
)
+
const canSelect = (name: string) => !!onselect && (selectable?.(name) ?? true)
/** The parts of a node's labels that did not fit, in full, for its tooltip. */
@@ -181,6 +191,7 @@
edges={graph.edges.length > 0}
fanIn={graph.fanIn.size > 0}
forEach={hasForEach}
+ chain={hasChain}
done={doneSteps.length > 0}
active={activeStep !== undefined}
failed={failedStep !== undefined}
@@ -251,7 +262,7 @@
clip-path={boxH > BOX_H
? `url(#flow-nodebox-${boxH})`
: 'url(#flow-nodebox)'}
- aria-label={`step ${node.name}, ${kindLabel(node.kind)}${stateOf(node.name) ? ', ' + stateOf(node.name) : ''}${node.isEntryPoint ? ', entry point' : ''}${fanIn ? ', fan-in: ' + fanIn.label : ''}${node.forEach ? (node.members?.length ? `, for_each with ${node.members.length} entries` : ', for_each') : ''}`}
+ aria-label={`step ${node.name}, ${kindLabel(node.kind)}${stateOf(node.name) ? ', ' + stateOf(node.name) : ''}${node.isEntryPoint ? ', entry point' : ''}${fanIn ? ', fan-in: ' + fanIn.label : ''}${node.forEach ? (node.members?.length ? `, for_each with ${node.members.length} entries` : ', for_each') : ''}${node.chain && !node.forEach ? (node.chain.segments ? `, chain of ${node.chain.segments} segments` : ', chain') : ''}`}
{...nodeAttributes(node.name)}
>
{#if overflowTitle(node)}
@@ -330,6 +341,12 @@
for_each
+ {:else if node.chain}
+ ): DataFlowGraph {
producedCount: producedCount(step),
forEach: FOR_EACH_KEY in step,
members: forEachMembers(workflow, step),
+ chain: chainSummary(workflow, step),
}
})
const edges: FlowEdge[] = []
diff --git a/ui/src/lib/flowChain.test.ts b/ui/src/lib/flowChain.test.ts
new file mode 100644
index 000000000..43b2bee06
--- /dev/null
+++ b/ui/src/lib/flowChain.test.ts
@@ -0,0 +1,86 @@
+import { describe, expect, it } from 'vitest'
+import { chainSummary } from './flowChain'
+
+describe('chained segments', () => {
+ const chainStep = (chain: Record) => ({
+ name: 'clip',
+ pipeline: { configuration: { component_type: 'ModularPipeline' }, chain },
+ })
+ const summary = (
+ chain: Record,
+ variables: Record = {},
+ ) => chainSummary({ variables }, chainStep(chain))
+
+ it("reads a declared segment count and the seam's last-frame bookkeeping", () => {
+ // The chained-segments template's shape: the count and continuity are
+ // variables, which a realized workflow always carries
+ expect(
+ summary(
+ {
+ segments: 'variable:segments',
+ continuity: 'variable:continuity',
+ anchor: 'variable:anchor',
+ trim_frames: 2,
+ crossfade_ms: 80,
+ },
+ { segments: 3, continuity: 'last_frame', anchor: 'segment_1' },
+ ),
+ ).toEqual({
+ segments: 3,
+ fromAudio: false,
+ seam: 'last frame · trim 2 · 80ms',
+ anchor: true,
+ })
+ })
+
+ it('falls back to the engine defaults the block leaves out', () => {
+ expect(chainSummary({}, chainStep({ segments: 2 }))).toEqual({
+ segments: 2,
+ fromAudio: false,
+ seam: 'last frame · trim 1 · 75ms',
+ anchor: false,
+ })
+ })
+
+ it('names what a guide or a video reference carries across the seam', () => {
+ const seam = (chain: Record) =>
+ summary({ segments: 2, ...chain })!.seam
+ expect(seam({ continuity: 'guide' })).toBe('guide · 22 frames')
+ expect(seam({ continuity: 'guide', guide_frames: 39 })).toBe(
+ 'guide · 39 frames',
+ )
+ expect(seam({ continuity: 'last_segment' })).toBe('video reference')
+ expect(
+ seam({
+ continuity: 'last_segment',
+ carry_frames: 24,
+ carry_audio: false,
+ }),
+ ).toBe('video reference · 24 frames · no audio')
+ expect(seam({ continuity: 'variable:unset' })).toBe('carry')
+ })
+
+ it('keeps the audio under a last-frame seam, whatever carry_audio says', () => {
+ expect(summary({ segments: 2, carry_audio: false })!.seam).toBe(
+ 'last frame · trim 1 · 75ms',
+ )
+ })
+
+ it('leaves the count unknown when the length comes from audio or an unset variable', () => {
+ const fromAudio = summary({ match_audio: true })!
+ expect(fromAudio.segments).toBeNull()
+ expect(fromAudio.fromAudio).toBe(true)
+ const unset = summary({ segments: 'variable:missing' })!
+ expect(unset.segments).toBeNull()
+ expect(unset.fromAudio).toBe(false)
+ })
+
+ it('is null on a step with no chain', () => {
+ expect(
+ chainSummary({}, { name: 'plain', task: { command: 'noop' } }),
+ ).toBeNull()
+ expect(
+ chainSummary({}, { name: 'plain', pipeline: { chain: null } }),
+ ).toBeNull()
+ })
+})
diff --git a/ui/src/lib/flowChain.ts b/ui/src/lib/flowChain.ts
new file mode 100644
index 000000000..4d3640bec
--- /dev/null
+++ b/ui/src/lib/flowChain.ts
@@ -0,0 +1,107 @@
+/** A chained pipeline step's `chain` block, summarized for the data-flow
+ * graph (flow.ts). The pipeline runs once per segment inside the one step,
+ * carrying continuity across each seam, and the segments are stitched into
+ * one video (dw/pipeline_processors/chain.py). */
+
+import { VARIABLE, referenceName } from './references'
+
+export interface ChainSummary {
+ /** The segment count, when the definition gives it - a literal or a
+ * declared variable, which a realized workflow always holds. Null under
+ * `match_audio`, or for a variable left unset. */
+ segments: number | null
+ /** True when the audio's length decides the count at run time. */
+ fromAudio: boolean
+ /** What each seam carries into the next segment, in a few words. */
+ seam: string
+ /** True under `anchor: "segment_1"`: segment 1's latents anchor every
+ * later segment's (#1166). */
+ anchor: boolean
+}
+
+// The chain block's engine defaults (dw/workflow_schema.json $defs/chain)
+const CHAIN_DEFAULTS: Record = {
+ continuity: 'last_frame',
+ trim_frames: 1,
+ crossfade_ms: 75,
+ guide_frames: 22,
+ carry_audio: true,
+ anchor: 'none',
+}
+
+/** A chain value by key, the engine default standing in for one the block
+ * leaves out. */
+type ChainRead = (key: string) => unknown
+
+/** The block's reader. A value may be a `variable:`, read from the declared
+ * variables - undefined when it names none, or one left null. */
+function chainReader(
+ workflow: Record,
+ chain: Record,
+): ChainRead {
+ return (key) => {
+ const value = key in chain ? chain[key] : CHAIN_DEFAULTS[key]
+ const name = referenceName(value, VARIABLE)
+ if (name === null) return value
+ return workflow.variables?.[name] ?? undefined
+ }
+}
+
+/** `label`, then the frame count when the block gives a number. */
+function withFrames(label: string, frames: unknown): string[] {
+ return typeof frames === 'number' ? [label, `${frames} frames`] : [label]
+}
+
+function lastFrameSeam(read: ChainRead): string[] {
+ const trim = read('trim_frames')
+ const fade = read('crossfade_ms')
+ return [
+ 'last frame',
+ typeof trim === 'number' ? `trim ${trim}` : null,
+ typeof fade === 'number' ? `${fade}ms` : null,
+ ].filter((part): part is string => part !== null)
+}
+
+/** What the continuity mode carries across the seam, as label words. */
+function continuityParts(read: ChainRead): string[] {
+ switch (read('continuity')) {
+ case 'last_frame':
+ return lastFrameSeam(read)
+ case 'guide':
+ return withFrames('guide', read('guide_frames'))
+ case 'last_segment':
+ return withFrames('video reference', read('carry_frames'))
+ default:
+ return ['carry']
+ }
+}
+
+function seamLabel(read: ChainRead): string {
+ const parts = continuityParts(read)
+ if (read('continuity') !== 'last_frame' && read('carry_audio') === false)
+ parts.push('no audio')
+ return parts.join(' · ')
+}
+
+/** The declared count, or null when it is unknown here or the audio decides. */
+function segmentCount(segments: unknown, matchAudio: unknown): number | null {
+ return typeof segments === 'number' && segments >= 1 && !matchAudio
+ ? segments
+ : null
+}
+
+/** The step's `chain` block summarized for the graph, or null without one. */
+export function chainSummary(
+ workflow: Record,
+ step: Record,
+): ChainSummary | null {
+ const chain = step.pipeline?.chain
+ if (chain === null || typeof chain !== 'object') return null
+ const read = chainReader(workflow, chain)
+ return {
+ segments: segmentCount(read('segments'), chain.match_audio),
+ fromAudio: chain.match_audio === true,
+ seam: seamLabel(read),
+ anchor: read('anchor') === 'segment_1',
+ }
+}
diff --git a/ui/src/lib/generated/api-schema.ts b/ui/src/lib/generated/api-schema.ts
index a544f3126..ab4cfcae9 100644
--- a/ui/src/lib/generated/api-schema.ts
+++ b/ui/src/lib/generated/api-schema.ts
@@ -402,6 +402,8 @@ export interface paths {
* of different sizes get no metrics, and say why. Stills join a video
* comparison as the same image in every column. `max_total_bytes`
* shrinks the whole grid, and `downscaled_to` says to what.
+ * `texture_bands` (#1152) adds each output's band energy per moment,
+ * the baseline's included, and its ratio to the baseline's.
*/
post: operations["gallery_compare_api_gallery_compare_post"];
delete?: never;
@@ -618,6 +620,13 @@ export interface paths {
* refused (422) with the interval that fits; the default width is
* never refused for length (#823).
*
+ * A file with a soundtrack also carries `pcm_sha256`, the SHA-256 of
+ * its decoded samples (native dtype, interleaved, little-endian,
+ * trimmed to the container duration), with the `sample_format` and
+ * `frames` hashed: proof two tracks are sample-identical, comparable
+ * only when `sample_format`, `frames`, `sample_rate` and `channels`
+ * all agree (#1139). Only this route computes it.
+ *
* An image's `media` is its measured `{kind, width, height, mode}`,
* read from the file header (#922) - not the width/height arguments
* the pipeline was asked for. `include_workflow=false` drops the
@@ -2095,9 +2104,15 @@ export interface components {
max_total_bytes?: number | null;
/**
* Names
- * @description 2 to 8 outputs (or `asset:` references), one row each
+ * @description 2 to 8 outputs (or `asset:` references), one row each; 1 is enough with `texture_bands`
*/
names: string[];
+ /**
+ * Texture Bands
+ * @description Add per-output RMS energy in the fine (2-8 px), mid (8-32) and coarse (32-128) wavelength bands of the measured region, on luma, R-G and B-G, plus each band's ratio to the baseline - a measurement, not a score. Needs one shared `crop` (or none: the full frame) of at least 16 px a side
+ * @default false
+ */
+ texture_bands: boolean;
};
/** CopiedItems */
CopiedItems: {
diff --git a/ui/src/lib/generated/openapi.json b/ui/src/lib/generated/openapi.json
index d2801ffe2..81317a1a5 100644
--- a/ui/src/lib/generated/openapi.json
+++ b/ui/src/lib/generated/openapi.json
@@ -501,12 +501,18 @@
"title": "Max Total Bytes"
},
"names": {
- "description": "2 to 8 outputs (or `asset:` references), one row each",
+ "description": "2 to 8 outputs (or `asset:` references), one row each; 1 is enough with `texture_bands`",
"items": {
"type": "string"
},
"title": "Names",
"type": "array"
+ },
+ "texture_bands": {
+ "default": false,
+ "description": "Add per-output RMS energy in the fine (2-8 px), mid (8-32) and coarse (32-128) wavelength bands of the measured region, on luma, R-G and B-G, plus each band's ratio to the baseline - a measurement, not a score. Needs one shared `crop` (or none: the full frame) of at least 16 px a side",
+ "title": "Texture Bands",
+ "type": "boolean"
}
},
"required": [
@@ -6223,7 +6229,7 @@
},
"/api/gallery/compare": {
"post": {
- "description": "Two to eight outputs side by side at the same moments, and how far\neach is from a baseline (#871): one grid, a row per output and a\ncolumn per moment, plus PSNR, SSIM and mean absolute difference per\nnon-baseline output at source resolution. Moments resolve to one\nframe index against the baseline (or the first video) and every\noutput is read at that frame; one past an output's end is a blank\ncell. A single `crop` is the region shown and measured; one box per\nname is shown only, and the metrics stay on the full frame. Outputs\nof different sizes get no metrics, and say why. Stills join a video\ncomparison as the same image in every column. `max_total_bytes`\nshrinks the whole grid, and `downscaled_to` says to what.",
+ "description": "Two to eight outputs side by side at the same moments, and how far\neach is from a baseline (#871): one grid, a row per output and a\ncolumn per moment, plus PSNR, SSIM and mean absolute difference per\nnon-baseline output at source resolution. Moments resolve to one\nframe index against the baseline (or the first video) and every\noutput is read at that frame; one past an output's end is a blank\ncell. A single `crop` is the region shown and measured; one box per\nname is shown only, and the metrics stay on the full frame. Outputs\nof different sizes get no metrics, and say why. Stills join a video\ncomparison as the same image in every column. `max_total_bytes`\nshrinks the whole grid, and `downscaled_to` says to what.\n`texture_bands` (#1152) adds each output's band energy per moment,\nthe baseline's included, and its ratio to the baseline's.",
"operationId": "gallery_compare_api_gallery_compare_post",
"parameters": [
{
@@ -6841,7 +6847,7 @@
},
"/api/gallery/{name}/metadata": {
"get": {
- "description": "Generation metadata embedded in a saved image ('workflow' inside\nit is the full definition the editor can reopen), plus the job that\nproduced the file when history remembers one, plus - for audio and\nvideo - what the file itself holds: duration, format and level,\nwhich is how an agent that cannot listen checks a track. Only an\nimage embeds 'metadata' this way - it is always null for audio and\nvideo, since neither format has a slot this writer uses; recover\nthe recipe from 'job' (GET /api/jobs/{id}/workflow) when one is\nknown, or from nothing when it isn't (a kept asset has no job).\n\n`envelope=true` adds the soundtrack's level second by second, which\nis what says *where* in a track something is - whether a shot is\nstill voiced at its last frame, how deep the hole at a seam goes.\nOpt-in: a ten-minute track is 600 numbers, and the default call has\nto stay small. `interval_seconds` (default 1.0, 0.05 to 3600) is\nthe bin width: 0.1 places a cut on a vocal onset. A track that\nwould need more than 2400 bins at a finer-than-default width is\nrefused (422) with the interval that fits; the default width is\nnever refused for length (#823).\n\nAn image's `media` is its measured `{kind, width, height, mode}`,\nread from the file header (#922) - not the width/height arguments\nthe pipeline was asked for. `include_workflow=false` drops the\nembedded `metadata.workflow` (the whole definition, often 20 KB) and\nleaves the rest of `metadata` with `workflow_omitted: true`.\n\n`name` may also be an 'asset:' reference, and then it is the input\nasset of that name that is described rather than an output (#127).\nThe numbers here - duration, frame count, fps, sample rate - are\nwhat decide whether a call will work at all, and for a file the\ncaller is about to *consume* they were previously unobtainable:\nthe only way to read a wav's length was to run a job that copied it\ninto the output directory. `job` is null for an asset with no\nrecorded provenance, and `source` says which of the two roots\nanswered.\n\nA kept asset (`keep_output`) is not always provenance-blind: when\nthe file it was kept from still had a job in history, `keep_output`\nrecorded `{job, run_id, version}` beside it, and this route reads it\nback the same way it reads a kept file's shots (#556). `run_id`/\n`version` are that source run's own - `job.id` is still the one to\npass `get_job_workflow`, since a pruned run leaves no `workflow.json`\nof its own to reopen. `inputs` is the source run's too (#902).",
+ "description": "Generation metadata embedded in a saved image ('workflow' inside\nit is the full definition the editor can reopen), plus the job that\nproduced the file when history remembers one, plus - for audio and\nvideo - what the file itself holds: duration, format and level,\nwhich is how an agent that cannot listen checks a track. Only an\nimage embeds 'metadata' this way - it is always null for audio and\nvideo, since neither format has a slot this writer uses; recover\nthe recipe from 'job' (GET /api/jobs/{id}/workflow) when one is\nknown, or from nothing when it isn't (a kept asset has no job).\n\n`envelope=true` adds the soundtrack's level second by second, which\nis what says *where* in a track something is - whether a shot is\nstill voiced at its last frame, how deep the hole at a seam goes.\nOpt-in: a ten-minute track is 600 numbers, and the default call has\nto stay small. `interval_seconds` (default 1.0, 0.05 to 3600) is\nthe bin width: 0.1 places a cut on a vocal onset. A track that\nwould need more than 2400 bins at a finer-than-default width is\nrefused (422) with the interval that fits; the default width is\nnever refused for length (#823).\n\nA file with a soundtrack also carries `pcm_sha256`, the SHA-256 of\nits decoded samples (native dtype, interleaved, little-endian,\ntrimmed to the container duration), with the `sample_format` and\n`frames` hashed: proof two tracks are sample-identical, comparable\nonly when `sample_format`, `frames`, `sample_rate` and `channels`\nall agree (#1139). Only this route computes it.\n\nAn image's `media` is its measured `{kind, width, height, mode}`,\nread from the file header (#922) - not the width/height arguments\nthe pipeline was asked for. `include_workflow=false` drops the\nembedded `metadata.workflow` (the whole definition, often 20 KB) and\nleaves the rest of `metadata` with `workflow_omitted: true`.\n\n`name` may also be an 'asset:' reference, and then it is the input\nasset of that name that is described rather than an output (#127).\nThe numbers here - duration, frame count, fps, sample rate - are\nwhat decide whether a call will work at all, and for a file the\ncaller is about to *consume* they were previously unobtainable:\nthe only way to read a wav's length was to run a job that copied it\ninto the output directory. `job` is null for an asset with no\nrecorded provenance, and `source` says which of the two roots\nanswered.\n\nA kept asset (`keep_output`) is not always provenance-blind: when\nthe file it was kept from still had a job in history, `keep_output`\nrecorded `{job, run_id, version}` beside it, and this route reads it\nback the same way it reads a kept file's shots (#556). `run_id`/\n`version` are that source run's own - `job.id` is still the one to\npass `get_job_workflow`, since a pruned run leaves no `workflow.json`\nof its own to reopen. `inputs` is the source run's too (#902).",
"operationId": "gallery_metadata_api_gallery__name__metadata_get",
"parameters": [
{
diff --git a/ui/src/lib/pages/JobPage.svelte b/ui/src/lib/pages/JobPage.svelte
index 12b879860..a3955721b 100644
--- a/ui/src/lib/pages/JobPage.svelte
+++ b/ui/src/lib/pages/JobPage.svelte
@@ -6,6 +6,7 @@
import { groupResultFiles, unsavedSteps } from '../results'
import {
activeMember,
+ chainSegments,
finishedMembers,
finishedNodes,
flowNodeName,
@@ -175,6 +176,7 @@
// One grain finer than the group: which entries of a for_each step have
// finished and which is running, in the engine's own `group@entry` names
const finishedMemberSteps = $derived(finishedMembers(events as JobEvent[]))
+ const segments = $derived(chainSegments(events as JobEvent[]))
const activeMemberStep = $derived(
running ? activeMember(events as JobEvent[]) : undefined,
)
@@ -300,6 +302,7 @@
selectable={hasResult}
selectHint="click a step to see its results"
doneMembers={finishedMemberSteps}
+ activeSegments={segments}
{nodeTimes}
{memberTimes}
/>
diff --git a/ui/src/lib/runstate.test.ts b/ui/src/lib/runstate.test.ts
index 3acb7ece2..3faebc19a 100644
--- a/ui/src/lib/runstate.test.ts
+++ b/ui/src/lib/runstate.test.ts
@@ -3,6 +3,7 @@ import {
nodeDurations,
stepDurations,
activeMember,
+ chainSegments,
finishedMembers,
finishedNodes,
flowNodeName,
@@ -148,3 +149,45 @@ describe('step and node durations', () => {
expect(stepDurations([bare, ev('step_end', 'x', 3)])).toEqual({})
})
})
+
+describe('chainSegments', () => {
+ const start = (step: string, extra = {}) => ({
+ seq: 0,
+ event: 'step_start',
+ step,
+ ...extra,
+ })
+ const segment = (i: number, n: number) => ({
+ seq: 0,
+ event: 'phase',
+ phase: 'generating',
+ detail: `segment ${i}/${n}`,
+ })
+
+ it('names the segment each top-level step is on, from its generating phases', () => {
+ const events = [
+ start('clip'),
+ { seq: 0, event: 'phase', phase: 'loading', detail: 'MiniMax-H3' },
+ segment(1, 3),
+ segment(2, 3),
+ ] as JobEvent[]
+ expect(chainSegments(events)).toEqual({ clip: 2 })
+ })
+
+ it("leaves a sub-workflow's inner chain to no node", () => {
+ const events = [
+ start('compose'),
+ start('inner', { parent_step: 'compose' }),
+ segment(1, 2),
+ ] as JobEvent[]
+ expect(chainSegments(events)).toEqual({})
+ })
+
+ it("ignores a plain step's generating phase, which names the step", () => {
+ const events = [
+ start('gen'),
+ { seq: 0, event: 'phase', phase: 'generating', detail: 'gen' },
+ ] as JobEvent[]
+ expect(chainSegments(events)).toEqual({})
+ })
+})
diff --git a/ui/src/lib/runstate.ts b/ui/src/lib/runstate.ts
index ff952727d..a26b62db7 100644
--- a/ui/src/lib/runstate.ts
+++ b/ui/src/lib/runstate.ts
@@ -136,3 +136,30 @@ export function nodeDurations(events: JobEvent[]): Record {
[...nodes].map(([node, [start, end]]) => [node, end - start]),
)
}
+
+// The detail a chained step's generating phase carries for each segment
+// (Pipeline._run_once, dw/pipeline_processors/pipeline.py); an unchained
+// step's names the step instead
+const SEGMENT_DETAIL = /^segment (\d+)\/\d+$/
+
+/** The segment each chained top-level step is on, 1-based, keyed by the
+ * engine's step name: the latest `segment i/N` generating phase since that
+ * step started. A phase event names no step, so it belongs to the last
+ * step_start before it - and one inside a sub-workflow belongs to no chip
+ * this graph draws, for the reason `finishedNodes` gives. Under fan-out
+ * the chain reruns per iteration, and the latest segment is the one on. */
+export function chainSegments(events: JobEvent[]): Record {
+ const segments: Record = {}
+ let current: string | null = null
+ for (const event of events) {
+ if (event.event === 'step_start') {
+ current = event.parent_step ? null : (event.step as string)
+ continue
+ }
+ if (current === null || event.event !== 'phase') continue
+ if (event.phase !== 'generating') continue
+ const match = SEGMENT_DETAIL.exec(String(event.detail ?? ''))
+ if (match) segments[current] = Number(match[1])
+ }
+ return segments
+}
diff --git a/ui/src/lib/workspace.svelte.test.ts b/ui/src/lib/workspace.svelte.test.ts
index 527a1f31f..d053bc039 100644
--- a/ui/src/lib/workspace.svelte.test.ts
+++ b/ui/src/lib/workspace.svelte.test.ts
@@ -1,5 +1,7 @@
import { afterEach, beforeEach, expect, it, vi } from 'vitest'
+// Workspaces the server gained after the page loaded - a test pushes to it
+const created = vi.hoisted((): string[] => [])
const listWorkspaces = vi.hoisted(() =>
vi.fn(() =>
Promise.resolve({
@@ -22,6 +24,14 @@ const listWorkspaces = vi.hoisted(() =>
outputs: '/ws/studio/outputs',
prompts: null,
},
+ ...created.map((name) => ({
+ name,
+ default: false,
+ workflows: `/ws/${name}/workflows`,
+ assets: null,
+ outputs: `/ws/${name}/outputs`,
+ prompts: null,
+ })),
],
}),
),
@@ -33,6 +43,7 @@ vi.mock('./toast', () => ({ notify: { error: errorToast, success: vi.fn() } }))
beforeEach(() => {
listWorkspaces.mockClear()
errorToast.mockClear()
+ created.length = 0
localStorage.clear()
location.hash = ''
})
@@ -135,10 +146,29 @@ it('a missing workspace entered after the listing landed still redirects', async
expect(errorToast).not.toHaveBeenCalled()
location.hash = '#/ws/typo/gallery'
window.dispatchEvent(new HashChangeEvent('hashchange'))
+ // Refetched once before it is called missing - see the next test
+ await loadWorkspaces()
expect(location.hash).toBe('#/ws/default/overview')
expect(errorToast).toHaveBeenCalledWith(expect.stringContaining('typo'))
})
+it('a workspace created after the listing landed is refetched, not called missing', async () => {
+ location.hash = '#/ws/studio/gallery'
+ await import('./router.svelte')
+ const { loadWorkspaces, workspace } = await import('./workspace.svelte')
+ await loadWorkspaces()
+ // Created elsewhere (an agent's create_workspace) after this page loaded -
+ // the running chip's job link then corrects the URL into it
+ created.push('fresh')
+ location.hash = '#/ws/fresh/jobs/abc'
+ window.dispatchEvent(new HashChangeEvent('hashchange'))
+ await loadWorkspaces()
+ expect(workspace.names).toContain('fresh')
+ expect(location.hash).toBe('#/ws/fresh/jobs/abc')
+ expect(workspace.current).toBe('fresh')
+ expect(errorToast).not.toHaveBeenCalled()
+})
+
it('a failed listing does not read as every other workspace being gone', async () => {
listWorkspaces.mockRejectedValueOnce(new Error('down'))
location.hash = '#/ws/studio/gallery'
@@ -152,3 +182,42 @@ it('a failed listing does not read as every other workspace being gone', async (
expect(workspace.current).toBe('studio')
expect(errorToast).not.toHaveBeenCalled()
})
+
+it('a refetch that fails keeps the listing and still redirects the missing name', async () => {
+ // Driven through applyRouteWorkspace rather than a hashchange: the routers
+ // earlier tests imported still listen, and one would take the rejection
+ const { applyRouteWorkspace, loadWorkspaces, workspace } =
+ await import('./workspace.svelte')
+ applyRouteWorkspace({
+ kind: 'ws',
+ workspace: 'studio',
+ section: 'gallery',
+ rest: [],
+ })
+ await loadWorkspaces()
+ expect(workspace.names).toEqual(['default', 'studio'])
+ // The refetch a name the cache lacks triggers hits a server hiccup
+ listWorkspaces.mockRejectedValueOnce(new Error('down'))
+ applyRouteWorkspace({
+ kind: 'ws',
+ workspace: 'typo',
+ section: 'gallery',
+ rest: [],
+ })
+ await loadWorkspaces()
+ expect(workspace.names).toEqual(['default', 'studio'])
+ expect(location.hash).toBe('#/ws/default/overview')
+ expect(workspace.current).toBe('default')
+ expect(errorToast).toHaveBeenCalledWith(expect.stringContaining('typo'))
+})
+
+it('a first listing that fails falls back to the default alone, until one lands', async () => {
+ listWorkspaces.mockRejectedValueOnce(new Error('down'))
+ const { invalidateWorkspaces, loadWorkspaces, workspace } =
+ await import('./workspace.svelte')
+ await loadWorkspaces()
+ expect(workspace.names).toEqual(['default'])
+ invalidateWorkspaces()
+ await loadWorkspaces()
+ expect(workspace.names).toEqual(['default', 'studio'])
+})
diff --git a/ui/src/lib/workspace.svelte.ts b/ui/src/lib/workspace.svelte.ts
index f21aa1a27..7d94606f3 100644
--- a/ui/src/lib/workspace.svelte.ts
+++ b/ui/src/lib/workspace.svelte.ts
@@ -36,6 +36,13 @@ let lastView: RouteView | null = null
// the default alone so the sidebar has something to show, and that fallback
// must not be read as proof that every other workspace is gone.
let listed = false
+// Whether `names` was fetched for the check now reading it. The listing is
+// cached for the page's life, and a workspace can be created elsewhere after
+// it landed (an agent's create_workspace, another tab) - the running chip's
+// job link then corrects the URL into one the cache has never heard of. So
+// a name the cache lacks is refetched once before it is called missing.
+let fresh = false
+let refetching = false
export function applyRouteWorkspace(view: RouteView | null): void {
lastView = view
@@ -59,6 +66,14 @@ export function applyRouteWorkspace(view: RouteView | null): void {
export function checkWorkspaceExists(view: RouteView | null): void {
if (!listed || !workspace.names) return
if (workspace.names.includes(workspace.current)) return
+ if (!fresh) {
+ // The refetch's own landing re-runs this check against `lastView`
+ if (refetching) return
+ refetching = true
+ invalidateWorkspaces()
+ void loadWorkspaces().finally(() => (refetching = false))
+ return
+ }
const missing = workspace.current
workspace.current = DEFAULT_WORKSPACE
rememberWorkspace(DEFAULT_WORKSPACE)
@@ -68,6 +83,17 @@ export function checkWorkspaceExists(view: RouteView | null): void {
}
}
+/** The check with `names` counted as fetched for it - a name it lacks is
+ * missing, not refetched again. */
+function checkAgainstFreshListing(): void {
+ fresh = true
+ try {
+ checkWorkspaceExists(lastView)
+ } finally {
+ fresh = false
+ }
+}
+
// The in-flight/completed listing, shared across every caller - the sidebar
// and any page that needs names call loadWorkspaces() and would otherwise
// refetch a listing that has not changed.
@@ -94,10 +120,15 @@ export function loadWorkspaces(): Promise {
// A route naming a workspace that has gone away (deleted elsewhere,
// or a stale bookmark) is caught here for the route the app opened
// on; `applyRouteWorkspace` catches every one after
- checkWorkspaceExists(lastView)
+ checkAgainstFreshListing()
} catch {
- listed = false
- workspace.names = [DEFAULT_WORKSPACE]
+ // The first listing failing leaves the fallback. A refetch failing
+ // (the one `checkWorkspaceExists` runs for a name the cache lacks)
+ // keeps the listing that landed: a server hiccup does not empty the
+ // sidebar, and the name that listing lacks is still called missing,
+ // since that listing is the only answer there is.
+ if (listed) checkAgainstFreshListing()
+ else workspace.names = [DEFAULT_WORKSPACE]
}
})()
}
diff --git a/workflows/models/flux3-klein.json b/workflows/models/flux3-klein.json
index 1bf5457cd..0ddd763a1 100644
--- a/workflows/models/flux3-klein.json
+++ b/workflows/models/flux3-klein.json
@@ -21,7 +21,14 @@
"pipeline": {
"configuration": {
"component_type": "ModularPipeline",
+ "remove_blocks": ["prompt_upsampler"],
"load_components": {
+ "names": [
+ "text_encoder",
+ "tokenizer",
+ "transformer",
+ "vae"
+ ],
"dtype": "torch.bfloat16"
},
"components": {
diff --git a/workflows/templates/controlnet.json b/workflows/templates/controlnet.json
index 8c68b2a7e..619655abc 100644
--- a/workflows/templates/controlnet.json
+++ b/workflows/templates/controlnet.json
@@ -12,12 +12,12 @@
"command": "canny",
"arguments": {
"image": {
- "location": "https://pbs.twimg.com/media/GezfaKzaQAEwLJJ?format=jpg&name=medium",
- "low_threshold": 50,
- "high_threshold": 200,
- "detect_resolution": 1024,
- "image_resolution": 1024
- }
+ "location": "https://pbs.twimg.com/media/GezfaKzaQAEwLJJ?format=jpg&name=medium"
+ },
+ "low_threshold": 50,
+ "high_threshold": 200,
+ "detect_resolution": 1024,
+ "image_resolution": 1024
}
},
"result": {
diff --git a/workflows/templates/flux3/image-edit.json b/workflows/templates/flux3/image-edit.json
index c94480ddd..c9b7dc189 100644
--- a/workflows/templates/flux3/image-edit.json
+++ b/workflows/templates/flux3/image-edit.json
@@ -36,7 +36,14 @@
"pipeline": {
"configuration": {
"component_type": "ModularPipeline",
+ "remove_blocks": ["prompt_upsampler"],
"load_components": {
+ "names": [
+ "text_encoder",
+ "tokenizer",
+ "transformer",
+ "vae"
+ ],
"dtype": "torch.bfloat16"
},
"components": {
diff --git a/workflows/templates/flux3/layout-boxes.json b/workflows/templates/flux3/layout-boxes.json
index 3a0ec4242..c2baa227c 100644
--- a/workflows/templates/flux3/layout-boxes.json
+++ b/workflows/templates/flux3/layout-boxes.json
@@ -31,7 +31,14 @@
"pipeline": {
"configuration": {
"component_type": "ModularPipeline",
+ "remove_blocks": ["prompt_upsampler"],
"load_components": {
+ "names": [
+ "text_encoder",
+ "tokenizer",
+ "transformer",
+ "vae"
+ ],
"dtype": "torch.bfloat16"
},
"components": {
diff --git a/workflows/templates/flux3/multi-reference.json b/workflows/templates/flux3/multi-reference.json
index 76e867805..1ab436401 100644
--- a/workflows/templates/flux3/multi-reference.json
+++ b/workflows/templates/flux3/multi-reference.json
@@ -39,7 +39,14 @@
"pipeline": {
"configuration": {
"component_type": "ModularPipeline",
+ "remove_blocks": ["prompt_upsampler"],
"load_components": {
+ "names": [
+ "text_encoder",
+ "tokenizer",
+ "transformer",
+ "vae"
+ ],
"dtype": "torch.bfloat16"
},
"components": {
diff --git a/workflows/templates/flux3/text-to-image.json b/workflows/templates/flux3/text-to-image.json
index 2498f8c4d..30c278d14 100644
--- a/workflows/templates/flux3/text-to-image.json
+++ b/workflows/templates/flux3/text-to-image.json
@@ -1,6 +1,6 @@
{
"id": "Flux3KleinT2I",
- "description": "FLUX 3 Image Klein (distilled, 5.6B) text to image, the baseline of the FLUX 3 templates. The prompt here is already a full caption - BFL's own upsampled caption for their man walking a dog example - because Klein was trained on long upsampler captions and turns short prompts into generic pictures; upsample-then-generate shows how to get one from a short request. Sixteen steps with no guidance: the distilled model is a flow map, so it takes no guidance scale and no negative prompt. Width and height must be multiples of 16, which the workflow declares; BFL states sides of 64 to 4096, which is not checked. Every component is placed on the GPU explicitly, about 19 GB of bf16 weights, which fits a 24 GB card at 1024x1024 with under 2 GB to spare. The weights are gated under the FLUX Kommunity License: accept it on the model page and log in with hf auth login first.",
+ "description": "FLUX 3 Image Klein (distilled, 5.6B) text to image, the baseline of the FLUX 3 templates. The prompt here is already a full caption - BFL's own upsampled caption for their man walking a dog example - because Klein was trained on long upsampler captions and turns short prompts into generic pictures; upsample-then-generate shows how to get one from a short request. Sixteen steps with no guidance: the distilled model is a flow map, so it takes no guidance scale and no negative prompt. Width and height must be multiples of 16, which the workflow declares; BFL states sides of 64 to 4096, which is not checked. Only the four components the pipeline runs are loaded (the repo index also names an 18.8 GB prompt upsampler, which loading everything would put on the card too and overflow 24 GB). Every component is placed on the GPU explicitly, about 19 GB of bf16 weights, which fits a 24 GB card at 1024x1024 with under 2 GB to spare. The weights are gated under the FLUX Kommunity License: accept it on the model page and log in with hf auth login first.",
"cost_drivers": [
"width",
"height"
@@ -30,7 +30,14 @@
"pipeline": {
"configuration": {
"component_type": "ModularPipeline",
+ "remove_blocks": ["prompt_upsampler"],
"load_components": {
+ "names": [
+ "text_encoder",
+ "tokenizer",
+ "transformer",
+ "vae"
+ ],
"dtype": "torch.bfloat16"
},
"components": {
diff --git a/workflows/templates/flux3/upsample-then-generate.json b/workflows/templates/flux3/upsample-then-generate.json
index f9d8e5e88..6500a8e42 100644
--- a/workflows/templates/flux3/upsample-then-generate.json
+++ b/workflows/templates/flux3/upsample-then-generate.json
@@ -45,7 +45,14 @@
"pipeline": {
"configuration": {
"component_type": "ModularPipeline",
+ "remove_blocks": ["prompt_upsampler"],
"load_components": {
+ "names": [
+ "text_encoder",
+ "tokenizer",
+ "transformer",
+ "vae"
+ ],
"dtype": "torch.bfloat16"
},
"components": {
+ 
+