Spec 001 T046: the figures step

specs/001-pathway-pdfs/contracts/cli.md

@@ -40,6 +40,9 @@ contracts/figures.md).
   at `--content`; no graph checksum, no render `build`, or a render service not reading local files; or an `--out` that is not
   empty. A `build` or content release that changes during the run, a render or content
   service that stops answering, or the breaker, is exit 1 (contracts/figures.md).
+  Any other error (the graph failing during the checks, a file that cannot be written) is
+  exit 2, its error on stderr, the run stopped and its report written. The output's lines
+  and the report are in contracts/figures.md.
 
 ## `pdf-export book`
 

specs/001-pathway-pdfs/contracts/figures.md

@@ -102,6 +102,15 @@ What `pdf-export figures` writes and `pdf-export book` reads (SC-010, research R
   holding no names, only stable identifiers, kinds, times and counts. Any other file
   is ignored, so a folder drawn for a release can be read again by a book made from the
   same graph.
+- **The run report** (`--report`, JSON; T046), written however the run ends: `release`
+  (the fingerprint, as far as it was read), `refused` (why the run was refused before
+  drawing, exit 2), `preflight` (its faults, exit 1), `figures` (for each kind: `needed`,
+  `drawn` by the render service, with `totalMs` and `maxMs`, retries included, and
+  `written`, in the folder with its line), `missing` (each figure's own fault, by `stId`,
+  `cause` and `detail`), `stopped` (`reason`: `deployment-changed`,
+  `deployment-unanswered`, `breaker-unanswered`, `breaker-silent` or `error`, and its
+  `detail`), `complete` and `elapsedMs`. On standard output, one `fault <fault>` line for
+  each preflight fault and `fault missing-figure <stId> <cause>` for each figure's own.
 - **A figure that could not be drawn** has no file and no line: `figures` draws the rest
   (unless the run stops at once, above),
   exits 1 with no `complete` line, and the book refuses the folder.

specs/001-pathway-pdfs/tasks.md

@@ -403,7 +403,7 @@ passes.
   - **From #149's review:** (1) an error during the preflight was exit 1 with no fault line, the faults already found lost (a diagram file not JSON, a graph failing after its retries, `buildBook`'s "not read from the graph"); exit 1 is to mean faults. A diagram file that cannot be read as JSON is now a fault, `bad-file <name>` (contracts/cli.md), its labels unchecked and the run going on; any other error ends the command with exit 2, its error on stderr, after the faults found before it are printed (`preflight` adds each check's faults to a list its caller holds). Both seen failing first: a corrupt `diagram/R-HSA-90000130.json` with an empty `--figures` gave exit 1 and nothing printed, and a graph failing at the reactions' names exit 1 with `incomplete-folder` lost. (2) `retryTimeMs: 0` still makes one retry, and it applied to every read: the start's read now has a driver of its own with it (said within seconds), and the checks' reads the driver's default; the comment and contract say "bounds the retrying". (3) The website's diagram draws a node's attachments' `label`s (a modification's lettering); `src/preflight/labels.ts` now reads them. `figureNames` does not read a reaction figure's equivalent (its entities' modified residues' PSI-MOD labels), recorded, not built: the fixture has no modification to test it on. Recorded too: a normal pathway's diagram drawn beneath a disease diagram (`forNormalDraw` false) is not read apart from its file. (4) Tests: a stale folder that also has a figure with no line and a missing file gives the one fault; a selection that is the right one's beginning is `stale-figure`; a figure line whose selection is not all strings is not whole. (5) Recorded, not changed: the fingerprint's `content` is not compared with the graph's release by `book`, defence in depth left to T046's check before drawing; "ignoring case" is `toLowerCase`, not full case folding or Unicode normalisation (final sigma, ß, NFC against NFD); the text check reads strings that are never printed (web addresses, and whatever the model holds beside what the layout writes), so a fault can be false; T047 needs the fingerprint compared and `none` told from `incomplete` (now in T047's text); a truncated `.graph.json` is not found, only its presence checked. 347 tests before (on main with T027), 350 after. Each of these mutants made one fail: the refusal as a whole removed (the stale folder's test); the selection's length not compared (the beginning's case); the selection's ids not checked as strings (the unit case); attachments not read; a file not JSON thrown again (the bad-file test); the error not caught (the exit 2 test); the faults found not printed on an error (the same); the start's read without its bound (the 15 s bound). Of the earlier mutants, 26 were run again on this code and each still fails a test; the other 12 (among them the file-name rule's, the substitutions' and the unreachable graph's) were not re-run; the two equivalents still survive.
   - **From #150's review:** the start's driver was opened outside the `try`, and the driver throws at once on an address it cannot parse, so `NEO4J_URI=127.0.0.1:17700` gave exit 1 with a stack trace; it is opened inside it now, exit 2 with the error on stderr (seen failing: `Unknown scheme: null`; tested only with addresses that fail to parse, never one falling back to the default port). On an error the faults are printed first, then the error (seen failing, `err` before `out`). The faults are kept check by check, as contracts/cli.md now says: those of the check an error stops are lost. A test runs the command as a child process and holds it to 15 s, so a driver left open is caught. contracts/cli.md adds `bad-file` to `book`'s preflight and to `figures`' exit 1, and says `book` exits 2 on an error during its preflight. Each of these mutants made one fail: the start's driver not closed (the child process's test); the checks' driver not closed (the same, and the test file's run then never ends); the start's driver opened outside the `try` (the address test); the error written before the faults (the exit 2 test's order).
 
-- [ ] T046 [US1] `pdf-export figures`, `src/figures/step.ts` and `src/cli.ts` (contracts/cli.md, contracts/figures.md, SC-010).
+- [x] T046 [US1] `pdf-export figures`, `src/figures/step.ts` and `src/cli.ts` (contracts/cli.md, contracts/figures.md, SC-010).
   - Lists the book's figures by the Figure rules (T013's `diagramChoices` with `nearestAncestor` from `src/model/build.ts`, and the EHLD summary from `--files`: the same code the book uses), and draws each through T014's client into `--out`, `{stId}.svg`, written beside its name and moved into place.
   - After T024: runs its release-file and pre-drawing character checks first (`src/preflight/`), and the label check after each figure is drawn.
   - Asks the content service named by `--content` for its release, and refuses another (exit 2); refuses a render service whose health answer does not say `files: local` (exit 2; contracts/render.md).
@@ -415,6 +415,8 @@ passes.
 
   Tests, on the fixture against the stub render service: exit 0 with one file per figure the fixture's book needs, by kind, and a `complete` manifest that passes T024's folder checks, seen failing with a selection keyed by its ancestor; a non-empty `--out` is refused (exit 2); a `build` of `unknown`, and a graph with no checksum, are refused; a `build` changed between a figure's render and the read after it, and a content release changed likewise, each leave that figure with no file and no line, and the run exits 1 with no `complete`; a figure failing its label check is left with no file and no line; one figure the stub refuses with a 404 and one with a 500 (its health unchanged), and one failing its label check: the run draws all the others, then exits 1 naming all three, with no `complete`; a content service not answering the read after a figure stops the run, exit 1; the render stub stopping answering mid-run stops the run at the next failed figure, exit 1, without waiting through the rest, and its report names why; the stub answering `/health` but returning 504 to every render trips the breaker, exit 1, its report naming it; ten figures in a row answering 500 (cause `failed`) do not trip it, the run drawing the rest; nor do 500s answered slowly for longer than `5 × --timeout` in all, each within `--timeout`; a deployment change's stop writes the report too; an unreachable content service is exit 2; a content service answering another release is refused before any drawing; a planted missing release file and a planted uncovered character each fail with exit 1 before any request; a figure that times out once and then answers, and one that answers 504 once and then draws, are drawn (the retry); a refused figure leaves no partial file; exit 2 on the render service unreachable.
 
+  - **Done (#154):** `src/figures/step.ts` and `pdf-export figures` (`src/cli.ts`). **Order:** the settings and `--out` (absent or empty, else exit 2); the graph's release and checksum (none, or an unreachable graph, exit 2); the figures from `buildBook` and `expectedFigures` (T048's and T024's, the code `book` uses), then T024's release-file and pre-drawing character checks, before any request is made (a fault is exit 1, `fault <fault>` lines); then the render service's health (`build` not `unknown`, `files: local`) and the content release, which must be the graph's (exit 2). The file-name and folder checks are not run: T046 names only the files' and characters'. **Drawing:** `--parallel` workers over the figures; each answer checked by `labelsInBook` with a `BookCharacters` of its own (its fault the cause `label`, no file); written to `{stId}.svg.part`, synced, renamed, with nothing awaited between, so a partial file is left only by a crash (the folder itself is not synced); then health and content read again within `--timeout`, and the line appended only if both, and `files: local`, are the fingerprint's and the run has not stopped, else the file is removed. The manifest is one descriptor opened `wx`, each line written and synced synchronously. After an `unanswered` figure the same read is made. **The breaker:** `2 × --parallel` `unanswered` in a row, or no figure answered for `5 × --timeout`; an answer is a figure drawn (label faults included), `failed`, `refused` or `invalid`; the health reads do not count. A stop aborts the work in flight (the client's signal). An error of the step's own (a file that cannot be written) stops the run, `error`, exit 2. **The client:** `{ missing, cause }`; `retryUnanswered` (only `figures` sets it) retries a 504, or a timeout of the request or its body, once, after the 503's pause, one retry per figure in all; a refused connection and a body cut by a closed connection are not retried (gone, not slow); a non-OK status that is not 4xx or 5xx is `invalid`. `serve` is unchanged. **The report** (contracts/figures.md now gives its fields and the stop reasons; contracts/cli.md the other errors): written in a `finally`, refusals and preflight faults included; times are each figure's from its first request to its answer, retries included. `--timeout` takes fractional seconds. No signal handling: an interrupted run leaves a folder the book refuses. **Stubs:** the render stub gains 500 and 504 refusals, `refuseAll`, a settable `health`, an `onRender` hook (run for refusals too), `silence` and `slow`, and its `reset` drops held and delayed answers (a stopped run's leftover request had used up a counted refusal in a later test); `test/stubs/content.ts` answers `GET /data/database/version` as plain text, its release settable, and can be silenced. **Tests:** `test/figures/step.test.ts` (22: every case above, plus the read after an unanswered figure, a stop ending work in flight, the silent breaker, `files: local`, the settings, an address the driver cannot parse, and a graph failing during the checks, its faults so far printed first, as T024's final `preflight` does), `test/figures/client.test.ts` (2 more: causes; retries by caller), `test/stubs/render.test.ts` (3 more), `test/stubs/content.test.ts` (1), `test/cli.test.ts` (1, the command run as a process). 362 tests before (on main with T024 and T028), 391 after. Each of these mutants made one fail: a figure's file, or its line, keyed by its source (the complete manifest's test, through T024's `checkFolder`); a non-empty `--out` allowed; `unknown` accepted; `files` not checked at the start; no checksum accepted; the content release not compared at the start; the build, or the content release, not read again; the file left on a change; the line appended before the read; an unanswered read not a stop; no read after an unanswered figure; the label check skipped; a file written before the label check; `complete` despite own faults, or despite a stop; the count breaker removed; every missing figure counted; the silent breaker not refreshed, or removed; a stop not ending the work in flight; the files, or the characters, not checked first; `figures` without the retry; the report not written; `figures` not dispatched by the CLI; in the client, a 504, a timeout or a body's timeout not retried, `serve` retrying them, two retries, a 500 `unanswered`, a 504 `failed`, a 4xx `failed`, a non-SVG `refused`, a cut-short body `failed`; the start's driver opened outside the `try` (an unparseable address thrown); the faults found before an error not printed. None survive.
+
 - [x] T025 [US1] Parts, `src/print/parts.ts`.
   - Each of the book's PDFs is printed in parts (T048's documents).
   - A part is split at section boundaries past a page threshold: a setting, about 200 pages by default.

src/cli.ts

@@ -1,14 +1,16 @@
 #!/usr/bin/env node
 // pdf-export's command line (contracts/cli.md). Its commands come with spec 001's tasks:
 // serve (T022) runs the single-event service; preflight (T024) runs the book's preflight
-// alone; book, check-book and compare are to come.
+// alone; figures (T046) draws the book's figures into a folder; book, check-book and compare
+// are to come.
 
 export const USAGE = `Usage: pdf-export <command> [options]
 
 Commands:
   book         write the release's book (spec 001, User Story 1)
   check-book   check a book for FR-011's faults
   compare      compare with event-pdf's output, by meaning (FR-012)
+  figures      draw the book's figures into a folder, before the book (SC-010)
   preflight    check the book's inputs, as book does before printing
   serve        the website's single-event PDF service (User Story 2)
 `;
@@ -56,6 +58,12 @@ if (import.meta.main) {
       out: (text) => process.stdout.write(text),
       err: (text) => process.stderr.write(text),
     });
+  } else if (args[0] === 'figures') {
+    const { figuresCommand } = await import('./figures/step.ts');
+    process.exitCode = await figuresCommand(args.slice(1), process.env, {
+      out: (text) => process.stdout.write(text),
+      err: (text) => process.stderr.write(text),
+    });
   } else {
     const { code, out } = run(args);
     (code === 0 ? process.stdout : process.stderr).write(out);