diff --git a/specs/001-pathway-pdfs/contracts/cli.md b/specs/001-pathway-pdfs/contracts/cli.md index 04afc77..4033e5f 100644 --- a/specs/001-pathway-pdfs/contracts/cli.md +++ b/specs/001-pathway-pdfs/contracts/cli.md @@ -17,7 +17,7 @@ contracts/figures.md). | `--out` | | the folder written (contracts/figures.md) | | `--parallel` | | figures asked at once; default 2, the service's own default; set it to the service's `RENDER_CONCURRENCY` | | `--timeout` | | seconds the client waits for each attempt at a figure, and for each health or release read; default 60; above the render service's own timeout (contracts/render.md) | -| `--report` | | where its run report is written, as JSON | +| `--report` | | where its run report is written, as JSON; a file there already is refused (exit 2) | - **Which figures:** the events' Figure rules (data-model.md), from the graph and the EHLD summary, the same code `book` uses, so the folder holds exactly what the book @@ -29,17 +29,25 @@ contracts/figures.md). must answer the graph's release (`GET /data/database/version`), the graph must carry a `DBInfo` checksum, and the render service's health answer a `build` other than `unknown` and `files: local` (contracts/render.md), or the run is refused before anything is drawn - (exit 2). What this cannot see is in contracts/figures.md. + (exit 2). Both services are probed before the preflight, so one down is said at once, and + read again after it for the folder's fingerprint; the preflight asks for no figure. What + this cannot see is in contracts/figures.md. - **One run, one folder:** `--out` must be empty or absent (exit 2 otherwise); a stopped run is not resumed. The manifest (contracts/figures.md) records the folder's fingerprint and what each file draws. - **Exit 0:** every figure is in the folder. **Exit 1:** a preflight fault (`missing-file`, `uncovered-character`, `bad-file`), a label fault after drawing, or a figure that could not be drawn, each named by stable identifier or file name. **Exit 2:** bad - arguments; the render or content service unreachable at the start; another release + arguments; a `--report` file that exists already (the earlier run's report kept, and none + written); the render or content service unreachable at the start; another release at `--content`; no graph checksum, no render `build`, or a render service not reading local files; or an `--out` that is not empty. A `build` or content release that changes during the run, a render or content service that stops answering, or the breaker, is exit 1 (contracts/figures.md). + Any other error (the graph failing during the checks, a file that cannot be written) is + exit 2, its error on stderr, the figures' own faults so far printed, the run stopped and + its report written. The report is written however the run ends but two: a signal, and a + `--report` refused; one that cannot be written is said on stderr, and the run keeps its + exit. The output's lines and the report are in contracts/figures.md. ## `pdf-export book` diff --git a/specs/001-pathway-pdfs/contracts/figures.md b/specs/001-pathway-pdfs/contracts/figures.md index b2edd09..1ebe935 100644 --- a/specs/001-pathway-pdfs/contracts/figures.md +++ b/specs/001-pathway-pdfs/contracts/figures.md @@ -65,8 +65,8 @@ What `pdf-export figures` writes and `pdf-export book` reads (SC-010, research R but for `ancestor`). - **A deployment changed, a service gone, or the breaker tripped, during a run** stops it at once with exit 1 and no `complete` line. - Figures still being drawn when it stops get no line. The report is written either way, - naming the reason it stopped and each figure's own fault so far. + Figures still being drawn when it stops get no line. The report is written either way + (not on a signal), naming the reason it stopped and each figure's own fault so far. - **What the fingerprint cannot see** (research R10): - the render service's `build` hashes its own code, not the website's built page it loads, which is what draws; @@ -102,6 +102,19 @@ What `pdf-export figures` writes and `pdf-export book` reads (SC-010, research R holding no names, only stable identifiers, kinds, times and counts. Any other file is ignored, so a folder drawn for a release can be read again by a book made from the same graph. +- **The run report** (`--report`, JSON; T046), written beside its name and moved into + place, its folder made if need be, however the run ends but on a signal; a `--report` + that exists already is refused (exit 2) and no report is written, so an earlier run's + is never replaced; a report that cannot be written is said on stderr, the run keeping + its exit. Its fields: `release` + (the fingerprint, as far as it was read), `refused` (why the run was refused before + drawing, exit 2), `preflight` (its faults, exit 1), `figures` (for each kind: `needed`, + `drawn` by the render service, with `totalMs` and `maxMs`, retries included, and + `written`, in the folder with its line), `missing` (each figure's own fault, by `stId`, + `cause` and `detail`), `stopped` (`reason`: `deployment-changed`, + `deployment-unanswered`, `breaker-unanswered`, `breaker-silent` or `error`, and its + `detail`), `complete` and `elapsedMs`. On standard output, one `fault ` line for + each preflight fault and `fault missing-figure ` for each figure's own. - **A figure that could not be drawn** has no file and no line: `figures` draws the rest (unless the run stops at once, above), exits 1 with no `complete` line, and the book refuses the folder. diff --git a/specs/001-pathway-pdfs/tasks.md b/specs/001-pathway-pdfs/tasks.md index 8f239d8..a0f3c98 100644 --- a/specs/001-pathway-pdfs/tasks.md +++ b/specs/001-pathway-pdfs/tasks.md @@ -403,7 +403,7 @@ passes. - **From #149's review:** (1) an error during the preflight was exit 1 with no fault line, the faults already found lost (a diagram file not JSON, a graph failing after its retries, `buildBook`'s "not read from the graph"); exit 1 is to mean faults. A diagram file that cannot be read as JSON is now a fault, `bad-file ` (contracts/cli.md), its labels unchecked and the run going on; any other error ends the command with exit 2, its error on stderr, after the faults found before it are printed (`preflight` adds each check's faults to a list its caller holds). Both seen failing first: a corrupt `diagram/R-HSA-90000130.json` with an empty `--figures` gave exit 1 and nothing printed, and a graph failing at the reactions' names exit 1 with `incomplete-folder` lost. (2) `retryTimeMs: 0` still makes one retry, and it applied to every read: the start's read now has a driver of its own with it (said within seconds), and the checks' reads the driver's default; the comment and contract say "bounds the retrying". (3) The website's diagram draws a node's attachments' `label`s (a modification's lettering); `src/preflight/labels.ts` now reads them. `figureNames` does not read a reaction figure's equivalent (its entities' modified residues' PSI-MOD labels), recorded, not built: the fixture has no modification to test it on. Recorded too: a normal pathway's diagram drawn beneath a disease diagram (`forNormalDraw` false) is not read apart from its file. (4) Tests: a stale folder that also has a figure with no line and a missing file gives the one fault; a selection that is the right one's beginning is `stale-figure`; a figure line whose selection is not all strings is not whole. (5) Recorded, not changed: the fingerprint's `content` is not compared with the graph's release by `book`, defence in depth left to T046's check before drawing; "ignoring case" is `toLowerCase`, not full case folding or Unicode normalisation (final sigma, ß, NFC against NFD); the text check reads strings that are never printed (web addresses, and whatever the model holds beside what the layout writes), so a fault can be false; T047 needs the fingerprint compared and `none` told from `incomplete` (now in T047's text); a truncated `.graph.json` is not found, only its presence checked. 347 tests before (on main with T027), 350 after. Each of these mutants made one fail: the refusal as a whole removed (the stale folder's test); the selection's length not compared (the beginning's case); the selection's ids not checked as strings (the unit case); attachments not read; a file not JSON thrown again (the bad-file test); the error not caught (the exit 2 test); the faults found not printed on an error (the same); the start's read without its bound (the 15 s bound). Of the earlier mutants, 26 were run again on this code and each still fails a test; the other 12 (among them the file-name rule's, the substitutions' and the unreachable graph's) were not re-run; the two equivalents still survive. - **From #150's review:** the start's driver was opened outside the `try`, and the driver throws at once on an address it cannot parse, so `NEO4J_URI=127.0.0.1:17700` gave exit 1 with a stack trace; it is opened inside it now, exit 2 with the error on stderr (seen failing: `Unknown scheme: null`; tested only with addresses that fail to parse, never one falling back to the default port). On an error the faults are printed first, then the error (seen failing, `err` before `out`). The faults are kept check by check, as contracts/cli.md now says: those of the check an error stops are lost. A test runs the command as a child process and holds it to 15 s, so a driver left open is caught. contracts/cli.md adds `bad-file` to `book`'s preflight and to `figures`' exit 1, and says `book` exits 2 on an error during its preflight. Each of these mutants made one fail: the start's driver not closed (the child process's test); the checks' driver not closed (the same, and the test file's run then never ends); the start's driver opened outside the `try` (the address test); the error written before the faults (the exit 2 test's order). -- [ ] T046 [US1] `pdf-export figures`, `src/figures/step.ts` and `src/cli.ts` (contracts/cli.md, contracts/figures.md, SC-010). +- [x] T046 [US1] `pdf-export figures`, `src/figures/step.ts` and `src/cli.ts` (contracts/cli.md, contracts/figures.md, SC-010). - Lists the book's figures by the Figure rules (T013's `diagramChoices` with `nearestAncestor` from `src/model/build.ts`, and the EHLD summary from `--files`: the same code the book uses), and draws each through T014's client into `--out`, `{stId}.svg`, written beside its name and moved into place. - After T024: runs its release-file and pre-drawing character checks first (`src/preflight/`), and the label check after each figure is drawn. - Asks the content service named by `--content` for its release, and refuses another (exit 2); refuses a render service whose health answer does not say `files: local` (exit 2; contracts/render.md). @@ -415,6 +415,9 @@ passes. Tests, on the fixture against the stub render service: exit 0 with one file per figure the fixture's book needs, by kind, and a `complete` manifest that passes T024's folder checks, seen failing with a selection keyed by its ancestor; a non-empty `--out` is refused (exit 2); a `build` of `unknown`, and a graph with no checksum, are refused; a `build` changed between a figure's render and the read after it, and a content release changed likewise, each leave that figure with no file and no line, and the run exits 1 with no `complete`; a figure failing its label check is left with no file and no line; one figure the stub refuses with a 404 and one with a 500 (its health unchanged), and one failing its label check: the run draws all the others, then exits 1 naming all three, with no `complete`; a content service not answering the read after a figure stops the run, exit 1; the render stub stopping answering mid-run stops the run at the next failed figure, exit 1, without waiting through the rest, and its report names why; the stub answering `/health` but returning 504 to every render trips the breaker, exit 1, its report naming it; ten figures in a row answering 500 (cause `failed`) do not trip it, the run drawing the rest; nor do 500s answered slowly for longer than `5 × --timeout` in all, each within `--timeout`; a deployment change's stop writes the report too; an unreachable content service is exit 2; a content service answering another release is refused before any drawing; a planted missing release file and a planted uncovered character each fail with exit 1 before any request; a figure that times out once and then answers, and one that answers 504 once and then draws, are drawn (the retry); a refused figure leaves no partial file; exit 2 on the render service unreachable. + - **Done (#155, replacing #154 after its review):** `src/figures/step.ts` and `pdf-export figures` (`src/cli.ts`). **Order:** the settings and `--out` (absent or empty, else exit 2); the graph's release and checksum (none, or an unreachable graph, exit 2); a probe of the render service's health (`build` not `unknown`, `files: local`) and the content release, which must be the graph's (exit 2); the figures from `buildBook` and `expectedFigures` (T048's and T024's, the code `book` uses), then T024's release-file and pre-drawing character checks, before any figure is asked for (a fault is exit 1, `fault ` lines); then the same read again, its values the fingerprint (from #154's review, below). The file-name and folder checks are not run: T046 names only the files' and characters'. **Drawing:** `--parallel` workers over the figures; each answer checked by `labelsInBook` with a `BookCharacters` of its own (its fault the cause `label`, no file); written to `{stId}.svg.part`, synced, renamed, with nothing awaited between, so a partial file is left only by a crash (the folder itself is not synced); then health and content read again within `--timeout`, and the line appended only if both, and `files: local`, are the fingerprint's and the run has not stopped, else the file is removed. The manifest is one descriptor opened `wx`, each line written and synced synchronously. After an `unanswered` figure the same read is made. **The breaker:** `2 × --parallel` `unanswered` in a row, or no figure answered for `5 × --timeout` (at least three timeouts and the retry's pause, from #154's review); an answer is a figure drawn (label faults included), `failed`, `refused` or `invalid`; the health reads do not count. A stop aborts the work in flight (the client's signal). An error of the step's own (a file that cannot be written) stops the run, `error`, exit 2. **The client:** `{ missing, cause }`; `retryUnanswered` (only `figures` sets it) retries a 504, or a timeout of the request or its body, once, after the 503's pause, one retry per figure in all; a refused connection and a body cut by a closed connection are not retried (gone, not slow); a non-OK status that is not 4xx or 5xx is `invalid`. `serve` is unchanged. **The report** (contracts/figures.md now gives its fields and the stop reasons; contracts/cli.md the other errors): written in a `finally`, refusals and preflight faults included (not on a signal, nor for a `--report` refused: from #154's review); times are each figure's from its first request to its answer, retries included. `--timeout` takes fractional seconds. No signal handling: an interrupted run leaves a folder the book refuses. **Stubs:** the render stub gains 500 and 504 refusals, `refuseAll`, a settable `health`, an `onRender` hook (run for refusals too), `silence` and `slow`, and its `reset` drops held and delayed answers (a stopped run's leftover request had used up a counted refusal in a later test); `test/stubs/content.ts` answers `GET /data/database/version` as plain text, its release settable, and can be silenced. **Tests:** `test/figures/step.test.ts` (22: every case above, plus the read after an unanswered figure, a stop ending work in flight, the silent breaker, `files: local`, the settings, an address the driver cannot parse, and a graph failing during the checks, its faults so far printed first, as T024's final `preflight` does), `test/figures/client.test.ts` (2 more: causes; retries by caller), `test/stubs/render.test.ts` (3 more), `test/stubs/content.test.ts` (1), `test/cli.test.ts` (1, the command run as a process). 362 tests before (on main with T024 and T028), 391 after. Each of these mutants made one fail: a figure's file, or its line, keyed by its source (the complete manifest's test, through T024's `checkFolder`); a non-empty `--out` allowed; `unknown` accepted; `files` not checked at the start; no checksum accepted; the content release not compared at the start; the build, or the content release, not read again; the file left on a change; the line appended before the read; an unanswered read not a stop; no read after an unanswered figure; the label check skipped; a file written before the label check; `complete` despite own faults, or despite a stop; the count breaker removed; every missing figure counted; the silent breaker not refreshed, or removed; a stop not ending the work in flight; the files, or the characters, not checked first; `figures` without the retry; the report not written; `figures` not dispatched by the CLI; in the client, a 504, a timeout or a body's timeout not retried, `serve` retrying them, two retries, a 500 `unanswered`, a 504 `failed`, a 4xx `failed`, a non-SVG `refused`, a cut-short body `failed`; the start's driver opened outside the `try` (an unparseable address thrown); the faults found before an error not printed. For the survivors found by #154's review, see below. + - **From #154's review:** (1) a figure missing as `unanswered` was untested on the way to no `complete`: pushing to `report.missing` only for other causes passed every test, and with one figure refused 504 every time it wrote `complete` with 17 of 18. Now a test gives exactly that (exit 1, the figure named, no `complete`, the other 17 drawn), and `complete` also needs a line for every figure, counted by the one writer. (2) `--report`: an existing file is now refused first (exit 2, nothing asked, no report written, so an earlier run's is never replaced); the report's folder is made; it is written beside its name and renamed into place; and a report that cannot be written is said on stderr, the run keeping its exit (before, a refusal with `--report` in an absent `--out` threw ENOENT out of the `finally`, exit 1). contracts/cli.md and contracts/figures.md now say a signal writes none, nor does a `--report` refused. (3) The services are probed before the preflight, so one down is said at once, and read again after it for the fingerprint; "before any request" in the tests now means before any figure is asked for (the probes are expected). The silent breaker's silence is at least `3 × --timeout` plus the retry's pause, as at `--timeout 1` a healthy service's 5 s pause after a 503 tripped it. A figure's file and the manifest's lines are written with `writeFileSync` on the descriptor, which writes every byte or throws, so a short write on a full disk is not taken for a whole one. The figures' own faults so far are printed when the run ends on its own error (exit 2). The silence test takes its start from the first figure's request, after the preflight. The append's own `!stop.signal.aborted` was dropped: the read after a figure is already false once the run stops. (4) The content stub's `reset` drops its held answers. The render stub gains a `cut-body` refusal (the connection closed part way through the body) and each request's time, and the client's test shows that refusal is not retried. Recorded, not changed: every read after a figure is tried once, so one transient error of either service stops a run that cannot be resumed, which costs a rerun of hours; the contract asks for it, as an unanswered read cannot be told from a deployment gone. Tests: 9 more in `test/figures/step.test.ts` (31): the unanswered figure above, `files` changed in the read after a figure, unanswered figures with answers between them, a 503's pause at `--timeout 1`, an existing `--report`, a refusal's report in an absent folder, a report that cannot be written, the run's own error, and the fingerprint read after the preflight; the breaker's test now exact (one at a time, two figures, four requests), the silent breaker's timed and its silence checked; the unreachable services' test also with a planted missing file (exit 2, no preflight). 391 tests before, 400 after. Each of these mutants made one fail: `report.missing` only for causes other than `unanswered` (with the line-count guard, or without it); the count breaker `>` for `>=`; `answered()` not resetting the count; `files` not checked in the read after a figure; the silence `6 × --timeout`; the silence `5 × --timeout` alone; an existing `--report` not refused; the report's folder not made; a failed report write thrown; no probe before the preflight; the fingerprint taken from the probe; the figures' faults not printed on the run's own error; a closed connection's cut body retried; and each of #154's mutants, run again on this code. Three survive as equivalent: `complete` despite own faults, and despite a stop, each alone (the line count refuses both, and with it dropped too each fails a test); and the read after a figure not returning false once the run has stopped (its requests take the stop's signal, so a read in flight at a stop rejects; the only window is between its answers and its continuation, within one turn of the event loop). The write beside its name and the rename, for the report as for figures, and the whole writes are seen only on a crash or a full disk, which the tests do not make. + - [x] T025 [US1] Parts, `src/print/parts.ts`. - Each of the book's PDFs is printed in parts (T048's documents). - A part is split at section boundaries past a page threshold: a setting, about 200 pages by default. diff --git a/src/cli.ts b/src/cli.ts index 8840813..641fb8f 100755 --- a/src/cli.ts +++ b/src/cli.ts @@ -1,7 +1,8 @@ #!/usr/bin/env node // pdf-export's command line (contracts/cli.md). Its commands come with spec 001's tasks: // serve (T022) runs the single-event service; preflight (T024) runs the book's preflight -// alone; book, check-book and compare are to come. +// alone; figures (T046) draws the book's figures into a folder; book, check-book and compare +// are to come. export const USAGE = `Usage: pdf-export [options] @@ -9,6 +10,7 @@ Commands: book write the release's book (spec 001, User Story 1) check-book check a book for FR-011's faults compare compare with event-pdf's output, by meaning (FR-012) + figures draw the book's figures into a folder, before the book (SC-010) preflight check the book's inputs, as book does before printing serve the website's single-event PDF service (User Story 2) `; @@ -56,6 +58,12 @@ if (import.meta.main) { out: (text) => process.stdout.write(text), err: (text) => process.stderr.write(text), }); + } else if (args[0] === 'figures') { + const { figuresCommand } = await import('./figures/step.ts'); + process.exitCode = await figuresCommand(args.slice(1), process.env, { + out: (text) => process.stdout.write(text), + err: (text) => process.stderr.write(text), + }); } else { const { code, out } = run(args); (code === 0 ? process.stdout : process.stderr).write(out); diff --git a/src/figures/client.ts b/src/figures/client.ts index c1a412e..2e42908 100644 --- a/src/figures/client.ts +++ b/src/figures/client.ts @@ -1,8 +1,13 @@ -// The figures client (spec 001 T014, contracts/render.md): asks the website's render -// service for each figure as SVG. A 503 is tried once more after a pause; any other -// refusal, a second 503 or a timeout is a missing figure, with its reason, for the caller -// to fail the book on (FR-011) or go on without (a single-event PDF). The caller's own end -// (its signal) is thrown as it is, never read as a missing figure. +// The figures client (spec 001 T014, T046; contracts/render.md): asks the website's render +// service for each figure as SVG. A 503 is tried once more after a pause; when `figures` asks +// (`retryUnanswered`, T046), a 504 or a timeout (the body's included) is too, while `serve` +// keeps one try of them, as SC-005 bounds its time. Any other refusal, or the second try's, +// is a missing figure with its reason and its cause (contracts/figures.md): `unanswered` (no +// connection, a timeout, a body cut short, a 502, 503 or 504), `failed` (any other 5xx: the +// page failed on that figure), `refused` (a 4xx) or `invalid` (any other answer that is not +// an SVG drawing), for the caller to fail the book's figures on or go on without (a +// single-event PDF). The caller's own end (its signal) is thrown as it is, never read as a +// missing figure. import { setTimeout as pause } from 'node:timers/promises'; /** What to draw (data-model.md's Figure). */ @@ -11,14 +16,19 @@ export type FigureRequest = | { kind: 'ancestor'; stId: string; highlight: readonly string[]; token?: string } | { kind: 'reaction'; stId: string; token?: string }; -export type FigureAnswer = { svg: string } | { missing: string }; +/** Why a figure is missing (contracts/figures.md). */ +export type MissingCause = 'unanswered' | 'failed' | 'refused' | 'invalid'; + +export type FigureAnswer = { svg: string } | { missing: string; cause: MissingCause }; export interface ClientOptions { - /** How long one request may take. */ + /** How long one request may take, its body included. */ timeoutMs?: number; - /** The pause before the one retry of a 503 (the render service asks for 5 s). */ + /** The pause before the one retry (the render service asks for 5 s after a 503). */ retryPauseMs?: number; - /** The caller's own end: the service's deadline for the PDF. */ + /** Retry a 504 or a timeout once too, as a 503 is: `figures` asks it, `serve` does not. */ + retryUnanswered?: boolean; + /** The caller's own end: the service's deadline for the PDF, or `figures`' stop. */ signal?: AbortSignal; } @@ -38,10 +48,18 @@ export function figureUrl(base: string, request: FigureRequest): URL { return url; } +/** A status's cause: 502, 503 and 504 say the service did not draw for want of time or room. */ +function statusCause(status: number): MissingCause { + if (status === 502 || status === 503 || status === 504) return 'unanswered'; + if (status >= 500) return 'failed'; + if (status >= 400) return 'refused'; + return 'invalid'; +} + export async function fetchFigure( base: string, request: FigureRequest, - { timeoutMs = 60_000, retryPauseMs = 5_000, signal }: ClientOptions = {} + { timeoutMs = 60_000, retryPauseMs = 5_000, retryUnanswered = false, signal }: ClientOptions = {} ): Promise { const url = figureUrl(base, request); /** The caller's end is its own, not the render service failing: it is thrown, not missing. */ @@ -50,43 +68,75 @@ export async function fetchFigure( }; for (let attempt = 1; ; attempt++) { stopped(); - let response: Response; + const answer = await once(url, request.stId, timeoutMs, signal, stopped); + if (!('retry' in answer)) return answer; + if (attempt > 1 || !(answer.retry === 503 || retryUnanswered)) return answer.missing; + // the pause ends with the caller's end too try { - const timeout = AbortSignal.timeout(timeoutMs); - response = await fetch(url, { - signal: signal ? AbortSignal.any([timeout, signal]) : timeout, - }); + await pause(retryPauseMs, undefined, signal ? { signal } : {}); } catch (error) { stopped(); - return { missing: `${request.stId}: no answer (${reason(error)})` }; + throw error; } - if (response.ok) { - const type = response.headers.get('content-type') ?? ''; - if (!/^image\/svg\+xml(;|$)/.test(type)) { - await response.body?.cancel(); - return { missing: `${request.stId}: not an SVG (${type || 'no type'})` }; - } - // The timeout covers the body too: one cut short is a missing figure, not a failure - try { - return { svg: await response.text() }; - } catch (error) { - stopped(); - return { missing: `${request.stId}: the answer was cut short (${reason(error)})` }; - } + } +} + +type Attempt = + | FigureAnswer + /** A missing figure that may be tried again: a 503, or (for `figures`) a 504 or a timeout. */ + | { retry: 503 | 'other'; missing: FigureAnswer & { cause: MissingCause } }; + +/** One request for a figure, its answer read whole within `timeoutMs`. */ +async function once( + url: URL, + stId: string, + timeoutMs: number, + signal: AbortSignal | undefined, + stopped: () => void +): Promise { + const timeout = AbortSignal.timeout(timeoutMs); + let response: Response; + try { + response = await fetch(url, { signal: signal ? AbortSignal.any([timeout, signal]) : timeout }); + } catch (error) { + stopped(); + const missing = { + missing: `${stId}: no answer (${reason(error)})`, + cause: 'unanswered', + } as const; + return timedOut(error) ? { retry: 'other', missing } : missing; + } + if (response.ok) { + const type = response.headers.get('content-type') ?? ''; + if (!/^image\/svg\+xml(;|$)/.test(type)) { + await response.body?.cancel(); + return { missing: `${stId}: not an SVG (${type || 'no type'})`, cause: 'invalid' }; } - await response.body?.cancel(); - if (response.status === 503 && attempt === 1) { - // the pause ends with the caller's end too - try { - await pause(retryPauseMs, undefined, signal ? { signal } : {}); - } catch (error) { - stopped(); - throw error; - } - continue; + // The timeout covers the body too: one cut short is a missing figure, not a failure + try { + return { svg: await response.text() }; + } catch (error) { + stopped(); + const missing = { + missing: `${stId}: the answer was cut short (${reason(error)})`, + cause: 'unanswered', + } as const; + return timedOut(error) ? { retry: 'other', missing } : missing; } - return { missing: `${request.stId}: the render service answered ${String(response.status)}` }; } + await response.body?.cancel(); + const missing = { + missing: `${stId}: the render service answered ${String(response.status)}`, + cause: statusCause(response.status), + }; + if (response.status === 503) return { retry: 503, missing }; + if (response.status === 504) return { retry: 'other', missing }; + return missing; +} + +/** A request ended by its own timeout (fetch's, or the body's read). */ +function timedOut(error: unknown): boolean { + return error instanceof Error && error.name === 'TimeoutError'; } /** An error's name, and its cause's code where there is one (ECONNREFUSED, say). */ diff --git a/src/figures/step.ts b/src/figures/step.ts new file mode 100644 index 0000000..beee9b3 --- /dev/null +++ b/src/figures/step.ts @@ -0,0 +1,648 @@ +// `pdf-export figures` (spec 001 T046; contracts/cli.md, contracts/figures.md, SC-010): draws +// every figure the book needs, before the book, into a folder the book reads. In order: +// - the settings; a `--report` that exists already (exit 2, its file untouched); an `--out` +// that is absent or empty (exit 2 otherwise: no resuming); +// - the graph's release and checksum (none, or no graph, is exit 2); +// - a probe of the render service's health (`build` not `unknown`, `files: local`) and the +// content service's release, which must be the graph's (exit 2 otherwise), so a service +// down is said before the preflight's minutes; +// - the book's figures by the Figure rules, built as `book` builds them (T048's `buildBook` +// and the EHLD summary), and T024's release-file and pre-drawing character checks, before +// any figure is asked for (a fault is exit 1); +// - the same read again, its values the folder's fingerprint; +// - each figure drawn through T014's client, its 504s and timeouts retried once, checked for +// its labels' characters (T024's `labelsInBook`), written beside its name, synced and moved +// into place, and given its manifest line only once the render service's `build` and the +// content release, read again, still equal the fingerprint's. +// A figure's own fault (missing, or failing its label check) leaves it with no file and no +// line while the rest are drawn; a deployment changed or gone, or the breaker, stops the run at +// once. Either way the run ends with exit 1 and no `complete` line. Its report is written +// however the run ends, but for a signal and a `--report` refused. One writer appends the +// manifest, synchronously, so lines never interleave. +import { + closeSync, + existsSync, + fsyncSync, + mkdirSync, + openSync, + readdirSync, + renameSync, + rmSync, + statSync, + writeFileSync, +} from 'node:fs'; +import { dirname, join } from 'node:path'; + +import { + graphSettings, + openGraph, + type Graph, + type GraphOptions, + type GraphSettings, +} from '../graph/connection.ts'; +import { releaseInfo, type ReleaseInfo } from '../graph/queries.ts'; +import { BookCharacters } from '../layout/characters.ts'; +import { buildBook } from '../model/build.ts'; +import { checkCharacters } from '../preflight/characters.ts'; +import { SettingsError } from '../preflight/command.ts'; +import { checkFiles, neededFiles, readSummary, SUMMARY } from '../preflight/files.ts'; +import { + expectedFigures, + manifestLine, + MANIFEST, + type FigureRecord, + type Fingerprint, + type ManifestLine, +} from '../preflight/folder.ts'; +import { fetchFigure, type FigureRequest, type MissingCause } from './client.ts'; +import { labelsInBook } from './labels.ts'; + +export interface FiguresSettings { + graph: GraphSettings; + species: string; + /** The release's folder: the EHLD summary, and the files the preflight looks for. */ + files: string; + /** The render service's base URL. */ + render: string; + /** The content service's base URL, asked only for its release. */ + content: string; + out: string; + /** Figures asked at once. */ + parallel: number; + /** How long each attempt at a figure, and each health or release read, may take. */ + timeoutMs: number; + /** Where the run report is written, as JSON; none without `--report`. */ + report: string | null; +} + +const FLAGS = [ + '--graph', + '--database', + '--species', + '--files', + '--render', + '--content', + '--out', + '--parallel', + '--timeout', + '--report', +]; + +/** The settings from the flags and the environment; the credentials from the environment. */ +export function figuresSettings( + args: readonly string[], + env: Readonly> +): FiguresSettings { + const flags = new Map(); + for (let i = 0; i < args.length; i++) { + const [name = '', inline] = (args[i] ?? '').split(/=(.*)/s, 2); + if (!FLAGS.includes(name)) throw new SettingsError(`Unknown argument: ${name}`); + const value = inline ?? args[++i]; + if (value === undefined || value === '') throw new SettingsError(`${name} needs a value`); + if (flags.has(name)) throw new SettingsError(`${name} is given twice`); + flags.set(name, value); + } + const required = (flag: string, variable?: string): string => { + const value = flags.get(flag) ?? (variable === undefined ? undefined : env[variable]); + if (value === undefined || value === '') { + throw new SettingsError(`${flag}${variable ? ` (or ${variable})` : ''} is needed`); + } + return value; + }; + const graphEnv = { ...env }; + const uri = flags.get('--graph'); + if (uri !== undefined) graphEnv['NEO4J_URI'] = uri; + const database = flags.get('--database'); + if (database !== undefined) graphEnv['NEO4J_DATABASE'] = database; + let graph: GraphSettings; + try { + graph = graphSettings(graphEnv); + } catch { + throw new SettingsError('--graph (or NEO4J_URI) is needed'); + } + const parallel = flags.get('--parallel') ?? '2'; + if (!/^[1-9]\d*$/.test(parallel)) throw new SettingsError('--parallel must be a whole number'); + const timeout = flags.get('--timeout') ?? '60'; + if (!/^\d+(\.\d+)?$/.test(timeout) || Number(timeout) <= 0) { + throw new SettingsError('--timeout must be a number of seconds above 0'); + } + return { + graph, + species: flags.get('--species') ?? 'Homo sapiens', + files: required('--files'), + render: required('--render', 'RENDER_SERVICE'), + content: required('--content', 'CONTENT_SERVICE'), + out: required('--out'), + parallel: Number(parallel), + timeoutMs: Number(timeout) * 1000, + report: flags.get('--report') ?? null, + }; +} + +/** A read of a service that did not give what was asked: no answer, or a bad one. */ +class ReadError extends Error { + override name = 'ReadError'; +} + +const base = (url: string): string => (url.endsWith('/') ? url : `${url}/`); + +/** A GET within `timeoutMs` (or the caller's end), its body read whole, or a ReadError. */ +async function get(url: URL, timeoutMs: number, signal?: AbortSignal): Promise { + const timeout = AbortSignal.timeout(timeoutMs); + try { + const response = await fetch(url, { + signal: signal ? AbortSignal.any([timeout, signal]) : timeout, + }); + const body = await response.text(); + if (!response.ok) throw new ReadError(`${url.pathname} answered ${String(response.status)}`); + return body; + } catch (error) { + if (error instanceof ReadError) throw error; + const code = ( + error instanceof Error ? (error.cause as { code?: unknown } | undefined) : undefined + )?.code; + const why = error instanceof Error ? error.name : String(error); + throw new ReadError( + `${url.pathname}: no answer (${typeof code === 'string' ? `${why}: ${code}` : why})` + ); + } +} + +/** The render service's health answer (contracts/render.md). */ +export async function readHealth( + render: string, + timeoutMs: number, + signal?: AbortSignal +): Promise<{ build: string; files: string }> { + const body = await get(new URL('health', base(render)), timeoutMs, signal); + let health: unknown; + try { + health = JSON.parse(body); + } catch { + throw new ReadError('/health answered no JSON'); + } + const { build, files } = (health ?? {}) as { build?: unknown; files?: unknown }; + if (typeof build !== 'string') throw new ReadError('/health answered no build'); + return { build, files: typeof files === 'string' ? files : '' }; +} + +/** The content service's release, `GET /data/database/version`. */ +export async function readContentRelease( + content: string, + timeoutMs: number, + signal?: AbortSignal +): Promise { + const body = await get(new URL('data/database/version', base(content)), timeoutMs, signal); + if (!/^\s*\d+\s*$/.test(body)) throw new ReadError('/data/database/version answered no release'); + return Number(body.trim()); +} + +/** Why a figure has no file and no line: the client's cause, or its labels' check. */ +export type FaultCause = MissingCause | 'label'; + +/** Why a run stopped at once (contracts/figures.md). */ +export type StopReason = + | 'deployment-changed' + | 'deployment-unanswered' + | 'breaker-unanswered' + | 'breaker-silent' + | 'error'; + +export interface KindCounts { + needed: number; + drawn: number; + totalMs: number; + maxMs: number; + written: number; +} + +/** The run report (contracts/figures.md): stable identifiers, kinds, times and counts only. */ +export interface FiguresReport { + /** The fingerprint, as far as it was read. */ + release: Partial; + /** Why the run was refused before drawing (exit 2), if it was. */ + refused: string | null; + /** The preflight's faults (exit 1, nothing drawn). */ + preflight: string[]; + /** + * Each kind's figures: how many the book needs, how many the render service drew (and how + * long each took, its retry included), and how many are in the folder with their line. + */ + figures: Partial>; + /** Each figure with a fault of its own, in the order met. */ + missing: { stId: string; cause: FaultCause; detail: string }[]; + /** Why the run stopped at once, if it did. */ + stopped: { reason: StopReason; detail: string } | null; + /** Whether the manifest's `complete` line was written. */ + complete: boolean; + elapsedMs: number; +} + +/** The request that draws a figure's record. */ +export function figureRequest(record: FigureRecord): FigureRequest { + if (record.kind === 'ancestor') { + return { kind: 'ancestor', stId: record.source, highlight: record.highlight }; + } + return { kind: record.kind, stId: record.source }; +} + +interface Draw { + render: string; + content: string; + out: string; + parallel: number; + timeoutMs: number; + retryPauseMs: number; + fingerprint: Fingerprint; + report: FiguresReport; +} + +/** + * Draws each figure into `out`, appending the manifest after the fingerprint, and `complete` + * only when every figure has its line. Returns whether it was written; the report says why + * not. An error of its own (a file that cannot be written) stops the run and is thrown. + */ +export async function drawFigures(figures: readonly FigureRecord[], draw: Draw): Promise { + const { out, parallel, timeoutMs, fingerprint, report } = draw; + mkdirSync(out, { recursive: true }); + const manifest = openSync(join(out, MANIFEST), 'wx'); + /** The one writer: a line written and synced before anything else runs. */ + let lines = 0; + const append = (line: ManifestLine): void => { + // writeFileSync on a descriptor writes until every byte is written, or throws (a full disk) + writeFileSync(manifest, manifestLine(line)); + fsyncSync(manifest); + if (line.type === 'figure') lines += 1; + }; + const stop = new AbortController(); + const halt = (reason: StopReason, detail: string): void => { + if (stop.signal.aborted) return; + report.stopped = { reason, detail }; + stop.abort(new Error(`${reason}: ${detail}`)); + }; + // the breaker: figures unanswered in a row, or no answer of any kind for a while; never less + // than one figure's two tries and the pause between them, and a read, so a healthy service + // asking for its pause is not taken for a silent one + const silence = Math.max(5 * timeoutMs, 3 * timeoutMs + draw.retryPauseMs); + let unanswered = 0; + const breaker = setTimeout(() => { + halt('breaker-silent', `no figure answered for ${String(silence / 1000)} s`); + }, silence); + const answered = (): void => { + unanswered = 0; + breaker.refresh(); + }; + + /** The deployment read again: whether it is the fingerprint's; the run halted if not. */ + const unchanged = async (): Promise => { + let now: { build: string; files: string; content: number }; + try { + const [health, content] = await Promise.all([ + readHealth(draw.render, timeoutMs, stop.signal), + readContentRelease(draw.content, timeoutMs, stop.signal), + ]); + now = { ...health, content }; + } catch (error) { + if (!stop.signal.aborted) halt('deployment-unanswered', describe(error)); + return false; + } + if (stop.signal.aborted) return false; + if ( + now.build !== fingerprint.build || + now.content !== fingerprint.content || + now.files !== 'local' + ) { + halt( + 'deployment-changed', + `build ${now.build}, content ${String(now.content)}, files ${now.files || 'unsaid'}` + ); + return false; + } + return true; + }; + + const one = async (record: FigureRecord): Promise => { + const { stId, kind } = record; + const started = performance.now(); + const answer = await fetchFigure(draw.render, figureRequest(record), { + timeoutMs, + retryPauseMs: draw.retryPauseMs, + retryUnanswered: true, + signal: stop.signal, + }); + if ('missing' in answer) { + report.missing.push({ stId, cause: answer.cause, detail: answer.missing }); + if (answer.cause !== 'unanswered') { + answered(); + return; + } + unanswered += 1; + if (unanswered >= 2 * parallel) { + halt('breaker-unanswered', `${String(unanswered)} figures in a row unanswered`); + return; + } + await unchanged(); + return; + } + answered(); + const times = report.figures[kind]; + if (times) { + const ms = Math.round(performance.now() - started); + times.drawn += 1; + times.totalMs += ms; + times.maxMs = Math.max(times.maxMs, ms); + } + const characters = new BookCharacters(); + try { + labelsInBook(answer.svg, stId, characters); + } catch (error) { + report.missing.push({ stId, cause: 'label', detail: `unreadable: ${describe(error)}` }); + return; + } + if (characters.faults.length > 0) { + report.missing.push({ stId, cause: 'label', detail: characters.faults.join(', ') }); + return; + } + // written beside its name, synced, and moved into place, with nothing between + const file = join(out, `${stId}.svg`); + const part = `${file}.part`; + const fd = openSync(part, 'wx'); + try { + writeFileSync(fd, answer.svg); + fsyncSync(fd); + } finally { + closeSync(fd); + } + renameSync(part, file); + // unchanged() is false once the run has stopped, so no line follows a stop + if (await unchanged()) { + append({ type: 'figure', ...record }); + if (times) times.written += 1; + } else { + rmSync(file, { force: true }); + } + }; + + const queue = [...figures]; + /** A worker's own error (a file that cannot be written), thrown once the workers end. */ + const failed: { error?: Error } = {}; + /** Read afresh: another figure may have stopped the run while this one waited. */ + const halted = (): boolean => stop.signal.aborted; + const worker = async (): Promise => { + for (let next = queue.shift(); next && !halted(); next = queue.shift()) { + try { + await one(next); + } catch (error) { + if (halted()) return; + failed.error = error instanceof Error ? error : new Error(String(error)); + halt('error', describe(error)); + } + } + }; + try { + append({ type: 'fingerprint', ...fingerprint }); + await Promise.all(Array.from({ length: Math.min(parallel, figures.length) }, worker)); + if (failed.error) throw failed.error; + if (stop.signal.aborted || report.missing.length > 0 || lines !== figures.length) { + return false; + } + append({ type: 'complete', ...fingerprint }); + return true; + } finally { + clearTimeout(breaker); + closeSync(manifest); + } +} + +/** `--out`: absent, or an empty folder. */ +function usableOut(out: string): boolean { + if (!existsSync(out)) return true; + return statSync(out).isDirectory() && readdirSync(out).length === 0; +} + +/** `pdf-export figures`: its exit code; faults to `out`, the rest to `err`. */ +export async function figuresCommand( + args: readonly string[], + env: Readonly>, + { + out, + err, + open = openGraph, + retryPauseMs = 5000, + }: { + out: (text: string) => void; + err: (text: string) => void; + /** How the graph is opened (a test gives one that rewrites its rows). */ + open?: (settings: GraphSettings, options?: GraphOptions) => Graph; + /** The pause before a figure's retry (the render service asks for 5 s). */ + retryPauseMs?: number; + } +): Promise { + let settings: FiguresSettings; + try { + settings = figuresSettings(args, env); + } catch (error) { + if (!(error instanceof SettingsError)) throw error; + err(`${error.message}\n`); + return 2; + } + // an earlier run's report is never overwritten: refused, and no report written + if (settings.report !== null && existsSync(settings.report)) { + err(`--report must not exist: ${settings.report}\n`); + return 2; + } + const report: FiguresReport = { + release: {}, + refused: null, + preflight: [], + figures: {}, + missing: [], + stopped: null, + complete: false, + elapsedMs: 0, + }; + const started = performance.now(); + const refuse = (why: string): number => { + report.refused = why; + err(`${why}\n`); + return 2; + }; + try { + return await steps(settings, report, { out, err, open, retryPauseMs, refuse }); + } catch (error) { + err(`The figures' run stopped: ${describe(error)}\n`); + report.stopped ??= { reason: 'error', detail: describe(error) }; + return 2; + } finally { + report.elapsedMs = Math.round(performance.now() - started); + if (settings.report !== null) writeReport(settings.report, report, err); + } +} + +/** + * The report, written beside its name and moved into place, its folder made if need be. A + * report that cannot be written is said on `err`, and the run keeps the exit it earned. + */ +function writeReport(path: string, report: FiguresReport, err: (text: string) => void): void { + const part = `${path}.part`; + try { + mkdirSync(dirname(path), { recursive: true }); + writeFileSync(part, `${JSON.stringify(report, null, 2)}\n`); + renameSync(part, path); + } catch (error) { + err(`The report could not be written: ${describe(error)}\n`); + } +} + +async function steps( + settings: FiguresSettings, + report: FiguresReport, + { + out, + err, + open, + retryPauseMs, + refuse, + }: { + out: (text: string) => void; + err: (text: string) => void; + open: (settings: GraphSettings, options?: GraphOptions) => Graph; + retryPauseMs: number; + refuse: (why: string) => number; + } +): Promise { + if (!usableOut(settings.out)) { + return refuse(`--out must be absent or an empty folder: ${settings.out}`); + } + + // the start's read, its retrying bounded, so a graph that cannot be reached is said soon + // (opened inside the try: the driver throws at once on an address it cannot parse) + let start: Graph | undefined; + let release: ReleaseInfo; + try { + start = open(settings.graph, { connectionTimeoutMs: 5000, retryTimeMs: 0 }); + release = await releaseInfo(start); + } catch (error) { + return refuse(`The graph cannot be read: ${describe(error)}`); + } finally { + await start?.close(); + } + report.release.release = release.release; + if (release.checksum === null) return refuse('The graph has no DBInfo checksum'); + report.release.checksum = release.checksum; + + // the deployment probed before the preflight's minutes, so a service down is said at once + const probe = await readDeployment(settings, release.release, report, refuse); + if (typeof probe === 'number') return probe; + + // the figures by the Figure rules, and the preflight's checks, before any figure is asked for + const summary = readSummary(settings.files); + if (!summary) report.preflight.push(`missing-file ${SUMMARY}`); + let figures: FigureRecord[] = []; + if (summary) { + report.release.summary = summary.hash; + const graph = open(settings.graph); + try { + const docs = await buildBook(graph, { + species: settings.species, + illustrated: summary.illustrated, + }); + figures = [...expectedFigures(docs).values()]; + report.preflight.push(...checkFiles(settings.files, neededFiles(figures))); + report.preflight.push( + ...(await checkCharacters(graph, docs, { files: settings.files, expected: figures })) + ); + } catch (error) { + // the faults found before an error are printed, as the preflight's command prints them + for (const fault of report.preflight) out(`fault ${fault}\n`); + throw error; + } finally { + await graph.close(); + } + } + if (report.preflight.length > 0) { + for (const fault of report.preflight) out(`fault ${fault}\n`); + return 1; + } + for (const { kind } of figures) { + const entry = (report.figures[kind] ??= { + needed: 0, + drawn: 0, + totalMs: 0, + maxMs: 0, + written: 0, + }); + entry.needed += 1; + } + + // the deployment read again just before drawing: the fingerprint's values + const deployment = await readDeployment(settings, release.release, report, refuse); + if (typeof deployment === 'number') return deployment; + const { build, content } = deployment; + + const fingerprint: Fingerprint = { + release: release.release, + checksum: release.checksum, + summary: summary?.hash ?? '', + content, + build, + }; + let complete: boolean; + try { + complete = await drawFigures(figures, { + render: settings.render, + content: settings.content, + out: settings.out, + parallel: settings.parallel, + timeoutMs: settings.timeoutMs, + retryPauseMs, + fingerprint, + report, + }); + } finally { + // the figures' own faults so far, however the drawing ended (its own error included) + for (const { stId, cause } of report.missing) out(`fault missing-figure ${stId} ${cause}\n`); + } + report.complete = complete; + if (report.stopped) err(`stopped: ${report.stopped.reason}: ${report.stopped.detail}\n`); + const written = Object.values(report.figures).reduce((sum, kind) => sum + kind.written, 0); + err(`${String(written)} of ${String(figures.length)} figures in ${settings.out}\n`); + return complete ? 0 : 1; +} + +/** + * The render service's health and the content service's release, each refused as the start's + * check refuses them (its exit, 2), or the build and the release, recorded in the report. + */ +async function readDeployment( + settings: FiguresSettings, + release: number, + report: FiguresReport, + refuse: (why: string) => number +): Promise<{ build: string; content: number } | number> { + let health: { build: string; files: string }; + try { + health = await readHealth(settings.render, settings.timeoutMs); + } catch (error) { + return refuse(`The render service cannot be read: ${describe(error)}`); + } + if (health.build === 'unknown') return refuse("The render service's build is unknown"); + if (health.files !== 'local') return refuse('The render service does not read local files'); + report.release.build = health.build; + let content: number; + try { + content = await readContentRelease(settings.content, settings.timeoutMs); + } catch (error) { + return refuse(`The content service cannot be read: ${describe(error)}`); + } + report.release.content = content; + if (content !== release) { + return refuse( + `The content service answers release ${String(content)}, the graph is release ${String(release)}` + ); + } + return { build: health.build, content }; +} + +const describe = (error: unknown): string => + error instanceof Error ? error.message : String(error); diff --git a/test/cli.test.ts b/test/cli.test.ts index 9ed02a6..c81d041 100644 --- a/test/cli.test.ts +++ b/test/cli.test.ts @@ -1,4 +1,6 @@ import assert from 'node:assert/strict'; +import { spawnSync } from 'node:child_process'; +import { fileURLToPath } from 'node:url'; import { test } from 'node:test'; import { run, USAGE } from '../src/cli.ts'; @@ -13,3 +15,14 @@ test('an unknown command is refused with exit 2, naming it', () => { assert.equal(code, 2); assert.match(out, /^Unknown command: bake\n/); }); + +test('figures is a command: its bad arguments are exit 2, saying what is needed', () => { + const result = spawnSync( + process.execPath, + [fileURLToPath(new URL('../src/cli.ts', import.meta.url)), 'figures'], + { encoding: 'utf8', env: { PATH: process.env['PATH'] ?? '', NEO4J_URI: 'bolt://127.0.0.1:1' } } + ); + assert.equal(result.status, 2); + assert.match(result.stderr, /--files is needed/); + assert.match(USAGE, /^ {2}figures {6}/m); +}); diff --git a/test/figures/client.test.ts b/test/figures/client.test.ts index db71c98..3fd1a27 100644 --- a/test/figures/client.test.ts +++ b/test/figures/client.test.ts @@ -2,7 +2,7 @@ import assert from 'node:assert/strict'; import { after, afterEach, before, describe, test } from 'node:test'; import { fetchFigure, type FigureRequest } from '../../src/figures/client.ts'; -import { startRenderStub, type RenderStub } from '../stubs/render.ts'; +import { startRenderStub, type Refusal, type RenderStub } from '../stubs/render.ts'; const ALPHA = 'R-HSA-90000200'; const REACTION = 'R-HSA-90000211'; @@ -81,6 +81,51 @@ describe('the figures client', () => { assert.ok('missing' in json); assert.match(json.missing, /not an SVG/); }); + + test("names each missing figure's cause (contracts/figures.md)", async () => { + const cause = async (how: Refusal | null, stId = ALPHA): Promise => { + stub.reset(); + if (how !== null) stub.refuse(stId, how); + const answer = await ask({ kind: 'diagram', stId }); + return 'missing' in answer ? answer.cause : 'drawn'; + }; + assert.equal(await cause(404), 'refused'); + assert.equal(await cause(null, 'R-HSA-90000999'), 'refused'); + assert.equal(await cause(500), 'failed'); + assert.equal(await cause(503), 'unanswered'); + assert.equal(await cause(504), 'unanswered'); + assert.equal(await cause('timeout'), 'unanswered'); + assert.equal(await cause('stall-body'), 'unanswered'); + assert.equal(await cause('not-svg'), 'invalid'); + assert.equal(await cause(null), 'drawn'); + }); + + test('serve keeps one try of a 504 or a timeout; figures retries each once', async () => { + const tries = async (how: Refusal, retryUnanswered: boolean, times = -1) => { + stub.reset(); + stub.refuse(ALPHA, how, times); + const answer = await fetchFigure( + stub.url, + { kind: 'diagram', stId: ALPHA }, + { timeoutMs: 300, retryPauseMs: 10, retryUnanswered } + ); + return { drawn: 'svg' in answer, requests: stub.requests.length }; + }; + for (const how of [504, 'timeout', 'stall-body'] as const) { + assert.deepEqual(await tries(how, false), { drawn: false, requests: 1 }, String(how)); + assert.deepEqual(await tries(how, true), { drawn: false, requests: 2 }, String(how)); + assert.deepEqual(await tries(how, true, 1), { drawn: true, requests: 2 }, String(how)); + } + // a body cut off by its connection closing is gone, not slow: not retried + assert.deepEqual(await tries('cut-body', true), { drawn: false, requests: 1 }); + stub.reset(); + stub.refuse(ALPHA, 'cut-body'); + const cut = await ask({ kind: 'diagram', stId: ALPHA }); + assert.deepEqual('missing' in cut && cut.cause, 'unanswered'); + // a 500 or a 404 is the figure's own: never retried + assert.deepEqual(await tries(500, true), { drawn: false, requests: 1 }); + assert.deepEqual(await tries(404, true), { drawn: false, requests: 1 }); + }); }); test('a render service that is not there is a missing figure that says why', async () => { @@ -90,4 +135,12 @@ test('a render service that is not there is a missing figure that says why', asy const answer = await fetchFigure(url, { kind: 'diagram', stId: ALPHA }, { timeoutMs: 500 }); assert.ok('missing' in answer); assert.match(answer.missing, /ECONNREFUSED/); + assert.equal(answer.cause, 'unanswered'); + // no connection is not retried, even by figures: the service is gone, not slow + const again = await fetchFigure( + url, + { kind: 'diagram', stId: ALPHA }, + { timeoutMs: 500, retryPauseMs: 2000, retryUnanswered: true } + ); + assert.ok('missing' in again); }); diff --git a/test/figures/step.test.ts b/test/figures/step.test.ts new file mode 100644 index 0000000..c567c83 --- /dev/null +++ b/test/figures/step.test.ts @@ -0,0 +1,680 @@ +// `pdf-export figures` (spec 001 T046) on the fixture graph, against the render and content +// stubs: the folder it writes passes the book's checks, and each refusal, fault and stop is +// what contracts/figures.md says. The shared fixture is never changed: a missing checksum or +// a planted character comes from a graph that rewrites its rows. +import assert from 'node:assert/strict'; +import { + chmodSync, + cpSync, + existsSync, + mkdtempSync, + readdirSync, + readFileSync, + rmSync, + unlinkSync, + writeFileSync, +} from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { after, afterEach, before, describe, test } from 'node:test'; + +import { + graphSettings, + openGraph, + type Graph, + type GraphOptions, + type GraphSettings, +} from '../../src/graph/connection.ts'; +import { releaseInfo, type ReleaseInfo } from '../../src/graph/queries.ts'; +import { buildBook } from '../../src/model/build.ts'; +import { readSummary } from '../../src/preflight/files.ts'; +import { + checkFolder, + expectedFigures, + MANIFEST, + type FigureRecord, +} from '../../src/preflight/folder.ts'; +import { figuresCommand, figuresSettings, type FiguresReport } from '../../src/figures/step.ts'; +import { startContentStub, type ContentStub } from '../stubs/content.ts'; +import { startRenderStub, type RenderStub } from '../stubs/render.ts'; + +const skip = process.env['NEO4J_URI'] ? false : 'no fixture graph: set NEO4J_URI'; +const FILES = fileURLToPath(new URL('../../ci/fixture/files/', import.meta.url)); +const ALPHA = 'R-HSA-90000200'; +const ALPHA_SUB = 'R-HSA-90000210'; +const REACTION = 'R-HSA-90000211'; +const BETA = 'R-HSA-90000100'; +const DELTA = 'R-HSA-90000215'; + +/** A graph whose rows are rewritten: a checksum taken away, a character planted. */ +function rewriting( + graph: Graph, + rewrite: (row: Record, query: string) => Record +): Graph { + return { + async read(query: string, params?: Record): Promise { + const rows = await graph.read>(query, params); + return rows.map((row) => rewrite(row, query)) as T[]; + }, + close: () => graph.close(), + }; +} + +/** Every string in a value with `from` replaced by `to`. */ +function replaced(value: unknown, from: string, to: string): unknown { + if (typeof value === 'string') return value.split(from).join(to); + if (Array.isArray(value)) return value.map((item) => replaced(item, from, to)); + if (value !== null && typeof value === 'object') { + return Object.fromEntries( + Object.entries(value).map(([key, item]) => [key, replaced(item, from, to)]) + ); + } + return value; +} + +interface Run { + code: number; + out: string; + err: string; + dir: string; + report: FiguresReport; + /** The manifest's lines, parsed. */ + lines: Record[]; + /** The files in `--out`, sorted. */ + files: string[]; +} + +describe('pdf-export figures', { skip }, () => { + let render: RenderStub; + let content: ContentStub; + let release: ReleaseInfo; + let summary: string; + let expected: Map; + const dirs: string[] = []; + const scratch = (): string => { + const dir = mkdtempSync(join(tmpdir(), 'pdfx-figures-')); + dirs.push(dir); + return dir; + }; + + before(async () => { + render = await startRenderStub(); + content = await startContentStub(); + const graph = openGraph(graphSettings(process.env)); + try { + release = await releaseInfo(graph); + const read = readSummary(FILES); + if (!read) throw new Error('the fixture has no EHLD summary'); + summary = read.hash; + expected = expectedFigures( + await buildBook(graph, { species: 'Homo sapiens', illustrated: read.illustrated }) + ); + } finally { + await graph.close(); + } + }); + afterEach(() => { + render.reset(); + content.reset(); + }); + after(async () => { + await render.close(); + await content.close(); + for (const dir of dirs) rmSync(dir, { recursive: true, force: true }); + }); + + /** A run with the stubs, a new `--out` and a report beside it; `flags` added or replacing. */ + const figures = async ( + flags: Record = {}, + open?: (settings: GraphSettings, options?: GraphOptions) => Graph, + retryPauseMs = 10 + ): Promise => { + const work = scratch(); + const settings: Record = { + '--files': FILES, + '--render': render.url, + '--content': content.url, + '--out': join(work, 'figures'), + '--parallel': '2', + '--timeout': '0.5', + '--report': join(work, 'report.json'), + ...flags, + }; + let out = ''; + let err = ''; + const code = await figuresCommand(Object.entries(settings).flat(), process.env, { + out: (text) => (out += text), + err: (text) => (err += text), + retryPauseMs, + ...(open ? { open } : {}), + }); + const dir = settings['--out'] ?? ''; + const manifest = join(dir, MANIFEST); + return { + code, + out, + err, + dir, + report: existsSync(settings['--report'] ?? '') + ? (JSON.parse(readFileSync(settings['--report'] ?? '', 'utf8')) as FiguresReport) + : ({} as FiguresReport), + lines: existsSync(manifest) + ? readFileSync(manifest, 'utf8') + .split('\n') + .filter(Boolean) + .map((line) => JSON.parse(line) as Record) + : [], + files: existsSync(dir) ? readdirSync(dir).sort() : [], + }; + }; + + const figureLines = (run: Run): string[] => + run.lines.filter((line) => line['type'] === 'figure').map((line) => String(line['stId'])); + const hasComplete = (run: Run): boolean => run.lines.some((l) => l['type'] === 'complete'); + const renders = (): string[] => render.requests.filter((path) => path.startsWith('/render/')); + + test("draws one file per figure the fixture's book needs, and a manifest the book accepts", async () => { + const run = await figures(); + assert.equal(run.code, 0, run.err); + assert.deepEqual( + run.files, + [...[...expected.keys()].map((stId) => `${stId}.svg`), MANIFEST].sort() + ); + // every kind, each drawn by its source, an ancestor's with the event's selection + const counts = Object.fromEntries( + Object.entries(run.report.figures).map(([kind, c]) => [kind, [c.needed, c.written]]) + ); + assert.deepEqual(counts, { diagram: [5, 5], reaction: [8, 8], ancestor: [3, 3], ehld: [2, 2] }); + assert.ok(renders().includes(`/render/${ALPHA}.svg?highlight=${REACTION}%2CR-HSA-90000212`)); + assert.match(readFileSync(join(run.dir, `${ALPHA_SUB}.svg`), 'utf8'), /data-highlight=/); + assert.equal(run.lines[0]?.['type'], 'fingerprint'); + assert.deepEqual(run.lines[0], { + type: 'fingerprint', + release: release.release, + checksum: release.checksum, + summary, + content: 98, + build: '000000000000', + }); + assert.ok(hasComplete(run)); + assert.deepEqual(checkFolder(run.dir, { release, summary, expected }), []); + assert.equal(run.report.complete, true); + assert.equal(run.report.stopped, null); + assert.deepEqual(run.report.missing, []); + }); + + test('refuses an --out that is not empty, before any request', async () => { + const dir = scratch(); + writeFileSync(join(dir, 'old.svg'), ''); + const run = await figures({ '--out': dir }); + assert.equal(run.code, 2); + assert.deepEqual(run.files, ['old.svg']); + assert.deepEqual(render.requests, []); + assert.match(run.report.refused ?? '', /--out/); + }); + + test('refuses a render build of unknown, files that are not local, or a graph with no checksum', async () => { + render.health.build = 'unknown'; + const unknown = await figures(); + assert.equal(unknown.code, 2); + assert.match(unknown.err, /build is unknown/); + render.health.build = '000000000000'; + render.health.files = 'download'; + const remote = await figures(); + assert.equal(remote.code, 2); + assert.match(remote.err, /local files/); + render.reset(); + const noChecksum = await figures({}, (settings, options) => + rewriting(openGraph(settings, options), (row, query) => + query.includes('DBInfo') ? { ...row, checksum: null } : row + ) + ); + assert.equal(noChecksum.code, 2); + assert.match(noChecksum.err, /no DBInfo checksum/); + for (const run of [unknown, remote, noChecksum]) { + assert.deepEqual(run.files, []); + assert.ok(!existsSync(run.dir)); + } + assert.deepEqual(renders(), []); + }); + + test("a build changed between a figure's render and the read after it stops the run", async () => { + render.onRender((stId) => { + if (stId === REACTION) render.health.build = '111111111111'; + }); + const run = await figures({ '--parallel': '1' }); + assert.equal(run.code, 1); + assert.ok(!run.files.includes(`${REACTION}.svg`)); + assert.ok(!figureLines(run).includes(REACTION)); + assert.ok(!hasComplete(run)); + // stopped at once: nothing drawn after it + assert.equal(renders().at(-1), `/render/${REACTION}.svg?layout=reaction`); + assert.ok(figureLines(run).length < expected.size - 1); + // the report is written for a stop too + assert.equal(run.report.stopped?.reason, 'deployment-changed'); + assert.equal(run.report.complete, false); + }); + + test('a deployment changed is read after an unanswered figure, and stops the run there', async () => { + render.refuse(REACTION, 504); + render.onRender((stId) => { + if (stId === REACTION) render.health.build = '111111111111'; + }); + const run = await figures({ '--parallel': '1' }); + assert.equal(run.code, 1); + assert.equal(run.report.stopped?.reason, 'deployment-changed'); + // the reaction tried twice, and nothing asked after it + assert.deepEqual(renders().slice(-2), [ + `/render/${REACTION}.svg?layout=reaction`, + `/render/${REACTION}.svg?layout=reaction`, + ]); + }); + + test('a stop ends the figures in flight at once', async () => { + // two at once: the first figure held unanswered, the second changing the build + const [first, second] = [...expected.keys()]; + render.refuse(first ?? '', 'timeout'); + render.onRender((stId, refused) => { + if (stId === second && !refused) render.health.build = '111111111111'; + }); + const started = Date.now(); + const run = await figures({ '--timeout': '5' }); + assert.equal(run.code, 1); + assert.equal(run.report.stopped?.reason, 'deployment-changed'); + // not the held figure's two tries of 5 s + assert.ok(Date.now() - started < 2000, String(Date.now() - started)); + assert.ok(!run.files.includes(`${first ?? ''}.svg`)); + }); + + test('files no longer local, read after a figure, stop the run likewise', async () => { + render.onRender((stId) => { + if (stId === REACTION) render.health.files = 'download'; + }); + const run = await figures({ '--parallel': '1' }); + assert.equal(run.code, 1); + assert.ok(!run.files.includes(`${REACTION}.svg`)); + assert.ok(!figureLines(run).includes(REACTION)); + assert.equal(run.report.stopped?.reason, 'deployment-changed'); + }); + + test('a content release changed likewise stops the run', async () => { + render.onRender((stId) => { + if (stId === REACTION) content.release = 99; + }); + const run = await figures({ '--parallel': '1' }); + assert.equal(run.code, 1); + assert.ok(!run.files.includes(`${REACTION}.svg`)); + assert.ok(!figureLines(run).includes(REACTION)); + assert.ok(!hasComplete(run)); + assert.equal(run.report.stopped?.reason, 'deployment-changed'); + }); + + test("a 404, a 500 and a label fault are each a figure's own: the rest are drawn", async () => { + render.refuse(DELTA, 404); + render.refuse(REACTION, 500); + render.label(BETA, 'node \u2030'); + const run = await figures(); + assert.equal(run.code, 1); + const faults = new Map(run.report.missing.map(({ stId, cause }) => [stId, cause])); + assert.deepEqual( + faults, + new Map([ + [DELTA, 'refused'], + [REACTION, 'failed'], + [BETA, 'label'], + ]) + ); + assert.match( + run.report.missing.find(({ stId }) => stId === BETA)?.detail ?? '', + /uncovered-character U\+2030 R-HSA-90000100/ + ); + // its health unchanged, a 500 is no stop + assert.equal(run.report.stopped, null); + for (const stId of faults.keys()) { + assert.match(run.out, new RegExp(`^fault missing-figure ${stId} `, 'm')); + assert.ok(!run.files.includes(`${stId}.svg`), stId); + assert.ok(!figureLines(run).includes(stId), stId); + } + const drawn = [...expected.keys()].filter((stId) => !faults.has(stId)); + assert.deepEqual(figureLines(run).sort(), drawn.sort()); + for (const stId of drawn) assert.ok(run.files.includes(`${stId}.svg`), stId); + assert.ok(!hasComplete(run)); + // nothing left part written + assert.deepEqual( + run.files.filter((name) => name.endsWith('.part')), + [] + ); + }); + + test('a figure unanswered after its retry, health unchanged, is named and leaves no complete', async () => { + render.refuse(REACTION, 504); + const run = await figures(); + assert.equal(run.code, 1); + assert.equal(run.report.stopped, null); + assert.deepEqual( + run.report.missing.map(({ stId, cause }) => [stId, cause]), + [[REACTION, 'unanswered']] + ); + assert.match(run.out, new RegExp(`^fault missing-figure ${REACTION} unanswered$`, 'm')); + assert.ok(!hasComplete(run)); + assert.ok(!run.files.includes(`${REACTION}.svg`)); + const rest = [...expected.keys()].filter((stId) => stId !== REACTION); + assert.deepEqual(figureLines(run).sort(), rest.sort()); + }); + + test('a content service not answering the read after a figure stops the run', async () => { + render.onRender((stId) => { + if (stId === REACTION) content.silence(); + }); + const run = await figures({ '--parallel': '1' }); + assert.equal(run.code, 1); + assert.ok(!run.files.includes(`${REACTION}.svg`)); + assert.ok(!hasComplete(run)); + assert.equal(run.report.stopped?.reason, 'deployment-unanswered'); + }); + + test('a render service gone silent mid-run stops it at once, saying why', async () => { + let drawn = 0; + render.onRender(() => { + if (++drawn === 3) render.silence(); + }); + const started = Date.now(); + const run = await figures(); + assert.equal(run.code, 1); + assert.ok(!hasComplete(run)); + assert.equal(run.report.stopped?.reason, 'deployment-unanswered'); + // without waiting through the rest: 18 figures at two tries of 0.5 s each would be 9 s + assert.ok(Date.now() - started < 3000, String(Date.now() - started)); + assert.ok(renders().length < 8, String(renders().length)); + assert.match(run.err, /stopped: deployment-unanswered/); + }); + + test('a render service answering its health but 504 to every figure trips the breaker', async () => { + render.refuseAll(504); + // one at a time: the breaker is two figures unanswered in a row, each tried twice + const run = await figures({ '--parallel': '1' }); + assert.equal(run.code, 1); + assert.equal(run.report.stopped?.reason, 'breaker-unanswered'); + assert.ok(!hasComplete(run)); + assert.equal(run.report.missing.length, 2); + assert.equal(renders().length, 4); + assert.ok(run.report.missing.every(({ cause }) => cause === 'unanswered')); + }); + + test('figures unanswered with answers between them do not trip the breaker', async () => { + // one at a time, every third or so unanswered (by sources no other figure draws), never + // two in a row: each answer between them starts the count again + const sources = ['R-HSA-90000220', 'R-HSA-90000251', REACTION, 'R-HSA-90000216']; + for (const source of sources) render.refuse(source, 504); + const run = await figures({ '--parallel': '1' }); + assert.equal(run.code, 1); + assert.equal(run.report.stopped, null); + assert.deepEqual( + run.report.missing.map(({ stId }) => stId), + sources + ); + assert.equal(figureLines(run).length, expected.size - sources.length); + }); + + test('ten figures in a row failing (500) do not trip the breaker', async () => { + // the first ten figures, one at a time: by their sources, as two share one (a request + // left over from an earlier test cannot use up a refusal kept for each source) + const first = [...expected.values()].slice(0, 10); + for (const { source } of first) render.refuse(source, 500); + const run = await figures({ '--parallel': '1' }); + assert.equal(run.code, 1); + assert.equal(run.report.stopped, null); + assert.deepEqual( + run.report.missing.map(({ stId }) => stId), + first.map(({ stId }) => stId) + ); + assert.ok(run.report.missing.every(({ cause }) => cause === 'failed')); + assert.equal(figureLines(run).length, expected.size - 10); + }); + + test('nor do 500s answered slowly for longer than five timeouts in all', async () => { + render.refuseAll(500); + render.slow(200); + const started = Date.now(); + // 18 figures one at a time, 0.2 s each: 3.6 s, past five timeouts of 0.5 s + const run = await figures({ '--parallel': '1' }); + assert.ok(Date.now() - started > 2500); + assert.equal(run.code, 1); + assert.equal(run.report.stopped, null); + assert.equal(run.report.missing.length, expected.size); + }); + + test('no figure answered for five timeouts, its health answering, trips the breaker', async () => { + // every figure slower than its timeout, ten at once (the count's 20 is never reached), a + // retry's pause of 0.6 s: each figure takes 1.4 s, the eighteen 2.8 s, and the breaker's + // silence is 2 s (five timeouts of 0.4 s, above three and the pause, 1.8 s) + render.slow(5000); + const run = await figures({ '--parallel': '10', '--timeout': '0.4' }, undefined, 600); + const drawing = render.times.at(render.requests.findIndex((p) => p.startsWith('/render/'))); + const ended = Date.now(); + assert.equal(run.code, 1); + assert.equal(run.report.stopped?.reason, 'breaker-silent'); + assert.match(run.report.stopped.detail, /for 2 s$/); + // timed from the drawing's start, after the preflight + assert.ok(drawing !== undefined && ended - drawing < 2300, String(ended - (drawing ?? 0))); + assert.deepEqual(figureLines(run), []); + }); + + test("a healthy service's pause after a 503 does not trip the silent breaker", async () => { + // a timeout of 1 s, the service's own pause of 5 s: five timeouts would end the run + // during the first figure's pause + const [first] = [...expected.values()]; + render.refuse(first?.source ?? '', 503, 1); + const run = await figures({ '--parallel': '1', '--timeout': '1' }, undefined, 5000); + assert.equal(run.code, 0, JSON.stringify(run.report.stopped)); + assert.ok(hasComplete(run)); + }); + + test('an unreachable content or render service is exit 2', async () => { + const gone = await startContentStub(); + await gone.close(); + const noContent = await figures({ '--content': gone.url }); + assert.equal(noContent.code, 2); + assert.match(noContent.err, /content service cannot be read/); + const goneRender = await startRenderStub(); + await goneRender.close(); + const noRender = await figures({ '--render': goneRender.url }); + assert.equal(noRender.code, 2); + assert.match(noRender.err, /render service cannot be read/); + assert.deepEqual(renders(), []); + // probed before the preflight: said at once, though the preflight would find a fault + const files = scratch(); + cpSync(FILES, files, { recursive: true }); + unlinkSync(join(files, 'diagram', `${ALPHA}.graph.json`)); + const first = await figures({ '--render': goneRender.url, '--files': files }); + assert.equal(first.code, 2); + assert.deepEqual(first.report.preflight, []); + }); + + test("the fingerprint is the deployment's after the preflight, not the probe's", async () => { + // a build changed while the preflight runs: the folder is the new build's, and whole + const run = await figures({}, (settings, options) => + rewriting(openGraph(settings, options), (row, query) => { + if (query.includes('WHERE r:ReactionLikeEvent')) render.health.build = '222222222222'; + return row; + }) + ); + assert.equal(run.code, 0, run.err); + assert.equal(run.lines[0]?.['build'], '222222222222'); + assert.ok(hasComplete(run)); + }); + + test('a content service answering another release is refused before any drawing', async () => { + content.release = 97; + const run = await figures(); + assert.equal(run.code, 2); + assert.match(run.err, /release 97/); + assert.deepEqual(renders(), []); + assert.deepEqual(run.files, []); + }); + + test('a missing release file and an uncovered character each fail before any figure is asked for', async () => { + const files = scratch(); + cpSync(FILES, files, { recursive: true }); + unlinkSync(join(files, 'diagram', `${ALPHA}.graph.json`)); + const missing = await figures({ '--files': files }); + assert.equal(missing.code, 1); + assert.match(missing.out, new RegExp(`^fault missing-file diagram/${ALPHA}.graph.json$`, 'm')); + const uncovered = await figures({}, (settings, options) => + rewriting( + openGraph(settings, options), + (row) => replaced(row, 'A third placeholder', 'A third \u2030 placeholder') as typeof row + ) + ); + assert.equal(uncovered.code, 1); + assert.match(uncovered.out, /^fault uncovered-character U\+2030 R-HSA-90000211$/m); + // only the deployment's probes, made before the preflight + assert.deepEqual(render.requests, ['/health', '/health']); + assert.deepEqual(content.requests, ['/data/database/version', '/data/database/version']); + assert.deepEqual(uncovered.report.preflight, ['uncovered-character U+2030 R-HSA-90000211']); + }); + + test('a graph failing during the checks is exit 2, the faults found before it printed', async () => { + const files = scratch(); + cpSync(FILES, files, { recursive: true }); + unlinkSync(join(files, 'diagram', `${ALPHA}.graph.json`)); + const run = await figures({ '--files': files }, (settings, options) => { + const graph = openGraph(settings, options); + return { + read: (query: string, params?: Record): Promise => + query.includes('WHERE r:ReactionLikeEvent') + ? Promise.reject(new Error('the graph failed')) + : graph.read(query, params), + close: () => graph.close(), + }; + }); + assert.equal(run.code, 2); + assert.match(run.err, /the graph failed/); + assert.match(run.out, new RegExp(`^fault missing-file diagram/${ALPHA}.graph.json$`, 'm')); + assert.deepEqual(renders(), []); + }); + + test("an existing --report is refused before anything else, an earlier run's report kept", async () => { + const work = scratch(); + const report = join(work, 'report.json'); + writeFileSync(report, '{"complete":true}\n'); + const run = await figures({ '--out': join(work, 'figures'), '--report': report }); + assert.equal(run.code, 2); + assert.match(run.err, /--report must not exist/); + assert.equal(readFileSync(report, 'utf8'), '{"complete":true}\n'); + assert.deepEqual(render.requests, []); + assert.deepEqual(content.requests, []); + }); + + test("a refusal writes its report, the report's folder made, its exit kept", async () => { + content.release = 97; + const work = scratch(); + const report = join(work, 'runC', 'report.json'); + const run = await figures({ '--out': join(work, 'runC'), '--report': report }); + assert.equal(run.code, 2); + assert.match(run.report.refused ?? '', /release 97/); + assert.deepEqual(readdirSync(join(work, 'runC')), ['report.json']); + }); + + test('a report that cannot be written is said, and the run keeps its exit', async () => { + const work = scratch(); + const notFolder = join(work, 'a-file'); + writeFileSync(notFolder, ''); + const run = await figures({ '--report': join(notFolder, 'report.json') }); + assert.equal(run.code, 0, run.err); + assert.match(run.err, /The report could not be written/); + assert.ok(hasComplete(run)); + }); + + test("the run's own error is exit 2, the figures' faults so far printed", async () => { + const work = scratch(); + const dir = join(work, 'figures'); + const [first] = [...expected.values()]; + render.refuse(first?.source ?? '', 404); + let drawn = 0; + render.onRender((_stId, refused) => { + // the folder made read-only: the next file cannot be written + if (!refused && ++drawn === 2) chmodSync(dir, 0o500); + }); + try { + const run = await figures({ '--out': dir, '--parallel': '1' }); + assert.equal(run.code, 2); + assert.match(run.err, /The figures' run stopped: EACCES/); + assert.match(run.out, new RegExp(`^fault missing-figure ${first?.stId ?? ''} refused$`, 'm')); + assert.equal(run.report.stopped?.reason, 'error'); + assert.ok(!hasComplete(run)); + } finally { + chmodSync(dir, 0o700); + } + }); + + test('a figure timing out once, and one answering 504 once, are drawn on their retry', async () => { + render.refuse(REACTION, 'timeout', 1); + render.refuse(ALPHA, 504, 1); + const run = await figures(); + assert.equal(run.code, 0, run.err); + assert.ok(run.files.includes(`${REACTION}.svg`)); + assert.ok(run.files.includes(`${ALPHA}.svg`)); + assert.equal( + renders().filter((p) => p === `/render/${REACTION}.svg?layout=reaction`).length, + 2 + ); + assert.equal(renders().filter((p) => p === `/render/${ALPHA}.svg`).length, 2); + }); +}); + +test("figures' settings: defaults, and bad values refused", () => { + const env = { + NEO4J_URI: 'bolt://127.0.0.1:1', + RENDER_SERVICE: 'http://r', + CONTENT_SERVICE: 'http://c', + }; + const settings = figuresSettings(['--files', 'f', '--out', 'o'], env); + assert.equal(settings.parallel, 2); + assert.equal(settings.timeoutMs, 60_000); + assert.equal(settings.render, 'http://r'); + assert.equal(settings.content, 'http://c'); + assert.equal(settings.report, null); + assert.throws(() => figuresSettings(['--files', 'f'], env), /--out is needed/); + assert.throws( + () => figuresSettings(['--files', 'f', '--out', 'o', '--parallel', '0'], env), + /--parallel/ + ); + assert.throws( + () => figuresSettings(['--files', 'f', '--out', 'o', '--timeout', '0'], env), + /--timeout/ + ); + assert.throws( + () => figuresSettings(['--files', 'f', '--out', 'o'], { NEO4J_URI: 'bolt://x' }), + /--render \(or RENDER_SERVICE\)/ + ); +}); + +test('bad arguments are exit 2', async () => { + let err = ''; + const code = await figuresCommand( + ['--colour', 'blue'], + {}, + { + out: () => undefined, + err: (text) => (err += text), + } + ); + assert.equal(code, 2); + assert.match(err, /Unknown argument: --colour/); +}); + +test('a graph address the driver cannot parse is exit 2, not a throw', async () => { + let err = ''; + const code = await figuresCommand( + [ + ...['--graph', 'not a uri', '--files', 'f', '--render', 'http://r', '--content', 'http://c'], + ...['--out', join(tmpdir(), `pdfx-figures-never-${String(process.pid)}`)], + ], + {}, + { out: () => undefined, err: (text) => (err += text) } + ); + assert.equal(code, 2); + assert.match(err, /The graph cannot be read/); +}); diff --git a/test/stubs/content.test.ts b/test/stubs/content.test.ts new file mode 100644 index 0000000..c290c51 --- /dev/null +++ b/test/stubs/content.test.ts @@ -0,0 +1,28 @@ +import assert from 'node:assert/strict'; +import { after, before, test } from 'node:test'; + +import { startContentStub, type ContentStub } from './content.ts'; + +let stub: ContentStub; +before(async () => { + stub = await startContentStub(); +}); +after(async () => { + await stub.close(); +}); + +test('answers its release as ContentService does, as set, and nothing when silenced', async () => { + const version = () => + fetch(`${stub.url}/data/database/version`, { signal: AbortSignal.timeout(300) }); + const first = await version(); + assert.match(first.headers.get('content-type') ?? '', /^text\/plain/); + assert.equal(await first.text(), '98'); + stub.release = 99; + assert.equal(await (await version()).text(), '99'); + assert.equal((await fetch(`${stub.url}/data/other`)).status, 404); + stub.silence(); + await assert.rejects(version(), { name: 'TimeoutError' }); + stub.reset(); + assert.equal(await (await version()).text(), '98'); + assert.deepEqual(stub.requests, ['/data/database/version']); +}); diff --git a/test/stubs/content.ts b/test/stubs/content.ts new file mode 100644 index 0000000..6d27e5a --- /dev/null +++ b/test/stubs/content.ts @@ -0,0 +1,66 @@ +// The content stub (spec 001 T046): the one request `pdf-export figures` makes of the content +// service, `GET /data/database/version`, answered as ContentService answers it, the release's +// number as plain text. Its release is settable by a test, also between a figure's render and +// the read after it, and it can be silenced (every request held unanswered). +import { createServer, type Server, type ServerResponse } from 'node:http'; +import type { AddressInfo } from 'node:net'; + +export interface ContentStub { + /** The base URL: requests go to `${url}/data/database/version`. */ + url: string; + /** Every request's path, in order. */ + requests: string[]; + /** The release answered; the fixture's, 98, at the start. */ + release: number; + /** Hold every request from now on unanswered. */ + silence(): void; + /** The release back to 98, the silence ended, held answers dropped, the requests forgotten. */ + reset(): void; + close(): Promise; +} + +export async function startContentStub(): Promise { + const hanging = new Set(); + let silent = false; + const stub: ContentStub = { + url: '', + requests: [], + release: 98, + silence() { + silent = true; + }, + reset() { + for (const res of hanging) res.destroy(); + hanging.clear(); + silent = false; + stub.release = 98; + stub.requests.length = 0; + }, + async close() { + for (const res of hanging) res.destroy(); + server.closeAllConnections(); + await new Promise((resolve) => + server.close(() => { + resolve(); + }) + ); + }, + }; + const server: Server = createServer((req, res) => { + const path = new URL(req.url ?? '/', 'http://stub').pathname; + stub.requests.push(path); + if (silent) { + hanging.add(res); + return; + } + if (path !== '/data/database/version') { + res.writeHead(404, { 'Content-Type': 'application/json' }).end('{"code":404}'); + return; + } + res.writeHead(200, { 'Content-Type': 'text/plain;charset=UTF-8' }).end(String(stub.release)); + }); + await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)); + const { port } = server.address() as AddressInfo; + stub.url = `http://127.0.0.1:${String(port)}`; + return stub; +} diff --git a/test/stubs/render.test.ts b/test/stubs/render.test.ts index 3337aef..81ee6d6 100644 --- a/test/stubs/render.test.ts +++ b/test/stubs/render.test.ts @@ -101,6 +101,50 @@ describe('the render stub', () => { stub.reset(); assert.deepEqual(stub.requests, []); }); + + test('refuses with a 500 or a 504, for one figure or for every one', async () => { + stub.refuse(ALPHA, 500, 1); + assert.equal((await get(`/render/${ALPHA}.svg`)).status, 500); + assert.equal((await get(`/render/${ALPHA}.svg`)).status, 200); + stub.refuseAll(504, 2); + assert.equal((await get(`/render/${ALPHA}.svg`)).status, 504); + assert.equal((await get(`/render/${BETA}.svg`)).status, 504); + assert.equal((await get(`/render/${BETA}.svg`)).status, 200); + }); + + test('answers its health as set, and runs a hook as each figure is drawn', async () => { + assert.deepEqual(await (await get('/health')).json(), { + build: '000000000000', + files: 'local', + }); + const drawn: string[] = []; + stub.onRender((stId) => { + drawn.push(stId); + stub.health.build = 'abcdefabcdef'; + }); + await (await get(`/render/${REACTION}.svg?layout=reaction`)).text(); + assert.deepEqual(drawn, [REACTION]); + assert.equal( + ((await (await get('/health')).json()) as { build: string }).build, + 'abcdefabcdef' + ); + stub.reset(); + assert.equal( + ((await (await get('/health')).json()) as { build: string }).build, + '000000000000' + ); + }); + + test('silenced, it answers nothing, its health included; slowed, it answers late', async () => { + stub.slow(150); + const started = Date.now(); + assert.equal((await get(`/render/${ALPHA}.svg`)).status, 200); + assert.ok(Date.now() - started >= 140); + stub.silence(); + for (const path of ['/health', `/render/${ALPHA}.svg`]) { + await assert.rejects(get(path, AbortSignal.timeout(300)), { name: 'TimeoutError' }); + } + }); }); test( diff --git a/test/stubs/render.ts b/test/stubs/render.ts index c9b8c1c..7c91584 100644 --- a/test/stubs/render.ts +++ b/test/stubs/render.ts @@ -3,9 +3,13 @@ // answers (tools/render/service.ts): its type, a 400 for another format or a parameter // given twice, a JSON error body, and Retry-After on a 503. It answers 400 to a query // parameter the contract does not name, so a wrongly named one fails a test instead of -// passing silently, and refuses on request: 404, 503, a timeout, a body that stops part way, -// or an answer that is not SVG. A test can also plant text in a figure's first label. Its -// `/health` answers as the service's does. +// passing silently, and refuses on request: 404, 500, 503, 504, a timeout, a body that stops +// part way (held, or its connection closed), or an answer that is not SVG, for one figure or +// every one. A test can also plant text in a figure's first label, slow every answer, or +// silence the stub (every request held unanswered, `/health`'s too). Its `/health` answers as +// the service's does, its `build` and `files` settable, also from a hook run as each figure +// request is answered (T046), so a test can change the deployment between a figure's render +// and the read after it. import { readFileSync, existsSync } from 'node:fs'; import { createServer, type Server } from 'node:http'; import type { AddressInfo } from 'node:net'; @@ -23,18 +27,42 @@ const PARAMETERS: Readonly boolean>> = { * How to refuse: a status, no answer at all, an answer whose body stops part way, or an * answer that is not SVG. */ -export type Refusal = 404 | 503 | 'timeout' | 'stall-body' | 'not-svg'; +export type Refusal = 404 | 500 | 503 | 504 | 'timeout' | 'stall-body' | 'cut-body' | 'not-svg'; + +/** The service's health answer (contracts/render.md). */ +export interface Health { + build: string; + files: string; +} export interface RenderStub { /** The base URL: requests go to `${url}/render/{stId}.svg`. */ url: string; /** Every request's path and query, in order. */ requests: string[]; + /** When each request came, by `Date.now()`, beside `requests`. */ + times: number[]; /** Refuse requests for a stable id, `times` times (every time without it). */ refuse(stId: string, how: Refusal, times?: number): void; + /** Refuse requests for every figure, `times` times in all (every time without it). */ + refuseAll(how: Refusal, times?: number): void; + /** The health answer, changed in place by a test; its build a placeholder at the start. */ + health: Health; + /** + * Run `hook` with each figure request's stable id just before its answer is sent: a drawing + * or a refusal (`refused` true), a held one's as it is held. + */ + onRender(hook: ((stId: string, refused: boolean) => void) | null): void; + /** Hold every request from now on unanswered, `/health`'s too: a service gone silent. */ + silence(): void; + /** Wait `ms` before each figure's answer, a refusal's too. */ + slow(ms: number): void; /** Put `text` at the start of the first label of every figure drawn for a stable id. */ label(stId: string, text: string): void; - /** Forget the refusals, the planted labels and the recorded requests, for the next test. */ + /** + * Forget the refusals, the planted labels, the hook, the slowness, the silence, the health + * set and the recorded requests, and drop every answer held or delayed, for the next test. + */ reset(): void; close(): Promise; } @@ -66,22 +94,34 @@ function figure( export async function startRenderStub(): Promise { const requests: string[] = []; + const times: number[] = []; const refusals = new Map(); + let every: { how: Refusal; left: number } | null = null; const labels = new Map(); const hanging = new Set(); + const health: Health = { build: '000000000000', files: 'local' }; + let hook: ((stId: string, refused: boolean) => void) | null = null; + let silent = false; + let delayMs = 0; + const delayed = new Map(); const server: Server = createServer((req, res) => { const url = new URL(req.url ?? '/', 'http://stub'); requests.push(url.pathname + url.search); + times.push(Date.now()); + if (silent) { + hanging.add(res); + return; + } const error = (status: number, message: string, headers: Record = {}) => res .writeHead(status, { 'Content-Type': 'application/json; charset=utf-8', ...headers }) .end(JSON.stringify({ error: message })); if (url.pathname === '/health') { - // the service's health answer (contracts/render.md), its build a placeholder + // the service's health answer (contracts/render.md) res .writeHead(200, { 'Content-Type': 'application/json; charset=utf-8' }) - .end(JSON.stringify({ build: '000000000000', files: 'local' })); + .end(JSON.stringify(health)); return; } const match = /^\/render\/(R-[A-Z]{3}-\d+)\.([a-z]+)$/.exec(url.pathname); @@ -107,9 +147,41 @@ export async function startRenderStub(): Promise { return; } } - const refusal = refusals.get(stId); - if (refusal && refusal.left !== 0) { + if (delayMs > 0) { + const timer = setTimeout(() => { + delayed.delete(timer); + if (silent) hanging.add(res); + else answer(stId, url.searchParams, res, error); + }, delayMs); + delayed.set(timer, res); + return; + } + answer(stId, url.searchParams, res, error); + }); + + /** Every answer held or delayed, dropped, so none reaches a later test. */ + const drop = () => { + for (const [timer, res] of delayed) { + clearTimeout(timer); + res.destroy(); + } + delayed.clear(); + for (const res of hanging) res.destroy(); + hanging.clear(); + }; + + /** A figure's answer: its refusal, if it has one left, or its drawing. */ + const answer = ( + stId: string, + query: URLSearchParams, + res: import('node:http').ServerResponse, + error: (status: number, message: string, headers?: Record) => void + ) => { + const own = refusals.get(stId); + const refusal = own && own.left !== 0 ? own : every && every.left !== 0 ? every : undefined; + if (refusal) { refusal.left -= 1; + hook?.(stId, true); if (refusal.how === 'timeout') { hanging.add(res); return; @@ -120,21 +192,30 @@ export async function startRenderStub(): Promise { hanging.add(res); return; } + if (refusal.how === 'cut-body') { + // the connection closed part way through the body + res.writeHead(200, { 'Content-Type': 'image/svg+xml; charset=utf-8' }); + res.write(' res.destroy()); + return; + } if (refusal.how === 'not-svg') { res.writeHead(200, { 'Content-Type': 'application/json' }).end('{}'); return; } if (refusal.how === 503) error(503, 'Busy', { 'Retry-After': '5' }); + else if (refusal.how === 500) error(500, `The page failed on ${stId}`); + else if (refusal.how === 504) error(504, `No drawing in time for ${stId}`); else error(404, `No figure for ${stId}`); return; } - const svg = figure(stId, url.searchParams, labels.get(stId)); + const svg = figure(stId, query, labels.get(stId)); if (svg === undefined) { error(404, `No figure for ${stId}`); return; } + hook?.(stId, false); res.writeHead(200, { 'Content-Type': 'image/svg+xml; charset=utf-8' }).end(svg); - }); + }; await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)); const { port } = server.address() as AddressInfo; @@ -142,21 +223,42 @@ export async function startRenderStub(): Promise { return { url: `http://127.0.0.1:${String(port)}`, requests, + times, refuse(stId, how, times = -1) { refusals.set(stId, { how, left: times }); }, + refuseAll(how, times = -1) { + every = { how, left: times }; + }, + health, + onRender(next) { + hook = next; + }, + silence() { + silent = true; + }, + slow(ms) { + delayMs = ms; + }, label(stId, text) { labels.set(stId, text); }, reset() { + drop(); refusals.clear(); + every = null; labels.clear(); + hook = null; + silent = false; + delayMs = 0; + Object.assign(health, { build: '000000000000', files: 'local' }); requests.length = 0; + times.length = 0; }, async close() { // A timed-out request's response is held open; close it with the connections, so the // tests end even when a client never gave up on one. - for (const res of hanging) res.destroy(); + drop(); server.closeAllConnections(); await new Promise((resolve) => server.close(() => {