diff --git a/specs/001-pathway-pdfs/contracts/cli.md b/specs/001-pathway-pdfs/contracts/cli.md index 04afc77..fada505 100644 --- a/specs/001-pathway-pdfs/contracts/cli.md +++ b/specs/001-pathway-pdfs/contracts/cli.md @@ -40,6 +40,9 @@ contracts/figures.md). at `--content`; no graph checksum, no render `build`, or a render service not reading local files; or an `--out` that is not empty. A `build` or content release that changes during the run, a render or content service that stops answering, or the breaker, is exit 1 (contracts/figures.md). + Any other error (the graph failing during the checks, a file that cannot be written) is + exit 2, its error on stderr, the run stopped and its report written. The output's lines + and the report are in contracts/figures.md. ## `pdf-export book` diff --git a/specs/001-pathway-pdfs/contracts/figures.md b/specs/001-pathway-pdfs/contracts/figures.md index b2edd09..d16eb03 100644 --- a/specs/001-pathway-pdfs/contracts/figures.md +++ b/specs/001-pathway-pdfs/contracts/figures.md @@ -102,6 +102,15 @@ What `pdf-export figures` writes and `pdf-export book` reads (SC-010, research R holding no names, only stable identifiers, kinds, times and counts. Any other file is ignored, so a folder drawn for a release can be read again by a book made from the same graph. +- **The run report** (`--report`, JSON; T046), written however the run ends: `release` + (the fingerprint, as far as it was read), `refused` (why the run was refused before + drawing, exit 2), `preflight` (its faults, exit 1), `figures` (for each kind: `needed`, + `drawn` by the render service, with `totalMs` and `maxMs`, retries included, and + `written`, in the folder with its line), `missing` (each figure's own fault, by `stId`, + `cause` and `detail`), `stopped` (`reason`: `deployment-changed`, + `deployment-unanswered`, `breaker-unanswered`, `breaker-silent` or `error`, and its + `detail`), `complete` and `elapsedMs`. On standard output, one `fault ` line for + each preflight fault and `fault missing-figure ` for each figure's own. - **A figure that could not be drawn** has no file and no line: `figures` draws the rest (unless the run stops at once, above), exits 1 with no `complete` line, and the book refuses the folder. diff --git a/specs/001-pathway-pdfs/tasks.md b/specs/001-pathway-pdfs/tasks.md index 8f239d8..1bc088d 100644 --- a/specs/001-pathway-pdfs/tasks.md +++ b/specs/001-pathway-pdfs/tasks.md @@ -403,7 +403,7 @@ passes. - **From #149's review:** (1) an error during the preflight was exit 1 with no fault line, the faults already found lost (a diagram file not JSON, a graph failing after its retries, `buildBook`'s "not read from the graph"); exit 1 is to mean faults. A diagram file that cannot be read as JSON is now a fault, `bad-file ` (contracts/cli.md), its labels unchecked and the run going on; any other error ends the command with exit 2, its error on stderr, after the faults found before it are printed (`preflight` adds each check's faults to a list its caller holds). Both seen failing first: a corrupt `diagram/R-HSA-90000130.json` with an empty `--figures` gave exit 1 and nothing printed, and a graph failing at the reactions' names exit 1 with `incomplete-folder` lost. (2) `retryTimeMs: 0` still makes one retry, and it applied to every read: the start's read now has a driver of its own with it (said within seconds), and the checks' reads the driver's default; the comment and contract say "bounds the retrying". (3) The website's diagram draws a node's attachments' `label`s (a modification's lettering); `src/preflight/labels.ts` now reads them. `figureNames` does not read a reaction figure's equivalent (its entities' modified residues' PSI-MOD labels), recorded, not built: the fixture has no modification to test it on. Recorded too: a normal pathway's diagram drawn beneath a disease diagram (`forNormalDraw` false) is not read apart from its file. (4) Tests: a stale folder that also has a figure with no line and a missing file gives the one fault; a selection that is the right one's beginning is `stale-figure`; a figure line whose selection is not all strings is not whole. (5) Recorded, not changed: the fingerprint's `content` is not compared with the graph's release by `book`, defence in depth left to T046's check before drawing; "ignoring case" is `toLowerCase`, not full case folding or Unicode normalisation (final sigma, ß, NFC against NFD); the text check reads strings that are never printed (web addresses, and whatever the model holds beside what the layout writes), so a fault can be false; T047 needs the fingerprint compared and `none` told from `incomplete` (now in T047's text); a truncated `.graph.json` is not found, only its presence checked. 347 tests before (on main with T027), 350 after. Each of these mutants made one fail: the refusal as a whole removed (the stale folder's test); the selection's length not compared (the beginning's case); the selection's ids not checked as strings (the unit case); attachments not read; a file not JSON thrown again (the bad-file test); the error not caught (the exit 2 test); the faults found not printed on an error (the same); the start's read without its bound (the 15 s bound). Of the earlier mutants, 26 were run again on this code and each still fails a test; the other 12 (among them the file-name rule's, the substitutions' and the unreachable graph's) were not re-run; the two equivalents still survive. - **From #150's review:** the start's driver was opened outside the `try`, and the driver throws at once on an address it cannot parse, so `NEO4J_URI=127.0.0.1:17700` gave exit 1 with a stack trace; it is opened inside it now, exit 2 with the error on stderr (seen failing: `Unknown scheme: null`; tested only with addresses that fail to parse, never one falling back to the default port). On an error the faults are printed first, then the error (seen failing, `err` before `out`). The faults are kept check by check, as contracts/cli.md now says: those of the check an error stops are lost. A test runs the command as a child process and holds it to 15 s, so a driver left open is caught. contracts/cli.md adds `bad-file` to `book`'s preflight and to `figures`' exit 1, and says `book` exits 2 on an error during its preflight. Each of these mutants made one fail: the start's driver not closed (the child process's test); the checks' driver not closed (the same, and the test file's run then never ends); the start's driver opened outside the `try` (the address test); the error written before the faults (the exit 2 test's order). -- [ ] T046 [US1] `pdf-export figures`, `src/figures/step.ts` and `src/cli.ts` (contracts/cli.md, contracts/figures.md, SC-010). +- [x] T046 [US1] `pdf-export figures`, `src/figures/step.ts` and `src/cli.ts` (contracts/cli.md, contracts/figures.md, SC-010). - Lists the book's figures by the Figure rules (T013's `diagramChoices` with `nearestAncestor` from `src/model/build.ts`, and the EHLD summary from `--files`: the same code the book uses), and draws each through T014's client into `--out`, `{stId}.svg`, written beside its name and moved into place. - After T024: runs its release-file and pre-drawing character checks first (`src/preflight/`), and the label check after each figure is drawn. - Asks the content service named by `--content` for its release, and refuses another (exit 2); refuses a render service whose health answer does not say `files: local` (exit 2; contracts/render.md). @@ -415,6 +415,8 @@ passes. Tests, on the fixture against the stub render service: exit 0 with one file per figure the fixture's book needs, by kind, and a `complete` manifest that passes T024's folder checks, seen failing with a selection keyed by its ancestor; a non-empty `--out` is refused (exit 2); a `build` of `unknown`, and a graph with no checksum, are refused; a `build` changed between a figure's render and the read after it, and a content release changed likewise, each leave that figure with no file and no line, and the run exits 1 with no `complete`; a figure failing its label check is left with no file and no line; one figure the stub refuses with a 404 and one with a 500 (its health unchanged), and one failing its label check: the run draws all the others, then exits 1 naming all three, with no `complete`; a content service not answering the read after a figure stops the run, exit 1; the render stub stopping answering mid-run stops the run at the next failed figure, exit 1, without waiting through the rest, and its report names why; the stub answering `/health` but returning 504 to every render trips the breaker, exit 1, its report naming it; ten figures in a row answering 500 (cause `failed`) do not trip it, the run drawing the rest; nor do 500s answered slowly for longer than `5 × --timeout` in all, each within `--timeout`; a deployment change's stop writes the report too; an unreachable content service is exit 2; a content service answering another release is refused before any drawing; a planted missing release file and a planted uncovered character each fail with exit 1 before any request; a figure that times out once and then answers, and one that answers 504 once and then draws, are drawn (the retry); a refused figure leaves no partial file; exit 2 on the render service unreachable. + - **Done (#154):** `src/figures/step.ts` and `pdf-export figures` (`src/cli.ts`). **Order:** the settings and `--out` (absent or empty, else exit 2); the graph's release and checksum (none, or an unreachable graph, exit 2); the figures from `buildBook` and `expectedFigures` (T048's and T024's, the code `book` uses), then T024's release-file and pre-drawing character checks, before any request is made (a fault is exit 1, `fault ` lines); then the render service's health (`build` not `unknown`, `files: local`) and the content release, which must be the graph's (exit 2). The file-name and folder checks are not run: T046 names only the files' and characters'. **Drawing:** `--parallel` workers over the figures; each answer checked by `labelsInBook` with a `BookCharacters` of its own (its fault the cause `label`, no file); written to `{stId}.svg.part`, synced, renamed, with nothing awaited between, so a partial file is left only by a crash (the folder itself is not synced); then health and content read again within `--timeout`, and the line appended only if both, and `files: local`, are the fingerprint's and the run has not stopped, else the file is removed. The manifest is one descriptor opened `wx`, each line written and synced synchronously. After an `unanswered` figure the same read is made. **The breaker:** `2 × --parallel` `unanswered` in a row, or no figure answered for `5 × --timeout`; an answer is a figure drawn (label faults included), `failed`, `refused` or `invalid`; the health reads do not count. A stop aborts the work in flight (the client's signal). An error of the step's own (a file that cannot be written) stops the run, `error`, exit 2. **The client:** `{ missing, cause }`; `retryUnanswered` (only `figures` sets it) retries a 504, or a timeout of the request or its body, once, after the 503's pause, one retry per figure in all; a refused connection and a body cut by a closed connection are not retried (gone, not slow); a non-OK status that is not 4xx or 5xx is `invalid`. `serve` is unchanged. **The report** (contracts/figures.md now gives its fields and the stop reasons; contracts/cli.md the other errors): written in a `finally`, refusals and preflight faults included; times are each figure's from its first request to its answer, retries included. `--timeout` takes fractional seconds. No signal handling: an interrupted run leaves a folder the book refuses. **Stubs:** the render stub gains 500 and 504 refusals, `refuseAll`, a settable `health`, an `onRender` hook (run for refusals too), `silence` and `slow`, and its `reset` drops held and delayed answers (a stopped run's leftover request had used up a counted refusal in a later test); `test/stubs/content.ts` answers `GET /data/database/version` as plain text, its release settable, and can be silenced. **Tests:** `test/figures/step.test.ts` (22: every case above, plus the read after an unanswered figure, a stop ending work in flight, the silent breaker, `files: local`, the settings, an address the driver cannot parse, and a graph failing during the checks, its faults so far printed first, as T024's final `preflight` does), `test/figures/client.test.ts` (2 more: causes; retries by caller), `test/stubs/render.test.ts` (3 more), `test/stubs/content.test.ts` (1), `test/cli.test.ts` (1, the command run as a process). 362 tests before (on main with T024 and T028), 391 after. Each of these mutants made one fail: a figure's file, or its line, keyed by its source (the complete manifest's test, through T024's `checkFolder`); a non-empty `--out` allowed; `unknown` accepted; `files` not checked at the start; no checksum accepted; the content release not compared at the start; the build, or the content release, not read again; the file left on a change; the line appended before the read; an unanswered read not a stop; no read after an unanswered figure; the label check skipped; a file written before the label check; `complete` despite own faults, or despite a stop; the count breaker removed; every missing figure counted; the silent breaker not refreshed, or removed; a stop not ending the work in flight; the files, or the characters, not checked first; `figures` without the retry; the report not written; `figures` not dispatched by the CLI; in the client, a 504, a timeout or a body's timeout not retried, `serve` retrying them, two retries, a 500 `unanswered`, a 504 `failed`, a 4xx `failed`, a non-SVG `refused`, a cut-short body `failed`; the start's driver opened outside the `try` (an unparseable address thrown); the faults found before an error not printed. None survive. + - [x] T025 [US1] Parts, `src/print/parts.ts`. - Each of the book's PDFs is printed in parts (T048's documents). - A part is split at section boundaries past a page threshold: a setting, about 200 pages by default. diff --git a/src/cli.ts b/src/cli.ts index 8840813..641fb8f 100755 --- a/src/cli.ts +++ b/src/cli.ts @@ -1,7 +1,8 @@ #!/usr/bin/env node // pdf-export's command line (contracts/cli.md). Its commands come with spec 001's tasks: // serve (T022) runs the single-event service; preflight (T024) runs the book's preflight -// alone; book, check-book and compare are to come. +// alone; figures (T046) draws the book's figures into a folder; book, check-book and compare +// are to come. export const USAGE = `Usage: pdf-export [options] @@ -9,6 +10,7 @@ Commands: book write the release's book (spec 001, User Story 1) check-book check a book for FR-011's faults compare compare with event-pdf's output, by meaning (FR-012) + figures draw the book's figures into a folder, before the book (SC-010) preflight check the book's inputs, as book does before printing serve the website's single-event PDF service (User Story 2) `; @@ -56,6 +58,12 @@ if (import.meta.main) { out: (text) => process.stdout.write(text), err: (text) => process.stderr.write(text), }); + } else if (args[0] === 'figures') { + const { figuresCommand } = await import('./figures/step.ts'); + process.exitCode = await figuresCommand(args.slice(1), process.env, { + out: (text) => process.stdout.write(text), + err: (text) => process.stderr.write(text), + }); } else { const { code, out } = run(args); (code === 0 ? process.stdout : process.stderr).write(out); diff --git a/src/figures/client.ts b/src/figures/client.ts index c1a412e..2e42908 100644 --- a/src/figures/client.ts +++ b/src/figures/client.ts @@ -1,8 +1,13 @@ -// The figures client (spec 001 T014, contracts/render.md): asks the website's render -// service for each figure as SVG. A 503 is tried once more after a pause; any other -// refusal, a second 503 or a timeout is a missing figure, with its reason, for the caller -// to fail the book on (FR-011) or go on without (a single-event PDF). The caller's own end -// (its signal) is thrown as it is, never read as a missing figure. +// The figures client (spec 001 T014, T046; contracts/render.md): asks the website's render +// service for each figure as SVG. A 503 is tried once more after a pause; when `figures` asks +// (`retryUnanswered`, T046), a 504 or a timeout (the body's included) is too, while `serve` +// keeps one try of them, as SC-005 bounds its time. Any other refusal, or the second try's, +// is a missing figure with its reason and its cause (contracts/figures.md): `unanswered` (no +// connection, a timeout, a body cut short, a 502, 503 or 504), `failed` (any other 5xx: the +// page failed on that figure), `refused` (a 4xx) or `invalid` (any other answer that is not +// an SVG drawing), for the caller to fail the book's figures on or go on without (a +// single-event PDF). The caller's own end (its signal) is thrown as it is, never read as a +// missing figure. import { setTimeout as pause } from 'node:timers/promises'; /** What to draw (data-model.md's Figure). */ @@ -11,14 +16,19 @@ export type FigureRequest = | { kind: 'ancestor'; stId: string; highlight: readonly string[]; token?: string } | { kind: 'reaction'; stId: string; token?: string }; -export type FigureAnswer = { svg: string } | { missing: string }; +/** Why a figure is missing (contracts/figures.md). */ +export type MissingCause = 'unanswered' | 'failed' | 'refused' | 'invalid'; + +export type FigureAnswer = { svg: string } | { missing: string; cause: MissingCause }; export interface ClientOptions { - /** How long one request may take. */ + /** How long one request may take, its body included. */ timeoutMs?: number; - /** The pause before the one retry of a 503 (the render service asks for 5 s). */ + /** The pause before the one retry (the render service asks for 5 s after a 503). */ retryPauseMs?: number; - /** The caller's own end: the service's deadline for the PDF. */ + /** Retry a 504 or a timeout once too, as a 503 is: `figures` asks it, `serve` does not. */ + retryUnanswered?: boolean; + /** The caller's own end: the service's deadline for the PDF, or `figures`' stop. */ signal?: AbortSignal; } @@ -38,10 +48,18 @@ export function figureUrl(base: string, request: FigureRequest): URL { return url; } +/** A status's cause: 502, 503 and 504 say the service did not draw for want of time or room. */ +function statusCause(status: number): MissingCause { + if (status === 502 || status === 503 || status === 504) return 'unanswered'; + if (status >= 500) return 'failed'; + if (status >= 400) return 'refused'; + return 'invalid'; +} + export async function fetchFigure( base: string, request: FigureRequest, - { timeoutMs = 60_000, retryPauseMs = 5_000, signal }: ClientOptions = {} + { timeoutMs = 60_000, retryPauseMs = 5_000, retryUnanswered = false, signal }: ClientOptions = {} ): Promise { const url = figureUrl(base, request); /** The caller's end is its own, not the render service failing: it is thrown, not missing. */ @@ -50,43 +68,75 @@ export async function fetchFigure( }; for (let attempt = 1; ; attempt++) { stopped(); - let response: Response; + const answer = await once(url, request.stId, timeoutMs, signal, stopped); + if (!('retry' in answer)) return answer; + if (attempt > 1 || !(answer.retry === 503 || retryUnanswered)) return answer.missing; + // the pause ends with the caller's end too try { - const timeout = AbortSignal.timeout(timeoutMs); - response = await fetch(url, { - signal: signal ? AbortSignal.any([timeout, signal]) : timeout, - }); + await pause(retryPauseMs, undefined, signal ? { signal } : {}); } catch (error) { stopped(); - return { missing: `${request.stId}: no answer (${reason(error)})` }; + throw error; } - if (response.ok) { - const type = response.headers.get('content-type') ?? ''; - if (!/^image\/svg\+xml(;|$)/.test(type)) { - await response.body?.cancel(); - return { missing: `${request.stId}: not an SVG (${type || 'no type'})` }; - } - // The timeout covers the body too: one cut short is a missing figure, not a failure - try { - return { svg: await response.text() }; - } catch (error) { - stopped(); - return { missing: `${request.stId}: the answer was cut short (${reason(error)})` }; - } + } +} + +type Attempt = + | FigureAnswer + /** A missing figure that may be tried again: a 503, or (for `figures`) a 504 or a timeout. */ + | { retry: 503 | 'other'; missing: FigureAnswer & { cause: MissingCause } }; + +/** One request for a figure, its answer read whole within `timeoutMs`. */ +async function once( + url: URL, + stId: string, + timeoutMs: number, + signal: AbortSignal | undefined, + stopped: () => void +): Promise { + const timeout = AbortSignal.timeout(timeoutMs); + let response: Response; + try { + response = await fetch(url, { signal: signal ? AbortSignal.any([timeout, signal]) : timeout }); + } catch (error) { + stopped(); + const missing = { + missing: `${stId}: no answer (${reason(error)})`, + cause: 'unanswered', + } as const; + return timedOut(error) ? { retry: 'other', missing } : missing; + } + if (response.ok) { + const type = response.headers.get('content-type') ?? ''; + if (!/^image\/svg\+xml(;|$)/.test(type)) { + await response.body?.cancel(); + return { missing: `${stId}: not an SVG (${type || 'no type'})`, cause: 'invalid' }; } - await response.body?.cancel(); - if (response.status === 503 && attempt === 1) { - // the pause ends with the caller's end too - try { - await pause(retryPauseMs, undefined, signal ? { signal } : {}); - } catch (error) { - stopped(); - throw error; - } - continue; + // The timeout covers the body too: one cut short is a missing figure, not a failure + try { + return { svg: await response.text() }; + } catch (error) { + stopped(); + const missing = { + missing: `${stId}: the answer was cut short (${reason(error)})`, + cause: 'unanswered', + } as const; + return timedOut(error) ? { retry: 'other', missing } : missing; } - return { missing: `${request.stId}: the render service answered ${String(response.status)}` }; } + await response.body?.cancel(); + const missing = { + missing: `${stId}: the render service answered ${String(response.status)}`, + cause: statusCause(response.status), + }; + if (response.status === 503) return { retry: 503, missing }; + if (response.status === 504) return { retry: 'other', missing }; + return missing; +} + +/** A request ended by its own timeout (fetch's, or the body's read). */ +function timedOut(error: unknown): boolean { + return error instanceof Error && error.name === 'TimeoutError'; } /** An error's name, and its cause's code where there is one (ECONNREFUSED, say). */ diff --git a/src/figures/step.ts b/src/figures/step.ts new file mode 100644 index 0000000..a22a5c9 --- /dev/null +++ b/src/figures/step.ts @@ -0,0 +1,594 @@ +// `pdf-export figures` (spec 001 T046; contracts/cli.md, contracts/figures.md, SC-010): draws +// every figure the book needs, before the book, into a folder the book reads. In order: +// - the settings, and an `--out` that is absent or empty (exit 2 otherwise: no resuming); +// - the graph's release and checksum (none, or no graph, is exit 2); +// - the book's figures by the Figure rules, built as `book` builds them (T048's `buildBook` +// and the EHLD summary), and T024's release-file and pre-drawing character checks, before +// any request is made (a fault is exit 1); +// - the render service's health (`build` not `unknown`, `files: local`) and the content +// service's release, which must be the graph's (exit 2 otherwise); +// - each figure drawn through T014's client, its 504s and timeouts retried once, checked for +// its labels' characters (T024's `labelsInBook`), written beside its name, synced and moved +// into place, and given its manifest line only once the render service's `build` and the +// content release, read again, still equal the fingerprint's. +// A figure's own fault (missing, or failing its label check) leaves it with no file and no +// line while the rest are drawn; a deployment changed or gone, or the breaker, stops the run at +// once. Either way the run ends with exit 1 and no `complete` line, and its report is written +// however it ends. One writer appends the manifest, synchronously, so lines never interleave. +import { + closeSync, + existsSync, + fsyncSync, + mkdirSync, + openSync, + readdirSync, + renameSync, + rmSync, + statSync, + writeFileSync, + writeSync, +} from 'node:fs'; +import { join } from 'node:path'; + +import { + graphSettings, + openGraph, + type Graph, + type GraphOptions, + type GraphSettings, +} from '../graph/connection.ts'; +import { releaseInfo, type ReleaseInfo } from '../graph/queries.ts'; +import { BookCharacters } from '../layout/characters.ts'; +import { buildBook } from '../model/build.ts'; +import { checkCharacters } from '../preflight/characters.ts'; +import { SettingsError } from '../preflight/command.ts'; +import { checkFiles, neededFiles, readSummary, SUMMARY } from '../preflight/files.ts'; +import { + expectedFigures, + manifestLine, + MANIFEST, + type FigureRecord, + type Fingerprint, + type ManifestLine, +} from '../preflight/folder.ts'; +import { fetchFigure, type FigureRequest, type MissingCause } from './client.ts'; +import { labelsInBook } from './labels.ts'; + +export interface FiguresSettings { + graph: GraphSettings; + species: string; + /** The release's folder: the EHLD summary, and the files the preflight looks for. */ + files: string; + /** The render service's base URL. */ + render: string; + /** The content service's base URL, asked only for its release. */ + content: string; + out: string; + /** Figures asked at once. */ + parallel: number; + /** How long each attempt at a figure, and each health or release read, may take. */ + timeoutMs: number; + /** Where the run report is written, as JSON; none without `--report`. */ + report: string | null; +} + +const FLAGS = [ + '--graph', + '--database', + '--species', + '--files', + '--render', + '--content', + '--out', + '--parallel', + '--timeout', + '--report', +]; + +/** The settings from the flags and the environment; the credentials from the environment. */ +export function figuresSettings( + args: readonly string[], + env: Readonly> +): FiguresSettings { + const flags = new Map(); + for (let i = 0; i < args.length; i++) { + const [name = '', inline] = (args[i] ?? '').split(/=(.*)/s, 2); + if (!FLAGS.includes(name)) throw new SettingsError(`Unknown argument: ${name}`); + const value = inline ?? args[++i]; + if (value === undefined || value === '') throw new SettingsError(`${name} needs a value`); + if (flags.has(name)) throw new SettingsError(`${name} is given twice`); + flags.set(name, value); + } + const required = (flag: string, variable?: string): string => { + const value = flags.get(flag) ?? (variable === undefined ? undefined : env[variable]); + if (value === undefined || value === '') { + throw new SettingsError(`${flag}${variable ? ` (or ${variable})` : ''} is needed`); + } + return value; + }; + const graphEnv = { ...env }; + const uri = flags.get('--graph'); + if (uri !== undefined) graphEnv['NEO4J_URI'] = uri; + const database = flags.get('--database'); + if (database !== undefined) graphEnv['NEO4J_DATABASE'] = database; + let graph: GraphSettings; + try { + graph = graphSettings(graphEnv); + } catch { + throw new SettingsError('--graph (or NEO4J_URI) is needed'); + } + const parallel = flags.get('--parallel') ?? '2'; + if (!/^[1-9]\d*$/.test(parallel)) throw new SettingsError('--parallel must be a whole number'); + const timeout = flags.get('--timeout') ?? '60'; + if (!/^\d+(\.\d+)?$/.test(timeout) || Number(timeout) <= 0) { + throw new SettingsError('--timeout must be a number of seconds above 0'); + } + return { + graph, + species: flags.get('--species') ?? 'Homo sapiens', + files: required('--files'), + render: required('--render', 'RENDER_SERVICE'), + content: required('--content', 'CONTENT_SERVICE'), + out: required('--out'), + parallel: Number(parallel), + timeoutMs: Number(timeout) * 1000, + report: flags.get('--report') ?? null, + }; +} + +/** A read of a service that did not give what was asked: no answer, or a bad one. */ +class ReadError extends Error { + override name = 'ReadError'; +} + +const base = (url: string): string => (url.endsWith('/') ? url : `${url}/`); + +/** A GET within `timeoutMs` (or the caller's end), its body read whole, or a ReadError. */ +async function get(url: URL, timeoutMs: number, signal?: AbortSignal): Promise { + const timeout = AbortSignal.timeout(timeoutMs); + try { + const response = await fetch(url, { + signal: signal ? AbortSignal.any([timeout, signal]) : timeout, + }); + const body = await response.text(); + if (!response.ok) throw new ReadError(`${url.pathname} answered ${String(response.status)}`); + return body; + } catch (error) { + if (error instanceof ReadError) throw error; + const code = ( + error instanceof Error ? (error.cause as { code?: unknown } | undefined) : undefined + )?.code; + const why = error instanceof Error ? error.name : String(error); + throw new ReadError( + `${url.pathname}: no answer (${typeof code === 'string' ? `${why}: ${code}` : why})` + ); + } +} + +/** The render service's health answer (contracts/render.md). */ +export async function readHealth( + render: string, + timeoutMs: number, + signal?: AbortSignal +): Promise<{ build: string; files: string }> { + const body = await get(new URL('health', base(render)), timeoutMs, signal); + let health: unknown; + try { + health = JSON.parse(body); + } catch { + throw new ReadError('/health answered no JSON'); + } + const { build, files } = (health ?? {}) as { build?: unknown; files?: unknown }; + if (typeof build !== 'string') throw new ReadError('/health answered no build'); + return { build, files: typeof files === 'string' ? files : '' }; +} + +/** The content service's release, `GET /data/database/version`. */ +export async function readContentRelease( + content: string, + timeoutMs: number, + signal?: AbortSignal +): Promise { + const body = await get(new URL('data/database/version', base(content)), timeoutMs, signal); + if (!/^\s*\d+\s*$/.test(body)) throw new ReadError('/data/database/version answered no release'); + return Number(body.trim()); +} + +/** Why a figure has no file and no line: the client's cause, or its labels' check. */ +export type FaultCause = MissingCause | 'label'; + +/** Why a run stopped at once (contracts/figures.md). */ +export type StopReason = + | 'deployment-changed' + | 'deployment-unanswered' + | 'breaker-unanswered' + | 'breaker-silent' + | 'error'; + +export interface KindCounts { + needed: number; + drawn: number; + totalMs: number; + maxMs: number; + written: number; +} + +/** The run report (contracts/figures.md): stable identifiers, kinds, times and counts only. */ +export interface FiguresReport { + /** The fingerprint, as far as it was read. */ + release: Partial; + /** Why the run was refused before drawing (exit 2), if it was. */ + refused: string | null; + /** The preflight's faults (exit 1, nothing drawn). */ + preflight: string[]; + /** + * Each kind's figures: how many the book needs, how many the render service drew (and how + * long each took, its retry included), and how many are in the folder with their line. + */ + figures: Partial>; + /** Each figure with a fault of its own, in the order met. */ + missing: { stId: string; cause: FaultCause; detail: string }[]; + /** Why the run stopped at once, if it did. */ + stopped: { reason: StopReason; detail: string } | null; + /** Whether the manifest's `complete` line was written. */ + complete: boolean; + elapsedMs: number; +} + +/** The request that draws a figure's record. */ +export function figureRequest(record: FigureRecord): FigureRequest { + if (record.kind === 'ancestor') { + return { kind: 'ancestor', stId: record.source, highlight: record.highlight }; + } + return { kind: record.kind, stId: record.source }; +} + +interface Draw { + render: string; + content: string; + out: string; + parallel: number; + timeoutMs: number; + retryPauseMs: number; + fingerprint: Fingerprint; + report: FiguresReport; +} + +/** + * Draws each figure into `out`, appending the manifest after the fingerprint, and `complete` + * only when every figure has its line. Returns whether it was written; the report says why + * not. An error of its own (a file that cannot be written) stops the run and is thrown. + */ +export async function drawFigures(figures: readonly FigureRecord[], draw: Draw): Promise { + const { out, parallel, timeoutMs, fingerprint, report } = draw; + mkdirSync(out, { recursive: true }); + const manifest = openSync(join(out, MANIFEST), 'wx'); + /** The one writer: a line written and synced before anything else runs. */ + const append = (line: ManifestLine): void => { + writeSync(manifest, manifestLine(line)); + fsyncSync(manifest); + }; + const stop = new AbortController(); + const halt = (reason: StopReason, detail: string): void => { + if (stop.signal.aborted) return; + report.stopped = { reason, detail }; + stop.abort(new Error(`${reason}: ${detail}`)); + }; + // the breaker: figures unanswered in a row, or no answer of any kind for a while + const silence = 5 * timeoutMs; + let unanswered = 0; + const breaker = setTimeout(() => { + halt('breaker-silent', `no figure answered for ${String(silence / 1000)} s`); + }, silence); + const answered = (): void => { + unanswered = 0; + breaker.refresh(); + }; + + /** The deployment read again: whether it is the fingerprint's; the run halted if not. */ + const unchanged = async (): Promise => { + let now: { build: string; files: string; content: number }; + try { + const [health, content] = await Promise.all([ + readHealth(draw.render, timeoutMs, stop.signal), + readContentRelease(draw.content, timeoutMs, stop.signal), + ]); + now = { ...health, content }; + } catch (error) { + if (!stop.signal.aborted) halt('deployment-unanswered', describe(error)); + return false; + } + if (stop.signal.aborted) return false; + if ( + now.build !== fingerprint.build || + now.content !== fingerprint.content || + now.files !== 'local' + ) { + halt( + 'deployment-changed', + `build ${now.build}, content ${String(now.content)}, files ${now.files || 'unsaid'}` + ); + return false; + } + return true; + }; + + const one = async (record: FigureRecord): Promise => { + const { stId, kind } = record; + const started = performance.now(); + const answer = await fetchFigure(draw.render, figureRequest(record), { + timeoutMs, + retryPauseMs: draw.retryPauseMs, + retryUnanswered: true, + signal: stop.signal, + }); + if ('missing' in answer) { + report.missing.push({ stId, cause: answer.cause, detail: answer.missing }); + if (answer.cause !== 'unanswered') { + answered(); + return; + } + unanswered += 1; + if (unanswered >= 2 * parallel) { + halt('breaker-unanswered', `${String(unanswered)} figures in a row unanswered`); + return; + } + await unchanged(); + return; + } + answered(); + const times = report.figures[kind]; + if (times) { + const ms = Math.round(performance.now() - started); + times.drawn += 1; + times.totalMs += ms; + times.maxMs = Math.max(times.maxMs, ms); + } + const characters = new BookCharacters(); + try { + labelsInBook(answer.svg, stId, characters); + } catch (error) { + report.missing.push({ stId, cause: 'label', detail: `unreadable: ${describe(error)}` }); + return; + } + if (characters.faults.length > 0) { + report.missing.push({ stId, cause: 'label', detail: characters.faults.join(', ') }); + return; + } + // written beside its name, synced, and moved into place, with nothing between + const file = join(out, `${stId}.svg`); + const part = `${file}.part`; + const fd = openSync(part, 'wx'); + try { + writeSync(fd, answer.svg); + fsyncSync(fd); + } finally { + closeSync(fd); + } + renameSync(part, file); + if ((await unchanged()) && !stop.signal.aborted) { + append({ type: 'figure', ...record }); + if (times) times.written += 1; + } else { + rmSync(file, { force: true }); + } + }; + + const queue = [...figures]; + /** A worker's own error (a file that cannot be written), thrown once the workers end. */ + const failed: { error?: Error } = {}; + /** Read afresh: another figure may have stopped the run while this one waited. */ + const halted = (): boolean => stop.signal.aborted; + const worker = async (): Promise => { + for (let next = queue.shift(); next && !halted(); next = queue.shift()) { + try { + await one(next); + } catch (error) { + if (halted()) return; + failed.error = error instanceof Error ? error : new Error(String(error)); + halt('error', describe(error)); + } + } + }; + try { + append({ type: 'fingerprint', ...fingerprint }); + await Promise.all(Array.from({ length: Math.min(parallel, figures.length) }, worker)); + if (failed.error) throw failed.error; + if (stop.signal.aborted || report.missing.length > 0) return false; + append({ type: 'complete', ...fingerprint }); + return true; + } finally { + clearTimeout(breaker); + closeSync(manifest); + } +} + +/** `--out`: absent, or an empty folder. */ +function usableOut(out: string): boolean { + if (!existsSync(out)) return true; + return statSync(out).isDirectory() && readdirSync(out).length === 0; +} + +/** `pdf-export figures`: its exit code; faults to `out`, the rest to `err`. */ +export async function figuresCommand( + args: readonly string[], + env: Readonly>, + { + out, + err, + open = openGraph, + retryPauseMs = 5000, + }: { + out: (text: string) => void; + err: (text: string) => void; + /** How the graph is opened (a test gives one that rewrites its rows). */ + open?: (settings: GraphSettings, options?: GraphOptions) => Graph; + /** The pause before a figure's retry (the render service asks for 5 s). */ + retryPauseMs?: number; + } +): Promise { + let settings: FiguresSettings; + try { + settings = figuresSettings(args, env); + } catch (error) { + if (!(error instanceof SettingsError)) throw error; + err(`${error.message}\n`); + return 2; + } + const report: FiguresReport = { + release: {}, + refused: null, + preflight: [], + figures: {}, + missing: [], + stopped: null, + complete: false, + elapsedMs: 0, + }; + const started = performance.now(); + const refuse = (why: string): number => { + report.refused = why; + err(`${why}\n`); + return 2; + }; + try { + return await steps(settings, report, { out, err, open, retryPauseMs, refuse }); + } catch (error) { + err(`The figures' run stopped: ${describe(error)}\n`); + report.stopped ??= { reason: 'error', detail: describe(error) }; + return 2; + } finally { + report.elapsedMs = Math.round(performance.now() - started); + if (settings.report !== null) { + writeFileSync(settings.report, `${JSON.stringify(report, null, 2)}\n`); + } + } +} + +async function steps( + settings: FiguresSettings, + report: FiguresReport, + { + out, + err, + open, + retryPauseMs, + refuse, + }: { + out: (text: string) => void; + err: (text: string) => void; + open: (settings: GraphSettings, options?: GraphOptions) => Graph; + retryPauseMs: number; + refuse: (why: string) => number; + } +): Promise { + if (!usableOut(settings.out)) { + return refuse(`--out must be absent or an empty folder: ${settings.out}`); + } + + // the start's read, its retrying bounded, so a graph that cannot be reached is said soon + // (opened inside the try: the driver throws at once on an address it cannot parse) + let start: Graph | undefined; + let release: ReleaseInfo; + try { + start = open(settings.graph, { connectionTimeoutMs: 5000, retryTimeMs: 0 }); + release = await releaseInfo(start); + } catch (error) { + return refuse(`The graph cannot be read: ${describe(error)}`); + } finally { + await start?.close(); + } + report.release.release = release.release; + if (release.checksum === null) return refuse('The graph has no DBInfo checksum'); + report.release.checksum = release.checksum; + + // the figures by the Figure rules, and the preflight's checks, before any request + const summary = readSummary(settings.files); + if (!summary) report.preflight.push(`missing-file ${SUMMARY}`); + let figures: FigureRecord[] = []; + if (summary) { + report.release.summary = summary.hash; + const graph = open(settings.graph); + try { + const docs = await buildBook(graph, { + species: settings.species, + illustrated: summary.illustrated, + }); + figures = [...expectedFigures(docs).values()]; + report.preflight.push(...checkFiles(settings.files, neededFiles(figures))); + report.preflight.push( + ...(await checkCharacters(graph, docs, { files: settings.files, expected: figures })) + ); + } catch (error) { + // the faults found before an error are printed, as the preflight's command prints them + for (const fault of report.preflight) out(`fault ${fault}\n`); + throw error; + } finally { + await graph.close(); + } + } + if (report.preflight.length > 0) { + for (const fault of report.preflight) out(`fault ${fault}\n`); + return 1; + } + for (const { kind } of figures) { + const entry = (report.figures[kind] ??= { + needed: 0, + drawn: 0, + totalMs: 0, + maxMs: 0, + written: 0, + }); + entry.needed += 1; + } + + // the deployment: its build and files, and the content service's release + let health: { build: string; files: string }; + try { + health = await readHealth(settings.render, settings.timeoutMs); + } catch (error) { + return refuse(`The render service cannot be read: ${describe(error)}`); + } + if (health.build === 'unknown') return refuse("The render service's build is unknown"); + if (health.files !== 'local') return refuse('The render service does not read local files'); + report.release.build = health.build; + let content: number; + try { + content = await readContentRelease(settings.content, settings.timeoutMs); + } catch (error) { + return refuse(`The content service cannot be read: ${describe(error)}`); + } + report.release.content = content; + if (content !== release.release) { + return refuse( + `The content service answers release ${String(content)}, the graph is release ${String(release.release)}` + ); + } + + const fingerprint: Fingerprint = { + release: release.release, + checksum: release.checksum, + summary: summary?.hash ?? '', + content, + build: health.build, + }; + const complete = await drawFigures(figures, { + render: settings.render, + content: settings.content, + out: settings.out, + parallel: settings.parallel, + timeoutMs: settings.timeoutMs, + retryPauseMs, + fingerprint, + report, + }); + report.complete = complete; + for (const { stId, cause } of report.missing) out(`fault missing-figure ${stId} ${cause}\n`); + if (report.stopped) err(`stopped: ${report.stopped.reason}: ${report.stopped.detail}\n`); + const written = Object.values(report.figures).reduce((sum, kind) => sum + kind.written, 0); + err(`${String(written)} of ${String(figures.length)} figures in ${settings.out}\n`); + return complete ? 0 : 1; +} + +const describe = (error: unknown): string => + error instanceof Error ? error.message : String(error); diff --git a/test/cli.test.ts b/test/cli.test.ts index 9ed02a6..c81d041 100644 --- a/test/cli.test.ts +++ b/test/cli.test.ts @@ -1,4 +1,6 @@ import assert from 'node:assert/strict'; +import { spawnSync } from 'node:child_process'; +import { fileURLToPath } from 'node:url'; import { test } from 'node:test'; import { run, USAGE } from '../src/cli.ts'; @@ -13,3 +15,14 @@ test('an unknown command is refused with exit 2, naming it', () => { assert.equal(code, 2); assert.match(out, /^Unknown command: bake\n/); }); + +test('figures is a command: its bad arguments are exit 2, saying what is needed', () => { + const result = spawnSync( + process.execPath, + [fileURLToPath(new URL('../src/cli.ts', import.meta.url)), 'figures'], + { encoding: 'utf8', env: { PATH: process.env['PATH'] ?? '', NEO4J_URI: 'bolt://127.0.0.1:1' } } + ); + assert.equal(result.status, 2); + assert.match(result.stderr, /--files is needed/); + assert.match(USAGE, /^ {2}figures {6}/m); +}); diff --git a/test/figures/client.test.ts b/test/figures/client.test.ts index db71c98..a5dacc2 100644 --- a/test/figures/client.test.ts +++ b/test/figures/client.test.ts @@ -2,7 +2,7 @@ import assert from 'node:assert/strict'; import { after, afterEach, before, describe, test } from 'node:test'; import { fetchFigure, type FigureRequest } from '../../src/figures/client.ts'; -import { startRenderStub, type RenderStub } from '../stubs/render.ts'; +import { startRenderStub, type Refusal, type RenderStub } from '../stubs/render.ts'; const ALPHA = 'R-HSA-90000200'; const REACTION = 'R-HSA-90000211'; @@ -81,6 +81,45 @@ describe('the figures client', () => { assert.ok('missing' in json); assert.match(json.missing, /not an SVG/); }); + + test("names each missing figure's cause (contracts/figures.md)", async () => { + const cause = async (how: Refusal | null, stId = ALPHA): Promise => { + stub.reset(); + if (how !== null) stub.refuse(stId, how); + const answer = await ask({ kind: 'diagram', stId }); + return 'missing' in answer ? answer.cause : 'drawn'; + }; + assert.equal(await cause(404), 'refused'); + assert.equal(await cause(null, 'R-HSA-90000999'), 'refused'); + assert.equal(await cause(500), 'failed'); + assert.equal(await cause(503), 'unanswered'); + assert.equal(await cause(504), 'unanswered'); + assert.equal(await cause('timeout'), 'unanswered'); + assert.equal(await cause('stall-body'), 'unanswered'); + assert.equal(await cause('not-svg'), 'invalid'); + assert.equal(await cause(null), 'drawn'); + }); + + test('serve keeps one try of a 504 or a timeout; figures retries each once', async () => { + const tries = async (how: Refusal, retryUnanswered: boolean, times = -1) => { + stub.reset(); + stub.refuse(ALPHA, how, times); + const answer = await fetchFigure( + stub.url, + { kind: 'diagram', stId: ALPHA }, + { timeoutMs: 300, retryPauseMs: 10, retryUnanswered } + ); + return { drawn: 'svg' in answer, requests: stub.requests.length }; + }; + for (const how of [504, 'timeout', 'stall-body'] as const) { + assert.deepEqual(await tries(how, false), { drawn: false, requests: 1 }, String(how)); + assert.deepEqual(await tries(how, true), { drawn: false, requests: 2 }, String(how)); + assert.deepEqual(await tries(how, true, 1), { drawn: true, requests: 2 }, String(how)); + } + // a 500 or a 404 is the figure's own: never retried + assert.deepEqual(await tries(500, true), { drawn: false, requests: 1 }); + assert.deepEqual(await tries(404, true), { drawn: false, requests: 1 }); + }); }); test('a render service that is not there is a missing figure that says why', async () => { @@ -90,4 +129,12 @@ test('a render service that is not there is a missing figure that says why', asy const answer = await fetchFigure(url, { kind: 'diagram', stId: ALPHA }, { timeoutMs: 500 }); assert.ok('missing' in answer); assert.match(answer.missing, /ECONNREFUSED/); + assert.equal(answer.cause, 'unanswered'); + // no connection is not retried, even by figures: the service is gone, not slow + const again = await fetchFigure( + url, + { kind: 'diagram', stId: ALPHA }, + { timeoutMs: 500, retryPauseMs: 2000, retryUnanswered: true } + ); + assert.ok('missing' in again); }); diff --git a/test/figures/step.test.ts b/test/figures/step.test.ts new file mode 100644 index 0000000..613f499 --- /dev/null +++ b/test/figures/step.test.ts @@ -0,0 +1,545 @@ +// `pdf-export figures` (spec 001 T046) on the fixture graph, against the render and content +// stubs: the folder it writes passes the book's checks, and each refusal, fault and stop is +// what contracts/figures.md says. The shared fixture is never changed: a missing checksum or +// a planted character comes from a graph that rewrites its rows. +import assert from 'node:assert/strict'; +import { + cpSync, + existsSync, + mkdtempSync, + readdirSync, + readFileSync, + rmSync, + unlinkSync, + writeFileSync, +} from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { after, afterEach, before, describe, test } from 'node:test'; + +import { + graphSettings, + openGraph, + type Graph, + type GraphOptions, + type GraphSettings, +} from '../../src/graph/connection.ts'; +import { releaseInfo, type ReleaseInfo } from '../../src/graph/queries.ts'; +import { buildBook } from '../../src/model/build.ts'; +import { readSummary } from '../../src/preflight/files.ts'; +import { + checkFolder, + expectedFigures, + MANIFEST, + type FigureRecord, +} from '../../src/preflight/folder.ts'; +import { figuresCommand, figuresSettings, type FiguresReport } from '../../src/figures/step.ts'; +import { startContentStub, type ContentStub } from '../stubs/content.ts'; +import { startRenderStub, type RenderStub } from '../stubs/render.ts'; + +const skip = process.env['NEO4J_URI'] ? false : 'no fixture graph: set NEO4J_URI'; +const FILES = fileURLToPath(new URL('../../ci/fixture/files/', import.meta.url)); +const ALPHA = 'R-HSA-90000200'; +const ALPHA_SUB = 'R-HSA-90000210'; +const REACTION = 'R-HSA-90000211'; +const BETA = 'R-HSA-90000100'; +const DELTA = 'R-HSA-90000215'; + +/** A graph whose rows are rewritten: a checksum taken away, a character planted. */ +function rewriting( + graph: Graph, + rewrite: (row: Record, query: string) => Record +): Graph { + return { + async read(query: string, params?: Record): Promise { + const rows = await graph.read>(query, params); + return rows.map((row) => rewrite(row, query)) as T[]; + }, + close: () => graph.close(), + }; +} + +/** Every string in a value with `from` replaced by `to`. */ +function replaced(value: unknown, from: string, to: string): unknown { + if (typeof value === 'string') return value.split(from).join(to); + if (Array.isArray(value)) return value.map((item) => replaced(item, from, to)); + if (value !== null && typeof value === 'object') { + return Object.fromEntries( + Object.entries(value).map(([key, item]) => [key, replaced(item, from, to)]) + ); + } + return value; +} + +interface Run { + code: number; + out: string; + err: string; + dir: string; + report: FiguresReport; + /** The manifest's lines, parsed. */ + lines: Record[]; + /** The files in `--out`, sorted. */ + files: string[]; +} + +describe('pdf-export figures', { skip }, () => { + let render: RenderStub; + let content: ContentStub; + let release: ReleaseInfo; + let summary: string; + let expected: Map; + const dirs: string[] = []; + const scratch = (): string => { + const dir = mkdtempSync(join(tmpdir(), 'pdfx-figures-')); + dirs.push(dir); + return dir; + }; + + before(async () => { + render = await startRenderStub(); + content = await startContentStub(); + const graph = openGraph(graphSettings(process.env)); + try { + release = await releaseInfo(graph); + const read = readSummary(FILES); + if (!read) throw new Error('the fixture has no EHLD summary'); + summary = read.hash; + expected = expectedFigures( + await buildBook(graph, { species: 'Homo sapiens', illustrated: read.illustrated }) + ); + } finally { + await graph.close(); + } + }); + afterEach(() => { + render.reset(); + content.reset(); + }); + after(async () => { + await render.close(); + await content.close(); + for (const dir of dirs) rmSync(dir, { recursive: true, force: true }); + }); + + /** A run with the stubs, a new `--out` and a report beside it; `flags` added or replacing. */ + const figures = async ( + flags: Record = {}, + open?: (settings: GraphSettings, options?: GraphOptions) => Graph, + retryPauseMs = 10 + ): Promise => { + const work = scratch(); + const settings: Record = { + '--files': FILES, + '--render': render.url, + '--content': content.url, + '--out': join(work, 'figures'), + '--parallel': '2', + '--timeout': '0.5', + '--report': join(work, 'report.json'), + ...flags, + }; + let out = ''; + let err = ''; + const code = await figuresCommand(Object.entries(settings).flat(), process.env, { + out: (text) => (out += text), + err: (text) => (err += text), + retryPauseMs, + ...(open ? { open } : {}), + }); + const dir = settings['--out'] ?? ''; + const manifest = join(dir, MANIFEST); + return { + code, + out, + err, + dir, + report: JSON.parse(readFileSync(settings['--report'] ?? '', 'utf8')) as FiguresReport, + lines: existsSync(manifest) + ? readFileSync(manifest, 'utf8') + .split('\n') + .filter(Boolean) + .map((line) => JSON.parse(line) as Record) + : [], + files: existsSync(dir) ? readdirSync(dir).sort() : [], + }; + }; + + const figureLines = (run: Run): string[] => + run.lines.filter((line) => line['type'] === 'figure').map((line) => String(line['stId'])); + const hasComplete = (run: Run): boolean => run.lines.some((l) => l['type'] === 'complete'); + const renders = (): string[] => render.requests.filter((path) => path.startsWith('/render/')); + + test("draws one file per figure the fixture's book needs, and a manifest the book accepts", async () => { + const run = await figures(); + assert.equal(run.code, 0, run.err); + assert.deepEqual( + run.files, + [...[...expected.keys()].map((stId) => `${stId}.svg`), MANIFEST].sort() + ); + // every kind, each drawn by its source, an ancestor's with the event's selection + const counts = Object.fromEntries( + Object.entries(run.report.figures).map(([kind, c]) => [kind, [c.needed, c.written]]) + ); + assert.deepEqual(counts, { diagram: [5, 5], reaction: [8, 8], ancestor: [3, 3], ehld: [2, 2] }); + assert.ok(renders().includes(`/render/${ALPHA}.svg?highlight=${REACTION}%2CR-HSA-90000212`)); + assert.match(readFileSync(join(run.dir, `${ALPHA_SUB}.svg`), 'utf8'), /data-highlight=/); + assert.equal(run.lines[0]?.['type'], 'fingerprint'); + assert.deepEqual(run.lines[0], { + type: 'fingerprint', + release: release.release, + checksum: release.checksum, + summary, + content: 98, + build: '000000000000', + }); + assert.ok(hasComplete(run)); + assert.deepEqual(checkFolder(run.dir, { release, summary, expected }), []); + assert.equal(run.report.complete, true); + assert.equal(run.report.stopped, null); + assert.deepEqual(run.report.missing, []); + }); + + test('refuses an --out that is not empty, before any request', async () => { + const dir = scratch(); + writeFileSync(join(dir, 'old.svg'), ''); + const run = await figures({ '--out': dir }); + assert.equal(run.code, 2); + assert.deepEqual(run.files, ['old.svg']); + assert.deepEqual(render.requests, []); + assert.match(run.report.refused ?? '', /--out/); + }); + + test('refuses a render build of unknown, files that are not local, or a graph with no checksum', async () => { + render.health.build = 'unknown'; + const unknown = await figures(); + assert.equal(unknown.code, 2); + assert.match(unknown.err, /build is unknown/); + render.health.build = '000000000000'; + render.health.files = 'download'; + const remote = await figures(); + assert.equal(remote.code, 2); + assert.match(remote.err, /local files/); + render.reset(); + const noChecksum = await figures({}, (settings, options) => + rewriting(openGraph(settings, options), (row, query) => + query.includes('DBInfo') ? { ...row, checksum: null } : row + ) + ); + assert.equal(noChecksum.code, 2); + assert.match(noChecksum.err, /no DBInfo checksum/); + for (const run of [unknown, remote, noChecksum]) { + assert.deepEqual(run.files, []); + assert.ok(!existsSync(run.dir)); + } + assert.deepEqual(renders(), []); + }); + + test("a build changed between a figure's render and the read after it stops the run", async () => { + render.onRender((stId) => { + if (stId === REACTION) render.health.build = '111111111111'; + }); + const run = await figures({ '--parallel': '1' }); + assert.equal(run.code, 1); + assert.ok(!run.files.includes(`${REACTION}.svg`)); + assert.ok(!figureLines(run).includes(REACTION)); + assert.ok(!hasComplete(run)); + // stopped at once: nothing drawn after it + assert.equal(renders().at(-1), `/render/${REACTION}.svg?layout=reaction`); + assert.ok(figureLines(run).length < expected.size - 1); + // the report is written for a stop too + assert.equal(run.report.stopped?.reason, 'deployment-changed'); + assert.equal(run.report.complete, false); + }); + + test('a deployment changed is read after an unanswered figure, and stops the run there', async () => { + render.refuse(REACTION, 504); + render.onRender((stId) => { + if (stId === REACTION) render.health.build = '111111111111'; + }); + const run = await figures({ '--parallel': '1' }); + assert.equal(run.code, 1); + assert.equal(run.report.stopped?.reason, 'deployment-changed'); + // the reaction tried twice, and nothing asked after it + assert.deepEqual(renders().slice(-2), [ + `/render/${REACTION}.svg?layout=reaction`, + `/render/${REACTION}.svg?layout=reaction`, + ]); + }); + + test('a stop ends the figures in flight at once', async () => { + // two at once: the first figure held unanswered, the second changing the build + const [first, second] = [...expected.keys()]; + render.refuse(first ?? '', 'timeout'); + render.onRender((stId, refused) => { + if (stId === second && !refused) render.health.build = '111111111111'; + }); + const started = Date.now(); + const run = await figures({ '--timeout': '5' }); + assert.equal(run.code, 1); + assert.equal(run.report.stopped?.reason, 'deployment-changed'); + // not the held figure's two tries of 5 s + assert.ok(Date.now() - started < 2000, String(Date.now() - started)); + assert.ok(!run.files.includes(`${first ?? ''}.svg`)); + }); + + test('a content release changed likewise stops the run', async () => { + render.onRender((stId) => { + if (stId === REACTION) content.release = 99; + }); + const run = await figures({ '--parallel': '1' }); + assert.equal(run.code, 1); + assert.ok(!run.files.includes(`${REACTION}.svg`)); + assert.ok(!figureLines(run).includes(REACTION)); + assert.ok(!hasComplete(run)); + assert.equal(run.report.stopped?.reason, 'deployment-changed'); + }); + + test("a 404, a 500 and a label fault are each a figure's own: the rest are drawn", async () => { + render.refuse(DELTA, 404); + render.refuse(REACTION, 500); + render.label(BETA, 'node \u2030'); + const run = await figures(); + assert.equal(run.code, 1); + const faults = new Map(run.report.missing.map(({ stId, cause }) => [stId, cause])); + assert.deepEqual( + faults, + new Map([ + [DELTA, 'refused'], + [REACTION, 'failed'], + [BETA, 'label'], + ]) + ); + assert.match( + run.report.missing.find(({ stId }) => stId === BETA)?.detail ?? '', + /uncovered-character U\+2030 R-HSA-90000100/ + ); + // its health unchanged, a 500 is no stop + assert.equal(run.report.stopped, null); + for (const stId of faults.keys()) { + assert.match(run.out, new RegExp(`^fault missing-figure ${stId} `, 'm')); + assert.ok(!run.files.includes(`${stId}.svg`), stId); + assert.ok(!figureLines(run).includes(stId), stId); + } + const drawn = [...expected.keys()].filter((stId) => !faults.has(stId)); + assert.deepEqual(figureLines(run).sort(), drawn.sort()); + for (const stId of drawn) assert.ok(run.files.includes(`${stId}.svg`), stId); + assert.ok(!hasComplete(run)); + // nothing left part written + assert.deepEqual( + run.files.filter((name) => name.endsWith('.part')), + [] + ); + }); + + test('a content service not answering the read after a figure stops the run', async () => { + render.onRender((stId) => { + if (stId === REACTION) content.silence(); + }); + const run = await figures({ '--parallel': '1' }); + assert.equal(run.code, 1); + assert.ok(!run.files.includes(`${REACTION}.svg`)); + assert.ok(!hasComplete(run)); + assert.equal(run.report.stopped?.reason, 'deployment-unanswered'); + }); + + test('a render service gone silent mid-run stops it at once, saying why', async () => { + let drawn = 0; + render.onRender(() => { + if (++drawn === 3) render.silence(); + }); + const started = Date.now(); + const run = await figures(); + assert.equal(run.code, 1); + assert.ok(!hasComplete(run)); + assert.equal(run.report.stopped?.reason, 'deployment-unanswered'); + // without waiting through the rest: 18 figures at two tries of 0.5 s each would be 9 s + assert.ok(Date.now() - started < 3000, String(Date.now() - started)); + assert.ok(renders().length < 8, String(renders().length)); + assert.match(run.err, /stopped: deployment-unanswered/); + }); + + test('a render service answering its health but 504 to every figure trips the breaker', async () => { + render.refuseAll(504); + const run = await figures(); + assert.equal(run.code, 1); + assert.equal(run.report.stopped?.reason, 'breaker-unanswered'); + assert.ok(!hasComplete(run)); + // two at once: four figures unanswered, each tried twice + assert.ok(run.report.missing.length <= 5, String(run.report.missing.length)); + assert.ok(run.report.missing.every(({ cause }) => cause === 'unanswered')); + }); + + test('ten figures in a row failing (500) do not trip the breaker', async () => { + // the first ten figures, one at a time: by their sources, as two share one (a request + // left over from an earlier test cannot use up a refusal kept for each source) + const first = [...expected.values()].slice(0, 10); + for (const { source } of first) render.refuse(source, 500); + const run = await figures({ '--parallel': '1' }); + assert.equal(run.code, 1); + assert.equal(run.report.stopped, null); + assert.deepEqual( + run.report.missing.map(({ stId }) => stId), + first.map(({ stId }) => stId) + ); + assert.ok(run.report.missing.every(({ cause }) => cause === 'failed')); + assert.equal(figureLines(run).length, expected.size - 10); + }); + + test('nor do 500s answered slowly for longer than five timeouts in all', async () => { + render.refuseAll(500); + render.slow(200); + const started = Date.now(); + // 18 figures one at a time, 0.2 s each: 3.6 s, past five timeouts of 0.5 s + const run = await figures({ '--parallel': '1' }); + assert.ok(Date.now() - started > 2500); + assert.equal(run.code, 1); + assert.equal(run.report.stopped, null); + assert.equal(run.report.missing.length, expected.size); + }); + + test('no figure answered for five timeouts, its health answering, trips the breaker', async () => { + // every figure slower than its timeout, ten at once (the count's 20 is never reached), a + // retry's pause of 1 s: no figure has an answer of any kind before 1.5 s + render.slow(5000); + const started = Date.now(); + const run = await figures({ '--parallel': '10', '--timeout': '0.3' }, undefined, 1000); + assert.equal(run.code, 1); + assert.equal(run.report.stopped?.reason, 'breaker-silent'); + assert.ok(Date.now() - started < 2500, String(Date.now() - started)); + assert.deepEqual(figureLines(run), []); + }); + + test('an unreachable content or render service is exit 2', async () => { + const gone = await startContentStub(); + await gone.close(); + const noContent = await figures({ '--content': gone.url }); + assert.equal(noContent.code, 2); + assert.match(noContent.err, /content service cannot be read/); + const goneRender = await startRenderStub(); + await goneRender.close(); + const noRender = await figures({ '--render': goneRender.url }); + assert.equal(noRender.code, 2); + assert.match(noRender.err, /render service cannot be read/); + assert.deepEqual(renders(), []); + }); + + test('a content service answering another release is refused before any drawing', async () => { + content.release = 97; + const run = await figures(); + assert.equal(run.code, 2); + assert.match(run.err, /release 97/); + assert.deepEqual(renders(), []); + assert.deepEqual(run.files, []); + }); + + test('a missing release file and an uncovered character each fail before any request', async () => { + const files = scratch(); + cpSync(FILES, files, { recursive: true }); + unlinkSync(join(files, 'diagram', `${ALPHA}.graph.json`)); + const missing = await figures({ '--files': files }); + assert.equal(missing.code, 1); + assert.match(missing.out, new RegExp(`^fault missing-file diagram/${ALPHA}.graph.json$`, 'm')); + const uncovered = await figures({}, (settings, options) => + rewriting( + openGraph(settings, options), + (row) => replaced(row, 'A third placeholder', 'A third \u2030 placeholder') as typeof row + ) + ); + assert.equal(uncovered.code, 1); + assert.match(uncovered.out, /^fault uncovered-character U\+2030 R-HSA-90000211$/m); + assert.deepEqual(render.requests, []); + assert.deepEqual(content.requests, []); + assert.deepEqual(uncovered.report.preflight, ['uncovered-character U+2030 R-HSA-90000211']); + }); + + test('a graph failing during the checks is exit 2, the faults found before it printed', async () => { + const files = scratch(); + cpSync(FILES, files, { recursive: true }); + unlinkSync(join(files, 'diagram', `${ALPHA}.graph.json`)); + const run = await figures({ '--files': files }, (settings, options) => { + const graph = openGraph(settings, options); + return { + read: (query: string, params?: Record): Promise => + query.includes('WHERE r:ReactionLikeEvent') + ? Promise.reject(new Error('the graph failed')) + : graph.read(query, params), + close: () => graph.close(), + }; + }); + assert.equal(run.code, 2); + assert.match(run.err, /the graph failed/); + assert.match(run.out, new RegExp(`^fault missing-file diagram/${ALPHA}.graph.json$`, 'm')); + assert.deepEqual(render.requests, []); + }); + + test('a figure timing out once, and one answering 504 once, are drawn on their retry', async () => { + render.refuse(REACTION, 'timeout', 1); + render.refuse(ALPHA, 504, 1); + const run = await figures(); + assert.equal(run.code, 0, run.err); + assert.ok(run.files.includes(`${REACTION}.svg`)); + assert.ok(run.files.includes(`${ALPHA}.svg`)); + assert.equal( + renders().filter((p) => p === `/render/${REACTION}.svg?layout=reaction`).length, + 2 + ); + assert.equal(renders().filter((p) => p === `/render/${ALPHA}.svg`).length, 2); + }); +}); + +test("figures' settings: defaults, and bad values refused", () => { + const env = { + NEO4J_URI: 'bolt://127.0.0.1:1', + RENDER_SERVICE: 'http://r', + CONTENT_SERVICE: 'http://c', + }; + const settings = figuresSettings(['--files', 'f', '--out', 'o'], env); + assert.equal(settings.parallel, 2); + assert.equal(settings.timeoutMs, 60_000); + assert.equal(settings.render, 'http://r'); + assert.equal(settings.content, 'http://c'); + assert.equal(settings.report, null); + assert.throws(() => figuresSettings(['--files', 'f'], env), /--out is needed/); + assert.throws( + () => figuresSettings(['--files', 'f', '--out', 'o', '--parallel', '0'], env), + /--parallel/ + ); + assert.throws( + () => figuresSettings(['--files', 'f', '--out', 'o', '--timeout', '0'], env), + /--timeout/ + ); + assert.throws( + () => figuresSettings(['--files', 'f', '--out', 'o'], { NEO4J_URI: 'bolt://x' }), + /--render \(or RENDER_SERVICE\)/ + ); +}); + +test('bad arguments are exit 2', async () => { + let err = ''; + const code = await figuresCommand( + ['--colour', 'blue'], + {}, + { + out: () => undefined, + err: (text) => (err += text), + } + ); + assert.equal(code, 2); + assert.match(err, /Unknown argument: --colour/); +}); + +test('a graph address the driver cannot parse is exit 2, not a throw', async () => { + let err = ''; + const code = await figuresCommand( + [ + ...['--graph', 'not a uri', '--files', 'f', '--render', 'http://r', '--content', 'http://c'], + ...['--out', join(tmpdir(), `pdfx-figures-never-${String(process.pid)}`)], + ], + {}, + { out: () => undefined, err: (text) => (err += text) } + ); + assert.equal(code, 2); + assert.match(err, /The graph cannot be read/); +}); diff --git a/test/stubs/content.test.ts b/test/stubs/content.test.ts new file mode 100644 index 0000000..c290c51 --- /dev/null +++ b/test/stubs/content.test.ts @@ -0,0 +1,28 @@ +import assert from 'node:assert/strict'; +import { after, before, test } from 'node:test'; + +import { startContentStub, type ContentStub } from './content.ts'; + +let stub: ContentStub; +before(async () => { + stub = await startContentStub(); +}); +after(async () => { + await stub.close(); +}); + +test('answers its release as ContentService does, as set, and nothing when silenced', async () => { + const version = () => + fetch(`${stub.url}/data/database/version`, { signal: AbortSignal.timeout(300) }); + const first = await version(); + assert.match(first.headers.get('content-type') ?? '', /^text\/plain/); + assert.equal(await first.text(), '98'); + stub.release = 99; + assert.equal(await (await version()).text(), '99'); + assert.equal((await fetch(`${stub.url}/data/other`)).status, 404); + stub.silence(); + await assert.rejects(version(), { name: 'TimeoutError' }); + stub.reset(); + assert.equal(await (await version()).text(), '98'); + assert.deepEqual(stub.requests, ['/data/database/version']); +}); diff --git a/test/stubs/content.ts b/test/stubs/content.ts new file mode 100644 index 0000000..86a9008 --- /dev/null +++ b/test/stubs/content.ts @@ -0,0 +1,64 @@ +// The content stub (spec 001 T046): the one request `pdf-export figures` makes of the content +// service, `GET /data/database/version`, answered as ContentService answers it, the release's +// number as plain text. Its release is settable by a test, also between a figure's render and +// the read after it, and it can be silenced (every request held unanswered). +import { createServer, type Server, type ServerResponse } from 'node:http'; +import type { AddressInfo } from 'node:net'; + +export interface ContentStub { + /** The base URL: requests go to `${url}/data/database/version`. */ + url: string; + /** Every request's path, in order. */ + requests: string[]; + /** The release answered; the fixture's, 98, at the start. */ + release: number; + /** Hold every request from now on unanswered. */ + silence(): void; + /** The release back to 98, the silence ended and the requests forgotten. */ + reset(): void; + close(): Promise; +} + +export async function startContentStub(): Promise { + const hanging = new Set(); + let silent = false; + const stub: ContentStub = { + url: '', + requests: [], + release: 98, + silence() { + silent = true; + }, + reset() { + silent = false; + stub.release = 98; + stub.requests.length = 0; + }, + async close() { + for (const res of hanging) res.destroy(); + server.closeAllConnections(); + await new Promise((resolve) => + server.close(() => { + resolve(); + }) + ); + }, + }; + const server: Server = createServer((req, res) => { + const path = new URL(req.url ?? '/', 'http://stub').pathname; + stub.requests.push(path); + if (silent) { + hanging.add(res); + return; + } + if (path !== '/data/database/version') { + res.writeHead(404, { 'Content-Type': 'application/json' }).end('{"code":404}'); + return; + } + res.writeHead(200, { 'Content-Type': 'text/plain;charset=UTF-8' }).end(String(stub.release)); + }); + await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)); + const { port } = server.address() as AddressInfo; + stub.url = `http://127.0.0.1:${String(port)}`; + return stub; +} diff --git a/test/stubs/render.test.ts b/test/stubs/render.test.ts index 3337aef..81ee6d6 100644 --- a/test/stubs/render.test.ts +++ b/test/stubs/render.test.ts @@ -101,6 +101,50 @@ describe('the render stub', () => { stub.reset(); assert.deepEqual(stub.requests, []); }); + + test('refuses with a 500 or a 504, for one figure or for every one', async () => { + stub.refuse(ALPHA, 500, 1); + assert.equal((await get(`/render/${ALPHA}.svg`)).status, 500); + assert.equal((await get(`/render/${ALPHA}.svg`)).status, 200); + stub.refuseAll(504, 2); + assert.equal((await get(`/render/${ALPHA}.svg`)).status, 504); + assert.equal((await get(`/render/${BETA}.svg`)).status, 504); + assert.equal((await get(`/render/${BETA}.svg`)).status, 200); + }); + + test('answers its health as set, and runs a hook as each figure is drawn', async () => { + assert.deepEqual(await (await get('/health')).json(), { + build: '000000000000', + files: 'local', + }); + const drawn: string[] = []; + stub.onRender((stId) => { + drawn.push(stId); + stub.health.build = 'abcdefabcdef'; + }); + await (await get(`/render/${REACTION}.svg?layout=reaction`)).text(); + assert.deepEqual(drawn, [REACTION]); + assert.equal( + ((await (await get('/health')).json()) as { build: string }).build, + 'abcdefabcdef' + ); + stub.reset(); + assert.equal( + ((await (await get('/health')).json()) as { build: string }).build, + '000000000000' + ); + }); + + test('silenced, it answers nothing, its health included; slowed, it answers late', async () => { + stub.slow(150); + const started = Date.now(); + assert.equal((await get(`/render/${ALPHA}.svg`)).status, 200); + assert.ok(Date.now() - started >= 140); + stub.silence(); + for (const path of ['/health', `/render/${ALPHA}.svg`]) { + await assert.rejects(get(path, AbortSignal.timeout(300)), { name: 'TimeoutError' }); + } + }); }); test( diff --git a/test/stubs/render.ts b/test/stubs/render.ts index c9b8c1c..5ed93f5 100644 --- a/test/stubs/render.ts +++ b/test/stubs/render.ts @@ -3,9 +3,12 @@ // answers (tools/render/service.ts): its type, a 400 for another format or a parameter // given twice, a JSON error body, and Retry-After on a 503. It answers 400 to a query // parameter the contract does not name, so a wrongly named one fails a test instead of -// passing silently, and refuses on request: 404, 503, a timeout, a body that stops part way, -// or an answer that is not SVG. A test can also plant text in a figure's first label. Its -// `/health` answers as the service's does. +// passing silently, and refuses on request: 404, 500, 503, 504, a timeout, a body that stops +// part way, or an answer that is not SVG, for one figure or every one. A test can also plant +// text in a figure's first label, slow every answer, or silence the stub (every request held +// unanswered, `/health`'s too). Its `/health` answers as the service's does, its `build` and +// `files` settable, also from a hook run as each figure request is answered (T046), so a test can +// change the deployment between a figure's render and the read after it. import { readFileSync, existsSync } from 'node:fs'; import { createServer, type Server } from 'node:http'; import type { AddressInfo } from 'node:net'; @@ -23,7 +26,13 @@ const PARAMETERS: Readonly boolean>> = { * How to refuse: a status, no answer at all, an answer whose body stops part way, or an * answer that is not SVG. */ -export type Refusal = 404 | 503 | 'timeout' | 'stall-body' | 'not-svg'; +export type Refusal = 404 | 500 | 503 | 504 | 'timeout' | 'stall-body' | 'not-svg'; + +/** The service's health answer (contracts/render.md). */ +export interface Health { + build: string; + files: string; +} export interface RenderStub { /** The base URL: requests go to `${url}/render/{stId}.svg`. */ @@ -32,9 +41,25 @@ export interface RenderStub { requests: string[]; /** Refuse requests for a stable id, `times` times (every time without it). */ refuse(stId: string, how: Refusal, times?: number): void; + /** Refuse requests for every figure, `times` times in all (every time without it). */ + refuseAll(how: Refusal, times?: number): void; + /** The health answer, changed in place by a test; its build a placeholder at the start. */ + health: Health; + /** + * Run `hook` with each figure request's stable id just before its answer is sent: a drawing + * or a refusal (`refused` true), a held one's as it is held. + */ + onRender(hook: ((stId: string, refused: boolean) => void) | null): void; + /** Hold every request from now on unanswered, `/health`'s too: a service gone silent. */ + silence(): void; + /** Wait `ms` before each figure's answer, a refusal's too. */ + slow(ms: number): void; /** Put `text` at the start of the first label of every figure drawn for a stable id. */ label(stId: string, text: string): void; - /** Forget the refusals, the planted labels and the recorded requests, for the next test. */ + /** + * Forget the refusals, the planted labels, the hook, the slowness, the silence, the health + * set and the recorded requests, and drop every answer held or delayed, for the next test. + */ reset(): void; close(): Promise; } @@ -67,21 +92,31 @@ function figure( export async function startRenderStub(): Promise { const requests: string[] = []; const refusals = new Map(); + let every: { how: Refusal; left: number } | null = null; const labels = new Map(); const hanging = new Set(); + const health: Health = { build: '000000000000', files: 'local' }; + let hook: ((stId: string, refused: boolean) => void) | null = null; + let silent = false; + let delayMs = 0; + const delayed = new Map(); const server: Server = createServer((req, res) => { const url = new URL(req.url ?? '/', 'http://stub'); requests.push(url.pathname + url.search); + if (silent) { + hanging.add(res); + return; + } const error = (status: number, message: string, headers: Record = {}) => res .writeHead(status, { 'Content-Type': 'application/json; charset=utf-8', ...headers }) .end(JSON.stringify({ error: message })); if (url.pathname === '/health') { - // the service's health answer (contracts/render.md), its build a placeholder + // the service's health answer (contracts/render.md) res .writeHead(200, { 'Content-Type': 'application/json; charset=utf-8' }) - .end(JSON.stringify({ build: '000000000000', files: 'local' })); + .end(JSON.stringify(health)); return; } const match = /^\/render\/(R-[A-Z]{3}-\d+)\.([a-z]+)$/.exec(url.pathname); @@ -107,9 +142,41 @@ export async function startRenderStub(): Promise { return; } } - const refusal = refusals.get(stId); - if (refusal && refusal.left !== 0) { + if (delayMs > 0) { + const timer = setTimeout(() => { + delayed.delete(timer); + if (silent) hanging.add(res); + else answer(stId, url.searchParams, res, error); + }, delayMs); + delayed.set(timer, res); + return; + } + answer(stId, url.searchParams, res, error); + }); + + /** Every answer held or delayed, dropped, so none reaches a later test. */ + const drop = () => { + for (const [timer, res] of delayed) { + clearTimeout(timer); + res.destroy(); + } + delayed.clear(); + for (const res of hanging) res.destroy(); + hanging.clear(); + }; + + /** A figure's answer: its refusal, if it has one left, or its drawing. */ + const answer = ( + stId: string, + query: URLSearchParams, + res: import('node:http').ServerResponse, + error: (status: number, message: string, headers?: Record) => void + ) => { + const own = refusals.get(stId); + const refusal = own && own.left !== 0 ? own : every && every.left !== 0 ? every : undefined; + if (refusal) { refusal.left -= 1; + hook?.(stId, true); if (refusal.how === 'timeout') { hanging.add(res); return; @@ -125,16 +192,19 @@ export async function startRenderStub(): Promise { return; } if (refusal.how === 503) error(503, 'Busy', { 'Retry-After': '5' }); + else if (refusal.how === 500) error(500, `The page failed on ${stId}`); + else if (refusal.how === 504) error(504, `No drawing in time for ${stId}`); else error(404, `No figure for ${stId}`); return; } - const svg = figure(stId, url.searchParams, labels.get(stId)); + const svg = figure(stId, query, labels.get(stId)); if (svg === undefined) { error(404, `No figure for ${stId}`); return; } + hook?.(stId, false); res.writeHead(200, { 'Content-Type': 'image/svg+xml; charset=utf-8' }).end(svg); - }); + }; await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)); const { port } = server.address() as AddressInfo; @@ -145,18 +215,37 @@ export async function startRenderStub(): Promise { refuse(stId, how, times = -1) { refusals.set(stId, { how, left: times }); }, + refuseAll(how, times = -1) { + every = { how, left: times }; + }, + health, + onRender(next) { + hook = next; + }, + silence() { + silent = true; + }, + slow(ms) { + delayMs = ms; + }, label(stId, text) { labels.set(stId, text); }, reset() { + drop(); refusals.clear(); + every = null; labels.clear(); + hook = null; + silent = false; + delayMs = 0; + Object.assign(health, { build: '000000000000', files: 'local' }); requests.length = 0; }, async close() { // A timed-out request's response is held open; close it with the connections, so the // tests end even when a client never gave up on one. - for (const res of hanging) res.destroy(); + drop(); server.closeAllConnections(); await new Promise((resolve) => server.close(() => {