From e4b1be4df51ae994c4fdd9ee55d6bd8e007bc85e Mon Sep 17 00:00:00 2001 From: Adam Wright Date: Tue, 3 Nov 2026 20:43:36 +0000 Subject: [PATCH 1/2] Spec 001 T028: the book's contents numbered, its front part printed again reprintFront reads each probe's page in the joined PDF, prints the front part again with the numbers and swaps its pages in before the links are rewritten and the pages stamped, deleting what only the first print held. A reprint whose pages or front headings move is refused. From #148's review: the stamp-twice guard is held before the first await; marginText collapses only CSS's collapsible white space; a name leaving no header is refused; the join measurement embeds Roboto's WOFF file; T027's note corrected. Co-Authored-By: Claude Opus 5.5 --- ci/measure/join.ts | 6 +- specs/001-pathway-pdfs/tasks.md | 8 +- src/print/contents.ts | 154 +++++++++- src/print/links.ts | 26 ++ src/print/print.ts | 16 +- src/print/stamp.ts | 36 ++- test/print/contents.test.ts | 500 ++++++++++++++++++++++++++++++++ test/print/print.test.ts | 4 +- test/print/stamp.test.ts | 49 +++- 9 files changed, 759 insertions(+), 40 deletions(-) create mode 100644 test/print/contents.test.ts diff --git a/ci/measure/join.ts b/ci/measure/join.ts index 05eb236..5f63484 100644 --- a/ci/measure/join.ts +++ b/ci/measure/join.ts @@ -215,13 +215,15 @@ async function join(job: JoinJob, report: (m: JoinMessage) => void): Promise { doc.registerFontkit(fontkit); const file = path.join( path.dirname(new URL(import.meta.resolve('@fontsource/roboto/package.json')).pathname), 'files', - 'roboto-latin-400-normal.woff2' + 'roboto-latin-400-normal.woff' ); const font = await doc.embedFont(readFileSync(file), { subset: true }); const hex = themeVariables().get('--mat-sys-on-surface-variant') ?? ''; diff --git a/specs/001-pathway-pdfs/tasks.md b/specs/001-pathway-pdfs/tasks.md index e7fb471..3fff00a 100644 --- a/specs/001-pathway-pdfs/tasks.md +++ b/specs/001-pathway-pdfs/tasks.md @@ -448,13 +448,17 @@ passes. - **Done (#148, replacing #147 after its review):** `stampPages` (`src/print/stamp.ts`) stamps each page of a joined PDF, in place, in `printBook`'s `joined` callback after T026's `rewriteLinks` (whose targets give where the front parts' headings landed), `await stampPages(joined, rewriteLinks(pdf))` beside `writeOutline`, which T029 wires into `book`. `pageStamps` gives each page its number and header from the parts' page counts: the PDF's pages numbered from the first, across its parts, the cover's included, as a single PDF's `counter(page)` does, so the introduction is page 2; the pages before the introduction's heading are the cover's, with neither; each front page is named for the front part begun on or before it (`FRONT_NAMES`, now in `src/layout/front.ts` and shared with the single PDF's margin boxes in `document.ts`); every page after the front part carries the top-level pathway's name (`doc.cover.title`) after FR-017's substitutions (`substituted`, added to `src/layout/characters.ts`: the list alone, uncounted, as the name is counted where its section prints it). Each stamp is set as the single PDF's margin box sets it: the same text (white space collapsed and characters never drawn left out, `marginText`, as the box does; the number alone, not T011's "12 of 3,760"; research R4 now says so), in runs by the face the box's font list (`MARGIN_FAMILIES`, now shared with `document.ts`'s margin boxes: Roboto, then the Noto fonts, at weight 400) gives each character, each face embedded with pdf-lib as a subset of its WOFF file, not the WOFF2 the pages load (below), named with a six-capital tag from its face and characters (`ABCDEF+Roboto-Regular`, not pdf-lib's random suffix, so the same stamps write the same names); the theme's body-small size (9 pt) and on-surface-variant colour, read from the compiled theme's stylesheet; centred on each page's own width, a landscape page's included; its baseline where Chromium puts a margin box's line: the first face's `normal` line (Roboto's hhea metrics) centred in the 20 mm margin, 31.42 pt from the top edge and 25.27 from the bottom, where the single PDF's are 31.50 and 25.17 (24.96 on a landscape page), Chromium rounding to its pixels. Refused, stamping nothing: a PDF stamped already (held in memory, not marked in the file, as a stamped PDF is saved and done with), parts whose pages are not the PDF's, a front part with no introduction probe or a front heading outside the front part, and a character no face draws (named by code point). Each face is embedded once in a PDF, and on each page its runs are drawn together with the font set once, as pdf-lib gives the page another name for a font each time it is set. There is no shaping: no kerning, ligatures, bidirectional text or joined scripts (Arabic); a stamp is a pathway's name, a front part's or a number, and today's need none. - **Tests (#147):** `test/print/stamp.test.ts` (11) prints the fixture's book with the threshold at 1 page, as T025's and T026's do (each PDF a front part of 3 pages and three parts of sections), rewrites its links, writes its outline and stamps it, reading the pages back with pdf.js: every page's number its place in the PDF, across the parts, none on the cover; the cover unchanged; each page's header the front part's whose heading (read from the printed pages, not the probes) begins on or before it, then the pathway's, checked either side of every part boundary; every header and number at the single PDF's margin boxes' heights (within 0.5 pt) and size, centred within 1 pt, on Gamma's landscape pages too; on every page one font more, `^[A-Z]{6}\+Roboto-Regular$`, its embedded `FontFile2` a subset (under 40 glyphs) and every fill colour the stamps set the theme's on-surface-variant, at 9 pt; a name planted in the test, `Alpha‑pathway ‑ one` with U+2011, stamped `Alpha-pathway - one`; a name in Greek, Cyrillic and an arrow drawn in three or more faces, a Noto one among them, read back whole and centred; a character no face draws (U+E000) and parts that are not the PDF's refused, nothing stamped; and `pageStamps` and `stampLook` alone. `test/layout/characters.test.ts` adds `substituted`'s test. 303 tests before, 315 after. Each of these mutants made one fail: each part numbered on its own (the numbers test, and `pageStamps` alone); the first section page given the front part's header (the header test, the planted-name tests, and `pageStamps` alone); every front page given the first front part's name (the header test, and `pageStamps` alone); the cover numbered (the numbers, cover and font tests); the name stamped as it is (the U+2011 test, which then refuses U+2011 as no face draws it); every character in Roboto's Latin face (the several-faces test); the baseline at the margin's middle (the placement test); centring on the portrait width (the placement test, on the landscape pages); the theme's on-surface colour (the font and colour test, and `stampLook` alone); the whole font embedded (the font test); a font given to every `drawText`, two names on a page (the font test); and the parts' pages unchecked (the refusal test). These tests read the text layer only, which is how the stamps of #147 passed them all without being drawn (below). - - **From #147's review:** the stamps were not drawn. Every face was embedded with `subset: true` from its @fontsource WOFF2 file, and fontkit 2.0.4's subsetter copies a WOFF2 file's `glyf` bytes as they are, transformed, so the subset's glyphs had no outlines: the text layer read back (every test above passed), but nothing was painted (Ghostscript at 300 dpi showed no header or number; pdf.js warned `Required "glyf" table is not found`). Each face is now embedded from the WOFF (1) file each @fontsource package ships beside its WOFF2 (`stampFile`, refused if there is none), which fontkit subsets correctly: on the fixture Ghostscript's `bbox` device now finds ink from 25.27 pt to 817.25 pt on a stamped page, 91.82 to 770.67 before stamping. The several faces' subsets are a few hundred bytes for two glyphs each (Noto Sans Math's was 135 KB for two from WOFF2). **T011's stamping measurement** (research R4) embedded the same broken WOFF2 subsets, so its stamp stage's size and time may change with outlines in them; T029's run measures it again. A name holding a tab, a newline, a variation selector or another character never drawn (FR-017's `NEVER_DRAWN`, now exported from `characters.ts`) prints in its section but refused the whole PDF at stamping, as no face's character map holds it; `marginText` now collapses white space and leaves those out, as a margin box does, after the substitutions. Each face is embedded once a PDF, and tested so; a PDF stamped twice is refused; `pageStamps`' test covers the analysis front part; the comment says there is no shaping. Tests added: every glyph of the stamps' embedded Roboto, read from its `FontFile2` with fontkit, has an outline, one for each character stamped but the space, and the stamps' fonts are one Roboto a PDF; each stamped page rendered by pdf.js (with `@napi-rs/canvas`, pdfjs-dist's optional dependency) has ink in its top and bottom margins, which were blank before stamping, and the cover none, on the first page after the cover and every landscape page; the several faces' page has each face once, each a subset under 10 KB whose glyphs but the space are outlined, and ink in its margins; a name planted with a leading space, a tab, a newline, U+FE0F, U+200D and U+0007 stamped `Alpha pathway one`; a second stamping refused, changing nothing; `marginText`'s cases one by one (tab, newline, CR LF, leading and trailing space, U+FE0F alone and between spaces, U+200D, U+0007, U+00AD, Greek kept); the analysis front part named `Analysis properties`. 316 tests before (main has gained one since #147), 322 after. Each of these mutants made one fail: the WOFF2 file embedded, as in #147 (the outline, ink and several-faces tests); the text drawn invisible, `renderMode` 3 (the ink and several-faces tests); each face embedded again on every page (the outline test, by its one-font-a-PDF check); no `marginText` (the planted white-space test and `marginText`'s); white space not collapsed (the same two); stamping twice not refused (its test); the analysis front part not named (its test); a font given to every `drawText` (the colour and font test, and the several-faces test). Every mutant of #147 still fails as before, more tests catching most: per part numbering (the numbers test, `pageStamps`); the first section page given the front part's header (the header test, the planted-name tests, `pageStamps`, the analysis test); every front page the first front part's (the header test, `pageStamps`, the analysis test); the cover numbered (the numbers, cover, font and ink tests, `pageStamps`); the name without the substitutions (the U+2011 test, and `marginText`'s through `pageStamps`); every character in Roboto Latin (the several-faces test); the baseline at the margin's middle, and centring on the portrait width (the placement test); the theme's on-surface colour (the colour test, `stampLook`); the whole font embedded (the font, outline and several-faces tests); the parts' pages unchecked (the refusal test). + - **From #147's review:** the stamps were not drawn. Every face was embedded with `subset: true` from its @fontsource WOFF2 file, and fontkit 2.0.4's subsetter copies a WOFF2 file's `glyf` bytes as they are, transformed, so the subset's glyphs had no outlines: the text layer read back (every test above passed), but nothing was painted (Ghostscript at 300 dpi showed no header or number; pdf.js warned `Required "glyf" table is not found`). Each face is now embedded from the WOFF (1) file each @fontsource package ships beside its WOFF2 (`stampFile`, refused if there is none), which fontkit subsets correctly: on the fixture Ghostscript's `bbox` device now finds ink from 25.27 pt to 817.25 pt on a stamped page, 91.82 to 770.67 before stamping. The several faces' subsets are a few hundred bytes for two glyphs each (Noto Sans Math's was 135 KB for two from WOFF2). **T011's stamping measurement** (research R4) embedded the same broken WOFF2 subsets, so its stamp stage's size and time may change with outlines in them; T029's run measures it again. A name holding a tab, a newline, a variation selector or another character never drawn (FR-017's `NEVER_DRAWN`, now exported from `characters.ts`) prints in its section but refused the whole PDF at stamping, as no face's character map holds it; `marginText` now collapses white space and leaves those out, close to a margin box, after the substitutions. Each face is embedded once a PDF, and tested so; a PDF stamped twice is refused; `pageStamps`' test covers the analysis front part; the comment says there is no shaping. Tests added: every glyph of the stamps' embedded Roboto, read from its `FontFile2` with fontkit, has an outline, one for each character stamped but the space, and the stamps' fonts are one Roboto a PDF; the cover, the first page after it and every landscape page, rendered by pdf.js (with `@napi-rs/canvas`, pdfjs-dist's optional dependency; no other page is rendered): each but the cover has ink in its top and bottom margins, which were blank before stamping, and the cover none; the several faces' page has each face once, each a subset under 10 KB whose glyphs but the space are outlined, and ink in its margins; a name planted with a leading space, a tab, a newline, U+FE0F, U+200D and U+0007 stamped `Alpha pathway one`; a second stamping refused, changing nothing; `marginText`'s cases one by one (tab, newline, CR LF, leading and trailing space, U+FE0F alone and between spaces, U+200D, U+0007, U+00AD, Greek kept); the analysis front part named `Analysis properties`. 316 tests before (#147's own count, 315 after it, was one short), 322 after. Each of these mutants made one fail: the WOFF2 file embedded, as in #147 (the outline, ink and several-faces tests); the text drawn invisible, `renderMode` 3 (the ink and several-faces tests); each face embedded again on every page (the outline test, by its one-font-a-PDF check); no `marginText` (the planted white-space test and `marginText`'s); white space not collapsed (the same two); stamping twice not refused (its test); the analysis front part not named (its test); a font given to every `drawText` (the colour and font test, and the several-faces test). Every mutant of #147 still fails as before, more tests catching most: per part numbering (the numbers test, `pageStamps`); the first section page given the front part's header (the header test, the planted-name tests, `pageStamps`, the analysis test); every front page the first front part's (the header test, `pageStamps`, the analysis test); the cover numbered (the numbers, cover, font and ink tests, `pageStamps`); the name without the substitutions (the U+2011 test, and `marginText`'s through `pageStamps`); every character in Roboto Latin (the several-faces test); the baseline at the margin's middle, and centring on the portrait width (the placement test); the theme's on-surface colour (the colour test, `stampLook`); the whole font embedded (the font, outline and several-faces tests); the parts' pages unchecked (the refusal test). -- [ ] T028 [US1] The book's contents, second pass: T019's `src/print/contents.ts`, extended to read where each section landed in each joined PDF and reprint only that PDF's front part. The reprinted front part is swapped in before T026's links and T027's stamping run, so its links are rewritten and its pages stamped like the rest. Tests: every entry's number is its target's page, across parts; the contents' links land; the front pages carry their headers and numbers. +- [x] T028 [US1] The book's contents, second pass: T019's `src/print/contents.ts`, extended to read where each section landed in each joined PDF and reprint only that PDF's front part. The reprinted front part is swapped in before T026's links and T027's stamping run, so its links are rewritten and its pages stamped like the rest. Tests: every entry's number is its target's page, across parts; the contents' links land; the front pages carry their headers and numbers. + - **Done (#PR):** `reprintFront` (`src/print/contents.ts`) runs first in `printBook`'s `joined` callback, before `rewriteLinks`, `writeOutline` and `stampPages` (T029 wires the four into `book`, in that order, as its task now says). It reads each probe's page in the joined PDF without changing anything (`probeTargets`, added to `src/print/links.ts`, refusing an anchor probed twice), numbers each contents entry from its target's page counted from the PDF's first (the cover's included, as the stamps count, T027), lays the front part out again (`layoutBookFront`), fills the placeholders (T019's `fillContents`), prints it through the same `PartPrinter` and swaps its pages in for the first print's (`removePage`, `insertPage`). The objects only the first print's front pages held are deleted (pdf-lib's `removePage` deletes a page's own object, not its contents, resources or annotations, which saving would write, placeholders and probes among them): those reachable from the old pages, not through `/Parent`, less what is still reachable from the trailer, so nothing the PDF uses goes even if a page shared it. That second walk covers the whole PDF, linear in its objects (inference: tens of milliseconds to about a second for the largest PDF; T029 measures it). Refused before anything is changed, each naming what is wrong: a first part that is not the front part, parts whose pages are not the PDF's, an entry whose target has no probe (as after `rewriteLinks`), an anchor probed twice, and a reprint whose pages differ from the first print's, in their count or a front heading's page (T019's `movedPages`, moved here from `print.ts` for both passes): every later page and number would move with it, so it is refused rather than handled (the numbers sit in boxes of fixed width, T019, so on the fixture nothing moves). The reprint's layout report is dropped: the first print's is the PDF's, so nothing in the front part is counted twice (T029's character test). The reprint holds the join's slots, as `joined` does. + - **Tests:** `test/print/contents.test.ts` (7) prints the fixture's book with the threshold at 1 page, as T025's to T027's do, and in `joined` saves each PDF as joined, reprints its front part, saves it, rewrites its links, writes its outline and stamps it: every contents number, read from the finished PDF's contents pages, is the page on which the entry's title is first a heading (found in the text, not from the probes) and that page's stamped number, the targets spread over the front part and at least two parts of sections, every number a placeholder `0` as joined; only the front part printed again, once, every other part once, the swapped PDF's pages as joined but for the front part's numbers; a reprint a page longer (a block pushing the introduction on) and one of the same length with its headings moved (its cover moved to its end) refused, naming the pages and the anchors, the PDF unchanged, and an entry with no probe, the parts not the PDF's (each way) and an anchor probed twice refused; the reprinted front part's contents links, matched by page and rectangle, landing on their headings, one for each entry, the outline the contents list, each entry's destination its link's, and no placeholder or probe address left in the file; the front pages stamped, the cover bare, `Introduction` and `Contents` and their numbers; an object the first print's front shares with a later page (planted, as parts share none) kept while its own annotations go; and no object left that nothing reaches, as joined, swapped and finished. 322 tests before, 330 after (8: these 7, and the stamping race below). Each of these mutants made one fail: no swap (the numbers, reprint and shared-object tests); the front part printed again without its numbers (the numbers and reprint tests); every number a page low (the numbers test); the first print's objects left in the file (the links test, by the placeholder address left, the shared-object and nothing-left tests); every object the old pages reach deleted, live or not (the shared-object test); moved pages not refused, and only the page count compared (the refusal test, each); entries with no probe not refused, the parts' pages unchecked, an anchor probed twice not refused (the refusal test, each); the reprinted pages inserted in reverse (every test, through the setup's run failing). In the shared-object test, an `assert.equal` of a found object against `undefined` never finished printing the object (the run ended with a promise pending), so it asserts with `assert.ok`. + - **From #148's review:** the stamp-twice guard raced (`stamped.add` came after an `await`, so two stampings at once both passed); the PDF is now held once its stamps are checked, before the first `await`, and a test stamps it twice at once: one stamps, the other is refused, the pages as stamped once. `marginText` collapsed JavaScript's `\s`, which takes in a no-break, thin or ideographic space that Chromium keeps; it now collapses only space, tab, line feed, carriage return and form feed, leaves out only controls and default-ignorable characters (FR-017's `NEVER_DRAWN` but its white space), trims only spaces, and is described as close to a margin box, not as one. A name that leaves no header (all white space and characters never drawn) is refused, naming it, not stamped bare. `ci/measure/join.ts` now embeds Roboto's WOFF file, as `stamp.ts` does, so its stamp stage draws outlines. T027's note is corrected (its test count, and which pages are rendered). Each of these mutants made one fail: the guard after the first `await` (the race test, and the stamp-twice test); `\s` collapsed, `trim()`, `NEVER_DRAWN` left out whole, and a form feed not collapsed (`marginText`'s test, each); the empty name not refused (`marginText`'s test through `pageStamps`, and the refusal test). - [ ] T029 [US1] `book` end to end, `src/cli.ts` (contracts/cli.md). - **Flags and their environment variables** (contracts/cli.md): `--graph` (`NEO4J_URI`), `--database` (`NEO4J_DATABASE`), `--figures`, `--site` (`REACTOME_SITE`), `--files`, `--species`, `--out`, `--parallel`, `--joins` and `--report`. Credentials come from `NEO4J_USER` and `NEO4J_PASSWORD`, never the command line. - **Writing:** `--out` must be absent or empty (exit 2); the folder is written beside its name and moved into place; each PDF is named as today's (the pathway's name and `.pdf`). - **Memory (research R4, T011):** the join's memory, object streams on and off, measured on the real book; the join limit's default (1, raised to 2 only if two real joins fit beside printing), given a flag `--joins` in contracts/cli.md, and the object streams' setting chosen from it; if one real join does not fit beside printing, T012 is done before T029 closes. + - **What is done with each joined PDF,** in `printBook`'s `joined` callback, in this order: T028's `reprintFront` (it reads the probes, which `rewriteLinks` removes), then `rewriteLinks`, `writeOutline` and `stampPages`. - **A refused PDF is discarded:** T026's `rewriteLinks` and `writeOutline` change the joined PDF in place and do not undo a part done when they throw (an outline refused leaves its links rewritten), so on any throw in `joined` the PDF is not saved, and the run fails as a run does. - **The run report:** the release; the characters FR-017's list replaced, by code point and count; each stage's time; the page count and size of each PDF and in all; figures by kind; the figures whose labels print below 1 pt (research R4); missing figures and missing release files; and each FR-011 check's result. Everything is named by stable identifier, never a person. diff --git a/src/print/contents.ts b/src/print/contents.ts index 24d282c..7517242 100644 --- a/src/print/contents.ts +++ b/src/print/contents.ts @@ -1,11 +1,36 @@ -// The contents list's page numbers (spec 001 T019, research R4): the first print carries a -// placeholder in a box of fixed width for each entry; where each target landed is read back -// from that print, and the numbers go into the same boxes, so nothing else moves and the -// second print's pages are the first's. Chromium writes a named destination for each -// anchor a link points at, and every entry links to its target, so each target's page is -// its destination's. +// The contents list's page numbers (spec 001 T019, T028, research R4): the first print +// carries a placeholder in a box of fixed width for each entry; where each target landed is +// read back, and the numbers go into the same boxes, so nothing else moves and the second +// print's pages are the first's. +// +// A single event's PDF (T019) is printed whole: Chromium writes a named destination for +// each anchor a link points at, and every entry links to its target, so each target's page +// is its destination's. +// +// A book's PDF (T028) is printed in parts and joined, its links placeholders: each target's +// page is read from its probe in the joined PDF, as T026's links are, the front part alone +// is printed again with the numbers, and its pages are swapped for the first print's in the +// joined PDF, before its links are rewritten and its pages stamped, so the new front pages +// are linked and stamped as the rest are. The front part must print the same pages, its +// own headings on the same pages, or every later page and number would move: a reprint +// that does not is refused. The reprint's layout report is not kept: the first print's is +// the PDF's, so nothing in the front part is counted twice. +import { + PDFArray, + PDFDict, + PDFDocument, + PDFName, + PDFRef, + PDFStream, + type PDFContext, + type PDFObject, +} from '@cantoo/pdf-lib'; import { getDocument } from 'pdfjs-dist/legacy/build/pdf.mjs'; +import { layoutBookFront, type LayoutOptions } from '../layout/document.ts'; +import { probeTargets, type Target } from './links.ts'; +import type { Joined, PartPrinter } from './parts.ts'; + /** An entry's placeholder, as the layout writes it (src/layout/front.ts). */ const PLACEHOLDER = /[^<]*<\/span>/g; @@ -58,3 +83,120 @@ export function fillContents(html: string, pages: ReadonlyMap): return whole.replace(/>[^<]*<\/span>$/, `>${String(page)}`); }); } + +/** + * How a second print's pages differ from the first's, or null: another page count, or a + * target on another page. Either means the numbers printed are not the pages. + */ +export function movedPages( + first: Landed, + second: Landed, + targets: readonly string[] +): string | null { + const moved = targets.filter((t) => second.pages.get(t) !== first.pages.get(t)); + if (second.pageCount === first.pageCount && moved.length === 0) return null; + return `${String(first.pageCount)} pages, then ${String(second.pageCount)}; moved: ${moved.join(', ') || 'none'}`; +} + +const PARENT = PDFName.of('Parent'); + +/** + * Every object reachable by reference from those given; through `/Parent` only when asked, + * as a page's leads to the whole page tree. + */ +function reachable(context: PDFContext, from: readonly PDFObject[], parents: boolean): Set { + const seen = new Set(); + const stack = [...from]; + for (let object = stack.pop(); object !== undefined; object = stack.pop()) { + if (object instanceof PDFRef) { + if (seen.has(object)) continue; + seen.add(object); + const found = context.lookup(object); + if (found !== undefined) stack.push(found); + } else if (object instanceof PDFDict) { + for (const [key, value] of object.entries()) { + if (parents || key !== PARENT) stack.push(value); + } + } else if (object instanceof PDFArray) { + for (const value of object.asArray()) stack.push(value); + } else if (object instanceof PDFStream) { + stack.push(object.dict); + } + } + return seen; +} + +/** Each probed anchor's page, counted from 1, for those on the first `count` pages. */ +function pagesOf(targets: ReadonlyMap, count: number): Map { + const pages = new Map(); + for (const [anchor, target] of targets) { + if (target.index < count) pages.set(anchor, target.index + 1); + } + return pages; +} + +/** + * A joined book PDF's contents numbered (T028): each entry's target's page read from its + * probe, the front part printed again with the numbers, and its pages put in place of the + * first print's, in place, the first print's objects deleted with them. Runs before + * `rewriteLinks`, which removes the probes, and so before `stampPages`. Throws, changing + * nothing, when the first part is not the front part or the parts' pages are not the PDF's, + * an entry's target has no probe, or the reprint's pages differ from the first print's (its + * page count, or a front heading on another page), which would move every later page. + */ +export async function reprintFront( + joined: Pick, + printer: PartPrinter, + layout: LayoutOptions +): Promise { + const { doc, pdf, parts } = joined; + const front = parts[0]; + if (!front || front.places.length > 0) { + throw new Error("The joined PDF's first part is not its front part"); + } + const counted = parts.reduce((n, part) => n + part.pages, 0); + if (counted !== pdf.getPageCount()) { + throw new Error( + `The parts hold ${String(counted)} pages, the joined PDF ${String(pdf.getPageCount())}` + ); + } + const html = layoutBookFront(doc, layout).html; + const targets = contentsTargets(html); + if (targets.length === 0) return; + const probes = probeTargets(pdf); + const unprobed = targets.filter((target) => !probes.has(target)); + if (unprobed.length > 0) { + throw new Error( + `The contents' entries with no probe in the joined PDF (run before rewriteLinks): ${unprobed.join(', ')}` + ); + } + const reprinted = await PDFDocument.load( + await printer.part(fillContents(html, pagesOf(probes, counted))) + ); + // the front part's own headings, where they landed the first time and the second + const first = { pages: pagesOf(probes, front.pages), pageCount: front.pages }; + const pageCount = reprinted.getPageCount(); + const second = { pages: pagesOf(probeTargets(reprinted), pageCount), pageCount }; + const anchors = [...new Set([...first.pages.keys(), ...second.pages.keys()])]; + const problem = movedPages(first, second, anchors); + if (problem) throw new Error(`The front part's numbers moved its pages: ${problem}`); + // the first print's page dictionaries: pdf-lib's removePage deletes a page's own object, + // not what it refers to (its contents, resources, annotations), which saving would write + const old = pdf + .getPages() + .slice(0, front.pages) + .map((page) => page.node); + const copied = await pdf.copyPages(reprinted, reprinted.getPageIndices()); + copied.forEach((page, i) => { + pdf.removePage(i); + pdf.insertPage(i, page); + }); + // what only the first print's pages held is deleted, never what the PDF still uses + const { context } = pdf; + const { Root, Info, Encrypt } = context.trailerInfo; + const roots = [Root, Info, Encrypt].filter((o) => o !== undefined); + const live = reachable(context, roots, true); + for (const ref of reachable(context, old, false)) { + if (!live.has(ref)) context.delete(ref); + } +} diff --git a/src/print/links.ts b/src/print/links.ts index ca7004c..1c746b6 100644 --- a/src/print/links.ts +++ b/src/print/links.ts @@ -54,6 +54,32 @@ export function destination(pdf: PDFDocument, target: Target): PDFArray { return dest; } +/** + * Where each anchor's probe is in a PDF, read without changing anything: T028 numbers the + * contents from them before `rewriteLinks` runs. Throws naming every anchor probed twice. + */ +export function probeTargets(pdf: PDFDocument): Map { + const targets = new Map(); + const twice = new Set(); + pdf.getPages().forEach((page, index) => { + const annots = page.node.Annots(); + for (let i = 0; i < (annots?.size() ?? 0); i++) { + const annot = pdf.context.lookup(annots?.get(i)); + const uri = annot instanceof PDFDict ? uriOf(annot) : null; + if (!(annot instanceof PDFDict) || !uri?.startsWith(PROBE)) continue; + const anchor = decodeURIComponent(uri.slice(PROBE.length)); + if (targets.has(anchor)) twice.add(anchor); + targets.set(anchor, { page: page.ref, index, y: topOf(annot) }); + } + }); + if (twice.size > 0) { + throw new Error( + `The PDF's probes: ${[...twice].map((a) => `${a} is probed twice`).join('; ')}` + ); + } + return targets; +} + /** * Each placeholder link in a joined PDF made a destination at its anchor's probe, and the * probes removed. Returns each anchor's target, for the outline. Throws, changing nothing, diff --git a/src/print/print.ts b/src/print/print.ts index e392c00..49773ea 100644 --- a/src/print/print.ts +++ b/src/print/print.ts @@ -11,21 +11,7 @@ import { pathToFileURL } from 'node:url'; import { chromium, type Browser } from 'playwright-core'; -import { contentsTargets, fillContents, landed, type Landed } from './contents.ts'; - -/** - * How a second print's pages differ from the first's, or null: another page count, or a - * target on another page. Either means the numbers printed are not the pages. - */ -export function movedPages( - first: Landed, - second: Landed, - targets: readonly string[] -): string | null { - const moved = targets.filter((t) => second.pages.get(t) !== first.pages.get(t)); - if (second.pageCount === first.pageCount && moved.length === 0) return null; - return `${String(first.pageCount)} pages, then ${String(second.pageCount)}; moved: ${moved.join(', ') || 'none'}`; -} +import { contentsTargets, fillContents, landed, movedPages } from './contents.ts'; export class Printer { readonly #browser: Browser; diff --git a/src/print/stamp.ts b/src/print/stamp.ts index de70fa1..2236dcf 100644 --- a/src/print/stamp.ts +++ b/src/print/stamp.ts @@ -8,8 +8,10 @@ // where the box puts it. The cover has neither. A top-level pathway's name is stamped, // not laid out, so FR-017's substitutions are made here; they are not counted in the run // report, which counts the name where its section prints it. White space is collapsed and -// characters never drawn are left out, as a margin box does, so a stray tab or variation -// selector that prints in a section does not refuse the PDF here. +// characters never drawn are left out, close to a margin box, so a stray tab or variation +// selector that prints in a section does not refuse the PDF here. Only CSS's collapsible +// white space is collapsed (space, tab, line feed, carriage return, form feed): a +// no-break, thin or ideographic space is kept, as Chromium keeps it. // // Each face is embedded from its WOFF file, not the WOFF2 the pages load: fontkit's // subsetter copies a WOFF2 file's `glyf` bytes as they are, transformed, and the subset's @@ -23,7 +25,7 @@ import { existsSync, readFileSync } from 'node:fs'; import { ColorTypes, type PDFDocument, type PDFFont, type RGB } from '@cantoo/pdf-lib'; import * as fontkit from 'fontkit'; -import { NEVER_DRAWN, substituted } from '../layout/characters.ts'; +import { substituted } from '../layout/characters.ts'; import { MARGIN_FAMILIES, pageStyles } from '../layout/document.ts'; import { PAGE } from '../layout/figure.ts'; import { codePointName, fontFaces, type FontFace } from '../layout/fonts.ts'; @@ -42,15 +44,24 @@ export interface FrontTarget { } /** - * A text as a margin box draws it: each run of white space one space, characters never - * drawn (controls, variation selectors, joiners; FR-017's) left out, and trimmed. + * What a margin box leaves out: FR-017's characters never drawn (`NEVER_DRAWN`) but its + * white space, which is collapsed or kept (below), so controls and default-ignorable + * characters (variation selectors, joiners, a soft hyphen). + */ +const LEFT_OUT = /[\p{Cc}\p{Default_Ignorable_Code_Point}]/u; + +/** + * A text close to how a margin box draws it: each run of collapsible white space (space, + * tab, line feed, carriage return, form feed; not JavaScript's `\s`, which takes in a + * no-break, thin or ideographic space that Chromium keeps) one space, `LEFT_OUT`'s + * characters left out, and a space at either end trimmed. */ export function marginText(text: string): string { let out = ''; - for (const char of text.replace(/\s+/g, ' ')) { - if (char === ' ' || !NEVER_DRAWN.test(char)) out += char; + for (const char of text.replace(/[ \t\n\r\f]+/g, ' ')) { + if (char === ' ' || !LEFT_OUT.test(char)) out += char; } - return out.replace(/ {2,}/g, ' ').trim(); + return out.replace(/ {2,}/g, ' ').replace(/^ | $/g, ''); } /** @@ -82,6 +93,9 @@ export function pageStamps( throw new Error(`Front parts outside the front part's pages 2–${String(frontPages)}: ${named}`); } const header = marginText(substituted(name)); + if (header === '') { + throw new Error(`The pathway's name ${JSON.stringify(name)} leaves no header to stamp`); + } const stamps: (PageStamp | null)[] = []; for (let i = 0; i < pages; i++) { const number = String(i + 1); @@ -217,7 +231,8 @@ const stamped = new WeakSet(); * front part is the PDF's first part; `targets` are where its headings landed, as * `rewriteLinks` gives them, so it runs after that. Throws, stamping nothing, when the * PDF is stamped already, the parts' pages are not the PDF's, a front part's heading is - * not in the front part, or a character of a stamp is one no font draws. + * not in the front part, the name leaves no header, or a character of a stamp is one no font + * draws. */ export async function stampPages( joined: Pick, @@ -253,6 +268,8 @@ export async function stampPages( used.set(run.face, chars); } } + // held before the first await, so two stampings of the PDF at once cannot both pass + stamped.add(pdf); pdf.registerFontkit(fontkit); const fonts = new Map(); for (const [face, chars] of used) { @@ -272,7 +289,6 @@ export async function stampPages( const top = (PAGE.margin - line) / 2 + metrics.ascent * em; const bottom = PAGE.margin - top; const { size, colour: color } = look; - stamped.add(pdf); pages.forEach((page, i) => { const stamp = laid[i]; if (!stamp) return; diff --git a/test/print/contents.test.ts b/test/print/contents.test.ts new file mode 100644 index 0000000..f9466b8 --- /dev/null +++ b/test/print/contents.test.ts @@ -0,0 +1,500 @@ +// The book's contents, second pass (spec 001 T028, research R4), on the fixture's book +// printed in parts with the threshold lowered so each PDF splits, as T025's to T027's tests +// do, each joined PDF's front part printed again with the numbers and swapped in before its +// links are rewritten, its outline written and its pages stamped: every entry's number is +// its target's page, across the parts; the front part keeps its length, and a reprint that +// would not is refused; only the front part is printed again; the reprinted front part's +// links land, its pages are stamped, the outline lands with them; and nothing of the first +// print is left in the file. +import assert from 'node:assert/strict'; +import { after, before, describe, test } from 'node:test'; + +import { + PDFArray, + PDFDict, + PDFDocument, + PDFName, + PDFRef, + PDFStream, + type PDFObject, +} from '@cantoo/pdf-lib'; +import { getDocument } from 'pdfjs-dist/legacy/build/pdf.mjs'; + +import { layoutBookFront, pageStyles, type LayoutOptions } from '../../src/layout/document.ts'; +import { PAGE } from '../../src/layout/figure.ts'; +import { PLACEHOLDER, PROBE } from '../../src/layout/html.ts'; +import type { Document } from '../../src/model/types.ts'; +import { reprintFront } from '../../src/print/contents.ts'; +import { rewriteLinks } from '../../src/print/links.ts'; +import { writeOutline } from '../../src/print/outline.ts'; +import { joinParts, printBook, type Part, type PartPrinter } from '../../src/print/parts.ts'; +import { Printer } from '../../src/print/print.ts'; +import { stampPages } from '../../src/print/stamp.ts'; +import { buildFixture, fixtureFigures, skipWithoutGraph, type Built } from '../layout/fixture.ts'; + +type PdfJs = Awaited['promise']>; + +/** A PDF read with pdf.js, let go after. */ +async function reading(pdf: Uint8Array, read: (doc: PdfJs) => Promise): Promise { + const task = getDocument({ data: pdf.slice() }); + try { + return await read(await task.promise); + } finally { + await task.destroy(); + } +} + +interface Item { + str: string; + x: number; + y: number; + size: number; +} + +interface PageRead { + width: number; + height: number; + items: Item[]; + links: { rect: number[]; url: string | null; dest: unknown[] | null }[]; +} + +/** Each page's text items and link annotations. */ +function pagesOf(pdf: Uint8Array): Promise { + return reading(pdf, async (doc) => { + const pages: PageRead[] = []; + for (let i = 1; i <= doc.numPages; i++) { + const page = await doc.getPage(i); + const [, , width = 0, height = 0] = page.view; + const items = (await page.getTextContent()).items.flatMap((item) => { + if (!('str' in item) || item.str.trim() === '') return []; + const [, , c = 0, d = 0, x = 0, y = 0] = item.transform as number[]; + return [{ str: item.str, x, y, size: Math.hypot(c, d) }]; + }); + const links = (await page.getAnnotations()).flatMap((a) => { + const { subtype, rect, url, dest } = a as { + subtype?: unknown; + rect?: number[]; + url?: unknown; + dest?: unknown; + }; + if (subtype !== 'Link' || !rect) return []; + return [ + { + rect, + url: typeof url === 'string' ? url : null, + dest: Array.isArray(dest) ? (dest as unknown[]) : null, + }, + ]; + }); + pages.push({ width, height, items, links }); + } + return pages; + }); +} + +/** A page's running header and number, as stamped: the text in its top and bottom margins. */ +const header = (page: PageRead) => + page.items + .filter((item) => item.y > page.height - PAGE.margin) + .map((item) => item.str) + .join(''); +const footer = (page: PageRead) => + page.items + .filter((item) => item.y < PAGE.margin) + .map((item) => item.str) + .join(''); + +/** + * The contents' numbers as printed, in order: digits at the right of the body of the front + * part's pages from its contents' heading (18 pt) on, the last front part. + */ +function contentsNumbers(pages: readonly PageRead[], front: number): number[] { + const first = pages.findIndex((page) => + page.items.some((item) => item.str === 'Contents' && item.size > 17) + ); + assert.ok(first > 0 && first < front, `the contents' heading on page ${String(first + 1)}`); + return pages + .slice(first, front) + .flatMap((page) => + page.items.flatMap((item) => + /^\d+$/.test(item.str) && + item.y > PAGE.margin && + item.y < page.height - PAGE.margin && + item.x > page.width / 2 + ? [Number(item.str)] + : [] + ) + ); +} + +/** + * The page, counted from 1, where a title is first a heading (above 11 pt: body text and + * contents entries are 10.5) after the cover, found from the text alone. + */ +function headingPage(pages: readonly PageRead[], title: string): number | null { + const i = pages.findIndex( + (page, k) => k > 0 && page.items.some((item) => item.str === title && item.size > 11) + ); + return i < 0 ? null : i + 1; +} + +/** Where a destination lands, and whether the title is a heading just below it (T026's test). */ +async function landing(doc: PdfJs, dest: unknown[], title: string): Promise { + const [ref, kind, , y] = dest; + const page = (await doc.getPageIndex(ref as { num: number; gen: number })) + 1; + if ((kind as { name?: unknown } | undefined)?.name !== 'XYZ' || typeof y !== 'number') { + return 'not an /XYZ destination with a height'; + } + if (page === 1) return 'lands on the cover'; + const items = (await (await doc.getPage(page)).getTextContent()).items; + const heading = items.find((item) => { + if (!('str' in item) || item.str !== title) return false; + const size = Number(item.transform[3]); + const above = y - Number(item.transform[5]); + return size > 11 && above >= 0.9 * size && above <= 1.6 * size; + }); + return heading ? null : `no heading "${title}" just below ${String(y)} on page ${String(page)}`; +} + +/** The indirect objects of a PDF that nothing reaches from its trailer. */ +function unreachable(pdf: PDFDocument): string[] { + const { context } = pdf; + const seen = new Set(); + const { Root, Info } = context.trailerInfo; + const stack: PDFObject[] = [Root, Info].filter((o) => o !== undefined); + for (let object = stack.pop(); object !== undefined; object = stack.pop()) { + if (object instanceof PDFRef) { + if (seen.has(object)) continue; + seen.add(object); + const found = context.lookup(object); + if (found) stack.push(found); + } else if (object instanceof PDFDict) stack.push(...object.values()); + else if (object instanceof PDFArray) stack.push(...object.asArray()); + else if (object instanceof PDFStream) stack.push(object.dict); + } + return context + .enumerateIndirectObjects() + .flatMap(([ref]) => (seen.has(ref) ? [] : [String(ref)])); +} + +/** The pages at which the fixture's parts end, as T025's tests: about three sections. */ +const THRESHOLD = 1; + +interface Made { + doc: Document; + parts: Part[]; + /** Saved as joined; with the front part reprinted; and with links, outline and stamps. */ + joined: Uint8Array; + swapped: Uint8Array; + done: Uint8Array; +} + +describe("the book's contents, second pass", { skip: skipWithoutGraph }, () => { + let built: Built; + let started: Printer | undefined; + let options: LayoutOptions; + const made = new Map(); + /** Every page of HTML printed, in order. */ + const printed: string[] = []; + + before(async () => { + built = await buildFixture(); + const printer = await Printer.start(); + started = printer; + options = { + site: 'https://site.example', + date: new Date('2026-10-30T12:00:00Z'), + figures: fixtureFigures(...built.book), + styles: pageStyles(), + }; + const counting: PartPrinter = { + part: (html) => { + printed.push(html); + return printer.part(html); + }, + }; + await printBook(built.book, { + printer: counting, + layout: options, + settings: { threshold: THRESHOLD, parallel: 2, joins: 1 }, + joined: async (j) => { + const joined = await j.pdf.save({ useObjectStreams: false }); + await reprintFront(j, counting, options); + const swapped = await j.pdf.save({ useObjectStreams: false }); + const targets = rewriteLinks(j.pdf); + writeOutline(j.pdf, j.doc.contents ?? [], targets); + await stampPages(j, targets); + const done = await j.pdf.save({ useObjectStreams: false }); + made.set(j.doc.cover.title, { doc: j.doc, parts: j.parts, joined, swapped, done }); + }, + }); + }); + after(async () => { + await started?.close(); + }); + + const madeOf = (title: string) => { + const found = made.get(title); + if (!found) throw new Error(`${title} was not made`); + return found; + }; + + test("every contents entry's number is its target's page, across the parts", async () => { + assert.deepEqual([...made.keys()].sort(), ['Alpha pathway', 'Gamma pathway', 'beta pathway']); + for (const [name, { doc, parts, joined, done }] of made) { + const front = parts[0]?.pages ?? 0; + const pages = await pagesOf(done); + const entries = doc.contents ?? []; + const expected = entries.map((entry) => headingPage(pages, entry.title)); + assert.ok( + expected.every((page) => page !== null), + `${name}: ${String(expected)}` + ); + assert.deepEqual(contentsNumbers(pages, front), expected, name); + // each number is also the page's stamped number + for (const page of expected) { + const at = pages[page - 1]; + assert.ok(at); + assert.equal(footer(at), String(page), `${name}: page ${String(page)}`); + } + // targets in the front part and in at least two parts of sections + const partOf = (page: number) => { + let start = 0; + return parts.findIndex((part) => (start += part.pages) >= page); + }; + const reached = new Set(expected.map(partOf)); + assert.ok(reached.has(0) && reached.size >= 3, `${name}: parts ${[...reached].join(', ')}`); + // as joined, before the second pass, every number was the placeholder + const placeholders = contentsNumbers(await pagesOf(joined), front); + assert.ok(placeholders.length >= entries.length, name); + assert.ok( + placeholders.every((n) => n === 0), + `${name}: ${String(placeholders)}` + ); + } + }); + + test('only the front part is printed again, keeping its length', async () => { + for (const [name, { doc, parts, joined, swapped }] of made) { + const front = layoutBookFront(doc, options).html; + // the front part twice, the second time with the numbers; every other part once + const own = printed.filter((html) => html.includes(`${name}`)); + assert.equal(own.filter((html) => html === front).length, 1, name); + const again = own.filter((html) => html !== front && html.includes('data-part="contents"')); + assert.equal(again.length, 1, name); + assert.equal(own.length, parts.length + 1, name); + // the same pages, the front part's with its numbers in, the rest as joined + const [was, now] = [await pagesOf(joined), await pagesOf(swapped)]; + assert.equal(now.length, was.length, name); + const frontPages = parts[0]?.pages ?? 0; + const text = (page: PageRead | undefined, digits: boolean) => + (page?.items ?? []).map((item) => (digits ? item.str : item.str.replace(/\d+/g, '#'))); + for (const [i, page] of now.entries()) { + const label = `${name}, page ${String(i + 1)}`; + if (i < frontPages) assert.deepEqual(text(page, false), text(was[i], false), label); + else assert.deepEqual(page, was[i], label); + } + assert.notDeepEqual( + now.slice(0, frontPages).map((page) => text(page, true)), + was.slice(0, frontPages).map((page) => text(page, true)), + name + ); + } + }); + + test('a reprint that would move the pages is refused, changing nothing', async () => { + const alpha = madeOf('Alpha pathway'); + const front = alpha.parts[0]?.pages ?? 0; + const printer = started; + assert.ok(printer); + const refused = async (part: PartPrinter['part'], expected: RegExp) => { + const pdf = await PDFDocument.load(alpha.joined); + await assert.rejects( + reprintFront({ doc: alpha.doc, pdf, parts: alpha.parts }, { part }, options), + expected + ); + assert.deepEqual(await pagesOf(await pdf.save()), await pagesOf(alpha.joined)); + }; + // a page more: the introduction pushed onto a page of its own + await refused( + (html) => + printer.part(html.replace('

$&')), + new RegExp( + `The front part's numbers moved its pages: ${String(front)} pages, then ${String(front + 1)}; moved: introduction, contents` + ) + ); + // the same pages, the headings on others: the reprint's cover moved to its end + await refused( + async (html) => { + const printed = await PDFDocument.load(await printer.part(html)); + const moved = await PDFDocument.create(); + const [cover, ...rest] = printed.getPageIndices(); + const pages = await moved.copyPages(printed, [...rest, cover ?? 0]); + for (const page of pages) moved.addPage(page); + return moved.save(); + }, + new RegExp(`${String(front)} pages, then ${String(front)}; moved: introduction, contents`) + ); + // after the links are rewritten, the probes are gone + const linked = await PDFDocument.load(alpha.joined); + rewriteLinks(linked); + await assert.rejects( + reprintFront({ doc: alpha.doc, pdf: linked, parts: alpha.parts }, printer, options), + /entries with no probe in the joined PDF \(run before rewriteLinks\): introduction, / + ); + // and parts that are not the PDF's + await assert.rejects( + reprintFront( + { doc: alpha.doc, pdf: await PDFDocument.load(alpha.joined), parts: alpha.parts.slice(1) }, + printer, + options + ), + /first part is not its front part/ + ); + await assert.rejects( + reprintFront( + { + doc: alpha.doc, + pdf: await PDFDocument.load(alpha.joined), + parts: alpha.parts.slice(0, -1), + }, + printer, + options + ), + /The parts hold \d+ pages, the joined PDF \d+/ + ); + // and the PDF joined twice over, every anchor probed twice + const { pdf: twice } = await joinParts([alpha.joined, alpha.joined]); + await assert.rejects( + reprintFront( + { doc: alpha.doc, pdf: twice, parts: [...alpha.parts, ...alpha.parts] }, + printer, + options + ), + /introduction is probed twice/ + ); + }); + + test("the reprinted front part's links land, and the outline lands with them", async () => { + for (const [name, { doc, parts, swapped, done }] of made) { + const front = parts[0]?.pages ?? 0; + const titleOf = new Map((doc.contents ?? []).map((e) => [e.target, e.title])); + // the reprinted front part's placeholders, by page and rectangle + const placeholders = (await pagesOf(swapped)).slice(0, front).flatMap((page, i) => + page.links.flatMap((link) => + link.url?.startsWith(PLACEHOLDER) + ? [ + { + page: i, + rect: link.rect, + anchor: decodeURIComponent(link.url.slice(PLACEHOLDER.length)), + }, + ] + : [] + ) + ); + // a link for every contents entry + assert.deepEqual( + [...new Set(placeholders.map((p) => p.anchor))].sort(), + [...titleOf.keys()].sort(), + name + ); + const pages = await pagesOf(done); + await reading(done, async (pdf) => { + const dests = new Map(); + for (const placeholder of placeholders) { + const label = `${name}: page ${String(placeholder.page + 1)}, to ${placeholder.anchor}`; + const link = pages[placeholder.page]?.links.find((l) => + l.rect.every((v, k) => Math.abs(v - (placeholder.rect[k] ?? NaN)) < 0.01) + ); + assert.ok(link?.dest, `${label}: no destination`); + assert.equal( + await landing(pdf, link.dest, titleOf.get(placeholder.anchor) ?? ''), + null, + label + ); + dests.set(placeholder.anchor, link.dest); + } + // the outline: the contents list, each entry where its link lands + interface Entry { + title: string; + dest: unknown; + items: Entry[]; + } + const flat = (items: Entry[]): Entry[] => items.flatMap((e) => [e, ...flat(e.items)]); + const outline = flat(await pdf.getOutline()); + const entries = doc.contents ?? []; + assert.deepEqual( + outline.map((e) => e.title), + entries.map((e) => e.title), + name + ); + for (const [i, entry] of entries.entries()) { + assert.deepEqual(outline[i]?.dest, dests.get(entry.target), `${name}: ${entry.title}`); + } + }); + // and no placeholder or probe left, in the annotations or the file + const file = Buffer.from(done); + for (const address of [PLACEHOLDER, PROBE]) + assert.ok(!file.includes(address), `${name}: ${address}`); + } + }); + + test('the reprinted front pages are stamped as the rest are', async () => { + for (const [name, { parts, done }] of made) { + const front = parts[0]?.pages ?? 0; + const pages = await pagesOf(done); + const headers = pages.slice(0, front + 1).map(header); + assert.equal(headers[0], '', `${name}: the cover`); + assert.equal(headers[1], 'Introduction', name); + assert.equal(headers.at(-2), 'Contents', name); + assert.equal(headers.at(-1), name, name); + assert.deepEqual( + pages.slice(0, front + 1).map(footer), + pages.slice(0, front + 1).map((_, i) => (i === 0 ? '' : String(i + 1))), + name + ); + } + }); + + test('what the first print shares with a later page is kept', async () => { + // a joined PDF's parts share no objects, each copied apart; one shared here by hand, + // the introduction page's content stream held by the first page of sections too + const alpha = madeOf('Alpha pathway'); + const front = alpha.parts[0]?.pages ?? 0; + const printer = started; + assert.ok(printer); + const pdf = await PDFDocument.load(alpha.joined); + const [, introduction] = pdf.getPages(); + const later = pdf.getPage(front); + assert.ok(introduction); + const refs = (value: PDFObject | undefined) => + value instanceof PDFRef ? [value] : value instanceof PDFArray ? value.asArray() : []; + const shared = introduction.node.get(PDFName.of('Contents')); + const own = refs(introduction.node.get(PDFName.of('Annots'))); + assert.ok(refs(shared).length > 0 && own.length > 0); + later.node.set(PDFName.of('Shared'), shared ?? PDFName.of('None')); + await reprintFront({ doc: alpha.doc, pdf, parts: alpha.parts }, printer, options); + for (const ref of refs(shared)) { + assert.ok(pdf.context.lookup(ref) !== undefined, `${String(ref)} deleted`); + } + // while the introduction's own annotations went with it + for (const ref of own) { + // assert.ok, not equal: an object found is not to be printed, which never ends + assert.ok(pdf.context.lookup(ref) === undefined, `${String(ref)} kept`); + } + assert.deepEqual(unreachable(pdf), []); + }); + + test("nothing of the first print's front part is left in the file", async () => { + for (const [name, { joined, swapped, done }] of made) { + for (const [stage, bytes] of [ + ['joined', joined], + ['swapped', swapped], + ['done', done], + ] as const) { + assert.deepEqual(unreachable(await PDFDocument.load(bytes)), [], `${name}, ${stage}`); + } + } + }); +}); diff --git a/test/print/print.test.ts b/test/print/print.test.ts index 061f595..f4d3e40 100644 --- a/test/print/print.test.ts +++ b/test/print/print.test.ts @@ -17,8 +17,8 @@ import { } from '../../src/layout/document.ts'; import { PLACEHOLDER, PROBE } from '../../src/layout/html.ts'; import { PAGE } from '../../src/layout/figure.ts'; -import { contentsTargets } from '../../src/print/contents.ts'; -import { Printer, movedPages } from '../../src/print/print.ts'; +import { contentsTargets, movedPages } from '../../src/print/contents.ts'; +import { Printer } from '../../src/print/print.ts'; import { bookPdf, buildFixture, diff --git a/test/print/stamp.test.ts b/test/print/stamp.test.ts index 6627d24..4c8b6c0 100644 --- a/test/print/stamp.test.ts +++ b/test/print/stamp.test.ts @@ -400,7 +400,7 @@ describe('page numbers and running headers', { skip: skipWithoutGraph }, () => { assert.ok(Math.abs((left + right) / 2 - read.width / 2) < 1, 'off centre'); }); - test('white space and characters never drawn in a name are as a margin box draws them', async () => { + test('white space and characters never drawn in a name are close to a margin box', async () => { const alpha = stamped.get('Alpha pathway'); assert.ok(alpha); // a tab, a newline, a variation selector, a zero-width joiner and a control character, @@ -429,6 +429,27 @@ describe('page numbers and running headers', { skip: skipWithoutGraph }, () => { assert.deepEqual(once, await pagesOf(alpha.after)); }); + test('two stampings of a PDF at once: one stamps, the other is refused', async () => { + const alpha = stamped.get('Alpha pathway'); + assert.ok(alpha); + const pdf = await PDFDocument.load(alpha.before); + const joined = { doc: alpha.doc, pdf, parts: alpha.parts }; + const results = await Promise.allSettled([ + stampPages(joined, alpha.targets), + stampPages(joined, alpha.targets), + ]); + assert.deepEqual( + results.map((r) => r.status), + ['fulfilled', 'rejected'] + ); + const [, second] = results; + assert.match(second.status === 'rejected' ? String(second.reason) : '', /stamped already/); + assert.deepEqual( + await pagesOf(await pdf.save({ useObjectStreams: false })), + await pagesOf(alpha.after) + ); + }); + test('a character no font draws, or parts that are not the PDF, are refused', async () => { const alpha = stamped.get('Alpha pathway'); assert.ok(alpha); @@ -441,6 +462,18 @@ describe('page numbers and running headers', { skip: skipWithoutGraph }, () => { stampPages({ doc: planted, pdf, parts: alpha.parts }, alpha.targets), /U\+E000, which no font draws/ ); + // a name with nothing to draw once a margin box's white space and controls are out + await assert.rejects( + stampPages( + { + doc: { ...alpha.doc, cover: { ...alpha.doc.cover, title: ' \t\uFE0F\n ' } }, + pdf, + parts: alpha.parts, + }, + alpha.targets + ), + /leaves no header to stamp/ + ); const short = alpha.parts.slice(0, -1); await assert.rejects( stampPages({ doc: alpha.doc, pdf, parts: short }, alpha.targets), @@ -586,12 +619,18 @@ describe('pageStamps', () => { ); }); - test('a margin box collapses white space and leaves out what is never drawn', () => { + test("a margin box's white space collapsed, other spaces kept, and what is never drawn out", () => { for (const [given, drawn] of [ ['a\tb', 'a b'], ['a\nb', 'a b'], ['a\r\n b', 'a b'], + ['a\fb', 'a b'], [' a b ', 'a b'], + // a no-break, thin or ideographic space is not collapsed, as Chromium keeps it + ['a\u00A0b', 'a\u00A0b'], + ['a\u2009\u2009b', 'a\u2009\u2009b'], + ['a\u3000b', 'a\u3000b'], + [' \u00A0a\u00A0 ', '\u00A0a\u00A0'], ['a\uFE0Fb', 'ab'], ['a \uFE0F b', 'a b'], ['a\u200Db', 'ab'], @@ -601,7 +640,11 @@ describe('pageStamps', () => { ] as const) { assert.equal(marginText(given), drawn, JSON.stringify(given)); } - // after the substitutions, in pageStamps + // after the substitutions, in pageStamps, which refuses a name left empty + assert.throws( + () => pageStamps(' \t\uFE0F\u200D ', [2, 1], new Map([['introduction', { index: 1 }]])), + /The pathway's name " \\t.*" leaves no header to stamp/ + ); assert.equal( pageStamps('P\u2011Q\t', [2, 1], new Map([['introduction', { index: 1 }]]))[2]?.header, 'P-Q' From 01dafd3054564f591bf45fd909cc3f051b60a901 Mon Sep 17 00:00:00 2001 From: Adam Wright Date: Tue, 3 Nov 2026 20:44:17 +0000 Subject: [PATCH 2/2] Spec 001 T028: the PR's number Co-Authored-By: Claude Opus 5.5 --- specs/001-pathway-pdfs/tasks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/001-pathway-pdfs/tasks.md b/specs/001-pathway-pdfs/tasks.md index 3fff00a..5dffe53 100644 --- a/specs/001-pathway-pdfs/tasks.md +++ b/specs/001-pathway-pdfs/tasks.md @@ -451,7 +451,7 @@ passes. - **From #147's review:** the stamps were not drawn. Every face was embedded with `subset: true` from its @fontsource WOFF2 file, and fontkit 2.0.4's subsetter copies a WOFF2 file's `glyf` bytes as they are, transformed, so the subset's glyphs had no outlines: the text layer read back (every test above passed), but nothing was painted (Ghostscript at 300 dpi showed no header or number; pdf.js warned `Required "glyf" table is not found`). Each face is now embedded from the WOFF (1) file each @fontsource package ships beside its WOFF2 (`stampFile`, refused if there is none), which fontkit subsets correctly: on the fixture Ghostscript's `bbox` device now finds ink from 25.27 pt to 817.25 pt on a stamped page, 91.82 to 770.67 before stamping. The several faces' subsets are a few hundred bytes for two glyphs each (Noto Sans Math's was 135 KB for two from WOFF2). **T011's stamping measurement** (research R4) embedded the same broken WOFF2 subsets, so its stamp stage's size and time may change with outlines in them; T029's run measures it again. A name holding a tab, a newline, a variation selector or another character never drawn (FR-017's `NEVER_DRAWN`, now exported from `characters.ts`) prints in its section but refused the whole PDF at stamping, as no face's character map holds it; `marginText` now collapses white space and leaves those out, close to a margin box, after the substitutions. Each face is embedded once a PDF, and tested so; a PDF stamped twice is refused; `pageStamps`' test covers the analysis front part; the comment says there is no shaping. Tests added: every glyph of the stamps' embedded Roboto, read from its `FontFile2` with fontkit, has an outline, one for each character stamped but the space, and the stamps' fonts are one Roboto a PDF; the cover, the first page after it and every landscape page, rendered by pdf.js (with `@napi-rs/canvas`, pdfjs-dist's optional dependency; no other page is rendered): each but the cover has ink in its top and bottom margins, which were blank before stamping, and the cover none; the several faces' page has each face once, each a subset under 10 KB whose glyphs but the space are outlined, and ink in its margins; a name planted with a leading space, a tab, a newline, U+FE0F, U+200D and U+0007 stamped `Alpha pathway one`; a second stamping refused, changing nothing; `marginText`'s cases one by one (tab, newline, CR LF, leading and trailing space, U+FE0F alone and between spaces, U+200D, U+0007, U+00AD, Greek kept); the analysis front part named `Analysis properties`. 316 tests before (#147's own count, 315 after it, was one short), 322 after. Each of these mutants made one fail: the WOFF2 file embedded, as in #147 (the outline, ink and several-faces tests); the text drawn invisible, `renderMode` 3 (the ink and several-faces tests); each face embedded again on every page (the outline test, by its one-font-a-PDF check); no `marginText` (the planted white-space test and `marginText`'s); white space not collapsed (the same two); stamping twice not refused (its test); the analysis front part not named (its test); a font given to every `drawText` (the colour and font test, and the several-faces test). Every mutant of #147 still fails as before, more tests catching most: per part numbering (the numbers test, `pageStamps`); the first section page given the front part's header (the header test, the planted-name tests, `pageStamps`, the analysis test); every front page the first front part's (the header test, `pageStamps`, the analysis test); the cover numbered (the numbers, cover, font and ink tests, `pageStamps`); the name without the substitutions (the U+2011 test, and `marginText`'s through `pageStamps`); every character in Roboto Latin (the several-faces test); the baseline at the margin's middle, and centring on the portrait width (the placement test); the theme's on-surface colour (the colour test, `stampLook`); the whole font embedded (the font, outline and several-faces tests); the parts' pages unchecked (the refusal test). - [x] T028 [US1] The book's contents, second pass: T019's `src/print/contents.ts`, extended to read where each section landed in each joined PDF and reprint only that PDF's front part. The reprinted front part is swapped in before T026's links and T027's stamping run, so its links are rewritten and its pages stamped like the rest. Tests: every entry's number is its target's page, across parts; the contents' links land; the front pages carry their headers and numbers. - - **Done (#PR):** `reprintFront` (`src/print/contents.ts`) runs first in `printBook`'s `joined` callback, before `rewriteLinks`, `writeOutline` and `stampPages` (T029 wires the four into `book`, in that order, as its task now says). It reads each probe's page in the joined PDF without changing anything (`probeTargets`, added to `src/print/links.ts`, refusing an anchor probed twice), numbers each contents entry from its target's page counted from the PDF's first (the cover's included, as the stamps count, T027), lays the front part out again (`layoutBookFront`), fills the placeholders (T019's `fillContents`), prints it through the same `PartPrinter` and swaps its pages in for the first print's (`removePage`, `insertPage`). The objects only the first print's front pages held are deleted (pdf-lib's `removePage` deletes a page's own object, not its contents, resources or annotations, which saving would write, placeholders and probes among them): those reachable from the old pages, not through `/Parent`, less what is still reachable from the trailer, so nothing the PDF uses goes even if a page shared it. That second walk covers the whole PDF, linear in its objects (inference: tens of milliseconds to about a second for the largest PDF; T029 measures it). Refused before anything is changed, each naming what is wrong: a first part that is not the front part, parts whose pages are not the PDF's, an entry whose target has no probe (as after `rewriteLinks`), an anchor probed twice, and a reprint whose pages differ from the first print's, in their count or a front heading's page (T019's `movedPages`, moved here from `print.ts` for both passes): every later page and number would move with it, so it is refused rather than handled (the numbers sit in boxes of fixed width, T019, so on the fixture nothing moves). The reprint's layout report is dropped: the first print's is the PDF's, so nothing in the front part is counted twice (T029's character test). The reprint holds the join's slots, as `joined` does. + - **Done (#151):** `reprintFront` (`src/print/contents.ts`) runs first in `printBook`'s `joined` callback, before `rewriteLinks`, `writeOutline` and `stampPages` (T029 wires the four into `book`, in that order, as its task now says). It reads each probe's page in the joined PDF without changing anything (`probeTargets`, added to `src/print/links.ts`, refusing an anchor probed twice), numbers each contents entry from its target's page counted from the PDF's first (the cover's included, as the stamps count, T027), lays the front part out again (`layoutBookFront`), fills the placeholders (T019's `fillContents`), prints it through the same `PartPrinter` and swaps its pages in for the first print's (`removePage`, `insertPage`). The objects only the first print's front pages held are deleted (pdf-lib's `removePage` deletes a page's own object, not its contents, resources or annotations, which saving would write, placeholders and probes among them): those reachable from the old pages, not through `/Parent`, less what is still reachable from the trailer, so nothing the PDF uses goes even if a page shared it. That second walk covers the whole PDF, linear in its objects (inference: tens of milliseconds to about a second for the largest PDF; T029 measures it). Refused before anything is changed, each naming what is wrong: a first part that is not the front part, parts whose pages are not the PDF's, an entry whose target has no probe (as after `rewriteLinks`), an anchor probed twice, and a reprint whose pages differ from the first print's, in their count or a front heading's page (T019's `movedPages`, moved here from `print.ts` for both passes): every later page and number would move with it, so it is refused rather than handled (the numbers sit in boxes of fixed width, T019, so on the fixture nothing moves). The reprint's layout report is dropped: the first print's is the PDF's, so nothing in the front part is counted twice (T029's character test). The reprint holds the join's slots, as `joined` does. - **Tests:** `test/print/contents.test.ts` (7) prints the fixture's book with the threshold at 1 page, as T025's to T027's do, and in `joined` saves each PDF as joined, reprints its front part, saves it, rewrites its links, writes its outline and stamps it: every contents number, read from the finished PDF's contents pages, is the page on which the entry's title is first a heading (found in the text, not from the probes) and that page's stamped number, the targets spread over the front part and at least two parts of sections, every number a placeholder `0` as joined; only the front part printed again, once, every other part once, the swapped PDF's pages as joined but for the front part's numbers; a reprint a page longer (a block pushing the introduction on) and one of the same length with its headings moved (its cover moved to its end) refused, naming the pages and the anchors, the PDF unchanged, and an entry with no probe, the parts not the PDF's (each way) and an anchor probed twice refused; the reprinted front part's contents links, matched by page and rectangle, landing on their headings, one for each entry, the outline the contents list, each entry's destination its link's, and no placeholder or probe address left in the file; the front pages stamped, the cover bare, `Introduction` and `Contents` and their numbers; an object the first print's front shares with a later page (planted, as parts share none) kept while its own annotations go; and no object left that nothing reaches, as joined, swapped and finished. 322 tests before, 330 after (8: these 7, and the stamping race below). Each of these mutants made one fail: no swap (the numbers, reprint and shared-object tests); the front part printed again without its numbers (the numbers and reprint tests); every number a page low (the numbers test); the first print's objects left in the file (the links test, by the placeholder address left, the shared-object and nothing-left tests); every object the old pages reach deleted, live or not (the shared-object test); moved pages not refused, and only the page count compared (the refusal test, each); entries with no probe not refused, the parts' pages unchecked, an anchor probed twice not refused (the refusal test, each); the reprinted pages inserted in reverse (every test, through the setup's run failing). In the shared-object test, an `assert.equal` of a found object against `undefined` never finished printing the object (the run ended with a promise pending), so it asserts with `assert.ok`. - **From #148's review:** the stamp-twice guard raced (`stamped.add` came after an `await`, so two stampings at once both passed); the PDF is now held once its stamps are checked, before the first `await`, and a test stamps it twice at once: one stamps, the other is refused, the pages as stamped once. `marginText` collapsed JavaScript's `\s`, which takes in a no-break, thin or ideographic space that Chromium keeps; it now collapses only space, tab, line feed, carriage return and form feed, leaves out only controls and default-ignorable characters (FR-017's `NEVER_DRAWN` but its white space), trims only spaces, and is described as close to a margin box, not as one. A name that leaves no header (all white space and characters never drawn) is refused, naming it, not stamped bare. `ci/measure/join.ts` now embeds Roboto's WOFF file, as `stamp.ts` does, so its stamp stage draws outlines. T027's note is corrected (its test count, and which pages are rendered). Each of these mutants made one fail: the guard after the first `await` (the race test, and the stamp-twice test); `\s` collapsed, `trim()`, `NEVER_DRAWN` left out whole, and a form feed not collapsed (`marginText`'s test, each); the empty name not refused (`marginText`'s test through `pageStamps`, and the refusal test). - [ ] T029 [US1] `book` end to end, `src/cli.ts` (contracts/cli.md).