` in their summation, so a table wider than the text
column, which Chromium printed by scaling its whole part down below the theme's sizes,
would have reached the real book: the layout now keeps realistic content inside the
- column (pages.css; a table of very many columns can still be wider), and the check
+ column (pages.css; a table whose longest words, side by side, are wider than the
+ column can still be wider: a token of up to 40 characters does not break, a longer one
+ breaks into pieces of 13 to 20. Measured by `ci/measure/breaks.ts`, a table of two
+ rows laid out by Chromium with the book's stylesheets and print media in a
+ `.summation` 170 mm wide, the text column; its output in
+ `ci/measure/results/2026-11-04/breaks/`. With lowercase text cut from biochemical
+ words, roughly five columns of 20- to 25-character words are wider, four of 30,
+ three of 35 to 40, and five to seven of longer tokens; with `m` alone, two to four.
+ The figures depend on the letters, so they are approximate), and the check
reads headings against the section's own body text, so a part printed scaled down is
not a fault. No human summation holds a
`` block, and no human event's name holds a soft hyphen, a zero-width or
diff --git a/specs/001-pathway-pdfs/tasks.md b/specs/001-pathway-pdfs/tasks.md
index 8909ecd..359a779 100644
--- a/specs/001-pathway-pdfs/tasks.md
+++ b/specs/001-pathway-pdfs/tasks.md
@@ -536,6 +536,11 @@ passes.
- **`overflow-wrap: anywhere` broke short columns' words (high).** In a table it let every column narrow to a character, so a table wider than the column broke its short columns' words (`Da`/`te`). `body` now has `overflow-wrap: break-word`, which keeps each column at its longest word, and the layout gives a token longer than 40 characters a `` every 20 (`text()` in `src/layout/html.ts`, grapheme by grapheme, for every text it writes; `markupBreaks` for a summation's text runs, a character reference one unit; an attribute's value and the page's `` never, through `plainText`), which adds no character to what is read. An edit's date is kept whole (`white-space: nowrap`), as a column narrowed to its longest word broke `2026-01-04` at its hyphens. Test: an edit's names, 30 of them, and a summation table of `Phosphofructokinase` beside a column of prose: exit 0, `Date` and `Action` whole and no fragment of them, every date whole, `Phosphofructokinase` whole four times; on #160's CSS it read `Da`, `te`, `Ac`. The overflow test is kept: its table is now four columns of long words, as wide as release 98's are (six columns of 28-letter words are wider than the column whatever breaks, and a part holding one is printed scaled down, which the check tolerates), and its 140-character token also sits in a table cell, where only its ``s let the column narrow.
- **Lows.** The docs no longer say nothing overflows: the layout keeps realistic content inside the column, a table of very many columns can still be wider, and the check's measure against the section tolerates a part printed scaled down (contracts/cli.md, research R8). `folded()` builds its pattern from FR-017's `NEVER_DRAWN` (`src/layout/characters.ts`), not a copy. `identifies()` reads a candidate's section only to the next place a section may start (an event's first link to its page, or an outline entry with an event's name). Recorded: the failed-removal test (`test/print/temporary.test.ts`) relies on `chmod` keeping a folder from being emptied, so it does not hold when run as root.
- **Tests:** 439 before, 441 after (the labels and the short-columns tests). Each of these mutants made one fail: the page's commonest size with the figures' labels counted, as before (the labels test); `overflow-wrap: anywhere` again, and the dates allowed to break (the short-columns test, each); no `` at all, and none in summations (the overflow test, each). Survived: the label's size alone, and the page's size without the figures' labels alone (each covers the other on the fixture; both kept, the second as the fallback for a section with no label); `identifies()` unbounded (no fixture section lacks its label, the only case it changes); no `overflow-wrap` and the tables' `max-width: 100%` (with the ``s, nothing tested is wider than a line; both kept, as the review asked); the facts' grid at `minmax(0, 1fr)` and the contents' links at `min-width: 0`, taken out again.
+ - **Follow-up (#164, replacing #163 after its review):** #162's review's five lows. A token ends only at a space Chromium breaks a line at, as measured (`ci/measure/breaks.ts`): tab, line feed, carriage return, space, U+1680, U+2000 to U+2006, U+2008 to U+200B, U+2028, U+2029, U+205F and U+3000, and a numeric reference to one; a no-break space, U+2007, U+202F, U+FEFF, U+000B and U+000C no longer end one. A summation's markup is read whole (`markupBreaks`, which `markupText` no longer feeds run by run), its tags never counted or broken into, and a word counted whole through the inline tags (`a`, `b`, `em`, `i`, `span`, `strong`, `sub`, `sup`, `u`) and comments, any other tag ending it; it relies on `sanitiseSummation`, which the model applies to every summation. A token over 40 breaks into equal pieces of at most 20 (41 is 14, 14 and 13, not 20, 20 and 1). The docs (contracts/cli.md, research R8) say what can still overflow, measured by `ci/measure/breaks.ts` (output in `ci/measure/results/2026-11-04/breaks/`) and approximate, as it depends on the letters: with lowercase text cut from biochemical words, five columns of 20- to 25-character words, four of 30, three of 35 to 40, five to seven of longer tokens (broken into pieces of 13 to 20); with `m` alone, two to four. Tests: `test/layout/html.test.ts` (11) for `text()` and `markupBreaks`, and in `test/model/build.test.ts` every built summation sanitised; 441 before, 452 after. Each of these mutants made one fail: the old `/^\s$/u` space (the spaces test, and the markup one through a no-break space); a carriage return and line feed not taken as a space (the spaces test); character references not split out (`&` broken into, the references test); `text()` escaping before it breaks (the escaped test); tags taken as text (the attribute and inline-tags tests); every tag ending a word, and none (the inline-tags test, each); tags counted in a word's length (the same); a `<` before no tag name taken as a tag (the markup test); a break every 20 again (five tests).
+ - **From #163's review:** two mediums, each reproduced, and lows.
+ - **Breaking spaces dropped (medium, a regression).** #163 ended a token only at a space, tab, line feed or carriage return, so two words beside a thin space (or an en, em or ideographic space, or U+200B, which main also missed) counted as one token, and a long pair broke mid-word; 9 of release 98's human summations hold such spaces (measured on the private copy). `BREAKABLE_SPACE` is now the set Chromium was measured to break at, which `ci/measure/breaks.ts` checks (exit 1 on a disagreement); the review's rule, `\s` without U+00A0, U+202F and U+FEFF and with U+200B, would also have ended tokens at U+2007, U+000B and U+000C, at which Chromium does not break. Tests: two 28- and 17-letter words beside a thin space kept whole; every measured space, breaking and joining. Each of these mutants made one fail: #163's set (the thin-space, spaces and reference tests); the set without U+2000 to U+200B (the same); the review's rule, and U+2007 taken as a space (the spaces test, each); numeric references never taken as spaces (the reference test).
+ - **The overflow figures (medium).** #163's figures, measured with the one word `phosphatidylinositoltrisphosphatekinases`, did not hold for other letters, and said pieces of 14 to 20 where 41 gives 13. Re-measured by a script that can be re-run (`ci/measure/breaks.ts`), the letters and the method stated, the figures given as approximate (above).
+ - **Lows.** The tag reader's branches are tested: a quoted `>` in an attribute, a comment within a word, `a` and `b` inline, and a tag in capitals; each of these mutants made the test fail: quotes not read in a tag, comments ending a word, `a` and `b` not inline, tag names not lowercased. `markupBreaks`'s doc comment says it relies on `sanitiseSummation`, and that raw markup (CDATA, a doctype, a processing instruction, a script) is not read rightly; a test gives such raw markup through the sanitiser and finds no `` inside a tag, and the model's test checks every summation it builds is the sanitiser's fixed point (with the model not sanitising, it fails, as the existing check for scripts does).
- **Known gaps:** the rules are measured on the fixture alone, whose names are short and whose figures small: T031 runs the check on release 98's book, and its time against R7's 9 minutes. A figure is known by its link: a drawing lost with its link kept is not seen. A heading that lost its link and every entry naming it is found nowhere, and is then three faults. A link landing on a page but not on its target is not a fault (R8: every link landing on a page). Between `bookCommand` returning and the process ending there are no handlers, so a signal then is the default action, exit 128 and its number, the book and its report already in place.
- [ ] T031 [US1] The book on a private copy of release 98's graph.
diff --git a/src/layout/html.ts b/src/layout/html.ts
index d6f1a36..e865ee7 100644
--- a/src/layout/html.ts
+++ b/src/layout/html.ts
@@ -12,34 +12,75 @@ export function plainText(value: string): string {
}
/**
- * A token longer than this, with no space in it, may break every BREAK_EVERY characters:
- * the pages break words only where they may (`overflow-wrap: break-word`, pages.css), so
- * that a table's short columns keep their words whole, and a token longer than the text
- * column (a run of letters, a long address) would otherwise overflow it, and Chromium then
- * prints its whole part scaled down.
+ * A token longer than this, with no space in it, is given places to break, in equal pieces
+ * of at most BREAK_EVERY characters: the pages break words only where they may
+ * (`overflow-wrap: break-word`, pages.css), so that a table's short columns keep their
+ * words whole, and a token longer than the text column (a run of letters, a long address)
+ * would otherwise overflow it, and Chromium then prints its whole part scaled down.
*/
export const LONG_TOKEN = 40;
export const BREAK_EVERY = 20;
/**
- * Units of text (characters as a reader sees them, and a markup run's character references)
- * with `` between every BREAK_EVERY of a token longer than LONG_TOKEN: a break
+ * The spaces a token ends at: those Chromium breaks a line at, measured
+ * (`ci/measure/breaks.ts`): tab, line feed, carriage return (a carriage return and line
+ * feed are one grapheme), space, U+1680, U+2000 to U+2006, U+2008 to U+200A (the en, em,
+ * thin and hair spaces among them), U+200B (zero width), U+2028, U+2029, U+205F and U+3000.
+ * The others `\s` holds join their neighbours, and a token runs through them: U+000B,
+ * U+000C, U+00A0 (no-break), U+2007 (figure), U+202F and U+FEFF. A space left out of this
+ * list would join two words into one token, and a long one would break mid-word.
+ */
+export const BREAKABLE_SPACE =
+ /^[\t\n\r \u1680\u2000-\u2006\u2008-\u200b\u2028\u2029\u205f\u3000]+$/u;
+
+/**
+ * A unit of text: a character as a reader sees it, a character reference, or a tag, which
+ * is no character, and which ends the token it is in unless it is inline (`joins`).
+ */
+interface Unit {
+ readonly text: string;
+ readonly tag?: 'joins' | 'ends';
+ /** A character reference to a space a line breaks at. */
+ readonly space?: boolean;
+}
+
+/**
+ * Units with `` between the pieces of a token longer than LONG_TOKEN, each piece of
+ * at most BREAK_EVERY units and none more than one shorter than another: a break
* opportunity, which adds no character to the printed text.
*/
-function breakTokens(units: readonly string[], escape: (unit: string) => string): string {
+function breakTokens(units: readonly Unit[], escape: (unit: string) => string): string {
let out = '';
- let token: string[] = [];
+ let token: Unit[] = [];
const flush = () => {
- for (let i = 0; i < token.length; i++) {
- if (token.length > LONG_TOKEN && i > 0 && i % BREAK_EVERY === 0) out += '';
- out += escape(token[i] ?? '');
+ const length = token.filter((unit) => unit.tag === undefined).length;
+ const pieces = length > LONG_TOKEN ? Math.ceil(length / BREAK_EVERY) : 1;
+ // the k-th piece ends at ceil(k * length / pieces) units, so the pieces differ by one at most
+ let next = 1;
+ let counted = 0;
+ for (const unit of token) {
+ if (unit.tag !== undefined) {
+ out += unit.text;
+ continue;
+ }
+ if (next < pieces && counted === Math.ceil((next * length) / pieces)) {
+ out += '';
+ next++;
+ }
+ out += escape(unit.text);
+ counted++;
}
token = [];
};
for (const unit of units) {
- if (/^\s$/u.test(unit)) {
+ if (unit.tag === 'joins') {
+ token.push(unit);
+ } else if (unit.tag === 'ends') {
flush();
- out += escape(unit);
+ out += unit.text;
+ } else if (unit.space === true || BREAKABLE_SPACE.test(unit.text)) {
+ flush();
+ out += escape(unit.text);
} else {
token.push(unit);
}
@@ -50,8 +91,8 @@ function breakTokens(units: readonly string[], escape: (unit: string) => string)
/** Characters as a reader sees them: a break never parts a mark from its letter. */
const GRAPHEMES = new Intl.Segmenter('en', { granularity: 'grapheme' });
-const graphemes = (value: string): string[] =>
- Array.from(GRAPHEMES.segment(value), (part) => part.segment);
+const graphemes = (value: string): Unit[] =>
+ Array.from(GRAPHEMES.segment(value), (part) => ({ text: part.segment }));
/** Text for an element's content, escaped, a long token given places to break. */
export function text(value: string): string {
@@ -59,14 +100,55 @@ export function text(value: string): string {
}
/**
- * A run of markup's text (raw: its character references as they are) given places to break
- * in a long token, as `text` gives plain text: for `markupText`'s runs of a summation.
+ * The inline tags of a summation (the sanitiser's, rules.ts): a word may run through them,
+ * as `Na2` or a word half in italics, and is counted whole. Any other tag, a
+ * paragraph's or a cell's, ends the word.
+ */
+const INLINE_TAGS = new Set(['a', 'b', 'em', 'i', 'span', 'strong', 'sub', 'sup', 'u']);
+
+/**
+ * A tag (its attributes' quoted values may hold `>`), or a comment; a `<` before neither is
+ * text. Enough for the sanitiser's output, which holds no CDATA, doctype, processing
+ * instruction or script or style element: raw markup is not read rightly.
+ */
+const TAG = /(|$)|<\/?([a-z][a-z0-9]*)(?:[^>"']|"[^"]*"|'[^']*')*>?)/giu;
+const REFERENCE = /(&(?:#[0-9]+|#x[0-9a-f]+|[a-z][a-z0-9]*);)/giu;
+
+/**
+ * The character a numeric reference stands for; a named one is left as it is, as the
+ * sanitiser writes every character but `&`, `<`, `>` and `"` as itself.
+ */
+function referenced(reference: string): string {
+ const hex = /^([0-9a-f]+);$/iu.exec(reference)?.[1];
+ const decimal = /^([0-9]+);$/u.exec(reference)?.[1];
+ const code =
+ hex !== undefined ? parseInt(hex, 16) : decimal !== undefined ? Number(decimal) : NaN;
+ return code <= 0x10ffff ? String.fromCodePoint(code) : reference;
+}
+
+/**
+ * A summation's markup, sanitised by `sanitiseSummation` (src/model/rules.ts), which the
+ * model applies to every summation it builds, a long token in its text given places to
+ * break as `text` gives plain text: each character reference one unit, a tag never broken
+ * into and never counted, and a word counted whole through inline tags. It relies on the
+ * sanitiser: on raw markup (CDATA, a doctype, a script) it may break where it should not.
*/
-export function markupBreaks(run: string): string {
- // each character reference one unit, every other text by its characters
- const units = run
- .split(/(&(?:#[0-9]+|#x[0-9a-f]+|[a-z][a-z0-9]*);)/giu)
- .flatMap((part, i) => (i % 2 === 1 ? [part] : graphemes(part)));
+export function markupBreaks(markup: string): string {
+ const units: Unit[] = [];
+ const parts = markup.split(TAG);
+ // split gives text, then each tag with its name (or undefined), then text again
+ for (let i = 0; i < parts.length; i += 3) {
+ const between = parts[i] ?? '';
+ between.split(REFERENCE).forEach((part, j) => {
+ if (j % 2 === 1) units.push({ text: part, space: BREAKABLE_SPACE.test(referenced(part)) });
+ else units.push(...graphemes(part));
+ });
+ const tag = parts[i + 1];
+ if (tag === undefined) continue;
+ const name = parts[i + 2]?.toLowerCase();
+ const joins = tag.startsWith('', ''],
+ ['', ''],
+ ['', ''],
+ ['', ''],
+ ] as const) {
+ const out = markupBreaks(`${x}${open}${y}${close}`);
+ assert.equal(out.split('').length, 3, open);
+ assert.equal(out.replaceAll('', ''), `${x}${open}${y}${close}`, open);
+ }
+ });
+
+ test("it is given the sanitiser's markup, and breaks nothing inside a tag of it", () => {
+ const raw =
+ `` +
+ `${'w'.repeat(50)}
`;
+ const out = markupBreaks(sanitiseSummation(raw));
+ assert.doesNotMatch(out, /<[^>]*/);
+ assert.match(out, /^w+w+w+<\/p>$/);
+ });
+});
diff --git a/test/model/build.test.ts b/test/model/build.test.ts
index ca58255..e197270 100644
--- a/test/model/build.test.ts
+++ b/test/model/build.test.ts
@@ -6,6 +6,7 @@ import { after, before, describe, test } from 'node:test';
import { graphSettings, openGraph, type Graph } from '../../src/graph/connection.ts';
import { eventByIdentifier } from '../../src/graph/queries.ts';
import { buildBook, buildSingle } from '../../src/model/build.ts';
+import { sanitiseSummation } from '../../src/model/rules.ts';
import type { Document, Place } from '../../src/model/types.ts';
const skip = process.env['NEO4J_URI'] ? false : 'no fixture graph: set NEO4J_URI';
@@ -205,6 +206,12 @@ describe('the model', { skip }, () => {
]);
const alpha = fullOf(pdfs, 'R-HSA-90000200');
assert.doesNotMatch(alpha.summations.join(''), /script|onerror|javascript/);
+ // every summation sanitised, as the layout's breaks (markupBreaks) rely on it
+ for (const doc of pdfs)
+ for (const p of doc.places)
+ if (p.full)
+ for (const summation of p.section.summations)
+ assert.equal(sanitiseSummation(summation), summation, p.section.event.stId);
assert.deepEqual(
fullOf(pdfs, 'R-HSA-90000120').inferredFrom.map((s) => s.speciesName),
['Mus musculus', 'Rattus norvegicus']