Skip to content

Every prose word against a dictionary (check:vocabulary) - #331

Merged
oblomov-dev merged 1 commit into
mainfrom
claude/funny-clarke-8i7ydw
Sep 21, 2026
Merged

oblomov-dev merged 1 commit into
mainfrom
claude/funny-clarke-8i7ydw

Conversation

@oblomov-dev

Copy link
Copy Markdown
Member

test/spelling.test.mjs answers which spelling of a word — it names British forms from a list and rewrites them. What it cannot do is notice a word that is neither spelling of anything.

thant, and two days later havte, fullfll and requoirememnts, went in through the browser editor, past all fourteen gates, through the deploy and out to the published site. The second set stood in the first sentence of a page — which is where scripts/lib/pages.mjs takes the page description from, so it read wrong in the search index, in llms.txt and under the card as well.

The gate

Every prose word is looked up in dictionary-en (the Hunspell en_US), then in scripts/lib/vocabulary.txt — 215 words this project uses that a dictionary does not carry, grouped and commented, built from the corpus in one pass. The prose comes from scripts/lib/prose.mjs, the same reader the spelling test uses, so an ABAP keyword, a CSS property or a path is never a word here.

Two new dependencies, 704 kB, MIT and BSD: nspell and dictionary-en.

The tokenizer is most of the work, and the manual proved each rule

rule what the manual did without it
\p{L}, not [A-Za-z] started a new token at the ller of Müller and reported it
emphasis joined first **f**rontend was read as rontend
a digit or _ means identifier abap2UI5 split at the digit and reported UI 1059 times
an acronym is not a word ICF, APIs, SEGW, GUID
camelCase is an identifier liveChange
a hyphenated compound is judged part by part minus the fragments a hyphen leaves behind, so un-build is not a word un

Each rule cut the unknown words: 821 distinct → 266 → 249. Those 249 are all real — project vocabulary, SAP's own words, the names of tools and people — which is why the list is exactly that long and no page changed in this PR.

The list

A word belongs in vocabulary.txt when it is right and the dictionary simply does not have it. A word you are unsure about does not: a list that absorbs doubt is how a gate like this stops catching anything. node scripts/check-vocabulary.mjs --unused names entries no page uses any more; right now every entry is earning its line.

Verified against the real case

The first case in test/vocabulary.test.mjs is the four typos themselves. Re-inserted into the live article, the gate reports:

check-vocabulary: 3 word(s) that neither the dictionary nor this project knows:
  havte
    docs/advanced/insights/03-one-class-no-programming-model.md:3
  fullfll
    docs/advanced/insights/03-one-class-no-programming-model.md:3
  requoirememnts
    docs/advanced/insights/03-one-class-no-programming-model.md:3

Wiring

Fifteenth gate, so it is named in the check script, in check.yml, in deploy.yml, and in the four documents that count them in prose — test/gates.test.mjs holds all three lists and the count against each other. The floor: the gate exits 1 when it walked no page or read an empty word list.

Gates

All fifteen pass locally except the two this environment cannot run: build (needs the playground checkout or network the proxy blocks) and check:samples (skips without a samples checkout).

🤖 Generated with Claude Code

https://claude.ai/code/session_0182AiPodwfGRKPNZG9D5epe


Generated by Claude Code

test/spelling.test.mjs answers WHICH spelling of a word - it names British
forms from a list and rewrites them. What it cannot do is notice a word
that is neither spelling of anything. "thant", and two days later "havte",
"fullfll" and "requoirememnts", went in through the browser editor, past
all fourteen gates, through the deploy and out to the published site. The
second set stood in the FIRST SENTENCE of a page, which is where
scripts/lib/pages.mjs takes the page description from - so it read wrong in
the search index, in llms.txt and under the card as well.

So: every prose word is looked up in dictionary-en (the Hunspell en_US),
and what a dictionary cannot know about this project is committed beside it
in scripts/lib/vocabulary.txt - 215 words, grouped and commented, built
from the corpus in one pass. The prose comes from scripts/lib/prose.mjs,
the same reader the spelling test uses, so an ABAP keyword, a CSS property
or a path is never a word here.

The tokenizer is most of the work, and the manual proved each rule:

- \p{L}, not [A-Za-z]: the latter starts a new token at the "ller" of
  "Müller" and reports it.
- emphasis inside a word is joined first: `**f**rontend` was read as
  "rontend".
- a token with a digit or an underscore is an identifier (abap2UI5,
  z2ui5_if_app), so is an acronym (ICF, APIs) and so is camelCase
  (liveChange) - none of them is English to be spelled. Without this the
  first probe reported "UI" 1059 times, from abap2UI5 split at the digit.
- a hyphenated compound is judged part by part, minus the fragments a
  hyphen leaves behind ("un-build" is not a word "un").

Each rule cut the unknown words: 821 distinct, then 266, then 249 - and
those 249 are all real, which is why the list is exactly that long and no
page changed.

Fifteenth gate: named in the check script, both workflows and the four
documents that count them. --unused names entries no page uses any more.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182AiPodwfGRKPNZG9D5epe
@oblomov-dev
oblomov-dev merged commit c99b6be into main Sep 21, 2026
1 check passed
@oblomov-dev
oblomov-dev deleted the claude/funny-clarke-8i7ydw branch September 21, 2026 14:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants