Every prose word against a dictionary (check:vocabulary) - #331
Merged
Merged
Conversation
test/spelling.test.mjs answers WHICH spelling of a word - it names British
forms from a list and rewrites them. What it cannot do is notice a word
that is neither spelling of anything. "thant", and two days later "havte",
"fullfll" and "requoirememnts", went in through the browser editor, past
all fourteen gates, through the deploy and out to the published site. The
second set stood in the FIRST SENTENCE of a page, which is where
scripts/lib/pages.mjs takes the page description from - so it read wrong in
the search index, in llms.txt and under the card as well.
So: every prose word is looked up in dictionary-en (the Hunspell en_US),
and what a dictionary cannot know about this project is committed beside it
in scripts/lib/vocabulary.txt - 215 words, grouped and commented, built
from the corpus in one pass. The prose comes from scripts/lib/prose.mjs,
the same reader the spelling test uses, so an ABAP keyword, a CSS property
or a path is never a word here.
The tokenizer is most of the work, and the manual proved each rule:
- \p{L}, not [A-Za-z]: the latter starts a new token at the "ller" of
"Müller" and reports it.
- emphasis inside a word is joined first: `**f**rontend` was read as
"rontend".
- a token with a digit or an underscore is an identifier (abap2UI5,
z2ui5_if_app), so is an acronym (ICF, APIs) and so is camelCase
(liveChange) - none of them is English to be spelled. Without this the
first probe reported "UI" 1059 times, from abap2UI5 split at the digit.
- a hyphenated compound is judged part by part, minus the fragments a
hyphen leaves behind ("un-build" is not a word "un").
Each rule cut the unknown words: 821 distinct, then 266, then 249 - and
those 249 are all real, which is why the list is exactly that long and no
page changed.
Fifteenth gate: named in the check script, both workflows and the four
documents that count them. --unused names entries no page uses any more.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0182AiPodwfGRKPNZG9D5epe
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
test/spelling.test.mjsanswers which spelling of a word — it names British forms from a list and rewrites them. What it cannot do is notice a word that is neither spelling of anything.thant, and two days laterhavte,fullfllandrequoirememnts, went in through the browser editor, past all fourteen gates, through the deploy and out to the published site. The second set stood in the first sentence of a page — which is wherescripts/lib/pages.mjstakes the page description from, so it read wrong in the search index, inllms.txtand under the card as well.The gate
Every prose word is looked up in
dictionary-en(the Hunspell en_US), then inscripts/lib/vocabulary.txt— 215 words this project uses that a dictionary does not carry, grouped and commented, built from the corpus in one pass. The prose comes fromscripts/lib/prose.mjs, the same reader the spelling test uses, so an ABAP keyword, a CSS property or a path is never a word here.Two new dependencies, 704 kB, MIT and BSD:
nspellanddictionary-en.The tokenizer is most of the work, and the manual proved each rule
\p{L}, not[A-Za-z]llerof Müller and reported it**f**rontendwas read asrontend_means identifierabap2UI5split at the digit and reportedUI1059 timesICF,APIs,SEGW,GUIDliveChangeun-buildis not a wordunEach rule cut the unknown words: 821 distinct → 266 → 249. Those 249 are all real — project vocabulary, SAP's own words, the names of tools and people — which is why the list is exactly that long and no page changed in this PR.
The list
A word belongs in
vocabulary.txtwhen it is right and the dictionary simply does not have it. A word you are unsure about does not: a list that absorbs doubt is how a gate like this stops catching anything.node scripts/check-vocabulary.mjs --unusednames entries no page uses any more; right now every entry is earning its line.Verified against the real case
The first case in
test/vocabulary.test.mjsis the four typos themselves. Re-inserted into the live article, the gate reports:Wiring
Fifteenth gate, so it is named in the
checkscript, incheck.yml, indeploy.yml, and in the four documents that count them in prose —test/gates.test.mjsholds all three lists and the count against each other. The floor: the gate exits 1 when it walked no page or read an empty word list.Gates
All fifteen pass locally except the two this environment cannot run:
build(needs the playground checkout or network the proxy blocks) andcheck:samples(skips without a samples checkout).🤖 Generated with Claude Code
https://claude.ai/code/session_0182AiPodwfGRKPNZG9D5epe
Generated by Claude Code