diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index e5e47255..b73256c2 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -30,8 +30,10 @@ jobs: # this suite might grow, even though it no longer matters for # `tests/test_repo_hygiene.py::test_the_handbook_follows_the_readmes_ # outline` itself — #91 moved that check from diffing against a named - # commit to hashing the working tree's own README.md, which needs no - # history at all and runs the same in a shallow clone. + # commit to hashing the working tree's own README.md, and #519 to one + # hash per README `## ` section, each recorded by the handbook section + # that reads it. Neither needs history, and both run the same in a + # shallow clone. - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 with: fetch-depth: 0 @@ -152,8 +154,9 @@ jobs: # The SAME two commits the ubuntu job pins. Two jobs resolving two # versions of the same action is the drift this repository spends a # README section on. `fetch-depth: 0` for the reason that job gives — - # kept as a generally useful default now that #91 stopped the - # handbook's own staleness check from needing history at all. + # kept as a generally useful default now that #91 and #519's + # per-section hashes stopped the handbook's own staleness check from + # needing history at all. - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 with: fetch-depth: 0 @@ -224,8 +227,9 @@ jobs: # The SAME two commits the other two jobs pin. Three jobs resolving # three versions of one action is the drift this repository spends a # README section on. `fetch-depth: 0` for the reason the ubuntu job - # gives — kept as a generally useful default now that #91 stopped the - # handbook's own staleness check from needing history at all. + # gives — kept as a generally useful default now that #91 and #519's + # per-section hashes stopped the handbook's own staleness check from + # needing history at all. - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 with: fetch-depth: 0 diff --git a/docs/handbook.html b/docs/handbook.html index 4e01a52b..383e80e7 100644 --- a/docs/handbook.html +++ b/docs/handbook.html @@ -468,7 +468,16 @@
-
+
+ + + + + + + + +

Ratified standard · Apache-2.0

openRepoShape

@@ -548,6 +557,9 @@

The fastest path — macOS, Linux, WSL2

+ + +

The one thing to read before electing it

The shape is elective and confers nothing. Electing it changes no gate, @@ -596,6 +608,9 @@

The one thing to read before electing it

+ + +

Setup

Start here

Pick your machine. Each panel says what the machine needs, how the tool offers to supply @@ -1005,6 +1020,9 @@

What you have afterwards — the same on every platform

+ + +

Setup

If it fails part-way

Nothing is created before the typed yes: the preflight, the naming check and @@ -1037,6 +1055,11 @@

If it fails part-way

+ + + + +

Setup

Reading private legs in CI

The validate workflow’s default GITHUB_TOKEN cannot clone a @@ -1090,6 +1113,11 @@

Reading private legs in CI

+ + + + +

Reference

The three legs, and naming

@@ -1165,6 +1193,9 @@

The three legs, and naming

+ + +

Reference

Adopting an existing repository

A repository that already exists is converted in place. It KEEPS ITS NAME, @@ -1212,6 +1243,9 @@

Adopting an existing repository

+ + +

Reference

Families: a holder for projects that ship separately

A family is a repository that pins other projects’ assembly @@ -1315,6 +1349,11 @@

The InkRouter example

+ + + + +

Reference

The double pin, and the lockstep invariant

Each leg is pinned TWICE, in the same commit: by the gitlink git records @@ -1376,6 +1415,9 @@

Advancing a leg

+ + +

Reference

Bootstrap is COPIED into the project, not fetched

scripts/bootstrap.py, the validators and AGENTS-shape.md are @@ -1407,6 +1449,9 @@

Bootstrap is COPIED into the project, not fetched

+ + +

Reference

Keeping a project’s shape current

The copies are the trade: a project runs its own gate, which never consults this standard @@ -1446,8 +1491,10 @@

Keeping a project’s shape current

away. Once a human merges a shape/ copy and removes it, check calls its row merged-away and apply drops the row.
  • A file the standard ADDS later is reported as upstream-added and counts as something to do. apply writes NONE of them unless --add <path> - names one, per file; --add refuses a path that is already pinned, one the - target’s lists do not name, and one the root ALREADY has a file at.
    • + names one, per file, and names each file it adds in the line-ending block of an adopted + root’s own .gitattributes when that file carries #471’s block; + --add refuses a path that is already pinned, one the target’s lists do + not name, and one the root ALREADY has a file at.

    --upstream takes a path to a clone (offline) or an owner/repo it bare-clones itself; it defaults to the repository the pin names. make bootstrap @@ -1460,6 +1507,9 @@

    Keeping a project’s shape current

    + + +

    Reference

    Is this repository compliant? openRepoShape --doctor

    Five commands answered that question and the order of them was tribal knowledge. @@ -1591,6 +1641,11 @@

    Is this repository compliant? openRepoShape --doctor

    + + + + +

    Reference

    Carrying in-flight work to another workstation

    Two verbs on the assembly root, for the day the machine your half-finished work is on is @@ -1724,6 +1779,9 @@

    Who implements it, and what the record is

    + + +

    Reference

    The degrade rule

    make bootstrap looks for a wallet review-authority register at @@ -1738,6 +1796,9 @@

    The degrade rule

    + + +

    Reference

    Layout of this repository

    @@ -1851,6 +1912,9 @@

    For AI assistants

    diff --git a/tests/test_agents_md_codes.py b/tests/test_agents_md_codes.py index 89b1b343..171db24a 100644 --- a/tests/test_agents_md_codes.py +++ b/tests/test_agents_md_codes.py @@ -16,8 +16,8 @@ This is the guard, in both directions, and it is the kind this repository already keeps for other copied facts: the Windows two-liner's file names, the -handbook's README blob, and `docs/cli.md` against `--help` -(`tests/test_cli_reference.py`). +handbook's hash of each README section it reads, and `docs/cli.md` against +`--help` (`tests/test_cli_reference.py`). WHICH NAMES. Every backticked token in the document whose whole text matches `CODE_SHAPE` (`[a-z]+(-[a-z0-9]+)+`), never a pattern read out of a refusal diff --git a/tests/test_readme_sections.py b/tests/test_readme_sections.py new file mode 100644 index 00000000..dc4fcc0b --- /dev/null +++ b/tests/test_readme_sections.py @@ -0,0 +1,530 @@ +# SPDX-License-Identifier: Apache-2.0 +"""The machinery #519 put behind the handbook and README.md's line cap. + +`tests/test_repo_hygiene.py` runs it on the real files: README.md cut into +segments, each page section of `docs/handbook.html` recording the segments +it reads and their hashes, and the cap read from `tests/readme-cap.d/`. Those +two tests say whether the files agree TODAY; these say whether the checks +would notice when they stop agreeing, and what they would say. So every case +here is a few lines of fixture text, never the real files, and every message +the checks print is held to what it must tell the person who reads it. + +OS-neutral on purpose: a fixture spells its own line endings, so nothing here +depends on how the checkout wrote README.md (Windows CI checks out LF only +because `.gitattributes` says so), and nothing runs git. +""" + +from __future__ import annotations + +import pytest + +from test_repo_hygiene import ( + README_CAP_BASE, README_CAP_FORMAT, handbook_footer_problems, + handbook_marker_problems, readme_cap_ledger_refusal, readme_cap_raises, + readme_cap_refusal, readme_cap_sum, readme_fingerprint, readme_segments) + +#: A README in miniature: a preamble, a `## ` heading holding a `### ` and a +#: fenced `## ` line, and a heading with a colon in it holding a tilde fence. +README = ( + "# Thing\n" + "\n" + "Intro.\n" + "\n" + "## First\n" + "\n" + "```sh\n" + "## not a heading, a shell comment\n" + "```\n" + "\n" + "### Sub\n" + "\n" + "Body.\n" + "\n" + "## Second: with a colon\n" + "\n" + "~~~\n" + "## nor this\n" + "~~~\n" + "Tail.\n" +) + +#: Which README segments each part of the fixture page reads, in the order +#: its markers are written. `masthead` and `licence` become a `
    ` and +#: a `
    `, as on the real page; `notes` reads nothing, as `agents` does. +READS = { + "masthead": ["preamble", "First"], + "first": ["First"], + "second": ["Second: with a colon"], + "notes": [], + "licence": ["Second: with a colon"], +} + +HEADING_TO_ID = {"First": "first", "Second: with a colon": "second"} + +#: git's own blob hashes of three texts (`git hash-object --stdin`), so the +#: fingerprint is held to git rather than to a second copy of its formula. +GIT_BLOBS = { + "hello\n": "ce013625030ba8dba906f756967f9e9ca394464a", + "": "e69de29bb2d1d6434b8b29ae775ad8c2e48c5391", + "a\nb\n": "422c2b7ab3b3c668038da977e4e93a5fc623169c", +} + + +def table(reads): + """`reads` as HANDBOOK_READS spells a table: a frozenset per part.""" + return {part: frozenset(names) for part, names in reads.items()} + + +def blobs_of(readme): + return {name: readme_fingerprint(text) + for name, text in readme_segments(readme)} + + +def marker(name, blob): + return f" " + + +def page_reading(readme, reads=None, *, blob=None): + """A page whose parts carry markers for `reads`, at the top of each part, + alone between blank lines as on the real page, each at the hash `readme` + gives it now unless `blob` maps `(part, segment)` elsewhere.""" + current = blobs_of(readme) + lines = ["
    "] + for part, names in (READS if reads is None else reads).items(): + tag = {"masthead": "header", "licence": "footer"}.get(part, "section") + lines.append(f'<{tag} id="{part}">') + for name in names: + lines += ["", marker(name, (blob or {}).get((part, name)) + or current[name])] + lines += ["", f"

    What {part} says.

    ", f""] + lines += ["
    ", ""] + return "\n".join(lines) + + +def problems_with(page, readme=README, heading_to_id=None, reads=None): + return handbook_marker_problems( + readme, page, HEADING_TO_ID if heading_to_id is None + else heading_to_id, table(READS if reads is None else reads)) + + +def only(problems, *fragments): + """The one problem holding every fragment; fails if there is not one.""" + found = [p for p in problems if all(f in p for f in fragments)] + assert len(found) == 1, (fragments, problems) + return found[0] + + +# ------------------------------------------------------------- the segments + + +def test_the_segments_hold_every_byte_once_in_order(): + segments = readme_segments(README) + assert [name for name, _ in segments] == [ + "preamble", "First", "Second: with a colon"] + assert "".join(text for _, text in segments) == README + assert segments[0][1] == "# Thing\n\nIntro.\n\n" + assert segments[1][1].startswith("## First\n") + assert segments[2][1].endswith("Tail.\n") + + +def test_a_fenced_heading_and_a_subsection_stay_in_their_segment(): + segments = dict(readme_segments(README)) + assert "## not a heading, a shell comment\n" in segments["First"] + assert "### Sub\n" in segments["First"] + assert "## nor this\n" in segments["Second: with a colon"] + + +def test_the_preamble_comes_first_and_an_empty_one_asks_for_no_marker(): + readme = "## Only\n\ntext\n" + assert readme_segments(readme) == [("preamble", ""), + ("Only", "## Only\n\ntext\n")] + reads = {"only": ["Only"]} + assert problems_with(page_reading(readme, reads), readme, + {"Only": "only"}, reads) == [] + + +def test_crlf_cuts_and_hashes_as_lf_and_is_refused_in_the_readme(): + crlf = README.replace("\n", "\r\n") + assert [n for n, _ in readme_segments(crlf)] == [ + n for n, _ in readme_segments(README)] + assert "".join(t for _, t in readme_segments(crlf)) == crlf + assert blobs_of(crlf) == blobs_of(README) + problems = problems_with(page_reading(README), crlf) + said = only(problems, "README.md:1: a carriage return (20 in the file)") + assert "LF-only" in said and ".gitattributes" in said + assert len(problems) == 1, problems + + +def test_a_lone_carriage_return_is_refused_where_it_is(): + readme = README.replace("Body.\n", "Body.\rStill body.\n") + only(problems_with(page_reading(readme), readme), + "README.md:13: a carriage return (1 in the file)") + + +@pytest.mark.parametrize("text", sorted(GIT_BLOBS)) +def test_a_fingerprint_is_gits_blob_hash(text): + assert readme_fingerprint(text) == GIT_BLOBS[text] + assert readme_fingerprint(text.replace("\n", "\r\n")) == GIT_BLOBS[text] + + +def test_a_fingerprint_moves_with_one_byte(): + assert readme_fingerprint("a\nb\n") != readme_fingerprint("a\nb \n") + + +@pytest.mark.parametrize("added, setext", [ + (" ## Indented\n", False), + (" ## Indented three\n", False), + ("##\tTabbed\n", False), + ("##\n", False), + ("Paragraph text\n---\n", True), + ("Paragraph text\n -\n", True), + ("- item\n\nTop-level paragraph after the list\n---\n", True), +]) +def test_a_heading_github_renders_that_the_cut_misses_is_refused(added, + setext): + readme = README + "\n" + added + said = only(problems_with(page_reading(readme), readme), + "makes a `##` heading on GitHub") + line = README.count("\n") + 1 + added.count("\n") + assert said.startswith(f"README.md:{line}: `{added.splitlines()[-1]}`") + assert "write it as `## ` at the start of the line" in said + hint = "or, if you meant a horizontal rule, put a blank line above it" + assert (hint in said) is setext + + +@pytest.mark.parametrize("added", [ + "##Name with no space is a paragraph\n", + "---\n", + "| a | b |\n|---|---|\n| 1 | 2 |\n", + "- item\n---\n", + "# Title\n---\n", + "```\n ## inside a fence\n```\n", + " ## indented code\n", + # The review of PR #551's three: a horizontal rule after a list item's + # continuation line, after a lazy block-quote line, and after a line + # indented by a tab or four spaces. None is a heading. + "- item\n its continuation\n---\n", + "> quoted\nlazy continuation of the quote\n---\n", + "\tindented by a tab\n---\n", + " indented four spaces\n---\n", + "- item\n\n a second paragraph of the item\n---\n", +]) +def test_what_github_does_not_render_as_a_heading_passes(added): + readme = README + "\n" + added + assert problems_with(page_reading(readme), readme) == [] + + +# -------------------------------------------------------------- the markers + + +def test_a_page_that_read_every_segment_as_it_is_has_no_problem(): + assert problems_with(page_reading(README)) == [] + + +def test_a_renamed_heading_names_the_marker_and_the_headings_there_are(): + renamed = README.replace("## First\n", "## Primero\n") + problems = problems_with(page_reading(README), renamed, + {"Primero": "first"}) + said = only(problems, "names README.md § First,", "`first`") + assert "" + in said) + assert "row in HANDBOOK_READS" in said + assert len(problems) == 1, problems + + +def test_a_stale_segment_names_every_section_reading_it_and_the_new_value(): + moved = README.replace("Body.\n", "Body, changed.\n") + old, new = blobs_of(README)["First"], blobs_of(moved)["First"] + problems = problems_with(page_reading(README), moved) + said = only(problems, "README.md § First has moved") + assert "Page sections that read it: `masthead`, `first`." in said + assert f"blob {old}" in said + assert f"set each of their markers to blob {new}" in said + for meaning in ("Re-read what changed", "own design and voice", + "#89 is the precedent", + "moving the markers alone is honest", + "hand-editing a pin"): + assert meaning in said, meaning + assert len(problems) == 1, problems + + +def test_a_half_moved_segment_names_only_the_marker_still_behind(): + moved = README.replace("Body.\n", "Body, changed.\n") + new = blobs_of(moved)["First"] + page = page_reading(README, blob={("masthead", "First"): new}) + said = only(problems_with(page, moved), "README.md § First has moved") + assert "the marker in `first` (docs/handbook.html:" in said + assert "`masthead` (" not in said + assert "Page sections that read it: `masthead`, `first`." in said + + +def test_a_dropped_marker_the_table_lists_is_refused(): + """The review of PR #551 dropped the five secondary § Starting markers + and the old check passed: a section had to carry one marker, not every + one its reading needs. The table makes each of them owed.""" + page = page_reading(README, dict(READS, masthead=["preamble"])) + blob = blobs_of(README)["First"] + said = only(problems_with(page), "page section `masthead` reads " + "README.md § First by its row") + assert f"" in said + assert "changes when the page's structure does" in said + + +def test_a_marker_the_table_does_not_list_is_refused(): + page = page_reading(README, dict(READS, notes=["First"])) + said = only(problems_with(page), "page section `notes` carries a marker " + "for README.md § First") + assert "add it to the row" in said and "drop the marker" in said + + +def test_the_table_has_a_row_for_every_part_and_no_other(): + reads = dict(READS, extra=["First"]) + page = page_reading(README, reads) + said = only(problems_with(page), "page section `extra` has no row in " + "HANDBOOK_READS") + assert "changes when the page's structure does" in said + assert "a section added is a row added" in said + only(problems_with(page_reading(README), reads=reads), + "HANDBOOK_READS has a row for `extra`, and the page has no section") + + +def test_the_table_names_only_segments_the_readme_has(): + reads = dict(READS, first=["First", "Nope"]) + only(problems_with(page_reading(README), reads=reads), + "HANDBOOK_READS lists README.md § Nope for `first`") + + +def test_a_mapped_heading_is_in_its_own_sections_row(): + problems = problems_with(page_reading(README), + heading_to_id={"First": "second"}) + said = only(problems, "maps README.md § First to page section `second`") + assert "`second`'s row in HANDBOOK_READS does not list it" in said + + +FIRST = marker("First", blobs_of(README)["First"]) +PREAMBLE = marker("preamble", blobs_of(README)["preamble"]) + + +@pytest.mark.parametrize("before, after, why", [ + (f"{PREAMBLE}\n\n{FIRST}\n", f"{PREAMBLE} {FIRST.strip()}\n", + "shares its line"), + (f'
    \n\n{FIRST}', f'
    \n{FIRST}', + "has no blank line above it"), + (f"{FIRST}\n\n

    What first says.

    ", + f"{FIRST}\n

    What first says.

    ", "has no blank line below it"), + (f"{FIRST}\n\n

    What first says.

    ", + f"{FIRST}

    What first says.

    ", "shares its line"), + (f'
    \n\n{FIRST}\n\n

    What first says.

    \n', + f'
    \n\n

    What first says.

    \n\n{FIRST}\n\n', + "is not at the top of its section"), +]) +def test_a_marker_stands_alone_between_blank_lines(before, after, why): + """A marker beside page text conflicts with the pull request that + rewords that text for another README section; two on one line conflict + with each other. The review of PR #551 merged exactly that conflict, and + then a marker moved below a paragraph, blank lines and all, conflicting + with a branch that deleted the paragraph: so a marker is at the top.""" + page = page_reading(README) + assert before in page + said = only(problems_with(page.replace(before, after, 1)), why) + assert "stands alone on its line, with a blank line above and below" \ + in said + + +@pytest.mark.parametrize("comment", [ + "", + "", + "", + "", +]) +def test_a_mistyped_marker_is_refused_rather_than_skipped(comment): + page = page_reading(README).replace("

    What first says.

    ", comment) + said = only(problems_with(page), "reads like a marker and is not one") + assert comment in said and "docs/handbook.html:" in said + + +def test_a_marker_outside_every_page_section_is_refused(): + page = page_reading(README).replace("", FIRST.strip()) + only(problems_with(page), "§ First sits in no page section", + "docs/handbook.html:1:") + + +def test_one_section_reads_one_segment_once(): + page = page_reading(README, dict(READS, first=["First", "First"])) + only(problems_with(page), "page section `first` carries 2 markers for " + "README.md § First") + + +def test_two_segments_of_one_name_are_refused_and_offered_no_hash(): + readme = README + "\n## First\n\nAgain.\n" + problems = problems_with(page_reading(README), readme) + only(problems, "more than one segment called `First`") + assert not [p for p in problems if "§ First has moved" in p], problems + + +# --------------------------------------------------------------- the footer + +FOOTER = ('

    First cut (#71); a row ' + "added (#97); the verdict added (#502). That\n list is " + "history, and it ends there.

    \n

    Each section " + "that reads the README records it (#519).

    ") +CLAUSES = (97, 502) + + +def footer_problems(page): + return handbook_footer_problems(page, CLAUSES) + + +def test_a_closed_footer_history_has_no_problem(): + assert footer_problems(FOOTER) == [] + + +def test_a_new_clause_before_the_history_closes_is_refused(): + """The review of PR #551 slipped `a new row, added (#600)` in before the + closing sentence and the check, which looked only after it, passed. The + history is a fixed list of numbers now.""" + new = FOOTER.replace("(#502). That", "(#502); a new row, added (#600). " + "That") + said = only(footer_problems(new), "gives \"added (#N)\" clauses for " + "[97, 502, 600]") + assert "frozen at HANDBOOK_HISTORY_CLAUSES, [97, 502]" in said + assert "[600] is new" in said + + +def test_the_frozen_history_keeps_its_order(): + swapped = FOOTER.replace("added (#97)", "added (#X)").replace( + "added (#502)", "added (#97)").replace("added (#X)", "added (#502)") + said = only(footer_problems(swapped), "clauses for [502, 97]") + assert "in that order" in said and "is new" not in said + + +@pytest.mark.parametrize("retired", [ + "README.md blob ae817c5fdb324ff60440b959a15f3ad3b2a18b43", + "README.md\n blob ae817c5", + "README.md blob AE817C5FDB324FF6", +]) +def test_a_hash_of_the_whole_readme_is_refused_in_any_spelling(retired): + said = only(footer_problems(FOOTER.replace( + "
    ", f"

    {retired}

    ")), "retired by #519") + assert " ".join(retired.split()) in said + + +def test_an_added_clause_after_the_history_closes_is_refused(): + late = FOOTER.replace("records it (#519).", + "records it (#519); a section added (#560).") + only(footer_problems(late), "adds ['added (#560)'] after the footer's " + "history closed") + + +def test_the_history_closes_once(): + only(footer_problems(FOOTER.replace("it ends there", "it is")), + "must close the footer's history once", "it does 0 times") + + +# ---------------------------------------------------------- the cap ledger + + +def write_ledger(root, files): + ledger = root / "readme-cap.d" + ledger.mkdir() + for name, data in files.items(): + (ledger / name).write_bytes(data) + return ledger + + +def test_no_ledger_raises_nothing(tmp_path): + assert readme_cap_raises(tmp_path / "readme-cap.d") == ([], []) + assert readme_cap_sum([]).startswith(f"{README_CAP_BASE}, with no raise") + + +def test_each_raise_adds_its_lines_whatever_the_line_endings(tmp_path): + ledger = write_ledger(tmp_path, { + "519.txt": b"+8\n2026-10-14\nWhy the README earns eight lines.\n", + "1000.txt": b"+3\r\n2026-10-15\r\nA reason, on\r\ntwo lines.\r\n", + "1001.txt": b"\xef\xbb\xbf+2\r\n2026-10-16\r\nPowerShell's BOM.\r\n", + }) + raises, problems = readme_cap_raises(ledger) + assert problems == [] + assert [(path.name, n) for path, n in raises] == [ + ("519.txt", 8), ("1000.txt", 3), ("1001.txt", 2)] + assert readme_cap_sum(raises) == ( + f"{README_CAP_BASE} + 8 (519.txt) + 3 (1000.txt) + 2 (1001.txt) = " + f"{README_CAP_BASE + 13}") + + +def test_dotfiles_and_editor_backups_are_not_read(tmp_path): + ledger = write_ledger(tmp_path, { + "519.txt": b"+8\n2026-10-14\nA good one.\n", + ".DS_Store": b"\x00\x00\x00\x01Bud1", + "519.txt~": b"old", + ".519.txt.swp": b"\x00swap", + "519.txt.bak": b"+80\n", + "519.txt.orig": b"+80\n", + }) + raises, problems = readme_cap_raises(ledger) + assert problems == [] + assert [path.name for path, _ in raises] == ["519.txt"] + + +@pytest.mark.parametrize("name, data, fragment", [ + ("notes.md", b"+3\n2026-10-14\nr\n", "is not named `"), + ("0519.txt", b"+3\n2026-10-14\nr\n", "is not named"), + ("519.txt.new", b"+3\n2026-10-14\nr\n", "is not named"), + ("520.txt", b"8\n2026-10-14\nr\n", "line 1 must be `+N`"), + ("521.txt", b"+0\n2026-10-14\nr\n", "line 1 must be `+N`"), + ("522.txt", b"", "line 1 must be `+N`"), + ("523.txt", b"+3\n14 October 2026\nr\n", "line 2 must be"), + ("524.txt", b"+3\n2026-02-30\nr\n", "line 2 must be"), + ("525.txt", b"+3\n2026-10-14\n", "gives no reason"), + ("526.txt", b"+3\n2026-10-14\n\n \n", "gives no reason"), + ("527.txt", b"+3\n2026-10-14\n\xff\n", + "is not UTF-8 (invalid start byte at byte 14); save it as UTF-8 " + "(PowerShell: Set-Content -Encoding utf8)"), + ("528.txt", "+3\n2026-10-14\nr\n".encode("utf-16"), + "is UTF-16; save it as UTF-8 (PowerShell: Set-Content -Encoding utf8)"), + ("529.txt", "+3\n2026-10-14\nr\n".encode("utf-16-le"), "is UTF-16"), +]) +def test_a_malformed_raise_is_named_by_its_path(tmp_path, name, data, + fragment): + ledger = write_ledger(tmp_path, { + "519.txt": b"+8\n2026-10-14\nA good one.\n", name: data}) + raises, problems = readme_cap_raises(ledger) + assert [path.name for path, _ in raises] == ["519.txt"] + assert len(problems) == 1 and fragment in problems[0], problems + assert problems[0].startswith((ledger / name).as_posix()) + said = readme_cap_ledger_refusal(problems) + assert said.startswith(problems[0]) and README_CAP_FORMAT in said + + +def test_a_directory_in_the_ledger_is_named_too(tmp_path): + ledger = write_ledger(tmp_path, {}) + (ledger / "530.txt").mkdir() + _, problems = readme_cap_raises(ledger) + assert problems == [f"{(ledger / '530.txt').as_posix()} is not a file, " + "and the ledger holds nothing else"] + + +def test_a_readme_over_the_cap_is_told_how_to_raise_it(tmp_path): + ledger = write_ledger(tmp_path, {"519.txt": b"+8\n2026-10-14\nWhy.\n"}) + raises, _ = readme_cap_raises(ledger) + said = readme_cap_refusal(README_CAP_BASE + 9, raises) + assert said.startswith( + f"README.md is {README_CAP_BASE + 9} lines; the cap is " + f"{README_CAP_BASE} + 8 (519.txt) = {README_CAP_BASE + 8}.") + assert README_CAP_FORMAT in said + for part in ("line 1 `+N`", "line 2 the date, `YYYY-MM-DD`", + "tests/readme-cap.d/519.txt", "UTF-8"): + assert part in said, part diff --git a/tests/test_repo_hygiene.py b/tests/test_repo_hygiene.py index 3761eb76..8df3a3e6 100644 --- a/tests/test_repo_hygiene.py +++ b/tests/test_repo_hygiene.py @@ -4,6 +4,8 @@ from __future__ import annotations import ast +import datetime +import hashlib import html import importlib.util import os @@ -697,8 +699,19 @@ def test_readme_is_short_enough_to_be_read(): first question anyone asks about it — family or one project? — is answered by a table and an example rather than by a definition, and because "what a holder does NOT confer" is the half that keeps it from - becoming a governance boundary.""" - lines = (REPO / "README.md").read_text().splitlines() + becoming a governance boundary. + + FROM 2026-10-14 A RAISE IS A FILE, NOT A COMMENT (#519). Every README + pull request used to edit this one function — a dated comment below and + the number in the assertion — so any two open at once conflicted here. + Brett Heap's ruling 29: *"Each line-cap raise becomes its own small file + with its reason."* The cap is now `README_CAP_BASE`, 1422, plus the sum of + the raises in `tests/readme-cap.d/`, one file per issue or pull request + that grows the README (`README_CAP_FORMAT` says what a file holds), and + each file's reason is argued the way this history argues each raise. This + docstring and the dated comments below are that history up to 1422, kept + as they were written; they do not grow.""" + lines = (REPO / "README.md").read_bytes().decode("utf-8").splitlines() # 2026-09-04: 508 -> 509 — the election reference paragraph now names the # ratified docs/project-repo-schema.md and keeps the staged path valid for # projects elected before ratification (Brett Heap's edit, PR #19). @@ -1183,8 +1196,145 @@ def test_readme_is_short_enough_to_be_read(): # shim's answer there changed from a clone of the upstream to a few # lines of its own, and a README that still promised the flag list # would send a reader to a command that no longer prints it. - assert len(lines) <= 1422, ( - f"README.md is {len(lines)} lines; the cap is 1422") + # 2026-10-14: the log above is history and stops here (#519). A raise from + # this date on is its own file in tests/readme-cap.d/, with its reason. + raises, problems = readme_cap_raises(README_CAP_LEDGER) + assert not problems, readme_cap_ledger_refusal(problems) + cap = README_CAP_BASE + sum(n for _, n in raises) + print(f"README.md is {len(lines)} lines; the cap is " + f"{readme_cap_sum(raises)}") + assert len(lines) <= cap, readme_cap_refusal(len(lines), raises) + + +#: README.md's line cap on 2026-10-14, the day its raises stopped being +#: comments in the test above (#519). Never moved again: a raise is a file. +README_CAP_BASE = 1422 + +#: The ledger of raises since then: one file per issue or pull request that +#: grows the README, so two README pull requests never edit one line (#519). +#: It does not exist until the first raise, and an absent ledger raises +#: nothing. Its format is carried by `README_CAP_FORMAT`, in every message +#: about it, rather than by a README of its own a reader may never open. +README_CAP_LEDGER = REPO / "tests" / "readme-cap.d" + +README_CAP_FORMAT = ( + "one UTF-8 text file per issue or pull request that grows README.md, " + "named for its number (`tests/readme-cap.d/519.txt`): line 1 `+N`, N " + "the lines it adds, a positive integer; line 2 the date, `YYYY-MM-DD`; " + "then one or more lines saying why the README earns them, argued the " + "way the history in `test_readme_is_short_enough_to_be_read` argues " + "each raise") + +#: A ledger file's name, line 1 and line 2, as `README_CAP_FORMAT` says them. +README_CAP_NAME = re.compile(r"[1-9][0-9]*\.txt") +#: What the ledger may hold besides raises, unread: a dotfile (`.DS_Store`, a +#: vim swap file) or an editor's backup. Any other name is refused. +README_CAP_IGNORED = re.compile(r"\..*|.*(?:~|\.swp|\.bak|\.orig)") +#: How Windows PowerShell 5.1's `>` and `Out-File` write a file unless told +#: otherwise: UTF-16 with a byte-order mark. NULs between ASCII bytes are the +#: same encoding without one. +README_CAP_UTF16 = (b"\xff\xfe", b"\xfe\xff") +README_CAP_SAVE = "save it as UTF-8 (PowerShell: Set-Content -Encoding utf8)" +README_CAP_RAISE = re.compile(r"\+([1-9][0-9]*)") +README_CAP_DATE = re.compile(r"[0-9]{4}-[0-9]{2}-[0-9]{2}") + + +def _shown(path): + """A path as a message names it: from the repository root, if under it.""" + try: + return path.resolve().relative_to(REPO).as_posix() + except ValueError: + return path.as_posix() + + +def readme_cap_raise_problem(name, data): + """What is wrong with one ledger file, or None (#519). + + `name` is the file's name and `data` its bytes, so a fixture can be + checked without a directory. Line endings are whatever the checkout + wrote: `splitlines` reads LF and CRLF alike. A UTF-8 byte-order mark, + which PowerShell 5.1's `-Encoding utf8` writes, is read past. + """ + if not README_CAP_NAME.fullmatch(name): + return ("is not named `.txt`, and the ledger " + "holds nothing else") + if data.startswith(README_CAP_UTF16) or b"\0" in data: + return f"is UTF-16; {README_CAP_SAVE}" + try: + lines = data.decode("utf-8-sig").splitlines() + except UnicodeDecodeError as error: + return (f"is not UTF-8 ({error.reason} at byte {error.start}); " + f"{README_CAP_SAVE}") + first = lines[0].strip() if lines else "" + if not README_CAP_RAISE.fullmatch(first): + return (f"line 1 must be `+N`, N a positive integer; it is " + f"{first!r}") + second = lines[1].strip() if len(lines) > 1 else "" + try: + dated = (README_CAP_DATE.fullmatch(second) + and datetime.date.fromisoformat(second)) + except ValueError: + dated = None + if not dated: + return (f"line 2 must be the date the raise was made, `YYYY-MM-DD`; " + f"it is {second!r}") + if not any(line.strip() for line in lines[2:]): + return ("gives no reason: line 3 on says why the README earns the " + "lines, and a raise with no reason is the cap moving for " + "prose") + return None + + +def readme_cap_raises(ledger): + """Every raise in the ledger as `(path, N)`, and every malformed file. + + The second list holds one message per file that is not a raise, each + naming its path. A ledger that is not there raises nothing, and a + dotfile or an editor's backup in it is not read (`README_CAP_IGNORED`). + """ + raises, problems = [], [] + if not ledger.is_dir(): + return raises, problems + for path in sorted(ledger.iterdir(), key=lambda p: (len(p.name), p.name)): + if README_CAP_IGNORED.fullmatch(path.name): + continue + if not path.is_file(): + problems.append(f"{_shown(path)} is not a file, and the ledger " + "holds nothing else") + continue + data = path.read_bytes() + problem = readme_cap_raise_problem(path.name, data) + if problem: + problems.append(f"{_shown(path)} {problem}") + continue + first = data.decode("utf-8-sig").splitlines()[0].strip() + raises.append((path, int(first[1:]))) + return raises, problems + + +def readme_cap_sum(raises): + """The cap, and how it is made: `1422 + 8 (519.txt) = 1430`.""" + if not raises: + return (f"{README_CAP_BASE}, with no raise in " + f"{_shown(README_CAP_LEDGER)}/") + terms = " + ".join(f"{n} ({path.name})" for path, n in raises) + total = README_CAP_BASE + sum(n for _, n in raises) + return f"{README_CAP_BASE} + {terms} = {total}" + + +def readme_cap_refusal(count, raises): + """The message when README.md is longer than the cap allows.""" + return (f"README.md is {count} lines; the cap is " + f"{readme_cap_sum(raises)}. A README that has to grow raises the " + f"cap in a file of its own - {README_CAP_FORMAT}. A cap that " + "never moves for a rule pushes the rule into tribal memory; a cap " + "that moves for prose is not a cap.") + + +def readme_cap_ledger_refusal(problems): + """The message when a ledger file is not a raise: each path, then how.""" + return ("\n".join(problems) + + f"\nA raise of README.md's line cap is {README_CAP_FORMAT}.") #: #476: what each document that carries the first rule says an assistant does @@ -1646,11 +1796,15 @@ def test_every_naming_families_count_matches_the_policy(): #: for the assistant rules. A section appearing with no entry in either table #: fails, so a page section can never become unexplained. HANDBOOK_SECTION_WITHOUT_A_README_H2 = { - "confers": "the README's `> [!IMPORTANT]` callout, promoted to the top", + "confers": "the README's `> [!IMPORTANT]` callout, promoted to the top, " + "with the rest of the preamble around it", "partway": "the `> [!WARNING]` inside `### A worked example: Northwind " - "starts Atlas`", + "starts Atlas`, inside § Starting a project in a new " + "organisation", + #: It reads § Starting too: the worked example's own paragraph on + #: private legs is said on the page here and nowhere else (#519). "ci": "`### Reading private legs in CI: a GitHub App first, " - "SHAPE_LEGS_TOKEN as fallback`", + "SHAPE_LEGS_TOKEN as fallback`, inside § The three legs", "agents": "AGENTS.md — the rules that outrank the rest, not README.md", #: `### Is this repository compliant? openRepoShape --doctor`, which sits #: under § Keeping a project's shape current in the README (#95). Its own @@ -1662,42 +1816,579 @@ def test_every_naming_families_count_matches_the_policy(): "inside § Keeping a project's shape current", } -#: The footer carries the page's history as PROSE — "first cut from README.md -#: at c093813 (#71); regenerated at cbca5b4 (#89)" — and, separately, its ONE -#: MACHINE-CHECKED fact: `README.md blob ` (#91). A commit sha named -#: where the page came FROM and had to exist as a commit object to diff -#: against; a blob hash names WHAT THE PAGE SAYS and needs no history at all -#: — `git hash-object README.md` on the bare working tree file, in a shallow -#: clone or a deep one, is the whole check. Matched on the literal word -#: "blob" rather than a bare hex run, so it can never pick up either commit -#: sha sitting in the prose sentence beside it, and `{40}` rather than -#: `{7,40}` because `git hash-object` always prints the full hash. -HANDBOOK_README_BLOB = re.compile(r"README\.md blob ([0-9a-f]{40})\b") - +#: The footer once carried the page's ONE MACHINE-CHECKED fact, `README.md +#: blob ` (#91): the blob hash of the whole README, which had to equal +#: `git hash-object README.md`. Every change to the README moved it, so every +#: README pull request edited that one footer line - and, by convention, an +#: "added (#N)" clause in the prose beside it - and any two open at once +#: conflicted there (#519); and a red check named the whole page to re-read +#: when one section of the README had moved. Brett Heap's ruling 29 +#: (2026-10-14) retired it for the markers below, and the footer's prose is +#: kept as history. This pattern stays to assert that the fact is GONE in any +#: spelling a re-wrap or a short hash could give it: `\s+` crosses a line +#: break, and 7 to 40 hex digits in either case is every way git prints one. +HANDBOOK_RETIRED_README_BLOB = re.compile( + r"README\.md\s+blob\s+[0-9a-fA-F]{7,40}") + +#: The sentence that closes the footer's history (#519). Before it are the +#: "added (#N)" clauses README pull requests wrote there until ruling 29, +#: kept as they were written, and nothing follows it. +HANDBOOK_HISTORY_END = re.compile( + r"That\s+list\s+is\s+history,\s+and\s+it\s+ends\s+there\.") +HANDBOOK_ADDED_CLAUSE = re.compile(r"added\s+\(#([0-9]+)\)") + +#: THE HISTORY, FROZEN (#519): the issue or pull request numbers of the +#: footer's "added (#N)" clauses, in the order the footer gives them. The +#: footer carries exactly these, in this order and before the closing +#: sentence, so a new clause is refused wherever it is put - before the +#: closing sentence as much as after it. The convention retired with +#: ruling 29; a section's markers say what it read, and the test checks +#: them. This tuple does not grow: its last entry, #471, is the clause #511 +#: wrote under the old rule, which landed in v1.0.0 before this check did. +HANDBOOK_HISTORY_CLAUSES = ( + 97, 99, 154, 163, 211, 207, 288, 289, 383, 341, 477, 411, 424, 476, 492, + 483, 498, 462, 487, 500, 499, 502, 471) + +#: What a marker calls the text before README.md's first `## ` heading. Every +#: other segment is called by its heading's text. +README_PREAMBLE = "preamble" + +#: HOW THE PAGE SAYS WHAT IT READ (#519). Every page section that reads a +#: README segment carries one HTML comment per segment it reads, exactly +#: +#: +#: +#: the hash being `readme_fingerprint` of that segment as the section last +#: read it. The markers stand at the TOP of the section, right after its +#: opening tag, each ALONE ON ITS LINE WITH A BLANK LINE ABOVE AND BELOW IT, +#: and the check says so. Git merges two pull requests that each change one +#: line only when an unchanged line stands between them, so a marker beside +#: page text conflicts with whichever pull request rewords that text for +#: another README section, and two markers side by side conflict with each +#: other; keeping two README pull requests apart is what the markers are for. +#: `[^\n]+?` and the literal `: blob ` after it read a heading that holds a +#: colon (`Families: a holder ...`) whole. +HANDBOOK_MARKER = re.compile( + r"") + +#: Every HTML comment on the page. One that mentions the README, or opens +#: with `reads`, is a marker or a mistake: a mistyped marker would drop out of +#: every check below without a word, so it is refused instead. +HANDBOOK_COMMENT = re.compile(r"", re.S) + +#: The parts of the page a marker sits in: the masthead (`
    `), each `
    `, and the `