[brettheap]

Closes #1285

Milestone v1.2.2: v1.2.1 is frozen, and lane 3 places this PR by its READY time.

Base: 3add78e1c (3add78e), which was origin/main when the branch was made, about 12:27Z (lane 1 saw it at 12:20Z too). Head: bd84004f3 (bd84004), three commits on it: d4f5283b0 (d4f5283: the reads, the writes and row 16), 84e006269 (84e0062: three asserts that judge a read's decoding, the issue's third box) and bd84004f3 (row 11's bytes compare held in a name, see Why). origin/main has moved since: at 17:14Z it is dc9a0bcc7 (dc9a0bc), four commits on 3add78e1c that change adopt-project.py and six test files (tests/test_adopt_github_exits.py, tests/test_adopt_partial_clone_source.py, tests/test_adopt_plan_against_its_source.py, tests/test_adopt_plan_and_remote_paths_spelled.py, tests/test_adopt_plan_value_order.py and tests/test_adopt_relative_submodule_url.py), none of them these two; git merge-tree --write-tree bd84004f3 dc9a0bcc7 is clean (exit 0, tree d55c242a97406d263fe2922eef9184cdc59c849a), and the tests were not run on that merge. The PR changes two test files and nothing else: tests/test_family.py (blob 16e2a4a42 to 4a7820b03) and tests/test_pinned_copy_text.py (blob d99414c87 to 376016906), +71 -30 against 3add78e1c. No tool, no template and no tests/conftest.py changes. This body is ASCII, apart from the footer's emoji: [caf\xe9] is pytest's id for the name caf plus U+00E9.

What

Under an ASCII locale (LC_ALL=C PYTHONUTF8=0 PYTHONCOERCECLOCALE=0) the 16 rows of #1285 failed on main. Re-measured on 3add78e1c: the same 16 ids fail, no id of the issue passes, and no other id of the two files fails. "Row N" below is the issue's number N.

Rows 1 to 15, tests/test_family.py: each failed with UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 at a read_text() with no encoding, of a file that holds U+2014 as UTF-8 (the Makefile, README.md, AGENTS-shape.md and .gitattributes of a family root, family.yaml, the copied and the upstream validate-family.py, a sibling's project.yaml). Now the 18 bare read_text() calls the 15 rows reach read with encoding="utf-8", as #1278's test_init_writes_the_holders_agent_files does, and the 13 write_text() calls that put such text back take encoding="utf-8" too. No helper serves the rows: each reads inline, so the keyword goes on each call (22 hunks, every one inside one of the 15 test bodies).

Row 16, tests/test_pinned_copy_text.py: test_a_printable_pin_source_that_is_not_a_repository_is_named_as_typed[caf\xe9] failed with UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' at source.mkdir(), before any tool ran. It now asks skip_where_the_environment_cannot_carry(name) just before source.mkdir(), as #1278's rows 9 to 12 do. Under ASCII [caf\xe9] skips with the guard's reason, this system's file name encoding (ascii) cannot spell 'caf\xe9' (tests/conftest.py:1281), and [plain] still runs and passes. No assert ran before that line, so none is lost.

The third box, rows 1, 3 and 11, tests/test_family.py: the issue's third box asks that "a mutant that reads one of the files as Latin-1 fails the row, as RV1278's MX1 did for row 7". On d4f5283b0 it was not met: three mutants that read a file as Latin-1 (M8, M9, M11) survived in every locale, because no assert of those rows saw a character that is not ASCII. Three asserts, in the three rows, now do, and the box is met by them for those three files. Other reads of the 15 rows are still judged by the ASCII locale alone (Known limits):

• Row 1, test_init_copies_the_siblings_utility_and_the_makefile_target: after its two in makefile asserts, it asserts the phrase WORKING CLONES BESIDE THIS HOLDER, a space, the U+2014 and not in, from line 16 of the Makefile, as tests: under a Latin-1 locale 12 of the 13 rows of #1259 skip on their own premise or read their UTF-8 file as UTF-8, and the 13th, the assembly root bootstrap.py saying it could not create a branch it made, is left failing for its own issue (#1259) #1278's test_init_writes_the_holders_agent_files asserts its own U+2014 (the character is written as itself in the test's source). That line is the only one of the Makefile with a character that is not ASCII. It says where the targets that go through siblings.py --make (park and resume) work: in the members' working clones BESIDE the holder, which make siblings places, not in members/<Project>. Read as Latin-1, the phrase is not in the text. This kills M9.
• Row 3, test_validate_finds_a_pin_that_disagrees_with_the_gitlink: it takes family.yaml's bytes before the rewrite, asserts they are not ASCII (the premise), and asserts that the file's bytes after the rewrite equal before.replace(commit.encode("ascii"), parent.encode("ascii")), as the row's own text.replace does on the text. A read as Latin-1 written back as UTF-8 doubles every U+2014, so the bytes differ. This kills M8.
• Row 11, test_update_shape_reads_a_family_root_and_mirrors_into_family_yaml: the same for the upstream validate-family.py: its old bytes are not ASCII, and its bytes after the rewrite equal the old bytes plus the appended line encoded as UTF-8. This kills M11.

Each failure message says what broke ("the rewrite changed more than the pin's commit: family.yaml was read or written as something other than UTF-8"), and each premise says what drifted ("fixture drift: family.yaml holds no character outside ASCII, so the next assert cannot tell a read as UTF-8 from a read as anything else"). M8, M9 and M11 now fail under ASCII, UTF-8 and Latin-1 (see Mutants); no id of the two files changes its outcome in any locale against d4f5283b0's runs.

Under ASCII the two files go from 16 failed, 221 passed, 6 skipped in 160.97s to 236 passed, 7 skipped in 204.28s. Under UTF-8 (243 passed in 219.47s) and Latin-1 (236 passed, 7 skipped in 200.97s) no id changes its outcome (see Measured).

Why

The tool is right; the rows read UTF-8 files in the locale's encoding. The 15 failing reads are UnicodeDecodeError at the first byte 0xe2 (positions 133, 135, 916, 1118 or 3020 of the file read). In the templates, the first non-ASCII byte of family.yaml (133), Makefile (916), .gitattributes (1118) and validate-family.py (3020) is the start of U+2014 (e2 80 94), checked with a script; position 135 is a sibling's rendered project.yaml, not checked. Nothing ran the tool wrongly: under the same ASCII locale 243 ids of the two files give 236 passed, 7 skipped in 204.28s once the reads say what the files are. Latin-1 differs only in that it never fails: it decodes every byte, so on main those reads return mojibake and the rows pass on ASCII-only phrases. That is why they were not in #1259's count, which is Latin-1's.

The decision: encoding="utf-8" on each read, and on each write of what was read. It is #1278's keyword on row 7, one call at a time. The writes need it for the same reason: text read as UTF-8 holds U+2014, which ASCII cannot write and Latin-1 cannot encode (M4 and M10 below show the write alone failing). Without the writes the rows would still fail under ASCII, at the write, and would newly fail under Latin-1, where on main they pass.

Rejected:

• A helper (read_utf8, or one that reads, replaces and writes back). Each of the 15 bodies would change to call it, the keyword is the whole fix, and a helper for the 3 identical source.write_text(source.read_text() + ...) lines would be new text to carry. tests: under a Latin-1 locale 12 of the 13 rows of #1259 skip on their own premise or read their UTF-8 file as UTF-8, and the 13th, the assembly root bootstrap.py saying it could not create a branch it made, is left failing for its own issue (#1259) #1278 chose the keyword in line.
• read_bytes().decode("utf-8"). The same reading in more text; row 12 already does it for a README, and the file's other reads use the keyword.
• Skipping the 15 rows under ASCII. The rows do not need a name ASCII cannot carry: the tool answers rightly, and each judges under ASCII once it reads the file as the UTF-8 it is. A skip would hide that.
• Guarding row 16 with the literal "caf\u00e9". The literal skips [plain] too, which passes under ASCII. The guard is asked of name, so only [caf\xe9] skips. A skipif on the locale, or a guard at the top of the row, loses nothing here (no assert precedes the line) but is not what tests: under a Latin-1 locale 12 of the 13 rows of #1259 skip on their own premise or read their UTF-8 file as UTF-8, and the 13th, the assembly root bootstrap.py saying it could not create a branch it made, is left failing for its own issue (#1259) #1278 did for rows 9 to 12.
• A conftest change (an autouse PYTHONUTF8, or a helper in tests/conftest.py). It would change what every other module reads, which this issue does not ask.

The third box: three asserts, each on a file its row reads, and each judging the bytes. The reads the issue's 15 rows make go to ASCII phrases, or are written back with a comment's bytes changed, so a read as Latin-1 changes no assert's answer there (w1285a's M8, M9 and M11 passed in all three locales). A row has to see a character that is not ASCII to judge the decoding. Rows 1, 3 and 11 are the rows of M8, M9 and M11: a read-and-phrase (row 1) and two read-then-rewrite rows whose file the test writes back (rows 3 and 11). The asserts are the shortest that do it: row 1 a phrase, rows 3 and 11 a compare of bytes with the premise that the file holds a character that is not ASCII, because without the premise a file that lost its non-ASCII text (a template edited to plain ASCII) would make the compare pass for any codec (M16 and M19 below show it).

Rejected for the third box:

• Comparing the text read to read_bytes().decode("utf-8"). It judges the read only. The compare of bytes after the rewrite judges the read and the write together, and a Latin-1 read written back as UTF-8, the mutant, is exactly what it fails (M8, M11).
• A phrase from the Makefile's first lines. They are all ASCII; line 16 is the one line with the U+2014, so any other phrase would not judge the decoding.
• An assert on every read of the 15 rows. Each would pin more of a template's wording and the box asks for one file's mutant to fail its row. Rows 1, 3 and 11 meet it for three files; the other reads are guarded by the ASCII locale (see Known limits).
• assert source.read_bytes() == old_bytes + ... for row 11. This was the first version. A mutant that read the file as Latin-1 made it fail, and pytest then spent more than four minutes of CPU before it reported the failure (stopped by me with TERM, no result). The cause is pytest's diff of the two values for the failing ==, validate-family.py being 268 KB: _compare_eq_iterable runs it at any verbosity on CI, and called alone on those two values (_compare_eq_any, -v) it had not finished when timeout stopped it at 150 s. The compare is now held in a name, rewritten, which the assert names, and a comment says why; the rerun of the same mutant on the new head failed in 14 to 18 seconds. Row 3's file, family.yaml, is 4.6 KB and keeps its direct compare, which fails in seconds (M8, M17).

Measured

Names in parentheses below, such as cmpids.py, reads.py, importers.txt and logs/, are files in lane 1's scratch, not in this PR.

Python 3.12.3, pytest 9.1.1, git 2.43.0, in the probe environment (a stub gh first on PATH, GH_TOKEN=bogus-w1285a for main's and the first head's runs and bogus-w1285b for the runs of this round, an empty GH_CONFIG_DIR, GIT_CONFIG_GLOBAL set to the null device, GIT_SSH_COMMAND=false, EMAIL set, LANES_LANE unset). A Python started in each locale reports ascii, utf-8 and iso8859-1 for sys.getfilesystemencoding() (ASCII is LC_ALL=C PYTHONUTF8=0 PYTHONCOERCECLOCALE=0; UTF-8 is LC_ALL=C.UTF-8; Latin-1 is LOCPATH of the build lane 1's writers made with localedef -c, LC_ALL=en_US.ISO-8859-1, PYTHONUTF8=0, as tests/test_adopt_one_byte_locale.py's latin1 fixture builds it). The host ran other lanes' tests (load average 25 to 55), so the times are not comparable with a quiet run's.

The two files, -v -rfs, on main 3add78e1c (w1285a's runs) and on head bd84004f3 (this round's runs):

Locale	main	head
ASCII	16 failed, 221 passed, 6 skipped in 160.97s	236 passed, 7 skipped in 204.28s
UTF-8	243 passed in 198.24s	243 passed in 219.47s
Latin-1	236 passed, 7 skipped in 166.32s	236 passed, 7 skipped in 200.97s

By test id, the 243 ids of the two files (FAILED, PASSED, SKIPPED):

#	Test id (file)	ASCII	UTF-8	Latin-1
1	test_init_copies_the_siblings_utility_and_the_makefile_target	FAILED -> PASSED	PASSED -> PASSED	PASSED -> PASSED
2	test_the_holder_says_what_its_bytes_are	FAILED -> PASSED	PASSED -> PASSED	PASSED -> PASSED
3	test_validate_finds_a_pin_that_disagrees_with_the_gitlink	FAILED -> PASSED	PASSED -> PASSED	PASSED -> PASSED
4	test_validate_accepts_a_member_bump_that_is_staged_but_not_committed	FAILED -> PASSED	PASSED -> PASSED	PASSED -> PASSED
5	test_validate_finds_a_digest_that_does_not_recompute	FAILED -> PASSED	PASSED -> PASSED	PASSED -> PASSED
6	test_validate_finds_an_id_that_is_not_the_project_mounted_there	FAILED -> PASSED	PASSED -> PASSED	PASSED -> PASSED
7	test_validate_finds_an_edited_shape_copy	FAILED -> PASSED	PASSED -> PASSED	PASSED -> PASSED
8	test_bump_to_the_commit_already_pinned_leaves_an_uncommitted_edit_alone	FAILED -> PASSED	PASSED -> PASSED	PASSED -> PASSED
9	test_bump_finishes_a_bump_an_earlier_run_staged_and_did_not_commit	FAILED -> PASSED	PASSED -> PASSED	PASSED -> PASSED
10	test_bump_to_the_pinned_commit_repairs_a_digest_that_does_not_recompute	FAILED -> PASSED	PASSED -> PASSED	PASSED -> PASSED
11	test_update_shape_reads_a_family_root_and_mirrors_into_family_yaml	FAILED -> PASSED	PASSED -> PASSED	PASSED -> PASSED
12	test_update_shape_moves_the_holder_readmes_shape_line_with_the_pin	FAILED -> PASSED	PASSED -> PASSED	PASSED -> PASSED
13	test_update_shape_with_no_shape_block_names_the_family_validator	FAILED -> PASSED	PASSED -> PASSED	PASSED -> PASSED
14	test_no_members_still_finds_an_edited_copy	FAILED -> PASSED	PASSED -> PASSED	PASSED -> PASSED
15	test_siblings_skips_a_clone_whose_project_id_is_not_the_rows	FAILED -> PASSED	PASSED -> PASSED	PASSED -> PASSED
16	test_a_printable_pin_source_that_is_not_a_repository_is_named_as_typed[caf\xe9] (test_pinned_copy_text.py)	FAILED -> SKIPPED	PASSED -> PASSED	PASSED -> PASSED
	the other 227 ids	221 PASSED and 6 SKIPPED, the same ids on both	227 PASSED, the same	220 PASSED and 7 SKIPPED, the same ids on both

Compared id by id with a script over the saved -v output of the runs (cmpids.py): main against head, 16 ids change under ASCII, none under UTF-8, none under Latin-1, and the 243 ids are the same set in all six runs; head against d4f5283b0, no id changes in any locale, and each locale's saved list (the sorted id OUTCOME lines of the -v output) is byte for byte the one of d4f5283b0 (equal sha256). The skips main already had are unchanged: under ASCII 6 (tests/conftest.py:1323, the stdout cannot spell U+2014 or U+2192), under Latin-1 7 (the same 6 and tests/conftest.py:1334, Co-Authored-By: Jos\xe9). The one new skip is the guard's, tests/conftest.py:1281, under ASCII only.

The other reads in tests/test_family.py

Every read_text(), write_text(), read_bytes(), open() and .decode() of the file, at head (an ast scan of the file, reads.py), after the 18 and 13 of the first commit:

Kind	Count	Verdict
read_text(encoding=...)	29 (11 on main, 18 added)	all "utf-8"
bare read_text(), left	5	none can read a byte that is not ASCII (below)
write_text(..., encoding=...)	19 (6 on main, 13 added)	all "utf-8"
bare write_text(), left	10	each writes an ASCII literal (below)
read_bytes()	21 calls on 18 lines (17 on 14 at d4f5283b0; this round adds 4: before and the compare in row 3, old_bytes and rewritten in row 11)	each compares or holds bytes and never decodes them
.decode("utf-8")	2 (readme_shape_lines(before.decode(...)), and after, in row 12)	already UTF-8
open() of a file	0 (Family.open(holder) is not one)	none

The 5 bare reads left, by head's line (the same five as at d4f5283b0, 12 and 25 lines further down):

Line	Read	Can it read a byte that is not ASCII?
1240	(member / "NOTES.md").read_text() == "edited in place\n"	no: the test wrote it at line 1230, "edited in place\n"
1554	"IRSS" not in (holder / ".gitmodules").read_text()	no, in these runs: git's file after remove, holding members/IRRS and the url of the fixture's own tmp_path, which is ASCII under pytest's default basetemp; the assert asks for an ASCII word absent. Left (see Known limits)
2105	(sibling / "SCRATCH.md").read_text() == "uncommitted work\n"	no: the test wrote it
2128	(wrong / "MINE.md").read_text() == "somebody else's work\n"	no: the test wrote it
2144	(estate["folder"] / "IRRS" / "notes.md").read_text() == "mine\n"	no: the test wrote it

The 10 bare writes left, by head's line, are 113, 1070, 1229, 1230, 1429, 1990, 2097, 2120, 2140 and 2406; each writes an ASCII literal (NOTES.md, MOVED.md, UNTRACKED.md, TRAILED.md, MINE.md, SCRATCH.md, notes.md, and the .gitmodules text of MEMBERS = ("IRRS", "IRSS")), checked by ast.get_source_segment(...).isascii() at d4f5283b0 (the lines have only moved since). The file has two non-ASCII string literals in its code, both asserted on purpose: the dash in test_init_writes_the_holders_agent_files's "Read AGENTS-shape.md first ..." (line 364; that row already reads with encoding="utf-8") and the dash of row 1's phrase (line 323, this round's); line 1531's Jos\udce9 is an ASCII escape, and every other non-ASCII character is in a docstring or a comment.

Mutants

Twenty-five mutants, each applied alone in a scratch clone of head (bd84004f3, git clone --shared, fast-forwarded from d4f5283b0), run over the ids of the mutated row only, under ASCII, UTF-8 and Latin-1. M1 to M14 are w1285a's, run again on this head (M5's and M11's anchor changed with row 11's rewrite, which now ends + appended,); M15 to M25 are new. After each, every file the mutant touched was written back from git cat-file blob HEAD:<path> by a script and git hash-object printed the blob at HEAD, EQUAL each time (26 restores, one for each file a mutant touched, logs/mut-restore.out): 4a7820b030d62fd04775364dd181707e89e7ee90 for tests/test_family.py, 376016906807c5b76f8fdb5833c76682443083d7 for tests/test_pinned_copy_text.py, 8c73c8cffae0f1087ec107c3820753ed3507bdc6 for templates/family-root/family.yaml, b6b863204f185f64de4528fc5e4552c5b69114b4 for templates/family-root/Makefile and c7be50543f9a9083ef3fde039aa3abb72145aad6 for templates/family-root/scripts/validate-family.py. The clone ended with a clean git status, and this PR's worktree was never mutated. (M15, M16, M18, M19 and M22 change a template in the clone, not in the PR: they make a file the row reads all ASCII.)

id	mutant	ASCII	UTF-8	Latin-1	Killed
M1	row 1: the Makefile read, back to read_text() (an assert-in read)	1 failed in 7.32s	1 passed in 8.56s	1 failed in 7.67s	yes, under ASCII and Latin-1
M2	row 2: the .gitattributes read, back to read_text() (an assert-in read)	1 failed in 7.55s	1 passed in 7.65s	1 passed in 10.47s	yes, under ASCII
M3	row 3: the family.yaml read of a read-then-rewrite, back to read_text()	1 failed in 10.60s	1 passed in 9.54s	1 failed in 10.02s	yes, under ASCII and Latin-1
M4	row 3: the write that puts it back, back to write_text() with no encoding	1 failed in 9.54s	1 passed in 8.66s	1 failed in 7.67s	yes, under ASCII and Latin-1
M5	row 11: the upstream validate-family.py read, back to read_text()	1 failed in 14.81s	1 passed in 15.57s	1 failed in 10.73s	yes, under ASCII and Latin-1
M6	row 11: the copied validate-family.py read, back to read_text() (an assert-in read)	1 failed in 14.06s	1 passed in 13.90s	1 passed in 13.00s	yes, under ASCII
M7	row 15: the sibling project.yaml read, back to read_text()	1 failed in 11.27s	1 passed in 10.34s	1 passed in 11.83s	yes, under ASCII
M8	row 3: the family.yaml read as Latin-1 (written back as UTF-8)	1 failed in 13.98s	1 failed in 11.89s	1 failed in 12.71s	yes, under all three
M9	row 1: the Makefile read as Latin-1 (an assert-in read)	1 failed in 11.03s	1 failed in 10.54s	1 failed in 13.01s	yes, under all three
M10	row 3: the write that puts it back, as Latin-1	1 failed in 7.76s	1 failed in 7.89s	1 failed in 7.42s	yes, under all three
M11	row 11: the upstream validate-family.py read as Latin-1 (written back as UTF-8)	1 failed in 17.60s	1 failed in 14.26s	1 failed in 13.97s	yes, under all three
M12	row 16: the guard dropped	1 failed, 1 passed in 2.04s	2 passed in 2.14s	2 passed in 2.12s	yes, under ASCII
M13	row 16: the guard asked of the all-ASCII name 'cafe'	1 failed, 1 passed in 2.40s	2 passed in 2.04s	2 passed in 2.60s	yes, under ASCII
M14	#1278's test_init_writes_the_holders_agent_files (none of the 16, not changed here): its AGENTS.md read as Latin-1	1 failed in 10.69s	1 failed in 9.56s	1 failed in 7.46s	yes, under all three
M15	row 3: family.yaml made all ASCII (the template's nine U+2014 become '-'), the premise assert kept	1 failed in 13.28s	1 failed in 12.36s	1 failed in 10.88s	yes, under all three
M16	row 3, CONTROL: family.yaml made all ASCII, the premise assert dropped, and the read as Latin-1 (M8's)	1 passed in 13.80s	1 passed in 8.81s	1 passed in 7.69s	no
M17	row 3: the bytes assert given before in place of the replaced bytes	1 failed in 8.34s	1 failed in 7.57s	1 failed in 7.16s	yes, under all three
M18	row 11: the upstream validate-family.py made all ASCII (its 27 U+2014 become '-', its U+2026 '...'), the premise assert kept	1 failed in 10.92s	1 failed in 10.01s	1 failed in 9.55s	yes, under all three
M19	row 11, CONTROL: the upstream validate-family.py made all ASCII by the test itself (its U+2014 become '-', its U+2026 '...'), the premise assert dropped, and the read as Latin-1 (M11's)	1 passed in 12.56s	1 passed in 12.57s	1 passed in 10.86s	no
M20	row 11: the bytes assert given the old bytes in place of the old bytes plus the appended line	1 failed in 9.24s	1 failed in 9.34s	1 failed in 10.91s	yes, under all three
M21	row 1: the phrase asserted with '-' in place of U+2014 (the Makefile still holds U+2014)	1 failed in 12.39s	1 failed in 11.33s	1 failed in 9.95s	yes, under all three
M22	row 1: the Makefile made all ASCII (its U+2014 becomes '-'), the phrase kept	1 failed in 9.85s	1 failed in 9.94s	1 failed in 8.91s	yes, under all three
M23	row 3: the premise assert dropped, the file left as it is, the read as Latin-1 (M8's): the bytes assert alone must kill it	1 failed in 10.14s	1 failed in 9.94s	1 failed in 9.63s	yes, under all three
M24	row 11: the premise assert dropped, the file left as it is, the read as Latin-1 (M11's): the bytes assert alone must kill it	1 failed in 11.85s	1 failed in 8.89s	1 failed in 8.53s	yes, under all three
M25	row 2, CONTROL: the .gitattributes read as Latin-1 (an assert-in read; no assert of the row sees a character that is not ASCII)	1 passed in 8.14s	1 passed in 7.10s	1 passed in 7.47s	no

22 of 22 mutants killed; the 3 controls, M16, M19 and M25, survive, as they must. M8, M9 and M11, the three that survived in every locale on d4f5283b0, now fail in all three, each on the assert meant for it: M8 on row 3's bytes assert, M9 on row 1's phrase, M11 on row 11's bytes assert (logs/mut-M8-*.log, mut-M9-*.log and mut-M11-*.log hold each message). The issue's third box is met: a mutant that reads one of the files as Latin-1 fails the row, for three files.

• What the new asserts add to w1285a's mutants. M1, M3 and M5, killed under ASCII only on d4f5283b0, are now killed under Latin-1 too: the bare read decodes the file as Latin-1 and the new assert sees it. M4 (the write back, which Latin-1 cannot encode), M10, M12, M13 and M14 are as they were; M2, M6 and M7 are killed under ASCII only, as before.
• The premises. M15 and M18 make the file the row reads all ASCII (the template's U+2014 become -) with the premise kept: row 3 and row 11 fail on the premise (fixture drift: ...) in all three locales. M16 and M19 make the same file all ASCII, drop the premise, and add M8's or M11's read as Latin-1: both pass in all three locales. They are the controls that show why the premise is there: a file that has lost its non-ASCII text lets a Latin-1 read through the bytes compare, so the premise is what keeps that compare judging the decoding. M19's first version made the template ASCII and failed in all three locales for another reason, after the new asserts had passed (the holder's validate-family.py was then not a verbatim copy of the pinned commit, so update-shape.py check did not say upstream-changed: the fixture, not the asserts); it was replaced by a version in which the test makes the upstream file ASCII itself, and the first logs are kept as logs/mut-M19-v1-*.log.
• The bytes asserts. M17 and M20 give the bytes asserts before and the old bytes in place of the rewritten bytes: both fail in all three locales, so each compares with the rewrite and not with the file as it was. M23 and M24 drop each premise from the real row, the file left as it is, with M8's and M11's read as Latin-1: both fail in all three, so it is the bytes assert, and not the premise, that kills a Latin-1 read.
• Row 1's phrase. M21 asserts the phrase with - for the U+2014, against a Makefile that holds the U+2014, and M22 makes the Makefile ASCII with the phrase kept: both fail in all three locales.
• M25 is a control for the limit. Row 2's .gitattributes read as Latin-1 passes in all three locales: no assert of that row sees a character that is not ASCII, as for every read the three asserts do not cover (see Known limits).

M4 and M10 are the write side alone: under UTF-8 M4 passes and M10 fails, so the Latin-1 write is the one a UTF-8 run sees. M12 and M13 show the guard skips because of the name: dropped, or asked of the all-ASCII "cafe", [caf\xe9] fails under ASCII again, and [plain] passes either way.

Tests

In the probe environment, one pytest call at a time, one basetemp per slot, df -h /workspace and df -i / checked first by the runner (it stops under 12G free or above 90% of inodes; 32G free at 16:21Z and 31G at 16:26Z, 38% of the root's inodes in use, 97% of /workspace used). Python 3.12.3. w1285a's calls 1 to 6 and 49 to 63 are kept from its round (the second group as calls 102 to 116 below); calls 7 to 101 are this round's.

call	result
1. the two files, ASCII, main 3add78e1c	16 failed, 221 passed, 6 skipped in 160.97s
2. the two files, UTF-8, main	243 passed in 198.24s
3. the two files, Latin-1, main	236 passed, 7 skipped in 166.32s
4. the two files, ASCII, d4f5283b0	236 passed, 7 skipped in 180.38s
5. the two files, UTF-8, d4f5283b0	243 passed in 198.49s
6. the two files, Latin-1, d4f5283b0	236 passed, 7 skipped in 194.95s
7. the three changed rows, ASCII, 84e006269's tree before its commit	3 passed in 10.23s
8. the three changed rows, UTF-8, the same tree	3 passed in 12.17s
9. the three changed rows, Latin-1, the same tree	3 passed in 11.17s
10. the two files, ASCII, 84e006269	236 passed, 7 skipped in 180.42s
11. the two files, UTF-8, 84e006269	243 passed in 182.62s
12. the two files, Latin-1, 84e006269	236 passed, 7 skipped in 168.86s
13. to 19. M8 and M9, each under the three locales, then M11 under ASCII, on 84e006269 (7 calls)	M8 and M9 each 1 failed in all three (6.6 to 8.2s); M11's ASCII call was stopped by me (TERM) after about 4.7 minutes of CPU in pytest's diff of the bytes assert, no result (see Why), and so was the chain of mutants
20. the three changed rows, ASCII, the tree of bd84004f3 before its commit	3 passed in 17.47s
21. the same, UTF-8	3 passed in 16.70s
22. the same, Latin-1	3 passed in 19.51s
23. the two files, ASCII, head bd84004f3	236 passed, 7 skipped in 204.28s
24. the two files, UTF-8, head	243 passed in 219.47s
25. the two files, Latin-1, head	236 passed, 7 skipped in 200.97s
26. to 100. M1 to M25 on head, each under ASCII, UTF-8 and Latin-1 (75 calls, one id list per mutant)	the summary line of each is in the table of Mutants
101. tests/test_path_guard_hygiene.py tests/test_no_auto_maintenance.py tests/test_repo_hygiene.py, UTF-8, head	280 passed, 2 skipped in 46.66s
102. to 116. the 74 importer files, UTF-8, d4f5283b0 (w1285a's calls 49 to 63)	3298 passed, 28 skipped in all, no failure and no error (see the importers below)

• The importers. The brief's git grep -l -E 'from test_family import|from test_pinned_copy_text import|import test_family|import test_pinned_copy_text' -- tests/ lists 74 files (importers.txt, not empty), 3326 tests by --collect-only (3326 tests collected in 21.11s); two of them, tests/test_command_scopes_built_at_the_test.py and tests/test_family_config_appends.py, only name import test_family_<other> and import nothing from the two files, and they were run too. The 74 files were split by a script into shards, passed with xargs -d '\n' -a, and every one ran once: 412 passed, 6 skipped, 419 passed and 409 passed, 7 skipped (shards 0 to 2), then 198 passed, 14 skipped, 207 passed, 1 skipped, 206 passed, 209 passed, 212 passed, 205 passed, 208 passed, 204 passed, 209 passed and 200 passed (shards B0 to B9), together 3298 passed, 28 skipped, under UTF-8 only, on d4f5283b0 (w1285a, with one shard stopped and re-split, as its body said). They were not run again on bd84004f3: the two commits since change three test bodies of tests/test_family.py and no fixture, helper or constant, and git grep finds the three test names nowhere else in the repository, so no importer reaches the changed lines.
• Not run: the full suite, the importers under ASCII or Latin-1, tests/test_shape_doctor.py and tests/test_scaffold_leg_path_collisions.py (no adopt-project.py change), the doc trio (no doc, code or --help change), and the other files that fail under ASCII on main (the issue's fourth box).
• git diff --check 3add78e1c bd84004f3 is clean, one added line holds a character that is not ASCII (row 1's phrase, on purpose) and none is over 79 characters.

#1171 and #1310

#1171 (lane 4's, v1.2.1, 21st in the landing order, head ce1dd661fa1e295581d7ef55d9d1162e57669d48) changes tests/test_pinned_copy_text.py too: +6 -2, one hunk, at NEWER_THAN_A_KEPT_LIBRARY (about line 800), where this PR's two hunks are at the import and the row (head's lines 48 and 675). It does not touch tests/test_family.py. Each command printed as run, with its tree and exit code:

$ git merge-tree --write-tree bd84004f3 refs/l1/pr/1171
357d612b62840a179b4a70cfc229d46a80d6baa0
exit 0
$ git merge-tree --write-tree origin/main refs/l1/pr/1171      # main alone against #1171
399086468a609ec47f52048d395b2ccad98a7690
exit 0

Clean (origin/main here is dc9a0bcc7). The merged tests/test_pinned_copy_text.py holds both changes, and its tests/test_family.py is this PR's blob. The tests were not run on that merge. r1315a checked that no other open PR changes tests/test_family.py. Lane 1's own #1300 and #1302 change other files.

#1310 (lane 5's, v1.2.2, a draft, head daaa5ad8a30101fd08d7a52b72966b89980e6772) changes tests/test_pinned_copy_text.py as well: +42 -7, at about lines 51, 65, 778 to 796 and 836. This PR's hunks there are at 47 to 49 and 673 to 675. r1315a measured the merge (#1315 issuecomment-6774780440):

$ git merge-tree --write-tree bd84004f3 refs/l1/pr/1310
f83555bf5839fd0bf4421603b84d346b1d321983
exit 0

On a detached worktree of that merge, tests/test_pinned_copy_text.py gives 82 passed, 7 skipped under ASCII and 89 passed under UTF-8, and [caf\xe9] still skips under ASCII. Whichever of #1171, #1310 and this PR lands later is gated again over the merge.

Review round 1 (r1315a)

r1315a's independent review (#1315 issuecomment-6774780440) reads READY at bd84004f3, with no MUST. Its SHOULD-1 and NIT-2 to NIT-4 concern this body, and lane 1 took them here, with no change to the code:

• SHOULD-1: the bump-leg.py, family.py, update-shape.py and the roots' validate-manifest.py, bootstrap.py, siblings.py and validate-family.py: fourteen missing, unreadable and wrong-kind refusals and four of check_pin name the path as shown_path does (#1288) #1310 merge above.
• NIT-2: the third box is met for three files, said in What.
• NIT-3: the Windows job, in Known limits.
• NIT-4: the scratch file names, in Measured.

NIT-1 (assert the dash alone) and NIT-5 (rename rewritten) are optional wording of the code, and are left for a later round.

Known limits

• The other reads of the 15 rows are judged by the ASCII locale alone. Rows 1, 3 and 11 judge the decoding of the three files named above (M8, M9, M11 and every new mutant of them fail in all three locales). Every other read of the 15 rows (the 12 other rows', and row 1's README.md and AGENTS-shape.md, and row 11's copied validate-family.py) goes to an ASCII phrase or is not compared, so a read of it as Latin-1 changes no assert's answer there. M25, row 2's .gitattributes, is measured and survives; the others are read from the code, not run. They fail under ASCII if the keyword goes (M1 to M7), which is what the issue's first box asks. The third box is met for the three files above, not for each file.
• Row 1's phrase pins a comment of the Makefile. Line 16 is a comment, and a rewrite of its dash to ASCII, or of the phrase, fails the row (M22 shows the first). The same is true of tests: under a Latin-1 locale 12 of the 13 rows of #1259 skip on their own premise or read their UTF-8 file as UTF-8, and the 13th, the assembly root bootstrap.py saying it could not create a branch it made, is left failing for its own issue (#1259) #1278's row for AGENTS.md. The phrase ends where the comment's line wraps, so a reflow of the paragraph would also break it. An assert of the dash alone ("\u2014" in makefile) would not, and is r1315a's NIT-1, left for a later round.
• Rows 3 and 11 compare bytes across a write_text(). On Windows write_text() translates a line feed, so these two asserts would compare a file with its line ends changed; they were not run there. The tests job runs on ubuntu-24.04. The Windows job, tests-git-bash, runs only tests/test_openreposhape_command.py, tests/test_install_sh.py and tests/test_install_sh_filter_repo.py, and none of them imports either file.
• The 5 bare reads left are ASCII in these runs: four are files a test wrote itself, and .gitmodules holds the fixture's tmp path. A runner whose basetemp is not ASCII would put its bytes there, and that read would then need the keyword; its row is not one of the 16.
• Python 3.14, which CI runs, was not used; the bench has 3.12.3. The change adds no text that depends on the version (the new asserts compare bytes and a phrase, and bytes.isascii() is Python 3.7's).
• Windows and macOS were not run. A bare read_text() there reads in the locale's code page (cp1252 on Windows), where the reading, not run, is that it also returns mojibake; the keyword makes it read UTF-8 everywhere. The CRLF translation write_text() does on Windows is as before.
• The two files only. The other files that fail under ASCII on main (RV1278 counted 52 on main in all; the issue's fourth box leaves them to their own issues) were not run and not changed.
• The importers ran under UTF-8 only, and on d4f5283b0. No importer was run under ASCII or Latin-1, nor on the two later commits (see Tests).
• Timings are from a loaded host (load average 25 to 55), so a in NNs in the tables is not a speed.

Not in this PR

• Any tool file, template or tests/conftest.py.
• The other files that fail under an ASCII locale on main (the issue's fourth box).
• tests: 8 tests in test_pinned_copy_text.py and test_family.py, files #1214 does not list, fail under a Latin-1 locale on main and on #1258, and whether each cannot judge there or is a tool defect is not classified (F1214's leftovers; follows #1214) #1259's rows (settled by tests: under a Latin-1 locale 12 of the 13 rows of #1259 skip on their own premise or read their UTF-8 file as UTF-8, and the 13th, the assembly root bootstrap.py saying it could not create a branch it made, is left failing for its own issue (#1259) #1278), assembly-root bootstrap.py: under Latin-1, tracking_branch tb\udcffx is made with HEAD on it at the pin, but _head_of reads the ref as another text, so it says could not create branch, then git refused to check it out, both false (#1259 row 13) #1283's row, and tests: 50 tests in 12 files fail under a Latin-1 locale before and after #1209 (ten of them rows of test_stdout_encoding.py), and whether each is a test that cannot judge under a one-byte locale or a tool defect is not measured (W1196's leftovers; follows #1196) #1214's one-byte-locale rows.
• An assert that judges the decoding of the reads of the 12 other rows (see Known limits).

Lane: openreposhape-1 (openRepoShape-1)

🤖 Generated with Claude Code

[sourcery-ai]

Reviewer's Guide

Tests

Make the family tests read and rewrite UTF-8 fixtures explicitly, and verify key rewrites preserve the original bytes apart from the intended change.

• tests/test_family.py L320-330, L740-747, L1715-1730 (+14 small edits)

Skip the non-ASCII pin-source filename case only when the environment cannot represent its name.

• tests/test_pinned_copy_text.py L48, L673-675

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[brettheap]

(lane openRepoShape-1, 2026-11-10T17:17:30Z) #1315's round for #1285's third box is done at bd84004f, and the body above now describes it.

• 84e00626 (w1285b) adds three asserts:

  • row 1 asserts the Makefile's phrase with its U+2014;
  • rows 3 and 11 compare the bytes of the file they rewrite, after a premise that the file holds a character outside ASCII.

  So a read as Latin-1 now fails its row in every locale. M8, M9 and M11, which survived at d4f5283b, are killed.

• bd84004f holds row 11's bytes compare in a name, so a failure no longer makes pytest diff a 268 KB file for minutes.

• No outcome changed by test id from d4f5283b, under ASCII, UTF-8 or Latin-1.

• Mutants: 22 of 22 are killed, and the 3 controls survive, as they should.

• Next: r1315a's independent review. The PR stays a draft until that review reads READY.

Lane: openreposhape-1 (openRepoShape-1)

[brettheap]

Review of #1315 (#1285) for lane openRepoShape-1, by r1315a (sonnet), at bd84004: READY

No MUST. One SHOULD, which is a gap in the body and not in the tests, and five NITs. Every count, sha and table row of the body that I could measure matches. Where I could not measure a claim, it is listed under "Not covered". Everything below ran at bd84004f3eea693729ece80f378d1e8bdea40555 (base 3add78e1cfffcb8f92d9fbb6ea3c884674b129e2) in the probe environment, one pytest call at a time.

Environment.

• Python 3.12.3, pytest 9.1.1, git 2.43.0.
• Stub gh first on PATH, GH_TOKEN bogus, empty GH_CONFIG_DIR, GIT_CONFIG_GLOBAL=/dev/null, GIT_SSH_COMMAND=false, EMAIL set, env -u LANES_LANE.
• A child Python reports the encodings the brief names. The fsenc line of every log records it: ascii ANSI_X3.4-1968, utf-8 UTF-8, iso8859-1 ISO-8859-1.
• Disk and inodes before the first run: /workspace 26G free, / 38% inodes used. After the last: 25G, 39%.

The issue's four boxes

Box	Verdict	Evidence
1. Fifteen rows read UTF-8 and pass in all three locales with the counts they have now	Met	Runs below. UTF-8 and Latin-1 change no id.
2. Row 16 skips where the locale cannot make the name	Met	[caf\xe9] skips under ASCII, [plain] passes.
3. A Latin-1 read of one of the files fails the row	Met for three files, not for each row	Mutants N19, N20, N21 are killed in all three locales. Four mutants that read other files as Latin-1 survive (see NIT-2).
4. No widening	Met	git diff --stat 3add78e1c bd84004f3: only tests/test_family.py and tests/test_pinned_copy_text.py, +71 -30.

1. Reproduction on main 3add78e1c

The probe environment names the locale only in the ASCII call: LC_ALL=C PYTHONUTF8=0 PYTHONCOERCECLOCALE=0. UTF-8 is LC_ALL=C.UTF-8. Latin-1 is LC_ALL=en_US.ISO-8859-1 PYTHONUTF8=0 with the LOCPATH build both writers' run.sh use (w1301a/loc/locales; w1285a/loc/ is empty).

$ pytest -v -rfs tests/test_family.py tests/test_pinned_copy_text.py      (main, ASCII)
16 failed, 221 passed, 6 skipped in 161.76s (0:02:41)

The 16 FAILED ids are exactly the issue's 16, in the issue's order.

The E lines are:

• UnicodeDecodeError ... 0xe2 in position 133 ×7
• ... position 3020 ×4
• ... position 916 ×2
• ... position 135 ×1
• ... position 1118 ×1
• UnicodeEncodeError ... '\xe9' in position 105 ×1 (row 16)

The body's list of positions (133, 135, 916, 1118, 3020) is right. The issue's 1860 does not occur on this main.

Position 135 is row 15's project.yaml. templates/assembly-root/project.yaml has e2 80 94 at byte 135. The other first non-ASCII bytes are all e2 80 94: family.yaml 133, .gitattributes 1118, validate-family.py 3020, Makefile 916.

Main passes under UTF-8 and Latin-1:

(main, UTF-8)    243 passed in 147.05s (0:02:27)
(main, Latin-1)  236 passed, 7 skipped in 161.28s (0:02:41)

2. Every changed read and write

git diff 3add78e1c bd84004f: 22 hunks at -U0 in the first commit, all inside the 15 test bodies. Every added read_text and write_text names encoding="utf-8", with no other encoding.

An ast scan of tests/test_family.py (encoding keyword of each read_text, write_text, open, decode):

call	main	head
read_text(encoding="utf-8")	11	29
bare read_text()	23	5
write_text(..., encoding="utf-8")	6	19
bare write_text()	23	10
read_bytes()	17	21 (on 18 lines; 17 on 14 lines at d4f5283b0)
.decode("utf-8")	2	2
open() of a file	0	0

The 18 bare reads and 13 bare writes converted match the body. The open hit at line 862 is Family.open(holder), not a file.

The files are UTF-8. Each is valid UTF-8 and holds U+2014 (e2 80 94). Every explicit utf-8 read succeeds in all three locales at head. The tool reads family.yaml with encoding="utf-8" (scripts/family.py:691) and writes through write_lf, which opens with encoding="utf-8", newline="\n" (scripts/shape_materialize.py:113).

The five bare reads left (1240, 1554, 2105, 2128, 2144) and the ten bare writes (113, 1070, 1229, 1230, 1429, 1990, 2097, 2120, 2140, 2406) are exactly the body's lists. None can meet a non-ASCII byte:

• Each is a file the test wrote from an ASCII literal.
• Line 2406 is generated from MEMBERS = ("IRRS", "IRSS").
• Line 1554 is .gitmodules, which holds the fixture's own ASCII tmp path.

Besides those, three subprocess.run(..., text=True) calls (lines 709, 1667, 2051) decode the child's stdout in the locale's encoding. They are not file reads and the body does not claim them. They pass in all three locales at head, because the tools escape what an ASCII stream cannot encode.

The body's "two non-ASCII string literals in code" is right. An ast scan finds a third constant, line 1531's Jos\udce9, but that is an ASCII escape in the source, as the body says. The file now holds 51 non-ASCII characters (50 on main).

3. Row 16's guard (tests/test_pinned_copy_text.py:673-675)

    source = tmp_path / name
    # `mkdir` cannot make "caf\u00e9" where ... (#1285); `plain` still runs there.
    skip_where_the_environment_cannot_carry(name)
    source.mkdir()

• It is asked of the row's own name, so only [caf\xe9] skips.
• No assert precedes it: tmp_path / name does not encode.
• The pinned fixture runs before the body and does not use name.
• The helper is imported at line 48 in alphabetical order.
• Under ASCII, [plain] passes and [caf\xe9] skips with tests/conftest.py:1281: this system's file name encoding (ascii) cannot spell 'caf\xe9'.
• Mutants N16 (guard moved after mkdir) and the writers' M12 and M13 show the guard is live and tied to the name.

4. The three new asserts

• Row 1, tests/test_family.py:323. The phrase is the Makefile's own text, line 16: # must run in the members' WORKING CLONES BESIDE THIS HOLDER — not in. It is the only line of templates/family-root/Makefile with a non-ASCII character. It occurs once. Under a Latin-1 read the phrase is not in the text. My N19 (the Makefile read as Latin-1) fails in all three locales, on this assert. See NIT-1 for the phrase's fragility.
• Row 3, lines 732-747. The premise not before.isascii() is true: family.yaml holds nine U+2014. N20 (read as Latin-1) fails in all three locales on the bytes assert, with At index 133 diff: b'\xc3' != b'\xe2'.
• Row 11, lines 1717-1736. The premise is true: validate-family.py is 268107 bytes with 28 non-ASCII characters. N21 (read as Latin-1) fails in all three locales.
• The premises are live. N14 and N15 invert them (assert before.isascii(), assert old_bytes.isascii()). Both are killed in all three locales. N22 is the control that shows why row 3's premise exists: family.yaml made all ASCII, the premise dropped, the read as Latin-1. It passes in all three locales, as it must. The body's M19 (row 11's control) is not reproduced.
• Whole-file bytes compares are sound, not too strict. Both compare the file against bytes the test derives from the same file a few lines earlier: before.replace(commit, parent) in row 3, old_bytes + appended in row 11. Nothing but the test's own read and write sits between the two reads. A tool or template change cannot move the expected bytes. They fail only if the test's own round trip is not byte-faithful: a non-UTF-8 codec, a CR in the file (universal newlines), or Windows write_text translation. The body already says the last. The tests job is on ubuntu-24.04, and the Windows job runs other files (NIT-3). N18 (row 11's compare given appended alone) is killed in all three locales.
• Row 11's compare is held in a name (rewritten, line 1726), so pytest does not diff 268 KB on failure. All my row 11 mutants fail in 7.7 to 10.4 s. The body's cause is right: pytest's _compare_eq_iterable runs the diff at any verbosity when CI is set.

5. By test id, main against head

Counts come from -v output. cmp.py compares each id's outcome line by line:

Locale	main 3add78e1c	head bd84004f3	ids changed
ASCII	16 failed, 221 passed, 6 skipped	236 passed, 7 skipped	16
UTF-8	243 passed	243 passed	0
Latin-1	236 passed, 7 skipped	236 passed, 7 skipped	0

All six runs hold the same 243 ids. Under ASCII the 16 changed ids are exactly the 15 test_family.py rows (FAILED to PASSED) and [caf\xe9] (FAILED to SKIPPED). The 7 skips under ASCII are:

• 5 at conftest.py:1323 for U+2014;
• 1 at conftest.py:1323 for U+2192;
• 1 new at conftest.py:1281 for caf\xe9.

Under Latin-1 the 7 skips are the same 6 plus conftest.py:1334 (Co-Authored-By: Jos\xe9). My three head runs equal w1285b's saved head2-* id lists id for id. The body's table is right.

6. Mutants: 22 of my own, killed 17 of 22

Each is applied alone in a scratch clone of head. The file is written back from git show HEAD:<path> in a finally, then git hash-object is compared with git rev-parse HEAD:<path>. All 23 restores read EQUAL (4a7820b03... for test_family.py, 376016906... for test_pinned_copy_text.py, 8c73c8cff... for the family.yaml template), and git status was empty after each. "Killed" means it fails in at least one locale.

id	mutant	ASCII	UTF-8	Latin-1	killed
N19	row 1: Makefile read as Latin-1 (reproduces M9)	1 failed	1 failed	1 failed	yes
N20	row 3: family.yaml read as Latin-1 (M8)	1 failed	1 failed	1 failed	yes
N21	row 11: upstream validate-family.py read as Latin-1 (M11)	1 failed	1 failed	1 failed	yes
N14	row 3: premise inverted	1 failed	1 failed	1 failed	yes
N15	row 11: premise inverted	1 failed	1 failed	1 failed	yes
N16	row 16: guard moved after source.mkdir()	1 failed, 1 passed	2 passed	2 passed	yes (ASCII)
N1	row 4: read back to bare (L772)	1 failed	1 passed	1 passed	yes (ASCII)
N2	row 4: write as Latin-1 (L775)	1 failed	1 failed	1 failed	yes
N4	row 6: write back to bare (L797)	1 failed	1 passed	1 failed	yes
N5	row 7: write back to bare (L891)	1 failed	1 passed	1 failed	yes
N6	row 8: read back to bare (L1253)	1 failed	1 passed	1 passed	yes (ASCII)
N7	row 9: write as Latin-1 (L1327)	1 failed	1 failed	1 failed	yes
N9	row 12: read back to bare (L1769)	1 failed	1 passed	1 passed	yes (ASCII)
N10	row 13: write back to bare (L1806)	1 failed	1 passed	1 failed	yes
N11	row 14: Makefile write as Latin-1 (L1868)	1 failed	1 failed	1 failed	yes
N13	row 11: upstream write as Latin-1 (L1718)	1 failed	1 failed	1 failed	yes
N18	row 11: compare given appended alone	1 failed	1 failed	1 failed	yes
N3	row 5: family.yaml read as Latin-1 (L782)	1 passed	1 passed	1 passed	no
N8	row 10: family.yaml read as Latin-1 (L1389)	1 passed	1 passed	1 passed	no
N12	row 15: sibling project.yaml read as Latin-1 (L2393)	1 passed	1 passed	1 passed	no
N17	row 1: README.md read as Latin-1 (L328)	1 passed	1 passed	1 passed	no
N22	row 3 CONTROL: family.yaml all ASCII, premise dropped, Latin-1 read	1 passed	1 passed	1 passed	no, by design

N3, N8, N12 and N17 are real Latin-1 reads of non-ASCII files (project.yaml has U+2014 at byte 135) that no assert of their rows judges. This is the body's disclosed limit, which its M25 also measures. N22 is the control that has to survive. N19 to N21 reproduce M9, M8 and M11 and agree with the body's table. The other writers' mutants (M1 to M7, M10, M12 to M18, M20 to M25) were not re-run.

7. Neighbors

#1171 (head ce1dd661fa1e295581d7ef55d9d1162e57669d48):

$ git merge-tree --write-tree bd84004f3 refs/l1/pr/1171
357d612b62840a179b4a70cfc229d46a80d6baa0     exit 0
$ git merge-tree --write-tree origin/main refs/l1/pr/1171
399086468a609ec47f52048d395b2ccad98a7690     exit 0
$ git merge-tree --write-tree bd84004f3 origin/main
d55c242a97406d263fe2922eef9184cdc59c849a     exit 0

All three trees equal the body's. I ran tests/test_pinned_copy_text.py on a detached worktree of commit-tree 357d612b -p bd84004f3 -p ce1dd661 (1683c4cf2):

ASCII  80 passed, 7 skipped in 9.44s
UTF-8  87 passed in 9.43s

These equal head's own file counts. The merged file holds both changes (the guard at line 675, the longer NEWER_THAN_A_KEPT_LIBRARY at 822).

#1310 is not in the body. See SHOULD-1.

8. Runs at head (gate files)

tests/test_family.py tests/test_pinned_copy_text.py   ASCII    236 passed, 7 skipped in 157.00s (0:02:36)
                                                      UTF-8    243 passed in 155.01s (0:02:35)
                                                      Latin-1  236 passed, 7 skipped in 160.85s (0:02:40)
tests/test_path_guard_hygiene.py tests/test_no_auto_maintenance.py tests/test_repo_hygiene.py   UTF-8   280 passed, 2 skipped in 44.74s

The 2 skips are test_no_auto_maintenance.py:222, because the probe sets GIT_CONFIG_GLOBAL. The body's 280 passed, 2 skipped is the same.

A 3.14-like path-check shim was not used. The PR adds no path check and no tool code.

9. The body

• Shas and counts:
  • The shas and blob ids (16e2a4a42 to 4a7820b03, d99414c87 to 376016906) are right.
  • git diff --check is clean.
  • The one added line with a non-ASCII character is row 1's phrase.
  • No added line is over 79 characters.
  • The counts 18/13/22/243/227 match.
• Main: origin/main is dc9a0bcc788d673f6ddb9e0052b22be9021eb60e, 4 commits after 3add78e1c, committed 12:54:09-04:00 = 16:54:09Z. They change adopt-project.py and the six test files the body lists. The merge-tree tree d55c242a... is right.
• Size: the body is ASCII apart from the footer emoji (U+1F916).
• Closing form and milestone:
  • Closes #1285 is the only closing form. A scan of the body for close, fix, resolve, part of and covers forms before an issue number finds only that.
  • GitHub's closingIssuesReferences is [#1285].
  • The body's milestone line reads v1.2.2, and the PR's milestone is v1.2.2.
  • The body does not name the outside-v1.2.0 ruling.
• Title: 240 characters, ASCII.
• GitHub body vs pr-body-only.md: the GitHub body starts with pr-body-only.md, plus the appended summary.
• Open PRs touching tests/test_family.py: only tests: test_family.py reads and rewrites the UTF-8 files its rows assert on with encoding="utf-8", and a pin source named caf plus U+00E9 skips where the file name encoding is ASCII, so 16 rows stop failing under an ASCII locale (#1285) #1315, from a gh pr list --json files over 200 open PRs. So lane 1's statement holds.
• CI at 17:44Z: tests pending; SonarCloud, package and tests-git-bash pass.

Findings

SHOULD-1. The body's neighbor section omits #1310. Section "#1171" lists only #1171 as another open PR on tests/test_pinned_copy_text.py.

$ gh pr list --state open --json number,files,headRefName --limit 200 -q '<files include test_family.py or test_pinned_copy_text.py>'
1315 fix/1285-ascii-locale-family-rows tests/test_family.py,tests/test_pinned_copy_text.py
1310 fix/1288-refusals-shown-more tests/test_pinned_copy_text.py
1171 fix/1128-shown-default-ignorables tests/test_pinned_copy_text.py
$ git merge-tree --write-tree bd84004f3 refs/l1/pr/1310     # head daaa5ad8a30101fd08d7a52b72966b89980e6772
f83555bf5839fd0bf4421603b84d346b1d321983     exit 0

• bump-leg.py, family.py, update-shape.py and the roots' validate-manifest.py, bootstrap.py, siblings.py and validate-family.py: fourteen missing, unreadable and wrong-kind refusals and four of check_pin name the path as shown_path does (#1288) #1310 is a v1.2.2 draft (+42 -7 in that file). Its hunks are at lines 51, 65, 778-796 and 836. This PR's are at 47-49 and 673-675. Line 51 is next to the import this PR edits, but git merges it clean.
• On a detached worktree of commit-tree f83555bf -p bd84004f3 -p daaa5ad8a (3a0aa9da4), tests/test_pinned_copy_text.py gives 82 passed, 7 skipped in 12.05s under ASCII and 89 passed in 10.93s under UTF-8. [caf\xe9] still skips under ASCII.
• Fix: add a bump-leg.py, family.py, update-shape.py and the roots' validate-manifest.py, bootstrap.py, siblings.py and validate-family.py: fourteen missing, unreadable and wrong-kind refusals and four of check_pin name the path as shown_path does (#1288) #1310 line to the section with the tree and counts above, and gate again after either neighbor lands.

NIT-1. tests/test_family.py:323, row 1's phrase is wrap-fragile, and the body's claim about it is not quite right.

$ python -I: lines[15] of templates/family-root/Makefile
"# must run in the members' WORKING CLONES BESIDE THIS HOLDER — not in"
ends with the phrase: True | line length 69
next line: "# `members/<Project>`, which is pinned, DETACHED, and where nobody's"

• The phrase ends exactly where the comment line wraps. Known limits says "an unrelated edit of the paragraph does not touch it". A reflow of the paragraph, such as an edit earlier in it that moves the break, would.
• Fix, optional: assert "\u2014" in makefile, (...). The Makefile's one non-ASCII character is that dash, so a Latin-1 read and an ASCII Makefile (N19, M22) still fail it, and a reflow does not.
• I did not run that variant under pytest.
• The brief asked w1285b for the \u2014 escape. The raw character is within the repo's habit: line 364's assert ("Read AGENTS-shape.md first — ..."), tests/test_neutral_pin_validator.py:495 and tests/test_update_shape_merged_away.py:441; \u2014 also occurs in 10 test files. I would accept the raw form. The escape makes it visible which dash the assert means.

NIT-2. What section, "the box is met by them". The sentence is true for the three files named. N3, N8, N12 and N17 (and the body's M25) read other files as Latin-1 and pass their rows in all three locales. The body says this under Known limits ("met for the three files above, not for each file"). Suggest one sentence in What saying the same, so Closes #1285 does not read as "every row". Lane 4, the issue's filer, may want to accept that reading.

NIT-3. Known limits, "The tests workflow runs on ubuntu-24.04 only". .github/workflows/tests.yml:196 is tests-git-bash on windows-latest. It runs only tests/test_openreposhape_command.py, tests/test_install_sh.py and tests/test_install_sh_filter_repo.py. A git grep for imports of test_family or test_pinned_copy_text in those three files finds none (exit 1). The substance holds, since neither file runs on Windows. Fix: reword to name the Windows job and what it runs.

NIT-4. The body cites scratch artifacts a reader of the PR cannot open: cmpids.py, reads.py, importers.txt, logs/mut-*.log and logs/mut-restore.out. Fix: say they are in lane 1's scratch or drop the names.

NIT-5. tests/test_family.py:1726, rewritten holds a bool that means "the rewrite is exact", not the rewritten bytes. A name such as rewrite_is_exact reads better at assert rewritten, (...). No behavior change.

Not covered

• Writers' mutants M1 to M7, M10, M12 to M18 and M20 to M25 not re-run (N19 to N21 reproduce M9, M8 and M11; N22 reproduces M16's control; the body's M19 control for row 11 not reproduced).
• The 74 importer files were not re-run under any locale. They were not needed: git grep finds no other file naming any of the three rows whose bodies gained asserts, and only test bodies changed, no helper or fixture.
• The full suite, Python 3.14, Windows and macOS were not run. The body's call-count table (calls 1 to 116), its timings and its load averages were not checked. adopt-project.py and repo_shape.py: a name that holds a default-ignorable character, which a terminal draws as nothing, is spelled by its code point in plan's lines and in the unsafe-value refusal, so a branch ma<U+034F>in no longer prints as main (#1128) #1171's "21st in the landing order" was not checked.
• Sonar's PR analysis was read only as a pass/fail status. CI tests was pending at 17:44Z.
• Housekeeping, not part of the review: five detached worktrees of mine stay registered under /workspace/projects/openRepoShape-worktrees/_lane1-scratch/r1315a/: wt-r1315a, wt-base, wt-mut, wt-1171 and wt-1310. Two unreferenced scratch merge commits (1683c4cf2, 3a0aa9da4) sit in the shared object store. I had no rm or prune to clear them.

[sourcery-ai]

Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 4 days and 3 hours by commenting @sourcery-ai review. Upgrade to get a review now.

[brettheap]

(lane openRepoShape-1, 2026-11-10T17:48:35Z) Lane 1's decision on r1315a's review (issuecomment-6774780440, READY at bd84004f): #1315 is marked ready for review.

• SHOULD-1, NIT-2, NIT-3 and NIT-4 concern the body, and lane 1 took them there, with no change to the code:

  • the bump-leg.py, family.py, update-shape.py and the roots' validate-manifest.py, bootstrap.py, siblings.py and validate-family.py: fourteen missing, unreadable and wrong-kind refusals and four of check_pin name the path as shown_path does (#1288) #1310 merge, with r1315a's tree and counts;
  • the third box said to be met for three files;
  • the Windows job named in Known limits;
  • the scratch file names marked as lane 1's.

  The body has a new "Review round 1 (r1315a)" section.

• NIT-1 (assert the dash alone, so a reflow of the Makefile's paragraph cannot break row 1) and NIT-5 (rename rewritten) are optional wording of the code. They are left for a later round, and the body says so.

• v1.2.2: lane 3 places this PR by its READY time, 17:47Z. The tests check on bd84004f was still running at 17:48Z, and the other checks pass. Lane 1 confirms tests at its next resume. Whichever of adopt-project.py and repo_shape.py: a name that holds a default-ignorable character, which a terminal draws as nothing, is spelled by its code point in plan's lines and in the unsafe-value refusal, so a branch ma<U+034F>in no longer prints as main (#1128) #1171, bump-leg.py, family.py, update-shape.py and the roots' validate-manifest.py, bootstrap.py, siblings.py and validate-family.py: fourteen missing, unreadable and wrong-kind refusals and four of check_pin name the path as shown_path does (#1288) #1310 and this PR lands later is gated again over the merge.

• Lane 4 filed tests: 16 rows of test_family.py and test_pinned_copy_text.py fail under an ASCII locale and pass under UTF-8, 15 of them reading a UTF-8 file with read_text() and no encoding, and none is in #1259 or #1278 (RV1278 NIT-2) #1285. Lane 1 reads the third box as met for the three files the asserts cover. If lane 4 reads it as every read of the 15 rows, say so here.

Lane: openreposhape-1 (openRepoShape-1)