[brettheap]

Lane: openreposhape-5 (openRepoShape-5)
Claim: #663 (comment)

Test only. No tool and no template changes. templates/assembly-root/scripts/validate-manifest.py is a pinned copy and is not edited.

No Windows run was made, and no macOS run. CI is Linux only since #597. What Windows and macOS answer is emulated: os.fsencode is replaced for one test by Windows' (text.encode("utf-8", "surrogatepass"), three bytes for U+DCFF), and macOS's EILSEQ by a mkdir that answers it.

Fix round (head 6ba8c5d2)

The review of 0b1d1ddb found no MUST and one SHOULD and two NITs. All three are taken, in one commit, 6ba8c5d2, in three test files: tests/conftest.py, tests/test_non_utf8_name_helpers.py and tests/test_non_utf8_git_reads.py. No tool file changed.

• SHOULD-1: skip_message had no test of its own.
  • A skip that escapes skip_message ends the test SKIPPED, and no run fails on a skip. The review's mutant SK1 (a skip escapes) gave 10 passed, 3 skipped on the helpers file, so seven emulation tests skipped silently.
  • tests/test_non_utf8_name_helpers.py now holds it with three tests: it returns the reason a call skips with (skip_message(lambda: pytest.skip("x")) == "x"), it returns None where the call does not skip, and it lets any other exception go on (ZeroDivisionError, raised by a small function and not by 1 / 0, so no analyser reads a literal division by zero).
  • The first test catches an escaped skip and fails it with pytest.fail. A bare assert would not do: the escaped skip would end that test SKIPPED too, and SK1 would still pass.
  • SK1 now gives 1 failed, 14 passed, 3 skipped (before: 10 passed, 3 skipped), and the one failure is test_skip_message_returns_the_reason_a_call_skips_with: Failed: the skip escaped skip_message, which would end the test SKIPPED, and no run fails on that.
• NIT-2: windows_fsencode took str only. Real os.fsencode passes bytes through. windows_fsencode in tests/conftest.py now does (if isinstance(text, bytes): return text). Two tests hold it: a surrogate gives b"\xed\xb3\xbf" and text gives UTF-8, and bytes come back as they went in, compared with os.fsencode itself. Without the pass-through: AttributeError: 'bytes' object has no attribute 'encode'.
• NIT-1: the "\\x" not in shown assertion on a Latin-1 file name encoding.
  • There make_file writes café.yml as the single byte 0xE9 and the tool, rightly, prints caf\xe9.yml, so the assertion would fail on a tool that is right.
  • It is now assert_shown_without_hex_escapes(shown, output) in tests/test_non_utf8_git_reads.py. It skips where os.fsencode(UTF8) != UTF8.encode("utf-8") and then asserts.
  • The test calls it AFTER its other assertions (.yml ending, no traceback, nothing moved). The assertion moved from the middle to the end, so a skip loses none of the others.
  • Two tests hold it: under an emulated Latin-1 os.fsencode it skips, and under Windows' (emulated) it does not skip, still fails a line that holds \xc3\xa9, and passes café.yml.
  • Where the file name encoding cannot write café.yml at all (ASCII), make_file skipped already.

Fix-round tests (head 6ba8c5d2, /usr/bin/python3, gh stubbed, targeted files only)

file	before the round (0b1d1ddb)	after
tests/test_non_utf8_name_helpers.py	13	18 passed
tests/test_non_utf8_git_reads.py	50	52 passed
tests/test_bootstrap_branch_bytes.py	61	61 passed
tests/test_repo_hygiene.py	77	77 passed

That is 208 passed in two runs (156 in 14.01s and 52 in 19.40s), and no -rs line printed a skip. tests/test_adopt_pins.py and tests/test_pin_source_unreadable.py were not re-run in this round (227 passed on 0b1d1ddb): the round changes nothing they import, and neither file uses windows_fsencode or skip_message.

Fix-round mutants

Each in a throwaway clone, one change at a time, with the new test files in place.

mutant	result
SK1: a skip escapes skip_message (the try removed)	1 failed, 14 passed, 3 skipped: test_skip_message_returns_the_reason_a_call_skips_with
SK2: skip_message swallows every other exception	4 failed, 14 passed: DID NOT RAISE ZeroDivisionError, and the three OSError rows
SK3: skip_message returns None where a call skips	4 failed, 14 passed
SK4: skip_message returns "" where a call does not skip	8 failed, 10 passed
FE1: windows_fsencode without the bytes pass-through	1 failed, 17 passed: AttributeError: 'bytes' object has no attribute 'encode'
FE2: windows_fsencode without surrogatepass	3 failed, 15 passed
HX1: the skip removed from assert_shown_without_hex_escapes	1 failed, 1 passed (-k hex_escape): the Latin-1 emulation, '\\x' is contained here: M .github/workflows/caf\xe9.yml
HX2: the assert removed from it	1 failed, 1 passed: DID NOT RAISE AssertionError
HX3: its condition inverted	2 failed
R1 again, with the call in place: scripts/bump-leg.py:1126 decodes with "ascii"	2 failed, 4 passed (-k "utf8_under_quote_path_false or hex_escape"): [bump] and [dry-run], '\\x' is contained here: M .github/workflows/caf\xc3\xa9.yml

Not probed: the real test's call to the helper. Dropping it drops the check, and a Latin-1 host is not here to show it. R1 failing with the call in place is the evidence that the call asserts.

Also measured, on a host emulated as ASCII (PYTHONUTF8=0 LC_ALL=C PYTHONCOERCECLOCALE=0, sys.getfilesystemencoding() is ascii): the helpers file gives 18 passed, and the two utf8 rows skip in make_file ('ascii' codec can't encode character '\xe9'). No Latin-1 locale is installed, so the Latin-1 host itself is emulated by os.fsencode only, not run.

Merge check

git merge-tree --write-tree HEAD origin/main (2fac9790): clean, rc 0, tree 3e71db8d. The other branches below were checked at 0b1d1ddb; this round adds seven tests, one helper function and a small change to windows_fsencode, and was not re-run against them.

What changed

Base: origin/main 8062810d. Branch head: 6ba8c5d2 (three commits: d6d1df23 and 0b1d1ddb, then the fix round).

1. L2-52, one helper. tests/conftest.py holds skip_where_a_name_cannot_be_not_utf_8(directory). tests/test_adopt_pins.py and tests/test_pin_source_unreadable.py each had a copy of the body; both copies are gone and both call the shared one. It asks two questions, in this order:
   • skip_where_fsencode_gives_a_surrogate_no_byte(text): skips where text holds a byte-carrying surrogate (U+DC80 to U+DCFF) and len(os.fsencode("\udcff")) != 1. That is Windows.
   • The disk: it makes probe\udcff under the directory and removes it. EILSEQ (macOS) skips; any other OSError is raised.
2. make_file (R423 N1). tests/test_non_utf8_git_reads.py calls the first question before it writes, for a name that holds the byte. A name that is text (café.yml) is still written. The try/except stays, for APFS. The module docstring and make_file's docstring now say who skips and why.
3. The --branch a\udcffb test (R455 F1). tests/test_bootstrap_branch_bytes.py calls the shared helper first, in place of its own probe of a file. test_a_branch_the_system_carries_has_no_fault asked the same os.fsencode question inline; it now asks the same function, so the rule is written once. The file's own emulated test test_where_a_surrogate_is_three_bytes_and_no_byte_it_is_refused keeps the Windows half, unchanged.
4. The UTF-8 promise (R423 N2, mutant R1). test_a_name_that_is_utf8_under_quote_path_false_is_still_refused takes the one stderr line that starts M .github/workflows/caf and asserts it holds no literal \x, through assert_shown_without_hex_escapes, which skips where this system's file name encoding is not UTF-8 (fix round, NIT-1). A locale decode of UTF-8 é is mojibake at worst and never a \x, so the ASCII-ends caution stays.
5. The leg finding's name (R455 F3, mutant R2). A unit test asks _leg_branch_findings([{"role": "spec", "path": "contracts", "branch": "a\x00b"}]) and holds that the line starts FINDING manifest-leg-branch: leg 'spec': branch is 'a\x00b', which holds U+0000, a NUL and does not hold contracts. It runs for spec and code, on validate-manifest.py loaded by load_without_bytecode. The file's last test now takes the module from a module fixture, validator_module, which the new test shares.
6. New file tests/test_non_utf8_name_helpers.py (18 tests: 13 in the first round, 5 in the fix round) holds the helpers to this by emulation:
   • the system is asked first, and the disk is never touched where os.fsencode gives three bytes;
   • EILSEQ skips; EACCES, ENOSPC and EEXIST are raised;
   • the probe is removed;
   • text and a surrogate with no byte are not skipped;
   • Linux is not skipped;
   • (fix round) skip_message and windows_fsencode are held too.
7. Wiring tests under emulation. The --branch test and the no-fault test are each called, as they are, under Windows' os.fsencode, and must skip. make_file is called the same way: it skips and writes nothing, and a UTF-8 name is still written.

Shared helpers in tests/conftest.py, all in one block after a_hook_that_unstages, away from fix/645-fixture-copy-maintenance-race's hunks at :85 and :137:
NOT_UTF_8_PROBE, skip_where_fsencode_gives_a_surrogate_no_byte, skip_where_a_name_cannot_be_not_utf_8, windows_fsencode (which passes bytes through, as os.fsencode does) and skip_message. skip_message(call, *args) returns the reason a call skips with, or None. A bare pytest.raises(pytest.skip.Exception) is not enough for a test of a skip helper: a helper that skips where it should not ends the test SKIPPED, and no run fails on a skip.

Decisions

• Two functions, not one. make_file has no directory to probe: its write is the probe. So the system's question is its own function, and the directory helper calls it first. A change to what Windows answers is made once.
• The system first, then the disk. On Windows the disk takes the name, so a skip that waited for the disk would never come. test_where_fsencode_makes_three_bytes_the_disk_is_not_asked passes a directory that does not exist: an mkdir under it would raise FileNotFoundError, and the skip is the proof the disk was not asked.
• I read the Windows facts, I did not run them. The conftest docstring says "most likely" and "read, not run", as the issue does.
• windows_fsencode is in conftest. test_where_a_surrogate_is_three_bytes_and_no_byte_it_is_refused has the same lambda inline. I left it, to keep the hunks small.
• A skip test must catch an escaped skip (fix round). pytest.fail in an except pytest.skip.Exception is the only way the test for skip_message can fail on SK1.
• The Latin-1 check is a function (fix round). Inline in a test that needs the scaffold fixtures it could not be shown to skip. As assert_shown_without_hex_escapes it is called with an emulated os.fsencode, and fails a test when its skip or its assertion is removed.

Tests (first round, head 0b1d1ddb, /usr/bin/python3, gh stubbed, targeted files only)

file	result
tests/test_non_utf8_git_reads.py	50 passed (47 on base)
tests/test_bootstrap_branch_bytes.py	61 passed (54 on base)
tests/test_non_utf8_name_helpers.py	13 passed (new)
tests/test_adopt_pins.py + tests/test_pin_source_unreadable.py	227 passed (146 + 81, no change in count)
tests/test_repo_hygiene.py	77 passed

No test skipped on Linux (-rs printed none). No test file imports skip_where_* besides those four and the new file. The fix round's counts are above.

Mutation probes (first round)

Each in a throwaway clone at the head, one change at a time. "Base file" is the file as it was on 8062810d, run under the mutant.

mutant	the base file	the new tests
R1: scripts/bump-leg.py:1126 decodes with "ascii"	47 passed (R423's count)	2 failed: test_a_name_that_is_utf8_under_quote_path_false_is_still_refused[bump] and [dry-run], '\\x' is contained here: M .github/workflows/caf\xc3\xa9.yml
R2: leg.get('path') in place of leg.get('role')	54 passed (R455's count)	2 failed: test_the_leg_finding_names_the_leg_by_its_role_and_not_its_path[spec] and [code], leg 'contracts': branch is ...
guard removed from make_file		1 failed: test_make_file_skips_where_fsencode_makes_no_byte_and_writes_nothing, make_file wrote the name instead of skipping
helper call removed from the --branch test		1 failed: test_the_branch_test_skips_where_fsencode_makes_no_byte_of_the_surrogate
guard removed from the no-fault test		2 failed: the two skipped rows of test_the_no_fault_test_skips_where_fsencode_makes_no_byte_of_the_branch
guard removed from the shared helper		1 failed: ..._the_disk_is_not_asked (FileNotFoundError)
disk asked before the system		1 failed: the same test
helper skips on any OSError		3 failed: EACCES, ENOSPC and EEXIST rows of test_any_other_error_from_the_disk_is_the_tests_own (DID NOT RAISE OSError)
probe not removed		1 failed: test_a_disk_that_takes_the_name_is_not_skipped_and_the_probe_is_removed
guard skips any text		5 failed: the five rows of test_where_fsencode_makes_three_bytes_a_name_that_is_text_is_not_skipped
guard inverted (== 1)		8 failed

Reproduction

All rows are local; no gh, no network. The issue's rows are about what a test does on Windows or macOS, so they are shown by emulation on Linux, and by the mutants above (a mutant that drops a guard is what base did):

• L2-52: git grep -n 'skip_where_a_name_cannot_be_not_utf_8\|skip_where_no_file_name_is_not_utf_8' on base shows two bodies; on the head it shows one in tests/conftest.py and four call sites.
• N1, F1: on base, the guard-removed mutants are what a Windows run did: make_file writes caf\udce9.yml and the branch test runs on. On the head both skip under Windows' os.fsencode.
• N2, F3: R1 and R2, above.

Merge checks (first round, head 0b1d1ddb)

git merge-tree --write-tree HEAD <ref>, with the refs fetched read-only (and deleted after):

• origin/main de926142: clean. Main had moved since 8062810d by bump-leg.py and setup-project.py: the git switch -c line says under it what to do where an earlier act left that branch, so already exists is no dead end (#607's sentence, in the two tools it left) #657 (scripts/bump-leg.py), the Git Bash change and a doctor test; none touches these files. (The fix round's check against 2fac9790 is above.)
• fix/645-fixture-copy-maintenance-race: clean (its conftest hunks are at :85 and :141).
• fix/651-adopt-annotated-tag-pin (lane 1; edits tests/test_adopt_pins.py): clean.
• fix/664-surviving-mutants (lane 4; touches none of these files): clean.
• fix/655-adopt-doctor-plan-overrides (edits validate-manifest.py, which I do not): clean.
• Every other branch on origin (fix/388, 389, 417, 425, 610, 646, 648, 650, 653, 658, 659, 696, l3-668, 669, 677, 678, l3-rehearsal-v1.1.0): clean.
• docs/release-notes-v1.0.0 and l3-563-docs conflict (docs/release-notes/v1.0.0.md, docs/handbook.html); they conflict with origin/main alone, so it is not this change.
• Not checked: lane 4's fix/674-update-shape-apply was not on origin yet, and no l1-* branch existed on origin.

Left out

• Four more tests make a directory with a non-UTF-8 byte and skip on failure: tests/test_stdout_encoding.py:209, tests/test_adopt_git_reads.py:431, tests/test_adopt_template_values.py:296 and tests/test_clone_url_bytes.py:143 (directory_named). They build the name from bytes, which raises on Windows, so they skip there already. They are not in tests: the non-UTF-8 name and branch tests: one skip helper for a filesystem that refuses probe\udcff, Windows guards on make_file and the --branch a\udcffb case, and the UTF-8 promise and the leg finding's role pinned #663's lines.
• test_where_a_surrogate_is_three_bytes_and_no_byte_it_is_refused keeps its inline lambda.

Checklist candidates (for the coordinator to place; none filed)

• The other four byte-named-directory probes could use one helper. The four above each write their own "make a directory with a byte that is not UTF-8, or skip". Folding them into tests/conftest.py next to skip_where_a_name_cannot_be_not_utf_8 would give a fifth name for the same question one place to change. v1.1.1, optional.
• test_make_file_writes_a_name_that_is_utf8_where_fsencode_makes_no_byte fails on an ASCII file name encoding. Found in the fix round, and it was so at 0b1d1ddb: on a host with PYTHONUTF8=0 LC_ALL=C PYTHONCOERCECLOCALE=0 (sys.getfilesystemencoding() is ascii), make_file cannot write café.yml, skips with this filesystem cannot hold a file name that is not UTF-8, and the test's assert reason is None fails (1 failed, 2 passed for -k make_file). Nothing in CI has that encoding (Linux, macOS and Windows are UTF-8 by default), and a host needs three opt-outs to get it. The direction is the neighbouring control test's own two lines: if reason is not None and "filesystem" in reason: pytest.skip(reason). v1.1.1, optional.

Closes #663

🤖 Generated with Claude Code

Summary by Sourcery

Centralize and strengthen portability tests for non-UTF-8 filenames and branch names across supported filesystem behaviors.

Enhancements:

• Consolidate non-UTF-8 filename and branch portability checks into shared test helpers, including Windows encoding and skip handling.
• Add coverage for UTF-8 filename rendering and manifest leg findings to protect branch and diagnostic behavior.

Tests:

• Expand tests for Windows and macOS filesystem behavior through Linux emulation, including skip conditions, error propagation, and helper correctness.

[brettheap]

(lane openRepoShape-5, 2026-10-16T14:22Z) @codex review

[sourcery-ai]

Reviewer's Guide

Test-only changes consolidate non-UTF-8 platform guards, emulate Windows and macOS behavior on Linux, and add regression coverage for skip ordering, UTF-8 diagnostic rendering, and role-based manifest-leg findings; no tools, templates, or pinned production copies are modified.

File-Level Changes

Change	Details	Files
Centralize platform capability checks for tests that exercise non-UTF-8 names or branches.	Add shared helpers that check surrogate byte preservation before probing the filesystem, handle macOS EILSEQ explicitly, and re-raise unrelated filesystem errors. Replace duplicated probes and inline guards in adoption, pin-source, branch, and Git-read tests. Add Linux-based emulation tests covering Windows fsencode behavior, macOS EILSEQ, probe cleanup, error propagation, and text-name controls.	tests/conftest.py tests/test_adopt_pins.py tests/test_pin_source_unreadable.py tests/test_non_utf8_name_helpers.py tests/test_non_utf8_git_reads.py tests/test_bootstrap_branch_bytes.py
Strengthen Windows-guard coverage for file creation and non-UTF-8 branch tests without changing production code.	Make make_file skip before writing when os.fsencode cannot preserve a byte, while continuing to write ordinary UTF-8 names. Route the --branch and branch fault tests through the shared guard and verify they skip under emulated Windows behavior. Verify skipped paths perform no filesystem or bootstrap work.	tests/test_non_utf8_git_reads.py tests/test_bootstrap_branch_bytes.py
Add regression assertions for UTF-8 output and manifest-leg finding labels.	Assert UTF-8 filenames are not rendered with literal \x escapes in refusal output. Assert leg findings use the leg role rather than its path, including cases where those values differ. Share a module-scoped validator fixture for the new and existing validator tests.	tests/test_non_utf8_git_reads.py tests/test_bootstrap_branch_bytes.py

Assessment against linked issues

Issue	Objective	Addressed	Explanation
#663	Use a shared filesystem-capability check for tests involving non-UTF-8 names or branches, and guard make_file and the --branch a\udcffb case before relying on filesystem behavior.	✅
#663	Pin the promise that a UTF-8 filename is displayed as UTF-8 rather than escaped bytes, with an assertion that fails if decoding changes to ASCII.	✅
#663	Add a test proving that a manifest-leg branch finding identifies the leg by its role, not its path.	✅

Possibly linked issues

• tests: the non-UTF-8 name and branch tests: one skip helper for a filesystem that refuses probe\udcff, Windows guards on make_file and the --branch a\udcffb case, and the UTF-8 promise and the leg finding's role pinned #663: The PR directly implements the issue’s requested helper consolidation, Windows guards, UTF-8 output assertion, and validator leg-role test.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-16T14:27:47.746082Z	0b1d1dd	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. What shall we delve into next?

Reviewed commit: 0b1d1ddbee

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[sourcery-ai]

Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 6 hours and 50 minutes by commenting @sourcery-ai review. Upgrade to get a review now.