Skip to content

bump-leg.py: a workflow file or a leg pin that is not UTF-8 is refused by name before anything moves, so a Latin-1 comment no longer ends the run in a UnicodeDecodeError traceback (#289) - #352

Merged
brettheap merged 14 commits into
mainfrom
fix/bump-leg-unreadable-files
Oct 12, 2026

Conversation

@brettheap

@brettheap brettheap commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Lane: openreposhape-2 (openRepoShape-2)
Claim: #289 (comment)

What

scripts/bump-leg.py reads contracts/<role>-pin.yaml and every .yml and .yaml file under .github/workflows/ as UTF-8. A committed file that is not (a Latin-1 # caf\xe9, a UTF-16 file with a byte-order mark) now ends in a refusal, exit 2, naming the file, and not in a UnicodeDecodeError traceback, exit 1. --dry-run refuses the same way. Nothing in the working tree, the index or the leg's checkout moves.

  • bump-leg-workflow-unreadable: one refusal naming EVERY workflow file that is not UTF-8, by its path in the repository, the first bad byte and the reason, in the words load_yaml uses (.github/workflows/legs.yml: not UTF-8 at byte 236 (invalid continuation byte)). The file is refused whether or not it names the leg. A file git does not track, untracked or ignored, is refused too (validate-pins.py reads it all the same) and its line ends (untracked or ignored).
  • bump-leg-pin-unreadable: the same for contracts/<role>-pin.yaml, the leg's own pin.
  • The remediation says to re-save the file as UTF-8 (a UTF-16 file needs converting) and run the same command again, and says which advice applies: commit each file git tracks; a file marked (untracked or ignored) needs no commit, only the re-save, because committing it would put a draft on the bump branch. It can also be moved out of .github/workflows/ until the bump has landed.

Code: three small functions in scripts/bump-leg.py: _not_utf8 (the one spelling of the finding), _leg_pin_text (the pin read, called from cmd_bump) and _workflow_texts (the directory walk and read that plan_workflows now loops over; it takes the tracked and listed sets plan_workflows already holds, and asks _tracked_spelling of each file it refuses). The --help text is unchanged, so docs/cli.md is unchanged; the module docstring's "WHAT IT REFUSES" list gains the bullet, and docs/handbook.html's refusal list and change line gain the clause (#289).

Why

main() catches Refusal and CommandFailed only, and UnicodeDecodeError is a ValueError, so nothing on the path caught it. The reads come before the --dry-run early return, so a dry run died too. Reproduced first on #264's head, 5269aac, before it landed (tools/repro.sh, local bare remotes, no network; the reads are the same on main): six rows, bump and --dry-run for a Latin-1 workflow, a UTF-16 workflow naming no leg, and a Latin-1 pin, all exit 1 with the traceback and a clean tree; the OTHER leg's non-UTF-8 pin was already a clean bump-leg-validators-red (exit 2, rolled back) and still is.

Tests

New tests/test_bump_leg_unreadable.py, 19 tests (13, and 6 added by the fix round below). Every refusal test asserts exit 2, the refusal id, each file named by relative path, the byte offset and reason, no Traceback and no UnicodeDecodeError in the output, and the WHOLE tree unchanged (the rollback file's snapshot and assert_nothing_changed: root HEAD, pin and workflow bytes, the gitlink, git status, git diff --cached, the leg's HEAD and branch). Bytes are written with write_bytes.

  • a committed workflow naming the leg with # caf\xe9, bump and --dry-run (2);
  • a UTF-16 workflow with a BOM that names no leg, .yml and .yaml, bump and --dry-run (4);
  • two bad workflows are both named in ONE refusal (1);
  • --to the commit already pinned, with a bad workflow: exit 2, not "nothing to do" exit 1 (1);
  • a committed contracts/spec-pin.yaml with a Latin-1 byte, bump and --dry-run (2);
  • controls (3): the OTHER leg's pin stays bump-leg-validators-red; a UTF-8 workflow with non-ASCII comments, and a UTF-8-with-BOM one, still bump and the bytes survive; a non-UTF-8 notes.txt and NOTES.md, which the validator never reads, do not stop it;
  • added by the fix round (6):

Run against main's scripts/bump-leg.py (ab623dd; tools/x352-tests-at-main-code.log): 16 failed, 3 passed. The 15 that go through the CLI fail on assert 1 == 2 (the run exits 1) with the output holding UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 245 (a workflow), 0xe9 in position 16 (the drafts), 0xff in position 0 (the UTF-16 file) or 0xe9 in position 1880 (the pin); the seam test calls plan_workflows and gets the UnicodeDecodeError itself. The 3 that pass are the controls, which hold on both. (The original 13 gave 10 failed and 3 passed against 86da808 and against #264's head 5269aac, tools/new-tests-main-base.log.gz and tools/new-tests-base.log.gz.) Run against this branch's previous head, 107242d (tools/x352-tests-at-107242d-code.log): 7 failed, 12 passed: the 4 draft tests and the mixed test fail on the unmarked line and on "commit it", the tracked-file test on the new "Commit each file git tracks" sentence, and the seam test passes, because the head was right and nothing pinned it. At this head: 19 passed.

Mutants of the decisive lines, each killed. The original five (tools/mutant-*.log.gz): the pin read with errors="replace" (2 fail); _workflow_texts collecting the files and not raising (8 fail); the workflow read with errors="replace" (8 fail); refusing only files that hold an @ (5 fail: the UTF-16 file naming no leg); cmd_bump back on the raw pin read (2 fail). The marking, this round (tools/x352-mutants.log): an unreadable file git does not track skipped (5 fail; the one that survived the review), every line marked tracked (5), every line marked untracked (4), the exact-spelling comparison in place of _tracked_spelling (1: the seam test), the refusal naming git's spelling in place of the disk's (1: the seam test), the untracked advice dropped (5), the tracked advice dropped (3).

Targeted, at head 3416283 (797c96b plus the Windows fix below, a one-line change to one test; the counts are the same at both), git-filter-repo 2.47.0 on PATH, never the whole suite:

files tests
tests/test_bump_leg_unreadable.py 19
tests/test_bump_leg.py 39 passed, 1 skipped
tests/test_bump_leg_rollback.py 24
tests/test_git_output_decoding.py 67
tests/test_commit_trailers.py 125
tests/test_cli_reference.py + tests/test_repo_hygiene.py 13 + 50

82 passed, 1 skipped in the first three, which hold #344's tests as well as this PR's (tools/x352-run1-bump-leg.log; the skip reason is in it); 130 in test_git_output_decoding.py + test_cli_reference.py + test_repo_hygiene.py (tools/x352-run2-decoding-cli-hygiene.log); 212 passed, 1 skipped in all six files but test_commit_trailers.py, together, at 797c96b (tools/x352-run-final-797c96b.log); 125 in test_commit_trailers.py (tools/x352-run-trailers.log); all seven files together at 3416283: 337 passed, 1 skipped (tools/w352-run-targeted-3416283.log); scripts/render-cli-reference.py --check ok. The handbook's check, that it follows the README's outline and names the README's blob, is in test_repo_hygiene.py; README.md is not touched, so the blob the footer names still holds. The one skip is #344's test_a_tracked_workflow_respelled_by_case_moves_under_gits_spelling, which needs a disk that ignores case and runs on the macOS and Windows jobs. tests/test_shape_doctor.py is not run: neither adopt-project.py nor update-shape.py is touched.

Fix round (review of 107242d)

Lane openRepoShape-2's own review of 107242d found one P3 and three nits; this round takes all four.

Windows fix (3416283)

tests-windows was red at 797c96b (1 failed, 3635 passed): test_the_other_legs_pin_stays_the_validators_refusal asserted contracts/code-pin.yaml, and the Windows job prints contracts\code-pin.yaml. The file is named by the project's own validate-pins.py, whose repo_shape.load_yaml prints the Path it was given, absolute and in the platform's spelling (D:\a\_temp\t\...\Atlas\contracts\code-pin.yaml: not UTF-8 at byte 1880), and this tool echoes the validator's output under bump-leg-validators-red. The behaviour was right (that refusal, every byte rolled back), and the same test had failed Windows at b5f1ef2.

The test changed, not the tool. This tool's refusals name a repository path with as_posix() (_not_utf8, pin_rel) and every other assertion in the file reads that text; load_yaml's native spelling is pinned by other tests (f"REFUSED yaml-unreadable: {path}" in test_manifest_validator.py, test_adopt_execute_refusals.py, test_load_yaml_refusals.py), and changing it would move a pinned template copy in every project. The assertion is now str(Path("contracts") / "code-pin.yaml"): the same text on POSIX, and a substring of the validator's path on Windows. It is the one assertion in the file that reads validator text.

Checked on Linux by making this tool echo the validator's paths in Windows spelling (D:\a\_temp\t\Atlas\contracts\code-pin.yaml, tools/w352-win-shim.py): the old assertion fails exactly as CI's did (tools/w352-probe-old-assertion-win-shim.log), the Windows-spelled one passes (tools/w352-probe-native-assertion-win-shim.log), and the whole file, 19 tests, passes under it (tools/w352-probe-whole-file-win-shim.log). The mutants of the fix round were not re-run: the only line changed is in a control test, and on POSIX its assertion text is the one the mutants ran against.

Left alone, for follow-up issues: a file this tool cannot OPEN (below), and _ensure_root_clean's strict decoding of git status --porcelain for a dirty tracked workflow whose NAME is not UTF-8 under core.quotePath=false, which is a traceback on main and here alike.

Merged main after #344 landed

Merge commit 107242d merges origin/main ab623dd, which holds #344 as a2cb7d3 (closes #332), into this branch. One file conflicted, scripts/bump-leg.py, in the two add-vs-add hunks named below, and both resolved by keeping both sides. The net diff against ab623dd at 107242d was scripts/bump-leg.py and tests/test_bump_leg_unreadable.py and nothing else (the fix round adds docs/handbook.html); git merge-tree --write-tree origin/main HEAD is rc 0 at the current head too, and py_compile passes. The merge adds no code or test of its own beyond the resolution.

Composes with

This branch is main 86da808 (#264's squash) merged into the branch, one merge commit, b5f1ef2; the net diff against 86da808 is scripts/bump-leg.py and tests/test_bump_leg_unreadable.py and nothing else. git merge-tree --write-tree b5f1ef2 <head>, nothing fetched:

against rc conflict
origin/main 86da808 0 none (the merge is this commit)
origin/main 6f4ce47 (one family.py commit newer, seen after the merge) 0 none
#344 (landed on main as a2cb7d3, closes #332; was 7013335 on fix/bump-leg-case-only-rename) 1 at b5f1ef2, resolved in 107242d scripts/bump-leg.py, TWO hunks, both add-vs-add in plan_workflows' neighbourhood
origin/main ab623dd (holds #344), at 107242d and at 797c96b 0 none (the merge is 107242d)
origin/main 92e29d9, at 3416283 0 none

The merge itself conflicted in three hunks against #222's tracked-files change (#207) in plan_workflows, and resolved this way: (1) the helper block keeps main's _tracked_workflows and this change's _not_utf8, _leg_pin_text and _workflow_texts; (2) the docstring keeps both paragraphs; (3) the loop head reads tracked, untracked, then for path, text in _workflow_texts(root, workflows), main's own text = path.read_text(...) line going with the walk it belonged to. (The loop head now passes tracked, listed too.)

#344 landed as a2cb7d3, and its two hunks resolved by keeping both sides in the merge 107242d. (1) The helper block: this change's _not_utf8, _leg_pin_text, _workflow_texts, and #344's _same_file and _tracked_spelling, ours first. (2) The plan_workflows docstring: #344's "A TRACKED FILE THE DISK SPELLS ANOTHER WAY" paragraph, and this change's one sentence after it. The loop merges clean: #344 adds listed = {child.name for child in workflows.iterdir()} and resolves each row through _tracked_spelling(root, path, tracked, listed), and _workflow_texts yields the same paths, so a case-renamed Legs.yml that is not UTF-8 is refused as bump-leg-workflow-unreadable under the disk's spelling, before any spelling question is asked for a plan row (by construction, _workflow_texts runs first, and the fix round's seam test pins that exact case). Every workflow is still read as UTF-8 before anything is written, and a tracked file respelled by case is still planned under git's spelling. At 107242d, test_bump_leg_unreadable.py, test_bump_leg.py (with #344's tests) and test_bump_leg_rollback.py gave 76 passed, 1 skipped (tools/merge2-a.log), the numbers the scratch resolution gave beforehand (tools/with344.log); at 797c96b, 82 passed, 1 skipped, the six added tests being the difference.

Open PRs touching scripts/bump-leg.py or tests/test_bump_leg*: none. #344 landed (a2cb7d3), and the lane's review of 107242d re-checked the list at 14:28Z and 14:42Z, with origin/main at ab623dd both times. At 797c96b, git merge-tree --write-tree against each of the 18 other open heads the review listed (nothing fetched): 15 rc 0, and #358, #356 and #314 rc 1, in shape-doctor.py, adopt-project.py and adopt-project.py + scaffold-project.py, files this PR does not touch (the review found each already conflicts with ab623dd alone). docs/handbook.html conflicts with none of them (tools/x352-merge-tree-all-797c96b.txt).

Left out

  • scripts/validate-pins.py still raises the same traceback. The issue's "Not in this issue". Reproduced again at this head: on a tree whose workflow holds # caf\xe9, python3 scripts/validate-pins.py --root . exits 1 with UnicodeDecodeError from _workflow_refs (templates/assembly-root/scripts/validate-pins.py:111). The file is a pinned template copy: a fix moves the sha256 that contracts/shape-pin.yaml records for scripts/validate-pins.py in every project, which every project already cut then sees as upstream-changed. A separate change with its own cost, and worth filing; bump-leg.py no longer reaches it, since it refuses the file first, but CI and a hand run still do.
  • A file this tool cannot OPEN is the same crash, a different cause. A mode-000 pin or workflow file raises PermissionError out of the same two reads (_leg_pin_text, _workflow_texts), a traceback with no refusal, exit 1, the tree unchanged; on main too. It is not an encoding failure, and the refusal's "re-save as UTF-8" would be the wrong cure for it, so it is not folded in. On POSIX it needs core.trustctime=false: under the default config the chmod makes the root read dirty (bump-leg-root-dirty). Left for a follow-up issue.
  • One refusal names every bad WORKFLOW, not every bad file. cmd_bump reads the pin first, so a root with a bad pin AND a bad workflow names the pin alone on the first run, bump and --dry-run, and the workflows on the second. The reads were not reordered: no promise covers the order.
  • README.md is untouched. It sits at its 1419-line cap (tests/test_repo_hygiene.py; AGENTS.md, README and scaffold-project.py --help: say that a --spec-path or --code-path that is, holds or lies inside a file the shape writes, and a --pin of spec, code or shape or of one product twice, are refused before anything is created, and name those three in the pin refusal's remediation, so an agent briefed from AGENTS.md can say so before it runs the scaffold (#288) #309 raised it from 1416) and main has rewrapped the "It REFUSES ..." paragraph since this branch's base. The new refusals are in the module docstring's list, which is where the existing ones are spelled out, and in the handbook's.
  • The reads are not moved before fetch_leg. They stay where they were, after the fetch and the digest, so the object store is fetched into first, as the issue notes and --dry-run already says. Reordering would change which refusal wins for a root that has two faults.

Closes #289

🤖 Generated with Claude Code

Summary by Sourcery

Make bump-leg.py reject unreadable workflow and pin files clearly and safely before modifying the repository.

New Features:

  • Refuse non-UTF-8 workflow files and the target leg pin by name, including the first invalid byte and decoding reason, before any repository changes occur.
  • Report whether unreadable workflow files are tracked and provide matching remediation for tracked and untracked or ignored files.

Bug Fixes:

  • Replace uncaught UnicodeDecodeError tracebacks with controlled exit-2 refusals for bump and dry-run operations.
  • Refuse every unreadable YAML workflow, including files that do not reference the selected leg, so validation does not fail later.

Documentation:

  • Document the new unreadable workflow and pin refusals in the handbook and bump-leg refusal documentation.

Tests:

  • Add comprehensive coverage for unreadable workflows and pins, dry runs, multiple and untracked files, case-respelled tracked files, tree immutability, valid UTF-8 controls, and platform-specific output.

Brett Heap and others added 9 commits October 11, 2026 22:20
…rolls the pin, the workflow files, the leg's checkout and the index back, so a workflow file it cannot write, a validator line the stdout cannot encode or an interrupt no longer leaves the pin rewritten and the leg on the new commit behind a traceback (#253)

`_apply_rewrite` rolled back on `except (Refusal, CommandFailed)` and nothing
else. A `PermissionError` on a workflow file, a `UnicodeError` from echoing a
validator's output (the crash #230's `errors="replace"` moves the decode error
to) or a `KeyboardInterrupt` passed through it: `main()` catches the same two
types, so the run ended in a traceback with the pin rewritten, the leg on the
new commit and, once staged, the index changed.

The handler is now `except BaseException`, and the rollback it runs is three
helpers instead of a loop and a `try`. `_put_back` attempts each step (every
restored file, the leg's checkout, the index) whatever the steps before it
did, and prints, never raises, a step that fails; `_restore_file` writes a
file back only when it no longer holds its original bytes, so a read-only
file the write never opened is not reported as unrestorable. A `Refusal` or a
`CommandFailed` is re-raised as it was. An `OSError` or a `UnicodeError`
leaves as `REFUSED bump-leg-rewrite-failed` naming the exception and saying
whether the tree is back, the way `update-shape.py` turns an unwritable README
into `update-readme-unwritable`; anything else, an interrupt included, is
re-raised as itself once the tree is back.

tests/test_bump_leg_rollback.py: a read-only workflow file, a stdout that
cannot encode a validator's line, a write that dies half way, an interrupt
while writing and while validating, a defect in the tool, and a rollback step
that itself fails. Each compares the root's HEAD, the pin, every workflow, the
gitlink, `git diff --cached --quiet` and the leg's HEAD and branch with what
they were before the bump. All eight fail on the code before this change.

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…m, not from a codec setting, so they keep meaning the same thing once `main` escapes what a strict stdout cannot encode (#253)

The first cut of the echo test ran `bump-leg.py` with `PYTHONIOENCODING=cp1252`.
#214's `escape_what_stdout_cannot_encode` reconfigures exactly that stream to
`backslashreplace` at the top of `main`, so that test would have started to
see a bump that succeeds the day #214 landed. The echo test is now in-process
with a stdout whose `write` raises `UnicodeEncodeError` for what cp1252
cannot hold, beside a `UnicodeDecodeError` raised from the validator read, so
both ways a validator's output can fail the statement after the staging are
covered whichever of #214 and #230 is in.

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…while the bump was being applied, so an `OSError` from the leg's checkout itself is no longer described as happening after the leg was moved (#253)

The refusal said "after the leg was moved", which is false for an `OSError`
raised by the checkout that moves it (a `git` that cannot be started). The
rollback it follows is unchanged; only the sentence is.

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…then failed, and a second interrupt during the rollback are all put back, and no report claims a rollback that was not complete, so a signing key that cannot sign no longer leaves the bump staged for the re-run to refuse as a dirty root (#253)

Lane openRepoShape-2's review of a8d31ee found the rollback stopped short of
the PR's own promise in four places, and one report said too little:

- The commit ran after `_apply_rewrite` returned, outside the rollback. A
  commit git refuses (`commit.gpgsign` with a signer that fails, a
  `commit-msg` hook) left the pin, the workflows and the gitlink staged and
  the leg on the new commit, and the same command run again once the signer
  was fixed was refused as `bump-leg-root-dirty`. The commit is now the last
  statement inside the `try`. One `git rev-parse HEAD` is read just before
  it, and `_head_guard` compares the root's HEAD with it after any failure:
  unmoved, every byte goes back; moved (or unreadable), the bump is a commit,
  nothing is rolled back under it, and the report says it was committed.
- A `bump-leg-validators-red` refusal said every byte was rolled back even
  when a rollback step had failed. The sentence is now one constant,
  `ROLLED_BACK`, and the handler replaces it with `NOT_ROLLED_BACK` (THE
  ROLLBACK WAS NOT COMPLETE) when the rollback did not earn it, in every
  `Refusal`; a `CommandFailed` gets the same sentence on a line before
  `main()` prints it.
- `_put_back` caught `Exception` only, so a second Ctrl-C during the leg's
  checkout skipped the index step and left `MM` on the pin, the workflow and
  the gitlink. It now catches `KeyboardInterrupt` too, names the step as
  interrupted, and the steps after it still run. No signal handler: this is
  a library function.
- The leg was restored only if the forward checkout returned, but git moves
  HEAD before `post-checkout` runs, so a failing hook (git-lfs's without
  `git-lfs` on PATH) or an interrupt while the checkout was being waited on
  left the leg on the new commit. `_restore_leg_if_moved` reads the leg's
  branch and HEAD and restores only when either differs, so an unmoved leg
  is not checked out again through the hook that refused.
- An interrupt or a defect was re-raised with nothing saying whether the
  tree was back. One stderr line now says so before the traceback.

tests/test_bump_leg_rollback.py: a refused commit (a signer that does not
exist, and a `commit-msg` hook), a failure after the commit landed (an
interrupt, and a refusal), a forward checkout that moved the leg (an
interrupt, and a `post-checkout` hook that refuses the new commit only), an
interrupt inside a rollback step, a `git add` that fails beside a failing
rollback step, and the line before an interrupt's or a defect's traceback.
Twelve cases fail on a8d31ee's script, each for its finding's reason.

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e read before the checkout, so a checkout cut off before it moved the leg is reported as THE ROLLBACK WAS NOT COMPLETE, not as a tree exactly as it was (#253)

Re-verification 1 of #264 found it with a real Ctrl-C. git writes the new
commit's files before it moves the leg's index and HEAD, and a terminal's
SIGINT reaches git and this tool together, so a large forward checkout cut
off between the two leaves the leg on its branch at its commit with the
files it wrote in it. `_restore_leg_if_moved` read only the branch and the
HEAD, found the leg where it stood, and `_roll_back` returned True: the
report said every byte was back and the tree was exactly as it was, while
the leg held 851 files of the cut-off checkout, and the operator's re-run
failed on "untracked working tree files would be overwritten". Main prints
only the traceback there, so the false claim was new in fix round 1.

`_apply_rewrite` now reads the leg's `git status --porcelain -z
--untracked-files=all` as bytes, next to `placement` and before the
checkout. `_restore_leg_if_moved` restores the leg when its branch or HEAD
moved, as before, and then compares the leg's status with that one. When it
differs the step raises `LegNotAsFound`, `_put_back` names the step with a
sentence sending the operator to `git -C <leg> status`, and the verdict is
THE ROLLBACK WAS NOT COMPLETE. The comparison runs after a restore as well,
so a checkout that moved the leg and left a file the checkout back cannot
remove is reported the same way.

Nothing in the leg is removed. An untracked file of the operator's own is
allowed there (`_ensure_root_clean` ignores untracked leg content), and
removing only the paths that appeared and that the new commit's tree holds
is not provably safe: git checkout overwrites an IGNORED file without
asking, and a new `.gitignore` written before the cut can make that file
show as untracked, so such a path could be a file the operator had.

Tests: the cut-off checkout, interrupted and failing with exit 130, and a
checkout that moved the leg, left a file and was interrupted. Each asserts
the line says THE ROLLBACK WAS NOT COMPLETE, nothing says the tree is
exactly as it was, the step is named, and the operator's own untracked file
and the checkout's file are both still there. One more puts an operator's
untracked file in the leg beside a red validator, and shows the rollback
still says every byte is back and leaves the file alone.

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r the commit, an `OSError` after it, and a leg detached at the commit it stood on, so the three rules re-verification 1 could break without a test failing are now held by one each (#253)

Re-verification 1 of #264 ran 17 mutants of 67a1163 and three survived:

- 9: `_head_guard` rolls back when HEAD cannot be read after the commit
  was asked for. `test_a_head_that_cannot_be_read_after_the_commit_rolls_nothing_back`
  makes every `git rev-parse HEAD` fail once the commit has been asked for
  (`commit_once`'s read-back and the guard's), as the reviewer's real
  `post-commit` reproduction did, and asserts the commit landed, the tree
  agrees with it, and the refusal says the commit could not be read.
- 12: the leg compared by its HEAD alone.
  `test_a_leg_detached_at_the_commit_it_stood_on_is_put_back_on_its_branch`
  bumps `--to` the commit already pinned with a workflow reference left
  behind, so the forward checkout detaches the leg at the same commit; a
  red validator follows, and the leg must be back ON `main`.
- 16: an `OSError` after a landed commit made into
  `bump-leg-rewrite-failed`, whose remediation says nothing was committed.
  `test_a_failure_after_the_commit_landed_rolls_nothing_back` gains an
  `oserror` case: the error is re-raised as itself after a line saying the
  bump was committed.

The landed-commit assertions are one helper, shared by the guard's tests.
No code changes.

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d by name before anything moves, so a Latin-1 comment no longer ends the run in a UnicodeDecodeError traceback (#289)

`plan_workflows` read every `.yml` and `.yaml` file under
`.github/workflows/`, and `cmd_bump` read `contracts/<role>-pin.yaml`, with
`read_text(encoding="utf-8")` and nothing that caught the decode error, so a
committed Latin-1 `# caf\xe9` or a UTF-16 file with a byte-order mark ended
the run in a traceback, exit 1, naming Python's codec and not the file.
`--dry-run` died the same way, since both reads come before its early return.
`main()` catches `Refusal` and `CommandFailed` only, and
`UnicodeDecodeError` is a `ValueError`.

Each is now a refusal of its own, exit 2, under `--dry-run` as well:
`bump-leg-workflow-unreadable` names every workflow file that is not UTF-8 by
its path in the repository, the first bad byte and the reason (the words
`load_yaml` uses), and `bump-leg-pin-unreadable` names
`contracts/<role>-pin.yaml` the same way. A workflow file is read whether or
not it names the leg, because `validate-pins.py` reads every one on disk.
The read is before the leg moves, so #253's rollback is not involved and its
promises are untouched.

Refused, not read lossily: `errors="replace"` would let the run go on, but
the rewritten text is written back and the lockstep commit would carry U+FFFD
where the operator's byte was. Not skipped either: the validator would still
fail on the file.

Left out: the project's own `validate-pins.py` still raises the same
traceback on a non-UTF-8 workflow when it is run by hand or by CI. That file
is a pinned template copy, so a fix moves the sha256 `contracts/shape-pin.yaml`
records for it in every project already cut.

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… the width of the file, so the module reads as the rest of the suite does (#289)

No behaviour change: three lines of tests/test_bump_leg_unreadable.py were 81
characters, in a suite that holds to 79. The 13 tests are unchanged and pass.

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…le-files

The one conflicted file is scripts/bump-leg.py, in three hunks in
`plan_workflows`, all against #222's tracked-files change (#207): the helper
block keeps main's `_tracked_workflows` and this branch's `_not_utf8`,
`_leg_pin_text` and `_workflow_texts`; the docstring keeps both paragraphs;
the loop head reads `tracked`, `untracked`, then
`for path, text in _workflow_texts(root, workflows)`, main's own
`text = path.read_text(...)` line going with the walk it belonged to. The
net diff against main is scripts/bump-leg.py and
tests/test_bump_leg_unreadable.py, nothing else.

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@sourcery-ai

sourcery-ai Bot commented Oct 12, 2026

Copy link
Copy Markdown

Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 11 hours and 6 minutes by commenting @sourcery-ai review. Upgrade to get a review now.

@brettheap

Copy link
Copy Markdown
Contributor Author

@codex review

@sourcery-ai

sourcery-ai Bot commented Oct 12, 2026

Copy link
Copy Markdown

Reviewer's Guide

The PR prevents bump-leg.py from leaking UnicodeDecodeError tracebacks by validating the leg pin and every workflow as UTF-8 before any rewrite or dry-run decision, reporting named exit-2 refusals with actionable byte diagnostics, and adds regression tests covering failure aggregation, ordering, immutability, and valid-input controls.

Sequence diagram for UTF-8 validation before bump-leg changes

sequenceDiagram
    participant Operator
    participant BumpLeg as bump-leg.py
    participant Pin as LegPin
    participant Workflows as WorkflowFiles
    participant Refusal

    Operator->>BumpLeg: cmd_bump(args)
    BumpLeg->>Pin: _leg_pin_text(root, pin_path)
    Pin-->>BumpLeg: UTF-8 pin text
    BumpLeg->>Workflows: _workflow_texts(root, workflows)
    Workflows-->>BumpLeg: UTF-8 workflow texts
    BumpLeg->>BumpLeg: plan_workflows(root, repository, commit)
    BumpLeg-->>Operator: Apply or --dry-run result

    alt Pin is not UTF-8
        Pin-->>Refusal: bump-leg-pin-unreadable
        Refusal-->>Operator: Exit 2 with path, byte, and reason
    else Any workflow is not UTF-8
        Workflows-->>Refusal: bump-leg-workflow-unreadable
        Refusal-->>Operator: Exit 2 naming every bad workflow
    end
Loading

Flow diagram for unreadable bump-leg refusal

flowchart TD
    A[Start bump-leg command] --> B[_leg_pin_text]
    B -->|Invalid UTF-8| C[bump-leg-pin-unreadable]
    B -->|Valid UTF-8| D[_workflow_texts]
    D -->|One or more invalid files| E[bump-leg-workflow-unreadable]
    D -->|All workflows valid| F[plan_workflows and possible rewrite]
    C --> G[Exit 2 with byte diagnostics and remediation]
    E --> G
    G --> H[Working tree, index, and leg checkout unchanged]
    F --> I[Apply changes or complete --dry-run]
Loading

File-Level Changes

Change Details Files
Add explicit UTF-8 validation and named refusals before bump planning or file rewrites.
  • Centralize decode-error formatting with repository-relative path, byte offset, and reason.
  • Read the leg pin through a helper that raises bump-leg-pin-unreadable with remediation details.
  • Read all .yml and .yaml workflows up front, aggregate every unreadable file, and raise bump-leg-workflow-unreadable even when a file does not reference the leg.
  • Keep validation before the dry-run and rewrite stages, preserving the no-working-tree/index/checkout-change guarantee.
scripts/bump-leg.py
Add comprehensive regression coverage for unreadable inputs, ordering, preservation, and untouched repository state.
  • Cover Latin-1 and UTF-16 workflow failures for normal and dry-run commands, including aggregation of multiple bad files and precedence over “nothing to do.”
  • Cover unreadable leg pins with distinct refusal IDs and verify unrelated-leg behavior remains unchanged.
  • Verify exit 2, file paths, byte diagnostics, remediation text, absence of tracebacks, and complete tree/index/submodule immutability.
  • Verify valid UTF-8, UTF-8 BOM, non-ASCII workflow content, and ignored non-workflow files continue to work byte-for-byte.
tests/test_bump_leg_unreadable.py

Assessment against linked issues

Issue Objective Addressed Explanation
#289 Make bump-leg.py refuse non-UTF-8 workflow files and the selected leg pin by name, including the relative path, byte offset, and decoding reason, with exit 2 and no UnicodeDecodeError traceback. ✅
#289 Ensure the refusal applies during --dry-run and occurs before any working-tree, index, pin, workflow, or leg checkout changes, without lossily replacing bytes or silently skipping unreadable workflow files. ✅
#289 Add regression coverage for unreadable Latin-1 and UTF-16 workflow and pin files, including files that do not reference the selected leg, while preserving valid UTF-8 workflows and unrelated non-UTF-8 files. ✅

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-12T16:51:06.639969Z 3416283 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: b5f1ef2930

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…344 landed in bump-leg.py

Lane 3's #344 landed on main as a2cb7d3 (closes #332) and changed
scripts/bump-leg.py in the same two places as this branch, so the merge
conflicted in that one file, in two add/add regions. No other file conflicted.
Both sides are kept in each, because the two changes add different things and
neither replaces the other:

1. The helpers before `plan_workflows`. This branch adds `_not_utf8`,
   `_leg_pin_text` and `_workflow_texts` (#289); main adds `_same_file` and
   `_tracked_spelling` (#332). All five stay: ours first, then theirs.

2. The last paragraph of `plan_workflows`'s docstring. Main's "A TRACKED FILE
   THE DISK SPELLS ANOTHER WAY IS STILL TRACKED (#332)" paragraph stays whole,
   and this branch's one sentence on a file that is not UTF-8 follows it.

Git merged the body of `plan_workflows` with no conflict, and the composition
is the one both changes need: the loop reads
`for path, text in _workflow_texts(root, workflows)`, so every workflow is read
as UTF-8 before anything is written and a file that is not UTF-8 is refused by
name whatever its spelling on disk; each row is then spelled through
`_tracked_spelling(root, path, tracked, listed)`, so a tracked file respelled
by case is still planned under git's spelling.

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@brettheap

Copy link
Copy Markdown
Contributor Author

(lane openRepoShape-2, 2026-10-12T14:46Z)

This lane reviewed its own PR #352 at 107242d, read-only in its own detached worktrees, with nothing authored.

MERGEABLE WITH NITS. Merge only after tests-macos and tests-windows pass at 107242d; both were pending, and Linux tests had passed.

Findings:

Tests:

  • 107242d, which equals itself merged with ab623dd:
    • the three bump-leg test files: 76 passed, 1 skipped;
    • the decoding, CLI reference, hygiene and trailer tests: 255 passed;
    • render-cli-reference.py --check ok.
  • ab623dd with the new file: 10 failed and 3 passed, each failure the issue's traceback.
  • The issue's six rows exit 1 on main and exit 2 here, with the tree unchanged.
  • Mutants: 6 of 9 killed.

Merges: no other open head touches scripts/bump-leg.py or tests/test_bump_leg*. Against 18 open heads, 15 merge with rc 0. #358, #356 and #314 conflict only where they already conflict with main.

The full report is at handoffs/xFactory/attachments/lane-openRepoShape-2/review352-107242d/final-report.md in opensoft/brett-wip.

brettheap and others added 2 commits October 12, 2026 14:54
…ine marked untracked or ignored when git does not track it, and the remediation tells a tracked file's operator to commit it and an untracked one's not to, so no operator commits a draft onto the bump branch to clear the refusal (#289)

The review of #352 (107242d) found that the `bump-leg-workflow-unreadable`
remediation ended "re-save every file named as UTF-8 ... commit it, and run
the same command again" for every file it named. That is true of a file git
tracks and false of one it does not: re-saving an untracked draft is enough
(the bump then goes through and the draft is not in the lockstep commit),
committing it puts a draft workflow on the bump branch, which #207's
`bump-leg-workflow-untracked` refusal exists to prevent, and `git add`
refuses an ignored file outright.

`_workflow_texts` now takes `tracked` and `listed`, the two sets
`plan_workflows` already holds before it calls it, and asks
`_tracked_spelling` of each file it refuses. A line git does not track ends
`(untracked or ignored)`, and the remediation gives each kind its own advice:
"Commit each file git tracks." for a tracked one, and "needs no commit, only
the re-save" with the move-it-out alternative for the others. The question is
asked only of a file already refused and reads no file, so every read still
comes before anything is written, `--dry-run` included. `_tracked_spelling`
is the question #344 settled, so a tracked file the disk spells another way
is not marked untracked.

Tests, in tests/test_bump_leg_unreadable.py (19, from 13):
- an untracked and an ignored draft holding `# caf\xe9`, bump and `--dry-run`
  (4): exit 2, the line marked, the remediation without "commit it", then the
  file re-saved as UTF-8 and still untracked bumps and is not in the commit;
  at 107242d these fail on the unmarked line and on "commit it";
- an unreadable tracked file and an unreadable untracked one in one refusal (1);
- a tracked `legs.yml` holding a Latin-1 byte, renamed outside git to
  `Legs.yml`, through #344's `_same_file` seam so it runs on every OS (1):
  refused `bump-leg-workflow-unreadable` under the disk's spelling, not
  marked untracked, bytes unchanged. This pins the composition the review
  confirmed by probe; it passes at 107242d.

Mutants of the marking, each killed: an unreadable untracked file skipped
(the one that survived the review), every line marked tracked, every line
marked untracked, the exact-spelling comparison, and each half of the advice
dropped.

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ow file or the leg's pin that is not UTF-8, and the page's change line records it, so a reader of the handbook meets the refusal in the list and not at the command line (#289)

The review of #352 found that the handbook's "Refuses ..." list for
`bump-leg.py` omitted the two refusals this change adds
(`bump-leg-workflow-unreadable` and `bump-leg-pin-unreadable`), and that its
change line recorded #207's refusal the same way and not this one. Nothing
the page said was false. README.md is at its 1419-line cap, so it is left as
it is, and the README blob the footer names does not move.

The list gains one clause, with the one fact an operator needs about a file
git does not track (it needs the re-save and no commit), and the change line
gains (#289).

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@brettheap

Copy link
Copy Markdown
Contributor Author

(lane openRepoShape-2, 2026-10-12T14:59Z)

Fix round pushed: 797c96b (107242d..797c96b), answering this lane's review (#352 (comment)):

Targeted at 797c96b:

  • test_bump_leg_unreadable.py: 19 passed.
  • test_bump_leg.py: 39 passed, 1 skipped.
  • test_bump_leg_rollback.py: 24 passed.
  • test_git_output_decoding.py: 67 passed.
  • test_cli_reference.py: 13 passed.
  • test_repo_hygiene.py: 50 passed.
  • test_commit_trailers.py: 125 passed.
  • render-cli-reference.py --check ok.

The two follow-ups stay out of this PR: #383 (PermissionError), and the strict git status decode, still to file. This lane's reviewer re-verifies this head next.

Lane: openreposhape-2 (openRepoShape-2)

@brettheap

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

…e spelling the validator prints, so tests-windows passes where the validator names `contracts\code-pin.yaml` (#289)

`test_the_other_legs_pin_stays_the_validators_refusal` asserted the
POSIX text `contracts/code-pin.yaml`. That file is named by the
project's own `validate-pins.py`, whose `repo_shape.load_yaml` prints the
Path it was given, absolute and in the platform's own spelling, and
bump-leg echoes the validator's output under `bump-leg-validators-red`.
On the Windows job it reads `D:\a\_temp\t\...\Atlas\contracts\code-pin.yaml`,
so the assertion failed there (run 38266538839: 1 failed, 3635 passed).

The behaviour was right: the validator refuses `yaml-unreadable`, the
bump refuses `bump-leg-validators-red`, and every byte is rolled back.
The test, not the tool, changes. This tool's own refusals name a file by
its repository path, `as_posix()` (`_not_utf8`, `pin_rel`), and the rest
of the module asserts that text; `load_yaml`'s native spelling is pinned
by other tests as `f"REFUSED yaml-unreadable: {path}"`, and moving it
would change a pinned template copy in every project.

The assertion now looks for `str(Path("contracts") / "code-pin.yaml")`:
the same text as before on POSIX, and `contracts\code-pin.yaml`, a
substring of the validator's path, on Windows.

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@brettheap

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Another round soon, please!

Reviewed commit: 3416283779

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

brettheap added a commit that referenced this pull request Oct 12, 2026
…mp-leg-unreadable-oserror, so this stacked branch carries #352's Windows test fix

#352 moved from 797c96b to 3416283, one test-only commit: the test of
the other leg's pin in tests/test_bump_leg_unreadable.py looks for the
path in the spelling the validator prints, which is
`contracts\code-pin.yaml` on Windows. That inherited test was the only
tests-windows failure at e2c78ee. No file of this branch's own change is
touched, so the merge is clean.

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 12, 2026
@brettheap

Copy link
Copy Markdown
Contributor Author

(lane openRepoShape-2, 2026-10-12T17:41Z)

Re-verify of PR #352's fix round (797c96b), and the Windows fix at 3416283.

MERGEABLE. CI at 3416283 is green on all three OSes: tests, tests-macos and tests-windows passed (17:40Z).

@sonarqubecloud

Copy link
Copy Markdown

@brettheap

Copy link
Copy Markdown
Contributor Author

(lane openRepoShape-3, 2026-10-12T17:48:41Z) An identity notice for lane openRepoShape-2. Brett has set the estate's git identity to the service address in every container: the host's ~/.gitconfig is now that, and the repository-local user.email overrides under the projects folder are gone. Three commits from 2026-10-11 carry a profile address as author and committer: 3c499c5, a8d31ee and f73feec (bump-leg.py). They are on fix/bump-leg-unreadable-files (this PR, #352) and fix/bump-leg-unreadable-oserror (#403), and in merged #264.

Land #352 and #403 only by squash with an explicit subject and body. GitHub's default squash message adds a Co-authored-by: line for every commit author other than the PR's, which would copy that address onto main. Do not use a merge commit or a rebase merge either: both keep the commits themselves. Brett is deciding with GitHub Support what happens to the copies GitHub keeps under each PR.

@brettheap
brettheap merged commit 53bee5b into main Oct 12, 2026
5 checks passed
@brettheap
brettheap deleted the fix/bump-leg-unreadable-files branch October 12, 2026 18:46
brettheap added a commit that referenced this pull request Oct 12, 2026
… so the branch holds the head that squash-landed as 53bee5b

#352 landed on main as the squash 53bee5b. This branch was stacked on #352
and still holds #352's original commits (up to 3416283), while main holds
only their squash, so GitHub reads the retargeted #403 as CONFLICTING.

dfc998a is #352's final head: 3416283 plus a merge of main 45e01f4. It is
the first of the two merges that clear this: the merge is clean, since the
branch already holds 3416283, and it brings in main 45e01f4's own changes
(scripts/bump-leg.py auto-merged with no conflict).

The second merge, of main 53bee5b, follows.

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 12, 2026
…or, keeping this branch's own bump-leg.py, docs/handbook.html and tests/test_bump_leg_unreadable.py

#352 landed as the squash 53bee5b. This branch holds #352's original
commits, merged up to its final head dfc998a in the previous commit, so
git saw the squash and those commits as two edits of the same lines and
conflicted in scripts/bump-leg.py and docs/handbook.html.

Both took this branch's side (`git checkout --ours`). That is safe because
`git diff dfc998a 53bee5b -- scripts/bump-leg.py docs/handbook.html
tests/test_bump_leg_unreadable.py` is empty: the squash's content of #352's
three files is dfc998a's, which this branch holds with #403's own change on
top. tests/test_bump_leg_unreadable.py merged with no conflict. No other
file conflicted. The three files are byte for byte what they were before
this merge.

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
brettheap pushed a commit that referenced this pull request Oct 12, 2026
#357 landed as the squash 5678f68: main and this branch both carry doctor B's text from base 45e01f4 under two histories, so `shape-doctor.py` and `tests/test_doctor_validator_rows.py` conflict, and both resolve to this branch's side (#277)

#357 (doctor B) landed as a squash, a new commit, while this branch carries B as a merge of its head e44d860. Git sees one text added twice with different ancestry, so a merge of main conflicts in `shape-doctor.py` (content) and `tests/test_doctor_validator_rows.py` (add/add), and `gh pr update-branch` would refuse the pull request.

Both files resolve with `git checkout --ours`, because this branch is e44d860 plus this pull request's change and the squash's bytes for both files are e44d860's. Proved before this commit:

- `git show 5678f68:<file>` equals `git show e44d860:<file>` byte for byte for `shape-doctor.py` (sha256 c206264b...) and for `tests/test_doctor_validator_rows.py` (sha256 34075904...). Main has touched neither file since 5678f68.
- `git diff --cached de09b7c -- shape-doctor.py tests/test_doctor_validator_rows.py` is empty, where de09b7c is the tree the phase-2 reviewer tested for main cb7063f with 36321c4: 488 passed, 3 skipped.

Against main the tree differs only in this pull request's five files. Beyond de09b7c it differs only by main's later #352 (`scripts/bump-leg.py`, its test and `docs/handbook.html`).

Lane: openreposhape-4 (openRepoShape-4)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
brettheap pushed a commit that referenced this pull request Oct 12, 2026
… into fix/adopt-record-submodule-placement

Brings in f7f4b14 (#366), cb7063f (#390), 5678f68 (#357) and 53bee5b
(#289) on top of 3ef059c, which 563ea16 had merged. Merged by sha, no
rebase.

One conflict, kept on both sides:

- tests/test_manifest_validator.py, the import block. cb7063f adds
  `import os`, `import subprocess` and `import sys`; this branch had
  already added `import os` and `import sys`, so only `subprocess` was in
  conflict. The merge keeps all three, once each, as cb7063f's message
  says it would, with this branch's `from conftest import REPO, git,
  run_script` and `from repo_shape import parse_yaml,
  path_spelling_fault` after them.

`git merge-tree --write-tree` of the two parents gives the same tree in
every file but that one.

Checked:

- Every top-level definition of the eleven modules either side changed
  is the changing side's text, byte for byte. The only ones both sides
  changed are that import block and shape-doctor.py's module docstring,
  where the merged docstring differs from main's by exactly this branch's
  two hunks, and from this branch's by exactly 5678f68's one.
- No name is bound twice by the merge. scripts/shape_materialize.py
  imports `YamlError` and `parse_yaml` twice, but main 3ef059c does that
  too.
- Each module's names are the base's plus what each side added.

Lane: openreposhape-1 (openRepoShape-1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap pushed a commit that referenced this pull request Oct 12, 2026
…plan-manifest-invalid` joins `_entry_findings` after #255's values, and this branch's `plan-root-symlink` stays after the source's own submodules

origin/main moved 92e29d9 -> dd8d0ea (twelve commits), merged by sha:
- f0a8bf6 (#330 via #381), scripts/family.py;
- 87d2eb7 (#333 via #379), update-shape.py;
- cd1fc79 (#327 via #375), shape-doctor.py;
- 45e01f4 (#348 via #396), tests/test_agents_md_codes.py: every
  code-shaped name AGENTS.md spells is a code a tool prints;
- 3ef059c (#258 and #261 via #314), adopt-project.py, scaffold-project.py
  and scripts/shape_materialize.py: the project's YAML reader and its
  `validate-manifest.py` are asked about the manifest's values before
  anything exists; `check` reports `plan-manifest-invalid` through
  `_manifest_findings` in `_entry_findings`; `_repository_urls` refuses
  `adopt-name-too-long`;
- f7f4b14 (#366 via #399), scripts/family.py;
- cb7063f (#301 and #280 via #390), scripts/repo_shape.py and the
  template's validate-manifest.py: a bracket or brace in a plain flow
  scalar is refused;
- 5678f68 (#278, #297 and #298 via #357), shape-doctor.py;
- 53bee5b (#289 via #352), scripts/bump-leg.py;
- f5cb616 (#369 via #398), scripts/shape_materialize.py: the docstrings
  of `force_stage` and `stage_written`;
- a92e60b (#326 via #400), scripts/family.py;
- dd8d0ea (#292 via #353), adopt-project.py: `_elector` keeps a
  `user.name` byte that is not UTF-8, `_remote_repository` reads an
  origin URL that is not UTF-8 as none, and `adopt-no-org` is asked
  before `checked_value`.

scripts/shape_materialize.py, scripts/repo_shape.py and every other file
merge with no conflict. adopt-project.py conflicts in two hunks of
`_entry_findings`, each resolved by keeping both sides:
1. The docstring. Main's order, with #261's manifest findings after
   #255's values and before #190's relative urls; this branch's symlinks
   stay after the source's own submodules.
2. The return. Main's `_manifest_findings(plan)` call, and the findings
   in main's order: each entry's path (#196), coverage, the legs and what
   the root keeps where the shape writes, the source's own submodules
   (#166), this branch's `_root_symlink_findings`, #255's values, #261's
   manifest findings, and #190's relative urls last. The remediation and
   the NOTEs are main's.

How each of main's commits that reach this branch's code meets it:
- 3ef059c: `_manifest_findings` asks the validator the template ships
  about the values `execute` writes; it reads no symlink and no path
  the shape writes. Its materializer half adds the validator's helpers,
  `refuse_unstorable_names` and `MANIFEST_VALIDATOR`, and holds
  `election_date` to the validator's `DATE_RE`; it leaves `place`,
  `_materialize`, `copy_tree`, `template_files` and `_refuse_unwritable`
  as they were.
  In `execute`, its `adopt-name-too-long` (in `_repository_urls`) comes
  before #255's clone-URL refusal, `_refuse_what_check_finds` and this
  branch's `core.symlinks` probe, all before any leg. Its new paragraph
  of the module docstring follows the one 6b53a6d corrected; both merge
  with no conflict.
- f5cb616: docstrings only.
- dd8d0ea: `_remote_repository`, `_checked_plan_inputs`' organisation and
  `_elector`, none of them this branch's.
- cb7063f: the plans `check` and `execute` read go through this reader.
  The plans `plan` writes for this branch's tests hold their entries as
  block mappings, each path `double_quoted` where it needs to be, and
  the rehearsal below read every one of them.

Checked on the merged tree:
- AST (fin183g/names.py, base 92e29d9): no top-level name is bound twice
  in adopt-project.py, scaffold-project.py or scripts/repo_shape.py, and
  none is loaded but bound nowhere. scripts/shape_materialize.py binds
  `YamlError` and `parse_yaml` twice, as main dd8d0ea does (1f90a0f's
  import and 3ef059c's, the same objects); the merge adds none.
- Both sides intact (fin183g/sides_intact.py): the only lines that differ
  from either side's own change are those of the two hunks.
- A rehearsal of this merge (tree d21f8b4, the same as this one) passed
  tests/test_adopt_root_symlinks.py with main's
  test_adopt_{manifest_values,git_reads,relative_submodule_url}.py,
  test_scaffold_manifest_values.py, test_agents_md_codes.py and
  test_yaml_flow_brackets.py: `440 passed`.

Lane: openreposhape-1 (openRepoShape-1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 12, 2026
… lines and clones nothing, and render-cli-reference.py refuses a tree that is not a checkout, so asking a download for help is no longer a clone of GitHub (#341) (#452)

From anywhere but a checkout of the standard (a downloaded `setup.sh`, `curl … | bash -s -- --help`, a `git archive` extraction), `setup.sh --help` went down section 0 like any other run. It cloned the public upstream before printing a word, died with git's exit 128 offline, and otherwise printed the CLONE's `setup-project.py` usage. `scripts/render-cli-reference.py` inherited all of it in such a tree: every rendering, `--check` included, reached GitHub and then documented upstream's flags under that tree's `setup.sh` heading. The clone attempts were reproduced on 87d2eb7, offline (`GIT_ALLOW_PROTOCOL=file`); what the generator renders when that clone succeeds is read from the code, not reproduced:

| | base 87d2eb7 | this branch |
|---|---|---|
| lone `setup.sh`, `bash setup.sh --help` | `(0) self-bootstrap`, `git clone --quiet --depth 1 -- https://github.com/opensoft/openRepoShape.git <tmp>`, `fatal: transport … not allowed`, exit 128 | the shim's own text, exit 0, no clone (`GIT_TRACE`), no temporary directory |
| same, `OPENREPOSHAPE_REPO=<checkout>` | a local clone, then the clone's `usage: setup-project.py …` | the shim's own text, no clone |
| `bash -s -- -h < setup.sh` | the clone, exit 128 | the shim's own text, exit 0 |
| `render-cli-reference.py --check` in a copy with no `.git` | tried the clone (stopped only by `GIT_ALLOW_PROTOCOL`, surfacing as `setup.sh --help exited 128`) | refuses, naming the tree, exit 2, no clone |

**What changed.**

**`setup.sh`**
- Section 0's one scan of the command line now also notes `-h` and `--help` before the first `--`.
- The shim answers them first: before the `$OPENREPOSHAPE_REPO` guard, before `mktemp`, before `git clone`. That is where `setup-project.py` answers them: before it looks at anything.
- The text says:
  - this is the bash front door to `setup-project.py`;
  - any other run from here first clones the standard into a temporary directory, from `$OPENREPOSHAPE_REPO` or, when that is unset or empty, `https://github.com/opensoft/openRepoShape.git`;
  - the one flag list is `python3 setup-project.py --help`, which answers with no network from a checkout or from that one file downloaded on its own, and `docs/cli.md` in the repository.
- It lists no flag, so #50's one `--help` still holds.
- The default is now one constant, `DEFAULT_SHAPE_REPO` (the name `setup-project.py` gives it), read by both the clone and the help. The help never prints the variable's value.
- Inside a checkout nothing changes: the shim still execs `setup-project.py` before section 0 reads a word.
- The header comment no longer says the shim "parses nothing". It now says the shim parses nothing of the RUN, and names the three things section 0 reads. The comment above the scan no longer says the scan never decides what the run does: the first two reads only decide whether a temporary directory is removed and whether a ref is checked out, and the third decides whether there is a run at all.

**`scripts/render-cli-reference.py`**
- `require_a_checkout()` asks `setup.sh`'s own `is_shape_checkout` question: `git -C <tree> rev-parse --show-toplevel`, and `scaffold-project.py` plus `contracts/repository-naming.yaml` at that top level.
- It also asks that the top level IS the tree (`os.path.samefile`). A copy nested inside another checkout passes the shim's probe, and its `setup.sh --help` would print that checkout's usage.
- It is called in `main()`, before the no-bash logic, and at the top of `render()`, which `tests/test_cli_reference.py` calls directly. The refusal names the tree and says to run it in a `git clone`, exit 2.
- It reads git's answer the way the shim's `$(...)` does: `os.fsdecode(...).rstrip("\r\n")`, so a checkout whose path ends in a space, or is not UTF-8, is a checkout here too.
- Every child now runs under `GIT_ALLOW_PROTOCOL=file`, so a regression fails closed.
- Inside a checkout neither measure changes a byte of a rendered help block. `--check` was `[ok]` before, and `[ok]` after regenerating.

**Docs**
- `README.md`: "`python3 setup-project.py --help` and, from a checkout, `setup.sh --help` list every flag, this one included. From a download, `setup.sh --help` clones nothing and lists none: it names `setup-project.py --help` … and `docs/cli.md`". That is +3 lines against main; the README cap in `tests/test_repo_hygiene.py` goes 1419 -> 1422, with its dated paragraph. `openRepoShape --help` is not the subject any more, because it names eight of `setup-project.py`'s fourteen flags besides `--help`. § What setup.sh does no longer gives the shim a line count ("about seventy lines of code"; it was 93 before this change and is 122 now), here, in the handbook and in `setup-project.py`'s docstring.
- `docs/handbook.html`: the page's copy of that sentence and of the shim's description, the footer's history clause, and the new README blob (`5465d135…`).
- `openRepoShape --help`: its one sentence "`./setup.sh --help` prints setup-project.py's usage, which lists them all" was false from a download. It now names `python3 setup-project.py --help`, and `./setup.sh --help` "in a checkout of this standard".
- `docs/cli.md` is regenerated: a note under `setup.sh` saying what `--help` does outside a checkout, and that `openRepoShape` sentence.
- `tests/test_setup_sh.py`: `test_no_test_here_depends_on_gh_being_authenticated`'s docstring is made true.

**Decisions** (the coordinator's; Brett may reverse).

1. **The shim's answer outside a checkout is option (a), with no flag list.** It is a short text of its own, exit 0, with no clone, no network and no temporary directory. Inside a checkout `--help` is still `setup-project.py`'s.
   - `-h` is treated like `--help`, because `setup-project.py` reads both.
   - Only arguments before `--` count. `setup.sh <Project> -- --help` is still a run, whose scaffold is asked for ITS help.
   - **Known false positives** (written into the shim's comment): runs `setup-project.py` would refuse anyway, such as `--org --help` or `--wat --help`, get the text and exit 0 instead of exit 2. Nothing is created either way.
   - **One known false negative**: `--visibility -- --help`, where `--` is a value, still clones as before. Fixing it would need the value-flag table in bash, which is the second parser #50 removed.
2. **Whose help.** The shim's text is its own, so it names no clone URL or ref. No path that prints a usage is left unattributed. `setup.sh <P> -- --help` prints the clone's scaffold usage under the `(0) self-bootstrap` clone line and `checkout: <tmp>`, which already say whose it is.
3. **The generator gets both measures**: the checkout refusal and `GIT_ALLOW_PROTOCOL=file`.
4. **Docs**: README, the handbook's reading of it, the shim's header, the test docstring and `openRepoShape --help`'s sentence, with `docs/cli.md` regenerated.
5. **The #50 guard is changed on purpose.** `tests/test_setup_project_py.py::test_setup_sh_parses_no_flags_of_its_own` refused any `--help` in the shim's code. It now excuses the spelling in exactly two places: the lines of `shim_help()`, each of which must be one `say "..."` with no `$(`, `${` or backtick in it, and section 0's one `-h|--help)` label. Outside those two, neither `--help` nor a bare `-h` may occur in the code. It still refuses `-y)`, `-h)`, `--yes)` and a second `--help)` label, and since fix round 1 it also refuses a quoted `"--help")`, a `[[ $1 == --help ]]` reader anywhere else (the developer path, the scan), and a reader hidden in a `say` line.

**Evidence.** The runs and the merge-tree below were taken at e2df9a8, the branch's head before the last merge of main (the last paragraph).

- **Targeted runs on e2df9a8** (d15a888, the fix round, merged with `origin/main` 53bee5b; the branch's first commit 968db85 is the writer's 16d1639 re-authored with the same tree 0d3243f), with a stub `gh` first on PATH, `GH_TOKEN` bogus, an empty `GH_CONFIG_DIR`, `GIT_ALLOW_PROTOCOL=file` and a blanked git identity:
  - `tests/test_setup_sh.py` and `tests/test_setup_project_py.py`: **132 passed**, 0 skipped;
  - `tests/test_cli_reference.py`, `tests/test_repo_hygiene.py`, `tests/test_agents_md_codes.py` and `tests/test_openreposhape_command.py`: **164 passed**, 0 skipped;
  - all six files together: **296 passed**, 0 skipped, 0 failed. Before the merge, on a throwaway merge of d15a888 and main 5678f68, the same six files gave 296 passed as well. review341 ran them on 968db85 merged with main 3ef059c: 132 + 162 passed.
- **The new tests against base 87d2eb7's code**: 15 failed. These are every new behaviour test (the shim's help ×4 spellings, its text, no echo, `bash -s`), the five generator refusal tests, `GIT_ALLOW_PROTOCOL` and the shim-probe tie. The 9 that pass are the "unchanged" guards (`-h`/`--help` inside a checkout, `-- --help` passthrough ×2) and pre-existing tests.
- **Mutation proof**: 18 single-hunk mutations in a throwaway worktree, each caught by a named test. They cover:
  - the help exit;
  - `-h`;
  - the `--` stop;
  - help after the env guard;
  - the default not named;
  - a flag line added;
  - the default not shared;
  - a second `--help)` label;
  - `main`'s guard, `render`'s guard, and both;
  - the top-level-is-the-tree check;
  - the files check;
  - `GIT_ALLOW_PROTOCOL`;
  - the files drifting from the shim;
  - the Tool note;
  - the `openRepoShape` sentence;
  - the README sentence.
- **The new tests**:
  - `tests/test_setup_sh.py` (`WINDOWS_SKIP` with the rest of the file):
    - `--help`, `-h`, after a run's arguments, and beside `--keep-shape-checkout --shape-ref`: exit 0, the shim's text, no `(0) self-bootstrap`, no usage, no `git clone` in `GIT_TRACE`, nothing in `TMPDIR`;
    - the text names the variable, its default, `setup-project.py --help` and `docs/cli.md`, and no flag but `--help`;
    - a forged `$OPENREPOSHAPE_REPO` is never echoed;
    - `bash -s -- --help`;
    - `<Project> -- --help` and `-- -h` still clone (locally) and reach the scaffold's usage. This is also the positive control that the trace sees a clone;
    - inside a checkout, `-h` and `--help` are `setup-project.py --help` byte for byte, on stdout and on stderr (stderr was not compared before the fix round; by probe it was already identical).
  - `tests/test_cli_reference.py`:
    - a `git ls-files` copy with no `.git`, one inside an unrelated work tree, and one inside a work tree that looks like a checkout of the standard: `--check` and a plain write both exit 2, naming the tree, with nothing written and no clone;
    - `render()` itself refuses;
    - the checkout refusal comes before the no-bash one;
    - `child_env()` pins `GIT_ALLOW_PROTOCOL=file`;
    - the generator's files are the ones the shim's probe reads;
    - (fix round 1) `--check` in a real `git init` copy whose path ends in a space, and in one whose path is not UTF-8, exits 0 with `[ok]`; each is skipped where the file system cannot hold the name (both on Windows; the second wherever creating the directory fails, which is expected on macOS, where no runner was available here).

**Merge-tree, as of e2df9a8 (re-run after merging `origin/main` 53bee5b into the branch).**

- At e2df9a8, `origin/main` (53bee5b) is an ancestor of the head: the branch carries a merge commit, so it was mergeable with main by construction. `render-cli-reference.py --check` was `[ok]` on it.
- **#352 landed as 53bee5b (#289) and conflicted in `docs/handbook.html`'s footer, as predicted.** The merge keeps both "added (#n)" clauses in landing order after "(#288)": (#289), then (#341), and keeps this branch's `README.md blob 5465d13`, which is `git hash-object README.md` again: main does not touch README.md and its own blob is `e8da9381…`.
- #314 has landed (3ef059c). The other PRs then open that share a file with this branch, re-checked against d15a888 before the merge (heads fetched at that time) and #403 again against e2df9a8:
  - clean against this branch: #394 (`tests/test_repo_hygiene.py`, not the README cap; it conflicts with main alone in `tests/test_manifest_validator.py`, a file this branch does not touch), #395 (one-line `visibility` assertion edits in `tests/test_setup_sh.py` and `tests/test_setup_project_py.py`, independent of this branch; its hunks in `docs/cli.md` are at :246-279), #397, #400 and #401 (each regenerates other sections of `docs/cli.md`);
  - **#403 (head 5b1da66) conflicted in the same footer hunk**: it merged clean with main alone and conflicted with this branch. It has since landed as dba5fb0 (#383, #403), and the merge of main into this branch (the last paragraph) keeps every clause in landing order after "(#289)", that is (#383) and then (#341), and names the blob of the README that lands, `5465d135ca7263010ee354424041f90c4553d2e7`. Main does not change README.md, so that is the blob this branch already named.
- `docs/cli.md`: #395, #397, #400 and #401 all merged clean with d15a888 plus main 5678f68, and the generator's `--check` was `[ok]` on each merged tree (#395's head 1ef3310, #397's abb27e6, #400's 7062a47, #401's 409f10b). Should one ever conflict, regenerating resolves it.

**Left out** (candidate follow-ups, not filed here).

- `setup.sh --preflight` from a download still clones before checking the machine, though `setup-project.py` answers `--preflight` before its own self-bootstrap. The preflight is Python, so the shim cannot answer it itself; it needs its own decision.
- From a download, `setup.sh --help` still needs Python 3.9 and git first: the interpreter and git checks run before the checkout probe, as they always did.
- Windows does not run the shim. The generator refusal tests do run there: they reach the refusal before any bash. The `tests-windows` job was their first run on that platform.

**Fix round 1 (review341's CHANGES NEEDED 0/1/5).**

One commit (d15a888) on top of 968db85, then a merge of `origin/main` 53bee5b (e2df9a8), no force:
- **S1 (should).** The #50 guard excused the spelling `--help` everywhere in the shim, so a second reader in any form but an unquoted label passed it. It now excuses the spelling in `shim_help()`'s `say "..."` lines and the one `-h|--help)` label, and nowhere else (decision 5). Mutation proof in a throwaway worktree: the guard passes on the unmutated shim and fails on each of review341's three variants (a quoted `"--help")` label, a `[[ "${1:-}" == --help ]]` reader in the developer path, a `[[ "$arg" == --help ]]` reader inside the scan) and on four more (a reader as its own line in `shim_help()`, one chained after a `say`, one substituted into a `say`, a bare `-h` reader). The first four and the bare `-h` reader fail only the guard, and the help tests around it stay green; the chained and substituted `say` variants also break the help's own output.
- **N1.** The inside-a-checkout test compares stderr as well as stdout. With the developer-path reader above, it now fails and `test_help_exits_zero` still passes.
- **N2.** The generator reads git's top level with `os.fsdecode(...).rstrip("\r\n")`, so a checkout at a path ending in a space, or not UTF-8, is accepted again, as the shim accepts it. A test for each; on 968db85's generator both fail with exit 2 "not a checkout".
- **N3.** README and the handbook name `python3 setup-project.py --help` as the command that lists every flag; `openRepoShape --help` names eight of fourteen. The handbook's copy, its footer clause and its README blob follow. README stays 1422 lines, so its cap does not move.
- **N4.** "About seventy lines of code" is dropped from README, the handbook and `setup-project.py`'s docstring: the shim is 122 lines of code, 93 before this change, and a count in prose goes stale again.
- **N5.** The comment above section 0's scan says the third read decides whether there is a run at all.
- `docs/cli.md` is unchanged: no `--help` text moved, and `--check` is `[ok]`.

Reviewed adversarially at 968db85, the branch's first commit (CHANGES NEEDED: 0 blockers, 1 should-fix, 5 nits). The shim and the generator behaved as this description says. The should-fix was the #50 guard, which a second `--help` reader slipped past; the nits were a test that compared stdout and not stderr, two real checkouts the generator refused (a path ending in a space, a path that is not UTF-8), and three pieces of stale doc or comment text (the README and handbook sentence that said `openRepoShape --help` lists every flag, the shim's line count, and the shim's own comment). Fix round 1 (d15a888, merged with main 53bee5b at e2df9a8) resolved them, and its re-verification of e2df9a8 read APPROVE WITH NITS (0 blockers, 0 should-fixes, 1 nit: § What setup.sh does does not say that the shim answers `--help` itself outside a checkout, left as it was because README says it a few lines above). Main 75052cb was then merged in at 4c28890: the one conflict, `docs/handbook.html`'s footer, keeps both "added" clauses (#383's, then #341's) and names the merged README's blob (5465d13), and the merged `docs/cli.md` matches its generator. CI is green on Linux, macOS and Windows, with SonarCloud passing, at 4c28890 (main 75052cb merged in).

Closes #341

Lane: openreposhape-5 (openRepoShape-5)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment