Repository navigation
adopt-project.py: _default_branch keeps a branch name's byte that is not UTF-8, so a source whose HEAD or origin/HEAD is on caf<e9> is refused or adopted past it, and no longer ends plan, check and execute in a traceback (#425) - #717
Merged
Merged
Conversation
…s not UTF-8 as a lone surrogate, so a source whose HEAD or `origin/HEAD` is on `caf<e9>` is refused as `unsafe-value` for `--tracking-branch`, or adopted past it, and no longer ends `plan`, `check` and `execute` in a UnicodeDecodeError traceback (#425) Lane: openreposhape-1 (openRepoShape-1) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Reviewer's GuideThe PR changes Sequence diagram for preserving non-UTF-8 default branch namessequenceDiagram
participant Git
participant DefaultBranch as _default_branch
participant SourceOpen as Source.open
participant Plan
participant CheckExecute as check_or_execute
DefaultBranch->>Git: symbolic-ref --short refs/remotes/origin/HEAD
Git-->>DefaultBranch: branch bytes
DefaultBranch->>DefaultBranch: decode utf-8 with surrogateescape
DefaultBranch-->>SourceOpen: caf\udce9
SourceOpen->>Git: rev-parse caf\udce9
Git-->>SourceOpen: branch commit
SourceOpen-->>Plan: default_branch and commit
Plan->>CheckExecute: plan result
CheckExecute->>Git: resolve tracked branch
Git-->>CheckExecute: branch commit
Flow diagram for non-UTF-8 default branch handlingflowchart TD
A[Read origin/HEAD or HEAD] --> B[Read Git output as bytes]
B --> C[Decode with surrogateescape]
C --> D{Tracking branch supplied?}
D -->|No| E[Existing unsafe-value validation refuses branch]
D -->|Yes| F[Resolve supplied text branch]
F --> G[Record default branch and commit in plan]
G --> H[check validates plan]
H --> I[execute adopts source without traceback]
E --> J[No plan or adoption side effects]
File-Level Changes
Assessment against linked issues
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
brettheap
marked this pull request as ready for review
October 16, 2026 17:13
|
Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days. You can request another review in 3 hours by commenting |
This was referenced Oct 16, 2026
|
brettheap
pushed a commit
that referenced
this pull request
Oct 17, 2026
main gained 25 pull requests since de92614: #634, #699, #671, #719, #701, #717, #700, #702, #722, #720, #725, #721, #728, #705, #727, #718, #744, #743, #749, #740, #737, #734, #742, #751 and #736. No conflict: AGENTS.md, adopt-project.py and docs/cli.md merged as text, and README.md and docs/handbook.html are unchanged by the merge. README.md is 1473 lines, its cap; AGENTS.md 494, its cap; every handbook marker is current; `render-cli-reference.py --check` is ok. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap
pushed a commit
that referenced
this pull request
Oct 17, 2026
…es every reason THIS checkout cannot answer, and the notes are current to main bf1f099 with the 25 pull requests since de92614 (#563, #683) - W683 and RV-f's N4 on #742: README's exit-3 list (Keeping a project's shape current, :1208-1220) and its handbook twin (`doctor`) named only a pin whose commit this checkout does not carry. They now say, as shape-doctor.py's EXIT CODES does: a pin's commit it lacks, a partial clone missing what is compared (#683), a read that fails (`update-upstream-unreadable`), a crashed validator of its own (#278, #410), or its own naming policy not loading (#678). +1 line, with tests/readme-cap.d/683.txt; README.md is 1474, its cap (1422 + 2 + 5 + 12 + 22 + 10 + 1). The two `Keeping a project's shape current` markers move: `doctor` text and marker, `current` marker only. - The notes, "What changed since v1.0.1", one entry per pull request from `git log --first-parent de92614..bf1f099`, each from its squash commit and the issues GitHub says it closes: - adopt-project.py: #699, #719, #749, #727, #717, #728 (with #621's hint), #743 + #720 + #718, #722, #721; - update-shape.py, bump-leg.py and a project's own scripts: #702 (with the pinned validators), #634; - scaffold, setup, family and the shared readers: #725, #671, #744; - shape-doctor.py: #742, #701, #737, #751, #736, #734, #700 + #705, #740; - install, CI and the docs: the exit-3 sentence above. - Counts (99 pull requests since bfed143; 109 of 135 milestone issues closed at 10:28Z), the "current to main" sha and the open list (26) are at main bf1f099, still placeholders for tag time. Left for round B: lane 5's eight lines (#630 issuecomment-6227129879), whose pull requests #726, #747, #682, #724, #746, #745 and #738 are open drafts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap
added a commit
that referenced
this pull request
Oct 17, 2026
…another spelling of its row's path moves it back to that path in the index alone, so the printed remedy no longer takes the member's gitlink away from its row (#646) (#670) **Goal (#646): no remedy `family.py` prints un-registers a member that is in the holder.** Three states are outside it, each under "Left out": a `.gitmodules` that a `git mv` made name the variant too (checklist candidate (e)); a file the index holds at or under the row's path, where #574's remedy is kept; and a file or link the index holds at `members` itself (candidate (h)). A member's only gitlink sat at `Members/<P>`, while its row and `.gitmodules` said `members/<P>`. This is #363's state after the fact. `family.py add` of ANOTHER member refused, as #574 rules (`family-members-dir-variant`). Its remedy was #574's remedy for any path under another spelling of `members/`: - `git rm -r --cached -- Members/<P>`, and commit the removal, where nothing is on disk in that spelling; - `git mv -- Members/<P> <new name>`, and commit the rename, where something is; - `git commit`, where HEAD alone holds it (the variant's removal staged, not committed). Each one, typed as printed, left no gitlink at the row's path. The holder's validator then refused `member-gitlink-absent`. Reproduced on BASE a524542 in all five layouts below. All in `scripts/family.py`. - `_members_spellings` takes the rows' own paths (`_rows_paths`: the one path `_member_path` gives a row, `members/<project>`; a row it refuses is `member-path`'s). - **Which path is the member's own.** The index must hold nothing at or under the row's path. Then a gitlink at a case variant of that path, in the index or in HEAD alone, is that member's. It gets a new remedy, `RESTORE`, placed in `REMEDY_ORDER` after the index's own remedies. - **The remedy** is the index-only move that `validate-family.py` prints for the same state since #639. Brett Heap accepted it on 2026-10-16T11:21:49Z ("Accept; merge #639 after #637"), in place of the "committed `git mv`" of his "Refuse on every disk" ruling, because `git mv` to the row's path fails or misplaces the gitlink: - `git rm -r --cached --ignore-unmatch -- <variants>`. It touches no file, and it is no error where HEAD alone holds the variant. Where a sparse checkout keeps a variant off disk (`skip-worktree`), the line says `git rm -r --cached --sparse --ignore-unmatch -- …`. - `git update-index --add --cacheinfo 160000,<commit>,members/<P>`, one line per row, at the commit the variant entry records. For an unmerged variant, that is the commit at stage 2. - A commit, where one is left to make. - **What the disk holds afterwards** gets the validator's own sentences, placed after every other remedy: - where nothing is checked out at the row's path, as in a fresh clone, `git submodule update --init --recursive`; - on a case-sensitive disk, `mv -- Members/<P> ../Members-<P>` for the directory left at the variant; - on macOS and Windows, leave it, since it is the row's own directory. - A directory on disk at a gitlink moved back from HEAD is the move's to name. It is not named a second time as untracked. The branch merges main 35133f2, which holds #639 (c6122e8), in one merge commit with no conflict. Example: the issue's state, the checkout at `members/IRRS`: ``` REFUSED family-members-dir-variant: the family root holds a path at or under another spelling of members/, which a case-insensitive disk takes for members/ itself — in the index: `Members/IRRS` (a submodule) Remediation: every member is mounted under members/, and a holder keeps that directory in one spelling on every disk. Move each member's gitlink held only under another spelling of its row's path, `Members/IRRS` for the row members/IRRS, to that path, at the commit it records, in the index alone, which touches no file (`git rm -r --cached --ignore-unmatch -- Members/IRRS`, then `git update-index --add --cacheinfo 160000,<commit>,members/IRRS`), and commit that; where nothing is checked out at members/IRRS, as in a fresh clone, `git submodule update --init --recursive` then checks the member out there; on a case-sensitive disk, as Linux's usually is, a directory left at Members/IRRS stays there, untracked, and add refuses it as family-members-dir-variant: move each one that is there out of the holder, `mv -- Members/IRRS ../Members-IRRS`; on macOS and Windows it is the row's own directory, so leave it; then re-run. ``` No pinned file changes: `scripts/family.py` is the standard's own tool, not a copy in a holder. No `--help` text changes, so `docs/cli.md` is untouched. A `git grep` of AGENTS.md, README.md, `docs/handbook.html` and the family-root templates found no sentence about `add`'s case-variant remedies, so none became false. 1. **The member's own gitlink: one whose path folds to a row's path, where the index holds nothing at or under that path.** The next commit records the index, and every remedy here ends in one. - If the index holds the row's gitlink, a gitlink at a variant is a second one. Taking it out removes nothing from the row, so it keeps #574's remedy, and moving it back would overwrite the row's gitlink. - If the index holds a FILE at the row's path, the move would replace it without a word. If it holds files UNDER the row's path, the move's second line fails (`appears as both a file and as a directory`) after the first has already taken the variant out. Both keep #574's remedy (lane 3's review RV670, N3). - A file at the variant, or anything under it, also keeps #574's remedy. - `test_only_the_members_own_gitlink_is_moved_back` holds all of these, 9 cases. 2. **The commit named is the one the variant entry records, so the remedy is a move, not a bump.** Brett Heap decided this at 2026-10-16T14:11:42Z: "take all recommendations for #670 and #681". That word replaces his 13:07:34Z take of "the row's pin". - Where a row has two variants, the first found is named, the index's before HEAD's. - **This departs from #646's Direction line (`<pin>`) and from #639's rule:** #639's restore in `validate-family.py` names the row's pin. The two tools give different lines where the variant records a commit the row does not pin, and, since S1, where a sparse checkout keeps the variant off disk: `add` prints `--sparse` and the validator does not (candidate (f)). - **The four reasons, as lane 3's review RV670 measured them:** 1. **A pure rename.** With the variant's commit, the move changes only the path. Where the commits differ, the gate then reports `member-gitlink-mismatch`, which has its own LOCKSTEP text and `bump` remedy, and exits 1, and an `add` of another member still goes through. `test_a_commit_the_row_does_not_pin_is_moved_not_bumped` types that end to end. (Where the variant equals the pin, the bytes are the same either way.) 2. **The pin would silently bump.** With the pin, the paste changes the commit the holder records: a roll-back or an advance, a bump done outside `bump`, which the LOCKSTEP text says the gate refuses. 3. **A row with no pin needs a fallback.** The validator never prints its remedy for such a row, because `member-pin` or `member-pin-commit` stops the row first. So #639's rule has no answer there, and a pin-based `add` would need the variant's commit anyway. 4. **A pin absent from the clone fails.** `update-index --cacheinfo` does not check that the object exists, and the printed `git submodule update --init --recursive` then fails with rc=128 (`Fetched in submodule path 'members/IRRS', but it did not contain 3333…`). The variant's commit is already recorded, so this remedy adds no new unreachable commit. - **The validator is not changed.** Naming the variant's commit there too (it already prints it, as `(at <commit>)`) would edit a pinned template, and that needs its own word, which nobody has given. 3. **An unmerged variant is moved back at its stage-2 commit** (RV670, N2). `ls-files -s` lists stage 1, the merge base, first, and the PR's first head took that one. - Stage 2 is HEAD's entry each time, so the move is a rename relative to HEAD, and the validator also answers an unmerged path from HEAD. In a merge and in a stash pop it is the holder's own side. In a rebase it is the upstream's side, which is HEAD while the rebase runs: after `git rebase --quit` the move names the upstream's commit, not the person's own, which is stage 3 (probed by rv670b, the re-verify of this PR). - The removal takes out every stage, and the commit after it goes through (`$Q/v112/w646b/probes/w646b-unmerged-probe.log`). - An unmerged variant with no gitlink at stage 2 is no member's own: HEAD's entry holds no gitlink there, whatever stages 1 and 3 hold. It keeps #574's remedy, where stage 2 is missing and where it is a file. A gitlink at stage 1, a file at stage 2 and a gitlink at stage 3 is an `OWN` state too (rv670b's NIT-3): git lists stage 1 first, and rv670b's mutant R1, which read the first record's mode in place of stage 2's, passed all 36 tests and moved the file's blob back as a gitlink. - The state is reachable only with unmerged entries and no in-progress mark. A conflicted `git stash pop` leaves it, and so do `git merge --quit` and `git rebase --quit` (rv670b probed both). `add` refuses a merge, rebase, cherry-pick, revert or `git am` in progress (`family-operation-in-progress`) before it gets here. 4. **A variant a sparse checkout keeps off disk is taken out with `--sparse`** (RV670, S1). Without it, `git rm -r --cached --ignore-unmatch` exits 0 and removes nothing, so the index then held two gitlinks, and the next `add` refused again. With it, the entry goes, measured on git 2.43 with and without `core.sparseCheckout` (`$Q/v112/w646b/probes/w646b-sparse-probe-*.log`). The flag is printed only where a variant needs it, so the line stays the validator's everywhere else. Git older than 2.34 has no `git rm --sparse`: see "Left out". 5. **The commit is printed only where one is left to make.** That is: - HEAD holds a variant (RV670's S2 state, HEAD holding the gitlink in both spellings at one commit, is one of these); - HEAD's gitlink at the row's path is another commit, or there is none; - or a pending change is committed first. A move to the variant that is only staged, with HEAD still holding the row's gitlink at that commit, leaves nothing to commit, and a printed `git commit` would fail as typed. 6. **The code stays `family-members-dir-variant`** (or `member-path-taken` where some path is also a variant of the new member's mount path). The add is still refused for this path, as for any path under another spelling of `members/`: with `core.ignorecase` set, git would stage the new gitlink as `Members/<new>` (#362). Only the remedy changes. 7. **The closing sentences come last**, after #574's disk remedies. - Where a `members/` renamed by hand is moved back (`mv -- Members members`), the checkout goes home with it. - If they came before that, the `mv` out of the holder would act on `Members/<P>` while it may still be the member's checkout. And `git submodule update` would make a `members/`, inside which `mv -- Members members` would then put `Members` (`members/Members`). - `test_with_the_other_remedies…[members/ renamed by hand]` holds the order: it types the move back and then `git submodule update`, which checks the member out in the `members/` moved back. 8. **One text on every disk.** The move reads the index and HEAD, never the disk. The dry run and the real run print the same two lines. Asked in process of an emulated folding disk and of a Linux one (`disk_as`, as `tests/test_family_members_case_variants.py` emulates it), each layout gives the same text the subprocess gives. 9. **The disk after the move.** - On a case-sensitive disk the variant keeps either a checkout (15 entries, `?? Members/`) or, after a fresh clone, an empty directory. The gate passes either way. - If the printed `mv` is not typed, the next `add` refuses `family-members-dir-variant ... on disk and untracked: Members/Atlas` with #574's `mv -- Members/Atlas <new name>`, and passes once that is typed. - If the printed `mv` is typed, the next `add` passes at once (`$Q/v110/w646/leftover.log`, `repro-head.log`). All runs used `$Q/v110/run9.sh`: a `gh` stub that exits 97 first on PATH, `GH_TOKEN=bogus-l5`, an empty `GH_CONFIG_DIR`, `/usr/bin/python3 -m pytest`, and local bare remotes only. `tests/test_family_add_member_variant.py` is new, with 37 tests. Every printed command is typed as printed. The holder's gate then passes (or, in the one case where the variant records a commit the row does not pin, names that drift), the member is where its row says at its commit, and the same `add` goes through. - `test_each_layout_moves_the_gitlink_back_as_printed` (4 layouts) and `test_a_fresh_clone_moves_the_gitlink_back_as_printed`. - `test_every_disk_gives_the_same_answer` (4 layouts × folding and Linux, in process). - `test_with_the_other_remedies_each_runs_in_its_order` (8 states): - beside a file under `Members/`; - after a pending change in HEAD; - with nothing left to commit; - `members/` renamed by hand; - HEAD alone, with the checkout at the variant; - HEAD holding both spellings at one commit, the row's removal staged (S2); - kept off disk by a sparse checkout (S1; skipped on git older than 2.34, as #574's sparse case is); - unmerged at stages 1, 2 and 3, with nothing in progress (N2). - `test_every_member_held_in_the_wrong_case_is_moved_back` (two rows). - `test_the_gitlink_is_recorded_at_the_commit_it_records` (4 states). - `test_a_commit_the_row_does_not_pin_is_moved_not_bumped` (S3): typed as printed, the gate exits 1 with `member-gitlink-mismatch` alone, and the same `add` goes through. - `test_only_the_members_own_gitlink_is_moved_back` (9 states, four of them new: a file at the row's path, a file under it, an unmerged variant with no gitlink at stage 2, and one with a gitlink at stage 1, a file at stage 2 and a gitlink at stage 3). - `test_the_rows_that_have_a_path_of_their_own`. Layouts that need two directories differing only in case skip on a disk that folds case. No test makes such directories there. Two existing lines change to the new shapes: - `Spelled` has two more fields, `recorded` and `sparse`, so a seven-field tuple compared in `tests/test_family_members_case_variants.py` gains `"", False`; - the three-argument stand-in for `_members_spellings` in `tests/test_family_add_refusals.py` takes `rows`. The S1 case sits in the new file, beside its other move-back states, and not in `test_family_members_case_variants.py` beside #574's sparse test: it needs the new file's helpers, and that file already imports from the other one. Results on this head. a837a02 is 2104a3d plus one test commit: it adds one state to `OWN` and changes no code, so the runs of 2104a3d below stand for every file it does not touch. | Run | Head | Result | |---|---|---| | the new file | a837a02 | `37 passed in 48.55s` | | `tests/test_family_members_case_variants.py` | a837a02 | `60 passed in 53.83s` | | the new file, with `scripts/family.py` put back to the merge's (bc48c2a) | 2104a3d's tests | `5 failed, 31 passed in 170.09s`: the sparse and unmerged states typed, and the OWN cases of a file at the row's path, a file under it, and an unmerged variant with no gitlink at stage 2 | | the new file, `test_family_members_case_variants.py`, `test_family_add_refusals.py` | 2104a3d | `161 passed in 809.30s` | | `test_family.py`, `test_family_member_path.py`, `test_validate_family_unnamed_mounts.py`, `test_readme_windows_quoting_sentence.py`, `test_repo_hygiene.py` | 2104a3d | `269 passed in 803.63s` | The S2 and S3 cases pass on the merge too, since they pin behaviour the PR already had. Lane 3's mutant X2 fails the S2 case (below). The S3 case holds the variant's-commit rule end to end. Earlier, at the PR's first heads (before the merge with main): | Run | Head | Result | |---|---|---| | the new file, on BASE a524542 | a524542 | `29 failed in 48.75s` | | every `tests/test_family*.py`, 21 files | 2216e1c | `851 passed, 19 skipped in 1248.90s` | | `tests/test_validate_family*.py` (5), `test_refusal_default_remediation.py`, `test_repo_hygiene.py`, `test_siblings_family_name.py`, `test_template_text_placeholders.py`, `test_clone_url_bytes.py`, `test_update_shape_header_comments.py`, `test_update_shape_pr_base.py`, `test_shape_doctor.py` | 2216e1c | `1014 passed, 3 skipped in 1323.30s` | Holder `Fam` with members Atlas and Nova, Atlas's gitlink moved to `Members/Atlas` in the index at its own commit and committed; then `family.py add` of Zed (`$Q/v110/w646/probes/repro.sh`), at the PR's first head. | Layout | BASE a524542 (`repro-base.log`) | head 79754b9 (`repro-head.log`) | |---|---|---| | L1, nothing on disk in that spelling | `git rm -r --cached -- Members/Atlas`, commit: no gitlink for Atlas; gate `member-gitlink-absent` | rm, `update-index`, commit: Atlas at `members/Atlas` at its commit; status clean; gate exit 0 in all three modes; next `add --dry-run` exit 0 | | L2, the only checkout at `Members/Atlas` | `git mv -- Members/Atlas renamed-1`, commit: gitlink at `renamed-1`; gate `member-gitlink-absent` | rm, `update-index`, commit, `git submodule update --init --recursive`, `mv` out: the same end state | | L3, both directories | `git mv`, commit: gitlink at `renamed-1`; `?? members/Atlas/`; gate `member-gitlink-absent` | rm, `update-index`, commit, `mv` out: the same end state | | L4, a fresh recursive clone (an empty `Members/Atlas`) | `git mv`, commit: gitlink at `renamed-1`; gate `member-gitlink-absent` | rm, `update-index`, commit, `git submodule update`, `mv` out: the same end state | | L5, the variant's removal staged, not committed | `git commit`: the removal committed; gate `member-gitlink-absent` | rm (`--ignore-unmatch`, no error), `update-index`, commit: the same end state | In every layout, on both sides, the dry run and the real run printed the same stderr, and the in-process run under a folding disk and under a Linux disk printed the same two lines as the subprocess. This round's states (S1, S2, S3, N2, N3) are reproduced by the tests above, which type each printed line. Not reproduced: a real macOS or Windows disk. The folding disk is emulated. CI runs Linux only (#597). **This round** (`$Q/v112/w646b/probes/w646b-mutate.py`, in throwaway clones at 2104a3d). Each probe changed one hunk or condition, and the new file was then run with `-x`. Every hunk of this round fails a test, and so do lane 3's X1 and X2 and rv670b's R1, which needed a new state (last row). | Probe | What it undoes | Result | First failing test | |---|---|---|---| | M1 | a variant never marked `sparse` | 1 failed, 19 passed | `test_with_the_other_remedies…[kept off disk by a sparse checkout]` | | M2 | `--sparse` never printed | 1 failed, 19 passed | the same | | M3 | back to "no GITLINK at the row's path" (the first heads' rule) | 1 failed, 31 passed | `test_only_the_members_own_gitlink…[a file at the row's path]` | | M4 = lane 3's X1 | "no entry AT the row's path", under it not asked | 1 failed, 32 passed | `…[a file under the row's path]` | | M5 | the first record git lists (stage 1), as the first heads had it | 1 failed, 20 passed | `test_with_the_other_remedies…[unmerged, with nothing in progress]` | | M6 | stage 0 alone: an unmerged variant is never the member's own | 1 failed, 20 passed | the same | | M7 | the last record (stage 3) | 1 failed, 20 passed | the same | | lane 3's X2 | `name in in_head or` deleted | 1 failed, 18 passed | `test_with_the_other_remedies…[HEAD holding both, the row's removal staged]` | | lane 3's X3 | `'posix'` dropped from the move back's three quotings | 36 passed (survives) | none on Linux: `shell_quoted` quotes for `os.name`, POSIX here (see "PowerShell" under "Left out") | | rv670b's R1 | `ours[0] == b"160000"` replaced by the mode of the first record git lists (`mode == 0o160000`), in place of stage 2's | 36 passed (survived) at 2104a3d. At a837a02: `1 failed, 34 passed in 44.07s` (`-x`), and `1 failed, 8 passed, 28 deselected` for `-k test_only_the_members_own_gitlink` | `test_only_the_members_own_gitlink…[unmerged, with a file at stage 2 and gitlinks at 1 and 3]` | The S3 test types the remedy end to end and asserts the gate's finding, so it holds the variant's-commit rule itself; no mutant of this round touches that rule. **The first heads** (`$Q/v110/w646/probes/mutate.py`, in a throwaway clone at 2216e1c). The new file was run with `-x`. | Probe | What it undoes | Result at 2216e1c | First failing test | |---|---|---|---| | H1 | `RESTORE` in `REMEDY_ORDER` | 1 failed | `test_each_layout…[nothing on disk in that spelling]` | | H2 | no member's own gitlink found | 1 failed | the same | | H3 | a gitlink at the row's path not asked | 1 failed, 22 passed | `test_only_the_members_own_gitlink…[a second gitlink beside the row's]` | | H4 | any index entry, not a gitlink alone | 1 failed, 23 passed | `…[a file at the variant]` | | H5 | the index's branch | 1 failed | `test_each_layout…[nothing on disk in that spelling]` | | H6 | the commit always printed | 1 failed, 15 passed | `test_with_the_other_remedies…[with nothing left to commit]` | | H7 | the commit printed only where HEAD holds the variant | 28 passed (survived) | killed at 79754b9: `test_the_gitlink_is_recorded…[staged at another commit, HEAD keeping the row's]` | | H8 | the commit read from HEAD's gitlink at the row's path first | 28 passed (survived) | killed at 79754b9, by the same case | | H9 | HEAD's branch | 1 failed, 3 passed | `test_each_layout…[the removal staged, not committed]` | | H10 | HEAD's file taken for a gitlink | 1 failed, 25 passed | `test_only_the_members_own_gitlink…[a file at the variant in HEAD alone]` | | H11 | the disk named again at a gitlink moved back from HEAD | 1 failed, 17 passed | `test_with_the_other_remedies…[HEAD alone, the checkout at the variant]` | | H12 | the commit not printed after a pending change | 1 failed, 14 passed | `…[after a pending change in HEAD]` | | H13 | no `--ignore-unmatch` | 1 failed | `test_each_layout…[nothing on disk in that spelling]` | | H14 | the last variant found names the commit | 1 failed, 20 passed | `test_the_gitlink_is_recorded…[two variants in the index]` | | H15 | the closing sentences before the disk's remedies | 1 failed, 16 passed | `test_with_the_other_remedies…[members/ renamed by hand]` | | H16 | the closing sentences dropped | 1 failed | `test_each_layout…[nothing on disk in that spelling]` | | H17 | `add` passes no rows | 1 failed | the same | | H18 | `_checked_mount_path` drops the rows | 1 failed | the same | | H19 | `_rows_paths` takes any row's path | 1 failed, 27 passed | `test_the_rows_that_have_a_path_of_their_own` | | H20 | `_rows_paths` does not check the project's type | 1 failed, 27 passed | the same | | H21 | the move's part dropped from the text | 1 failed | `test_each_layout…[nothing on disk in that spelling]` | The branch merges main 35133f2 in bc48c2a, with no conflict: its tree, 088aa26, is the one `merge-tree` predicted. Only #637, #622 and #639 had changed `scripts/family.py` since BASE. `git merge-tree --write-tree` of the head a837a02, last after a fetch at about 18:45Z. Every one is clean. - origin/main 2fac979 (tree 042d309). Since the last check (main de92614) it has landed #634 (30676c9), #659/#671 (6e37d0e), #658/#699 (b618ec1), #650/#719 (3dc34b8), #678/#701 (5c392dd) and #425/#717 (2fac979). The other rows are the heads as fetched now. - Lane 4's five branches that change `scripts/family.py`: `fix/684-family-holder-states` 7015386, `fix/686-bump-reachability` 7567e60, `fix/688-family-init-values` aa95b6b, `fix/689-next-push-refspec` fa51384 (it also changes `tests/test_family_add_refusals.py`) and `fix/693-local-remote-dir` 96ebe45. - Lane 4's other open heads: `fix/674-update-shape-apply` d069902, `fix/690-siblings-inputs` 38a40ab, #698 b069951, #700 fca40df, #702 48bac6e, #703 (`fix/696-setup-family-folder`) 4054ffd, #705 1bc63e8 and #706 (`fix/664-surviving-mutants`) 3e408b1. - Lane 2's #671 and #699 and lane 4's #701 are on main now, so the main row covers them. - Lane 5's #682 (#648, 8b39cd1), `fix/692-raw-folder-paths` ab55c29 (it changes `scripts/family.py` too) and `fix/704-bootstrap-leg-moves` 4e2b211. - **A `.gitmodules` that names the variant too** (a `git mv members/<P> Members/<P>`, committed or staged; checklist candidate (e) below). The move puts the gitlink back at the row's path, but the section still gives `Members/<P>`. So `git submodule update` fails with `No url found for submodule path 'members/Atlas'`, and the gate refuses `member-uninitialized` (`$Q/v110/w646/gitmv-head.log`). On BASE the same remedy moved the gitlink and the section to `renamed-1` (`gitmv-base.log`). #639's validator prints the same incomplete lines for this state. A fix needs a `.gitmodules` edit that the validator does not print. - **A file the index holds at or under the row's path, beside the variant's gitlink** (decision 1). #574's remedy is kept: it takes the variant out, and the file stays at the row's path. For a file at the row's path, typed, it leaves HEAD with `100644 blob … members/IRRS`, the gate refuses `member-gitlink-absent` (exit 2), and `add` of another member then goes through (exit 0), as rv670b ran it; a file under the row's path was not run. The move would replace that file, or stop half done, so no remedy here can both keep the file and put the member back. This state predates this PR. - **A file or link the index holds at `members` itself** (a file `members`, or a committed link `members -> Members` left after a hand rename; checklist candidate (h) below). The rule asks about entries at or under a row's path, not above it. With every member at a variant and `members` a file in the index, `add` prints the move back, and rv670b typed it line by line: `git rm` takes out both variants (exit 0), then each `git update-index --add --cacheinfo` fails with exit 128, `'members/IRRS' appears as both a file and as a directory`. The next `add` prints the same move again, now from HEAD. A committed link behaves the same. #574's remedy also un-registered the member here, so it is not a regression. Adding `or row.startswith(name + "/")` to the rule would keep #574's remedy there, which leaves the member with no gitlink, as the entry above does. `scripts/family.py` is not changed for it here. - **Git older than 2.34 and `--sparse`.** `git rm` has no `--sparse` option before 2.34 (git v2.33.0's `builtin/rm.c`), and it skips every `skip-worktree` entry (`if (ce_skip_worktree(ce)) continue;`), so there the line without the flag was the same silent no-op. The printed `--sparse` stops with an unknown-option usage error, exit 129 (from reading that source; not run). #574's `git mv --sparse` has the same limit. The S1 test is skipped on git older than 2.34, as #574's sparse test is. - **PowerShell.** The printed lines are quoted for a POSIX shell (#532), so paste them into Git Bash. RV670 notes (N4) that in PowerShell the bare `160000,<commit>,members/IRRS` becomes an array passed as three arguments, which `--cacheinfo` also accepts. Not run, by RV670 or here. On Linux, dropping `'posix'` from these lines changes nothing (lane 3's mutant X3), since `shell_quoted` quotes for `os.name`, which is POSIX there. - **The closing `mv` inherits #639's N-1 and N-2** (rv611c), and its N-3: - N-1: where the variant directory is a live checkout with an absorbed gitdir, the `mv` leaves its `.git` file pointing nowhere; - N-2: the `mv`'s condition is stated by OS, not by a check a person can run, so on a case-folding Linux mount (WSL's `/mnt/c`) typing it moves the member's own checkout out, and `git submodule update --init --recursive` then puts it back; - N-3: an empty `../<variant>` directory that already exists is moved into. - **A variant under `members/` itself.** `members/irrs` for the row `members/IRRS` is invisible to `add`'s `_spelled_target`, while the validator's casefold check names it (RV670, N6). This predates this PR. - **Case folding beyond ASCII**, as in #574: project names are ASCII, and git's `icase` folds ASCII only. - **(e) `family.py add` and `validate-family.py`: the move back of a member's gitlink leaves `.gitmodules` naming the variant after a `git mv`.** After `git mv members/<P> Members/<P>`, git rewrites the section's `path` to `Members/<P>`. The index-only move both tools print puts the gitlink back at `members/<P>` and leaves the section where it is. `git submodule update --init` then fails with `No url found`, and the conditional `mv` would move the member's only checkout out of the holder. Both could also print `git config -f .gitmodules submodule.<name>.path members/<P>` and `git add .gitmodules` where a section gives a variant's path (evidence `$Q/v110/w646/gitmv-head.log`). - **(f) `validate-family.py`: its move back of a member's gitlink is a silent no-op where a sparse checkout keeps the variant off disk.** Its `git rm -r --cached --ignore-unmatch -- <variant>` has no `--sparse`, so it exits 0 and removes nothing, and its `git update-index --add` then leaves two gitlinks in the index. This is the validator's half of RV670's S1, which RV670 assigns to #459. The validator is a pinned template. - **(g) `validate-family.py` and `family.py add`: a file the index holds at or under a row's path, beside the member's gitlink at a variant, has no remedy that keeps the member.** `add` now keeps #574's remedy there, which takes the variant out. The validator asks `recorded_gitlink`, which skips any entry that is not mode 160000, so by reading its code (not run) it prints its move back for this state too: `update-index --add` would replace a file at the row's path without a word, and fail on files under it after its first line has taken the variant out. This state predates #670. - **(h) `family.py add` (and probably the validator, not run): a file or link the index holds at `members` itself is never named.** The move printed beside it stops at its `update-index` line after `git rm` has taken every variant out (above, under "Left out"). Nothing here fixes it; the coordinator places it. Review: lane 3's RV670 found 79754b9 READY with three SHOULDs; the fix round took S1-S3, N2 and N3, rv670b's re-review found it ready with nits, and a837a02 takes Brett Heap's ruling of 2026-10-16T14:11:42Z that the restore keeps the variant's own commit. CI (Brett Heap's ruling 27 and his fast-refresh ruling of 2026-10-15T22:48:35Z): H0 a837a02 passed tests, tests-git-bash and SonarCloud; the head 6d7e4ff is a clean update-branch merge of main bf1f099, with tests-git-bash and SonarCloud green; and the local gate passed on its predicted squash tree b46e560 over main 6794a9f: the family_add_member_variant, family_add_refusals, family_members_case_variants, repo_hygiene, agents_md_codes, cli_reference sets, 308 passed. Landed on Brett Heap's word of 2026-10-17T10:59:39Z, relayed by lane 3 on #473: "merge lane 5's approved v1.1.0 drafts". Closes #646 Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap
added a commit
that referenced
this pull request
Oct 17, 2026
… verification row's colon, `run_validator`'s "costs no child an import" and the decoy test's `sitecustomize` claim say what main does (#666) (#730) Four comments and docstrings said something main no longer does. They now say what the code beneath them does. A fix round after review added two more of the same kind (see "Fix round"). Comments and docstrings only: no behaviour, no `--help` text, no pinned copy, so no README, handbook or `docs/cli.md` change. - **`adopt-project.py`, the module docstring.** It said the verification table reads `code: 0 of N source paths (seeded from template)`. The printed row has no colon (` {leg:<6} {count:>5} of {N} source paths{note}`). It now says the table's `code` row reads `0 of N source paths (seeded from template)`, the words `AGENTS.md` and `README.md` use since #505 and the docstring of the verification function uses too. `git grep 'code: 0 of'` finds nothing now. - **`shape-doctor.py`, `submodule_config`.** The last sentence of the docstring gave a reason that is true of the string and not decisive: "It is also what the row PRINTS, and a lone surrogate is a string no UTF-8 stream can write." The table goes through `ascii_text` and `--json` through `json.dumps`, so neither would raise on a lone surrogate. I dropped the sentence, the issue's first direction. The comparison with `origin_url`'s reading of a clone's remote is the reason, and the paragraph already gives it first. **This hunk carries the bytes of lane 3's #700**, which landed as `55c6ba7d`: the two removed lines and the one added line (` working clone.`) are the same. So the hunk is on main already and drops out of the squash (see "Evidence"). - **`shape-doctor.py`, `run_validator`.** "It costs no child an import" holds for an unedited copy only. A new paragraph says a root whose own copy of a validator gained an `import yaml` read `passes` while `site` was on, and now reads `crashed before reaching a verdict (exit 1): ModuleNotFoundError: No module named 'yaml'`, the crash its own CI gets, because `validate.yml` installs nothing. It calls that row a fair answer and says `-S` does not ask whether a copy was edited. No code change. - **`tests/test_shape_doctor.py`, the decoy test's two comments** (the block above `DECOY_REPO_SHAPE` and the inline comment before `reached`). `run_validator` starts every child as `[sys.executable, "-S", script, ...]`, so no child imports a `sitecustomize`. The comments now say that, say that replace-not-prepend and the #306 review S1 case are held by `test_run_validator_gives_the_child_exactly_the_pythonpath_it_is_given`, and say that the decoy test holds the template row's real verdict under an exported `PYTHONPATH`. The review of the first two commits approved with nits. Two new commits take S1 and N1: `7f2bb36b` makes both edits, and `426877bf` rewrites the last sentence of the N1 comment, which did not parse ("...a future removal of `-S`, which `test_run_validator_starts_every_child_with_no_site` ... is the test that holds"). It now ends "...a future removal of `-S`; `test_run_validator_starts_every_child_with_no_site` (in `tests/test_doctor_validator_site.py`) is the test that holds `-S`." Both are comments and docstrings only. - **S1, a third stale sentence.** The docstring of `test_run_validator_gives_the_child_exactly_the_pythonpath_it_is_given` said that for the callers which pass no `pythonpath`, "only a probe like this one, or (b)'s decoy, can see that they have none". That is false under `-S`: a `run_validator` that keeps the variable for those callers passes the decoy test in all three cases and fails the probe test twice (the second row of the table below). It now says only a probe like this one can see that they have none, and that the decoy of (b) could before `-S` (#342). It is the same defect as the issue's, in a place the issue did not name. "Could" holds for the `manifest` and `pins` cases: with `-S` removed, 2 of the 3 decoy cases fail, and the `family` case never caught it. - **N1, the inline comment overstated the loop.** The comment above `reached` said "What this loop still holds is that no row quotes the decoy or says it exited 86." Under `-S` the loop cannot fire. It now says so, says the loop is kept as a tripwire for a future removal of `-S`, and names `test_run_validator_starts_every_child_with_no_site` in `tests/test_doctor_validator_site.py` as the test that holds `-S`. That test is on main. - **N2, this body.** The merge checks and the notes on #630 were stale (they cited `de926142` and a conflict that is gone). They are rerun and rewritten below. **Only comments and docstrings changed.** `git diff 8062810 426877b` has 74 changed lines (44 added, 30 removed) in three files. A script checked each changed line against the file's comment tokens and docstring spans (both sides of the diff): 0 outside. A second check parsed each file at the base and at the head, blanked every docstring, and compared the ASTs: identical in all three files. For the two fix-round commits together, the AST of `tests/test_shape_doctor.py` with docstrings blanked is identical to the one at `a2113cff`, `git diff --check` is clean, and the longest added line is 75 columns (73 in the second commit). **What the squash holds.** `git merge-tree --write-tree` of this head with origin/main `20c8873b` gives the tree `6ead2ffc`. `git diff 20c8873 6ead2ffc` is 71 changed lines in three files, and `shape-doctor.py` has 8 of them: the new `run_validator` paragraph and nothing else. `git diff` of the two trees has no hunk that mentions `submodule_config`, `surrogate`, `PRINTS` or `working clone`. On main, `submodule_config`'s docstring has no "PRINTS" sentence and ends in `working clone.`. The same two checks on that tree: 0 lines outside comments and docstrings, ASTs identical. On main `2d988200` with the head `426877bf` the squash is the same: 71 changed lines, 8 of them in `shape-doctor.py`, 0 outside comments and docstrings, ASTs identical, and no hunk that mentions `submodule_config`. **`python3 scripts/render-cli-reference.py --check`**, at `426877bf`: `[ok] docs/cli.md is what the generator produces`. Only the first line of a module docstring is help text, and none of the first lines changed. **The decoy comments, measured.** I did not take the issue's reading on trust. In a throwaway clone at the base I put each #306 regression into `run_validator` and ran the two tests (`test_an_exported_pythonpath_does_not_reach_a_template_validator`, three cases, and `test_run_validator_gives_the_child_exactly_the_pythonpath_it_is_given`, six): | mutant | with `-S` (main) | with `-S` removed | |---|---|---| | the standard's `scripts/` put in FRONT of the operator's value | decoy test: 3 of 3 pass. Probe test: 1 of 6 fails | decoy test: 3 of 3 fail | | the variable kept for the callers that pass none (S1) | decoy test: 3 of 3 pass. Probe test: 2 of 6 fail | decoy test: 2 of 3 fail | | the operator's directory AHEAD of the standard's | decoy test: 3 of 3 fail. Probe test: 1 of 6 fails | not run | So on main the decoy test catches neither #306 regression and the probe test catches both, as lane 4's R342 measured on #636's head. The third row is why the comment says the raising `repo_shape.py` "still bites": it kills an operator-first order. The review re-ran these mutants on `a2113cff` and got the same counts. **The `run_validator` paragraph, measured.** A root whose own `validate-manifest.py` is `import yaml` plus a print, run through `run_validator` from this tree: exit 1 and `ModuleNotFoundError: No module named 'yaml'`. The same call with `-S` taken out of the child's argv: exit 0, `manifest ok`. `yaml` is importable from the machine's site-packages and not without `site`. With the real `gh` unreachable (a stub first on PATH, a bogus token, an empty `GH_CONFIG_DIR`). On `a2113cff`, before the fix round: - `tests/test_cli_reference.py`, `tests/test_repo_hygiene.py`, `tests/test_agents_md_codes.py`, `tests/test_scaffold_leg_path_collisions.py`: 250 passed (57.5 s). - `tests/test_doctor_gitmodules_not_utf8.py`, `tests/test_doctor_validator_rows.py`, `tests/test_doctor_validator_site.py`, `tests/test_doctor_validator_verdicts.py`, `tests/test_adopt_spec_only.py`, `tests/test_adopt_entry_paths.py`, `tests/test_adopt_submodules.py` (the files that drive `run_validator`, `submodule_config` or the seeded-leg row): 334 passed (28 min 18 s). - `tests/test_shape_doctor.py`, in two halves run side by side (the first 204 test ids, then the last 204): 204 passed in 11 min 31 s, and 201 passed, 3 skipped in 4 min 58 s. That is all 408 tests. The 3 skips are the PowerShell-line tests, which need `powershell.exe` on a Windows host. - The review ran all of `tests/test_shape_doctor.py` there: 405 passed, 3 skipped (512 s). On the predicted merge with main `2fac9790` it ran `test_shape_doctor.py`, `test_doctor_validator_site.py`, `test_doctor_validator_rows.py`, `test_doctor_validator_verdicts.py`, `test_doctor_gitmodules_not_utf8.py`, `test_repo_hygiene.py`, `test_cli_reference.py` and `test_agents_md_codes.py`: 632 passed, 3 skipped (760 s). On `426877bf`, after the fix round, which changes comments and docstrings only, so the runs above stand: - The two tests the fix round touches, `test_an_exported_pythonpath_does_not_reach_a_template_validator` (3 cases) and `test_run_validator_gives_the_child_exactly_the_pythonpath_it_is_given` (6 cases), run by node id: 9 passed (11.7 s). The same 9 also passed (16.1 s) at `7f2bb36b`. - `tests/test_repo_hygiene.py`: 77 passed (1.9 s). It also passed (2.0 s) at `7f2bb36b`. No test fails on the base, because nothing here changes behaviour. The AST comparison above is the proof in their place. `git merge-tree --write-tree` of this head against origin/main and against every ref on origin that could meet it. The full run was on 2026-10-16, with the head `7f2bb36b` (the one before the last comment commit) and main `20c8873b`: the 41 branches (`fix/*`, `l3-*`, `wip/*`, `docs/*` and `release/v1.0`) and the 55 PR heads from #630 up. That is 96 refs. I ran #731's head `ab55c29f` through the same list. After `426877bf` I re-ran main, now `2d988200`, and the six heads that edit `tests/test_shape_doctor.py`, the one file that commit touches (#705, #737, #740, `l3-669-members-row`, `l3-680-doctor-signals` and `l3-697-crash-cure-next`), and #630: all clean. - **origin/main: clean.** At `20c8873b`, with the head `7f2bb36b`, the predicted tree is `6ead2ffc`. At `2d988200` (three `adopt-project.py` commits later, with their tests and a change in `scripts/shape_materialize.py`), with the head `426877bf`, it is `7ee5caf4`, and the squash is the same three files (43 added, 28 removed). #731's head `ab55c29f` is clean with `2d988200` too. This head with #731's head: clean. - **#630's head `6fd33a93`: clean.** It conflicted with main in `docs/handbook.html` when this body was first drafted, and no longer does, with or without this change. - **#700 and #702 have landed** (`55c6ba7d` and `20c8873b`), and so has #701, and #699, #717 and #719 for `adopt-project.py`. Their heads still merge clean with this head. - **Every ref that edits `shape-doctor.py`, `tests/test_shape_doctor.py` or `adopt-project.py` merges clean with this head**, except the two old heads named in the bullet after next. They include the heads of PRs not on main yet (#705, #721, #737 and #742, and lane 3's `l3-680`, `l3-683`, `l3-687`, `l3-691` and `l3-697`, which were not on origin when this body was first drafted). #673 has no branch or PR head on origin. - **Four refs conflict with origin/main on their own**, and so with this head: `docs/release-notes-v1.0.0` and `release/v1.0`, and the pre-squash heads of the landed #637 and #641. Their conflicts do not come from this change. - **Two landed pre-squash heads conflict with this head only because they are old.** #632 conflicts with main on its own. #636 merges clean with main, but its old head conflicts with this head in `run_validator`'s docstring, next to the text #636 added; main has that text, and this head merges clean with main. - **Refs that conflict with main on their own are not made worse.** #682, #745, #747, `fix/648`, `fix/681` and `fix/704` conflict with main in `templates/family-root/scripts/validate-family.py`; #742 and `l3-683` in `shape-doctor.py`. Each merges clean with this head. - **Every other open head and branch: clean.** Nothing from the issue. Its four lines are done. The decoy test's `sitecustomize` machinery is dead code under `-S` and I left it, because the issue says comment only. The comment above `reached` now says it is kept as a tripwire. - **The decoy test's `sitecustomize.py` and exit-86 trap never run.** `DECOY_SITECUSTOMIZE`, `os._exit(86)` and the `(exit 86)` and `DECOY` clauses in `reached` can only fire for a child started without `-S`. They are kept as a tripwire and the comment says so. The other choice is to remove them; `test_run_validator_starts_every_child_with_no_site` already holds `-S`. Small, tests only. Review: rv666, lane 5's independent review, approved a2113cf with nits; 7f2bb36 and 426877b take its S1 and N1 (above), and codex found no major issue at a2113cf. CI (Brett Heap's ruling 27 and his fast-refresh ruling of 2026-10-15T22:48:35Z): H0 426877b passed tests, tests-git-bash and SonarCloud; the head 32722b3 is a clean update-branch merge of main bf1f099, with tests-git-bash and SonarCloud green; and the local gate passed on its predicted squash tree 55d5d3b over main dcf62ed: the shape_doctor, scaffold_leg_path_collisions, repo_hygiene, agents_md_codes, cli_reference sets, 686 passed, 3 skipped, 1 xfailed. Landed on Brett Heap's word of 2026-10-17T10:59:39Z, relayed by lane 3 on #473: "merge lane 5's approved v1.1.0 drafts". Closes #666 Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 17, 2026
brettheap
added a commit
that referenced
this pull request
Oct 17, 2026
…ame it is given, so a branch whose upstream is not UTF-8 is moved back, and a good branch this system's encoding cannot write gets a line of its own, not a traceback (#710) (#824) `_try_git` read every answer through `repo_shape.git_out`'s strict UTF-8 decode. It now reads bytes and decodes them so that a name git prints goes back to git as the same bytes: - on POSIX with `os.fsdecode`, as #745 (#704) made `_head_of` and `_in_progress` read theirs, the inverse of the `os.fsencode` that `subprocess` applies to each argument; - on Windows as UTF-8, since `subprocess` hands git its arguments through `CreateProcessW`. Measured on main 24a977c before this change: - lane 5's site on #710: a tracking branch past the pin with no commit of its own, its name and upstream not UTF-8, and the leg detached at the pin, ended in `UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff`, exit 1, from `_upstream_holding`. It is now moved back. - under `LC_ALL=C PYTHONUTF8=0`, `main` tracking `origin/日本` past the pin ended in `UnicodeEncodeError` once the name, read as UTF-8 text, was handed back to `rev-parse`. It is now moved back. - the bytes are stripped and not the text, because `str.strip` took a trailing U+00A0 off `refs/remotes/origin/a<U+00A0>`, and the branch was then not moved back. `git_out` is not changed, as #710's ruling took Option 2. The readers of `origin`'s URL in `shape-doctor.py` and in the advisory it reads take its `UnicodeDecodeError` to mean the URL names nothing (#385). `adopt-project.py`'s `_default_branch` decodes locally since #717 (#425). L4-124, as Brett Heap ruled at 2026-10-17T13:01:07Z ("go with (a) for L4-124"): `_argument_fault` also faults a character `os.fsencode` cannot write (`UNWRITABLE`). The leg is left as it is, under a line that says `did not try branch`. - The fault is asked last. A name that also holds a surrogate is faulted for the surrogate, which `validate-manifest.py` names too (#280). - The second line calls the name good only where `git check-ref-format --branch`, asked in the root, takes the name's UTF-8 bytes; in a leg whose `.git` file names a git dir that is gone it exits 128. The line then names the encoding and gives `LC_ALL=C.UTF-8` and `PYTHONUTF8=1`. - Where git refuses the name (`feature/日本..x`, `日本.lock`, `日本 x`), the line gives git's words and #455's way out. L4-125's own route (a leg on `a<ff>b`, bootstrapped twice) and L4-126 (the manifest's `"a\udcffb"` followed back to `main`) already exit 0 on main through #745's `_head_of`. L4-126 is answered by option (b), the root cause, as R455 recommended. The new tests pin both. No pinned copy or recorded digest names `bootstrap.py` in the repository's tests, as #644 and #745 found when they changed it, so nothing is re-pinned. `tests/test_bootstrap_non_utf8_branch.py` runs each case on the harness of `tests/test_bootstrap_after_bump.py`. It holds the rule on every platform, under an emulated `ascii` `os.fsencode` and a legacy-mode Windows `os` stand-in, and skips where pytest itself is not under UTF-8. `tests/test_bootstrap_branch_bytes.py`'s no-fault test skips the rows this system's encoding cannot write. RV824 (lane 3) reviewed it, and found it READY after two fix rounds. Closes #710 Lane: openreposhape-4 (openRepoShape-4) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lane: openreposhape-1 (openRepoShape-1)
Claim: #425 (comment)
What
adopt-project.py's_default_branchread the branch the source is on (git symbolic-ref --short refs/remotes/origin/HEAD, thenHEAD) throughgit_out, which decodes strictly. git keeps a branch name as the bytes it was given, sogit branch caf<e9>is legal (a name typed in a Latin-1 terminal is one), and a source whose HEAD ororigin/HEADnames such a branch endedplan,checkandexecutein aUnicodeDecodeErrortraceback, exit 1, fromSource.open: no plan written, no leg, and no word about the branch.Now the name is read as bytes and decoded with
surrogateescape, as_elector(#292) does and as_branches_of,_listed_branchesand_config_valuealready do for every other name git prints. The byte is kept as a lone surrogate (caf\udce9), and the checks that already existed answer:planwith no--tracking-branch: the source's default branch is the tracking branch it would take (adopt-project.py: a source whose default branch is notmainis adopted with both legs pushed, then its pull request is refused (--base main), and the printed way out repeats--base main#479), so it is refused asunsafe-valuefor--tracking-branch (not given, so the source's default branch), exit 2, before any plan is written.plan --tracking-branch main(a branch the source has): the source is read atcaf<e9>, the plan recordsdefault_branch: "caf\udce9"and that branch's commit, andcheckandexecuteadopt it.checkandexecuteof a plan that tracks a branch the source no longer has:plan-tracking-branch-missing(adopt-project.py: a source whose default branch is notmainis adopted with both legs pushed, then its pull request is refused (--base main), and the printed way out repeats--base main#479, adopt-project.py:checkfinds, andexecuterefuses before any leg, the branch namesexecutefailed on, soadopt_branch: a..b, a source branchAdoptor one hidden byhideRefsno longer passcheckand fail after both legs exist (#520, #420) #583), naming both branches as the plan spells them, andexecuterefuses it before any leg exists.One line of code, plus its docstring.
Why (repro on main)
Measured on main
1babe42c, and again on main78fe3fff(whoseadopt-project.pyandscripts/de926142keeps unchanged) with the same result in every row, git 2.43.0, Python 3.12.3, local sources, a stubgh,--local-remote-dir. Each source is the conftest tree (three commits);mainandcaf<e9>name the same commit unless a row says otherwise.caf<e9>:planUnicodeDecodeErrorfrom_default_branch(git_out), no plan fileREFUSED unsafe-value: --tracking-branch (not given, so the source's default branch) is 'caf\udce9', …, no plan filerefs/remotes/origin/HEAD->origin/caf<e9>, HEAD onmain:planmain, thenmainrenamedcaf<e9>:checkFINDING plan-tracking-branch-missing: the plan'smainis no branch of the source, whose branches arecaf\udce9and whose default branch iscaf\udce9execute --yes --local-remote-dirREFUSED plan-tracking-branch-missing, the remote directory emptycaf<e9>,maintoo:plan --tracking-branch maincheckplan okexecuteadoption verified, both legs pushedplanplan ok,adoption verifiedmain(control):planThe traceback on main:
How
_default_branch:git_out(args, cwd=path, binary=True).decode("utf-8", "surrogateescape").strip().git_outis inscripts/repo_shape.py, a pinned shape copy: it is not changed, and no project'scontracts/shape-pin.yamlseesupstream-changed.The name goes back to git as the byte it was.
Source.openhands it togit rev-parseas astr;subprocessencodes a lone surrogate back to its byte withsurrogateescapeon POSIX, under any locale (plan --tracking-branch main,checkandexecutewere also run underLC_ALL=C PYTHONUTF8=0, where the filesystem encoding is ASCII: exit 0 andadoption verified; probed, not a test)._config_value(thebranch.<name>.remoteread of the relative-submodule-url check) passes it the same way: a source with a relative submodule url on acaf<e9>branch that tracks a remote is read as the branch's remote, with no traceback (probed, not a test).No new finding code and no new sentence. The issue proposed a dedicated code,
source-branch-not-utf8. The refusal that exists ischecked_value's, in the pinnedrepo_shape.py, and what it refuses is the ALPHABET a branch may be written in (^[A-Za-z0-9._/@+~-]{1,255}$), which every non-ASCII name fails alike: a UTF-8cafédefault branch gets the same refusal, word for word, with'café'for the value. So "not UTF-8" is not what the person has to change; the refusal already shows the byte ascaf\udce9, names the flag that avoids it (--tracking-branch) and states the alphabet. A sentence only for the non-UTF-8 case would mean wrapping the pinned function at this one call and again wherecheckprints the name, and would give one refusal two spellings. If it is wanted, it is a small follow-up.The other
git_outreads in adopt-project.py. Every one that reads a NAME now keeps a byte that is not UTF-8:git for-each-ref %(refname)in_branches_of, and_listed_bythrough_listed_branches_git_path(#520)git config --get <key>in_config_value_git_pathgit config user.namein_electorsurrogateescape(#292)git remote get-url originin_origin_urlUnicodeDecodeErroris caught and means no origin (#292)rev-parse,rev-list --count,hash-object,ls-files -s(split on whitespace)git ls-remotethroughrunerrors="replace"(#217)Nothing else needed the change.
Tests
tests/test_adopt_git_reads.py(the #292 file) goes from 21 to 29 tests: 8 new cases from 6 functions, on POSIX, skipped where no branch can be named with the byte 0xE9 or git does not give the name back as written. The source for thewherecases hascaf<e9>ONE COMMIT AHEAD ofmain, so a read of the wrong branch adopts the wrong commit and the tests see it.test_plan_refuses_a_default_branch_that_is_not_utf8origin-head,head: exit 2, theunsafe-valuetext, no plan file, no leg, no tracebacktest_check_finds_a_plans_branch_gone_for_one_that_is_not_utf8checkfindsplan-tracking-branch-missing, naming both branchestest_execute_refuses_a_plans_branch_gone_for_one_that_is_not_utf8executerefuses it before any leg (needsgit-filter-repo)test_plan_and_check_go_past_a_default_branch_that_is_not_utf8origin-head,head, with--tracking-branch main: the plan records the branch and ITS commit,checksaysplan oktest_execute_adopts_a_source_whose_default_branch_is_not_utf8origin-head,head:adoption verified, and the split is made oncaf<e9>'s commit, notmain's (needsgit-filter-repo)Revert table. The new tests, run against
adopt-project.pywith_default_branchchanged back (the other 21 tests pass in each):_default_branchreadsgit_outstrictly (main)errors="replace"(a tempting fix)REFUSED git-failedfrom agit rev-parseof a name with U+FFFD in place of the byte, which is not the branchUnicodeDecodeErrorand falling back tomain(the issue's other idea)plansays(main)for a source oncaf<e9>and exits 0;checkandexecuteafter the rename end inREFUSED git-failedfromgit rev-parse main; the plan recordsdefault_branch: main; the split's parent ismain's commit, notcaf<e9>'ssurrogateescape(this PR)Targeted runs at
44e97113, on a clean tree, with a stubghfirst on PATH, a bogusGH_TOKENand an emptyGH_CONFIG_DIR,/usr/bin/python3 -m pytest, over the 86 test files that run or importadopt-project.py(grep -l -E 'adopt-project|adopt_project|\bADOPT\b' tests/*.py, which includestests/test_shape_doctor.pyandtests/test_scaffold_leg_path_collisions.py):git-filter-repoon PATHgit-filter-repohidden (PATH=/usr/local/bin:/usr/bin:/binafter the stub dir)tests/test_adopt_git_reads.pyalone, on the merge of this head with mainde926142(which merges cleanly): 29 passed withgit-filter-repoon PATH, and 23 passed and 6 skipped with it hidden.Windows
CI is Linux only (#597), and these tests are POSIX only in effect: a Windows command line is text and cannot carry the byte, so they are written to skip there:
with_default_branchwhere git refuses the name or does not give it back as written, andmoved_to_a_branch_that_is_not_utf8where git refuses it. Neither has run on Windows. Nothing is compared in bytes read back from a file a helper wrote withwrite_text, and no path the tests create is illegal on NTFS.Not in this PR
scripts/repo_shape.py'sgit_out(pinned): unchanged.docs/release-notes/v1.0.0.md(describes the released v1.0.0; its adopt-project.py_default_branchreads the source's branch name throughgit_out, which decodes strictly, so a source whose HEAD (ororigin/HEAD) names a branch that is not UTF-8 endsplan,checkandexecutein a UnicodeDecodeError traceback, exit 1, fromSource.open, with no plan written and no refusal naming the branch #425 known limit stays true of it).scripts/family.py's_member_head, which readsgit symbolic-ref -q HEADthroughgit_outand has the same strict decode: a member checkout on such a branch raisesUnicodeDecodeErrorthere. It is not family.py init_creator_namereadsgit config user.namethroughgit_out, which decodes strictly, so a name that is not UTF-8 (no--created-by, no GIT_AUTHOR_NAME) endsinit,--dry-runincluded, in a UnicodeDecodeError traceback, exit 1, naming neither the git setting nor the flag that avoids it, where the scaffold refuses the same name by name (#282, #351) #426, which was_creator_name'sgit config user.nameand is closed; it is outsideadopt-project.pyand left for a follow-up.symbolic-ref(bump-leg.py,update-shape.py,shape-doctor.py): not audited here, because the issue isadopt-project.py's.Known limits
--tracking-branchmust name a branch the source has (plan-tracking-branch-missing), and the finding's remediation says "re-runplanwithout--tracking-branch, which records that default branch", whichplanrefuses for such a name. This holds for a UTF-8caféalike and is not new: adopt-project.py: a source whose default branch is notmainis adopted with both legs pushed, then its pull request is refused (--base main), and the printed way out repeats--base main#479's alphabet.cafédefault branch underLC_ALL=C PYTHONUTF8=0endsplanin aUnicodeEncodeErrorfromgit rev-parse <branch>inSource.open, because the name is passed as astr(probed on main1babe42c, unchanged by this PR). That is the C locale's family: adopt-project.py underLC_ALL=C PYTHONUTF8=0: the--descriptionem dash,plan'sgit grepandcheck's submodule url still end in a UnicodeEncodeError before any leg #653 (Tracking: adopt-project.py, converting an existing repository in place #457's D4).origin/HEADnaming a branch the source has no local copy of (any name,devas well) isREFUSED git-failed: git rev-parse dev … ambiguous argument, becauseSource.openasks for<branch>and notorigin/<branch>. A refusal, not a traceback; not changed here.Closes #425
🤖 Generated with Claude Code
Summary by Sourcery
Preserve non-UTF-8 source branch names throughout adoption planning and execution so invalid names are handled by existing validation and valid explicit tracking workflows continue successfully.
Bug Fixes:
Enhancements:
Tests: