Repository navigation
family.py: a failed remove's by-hand lines spell a leg mounted at a path that is not UTF-8 as a printf word of octal escapes ending in a slash, so git -C <member>/<leg> and submodule update -- <leg> typed in any POSIX shell name that leg (#1086) - #1098
Conversation
…ath that is not UTF-8 as a printf word of octal escapes ending in a slash, so `git -C <member>/<leg>` and `submodule update -- <leg>` typed in any POSIX shell name that leg (#1086) When a `remove` whose rollback cannot check the member out again prints the lines to type, a leg's path inside the member is read from the member's `ls-files -z` with `surrogateescape`. For a path that is not UTF-8, `shlex.quote` printed Python's `\udcXX` escapes. Typed, the `submodule update` and the `git -C` named no path. `_path_by_hand` now spells such a path as `"$(printf '<escapes>/')"`: #1084's form for a branch, with a `/` at the end, so a trailing newline is never the last byte `$(...)` strips, and a leading `-` as `\055`, which the `printf` of sh, dash and bash reads as an option. Each word names a directory or a gitlink, and git reads `<dir>/` as `<dir>`. A UTF-8 path keeps `shlex.quote`'s form byte for byte. A new test mounts a leg at such paths in a real holder, runs every printed line in sh, dash, bash and zsh, and reads the leg's commit and branch back as bytes. Closes #1086 Lane: openreposhape-3 (openRepoShape-3) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reviewer's GuideRollback instructions now reconstruct non-UTF-8 submodule leg paths byte-for-byte across POSIX shells by using octal-escaped printf substitutions with a trailing slash, while retaining existing UTF-8 quoting; extensive real-repository and cross-shell tests validate the behavior. Sequence diagram for byte-safe rollback commandssequenceDiagram
participant PinRollback
participant Shell
participant Git
participant Filesystem
PinRollback->>PinRollback: _path_by_hand(leg)
alt UTF-8 path
PinRollback-->>Shell: shlex.quote(leg)
else Non-UTF-8 path
PinRollback->>PinRollback: _printf_escapes(raw bytes)
PinRollback-->>Shell: $(printf '<octal escapes>/')
end
PinRollback-->>Shell: git -C <member> submodule update -- <leg>/
Shell->>Git: Execute rollback command
Git->>Filesystem: Resolve exact leg bytes
Filesystem-->>Git: Leg gitlink and checkout
Flow diagram for path spelling selectionflowchart TD
A["Rollback needs a leg path"] --> B{"_is_utf_8(path)?"}
B -->|Yes| C["shlex.quote(path)"]
B -->|No| D["_printf_escapes(raw bytes)"]
D --> E{"Starts with -?"}
E -->|Yes| F["Prefix \055 escape"]
E -->|No| G["Keep octal escapes"]
F --> H["Append trailing slash"]
G --> H
C --> I["Emit one shell word"]
H --> I
I --> J["Use in git -C or submodule update pathspec"]
File-Level Changes
Assessment against linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
(lane openRepoShape-3, 2026-10-21T13:18:01Z) Independent review RV1098 of #1098 at eaa9e4c: READY. No MUST, no SHOULD, three NITs. Probes (own helpers, not W1086's):
Mutants, each against tests/test_family_leg_path_by_hand_bytes.py, family.py restored and checked by cmp each time:
Tests at the head, one at a time, stub gh: the new file 78 passed with no skips (-rs); test_family_head_by_hand_bytes.py 61 passed; test_family_bump_remove_rollback.py 31 passed. Merges: merge-tree with origin/main 6da757c is clean. No other open PR touches scripts/family.py (live list of 12). The new file and test_family_head_by_hand_bytes.py also pass on a local merge with main: 139 passed. Body: closingIssuesReferences [1086]; draft; no milestone; the follow-up sentence is line 2; the authored text ends Closes #1086, Lane, robot, with no other closing keyword and nothing above U+007F but the robot. NITs:
Lane 3's plan: READY as it stands. NIT-1 is a docstring's wording, and lane 3 takes it in a one-line change before landing, which RV1098 then checks. NIT-2 is unchanged behaviour from main and stays recorded here. NIT-3 needs nothing. #1098 stays a draft with no milestone and lands after the v1.2.0 tag, on a later word. Lane: openreposhape-3 (openRepoShape-3) |
… a guard kept as #1079 spells it, since bash 5.2 and zsh 5.9 with history on expand no `!` inside `"$(printf ...)"` (#1086) Review round 1 of #1098 (RV1098: READY), NIT-1. The docstring said a shell with history on reads the `!` of the double-quoted word. RV1098 measured that bash 5.2.21 (`-i`, from a pipe) and zsh 5.9 (`-f -i`, pipe and pty) expand the `!` of `"x!echo"` but none inside `"$(printf ...)"`. The docstring now says so. No code changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
|
(lane openRepoShape-3, 2026-10-21T13:23:03Z) RV1098 round 2 at dad7961: READY.
NIT-1 is resolved. NIT-2 (a UTF-8 path holding a newline or a tab keeps shlex.quote's raw byte, as it did before this PR) stays a possible follow-up, outside this PR. Lane 3's plan: #1098 is READY at dad7961. It stays a draft with no milestone and lands after the v1.2.0 tag, on a later word, by the usual pilot, with an explicit squash message. Lane: openreposhape-3 (openRepoShape-3) |
|
Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days. You can request another review in 8 hours and 24 minutes by commenting |
…nt of the commit `--to` names, so a tag above it reads as one that reaches it and a tag below it keeps the plain words, and `--points-at` for `--contains` no longer survives (#1019) (#1107) No test held that a tag at a DESCENDANT of the refused commit counts as a tag that reaches it, so `--points-at` in place of `--contains` survived in `bump-leg.py` and `family.py` (mutant M9 of #988's review, filed as #1019). This adds the rows the issue asks for to the two reachability test files, and no production code changes. Opened as a draft with no milestone: Brett Heap's ruling of 2026-10-21T10:32:04Z ("leave the follow-ups out of v1.2.0") held it until the v1.2.0 tag, and it is now on the v1.2.1 milestone. Measured on main `2a119f28` before any change, and checked again on `16f9a0e81` and `7e00834c` (the tag questions did not change, only their line numbers): - `scripts/bump-leg.py` `_remote_tag_reaches` and `scripts/family.py` `_origin_tag_reaches` list the tags with `git for-each-ref --contains <commit> --format=%(objectname) refs/tags/`. No other copy of either exists (the `--contains` in `on_a_remote_branch` and `_on_a_remote_branch` asks about branches and is not this question). - The mutant (`--contains` -> `--points-at` on that one line in each script) SURVIVES every row on main: `test_bump_leg_reachability_words.py` 34 passed, `test_family_bump_reachability_words.py` 20 passed, 54 rows, as the issue says. - The production code is right: the new rows pass on unmutated main. So they do not "fail on main"; they fail on the mutant, which is the issue's own acceptance ("With `--points-at`, that row must fail"). No defect was found. `tests/test_bump_leg_reachability_words.py`: - `made_in_the_leg(project, name, on="HEAD")` takes an optional `on`, the commit the new commit is made on top of. Every existing call is unchanged. - `test_a_tag_at_a_descendant_of_the_commit_reaches_it`: a commit `made` and a child `above` on top of it, on no branch of the leg's remote. The leg and the remote each tag `above` (`v1-w1019`), made apart, in three kinds: both annotated (the reviewer's case J), leg lightweight and remote annotated, and the reverse. Both runs (dry run and real) refuse `made` `bump-leg-commit-local-only` in the TAG words, through `origin` and through `--local-remote-dir` (3 kinds x 2 routes = 6 rows), with nothing moved and the leg's own tag as it was. The premises say the tag names the child, that no tag is AT `made` (`for-each-ref --points-at`), and, through #1087's `only_a_tag_reaches`, that a tag of the remote reaches it, no branch does and no remote-tracking ref of the leg does. - `test_a_tag_at_an_ancestor_of_the_commit_alone_keeps_the_plain_words`: the control (x 2 routes). The commit named is the child, and the leg and the remote tag only its parent. The plain words stay, "a pin the rest of the world cannot fetch" included. `tests/test_family_bump_reachability_words.py`: - `made_on(holder, env, parent, name)`: a commit never pushed, made in the mount on top of `parent`, through `commit_in_mount` (which checks `parent` out first), so the holder stays clean. - `test_a_tag_at_a_descendant_of_the_commit_reaches_it`: the same three kinds (3 rows), both `bump --dry-run` and the real `bump` refuse `member-to-local-only` in the tag words, with nothing moved and the mount's own tag as it was. - `test_a_tag_at_an_ancestor_of_the_commit_alone_keeps_the_plain_words`: the control (1 row). No assert joins conditions with `and` or `or`; each premise is its own assert. No README, docs, AGENTS.md or handbook text names these tests or their row counts (grep), so none changes. | Mutant | Rows before (main's) | With these rows | | --- | --- | --- | | `--points-at` in `bump-leg.py` | survives: 34 of 34 pass | killed: 6 failed, 36 passed (all six descendant rows; the two controls pass, as they should) | | `--points-at` in `family.py` | survives: 20 of 20 pass | killed: 3 failed, 21 passed (all three descendant rows) | | `--merged` in `bump-leg.py` (extra: a check read the other way round) | survives: 34 of 34 pass | killed: 8 failed (6 descendant rows and both controls) | | `--merged` in `family.py` (extra) | survives: 20 of 20 pass | killed: 4 failed (3 descendant rows and the control) | The failure is the one the issue names: the plain words ("is reachable from no branch of ...") where the tag words ("and a tag of ... reaches it, but no branch ... does") are expected. Logs: `r-mut-points-at-bump.log`, `r-mut-points-at-family.log`, `r-mut-merged-bump.log`, `r-mut-merged-family.log` (and `mut-main-*.log` for the 34 and 20 on `2a119f28`). On the pushed head `01fcb259`: - git 2.43.0 (`/usr/bin/git`): the two test files, 66 passed (42 + 24; 34 + 20 before). - git 2.43.0: the trio `tests/test_repo_hygiene.py`, `tests/test_agents_md_codes.py`, `tests/test_cli_reference.py` and `tests/test_update_shape_kept_helper.py`, 329 passed. On `1201680f` (the same two files on top of `16f9a0e8`, before `7e00834c`, whose three files no row of mine touches or runs): - git 2.43.0: the trio alone, 170 passed; the two test files that import helpers from mine (`test_bump_leg_fetch_refspecs.py`, `test_family_bump_fetch_refspecs.py`), 102 passed; the two test files, 42 and 24 passed. - git 2.47.1 (`$Q/v115/w764/git247/bin/git` first on PATH, `git --version` printed `git version 2.47.1`): the two test files, 66 passed. No script changed, so no other test runs a changed script. - `git merge-tree --write-tree HEAD origin/main` against a re-fetched `origin/main` (`7e00834c`, which is also the merge base): exit 0, tree `3a852e87fcfda41766ece8c1771e47689c622ad6`, equal to this head's tree. Against `16f9a0e8` before the last rebase: exit 0 too. - Open PRs read with `gh pr list --repo opensoft/openRepoShape --state open --json number,headRefName,isDraft,files`: 8 (#1102, #1101, #1098, #1097, #1091, #1090, #1088, #1075). None touches `tests/test_bump_leg_reachability_words.py` or `tests/test_family_bump_reachability_words.py`, so there is no merge-tree to run against any of them. - **The NIT.** The docstrings of the two descendant rows said "Every row before this one tagged the commit itself, so ... survived all 54 of them". 54 counts the two files together (34 + 20), several earlier rows tag nothing (the no-`origin` and prune rows), and a count goes stale with the next row. - **The fix, docstrings only.** Each now says only what holds: "No row before this one tags a descendant of the commit: the tags in them are at the commit itself or below it, so `--points-at` in place of `--contains`, the mutant M9 of #988's review, survived every row of this file." No row, assert or helper changed. The 54 in "What still held on main" above is the two files together, as it says. - **One commit on top of `01fcb259`, a fast-forward push, no rebase:** head `ee48b5fe9693ebda099d1c0aadaca7bdc16b91c1`, read back with `git ls-remote`. Its tree is `615e571bb62fd1cf481e4152278b0e3628bab059`. - **Runs** (git 2.43.0, through `run9.sh`, `EMAIL=service@opensoft.dev`): `tests/test_bump_leg_reachability_words.py` 42 passed, `tests/test_family_bump_reachability_words.py` 24 passed. They ran on `7186f3cf`, the same tree as the pushed head; only the commit message was shortened after. - **Merge-tree** against a re-fetched `origin/main` (`6da757cb`, merge base `7e00834c`): exit 0, tree `453566df33c886cf7fc2b1c39af02acbbff365dc`. Main moved by nine files, none of mine; its `scripts/bump-leg.py` change (9 lines added) leaves `_remote_tag_reaches` and its `--contains` line alone (now line 1214). Review: rv1104-1107, lane 5's independent opus reviewer, which wrote none of #1107: READY at `ee48b5fe`, the head that lands (issuecomment-6318124100), with no MUST or SHOULD and one optional test NIT that needs no round. CI (Brett Heap's ruling 27): H0 is the head ee48b5f itself, which passed tests, tests-git-bash and SonarCloud; main has touched none of its files since its base 7e00834; and the local gate passed on its predicted squash tree 2f7f587 over main 422a590: the bump_leg_reachability_words, family_bump_reachability_words, repo_hygiene, agents_md_codes, cli_reference sets, 237 passed. Landed on Brett Heap's word of 2026-10-23T13:10:26Z, relayed to every lane by lane 3 (#473 issuecomment-6365697123): "merge v1.2.1 PRs once reviewed and green". Closes #1019 Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ed into a folder this user cannot search is said once in `what is here` or `agent files`, `<name>: cannot be read (<reason>)`, never absent; a `.git` offers no way in (#1110) (#1144) Eight sites in `shape-doctor.py` and `scripts/shape_advisory.py` asked `exists()` or `is_file()` of a name directly in the root, and both calls follow a link. So a `.git` or an agent file that is a link into a folder this user cannot search made the doctor raise on Python 3.12, and, as 3.14's `is_file` answers False to any `OSError`, read as absent there. On 3.12 the doctor said `FINDING .git: cannot be read` four times over, and an unreadable `AGENTS.md` added `agent-files` to the verdict. Under a 3.14-like shim it said `there is no repository here to adopt` and offered the scaffold. - Each site now asks the file's own stat helper, which applies #430's rule: `stat` the path. ENOENT, ENOTDIR, EBADF and ELOOP (and Windows' 21, 123 and 1921) mean nothing is there, and any other `OSError` is raised so the caller can name it. The helper is `is_a` in `shape-doctor.py` and `_is_a` in `scripts/shape_advisory.py`. The four `.git` sites in `shape-doctor.py` share `carries_git(root)`, which accepts a gitdir file, a directory or a link to either. - Such a file is said once, in `what is here` (a directory that is no shape root) or `agent files` (in a project), as `<name>: cannot be read (<reason>)` with the chmod cure, never as absent. In `agent files` it is a `note`, and a FINDING only for the pinned `AGENTS-shape.md`, which the project's own `validate-pins.py` refuses by name. - An unreadable `.git` offers no way in, since it is not known whether there is a repository to adopt. `the way in` has no NEXT and its `detail.git_repository` is `null`. `naming` keeps the policy's own status and says that `origin`'s name was not read. `origin_org`, `repo_local_origin_name` and `repository_names` raise the refused `stat`, and `shape_advisory.read()` keeps its `not-answered` state, whose `why` now names the `.git` and its reason. - `tests/test_doctor_unsearchable_links.py` holds 25 rows, and the eight sites leave `KNOWN_OPEN` in `tests/test_path_guard_hygiene.py`. - Measured with Python 3.12.3 and a 3.14-like `sitecustomize` shim, over a folder `Atlas` whose `.git` links to the git directory of a checkout of `acme/Orbit` in a folder at mode 000: - On base, 3.12: `naming`, `what is here`, `the way in` and `the other way in` are each `FINDING .git: cannot be read (Permission denied)`, with no Triad row, and `NOT A SHAPE ROOT`, exit 2. - On base, the shim: `the way in` reads `there is no repository here to adopt`, with NEXT `setup.sh --org <your-org> --project Atlas`. - On head, 3.12 and the shim give byte-identical reports (`cmp`): the cure is said once, in `what is here`, `the way in` offers neither command, and `naming` says that `origin`'s name is not read. Exit 2. - The new rows: base 2 passed and 23 failed (the 2 are controls), on 3.12 and under the shim; head 25 passed on both. Mutants: 47 of 47 killed, the writer's 18 and the review's 29. - Known limits: - An unreadable leg `AGENTS.md` is still read as absent where `_leg_by_agents_md` reads it (`scripts/shape_advisory.py`), so `the way in` and the Triad row still offer adoption over it, as on main. It is not one of #1110's eleven sites, and #1110 lists it as a twelfth. `_leg_by_agents_md` is unchanged. - A `.git` that is a gitdir file pointing into a folder this user cannot search, and a root `.git` directory at mode 000, still read as a repository, so adoption is offered as before. - Python 3.14 itself and Windows were not run. The 3.14 statements come from the shim, and the mode rows skip on Windows and as root. - `NOT_ANSWERED`'s new `why` is printed by nothing, so it is reachable only in process, where the rows hold it. - Not in this PR: slice B's three sites (`scripts/family.py` `_gitmodules_names` and `_sections_giving`, and `templates/family-root/scripts/siblings.py` `submodule_urls`) stay on `KNOWN_OPEN` under #1110, waiting for other lanes' open DRAFTs on `family.py`'s `remove` (#1117 and #1098). A shape root whose own `.git` is a link into a folder this user cannot search still reads DRIFTED, and both of its pinned validators crash. Base and head behave alike, and it is filed as #1146. Review: lane 1's independent reviewers reviewed each round. r1144 found `eb3f098c` READY on condition, with no MUST, two SHOULDs and six NITs (#1144 issuecomment-6328153279). The SHOULDs were that the title and design point 1 claimed too much for an agent file, and that seven of the reviewer's mutants survived. The round after it narrowed the title, added 8 rows and left the two tools unchanged, and a last commit split two composite asserts for SonarCloud's `python:S9073`. r1144b found `89486f69` READY, with no MUST or SHOULD (issuecomment-6331490548). Each of the seven survivors is killed by its named row, and all 47 mutants are killed. Tests, under a stub `gh` with a bogus token: at `89486f69`, the gate in the PR body gave 1,141 passed and 3 skipped over its 9 rows. They covered the PR's test file, on 3.12 and under the shim, `tests/test_path_guard_hygiene.py`, `tests/test_shape_doctor.py` in 4 shards, `tests/test_scaffold_leg_path_collisions.py`, `tests/test_doctor_unreadable_paths.py` with `tests/test_doctor_unreadable_pin.py`, the other files that import or run `shape_advisory`, the doc trio, and `tests/test_doctor_unreadable_paths.py` under the shim. It lands by fast refresh (Brett Heap, 2026-10-15T22:48:35Z). Its H0 `89486f69` is green on Linux and SonarCloud. The branch was never rebased. After H0, the only commit is update-branch's clean merge of main, giving 5b38c3a, which is green on SonarCloud and Git Bash. Lane 1's gate passed on its tree: 7 pytest calls over `tests/test_doctor_unsearchable_links.py`, `tests/test_path_guard_hygiene.py`, `tests/test_agents_md_codes.py`, `tests/test_cli_reference.py`, `tests/test_repo_hygiene.py`, `tests/test_scaffold_leg_path_collisions.py`, `tests/test_no_auto_maintenance.py` and `tests/test_shape_doctor.py` in 4 shards: 925 passed, 5 skipped. Part of #1110 Lane: openreposhape-1 (openRepoShape-1) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…py: an `id:`, a date or a `repository:` ending in a line feed is refused by name, and a pin's `source_repository:` that is no path or `owner/repo` name is named as the pin's (#1139) (#1166) The value patterns the issue names refuse a value that ends in a line feed, by name, as #1126 (#1017) makes `COMMIT_RE`, `SHA256_RE` and `COMMIT_40` do: - `DATE_RE` and `QUALIFIED_RE` in `validate-manifest.py` and `validate-family.py`, `MEMBER_RE` in `scripts/family.py`, `DATE_RE` in `scripts/shape_advisory.py` and `REPOSITORY_RE` in `update-shape.py`; - `PROJECT_ID_RE`, read with `fullmatch` where the two validators and `family.py add` read an id from a file. A shape pin's `source_repository:` that is neither a path on this machine nor an `owner/repo` name is refused in the pin's name, not `--upstream`'s. Values without a line feed keep their outcome, and no refusal code is added or renamed. Two inputs do move from one existing code to another (see "Behaviour changed", below). **What this PR does not cover, and the title does not claim:** - the naming-policy patterns that `scripts/repo_shape.py` reads from `contracts/repository-naming.yaml`, which `validate-repository-naming.py` still reads with `.match`; - `update-upstream-not-a-repo`, which still says `--upstream` when a pin names a folder that exists and is not a repository. Both are follow-ups, with the other findings of the independent review, under "Follow-ups (not in this PR)". This does not need #1126 and does not touch its files or its test file. Measured on main `038dafba` (Python 3.12.3, git 2.43.0) by a probe of each tool, before any change. Each value is written in the file as a YAML double-quoted `"...\n"`, which the reader takes as the value and a line feed. | Read from | Value | On main | |---|---|---| | `project.yaml` (`validate-manifest.py`) | `id: "atlas\n"` | only `FINDING manifest-topic: topic is 'xf-project-atlas', expected 'xf-project-atlas\n' derived from id 'atlas\n'`, exit 1 | | `project.yaml` | `elected_on: "2026-09-02\n"` | `manifest ok`, exit 0 | | `project.yaml` | a leg's `repository: "testorg/Atlas-spec\n"` | `manifest ok`, exit 0 | | `family.yaml` (`validate-family.py`) | `id: "inkrouter\n"` | `family ok: InkRouter (inkrouter` and a line break, exit 0 | | `family.yaml` | `created_on: "2026-09-04\n"` | `family ok`, exit 0 | | `family.yaml` | a member's `repository: "InkRouter/IRRS\n"` | `FINDING member-repository-name: ... whose name is 'IRRS\n'`, exit 1 | | a member's `project.yaml`, read by `family.py add` | `id: "irqs\n"` | mounted and committed, exit 0, with the row `id: "irqs\n"`; the holder's gate then said `family ok` | | `contracts/shape-pin.yaml`, `update-shape.py check` with no `--upstream` | `source_repository: "opensoft/openRepoShape\n"` | `REFUSED unsafe-value: --upstream is 'opensoft/openRepoShape\n'`, exit 2 | | the same | `source_repository: "opensoft/open Repo"` | `REFUSED update-upstream-unresolvable: --upstream 'opensoft/open Repo' is neither ...`, exit 2 | | the same | `source_repository: "-opensoft/openRepoShape"` | `REFUSED unsafe-value: --upstream is '-opensoft/openRepoShape'`, exit 2 | | an adoption plan (`adopt-project.py check`) | `id: "northwind\n"` | `FINDING plan-manifest-invalid: id is 'northwind\n', which the project's own validate-manifest.py refuses: manifest-topic: ...`, exit 1; `execute` refused it, exit 2, before any leg | In-process on the same main: `repo_shape.PROJECT_ID_RE.match('atlas\n')`, `shape_advisory.DATE_RE.match('2026-09-02\n')` and `REPOSITORY_RE.match('a/b\n')` each matched, and `date.fromisoformat('2026-09-02\n')` raised `ValueError`. - **`DATE_RE` and `QUALIFIED_RE`** in `templates/assembly-root/scripts/validate-manifest.py` and `templates/family-root/scripts/validate-family.py` end in `\Z`. So do `MEMBER_RE` in `scripts/family.py` (the third spelling `tests/test_family_member_name.py` holds equal to the two `QUALIFIED_RE`), `DATE_RE` in `scripts/shape_advisory.py` and `REPOSITORY_RE` in `update-shape.py`. - **`PROJECT_ID_RE` keeps its `$`.** Every id refusal prints its text (`must match ^[a-z0-9][a-z0-9-]*$`, pinned in six test files), the validators run against each project's own copy of `repo_shape.py`, and its definition sits inside #1126's hunk. The three callers that read an id from a file ask it with `fullmatch` instead: `validate-manifest.py` (`manifest-id`), `validate-family.py` (`family-id`) and `family.py`'s reading of a member's `project.yaml` (`member-bad-id`, which `add` now says before anything is mounted). - **`update-shape.py`.** `open_upstream(spec, pin_path=None)` takes the pin the value was read from, which `prepare` passes when no `--upstream` was given. Then `update-upstream-unresolvable` says ``<root>/contracts/shape-pin.yaml: `source_repository:` is '...', which is neither a path on this machine nor an `owner/repo` name``, with a remedy to write the pin's field or pass `--upstream`, and `checked_value` names the same field. **Code kept:** `update-upstream-unresolvable`, the existing code for a value that is not an `owner/repo` name. A line feed now reaches it, since the pattern no longer takes one. Before, `checked_value` refused it as `unsafe-value`. - **One existing test line:** `tests/test_family_init_values.py` finds validate-family's `DATE_RE` line by its text, which now ends `\Z`. - **One existing stand-in:** the `temporaries` fixture in `tests/test_update_shape_unreadable.py` stands in for `open_upstream` with `def opening(spec)`, which the new two-argument call refused with `TypeError: ... takes 1 positional argument but 2 were given`. It now takes `pin_path=None` and hands it on to the real `open_upstream`, so `test_a_read_refused_in_prepare_closes_the_upstream[check|apply]` asks what it asked before. - **The rows** are in a new file of their own, `tests/test_value_pattern_line_break.py`. - **A second guard, in a second commit (`5102b4294`, a test only, no production file).** The first guard holds the four pattern names this PR ends in `\Z`, and nothing held `PROJECT_ID_RE`, which keeps its `$`. `test_every_reader_of_the_id_pattern_is_in_the_table` reads every tracked `.py` outside `tests/` with `ast` (it runs none) and holds each call of a method of `PROJECT_ID_RE` (`PROJECT_ID_RE.x(...)` or `repo_shape.PROJECT_ID_RE.x(...)`) to the table `ID_READERS`: - the three `fullmatch` readers of an id from a file (`validate-manifest.py`, `validate-family.py`, `scripts/family.py`); - the modules that still `match` it, each with its reason: argv `--id` in `scaffold-project.py` and `scripts/family.py`, where `checked_value` refuses a line feed after the pattern; argv `--id` in `adopt-project.py`, which the project's own `validate-manifest.py` refuses (`adopt-manifest-invalid`, below; `ID_READERS` still gives it the `checked_value` reason); and `adopt-project.py`'s two `plan-bad-id` readers, of the plan's `id:` and of the id derived from `legs.assembly`, which a line feed never passes (below). A new reader of `PROJECT_ID_RE`, `fullmatch` included, fails it by file and method until it is listed in `ID_READERS` with its reason. The table is exact both ways, so a reader taken away fails it too. **Left as they are, and why:** - the argv readers of `PROJECT_ID_RE` (`--id` in `scaffold-project.py`, `family.py init` and `adopt-project.py`). The first two hold the id to `checked_value` after the pattern, which refuses a line feed as `unsafe-value` naming `--id`, and `tests/test_derived_id_refusals.py` pins that answer for them. `adopt-project.py` never calls `checked_value` on `--id`: `plan` with `--id` set to `northwind` and a line feed exits 2 with `adopt-manifest-invalid`, in the words of the project's own `validate-manifest.py` (`manifest-id`), and writes no plan (lane 5's re-check, issuecomment-6371408915, NIT 2; measured on `plan` only); - `adopt-project.py`'s reading of a plan's `id:`: `check` and `execute` hold it to the project's own `validate-manifest.py`, which refused it on main (above) and now names it in `manifest-id`'s words. It is the same value, refused under the same code, `plan-manifest-invalid`, before any leg exists. The id derived from `legs.assembly` is read with `.match` too, and a line feed in `legs.assembly` is `unsafe-value` before that id is asked (the review measured both, on main and here alike). This keeps the open PRs on `adopt-project.py` (24 at 12:02Z) clear; - `PIN_ARG_RE`, and `NAME_RE`/`REF_RE`/`PATH_RE` in `setup-project.py`, as the issue says; - `update-upstream-not-a-repo` still says `--upstream`. A pin's value reaches it only when it names a path that exists, not through the pattern. **Behaviour changed, stated plainly, and what the issue's Verify means by "No refusal code changes":** the issue's Verify line reads as no code added or renamed, and none is. Two inputs do move from one existing code to another, exit 2 both before and after: - `update-shape.py --upstream "opensoft/openRepoShape<LF>"` was `unsafe-value` and is now `update-upstream-unresolvable`, still naming `--upstream`; - a pin's `source_repository:` that ends in a line feed, with no `--upstream`, was `unsafe-value` naming `--upstream` and is now `update-upstream-unresolvable` naming the pin's field. No test pinned either old answer: every gate below passed. `family.py add --member` is stripped before `MEMBER_RE`, so `add` answers as before. **Not changed, seen while measuring:** with `--upstream` passed, `update-shape.py` does not read the pin's `source_repository:` through `REPOSITORY_RE` at all, and it keeps the value as the upstream's repository name. `validate-family.py` does not hold a member row's `id:` to `PROJECT_ID_RE`; it compares the row with the member's manifest. Neither is a `$` question. Both are under "Follow-ups (not in this PR)". All in `tests/test_value_pattern_line_break.py`, run against main's production code: the five production files swapped in place from `origin/main` (`961640c68`), then put back with their sha256 matching (`dirty=5` in the run's START line is those five files). On the head `5102b4294` (30 rows) the result is **20 failed, 10 passed in 23.12s, git 2.43.0**. The 10 that pass are baselines: the seven `test_a_pattern_takes_its_value` rows, the spelling-table guard and the two double-quoted-without-a-line-feed rows. The first 19 are the first commit's rows (19 failed, 10 passed in 22.12s on its 29 rows at `038dafba`). The independent review reproduced them from main's bytes of the same five files (19 failed, 10 passed in 26.78s, the same 19 ids). The 20th, `test_every_reader_of_the_id_pattern_is_in_the_table`, is the second commit's: it fails on main because the three readers of an id from a file are `.match` there, not `.fullmatch`. - `test_a_pattern_refuses_its_value_and_a_line_feed[manifest-DATE_RE]` - `test_a_pattern_refuses_its_value_and_a_line_feed[manifest-QUALIFIED_RE]` - `test_a_pattern_refuses_its_value_and_a_line_feed[family-DATE_RE]` - `test_a_pattern_refuses_its_value_and_a_line_feed[family-QUALIFIED_RE]` - `test_a_pattern_refuses_its_value_and_a_line_feed[family.py-MEMBER_RE]` - `test_a_pattern_refuses_its_value_and_a_line_feed[shape_advisory-DATE_RE]` - `test_a_pattern_refuses_its_value_and_a_line_feed[update-shape-REPOSITORY_RE]` - `test_a_manifest_value_that_ends_in_a_line_break_is_a_finding[id]` - `test_a_manifest_value_that_ends_in_a_line_break_is_a_finding[elected_on]` - `test_a_manifest_value_that_ends_in_a_line_break_is_a_finding[leg-repository]` - `test_adopt_check_names_a_plan_id_that_ends_in_a_line_break_as_an_id` - `test_a_family_value_that_ends_in_a_line_break_is_a_finding[id]` - `test_a_family_value_that_ends_in_a_line_break_is_a_finding[created_on]` - `test_a_family_value_that_ends_in_a_line_break_is_a_finding[member-repository]` - `test_add_refuses_a_member_whose_id_ends_in_a_line_break` - `test_a_pins_source_that_is_no_owner_repo_is_named_as_the_pins[line-feed]` - `test_a_pins_source_that_is_no_owner_repo_is_named_as_the_pins[space]` - `test_a_pins_source_checked_value_refuses_is_named_as_the_pins` - `test_an_upstream_passed_with_a_line_feed_is_named_as_passed` - `test_every_reader_of_the_id_pattern_is_in_the_table` **13 of 13 killed.** Each mutant was applied alone to one file of the branch's production bytes (the same bytes as the first commit and as the head, which adds a test only), the rows were run, and the file was put back with its sha256 checked (`restored=yes` for all 13). Each run is `tests/test_value_pattern_line_break.py` plus `tests/test_family_member_name.py::test_every_spelling_of_the_member_name_rule_is_the_same_pattern`, git 2.43.0. | Mutant | File | Killed by | |---|---|---| | M1 `DATE_RE` back to `$` | `validate-manifest.py` | 2: `test_a_pattern_refuses_its_value_and_a_line_feed[manifest-DATE_RE]`, `test_a_manifest_value_that_ends_in_a_line_break_is_a_finding[elected_on]` | | M2 `QUALIFIED_RE` back to `$` | `validate-manifest.py` | 3: `...[manifest-QUALIFIED_RE]`, `...[leg-repository]`, `test_every_spelling_of_the_member_name_rule_is_the_same_pattern` | | M3 the id read with `.match` | `validate-manifest.py` | 2: `test_a_manifest_value_that_ends_in_a_line_break_is_a_finding[id]`, `test_adopt_check_names_a_plan_id_that_ends_in_a_line_break_as_an_id` | | M4 `DATE_RE` back to `$` | `validate-family.py` | 2: `...[family-DATE_RE]`, `test_a_family_value_that_ends_in_a_line_break_is_a_finding[created_on]` | | M5 `QUALIFIED_RE` back to `$` | `validate-family.py` | 3: `...[family-QUALIFIED_RE]`, `...[member-repository]`, `test_every_spelling_of_the_member_name_rule_is_the_same_pattern` | | M6 the id read with `.match` | `validate-family.py` | 1: `test_a_family_value_that_ends_in_a_line_break_is_a_finding[id]` | | M7 `MEMBER_RE` back to `$` | `scripts/family.py` | 2: `...[family.py-MEMBER_RE]`, `test_every_spelling_of_the_member_name_rule_is_the_same_pattern` | | M8 a member's id read with `.match` | `scripts/family.py` | 1: `test_add_refuses_a_member_whose_id_ends_in_a_line_break` | | M9 `DATE_RE` back to `$` | `scripts/shape_advisory.py` | 1: `...[shape_advisory-DATE_RE]` (its pattern row alone: `date.fromisoformat` refuses the line feed after the pattern, so no tool row sees it) | | M10 `REPOSITORY_RE` back to `$` | `update-shape.py` | 3: `...[update-shape-REPOSITORY_RE]`, `test_a_pins_source_that_is_no_owner_repo_is_named_as_the_pins[line-feed]`, `test_an_upstream_passed_with_a_line_feed_is_named_as_passed` | | M11 `prepare` passes no pin | `update-shape.py` | 3: `test_a_pins_source_that_is_no_owner_repo_is_named_as_the_pins[line-feed]` and `[space]`, `test_a_pins_source_checked_value_refuses_is_named_as_the_pins` | | M12 `checked_value` names `--upstream` again | `update-shape.py` | 1: `test_a_pins_source_checked_value_refuses_is_named_as_the_pins` | | M13 the pin's refusal sentence removed | `update-shape.py` | 2: `test_a_pins_source_that_is_no_owner_repo_is_named_as_the_pins[line-feed]` and `[space]` | No mutant is added for the stand-in in `tests/test_update_shape_unreadable.py`: it is a test change. The independent review re-ran M3, M8, M11 and M12, and one mutant of its own (X1: `prepare` names the pin even when `--upstream` is passed). All five were killed, and each file was put back with its sha256 checked. The other nine of the 13 are this writer's runs alone. **The second commit's guard: 6 of 6 probes caught.** Each was applied alone to one tracked file at `5102b4294`, `test_every_reader_of_the_id_pattern_is_in_the_table` was run, and the file was put back with its sha256 checked (`restored=yes` for all six). The first commit's guard, `test_every_spelling_of_a_value_pattern_is_in_the_table`, was run with each plant too: it passes every one of them, which is the gap the second guard closes. | Probe | File | The new guard | The first guard | |---|---|---|---| | P1 a new `PROJECT_ID_RE.match(value)` reader appended | `update-shape.py` | failed: `('update-shape.py', 'match'): 1` is more than the table holds | passed | | P2 a new `repo_shape.PROJECT_ID_RE.search(value)` reader appended | `scripts/shape_advisory.py` | failed | passed | | P3 `fullmatch` back to `match` | `templates/assembly-root/scripts/validate-manifest.py` | failed | passed | | P4 `fullmatch` back to `match` | `templates/family-root/scripts/validate-family.py` | failed | passed | | P5 `fullmatch` back to `match` | `scripts/family.py` | failed | passed | | P6 main's bytes of those three files together (the 20th row above) | the three | failed | not run | **The second commit `5102b4294` (dirty=0)**, Python 3.12.3, with the same stub `gh`, bogus token and no global git config: | Run | Git | Result | |---|---|---| | `tests/test_value_pattern_line_break.py` | 2.43.0 | **30 passed** in 23.00s | | `tests/test_value_pattern_line_break.py` | 2.47.1 | **30 passed** in 23.30s | | `tests/test_pinned_copy_text.py`, `tests/test_path_guard_hygiene.py`, `tests/test_repo_hygiene.py`, `tests/test_agents_md_codes.py`, `tests/test_cli_reference.py` | 2.43.0 | **306 passed** in 61.54s | That commit changes only the test file (62 lines added, no assert that joins conditions with `and` or `or`, ASCII), so the first commit's runs below are not repeated: they hold the production files and every other test file, and those are unchanged. CI on `5102b4294`, and an independent look at the second commit, were not done when this was written. **The independent review of the first commit** (rv1166c, READY, read-only) reports these gates on `2593952c9`, all passing: the stand-in gate's 6 files (298 passed), the new file on 2.47.1 (29 passed), the five hygiene files (306 passed), 15 more test files that name a changed function, pattern or code (1611 passed, 19 skipped) and `tests/test_shape_doctor.py` (445 passed, 3 skipped). It also read CI on that head at about 20:04Z: `tests` (the full suite on 3.14), `tests-git-bash`, `package` and SonarCloud all passed. **The first commit's gates ran on `2593952c` (dirty=0)**, Python 3.12.3, with the real `gh` unreachable (a stub first on `PATH`, a bogus token) and no global git config: | Run | Git | Result | |---|---|---| | the files that held every failure and setup error of the shards below: `tests/test_family.py`, `tests/test_update_shape_pr_base.py`, `tests/test_update_shape_unreadable.py`, `tests/test_update_shape_header_comments.py`, `tests/test_update_shape_messages.py` and the new `tests/test_value_pattern_line_break.py` | 2.43.0 | **298 passed** in 343.60s | | `tests/test_value_pattern_line_break.py` | 2.47.1 | **29 passed** in 21.31s | | `tests/test_pinned_copy_text.py`, `tests/test_path_guard_hygiene.py`, `tests/test_repo_hygiene.py`, `tests/test_agents_md_codes.py`, `tests/test_cli_reference.py` | 2.43.0 | **306 passed** in 69.12s | **The two shards, on the uncommitted tree** holding the same production bytes as the first commit (the 7 files' sha256 checked before and after), git 2.43.0: the validators' test files, the `REPOSITORY_RE`/`source_repository` files and the `update-shape.py`, `shape_advisory.py` and `family.py` files, 125 in all. - shard 1 (62 files): 4 failed, 5290 passed, 13 skipped in 56 min; - shard 2 (63 files): 3 failed, 4 setup errors, 3761 passed, 29 skipped in 50 min. Every failure and setup error is in the five existing files of the first row of the table above, which all pass on the first commit: - 2 were real: `test_a_read_refused_in_prepare_closes_the_upstream[check|apply]`, the `opening(spec)` stand-in under the two-argument call. The stand-in is fixed in the first commit. - The other 5 failures and the 4 setup errors were `update-conflict` on `scripts/validate-family.py` ("not a verbatim copy ... at the pinned commit"). This is an artifact of an uncommitted tree: `tests/test_family.py`'s `upstream_clone` commits the working tree into its clone, while `family.py init` pins the holder at the real `HEAD`, so an uncommitted `templates/family-root/` file is never verbatim at its pin. On the first commit they pass. Every other file of the two shards passed on these production bytes, so the shards were not re-run. `git merge-tree --write-tree <other> 5102b42` (git 2.43.0) at 2026-10-23T12:08Z, after re-fetching main (still `961640c6`; read again at 12:10Z, just before the push) and re-listing the open PRs at 12:02Z (55 open). Main has moved from the `038dafba` that this body measured to `961640c6` by one file, `docs/release-notes/v1.2.0.md`. Each open PR that touches one of this PR's 8 paths, at its current head, plus #1126: | Against | Head | Shares | Result | |---|---|---|---| | origin/main | `961640c6` | | clean, tree `4613729516ff` | | #1098 | `dad7961a` | `scripts/family.py` | clean, tree `4d1ff9517588` | | #1117 | `10dc011c` | `scripts/family.py` | `scripts/family.py` merges clean; conflict in `docs/handbook.html` only, which this PR does not touch (see below) | | #1124 | `8dfa32bd` | `validate-family.py` | clean, tree `cb2ec1073646` | | #1126 | `4c1f80cf` | none (its `scripts/repo_shape.py` and tests) | clean, tree `e82c8945ac37` | | #1133 | `a7bfee1b` | `update-shape.py` | clean, tree `a710870d1ffc` | | #1144 | `89486f69` | `scripts/shape_advisory.py` | clean, tree `2d3e3e1ef43d` | | #1147 | `b75babee` | `validate-manifest.py` | clean, tree `a82ac3b47a3f` | | #1159 | `66e68f94` | `scripts/shape_advisory.py` | clean, tree `4abeb7b38665` | | #1164 | `74096eb4` | `scripts/family.py` | clean, tree `ee285e1cb8cf` | | #1172 | `5aa0f116` | `validate-family.py` | clean, tree `aa32d41e6e2c` | | #1187 | `cc8a87ee` | `update-shape.py`, `tests/test_update_shape_unreadable.py` | clean, tree `25711eb1cac4` | | #1201 | `eac177af` | `scripts/family.py` | `scripts/family.py` merges clean; conflict in `docs/handbook.html` only (see below) | - #1117 and #1201 conflict in `docs/handbook.html` alone. #1201 is stacked on #1117 (their merge-base is #1117's head `10dc011c`). Each of them against origin/main alone gives the same conflict in the same file, so it is theirs: #1117's, from its older base `6da757cb`, which #1201 inherits. - #1187 and #1201 are new since the first merge-trees. #1187 shares `tests/test_update_shape_unreadable.py` with this PR; its diff adds or removes no line that names `open_upstream`, `def opening` or `REPOSITORY_RE`, so it brings no one-argument stand-in for the two-argument call. That was read for all 55 open heads at 12:12Z; only this PR's own lines name them. - No other open PR adds or removes a call of a method of `PROJECT_ID_RE`, which the second guard counts: the lines each of the 55 open heads adds or removes outside `tests/`, `docs/` and `*.md` were read at 12:04Z. Only this PR's own name it, and #1201's one new `MEMBER_RE.fullmatch` is another pattern. So a sibling landing first does not move the guard's table. Each is a candidate for its own issue. They come from the independent review (rv1166c, READY at `2593952c9`, issuecomment-6352173084), which measured each of them on main and at the head unless the item says it did not. The review's finding numbers are in brackets. - **The naming-policy patterns [1].** `scripts/repo_shape.py` reads the families, the roles and the topic pattern of `contracts/repository-naming.yaml` with `.match` (`family["_re"].match(name)`, `family["_split_re"].match(name)`, `role["_re"].match(name)`, and `topic_pattern`, which defaults to `^xf-project-[a-z0-9-]+$`). `validate-repository-naming.py --project project.yaml` runs in the root template's `Makefile` and `validate.yml`. At the head, and on main, it exits 0 on a leg `repository: "testorg/Atlas\n"` and prints the line feed inside its table; `-spec` and `-code` the same. The project's gate as a whole still fails, because `validate-manifest.py` now refuses that manifest. - **`update-upstream-not-a-repo` with a pin [3].** With no `--upstream`, a pin whose `source_repository:` names a folder that exists and is no repository (the review measured `contracts` and `scripts/repo_shape.py`) is `REFUSED update-upstream-not-a-repo: --upstream contracts exists but is not a git repository`, exit 2, on main and here. `pin_path` is in scope at `update-shape.py:686-687`, so that refusal can name the pin as `update-upstream-unresolvable` now does. - **A pin's line feed when `--upstream` is passed [9].** `update-shape.py check --upstream <clone>` over a pin whose `source_repository:` ends in a line feed exits 0, `nothing to do`. The value is kept as `upstream.repository`, which reaches the README shape line's check, `apply`'s commit subject and the pull request words (`update-shape.py:2525`, `3017` and `5121`). The review measured `check`, not `apply`. It belongs with the previous item. - **A member row's `id:` [4].** On main, `add` of a member whose `project.yaml` reads `id: "irqs\n"` commits the row `id: "irqs\n"`, and the holder's `validate-family.py` then says `family ok`. Under mutant M8 (this branch with main's `.match` in `add`) the same row is written, and the gate still says `family ok`. By code reading, not measured end to end: `validate-family.py` compares a row's id with the member manifest's by `str` equality (`member-id-mismatch`, `validate-family.py:1608`), and `_check_member_ids_unique` does the same, so neither asks `PROJECT_ID_RE` of a row. This PR makes `add` refuse such a member, so no new holder gets such a row. - **Values that no pattern reads [8].** The review's sweep found these `manifest ok` or `family ok`, exit 0, with a trailing line feed, at the head and on main alike: - in `project.yaml`: `name`, a spec or code leg's `path`, `shape.repository` and `shape.digests.digest_algorithm`; - in `family.yaml`: `org`, the family's own `repository`, `tracking_branch`, and the shape's and each member pin's `digest_algorithm`. No `^...$` pattern reads them, so they are not this PR's. A leg's `path` is checked only by `_leg_path_findings`, which asks no pattern, and the family's `tracking_branch` is refused in `project.yaml` (`manifest-tracking-branch`, U+000A) but not in `family.yaml`. `shape.commit` and `tree_sha256` with a line feed are also `manifest ok`; that is #1126's. The first commit's message says "the three readers of an id from a file ask it with `fullmatch` (`manifest-id`, `family-id`, `member-bad-id`)". That is true of those three, but it reads as if there were no others [2]. What the review measured instead: - `adopt-project.py` still reads the id derived from `legs.assembly` (`:6780`) and the plan's `id:` (`:6791`), both `plan-bad-id`, with `.match`; - neither lets a line feed through. A plan `id: "northwind\n"` is `plan-manifest-invalid`, now in `manifest-id`'s words. A plan with no `id:` and `legs.assembly: "NorthWind\n"` is `REFUSED unsafe-value: legs.assembly is 'NorthWind\n'`, exit 2, before the derived id is asked, on main and here alike. So it is the wording alone that overstates. The second commit's `ID_READERS` lists both readers with that reason. The title's pin clause is the narrower one (the pin's name only where the value is no path and no `owner/repo` name), and two inputs change from `unsafe-value` to `update-upstream-unresolvable` (see "Behaviour changed", above). Review: rv1166 and rv1166c, lane 5's independent opus reviewers, at `2593952c` (issuecomment-6352173084); then rc1166b, lane 5's independent sonnet re-checker, which wrote none of #1166: READY at `5102b429`, H0 of this landing (issuecomment-6371408915), with no MUST or SHOULD; its NITs 1 and 2 are corrected in the body. CI (Brett Heap's ruling 27 and his fast-refresh ruling of 2026-10-15T22:48:35Z): H0 5102b42 passed tests, tests-git-bash and SonarCloud; the head 5c8afef is a clean update-branch merge of main 93d2931, with tests-git-bash and SonarCloud green; and the local gate passed on its predicted squash tree 3c89065 over main 11560eb: the family_init_values, update_shape_unreadable, value_pattern_line_break, repo_hygiene, agents_md_codes, cli_reference sets, 365 passed, 19 skipped. Landed on Brett Heap's word of 2026-10-23T13:10:26Z, relayed to every lane by lane 3 (#473 issuecomment-6365697123): "merge v1.2.1 PRs once reviewed and green". Closes #1139 Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s message spells a path byte that is not UTF-8 as `_shown` does, an escape, and no longer says it names it with a `?` in its place, which #900 replaced (#1018) (#1104) `env_commit`'s docstring in `scripts/shape_materialize.py` says what the split commit's message does with a `root` or `drop` path that holds a byte that is not UTF-8, as the code does since #900. It said the message names such a path "with a `?` in place of that byte, a path that does not exist". That was true before #900, and it is not true now. A reader of the docstring is told about a `caf?.md` the tool no longer writes. Found by lane 1's writer on #900 and filed as #1018. Lane 5 took it outside v1.2.0, under Brett Heap's ruling of 2026-10-21T10:32:04Z. Measured on main `2a119f281` and again on `16f9a0e81` (main after #1087), with a small script that calls: `adopt-project.py`'s own `_split_message` and `_shown`, then `env_commit` and a real `git commit`, read back with `cat -A`. - The issue's claim held: the docstring (about line 2103 when filed, line 2268 on `2a119f281`, line 2349 on this branch) still said the message names the byte with a `?`. - The code does not. `_shown("caf\udce9.md")` is `` `caf\udce9.md` ``, and the split message for a `root` and a `drop` entry named that holds no lone surrogate. Through `env_commit` and `git cat-file commit HEAD`, `cat -A`: ``` DROPPED (in no leg and no longer here):$ `caf\udce9.md`$ STAYS IN THE ASSEMBLY ROOT:$ README.md$ `caf\udce9.md`$ ``` - What stays true of `env_commit` itself: a message that holds the lone surrogate, handed to it directly, is still written as `?`. `cat -A` reads `Name: caf?.md$`. So the `errors="replace"` paragraph is kept, and what changes is who it describes. - #1087 (main `16f9a0e81`) changed `scripts/shape_materialize.py` by 128 lines, none of them in `env_commit`, `COMMIT_COMMAND` or the message, and changed nothing in `adopt-project.py` (0 diff lines). The rebase onto it was clean and the docstring needed no other change. One file, `env_commit`'s docstring, in two paragraphs: the `errors="replace"` paragraph (round 1, 11 lines added and 7 removed) and the sentence above it (round 2, below; 3 added and 2 removed). Together 14 added and 9 removed against main `6da757cb`. The `errors="replace"` paragraph now says: - encoding, a character UTF-8 cannot carry is written as `?`, and nothing raises after both legs are pushed, as before; - `adopt-project.py` refuses a lone surrogate in a path it extracts (`entry_path_problem`), and a `root` or `drop` entry that holds one is listed in the split message but never raw: the message names each path as `plan`'s report does (`_shown`, #900), so the byte 0xE9 reads as an escape with no surrogate in it, `` `caf\\udce9.md` ``, the path the source holds; - written raw it would be a `?`, `caf?.md`, a path that does not exist, which is what `env_commit` still writes for a lone surrogate it is handed. The backslash is doubled in the source, as the neighbouring docstrings write `\\udcc3`. A single `\u` in a non-raw docstring would be an escape that makes a lone surrogate. Measured: the docstring holds no character from U+D800 to U+DFFF and holds the six characters `\udce9`. The longest docstring line is 78 characters. **The code is unchanged.** A script compares the AST of `scripts/shape_materialize.py` with every docstring blanked: the branch's file is identical to `origin/main`'s after the rebase onto `16f9a0e81` (round 1), and the merge of this head into `origin/main` `6da757cb` is identical to `6da757cb`'s file (round 2). **No pin moves.** `scripts/shape_materialize.py` is not a pinned copy. `COPIED_VERBATIM`, `COPIED_FROM_SHAPE`, `FAMILY_COPIED_VERBATIM` and `FAMILY_COPIED_FROM_SHAPE` do not list it, and `templates/assembly-root/scripts/validate-pins.py` says of it "which no project carries". So no `upstream-changed` row, no digest, no assembly copy and no `tests/test_pinned_copy_text.py` entry changes. **No other text pins these words.** Searched in `tests/`, `README.md`, `docs/`, `docs/handbook.html`, `AGENTS.md` and `templates/`: no test reads `env_commit`'s docstring (`__doc__`, `getdoc`), and none of them holds "in place of that byte" or "a path that does not exist" for this. The other places that name the `?` are about this function's behaviour or about history, and are still true: `_split_message`'s docstring says `caf\\udce9.md` "was `caf?.md`" (before #900), `_refused_by_policy` already says the message "lists it as `_shown` does (#900)", and `tests/test_adopt_spelled_names.py` quotes the message as it was on `c7a3ff16`. The `?` in `argv_text`, `trailer_line` and the commit-trailer text is about a `--trailer`, not the split message. No rows and no mutants: the change is a docstring, the code is the same AST, and no test pins the docstring. So there is no row to fail on main and nothing to mutate. All with `$Q/v110/run9.sh`, `EMAIL=service@opensoft.dev`, `GIT_SSH_COMMAND=false`, `GIT_CONFIG_GLOBAL=/dev/null`, the stub `gh`, a fake token, git 2.43.0 (`/usr/bin/git`). Two shards, run before the rebase, at the `2a119f281` base with this one-file change in the tree. Together they are every `tests/*.py` that names `shape_materialize` or `env_commit` (86 files, including `tests/test_adopt_spelled_names.py` and `tests/test_adopt_entry_paths.py`, which run `env_commit` and the #900 message) and `tests/test_agents_md_codes.py`: | run | files | result | |---|---|---| | shard A | 43 | 2873 passed, 4 skipped in 1707.51s, rc=0 | | shard B | 44 | 4611 passed, 31 skipped in 2462.01s, rc=0 | After the rebase onto `16f9a0e81`, on the committed head `e32b2703d733f6c60c1b2cba2d62d164d93d7b9a` (`dirty=0`): the trio, `tests/test_repo_hygiene.py`, `tests/test_agents_md_codes.py` and `tests/test_cli_reference.py`, **170 passed in 55.67s**, rc=0. The docstring-only change needed no new shards: the shards' counts are runs at the `2a119f281` base, and #1087 did not touch `env_commit`. Round 2, on the committed head `de89aa81b1533e32f1007f991aeae05cd915b36d` (`dirty=0`): the same trio, **170 passed in 61.77s**, rc=0. The round 2 commit is one docstring sentence, so it needed no new shards either. git 2.47.1: no run. The change adds no test file of its own, so there is none to run on it. `git merge-tree --write-tree HEAD <other>` with HEAD at `de89aa81b1533e32f1007f991aeae05cd915b36d`, after a re-fetch of `origin main` with `-c gc.auto=0 -c maintenance.auto=false` (round 2, about 14:03Z on 2026-10-21): | against | result | |---|---| | `origin/main` `6da757cbe9a4eb66d60c34f38dcc2b332d175e8b` | clean, rc 0, tree `15c11b6c959fc4c7540105d5d62e3f961c8e27fd` | | #1105 `fix/1048-makefile-follow-up-gnu-read-order` `5a4de7f763b5a3c32adce9614a3a539508416eb4`, an open PR that touches `scripts/shape_materialize.py` | clean, rc 0, tree `db05aead67052643942e35a9689d9dd4a0873fda` | | #1118 `fix/1089-named-link-skip-tests` `c60066cbeec18b8b6c926e254ed4bee6b9a486c2`, the other open PR that touches `scripts/shape_materialize.py` | clean, rc 0, tree `a40df3b63c32c2a3d8f4c40587b094940c8d6567` | Read with `gh pr list --state open --json number,headRefName,headRefOid,files`: 16 open PRs, three of them (this one, #1105 and #1118) touch `scripts/shape_materialize.py`, the only file this change touches. The other 13 (#1120, #1119, #1117, #1111, #1109, #1107, #1106, #1102, #1101, #1098, #1097, #1091, #1088) touch no file this change touches. The same command with HEAD at the reviewed head `e32b2703` gives rv1104's trees: `de31d857` against `6da757cb` and `195ffe19` against #1105. The round 1 merge against #1090 is moot: #1090 has landed in `6da757cb`. Held as a draft until the v1.2.0 tag; it is now on the v1.2.1 milestone. - **NIT 1, `scripts/shape_materialize.py` lines 2340-2343, a line round 1 did not change.** The paragraph above the reworded one said the adoption's split message "names each entry path as the source spells it". Since #900 that is true of a printable path alone: `_split_message` lists any other through `_shown`, as the escape the next paragraph describes. It now says "as the source spells it (a printable one; see below)". One new commit on top of `e32b2703`, `de89aa81b1533e32f1007f991aeae05cd915b36d`, 3 lines added and 2 removed, docstring only, pushed as a fast-forward (no rebase, the PR is open). - Measured: the AST with docstrings blanked is identical to `e32b2703`'s and, merged into `6da757cb`, to `6da757cb`'s. The docstring holds no lone surrogate, still holds the six characters `\udce9`, and its longest line is 78 characters. - Searched `tests/`, `README.md`, `docs/`, the handbook mirror, `AGENTS.md`, `templates/` and the scripts for "names each entry path", "as the source spells" and the new words: only `tests/test_adopt_entry_paths.py` line 233 matches, a test's docstring about `execute` handing each path on as the source spells it, which is another sentence and pins nothing here. - **NIT 2, this body.** It named main as `16f9a0e81` and #1090 as the one open PR that touches the file. #1090 has landed. Main is `6da757cb`, and the merge-trees above are redone against it, against #1105 and against #1118, a draft that opened since and also touches the file. Review: rv1104-1107, lane 5's independent opus reviewer, which wrote none of #1104: READY at `de89aa81`, H0 of this landing (issuecomment-6318111413), with no finding; NIT 1 and NIT 2 of the round before are fixed. CI (Brett Heap's ruling 27 and his fast-refresh ruling of 2026-10-15T22:48:35Z): H0 de89aa8 passed tests, tests-git-bash and SonarCloud; the head 92285d4 is a clean update-branch merge of main b66d8f0, with tests-git-bash and SonarCloud green; and the local gate passed on its predicted squash tree 6bd2128 over main b66d8f0: the repo_hygiene, agents_md_codes, cli_reference sets, 171 passed. Landed on Brett Heap's word of 2026-10-23T13:10:26Z, relayed to every lane by lane 3 (#473 issuecomment-6365697123): "merge v1.2.1 PRs once reviewed and green". Closes #1018 Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… and the `add` that `bump`'s `member-unknown` names carry `--family-root <holder>`, quoted for the host, so each runs as printed (#1005) (#1117) ## What `remove` and `add` require `--family-root`, as `bump` does. #1002 gave every remedy that names `family.py bump` the holder and left three that name another verb without it. Typed as printed, each ended in `family.py <verb>: error: the following arguments are required: --family-root`. Each now names `--family-root` and the resolved `family.root`, spelled by `shell_quoted`, as #1002's `bump` line and `_add_command` are. - **`scripts/family.py`:** - `add`'s `member-duplicate-id` (`_checked_member_id`): the `remove` it names is now `family.py remove --family-root <holder> --member <row>`. Its `bump` line was already right. - `bump`'s own `member-duplicate-id` (`_checked_bumped_id`): the `remove` it names is now `family.py remove --family-root <holder> --member <held row>`. - `bump`'s `member-unknown` (`cmd_bump`): the `add` it names is now `family.py add --family-root <holder> --member <org>/<P>`. `<org>` stays a blank for the person to fill in, as `<commit>` does. - **`README.md` and `docs/handbook.html`:** the "Windows, natively" sentence now names `family.py`'s `remove` line beside `bump`, since the new lines quote the holder for the shell they are printed in (`add` was already named). The handbook's seven `reads README.md` markers for the README section carry its new hash. - **`tests/test_readme_windows_quoting_sentence.py`:** the `NAMED` table counts the new host-dialect calls: `_checked_member_id` 2, `_checked_bumped_id` 1, `cmd_bump` 1. CI's full suite is where that table fails first, as it did on #1002, so it is in the gate here. The lines start with a bare `family.py`, as #1002's `bump` line does and as the lines in the same remedy sentence do. `validate-family.py`'s template copy already prints `--family-root` and the interpreter and script path, and is unchanged. ## Tests All runs used `/usr/bin/python3 -m pytest` under a stub `gh`, a bogus `GH_TOKEN`, an empty `GH_CONFIG_DIR`, `GIT_CONFIG_GLOBAL` an empty file, `GIT_CONFIG_NOSYSTEM=1` and no `GIT_SSH_COMMAND`. - **`tests/test_remove_add_remedies_family_root.py`, 45 rows:** each of the three refusals, over a holder in a plain folder, under `my work`, under `Dana's work`, reached by a relative `--family-root`, and reached through a symlink (skipped on Windows). Each checks three things: - the printed line is the expected text, and the host's own word-splitting gives the holder back as one word; - `family.py`'s own parser accepts it, with `<org>` filled in; - typed as printed from a folder that is not the holder's, it exits 0, leaves the holder clean and its gate green, and does what the remedy says: the refused `add` then goes through, or the refused `bump` then moves the row, or the member the refused `bump` named is in the family. - **Against main `2a119f281`:** all 45 fail and all 45 pass on this head. The `parses` and `runs` rows fail on `the following arguments are required: --family-root`, and the `carries the holder whole` rows fail on the printed line. - **`tests/test_family_add_member_id.py` and `tests/test_family_bump_to_checks.py`** follow the new words. The former's `remove` row now runs the printed line as it is, with no `--family-root` added by the test. - **Gate, at head `10dc011c9`, 1293 passed and 0 failed in three runs:** 927 (the new file, the doc and quoting files, and the family files that run `add`, `bump` and these codes), 178 and 188 (the validate-family, remedy and holder-state files). - Doc files in the gate: `test_readme_windows_quoting_sentence`, `test_repo_hygiene`, `test_readme_sections`, `test_setup_by_hand_commands`, `test_agents_md_codes`, `test_update_add_codes_documented`, `test_family_docs_one_spelling`, `test_family_printed_quoting` and `test_git_bash_dialect`. - **Mutants, 16 of 16 caught, each restored with its sha256 checked:** each remedy without `--family-root`; each with the holder unquoted; the wrong `--member`; `<org>/` dropped; each `shell_quoted` call told `'posix'` (the `NAMED` table); README or the handbook without `remove`; the handbook marker with the old hash; and the `NAMED` table without the new counts. ## Merges - **Main** `6da757cbe`: clean. - **#1098** (`scripts/family.py`): clean. - **#1101** (README and handbook, a lane 3 draft): conflicts in `docs/handbook.html` only, at the seven `reads README.md` markers of its `Starting a project in a new organisation` section, because both change that README section and so its hash. It is the known serialization of two README pull requests: the second to land recomputes the hash from the merged README. ## Notes - Outside this issue: `member-already-present`'s remedy says "`remove` takes it out" as a bare verb with no line to type. It is not one of the issue's three, so it is unchanged. Closes #1005 Lane: openreposhape-4 (openRepoShape-4) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>



What changed
A follow-up outside v1.2.0 (Brett Heap, 2026-10-21T10:32:04Z). It lands only after the v1.2.0 tag, on a later merge word.
On main. When
remove's rollback cannot check the member out again, it prints the lines to type (PinRollback._work_tree_by_hand). Two of them name a leg:git -C members/<P> -c protocol.allow=never submodule update --init --no-fetch -- <leg> ...;git -C members/<P>/<leg> checkout -q <branch> --, for a leg that was on a branch.A leg's path is read from the member's
ls-files -zwithsurrogateescape(_legs), soshlex.quotequoted the surrogate. Measured on main (2a119f28) with a leg mounted atcaf+ byte E9, the lines read... --no-fetch -- 'caf\udce9' specandgit -C 'members/Thing/caf\udce9' checkout -q side --. Typed, they name no path.The issue says "bump". These lines are
remove's:PinRollbackreads legs only withunmount=True, andbump's by-hand line names the member only.Now. A new
_path_by_hand(path)spells each word that holds a leg's path:shlex.quote, byte for byte as before;"$(printf '<escapes>/')". That is family.py: the by-hand checkout line a failed rollback prints names a branch that is not UTF-8 by its exact bytes, as"$(printf 'caf\351')", not as'caf\udce9', which typed names no branch (#1079) #1084's form for a branch, with a/at the end and a leading-as\055._head_by_handspells its-Cpath with it, and the legs'submodule updatespells each leg with it. A member's own path comes fromfamily.yaml, which is UTF-8, so the member's lines do not change._branch_by_hand's escape loop moved into_printf_escapes, which both use;_branch_by_hand's output is unchanged, and its test file passes.The chosen form, and why.
"$(printf 'caf\351')", not as'caf\udce9', which typed names no branch (#1079) #1084's printf word, plus two changes./at the end.$(...)strips only trailing newlines; with/last, nothing is stripped. Each word names a directory or a gitlink, and git reads<dir>/as<dir>.git -C <dir>/changes into the directory.git submodule update -- <leg>/matches the gitlink<leg>, because git matches a pathspec with a trailing slash to the directory of that name, and no index entry can sit under a gitlink.-. Theprintfof sh, dash and bash reads a format starting-cas an option (Illegal option -c,invalid option); zsh's prints it. So the first byte, if it is-, is\055.',\,%,!and every byte outside printable ASCII are three-digit octal escapes, as in family.py: the by-hand checkout line a failed rollback prints names a branch that is not UTF-8 by its exact bytes, as"$(printf 'caf\351')", not as'caf\udce9', which typed names no branch (#1079) #1084. So the line is ASCII and holds no newline.Rejected:
xstripped by${v%x}: it needs a variable, so two statements per leg, one variable per leg on the update line, and variables left in the person's shell;$'...': dash has none;git submodule updatewith no pathspec would also take legs that were never checked out, which it can only clone;Decided where the direction left it open:
/goes on every word for a path that is not UTF-8, not only on one that ends in a newline: one rule;-at the first byte is escaped, since only the first byte can be read as an option.Four-shell measurement
tests/test_family_leg_path_by_hand_bytes.pybuilds real repositories: a holder, one member atmembers/Thing, and two legs checked out inside the mount.specis detached. The odd leg is at the path below, on the branch below. The test records the rollback asremovedoes, then runsgit submodule deinit -fon the member asremovedoes, and takes_work_tree_by_hand()'s four lines. Each line is run as printed through<shell> -c. Then the leg is read back as bytes: a checkout is there, at its gitlink's commit, andsymbolic-ref HEADis the branch.specis read back at its commit, and the member is read back onmain.caf\xe9(the issue's)sidecaf\xe9\nsideit's 100%!\\\xe9\n\nsidesub/caf\xe9 \x01\x7f\xffcaf\xe9caf\xc3\xa9's leg(UTF-8: lines equalshlex.quote's)sideThe word alone:
printf %s <word>in each shell writes the path's bytes and then/. Rows that git cannot mount as a leg (a leading-:git submodule addrefuses one) are asked here.caf\xe9caf\xe9\n\n\n\xe9\n\n-caf\xe9--\xe9\na'b\\c%d!e\xe9%s%%\xff%d\n\xc3\n-, every byte 0x01-0xff, then\nHost: git 2.43.0, bash 5.2.21, zsh 5.9, dash as
/bin/sh. Every cell passed in the run below. No shell skipped (-rslists none).Out of scope
family.yaml, which is UTF-8, as the issue says.\udcXXincluded:_work_tree_said, and themember-store-gonerefusal for a leg left out. Those are words to read, not lines to type, and the issue names only the typed lines. This can be a follow-up.submodule updatehas no--literal-pathspecs, though the rollback's own run has it. So a leg path holding*,?or[is also a glob on that line, for a UTF-8 path as much as any. git tries the literal path first, so the leg itself always matches. This is not this issue.\udcXX, so the typedgit -C <member>/<leg>andsubmodule update -- <leg>name no path #1086 has no section headed "Needs a ruling" or "Documented limit".Tests
Each file was run alone, under a stub
ghfirst on PATH (it exits 97), withGH_TOKEN=bogus-W1086, an emptyGH_CONFIG_DIR,EMAIL=tests@openreposhape.invalid,GIT_SSH_COMMAND=falseandGIT_CONFIG_GLOBAL=/dev/null, astimeout 2400 /usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<scratch>/bt-gate -o tmp_path_retention_policy=failed -rs tests/<file>:tests/test_family_leg_path_by_hand_bytes.py(new):78 passed in 50.23stests/test_family_head_by_hand_bytes.py:61 passed in 10.58stests/test_family_bump_remove_rollback.py:31 passed in 52.64stests/test_readme_windows_quoting_sentence.py:44 passed in 22.69s. No host quoter was added (shell_quoted,shell_line,shell_word,quote_arg), so README, the handbook andNAMEDare unchanged.tests/test_repo_hygiene.py:100 passed in 3.27stests/test_no_auto_maintenance.py:58 passed, 2 skipped in 15.23s. Both skips arethe session began with GIT_CONFIG_GLOBAL set, from the exported/dev/null.tests/test_family_py_unreadable.py(it names the member's update line andPinRollback):75 passed in 60.01stests/test_family_member_branch_not_utf8.py(PinRollback):4 passed in 10.35stests/test_family_refuse_up_front.py(PinRollback):79 passed in 114.84stests/test_agents_md_codes.py:47 passed in 2.19stests/test_cli_reference.py:23 passed in 36.14stests/test_update_add_codes_documented.py:6 passed in 0.63spython3 -m pyflakes(4.0.0): no warning onscripts/family.pyat main or at the head, and none on the new test.Mutants
Each mutant was applied to
scripts/family.pyand the new test file was run. Then the file was restored and checked withcmp(equal each time).family.py: 74 failed, 4 passed. All 16 real-repo rows for a path that is not UTF-8 fail; the 4 UTF-8 rows pass.'{escapes}/'becomes'{escapes}'): killed, 53 failed. These include the real-repo rowsa-trailing-newlineandapostrophe-percent-bang-backslash-two-newlinesin all four shells, 40 word rows, andtest_a_trailing_newline_is_an_escape_before_the_slash.!passed through (!dropped from the escaped set): killed bytest_the_bang_is_an_octal_escape_as_in_a_branch, and by family.py: the by-hand checkout line a failed rollback prints names a branch that is not UTF-8 by its exact bytes, as"$(printf 'caf\351')", not as'caf\udce9', which typed names no branch (#1079) #1084'stest_the_bang_is_an_octal_escape_so_no_history_scanner_reads_the_line. Nosh -crun reads history, so only these spelling tests can see it.shlex.quotebranch removed): killed, 33 failed. That is all 13test_a_utf8_path_is_quoted_exactly_as_shlex_quote_has_always_donerows and all 20 real-repo rows (thespecword and the member's-Cchange).-left as is: killed, 10 failed. These are thea-leading-dash,two-dashes-and-a-newlineandevery-byte-after-a-dashword rows in sh, dash and bash, andtest_a_leading_dash_is_an_escape_so_printf_reads_no_option. The zsh rows pass under this mutant, because zsh'sprintfprints a leading-._head_by_handkeepsshlex.quote(path): killed, 18 failed: the 16 real-repo rows for a path that is not UTF-8, and 2 spelling tests.shlex.quote(leg): killed, 16 failed: the 16 real-repo rows for a path that is not UTF-8.Merges
git merge-tree --write-tree, with the headeaa9e4c0:2a119f28: clean;family.py bumpandbump-leg.pyno longer fail where the remote hasxandrelease/xand the repository's owngit fetch originworks (#1066) #1087 (a9289bd9): clean. shape_materialize.py: the fetch for a branch's answer leaves out every refspec of the repository's own onto the branches' names, whatever its source, and every one that puts a branch under a name not its own, sofamily.py bumpandbump-leg.pyno longer fail where the remote hasxandrelease/xand the repository's owngit fetch originworks (#1066) #1087 is the only open PR onscripts/family.py, from a livegh pr list.Review round 1 (RV1098: READY)
RV1098 found no MUST and no SHOULD. Lane 3 took NIT-1 only, in
dad7961a.test_the_bang_is_an_octal_escape_as_in_a_branchsaid a shell with history on reads the!of the double-quoted word. RV1098 measured that bash 5.2.21 (-i, from a pipe) and zsh 5.9 (-f -i, pipe and pty) expand the!of"x!echo", but none inside"$(printf '...')". The docstring now calls\041a guard kept as family.py: a rollback's by-hand checkout line spells a branch that is not UTF-8 as Python's\udce9, which typed names no branch #1079 spells it, for shells nobody has measured here. No code changes.tests/test_family_leg_path_by_hand_bytes.pyatdad7961a, with-rs:78 passed in 55.40s, no skips. pyflakes: no warning.git merge-tree --write-treewith origin/main6da757cb, which now holds shape_materialize.py: the fetch for a branch's answer leaves out every refspec of the repository's own onto the branches' names, whatever its source, and every one that puts a branch under a name not its own, sofamily.py bumpandbump-leg.pyno longer fail where the remote hasxandrelease/xand the repository's owngit fetch originworks (#1066) #1087: clean. No other open PR touches these files (a livegh pr list).Closes #1086
Lane: openreposhape-3 (openRepoShape-3)
🤖 Generated with Claude Code