Skip to content

family.py: the remove that both member-duplicate-id remedies name and the add that bump's member-unknown names carry --family-root <holder>, quoted for the host, so each runs as printed (#1005) - #1117

Merged
brettheap merged 3 commits into
mainfrom
fix/1005-family-remove-add-family-root
Nov 8, 2026

Conversation

@brettheap

@brettheap brettheap commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Lane: openreposhape-4 (openRepoShape-4) wrote this for issue #1005, which follows #927 (#1002).

Closes #1005

What

remove and add require --family-root, as bump does. #1002 gave every remedy that names family.py bump the holder and left three that name another verb without it. Typed as printed, each ended in family.py <verb>: error: the following arguments are required: --family-root. Each now names --family-root and the resolved family.root, spelled by shell_quoted, as #1002's bump line and _add_command are.

  • scripts/family.py:
    • add's member-duplicate-id (_checked_member_id): the remove it names is now family.py remove --family-root <holder> --member <row>. Its bump line was already right.
    • bump's own member-duplicate-id (_checked_bumped_id): the remove it names is now family.py remove --family-root <holder> --member <held row>.
    • bump's member-unknown (cmd_bump): the add it names is now family.py add --family-root <holder> --member <org>/<P>. <org> stays a blank for the person to fill in, as <commit> does.
  • README.md and docs/handbook.html: the "Windows, natively" sentence now names family.py's remove line beside bump, since the new lines quote the holder for the shell they are printed in (add was already named). The handbook's seven reads README.md markers for the README section carry its new hash.
  • tests/test_readme_windows_quoting_sentence.py: the NAMED table counts the new host-dialect calls: _checked_member_id 2, _checked_bumped_id 1, cmd_bump 1. CI's full suite is where that table fails first, as it did on family.py and bump-leg.py: every remedy that names family.py bump names --family-root, so each runs as printed (#927) #1002, so it is in the gate here.

The lines start with a bare family.py, as #1002's bump line does and as the lines in the same remedy sentence do. validate-family.py's template copy already prints --family-root and the interpreter and script path, and is unchanged.

Tests

All runs used /usr/bin/python3 -m pytest under a stub gh, a bogus GH_TOKEN, an empty GH_CONFIG_DIR, GIT_CONFIG_GLOBAL an empty file, GIT_CONFIG_NOSYSTEM=1 and no GIT_SSH_COMMAND.

  • tests/test_remove_add_remedies_family_root.py, 45 rows: each of the three refusals, over a holder in a plain folder, under my work, under Dana's work, reached by a relative --family-root, and reached through a symlink (skipped on Windows). Each checks three things:
    • the printed line is the expected text, and the host's own word-splitting gives the holder back as one word;
    • family.py's own parser accepts it, with <org> filled in;
    • typed as printed from a folder that is not the holder's, it exits 0, leaves the holder clean and its gate green, and does what the remedy says: the refused add then goes through, or the refused bump then moves the row, or the member the refused bump named is in the family.
  • Against main 2a119f281: all 45 fail and all 45 pass on this head. The parses and runs rows fail on the following arguments are required: --family-root, and the carries the holder whole rows fail on the printed line.
  • tests/test_family_add_member_id.py and tests/test_family_bump_to_checks.py follow the new words. The former's remove row now runs the printed line as it is, with no --family-root added by the test.
  • Gate, at head 10dc011c9, 1293 passed and 0 failed in three runs: 927 (the new file, the doc and quoting files, and the family files that run add, bump and these codes), 178 and 188 (the validate-family, remedy and holder-state files).
    • Doc files in the gate: test_readme_windows_quoting_sentence, test_repo_hygiene, test_readme_sections, test_setup_by_hand_commands, test_agents_md_codes, test_update_add_codes_documented, test_family_docs_one_spelling, test_family_printed_quoting and test_git_bash_dialect.
  • Mutants, 16 of 16 caught, each restored with its sha256 checked: each remedy without --family-root; each with the holder unquoted; the wrong --member; <org>/ dropped; each shell_quoted call told 'posix' (the NAMED table); README or the handbook without remove; the handbook marker with the old hash; and the NAMED table without the new counts.

Merges

Notes

  • Outside this issue: member-already-present's remedy says "remove takes it out" as a bare verb with no line to type. It is not one of the issue's three, so it is unchanged.

🤖 Generated with Claude Code
Lane: openreposhape-4 (openRepoShape-4)

Summary by Sourcery

Make every affected family remediation command self-contained by including its resolved family root.

Bug Fixes:

  • Ensure all printed family.py remove and add remediation commands include the required, correctly shell-quoted --family-root argument so they run successfully as printed.

Enhancements:

  • Update Windows shell-quoting documentation and generated handbook references to include remove commands.
  • Expand coverage to verify remediation commands parse and execute correctly across quoted, relative, and symlinked family roots.

Documentation:

  • Document that family.py remove commands are quoted for the shell in which they are printed on Windows.

Tests:

  • Add end-to-end coverage for the three affected add and bump remediation scenarios and update expected command output tests.

… and the `add` that `bump`'s `member-unknown` names carry `--family-root <holder>`, quoted for the host, so each runs as printed (#1005)

`remove` and `add` require `--family-root`, as `bump` does. #927 gave every
remedy that names `family.py bump` the holder and left these three, which
typed as printed ended in `the following arguments are required:
--family-root`:

- `add`'s `member-duplicate-id`: `family.py remove --member <P>`.
- `bump`'s own `member-duplicate-id`: `family.py remove --member <P>`.
- `bump`'s `member-unknown`: `family.py add --member <org>/<P>`.

Each now names `--family-root` and the resolved `family.root`, spelled by
`shell_quoted` as #927's `bump` line and `_add_command`'s are. `<org>` stays
a blank for the person to fill in.

README's and the handbook's "Windows, natively" sentence names `remove` among
the lines `family.py` quotes for the shell they are printed in, and the
handbook's markers carry README's new section hash. The table in
`tests/test_readme_windows_quoting_sentence.py` counts the new host-dialect
calls in `_checked_member_id`, `_checked_bumped_id` and `cmd_bump`.

`tests/test_remove_add_remedies_family_root.py` types each printed line from
another folder, over holders in a plain folder, under `my work` and
`Dana's work`, reached by a relative `--family-root` and through a symlink:
the line names the holder whole, `family.py`'s own parser accepts it, and it
runs as printed and does what the remedy says. `tests/test_family_add_member_id.py`
and `tests/test_family_bump_to_checks.py` follow the new words, and the former
now runs the `remove` line exactly as printed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@sourcery-ai

sourcery-ai Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Reviewer's Guide

Implementation

Add the required, host-shell-quoted --family-root to the three remove and add remediation commands so they run as printed.

  • scripts/family.py L6602-6605, L9385-9394 (+3 small edits)

Documentation

Document that family.py's remove commands are quoted for the host shell and update the handbook markers to the revised README section hash.

  • README.md L161-162
  • docs/handbook.html L475-1763 (8 small edits)

Tests

Update remedy expectations and verify the new commands parse, preserve holder paths as single shell words, and execute as printed; extend the Windows quoting checks for the added host-dialect calls.

  • tests/test_family_add_member_id.py L74-183 (4 small edits)
  • tests/test_family_bump_to_checks.py L83-87 (+5 small edits)
  • tests/test_readme_windows_quoting_sentence.py L64-65, L73-77
  • tests/test_remove_add_remedies_family_root.py L1-282

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@brettheap

Copy link
Copy Markdown
Contributor Author

RV1117: independent review of #1117 at 10dc011c: READY

No MUST and no SHOULD. Seven NITs, all for follow-ups and none a reason to hold this PR.

What I measured

  • Head and main. Head 10dc011c9, main 6da757cbe, which is still the PR's base. git merge-tree --write-tree of the head against origin/main is clean. Its tree, de872303f, is the head's own tree and differs from main in the PR's 7 files only. The gate ran on a scratch merge commit, 48d6a6155.
  • Siblings. scaffold-project.py: the --reuse-empty-repo help, its docs and each remedy that sends a person into that re-run say ssh, a credential helper or askpass can still make it wait, and how to keep it from waiting (#915) #1101 at 2cbf54b5 conflicts in docs/handbook.html only, at the seven reads README.md markers, which is the known serialization of two README PRs. family.py: a failed remove's by-hand lines spell a leg mounted at a path that is not UTF-8 as a printf word of octal escapes ending in a slash, so git -C <member>/<leg> and submodule update -- <leg> typed in any POSIX shell name that leg (#1086) #1098 at dad7961a is clean. For the second to land: the README section hash after merging this head with scaffold-project.py: the --reuse-empty-repo help, its docs and each remedy that sends a person into that re-run say ssh, a credential helper or askpass can still make it wait, and how to keep it from waiting (#915) #1101 is b102a1cd7ec797cf9aa1daa80fbf3a5e60ae3b72. Recompute it on the real merge.
  • Marker hash, recomputed independently. I cut README.md at ## lines outside fences and ran git hash-object --no-filters on "Starting a project in a new organisation". Main gives 8f25ef0d... and the head gives 2085e8ac35de0506c5ea51774a105e0e400956d7. All 7 handbook markers carry the new hash and none the old. README keeps its line count (+2 -2), so readme-cap.d is not touched.
  • Scope. I searched scripts/, templates/ and the root tools for family.py (remove|add|bump|siblings). These three were the only family.py command lines without --family-root. Every other hit is prose, init (which takes no --family-root), or already carries it.
    • The template's validate-family.py prints its member-duplicate-id line as {PYTHON} scripts/family.py remove --family-root <root> --member <row>, each value shlex.quoted. It is unchanged and agrees with this PR.
    • AGENTS.md names neither code, so its table needs no change.
  • POSIX shells, with my own probe file (not in the PR). Each of the three lines was typed as printed in /bin/bash, /bin/dash and /usr/bin/zsh, from a folder that is not the holder's. A family.py wrapper on PATH logged its argv. The holders contained:
    • a $HOME b, a backtick pair, ; and &, !, and U+2019 (the typographic apostrophe);
    • a -lead dash component, glob characters, ~tilde, =eq, and a newline.
    • Result: 90 of 90 passed. Each run exited 0, family.py got the holder as one argv word, the holder stayed clean, its gate stayed green, and the repair the remedy names was done.
  • PowerShell. I drove _checked_member_id, _checked_bumped_id and cmd_bump in-process with host_shell_dialect forced to nt. For C:\Users\Dana's work\Orbit and three more Windows paths, powershell_words reads [verb, --family-root, <holder>, --member, <name>] from each line: 4 of 4.
  • The <org> blank. It is never mistaken for a value.
    • Pasted unfilled into bash, dash or zsh, it is a redirection: org: No such file or directory, exit 1 or 2, and family.py never runs.
    • With a file named org in the cwd, the shell instead tries to create /IRQS, which fails with permission denied for an ordinary user.
    • In PowerShell, < is a reserved operator, so the line is a parse error.
  • Leading dash, relative root, symlinked holder. The printed holder is Family.open's Path(...).expanduser().resolve(). It is absolute, so it never starts with -. A relative or symlinked --family-root prints the resolved path. Mutant M5 below shows the 10 relative and symlink rows are the ones that catch an unresolved spec.

Tests

All runs used the landing gate's environment: /usr/bin/python3 -m pytest -p no:cacheprovider with retention=failed, a stub gh, GH_TOKEN=bogus, an empty GH_CONFIG_DIR, an empty GIT_CONFIG_GLOBAL and GIT_CONFIG_NOSYSTEM=1.

  • The new file against main 6da757cbe: 45 failed, each for the right reason.
    • 15 runs rows: the following arguments are required: --family-root.
    • 15 parses rows: argparse's SystemExit: 2.
    • 15 carries the holder whole rows: the printed line differs.
  • The same file at the head: 45 passed.
  • Gate on the scratch merge: 1191 passed, 0 failed, in three shards.
    • Shard 1, 552 passed: test_readme_windows_quoting_sentence, test_repo_hygiene, test_readme_sections, test_family_docs_one_spelling, test_family_printed_quoting, test_agents_md_codes, test_update_add_codes_documented, test_setup_by_hand_commands, test_git_bash_dialect, test_family_add_member_id, test_family_bump_to_checks and test_remove_add_remedies_family_root.
    • Shard 2, 354 passed: test_family, test_family_dry_run, test_family_members_not_a_list and test_family_switch_branch_exists.
    • Shard 3, 285 passed: test_bump_remedies_family_root, test_family_tracking_branch, test_family_add_refusals, test_family_member_path, test_validate_family_branch_step, test_validate_family_duplicates, test_family_bump_remove_rollback and test_family_holder_remedies_compose.
  • Probe file: 97 passed, 2 skipped, 1 failed. The failure is the tracking-branch row (N1). The two skips are the remove lines, which a real add or bump does not reach on the tracking branch.
  • CI when I looked: tests-git-bash, package and SonarCloud passed, and tests was pending.

Mutants

These are mine. Each was restored after its run and its sha256 checked.

id mutation result
M1 add's remove line without --family-root caught, 21 failed: 15 new rows, 5 in test_family_add_member_id, NAMED
M2 bump's remove line without --family-root caught, 19: 15 new, 3 in test_family_bump_to_checks, NAMED
M3 member-unknown's add line without --family-root caught, 16: 15 new, NAMED
M4 add's remove line with the holder unquoted caught, 7: the 6 rows under my work and Dana's work, NAMED
M5 member-unknown prints the --family-root as typed, unresolved caught, 5: the relative and symlink rows
M6 bump's remove names the bumped row, not the row holding the id caught, 18: 15 new, 3 in bump_to_checks
M7 add's remove names the new name, not the pinned row caught, 20: 15 new, 5 in add_member_id
M8 member-unknown drops <org>/ caught, 15: the new rows
M11 bump's remove line quoted 'posix' caught, 1: NAMED

9 of 9 caught. Four mutants were not run:

  • M10, shlex.quote in cmd_bump, was stopped mid-run by a lane checkpoint. It prints the same bytes on Linux, so only NAMED could catch it, and M3 already shows cmd_bump's NAMED entry failing when its quoter is gone.
  • The three doc mutants (the old marker hash, the handbook without remove, and both docs without remove). The hash was recomputed by hand instead, and the README/handbook equality test passes.

Findings

  • N1 (NIT, follow-up, outside family.py: the member-duplicate-id and member-unknown remedies name family.py remove and family.py add without --family-root, so each fails as printed #1005). A fresh clone stands on the holder's tracking branch. There, bump's member-unknown is said before _checked_branch, so the add it names is refused family-on-tracking-branch.
    • Measured: exit 2, with that refusal's own git -C <holder> switch -c family/add-irqs. The chain still ends in a command that works.
    • A real add or bump asks the branch before the id, so it never reaches the two remove lines there. --dry-run on the tracking branch does reach them.
    • Fix: name the switch -c before the line, as validate-family.py's member-duplicate-id does before its remove, or say the line runs on a branch other than the tracking branch.
  • N2 (NIT). The lines start with a bare family.py, as family.py and bump-leg.py: every remedy that names family.py bump names --family-root, so each runs as printed (#927) #1002's bump lines do, so only the words after it run as printed. _add_command and validate-family.py print {PYTHON} <script>. Fix: a follow-up that gives every family.py remedy line one spelling.
  • N3 (NIT). <org> is left blank although the family knows an org: family.yaml's org:, which _add_command already prints. The person may also have typed --member <org>/<P> to bump. Unfilled, <org>/IRQS is a redirection that tries to create /IRQS when a file named org is in the cwd. Fix: print the family's org, or use a blank that is not a redirection.
  • N4 (NIT, pre-existing). --member {name} prints the row's project: raw, where validate-family.py shlex.quotes it. It changes no outcome today: remove's checked_value refuses any name that would need quoting.
  • N5 (NIT, pre-existing test design, by reading the test, not run). Nothing ties the README sentence's list of verbs to NAMED. The tests hold README to the handbook and NAMED to the code. Dropping remove from both docs, with the hash recomputed, would pass.
  • N6 (NIT, pre-existing). The command sits between backticks, so a holder path with a backtick reads ambiguously, and the PR's own the_line regex cuts at the path's backtick (probe). The shells still read the line whole: those probe rows passed.
  • N7 (NIT). As the PR's notes say, member-already-present's "remove takes it out" gives no line to type. A follow-up could print the remove line beside its bump.

Lane: openreposhape-4 (openRepoShape-4)

This was referenced Oct 21, 2026
…amily-root (#1005)

One file conflicted, docs/handbook.html, and only at the seven
`reads README.md, section Starting a project in a new organisation: blob ...`
markers (the masthead, `start`, `partway`, `ci`, `legs`, `doublepin` and
`park`), the known serialization of two README pull requests:

- this branch changed that README section's "Windows, natively" sentence, which
  now names `remove` among the lines `family.py` quotes for the shell they are
  printed in, so its markers carried blob 2085e8ac35de0506c5ea51774a105e0e400956d7;
- main's #1101 (7cd6690) added a paragraph to the same section, about the
  ssh, helper and askpass prompts a `--reuse-empty-repo` re-run can still wait
  on, so its markers carried blob ffaedeefe61720051098400cbf2212de50b923f3.

README.md and the handbook's prose merged without a conflict, and the merged
README holds both edits. Each of the seven markers is now the hash of that
section of the merged README, b102a1cd7ec797cf9aa1daa80fbf3a5e60ae3b72,
computed with the repository's own `readme_segments` and `readme_fingerprint`
(tests/test_repo_hygiene.py). No other README section changed on either side,
so no other marker moved, and no count, list or table in a test needed a new
value: main adds no `shell_quoted` call to `family.py` or `setup-project.py`,
and README's line count is unchanged by this branch (+2 -2).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@brettheap

Copy link
Copy Markdown
Contributor Author

W1117m: #1117 merged with main 961640c6 at 219e57c2

Merge commit 219e57c28a9f07ea5203ca06c3d4e83536e1d8fc. Its parents are 10dc011c9 (the PR head) and 961640c68 (main, v1.2.0). It went up as a plain fast-forward push, no force. GitHub now reports #1117 as MERGEABLE (mergeStateStatus CLEAN) at head 219e57c28.

The conflict

The resolution

  • All 7 markers now carry b102a1cd7ec797cf9aa1daa80fbf3a5e60ae3b72, the hash of that section of the merged README. It was computed with the repository's own readme_segments and readme_fingerprint from tests/test_repo_hygiene.py, and it matches the hash RV1117 predicted on 2026-10-21.
  • Only the Starting segment's hash differs between main, the PR and the merge, so no other marker moved.
  • No count, list or table needed a new value: main added no shell_quoted call to family.py or setup-project.py, and README's line count is unchanged by this branch.
  • git diff-tree origin/main HEAD is exactly this PR's 7 files, 337 insertions and 35 deletions. git merge-tree --write-tree origin/main HEAD is clean, with tree 8d9f3a712a6e0e5a992ca7f38c60ddd9023b3425, equal to HEAD's tree.

Tests on the merge commit (45 targeted files in 3 shards, the landing gate's environment: stub gh, GH_TOKEN=bogus, empty GH_CONFIG_DIR, empty gitconfig; the full suite was not run)

  • shard a (16 files: this PR's own tests, the README, handbook, docs and hygiene pins, and test_path_guard_hygiene and test_visibility_field_words from main): 704 passed in 297.80s.
  • shard b (12 files: the family.py and validate-family.py tests): 639 passed in 677.16s.
  • shard c (17 files: the other family quoting and remedy files, plus test_scaffold_reuse_seeded_legs): 684 passed in 1337.70s.
  • Total 2027 passed, 0 failed, 0 errors.

Lane: openreposhape-4 (openRepoShape-4)

@brettheap

Copy link
Copy Markdown
Contributor Author

RV1117r2: delta review of #1117 at 219e57c2: READY

No MUST, no SHOULD and no NIT of my own. RV1117's READY at 10dc011c carries to 219e57c2: the merge is clean apart from the 7 marker blobs, the new blob is right, and the tests pass on the head and on a merge with current main. RV1117's seven NITs stand as they were, since the code is byte-identical (below).

What I measured

  • Parents. 219e57c28a9f07ea5203ca06c3d4e83536e1d8fc has parents 10dc011c9ea5c0c72e108d14a107a7574d930960 (the reviewed head) and 961640c681b54d9258e6c9d3eab8c6f46a26008b (main at v1.2.0). As W1117m said.
  • Tree against a fresh merge. git merge-tree --write-tree 961640c68 10dc011c9 conflicts in docs/handbook.html only (stages 1, 2 and 3), tree 5a372476a with conflict markers. 219e57c2's tree is 8d9f3a712a6e0e5a992ca7f38c60ddd9023b3425. git diff-tree -r of the two trees names docs/handbook.html and nothing else. Diffing that file, the only changed lines are the 7 conflict blocks, each replaced by one marker line carrying b102a1cd.... Every auto-merged hunk, including main's scaffold-project.py: the --reuse-empty-repo help, its docs and each remedy that sends a person into that re-run say ssh, a credential helper or askpass can still make it wait, and how to keep it from waiting (#915) #1101 list item, is the same in both.
  • Against both parents.
  • The blob. I recomputed it without the repository's helper: cut README.md at ## lines outside code fences, sha1("blob <len>\0" + bytes), cross-checked with git hash-object --no-filters --stdin. "Starting a project in a new organisation" is b102a1cd7ec797cf9aa1daa80fbf3a5e60ae3b72. All 12 README segments match their markers (0 mismatches), no marker names a missing segment, the handbook holds 7 markers for that section and 0 carry ffaedeef or 2085e8ac. The pin test (tests/test_repo_hygiene.py::test_the_handbook_follows_the_readmes_outline) computes the same value: its failure text, with the marker mutated, says the section "hashes to b102a1cd7ec797cf9aa1daa80fbf3a5e60ae3b72 now".
  • No other pin needed a value.
    • Main added no shell_quoted or host-dialect call to family.py, scaffold-project.py, setup-project.py or shape-doctor.py since v1.2.0 (occurrence counts 40, 3, 4 and 18 on 961640c68 and on current main; the PR's own 3 new shell_quoted calls in family.py are the NAMED table rows RV1117 reviewed).
    • README's line count is unchanged by this branch, readme-cap.d is untouched, and test_readme_is_short_enough_to_be_read passes.
    • The quoters sentence, tests/test_readme_windows_quoting_sentence.py (44 rows) and the handbook sentence agree.
    • I searched the merged tree for printed family.py remove|add|bump|siblings lines without --family-root: every hit is prose, as RV1117 found.
  • CI on 219e57c2: tests, tests-git-bash, package and SonarCloud all SUCCESS.

Current main

Tests

Landing-gate environment: /usr/bin/python3 -m pytest -p no:cacheprovider -o tmp_path_retention_policy=failed, a stub gh first on PATH, GH_TOKEN=bogus, an empty GH_CONFIG_DIR, an empty GIT_CONFIG_GLOBAL, GIT_CONFIG_NOSYSTEM=1, no GIT_SSH_COMMAND or GIT_ALLOW_PROTOCOL, EMAIL set. Started at 45G free or more.

The 14 files (646 tests): test_remove_add_remedies_family_root 45, test_family_add_member_id 14, test_family_bump_to_checks 6, test_readme_windows_quoting_sentence 44, test_repo_hygiene 101, test_readme_sections 76, test_setup_by_hand_commands 5, test_agents_md_codes 47, test_update_add_codes_documented 6, test_family_docs_one_spelling 32, test_family_printed_quoting 84, test_git_bash_dialect 93, test_adopt_make_reads_first 33, test_doctor_plan_overrides 60.

run tree result
PR head 219e57c2, the 14 files 8d9f3a712 646 passed, 0 skipped, 346 s
scratch merge of 219e57c2 and main ac5f28d2a (the main of my first pass), the 14 files dfa379dbc 646 passed, 0 skipped, 358 s
scratch merge 103cbc40c of 219e57c2 and current main ef619c74a, the 14 files ec97278a4 646 passed, 0 skipped, 276 s
same merge, 7 more family files that run add, bump, remove and #1098's new code: test_family_leg_path_by_hand_bytes, test_bump_remedies_family_root, test_family_holder_remedies_compose, test_family_add_refusals, test_family_tracking_branch, test_family_bump_remove_rollback, test_validate_family_duplicates ec97278a4 297 passed, 0 skipped, 417 s

A host reboot killed one run (the 7-file family run on the old merge); I re-ran it on the merge with current main.

Mutants

Own handbook mutants, in a separate worktree at 219e57c2. Each file was restored and its sha256 checked (9ad794c4... before and after each).

id mutation result
H1 one of the 7 markers carries 2085e8ac (the head's pre-merge hash) caught: test_the_handbook_follows_the_readmes_outline fails, naming the masthead marker
H2 all 7 markers carry ffaedeef (main's pre-merge hash) caught: the same test fails
H3 the handbook's Windows sentence without remove, markers untouched caught: test_the_handbooks_windows_panel_says_the_same_words fails

3 of 3 caught. I ran no mutants of the PR's code guards: the code is byte-identical to what RV1117 mutated (9 of 9 caught).

Findings

None. If a README pull request changes "Starting a project in a new organisation" before this one lands, its 7 markers must be recomputed again; that is the known serialization, not a defect here.

Lane: openreposhape-4 (openRepoShape-4)

brettheap added a commit that referenced this pull request Nov 8, 2026
…ed into a folder this user cannot search is said once in `what is here` or `agent files`, `<name>: cannot be read (<reason>)`, never absent; a `.git` offers no way in (#1110) (#1144)

Eight sites in `shape-doctor.py` and `scripts/shape_advisory.py` asked `exists()` or `is_file()` of a name directly in the root, and both calls follow a link. So a `.git` or an agent file that is a link into a folder this user cannot search made the doctor raise on Python 3.12, and, as 3.14's `is_file` answers False to any `OSError`, read as absent there. On 3.12 the doctor said `FINDING .git: cannot be read` four times over, and an unreadable `AGENTS.md` added `agent-files` to the verdict. Under a 3.14-like shim it said `there is no repository here to adopt` and offered the scaffold.

- Each site now asks the file's own stat helper, which applies #430's rule: `stat` the path. ENOENT, ENOTDIR, EBADF and ELOOP (and Windows' 21, 123 and 1921) mean nothing is there, and any other `OSError` is raised so the caller can name it. The helper is `is_a` in `shape-doctor.py` and `_is_a` in `scripts/shape_advisory.py`. The four `.git` sites in `shape-doctor.py` share `carries_git(root)`, which accepts a gitdir file, a directory or a link to either.
- Such a file is said once, in `what is here` (a directory that is no shape root) or `agent files` (in a project), as `<name>: cannot be read (<reason>)` with the chmod cure, never as absent. In `agent files` it is a `note`, and a FINDING only for the pinned `AGENTS-shape.md`, which the project's own `validate-pins.py` refuses by name.
- An unreadable `.git` offers no way in, since it is not known whether there is a repository to adopt. `the way in` has no NEXT and its `detail.git_repository` is `null`. `naming` keeps the policy's own status and says that `origin`'s name was not read. `origin_org`, `repo_local_origin_name` and `repository_names` raise the refused `stat`, and `shape_advisory.read()` keeps its `not-answered` state, whose `why` now names the `.git` and its reason.
- `tests/test_doctor_unsearchable_links.py` holds 25 rows, and the eight sites leave `KNOWN_OPEN` in `tests/test_path_guard_hygiene.py`.
- Measured with Python 3.12.3 and a 3.14-like `sitecustomize` shim, over a folder `Atlas` whose `.git` links to the git directory of a checkout of `acme/Orbit` in a folder at mode 000:
  - On base, 3.12: `naming`, `what is here`, `the way in` and `the other way in` are each `FINDING .git: cannot be read (Permission denied)`, with no Triad row, and `NOT A SHAPE ROOT`, exit 2.
  - On base, the shim: `the way in` reads `there is no repository here to adopt`, with NEXT `setup.sh --org <your-org> --project Atlas`.
  - On head, 3.12 and the shim give byte-identical reports (`cmp`): the cure is said once, in `what is here`, `the way in` offers neither command, and `naming` says that `origin`'s name is not read. Exit 2.
  - The new rows: base 2 passed and 23 failed (the 2 are controls), on 3.12 and under the shim; head 25 passed on both. Mutants: 47 of 47 killed, the writer's 18 and the review's 29.
- Known limits:
  - An unreadable leg `AGENTS.md` is still read as absent where `_leg_by_agents_md` reads it (`scripts/shape_advisory.py`), so `the way in` and the Triad row still offer adoption over it, as on main. It is not one of #1110's eleven sites, and #1110 lists it as a twelfth. `_leg_by_agents_md` is unchanged.
  - A `.git` that is a gitdir file pointing into a folder this user cannot search, and a root `.git` directory at mode 000, still read as a repository, so adoption is offered as before.
  - Python 3.14 itself and Windows were not run. The 3.14 statements come from the shim, and the mode rows skip on Windows and as root.
  - `NOT_ANSWERED`'s new `why` is printed by nothing, so it is reachable only in process, where the rows hold it.
- Not in this PR: slice B's three sites (`scripts/family.py` `_gitmodules_names` and `_sections_giving`, and `templates/family-root/scripts/siblings.py` `submodule_urls`) stay on `KNOWN_OPEN` under #1110, waiting for other lanes' open DRAFTs on `family.py`'s `remove` (#1117 and #1098). A shape root whose own `.git` is a link into a folder this user cannot search still reads DRIFTED, and both of its pinned validators crash. Base and head behave alike, and it is filed as #1146.

Review: lane 1's independent reviewers reviewed each round. r1144 found `eb3f098c` READY on condition, with no MUST, two SHOULDs and six NITs (#1144 issuecomment-6328153279). The SHOULDs were that the title and design point 1 claimed too much for an agent file, and that seven of the reviewer's mutants survived. The round after it narrowed the title, added 8 rows and left the two tools unchanged, and a last commit split two composite asserts for SonarCloud's `python:S9073`. r1144b found `89486f69` READY, with no MUST or SHOULD (issuecomment-6331490548). Each of the seven survivors is killed by its named row, and all 47 mutants are killed.

Tests, under a stub `gh` with a bogus token: at `89486f69`, the gate in the PR body gave 1,141 passed and 3 skipped over its 9 rows. They covered the PR's test file, on 3.12 and under the shim, `tests/test_path_guard_hygiene.py`, `tests/test_shape_doctor.py` in 4 shards, `tests/test_scaffold_leg_path_collisions.py`, `tests/test_doctor_unreadable_paths.py` with `tests/test_doctor_unreadable_pin.py`, the other files that import or run `shape_advisory`, the doc trio, and `tests/test_doctor_unreadable_paths.py` under the shim.

It lands by fast refresh (Brett Heap, 2026-10-15T22:48:35Z). Its H0 `89486f69` is green on Linux and SonarCloud. The branch was never rebased. After H0, the only commit is update-branch's clean merge of main, giving 5b38c3a, which is green on SonarCloud and Git Bash. Lane 1's gate passed on its tree: 7 pytest calls over `tests/test_doctor_unsearchable_links.py`, `tests/test_path_guard_hygiene.py`, `tests/test_agents_md_codes.py`, `tests/test_cli_reference.py`, `tests/test_repo_hygiene.py`, `tests/test_scaffold_leg_path_collisions.py`, `tests/test_no_auto_maintenance.py` and `tests/test_shape_doctor.py` in 4 shards: 925 passed, 5 skipped.

Part of #1110

Lane: openreposhape-1 (openRepoShape-1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Nov 8, 2026
…py: an `id:`, a date or a `repository:` ending in a line feed is refused by name, and a pin's `source_repository:` that is no path or `owner/repo` name is named as the pin's (#1139) (#1166)

The value patterns the issue names refuse a value that ends in a line feed, by name, as #1126 (#1017) makes `COMMIT_RE`, `SHA256_RE` and `COMMIT_40` do:
- `DATE_RE` and `QUALIFIED_RE` in `validate-manifest.py` and `validate-family.py`, `MEMBER_RE` in `scripts/family.py`, `DATE_RE` in `scripts/shape_advisory.py` and `REPOSITORY_RE` in `update-shape.py`;
- `PROJECT_ID_RE`, read with `fullmatch` where the two validators and `family.py add` read an id from a file.

A shape pin's `source_repository:` that is neither a path on this machine nor an `owner/repo` name is refused in the pin's name, not `--upstream`'s. Values without a line feed keep their outcome, and no refusal code is added or renamed. Two inputs do move from one existing code to another (see "Behaviour changed", below).

**What this PR does not cover, and the title does not claim:**
- the naming-policy patterns that `scripts/repo_shape.py` reads from `contracts/repository-naming.yaml`, which `validate-repository-naming.py` still reads with `.match`;
- `update-upstream-not-a-repo`, which still says `--upstream` when a pin names a folder that exists and is not a repository.

Both are follow-ups, with the other findings of the independent review, under "Follow-ups (not in this PR)".

This does not need #1126 and does not touch its files or its test file.

Measured on main `038dafba` (Python 3.12.3, git 2.43.0) by a probe of each tool, before any change. Each value is written in the file as a YAML double-quoted `"...\n"`, which the reader takes as the value and a line feed.

| Read from | Value | On main |
|---|---|---|
| `project.yaml` (`validate-manifest.py`) | `id: "atlas\n"` | only `FINDING manifest-topic: topic is 'xf-project-atlas', expected 'xf-project-atlas\n' derived from id 'atlas\n'`, exit 1 |
| `project.yaml` | `elected_on: "2026-09-02\n"` | `manifest ok`, exit 0 |
| `project.yaml` | a leg's `repository: "testorg/Atlas-spec\n"` | `manifest ok`, exit 0 |
| `family.yaml` (`validate-family.py`) | `id: "inkrouter\n"` | `family ok: InkRouter (inkrouter` and a line break, exit 0 |
| `family.yaml` | `created_on: "2026-09-04\n"` | `family ok`, exit 0 |
| `family.yaml` | a member's `repository: "InkRouter/IRRS\n"` | `FINDING member-repository-name: ... whose name is 'IRRS\n'`, exit 1 |
| a member's `project.yaml`, read by `family.py add` | `id: "irqs\n"` | mounted and committed, exit 0, with the row `id: "irqs\n"`; the holder's gate then said `family ok` |
| `contracts/shape-pin.yaml`, `update-shape.py check` with no `--upstream` | `source_repository: "opensoft/openRepoShape\n"` | `REFUSED unsafe-value: --upstream is 'opensoft/openRepoShape\n'`, exit 2 |
| the same | `source_repository: "opensoft/open Repo"` | `REFUSED update-upstream-unresolvable: --upstream 'opensoft/open Repo' is neither ...`, exit 2 |
| the same | `source_repository: "-opensoft/openRepoShape"` | `REFUSED unsafe-value: --upstream is '-opensoft/openRepoShape'`, exit 2 |
| an adoption plan (`adopt-project.py check`) | `id: "northwind\n"` | `FINDING plan-manifest-invalid: id is 'northwind\n', which the project's own validate-manifest.py refuses: manifest-topic: ...`, exit 1; `execute` refused it, exit 2, before any leg |

In-process on the same main: `repo_shape.PROJECT_ID_RE.match('atlas\n')`, `shape_advisory.DATE_RE.match('2026-09-02\n')` and `REPOSITORY_RE.match('a/b\n')` each matched, and `date.fromisoformat('2026-09-02\n')` raised `ValueError`.

- **`DATE_RE` and `QUALIFIED_RE`** in `templates/assembly-root/scripts/validate-manifest.py` and `templates/family-root/scripts/validate-family.py` end in `\Z`. So do `MEMBER_RE` in `scripts/family.py` (the third spelling `tests/test_family_member_name.py` holds equal to the two `QUALIFIED_RE`), `DATE_RE` in `scripts/shape_advisory.py` and `REPOSITORY_RE` in `update-shape.py`.
- **`PROJECT_ID_RE` keeps its `$`.** Every id refusal prints its text (`must match ^[a-z0-9][a-z0-9-]*$`, pinned in six test files), the validators run against each project's own copy of `repo_shape.py`, and its definition sits inside #1126's hunk. The three callers that read an id from a file ask it with `fullmatch` instead: `validate-manifest.py` (`manifest-id`), `validate-family.py` (`family-id`) and `family.py`'s reading of a member's `project.yaml` (`member-bad-id`, which `add` now says before anything is mounted).
- **`update-shape.py`.** `open_upstream(spec, pin_path=None)` takes the pin the value was read from, which `prepare` passes when no `--upstream` was given. Then `update-upstream-unresolvable` says ``<root>/contracts/shape-pin.yaml: `source_repository:` is '...', which is neither a path on this machine nor an `owner/repo` name``, with a remedy to write the pin's field or pass `--upstream`, and `checked_value` names the same field. **Code kept:** `update-upstream-unresolvable`, the existing code for a value that is not an `owner/repo` name. A line feed now reaches it, since the pattern no longer takes one. Before, `checked_value` refused it as `unsafe-value`.
- **One existing test line:** `tests/test_family_init_values.py` finds validate-family's `DATE_RE` line by its text, which now ends `\Z`.
- **One existing stand-in:** the `temporaries` fixture in `tests/test_update_shape_unreadable.py` stands in for `open_upstream` with `def opening(spec)`, which the new two-argument call refused with `TypeError: ... takes 1 positional argument but 2 were given`. It now takes `pin_path=None` and hands it on to the real `open_upstream`, so `test_a_read_refused_in_prepare_closes_the_upstream[check|apply]` asks what it asked before.
- **The rows** are in a new file of their own, `tests/test_value_pattern_line_break.py`.
- **A second guard, in a second commit (`5102b4294`, a test only, no production file).** The first guard holds the four pattern names this PR ends in `\Z`, and nothing held `PROJECT_ID_RE`, which keeps its `$`. `test_every_reader_of_the_id_pattern_is_in_the_table` reads every tracked `.py` outside `tests/` with `ast` (it runs none) and holds each call of a method of `PROJECT_ID_RE` (`PROJECT_ID_RE.x(...)` or `repo_shape.PROJECT_ID_RE.x(...)`) to the table `ID_READERS`:
  - the three `fullmatch` readers of an id from a file (`validate-manifest.py`, `validate-family.py`, `scripts/family.py`);
  - the modules that still `match` it, each with its reason: argv `--id` in `scaffold-project.py` and `scripts/family.py`, where `checked_value` refuses a line feed after the pattern; argv `--id` in `adopt-project.py`, which the project's own `validate-manifest.py` refuses (`adopt-manifest-invalid`, below; `ID_READERS` still gives it the `checked_value` reason); and `adopt-project.py`'s two `plan-bad-id` readers, of the plan's `id:` and of the id derived from `legs.assembly`, which a line feed never passes (below).

  A new reader of `PROJECT_ID_RE`, `fullmatch` included, fails it by file and method until it is listed in `ID_READERS` with its reason. The table is exact both ways, so a reader taken away fails it too.

**Left as they are, and why:**
- the argv readers of `PROJECT_ID_RE` (`--id` in `scaffold-project.py`, `family.py init` and `adopt-project.py`). The first two hold the id to `checked_value` after the pattern, which refuses a line feed as `unsafe-value` naming `--id`, and `tests/test_derived_id_refusals.py` pins that answer for them. `adopt-project.py` never calls `checked_value` on `--id`: `plan` with `--id` set to `northwind` and a line feed exits 2 with `adopt-manifest-invalid`, in the words of the project's own `validate-manifest.py` (`manifest-id`), and writes no plan (lane 5's re-check, issuecomment-6371408915, NIT 2; measured on `plan` only);
- `adopt-project.py`'s reading of a plan's `id:`: `check` and `execute` hold it to the project's own `validate-manifest.py`, which refused it on main (above) and now names it in `manifest-id`'s words. It is the same value, refused under the same code, `plan-manifest-invalid`, before any leg exists. The id derived from `legs.assembly` is read with `.match` too, and a line feed in `legs.assembly` is `unsafe-value` before that id is asked (the review measured both, on main and here alike). This keeps the open PRs on `adopt-project.py` (24 at 12:02Z) clear;
- `PIN_ARG_RE`, and `NAME_RE`/`REF_RE`/`PATH_RE` in `setup-project.py`, as the issue says;
- `update-upstream-not-a-repo` still says `--upstream`. A pin's value reaches it only when it names a path that exists, not through the pattern.

**Behaviour changed, stated plainly, and what the issue's Verify means by "No refusal code changes":** the issue's Verify line reads as no code added or renamed, and none is. Two inputs do move from one existing code to another, exit 2 both before and after:
- `update-shape.py --upstream "opensoft/openRepoShape<LF>"` was `unsafe-value` and is now `update-upstream-unresolvable`, still naming `--upstream`;
- a pin's `source_repository:` that ends in a line feed, with no `--upstream`, was `unsafe-value` naming `--upstream` and is now `update-upstream-unresolvable` naming the pin's field.

No test pinned either old answer: every gate below passed. `family.py add --member` is stripped before `MEMBER_RE`, so `add` answers as before.

**Not changed, seen while measuring:** with `--upstream` passed, `update-shape.py` does not read the pin's `source_repository:` through `REPOSITORY_RE` at all, and it keeps the value as the upstream's repository name. `validate-family.py` does not hold a member row's `id:` to `PROJECT_ID_RE`; it compares the row with the member's manifest. Neither is a `$` question. Both are under "Follow-ups (not in this PR)".

All in `tests/test_value_pattern_line_break.py`, run against main's production code: the five production files swapped in place from `origin/main` (`961640c68`), then put back with their sha256 matching (`dirty=5` in the run's START line is those five files). On the head `5102b4294` (30 rows) the result is **20 failed, 10 passed in 23.12s, git 2.43.0**. The 10 that pass are baselines: the seven `test_a_pattern_takes_its_value` rows, the spelling-table guard and the two double-quoted-without-a-line-feed rows.

The first 19 are the first commit's rows (19 failed, 10 passed in 22.12s on its 29 rows at `038dafba`). The independent review reproduced them from main's bytes of the same five files (19 failed, 10 passed in 26.78s, the same 19 ids). The 20th, `test_every_reader_of_the_id_pattern_is_in_the_table`, is the second commit's: it fails on main because the three readers of an id from a file are `.match` there, not `.fullmatch`.

- `test_a_pattern_refuses_its_value_and_a_line_feed[manifest-DATE_RE]`
- `test_a_pattern_refuses_its_value_and_a_line_feed[manifest-QUALIFIED_RE]`
- `test_a_pattern_refuses_its_value_and_a_line_feed[family-DATE_RE]`
- `test_a_pattern_refuses_its_value_and_a_line_feed[family-QUALIFIED_RE]`
- `test_a_pattern_refuses_its_value_and_a_line_feed[family.py-MEMBER_RE]`
- `test_a_pattern_refuses_its_value_and_a_line_feed[shape_advisory-DATE_RE]`
- `test_a_pattern_refuses_its_value_and_a_line_feed[update-shape-REPOSITORY_RE]`
- `test_a_manifest_value_that_ends_in_a_line_break_is_a_finding[id]`
- `test_a_manifest_value_that_ends_in_a_line_break_is_a_finding[elected_on]`
- `test_a_manifest_value_that_ends_in_a_line_break_is_a_finding[leg-repository]`
- `test_adopt_check_names_a_plan_id_that_ends_in_a_line_break_as_an_id`
- `test_a_family_value_that_ends_in_a_line_break_is_a_finding[id]`
- `test_a_family_value_that_ends_in_a_line_break_is_a_finding[created_on]`
- `test_a_family_value_that_ends_in_a_line_break_is_a_finding[member-repository]`
- `test_add_refuses_a_member_whose_id_ends_in_a_line_break`
- `test_a_pins_source_that_is_no_owner_repo_is_named_as_the_pins[line-feed]`
- `test_a_pins_source_that_is_no_owner_repo_is_named_as_the_pins[space]`
- `test_a_pins_source_checked_value_refuses_is_named_as_the_pins`
- `test_an_upstream_passed_with_a_line_feed_is_named_as_passed`
- `test_every_reader_of_the_id_pattern_is_in_the_table`

**13 of 13 killed.** Each mutant was applied alone to one file of the branch's production bytes (the same bytes as the first commit and as the head, which adds a test only), the rows were run, and the file was put back with its sha256 checked (`restored=yes` for all 13). Each run is `tests/test_value_pattern_line_break.py` plus `tests/test_family_member_name.py::test_every_spelling_of_the_member_name_rule_is_the_same_pattern`, git 2.43.0.

| Mutant | File | Killed by |
|---|---|---|
| M1 `DATE_RE` back to `$` | `validate-manifest.py` | 2: `test_a_pattern_refuses_its_value_and_a_line_feed[manifest-DATE_RE]`, `test_a_manifest_value_that_ends_in_a_line_break_is_a_finding[elected_on]` |
| M2 `QUALIFIED_RE` back to `$` | `validate-manifest.py` | 3: `...[manifest-QUALIFIED_RE]`, `...[leg-repository]`, `test_every_spelling_of_the_member_name_rule_is_the_same_pattern` |
| M3 the id read with `.match` | `validate-manifest.py` | 2: `test_a_manifest_value_that_ends_in_a_line_break_is_a_finding[id]`, `test_adopt_check_names_a_plan_id_that_ends_in_a_line_break_as_an_id` |
| M4 `DATE_RE` back to `$` | `validate-family.py` | 2: `...[family-DATE_RE]`, `test_a_family_value_that_ends_in_a_line_break_is_a_finding[created_on]` |
| M5 `QUALIFIED_RE` back to `$` | `validate-family.py` | 3: `...[family-QUALIFIED_RE]`, `...[member-repository]`, `test_every_spelling_of_the_member_name_rule_is_the_same_pattern` |
| M6 the id read with `.match` | `validate-family.py` | 1: `test_a_family_value_that_ends_in_a_line_break_is_a_finding[id]` |
| M7 `MEMBER_RE` back to `$` | `scripts/family.py` | 2: `...[family.py-MEMBER_RE]`, `test_every_spelling_of_the_member_name_rule_is_the_same_pattern` |
| M8 a member's id read with `.match` | `scripts/family.py` | 1: `test_add_refuses_a_member_whose_id_ends_in_a_line_break` |
| M9 `DATE_RE` back to `$` | `scripts/shape_advisory.py` | 1: `...[shape_advisory-DATE_RE]` (its pattern row alone: `date.fromisoformat` refuses the line feed after the pattern, so no tool row sees it) |
| M10 `REPOSITORY_RE` back to `$` | `update-shape.py` | 3: `...[update-shape-REPOSITORY_RE]`, `test_a_pins_source_that_is_no_owner_repo_is_named_as_the_pins[line-feed]`, `test_an_upstream_passed_with_a_line_feed_is_named_as_passed` |
| M11 `prepare` passes no pin | `update-shape.py` | 3: `test_a_pins_source_that_is_no_owner_repo_is_named_as_the_pins[line-feed]` and `[space]`, `test_a_pins_source_checked_value_refuses_is_named_as_the_pins` |
| M12 `checked_value` names `--upstream` again | `update-shape.py` | 1: `test_a_pins_source_checked_value_refuses_is_named_as_the_pins` |
| M13 the pin's refusal sentence removed | `update-shape.py` | 2: `test_a_pins_source_that_is_no_owner_repo_is_named_as_the_pins[line-feed]` and `[space]` |

No mutant is added for the stand-in in `tests/test_update_shape_unreadable.py`: it is a test change.

The independent review re-ran M3, M8, M11 and M12, and one mutant of its own (X1: `prepare` names the pin even when `--upstream` is passed). All five were killed, and each file was put back with its sha256 checked. The other nine of the 13 are this writer's runs alone.

**The second commit's guard: 6 of 6 probes caught.** Each was applied alone to one tracked file at `5102b4294`, `test_every_reader_of_the_id_pattern_is_in_the_table` was run, and the file was put back with its sha256 checked (`restored=yes` for all six). The first commit's guard, `test_every_spelling_of_a_value_pattern_is_in_the_table`, was run with each plant too: it passes every one of them, which is the gap the second guard closes.

| Probe | File | The new guard | The first guard |
|---|---|---|---|
| P1 a new `PROJECT_ID_RE.match(value)` reader appended | `update-shape.py` | failed: `('update-shape.py', 'match'): 1` is more than the table holds | passed |
| P2 a new `repo_shape.PROJECT_ID_RE.search(value)` reader appended | `scripts/shape_advisory.py` | failed | passed |
| P3 `fullmatch` back to `match` | `templates/assembly-root/scripts/validate-manifest.py` | failed | passed |
| P4 `fullmatch` back to `match` | `templates/family-root/scripts/validate-family.py` | failed | passed |
| P5 `fullmatch` back to `match` | `scripts/family.py` | failed | passed |
| P6 main's bytes of those three files together (the 20th row above) | the three | failed | not run |

**The second commit `5102b4294` (dirty=0)**, Python 3.12.3, with the same stub `gh`, bogus token and no global git config:

| Run | Git | Result |
|---|---|---|
| `tests/test_value_pattern_line_break.py` | 2.43.0 | **30 passed** in 23.00s |
| `tests/test_value_pattern_line_break.py` | 2.47.1 | **30 passed** in 23.30s |
| `tests/test_pinned_copy_text.py`, `tests/test_path_guard_hygiene.py`, `tests/test_repo_hygiene.py`, `tests/test_agents_md_codes.py`, `tests/test_cli_reference.py` | 2.43.0 | **306 passed** in 61.54s |

That commit changes only the test file (62 lines added, no assert that joins conditions with `and` or `or`, ASCII), so the first commit's runs below are not repeated: they hold the production files and every other test file, and those are unchanged. CI on `5102b4294`, and an independent look at the second commit, were not done when this was written.

**The independent review of the first commit** (rv1166c, READY, read-only) reports these gates on `2593952c9`, all passing: the stand-in gate's 6 files (298 passed), the new file on 2.47.1 (29 passed), the five hygiene files (306 passed), 15 more test files that name a changed function, pattern or code (1611 passed, 19 skipped) and `tests/test_shape_doctor.py` (445 passed, 3 skipped). It also read CI on that head at about 20:04Z: `tests` (the full suite on 3.14), `tests-git-bash`, `package` and SonarCloud all passed.

**The first commit's gates ran on `2593952c` (dirty=0)**, Python 3.12.3, with the real `gh` unreachable (a stub first on `PATH`, a bogus token) and no global git config:

| Run | Git | Result |
|---|---|---|
| the files that held every failure and setup error of the shards below: `tests/test_family.py`, `tests/test_update_shape_pr_base.py`, `tests/test_update_shape_unreadable.py`, `tests/test_update_shape_header_comments.py`, `tests/test_update_shape_messages.py` and the new `tests/test_value_pattern_line_break.py` | 2.43.0 | **298 passed** in 343.60s |
| `tests/test_value_pattern_line_break.py` | 2.47.1 | **29 passed** in 21.31s |
| `tests/test_pinned_copy_text.py`, `tests/test_path_guard_hygiene.py`, `tests/test_repo_hygiene.py`, `tests/test_agents_md_codes.py`, `tests/test_cli_reference.py` | 2.43.0 | **306 passed** in 69.12s |

**The two shards, on the uncommitted tree** holding the same production bytes as the first commit (the 7 files' sha256 checked before and after), git 2.43.0: the validators' test files, the `REPOSITORY_RE`/`source_repository` files and the `update-shape.py`, `shape_advisory.py` and `family.py` files, 125 in all.

- shard 1 (62 files): 4 failed, 5290 passed, 13 skipped in 56 min;
- shard 2 (63 files): 3 failed, 4 setup errors, 3761 passed, 29 skipped in 50 min.

Every failure and setup error is in the five existing files of the first row of the table above, which all pass on the first commit:
- 2 were real: `test_a_read_refused_in_prepare_closes_the_upstream[check|apply]`, the `opening(spec)` stand-in under the two-argument call. The stand-in is fixed in the first commit.
- The other 5 failures and the 4 setup errors were `update-conflict` on `scripts/validate-family.py` ("not a verbatim copy ... at the pinned commit"). This is an artifact of an uncommitted tree: `tests/test_family.py`'s `upstream_clone` commits the working tree into its clone, while `family.py init` pins the holder at the real `HEAD`, so an uncommitted `templates/family-root/` file is never verbatim at its pin. On the first commit they pass.

Every other file of the two shards passed on these production bytes, so the shards were not re-run.

`git merge-tree --write-tree <other> 5102b42` (git 2.43.0) at 2026-10-23T12:08Z, after re-fetching main (still `961640c6`; read again at 12:10Z, just before the push) and re-listing the open PRs at 12:02Z (55 open). Main has moved from the `038dafba` that this body measured to `961640c6` by one file, `docs/release-notes/v1.2.0.md`. Each open PR that touches one of this PR's 8 paths, at its current head, plus #1126:

| Against | Head | Shares | Result |
|---|---|---|---|
| origin/main | `961640c6` | | clean, tree `4613729516ff` |
| #1098 | `dad7961a` | `scripts/family.py` | clean, tree `4d1ff9517588` |
| #1117 | `10dc011c` | `scripts/family.py` | `scripts/family.py` merges clean; conflict in `docs/handbook.html` only, which this PR does not touch (see below) |
| #1124 | `8dfa32bd` | `validate-family.py` | clean, tree `cb2ec1073646` |
| #1126 | `4c1f80cf` | none (its `scripts/repo_shape.py` and tests) | clean, tree `e82c8945ac37` |
| #1133 | `a7bfee1b` | `update-shape.py` | clean, tree `a710870d1ffc` |
| #1144 | `89486f69` | `scripts/shape_advisory.py` | clean, tree `2d3e3e1ef43d` |
| #1147 | `b75babee` | `validate-manifest.py` | clean, tree `a82ac3b47a3f` |
| #1159 | `66e68f94` | `scripts/shape_advisory.py` | clean, tree `4abeb7b38665` |
| #1164 | `74096eb4` | `scripts/family.py` | clean, tree `ee285e1cb8cf` |
| #1172 | `5aa0f116` | `validate-family.py` | clean, tree `aa32d41e6e2c` |
| #1187 | `cc8a87ee` | `update-shape.py`, `tests/test_update_shape_unreadable.py` | clean, tree `25711eb1cac4` |
| #1201 | `eac177af` | `scripts/family.py` | `scripts/family.py` merges clean; conflict in `docs/handbook.html` only (see below) |

- #1117 and #1201 conflict in `docs/handbook.html` alone. #1201 is stacked on #1117 (their merge-base is #1117's head `10dc011c`). Each of them against origin/main alone gives the same conflict in the same file, so it is theirs: #1117's, from its older base `6da757cb`, which #1201 inherits.
- #1187 and #1201 are new since the first merge-trees. #1187 shares `tests/test_update_shape_unreadable.py` with this PR; its diff adds or removes no line that names `open_upstream`, `def opening` or `REPOSITORY_RE`, so it brings no one-argument stand-in for the two-argument call. That was read for all 55 open heads at 12:12Z; only this PR's own lines name them.
- No other open PR adds or removes a call of a method of `PROJECT_ID_RE`, which the second guard counts: the lines each of the 55 open heads adds or removes outside `tests/`, `docs/` and `*.md` were read at 12:04Z. Only this PR's own name it, and #1201's one new `MEMBER_RE.fullmatch` is another pattern. So a sibling landing first does not move the guard's table.

Each is a candidate for its own issue. They come from the independent review (rv1166c, READY at `2593952c9`, issuecomment-6352173084), which measured each of them on main and at the head unless the item says it did not. The review's finding numbers are in brackets.

- **The naming-policy patterns [1].** `scripts/repo_shape.py` reads the families, the roles and the topic pattern of `contracts/repository-naming.yaml` with `.match` (`family["_re"].match(name)`, `family["_split_re"].match(name)`, `role["_re"].match(name)`, and `topic_pattern`, which defaults to `^xf-project-[a-z0-9-]+$`). `validate-repository-naming.py --project project.yaml` runs in the root template's `Makefile` and `validate.yml`. At the head, and on main, it exits 0 on a leg `repository: "testorg/Atlas\n"` and prints the line feed inside its table; `-spec` and `-code` the same. The project's gate as a whole still fails, because `validate-manifest.py` now refuses that manifest.
- **`update-upstream-not-a-repo` with a pin [3].** With no `--upstream`, a pin whose `source_repository:` names a folder that exists and is no repository (the review measured `contracts` and `scripts/repo_shape.py`) is `REFUSED update-upstream-not-a-repo: --upstream contracts exists but is not a git repository`, exit 2, on main and here. `pin_path` is in scope at `update-shape.py:686-687`, so that refusal can name the pin as `update-upstream-unresolvable` now does.
- **A pin's line feed when `--upstream` is passed [9].** `update-shape.py check --upstream <clone>` over a pin whose `source_repository:` ends in a line feed exits 0, `nothing to do`. The value is kept as `upstream.repository`, which reaches the README shape line's check, `apply`'s commit subject and the pull request words (`update-shape.py:2525`, `3017` and `5121`). The review measured `check`, not `apply`. It belongs with the previous item.
- **A member row's `id:` [4].** On main, `add` of a member whose `project.yaml` reads `id: "irqs\n"` commits the row `id: "irqs\n"`, and the holder's `validate-family.py` then says `family ok`. Under mutant M8 (this branch with main's `.match` in `add`) the same row is written, and the gate still says `family ok`. By code reading, not measured end to end: `validate-family.py` compares a row's id with the member manifest's by `str` equality (`member-id-mismatch`, `validate-family.py:1608`), and `_check_member_ids_unique` does the same, so neither asks `PROJECT_ID_RE` of a row. This PR makes `add` refuse such a member, so no new holder gets such a row.
- **Values that no pattern reads [8].** The review's sweep found these `manifest ok` or `family ok`, exit 0, with a trailing line feed, at the head and on main alike:
  - in `project.yaml`: `name`, a spec or code leg's `path`, `shape.repository` and `shape.digests.digest_algorithm`;
  - in `family.yaml`: `org`, the family's own `repository`, `tracking_branch`, and the shape's and each member pin's `digest_algorithm`.

  No `^...$` pattern reads them, so they are not this PR's. A leg's `path` is checked only by `_leg_path_findings`, which asks no pattern, and the family's `tracking_branch` is refused in `project.yaml` (`manifest-tracking-branch`, U+000A) but not in `family.yaml`. `shape.commit` and `tree_sha256` with a line feed are also `manifest ok`; that is #1126's.

The first commit's message says "the three readers of an id from a file ask it with `fullmatch` (`manifest-id`, `family-id`, `member-bad-id`)". That is true of those three, but it reads as if there were no others [2]. What the review measured instead:
- `adopt-project.py` still reads the id derived from `legs.assembly` (`:6780`) and the plan's `id:` (`:6791`), both `plan-bad-id`, with `.match`;
- neither lets a line feed through. A plan `id: "northwind\n"` is `plan-manifest-invalid`, now in `manifest-id`'s words. A plan with no `id:` and `legs.assembly: "NorthWind\n"` is `REFUSED unsafe-value: legs.assembly is 'NorthWind\n'`, exit 2, before the derived id is asked, on main and here alike.

So it is the wording alone that overstates. The second commit's `ID_READERS` lists both readers with that reason.

The title's pin clause is the narrower one (the pin's name only where the value is no path and no `owner/repo` name), and two inputs change from `unsafe-value` to `update-upstream-unresolvable` (see "Behaviour changed", above).

Review: rv1166 and rv1166c, lane 5's independent opus reviewers, at `2593952c` (issuecomment-6352173084); then rc1166b, lane 5's independent sonnet re-checker, which wrote none of #1166: READY at `5102b429`, H0 of this landing (issuecomment-6371408915), with no MUST or SHOULD; its NITs 1 and 2 are corrected in the body. CI (Brett Heap's ruling 27 and his fast-refresh ruling of 2026-10-15T22:48:35Z): H0 5102b42 passed tests, tests-git-bash and SonarCloud; the head 5c8afef is a clean update-branch merge of main 93d2931, with tests-git-bash and SonarCloud green; and the local gate passed on its predicted squash tree 3c89065 over main 11560eb: the family_init_values, update_shape_unreadable, value_pattern_line_break, repo_hygiene, agents_md_codes, cli_reference sets, 365 passed, 19 skipped. Landed on Brett Heap's word of 2026-10-23T13:10:26Z, relayed to every lane by lane 3 (#473 issuecomment-6365697123): "merge v1.2.1 PRs once reviewed and green".

Closes #1139

Lane: openreposhape-5 (openRepoShape-5)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Nov 8, 2026
…s message spells a path byte that is not UTF-8 as `_shown` does, an escape, and no longer says it names it with a `?` in its place, which #900 replaced (#1018) (#1104)

`env_commit`'s docstring in `scripts/shape_materialize.py` says what the split commit's message does with a `root` or `drop` path that holds a byte that is not UTF-8, as the code does since #900. It said the message names such a path "with a `?` in place of that byte, a path that does not exist". That was true before #900, and it is not true now. A reader of the docstring is told about a `caf?.md` the tool no longer writes.

Found by lane 1's writer on #900 and filed as #1018. Lane 5 took it outside v1.2.0, under Brett Heap's ruling of 2026-10-21T10:32:04Z.

Measured on main `2a119f281` and again on `16f9a0e81` (main after #1087), with a small script that calls: `adopt-project.py`'s own `_split_message` and `_shown`, then `env_commit` and a real `git commit`, read back with `cat -A`.

- The issue's claim held: the docstring (about line 2103 when filed, line 2268 on `2a119f281`, line 2349 on this branch) still said the message names the byte with a `?`.
- The code does not. `_shown("caf\udce9.md")` is `` `caf\udce9.md` ``, and the split message for a `root` and a `drop` entry named that holds no lone surrogate. Through `env_commit` and `git cat-file commit HEAD`, `cat -A`:

  ```
  DROPPED (in no leg and no longer here):$
    `caf\udce9.md`$
  STAYS IN THE ASSEMBLY ROOT:$
    README.md$
    `caf\udce9.md`$
  ```
- What stays true of `env_commit` itself: a message that holds the lone surrogate, handed to it directly, is still written as `?`. `cat -A` reads `Name: caf?.md$`. So the `errors="replace"` paragraph is kept, and what changes is who it describes.
- #1087 (main `16f9a0e81`) changed `scripts/shape_materialize.py` by 128 lines, none of them in `env_commit`, `COMMIT_COMMAND` or the message, and changed nothing in `adopt-project.py` (0 diff lines). The rebase onto it was clean and the docstring needed no other change.

One file, `env_commit`'s docstring, in two paragraphs: the `errors="replace"` paragraph (round 1, 11 lines added and 7 removed) and the sentence above it (round 2, below; 3 added and 2 removed). Together 14 added and 9 removed against main `6da757cb`. The `errors="replace"` paragraph now says:

- encoding, a character UTF-8 cannot carry is written as `?`, and nothing raises after both legs are pushed, as before;
- `adopt-project.py` refuses a lone surrogate in a path it extracts (`entry_path_problem`), and a `root` or `drop` entry that holds one is listed in the split message but never raw: the message names each path as `plan`'s report does (`_shown`, #900), so the byte 0xE9 reads as an escape with no surrogate in it, `` `caf\\udce9.md` ``, the path the source holds;
- written raw it would be a `?`, `caf?.md`, a path that does not exist, which is what `env_commit` still writes for a lone surrogate it is handed.

The backslash is doubled in the source, as the neighbouring docstrings write `\\udcc3`. A single `\u` in a non-raw docstring would be an escape that makes a lone surrogate. Measured: the docstring holds no character from U+D800 to U+DFFF and holds the six characters `\udce9`. The longest docstring line is 78 characters.

**The code is unchanged.** A script compares the AST of `scripts/shape_materialize.py` with every docstring blanked: the branch's file is identical to `origin/main`'s after the rebase onto `16f9a0e81` (round 1), and the merge of this head into `origin/main` `6da757cb` is identical to `6da757cb`'s file (round 2).

**No pin moves.** `scripts/shape_materialize.py` is not a pinned copy. `COPIED_VERBATIM`, `COPIED_FROM_SHAPE`, `FAMILY_COPIED_VERBATIM` and `FAMILY_COPIED_FROM_SHAPE` do not list it, and `templates/assembly-root/scripts/validate-pins.py` says of it "which no project carries". So no `upstream-changed` row, no digest, no assembly copy and no `tests/test_pinned_copy_text.py` entry changes.

**No other text pins these words.** Searched in `tests/`, `README.md`, `docs/`, `docs/handbook.html`, `AGENTS.md` and `templates/`: no test reads `env_commit`'s docstring (`__doc__`, `getdoc`), and none of them holds "in place of that byte" or "a path that does not exist" for this. The other places that name the `?` are about this function's behaviour or about history, and are still true: `_split_message`'s docstring says `caf\\udce9.md` "was `caf?.md`" (before #900), `_refused_by_policy` already says the message "lists it as `_shown` does (#900)", and `tests/test_adopt_spelled_names.py` quotes the message as it was on `c7a3ff16`. The `?` in `argv_text`, `trailer_line` and the commit-trailer text is about a `--trailer`, not the split message.

No rows and no mutants: the change is a docstring, the code is the same AST, and no test pins the docstring. So there is no row to fail on main and nothing to mutate.

All with `$Q/v110/run9.sh`, `EMAIL=service@opensoft.dev`, `GIT_SSH_COMMAND=false`, `GIT_CONFIG_GLOBAL=/dev/null`, the stub `gh`, a fake token, git 2.43.0 (`/usr/bin/git`).

Two shards, run before the rebase, at the `2a119f281` base with this one-file change in the tree. Together they are every `tests/*.py` that names `shape_materialize` or `env_commit` (86 files, including `tests/test_adopt_spelled_names.py` and `tests/test_adopt_entry_paths.py`, which run `env_commit` and the #900 message) and `tests/test_agents_md_codes.py`:

| run | files | result |
|---|---|---|
| shard A | 43 | 2873 passed, 4 skipped in 1707.51s, rc=0 |
| shard B | 44 | 4611 passed, 31 skipped in 2462.01s, rc=0 |

After the rebase onto `16f9a0e81`, on the committed head `e32b2703d733f6c60c1b2cba2d62d164d93d7b9a` (`dirty=0`): the trio, `tests/test_repo_hygiene.py`, `tests/test_agents_md_codes.py` and `tests/test_cli_reference.py`, **170 passed in 55.67s**, rc=0. The docstring-only change needed no new shards: the shards' counts are runs at the `2a119f281` base, and #1087 did not touch `env_commit`.

Round 2, on the committed head `de89aa81b1533e32f1007f991aeae05cd915b36d` (`dirty=0`): the same trio, **170 passed in 61.77s**, rc=0. The round 2 commit is one docstring sentence, so it needed no new shards either.

git 2.47.1: no run. The change adds no test file of its own, so there is none to run on it.

`git merge-tree --write-tree HEAD <other>` with HEAD at `de89aa81b1533e32f1007f991aeae05cd915b36d`, after a re-fetch of `origin main` with `-c gc.auto=0 -c maintenance.auto=false` (round 2, about 14:03Z on 2026-10-21):

| against | result |
|---|---|
| `origin/main` `6da757cbe9a4eb66d60c34f38dcc2b332d175e8b` | clean, rc 0, tree `15c11b6c959fc4c7540105d5d62e3f961c8e27fd` |
| #1105 `fix/1048-makefile-follow-up-gnu-read-order` `5a4de7f763b5a3c32adce9614a3a539508416eb4`, an open PR that touches `scripts/shape_materialize.py` | clean, rc 0, tree `db05aead67052643942e35a9689d9dd4a0873fda` |
| #1118 `fix/1089-named-link-skip-tests` `c60066cbeec18b8b6c926e254ed4bee6b9a486c2`, the other open PR that touches `scripts/shape_materialize.py` | clean, rc 0, tree `a40df3b63c32c2a3d8f4c40587b094940c8d6567` |

Read with `gh pr list --state open --json number,headRefName,headRefOid,files`: 16 open PRs, three of them (this one, #1105 and #1118) touch `scripts/shape_materialize.py`, the only file this change touches. The other 13 (#1120, #1119, #1117, #1111, #1109, #1107, #1106, #1102, #1101, #1098, #1097, #1091, #1088) touch no file this change touches.

The same command with HEAD at the reviewed head `e32b2703` gives rv1104's trees: `de31d857` against `6da757cb` and `195ffe19` against #1105. The round 1 merge against #1090 is moot: #1090 has landed in `6da757cb`.

Held as a draft until the v1.2.0 tag; it is now on the v1.2.1 milestone.

- **NIT 1, `scripts/shape_materialize.py` lines 2340-2343, a line round 1 did not change.** The paragraph above the reworded one said the adoption's split message "names each entry path as the source spells it". Since #900 that is true of a printable path alone: `_split_message` lists any other through `_shown`, as the escape the next paragraph describes. It now says "as the source spells it (a printable one; see below)". One new commit on top of `e32b2703`, `de89aa81b1533e32f1007f991aeae05cd915b36d`, 3 lines added and 2 removed, docstring only, pushed as a fast-forward (no rebase, the PR is open).
  - Measured: the AST with docstrings blanked is identical to `e32b2703`'s and, merged into `6da757cb`, to `6da757cb`'s. The docstring holds no lone surrogate, still holds the six characters `\udce9`, and its longest line is 78 characters.
  - Searched `tests/`, `README.md`, `docs/`, the handbook mirror, `AGENTS.md`, `templates/` and the scripts for "names each entry path", "as the source spells" and the new words: only `tests/test_adopt_entry_paths.py` line 233 matches, a test's docstring about `execute` handing each path on as the source spells it, which is another sentence and pins nothing here.
- **NIT 2, this body.** It named main as `16f9a0e81` and #1090 as the one open PR that touches the file. #1090 has landed. Main is `6da757cb`, and the merge-trees above are redone against it, against #1105 and against #1118, a draft that opened since and also touches the file.

Review: rv1104-1107, lane 5's independent opus reviewer, which wrote none of #1104: READY at `de89aa81`, H0 of this landing (issuecomment-6318111413), with no finding; NIT 1 and NIT 2 of the round before are fixed. CI (Brett Heap's ruling 27 and his fast-refresh ruling of 2026-10-15T22:48:35Z): H0 de89aa8 passed tests, tests-git-bash and SonarCloud; the head 92285d4 is a clean update-branch merge of main b66d8f0, with tests-git-bash and SonarCloud green; and the local gate passed on its predicted squash tree 6bd2128 over main b66d8f0: the repo_hygiene, agents_md_codes, cli_reference sets, 171 passed. Landed on Brett Heap's word of 2026-10-23T13:10:26Z, relayed to every lane by lane 3 (#473 issuecomment-6365697123): "merge v1.2.1 PRs once reviewed and green".

Closes #1018

Lane: openreposhape-5 (openRepoShape-5)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@brettheap
brettheap marked this pull request as ready for review November 8, 2026 19:14
@sourcery-ai

sourcery-ai Bot commented Nov 8, 2026

Copy link
Copy Markdown

Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 6 days and 18 hours by commenting @sourcery-ai review. Upgrade to get a review now.

@sonarqubecloud

sonarqubecloud Bot commented Nov 8, 2026

Copy link
Copy Markdown

@brettheap
brettheap merged commit 9252ef2 into main Nov 8, 2026
5 checks passed
@brettheap
brettheap deleted the fix/1005-family-remove-add-family-root branch November 8, 2026 19:16
brettheap pushed a commit that referenced this pull request Nov 8, 2026
Lane: openreposhape-5 (openRepoShape-5)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap pushed a commit that referenced this pull request Nov 8, 2026
…e for: scripts/family.py tests/test_family_add_member_id.py tests/test_family_bump_to_checks.py tests/test_readme_windows_quoting_sentence.py tests/test_remove_add_remedies_family_root.py

Lane: openreposhape-5 (openRepoShape-5)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

family.py: the member-duplicate-id and member-unknown remedies name family.py remove and family.py add without --family-root, so each fails as printed

1 participant