[brettheap]

Lane: openreposhape-5 (openRepoShape-5)
Claim: #648 (comment)

Head f74f640. This branch holds #639 (#611), which landed as c6122e8, and main 2fac979, merged in bad6c00. Its diff against main is this PR's own change. git merge-tree --write-tree f74f640c origin/main is clean at 2a33a56.

README PR. It changes one sentence of README.md § Families, which this change makes false, and its twin in docs/handbook.html. The handbook's § Families marker is the segment's new blob, b3d19092. README stays at 1459 lines, its cap, and tests/readme-cap.d/ is not raised. It lands in lane 3's README turn.

Pinned copies. templates/family-root/scripts/bootstrap.py and templates/family-root/scripts/validate-family.py change. Each holder sees them as upstream-changed on its next update-shape.py check. No test pins either file's bytes or digest; the tests that copy them into a holder take them from the template. Rounds 2 and 3 change bootstrap.py alone. By Brett's 13:36:23Z word (R0) the family root's pinned copy is re-pinned once, in v1.1.0's shape bump, so this PR re-pins nothing.

Fix round 3 (rv648's verify at 23e1105: 1 MUST)

One commit, f74f640, on 23e1105, in the same two files.

MM1. FIXED: a .gitmodules a merge left unmerged

In a plain clone, a merge conflicts in .gitmodules, and the file is resolved in the work tree but not added. The index then holds it only at its three conflict stages (git ls-files -u lists 3). Git reads no .gitmodules at all while it is unmerged: repo_read_gitmodules asks is_gitmodules_unmerged before it reads any file. So git's update dies at the first gitlink, No url found for submodule path 'members/Atlas', exit 128. On 23e1105, bootstrap read the work tree's file, found every section with its url, and ended family bootstrap ok, exit 0 (rv648's p10-head.log).

gitmodules_sections now asks git ls-files -u -z -- .gitmodules first. Where the index holds the file at a conflict stage, it returns nothing, with Gitmodules.unmerged set. unfetchable asks that before every other reason, since no credential helps while git reads no section:

  NOT CHECKED OUT, and not for want of a credential: members/Atlas (git reads no .gitmodules while a merge leaves it unmerged), members/Nova (git reads no .gitmodules while a merge leaves it unmerged). Resolve the merge in .gitmodules and `git add .gitmodules`, then run this again.

With the file added, the same run checks both members out. That is all the cure needs; committing the merge is the person's own step.

Test. test_a_gitmodules_a_merge_left_unmerged_says_to_add_it builds MM1. The two sides each give OTHER's section another branch: dev, pushed from the holder, and main, committed in the clone. The merge exits 1, ls-files -u lists 3, and the work tree takes stage 2. The run exits 1, with the last two stderr lines exact and git's No url found ... 'members/IRRS'. Then git add .gitmodules, and the run is a good holder's. Against 23e1105's bootstrap.py the test fails on exit 0.

Mutants. Two more, both killed:

mutant	result	tests that fail
U: the unmerged check dropped, so the work tree's file is read as before	1 failed, 33 passed	MM1
U2: the file read as unmerged, but the reason not given, so the rows fall to the next one	1 failed, 33 passed	MM1

Round 2's eight were re-run on f74f640's two files. All are still killed, by the same tests.

Results on f74f640.

• rv648's lean set: 483 passed in 313.92s (0:05:13). That is 482 at 23e1105, plus MM1.
• The four files that load the family bootstrap.py (tests/test_update_shape_kept_helper.py tests/test_windows_paths.py tests/test_pinned_copy_text.py tests/test_cli_reference.py): 259 passed in 41.20s.
• rv648's p10.sh: A1T and H2T as on 23e1105, each typed through, with the gate passing in all three modes. MM1 now exits 1 with the line above.

A sentence of this body was false. "Not changed" said "A fresh clone, where bootstrap runs, has the work tree's file and no merge". MM1 is a fresh clone with a merge. That bullet now says what is left.

Candidate 6 (what git dies on while reading .gitmodules) is filed as a v1.1.1 issue (#768), with the approximations below.

Fix round 2 (rv648's verify at 45c512e: 1 MUST, 1 SHOULD, 3 NIT)

One commit, 23e1105, on 45c512e. It changes templates/family-root/scripts/bootstrap.py and tests/test_family_bootstrap_gate_order.py only. README, the handbook and validate-family.py are as they were.

M1. FIXED: .gitmodules is read as git's own parser reads it

init_stop read .gitmodules with git config --file --list. That keeps entries git's submodule parser drops or overrides. So in three states git's init died and bootstrap still said ok:

state (plain clone)	git's update	45c512e	23e1105
D1: a second section nova2 gives members/Nova with no url, after Nova's own	128, No url found ... 'members/Nova'	family bootstrap ok, exit 0	exit 1, both rows named, --no-members
Q1: Nova's section is named ../Nova	128, ignoring suspicious submodule name, then No url found	ok, exit 0	exit 1, both rows named, section written back
O1: Nova's url is -nowhere	128, ignoring ... may be interpreted as a command-line option, then No url found	ok, exit 0	exit 1, both rows named, section written back

The parser rules, as built. gitmodules_sections now returns a Gitmodules. Its read ({path: name}) and urls ({name: url}) follow git 2.43's submodule-config.c, entry by entry, in file order:

• A section whose name is empty, or has a .. component (split on / and on a backslash), is ignored whole. This is check_submodule_name.
• A path or url value that begins with - is dropped, and the value before it stands. This is looks_like_command_line_option.
• Where two sections give one path, the later in the file decides (cache_put_path replaces the entry). A section that gives a path again lets go of the path it gave before, even where a later section gave that path too, and then git reads no section there (cache_remove_path).

Its given counts every section that gives each path, as validate-family.py reads them. That count is what member-gitmodules-absent and gitmodules-path-twice see, so the line's cures stay true.

has_url asks the section git reads for a path, then the git config under that section's name. Where git reads no section for the path, there is no url, whatever the config holds: git's init dies on !sub before it looks at the config.

Checked against git 2.43's own source (submodule-config.c, submodule.c, builtin/submodule--helper.c and config.c at v2.43.0) and measured with git 2.43.0 in plain clones (p8.log):

.gitmodules form	git 2.43's update
name "", nova\..\x, a/../Nova	ignored; No url found, 128
name ..Nova.. (dots that are no .. component)	read; exit 0
path = -members/Nova	dropped; No url found, 128
a good url, then url = -x	the -x dropped, the good url kept; exit 0
a second section with Nova's url first, Nova's own with no url after	the later read; No url found, 128
nova1 gives the path, nova2 gives it too, then nova1 gives another path	members/Nova maps to nothing; No url found, 128
[submodule.Nova] (the old header)	read, under the name nova; exit 0

What else git does with .gitmodules. It drops nothing else. But it DIES reading the file, exit 128 before it checks out any member, on these, which git config --list reads without complaint:

• a path, url, update or ignore key with no value (path alone): bad config line N;
• update = !command, or any update value it does not know: invalid value for 'submodule.<name>.update';
• a shallow value that is no boolean: bad boolean config value;
• a fetchRecurseSubmodules value it does not know: bad submodule.<name>.fetchrecursesubmodules argument.

init_stop does not model these, and bootstrap still says ok over them where every row's section has a url (ignore = bogus is only a warning). No tool here writes any of them. This is a checklist candidate below, not part of this round.

D1's cure. Writing the section back would add a third section. So where more than one section gives the path git has no url for, the line names validate-family.py --no-members, whose gitmodules-path-twice names both sections and says to take the others out:

  NOT CHECKED OUT, and not for want of a credential: members/Atlas (git stopped at members/Nova, a gitlink it has no url for), members/Nova (git has no url for that path). `python3 scripts/validate-family.py --no-members` names each path more than one section of .gitmodules gives, and how to take the others out.

With the duplicate taken out and committed, the same run checks both members out.

The stop's sentence was "a gitlink no section of .gitmodules gives a url". That is false in D1, where Nova's own section does give one. It now says "a gitlink it has no url for", which is true in every state here. test_doctor_legs_members_next.py's git stopped at members/Ghost still matches.

S1. FIXED: A1, a member git has no url for, is its own reason

In a subset clone (--recurse-submodules=members/Atlas) where Nova's section has no url, git exits 0. On 45c512e the line printed the exit-0 skip's git submodule update --init --checkout --recursive -- members/Nova, which exits 128 (No url found) when typed.

"git has no url for that path" (has_url: the section git reads for it gives none, and the config has none under its name) is now its own reason. It is asked after "no section gives that path" and before "git exited 0", and its cure is the section written back. In A1:

  NOT CHECKED OUT, and not for want of a credential: members/Nova (git has no url for that path). Write the section of .gitmodules for members/Nova back as `family.py add` writes one, with its path and url, and commit .gitmodules.

Written back and committed, the next run is the exit-0 skip, and its printed update, typed, checks Nova out. In U1, Q1 and O1 the member's own row now gives this reason too, and the other row gives "git stopped at members/Nova". The cure is said once.

NIT 1. FIXED: a stop outside members/ (H2)

A hand-mounted vendor/Zed stops git's init, and the line named --no-members, which reads only members/ and says family ok there. Both cures were measured in H2's clone (p9-head.log):

• git rm --cached -- vendor/Zed and a commit: bootstrap then checks both members out and ends ok, and the gate passes in all three modes.
• A section with its path and url written for vendor/Zed and committed: bootstrap ends ok, but the gate refuses gitmodules-path-unnamed in all three modes.

So the line now says: No row names vendor/Zed and no clone can check it out: `git rm --cached -- vendor/Zed` takes its gitlink out, then commit that. "Outside members/" is the validator's own test: the first path component, casefolded, is not members.

NIT 2. FIXED: the cures docstring

It now says that, members checked, the validator refuses member-uninitialized where every row's gitlink is there (a stray, a hand mount), and member-gitlink-absent with the move where a row's is only in another case (#363), which the validator sentence names.

NIT 3. FIXED: this body

This section; the parser rules in Decision 2 and in (a)'s reasons; and a new merge table, against main a27c371, #745 and #747.

Tests (round 2)

tests/test_family_bootstrap_gate_order.py has 33 tests now (23 before):

• U1, Q1 and O1, one parametrized test (no-url, dotdot-name, empty-name, backslash-name, option-url). It replaces the U1 test. Each exits 1 with the last two stderr lines exact, git's No url found and its warning shown. Each is written back as family.py add wrote it and committed, and the same run ends ok.
• D1: exits 1 with the --no-members sentence. --no-members names gitmodules-path-twice for both sections; the duplicate is taken out as that finding says and committed, and the run ends ok.
• What git reads past (duplicate-first, which is D2; dots-in-a-name; option-url-after-a-url): with no credential for the members, each is still the credential's skip and ends ok; with the credential, each is a good holder's run. The no-credential run is what makes a member be asked at all, so these tests catch a rule read too widely.
• A1: the write-it-back line, then the exit-0 skip's update line, typed as printed, then a good holder's run, stderr empty each time but for the two lines.
• H2: the git rm --cached line, typed and committed; the run ends ok and the gate passes in every mode. A premise checks that --no-members says family ok there.
• The variant-plain, stray, hand-mount and round-trip expectations take the new stop sentence.

On 45c512e the new file gives 12 failed, 21 passed. Q1 (three names), O1 and D1 fail on exit 0; A1 and H2 fail on the line; U1, variant-plain, the stray, the hand mount and the round trip fail on the reworded sentences.

Mutation probes (round 2)

Each was applied alone to a throwaway clone holding this commit's two files, and the test file was run against it. All eight are killed:

mutant	result	tests that fail
L: the first section for a path decides, not the later	2 failed, 31 passed	D1; duplicate-first
N: a section with a suspicious name is read	3 failed, 30 passed	dotdot-name, empty-name, backslash-name
N2: any .. in a name, component or not, ignores the section	1 failed, 32 passed	dots-in-a-name
O: a value that begins with - is kept	1 failed, 32 passed	option-url
D: no --no-members sentence where two sections give the path	1 failed, 32 passed	D1
A: "git has no url for that path" not asked	7 failed, 26 passed	the five no-url cases, D1, A1
A2: that reason asked after "git exited 0"	1 failed, 32 passed	A1
H: a stop outside members/ gets the --no-members sentence	1 failed, 32 passed	H2

N2 was not killed by a first version of the read-past tests, which ran with the credential: git then checks both members out and bootstrap asks nothing. They now run without it first.

Probes (round 2)

rv648's p6.sh and p7.sh and w648b's p5.sh were re-run on 23e1105, and this round's p9.sh was run on 45c512e and on 23e1105. All use local bare remotes and the gh stub, with git 2.43.0. "ok" is family bootstrap ok, exit 0.

state	git's update	45c512e	23e1105
D1	128	ok	exit 1, the --no-members cure; the duplicate taken out and committed, then ok
D2 (the duplicate first)	0, both checked out	ok	ok
D3: the duplicate with Nova's url first, Nova's own with no url after	128	ok	exit 1, the --no-members cure
Q1: names ../Nova, "", nova\..\x, a/../Nova	128	ok	exit 1, section written back; then ok
Q2: name ..Nova..	0	ok	ok
O1: url -nowhere	128	ok	exit 1, section written back; then ok
O2: a good url, then url = -x	0	ok	ok
O3: path -members/Nova	128	exit 1	exit 1, the validator's cure, whose member-gitmodules-absent names it
U1	128	exit 1	exit 1; written back, then ok
A1	0	exit 1, the update line, which exits 128 typed	exit 1, section written back; then the update line, exit 0 typed; then ok
H2	128	exit 1, --no-members, which says family ok	exit 1, git rm --cached -- vendor/Zed; typed and committed, ok, and the gate passes in all three modes
G1, G3, S2	0	ok	ok
G2, X1, X2, I1 (no credential), W1 (a wrong url)	non-zero	the skip, ok	the skip, ok
S1, H1, V1	128	exit 1	exit 1, the same rows and cures, with the stop's new sentence
P1, N1	0	exit 1, the update line, exit 0 typed, then ok	the same

Every line of p5.sh but S1's, H1's, U1's and V1's reworded sentences is the same on 45c512e and 23e1105, gate lines included.

Fix round 1 (rv648 on 8b39cd1: 1 MUST, 2 SHOULD, 5 NIT)

Round 2 rewords two of round 1's sentences: the stop's why is now "a gitlink it has no url for", and in U1 the member's own row gives "git has no url for that path". The examples below are round 1's.

Three commits: bad6c00 merges main, 1a69505 answers the review, and 45c512e updates one doctor test that pinned the old ok (below).

The merge with main (bad6c00)

One file conflicted with #634 (#610): templates/family-root/scripts/validate-family.py, in two regions. This PR splits _check_member in two, and #634 made THE LOCKSTEP RULE a per-member value, rule = _lockstep(root, project, str(row.get("id") or project)), passed to the gitlink and tree checks.

• _check_member passes rule to _check_member_gitlink, as Pinned shape copies: one bump, so AGENTS-shape.md is true of every root, the scaffold's chain WARNING offers only its own sources and spells --pin-source, the root's not-used note spells it too, and THE LOCKSTEP RULE's bump runs (#610) #634 has it.
• Gitlinked gains a sixth field, rule, which _check_member returns.
• _check_member_checkout passes asked.rule to _check_member_tree.

So each member's gitlink and digest findings end at the rule #634 gives them, naming that member. The closing count's _lockstep(root) is #634's and merged clean. This is rv648's resolution in rv648-x634, plus one line of Gitlinked's docstring. It was built against #634's file, which is the same blob on main (519b71fc). Mutant X1 below checks it.

M1. FIXED: bootstrap no longer says ok where git died at another gitlink

In a plain clone, git submodule update --init dies in its init step at the first gitlink, in index order, that no section of .gitmodules gives a url. It then checks out no member. unfetchable asked only each row's own gitlink and section, so every row passed as the credential skip.

Measured on 8b39cd1 (p5-base.log, rv648's states with local bare remotes and the gh stub):

state	git's update	8b39cd1	1a69505
S1: a stray Members/Nova beside members/Nova, plain clone	128, No url found ... 'Members/Nova'	family bootstrap ok, exit 0	exit 1, both rows named
H1: members/Zed mounted by hand, no row, no section, plain clone	128, No url found ... 'members/Zed'	ok, exit 0	exit 1, both rows named
U1: Nova's section with no url, plain clone	128, No url found ... 'members/Nova'	ok, exit 0	exit 1, both rows named
V1: #363's state, plain clone	128, at Members/Nova	exit 1, Nova alone	exit 1, Atlas and Nova

rv648's prototype, measured before it was taken (p5-proto.log). It fixed S1, H1 and U1, but failed three states that are a missing credential, where 8b39cd1 rightly says ok:

state	8b39cd1	prototype	1a69505
X1: the stray, recursive clone, no credential	ok	FAILED, "git stopped at Members/Nova"	ok
X2: the hand mount, recursive clone, no credential	ok	FAILED, "git stopped at members/Zed"	ok
I1: Nova's url registered by git submodule init, then gone from the work tree's section, no credential	ok	FAILED, "git stopped at members/Nova"	ok

The prototype also said "no section of .gitmodules gives that path" for U1's Nova, whose section gives the path and lacks only the url.

What changed. init_stop asks the holder's records which gitlink git's init stops at, keeping two rules of git 2.43's init:

• submodule.active. Where the git config sets it, as a recursive clone sets ., git asks only the gitlinks a section gives a path. A gitlink no section gives is never active, so it stops nothing there (X1, X2).
• The config's url. A section with no url stops git only where the config has no url under the section's name, as an earlier init writes one (I1).

gitmodules_sections replaces gitmodules_paths and reads each section's keys. Where git exited non-zero and init stops at a gitlink, each row not checked out is named git stopped at <gitlink>, a gitlink no section of .gitmodules gives a url, and the run fails. A row with its own reason (no gitlink, no section) keeps that reason.

What the line says cures it. Each reason now has its own sentence:

• No gitlink, no section: the sentence it had, python3 scripts/validate-family.py names the cause and the commit that cures it.
• Git stopped at a gitlink no row names (S1, H1, V1): python3 scripts/validate-family.py --no-members names each gitlink in members/ that no row names, and the commit that cures it.
  • The plain validator would not help here: members checked, it refuses member-uninitialized for the first row (rv648's candidate 3, still open).
  • S1 and H1 are typed through: the finding's lines, then a commit. The hand mount's family.py add line is not typed, since it mounts the member again. The same run then checks both members out.
• Git stopped at a row's own gitlink (U1): no mode of the validator names it. So the line says: Write the section of .gitmodules for members/Nova back as family.py add writes one, with its path and url, and commit .gitmodules. Written back and committed, the same run checks both members out.

S1 now prints, for example:

  NOT CHECKED OUT, and not for want of a credential: members/Atlas (git stopped at Members/Nova, a gitlink no section of .gitmodules gives a url), members/Nova (git stopped at Members/Nova, a gitlink no section of .gitmodules gives a url). `python3 scripts/validate-family.py --no-members` names each gitlink in members/ that no row names, and the commit that cures it.
family bootstrap FAILED: Atlas (not checked out), Nova (not checked out)

Unchanged (p5-head.log): G1 and G3 (good holders, recursive and plain), G2 (plain, no credential), S2 (the stray in a recursive clone) and W1 (a wrong url, which looks like the missing credential).

Tests. The variant-plain case and the round trip now name MEMBER too, with both sentences. One test each covers S1, H1 and U1, and two cover the misfires: X1 and X2 in one parametrized test, and I1.

S1. FIXED: an exit-0 skip prints the update that checks the member out

For "git exited 0 and checked nothing out there", the line now prints the update that checks the member out, in place of the validate-family.py sentence:

  NOT CHECKED OUT, and not for want of a credential: members/Nova (git exited 0 and checked nothing out there). `git submodule update --init --checkout --recursive -- members/Nova` checks it out.

• --init with the path registers a member a subset clone left out. A pathspec lifts git's submodule.active filter, and init marks the member active.
• --checkout overrides an update = none.

Measured in P1 (a --recurse-submodules=members/Atlas clone) and N1 (update = none committed for Nova), on 8b39cd1 and on the head:

• 8b39cd1: the validator sentence, whose member-uninitialized remedy exits 0 and leaves Nova empty.
• 1a69505: the printed line, typed as printed, exits 0 and checks Nova out (15 entries). Bootstrap then ends every member and leg fetched, ok, and the gate passes in every mode.

test_an_update_that_exits_0_and_skips_a_member_fails is parametrized update-none and subset. It checks that the validator's remedy checks nothing out, types the printed line, and runs bootstrap again: exit 0, stdout exact, stderr empty.

The behaviour change (a subset clone and an update = none member now fail make bootstrap) belongs in v1.1.0's release notes, lane 3's #630 (candidate 4).

S2. FIXED: the credential test asserts no "every member fetched"

test_a_missing_credential_is_still_the_skip_it_was asserts " every member and leg fetched" not in result.stdout.splitlines(). rv648's mutant R2 (elif proc.returncode == 0: made else:) now fails it (mutants below).

NITs

1. FIXED. README: member's. `make bootstrap` reports and skips one it lacks a credential for. (75 characters, the line count unchanged). Handbook: A member <code>make bootstrap</code> lacks a credential for is reported and skipped, not a failure. § Families marker: c40baa82 → b3d19092, computed with test_repo_hygiene.py's own readme_segments and readme_fingerprint.
2. FIXED. gitmodules_paths is now gitmodules_sections, whose docstring says "An empty mapping where there is no such file".
3. FIXED. Bootstrap's (a) docstring says "128 (No url found for submodule path) in a plain clone; a missing credential exits non-zero too". It also names the new reason and each cure.
4. FIXED. The validator's WHAT IS CHECKED says the move under --no-members is printed "where its row's path is its canonical members/<Project> and its pin a 40-hex commit".
5. FIXED. The merge table below replaces the stale one.

What changed and why

(a) bootstrap.py no longer ends ok over members it could not check out

After the fetch, bootstrap asks the holder's own records, never git's words, about each row whose member is not checked out. The first reason that holds is the row's:

1. A merge left .gitmodules unmerged in the index, so git reads none of it (Gitmodules.unmerged; MM1). This is asked first: it holds for every row, and no credential helps.
2. No gitlink is recorded at the row's path (recorded_gitlink, the reading member-gitlink-absent uses), as in family.py add, under core.ignorecase=true, commits the member's gitlink as Members/<P> when HEAD holds a path under a case variant of members/ that the index no longer does (a git rm -r Members not yet committed), so .gitmodules and family.yaml say members/<P>, the holder's own validator passes in the working clone, and every fresh clone of the pushed holder leaves the member's directory empty and is refused member-gitlink-absent #363's state.
3. No section of .gitmodules gives that path, as where HEAD lost the file. Every section counts here, as validate-family.py reads them.
4. Git has no url for that path: the section git reads for it gives none, and the git config has none under that section's name (has_url; U1, A1, D1, Q1, O1). No credential gives a url, whatever git exited.
5. Git exited 0, so it lacked nothing a credential gives.
6. Git exited non-zero, and its init stopped at a gitlink it has no url for (init_stop, M1 of both rounds).

Reasons 4 and 6 read .gitmodules as git 2.43's own parser reads it, not as git config --list lists it (gitmodules_sections): a section whose name is empty or has a .. component, split on / and on a backslash, is ignored; a path or url that begins with - is dropped, and the value before it stands; and where two sections give one path, the later in the file decides.

Each such row is named on one line, with what cures each reason, and the run fails. "every member and leg fetched" is printed only where git exited 0 and every row is checked out. A row git could not fetch for another reason is still NOT CHECKED OUT; skipped, and the run still ends ok: that is the state a missing credential leaves. test_family.py's test_bootstrap_degrades_when_a_member_cannot_be_fetched and bootstrap's docstring (c) say so.

(b) The gate asks every row's gitlink before any row's checkout

_check_member asked each row whole, gitlink then checkout, before it asked the next. In a plain clone of #363's state plus git submodule update --init --recursive (CI's and make bootstrap's flow), the update dies at Members/<P> before it checks out any member. So where the variant's row was not first, the gate refused member-uninitialized for the first row, naming that same update.

Now _check_member stops at the gitlink and returns the member (Gitlinked). _check_member_checkout asks each member's digest and identity once every row's gitlink has been asked. The refusal there is now #611's member-gitlink-absent with the move, whichever row it is, members checked and --pins alike.

(c) --no-members prints the move members mode prints

Under --no-members, the mode CI falls back to in (b)'s state, member-gitlink-unnamed for a member's only gitlink in the wrong case printed no command. Where the row's path is its canonical members/<P> and its pin is a 40-hex commit, it now prints the same text and two lines the row's members-mode refusal prints. One helper, _wrong_case_move, builds both. The commit is the row's pin, not the variant's gitlink, and every variant goes on one line.

Decisions and evidence

1. The holder's records decide, not git's exit or words. Git exits 0 and 128 over the same fault (HEAD lost .gitmodules: 0 after a recursive clone, 128 in a plain one), and a missing credential exits non-zero too. No url found is git's own message, and git translates it.
2. init_stop keeps git's rules, and no more. rv648's prototype failed X1, X2 and I1, so the submodule.active filter and the config's url are kept (round 1's M1). Round 2 adds the three rules by which git's own parser reads .gitmodules, each checked against git 2.43's source and measured: the suspicious name, the value that begins with -, and the later section for a path (round 2's M1). It does not match submodule.active's pathspecs: under a subset clone's value, a url-less section of a member outside the subset counts as a stop. That is reached only where git also fails for another reason (Not changed, below).
3. The cure is per reason. The validator's members mode refuses member-uninitialized in S1, H1 and U1, and nothing names U1, so one sentence cannot be true of every reason. Two sections for one path get --no-members, since a section written back would be a third (D1), and a gitlink outside members/ gets git rm --cached, the one cure measured to leave the gate passing (H2). Each sentence was typed through in its state (Tests below).
4. The README sentence. README is at its 1459-line cap, so the sentence fits its line. It cannot also name the new failures; bootstrap's docstring does.

Tests

tests/test_family_bootstrap_gate_order.py, new, 34 tests. Round 1's 23 are listed here, and rounds 2's ten and 3's one in their own sections above:

• (a), four states: HEAD lost .gitmodules and family.py add, under core.ignorecase=true, commits the member's gitlink as Members/<P> when HEAD holds a path under a case variant of members/ that the index no longer does (a git rm -r Members not yet committed), so .gitmodules and family.yaml say members/<P>, the holder's own validator passes in the working clone, and every fresh clone of the pushed holder leaves the member's directory empty and is refused member-gitlink-absent #363's state, each after a recursive clone and after a plain one. Each exits 1, stdout and the last two stderr lines are exact, and git's No url found shows in the plain ones.
• The good and the skipped: a good holder's run, exact; the credential skip still ends ok, with no new line and no "every member fetched".
• Git exits 0 and skips a member: update = none and a subset clone. The run fails on that member alone, and the printed update, typed, lets the run end ok.
• Git stops at another gitlink: S1 and H1 (typed through --no-members and committed), U1 (the section written back), and the misfires X1 and X2 and I1, which are still the skip.
• A row written twice is named once.
• The round trip: a plain clone of family.py add, under core.ignorecase=true, commits the member's gitlink as Members/<P> when HEAD holds a path under a case variant of members/ that the index no longer does (a git rm -r Members not yet committed), so .gitmodules and family.yaml say members/<P>, the holder's own validator passes in the working clone, and every fresh clone of the pushed holder leaves the member's directory empty and is refused member-gitlink-absent #363's state with the variant at the SECOND row. Bootstrap fails, the gate refuses with the move, the lines are typed, and bootstrap then checks both members out, runs each one's own bootstrap and ends ok. The gate passes in every mode.
• (b): a plain clone plus git submodule update --init --recursive (exit 128), with the variant in the first row and in the second.
• (c): on a Linux disk and an emulated folding one, one variant and two, at a commit that is not the pin. A row whose pin is no commit still prints no command.

Changed: tests/test_validate_family_unnamed_mounts.py, state (ii) under --no-members. It printed no command; it prints the move now.

Results on 1a69505, Linux, /usr/bin/python3 -m pytest through run9.sh (gh stub first on PATH). 45c512e changes one test file, which was run again on it:

• Batch A, the brief's named files: tests/test_family_bootstrap_gate_order.py tests/test_validate_family_unnamed_mounts.py tests/test_family.py tests/test_pinned_copy_text.py tests/test_validate_family_branch_step.py tests/test_family_members_case_variants.py tests/test_family_head_lost_gitmodules.py tests/test_family_bump_to_local_only.py tests/test_repo_hygiene.py tests/test_readme_sections.py tests/test_template_text_placeholders.py tests/test_cli_reference.py: 698 passed in 638.20s (0:10:38).
• Batch B, every other test file that names validate-family or the family's bootstrap.py (49 files, git grep -l): 1 failed, 3797 passed, 32 skipped in 2483.22s (0:41:23).
  • The failure was tests/test_doctor_legs_members_next.py::test_414_3_there_the_bootstrap_cures_nothing_and_the_git_line_cures_one. It builds rv648's U1 state (a member, Ghost, whose section gives no url) and asserted bootstrap's exit 0: the false ok M1 ends.
  • 45c512e asserts exit 1 and git stopped at members/Ghost, and its point stands: the bootstrap cures nothing there, and the doctor's git line cures the other member. That file alone on 45c512e: 124 passed in 153.99s (0:02:33).
  • The skips are the Windows-only lines, as on main.
• Round 2, on 23e1105. rv648's lean set, tests/test_family_bootstrap_gate_order.py tests/test_validate_family_unnamed_mounts.py tests/test_family_head_lost_gitmodules.py tests/test_validate_family_branch_step.py tests/test_family_members_case_variants.py tests/test_doctor_legs_members_next.py tests/test_family_bump_to_local_only.py tests/test_repo_hygiene.py tests/test_readme_sections.py and test_family.py's three bootstrap tests: 482 passed in 263.30s (0:04:23). That is 472 at 45c512e plus round 2's ten. The files that load the family bootstrap.py by path or check its imports and text, tests/test_update_shape_kept_helper.py tests/test_windows_paths.py tests/test_pinned_copy_text.py tests/test_cli_reference.py: 259 passed in 30.69s. Batch B was not re-run in round 2.
• Not run: the 22 files that name only the assembly root's bootstrap.py (adopt, scaffold, bump-leg.py and setup tests), which this PR does not change. Main's six commits after 55c6ba7 were not run against this head either; merge-tree with them is clean.

Mutation probes (round 1)

Round 2's eight are in its own section above. Each of round 1's mutants was applied alone to a throwaway shared clone at 1a69505 (exact text replacements), and its test file was run against it. Each one is killed:

mutant	result	tests that fail
M1a the "git stopped at" reason never given	5 failed, 18 passed	variant-plain, S1, H1, U1, the round trip
M1b submodule.active not asked	2 failed, 21 passed	the recursive clones with no credential, stray and hand mount
M1c the config's url not asked	1 failed, 22 passed	the url an earlier init registered
M1d a url-less section counts as having one	1 failed, 22 passed	U1
M1e no --no-members sentence	4 failed, 19 passed	variant-plain, S1, H1, the round trip
M1f a row's own stop given the --no-members sentence	1 failed, 22 passed	U1
S1a the validator sentence for an exit-0 skip	2 failed, 21 passed	update-none, subset
S1b no update line	2 failed, 21 passed	update-none, subset
S1c the update line without --checkout	2 failed, 21 passed	update-none, subset
S1d the update line without --init	2 failed, 21 passed	update-none, subset
R2 (rv648's) "every member and leg fetched" over a failed fetch	1 failed, 22 passed	the credential skip
X1 the digest finding's rule not the member's (the merge)	1 failed, 22 passed	test_validate_family_branch_step.py's test_a_digest_s_rule_names_its_own_member
N1 the § Families marker left at c40baa82	1 failed, 76 passed	test_repo_hygiene.py's test_the_handbook_follows_the_readmes_outline

S1c and S1d were also run with the test's expected line edited to match, so that only typing the printed line can catch them. Each is still killed, by the state that needs the flag: without --checkout, update-none fails (1 failed, 1 passed); without --init, subset fails (1 failed, 1 passed).

Merge checks (git merge-tree --write-tree f74f640c <ref>, 2026-10-17 at 00:35Z)

Merged with	Head	Result
origin/main	2a33a56	clean
lane 5's fix/704-bootstrap-leg-moves (#745: #704, #694), restacked on 23e1105	f4fa602	clean
lane 5's fix/681-validate-family-remedies (#747: #681), restacked on 23e1105	03e0b5a	clean
lane 4's fix/690-siblings-inputs	38a40ab	clean
lane 5's fix/695-family-docs (#738)	c42286d	CONFLICT in docs/handbook.html, the § Families marker line only, as before. Rounds 2 and 3 change neither README nor the handbook.
every other fix/* and l3-* branch on origin, 26 more		clean

README is 1459 lines at f74f640 and in its merge with main.

Not changed, and why

• The validator's members mode in S1, H1 and U1. It still refuses member-uninitialized for the first row, with a remedy that dies the same way, and no mode names U1. Bootstrap's line now names the cause in all three and the cure, so the person is no longer sent there. Candidate 3.
• What git dies on while reading .gitmodules. Git 2.43 exits 128 before it checks out any member over a path, url, update or ignore key with no value, an update of !command or a value it does not know, a shallow that is no boolean, and a fetchRecurseSubmodules value it does not know (p8.log). init_stop does not model these, so where every row's section has a url, bootstrap still ends ok over them. Filed as a v1.1.1 issue (bootstrap.py and validate-family.py: a .gitmodules value git dies on while reading the file fails make bootstrap naming the key, and the family gate names it, so update = None no longer ends family bootstrap ok (#648) #768), which fixes the family validator in the same change.
• Where git reads .gitmodules from. Git reads the work tree's file; where the work tree has none, the index's, then HEAD's; and none at all while a merge leaves it unmerged. Since round 3 the unmerged file is its own reason (MM1). The fallback to the index's or HEAD's file is still not followed: gitmodules_sections reads the work tree's alone. That is reached only where the work tree lacks the file while the index or HEAD holds it, as a deletion not yet committed or a sparse checkout that leaves it out leaves it. There every row git left not checked out is said to have no section, and the run fails, even where a credential was all git lacked. In the v1.1.1 issue.
• submodule.<name>.active. Where submodule.active is set, git asks a member's own active key before the pathspec. init_stop asks neither, as round 1 did. In the v1.1.1 issue.
• submodule.active pathspecs. init_stop takes the key's presence, not its patterns. Under a subset clone's members/Atlas, a url-less section of a member outside the subset counts as a stop, though git's init skips it. That is reached only where git's exit is non-zero for another reason, and then the line names that section, which is a real defect. In the v1.1.1 issue.
• macOS and Windows were not run (CI is Linux only since ci: run the test suite on Linux only #597).
  • The variant states are index records; the holder's own are asked again with core.ignorecase=true.
  • A clone of S1 gets Members/Nova and members/Nova as two empty mounts. On a disk that folds case, git checks them out as one directory and warns. No assertion reads either directory, and git's init dies at the stray on every disk.
  • Every file is written as bytes, and url_edited reads a CRLF checkout too.
  • The backslash-name case (IRSS\..\x): git splits names on both / and a backslash on every system (is_xplatform_dir_sep), so it is ignored on Windows too.

Checklist candidates

1. A half-typed move (rv648's 1). After typing the move and before committing, both modes print STRAY's git rm --cached -- Members/Nova, which fails pathspec ... did not match (exit 128). It is a v1.0.0 known limit. validate-family.py: a staged git rm is named again and fails retyped, the member-gitlink-absent remedy ends in a refused add, a shape-pin row with no path: reads None, and #639's mv orphans a live checkout and is stated by OS #681's branch says to commit a staged removal and prints no git rm there.
2. The state a git mv members/Nova Members/Nova leaves (rv648's 2). The move fixes only the gitlink, and two more rounds follow.
3. Members mode in S1, H1 and U1 (rv648's 3). It refuses member-uninitialized for the first row, with a remedy that dies the same way. No mode names a section with no url.
4. v1.1.0's release notes for bootstrap.py and validate-family.py: family bootstrap ok with no member checked out, an earlier row's member-uninitialized hiding a case-variant refusal, and member-gitlink-unnamed with no command under --no-members #648, in lane 3's README, AGENTS.md, docs/cli.md and the handbook: pipx install openreposhape is the second way in and the first on native Windows, git-filter-repo is installed in install.sh's order and never with a bare pip install, and a first draft of the v1.1.0 release notes (#563) #630. A subset clone, an update = none member, and S1, H1 and U1 now fail make bootstrap. The line prints the cure for each.
5. member-gitlink-unnamed says "a recursive clone of this holder fetches it" for a gitlink no section of .gitmodules gives. A recursive clone fetches nothing there, as S2 and X1 show for the stray.
6. Bootstrap still says ok where git dies reading .gitmodules. Filed as a v1.1.1 issue (bootstrap.py and validate-family.py: a .gitmodules value git dies on while reading the file fails make bootstrap naming the key, and the family gate names it, so update = None no longer ends family bootstrap ok (#648) #768).

Closes #648

🤖 Generated with Claude Code

Summary by Sourcery

Make family bootstrap accurately distinguish credential-limited members from checkout failures and provide actionable repairs for invalid submodule state.

Bug Fixes:

• Make family bootstrap fail when members cannot be checked out for repository-state reasons rather than silently reporting success, while continuing to skip members unavailable only because credentials are missing.
• Detect malformed, ambiguous, or unmerged .gitmodules states and report actionable remediation for each affected member.
• Ensure validation checks every member gitlink before checking out any member, and provide the case-variant gitlink move under --no-members.

Enhancements:

• Align family bootstrap's .gitmodules interpretation and submodule-stop detection with Git's behavior, including duplicate sections, ignored names, option-like values, and configured URLs.

Documentation:

• Update README and handbook family-bootstrap documentation to distinguish credential skips from bootstrap failures.

Tests:

• Add comprehensive bootstrap, gitlink-ordering, .gitmodules, merge-conflict, credential, and case-variant regression coverage.

[brettheap]

(lane openRepoShape-5, 2026-10-16T13:04Z) @codex review

[sourcery-ai]

Reviewer's Guide

The PR hardens family bootstrap and validation around members that cannot be checked out: bootstrap now uses holder metadata to fail only for non-credential causes, validation checks every row’s gitlink before checkout-dependent checks, and --no-members emits the canonical case-variant index move, with extensive regression coverage and synchronized documentation.

Sequence diagram for credential-aware family bootstrap

sequenceDiagram
    participant Bootstrap as bootstrap.py
    participant Git as git submodule update
    participant Holder as Holder records
    participant Validator as validate-family.py

    Bootstrap->>Git: git submodule update --init --recursive
    Git-->>Bootstrap: exit status and output
    Bootstrap->>Holder: unfetchable(rows, exited_0)
    Holder-->>Bootstrap: rows not checked out with recorded cause
    alt non-credential cause found
        Bootstrap->>Bootstrap: report NOT CHECKED OUT
        Bootstrap->>Validator: name member-gitlink-absent or member-gitmodules-absent
        Bootstrap-->>Bootstrap: family bootstrap FAILED, exit 1
    else only credential-related skips
        Bootstrap-->>Bootstrap: report skipped members, family bootstrap ok
    end

Loading

Sequence diagram for case-variant repair in no-members mode

sequenceDiagram
    participant Validator as validate-family.py
    participant Index as Git index
    participant Operator as Operator

    Validator->>Index: inspect member gitlinks
    Index-->>Validator: case-variant gitlink at variant path
    Validator->>Validator: _wrong_case_move(row_path, pin, variants)
    Validator-->>Operator: print member-gitlink-unnamed and canonical move
    Operator->>Index: git rm -r --cached -- variant
    Operator->>Index: git update-index --add --cacheinfo 160000,pin,row_path
    Operator-->>Validator: commit repaired gitlink

Loading

Flow diagram for ordered family validation checks

flowchart LR
    Rows["Family rows"] --> Gitlinks["Check every row's gitlink"]
    Gitlinks --> Findings["Report gitlink findings"]
    Findings --> Checkouts["Check checkout-dependent state"]
    Checkouts --> Result["Pass or refuse with remediation"]

Loading

File-Level Changes

Change	Details	Files
Make bootstrap fail when a member is not checked out for a repository-state reason rather than a missing credential.	Inspect each unchecked-out row using its recorded gitlink and .gitmodules path entries. Distinguish credential skips from missing gitlinks, missing .gitmodules sections, and successful updates that checked out nothing. Report affected members and remediation guidance, suppress the success message, and return failure while preserving credential-skip behavior.	templates/family-root/scripts/bootstrap.py README.md docs/handbook.html
Separate gitlink validation from checkout validation so all rows’ gitlinks are examined before any checkout-dependent checks.	Return an intermediate gitlinked-row record from the per-member check. Run digest and identity checks in a second pass after every row’s gitlink has been evaluated. Ensure case-variant gitlink refusals are surfaced even when submodule update stopped earlier in the row list.	templates/family-root/scripts/validate-family.py
Provide the same corrective case-variant gitlink move under --no-members as in members mode.	Centralize construction of the index move and disk-cleanup remediation. Use the row’s pinned commit and include all case variants in the removal command. Restrict the command to canonical row paths with valid commit pins; retain no-command behavior otherwise.	templates/family-root/scripts/validate-family.py tests/test_validate_family_unnamed_mounts.py
Add focused regression coverage for bootstrap failure classification, validation ordering, and remediation output.	Cover recursive and plain clones with missing .gitmodules sections, case-variant gitlinks, credential failures, skipped updates, and duplicate rows. Exercise the end-to-end printed remediation through checkout and validation in members, pins, and no-members modes. Verify one- and two-variant cases, case-folding behavior, non-commit pins, exact output, and preserved good-holder behavior.	tests/test_family_bootstrap_gate_order.py tests/test_validate_family_unnamed_mounts.py

Assessment against linked issues

Issue	Objective	Addressed	Explanation
#648	Make bootstrap fail with a clear explanation and a non-zero exit when a member is left unchecked out for a reason a credential cannot fix, rather than claiming all members were fetched or ending successfully.	✅
#648	Validate every member row's gitlink before checking out any member, so an earlier member-uninitialized refusal cannot hide a later case-variant gitlink refusal.	✅
#648	Under --no-members, give member-gitlink-unnamed the same index-only move command as members mode when a member's gitlink is recorded only in the wrong case.	✅

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-16T13:14:30.341287Z	8b39cd1	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8b39cd195e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Require a URL before treating a submodule path as fetchable

When a .gitmodules section retains path = members/X but loses its url, git submodule update exits 128 and leaves the member unchecked out, yet this helper adds the path to given. Because the Git command did not exit 0, unfetchable() then reports no failure, the member is skipped, and bootstrap exits 0 with family bootstrap ok; this is a structural error that no credential can fix. Track complete path-and-URL sections rather than paths alone.

Useful? React with 👍 / 👎.