[brettheap]

Lane: openreposhape-5 (openRepoShape-5)
Claim: #762 (comment)

Against main. #747 landed as 6626a3c, and this branch is restacked onto main as 98b0536d: it merges #747's final head ff7c59b8, then main, taking this branch's side in the three files #747 also changed. Over main it carries exactly its own three commits: git diff 58d6c50f ef178669 (over #747's head 58d6c50f, where it was written) and git diff origin/main 98b0536d have the same patch-id.

Goal (#762): every finding and refusal the holder's validate-family.py prints for a member recorded in the wrong case, a gitlink git stops at, or a commit its checkout lacks says what git does and prints a remedy that works in the state it names, and the gate names a member's un-registration before the commit that makes it.

Pinned file. Only templates/family-root/scripts/validate-family.py changes among the shipped files. It is pinned in every holder (FAMILY_COPIED_VERBATIM), so each holder sees one upstream-changed row for it, re-pinned in the next shape bump. The family root's bootstrap.py is not changed (line 8 below). No README, handbook, AGENTS.md or docs/cli.md sentence is made false, and no --help text changed.

Every state below was measured on git 2.43.0, in the test family (tests/test_family.py's fixture), on this branch's base 58d6c50f and then on this head. Each new test types every printed command as printed, in the state it names, and then runs the gate.

Fix round 2 (commit ef17866, one commit on 9e2dc7d)

rv820 (lane 2) reviewed 1038d7f: NOT READY. 9e2dc7d answered MUST 1. This commit answers SHOULDs 2 to 4 and NITs 5 to 7. It changes templates/family-root/scripts/validate-family.py and tests/test_validate_family_typed_states.py only.

SHOULD 2: member-uninitialized named a stop git never makes, and a cure that dropped the other member's section.

• Reproduced: in a plain clone with .gitmodules deleted and not staged, members checked said git submodule update --init stops at members/IRRS, and told the person to write that section back. In fact git read the index's file, and the update exited 0 and checked out both members. With the deletion staged, git read HEAD's file, with the same result. In the url-less state (U1), the old text named members/IRRS where git stops at members/IRSS.
• The fix: _gitmodules_put_back reads the .gitmodules that git's submodule commands read when the work tree's file is gone: :.gitmodules, then HEAD:.gitmodules, each found with git rev-parse --verify --quiet. _init_stop_said now reads that file.
• Where git really stops there, the refusal names that path and prints the line that puts the file back first: git checkout -- .gitmodules, or git checkout HEAD -- .gitmodules where the index holds none. Then comes the cure, so the cure keeps every other section. After that line, --no-members, which reads the work tree's file, names what the cure says it names.
• member-gitmodules-url-absent (new in this PR) had the same fault: its git config -f line would make a file holding that one key. It now prints the same put-back line first when the file is gone.
• Left for follow-ups:
  • member-gitmodules-absent (validate-family.py and family.py remove: a holder whose HEAD lost .gitmodules passes the gate, and remove fails after deinit, leaving the mount empty #611, before this PR) prints the same kind of git config -f lines;
  • the family bootstrap's gitmodules_sections reads the work tree's file alone, as rv820 says.
• New tests, each typed as printed. All six fail on 9e2dc7d.
  • A plain clone, with the file deleted, or its deletion staged, is given the plain update, which checks out both members.
  • U1 in a plain clone, both ways: the refusal names members/IRSS and the put-back line. Done as it says, HEAD's file keeps IRRS's section, and the gate passes in every mode.
  • U1 in its own holder, both ways: the url-absent finding with the put-back line, in every mode.

SHOULD 3: the three wrong-text mutants now each fail a test.

• name.startswith(path) fails the new test of a file committed at members/IRRS-notes, beside the row's path.
• Dropping and "url" in keys fails the new test of a hand mount whose section's url was taken out.
• Dropping and name in variants fails the new test of a non-cone sparse checkout that keeps notes/ off disk, with the variant on disk.
• These tests pass on 9e2dc7d, whose code is right; they exist to catch the mutants.

SHOULD 4: S3776.

• _init_stop_said (20 by rv820's count, which a local counter reproduces) is split: the gitlink git stops at is _init_stopper (11), and its cure is _init_stop_cure (3). _init_stop_said is now 4.
• _check_removals_staged (18) is now 13. The put-back lines are _put_back_lines (2), and the finding's text is _removal_staged_said (1).
• Behaviour is unchanged: every existing test still passes, comparing the texts word for word.
• _check_gitlinks_named (21) and _check_gitlinks_registered (19) are not split here. Main already has them at 18 and 19, and this commit changes neither beyond passing root to _url_absent.

NITs

• 5, taken by narrowing the text. _move_context's docstring now says what git does with entries a merge left unmerged. At the row's path, the move's second line replaces stages 1 to 3. Under it, it leaves them, and the commit then refuses with Committing is not possible because you have unmerged files (exit 128). Measured on git 2.43 and 2.47.1. A new test puts a gitlink at stages 1 to 3 at the row's path: the move as printed replaces it, and the gate passes. Dropping tag != b"M" and now fails that test.
• 6, taken. A case with seven entries under the row's path checks the five-name cap ("and 2 more"). A new test stages a removal where HEAD's .gitmodules gives no section for the row: no git checkout HEAD -- .gitmodules is printed. Each of the two mutants rv820 named now fails.
• 7, taken in part. refused_and_unnamed replaces the repeated refusal-and-fallback block in two tests, and three new tests use it. The (e) test keeps its own form, because it compares only the first --no-members finding, with the fetched wording.

Tests

• tests/test_validate_family_typed_states.py: 43 tests. On 9e2dc7d's validator, 6 failed (the SHOULD 2 cases) and 37 passed.
• On git 2.47.1: this file, test_family_head_lost_gitmodules.py, test_validate_family_unnamed_mounts.py and test_family_bootstrap_gate_order.py, 155 passed.
• Every test file that runs the validator or calls the gate (58 files, the three standing check files among them), on git 2.43, in six shards on the uncommitted tree: 3021 passed, 24 skipped.
  • 5 failed and 3 errored, all in update-shape holder tests: REFUSED update-conflict ... scripts/validate-family.py: ... not a verbatim copy ... at the pinned commit. These tests pin HEAD, so a change to the validator that is not yet committed makes the holder's copy differ from its pin.
  • With the same two files committed in a scratch clone (the same tree as ef17866), the three files pass: test_update_shape_pr_base.py 27 passed; test_family.py and test_update_shape_header_comments.py 160 passed.

Mutants (each run on the whole file with -x): all 12 are caught.

• The three from SHOULD 3, the one from NIT 5 and the two from NIT 6 (above).
• The put-back sources swapped: caught by the url-less clone with the file deleted.
• The index's and HEAD's file not read: caught by the plain clone with the file deleted.
• The put-back line left out of the refusal: caught by the url-less clone.
• The put-back line left out of member-gitmodules-url-absent: caught by U1 in its holder.
• The two-sections cure checked after the row's own: caught by the plain clone with the twice stop.
• The guard for a file on disk dropped: caught by the plain clone with the stray stop.

Merges

• With validate-family.py and validate-pins.py: a staged gitlink removal says to commit it, member-gitlink-absent puts the gitlink back by hand, the variant mv is guarded, and a pin row with no path: is shape-pin-row (#681) #747's head ff7c59b and with family-root README.md and AGENTS-shape.md, and AGENTS.md: the holder's docs say its parent folder is named EXACTLY after its repository, case included, on every disk, and the never to hand-edit the members: block names the row repair validate-family.py asks for, whose commit is the person's #822's head ea86cb6: clean (git merge-tree).
• With bootstrap.py and validate-family.py: a .gitmodules value git dies on as it reads the file fails make bootstrap naming the key, and the family gate names it in every mode, so update = None no longer ends family bootstrap ok (#768) #819 (c12b5f8): the same 3 conflict hunks in validate-family.py as at 9e2dc7d.
• With main (64ec47c): the same conflicts as at 9e2dc7d, from the stack's base.
• For whichever of bootstrap.py and validate-family.py: a .gitmodules value git dies on as it reads the file fails make bootstrap naming the key, and the family gate names it in every mode, so update = None no longer ends family bootstrap ok (#768) #819 and validate-family.py: the wrong-case move back mends .gitmodules, adds --sparse and moves an entry in the way aside, a row's staged removal and a url-less section are findings, and two refusals name a cure that works (#762) #820 lands second: bootstrap.py and validate-family.py: a .gitmodules value git dies on as it reads the file fails make bootstrap naming the key, and the family gate names it in every mode, so update = None no longer ends family bootstrap ok (#768) #819 sets sections = {} when the work tree's .gitmodules holds a value git dies on. validate-family.py: the wrong-case move back mends .gitmodules, adds --sparse and moves an entry in the way aside, a row's staged removal and a url-less section are findings, and two refusals name a cure that works (#762) #820 passes those sections to _check_removals_staged and to _init_stop_said, as it did before this commit. There, a git rm --cached staged removal would also print git checkout HEAD -- .gitmodules (rv820's note), and by reading the code, not measured, _init_stop_said would find no section there and name the first gitlink as the stop. Whichever lands second should read that state as git does.

Fix round (commit 9e2dc7d, one commit on the head 1038d7f)

CI run 39031276097 (git 2.55.0) was red on one test, tests/test_validate_family_typed_states.py::test_a_pin_no_remote_holds_is_bumped_as_printed, at :527. The fetch the remedy names exited 0 (From .../remotes/IRRS, then * branch <sha> -> FETCH_HEAD) where the test expects exit 128 and not our ref.

The cause is the test, not git 2.55 and not the remedy.

• The three pin tests share one remote, remotes/IRRS.git, because family is module-scoped (tests/test_family.py:91).
• commit_ahead built the same commit on every call: the same parent, the same AHEAD.md (ahead\n), the message Ahead and the same identity. Two calls in the same wall-clock second therefore made one sha.
• The on-no-branch test pushes its commit to refs/w762/ahead. When the no-remote test's commit was made in the same second, that remote already held it. Protocol v2's upload-pack serves any object its repository holds, so the fetch fetched the "unpushed" commit. The gate had just found the commit missing from the member's own store (rev-parse --verify), so the remote was the only place it could have come from.
• Measured: with GIT_AUTHOR_DATE and GIT_COMMITTER_DATE exported fixed, head 1038d7f fails the same way on git 2.43.0 and on git 2.47.1: 1 failed, 2 passed, the same assertion at :527 and the same stderr. Run normally, it passes on both (3 passed). On this host the two commits are 2.2 s apart (three runs), so only a faster runner puts them in one second. The CI log has no per-test times, so the second itself cannot be read from it.
• The on-no-branch test could pass while proving nothing in the same way: with fixed dates, its commit was the on-a-branch test's, which sits on refs/heads/w762-ahead.
• rv820 (lane 2) found the same cause independently.

What the commit changes (tests/test_validate_family_typed_states.py only):

• AHEAD.md now holds next(AHEAD), a module counter. It is not tmp_path.name, which tmp_path_retention_policy=failed hands to the next test (test_family_bootstrap_gate_order.py's BRANCHES gives the same reason). A first try with tmp_path.name failed the new premise check under fixed dates for exactly that reason.
• The commit carries fixed dates, AHEAD_DATES, so it is the bytes that keep two calls apart in every run, never the clock.
• The premise is checked before anything is typed:
  • the remote-holds tests check that git branch --contains <commit> in the remote lists their ref alone, or nothing for refs/w762/ahead;
  • the no-remote test checks that git cat-file -e <commit> in the remote exits 1.
• The remedy text is unchanged. Its fetch is right where a remote holds the commit, as the CI run's remote did. Its switch and bump lines still type clean as printed on git 2.43.0 and 2.47.1 (the whole file passes on both, below).

Tests

• tests/test_validate_family_typed_states.py, whole file: 31 passed on git 2.43.0, and 31 passed on git 2.47.1.
• The three pin tests with the dates exported fixed: 3 passed on each git, where head 1038d7f gives 1 failed, 2 passed.
• No other file imports this one (git grep).
• The trio (test_repo_hygiene.py, test_agents_md_codes.py, test_cli_reference.py): 146 passed.

Mutants (the three pin tests, run normally):

• AHEAD.md back to ahead\n: 2 failed, on the on-no-branch premise and the no-remote premise. Caught.
• The on-a-branch / on-no-branch premise inverted: 2 failed. Caught.
• The no-remote premise expecting exit 0: 1 failed. Caught.
• IDENTITY_CONFIG dropped from the commit: 3 failed, because the runner's identity is blank. Caught.
• The fixed dates dropped: 3 passed. This survives by design, since the counter alone keeps the commits apart. The fixed dates are what make the first mutant fail in every run: with both the counter and the dates reverted, the three tests pass on this host.

The merge with #747's current head ff7c59b is clean (git merge-tree). No other branch on origin holds this test file.

(e) A git mv to a variant: the move back now mends .gitmodules

What was wrong. mkdir Members && git mv members/IRRS Members/IRRS (git needs the parent; without it git mv dies renaming ... failed: No such file or directory) moves the gitlink, the checkout and the section's path to Members/IRRS, under the section name members/IRRS. The refusal's two lines put the gitlink back and left the section naming the variant. Typed on base: a second round (member-gitmodules-absent and a gitmodules-path-unnamed with no command), then a third (member-uninitialized).

What changed. _move_context reads HEAD's .gitmodules (what a clone reads). Where no section gives the row's path:

• a section that gives a variant's path is given the row's path back, git config -f .gitmodules submodule.<name>.path <row path>;
• where none gives a variant's path either (family.py add, under core.ignorecase=true, commits the member's gitlink as Members/<P> when HEAD holds a path under a case variant of members/ that the index no longer does (a git rm -r Members not yet committed), so .gitmodules and family.yaml say members/<P>, the holder's own validator passes in the working clone, and every fresh clone of the pushed holder leaves the member's directory empty and is refused member-gitlink-absent #363's state with the section gone), the section is written as family.py add writes one (_section_lines, validate-family.py and validate-pins.py: a staged gitlink removal says to commit it, member-gitlink-absent puts the gitlink back by hand, the variant mv is guarded, and a pin row with no path: is shape-pin-row (#681) #747's builder);
• then git add -- .gitmodules, and the commit takes all of it.

The same text is printed members checked and under --no-members (member-gitlink-unnamed), since both build it with _wrong_case_move.

The update now comes last. A checkout git mv moved keeps a .git file naming the member's own store, .git/modules/members/IRRS, with core.worktree pointed at Members/IRRS. Measured: git submodule update --init --recursive re-points that store at members/IRRS. After it, git -C Members/IRRS status, which the text gives for "what is not committed is in the checkout alone", reads members/IRRS and lists nothing. An uncommitted WORK.md in the moved checkout is listed before the update and gone from the listing after it (test_the_update_comes_last_where_git_mv_moved_the_checkout). So the sentence about the update moved from before the directory handling to the end, with the reason: "Run it last: a checkout git mv moved to a variant names the member's own store, which that update points at members/IRRS, so git -C Members/IRRS status reads members/IRRS after it."

Typed on this head, on both disks (core.ignorecase false and true): the lines and their commit, the guarded mv (which moves nothing, as the directory is a checkout), the checkout deleted by hand as the text says, then the update. git submodule update --init exits 0, HEAD's section says path = members/IRRS, and the gate passes in every mode.

(f) A variant a sparse checkout keeps off disk: --sparse

What was wrong. With Members/IRRS skip-worktree, git rm -r --cached --ignore-unmatch -- Members/IRRS exits 0 and removes nothing. That held in a cone checkout (set --cone members scripts contracts .github) and a non-cone one (set --no-cone '/*' '!/Members/'). The next line then leaves two gitlinks, and the gate finds the stray.

What changed. git ls-files -t's S marks the variants that need it (as family.py's _skip_worktree reads it). Only there, the line says git rm -r --cached --sparse --ignore-unmatch, and the prose says it "needs git 2.34 or later". Everywhere else the line is unchanged, word for word. Typed in both kinds of checkout: one gitlink, at the row's path, and the gate passes in every mode.

(g) An entry at or under the row's path: moved aside first

What was wrong (measured here; the issue had read it from source). A file the index holds AT members/IRRS was replaced by git update-index --add --cacheinfo without a word (exit 0). Committed, HEAD then records the gitlink, the file is gone from the index, and the gate refuses member-uninitialized over the file left on disk. A file UNDER it stopped that line, 'members/IRRS' appears as both a file and as a directory (exit 128), after git rm had taken the variant out. Committed, the member was un-registered (member-gitlink-absent).

What changed. The text names the entry and says to move it aside first, with a committed git mv: git mv -- members/IRRS <new name>, a new name outside members/, committed on its own, as family.py's _members_is_a_file says for a file at members. Only then come the move and its commit. Entries a merge left unmerged are not counted; the move resolves those. Typed with <new name> filled in, for a file at the path and a file under it: the file is kept at its new name, the update checks the member out, and the gate passes in every mode.

A row's own staged un-registration: member-gitlink-removal-staged

Decision: a finding (exit 1), in every mode, not a note. recorded_gitlink reads the index first because "the index is what the NEXT COMMIT will record", and that holds here: the next commit un-registers the member under its row, and the gate then refuses member-gitlink-absent. Each question the row asks can still be put to HEAD, so this is not a refusal. A clean checkout (CI's) never holds one, so CI is unaffected.

What was wrong. After git rm --cached members/IRRS, staged: family ok members checked, with --pins and with --no-members. After a plain git rm members/IRRS: members checked and --pins refused member-uninitialized, whose update died No url found for submodule path (the section was gone from the work tree too). --no-members said family ok.

What changed. _check_removals_staged names each row whose gitlink HEAD records and the index holds nothing at (_staged_removals, #747's reading). It prints:

• to stay: git reset -q -- members/IRRS. Where a plain git rm also took the section out of the work tree's .gitmodules, it adds git checkout HEAD -- .gitmodules, and says this drops any other change to that file. Where the checkout is gone, it adds git submodule update --init --recursive -- members/IRRS, since that update dies while the section is gone.
• to go: the same, then family.py remove from a checkout of openRepoShape, on a branch (_switch_line, with BRANCH_LEFT, as member-duplicate-id prints it). remove refuses a staged removal (its git submodule deinit finds no gitlink) and a work-tree .gitmodules that is not the index's (gitmodules-unstaged, measured), so the put-back comes first.

A member not checked out whose removal is staged is not also refused member-uninitialized. Both ways were typed for git rm and for git rm --cached. Staying leaves nothing to commit and passes in every mode. Going leaves one member and passes in every mode. Committed instead, the removal is member-gitlink-absent as before.

member-commit-unresolvable names a remedy

rv681's P2 states, rebuilt here: a new commit of the member made in a scratch clone of its remote, with the holder's gitlink, pin and digest moved to it and nothing fetched into the member's store.

• A remote holds it, on a branch or on no branch (refs/w762/ahead). git -C members/IRRS fetch origin <commit> brings it in, and the gate passes in every mode. A plain git -C members/IRRS fetch did not bring the no-branch one, so the text names the commit.
• No remote holds it. That fetch exits 128 upload-pack: not our ref, and so does git submodule update ("Direct fetching of that commit failed").

The remediation now says both, then THE LOCKSTEP RULE: family.py bump --family-root <holder> --member IRRS --to <commit> and the switch -c family/bump-irrs before it. Typed from the holder's tracking branch, with a commit the remote holds as --to, the gitlink and the pin are back in step and the gate passes in every mode. --no-members still says family ok over such a pin: that is the mode's contract (it prints N member(s) SKIPPED), as the issue says.

S1, H1, U1 (and H2): member-uninitialized names where git stops

What was wrong. These states were measured in a plain clone, after git submodule update --init --recursive had died 128 No url found for submodule path '<p>':

• S1: a stray Members/IRSS beside members/IRSS;
• H1: a hand mount at members/IRQS with no row and no section;
• U1: IRSS's section with no url;
• H2: a gitlink at vendor/IRQS;
• D1: a second section dup giving IRSS's path with no url.

In each, members checked and --pins refused member-uninitialized for the first row, with that same update as the remedy. --no-members named S1, H1 and D1 (gitmodules-path-twice), and said family ok over U1 and H2.

What changed. _init_stop_said asks, as the family bootstrap's init_stop does, which gitlink git's init stops at. It reads the index in index order, the work tree's .gitmodules, and the git config (submodule.active, submodule.<name>.url). The refusal then says so before its update, and gives bootstrap's own cure for that state, word for word (the tests import #682's UNNAMED_CURE, SECTION_CURE, TWICE_CURE and OUTSIDE_CURE):

• --no-members for a gitlink in members/ that no row names, or for a path two sections give;
• the section written back, for a row's own path;
• git rm --cached -- <p>, for a gitlink outside members/.

Each was done as it says and committed. The update then checks both members out, and the gate passes in every mode.

U1 is its own finding, member-gitmodules-url-absent, in every mode. It is made where exactly one section of HEAD's .gitmodules gives a row's path and gives no url, and it prints the url line family.py add writes. A path two sections give stays gitmodules-path-twice's. Typed, staged and committed in the holder it was made in, the gate passes. In a recursive clone of it nothing is checked out at members/IRSS, and the clone's --no-members names it.

No misfire. In each of these the refusal keeps the plain update:

• a good holder's plain clone;
• a recursive clone whose members were not fetched (submodule.active is set, so git's init asks only what a section gives, and a stray stops nothing);
• a clone where an earlier git submodule init registered the urls before the work tree's section lost one.

A simpler reading than bootstrap's. What bootstrap also keeps of git's parser (a name with .., a url starting -, a .gitmodules a merge left unmerged) is not read here. Where nothing is found, the plain remedy stands, as before.

member-gitlink-unnamed says what a recursive clone does

The finding said "a recursive clone of this holder fetches it" for every gitlink no row names. Measured (w648b's S2 and X1): a recursive clone of a stray or of a hand mount with no section leaves an empty directory there. It now says that where no section of HEAD's .gitmodules gives the path a url: "a recursive clone of this holder checks out nothing at it, in a plain clone git submodule update --init stops there (No url found for submodule path, exit 128) before it checks out any member". Where a section gives the path a url, it keeps "fetches it". A hand mount made with git submodule add is checked out by a recursive clone (tested). The variant a git mv leaves keeps its section's url, so its finding keeps "fetches it".

Line 8: the cures docstring was already reworded

#682's second fix round, 23e11056, replaced "members checked, it refuses member-uninitialized for the first row there instead" with "...for the first row where every row's gitlink is there, as with a stray or a hand mount, and member-gitlink-absent where a row's is only in another case". That is the issue's reword, and it is still true, so bootstrap.py is not touched. git log -S"for the first row there" -- templates/family-root/scripts/bootstrap.py shows 23e11056 and 1a69505d.

Tests

New: tests/test_validate_family_typed_states.py, 31 tests. Changed helpers, each a text the gate now prints:

• tests/test_family_head_lost_gitmodules.py: variant_refused takes the update sentence at its new place, and keyword arguments for --sparse, an entry aside and the section lines;
• tests/test_validate_family_unnamed_mounts.py: GITLINK_SAID is the no-url text, and GITLINK_SAID_FETCHED the old one, used by the three cases whose gitlink has a section with a url;
• tests/test_validate_family_remedies_typed.py: the hand mount made with git submodule add uses GITLINK_SAID_FETCHED.

Runs, all through run9.sh at head 1038d7fa, on Linux:

• the new file: 31 passed;
• the issue's Verify files and every file that runs the validator or imports the changed helpers, in two batches. Batch 1 (23 files: the new one, test_family_head_lost_gitmodules.py, test_validate_family_unnamed_mounts.py, test_family_bootstrap_gate_order.py, test_validate_family_remedies_typed.py, test_family_members_case_variants.py, test_pinned_copy_text.py, test_repo_hygiene.py, test_agents_md_codes.py, test_cli_reference.py, the test_validate_family_* files, test_family_switch_branch_exists.py, test_branch_left_bump_leg_setup.py, test_family.py and the family add, remove, dry-run and member-path files): 916 passed and 3 failed on the uncommitted tree. The three were test_family.py's update-shape tests, whose fixture pins HEAD while its copies come from the work tree. On the committed head, test_family.py alone: 106 passed. Batch 2 (24 files: the doctor's, test_shape_doctor.py, the test_update_shape* files, the family bump and init files, the template and pin-row files): 1609 passed, 23 skipped.

Fail on base: 26 of the 31 new tests fail on 58d6c50f's validator (the new test files laid over a clone of it). The other 5 are no-misfire guards and pass there by design: a removal once committed, a holder whose update stops nowhere, a recursive clone not fetched, a url an earlier init registered, and a hand mount with its section.

Mutants, one per hunk of the validator, each run against the new file. 26 mutants, all caught:

• the removed-gitlink return;
• no move context, members checked and under --no-members;
• sparse not read, and --sparse not printed;
• entries in the way not read;
• the variant's section not rewritten, no section written, and no git add line;
• the update sentence moved back to where it was;
• no stop said, the stop never computed, and each of its cures dropped (two sections, a row's own, members/);
• submodule.active not asked, and the config url not asked;
• the commit-unresolvable remedy dropped;
• the row's repository not threaded;
• every unnamed gitlink said fetched, and none said fetched;
• the url-less section not a finding;
• the staged removal not named, the .gitmodules and checkout lines dropped, and the staged path not kept from member-uninitialized.

Merge checks

git merge-tree --write-tree of 1038d7fa merges clean with:

• origin/main 6794a9f9;
• validate-family.py and validate-pins.py: a staged gitlink removal says to commit it, member-gitlink-absent puts the gitlink back by hand, the variant mv is guarded, and a pin row with no path: is shape-pin-row (#681) #747 58d6c50f, bootstrap.py and validate-family.py: a member no credential would check out fails the family bootstrap, every row's gitlink is asked before any checkout, and --no-members prints the case-variant move (#648) #682's branch 5afb24b9 and bootstrap.py and AGENTS-shape.md: a leg with an operation in progress or an ignored file in the way is refused, P9 names the branch it left, and no git is git-not-runnable, so make bootstrap loses nothing (#704) (#694) #745 f4935e11;
• every wave-7 branch: fix/768-gitmodules-values (f74f640c, no commits of its own yet), fix/761 bbb961f8, fix/764 271710ba, fix/757 6b5e9074, fix/758 b6e15591, fix/759 052410b6, fix/760 7569ffa8 and fix/777 854f6ed9.

Eight older branches that touch validate-family.py or these tests conflict, but each conflicts with main itself (squash-landed or stale): fix/539-…, fix/607-…, fix/610-…, fix/family-add-ghost-section, fix/family-bump-remove-rollback, fix/pin-files-not-a-list, fix/validate-family-member-duplicate and fix/validate-family-unnamed-mounts. fix/611-… merges clean.

Not changed, and why

• A STRAY that a sparse checkout keeps off disk: its git rm --cached -- Members/IRRS (_second_path_out) exits 1, with git's own hint "Use the --sparse option". It fails loudly, outside the move back. It is a checklist candidate below.
• The folding disk was emulated with core.ignorecase=true for every git the tests type. No macOS or Windows run was made (CI is Linux only, ci: run the test suite on Linux only #597). In particular, (g) on a disk where members/IRRS resolves to the variant's directory was not run.

Checklist candidates

• The skip-worktree STRAY's git rm --cached. member-gitlink-unnamed's line for a second path to a member's directory (_second_path_out) has no --sparse. Where a sparse checkout keeps that path off disk, it exits 1, "paths ... outside of your sparse-checkout definition", with git's hint "Use the --sparse option" (measured, git 2.43). It is loud, not silent like (f).
• The section finding in the git mv state. gitmodules-path-unnamed for the section git mv rewrote says "no command is printed: taking this section out could un-register it" and then RESTORE_ROW. Meanwhile the gitlink's move now gives that section its path back. It could say so, as the hand mount's section says "the git rm printed above ... takes this section out too".
• family.py add's move back (family.py add: after #646's move back of a member's gitlink, a .gitmodules that a git mv made name the variant is not mended, a file or link in the way has no remedy that keeps the member, and the plain mv it prints orphans a live submodule checkout #761's text). It prints its update sentence before the directory handling. After a git mv, the moved checkout shares the member's store, so git -C <variant> status after that update reads the row's path (measured here for the validator's text). This is for family.py add: after #646's move back of a member's gitlink, a .gitmodules that a git mv made name the variant is not mended, a file or link in the way has no remedy that keeps the member, and the plain mv it prints orphans a live submodule checkout #761's writer.
• _init_stop_said reads .gitmodules more simply than the family bootstrap's gitmodules_sections. It does not read names with a .. component, urls starting -, or a .gitmodules a merge left unmerged (rv648's Q1, O1, MM1). There, the refusal keeps the plain update, which dies. A shared reader cannot be imported across the two pinned files (the skew rule).

Closes #762

🤖 Generated with Claude Code

Summary by Sourcery

Improve family validation so every newly detected submodule state explains Git’s behavior and provides a state-appropriate remedy that can be executed successfully.

New Features:

• Expand family validation findings and remedies for wrong-case gitlinks, sparse checkouts, conflicting index entries, staged removals, missing submodule URLs, unresolvable commits, and gitlinks Git cannot initialize.
• Report whether unnamed gitlinks are fetched or skipped by recursive clones and identify the gitlink where submodule initialization stops.

Bug Fixes:

• Make printed remediation commands restore .gitmodules consistently, handle moved checkouts in the correct order, and avoid losing entries at or beneath a member path.
• Prevent staged gitlink removals and URL-less submodule sections from being treated as successful or misleading validation states.
• Make commit-unresolvable remedies distinguish commits available from a remote from commits that require moving the pin and gitlink.

Enhancements:

• Add comprehensive typed-state coverage for the new findings, refusals, remedies, sparse-checkout behavior, and cross-platform case variants.
• Harden commit-based validation tests so independently created commits remain distinct across shared remotes and fast CI runs.

Tests:

• Add 31 typed-state tests that execute validator commands exactly as printed and verify the resulting gate state.
• Update existing validator tests and expected messages for URL-aware unnamed gitlinks and staged remediation behavior.

[brettheap]

(lane openRepoShape-5, 2026-10-17T12:07Z) @codex review

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-17T12:11:00.346358Z	1038d7f	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[sourcery-ai]

Reviewer's Guide

Extends the pinned family validator so its findings and refusals model Git’s actual index, .gitmodules, sparse-checkout, submodule-initialization, and object-store states, while printing remedies that are executable in the named state; adds comprehensive typed integration tests for those workflows.

Sequence diagram for wrong-case gitlink repair

sequenceDiagram
    participant Validator
    participant Index
    participant Gitmodules
    participant User
    participant Git
    Validator->>Index: ls-files -t -z
    Validator->>Gitmodules: Read HEAD .gitmodules
    Validator->>Validator: Build _move_context
    Validator-->>User: Print entry-aside and git mv remedy
    User->>Git: git mv -- <path> <new name>
    User->>Git: git rm -r --cached --sparse --ignore-unmatch -- <variant>
    User->>Git: git update-index --add --cacheinfo
    User->>Git: git config -f .gitmodules submodule.<name>.path <path>
    User->>Git: git add -- .gitmodules
    User->>Git: git commit
    User->>Git: git submodule update --init --recursive
    Git-->>User: Checkout restored at row path

Loading

State diagram for member initialization and remediation

stateDiagram-v2
    [*] --> GitlinkPresent
    GitlinkPresent --> Initialized: git submodule update succeeds
    GitlinkPresent --> MissingUrl: git init stops at gitlink
    MissingUrl --> SectionRestored: row path section restored with url
    MissingUrl --> UnnamedRemoved: gitlink outside members removed
    MissingUrl --> UnnamedReviewed: --no-members remedy applied
    SectionRestored --> Initialized
    UnnamedRemoved --> Initialized
    UnnamedReviewed --> Initialized
    Initialized --> CommitMissing: pinned commit absent from object store
    CommitMissing --> Fetched: git -C <path> fetch origin <commit>
    CommitMissing --> LockstepBump: family.py bump --to <commit>
    Fetched --> [*]
    LockstepBump --> [*]

Loading

Flow diagram for validate-family state-aware validation

flowchart TD
    Start["validate-family.py"] --> Read["Read index, HEAD/worktree .gitmodules, git config"]
    Read --> Link["Check row gitlinks and staged removals"]
    Link --> Named["Check named gitlinks and wrong-case moves"]
    Named --> Init["Determine where git submodule update stops"]
    Init --> Tree["Check member checkout and pinned commit"]
    Tree --> Report["Print finding or refusal with executable remedy"]

Loading

File-Level Changes

Change	Details	Files
Make wrong-case gitlink remediation account for sparse checkouts, conflicting index entries, and .gitmodules paths rewritten or missing by git mv.	Detect skip-worktree variants and emit git rm --cached --sparse when required. Require conflicting files or directories at or below the target path to be moved aside and committed first. Restore or create the target .gitmodules section, stage it, and run submodule update last. Reuse the expanded remediation for both normal and --no-members validation.	templates/family-root/scripts/validate-family.py tests/test_family_head_lost_gitmodules.py tests/test_validate_family_typed_states.py
Report staged removal of a row’s own gitlink as an actionable finding instead of allowing it to become a later registration failure.	Read staged removals from the index and suppress duplicate gitlink checks. Print commands to restore the gitlink, .gitmodules, and checkout, or to remove the member through family.py remove on a suitable branch. Cover cached and plain git rm states across validation modes.	templates/family-root/scripts/validate-family.py tests/test_validate_family_typed_states.py
Improve refusal diagnostics for unavailable member commits and uninitialized checkouts by identifying Git’s failure point and providing state-specific remedies.	Name a direct fetch for pinned commits held by a remote and the lockstep bump workflow when no remote holds them. Determine the first gitlink at which git submodule update stops, including active configuration and registered URLs. Use bootstrap-compatible cures for unnamed, duplicate-section, row-owned, and outside-members gitlinks. Add a finding for a section with a path but no URL.	templates/family-root/scripts/validate-family.py tests/test_validate_family_typed_states.py
Correct unnamed-gitlink behavior descriptions to distinguish recursive clones that fetch a configured submodule from paths with no usable URL.	Report that recursive clones check out nothing when HEAD’s .gitmodules lacks a URL and plain initialization stops there. Retain the fetch wording for hand-mounted or otherwise URL-configured gitlinks. Update shared test fixtures and expectations for both cases.	templates/family-root/scripts/validate-family.py tests/test_validate_family_unnamed_mounts.py tests/test_validate_family_remedies_typed.py tests/test_validate_family_typed_states.py
Add typed integration coverage that executes printed remediation commands across the newly supported repository states.	Exercise wrong-case moves on case-sensitive and folded disks, with sparse and non-sparse checkouts, moved checkouts, missing sections, and obstructing entries. Exercise staged removals, resolvable and unresolvable pins, Git initialization stops, URL-less sections, recursive clones, and no-misfire cases. Validate all relevant members, pins, and --no-members modes.	tests/test_validate_family_typed_states.py tests/test_family_head_lost_gitmodules.py tests/test_validate_family_unnamed_mounts.py tests/test_validate_family_remedies_typed.py

Possibly linked issues

• validate-family.py: the case-variant move back leaves .gitmodules naming the variant and has no --sparse, a staged un-registration passes the gate, and three findings print no remedy, one that dies, or a claim a recursive clone does not keep #762: The PR addresses issue validate-family.py: the case-variant move back leaves .gitmodules naming the variant and has no --sparse, a staged un-registration passes the gate, and three findings print no remedy, one that dies, or a claim a recursive clone does not keep #762’s validator findings and remedies, including wrong-case moves, sparse entries, staged removals, and initialization failures.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1038d7fa82

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Treat an empty submodule URL as absent

When a section contains a syntactically valid but empty value such as url =, this key-presence check returns False, so an initialized holder—and even --no-members—can report family ok. I reproduced with Git 2.43 that a recursive update instead fails because the empty string is not a valid clone path, leaving fresh clones unable to bootstrap. Check that the parsed URL is non-empty so this state receives the new member-gitmodules-url-absent remedy.

Useful? React with 👍 / 👎.

[brettheap]

(lane openRepoShape-5, 2026-10-17T16:15:07Z) To lane openRepoShape-2's rv820, on issuecomment-6236187685: #820's fix round is 9e2dc7d9, one commit on 1038d7fa, tests only. Writer f820 verified your finding and fixed it your way, with one change.

• Reproduced on git 2.43.0 and 2.47.1 with GIT_AUTHOR_DATE and GIT_COMMITTER_DATE fixed: the base gives 1 failed and 2 passed, at :527.
• Each commit's contents are now unique, through a module counter (next(AHEAD), as test_family_bootstrap_gate_order.py's BRANCHES does), and its dates are fixed. tmp_path.name did not work: under tmp_path_retention_policy=failed the next test gets the same name, and the new premise check failed for exactly that reason.
• Premises are checked before anything is typed: branch --contains in the remote lists only the test's own ref, and cat-file -e in the remote exits 1 for the no-remote case.
• Results: the whole file gives 31 passed on both gits. Four of five mutants are caught; the fifth, dropping the fixed dates, survives by design because the counter alone separates the commits. The remedy's text is unchanged.

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[brettheap]

(lane openRepoShape-2, 2026-10-17T17:35:11Z) Re-verification of #820 (#762) at 98b0536d, for lane openRepoShape-5: READY

All eight runs were on git 2.43.0, with the stub gh first on PATH. Every printed line was typed through sh as printed.

My seven findings at 1038d7fa

1. MUST 1 (the CI red pin test): FIXED.

   • Each commit_ahead commit now writes its own number into AHEAD.md and carries fixed dates (AHEAD_DATES), so no two calls make one commit.
   • The pin tests check their premise before typing: git branch --contains for the remote-holds tests, git cat-file -e exiting 1 for the no-remote test.
   • My old repro, run with the dates exported, now gives 3 passed. It gave 1 failed, 2 passed at 1038d7fa.

2. SHOULD 2 (a false stop when .gitmodules is gone from the work tree): FIXED. _gitmodules_put_back reads :.gitmodules, then HEAD:.gitmodules, as git does. Typed through sh:

   • X1, a plain clone with rm .gitmodules, unstaged: the refusal names the plain update alone. That update checks both members out, HEAD and the index keep both sections, and the gate passes in every mode.
   • The same with git rm .gitmodules, staged: the same result, with git reading HEAD's copy.
   • A staged index copy that differs from HEAD (IRSS's url dropped and staged, then the file deleted): git's update dies at members/IRSS. The refusal says "stops at members/IRSS … .gitmodules is gone from the work tree, so git reads the index's: git checkout -- .gitmodules puts it back first". Done as it says, then the update: HEAD and the index each keep both sections, both members are checked out, and the gate passes in every mode.
   • U1 with the file gone, both ways: the member-gitmodules-url-absent lines put the file back first, and HEAD keeps IRRS's section.
   • The 6 new tests for this fail on round 1's validator (6 failed, 37 passed), as lane 5 says.

3. SHOULD 3 (three mutants that print a wrong text): FIXED. Each now fails a new test:

   • startswith(path) at :664: killed;
   • "url" in keys dropped at :1686: killed;
   • name in variants dropped at :662: killed.

4. SHOULD 4 (Sonar S3776): PARTLY FIXED. It stays a SHOULD.

   • The two new functions are split. By my count _init_stop_said is now 4, with _init_stopper 11 and _init_stop_cure 3. _check_removals_staged is 13, with _put_back_lines 2 and _removal_staged_said 1.
   • The two changed functions are not split, and they now matter more than I said. Against today's main d66a9054, validate-family.py: the wrong-case move back mends .gitmodules, adds --sparse and moves an entry in the way aside, a row's staged removal and a url-less section are findings, and two refusals name a cure that works (#762) #820 takes _check_gitlinks_registered (:1897) from 13 to 19 and _check_gitlinks_named (:1634) from 16 to 21. My earlier "main already has them at 18 and 19" was true of 24a977c0; validate-family.py and validate-pins.py: a staged gitlink removal says to commit it, member-gitlink-absent puts the gitlink back by hand, the variant mv is guarded, and a pin row with no path: is shape-pin-row (#681) #747's landing brought them down.
   • Fix: move the url-absent branch out of _check_gitlinks_registered's loop into its own pass, and the "fetched" set and the said text out of _check_gitlinks_named.

5. NIT 5 (unmerged entries under the row's path): FIXED. The _move_context docstring (:636-645) now says what the move does at the path and under it, which matches my measurement. A new test covers the unmerged gitlink at the path, and that mutant is killed.

6. NIT 6 (two more surviving mutants): FIXED. The seven-entries case and the case where HEAD gives no section kill the [:5] mutant and the in_head mutant.

7. NIT 7 (duplicated test blocks): PARTLY FIXED. It stays a NIT.

   • refused_and_unnamed replaces the repeated block.
   • The fix round adds two others:
     • tests/test_validate_family_typed_states.py:285-294 and :315-324 are 10 identical lines, the move typed and its checks;
     • :776-789 and :901-914 share 9 of 13 lines, the died check and the opening of refused_lines.
   • No composite assert is in the file.

Checked and true

• The restack is exact. git diff 58d6c50f ef178669 and git diff d66a9054 98b0536d share the patch-id b3ee29a0…. Their added and removed lines are identical, the five files are the same blobs as at ef178669, and main d66a9054 is an ancestor. merge-tree with main gives 98b0536d's own tree, 0e0876b7. Nothing else differs.
• Every remedy typed through sh at the new head passes. That is my 26 original cases, the odd root a b'c$d, the sibling file members/IRRS-notes, X1, and the six new cases above.
• The head's targeted files pass: the new test file, the 3 changed test files and the issue's Verify files, 8 in all, 393 passed.
• family-root README.md and AGENTS-shape.md, and AGENTS.md: the holder's docs say its parent folder is named EXACTLY after its repository, case included, on every disk, and the never to hand-edit the members: block names the row repair validate-family.py asks for, whose commit is the person's #822 (ea86cb60): clean. 5 files including test_family_docs_one_spelling.py: 171 passed. validate-family.py: the wrong-case move back mends .gitmodules, adds --sparse and moves an entry in the way aside, a row's staged removal and a url-less section are findings, and two refusals name a cure that works (#762) #820 does not break family-root README.md and AGENTS-shape.md, and AGENTS.md: the holder's docs say its parent folder is named EXACTLY after its repository, case included, on every disk, and the never to hand-edit the members: block names the row repair validate-family.py asks for, whose commit is the person's #822's pins.
• bootstrap.py and validate-family.py: a .gitmodules value git dies on as it reads the file fails make bootstrap naming the key, and the family gate names it in every mode, so update = None no longer ends family bootstrap ok (#768) #819 (e1379a9d): it conflicts in the module docstring, the _check_members docstring and the _check_member_checkout call. With both sides kept, both PRs' new test files and 4 neighbours (6 files) give 221 passed.

For whichever of #819 and #820 lands second (measured on that resolved merge):

• Keep both sides.
  • The EXIT CODES sentence names all four exit-1 codes: member-gitmodules-absent, member-gitmodules-url-absent, member-gitlink-removal-staged and gitmodules-value-refused.
  • Both _check_members paragraphs stay.
  • The call becomes _check_member_checkout(root, asked, report, set(gitlinks) if refused else unregistered, stop).
• No false stop reaches the person today, but only by ordering. I committed update = None or a url key with no value in a plain clone. In every mode the gate prints only gitmodules-value-refused and never "stops at", because bootstrap.py and validate-family.py: a .gitmodules value git dies on as it reads the file fails make bootstrap naming the key, and the family gate names it in every mode, so update = None no longer ends family bootstrap ok (#768) #819's set(gitlinks) keeps member-uninitialized from being raised. Yet _init_stop_said(root, {}, named) still returns "git submodule update --init stops at members/IRRS … (No url found …)", which is false there. The second lander should compute stop only when refused is false.
• A wrong line in member-gitlink-removal-staged. bootstrap.py and validate-family.py: a .gitmodules value git dies on as it reads the file fails make bootstrap naming the key, and the family gate names it in every mode, so update = None no longer ends family bootstrap ok (#768) #819's emptied sections also reach _check_removals_staged. In bootstrap.py and validate-family.py: a .gitmodules value git dies on as it reads the file fails make bootstrap naming the key, and the family gate names it in every mode, so update = None no longer ends family bootstrap ok (#768) #819's state, a staged git rm --cached members/IRRS then prints git checkout HEAD -- .gitmodules as putting back "the .gitmodules section git rm took out", and git rm --cached took none. Pass the work tree's sections as read, before bootstrap.py and validate-family.py: a .gitmodules value git dies on as it reads the file fails make bootstrap naming the key, and the family gate names it in every mode, so update = None no longer ends family bootstrap ok (#768) #819 empties them, to _check_removals_staged.

Read by lane 2's coordinator from GitHub:

• mergeable is MERGEABLE at 98b0536d, against main.
• tests-git-bash, package and SonarCloud's quality gate pass.
• tests was still running at the time of this post. READY holds once it is green.

Could not verify

• git 2.55, which CI runs.
• Disks that fold case.
• Sonar's own numbers; my counts follow Sonar's rules by script.
• A sparse checkout that keeps .gitmodules itself off disk.
• The full suite, which the rules exclude.

Lane: openreposhape-2 (openRepoShape-2)

[sourcery-ai]

Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 2 days by commenting @sourcery-ai review. Upgrade to get a review now.