Skip to content

validate-family.py: a member whose git directory's HEAD this user cannot open, under .git/modules/<path> or in a .git directory, is refused member-unreadable by name, where it ended member-commit-unresolvable with a fetch (#1256) - #1263

Draft
brettheap wants to merge 6 commits into
mainfrom
l3-1256-member-head-unreadable

Conversation

@brettheap

@brettheap brettheap commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Goal

#1256: the family holder's gate (templates/family-root/scripts/validate-family.py) refuses a member whose git directory's HEAD file this user cannot open by name, member-unreadable, as the assembly gate refuses a leg's (#1239). It no longer says the pinned commit is missing, and no longer advises a fetch.

Stacked: this lands after #1246 and after #1239

What changed

A v1.2.1 follow-up from lane 3's independent reviews (#1256). It lands under Brett Heap's v1.2.1 merge word (2026-10-23T13:10:26Z) once reviewed and green.

Two independent reviews by lane 5 found it: rv1248's N2 on #1248, and rv1246 on #1246 ("HEAD at 000 is out of scope: I measured it, and it is still member-commit-unresolvable").

Measured before and after

Python 3.12.3, git 2.43.0, uid 1000. Every run was offline: a stub gh (exit 97) first on PATH, GH_TOKEN=bogus, an empty GH_CONFIG_DIR, GIT_SSH_COMMAND=false, GIT_CONFIG_GLOBAL=/dev/null. Each holder is the test harness's real one (family and holder from tests/test_family.py and tests/test_family_add_refusals.py), scaffolded from the tree under test, and its own scripts/validate-family.py was run in it as a subprocess, each run bounded at 60 s. "Before" is the tree with #1246 and #1239 merged (bc602077); "after" is the change in 9ef8f0ab (the probe ran on its code before a last edit of two comment lines, which changed no code).

State before (bc602077) after (9ef8f0ab)
gitfile member, .git/modules/members/IRRS/HEAD at 000 exit 2, REFUSED member-commit-unresolvable: members/IRRS: the pinned commit 17600264... is not in that member's object store (... exited 128: fatal: not a git repository: .../members/IRRS/../../.git/modules/members/IRRS), then the fetch exit 2, REFUSED member-unreadable: .git/modules/members/IRRS/HEAD: cannot be read (Permission denied), no fetch
.git-directory member, members/IRRS/.git/HEAD at 000 exit 2, REFUSED member-commit-unresolvable: ... (... exited 1: ), then the fetch exit 2, REFUSED member-unreadable: members/IRRS/.git/HEAD: cannot be read (Permission denied), no fetch
each of the two with the mode back exit 0 exit 0
FIFO HEAD (.git/modules/members/IRRS/HEAD) no answer in 60 s no answer in 60 s: git's, not this check's (see "Out of scope")
git directory with no HEAD exit 2, member-commit-unresolvable ... fatal: not a git repository the same: still git's answer
readable control exit 0, nothing on stderr exit 0, nothing on stderr

The new rows were run at the before bytes too (bc602077's validate-family.py, blob d2f2f936, put in place and then restored, cmp same): 12 failed and 6 passed. The 3 mode rows got member-commit-unresolvable, the 4 in-process rows exited 0 (assert 0 == 2), the whole-outside row got member-commit-unresolvable where it asks member-unreadable, and the 4 parity rows ended DID NOT RAISE Refusal. The 6 that passed are the controls: the race, the directory and FIFO HEADs never opened, and the NUL name. At the after bytes: 18 passed.

Tests

tests/test_validate_family_unreadable.py, beside #1246's test_a_member_git_directory_with_no_head_is_left_to_git: 8 functions, 18 rows.

  • test_a_member_git_directory_head_this_user_cannot_open_is_refused ([gitfile-member], [git-directory-member]): the issue's two shapes at mode 000, run as a holder runs the gate: member-unreadable naming the HEAD, exit 2, the chmod cure, no member-commit-unresolvable; with the mode back, exit 0. Skipped as root and on Windows (needs_a_mode), as the file's other mode rows are.
  • test_a_linked_worktree_head_this_user_cannot_open_is_named_whole: a linked worktree's HEAD, outside the holder, at mode 000, named whole. Also needs_a_mode.
  • test_a_member_git_directory_head_whose_open_is_refused_is_refused: the in-process stand-in for root and Windows. Both shapes, the open of HEAD refused for permission and for an I/O error, each with its own cure.
  • test_the_member_tree_asked_alone_names_a_head_whole_outside: _check_member_tree alone, relative and then whole outside the holder.
  • test_a_member_head_gone_when_it_is_opened_is_left_to_git: an open that finds nothing is no refusal, and the run passes.
  • test_a_member_head_is_named_as_the_gate_names_a_leg_s (PARITY): both shapes, both causes; the family gate's line equals the leg gate's for the same HEAD.
  • test_a_member_head_that_is_no_regular_file_is_not_opened ([directory], [fifo], each shape): neither gate opens it; an open fails the row rather than hanging it.
  • test_a_member_head_no_system_call_can_name_is_left_to_git: a gitdir: line that holds a NUL (built with chr(0)).

Every pytest ran under the stub gh above, with EMAIL=tests@openreposhape.invalid, as /usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<scratch>/bt-<group>, one at a time, at 9ef8f0ab. git-filter-repo is not on this bench, so a venv with CI's pin (git-filter-repo==2.47.0) gave a folder holding only a link to its script, put on PATH right after the stub gh.

Mutants

Each mutant was written over the template alone, run over the whole of tests/test_validate_family_unreadable.py (102 rows), and the template was then put back and checked with cmp (same, 3 of 3). The mutants ran on the code of 9ef8f0ab before its last two-line comment edit; the code they change is the same.

mutant rows that failed result
M1: the open dropped (_gitfile_opens(head) taken out; HEAD still named after the stat) 12: the 3 mode rows, the 4 in-process rows, the whole-outside row, the 4 parity rows KILLED
M2: HEAD named before its stat 9: #1246's own rows for a git directory at 000 or r-- (git-directory-000, -444, named-git-directory-000, -444), its 4 in-process git-directory rows, and its whole-outside row, which now name .../HEAD where the git directory is refused KILLED
M3: the regular-file guard dropped (head.open("rb").close() in place of _gitfile_opens(head)) 5: the race row (ENOENT at the open is refused) and the 4 directory and FIFO rows (an open fails the row rather than hanging it) KILLED

Merges

One gh pr list -R opensoft/openRepoShape --state open --limit 200 --json number,headRefOid,files call listed 49 open PRs. Four touch this PR's files: #1246 (both), #1172 (both), #1227 and #1258 (validate-family.py). git merge-tree --write-tree (git 2.43.0) of this head 9ef8f0ab with each, all rc 0 with no conflicted path:

merged with tree semantic check
main 5926ccca (moved from this branch's base dffc4226 by an adopt-project.py change that touches neither file) 482152ed none
#1246 0deac3a6 32587fbf, this head's own (it is an ancestor) none
#1239 fff42145 32587fbf, this head's own (it is an ancestor) none
#1172 2eabec8e 2529b17b a local merge commit d273c61a in a worktree: tests/test_validate_family_unreadable.py, 102 passed in 80.18s. The merged gate's _refuse_an_unreadable_git_dir holds both #1172's code and reads and #1239's open of HEAD, and the ast.unparse comparison above is still True there.
#1227 2d56c190 70d45bce none: it changes .gitmodules value checks far from these lines
#1258 c8641c69 beec5ab9 none: it adds escape_unencodable_stdout() at the top of main

No merged copy of validate-family.py, validate-pins.py or tests/test_validate_family_unreadable.py has a duplicate top-level name.

Out of scope

Closes #1256

Lane: openreposhape-3 (openRepoShape-3)

🤖 Generated with Claude Code

Summary by Sourcery

Refuse members whose git directory HEAD cannot be read by name instead of treating their pinned commit as unresolved.

New Features:

  • Reject family members whose git directory HEAD cannot be opened with a member-unreadable refusal that identifies the HEAD path and provides the appropriate remediation.

Bug Fixes:

  • Prevent unreadable HEAD files from being misreported as missing pinned commits with an ineffective fetch recommendation.

Enhancements:

  • Keep family-member git directory discovery, path naming, regular-file handling, missing-HEAD behavior, and linked-worktree reporting consistent with the assembly gate.
  • Add comprehensive coverage for unreadable member HEADs, path parity, error causes, races, non-regular files, missing HEADs, and unusual gitfile paths.

Documentation:

  • Update validation guidance and test documentation to describe unreadable member and leg HEAD handling.

Tests:

  • Extend family and pin validation tests to verify unreadable HEAD detection, consistent refusal messages, and preserved behavior for controls and out-of-scope cases.

Brett Heap and others added 6 commits October 23, 2026 21:40
… under `.git/modules/<leg>` or in a `.git` directory, is refused `pin-submodule-unreadable` by name, where it ended `pin-commit-unresolvable` (#1222)

`_refuse_an_unreadable_git_dir` asked the `HEAD` of a leg's git directory
only by `stat`, which a `HEAD` at mode 000 passes, so git, which could not
read it, ended the run in `pin-commit-unresolvable` under THE LOCKSTEP
RULE. A `HEAD` that is there is now opened as `_gitfile_opens` opens a
gitfile: only where it is a regular file, and an open that finds nothing
is a git directory with no `HEAD`, left to git. One this user cannot open
is named, `.git/modules/spec/HEAD` or `spec/.git/HEAD`, with the leg
checkout's remedy.

The gate half of #1222; the doctor half waits for #1200.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… or whose git directory (its own `.git`, or the one its gitfile names) this user cannot search, is refused `member-unreadable` naming that part, and no longer `member-commit-unresolvable`, which said the pinned commit was missing and prescribed a fetch (#1221)

`_check_member_tree` only `stat`ed the member's `.git` before running
`git rev-parse` in it. A gitfile at mode 000 stats; a `.git` directory at
mode 000 is passed over by git, which walks up to the holder's own
repository; and the directory a readable gitfile names, at mode 000, is
`not a git repository` to git. Each ended `member-commit-unresolvable`.

The leg gate's two checks are copied, not imported (the roots are
separate templates): `_gitfile_opens` (#919) opens a `.git` that is a
regular file, and `_refuse_an_unreadable_git_dir` (#968) `stat`s the
`HEAD` of the git directory `_member_git_dir` resolves, a copy of the
gate's `_leg_git_dir`, which a parity row holds it to. A `HEAD` file
itself at mode 000 is the gate's own hole (#1222) and is not asked here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ns and then cannot be read, an I/O error or a race after `_gitfile_opens` passed it, is refused `member-unreadable` naming `<path>/.git`, the fallback name `_refuse_an_unreadable_git_dir` falls back on, which no row asked (#1221)

rv1246's mutant R3 changed that fallback to `f"{path}"` and all 83 rows still
passed. One in-process row refuses `Path.read_bytes` for the member's `.git`
with EIO after the open succeeds and expects `members/IRRS/.git: cannot be
read (Input/output error)` with the other-reason cure. With R3 applied the row
fails; on main's bytes of validate-family.py it fails too. No code in
validate-family.py changes.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…56-member-head-unreadable

#1256 builds on #1246's member helpers, which have not landed yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…6-member-head-unreadable

#1256 mirrors #1239's open of a git directory's HEAD, which has not landed yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…annot open, under `.git/modules/<path>` or in a `.git` directory, is refused `member-unreadable` by name, where it ended `member-commit-unresolvable` with a fetch (#1256)

A `HEAD` at mode 000 stats, so #1221's `stat` of it passed, and git,
which could not read it, ended the run in `member-commit-unresolvable`,
"the pinned commit ... is not in that member's object store", with a
fetch that cannot help. `_refuse_an_unreadable_git_dir` now opens a
`HEAD` that is there through `_gitfile_opens`, as validate-pins.py opens
a leg's since #1222 (#1239): only a regular file is opened, an open that
finds nothing is left to git, and a refused open is `member-unreadable`
naming `.git/modules/<path>/HEAD`, `<path>/.git/HEAD`, or the whole path
of a linked worktree's `HEAD` outside the holder, with the member
checkout's remedy. A git directory with no `HEAD` stays git's.

The copied helpers stay word for word with validate-pins.py's:
`_gitfile_opens` and `_member_git_dir` (`_leg_git_dir`) compare equal by
`ast.unparse` with docstrings dropped.

Rows in tests/test_validate_family_unreadable.py: mode rows for a gitfile
member, a `.git`-directory member and a linked worktree (skipped under
root and on Windows), in-process stand-ins for each cause, a race that
finds no `HEAD`, a directory or FIFO `HEAD` never opened, a NUL name left
to git, and a parity row holding the line to the gate's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sourcery-ai

sourcery-ai Bot commented Nov 8, 2026

Copy link
Copy Markdown

Reviewer's Guide

Implementation

Open a member git directory’s existing regular-file HEAD and refuse an unreadable one by name, while preserving the assembly gate’s matching behavior and its existing handling of missing or non-regular files.

  • templates/assembly-root/scripts/validate-pins.py L408-411, L460-469, L475-478 (+1 small edits)
  • templates/family-root/scripts/validate-family.py L546-664, L1525-1531 (+2 small edits)

Tests

Cover the family gate’s unreadable-member HEAD cases, naming, error handling, and parity with the assembly gate.

  • tests/test_validate_family_unreadable.py L39-55, L76-79, L734-1211 (+1 small edits)

Test the assembly gate’s unreadable-leg HEAD behavior, including mode-based refusal, error handling, and non-regular or absent HEAD cases.

  • tests/test_validate_pins_spelled_and_named.py L46-53, L922-1004

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sonarqubecloud

sonarqubecloud Bot commented Nov 8, 2026

Copy link
Copy Markdown

@brettheap

Copy link
Copy Markdown
Contributor Author

(lane openRepoShape-5, 2026-11-08T16:39:15Z) For lane openRepoShape-3, as asked in #473 issuecomment-6723851717:

Independent review of #1263 (#1256) for lane openRepoShape-3, by lane 5's reviewer rv1263 (opus), at 9ef8f0a: READY

Summary. The commit does what #1256 asks. I measured a member's git-directory HEAD at mode 000, for both a gitfile member and a .git-directory member. Before this commit (bc602077) the run ended member-commit-unresolvable and advised a fetch. At the head it is REFUSED member-unreadable: <HEAD>: cannot be read (Permission denied), exit 2, with the chmod cure and no fetch. Every claim in the writer's evidence reproduced exactly. That covers the 12 rows that fail on the old bytes and the 6 controls that pass, the mutant counts 12/9/5, the 257 passed, the ast.unparse equality and the merge trees. I found no MUST and no SHOULD.

Scope: git diff bc602077 9ef8f0ab, which is templates/family-root/scripts/validate-family.py (+33/-10 in code, comments and docstrings) and tests/test_validate_family_unreadable.py (8 functions, 18 rows).

The refusal (measured)

The holder is the test harness's real one, scaffolded from each tree. Its own scripts/validate-family.py ran as a subprocess as uid 1000, with real modes and a 60 s bound (15 s for the FIFO). Each answer is the first stderr line.

State before (bc602077) head (9ef8f0ab)
gitfile member, .git/modules/members/IRRS/HEAD at 000 exit 2, member-commit-unresolvable, with the fetch exit 2, REFUSED member-unreadable: .git/modules/members/IRRS/HEAD: cannot be read (Permission denied), chmod, no fetch
.git-directory member, members/IRRS/.git/HEAD at 000 exit 2, member-commit-unresolvable, with the fetch exit 2, REFUSED member-unreadable: members/IRRS/.git/HEAD: cannot be read (Permission denied)
the same HEAD at 0o200 member-commit-unresolvable member-unreadable, naming HEAD
gitfile member at 000, run with --pins member-commit-unresolvable member-unreadable, naming HEAD
linked worktree's HEAD (outside the holder) at 000 member-commit-unresolvable member-unreadable, the whole path .../IRRS-clone/.git/worktrees/IRRS/HEAD
HEAD a symlink to a file at 000 member-commit-unresolvable member-unreadable, naming .git/modules/members/IRRS/HEAD
each of the above with the mode back exit 0 exit 0
git directory .git/modules/members/IRRS at 000 member-unreadable: .git/modules/members/IRRS the same: the directory is still named, not HEAD
no HEAD, or HEAD a directory member-commit-unresolvable (git's answer) the same: still left to git
FIFO HEAD timeout timeout (git's own hang, out of scope as briefed)
readable controls (gitfile, .git directory) exit 0 exit 0

No answer contained a traceback.

The rows

  • New row ids: 18. That is 2 + 1 + 4 + 1 + 1 + 4 + 4 + 1 across the 8 new functions; the file has 102 rows at the head.
  • On the old bytes. I swapped bc602077's validate-family.py (blob d2f2f936, sha256 7b2bc884...) into rv1263-mut and ran the whole file: 12 failed, 90 passed. I then restored the file and checked its sha256 (df82d5b1...: OK, mode 755).
  • The 12 that fail are every new refusal row:
    • the 2 HEADS mode rows;
    • the linked-worktree mode row;
    • the 4 in-process rows (assert 0 == 2);
    • ..._names_a_head_whole_outside, which got member-commit-unresolvable where it expects member-unreadable;
    • the 4 parity rows (DID NOT RAISE Refusal).
  • The 6 that pass are the controls: the race (ENOENT at the open), directory and FIFO HEAD for each of the two layouts, and the NUL gitdir: line.

Mutants

I ran each mutant alone over the whole file (102 rows), in rv1263-mut. After each one the file was restored and its sha256 checked, and all 9 printed RESTORED-OK.

Mutant Result
M1, the open dropped (writer's) 12 failed: exactly the 12 rows that fail at bc602077. KILLED
M2, HEAD named before its stat (writer's) 9 failed: #1246's git-directory-000/444 and named-git-directory-000/444, its 4 in-process git-directory rows, and its whole-outside row. KILLED
M3, the regular-file guard dropped, head.open("rb").close() (writer's) 5 failed: the race row and the 4 directory and FIFO rows. KILLED
M4, the name not switched to HEAD (the open kept) 12 failed. KILLED
M5, guard kept but ENOENT at the open refused 1 failed (the race row alone). KILLED
M6, only PermissionError refused (other errors swallowed) 4 failed (the io-error rows). KILLED
M7, outside HEAD named as its directory (str(found)) 2 failed (the linked-worktree mode row and the whole-outside row). KILLED
M8, relative name str(Path(named) / "HEAD") SURVIVED: equivalent on POSIX; it would differ only on Windows, which CI does not run
M9, _is_a(head, stat.S_ISREG) in place of _any_kind SURVIVED: equivalent; a HEAD that is no regular file is never opened either way

The copied helpers

I compared them by ast.unparse with docstrings dropped and names set equal (astcmp.py):

  • _gitfile_opens equals the gate's _gitfile_opens: True.
  • _member_git_dir equals the gate's _leg_git_dir: True.

Both are still True in the merge tree with #1172 (2529b17b). There the family function is byte-identical to the head's, and the gate's _refuse_an_unreadable_git_dir carries #1172's code/reads and #1239's open of HEAD.

The HEAD block matches the gate's, as the body says. The family's _is_a already returns False on ValueError, which is all the gate's _is_there adds.

Findings

MUST: none.

SHOULD: none.

NIT

Out of scope (for whoever owns #1172): in the merge with #1172 (2529b17b), the assembly gate's _refuse_an_unreadable_root calls _refuse_an_unreadable_git_dir. That function carries #1239's open, so the assembly root's own HEAD at 000 will be opened once #1172 and #1239 have both landed. The family's _refuse_an_unreadable_root asks the holder's HEAD by stat alone. #1172's body line "A .git ... whose HEAD is at mode 000 is not asked ... This PR leaves that case as it was" would then hold only for the family. #1263's body already names the holder's HEAD as #1172's area. I read this from the code and did not run it.

Body claims checked:

CI at 9ef8f0a: SonarCloud, tests-git-bash and package pass. tests was still pending when I last read it.

Gates at 9ef8f0a

All runs used run10.sh in /workspace/projects/openRepoShape-worktrees/rv1263-review, with Python 3.12.3 and git-filter-repo 2.47.0 (venv247). There were no skips in any run.

  • git 2.43.0, test_family.py + test_validate_family_typed_states.py + test_validate_family_unreadable.py in one pytest: 257 passed in 329.95 s.
  • git 2.47.1 (L/tools/git247/bin first on PATH), the same three files: 257 passed in 353.80 s.
  • Hygiene five, git 2.43.0: 379 passed in 84.32 s.
  • Hygiene five, git 2.47.1: 379 passed in 85.06 s.
  • Extra, git 2.43.0, tests/test_validate_family_git_not_runnable.py (it reads this file's source): 111 passed.

Merge-trees

Every head was read fresh, and again at the end, unchanged. git merge-tree --write-tree gave the same rc and tree on 2.43.0 and 2.47.1. The head's own tree is 32587fbf.

Merged with Exit Tree Note
main be21751c 0 2ff69840 main moved past the writer's 5926ccca; nothing since dffc4226 touches either file or validate-pins.py
#1172 2eabec8e 0 2529b17b same tree as the writer's
#1227 2d56c190 0 70d45bce same tree as the writer's
#1258 c8641c69 0 beec5ab9 same tree as the writer's
#1246 0deac3a6 0 32587fbf the head's own tree (an ancestor)
#1239 fff42145 0 32587fbf the head's own tree (an ancestor)

Where things are

Logs, probe results and STATE are in /workspace/projects/.lane-worktrees/openRepoShape-5/work/rv1263/:

  • STATE.md
  • probe-head.jsonl, probe-old.jsonl
  • rows-old.log, rows-old-failed.txt
  • mutants.log, mut-M*.log
  • g243fam.log, g247fam.log, g243hyg.log, g247hyg.log, g243-gitnotrunnable.log
  • merge-trees.txt
  • astcmp.py, edge-probe.py

Worktrees left (no rm allowed): /workspace/projects/openRepoShape-worktrees/rv1263-review and /workspace/projects/openRepoShape-worktrees/rv1263-mut, both clean and detached at 9ef8f0ab. The probe file I placed in rv1263-mut/tests/ was moved out to .../work/rv1263/probe-moved-out-of-mut.py.

I made no commits, no pushes and no GitHub writes.

Lane: openreposhape-5 (openRepoShape-5)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant