Skip to content

validate-family.py keeps checking a member at the row's path: after it has found that path wrong (member-path), so a hand-edited path: "members/*" reaches recorded_gitlink as a glob pathspec, which answers with a neighbour's gitlink, and the run ends in member-uninitialized, whose remedy (git submodule update --init, make bootstrap) cannot help, while the member-path finding that names the hand edit is gathered and never printed #367

Closed

Description

@brettheap

Lane: openreposhape-4 (openRepoShape-4)

Named out of scope by #299's writer (PR #315, head 58aa4e3), "repo_shape.recorded_gitlink hands a row's path to git ls-files/ls-tree as a pathspec, read-only, from validate-family.py and shape-doctor.py". This lane's adversarial review of #299 recorded the same as OOS-3: "a glob there could read a neighbour's gitlink". Re-verified on main c7c02fa and on a scratch merge of #318's head 0f2a0dd (which carries c7c02fa) with #317's head c23e2b0, with local bare remotes, an empty global git config, GIT_ALLOW_PROTOCOL=file and a gh that exits 97 (never called). What did not reproduce: a wrong verdict. Every row path other than members/<project> is already a member-path finding, so the glob read never turns a run green. What does reproduce is the misleading way the run ends, below, and this issue is written about that. Not probed: the other callers of recorded_gitlink (bump-leg.py, the assembly root's validate-pins.py and bootstrap.py, the doctor's legs row and #211's family_beside_project), whose paths come from files their own validators check.

What is wrong. _check_member (templates/family-root/scripts/validate-family.py:358 at c7c02fa) calls _check_member_repository_and_path, which records member-path for a row path that is not members/<project>. That function then returns the row's path anyway, and the gitlink, tree and identity checks run against it. recorded_gitlink (scripts/repo_shape.py:999) lists git ls-files -s -- <path> and then git ls-tree HEAD -- <path>, both pathspecs, and takes the first gitlink line. _check_member_tree then refuses for the missing directory. main prints a refusal and returns 2 before it prints any finding:

U=<openRepoShape checkout>   # a clean checkout at c7c02fa
# Runs from a clean checkout of openRepoShape, in bash, on Linux (GNU coreutils). Nothing leaves the machine: members and
# holder live in local bare repositories under $R, git reads an empty global config, GIT_ALLOW_PROTOCOL=file makes any
# accidental https/ssh clone fail, and a `gh` that exits 97 stands first on PATH.
R=$(mktemp -d); mkdir $R/bin; printf '#!/bin/sh\nexit 97\n' > $R/bin/gh; chmod +x $R/bin/gh; touch $R/gitconfig
export PATH=$R/bin:$PATH GIT_CONFIG_GLOBAL=$R/gitconfig GIT_CONFIG_NOSYSTEM=1 GIT_ALLOW_PROTOCOL=file \
       GIT_AUTHOR_NAME=t GIT_AUTHOR_EMAIL=t@t.invalid GIT_COMMITTER_NAME=t GIT_COMMITTER_EMAIL=t@t.invalid
for P in Atlas Nova Zed; do
  python3 $U/scaffold-project.py --org testorg --project $P --elected-by 'Test Human' --elected-on 2026-10-12 \
      --local-remote-dir $R/remotes --work-dir $R/work >/dev/null
done
python3 $U/scripts/family.py init --org testorg --family Fam --created-by 'Test Human' --created-on 2026-10-12 \
    --local-remote-dir $R/fam --work-dir $R/h >/dev/null 2>&1
H=$R/h/Fam; cd $H
add() { python3 $U/scripts/family.py add --family-root $H --member "$1" --local-remote-dir $R/remotes; }
add testorg/Atlas >/dev/null; add testorg/Nova >/dev/null
python3 - <<'PY'                    # a hand edit of Nova's row: path: "members/*"
t = open("family.yaml").read()
open("family.yaml", "w").write(t.replace("    path: members/Nova\n", '    path: "members/*"\n'))
PY
python3 scripts/validate-family.py; echo "validate: exit $?"
python3 - <<'PY'                    # what the validator read and gathered before that refusal
import importlib.util, sys
from pathlib import Path
spec = importlib.util.spec_from_file_location("vf", "scripts/validate-family.py")
vf = importlib.util.module_from_spec(spec); spec.loader.exec_module(vf)
rows = {r["project"]: r for r in vf.load_yaml(Path("family.yaml"))["members"]}
print("recorded_gitlink('members/*') =", vf.recorded_gitlink(Path("."), "members/*")[:12],
      "| Atlas pin", rows["Atlas"]["pin"]["commit"][:12], "| Nova pin", rows["Nova"]["pin"]["commit"][:12])
report = vf.Report()
try:
    vf._check_member(Path(".").resolve(), rows["Nova"], {}, report)
except vf.Refusal as exc:
    print("raised", exc.code, "after gathering:")
for finding in report.findings:
    print("  " + finding.splitlines()[0][:104])
PY

Output at c7c02fa (the merge of #318 and #317 prints the same, commit ids aside):

REFUSED member-uninitialized: members/* is not an initialized submodule checkout, so neither the pinned digest nor the project it claims can be read. Presence is not identity and an unreadable surface must fail rather than degrade.
Remediation: `git submodule update --init --recursive`, or `make bootstrap`, which resolves a credential for private members first.
validate: exit 2
recorded_gitlink('members/*') = e2b66c353f5f | Atlas pin e2b66c353f5f | Nova pin 0e8d09c9656b
raised member-uninitialized after gathering:
  FINDING member-path: Nova: path is 'members/*', expected 'members/Nova'
  FINDING member-gitlink-mismatch: members/*: gitlink e2b66c353f5f42522565f305341602bba2c2db98 != family.y

So the operator sees one refusal, and its remedy cannot help. Two findings were gathered and dropped: the one that names the hand edit, and a member-gitlink-mismatch that pairs Nova's pin with Atlas's commit, a false statement that would mislead if it were printed. The same run ends the same way, in both the default mode and --pins, for members/A*, members/[AN]*, :(glob)members/Nova and members/Nova/... A trailing slash (members/Nova/) gives only the member-path finding (exit 1), which is right.

Why. Three things combine:

  1. The member checks go on at a path the row has just been found not to have.
  2. recorded_gitlink reads that path as a pathspec. Taken literally, the answer would be "no gitlink there".
  3. A refusal discards the findings gathered before it.

Since #299 (PR #315), bump and remove refuse such a row before any git call (_member_path). The validator, which ships into every holder and runs in CI and in the doctor, is now the one reader that acts on it.

What it should do. A hand-edited path: is reported as member-path, and nothing read through it is reported as a fact about the member. The decisions are the author's:

  1. After member-path, skip that row's gitlink, tree and identity checks, or run them at members/<project>, the one path the row may have.
  2. recorded_gitlink takes its path literally, for every caller: the global --literal-pathspecs on both listings. Not :(literal) magic, which a caller's GIT_LITERAL_PATHSPECS=1 turns off (family.py: bump and remove take the git pathspec from the manifest row's path: with no check and no --literal-pathspecs, so a hand-edited path: members/* makes remove --member Atlas exit 0 with ONE commit that unmounts every member while dropping only Atlas's row, and path: . makes it git rm -r -f the holder's own files and die on a FileNotFoundError, though validate-family.py already has a rule, member-path, for what path: must equal #299's last fix was exactly that).
  3. A refusal prints the findings gathered before it, or a member's refusals become findings, so that one bad row does not hide what the others say.
  4. Both validate-family.py and repo_shape.py are copied into holders, so a fix reaches them through update-shape.py.

How to test. The repro's row as a validator test: member-path is printed and the run does not end in member-uninitialized (whichever of decisions 1 and 3 is taken). A unit test that recorded_gitlink(root, "members/*") is None in a holder with members (decision 2). Both are built in git's objects, with no file named * on disk, so they run on Windows too.

Verified at main c7c02fa and at the scratch merge of #318's head 0f2a0dd with #317's head c23e2b0 (validate-family.py sha256 7ec3eec0…); neither PR touches these functions, and the result survives the merge.

Related: #299 / PR #315 (_member_path, which refuses the same row in bump and remove), #20 (recorded_gitlink's index-first order), #361 (what the validator does not ask about members/), #368 (the same validator without git).

Activity

  1. brettheap commented on Oct 12, 2026

    @brettheap
    ContributorAuthor

    CLAIMED — lane openRepoShape-4, session e7cd5391-c790-49b0-a88f-a97d34b22582@Eagle, 2026-10-12T15:17:51Z, for #367

    Logged in lanes/log/openRepoShape-4.md at e2fed18fb8b695a7fb837eacdd1c295f49093d63.

    The three reads (lane-collision-protocol Rule 1):

    1. existing `CLAIMED —` comments on opensoft/openRepoShape#367:
       none
    2. `gh pr list --repo opensoft/openRepoShape --state all --search "#367"`, kept where the row or its body names `#367`:
       none
    3. `git ls-remote --heads git@github.com:opensoft/openRepoShape` branches naming `367` as a whole token:
       none
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions