Repository navigation
validate-family.py: a gitlink in members/ or a .gitmodules section whose path no row names is a finding in every mode, a hand-edited path: is member-path with nothing read through it and recorded_gitlink reads its path literally, and no git to run refuses git-not-runnable instead of a traceback that exits 1 (#361, #367, #368) - #406
Merged
Merged
Conversation
…n whose path no row names is a finding in every mode, a hand-edited `path:` is `member-path` with nothing read through it and `recorded_gitlink` reads its path literally, and no git to run refuses `git-not-runnable` instead of a traceback that exits 1 (#361, #367, #368) #361. Every member check started from a row: `_check_members` asked each `members:` row for its gitlink, digest and identity, and nothing walked back from what the holder records to a row. A member mounted by hand with `git submodule add`, a row deleted by hand, the section `family.py remove` leaves of two for one path (#360), and a section for `Members/Atlas` that `add` commits beside the member's own all passed the holder's own gate, and the doctor called the holder COMPLIANT. Two findings now, each with its own code, both made before the `--no-members` return, as #350's `member-duplicate` is, because a private family's CI falls back to that mode: - `member-gitlink-unnamed`: every gitlink in `members/` (the directory in any case, so `Members/Zed` counts and `tools/Zed` does not), listed from the whole index at stage 0, else HEAD, the order `recorded_gitlink` reads one in, with no pathspec and `-z`. A HEAD git cannot list records nothing, as `recorded_gitlink` reads it; an index git cannot list is a new refusal, `gitlinks-unreadable`, since this is the one check of `members/` that `--no-members` runs. - `gitmodules-path-unnamed`: every section whose path no row names, read from `.gitmodules` as git reads it, in one reading shared with #300's `gitmodules-path-twice` (`_gitmodules_sections`, which now keeps every key a section holds, for the remedy). "Unnamed" is decided EXACTLY against the path each row gives, its `path:` as written or `members/<project>`. A path a row names under `str.casefold()` is still a finding, and says it is the case variant of that row's path, since macOS and Windows hold the two as one directory. `family.py add`'s `member-path-registered` is unchanged (#362). The remedies print each command on its own indented line, with no backticks, no `;` and no trailing comment. A section is taken out by key, #300's removal, every key it holds. A gitlink a row names in another case is taken out with `git rm --cached`, which touches no file, because a plain `git rm` would take the member's own checkout with it where case is folded. Any other unnamed gitlink is either a row to put back from git, or a hand mount taken out with a plain `git rm` and mounted with `family.py add`. NOT `git rm --cached`, the remedy the ruling named: it leaves the checkout at the mount path, and `add` then refuses `member-path-taken` (probed). After a plain `git rm`, which refuses while the checkout holds changes and takes its section out too, `add` names the object store `git rm` leaves in `.git/modules` (`member-git-dir-cached`), and once that is gone it mounts and pins the member. A test runs that chain as printed. #367. `_check_member` found a hand-edited `path:` wrong, recorded `member-path`, and then checked the gitlink, tree and identity AT that path. At `members/*` the gitlink listing read a glob and answered with a neighbour's commit (a false `member-gitlink-mismatch`), the tree check refused `member-uninitialized`, whose remedy cannot help, and the refusal dropped both findings unprinted. Now the row's checks stop once its pin is read, the pin being a fact of the row itself, and `member-path` names the one path the row may have. `recorded_gitlink` passes git's global `--literal-pathspecs` to both of its listings (`GITLINK_LISTINGS`), never `:(literal)` magic, which a caller's `GIT_LITERAL_PATHSPECS=1` turns off. git refuses that option beside an exported `GIT_GLOB_PATHSPECS` or `GIT_ICASE_PATHSPECS` (exit 128), and `recorded_gitlink` reads a refused listing as "no gitlink here", so those two variables are kept out of those two calls' environment (`git_out` takes an `env` now), or an operator who exports one would see every gate refuse a gitlink that is there. There is ONE `repo_shape.py`, `scripts/repo_shape.py`: the templates carry none, and `scaffold-project.py` and `family.py init` copy it in (`REPO_SHAPE` in `COPIED_FROM_SHAPE` and `FAMILY_COPIED_FROM_SHAPE`), so the fix reaches `validate-family.py`, the assembly root's `validate-pins.py` and `bootstrap.py`, `bump-leg.py` and `shape-doctor.py`, its callers. The issue's decision 3, a refusal that prints the findings gathered before it, is not taken. #368. `git_out` raises `OSError` when there is no git to start, and only #317's `.gitmodules` reader caught it, so the default and `--pins` modes ended in a `FileNotFoundError` traceback, exit 1, a finding's code. The validator now runs every git-reading check inside `_git_runs`: the `.gitmodules` reader, #361's gitlink listing, each member's gitlink and its tree. It refuses ONE new code, `git-not-runnable`, exit 2, naming the check and that git could not be run. #317's no-git clause moves to that code too, so `gitmodules-unreadable` means only a file git ran and could not read. `recorded_gitlink`'s `except Refusal` lets the `OSError` pass, so the missing git never reads as "no gitlink". `_git_runs` wraps the git calls alone, not the identity check, which reads a file. `--no-members` now runs git for the gitlink listing even with no `.gitmodules`, so without git it refuses where it used to pass. The assembly root's `validate-pins.py` ends in the same traceback, and is not changed here. Also: `_check_members`'s docstring said "the two checks that read a file alone" where #317 had made three; it now names them, without a count. Tests, one new file per issue, every state built in git's index and objects with bytes on stdin but the one that runs the remedy chain on a real mount: - tests/test_validate_family_unnamed_mounts.py (#361): the issue's rows (a), (b) and (c), each a finding in the default mode, `--pins` and `--no-members`; a gitlink at a case variant, and its remedy run as printed; the index-first, HEAD-second order; `members/` alone in any case; a deleted row; each section's removal run as printed; an index git cannot list; the hand-mount remedy chain; and the controls, a holder whose rows name every gitlink and section, and one `add` grew. - tests/test_validate_family_member_path_glob.py (#367): six hand-edited spellings in both member modes print `member-path` and never end in `member-uninitialized`; the other rows are still checked; `recorded_gitlink(root, "members/*")` and three other globs are None in a holder with members and no file named `*`; and the four pathspec variables neither widen nor break it. - tests/test_validate_family_without_git.py (#368): in-process, git failing as a missing (or non-executable) program does, exit 2, the one code and no traceback, in the default mode, `--pins` and `--no-members`, with and without `.gitmodules`, with git failing only in `recorded_gitlink`, and only inside a member. - tests/test_family_add_refusals.py: #317's no-git test expects the new code, and its add-past-a-section control expects the one new finding for a section at another path. - tests/test_validate_family_duplicates.py: five of #331's tests rewrote `members:` without the row of a member the holder still mounts, the state #361 now finds, so each keeps a row for every mounted member (`keeping_mounts`) and still asks only about the names it spells. Mutants, each run against these files: the #361 checks moved after the `--no-members` return; `--literal-pathspecs` removed from both listings; the `OSError` clause in `_git_runs` removed; and also the `member-path` skip removed, and the environment scrub removed. Each fails at least one test, listed in the pull request. Lane: openreposhape-4 (openRepoShape-4) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days. You can request another review in 2 days and 3 hours by commenting |
Reviewer's GuideThe PR hardens family validation by finding unregistered gitlinks and .gitmodules paths in every mode, preventing malformed member paths from triggering checks through glob-like paths, and replacing missing-git tracebacks with explicit exit-2 refusals; shared literal gitlink lookup behavior is implemented in scripts/repo_shape.py and covered by targeted regression tests. Flow diagram for family validation findings and refusalsflowchart TD
Start["validate-family.py"] --> Checks["Run checks before --no-members return"]
Checks --> Gitmodules["Read .gitmodules sections"]
Checks --> Gitlinks["List index and HEAD gitlinks"]
Checks --> Rows["Build paths named by members rows"]
Gitmodules --> UnnamedSection{"Section path named by a row?"}
Gitlinks --> UnnamedLink{"Gitlink path named exactly?"}
Rows --> MemberPath{"Row path is expected?"}
UnnamedSection -->|No| SectionFinding["gitmodules-path-unnamed"]
UnnamedLink -->|No| LinkFinding["member-gitlink-unnamed"]
MemberPath -->|No| PathFinding["member-path; stop row checks"]
Checks --> GitRunnable{"Git can be started?"}
GitRunnable -->|No| GitRefusal["git-not-runnable; exit 2"]
GitRunnable -->|Yes| Continue["Continue validation"]
File-Level Changes
Assessment against linked issues
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
This was referenced Oct 12, 2026
…riant of a row's path is told to be taken out (`git rm --cached`, `--unset-all`) only where it is a second path to a member that still stands, so a row hand-edited to `Members/IRSS` is said to be the edit and a member recorded only at `members/irss` is said to be in the wrong case, where the command took the member's own record out and `--no-members` then said `family ok` (#361, #367) Review vf-423fb25 of PR #406: SHOULD-1 and NITs 1, 3 and 4. SHOULD-1. The case-variant branch of `_check_gitlinks_named` and `_check_gitmodules_named` took every path that folds to a row's path for the stray and printed `git rm --cached -- <path>`, or the `--unset-all` lines, for it. That is right in one state: the row's `path:` is its canonical `members/<project>` AND that exact path also holds a gitlink (a section), so the path found is a second one to a member that stands. In two others the command took the member's own record out, and the `--no-members` gate, the run a private family's CI falls back to, then said `family ok` over a holder with a gitlink fewer than its rows. Probed by the reviewer, and built here as tests. `_variant_case` tells the three apart, and `_row_paths` carries each row's canonical path for it: 1. STRAY, the row's path is canonical and that exact path holds a gitlink (a section) too: today's `git rm --cached -- <path>` and `--unset-all` lines, unchanged. 2. ROW_EDITED, the path found IS the row's canonical `members/<project>` and the row spells it in another case: the row is the hand edit, so the finding says to put `members/<project>` back in the row's `path:` and prints no command. Under `--no-members` no `member-path` finding says so, which is why this must. 3. WRONG_CASE, anything else, the row canonical and the only record of the member at a case variant: the member is recorded in the wrong case, the `git log -p -- family.yaml` restore sentence the unnamed gitlink always had, no command. A row that is itself hand-edited and a gitlink at a third spelling, and a row with no `project:`, are decided here too (nothing says either is a second path to a member that stands), as the brief's "otherwise". NIT-1. A hand mount is two findings, and the gitlink's `git rm -- <path>` takes the `.gitmodules` section out too, so the section's `--unset-all` lines exit 5 after it. The section finding now says the `git rm` above takes it out and prints no line. Where two sections give the path (`gitmodules-path-twice`) git removes only the one it reads, the last in the file, so each keeps its own lines (probed on git 2.43; the test asserts one section is left). NIT-3. `_member_gitlinks`'s stage-0 filter had no test. A gitlink held at stages 1-3 only is no gitlink the holder records, and one HEAD records and the index holds only unmerged is HEAD's. Built with `update-index --index-info` bytes. NIT-4. The add remedy quotes its project as the paths beside it are: `--member <org>/'My Proj'`. `<org>` is the operator's to fill in and stays outside the quotes. (`shlex.quote` is the file's own helper.) The PR body's caller list is corrected separately. NIT-2 is not done: it goes to the coordinator as a follow-up. Tests: tests/test_validate_family_unnamed_mounts.py 24 -> 39. Every state is built in git's index and objects (`stage_gitlink`, `git rm --cached`, `update-index --index-info`, a hashed `.gitmodules`) and never as two directories, so a case-insensitive disk holds them: the one real checkout stays the one directory it is. Mutants, each run against that file at this head: SHOULD-1 reverted to the old branch (`_variant_case` always STRAY) killed, 8 failed 31 passed, both state tests among them; the stage-0 filter dropped, 2 failed 37 passed; NIT-1's `--unset-all` lines printed again, 4 failed 35 passed. Also killed: ROW_EDITED never (3), NIT-1 for two sections (1), the member unquoted (2), STRAY without the exact path held (4). Lane: openreposhape-4 (openRepoShape-4) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This was referenced Oct 12, 2026
|
brettheap
pushed a commit
that referenced
this pull request
Oct 12, 2026
…C`) is `git-not-runnable` too, so the rule is any `OSError` whose `filename` is `git`, and the test picks each git call by its subcommand, so #406's `git --literal-pathspecs ls-files` no longer fails eight of its cases on a merge (#412) Fix round 1 of the review of d4b371f. MUST-1. `tests/test_validate_pins_without_git.py` chose the call to fail by `args[0] == "ls-files"` (and `"rev-parse"`, `"ls-tree"`). #406 makes `recorded_gitlink` run `git --literal-pathspecs ls-files -s -- <path>`, so on a merge with it the predicate never matched, the "gitlink" cases failed nothing, and eight of the file's tests went red: whichever of #406 and this PR landed second would have failed CI. Each predicate now reads the git SUBCOMMAND, wherever it sits among git's global options (`subcommand()`, which also skips the value of `-C` and `-c`), and a test of its own feeds each predicate the argv with options in front of it, so the file says so on its own head and not only on a merge with #406. NIT-1. A `git` on PATH that exists and cannot be executed (a binary for another architecture, a script with no shebang) fails `ENOEXEC`, a plain `OSError` whose `filename` is `git`, and ended in the same traceback, exit 1, that #412 is about. `_git_runs` now counts any `OSError` whose `filename` is `git`, and a `FileNotFoundError` or `PermissionError` whose `filename` is None (Windows' `CreateProcess` failures carry no filename, so None is required there and is kept for those two classes only). Every exclusion the tests pin stands: an error naming a directory or another file, and a nameless `OSError` that is neither class, leave the wrapper unchanged. Tests: `OSError(ENOEXEC, ..., "git")` from the fake in every mode and at each of the three calls, a nameless `PermissionError`, and an `ENOEXEC` that names no file or a directory (not git, so not counted). With a real `git` file with no shebang first on PATH, `subprocess` raises `OSError: [Errno 8] ... 'git'` and the run ends in the refusal (checked in scratch, Linux, Python 3.12). Lane: openreposhape-4 (openRepoShape-4) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
brettheap
pushed a commit
that referenced
this pull request
Oct 12, 2026
…o fix/family-bump-remove-rollback, so CI tests the pair that lands together (#365, #360) #406 (`validate-family.py` and unnamed mounts) lands before this PR, and both edit `tests/test_family_add_refusals.py`. Merging it here lets CI test the two together, so this PR can land right after #406 without another cycle. The merge was clean: git merged `tests/test_family_add_refusals.py` with no conflict, and no other file is touched by both branches. Lane: openreposhape-4 (openRepoShape-4) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This was referenced Oct 12, 2026
brettheap
added a commit
that referenced
this pull request
Oct 12, 2026
…fuses `git-not-runnable`, exit 2, naming the leg's gitlink, pinned commit or tree digest it could not read, instead of a traceback that exits 1 where 1 is a finding, and only a git that will not start counts, so a leg directory nobody may enter is not told to put git on PATH (#412) (#446) `git_out` raises `FileNotFoundError` when there is no git on PATH, `PermissionError` when the `git` found there will not run, and a plain `OSError` (`ENOEXEC`) when it cannot be executed (a binary for another architecture, a script with no shebang), and nothing in the validator caught any of them. A scaffolded root on such a machine ended in a traceback out of `recorded_gitlink`: exit 1, which this script's own exit codes give to "a FINDING: drift a reviewer can act on", though nothing was found, and a person or a CI step reading the code took it for drift. The doctor's `pins` row said so too, in its words: "reports a finding (exit 1): FileNotFoundError: [Errno 2] No such file or directory: 'git'". The three calls that run git now run inside `_git_runs`: the leg's gitlink (`recorded_gitlink`), the pinned commit's lookup in the submodule (`git_out`) and the tree digest (`tree_digest`). It refuses `git-not-runnable`, exit 2, "the question could not be asked", naming which of the three it was and saying to put git on PATH and run `scripts/validate-pins.py` again. The code is the one `validate-family.py` chose for the same fact (#368, #406), so a project and a holder say the same thing; the wording is this validator's own and uses no holder words. Three decisions the issue left to its author: - WHERE: in this validator, as #368's fix did in `validate-family.py`, and not in `git_out` or `repo_shape.py`, which every tool shares. The change adds nothing to the names this file imports from `repo_shape`, so a root whose `repo_shape.py` is older than this copy still imports it. - ONLY A GIT THAT WILL NOT START COUNTS. #406's wrapper caught every `OSError` and told an operator whose directory nobody could read to put git on PATH. This one is told apart two ways. The block is around the three git calls alone, never a whole check, so the `(submodule / ".git").exists()` before them, a pin file or a workflow nobody may read stay what they were. And inside a git call the rule is the NAME `subprocess` gives the error: any `OSError` whose `filename` is `git` counts (a missing git, one that will not run, one that cannot be executed), and so does a `FileNotFoundError` or `PermissionError` whose `filename` is None, those two classes only (Windows' `CreateProcess` failures carry no filename). `subprocess` raises the same two classes when the DIRECTORY it was to run git in cannot be entered, and on POSIX names that directory where a program that will not start names `git` (probed on Linux, Python 3.12: no git on PATH gives `filename 'git'`; a `git` that is not executable gives a PermissionError, `'git'`; a `git` with no shebang gives `OSError: [Errno 8]`, `'git'`; a missing, unreadable or non-directory `cwd` gives the directory). Any other error leaves the wrapper as it entered. - `recorded_gitlink`'s `except Refusal` does not turn the missing git into "no gitlink recorded": an `OSError` is no `Refusal`, passes straight through it and reaches `_git_runs`, so there is no false `pin-gitlink-absent`. The commit lookup's `_git_runs` is OUTSIDE its `try`, for the same reason one step on: inside, its own `except Refusal` would say `pin-commit- unresolvable`, which tells the operator to fetch a commit. `_digest_from_source` (a neutral product's `--pin-source`) keeps its own `except OSError`: it is reached only after every leg's gitlink was read, so no git means the refusal above came first. This file is a pinned copy and reaches existing roots through `update-shape.py` as `upstream-changed`; nothing more is needed here. tests/test_validate_pins_without_git.py (58 tests, run in process, the validator's own `git_out` and the one `recorded_gitlink` and `tree_digest` call in `repo_shape` patched to raise what a git that will not start raises, no PATH changed): exit 2, the code and no `Traceback` in each of three runs (plain, `--quiet`, `--pin-source`) and for each of five errors a git that will not start raises (missing, not executable, cannot be executed, and two that name no file); each call named by the check it is; the missing git is neither `pin-gitlink-absent` nor `pin-commit-unresolvable`, each with its control where git runs; the same root says `pins ok` through the same harness; seven errors that are no git that will not start, at each of the three calls, leave the wrapper unchanged; and one POSIX test with no fake, PATH an empty directory. The fake picks the call to fail by the git SUBCOMMAND, wherever it sits among git's global options, and never by `args[0]`: #406's `recorded_gitlink` runs `git --literal-pathspecs ls-files ...`, which an `args[0]` predicate stops matching, and a test of its own pins that. 27 fail on main's validator. Mutants, each killed (failing of 58): the `OSError` clause removed (27), any `OSError` counts (21), the classes narrow but the directory counts (12), `OSError` broad but a nameless error of any class counts (9), the refusal exits 1 (29), the commit lookup's wrapper inside its `try` (4), each of the three wrappers removed (20, 4, 3), `recorded_gitlink` swallowing `OSError` in `repo_shape.py` (27), the `ENOEXEC` clause dropped (6), the nameless clause dropped (6), narrowed to `FileNotFoundError` (3), `ENOEXEC` counted by errno (6), the test's gitlink predicate back to `args[0]` (4 here, 15 on a merge with #406). Closes #412 Lane: openreposhape-4 (openRepoShape-4) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
14 of 17 tasks
brettheap
pushed a commit
that referenced
this pull request
Oct 12, 2026
…t: `check_pin` takes the root its `update-pin-row` remediation reads, so #290's refusal survives #267's split instead of a NameError (#290) Main 580d7df carries #267 (decff87), which split update-shape.py's `read_pin(root)` into the read and `check_pin(pin, pin_path)`, so that shape-doctor.py can check the pins in a root's history in memory. The `update-pin-row` raises moved into `check_pin`, and on this branch they call `pin_rows_remediation(root)`, #290's `git -C <root> log` advice, where `root` is not a name. The textual merge is clean and the tree dies at run time: `NameError: name 'root' is not defined` for a `files:` of `5`, `true` or `1.5` in `check`, `apply` and shape-doctor.py, which then prints no report at all. This commit takes the planned #267 composition, the PR's two attachments fix1-compose-267-resolution-v2.patch and root-pin-faults-after-267.patch, applied unchanged; main moved their hunks by offset only (shape-doctor.py 526 lines, tests/test_shape_doctor.py 521, update-shape.py 1): - update-shape.py: `check_pin(pin, pin_path, root)`, `read_pin` passes its root, and the docstring says `root` is the repository whose history a refusal's `git log` reads. - shape-doctor.py `pin_reads_at` passes `ctx.root`. Its docstring says `check_pin` refuses `files: 5` as `update-pin-row`, and `TypeError` leaves its `except` tuple: no pin's bytes raise one now, so one means a caller out of step with `check_pin`, and the doctor fails loudly rather than answer that no commit holds a pin to put back. - tests/test_shape_doctor.py: #267's ROOT_PIN_FAULTS gains `files: 5`, `true` and `1.5`, each `update-pin-row`, so each is INVALID, exit 1, with `checkout HEAD` as its NEXT, which run as printed leaves the root COMPLIANT. Main's other changes to this PR's files merge as text away from these lines: #446 (245d775) in validate-pins.py, #406 (ec8e09a) in validate-family.py, #379 (87d2eb7) in update-shape.py, and #375 (cd1fc79) and #357 (5678f68) in shape-doctor.py. Neither module has changed on main since decff87. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap
pushed a commit
that referenced
this pull request
Oct 12, 2026
…landed: `_commit_the_split` keeps #445's reported steps and reads `follow_ups:` through `_plan_follow_ups` Main moved from dd8d0ea to 580d7df: decff87 (#267), dba5fb0 (#403), 245d775 (#446), 75052cb (#445), ec8e09a (#406) and 580d7df (#416). Only #445 touches adopt-project.py, and it conflicted in one hunk of `_commit_the_split`. #445 wraps that function's body in a `try` that names each step (cloning the source for the split, mounting the legs, committing the split, pushing the split branch), so a failure once both legs are pushed flushes stdout, prints `REFUSED <step>.`, what failed and one notice of what the run left. This branch reads `follow_ups:` there through `_plan_follow_ups`, the one read of it (#293). The resolution keeps #445's `try`, steps and reports whole, and carries this branch's read into its "committing the split" step: git had kept main's `[str(f) for f in plan.get("follow_ups", [])]` inside the `try`. The two contracts meet without overlap. `plan-bad-follow-ups` and `election-date-malformed` are findings of `_entry_findings`, which `execute` refuses through `_refuse_what_check_finds`. That runs before #445's `_refuse_an_occupied_work_dir`, `_refuse_a_split_branch_left_behind` and `_create_leg_remotes`, so these are refusals before any leg exists: exit 2, `REFUSED <code>:` and the remediation that says no leg repository was created and nothing was pushed, as #445's own refusals before the legs say, and no notice. #445's report covers the failures once a leg exists, which these values no longer reach. The guard in `_manifest_values` is unchanged; #445 does not touch it. #445's comment over `_template_values` in `cmd_execute` named "an `elected_on` that is not a date" as a refusal the table can raise. With #294's finding, `_refuse_what_check_finds` refuses such a date first, so the comment now says the date no longer reaches the table. #445's test of a table refusal stubs `_template_values` and is unchanged. tests/test_adopt_execute_refusals.py: the two end-to-end runs through `main` (#293, #294) now also assert that the refusal says no leg was created and that #445's notice ("NOTHING has been rolled back") does not follow it. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 12, 2026
brettheap
pushed a commit
that referenced
this pull request
Oct 12, 2026
…_spelled` now spells every symlink this branch names, and #283's tracking-branch question is not asked by adopt-project.py, so it meets none of this branch's refusals Brings in, by sha, main's ten commits since 75052cb: ec8e09a (#406), 580d7df (#416), 1184f67 (#404), d6df36b (#346), 37665ff (#452), 36538fd (#378, #329), 6c2a371 (#355, #283), 8782333 (#401, #340), ec4a7a0 (#456, #391) and 437a360 (#343, #290). No conflict. The tree is `git merge-tree --write-tree 437a360 95f6fc7`'s, ae7be0a. Three files changed on both sides merged by git alone: - adopt-project.py: 36538fd rewrites `_spelled`'s docstring and `_spelled_character` (a byte that is not UTF-8 is `\udcNN`, a character `\xNN`, by `repo_shape.double_quoted`), and two docstrings (`_refused_by_policy`, `_index_entries`); 8782333 adds `_leg_path_help` and the `help=` of `plan`'s `--spec-path` and `--code-path`. This branch's hunks are elsewhere. - scripts/shape_materialize.py: 6c2a371 adds `import tempfile` and the branch-name question (`_ask_git_branch`, `_branch_verdict`, `branch_problem`, `refuse_bad_branch`) after `run`. This branch's hunks are elsewhere, and it adds no import of `tempfile` to either file (adopt-project.py has had one since before 75052cb), so no name is bound twice. - tests/test_adopt_finding_spelling.py: 36538fd's `\udcfe` lines and this branch's `follows too many symlinks` line are more than 100 lines apart. How main's changes meet this branch's code: - 36538fd: every source path and symlink target this branch's findings and the materializer's refusals name is spelled by `_spelled` (`execute` hands it to the materializer as `spell`), so they now spell a byte that is not UTF-8 as `\udcNN`, as the plan's own `path:` does. One test of this branch pinned the old spelling, `nowhere\xff`; the next commit pins `nowhere\udcff`. Nothing in this branch has an escape table of its own. - 6c2a371: `refuse_bad_branch` is called by scaffold-project.py and scripts/family.py only. adopt-project.py's `--tracking-branch` is not asked of git (6c2a371 leaves it as its own change), so the question falls nowhere among `check`'s findings or `execute`'s refusals. - 8782333: `plan --help` text only; `plan` refuses nothing new, and the help names `shape/` as where `execute` writes its copy, which this branch does not move. Checked on the merged tree: no top-level name in adopt-project.py, scaffold-project.py, scripts/repo_shape.py, scripts/shape_materialize.py or scripts/family.py is bound twice that was not already bound twice on main (shape_materialize.py's `YamlError` and `parse_yaml`, as on main), no name is loaded and bound nowhere, and each side's own change reappears whole in the merge (no line of either side lost). Lane: openreposhape-1 (openRepoShape-1) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap
added a commit
that referenced
this pull request
Oct 13, 2026
… is `PRODUCT=PATH` only when its key names a chain link, and a link source that cannot be entered is a WARNING with the system's reason, not a traceback (#338) (#453) **What was wrong.** Two faults, one in the parser and one in the lookup. - **The parser.** `_link_sources_from_args` split every `--link-source` value at its first `=`. A link tree under a directory named `a=b`, passed bare, became a source keyed to a link `a` that no chain names. Nothing asked for it, so the link stayed `declared-unverified`, exit 0. The WARNING then offered `--link-source openXdox=<path>` to the person who had passed exactly that. A mistyped key failed the same silent way. - **The lookup.** `repo_shape.link_manifest_path` asked `Path.is_dir()` and then `is_file()`. Before Python 3.14 those swallow only ENOENT, ENOTDIR, EBADF and ELOOP. So each of these ended the naming gate in a traceback, exit 1: - a keyed source under a parent this user cannot enter; - `SHAPE_PIN_SOURCE_<LINK>` pointing there; - a checkout `../openXdox` at mode 000, which every `--project` run asks for; - a 300-character name. **What changed.** **How a value is read** (`LinkSources` in `scripts/validate-repository-naming.py`). This mirrors #312's `PinSources`, with a chain link where that file has a declared pin: - The text before the first `=` is a LINK only when it casefolds to a link of a chain this run reads. Those are the entries of `--referent-chain`, and of each leg's `naming.referent_chain:` under `--project`. Both are read before any leg is classified. - The keyed reading wins even over a directory spelled like the whole value. `./` reads that directory. - Any other value is a bare PATH, taken whole. **What a lookup says** (`scripts/repo_shape.py`): - `link_manifest_lookup` asks `stat()` inside a guard, as `_digest_from_source` does in #312. It returns the system's reason when a source cannot answer, or says which of four plain cases it is (`no such file or directory`, `it holds no project.yaml`, `its project.yaml is not a file`, `it is neither a directory nor a file`), and it never raises. It replaces `link_manifest_path`, which is gone: nothing calls it. `link_declarations` gives the reason a manifest could not be read, and `declared_neutral_pins`, which it left with no caller, is removed (fix round 1). - `resolve_link_source` and `link_pins_from_trees` take a keyword-only `tried`. It records each source asked, in order, with its reason. Nothing in `repo_shape` prints. - `link_pins_from_trees` also takes a keyword-only `with_reasons` (fix round 2). Asked, it returns a `LinkPins`, a dict equal to the plain mapping that also remembers why a tree it reached could not be read. Only the scaffold asks. **What the validator prints** (stderr, after each name's WARNINGs): - `WARNING link-source-unreadable: <path> (<how>) could not answer for <link>: <reason>`. This covers a `--link-source` asked for a link that holds a pin, a `SHAPE_PIN_SOURCE_<LINK>` likewise, and a checkout beside the project that exists and still cannot answer. A keyed value that also names a directory gets a second line saying how to ask for that directory with `./`. A bare value holding `=` says it was read as a PATH. - `NOTE --link-source <typed>: not used — <why>.` for a value no such link asked for. The why is one of five: a later value replaced it; no name reads that link's own declaration (a keyed value); no name records a chain at all; no name reads any chain link's own declaration (a bare value, where chains exist and none is read); or every link that is read had a more specific source. - Three kinds of source are never named: a variable or a checkout asked only for a chain's LAST link (no one ever reads that link's own declaration); a variable or a checkout for a chain no classification reads (fix round 1: a spec leg's chain, a name with no referent, a chain whose shape fails; a `--link-source` value for such a chain is the NOTE); and a checkout that is absent or holds no manifest. - `--quiet` drops both, with the WARNINGs. So `make bootstrap` and the doctor's `naming` row print nothing new. `--help`, the module docstring and `docs/cli.md` (regenerated) say all of this. **Fix round 1 (lane 5's review of 40762ca: 0 blockers, 2 should, 4 nits).** Second commit, 4b21263. Each review finding, in order: - **A link manifest whose `neutral_product_pins:` is not a list of names is a tree that cannot answer** (SHOULD-1). `link_declarations` iterated `data.get(key) or []` as it stood, against the `link_pins_from_trees` docstring this change wrote ("nothing a source does raises out of here"). `5` ended the naming gate, the scaffold and the project's `validate-manifest.py` in `TypeError: 'int' object is not iterable`. A scalar `openDox` was read as the characters `D`, `e`, `n`, `o`, `p`, `x`, so the chain broke, and under `--project` that is a FINDING. A mapping was read as its keys, so `openDox: yes` DECLARED `openDox`. Each is now `(None, why)`, as `not a mapping` is: declared-unverified, with the reason named ("`neutral_product_pins` is not a list", "holds an entry that is not text"). An absent or null value, an empty list, and a blank entry (`null`, `''`) in a list, declare nothing, as ever. The fix is in the shared function, so the scaffold and `validate-manifest.py` are reached too. **Three behaviours change on purpose here, and the tests of fix round 2 pin each:** - **A falsy value is no longer an empty list.** `neutral_product_pins: 0`, `false` or `''` was read as no pin at all (`or []`), so a chain through the tree was BROKEN, and under `--project` that is a FINDING. It is now a tree that cannot answer: the link is declared-unverified, a WARNING, exit 0. - **An entry that is not text is no longer read as text.** `[openInk, 0]` and `[openInk, false]` dropped the entry as a blank, and `[openInk, true]` read it as the text `True`, so a chain through the list VERIFIED. Each is now declared-unverified, with a WARNING saying the list "holds an entry that is not text". That is stricter than the link tree's own `validate-manifest.py`, which takes `str(pin)` of an entry (`_declared_pins`, `templates/assembly-root/scripts/validate-manifest.py:85`): a chain is better left unverified than verified through a name nobody wrote. - **The scaffold no longer refuses `chain-not-declared` over a pins value that is a scalar.** With `--pin-source` a tree whose `neutral_product_pins:` is the scalar `openInk`, the scaffold used to refuse (`REFUSED chain-not-declared: ... openXdox declares I, e, k, n, o, p, not openInk`, exit 2, from the characters). It is now the declared-unverified WARNING and the scaffold goes on, exit 0, which is its documented rule for a tree that cannot answer: "a warning, never a refusal, because the check is offline". The warning says the tree was reached and could not be read (fix round 2, below). - **The reach wording is corrected** (SHOULD-2): "Who else this reaches" below, and the docstring of the test that made the claim. What stops crashing is the naming and manifest steps. `make validate`, `validate.yml` and `make bootstrap` still end in a traceback on the sibling row before Python 3.14, at a file this change does not touch (below). On 3.14 the sibling is passed over without a word there instead, which is #430. - **The mutation gaps** (nit). Six of the review's ten mutations survived. Five now die, each by a new test: a `project.yaml` that is a directory; a device or a FIFO as the source; the order A, B, A; a value holding `=` with no chain at all; and every failure of `SHAPE_PIN_SOURCE_<LINK>` for a link that holds a pin (a path that is not there, a directory with no manifest, a file that is no manifest), not only a permission. The sixth, the `typed in self.used` test in `unread_lines`, could not be killed because it was dead: a value is read as one link's keyed source or is the one bare PATH, so every lookup asks it of the same path and it is used or unread, never both. That was asserted over 293 tests and the test is removed. - **A chain no classification reads** (nit). `classify` consults a link's own declaration only for a name that claims a referent, with a recorded chain whose shape holds. For any other chain (one recorded on a spec leg; one on a name with no `open<Product>` to reach; one that does not begin at a held pin or does not end at a referent of the name) a source was still asked, so a path that was not there was reported as `link-source-unreadable` and a good tree was counted used, each for a question nobody put. `NamingPolicy.reads_link_declarations` now says when `resolve_referent` reads a chain, and `LinkSources.link_pins` asks it first: such a chain asks nothing, and a value for it is the NOTE that no name reads that link's declaration. A test runs `resolve_referent` with a mapping that records whether it was read, over every way a chain is and is not, and holds the new method to it. - **Dead code and stale sentences** (nit). `declared_neutral_pins` is removed: `git grep` over the tree, every pushed branch and every sibling worktree finds no caller. `link_manifest_lookup`'s docstring no longer says "replaces" over a name that is gone. This description no longer says `tests/test_naming_policy.py` takes `SEARCH_DENIED` from #312: it defines a mark of its own, with the same condition and its own reason text. **Fix round 2 (lane 5's re-review of 4b21263: 0 blockers, 0 should, 3 nits).** Third commit, 2ac911c. Each finding, in order: - **The three behaviours the body did not state are stated and tested** (nit 1). Round 1's not-a-list fix changed three behaviours on purpose, each listed under SHOULD-1 above, and the re-review found two of them had no test. R1 (`if declared is None` read as `if not declared`, which makes `0`, `false` and `''` declare nothing again) and R2 (`if pin is None or pin == ""` read as `if not pin`, which drops a `0` or `false` entry as a blank) both passed all 210 cases of `tests/test_naming_policy.py` at 4b21263. Six rows join `NOT_A_LIST_OF_NAMES`, which runs at the CLI and in process: `0`, `false` and `''` as the value, and `0`, `false` and `true` among the names. R1 now fails 9 cases and R2 fails 6. - **The scaffold's warning is true for a tree it reached** (nit 1, the third behaviour). Over a `--pin-source` whose `neutral_product_pins:` is not a list, round 1 turned the `chain-not-declared` refusal into a declared-unverified WARNING that says the tree "is not reachable here (a sibling checkout, SHAPE_PIN_SOURCE_<LINK>, or --link-source <LINK>=<path> would let it be read)". `--pin-source` did reach the tree, and only its declaration could not be read, so that blamed a path that was right. Now `link_pins_from_trees` takes a keyword-only `with_reasons`; asked, it returns a `LinkPins`, a dict equal to the plain mapping that also keeps `unreadable`, `{link: why}` for each link whose source answered and whose manifest could not be read. `_chain_link_findings` words such a link "which was reached but could not be read as a declaration (<why>)", in a branch of its own, and only the scaffold asks. The new text is asserted for each of the eleven not-a-list rows, with `scaffold-project.py --dry-run` run as a subprocess. - **Every other path's text is the bytes it was.** The sentence for a link no source reached is the same code in the same lines, and four tests hold it: the generic warning for a plain mapping, for an empty `LinkPins` and for an absent tree; the scaffold's own old warning for a clone that holds no `project.yaml`; the naming validator's two lines (its warning, and the `link-source-unreadable` line that names the reason); and `validate-manifest.py`'s. The naming validator and `validate-manifest.py` do not ask for reasons, so they print what they printed. - **One more scaffold path changes with it, on purpose.** A tree the scaffold reached and could not read as a declaration for any other reason (a manifest that does not parse, or is not a mapping) used to say "not reachable here" too, and was no truer. It now says what it is, with the system's reason, and a test for each of the two (a manifest that is not a mapping, one that does not parse) holds it, at the scaffold's command line and in process: narrowing the wording to the not-a-list reasons fails both. - `_declaration` still returns its four values, so nothing else in `scaffold-project.py` moved. - **A qualified chain link in the spy test** (nit 2). `_chain_read_cases` had no case whose chain link is qualified, so `reads_link_declarations` could take the chain as typed and not through `_chain_links` (R3) and pass every test. With `--referent-chain testorg/openXdox,openDox` that turns a verified chain into a declared-unverified one. `("codexDox", {"openXdox"}, ("testorg/openXdox", "openDox"), True)` is a case now, and R3 fails it. - **The description** (nit 3): the ten corrections of the re-review are applied (the reason a lookup returns, the fifth NOTE reason, the third kind of silent source, the SHOULD-1 bullet, "before Python 3.14", the `validate-pins.py` bullet, the 3.14.7 rows, the merge-tree section, the filed numbers, and the project's own `validate-manifest.py`). The mutation proofs: all three survivors were reproduced first at 4b21263 (210 passed each) and each dies at 2ac911c. Six mutations of the new code each die too: the warning ignoring the reasons (12 cases fail), the lookup recording none (12), the scaffold not asking (11), `with_reasons` on by default (3), an absent tree called reached (3), and the reached sentence used for a tree nothing reached (17). **Where it departs from #312, and why.** - **An explicit source that cannot answer stays a WARNING, exit 0.** #312 made an unreadable `--pin-source` a finding. Three reasons for the difference: - Exit 1 here means "a name is wrong", and this module has always said an unread link tree is a warning. - `SHAPE_PIN_SOURCE_<PRODUCT>` also answers `validate-pins.py`'s digest question, from a checkout that need not hold a manifest. - The lookup falls through to the next source, which may still verify the link. - **The fall-through is kept**, as it always was. With `--pin-source`, a keyed value is final. - **The lines go to stderr, spelled `WARNING`/`NOTE` as this file spells them.** #312's notes are ` note` lines on stdout, where that file's report lives. **Who else this reaches.** The guard is in `repo_shape.py`, so the same `link_pins_from_trees` lookup stops crashing for these callers too: - **The project's own `validate-manifest.py`.** It reaches the lookup through `SHAPE_PIN_SOURCE_<LINK>` or a sibling checkout, and now gives its usual declared-unverified WARNING instead of the traceback. - **`scaffold-project.py --referent-chain`.** A `SHAPE_PIN_SOURCE_<LINK>` under a locked parent no longer kills the plan, and `--pin-source` verifies the link. Neither of these prints the reason a source could not be entered; the scaffold says it only for a tree it reached and could not read (fix round 2, above). A `--pin-source` the scaffold cannot enter is still a traceback: that is #339 (`pin_digest_from_source`), and this PR does not touch it. **What this does not reach, and what the doctor says.** - **The project's gates still end in a traceback on the sibling row.** The NAMING and MANIFEST steps of `make validate`, `validate.yml` and `make bootstrap` stop crashing for a checkout `../openXdox` at mode 000. Those targets also run `validate-pins.py`, whose `resolve_neutral_pin_source` asks `(sibling / ".git").exists()` with no guard (`templates/assembly-root/scripts/validate-pins.py:463` on main), so before 3.14 a sibling at mode 000 still ends them in a `PermissionError`. That file is #312's, and `fix/pin-source-env-collisions` (PR #409, since landed), `fix/adopt-pin-writes-pin-file` and open PRs #343 and #419 touched it when this was written, so this PR does not edit it. The same line is listed for Python 3.14 by #430 (lane 2), where the helper that would cure both halves is asked for; the pre-3.14 traceback reproduction is added there as a comment, not filed apart. A `SHAPE_PIN_SOURCE_<LINK>` under a locked parent is different: `validate-pins.py` makes that a `neutral-pin-source-unreadable` FINDING (#312), and that stays. - **The doctor's `naming` row** runs the STANDARD's copy of the validator, so it stops crashing at once. **Its `manifest` row** runs the project's OWN `validate-manifest.py` on the project's own `repo_shape.py`, so it changes only once the project has applied this change. Over an existing project the doctor says `INVALID (manifest, pins)` until `update-shape.py apply` lands the two `upstream-changed` rows below. - **The verdict.** For `SHAPE_PIN_SOURCE_<LINK>` under a locked parent, on a project that carries these copies, `INVALID (naming, manifest, pins)` becomes `INVALID (pins)`, and `pins` is #312's finding for that locked variable, which is right. For a sibling at mode 000 the `pins` that remains is the traceback above, not a finding. The two copied scripts this change edits (`scripts/repo_shape.py` and `scripts/validate-repository-naming.py`) are copied into every project and pinned there. A project takes the change through `update-shape.py` (two `upstream-changed` rows), and a family holder takes the `repo_shape.py` row. Neither file has a second copy in this repository. **Evidence.** - **Issue rows, main ab623dd** (`repro338.sh`, a `gh` stub first on PATH). Under 3.10.21 and 3.12.3, rows 4-7 (keyed under a mode-000 parent, `SHAPE_PIN_SOURCE_OPENXDOX` there, a sibling at mode 000, a 300-character name) are a traceback, exit 1. Rows 2-3 (`a=b`, the typo) are a silent declared-unverified, exit 0. Under 3.14.7 row 1 verifies, and rows 2-7 are the silent declared-unverified WARNING, exit 0. - **Issue rows, this PR.** Under 3.10.21, 3.11.16, 3.12.3, 3.12.14, 3.13.15, 3.14.7 and 3.15.0rc2 (first commit), all 7 rows exit 0 with no traceback, and the output is byte-identical across versions. `a=b` verifies. The other five rows each name their source with its reason. Re-run on the fix-round head, 4b21263, under 3.10.21, 3.12.3 and 3.14.7: byte-identical to the first commit's run. Re-run on the fix-round-2 head, 2ac911c, under the same three: all 7 rows exit 0 with no traceback, the output is byte-identical across the three versions, and it is identical to the re-review's runs of the previous head over main. - **Reach on main.** Main gives a `PermissionError` traceback, exit 1, for `scaffold --dry-run`, the project's `validate-manifest.py`, `validate-repository-naming.py --project` and its `--quiet` form, each with `SHAPE_PIN_SOURCE_OPENXDOX` under a locked parent. This PR gives exit 0 for all four. `validate-pins.py` on the sibling row is the exception said above. - **New tests.** 17 cases (11 functions) in the first commit; 16 fail on ab623dd. The fix round adds 31 cases in `tests/test_naming_policy.py`, and 17 of them fail on 40762ca: the five shapes of pins that are not a list (each at the CLI, in-process and, for the scalar, under `--project`), and the chains nobody reads. The other 14 guard what a mutation could break (below) or the side the fix must not move (an absent, null or blank pin list still answers). - **Targeted run on 4b21263**, files `test_naming_policy`, `test_manifest_validator`, `test_scaffold_referent_chain_links`, `test_cli_reference`, `test_repo_hygiene`, `test_neutral_pin_validator`, `test_scaffold_e2e`, `test_shape_doctor`, `test_git_output_decoding` and `test_pin_validator`: **765 passed, 3 skipped**. The 3 skips are the PowerShell-only rows. (40762ca: 734 passed, 3 skipped.) `render-cli-reference.py --check` is ok. - **Mutation probes, first commit.** Reverting the stat guard, the parse, keyed-over-directory, the NOTE lines, the chain's-end restriction, the sibling-absence filter, the manifest reason, the `--quiet` drop, the "read as a PATH" sentence, the `./` line, the project-chain read, the WARNING lines or the env reporting each fails at least one new test. - **Targeted runs on 2ac911c (fix round 2).** Under 3.12.3, `test_naming_policy` (238 cases: 210 and 28 new), `test_cli_reference`, `test_manifest_validator`, `test_scaffold_referent_chain_links`, `test_repo_hygiene` and `test_setup_project_py`: **468 passed**; and `test_scaffold_e2e`, `test_scaffold_pin_and_reuse`, `test_scaffold_leg_path_collisions`, `test_scaffold_pin_collisions`, `test_neutral_pin_validator`, `test_git_output_decoding` and `test_pin_validator`: **312 passed**. Under 3.10.21 and under 3.14.7, `test_naming_policy`, `test_manifest_validator` and `test_scaffold_referent_chain_links`: **321 passed** each. `render-cli-reference.py --check` is ok, and `ast.parse(feature_version=(3, 9))` passes on the three changed files. - **Mutation probes, fix round.** The review's ten, re-run on 4b21263: MW1, MW2, MW3, MW5 and MW11 now die (they survived), and MW4, MW7, MW12 and MW13 still die; MW6 is the dead test, removed. (Fix round 2 kills the three the re-review found, R1 to R3, and six more: see above.) Nine more for the new code (the list check, the entry check, the null and blank tolerances, `reads_link_declarations` always true, without the shape check, without the held pins, and the validator ignoring it): eight die. The ninth, `reads_link_declarations` ignoring whether the name has a referent, survives as an equivalent mutant: with no referent `_chain_shape_problem`'s last question always reports a problem, so the explicit early return is redundant, and is kept because `resolve_referent` has the same first line. The spy test holds the two together. - **Windows.** The mode-000 rows skip on Windows and as root, using a `SEARCH_DENIED` mark this file defines, with the same condition as #312's. The new FIFO and `/dev/null` rows skip where those do not exist. `a=b` and the 300-character name run everywhere. On Windows the long-name row asserts the line, not POSIX's reason. None of it had been run on Windows or macOS when this was written; the last paragraph gives the CI result. **Merge-tree.** Recomputed at 19:04Z with `git merge-tree --write-tree` of 2ac911c against `origin/main` as it was then, 53bee5b, and against every one of the other 37 pushed branches that touches a file this PR changes (a scan of each head against its merge-base with main): - **Clean:** origin/main 53bee5b (it carries #314 and #390); #394 `fix/adopt-record-submodule-placement` 563ea16 (`scripts/repo_shape.py` and `validate-manifest.py`) and #406 `fix/validate-family-unnamed-mounts` 369af2e (`scripts/repo_shape.py`); #397 abb27e6, #400 7062a47, #401 409f10b, #416 7cca975, #452 e2df9a8 and #395 1ef3310 (`docs/cli.md`); #355 6fdbe3d, #373 741cbf2 and #395 (`scaffold-project.py`). - **Conflict, four hunks each, all in `scripts/repo_shape.py`:** `fix/pin-source-env-collisions` (PR #409) 6494714, and `fix/adopt-pin-writes-pin-file` 421e4e5 (no PR). Both also touch `scaffold-project.py`, and that merges clean with this PR's one edit there. Fix round 2's new hunks merge with neither: #409's change to the "not reachable here" sentence (the `offered` text, when a variable is shared) and the reached branch sit in different lines of `_chain_link_findings`. The four hunks, each taking both sides: 1. the `resolve_link_source` signature: `default_source=None, env_shared: bool = False, *, tried: list | None = None`; 2. its docstring: both paragraphs; 3. the `link_pins_from_trees` docstring and the lines after it: both paragraphs, then `found`, this PR's `unreadable` and #409's `links` and `shared`; 4. the call: `resolve_link_source(name, root, keyed_sources, default_source, pin_product_key(name) in shared, tried=asked)` (the same spelling on both branches), followed by `if tried is not None: tried[name] = asked`. Resolved that way in a throwaway and committed there, the merge with 6494714 passed **496 tests** (`test_naming_policy`, `test_pin_source_env_collisions`, `test_neutral_pin_validator`, `test_scaffold_referent_chain_links`, `test_manifest_validator`, `test_cli_reference`), and the merge with 421e4e5 passed **559 tests** (the same files and `test_pin_source_unreadable`). `render-cli-reference.py --check` is ok on both, pyflakes is clean, and the grammar parses as 3.9's. Neither branch carried the resolution when this was measured. #409 has since landed, and the merge of main into this branch (the last paragraph) takes those four hunks both ways. The earlier counts (468 and 434 from the first fix round, 430 from the re-review) are those runs' own, at their heads. **Left out.** Each of these was found by lane 5's reviews of this change, and each is filed or folded: - **A bare `--link-source` answers for every link with no more specific source** (#441). In a chain of three or more links, one bare tree is read as two links' declarations, and a chain that holds is reported broken, a FINDING for a name with no direct pin on its referent. Pre-existing, the same on main; `validate-pins.py`'s `--pin-source` has the same documented rule and was not examined. - **An empty path after a key (`--link-source openXdox=`) is `Path("")`, the current directory** (#442). Run from a project root, the project's own `project.yaml` is read as the link's declaration. Pre-existing; `validate-pins.py`'s `--pin-source openXdox=` has the same shape (there a finding naming `.`). - **`validate-pins.py`'s `(sibling / ".git").exists()` is unguarded** (the sibling row above), folded into lane 2's #430, which already covers the Python 3.14 half: the pre-3.14 traceback and its reproduction are added there as a comment. #312's file. - **The project's OWN manifest** with `neutral_product_pins: 5` ends in a `TypeError` in two gates before either can say anything about it (#443): `_pins_from_project` in the naming gate (`scripts/validate-repository-naming.py:302` on main), and `_declared_pins` in the project's own `validate-manifest.py` (`templates/assembly-root/scripts/validate-manifest.py:85`), which ends it before its own `manifest-neutral-product-pins` finding (`_neutral_product_pins_type_finding`, `:524` on main) can print. Reproduced on 3ef059c. That is the project's own declaration, read by the project's own gates, and not a link source, so it is outside this change; "read by the project's own gates" covers two crashing gates. Also not done here: - **Python 3.9 (the floor) was not run** because it is not installed here. The grammar of the three changed files was checked with `ast.parse(feature_version=(3, 9))`. Windows and macOS were not run here; CI ran them (the last paragraph). Reviewed adversarially at 40762ca, the branch's first commit (CHANGES NEEDED: 0 blockers, 2 should-fixes, 4 nits). The head stopped every crash in the issue's table, and the should-fixes were a docstring claim the code did not keep for a `neutral_product_pins:` that is not a list, and reach claims that overstated what the project's gate does on the sibling row. Fix round 1 (4b21263) resolved every finding, and the re-review of 4b21263 read APPROVE WITH NITS (0 blockers, 0 should-fixes, 3 nits: three behaviour changes this description had not stated, two of them untested; a spy-test case for a qualified chain link; and description corrections). Fix round 2 (2ac911c) resolved those three, and its re-verification of 2ac911c read APPROVE WITH NITS (0 blockers, 0 should-fixes, 2 nits: no test of the scaffold's wider wording for a manifest that is not a mapping or does not parse, now pinned by two tests that each fail when the wording is narrowed, and a cross-reference in this description that pointed the wrong way, now corrected). Main (9736424, #409's squash) was merged into the branch as ced8fac. `scripts/repo_shape.py` conflicted in four hunks; each takes both sides, as the review of 2ac911c resolved it: `resolve_link_source(..., default_source=None, env_shared: bool = False, *, tried=None)`, both docstrings kept, `link_pins_from_trees` opening with `found`, `unreadable`, `links` and `shared`, and the call passing `pin_product_key(name) in shared` with `tried=asked`. Nothing else conflicted. After the merge, `tests/test_pin_source_env_collisions.py` and the targeted set passed (667 and 312), and `render-cli-reference.py --check` printed `[ok]`. CI is green on Linux, macOS and Windows, with SonarCloud passing, at 938a131 (main 48585e6 merged in). Its merge with main 994be39 passed the targeted set (the naming, manifest, scaffold, pin and setup test files): 983 passed. Closes #338 Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Lane: openreposhape-4 (openRepoShape-4)
Claim: #361 (comment)
Claim: #367 (comment)
Claim: #368 (comment)
Three defects in the holder's own gate,
templates/family-root/scripts/validate-family.py, fixed in one change because all three are in_check_membersand the member checks under it. The third also reaches the sharedscripts/repo_shape.py.#361: a gitlink or a
.gitmodulessection that no row namesWhat was wrong. Every member check started from a row.
_check_membersasked eachmembers:row for its gitlink, digest and identity, and nothing walked back from what the holder records to a row. These all passed the holder's own gate, and the doctor called the holder COMPLIANT:git submodule add;family.py removeleaves of two sections for one path (family.pyremove, when.gitmodulesgives the member's path to two sections, takes out only the one git finds by that path (the later in the file) and exits 0, so its commit still registersmembers/<P>with no gitlink and no row,validate-family.pypasses the result (after #317 it reported the doubled file and stops), and the nextaddof that member is refusedmember-path-registeredor silently reuses the leftover section and the.git/configregistrationdeinitnever touched #360);Members/Atlas, whichaddcommits beside the member's own.What it does now. Two findings, each with its own code. Both are made before the
--no-membersreturn, as #350'smember-duplicateis, because a private family's CI falls back to that mode.member-gitlink-unnamedmembers/, the directory in any case (Members/Zedcounts,tools/Zeddoes not). It reads the whole index at stage 0, else HEAD, the orderrecorded_gitlinkreads one in, with no pathspec and-z.gitmodules-path-unnamed.gitmodulessection whose path no row names. It shares one reading of the file, as git reads it, with #300'sgitmodules-path-twice."Unnamed" is decided exactly, against the path each row gives: its
path:as written, ormembers/<project>. A path a row names understr.casefold()is still a finding, because macOS and Windows hold the two as one directory. Which of three things it is decides what the finding says to run (fix round 1, below).family.py add'smember-path-registeredis unchanged (#362).Remedies. Each command is printed on its own indented line, with no backticks, no
;and no trailing comment.git config -f .gitmodules --unset-allper key it holds. None when its gitlink'sgit rmbelow takes it out too (fix round 1).path:is its canonicalmembers/<project>and that exact path holds one toogit rm --cached -- <path>, which touches no file (a plaingit rmwould delete the member's own checkout on a disk that folds case), or the--unset-alllinesmembers/<project>back in itspath:(fix round 1)git log -p -- family.yamlrestore sentence (fix round 1)git log -p -- family.yaml), or, for a hand mount:git rm -- <path>, thenpython3 scripts/family.py add --family-root <holder> --member <org>/<Project>from a checkout of openRepoShape, the project shell-quoted as the paths areThis departs from the ruling's
git rm --cached. Probed on the issue's repro row (a): aftergit rm --cached members/Zed, the checkout stays at the mount path, andaddrefusesREFUSED member-path-taken: members/Zed already exists in the family root but no `members:` row names it. A plaingit rm members/Zedworks instead:.gitmodulessection out with it;addthen refusesmember-git-dir-cached, naming the object storegit rmleaves in.git/modules(the remedy says it will);addmounts and pins the member, and the holder saysfamily ok.test_a_hand_mount_is_taken_out_and_added_back_as_printedruns that chain as printed.Smaller decisions.
gitlinks-unreadable, for an index git ran and could not list. The gitlink listing is the one check ofmembers/that--no-membersruns, so a failed listing must not pass silently. A HEAD git cannot list records nothing, asrecorded_gitlinkreads it (a holder before its first commit).familyrow reports what this validator finds.#367: a hand-edited
path:was read as a globWhat was wrong.
_check_memberfound the row'spath:wrong (member-path), then checked the gitlink, tree and identity at that path anyway. Atmembers/*:member-gitlink-mismatch;member-uninitialized, whose remedy cannot help;What it does now.
member-pathnow names the one path the row may have.recorded_gitlinkpasses git's global--literal-pathspecsto both of its listings. It never uses:(literal)magic, which a caller'sGIT_LITERAL_PATHSPECS=1turns off.--literal-pathspecsbeside an exportedGIT_GLOB_PATHSPECSorGIT_ICASE_PATHSPECS: "global 'literal' pathspec setting is incompatible with all other global pathspec settings", exit 128 (probed on git 2.43).recorded_gitlinkreads a refused listing as "no gitlink here", so an operator exporting either variable would have seen every gate refuse a gitlink that is there. Those two variables are now kept out of those two git calls' environment;git_outgained an optionalenvfor this.One copy of
repo_shape.py:scripts/repo_shape.py. The templates carry none.scripts/shape_materialize.pycopies it into projects (COPIED_FROM_SHAPE) and holders (FAMILY_COPIED_FROM_SHAPE), run byscaffold-project.pyandfamily.py init, so the fix reaches every caller ofrecorded_gitlink:validate-family.py;validate-pins.py(187) andbootstrap.py(114);bump-leg.py(740);shape-doctor.py, at three call sites (line numbers as on main 45e01f4, and the same at cb7063f):family_beside_project(1462),check_legs, thelegsrow (2167), andmembers_verdict, themembersrow (3802).family.py(107) importsrecorded_gitlinkbut never calls it. Every caller passes a declared path, and none relies on globbing. Existing projects and holders get the fix throughupdate-shape.py.Decision 3 is not taken. A refusal still prints none of the findings gathered before it.
#368: no git on PATH ended in a traceback, exit 1
What was wrong.
git_outraisesOSErrorwhen there is no git to start, and only #317's.gitmodulesreader caught it. The default and--pinsmodes ended in aFileNotFoundErrortraceback, exit 1, which is a finding's code.What it does now.
_git_runs: the.gitmodulesreader, validate-family.py asks everymembers:row for its gitlink and never the reverse, so a gitlink undermembers/that no row names (a member mounted by hand withgit submodule add, or a row deleted by hand) and a.gitmodulessection whose path no row names (whatfamily.py removeleaves of two sections for one path, or a section for a case variant such asMembers/Atlas, whichaddcommits beside the member's own even after #317) all pass the holder's own gate, and the doctor calls the holder COMPLIANT #361's gitlink listing, and each member's gitlink and tree.git-not-runnable, exit 2, naming the check and that git could not be run.addrefusesmember-path-registeredwhen a.gitmodulessection under another name already gives the mount path, andvalidate-family.pyfindsgitmodules-path-twice, soaddno longer commits two sections for one path that a recursive clone resolves to the later one, and a holder that already has such a ghost section fails its own gate (#300) #317's no-git clause moves to that code too, sogitmodules-unreadablenow means only a file git ran and could not read. family.py:addrefusesmember-path-registeredwhen a.gitmodulessection under another name already gives the mount path, andvalidate-family.pyfindsgitmodules-path-twice, soaddno longer commits two sections for one path that a recursive clone resolves to the later one, and a holder that already has such a ghost section fails its own gate (#300) #317's no-git test is updated to match.recorded_gitlink'sexcept Refusallets theOSErrorthrough, so a missing git never reads as "no gitlink"._git_runswraps the git calls alone, not the identity check, which reads a file.--no-membersnow runs git (the validate-family.py asks everymembers:row for its gitlink and never the reverse, so a gitlink undermembers/that no row names (a member mounted by hand withgit submodule add, or a row deleted by hand) and a.gitmodulessection whose path no row names (whatfamily.py removeleaves of two sections for one path, or a section for a case variant such asMembers/Atlas, whichaddcommits beside the member's own even after #317) all pass the holder's own gate, and the doctor calls the holder COMPLIANT #361 gitlink listing) even with no.gitmodules. Without git it now refuses, where it used to pass.Not fixed here: the assembly root's
validate-pins.pyends in the same traceback (the issue's last line). It is #412, fixed in PR #446 with the samegit-not-runnablecode.Also
_check_members's docstring said "the two checks that read a file alone" where #317 had made three. It now names the checks that need no member checked out, without a count (the nit from the review of #331, PR #350).No
--helptext changed, sodocs/cli.mdis unchanged.Fix round 1 (review vf-423fb25)
The review found each issue's repro fixed, the five departures right and the decisive run green (1455 passed, 4 skipped on the scratch merge). It held the head on one remedy that misdirects and left four nits. Head after this round:
369af2e, one commit on423fb25.--no-membersgit rmabove takes it out and prints no--unset-alllinegit-not-runnablewrapper catches more than git_member_gitlinksis unpinnedmembersrow, and the three doctor call sites with lines) and the copier (shape_materialize.py); the--membervalue in the add remedy is shell-quoted, withshlex.quote, the file's own helperSHOULD-1. A path that folds to a row's path was always taken for the stray, and the finding printed
git rm --cached -- <path>(or the--unset-alllines). That is right in one state. In two others it took the member's own record out of the index (or.gitmodules), after which--no-members, a private family's CI fallback, saidfamily ok. Now_variant_casetells the three apart (_row_pathscarries each row's canonicalmembers/<project>for it):path:is its canonical path AND that exact path also holds a gitlink (a section). Today's command, unchanged.path: Members/IRSSover a mount atmembers/IRSS): the finding says the row is the hand edit and to putmembers/IRSSback in the row'spath:. It prints no command. Under--no-membersnomember-pathfinding says so, which is why this one must.git log -p -- family.yamlrestore sentence the unnamed gitlink always had. No command.Decisions the ruling left open:
Members/IRSSrow, gitlink atMEMBERS/IRSS), and a row with noproject:. Nothing says either is a second path to a member that stands, so neither is told to remove it. The third case says "could un-register it" for that reason.git log -p -- family.yamlparenthesis, as the brief names it. The section's non-variant text, which has none, is unchanged.members/<project>for a row with aproject:, and unknown for one without.NIT-1.
git rm -- <path>takes the.gitmodulessection out too, so the section's--unset-alllines exit 5 after it. Where two sections give one path (gitmodules-path-twice),git rmremoves only the one git reads for it, the last in the file (probed on git 2.43, and the test asserts one section is left), so each of those keeps its own lines.NIT-3. A gitlink at stages 1-3 only (made with
update-index --index-infobytes) is no gitlink the holder records, and one HEAD records and the index holds only unmerged is named at HEAD's commit, not at a stage's.How the state tests hold on a case-insensitive disk (macOS, Windows). None puts a second directory on disk. A gitlink is only an index record (
update-index --index-infobytes,git rm --cached) and a commit of the index, which reads no directory, so the checkout atmembers/IRSSstays the one directory it is while the index saysmembers/irss. The.gitmodulesedit is hashed into git's objects and the same bytes are written to disk. The one hand edit,family.yaml, is a single file written as bytes. Members checked, state (ii) is the refusalmember-gitlink-absenton a git that matches a pathspec exactly, which prints no finding (decision 3 of #367 is not taken), so only--no-membersshows its finding. A git that matched the path in any case would find the variant and print the finding, so the test accepts either, and asserts that no command is printed in both. The macOS and Windows legs were not run here: CI is the gate.Tests
One new file per issue. Every state is built in git's index and objects, with bytes on stdin, except one test that runs the remedy chain on a real
git submodule add.tests/test_validate_family_unnamed_mounts.py(validate-family.py asks everymembers:row for its gitlink and never the reverse, so a gitlink undermembers/that no row names (a member mounted by hand withgit submodule add, or a row deleted by hand) and a.gitmodulessection whose path no row names (whatfamily.py removeleaves of two sections for one path, or a section for a case variant such asMembers/Atlas, whichaddcommits beside the member's own even after #317) all pass the holder's own gate, and the doctor calls the holder COMPLIANT #361):--pinsand--no-members;members/alone, in any case;initandaddmade, and oneaddgrew, pass in every mode;Members/IRSS(state (i)), a member recorded only atmembers/irss(state (ii)), and a section whose only path is a case variant, each in every mode, with the finding's code, its sentence and nogit rmor--unset-all; a third spelling beside an edited row; the hand mount's section naming thegit rmabove it, and two sections for that path keeping their lines; an unmerged gitlink, alone and beside HEAD's; and the add remedy quoting a project with a space or a quote.tests/test_validate_family_member_path_glob.py(validate-family.py keeps checking a member at the row'spath:after it has found that path wrong (member-path), so a hand-editedpath: "members/*"reachesrecorded_gitlinkas a glob pathspec, which answers with a neighbour's gitlink, and the run ends inmember-uninitialized, whose remedy (git submodule update --init,make bootstrap) cannot help, while themember-pathfinding that names the hand edit is gathered and never printed #367):members/*,members/I*,members/[I]*,:(glob)members/IRSS,members/IRSS/..,members/IRSS/) in both member modes printmember-path, nevermember-uninitializedor a false mismatch, exit 1;recorded_gitlink(root, "members/*")and three other globs areNonein a holder with members and no file named*;tests/test_validate_family_without_git.py(validate-family.py's default and--pinsmodes end in aFileNotFoundErrortraceback, exit 1 (a finding's code), when git is not on PATH: on main for every holder with a member (recorded_gitlinkthroughrepo_shape.git_out), and after #317 still for a holder whose.gitmodulesis gone, though #317's own check answersgitmodules-unreadable, exit 2, whenever the file is there #368), in-process, with the git runner patched to raise as a missing (or non-executable) program does. In each case: exit 2, the one code, no traceback.--pinsand--no-members;.gitmodules;recorded_gitlink;tests/test_family_add_refusals.py:addrefusesmember-path-registeredwhen a.gitmodulessection under another name already gives the mount path, andvalidate-family.pyfindsgitmodules-path-twice, soaddno longer commits two sections for one path that a recursive clone resolves to the later one, and a holder that already has such a ghost section fails its own gate (#300) #317's no-git test expectsgit-not-runnable;gitmodules-path-unnamedfinding for a section at another path.Local targeted run. Linux, git 2.43.0,
/usr/bin/python3 -m pytest -q, 16 files: the three new ones,test_validate_family_duplicates.py,test_family.py,test_family_add_refusals.py,test_family_member_path.py,test_family_add_excludes.py,test_family_dry_run.py,test_shape_doctor.py(the doctor callsrecorded_gitlink),test_bump_leg.py,test_bump_leg_rollback.py,test_pin_validator.py,test_scaffold_e2e.py(the assembly root'svalidate-pins.pyandbootstrap.py),test_repo_hygiene.pyandtest_template_values_single_pass.py.5eba7d5, never pushed): 924 passed, 7 failed, 4 skipped. The 7 were validate-family.pymember-duplicateruns only with the members checked and comparesproject:exactly, so--no-memberspasses afamily.yamlthat names one row twice, and rowsIRRSandirrswith distinct ids pass, thoughmembers/IRRSandmembers/irrsare one directory on macOS and Windows #331's tests intest_validate_family_duplicates.pythat dropped a mounted member's row, now fixed withkeeping_mounts.423fb25, every other file byte-identical, checked by sha256): that file 20 of 20 passed.Fix round 1 runs (Linux, git 2.43.0,
/usr/bin/python3 -m pytest -q, committed bytes, sha256 recorded first):test_repo_hygiene.pyandtest_template_values_single_pass.py: 472 passed;test_agents_md_codes.pyincluded, on a scratch merge of this head onto maincb7063f(that test landed on main after this branch's base): 305 passed.CI's three OSes are the gate.
Mutants, each run in a scratch worktree at
5eba7d5, whose code and mutant test files are byte-identical to the pushed head's, against the files named:--no-membersreturntest_validate_family_unnamed_mounts.py--literal-pathspecsremoved from both listingstest_validate_family_member_path_glob.pyOSErrorclause in_git_runsremovedtest_validate_family_without_git.py+ #317's no-git testmember-pathskip removed (the row goes on at its path)test_validate_family_member_path_glob.pytest_validate_family_member_path_glob.py -k pathspec_settingFix round 1 mutants, each run against
tests/test_validate_family_unnamed_mounts.pyat this head:_variant_casealways the stray)--no-members(members checked it is a refusal, which prints no finding), the section state and the third spelling._member_gitlinks--unset-alllines printed again--membervalue unquotedFollow-ups, not done here
validate-pins.pyends in a traceback without git (validate-family.py's default and--pinsmodes end in aFileNotFoundErrortraceback, exit 1 (a finding's code), when git is not on PATH: on main for every holder with a member (recorded_gitlinkthroughrepo_shape.git_out), and after #317 still for a holder whose.gitmodulesis gone, though #317's own check answersgitmodules-unreadable, exit 2, whenever the file is there #368's last row): filed as The assembly root'sscripts/validate-pins.pyends in aFileNotFoundErrortraceback, exit 1, a finding's code, when git is not on PATH, becauserecorded_gitlinkreachesrepo_shape.git_outand nothing catchesOSError, where the file's own exit codes give 1 to drift and 2 to a question that could not be asked: the sibling of #368, whose fix is ruled intovalidate-family.pyonly #412 and fixed in PR validate-pins.py: no git to run, or a git that cannot be executed, refusesgit-not-runnable, exit 2, naming the leg's gitlink, pinned commit or tree digest it could not read, instead of a traceback that exits 1 where 1 is a finding, and only a git that will not start counts, so a leg directory nobody may enter is not told to put git on PATH (#412) #446, with the same code.git-not-runnablewrapper also catches anOSErrorthat is not git's, so an unreadablemembers/<P>with git on PATH is told git could not be run.recorded_gitlinkstill answers with the first gitlink under a path that is a directory (members) or climbs with..(members/Nova/..). git normalizes..and matches a leading directory even with--literal-pathspecs(probed). The validator no longer reaches it (themember-pathskip), but other callers could. The fix would be an exact-path check on the listed record, or a listing with no pathspec, asfamily.py's_index_entriesdoes.Closes #361
Closes #367
Closes #368
🤖 Generated with Claude Code
Summary by Sourcery
Strengthen family validation by auditing recorded members in reverse, treating member paths literally, and converting unavailable Git into explicit refusals.
New Features:
members/and.gitmodulessections whose paths are not named by family rows, including case-variant paths, in all validation modes.Bug Fixes:
git-not-runnablerefusal instead of an uncaught traceback.Enhancements:
Tests: