validate-pins.py and repo_shape.py: `--root` and gh's stderr are spelled, what stands in a pin's, the manifest's or a shape copy's place is named, and a leg's `.git` gitfile this user cannot open is refused by name (#919) by brettheap · Pull Request #952 · opensoft/openRepoShape

brettheap · 2026-10-18T17:05:17Z

Lane: openreposhape-5 (openRepoShape-5)
Claim: #919 (comment)

The four parts of #919, in one shape bump.

Base: origin/main 50178305, the v1.1.1 release commit. The head is 8a7671b3.
Probes: a scaffold into local bare remotes, run through its own validate-pins.py, with a stub gh first on PATH, on git 2.43.0. Below, <ESC> is the byte 0x1B, and a "before" line shows a raw ESC the way cat -v does (^[).

U176: `--root` is spelled

Change. find_repo_root (scripts/repo_shape.py) names start with shown_path: as it is, unless a character in it is not printable, and then repr'd. Only the message changed. The (candidate / ".git").exists() guard is #430's.

Probe. validate-pins.py --root '<dir>/x<ESC>[31mRED', outside any repository. Exit 2 before and after.

Before: REFUSED not-a-git-repo: no .git found at or above <dir>/x^[[31mRED
After: REFUSED not-a-git-repo: no .git found at or above '<dir>/x\x1b[31mRED'
A printable --root is named byte for byte as before.

The same holds for every tool that calls find_repo_root.

U177: gh's stderr is spelled, on one line

Change. tree_digest_from_gh's gh-unreadable detail spells what gh printed with spelled, as one line. It is the choice git_out makes for what git printed (#610), in the same file. A line break gh prints becomes \n, so gh's two-line not-logged-in help stays on one line. The pin's repository in the same detail is named with shown_path, as git_out names its folder. gh's stderr is read in text mode, so a CR is a line break by the time it is spelled.

Probe. A declared openXdox pin with no local source, and a stub gh that exits 1 with stderr one<ESC>[31mred<CR>two<LF>three<LF>.

Before: the skip line held a raw ESC and two raw line breaks, so it printed as three lines: ... could not read testorg/openXdox @ cccc…: one^[[31mred, then two, then three). A --pin-source, ...
After, one line:

  skip openXdox: pin commit cccccccccccc digest NOT rechecked — no local checkout (gh-unreadable: `gh api` could not read testorg/openXdox @ cccccccccccccccccccccccccccccccccccccccc: one\x1b[31mred\ntwo\nthree). A --pin-source, SHAPE_PIN_SOURCE_OPENXDOX, or a checkout at ../openXdox beside this assembly root would verify it offline.

shape-doctor.py already joins a skip that runs over several lines (reported). A root with older pinned copies still prints one, so the doctor keeps reading both.

U178: what stands in a file's place is named

Change. In validate-pins.py, _not_a_file asks lstat what stands where _is_file found no regular file. It answers is a directory, not a file, is a FIFO, …, is a symlink to nothing, …, is a symlink loop, … or is a symlink to a directory, …, and does the same for a socket or a device. It opens nothing. _no_file keeps each refusal's code and the words before "does not exist", says what stands there, and gives a remedy that fits:

Remediation: this validator reads each leg's pin, the shape pin, each neutral-product pin, project.yaml and every copy the shape pin lists, and one it cannot read is a question it cannot answer, never a pass. Something that is not a file stands in its place: move it aside, or take it away if nobody wants it, and restore the file from the project's history. Then run the same command again.

The first sentence is _unreadable_file's own, now shared through _remedy.

Site	Before, for a directory, a FIFO, a symlink to nothing or a symlink loop	After (a directory)
leg pin	`REFUSED pin-missing: … but contracts/spec-pin.yaml does not exist`, lockstep remedy	`… but contracts/spec-pin.yaml is a directory, not a file`, the remedy above
shape pin	`REFUSED shape-pin-missing: contracts/shape-pin.yaml does not exist`, lockstep remedy	`… is a directory, not a file`, the remedy above
neutral-product pin	`REFUSED neutral-pin-missing: … does not exist`, the `scaffold-project.py --pin` remedy	`… is a directory, not a file`, the remedy above
`project.yaml`	`REFUSED manifest-missing: project.yaml does not exist, so there is no declaration …`, lockstep remedy	`REFUSED manifest-missing: project.yaml is a directory, not a file`, the remedy above
shape copy	`FINDING shape-copy-missing: contracts/shape-pin.yaml: Makefile is pinned but absent`	`… Makefile is pinned but is a directory, not a file`, with the line `A shape file's place is taken by something that is not a file. Move it aside, or take it away if nobody wants it, and restore the copy from the project's history.`

Five sites, not three. The issue's State names the leg pin, the shape pin and the shape copy. Its "What happens" names the manifest too, and a neutral-product pin is a pin. All five _is_file sites had the same defect, so all five use the same helper.
No new code. Each site keeps its code and its line's form. A file that is not there keeps today's words, and its remedy, byte for byte.
Skew. A site that had no remedy of its own still builds its refusal with none, not with an empty one. Beside a repo_shape.py older than repo_shape.py's Refusal gives every refusal built without a remediation the assembly root's own (run git submodule update --init --recursive, then python3 scripts/validate-pins.py ... contracts/<leg>-pin.yaml), so in a family holder family.py add over an unparsable .gitmodules (git-failed), bump on a family.yaml of the wrong kind, and the holder's own validate-family.py on a short pin.commit each tell the operator to run a file no holder carries and to move a leg pin a holder does not have #370, Refusal's own default is the lockstep remedy, and an empty one would print a blank line.
The doctor's reading. shape-doctor.py reads <pin>: <path> is pinned but absent only to find a merged-away shape/ copy. update-shape.py's merged_into allows that only where nothing at all is at the path, so a non-file was never one. Neither file changes.

U179: a gitfile this user cannot open is refused by name

Change. _check_pin_digest asks _is_there(.git) and _gitfile_opens(.git). _gitfile_opens opens a .git that is a regular file, a gitfile. It reads the open's error by _is_a's own errno set, _ABSENT_ERRNOS and _ABSENT_WINERRORS, with no new set:

nothing there is an uninitialized leg;
any other error is raised, and becomes pin-submodule-unreadable through _unreadable_file, as a refused stat already did.

A .git directory or a FIFO is not opened.

Probe. spec/.git, the gitfile a recursive clone gives the leg, at mode 000.

Before: REFUSED pin-commit-unresolvable: spec: the pinned commit 4be308ba… is not in that submodule's object store (… exited 128: fatal: error opening '<root>/spec/.git': Permission denied), under the lockstep remedy.
After: REFUSED pin-submodule-unreadable: spec/.git: cannot be read (Permission denied), with the checkout remedy that validate-pins.py and repo_shape.py: a pin, the manifest or a shape copy the gate cannot reach or read is refused by name, relative to the root, once and with its cause's advice, and every name and path it prints is spelled (#712) #827 gives a leg this user cannot search.

Pinned copies

Pinned copies: every project's repo_shape.py and validate-pins.py take an upstream-changed row each, and re-pin them.

validate-pins.py imports nothing new from repo_shape (#370, #610). Its names are unchanged, and shown_path and spelled are used only inside repo_shape.py.

Tests

tests/test_validate_pins_spelled_and_named.py, new, 51 tests:

U176 (6): the issue's --root, exact stderr; a printable --root, exact; and find_repo_root in-process for a newline, a bidi control, a C1 control and an ESC.
U177 (9): the stub gh (POSIX), with the skip line exact and on one line; gh-unreadable's detail in-process for one printable line, gh's two-line help, controls with a bidi control, and UTF-8 left unchanged; and the repository spelled.
U178 (30):
- each of the four refusal sites with a directory, a FIFO, a symlink to nothing, a symlink loop and a symlink to a directory, the whole of stderr exact;
- the shape copy with the same five;
- the five sites with nothing there, byte for byte.
- FIFO and symlink rows skip on Windows.
U179 (6):
- the gitfile at mode 000, run as a project runs it, with the mode put back in a finally. It skips as root and on Windows.
- In-process, on every platform: a refused open (permission, and an I/O error), an open that finds nothing, and a .git directory or FIFO that is never opened.

tests/test_doctor_validator_rows.py: one docstring now says the skip embedded gh's stderr verbatim before #919. No assertion changed.

Fail on base. The new file run against 50178305's files: 43 failed, 8 passed. The 8 pass there by design:

the five byte-for-byte rows for a file that is not there;
the printable --root;
gh's one printable line;
the UTF-8 row.

Runs

Through run9s.sh, at head 8a7671b3. The coordinator held this lane to one pytest process at a host load of 85-270, so each run is one shard (SHARDS=1), and he scoped the git 2.47.1 run.

git 2.43.0, 26 files: 2043 passed, 3 skipped, rc 0, in 63 min. They are:
- the new file;
- the 19 test files git grep -l -E 'no \.git found at or above|gh-unreadable|pin-missing|shape-pin-missing|shape-copy-missing|pinned but absent|pin-submodule-unreadable|not-a-git-repo' -- tests finds (tests/readme-cap.d/460.txt is no test). They include the four doctor validator files, test_shape_doctor.py, the four update-shape files and test_pin_validator.py;
- test_pinned_copy_text.py, test_bom_reads.py and test_validator_child_encoding.py;
- the trio: test_repo_hygiene.py, test_agents_md_codes.py and test_cli_reference.py.
git 2.47.1, 6 files: 308 passed, rc 0: the new file, test_pin_validator.py, test_pinned_copy_text.py and the trio.
Left to CI's tests job on git 2.47.1: test_adopt_ignored_shape_files.py, test_adopt_pins.py, test_bump_leg_unopenable.py, test_doctor_unreadable_pin.py, test_doctor_validator_rows.py, test_doctor_validator_site.py, test_doctor_validator_verdicts.py, test_neutral_pin_validator.py, test_non_list_project_fields.py, test_shape_doctor.py, test_shape_pin_row_path.py, test_update_shape.py, test_update_shape_landing.py, test_update_shape_merged_away.py, test_update_shape_rewrite_symlink.py, test_validate_pins_lockstep_lines.py, test_validate_pins_named_and_spelled.py, test_validate_pins_unreadable_workflows_dir.py, test_bom_reads.py, test_validator_child_encoding.py. Each of these passed on git 2.43.0 above.

Mutants

Each mutant is applied alone to a scratch tree at the head and run against its part's tests in the new file. All 15 are killed.

Part	Mutant	Fails
U176	`start` named raw (no `shown_path`)	5: the issue's `--root` and the four in-process names
U177	gh's text raw (no `spelled`)	3: the stub `gh` line, gh's two-line help, the controls
U177	each line spelled but joined with a line feed (no one line)	3: the same three
U177	the repository raw	4: the four unprintable repositories
U178	leg pin back to "does not exist"	5: that site's five rows
U178	shape pin back to "does not exist"	5: that site's five rows
U178	neutral-product pin back to "does not exist"	5: that site's five rows
U178	`project.yaml` back to "does not exist"	5: that site's five rows
U178	shape copy back to "pinned but absent"	5: that site's five rows
U178	a symlink loop read as a symlink to nothing	5: the loop row at each site
U179	no read check (`_is_there` alone)	4: mode 000, permission, I/O error, gone-when-opened
U179	`_ABSENT_ERRNOS` widened with EACCES, EPERM and EIO	3: mode 000, permission, I/O error
U179	the open's errno set widened the same way	3: the same three
U179	the open's errno set dropped (every error raised)	1: gone-when-opened
U179	a `.git` that is no regular file opened too	2: the directory and the FIFO rows

No survivor.

Merge checks

origin/main: git merge-tree --write-tree is clean against 50178305, 64464d31 and 01a2b016 (tree 1d7066ce, measured by rv919). Main has moved on since; the landing gate runs on the predicted squash over it.
The open fix/* and l3-* branches (10): only l3-677-pins-hand-step-wip (1ef46b3f) touches either file, validate-pins.py. It conflicts with this head, but it conflicts with main alone in the same four files. It is a WIP of validate-pins.py and shape-doctor.py: one leg's gitlink off its pin beside another leg's digest or workflow reference names the step that comes first, that gitlink put back by hand, and then the other leg's bump-leg.py line, so neither finding says "First clear that finding" over a finding no line can clear first (#677) #702's fix round from 2026-10-16, based 119 commits behind main.
Lane 3's l3-917-yaml-reader (repo_shape.py's YAML reader: a key written twice keeps the last without a word, and a block-scalar line indented less than its block loses its first characters #917, d3a2ee61) also edits repo_shape.py. It merges clean with this head (tree 9c14dbf2, measured by rv919).

Closes #919

🤖 Generated with Claude Code

Summary by Sourcery

Harden repository-shape validation so diagnostics safely name untrusted paths and output, distinguish missing files from non-files, and reject inaccessible submodule gitfiles clearly.

Bug Fixes:

Make validation refuse unreadable .git gitfiles by name instead of misclassifying them as unresolved commits.
Preserve single-line, terminal-safe diagnostics when --root paths or gh stderr contain control characters or line breaks.
Identify directories, FIFOs, symlinks, sockets, and devices occupying expected file paths, with appropriate remediation guidance.

Enhancements:

Apply consistent path spelling and file-state diagnostics across pin, manifest, shape-pin, neutral-pin, and shape-copy validation.

Tests:

Add comprehensive coverage for escaped paths and gh output, non-file replacements, and unreadable gitfiles across supported platforms.

Chores:

Re-pin the affected project copies of repo_shape.py and validate-pins.py.

…led, what stands in a pin's, the manifest's or a shape copy's place is named with a remedy that fits it, and a leg's `.git` gitfile this user cannot open is refused by name, so no ESC or line break reaches the terminal raw, a directory is no longer "does not exist", and a gitfile at mode 000 no longer ends in `pin-commit-unresolvable` (#919) U176. `find_repo_root` names `start` with `shown_path`: as it is, unless a character in it is not printable, and then `repr`'d. `--root '<dir>/x<ESC>[31mRED'` outside any repository put the ESC on stderr from every tool that asks it. Only the message changed; the `.git` guard is #430's. U177. `tree_digest_from_gh`'s `gh-unreadable` detail spells what gh printed with `spelled`, as ONE line, and names the pin's repository with `shown_path`, as `git_out` spells its own two parts (#610). `validate-pins.py` prints the detail in its `skip` line, where an ESC was the control it is and each line break began a line. U178. A directory, a FIFO, a socket, a device, a symlink to nothing, a symlink loop or a symlink to a directory in the place of a leg pin, the shape pin, a neutral-product pin or `project.yaml` was refused under "does not exist" and the lockstep remedy; in a shape copy's place it was "pinned but absent". `_not_a_file` asks `lstat` what stands there, and `_no_file` keeps each code and the words before "does not exist", names it (`is a directory, not a file`), and gives `_remedy`'s first sentence with `NOT_A_FILE`: move it aside, or take it away, and restore the file. The copy's finding keeps `<pin>: <path> is pinned but` and gets a line under it. Nothing at the name keeps today's words byte for byte, and a site with no remedy of its own still builds its refusal with none, so a root beside a `repo_shape.py` older than #370 prints what it printed. U179. A leg's `.git` that is a regular file is opened (`_gitfile_opens`), its errors read by `_is_a`'s own errno set: nothing there is an uninitialized leg, and a refused open is `pin-submodule-unreadable` through `_unreadable_file`, as a refused `stat` already was. A directory or a FIFO is not opened. Both files are pinned copies; `validate-pins.py` imports nothing new from `repo_shape`. `tests/test_validate_pins_spelled_and_named.py` holds the four parts, and `tests/test_doctor_validator_rows.py`'s two-line gh test says the skip embedded gh's stderr verbatim before #919. Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

brettheap · 2026-10-18T17:05:19Z

(lane openRepoShape-5, 2026-10-18T17:05Z) @codex review

chatgpt-codex-connector · 2026-10-18T17:05:31Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-18T17:09:50.785268Z	`8a7671b`	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

sourcery-ai · 2026-10-18T17:05:31Z

Reviewer's Guide

The PR hardens user-facing validator diagnostics by escaping untrusted paths and gh output, identifies what non-file objects occupy expected pin and manifest paths, and explicitly rejects unreadable regular-file submodule gitfiles; extensive regression tests verify exact output, preserved legacy behavior for absent files, and platform-specific filesystem cases.

Sequence diagram for validator diagnostics and file checks

sequenceDiagram
    participant Validator as validate-pins.py
    participant Shape as repo_shape.py
    participant FS as Filesystem
    participant GH as gh
    participant Git as Git

    Validator->>Shape: find_repo_root(start)
    Shape->>Shape: shown_path(start)
    Shape-->>Validator: Refusal not-a-git-repo with escaped path

    Validator->>GH: gh api
    GH-->>Validator: stderr
    Validator->>Shape: spelled(stderr)
    Shape-->>Validator: gh-unreadable with one-line escaped detail

    Validator->>FS: _is_file(path)
    FS-->>Validator: false
    Validator->>FS: _not_a_file(path)
    FS-->>Validator: object kind or absent
    Validator-->>Validator: _no_file(...)
    Validator-->>Validator: Refusal or shape-copy-missing finding

    Validator->>FS: _is_there(submodule/.git)
    FS-->>Validator: present
    Validator->>FS: _gitfile_opens(submodule/.git)
    FS-->>Validator: permission or I/O error
    Validator-->>Validator: pin-submodule-unreadable

File-Level Changes

Change	Details	Files
Sanitize repository-root error paths and GitHub CLI diagnostics before embedding them in user-facing output.	Format non-printable --root paths with shown_path while preserving printable paths. Escape gh stderr with spelled and collapse line breaks to keep skip output on one line. Use shown_path for the repository named in gh-unreadable errors.	`scripts/repo_shape.py`
Improve validator diagnostics for non-file filesystem entries and unreadable submodule gitfiles while preserving existing refusal codes and absent-file behavior.	Centralize remediation construction and distinguish directories, FIFOs, sockets, devices, broken symlinks, and symlink loops via lstat/stat. Apply contextual non-file diagnostics to leg pins, shape pins, neutral pins, project.yaml, and shape copies. Probe regular-file .git gitfiles before digest checks, converting non-absent open failures into pin-submodule-unreadable refusals without opening .git directories or FIFOs.	`templates/assembly-root/scripts/validate-pins.py`
Add comprehensive regression coverage for escaping, filesystem-shape diagnostics, gitfile access handling, and compatibility with existing output.	Cover exact CLI and in-process output for roots, gh stderr, repository names, and control characters. Exercise all five file-validation sites across directories, FIFOs, symlinks, and absent paths. Verify permission, I/O, disappearance, and non-regular .git cases across supported platforms and execution modes.	`tests/test_validate_pins_spelled_and_named.py`
Update doctor test documentation to reflect that newer validators escape and single-line GitHub stderr while older pinned copies may retain legacy behavior.	Revise the affected test docstring without changing assertions.	`tests/test_doctor_validator_rows.py`

Assessment against linked issues

Issue	Objective	Addressed
#919	Spell the `--root` path in `not-a-git-repo` refusals and spell `gh` stderr as a single line in `gh-unreadable` details.	✅
#919	When a pin, manifest, or shape-copy path exists but is not a regular file, identify what stands there and give a remedy suited to that case, while preserving the existing wording when the path is absent.	✅
#919	Refuse an unreadable regular-file `.git` gitfile by name instead of treating the leg as initialized and later reporting its pinned commit as unresolvable.	✅

Tips and commands

Interacting with Sourcery

Trigger a new review: Comment @sourcery-ai review on the pull request.
Continue discussions: Reply directly to Sourcery's review comments.
Generate a GitHub issue from a review comment: Ask Sourcery to create an
issue from a review comment by replying to it. You can also reply to a
review comment with @sourcery-ai issue to create an issue from it.
Generate a pull request title: Write @sourcery-ai anywhere in the pull
request title to generate a title at any time. You can also comment
@sourcery-ai title on the pull request to (re-)generate the title at any time.
Generate a pull request summary: Write @sourcery-ai summary anywhere in
the pull request body to generate a PR summary at any time exactly where you
want it. You can also comment @sourcery-ai summary on the pull request to
(re-)generate the summary at any time.
Generate reviewer's guide: Comment @sourcery-ai guide on the pull
request to (re-)generate the reviewer's guide at any time.
Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
pull request to resolve all Sourcery comments. Useful if you've already
addressed all the comments and don't want to see them anymore.
Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
request to dismiss all existing Sourcery reviews. Especially useful if you
want to start fresh with a new review - don't forget to comment
@sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

Enable or disable review features such as the Sourcery-generated pull request
summary, the reviewer's guide, and others.
Change the review language.
Add, remove or edit custom review instructions.
Adjust other review settings.

Getting Help

Contact our support team for questions or feedback.
Visit our documentation for detailed guides and information.
Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

sonarqubecloud · 2026-10-18T17:06:41Z

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
2.1% Duplication on New Code

See analysis details on SonarQube Cloud

chatgpt-codex-connector · 2026-10-18T17:09:49Z

Codex Review: Didn't find any major issues. Swish!

Reviewed commit: 8a7671b3f3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

brettheap · 2026-10-18T18:03:58Z

Independent review of #952 (#919) at 8a7671b3, for lane openRepoShape-5: READY

I found no MUST and no SHOULD. Every claim in the body that I measured again holds. Where the body says output is unchanged, a probe shows it is unchanged byte for byte. The four NITs below can be fixed now or in a follow-up.

Findings

NIT 1: one rule in the change has no test. The rule is at templates/assembly-root/scripts/validate-pins.py:503: a refusal with no remedy of its own is built with none, not with an empty one.

Scenario: a later edit passes "" there. Beside a repo_shape.py older than repo_shape.py's Refusal gives every refusal built without a remediation the assembly root's own (run git submodule update --init --recursive, then python3 scripts/validate-pins.py ... contracts/<leg>-pin.yaml), so in a family holder family.py add over an unparsable .gitmodules (git-failed), bump on a family.yaml of the wrong kind, and the holder's own validate-family.py on a short pin.commit each tell the operator to run a file no holder carries and to move a leg pin a holder does not have #370, every "does not exist" refusal would then print a blank line before the lockstep remedy.
Evidence: my mutant r4 (*(remediation or ("",))) survives: 54 passed. My probe shows the rule is right today: with the pre-repo_shape.py's Refusal gives every refusal built without a remediation the assembly root's own (run git submodule update --init --recursive, then python3 scripts/validate-pins.py ... contracts/<leg>-pin.yaml), so in a family holder family.py add over an unparsable .gitmodules (git-failed), bump on a family.yaml of the wrong kind, and the holder's own validate-family.py on a short pin.commit each tell the operator to run a file no holder carries and to move a leg pin a holder does not have #370 repo_shape.py (b0826e02), head and base print the same bytes at all five sites when the file is absent.
Fix: about six lines in the test file. Monkeypatch validator.Refusal with a class whose default remedy is not empty, and assert that the absent branch keeps that default.

NIT 2: the body's merge section is out of date. It says "No open branch touches repo_shape.py". That is now false:

l3-917-yaml-reader (d3a2ee61, committed 16:47:52Z) edits scripts/repo_shape.py. It merges clean with this head (tree 9c14dbf2).
Main is now 01a2b016, and the head merges clean with it (tree 1d7066ce).

NIT 3: the same function still names the repository raw in two places. In scripts/repo_shape.py, gh-unreadable now spells the repository, but gh-response-unreadable (:1301-1302) and gh-tree-truncated (:1305-1306) still name {repository} raw. All three details end up in the same skip line. In practice nobody reaches them, because they need gh to exit 0 for a repository name that holds a control character.

NIT 4: three of my mutants survive on unchecked branches.

r2: the socket named "a FIFO" survives. A socket row could be tested on Linux. My probe made one by binding to a relative path, which avoids the AF_UNIX length limit.
r11: a character device left unnamed survives. Making a device needs root.
r9: dropping the Windows winerror check in _gitfile_opens survives. Linux cannot reach that branch.

Checked and true

Base-fail. The new file run against 50178305's code gives 43 failed, 8 passed. The 8 that pass are the ones the body names.

Runs at the head, through run9.sh:

git	Files	Result
2.43	the new file, `test_doctor_validator_rows.py`, `test_validate_pins_named_and_spelled.py`, `test_pin_validator.py`, `test_neutral_pin_validator.py`, `test_pinned_copy_text.py`, the trio	748 passed, rc 0
2.43	`test_shape_doctor.py`, `test_doctor_validator_site.py`, `test_doctor_validator_verdicts.py`, `test_doctor_unreadable_pin.py`, `test_update_shape_merged_away.py`	569 passed, 3 skipped, rc 0 (the skips are the Windows-only PowerShell rows)
2.47.1	the new file	51 passed

The writer's counts. Collecting the writer's two run lists gives 26 files with 2046 tests (2043 + 3, as the body says) and 6 files with 308 tests.

Probe. I ran 164 cases on one scaffold, with four pairs of files: head, base, head beside the pre-#370 repo_shape.py, and base beside it.

U176: a --root holding an ESC, a newline or a non-UTF-8 byte is now repr'd. A printable --root is byte-identical to base.
U177: a gh stderr of ESC + CR + LF, of CRLF, or the two-line not-logged-in help now prints as one line. An empty stderr is byte-identical to base.
U178: each of these stands at each of the five sites: a directory, a FIFO, a dangling link, a self loop, a two-link loop, a link to a directory, a link to a FIFO, and a socket. Each is named truly ("a directory", "a FIFO", "a symlink to nothing", "a symlink loop", ...), with the new remedy. A file that is simply absent is byte-identical to base at all 5 sites, in both pairings.
U179: a gitfile at mode 000 is refused as pin-submodule-unreadable: spec/.git: cannot be read (Permission denied), beside either library.
Unchanged, byte for byte: a clean root, the gitfile with its mode put back, and contracts replaced by a file.

Five sites is right. They are exactly the five _is_file calls (validate-pins.py:647, :1256, :1309, :1591, :1920). The issue's own "What happens" names "a pin, the shape pin or the manifest", and a neutral-product pin is a pin.

The errno rule (slice Z). _gitfile_opens reads an error at :414-415 with the same test _is_a uses at :275-276, on the same constants (:253-254). There is no new set.

What the doctor and update-shape.py read. No code and no line form changed. An absent copy still ends in is pinned but absent (shape-doctor.py:3090). A copy that is not a file now falls out of the doctor's merged-away reader. That is the same answer as before, because merged_into's lexists check refused it anyway (update-shape.py:694-721). I also ran the head doctor on a root whose scripts/validate-pins.py is a directory: it says DRIFTED (1 copy-missing) and quotes the new line.

Mutants.

I re-ran three of the writer's, and each fails the same number of tests the writer reported:
- a symlink loop read as a symlink to nothing: 5 failed;
- _ABSENT_ERRNOS widened with EACCES, EPERM and EIO: 3 failed;
- gh's lines joined with a line feed: 3 failed.
I wrote 11 of my own. 7 are killed (r1: 2, r3: 20, r5: 15, r6: 4, r7: 7, r8: 5, r10: 1). The 4 survivors are in NIT 1 and NIT 4.

Merges.

Against 01a2b016 the merge is clean, as is l3-917 (both above).
l3-677-pins-hand-step-wip (1ef46b3f) conflicts in the same 12 regions and the same 4 files whether merged with this head or with main alone, so validate-pins.py and repo_shape.py: --root and gh's stderr are spelled, what stands in a pin's, the manifest's or a shape copy's place is named, and a leg's .git gitfile this user cannot open is refused by name (#919) #952 adds no conflict.
No other branch of the 30 on origin touches the four files.
validate-manifest.py, repo_shape.py and scaffold-project.py: a chain source that is there and cannot answer is still offered, one SHAPE_PIN_SOURCE_* can answer for two chains, and a Kelvin-sign link is recorded as typed #909 has no branch or pull ref on origin: couldn't find remote ref pull/909/head.

Sonar-style checks.

Cognitive complexity, counted by hand: all under 15.
- _gitfile_opens 5, _not_a_file 5, _named_kind 2-3, _no_file 1, _unreadable_file 1.
- _check_shape_copy 8 (6 at base).
- _check_pin_digest 11 (10 at base). The writer counts 12 and 11.
- find_repo_root 3.
Line widths: no added line widens a file. The widest added lines are 76, 78 and 76 characters, against file widths of 93, 84 and 80. The new test file is 79 characters wide.
No assert joins conditions with and, and I found no duplicated block. The new test file holds a raw U+2014 on purpose, to match the validator's own em dash; body-guard returns rc 0 on it.

Pinned copies and docs.

No import line changed. shown_path and spelled are used only inside repo_shape.py, and test_pinned_copy_text.py passes.
README.md:752 and docs/handbook.html:1254 stay true.
The changed docstring in tests/test_doctor_validator_rows.py is true. The multi-line join in the doctor is still tested in-process by test_skipped_keeps_the_whole_block_and_stops_at_the_next_item.
The commit is by Brett Heap <service@opensoft.dev> and carries both trailers.

Could not verify

Windows and macOS behaviour, since CI runs Linux only. That includes the symlink-loop winerror 1921 check, junctions, and the in-process rows there.
How a device is named (needs root), and git 2.55.0, which CI runs.
21 of the writer's 26 files on 2.43 (I ran 5), and 20 of them on 2.47.1. I collected all 26, and the counts match.
The writer's checklist candidates 3 to 5, which I did not measure again.

Checklist candidates

A leg's git directory this user cannot read. Measured on head and base, the same output:
- With spec/.git a directory at mode 000, git walks up to the root's own repository. The run ends REFUSED pin-commit-unresolvable: ... exited 1: (the reason is empty), under the lockstep remedy.
- With .git/modules/spec at mode 000 (the writer's candidate 2), it ends in fatal: not a git repository, under the same remedy.
gh stderr that is not UTF-8 ends in a UnicodeDecodeError traceback, exit 1, on both base and head. This is the writer's candidate 1, confirmed.
contracts replaced by a regular file: every pin still reads "does not exist", with the lockstep remedy. This is unchanged from base, and is the same family as U178.
shape-doctor.py docstrings (lane 2's slice C owns that file):
- :1765-1767 says a skip embeds gh's stderr verbatim, which is now true only of roots older than validate-pins.py and repo_shape.py: --root and gh's stderr are printed raw, a non-file in a pin's place is called absent, and a leg's .git gitfile at mode 000 passes #919;
- the sentence at :9300-9301 says "pinned but absent" where the copy may now be called a directory.
A note for slice Z: _not_a_file (validate-pins.py:465-466) tests errno.ELOOP and winerror 1921 directly, to name a loop, not to decide that nothing is there. A parity scan should allow that.

sourcery-ai · 2026-10-18T18:19:32Z

Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 3 days and 1 hour by commenting @sourcery-ai review. Upgrade to get a review now.

…a skip and no traceback, JSON that is not a tree listing is `gh-response-unreadable`, every gh detail spells the repository, a leg's git directory this user cannot search is refused by name, and what stands where `contracts` should be is named (#968) (#985) What #919 left, from rv919's review of #952 at `8a7671b3`: the five items of #968, and JSON from gh that is not a tree listing, which ended in a traceback too. - **Base:** main `2f5cabf5` (#952). The head is `c8b6d0fa`. - **Probes:** a scaffold into local bare remotes, run through its own `validate-pins.py` with a stub `gh` first on PATH, on git 2.43.0. Below, `<ESC>` is the byte 0x1B and `<FF>` the byte 0xFF. **Change.** `tree_digest_from_gh` (`scripts/repo_shape.py`) reads gh's two streams as bytes and decodes each with a new helper, `_child_text`. A CR is still a line break, as text mode made it. - **stderr** is decoded with `errors="replace"`, as `git_out` decodes git's. It is words for a person, and `spelled` already makes it one printable line (#919); a replaced byte (U+FFFD) says where a byte was. - **stdout stays strict** (#217): this JSON becomes a digest. A byte that is not UTF-8 there is `gh-response-unreadable`. - **JSON that is not a tree listing** (a list, `null`, `tree` that is a number, a row that is not a mapping or has no `mode`, a path holding a lone surrogate) is `gh-response-unreadable`, `<repository> @ <commit>: the answer is not a tree listing (<error>)`. **Probe.** A declared `openXdox` pin with no local source. - gh exits 1 printing `bad<FF>byte`. Before: a `UnicodeDecodeError` traceback from `subprocess`'s `_translate_newlines`, exit 1. After, exit 0 and `pins ok`, with the skip `... (gh-unreadable: `gh api` could not read testorg/openXdox @ cccc...: bad<U+FFFD>byte). ...` - gh exits 0 printing `{"tree": [], "x": "<FF>"}`. Before: the same traceback, exit 1. After: the skip names `gh-response-unreadable: testorg/openXdox @ cccc...: 'utf-8' codec can't decode byte 0xff in position 19: invalid start byte`. - gh exits 0 printing `[]`. Before: `AttributeError: 'list' object has no attribute 'get'`, a traceback, exit 1. After: the skip names `gh-response-unreadable: ... the answer is not a tree listing (AttributeError: 'list' object has no attribute 'get')`. `null`, `{"tree": 5}`, `{"tree": ["x"]}`, a row with no `mode` and a lone surrogate read the same way, each with its own error. - A valid listing, CR LF between its lines, digests to the same value before and after, and beside the pre-#370 `repo_shape.py`. **Change.** `_check_pin_digest` (`validate-pins.py`) calls `_refuse_an_unreadable_git_dir` after the uninitialized check. It finds the leg's git directory with `_leg_git_dir`: the `.git` itself where it is a directory, or the directory named by a gitfile's `gitdir: ` line, with line ends stripped as git strips them. Then it stats the `HEAD` in it, as `_is_there` asks. A refused `stat` is `pin-submodule-unreadable` through `_unreadable_file`, with `_is_a`'s errno set and the leg checkout's remedy that #827 gives a checkout this user cannot search. A git directory with no `HEAD`, or a gitfile that names none, is left to git, as before. **Probe.** | State | Before | After | |---|---|---| | `.git/modules/spec` at mode 000 (or r--), gitfile readable | `REFUSED pin-commit-unresolvable: spec: the pinned commit ... (... exited 128: fatal: not a git repository: <root>/spec/../.git/modules/spec)`, lockstep remedy | `REFUSED pin-submodule-unreadable: .git/modules/spec: cannot be read (Permission denied)`, the checkout remedy | | `spec/.git` a directory at mode 000 (or r--) | `REFUSED pin-commit-unresolvable: ... exited 1: )`, lockstep remedy | `REFUSED pin-submodule-unreadable: spec/.git: cannot be read (Permission denied)`, the checkout remedy | The same leg with `spec/.git` a searchable directory passes before and after, and so does a gitfile ending in CR LF. **Change.** `_no_file` (`validate-pins.py`) now takes the root. Where nothing stands at the file's name, it asks `_not_a_directory` about the directory the file is in. Something that is not a directory there is named, with a remedy of its own: ``` Remediation: this validator reads each leg's pin, the shape pin, each neutral-product pin, project.yaml and every copy the shape pin lists, and one it cannot read is a question it cannot answer, never a pass. Something that is not a directory stands where the directory that holds it should be: move it aside, or take it away if nobody wants it, and restore the directory from the project's history. Then run the same command again. ``` `_what_stands` is #919's `_not_a_file` reading, now shared by both: `lstat`, then `stat` through a symlink, opening nothing. **Probe.** `contracts` moved aside and replaced. The first pin a run asks is the leg's. - Before, for each: `REFUSED pin-missing: leg 'spec' declares testorg/Atlas-spec at 'spec' but contracts/spec-pin.yaml does not exist`, lockstep remedy. - After: `... but contracts/spec-pin.yaml is under contracts, which is a file, not a directory`, and the same with `a FIFO`, `a socket`, `a symlink to nothing`, `a symlink loop` and `a symlink to a file`, each with the remedy above. - `contracts` absent, and `contracts` a symlink to the real directory, print what they printed. The shape pin and a neutral-product pin, which a run reaches only after the legs, say the same in-process. The manifest's directory is the root, which always is one. **Change.** `gh-response-unreadable` and `gh-tree-truncated` name the pin's repository with `shown_path`, as #919 did for `gh-unreadable`. **Probe**, in-process, with the repository `testorg/x<ESC>[31mRED`: - Before: `REFUSED gh-tree-truncated: the forge truncated its tree listing for testorg/x<ESC>[31mRED @ cccc..., so ...`, with the ESC raw. - After: `... for 'testorg/x\x1b[31mRED' @ cccc..., so ...`. `gh-response-unreadable` is the same. A printable repository is named byte for byte as before. - **r2:** a socket in each file's place: the four refusal sites and the shape copy (`is a socket, not a file`). It is bound by its name after a `chdir`, because AF_UNIX takes about 100 bytes and a temporary path need not fit. POSIX only. - **r4:** `Refusal`, as the gate builds one, replaced by a subclass whose default is a remedy, as a pre-#370 `repo_shape.py`'s is. Then the leg pin, the shape pin and `project.yaml` are taken away. Each refusal keeps that default, so it was built with no remedy, not with an empty one. `probe968.py` is rv919's `probe919.py` with every case run on all four variants and 28 cases added: **89 cases, 356 records**. The variants are the base and the head `validate-pins.py`, each beside the base or the head `repo_shape.py`, and each beside the pre-#370 `repo_shape.py` (`b0826e02`). The probe tree is under `_lane5-scratch`, not `$Q`. - `cmp968.py`: **every case #968 does not name is byte-identical** between base and head (stdout, stderr and exit), beside both `repo_shape.py` versions. That includes rv919's 61 cases and the new controls: `contracts` absent or a symlink to the real directory, a leg's own `.git` directory that can be read, a gitfile ending in CR LF, a gitfile naming nothing, a garbage or empty gitfile, `HEAD` at mode 000, a truncated or valid listing, and JSON that does not parse. - The 21 cases it names differ, as above. - Skew: wherever the base printed the same beside both libraries, the head does too. Items 2 and 3 pass their remedies explicitly, so no lockstep remedy is added beside an older library. Pinned copies: every project re-pins `validate-pins.py` and `repo_shape.py`. `validate-pins.py` imports nothing new from `repo_shape` (#370, #610). `tests/test_validate_pins_spelled_and_named.py`, 106 tests (it had 51): - **Item 1:** the stub `gh` printing a byte that is not UTF-8 (POSIX), with the skip line exact; that byte in-process; stdout that is not UTF-8; six kinds of JSON that is not a listing; a valid listing's digest, with CR LF. - **Item 2:** `.git/modules/spec` and `spec/.git` at mode 000 and r--, run as a project runs them (they skip as root and on Windows); a refused `stat` of `HEAD` in-process, a permission and an I/O error; a missing `HEAD` left to git; `_leg_git_dir` for a relative, CR LF, unterminated, blank-line-after and absolute `gitdir: `, a directory and a gitfile naming none. - **Item 3:** `contracts` replaced by a file, a FIFO, a socket, a symlink to nothing, a loop and a symlink to a file, whole stderr exact; the shape-pin and neutral-pin sites in-process; `contracts` absent, byte for byte. - **Item 4:** `gh-response-unreadable` (bad JSON, and JSON that is not a listing) and `gh-tree-truncated` for four unprintable repositories and a printable one. - **Item 5:** the socket rows, and r4 at three sites. - The `subprocess.run` stand-in hands bytes, as the code now reads them. - `tests/test_git_output_decoding.py`: the `EXEMPT` row for `tree_digest_from_gh` is gone, since that call asks for no text now. **Fail on base.** Both files run against `2f5cabf5`'s code: **51 failed, 122 passed**. The new tests that pass there by design: - the r2 and r4 rows (five socket rows and three r4 rows): #919's code already does what they pin, and they exist to kill those mutants; - the controls: `contracts` absent, a valid listing's digest, the printable repository's rows for bad JSON and a truncated listing, and a missing `HEAD` left to git. Eight of #919's in-process gh rows fail there only because the stand-in now hands bytes, where the base asked for text. Each mutant is applied alone to a scratch tree at the head and run against the two changed test files. **All 23 are killed.** | Item | Mutant | First test it fails | |---|---|---| | 1 | gh's stderr decoded strictly | `test_gh_s_stderr_that_is_not_utf_8_is_a_skip_and_no_traceback` | | 1 | gh's stdout decoded with `replace` | `test_gh_json_that_is_not_utf_8_is_unreadable_and_no_traceback` | | 1 | a lone CR not read as a line break | `test_gh_s_stderr_in_the_skip_line_is_spelled_on_one_line` | | 1 | only `KeyError` caught for JSON that is not a listing | `test_every_gh_refusal_names_the_repository_spelled[not-a-listing-newline]` | | 1 | `UnicodeEncodeError` not caught | `test_json_that_is_not_a_tree_listing_is_unreadable[a-lone-surrogate]` | | 1 | the call back in text mode | `test_gh_s_stderr_in_the_skip_line_is_spelled_on_one_line` | | 2 | no git-directory check | `test_the_git_directory_a_leg_gitfile_names_unsearchable_is_refused[000]` | | 2 | the gitdir named unnormalized (`spec/../.git/modules/spec`) | `test_the_git_directory_a_leg_gitfile_names_unsearchable_is_refused[000]` | | 2 | the `gitdir: ` line's line end kept | `test_the_git_directory_a_leg_gitfile_names_unsearchable_is_refused[000]` | | 2 | the directory stat'ed, not its `HEAD` | `test_the_git_directory_a_leg_gitfile_names_unsearchable_is_refused[000]` | | 2 | the files' remedy, not the checkout's | `test_the_git_directory_a_leg_gitfile_names_unsearchable_is_refused[000]` | | 2 | a `.git` directory not asked | `test_a_leg_git_directory_this_user_cannot_search_is_refused[000]` | | 3 | no check of the directory the file is in | `test_what_stands_where_contracts_should_be_is_named[file]` | | 3 | that check asks for a file, not a directory | `test_a_file_that_is_not_there_keeps_its_words_byte_for_byte[leg-pin]` | | 3 | the not-a-file remedy given | `test_what_stands_where_contracts_should_be_is_named[file]` | | 3 | the file named, not its directory | `test_what_stands_where_contracts_should_be_is_named[file]` | | 3 | a regular file left unnamed (`which is not a directory`) | `test_what_stands_where_contracts_should_be_is_named[file]` | | 3 | "a symlink to" dropped | `test_what_stands_in_a_files_place_is_named_with_a_remedy_that_fits[leg-pin-symlink-to-a-directory]` | | 4 | `gh-response-unreadable` names the repository raw | `test_every_gh_refusal_names_the_repository_spelled[not-json-newline]` | | 4 | `gh-tree-truncated` names it raw | `test_every_gh_refusal_names_the_repository_spelled[truncated-newline]` | | 4 | "not a tree listing" names it raw | `test_every_gh_refusal_names_the_repository_spelled[not-a-listing-newline]` | | 5 (r2) | a socket named "a FIFO" | `test_what_stands_in_a_files_place_is_named_with_a_remedy_that_fits[leg-pin-socket]` | | 5 (r4) | an empty remedy where none was given | `test_a_refusal_with_no_remedy_of_its_own_is_built_with_none[leg-pin]` | Through `run9.sh` (stub `gh` first on PATH, bogus token), at the head's tree: - **git 2.43.0, 16 files: 1167 passed, 0 skipped, rc 0, in 8 min.** They are the two changed files, the other five `test_validate_pins*` files, `test_pinned_copy_text.py`, the trio (`test_repo_hygiene.py`, `test_agents_md_codes.py`, `test_cli_reference.py`), `test_pin_validator.py`, `test_neutral_pin_validator.py`, `test_doctor_validator_rows.py`, `test_doctor_validator_verdicts.py` and `test_pin_source_env_collisions.py`. - **git 2.47.1, the two changed files: 173 passed, rc 0.** - Left to CI: the rest of the suite. `git merge-tree --write-tree` is clean against origin/main `ebbd9fa4`, fetched before the push, and against lane 3's `l3-917-yaml-reader` `e63ca75c`, which edits `repo_shape.py`'s YAML reader. The commits on main since `2f5cabf5` touch none of this change's four files. Review: rv985, lane 5's independent reviewer, not the writer (w968): READY at `c8b6d0fa`, the head that lands (issuecomment-6254758943), with no MUST, SHOULD or NIT. Its six-variant probe finds no change outside what #968 names. CI (Brett Heap's ruling 27): H0 is the head c8b6d0f itself, which passed tests, tests-git-bash and SonarCloud; main has touched none of its files since its base 2f5cabf; and the local gate passed on its predicted squash tree 6e24cee over main c4dd19d: the git_output_decoding, validate_pins_spelled_and_named, repo_hygiene, agents_md_codes, cli_reference sets, 343 passed. Landed on Brett Heap's word of 2026-10-18T14:45:28Z, relayed to every lane by lane 3 (#473 issuecomment-6251255596): "merge v1.2.0 PRs once reviewed and green". Closes #968 Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

brettheap marked this pull request as ready for review October 18, 2026 18:19

brettheap merged commit 2f5cabf into main Oct 18, 2026
5 checks passed

brettheap deleted the fix/919-pins-spelled-and-named branch October 18, 2026 18:20

Navigation Menu

Repository navigation

validate-pins.py and repo_shape.py: `--root` and gh's stderr are spelled, what stands in a pin's, the manifest's or a shape copy's place is named, and a leg's `.git` gitfile this user cannot open is refused by name (#919) - #952

validate-pins.py and repo_shape.py: `--root` and gh's stderr are spelled, what stands in a pin's, the manifest's or a shape copy's place is named, and a leg's `.git` gitfile this user cannot open is refused by name (#919)#952
brettheap merged 1 commit into
mainfrom
fix/919-pins-spelled-and-named

brettheap commented Oct 18, 2026 •

edited

Loading

Uh oh!

brettheap commented Oct 18, 2026

Uh oh!

chatgpt-codex-connector Bot commented Oct 18, 2026 •

edited

Loading

Uh oh!

sourcery-ai Bot commented Oct 18, 2026

Interacting with Sourcery

Customizing Your Experience

Getting Help

Uh oh!

sonarqubecloud Bot commented Oct 18, 2026

Uh oh!

chatgpt-codex-connector Bot commented Oct 18, 2026

Uh oh!

brettheap commented Oct 18, 2026

Uh oh!

sourcery-ai Bot commented Oct 18, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

brettheap commented Oct 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

U176: --root is spelled

U177: gh's stderr is spelled, on one line

U178: what stands in a file's place is named

U179: a gitfile this user cannot open is refused by name

Pinned copies

Tests

Runs

Mutants

Merge checks

Summary by Sourcery

Uh oh!

brettheap commented Oct 18, 2026

Uh oh!

chatgpt-codex-connector Bot commented Oct 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

sourcery-ai Bot commented Oct 18, 2026

Reviewer's Guide

Sequence diagram for validator diagnostics and file checks

File-Level Changes

Assessment against linked issues

Interacting with Sourcery

Customizing Your Experience

Getting Help

Uh oh!

sonarqubecloud Bot commented Oct 18, 2026

Quality Gate passed

Uh oh!

chatgpt-codex-connector Bot commented Oct 18, 2026

Uh oh!

brettheap commented Oct 18, 2026

Findings

Checked and true

Could not verify

Checklist candidates

Uh oh!

sourcery-ai Bot commented Oct 18, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

brettheap commented Oct 18, 2026 •

edited

Loading

U176: `--root` is spelled

chatgpt-codex-connector Bot commented Oct 18, 2026 •

edited

Loading