Skip to content

validate-pins.py: a leg git directory's HEAD this user cannot open, under .git/modules/<leg> or in a .git directory, is refused pin-submodule-unreadable by name, where it ended pin-commit-unresolvable (#1222) - #1239

Draft
brettheap wants to merge 1 commit into
mainfrom
l3-1222-gate-head-unreadable

Conversation

@brettheap

@brettheap brettheap commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Goal

#1222, the gate half: a leg whose git directory's HEAD file this user cannot open is refused by name, pin-submodule-unreadable, by validate-pins.py, so the gate no longer says the pinned commit is missing. The doctor half follows in #1248, now that #1200 has landed (50869f38); see "What the doctor half will need".

What changed

A v1.2.1 follow-up from lane 3's independent reviews (#1222). It lands under Brett Heap's v1.2.1 merge word (2026-10-23T13:10:26Z) once reviewed and green.

It is RV1200's N2, from the review of #1200. One script, templates/assembly-root/scripts/validate-pins.py, and its test file: 2 files, +111 and -4 lines against e04826db.

  • _refuse_an_unreadable_git_dir asked the HEAD of a leg's git directory only by stat (validate-pins.py and repo_shape.py after #919: gh's stderr that is not UTF-8 ends in a traceback, a leg's git directory this user cannot read is called an unresolvable pin with no reason, and two gh details still name the repository raw #968). A stat of a file at mode 000 works while its directory can be searched, so a HEAD at 000 passed, and git rev-parse, which could not read it, ended the run in pin-commit-unresolvable under THE LOCKSTEP RULE.
  • After that stat, a HEAD that is there is now opened by _gitfile_opens, the gate's validate-pins.py and repo_shape.py: --root and gh's stderr are printed raw, a non-file in a pin's place is called absent, and a leg's .git gitfile at mode 000 passes #919 open of a gitfile, so it is opened the same way:
    • only where it is a regular file, since the open of a FIFO waits for a writer;
    • an open that finds nothing (_ABSENT_ERRNOS) is a git directory with no HEAD, which is git's to answer for;
    • a refused open is pin-submodule-unreadable, <HEAD>: cannot be read (<reason>), naming the file (.git/modules/spec/HEAD or spec/.git/HEAD), with the leg checkout's remedy and the advice for the cause, as the gate refuses an unreadable git directory today.
  • Unchanged:
    • a refused stat of HEAD still names the git directory (.git/modules/spec, spec/.git);
    • a git directory with no HEAD is still left to git;
    • a gitfile that names no directory is still left to git;
    • a gitdir: line that holds a NUL is still left to git: its HEAD is nothing there, as _is_there answers it.
  • Decided here:
    • _gitfile_opens is reused rather than renamed or copied. Its docstring now says it also asks a git directory's HEAD. A rename would move lines that other lanes' PRs may call, and a copy would repeat its body.
    • The open sits behind if _is_there(head):. The NUL row pins that guard: without it, a ValueError ends the run (mutant m5).
  • The docstrings of _gitfile_opens, _refuse_an_unreadable_git_dir and _check_pin_digest say all of this, and so does the test module's docstring.
  • It is a pinned copy. Holders see an upstream-changed row for scripts/validate-pins.py on their next update-shape.py check. A grep of tests/ found no byte pin or digest pin of this file's bytes. No file holds a 64-hex digest beside its name. test_pin_source_env_collisions.py compares the copy with a blob it reads when it runs. The gate's six test files and the docs guards were run (below).

Measured before the change

Python 3.12.3, git 2.43.0, uid 1000. Every run was offline: a stub gh (exit 97) first on PATH, GH_TOKEN=bogus, an empty GH_CONFIG_DIR, GIT_SSH_COMMAND=false and GIT_CONFIG_GLOBAL=/dev/null. The projects are the test harness's real scaffolds (project), kept by one run with -o tmp_path_retention_policy=all. The gate was run as python3 scripts/validate-pins.py in the project, with the bytes at main (e04826db) and then this branch's.

Leg HEAD at 000 main e04826db this branch
gitfile leg .git/modules/spec/HEAD REFUSED pin-commit-unresolvable: spec: the pinned commit 1f482c9f... is not in that submodule's object store (... exited 128: fatal: not a git repository: .../Atlas/spec/../.git/modules/spec), then the lockstep remedy. Exit 2. REFUSED pin-submodule-unreadable: .git/modules/spec/HEAD: cannot be read (Permission denied), then the leg checkout's remedy and the permission advice. Exit 2.
.git-directory leg spec/.git/HEAD REFUSED pin-commit-unresolvable: spec: the pinned commit 1f482c9f... is not in that submodule's object store (... exited 1: ). Exit 2. Git, which cannot read that HEAD, passes over spec/.git and walks up to the root's repository: git -C spec rev-parse --show-toplevel printed the root. REFUSED pin-submodule-unreadable: spec/.git/HEAD: cannot be read (Permission denied), with the same remedy. Exit 2.

With HEAD readable, this branch's gate exits 0 on both projects. Main's bytes exit 1 there, with one finding, shape-copy-drift for scripts/validate-pins.py. That finding comes from the probe, not from main: the projects were scaffolded from this branch's template, and the probe swapped main's bytes in.

The new rows were also run at main's gate before the change: 6 failed and 4 passed. The 2 mode rows got pin-commit-unresolvable, and the 4 in-process rows ended DID NOT RAISE Refusal. The 4 rows that passed are the controls.

Tests

tests/test_validate_pins_spelled_and_named.py, +91 lines: 5 functions and 10 rows (116 collected at this head, 106 at main), beside #968's rows.

  • test_a_leg_git_directory_head_this_user_cannot_open_is_refused ([gitfile-leg], [git-directory-leg]): the issue's two shapes, at mode 000, run as a project runs the gate. The control comes first: readable, the same leg passes, exit 0. Then exactly one pin-submodule-unreadable naming the HEAD, with the remedy and the permission advice, exit 2, and nothing written. Skipped as root and on Windows (CANNOT_RUN_AS_ROOT).
  • test_a_leg_git_directory_head_whose_open_is_refused_is_refused_everywhere: the in-process stand-in for those hosts. Both shapes, with the open of HEAD refused for permission and for an I/O error, each with its own advice.
  • test_a_head_gone_when_it_is_opened_is_left_to_git: an open that finds nothing is no refusal, and the run finds nothing.
  • test_a_head_that_is_no_regular_file_is_not_opened ([directory], [fifo]): an open of either fails the row rather than hanging it.
  • test_a_head_no_system_call_can_name_is_left_to_git: a gitdir: line that holds a NUL (built with chr(0)).
  • validate-pins.py and repo_shape.py after #919: gh's stderr that is not UTF-8 ends in a traceback, a leg's git directory this user cannot read is called an unresolvable pin with no reason, and two gh details still name the repository raw #968's test_a_leg_git_directory_with_no_head_is_left_to_git passes unchanged.

Every pytest ran under the stub gh above, with EMAIL=tests@openreposhape.invalid, /usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<scratch>/bt-<name> -o tmp_path_retention_policy=failed, one at a time:

shape-doctor.py is not changed, so tests/test_shape_doctor.py was not run.

Mutants

Each mutant was applied to the template, then the gate's leg rows were run (-k "head or no_head or git_directory or gitfile or no_system_call"). The file was then restored, and cmp with the saved good copy was clean each time.

Mutant Result
m1 the open dropped (_gitfile_opens(head) deleted) KILLED: both mode rows and all 4 in-process rows fail
m2 the refusal names the git directory, not its HEAD KILLED: the same 6 rows fail
m3 HEAD opened whatever it is, with head.open("rb") in place of _gitfile_opens KILLED: the gone-at-open row and both no-regular-file rows fail
m4 the open asked with no _is_there guard, as a direct open KILLED: the same 3 rows fail
m5 the if _is_there(head): guard made a bare statement, with the open kept KILLED: the NUL row fails with ValueError: embedded null byte

Merges

git merge-tree --write-tree of this head (fff42145) with:

What the doctor half will need (it follows in #1248, now that #1200 has landed)

At #1200's head, mount_read asks is_a(gitdir / "HEAD", lambda _mode: True), which is a stat and the gate's old check. To mirror this gate:

  • Where that HEAD is there, name it as the gate does, after its mount, with git_dir_named(root, rel, gitdir / "HEAD"): spec: .git/modules/spec/HEAD, or spec/.git/HEAD.
  • Open it only where it is a regular file, as _gitfile_opens does. An open that finds nothing stays git's. A refused open is cannot_be_read(<that name>, exc).
  • The doctor has no open helper of that shape yet, so it needs one. git_dir=False (leg shape files) still asks nothing.
  • tests/test_doctor_unreadable_paths.py then needs:
    • two GIT_DIR_CELLS: .git/modules/spec/HEAD at 000, named spec: .git/modules/spec/HEAD, and spec/.git/HEAD at 000 on the own-directory leg, named spec/.git/HEAD. The legs and members rows read INVALID with the gate's line and not DRIFTED, and placement does not say "not a git repository";
    • an in-process row with the open of HEAD refused;
    • the parity row, which runs the gate's _refuse_an_unreadable_git_dir and the doctor's mount_read over the same HEAD-open-refused shape and asserts that the doctor's line ends with the gate's detail, as the directory's parity row does now;
    • the no-regular-file and gone-at-open controls.

Out of scope

Part of #1222

Lane: openreposhape-3 (openRepoShape-3)

🤖 Generated with Claude Code

… under `.git/modules/<leg>` or in a `.git` directory, is refused `pin-submodule-unreadable` by name, where it ended `pin-commit-unresolvable` (#1222)

`_refuse_an_unreadable_git_dir` asked the `HEAD` of a leg's git directory
only by `stat`, which a `HEAD` at mode 000 passes, so git, which could not
read it, ended the run in `pin-commit-unresolvable` under THE LOCKSTEP
RULE. A `HEAD` that is there is now opened as `_gitfile_opens` opens a
gitfile: only where it is a regular file, and an open that finds nothing
is a git directory with no `HEAD`, left to git. One this user cannot open
is named, `.git/modules/spec/HEAD` or `spec/.git/HEAD`, with the leg
checkout's remedy.

The gate half of #1222; the doctor half waits for #1200.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@brettheap brettheap added this to the v1.2.1 milestone Oct 23, 2026
@sourcery-ai

sourcery-ai Bot commented Oct 23, 2026

Copy link
Copy Markdown

Reviewer's Guide

The gate now explicitly opens an existing regular HEAD in each leg’s git directory and reports unreadable heads as pin-submodule-unreadable, with targeted tests covering both git-directory shapes, platform-independent failures, and preserved Git-handled edge cases.

Sequence diagram for validating a leg git directory HEAD

sequenceDiagram
    participant Gate as validate-pins.py
    participant GitDir as Leg git directory
    participant HEAD as HEAD
    participant Git as Git

    Gate->>GitDir: _leg_git_dir(dotgit)
    Gate->>HEAD: _is_there(head)
    alt HEAD exists
        Gate->>HEAD: _gitfile_opens(head)
        alt HEAD is unreadable
            HEAD-->>Gate: OSError
            Gate-->>Gate: pin-submodule-unreadable
        else HEAD is readable
            HEAD-->>Gate: opened
        end
    else HEAD absent or unresolved
        Gate->>Git: leave HEAD handling to Git
    end
Loading

Flow diagram for unreadable submodule HEAD handling

flowchart TD
    A[Locate leg git directory] --> B[Build git directory HEAD path]
    B --> C{"_is_there(head)"}
    C -- No --> D[Leave missing HEAD to Git]
    C -- Yes --> E{"_gitfile_opens(head)"}
    E -- Regular and readable --> F[Continue pin validation]
    E -- Refused open --> G[Refuse with pin-submodule-unreadable]
    G --> H[Name .git/modules/<leg>/HEAD or <leg>/.git/HEAD]
Loading

File-Level Changes

Change Details Files
Validate a leg git directory’s HEAD by opening it when present, so unreadable files receive the existing pin-submodule-unreadable refusal instead of being misreported as an unresolved pinned commit.
  • Reuse _gitfile_opens for regular-file HEAD checks, preserving absent, non-regular, and invalid-path behavior for Git to handle.
  • Guard the open with _is_there and include HEAD in the reported path and remediation.
  • Expand documentation to describe the new gate behavior and its relationship to prior checks.
templates/assembly-root/scripts/validate-pins.py
Add coverage for unreadable HEAD files across both supported git-directory layouts and for the edge cases surrounding the guarded open.
  • Verify mode-000 and in-process permission/I/O failures produce the expected refusal, path, advice, exit status, and empty output.
  • Verify readable, missing-at-open, non-regular, FIFO, and NUL-containing gitfile cases remain controls delegated to Git.
  • Document the scenario and regression rationale in the test module.
tests/test_validate_pins_spelled_and_named.py

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sonarqubecloud

Copy link
Copy Markdown

@brettheap

Copy link
Copy Markdown
Contributor Author

(lane openRepoShape-3, 2026-10-24T14:41:31Z) RV1239, independent review of #1239 at fff4214 (Claude Opus 5.5): READY for the code. Two body edits are needed before landing; they are not commits.

Measured with my own probe. Each tree's gate ran in a project scaffolded by that tree, 45 s timeout. Main 50869f3 against this head:

  • HEAD at 000, gitfile leg and .git-directory leg: main pin-commit-unresolvable, exit 2. Head: pin-submodule-unreadable, ".git/modules/spec/HEAD: cannot be read (Permission denied)" or "spec/.git/HEAD: ...", exit 2.
  • A symlink HEAD to a file at 000 is now refused by name too.
  • Unchanged from main: a HEAD that is a directory, a dangling symlink (exit 0 when it points into refs/), no HEAD, a git directory at 000, and a readable leg (exit 0).
  • A FIFO HEAD hangs at main AND at this head (exit 124 at 45 s). The hang is git's own open of HEAD; git_out has no timeout. Not a regression and not this PR, but it wants a follow-up issue.
  • As root (sudo, a scratch safe.directory config), HEAD at 000 exits 0 at main and here: root keeps its old answer. The new rows as root: 11 passed, 2 skipped (the mode rows, CANNOT_RUN_AS_ROOT).

Tests at the head: the 27 leg rows passed; test_path_guard_hygiene, test_repo_hygiene and test_no_auto_maintenance gave 280 passed, 2 skipped (test_no_auto_maintenance.py:222, GIT_CONFIG_GLOBAL set). The full six gate files were not re-run (window).
Mutants, my own script, each restored and cmp-clean: k1 open dropped, k2 the directory named not its HEAD, k3 HEAD named before its stat, k4 a gone HEAD treated as refused, k5 no regular-file guard. All KILLED.
Pinned copy: no byte or digest pin of validate-pins.py in tests/ or anywhere in the merged tree.
Merge-tree clean against main 50869f3 (77d9529a) and #1147 569ea05, #1172, #1159, #1144, #1109, each against the head and against main+head.
Sonar: gate OK, 0.0% duplication, 0 issues. CI: 4/4 green on fff4214 (from before #1200 landed).

Doctor parity: on main+head, tests/test_doctor_unreadable_paths.py gave 68 passed. Yet with HEAD at 000 the gate refuses .git/modules/spec/HEAD while the doctor's mount_read answers (True, True, None). The parity rows do not catch that split. The doctor half must:

Body:
B1 strip the "Summary by Sourcery" block after the robot line.
B2 #1200 has landed (50869f3): update "waits for #1200", "(after #1200 lands)", main 0ed3e57 -> 50869f3, and #1147 b75babe -> 569ea05.
N1 the new rows are 10, not 11 (116 at the head, 106 at main).
N2 the 2 hygiene skips are test_no_auto_maintenance.py:222.

Lane: openreposhape-3 (openRepoShape-3)

brettheap pushed a commit that referenced this pull request Nov 8, 2026
…e its `stat` passes, as the gate does, so a root `HEAD` at mode 000 is `.git/HEAD: cannot be read (<reason>)` and `INVALID (pins)`, never `no gitlink recorded` and DRIFTED (rv1248's S1, N3, N4) (#1222)

S1 (rv1248's SHOULD, a MUST on a main that holds #1147). #1147's
`root_git_read` asked the root's `HEAD` by `stat` alone. A `HEAD` at mode
000 stats, so git could not read the root, the `legs` row said `no gitlink
recorded` of each leg and the verdict was `DRIFTED (1 leg(s) not at the
pin)`, while `validate-pins.py` (`_refuse_an_unreadable_root`, through
`_refuse_an_unreadable_git_dir` with #1239's open) refused
`git-repo-unreadable: .git/HEAD: cannot be read (Permission denied)`: the
#1146 defect again. `root_git_read` now names `HEAD` once its `stat` finds
it and opens it through `regular_file_opens`, only where it is a regular
file. A refused open is the root's line, so no gitlink is read and no leg is
compared with its pin, and the verdict is `INVALID (pins)`. A refused
`stat` still names the git directory; an absent `HEAD`, one gone at the
open, a directory and a FIFO are left to git as before. `named` starts from
the root's `.git` path, so `.git/HEAD` is spelled relative to the root, in
POSIX, on every platform, as the gate's `_named` spells it.

N3. `regular_file_opens` keeps its False/True answer, and its docstring now
says why: its two callers (`mount_read`, `root_git_read`) ask it only for
its raise, but it is the gate's `_gitfile_opens`, whose answer the gate
reads of a leg's gitfile (`initialized`, #919), so one question keeps one
answer in both files for any later caller here. A new parity row holds the
two to the same answer for each shape, so rv1248's m11 is no longer an
equivalent mutant.

N4. The leg parity row's `HEAD_SHAPES` gains a `HEAD` that links to a file
at mode 000, and a new row asks #1239's NUL `gitdir: ` line of both parity
pairs (a leg's gitfile, the root's gitfile): neither tool refuses it.

Rows: in `tests/test_root_git_unreadable.py`, the root `HEAD` at 000 (a mode
row, doctor and gate, COMPLIANT once put back), its open refused in process
for a permission and an I/O error (doctor and gate, each with its advice),
three open rows in the doctor-and-gate sentence row (the git directory, and
a gitfile naming one inside and outside the root), a `HEAD` gone at the
open, and a directory `HEAD` among the absent controls. In
`tests/test_doctor_unreadable_paths.py`, the parity row of `root_git_read`
against `_refuse_an_unreadable_root` over three root layouts and nine `HEAD`
shapes, the NUL row, and the `regular_file_opens` parity row.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant