Skip to content

adopt-project.py: check asks this machine's git what the split stores for each file the plan keeps or moves and each pinned copy, so one it would store as other bytes no longer passes check and fails after both legs exist (#654) - #743

Merged
brettheap merged 6 commits into
mainfrom
fix/654-adopt-reencoded-kept-files
Oct 16, 2026

Conversation

@brettheap

@brettheap brettheap commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Lane: openreposhape-1 (openRepoShape-1)
Claim: #654 (comment)

What

check now asks this machine's git what the split's checkout, the mount's git rm and the split's git add make of every file the plan keeps in the assembly root or sends to a leg (or drops), and of each pinned shape copy where the split writes it. A file git would store as other bytes than the split expects, or would refuse, is a new finding, plan-file-reencoded, and execute refuses it before any leg exists. Since #619, check asked this only about the source's own root .gitattributes. Every other such file passed check and failed after both leg remotes existed.

The finding comes in up to four groups, in this order. Each names up to ten paths in tree order, then counts the rest:

  • kept files the split's git add -A would store as other bytes, which the verification then reports as adopt-lost:

    FINDING plan-file-reencoded: the plan keeps 1 file in the assembly root, and this machine's `git add` would store it as other bytes than its blob once the split's checkout has written it: `notes.txt`. The split's `git add -A` stores again each one git reads again (every file of a small source; of a large one, those checked out in the second git wrote its index), and the verification then reports it as `adopt-lost`, in no leg and not in the root tree, after both legs exist. Commit it in the source as git stores it (`git add --renormalize -- <path>`, then commit), or change the rule that reaches it, and re-run `plan`.
    
  • sent to a leg or dropped: files the mount's git rm refuses as holding local modifications, so the split stops at mounting the legs. The way out is the same.

  • pinned copies the source's rules make git add store as other bytes than the split writes, over which execute would say adoption verified. The way out names, for each copy, the line that keeps the rule off it, such as /scripts/bootstrap.py -filter.

  • git's refusals, one finding for each step git refuses, in git's own words: the split's checkout, its git add -A of what the root keeps, and its add of the pinned copies. Under core.safecrlf=true:

    FINDING plan-file-reencoded: this machine's git refuses to store again what the plan keeps in the assembly root, as the split's `git add -A` does (git said `fatal: CRLF would be replaced by LF in notes.txt`), so `execute` would stop there, after the leg repositories exist. Commit the file git names in the source as git stores it, or change the rule or the setting that refuses it, and re-run `plan`.
    

Riding along:

Why (repro on main)

22 real adoptions (plan, check, execute --local-remote-dir) on main at a524542, with git 2.43.0 and git-filter-repo 2.47.0, a stub gh first on PATH (no network):

source (the file is notes.txt unless named) main this PR
kept CRLF under * text eol=lf; kept CRLF docs/.gitattributes; kept $Id: deadbeef $ under ident; kept, under a filter this machine defines; kept *.txt whose working-tree-encoding rule goes to a leg with the root .gitattributes (5) check 0, then adopt-lost after both legs, exit 1 found (kept)
the CRLF file, the ident file, a CRLF file under core.safecrlf=true and a src/*.py under that filter, each sent to a leg (4) check 0, then refused at mounting the legs (git rm: local modifications) found (sent; the filter case also finds the pinned scripts it reaches)
* working-tree-encoding=UTF-16LE with .gitattributes -working-tree-encoding refused at mounting the legs, on the mount's own .gitmodules found (a refusal of the copies)
a kept non-UTF-8 file under working-tree-encoding; a kept CRLF file under core.safecrlf=true; *.py working-tree-encoding=UTF-16LE-BOM (#530's rider) (3) refused committing the split found (refusals)
filter.<x>.required with no command refused extracting the spec leg found (a refusal)
/scripts/*.py filter=<upper> adoption verified over uppercased pinned scripts; the recursive clone's validate-pins.py dies with a SyntaxError found (copies)
CRLF under text=auto; LF under text; CRLF with no rule under core.autocrlf=true; a filter= with no driver here; a valid and a non-UTF-8 file under working-tree-encoding, sent to a leg (6) verified no finding, and verified
* working-tree-encoding=UTF-16LE #619's plan-attributes-reencoded the same, then this finding's lines

On main's code at this branch's base (78fe3ff), the new tests fail as On main's code under Tests says.

Timing is part of the defect. Git reads a checked-out file again, and stores it through its rules, only where it cannot trust the index's stat data. In three identical runs of the split's own clone, checkout -B and git add -A over 20,000 CRLF files under * text eol=lf, git stored 852, 1,577 and 6,803 of them again. On a small source that is every file. So check asks about every file a rule reaches. It finds the source that can fail, not only the run that would.

How

Cost, measured at 78ffd72 (this head adds only an environment variable and a helper) on a shared 32-core machine while other test runs went (load average 35 to 60), median of 3. check on main's code is 78fe3ff's adopt-project.py; "the question" is _reencoded_files_findings alone, in process.

source check on main's code check here the question
this repository at a524542 (277 files) 0.52 s 1.25 s 0.56 s
20,001 files, no rule 2.84 s 3.78 s 0.90 s
20,001 files under * text=auto eol=lf, sent to a leg 2.91 s 14.1 s 9.3 s
the same, kept 6.11 s 18.2 s 11.3 s
20,000 CRLF files under * text eol=lf, sent to a leg (found) 2.51 s 12.4 s 9.4 s
the same, kept (found) 5.68 s 28.5 s 22.7 s

Most of it is git's own work. On the text=auto source, checkout-index of the 20,000 files took 5.8 s of the question's 9.0 s (2.1 to 3.3 s in the design's quieter session, so the disk was busy); on the kept CRLF one, git add -u storing 20,000 blobs took 12.2 s of 22.5 s. For scale, a real execute of the text=auto source took 24.6 s.

Tests

tests/test_adopt_reencoded_files.py, 53 tests:

  • 17 sources check finds, each in its group's words: the 15 of the table above that pass check on main, the same CRLF file dropped by the plan, and a required smudge filter that fails;
  • 4 sources execute refuses before any leg exists, one for each group;
  • 8 sources the split stores as their blobs: no finding, and each adopts end to end (16; the adoptions need git filter-repo). They are the six right verifications of the table, pinned scripts under *.py text eol=crlf with core.safecrlf=true, which the block keeps LF, and a kept .github/workflows/ whose filter rule's .github/.gitattributes goes to a leg;
  • the words of each group, and the ten-name cap (4);
  • the root's own .gitattributes, named by adopt-project.py: check finds, and execute refuses before any leg, a kept nested .gitattributes that outranks the root's line-ending rule for a pinned copy, and a root .gitattributes that git add would re-encode (#530) #619's finding alone (1);
  • a source no rule reaches checks out only its .gitattributes, and the checkout's index holds no stat data (2);
  • GIT_ATTR_SOURCE=HEAD, GIT_ATTR_SOURCE at the empty tree and a command-scope attr.tree do not steer the answer (3);
  • two malformed plans get findings, not a traceback (2);
  • a stand-in for git-lfs's required filter, whose smudge fails unless GIT_LFS_SKIP_SMUDGE is 1, gets no finding (1);
  • a kept file named shape where the shape's shape/.gitattributes goes (1), the root asked as the split writes it under core.autocrlf=true and core.safecrlf=true (1), and a kept root .gitattributes symlink out of the source, which the question never writes through (1).

Edited: tests/test_adopt_nested_attributes.py (L1-11 (2) in two tests, and L1-12's words in the encoding test, which now also expects plan-file-reencoded after #619's finding) and tests/test_adopt_attributes_eol_block.py (L1-11 (1)).

On main's code. With 78fe3ff's adopt-project.py and scripts/shape_materialize.py and this PR's tests, 34 of the three files' 169 tests fail: 31 of the new module, and the three L1-11 and L1-12 assertions. The 22 new tests that pass there are the controls: the 8 sources stored as their blobs, checked and adopted (16), the two malformed plans, the kept file named shape, the root's own .gitattributes, the root link and the git-lfs stand-in.

Revert table. Each change reverted alone in a clone of this head, committed there, and run with git filter-repo on PATH over the new module (53 tests), with the test a row names run beside it, or over the one file a row names:

change reverted fails
the _entry_findings line 29: every test that runs check or execute for a finding (the 22 controls, the gate and the clock pass)
_entry_findings' one probe not given to the question 1: tests/test_adopt_root_symlinks.py's test_the_probe_is_made_once_however_many_walks_ask (run with it)
ls-files -s back to diff-index --cached against the commit 1: tests/test_adopt_submodules.py's unreadable .gitmodules kept, its submodule dropped (REFUSED git-failed; run with that test's 5 cases)
place writing through a symlink 1: the root link (the file it reaches is overwritten)
the root's .gitattributes link asked as a kept file 1: the root link (a second finding)
GIT_LFS_SKIP_SMUDGE=1 not set 1: the git-lfs stand-in (its required smudge fails, and the checkout is refused)
git add -u back to git add --renormalize -- . 9: the text=auto and core.autocrlf controls, checked and adopted (4), and five refusals it does not raise (a kept non-UTF-8 file under an encoding, a kept CRLF file under core.safecrlf, the refusal's words, the root asked as the split writes it)
checkout-index -u 1: the clock
the mount's side asked with git add -u and its refusal 3: a CRLF file sent to a leg under core.safecrlf (found as a refusal instead), and a non-UTF-8 file sent to a leg under an encoding, checked and adopted (a refusal the mount's git rm never meets)
the moved .gitattributes left in place 2: the kept .github/workflows/ whose rule goes to a leg, checked and adopted
the checkout without the .gitattributes files 26
the pinned copies not asked 6: the four copy cases, execute's copy case and the copies' words
#471's block not appended to the root's own 2: the pinned scripts under *.py text eol=crlf with core.safecrlf, checked and adopted
the shape's own .gitattributes not written at the root 1: the root asked as the split writes it
the gate (every file checked out) 1: a source no rule reaches
_inherited_env() back to os.environ 3: the three steering cases
a refusal read as success 9: every refusal case
the root's own .gitattributes asked again 31
no ten-name cap 1
shape/.gitattributes placed where the root keeps one 1: tests/test_adopt_nested_attributes.py's kept shape/.gitattributes (run with it)
a refused checkout not read 1: the smudge filter that fails
update-index --replace dropped 1: the kept file named shape (REFUSED git-failed)
SPLIT_TEMPLATE back to ASSEMBLY_TEMPLATE 16 of tests/test_adopt_plan_yaml_quoting.py's 524 (bare_template points ASSEMBLY_TEMPLATE at a copy without .gitattributes)
_named_and_counted named _named again 43 of tests/test_adopt_plan_yaml_quoting.py's 524 (it replaced #181's _named: a TypeError)
L1-11 (1): scripts/shape_materialize.py's sentence 1 of tests/test_adopt_attributes_eol_block.py's 37
L1-11 (2): linked always false 1 of tests/test_adopt_nested_attributes.py's 79
L1-12: encoded always false 1 of tests/test_adopt_nested_attributes.py's 79

Runs on this head (9cf5e48, made from main at 78fe3ff; git merge-tree merges it cleanly with main at 55c6ba7, whose newer commits #610, #658, #650 and #425 change scripts/shape_materialize.py or adopt-project.py outside this diff's hunks), with /usr/bin/python3 -m pytest, over 29 files: the three test files this diff touches, tests/test_shape_doctor.py, tests/test_scaffold_leg_path_collisions.py, tests/test_cli_reference.py, tests/test_agents_md_codes.py, tests/test_repo_hygiene.py, and 21 adopt test files that run check or execute over kept files, symlinks, submodules, unusual names and the temporary directory:

  • with git filter-repo on PATH: 2788 passed, 3 skipped, 0 failed;
  • with it hidden (PATH=/usr/local/bin:/usr/bin:/bin behind a stub gh): 2637 passed, 154 skipped, 0 failed.

On 78ffd72 merged locally with main at b618ec1, the reviewer ran the three test files this diff touches, tests/test_adopt_root_symlinks.py, tests/test_adopt_submodules.py, tests/test_shape_doctor.py and tests/test_scaffold_leg_path_collisions.py: 1000 passed and 3 skipped with git filter-repo on PATH, and 954 passed and 49 skipped with it hidden.

These runs found three defects of the question's own on earlier heads of this branch, each fixed here with a test: it made a second symlink probe (tests/test_adopt_root_symlinks.py); diff-index read a .gitmodules git cannot parse and refused git-failed (tests/test_adopt_submodules.py); and a root .gitattributes link out of the source was written through, outside the throwaway repository (found by a probe). CI's full Linux suite runs on this pull request.

Windows

CI is Linux only since #597. The new tests commit every file byte for byte (committed_raw, from tests/test_adopt_nested_attributes.py), set git's config through GIT_CONFIG_COUNT rather than a file, and read the plan back with read_text(). Their filter commands, tr a-z A-Z and false, are tools Git for Windows ships. Every path they make (notes.txt, notes/n01.txt, docs/a.md, more.txt, shape, a/b/Thing, victim.txt) is legal on NTFS. The root-link test makes a symlink on the disk and skips where the platform cannot, and it points TMPDIR, which Python's tempfile reads first on every platform, at a directory of its own.

Not in this PR

Known limits

Closes #654

🤖 Generated with Claude Code

Summary by Sourcery

Make adoption checks detect Git-dependent file transformations and refusals before leg repositories are created.

New Features:

  • Add preflight detection for files and pinned copies that Git would re-encode, transform, or reject during project adoption.

Bug Fixes:

  • Prevent adoptions from proceeding when Git would later lose kept files, reject moved files, or verify incorrectly transformed pinned copies.
  • Avoid following symlinked root .gitattributes files during rehearsal and clarify their resulting diagnostics.
  • Correct handling and messaging for root .gitattributes files affected by working-tree-encoding.

Enhancements:

  • Model the split's checkout, mount removal, root staging, and pinned-copy staging in an isolated Git rehearsal using the current machine's configuration.
  • Group and cap file-reencoding findings with actionable remediation guidance, including Git refusal messages and affected paths.

Documentation:

  • Clarify how Git treats symlinked .gitattributes files and update related nested-attribute diagnostics.

Tests:

  • Add comprehensive coverage for re-encoded files, Git refusals, transformed pinned copies, symlink safety, environment isolation, path limits, and successful control cases.

Brett Heap and others added 4 commits October 16, 2026 14:17
…eckout, `git rm` and `git add` make of every file the plan keeps or moves and of each pinned copy, so a file they would store as other bytes no longer passes `check` and ends `adopt-lost`, or stops `execute`, after both legs exist (#654)

Since #530's D5(a) line (#619), `check` asked this machine's `git add`
about the source's own root `.gitattributes`. Any other file still passed
`check` and failed after both leg remotes existed. Measured on main
a524542 with git 2.43 in real adoptions: a kept CRLF file under `* text
eol=lf`, a kept expanded `$Id$` under `ident`, a kept file under a filter
this machine defines, and a kept file whose `working-tree-encoding` rule
went to a leg ended `adopt-lost`; the same files sent to a leg were refused
by the mount's `git rm` ("local modifications") at mounting the legs; git's
refusal (`core.safecrlf=true`, an encoding) stopped the split committing;
the pinned scripts under `*.py working-tree-encoding=UTF-16LE-BOM` (#530's
rider) stopped it too; and `/scripts/*.py filter=<upper>` said `adoption
verified` over uppercased pinned scripts.

`check` now finds these as `plan-file-reencoded`, after
`plan-attributes-reencoded` in `_entry_findings`, and `execute` refuses
them before any leg exists. A throwaway repository borrows the source's
objects (alternates; nothing copied, the source never written) and runs
with this machine's config. `git check-attr --cached` of the six attributes
that can change bytes, as the source holds its rules and as the split's
root will, picks the files a rule reaches; only those and every
`.gitattributes` are checked out, through indexes with no stat data, so git
reads every one again. The mount's side is asked as `git rm` asks it
(`update-index --refresh`), the split's side with a plain `git add -u`
(never `--renormalize`), and each pinned copy with its own bytes where it
lands. One finding per group (kept, sent to a leg or dropped, pinned
copies), each naming ten paths, then each of git's refusals in git's words.

Riding along:
- L1-11: shape_materialize.py's follow-up for a symlinked root
  `.gitattributes` says git follows no such link and a checkout reads only
  the link's own text, a path; `plan-nested-attributes` names the rule
  #471's follow-up asks for where the root's file is a symlink.
- L1-12: where a `working-tree-encoding` rule reaches the root's own
  `.gitattributes`, `plan-attributes-reencoded` says the split stops at
  mounting the legs and that `git add --renormalize` is no way out, and
  names `.gitattributes -working-tree-encoding`.

Lane: openreposhape-1 (openRepoShape-1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…aterializer reads them and names its helper apart from `_named`, so `plan`'s value questions and a copy of `ASSEMBLY_TEMPLATE` a test points elsewhere no longer break it

- `_named_and_counted` (the ten names a finding gives) had been `_named`,
  which already names a plan value for #181's gate; the later definition
  replaced it, and `plan`, `check` and `execute` ended in a TypeError over
  a misread value (tests/test_adopt_plan_yaml_quoting.py).
- `SPLIT_TEMPLATE` is where `materialize_assembly_root(SHAPE_ROOT, ...)`
  copies the shape's files from: the pinned copies' bytes and the shape's
  own `.gitattributes` (`_root_attributes`) are read there, not from
  `ASSEMBLY_TEMPLATE`, which #181's gate reads and `bare_template` points
  at a copy holding only the templated YAML.
- Two tests: a kept FILE named `shape`, where the shape's own
  `shape/.gitattributes` would go beside the root's (`--replace` in the
  split's index), and no rule of the source's under `core.autocrlf=true`
  with `core.safecrlf=true`, which the split's `git add -A` refuses once it
  has written the shape's root `.gitattributes` (replayed by hand).
- Comment and docstring lines re-wrapped.

Lane: openreposhape-1 (openRepoShape-1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…and reads what `git add` stored from the index alone, so a link no walk can follow no longer has git asked twice, and a `.gitmodules` git cannot parse no longer ends `check` in `git-failed` (#654)

The targeted runs at 30972de found two defects of the question's own:

- It walked the plan with a probe of its own, so a source keeping links
  the walks cannot follow asked git twice whether it checks a symlink
  out as one (tests/test_adopt_root_symlinks.py,
  test_the_probe_is_made_once_however_many_walks_ask). `_entry_findings`
  now makes one probe and gives it to `_root_symlink_findings` and
  `_reencoded_files_findings`.
- `diff-index --cached <commit>` made git read the `.gitmodules` the
  checkout wrote, for the gitlink the commit holds, and a `.gitmodules`
  git cannot parse ended `check` in `REFUSED git-failed` where it names
  `plan-gitmodules-unreadable` (tests/test_adopt_submodules.py, the
  `.gitmodules` kept and the submodule dropped). What `git add -u` stored
  is now read with `ls-files -s` (`_SplitRehearsal.staged`) and compared
  with each blob, as the pinned copies already were.

And a control the revert table showed was missing: a kept
`.github/workflows/` whose filter rule's `.github/.gitattributes` goes to
a leg has no finding and adopts, which fails where the rehearsal leaves
the moved `.gitattributes` in its tree.

Lane: openreposhape-1 (openRepoShape-1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ace of a symlink its checkout wrote, never through it, so `check` of a source keeping a root `.gitattributes` link out of it no longer overwrites the file that link reaches (#654)

A probe at 0224f96: a source at `<base>/a/b/Thing` keeping a root
`.gitattributes` that links to `../../../victim.txt`, `check --work-dir`
with `TMPDIR` at `<base>/tmpd`. The link leads to `<base>/victim.txt`
from the source and from the throwaway repository alike. The walk takes
no `.gitattributes` there (`plan-root-symlink`), so the question laid the
shape's own at the root, and `write_bytes` followed the link the checkout
had written: `<base>/victim.txt` then held the shape's `.gitattributes`.

- `_SplitRehearsal.place` removes a symlink at its path before it writes,
  and the root's own `.gitattributes`, with #471's block appended, is
  written through it too.
- A root `.gitattributes` kept as a symlink is not asked again: the split
  never writes through it, and `plan-root-symlink` or #471's follow-up
  names it. Without that, the file written in its place read as a kept
  file `git add` stores otherwise.
- A test: that source's `check` names `plan-root-symlink` alone and leaves
  the file the link reaches as it was.

Lane: openreposhape-1 (openRepoShape-1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sourcery-ai

sourcery-ai Bot commented Oct 16, 2026

Copy link
Copy Markdown

Reviewer's Guide

The PR extends check with a Git-backed rehearsal of the split’s checkout, leg removal, root staging, and pinned-copy staging, reporting plan-file-reencoded findings before execution can create legs; it also refines symlink and root-attribute diagnostics and adds broad regression coverage.

Sequence diagram for preflight Git re-encoding checks

sequenceDiagram
    participant Check
    participant Rehearsal as SplitRehearsal
    participant Git
    participant Source
    Check->>Rehearsal: _reencoded_files_findings()
    Rehearsal->>Source: Read commit objects via alternates
    Rehearsal->>Git: check-attr --cached for source and split layouts
    Rehearsal->>Git: checkout-index -a -f
    alt checkout refused
        Git-->>Rehearsal: refusal text
        Rehearsal-->>Check: plan-file-reencoded
    else checkout succeeds
        Rehearsal->>Git: update-index --refresh and diff-files
        Git-->>Rehearsal: moved or dropped files with local modifications
        Rehearsal->>Git: add -u for files kept in the root
        Git-->>Rehearsal: stored blobs or refusal
        Rehearsal->>Git: literal-pathspecs add -f for pinned copies
        Git-->>Rehearsal: stored blobs or refusal
        Rehearsal-->>Check: grouped plan-file-reencoded findings
    end
    Check->>Check: _refuse_what_check_finds()
    Check-->>Check: execute refuses before _create_leg_remotes
Loading

Flow diagram for grouped re-encoding findings

flowchart TD
    Start[check] --> Layout[Build source and split index layouts]
    Layout --> Checkout[Rehearse checkout]
    Checkout -->|refused| Refusal[Record Git refusal]
    Checkout -->|succeeds| Mount[Rehearse moved or dropped files with git rm comparison]
    Mount --> Kept[Rehearse kept files with git add -u]
    Kept --> Copies[Rehearse pinned copies with git add -f]
    Copies --> Groups[Group findings in order]
    Groups --> KeptFinding[Kept files: adopt-lost risk]
    Groups --> MovedFinding[Sent or dropped files: git rm risk]
    Groups --> CopyFinding[Pinned copies: adoption verified drift]
    Groups --> RefusalFinding[Git refusals]
    KeptFinding --> Execute[execute refuses before legs exist]
    MovedFinding --> Execute
    CopyFinding --> Execute
    RefusalFinding --> Execute
Loading

File-Level Changes

Change Details Files
Added rehearsal-based detection of Git re-encoding and refusal cases across files affected by the split.
  • Introduced a temporary repository using source-object alternates and inherited Git configuration to model checkout, leg removal, root staging, and pinned-copy staging.
  • Determined which files are reached by conversion attributes in both source and split layouts, including text, encoding, ident, and filter rules.
  • Added grouped plan-file-reencoded findings for kept files, moved or dropped files, pinned copies, and Git refusals, with capped path lists and remediation guidance.
  • Made execute reject these findings before creating leg remotes.
  • Protected symlink handling, Git index/stat behavior, Git steering environment, and unreadable submodule metadata during rehearsal.
adopt-project.py
Clarified symlinked-root and encoded-root-attributes behavior to match Git and split execution semantics.
  • Updated documentation and follow-up wording to state that Git does not follow symlinked .gitattributes files for attribute discovery.
  • Adjusted nested-attribute findings when the root .gitattributes is a symlink.
  • Expanded root-attributes re-encoding diagnostics for working-tree-encoding, including why renormalization is not a remedy.
adopt-project.py
scripts/shape_materialize.py
Expanded regression coverage for re-encoding detection and updated affected wording assertions.
  • Added comprehensive tests for kept, moved, dropped, pinned-copy, and refusal scenarios across line endings, encodings, ident, filters, safecrlf, and required filters.
  • Verified successful controls, execute-time refusal before leg creation, ten-path reporting limits, malformed plans, environment steering, symlink safety, root layout, and stat-independent reads.
  • Updated nested-attribute and symlink follow-up expectations, including encoded root .gitattributes behavior.
tests/test_adopt_reencoded_files.py
tests/test_adopt_nested_attributes.py
tests/test_adopt_attributes_eol_block.py

Assessment against linked issues

Issue Objective Addressed Explanation
#654 Have check detect kept or moved files other than the root .gitattributes whose bytes Git would re-encode or refuse during the split, and make execute refuse before any leg exists. ✅
#654 Document the whole-source rehearsal design and its cost, and correct the related wording for symlinked attributes and working-tree-encoding failures. ✅

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

…and lays its findings out in a helper, so an LFS source is no longer refused for objects its throwaway repository cannot fetch, and the comment no longer says the source is never written (#654)

From r654's review of 78ffd72:

- Finding 2. The throwaway repository has no remote, so git-lfs's
  required smudge there either fails, a refusal the split's own clone
  never meets, or reads, links or fetches every LFS object.
  `_SplitRehearsal` now runs git with `GIT_LFS_SKIP_SMUDGE=1`: an LFS file
  is checked out as its pointer, and git-lfs's clean stores a pointer as
  it is, while a blob under `filter=lfs` that is not a pointer is still
  stored otherwise and found. A test emulates git-lfs, as the review did,
  with a required smudge that fails unless the variable is 1 (no git-lfs
  here); it fails with the variable removed.
- Finding 1, accepted as it is. Where the question stores a blob the
  source holds, git's object freshening sets the modification time of the
  source's pack or loose object file to now, through the alternates. No
  byte of the source changes. The comment said the source is never
  written; it now says that.
- Finding 5. SonarCloud's python:S3776: `_reencoded_files_findings` had a
  cognitive complexity of 19, 15 allowed. Its findings are laid out in
  `_grouped_findings`, in the same order and words.

Lane: openreposhape-1 (openRepoShape-1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sourcery-ai

sourcery-ai Bot commented Oct 16, 2026

Copy link
Copy Markdown

Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 3 hours and 34 minutes by commenting @sourcery-ai review. Upgrade to get a review now.

@sonarqubecloud

Copy link
Copy Markdown

@brettheap
brettheap merged commit 66b230e into main Oct 16, 2026
5 checks passed
@brettheap
brettheap deleted the fix/654-adopt-reencoded-kept-files branch October 16, 2026 20:17

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sourcery assessment

Approved.

brettheap pushed a commit that referenced this pull request Oct 17, 2026
main gained 25 pull requests since de92614: #634, #699, #671, #719,
#701, #717, #700, #702, #722, #720, #725, #721, #728, #705, #727, #718,
#744, #743, #749, #740, #737, #734, #742, #751 and #736. No conflict:
AGENTS.md, adopt-project.py and docs/cli.md merged as text, and
README.md and docs/handbook.html are unchanged by the merge. README.md
is 1473 lines, its cap; AGENTS.md 494, its cap; every handbook marker is
current; `render-cli-reference.py --check` is ok.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap pushed a commit that referenced this pull request Oct 17, 2026
…es every reason THIS checkout cannot answer, and the notes are current to main bf1f099 with the 25 pull requests since de92614 (#563, #683)

- W683 and RV-f's N4 on #742: README's exit-3 list (Keeping a project's
  shape current, :1208-1220) and its handbook twin (`doctor`) named only a
  pin whose commit this checkout does not carry. They now say, as
  shape-doctor.py's EXIT CODES does: a pin's commit it lacks, a partial
  clone missing what is compared (#683), a read that fails
  (`update-upstream-unreadable`), a crashed validator of its own (#278,
  #410), or its own naming policy not loading (#678). +1 line, with
  tests/readme-cap.d/683.txt; README.md is 1474, its cap (1422 + 2 + 5 +
  12 + 22 + 10 + 1). The two `Keeping a project's shape current` markers
  move: `doctor` text and marker, `current` marker only.
- The notes, "What changed since v1.0.1", one entry per pull request from
  `git log --first-parent de92614..bf1f099`, each from its squash commit and
  the issues GitHub says it closes:
  - adopt-project.py: #699, #719, #749, #727, #717, #728 (with #621's
    hint), #743 + #720 + #718, #722, #721;
  - update-shape.py, bump-leg.py and a project's own scripts: #702 (with
    the pinned validators), #634;
  - scaffold, setup, family and the shared readers: #725, #671, #744;
  - shape-doctor.py: #742, #701, #737, #751, #736, #734, #700 + #705,
    #740;
  - install, CI and the docs: the exit-3 sentence above.
- Counts (99 pull requests since bfed143; 109 of 135 milestone issues
  closed at 10:28Z), the "current to main" sha and the open list (26) are
  at main bf1f099, still placeholders for tag time.

Left for round B: lane 5's eight lines (#630 issuecomment-6227129879),
whose pull requests #726, #747, #682, #724, #746, #745 and #738 are open
drafts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 17, 2026
…efusals and `family-operation-in-progress` name a path as `shown_source` names it, so an ESC or a line break in a path no longer reaches the terminal (#692) (#731)

#622's folder refusals printed a path as it was. `family.py init --into <a file named wd-ESC[31mRED> --dry-run` gave `REFUSED family-folder-cannot-be-made: the family folder .../wd-^[[31mRED/Orbit cannot be made: ...` through `cat -v`. A name with a line break and then `WARNING forged-line: pwned` printed that line as a line of its own. R612's `family-operation-in-progress` printed the mark's path across two lines for a holder whose path holds a newline.

Each path is now spelled as `shape_materialize.shown_source` spells a `--pin-source`: as it is, and as Python's `repr` spells it where a character in it is not printable.

- `scripts/shape_materialize.py`: both answers of `folder_cannot_be_made` (a file or a bad stat in the way, and a folder this user cannot write in), the folder in `refuse_unmakeable_folder`, and the folder in `make_folder`.
- `scaffold-project.py`: `scaffold-target-exists`.
- `scripts/family.py`: `family-target-exists`, and the mark's path in `family-operation-in-progress` (`where.as_posix()`).

A printable path is named byte for byte, so no existing row changes and `why.endswith(CANNOT_WRITE_IN)` in `refuse_unmakeable_folder` still holds. No help text changed, so `docs/cli.md` is untouched. No README, handbook or template sentence says how a refusal names a path (`git grep` of `shown_source`, `target-exists` and `folder-cannot-be-made` in AGENTS.md, README.md, `docs/handbook.html` and the templates found none), so no doc changes.

- **`shown_source`, not `shown_path`.** Since #634, `shape_materialize.shown_source` returns `repo_shape.shown_path(source)` (`scripts/shape_materialize.py`: `return shown_path(source)`), so the seven call sites this change adds (four in `shape_materialize.py`, one in `scaffold-project.py` and two in `family.py`) spell a path as `shown_path` does. Renaming them to `shown_path` stays out of this PR: it would only move imports in `scaffold-project.py` and `family.py` and reword docstrings in `family.py`, `shape_materialize.py` and the test file, and the review's N1 left it optional. I did not take it. The predicted merge with main works with the alias: `family.py --help` and `scaffold-project.py --help` exit 0 on it, and `shown_source` and `shown_path` return the same string for an ESC name, a line-break name and a plain path.
- **`shown_source` is imported by name into each file's existing `from shape_materialize import (...)` block.** The first commit put it on a line of its own after the block; lane 4's #660, #688 and #693 add an import at that same place, and `merge-tree` reported add/add conflicts in `scripts/family.py` and `scaffold-project.py`. The second commit moves the name into the block, and those conflicts are gone.
- **"The holder in the other refusals of the same block" is one refusal.** The `# What the holder is in the middle of, and what its sparse checkout leaves out` block holds `family-operation-in-progress`, `family-sparse-excluded` and `member-path-unaddable`. Only the first prints a path of the holder's. `family-sparse-excluded` names `members/<Project>` and `.gitmodules`, and `member-path-unaddable` names `members/<Project>`.
- **No scaffold row for `--local-remote-dir`.** A control character there is refused earlier as `scaffold-unwritable-value`, which already names the path with `repr`. The unit test covers `make_folder`, which only a race reaches by the command line.

`tests/test_dry_run_parity.py`:
- `SCAFFOLD_CASES` gains five rows: ESC and line break in the `--work-dir` name, ESC in a folder name above it, and ESC and line break in a work dir that holds a file named `Atlas` (`scaffold-target-exists`).
- `FAMILY_CASES` gains five rows: ESC and line break in `--work-dir`, ESC in `--into` (`the family folder`), and ESC and line break in a work dir that holds a file named `Orbit` (`family-target-exists`).
- Two rows join `UNWRITABLE_FOLDER_AT` (ESC in the scaffold's and in `init`'s `--work-dir`, a folder this user cannot write in).
- Rows named `ESC-...` and `line-break-...` are skipped on Windows, whose file names cannot hold them (`POSIX_NAMES`, `case_params`). Every expectation spells the path with `repr`, and not with `shown_source`.
- `assert_refused` now fails any stderr line that starts with `WARNING forged-line`.
- Three unit tests: `test_each_folder_question_spells_a_name_that_is_not_printable` (ESC, line break; `folder_cannot_be_made`, `refuse_unmakeable_folder` and `make_folder` by themselves), `test_a_folder_this_user_cannot_write_in_is_spelled_safe_to_show`, and `test_a_printable_name_is_named_byte_for_byte` (a name with a space and an accented letter stays unquoted).

`tests/test_family_refuse_up_front.py`: `test_a_holder_path_with_a_newline_still_shows_its_merge` asserted the raw newline in the refusal (`"Ink\nRouter/.git/worktrees/" in ran.stderr`). It is now `test_a_holder_path_with_a_control_character_still_shows_its_merge`, over a line break, an ESC and a forged line, and asserts the one-line `repr` form (`refusal()` fails unless the stderr is two lines). It is the one existing test this change had to edit.

Results at head `ab55c29f` (Python 3.12.3, Linux):
- `tests/test_dry_run_parity.py tests/test_dry_run_asks_remotes.py tests/test_family_dry_run.py tests/test_family_init_landing.py tests/test_repo_hygiene.py tests/test_pin_source_unreadable.py tests/test_scaffold_leg_path_collisions.py tests/test_scaffold_reuse_seeded_legs.py`: **494 passed**.
- `tests/test_family.py tests/test_family_refuse_up_front.py tests/test_family_init_values.py tests/test_family_add_refusals.py`: **356 passed, 19 skipped** (the skips are `test_family_init_values.py:219` and `:291`, template-quoting premises).
- Not run: the full suite (the coordinator's gate), and macOS and Windows (CI is Linux-only since #597). The new hostile-name rows are POSIX only by design.

On the base code: the review overlaid this head's two test files on main `2fac9790` and ran `tests/test_dry_run_parity.py` and `tests/test_family_refuse_up_front.py` with `-k "dry_run_parity or control_character"`. The result is **20 failed, 87 passed** (76 deselected, 43.5 s). The 20 are exactly the new and edited tests: the 5 scaffold and 5 `init` rows, the 4 unwritable-folder rows, the 3 unit tests (`test_each_folder_question_spells_a_name_that_is_not_printable` in two cases, and `test_a_folder_this_user_cannot_write_in_is_spelled_safe_to_show`), and the 3 cases of the edited `test_a_holder_path_with_a_control_character_still_shows_its_merge`. The one new test that passes on main's code is `test_a_printable_name_is_named_byte_for_byte`, by design: a printable name is named byte for byte on both. An earlier draft of this body said 21 failed, 2 passed. That run was on `8062810d` with a wider selection, and it held a scaffold row for ESC in the local remote dir's name (`ESC-in-the-local-remote-dir-name`) that this head's tests no longer have; its second pass was the existing `test_the_refusal_quotes_gh_s_first_line_and_escapes_what_is_unprintable`, which also passes in the new run, one of the 87.

`family.py init` and `scaffold-project.py --dry-run` against a file named `wd-ESC[31mRED`, a file named `wd-\nWARNING forged-line: pwned`, and a work dir with a file `Atlas` or `Orbit` in it, run with a stub `gh` and no real `gh`:
- On main, through `cat -v`: `... the family folder /.../wd-^[[31mRED/Orbit cannot be made: /.../wd-^[[31mRED is a file, not a folder`, and a stderr line `WARNING forged-line: pwned cannot be made: ...` of its own.
- On this head: `REFUSED family-folder-cannot-be-made: the family folder '.../wd-\x1b[31mRED/Orbit' cannot be made: '.../wd-\x1b[31mRED' is a file, not a folder`, and one line for the forged name: `--work-dir '.../wd-\nWARNING forged-line: pwned' cannot be made: '...' is a file, not a folder`. `scaffold-target-exists` and `family-target-exists` are one line each in the same form.

Each spelled site reverted alone in a throwaway clone at `19d126cd` (the call sites are the same at `ab55c29f`); the new tests that fail for each:
- `folder_cannot_be_made`, the `_not_a_folder` answer: 8 failed (six CLI rows and the two unit tests).
- `folder_cannot_be_made`, the `CANNOT_WRITE_IN` answer: 5 failed (the four unwritable rows and the unit test).
- `refuse_unmakeable_folder`: 12 failed.
- `make_folder`: 2 failed (the unit test only).
- `scaffold-target-exists`: 2 failed (its two rows).
- `family-target-exists`: 2 failed (its two rows).
- `family-operation-in-progress`: 3 failed (line break, ESC, forged line).

`git merge-tree --write-tree` of this head `ab55c29f` against origin/main `20c8873b` and against every ref on origin that could meet it, fetched 2026-10-16: the 41 branches (`fix/*`, `l3-*`, `wip/*`, `docs/*` and `release/v1.0`) and the 55 PR heads from #630 up. That is 96 refs. (The same list was run for #730's head `7f2bb36b`.)
- **origin/main `20c8873b`: clean** (the predicted tree `00b5b24c`, which differs from main in these five files only: 216 lines added, 25 removed).
- **#630's head `6fd33a93`: clean.** It conflicted with main in `docs/handbook.html` when this body was first drafted, and no longer does, with or without this change. This change touches no README or handbook.
- **The refs that edit one of this change's five files**, and so could meet it, all merge clean with this head: `fix/646`, `fix/651`, `fix/654`, `fix/660`, `fix/661`, `fix/684`, `fix/686`, `fix/688`, `fix/689` and `fix/693`, the heads of #670, #725, #728, #743 and #744, and the heads of the landed #634, #638, #639, #640, #643 and #671. #634 has landed (`30676c90`), so the `fix/660` note about a conflict with it is gone.
- **Four refs conflict with origin/main on their own**, and so with this head: `docs/release-notes-v1.0.0` and `release/v1.0`, and the pre-squash heads of the landed #637 and #641. Their conflicts do not come from this change. Every other ref, 92 of the 96 in all, is clean with this head, including the refs that conflict with main in `templates/family-root/scripts/validate-family.py` (#682, #745, #747, `fix/648`, `fix/681`, `fix/704`) or in `shape-doctor.py` (#742, `l3-683`) on their own. So do the old heads of the landed #638 and #639, and lane 3's `l3-677-pins-hand-step-wip` branch, and each of them merges clean with this head too.
- **Tests on the predicted merge**, run by the review on the merge with main `2fac9790`: `test_dry_run_parity`, `test_family_refuse_up_front`, `test_pinned_copy_text`, `test_pin_source_unreadable`, `test_dry_run_asks_remotes`, `test_family_dry_run` and `test_family_init_landing`: 445 passed in 215 s. `test_family`, `test_family_init_values`, `test_family_add_refusals`, `test_derived_id_refusals`, `test_repo_hygiene`, `test_naming_policy`, `test_scaffold_leg_path_collisions` and `test_scaffold_reuse_seeded_legs`: 758 passed, 19 skipped in 243 s. I did not re-run them on `20c8873b`; the merge is clean, and the two commits since `2fac9790` (#700 and #702) change only `shape-doctor.py`, `templates/assembly-root/scripts/validate-pins.py` and four test files, none of them a file of this change.
- Before the second commit, `fix/660`, `fix/688` and `fix/693` each conflicted with this head in the import region of `scripts/family.py` or `scaffold-project.py` (a separate import line each side added at the same place). That is why the import joins the existing block.

#634 is on main, and this change does not edit `shown_source` or its docstring in `scripts/shape_materialize.py`.

The review approved with nits. Its S1, the stale base-run numbers and merge lines, is taken here, in this body: no code changed, and the head is still `ab55c29f`. Its N1, renaming the call sites to `shown_path`, is not taken (see "Decisions and their evidence"). It also re-ran 5 of the 7 mutation probes on `ab55c29f` and got the same failure counts as above (12, 2, 3, 2 and 2), and ran 144 hostile-name dry runs (16 names in 9 scenarios): on main 69 rows put a control character on stderr and 6 print a forged line of their own; on this head all 144 exit 2 with no traceback, no control character and no forged line.

- `scripts/family.py` `init`'s plan prints `folder` and `landing` lines with the path raw (`print(f"landing      {holder}   ...")`, `print(f"folder       {folder}   ...")`). On a dry run or a refused run the plan is on stdout before the refusal, so `WARNING forged-line: pwned/Orbit   (--work-dir: no family folder)` still appears there. It is the plan, not a refusal, and not one of the issue's lines.
- Other refusals that name a holder or folder path raw, none of them in the issue's lines: `family-holder-detached` and `family-on-tracking-branch` (`{family.root}`), `family-index-locked` (`lock.as_posix()` in the detail), `family-landing-in-standard` (`{holder}`, `{checkout}`, `{beside / family / family}`), `family-holder-already-here` (`{into}`), and `family-remote-exists` and `scaffold-remote-exists` (`{target}`, a local remote path).

1. **family.py `init`'s plan prints the holder and folder paths raw.** `scripts/family.py` `landing` and `folder` rows (about :1756-1759) print `holder` and `folder` as they are, so a forged line or an ESC reaches stdout in a `--dry-run` and in a run refused after the plan. Direction: spell them with `shown_source`.
2. **The same raw spelling in the refusals listed above.** `family-holder-detached`, `family-on-tracking-branch`, `family-index-locked`, `family-landing-in-standard`, `family-holder-already-here`, `family-remote-exists` and `scaffold-remote-exists` each print a path the person supplied or git keeps. Direction: the spelling this change gives its sites.

Review: rv692, lane 5's independent review, approved ab55c29 with nits, and its body fix (S1) is published. CI (Brett Heap's ruling 27 and his fast-refresh ruling of 2026-10-15T22:48:35Z): H0 ab55c29 passed tests, tests-git-bash and SonarCloud; the head 13f643d is a clean update-branch merge of main 431c8cf, with tests-git-bash and SonarCloud green; and the local gate passed on its predicted squash tree 4327947 over main 431c8cf: the dry_run_parity, family_refuse_up_front, repo_hygiene, agents_md_codes, cli_reference sets, 330 passed. Landed on Brett Heap's word of 2026-10-17T10:59:39Z, relayed by lane 3 on #473: "merge lane 5's approved v1.1.0 drafts".

Closes #692

Lane: openreposhape-5 (openRepoShape-5)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 17, 2026
…nd what they tell git to the `GIT_CONFIG_COUNT` entries the environment holds, so a pasted line keeps the session's maintenance settings and the Windows job's two (#758) (#818)

Against main. #748 landed as `431c8cf8`; this branch merged #748's final head and main (see Fix round 1). After that merge, #723 (`bad05d21`) and #746 (`fc867207`) landed. The branch is refreshed over main `fc867207` as `4a553e17`, one merge commit. Its only conflict was `tests/test_non_utf8_git_reads.py`'s `from conftest import` block, resolved by keeping both sides' names, as rv818 asked. The merged tree differs from main in #818's own 27 files, and #818's count scan flags no file. Tests only: 25 files under `tests/`, no shipped code, no pinned file, no README, handbook, `AGENTS.md` or `docs/cli.md` change.

Two new commits on `b6e15591`: the retarget merge `2908e46f`, then the fix commit `eaa72d9e`. Tests only, as before.

### The retarget: merge commit `2908e46f`
#748 landed as `431c8cf8` from its final head `8368d1f8`, and GitHub moved this PR's base to main. The merge commit has three parents: `b6e15591`, `8368d1f8` and main `fa04d2cd`. #748's head goes in first, so git sees the history the squash dropped.

- **#748's head did not merge clean**, as rv818 found. `tests/test_adopt_empty_symlinks.py` conflicts, because `8368d1f8` holds main `741679e7`, which carries #388.
  - I took w758's earlier resolution of that file, which rv818 checked. Main's side of the file is the same at `974d4827`, `8368d1f8` and `fa04d2cd`, so it still fits.
  - Both sides stay: #388's two targets (`with_each_target`, `SAID`, the `target` parameters) and #758's `symlinks_off()`, built at the call.
  - `test_the_link_given_a_target_adopts_end_to_end` calls `symlinks_off()` too. Git does not flag it, and left as it was the file fails at collection.
- **Main then conflicted in three files** (main was `d17564d9` when I resolved them): `tests/test_clone_just_pushed.py`, `tests/test_doctor_legs_members_next.py` and `tests/test_family_bump_remove_rollback.py`.
  - Main's copy of each was `8368d1f8`'s byte for byte.
  - This branch's copy is `8368d1f8`'s plus #758's own hunks, so this branch's side is taken.
  - Main's next two landings, #809 and #816, merged with no conflict.
- **rv818 MUST 1.** `tests/test_local_remote_dir_urls.py` came with #812 and imported `ALLOW_FILE_PROTOCOL` from `test_family`.
  - Without a change, the file fails at collection: `ImportError: cannot import name 'ALLOW_FILE_PROTOCOL' from 'test_family'`, and none of its 27 cases run.
  - It now imports `allow_file_protocol` and calls `pasted(lines, paste, allow_file_protocol())`.
  - `git grep` finds no other file on main that uses a name #758 removes (`SYMLINKS_OFF`, `SYMLINKS_ON`, `CONFIG_COUNT`, `CONFIG_VALUE`, `PASTE_ENV`, `ALLOW_FILE_PROTOCOL`). `NO_MAINTENANCE`, `QUOTE_PATH_OFF` and `tool_env` change shape, and each is used only in its own file.
- **Checked:**
  - Against main, the merged tree differs in #758's 25 files and in `tests/test_local_remote_dir_urls.py`.
  - Each of the 25 differs by exactly #758's own added and removed lines, `tests/test_adopt_empty_symlinks.py` aside.
  - Collecting the whole suite at the head finds 12,320 tests and no error.

### The fix commit `eaa72d9e`
- **SHOULD 2:** `configure` in `tests/test_adopt_reencoded_files.py`, the seventh append from #758's line 2, now sets what `appended_to_command_scope(os.environ, pairs)` returns. It gives the same entries in the same order, so no test can fail on the old form.
- **SHOULD 3:** the scan also reads a name the module binds to `"GIT_CONFIG_COUNT"` (`CONFIG_COUNT = "GIT_CONFIG_COUNT"`, annotated or not) as the count. It does so in a dict key, a setter's first argument and a subscript.
  - rv818's repro: append the base's two lines to `tests/test_adopt_root_symlinks.py`.
  - The old scan passes. The new one names line 2137.
- **SHOULD 4:** `counts_set_to_a_number` is split into one reader per node type: `in_a_dict`, `in_a_call` and `by_subscript`. The walk takes each node's reader from `READERS`. By hand, no function scores more than 5 on S3776. I could not run Sonar.
- **NIT 5:** `conftest.py` is no longer skipped. It scans clean, at the head and on today's main.
- **NIT 6:**
  - New "sees" cases: `os.putenv("GIT_CONFIG_COUNT", "1")` and a bare `setenv("GIT_CONFIG_COUNT", "1")`.
  - The walk is now `files_off_their_count(directory)`. A new test runs it over a folder of its own, and checks three things:
    - `conftest.py` and a helper module are read;
    - a `.txt` file is not;
    - a file `OWN_COUNTS` allows is named for fewer counts as well as for more.
  - Four of rv818's six surviving mutants now fail a case: `putenv` dropped, the bare-name setter dropped, `!=` changed to `>`, and `glob("test_*.py")`.
  - The fifth, `conftest.py` scanned, is now how the code works, and putting the skip back fails the new test.
  - The sixth, the count's value taking any string, still survives: it is a stricter scan. Git refuses a count that is not a number (`bogus count in GIT_CONFIG_COUNT`, exit 128), so a test that holds one fails on its own.
- **NIT 7, in part:**
  - An empty count (`""`) is now a count of its own. Git 2.43 and 2.47.1 read it as no entries at all. Measured: `GIT_CONFIG_COUNT= GIT_CONFIG_KEY_0=a.b GIT_CONFIG_VALUE_0=c git config --get a.b` exits 1.
  - The docstring says the scan reads the count's value as a constant only.
  - Left out:
    - a value built by a call (`str(len(pairs))`) or held in a variable;
    - an int;
    - `dict()`, or `environ={}` by keyword, handed to the helper;
    - `setenv` with keyword arguments;
    - an annotated subscript.

    rv818 judged only `str(len(entries))` likely, and telling it from a count worked out needs what the call means, not how it is spelled.
- **NIT 8:** the paste case deletes `GIT_CONFIG_PARAMETERS` first. Measured on the merge before the fix commit, with `GIT_CONFIG_PARAMETERS="'probe.param'='p'"`, the case failed (`Left contains one more item: 'probe.param=p'`). It passes now.
- **NIT 9, left as it is:** main's three appends built by hand each work the count out right: `tests/test_family_head_lost_gitmodules.py:97`, `tests/test_family_bootstrap_gate_order.py:135` and `tests/test_family_add_spellings_remedies.py:137`. Each of those files is also changed by another open PR (#828 `fix/761-add-move-back-gitmodules`, #820 `fix/762-validate-family-remedies-texts`, `fix/763-holder-docs-agree`), so converting them here would only add conflicts.
- **NIT 10:** the comment on `NO_MAINTENANCE` now says its `maintenance.auto` and `gc.auto` repeat the session's two, which reach these gits in the command scope. Only `gc.autoDetach` adds a setting. The two stay, so the constant alone still keeps maintenance off for these gits.
- **NIT 11:** the count was in lane 5's #473 summary, not in this body. The seven sets held 2,401 distinct passes, not 2,609: the 208-test run repeats three files of the 256-test run.

### Tests at `eaa72d9e`
All runs went through `run9.sh`, with the real `gh` unreachable. All passed, none skipped:
- **The 26 test files the PR changes against main**, in one full-file run each: 2,206 passed (1,108 + 1,098).
  - These are #758's 24 test files, `tests/test_local_remote_dir_urls.py` and `tests/test_adopt_reencoded_files.py`.
  - The count matches the collection exactly.
- **The 43 files that import any of those 26, with `tests/test_repo_hygiene.py`, `tests/test_agents_md_codes.py` and `tests/test_cli_reference.py`:** 1,646 passed (825 + 821).
- **The same merge made on main `d17564d9`** (a local commit, `c16a7e1a`, never pushed), before the fix commit: `tests/test_local_remote_dir_urls.py` and `tests/test_no_auto_maintenance.py`, 55 passed.

### Fails on the base
The new tests were run against the base's scan and walk, as `b6e15591` has them, with `GIT_CONFIG_PARAMETERS` set: 6 failed, 30 passed. The six:
- the paste case;
- the `nothing`, `a-name-bound-to-it`, `an-annotated-name` and `a-name-by-subscript` cases;
- the walk test.

The three other new cases (`a-bare-setenv`, `putenv`, `a-name-bound-to-another-key`) are there to kill mutants, and they pass on any correct scan. MUST 1 fails on main's form at collection, as above. SHOULD 2 and NIT 10 change no behaviour.

### Mutants
I made 26 mutants. 25 are caught, and one survives as rv818 said it would:
- **22 on the scan and the paste case**, each run against `tests/test_no_auto_maintenance.py`:
  - the delenv dropped;
  - `putenv` dropped from `SETTERS`, or the bare-name setter;
  - the name branch of `is_the_count` dropped, or any name taken as the count;
  - `""` no longer a count;
  - the `Assign` names dropped, or the `AnnAssign` names;
  - no names collected;
  - each reader ignoring the names;
  - each of the three readers dropped from `READERS`;
  - `conftest.py` skipped;
  - `!=` changed to `>` or to `<`;
  - `glob("test_*.py")`, or `glob("*")`;
  - `by_subscript` taking any value;
  - the walk pointed at an empty folder.

  The last passes on a clean tree, like any change to an assertion that nothing is found. It is caught by rv818's SHOULD 3 repro, where the real walk fails and the mutant passes.
- **The survivor:** the count's value taking any string. It makes the scan stricter (see NIT 6).
- **`configure`:** with `appended_to_command_scope({}, pairs)`, the scan names the file. With the pairs dropped, three of its tests fail.
- **MUST 1:** main's import put back fails collection with `ImportError`.

### Merge checks at `eaa72d9e`
I ran `git merge-tree --write-tree` against `origin/main` (`fa04d2cd`) and the 53 other `fix/*` and `l3-*` heads on origin:
- **Clean:** `origin/main` and 43 heads.
- **#792 (`l3-785-non-ascii-paths`)** merges clean, but `tests/test_doctor_non_ascii_paths.py` imports `PASTE_ENV`. Whichever PR lands second changes `:50` (the import), the `:266` docstring and `:268` to `paste_env()`.
- **#723 (`fix/663-non-utf8-name-tests`), #746 (`fix/672-bump-leg-branch-and-push-line`, which now holds #723) and #806 (`fix/757-non-utf8-test-dirs`)** conflict only in the `from conftest import` block of `tests/test_non_utf8_git_reads.py` (and of `tests/test_adopt_git_reads.py`, for #806). Both sides add names, so keep them all.
- **Seven heads conflict with main itself, not with this PR:**
  - `l3-732-latin1-section` and `l3-733-ignored-sections` (`tests/test_doctor_legs_members_next.py`);
  - `fix/712-validate-pins-contracts`, `fix/766-spelled-paths` and `fix/798-adopt-legs-asked-first`;
  - `l3-677-pins-hand-step-wip` and `l3-678-naming-checkout-wip`.
- I scanned each clean merged tree's changed test files for a name #758 removes, or a count the scan flags. Only #792 shows up.
- **Main moved after the push.** #723 landed as `bad05d21`, and the two now conflict in one place: the `from conftest import` block of `tests/test_non_utf8_git_reads.py`. The fix is to keep both sides' names: `ORG, PROJECT, REPO, appended_to_command_scope, cloned_just_pushed, git, run_script, skip_message, skip_where_fsencode_gives_a_surrogate_no_byte, windows_fsencode`. Every other change merges in. On that merged tree, with the block resolved that way, the scan flags no file, and no file uses a name #758 removes.

On `deb029c9` a probe test imported `PASTE_ENV` and pasted `git config --show-origin --list` through `paste()`:
- `PASTE_ENV` held `GIT_CONFIG_COUNT=1` (`protocol.file.allow`), a stale `GIT_CONFIG_KEY_1=user.email` left from collection, and no `GIT_CONFIG_GLOBAL`.
- `os.environ` at the call held a count of 4 and the session's global file.
- The pasted git listed one command-line entry, `protocol.file.allow=always`, and `git config --get maintenance.auto` exited 1. So a pasted line ran with git's automatic maintenance on, in both scopes, as the issue said.

`paste_env()` replaces it. It reads `os.environ` at each call (minus `PYTHONPATH`), adds the identity and `PYTHONDONTWRITEBYTECODE`, and appends `protocol.file.allow` with `appended_to_command_scope`. The six uses call it. The import-time copy also predated the session's `gh` guard on `PATH`, as `test_doctor_next_is_a_command.py`'s docstring says of its own environment; a pasted line now gets the `PATH` of the call. That docstring now says `paste_env()` reads at the call too.

- **The seven.** Six are on this base, each now one call to `appended_to_command_scope`: `test_adopt_git_reads.py`, `test_adopt_nested_attributes.py`, `test_adopt_ignored_shape_files.py`, `test_adopt_root_symlinks.py`, `test_bootstrap_after_bump.py`, `test_family_bump_remove_rollback.py`. The seventh, `test_adopt_reencoded_files.py`, came with #743 and is converted in fix round 1. `test_clone_just_pushed.py:96` stays, as the issue says: it is the helper's own test.
- **The other literals.** Each sets a count of its own and is laid over `os.environ`, where the session's two settings and the Windows job's two sit, so each replaced them (`NO_MAINTENANCE` put the two maintenance keys back and dropped the job's two). I read each; each is converted, none left:
  - `test_family.py`: `ALLOW_FILE_PROTOCOL` is now `allow_file_protocol()`; its two importers (`test_siblings_family_name.py`, `test_family_siblings_placement.py`) and the four uses in `test_family.py` call it.
  - `test_family_dry_run.py`: `NO_MAINTENANCE` is a tuple of entries, appended by `isolated()`.
  - `test_non_utf8_git_reads.py`: `QUOTE_PATH_OFF` is a tuple of entries, which `tool_env()` appends; `user_name()` appends too.
  - `test_adopt_empty_symlinks.py`: `SYMLINKS_OFF` is `symlinks_off()`.
  - `test_adopt_root_symlinks.py`: `SYMLINKS_OFF` and `SYMLINKS_ON` are gone; the four tests that set the first call `links_as_files` and the two that set the second call `told_git`, which already appended.
  - one-off environments: `test_scaffold_reuse_seeded_legs.py`, `test_adopt_split_branch.py`, `test_scaffold_free_text_values.py`, `test_adopt_plan_yaml_quoting.py`, `test_setup_project_py.py`, `test_setup_sh.py`, `test_doctor_next_is_a_command.py`.
  - The two tests that build a held environment on purpose to see it kept (`test_adopt_git_reads.py`, `test_clone_just_pushed.py`) now build it with `appended_to_command_scope({}, ...)`.

Measured with a probe on a throwaway bare remote after a push (18 loose objects): the hand-built options and `clone_just_pushed`'s give the same clone, one pack and no loose object, with `upload-pack` and `pack-objects` in `GIT_TRACE`, and `origin` keeps the plain path. A plain local clone of the same remote keeps all 18 loose objects. For a clone that is not recursive the helper's `insteadOf` rewrite is inert, because no submodule is cloned; the helper adds one place for the options, not a different read.
- `test_bump_leg_dry_run_truth.py:98`, the private bare copy of the session's spec remote: a session remote. `cloned_just_pushed(..., bare=True)`; the fixture now asserts the copy is bare.
- `test_bump_leg_dry_run_truth.py:110`, `private_work`, a clone of that private remote, which tests push tags into: `cloned_just_pushed(..., recurse=False)`.
- `test_doctor_legs_members_next.py:1828` and `:1929`, `Ghost`, a clone of the first member's checkout in the test's own holder copy: not a remote, and the hand-built `-c protocol.file.allow=always` was dead weight on a direct clone. Both call `clone_member`, one helper over `cloned_just_pushed(..., recurse=False)` that asserts the legs stay empty.
- `test_family_bump_to_checks.py:235`, a clone of `a_private_remote`, a bare repository filled by a push: `cloned_just_pushed(remote, work, "main", recurse=False, env=env)`.
- `test_doctor_pin_restore_merges.py:714` is **left as it is**: a `--filter=blob:none` clone of a file URI. The helper takes a path, and has no way to pass `--filter`.

- `test_family_dry_run.py`: the comment on `NO_MAINTENANCE` says what the entries are for, the gits `isolated()` hands an environment to, whose global file `excluding()` replaces, and that the session fixture covers the module's own setup commits, which pass no `env=`.
- `tests/conftest.py`: the two i645 replacements, "repacks once its estimate, taken from `objects/17` alone, is above 256 loose objects, so two loose objects there are enough", and the `objects changed` sentence. **#713's body holds the matching sentence and needs the same two replacements; I cannot edit a body.**

`tests/test_no_auto_maintenance.py` has 17 new cases (28 in the file, 11 before):
- `test_a_pasted_line_keeps_the_entries_the_environment_holds_at_the_call`: the environment holds two entries as the Windows job does, then the two settings as the session fixture appends them, set after the doctor tests were imported. `git config --show-origin --list` through `paste()` lists the four, in order, then `protocol.file.allow`, and reads both settings from the session's global file.
- `test_no_test_sets_a_git_config_count_of_its_own`: reads every `tests/*.py` but `conftest.py` with `ast` and fails a `GIT_CONFIG_COUNT` set to a number (dict, keyword, `setenv`/`setdefault`/`putenv`, subscript) and an `appended_to_command_scope` call handed a dict written out. Three files are allowed the ones that build a held environment on purpose, with a count each in `OWN_COUNTS`.
- 15 cases that hold the scan itself: seven spellings it must see, eight it must leave (`env.get("GIT_CONFIG_COUNT", "0")` is a read).

Runs at the head `b6e15591` through `run9.sh` (the real `gh` unreachable), all passed, none skipped:
- `test_no_auto_maintenance.py`, `test_doctor_legs_members_next.py`, `test_doctor_next_is_a_command.py`: 208 passed.
- `test_clone_just_pushed.py` and `test_doctor_validator_rows.py` (it imports the doctor file), with those three: 256 passed, run one commit earlier (the later commit only moved two blank lines and reworded one failure message).
- The sweep files: `test_adopt_git_reads.py`, `test_adopt_nested_attributes.py`, `test_adopt_root_symlinks.py`, `test_adopt_ignored_shape_files.py`, `test_bootstrap_after_bump.py`, `test_family_bump_remove_rollback.py`, `test_adopt_empty_symlinks.py`, `test_adopt_split_branch.py`: 445 passed.
- `test_non_utf8_git_reads.py`, `test_family.py`, `test_siblings_family_name.py`, `test_family_siblings_placement.py`, `test_scaffold_reuse_seeded_legs.py`, `test_scaffold_free_text_values.py`, `test_adopt_plan_yaml_quoting.py`: 999 passed.
- `test_family_dry_run.py`, the five files that import its `isolated` (`test_family_bump_to_a_tag.py`, `test_family_bump_to_checks.py`, `test_family_bump_to_local_only.py`, `test_family_refuse_up_front.py`, `test_family_remove_gitmodules.py`) and `test_bump_leg_dry_run_truth.py`: 216 passed.
- `test_setup_project_py.py`, `test_setup_sh.py` and their importers (`test_adopt_shape_pin_and_work_dir.py`, `test_dry_run_asks_remotes.py`, `test_clone_url_bytes.py`, `test_setup_by_hand_commands.py`): 313 passed.
- `test_repo_hygiene.py`, `test_agents_md_codes.py`, `test_cli_reference.py`, `test_doctor_pin_restore_merges.py`, `test_git_helper_identity.py`: 172 passed. `render-cli-reference.py --check` is ok.

**On the base (`deb029c9`)**, with the new `test_no_auto_maintenance.py` copied in: 2 of the 17 fail, the paste case and the scan (it finds 12 files). The other 15 hold the scan itself and pass on any tree.

Each of the new hunks was broken in a throwaway and run against its tests:
- `paste_env`: a literal count of 1, a snapshot taken at import, and `appended_to_command_scope({}, ...)`: each fails the paste case, and the first and third the scan too.
- The 18 sites that call `appended_to_command_scope(os.environ, ...)` (or `env`), each with `{}` in place of the environment: each fails the scan. Sites: the six of line 2, `allow_file_protocol`, `isolated`, `tool_env`, `user_name` (non-UTF-8), `symlinks_off`, `git_reads_github_from`, `github_at`, `user_name` (free text), `test_adopt_plan_yaml_quoting.py`, the two setup tests, `test_doctor_next_is_a_command.py`.
- The scan: eight changes (ignore dicts, keywords, `setenv` calls, subscripts, `appended_to_command_scope({}`; flag `os.environ`; flag a read; allow fewer in `OWN_COUNTS`) each fail a scan case.
- The clone sites: dropping `bare=True` from the private remote, `recurse=False` from `clone_member`, and `"main"` from the checks clone each fail. Two **survive as equivalents**: `recurse=False` in `private_work` (the spec leg has no `.gitmodules`, so a recursive clone of it is the same clone) and `env=env` in the checks clone (a clone reads no config the isolated environment changes).
- The two comments cannot be mutated.

`git merge-tree --write-tree` of this head against `origin/main` (`974d4827`), #748 (`deb029c9`), #713 (`979bb394`) and every `fix/*` and `l3-*` head on origin (46 refs):
- **Clean:** #748, #713, and 24 of the 46 heads: 21 `fix/*` heads (every lane-5 PR but #723 and #756, and most of lane 4's) and `l3-677`, `l3-678`, `l3-rehearsal-v1.1.0`.
- **`origin/main`: one file, `tests/test_adopt_empty_symlinks.py`** (four hunks), against #388's landing (`5c990708`). The parent #748 merges clean, so this arrives with the retarget. Resolve by keeping main's lines and my `symlinks_off()`. Two more spots need the same change and git does not flag them:
  - `tests/test_adopt_empty_symlinks.py::test_the_link_given_a_target_adopts_end_to_end` uses `SYMLINKS_OFF`, which this change removes: give it the `off` parametrization.
  - `tests/test_adopt_reencoded_files.py::configure`, the seventh hand-built append, goes to `appended_to_command_scope(os.environ, pairs)`.
  - `tests/test_family_head_lost_gitmodules.py::on_disk` (on main) is one more hand-built append: `{**env, **appended_to_command_scope(env, [("core.ignorecase", ignorecase)])}`. The scan does not read for hand-built appends, so it is not required, but it is the same thing.

  I made that merge in a scratch tree (main at `bf1f099b`), resolved it as above, and ran the files this change touches that main also changed, with `test_adopt_empty_symlinks.py`, `test_adopt_reencoded_files.py`, `test_family_head_lost_gitmodules.py`, `test_no_auto_maintenance.py` and the files that import `isolated` or the doctor helpers: 653 and 245 passed, in two runs. The scan finds no other file of main's with a count set by hand.
- **#723 (`fix/663-non-utf8-name-tests`):** `tests/test_non_utf8_git_reads.py`, the `from conftest import` block, where both add names.
- **`l3-732`, `l3-733`:** `tests/test_doctor_legs_members_next.py` (the `from conftest import` block) and `tests/test_adopt_empty_symlinks.py`.
- **19 more** (`fix/664`, `fix/729` (#756) and 17 `l3-*` heads): `tests/test_adopt_empty_symlinks.py` only, because they carry main's #388 as main does.
- **Not a textual conflict, but it breaks on merge:** `l3-785-non-ascii-paths` imports `PASTE_ENV` (`tests/test_doctor_non_ascii_paths.py:47` and `:223`); it must call `paste_env()`. #812 (`fix/693-local-remote-dir`) landed first, with `tests/test_local_remote_dir_urls.py` importing `ALLOW_FILE_PROTOCOL` (`:51`, `:349`). This PR's retarget merge makes it call `allow_file_protocol()` (rv818 MUST 1).

- A pasted `git submodule update --init` still clones a local path: `paste()` takes no remotes, so no `insteadOf` rewrite reaches it (rv662's "build it from `read_through_git`"). It runs under the session's maintenance settings now, which is what the issue asks.
- Three more appends built by hand beside `appended_to_command_scope`, in files this base does not hold: `tests/test_family_head_lost_gitmodules.py:97` (main), `tests/test_family_bootstrap_gate_order.py:135` (#682), `tests/test_family_add_spellings_remedies.py:137` (#767). The scan reads for counts set to a number, not for appends worked out by hand.

Review: lane 2's independent review (issuecomment-6236874291) and its re-verification of eaa72d9, READY (issuecomment-6238781272); its new SHOULD is taken in #848, and 4a553e1 merges main over eaa72d9. CI (Brett Heap's ruling 27): H0 is the head 4a553e1 itself, which passed tests, tests-git-bash and SonarCloud; main has touched none of its files since its base fc86720; and the local gate passed on its predicted squash tree 9d6c099 over main d8b8eb3: the adopt_empty_symlinks, adopt_git_reads, adopt_ignored_shape_files, adopt_nested_attributes, adopt_plan_yaml_quoting, adopt_reencoded_files, adopt_root_symlinks, adopt_split_branch, bootstrap_after_bump, bump_leg_dry_run_truth, clone_just_pushed, doctor_legs_members_next, doctor_next_is_a_command, family, family_bump_remove_rollback, family_bump_to_checks, family_dry_run, family_siblings_placement, local_remote_dir_urls, no_auto_maintenance, non_utf8_git_reads, scaffold_free_text_values, scaffold_reuse_seeded_legs, setup_project_py, setup_sh, siblings_family_name, repo_hygiene, agents_md_codes, cli_reference sets, 2359 passed. Landed on Brett Heap's word of 2026-10-17T21:47:28Z, in lane 5's session (posted on #473 as issuecomment-6241758069): "merge lane 5's v1.1.1 PRs once reviewed and green".

Closes #758

Lane: openreposhape-5 (openRepoShape-5)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant