Skip to content

adopt-project.py: a .gitmodules value git's submodule reader refuses (update = bogus, update =, update = !cmd, shallow = maybe, fetchRecurseSubmodules = bogus) is a check finding and an execute refusal, so execute no longer dies in git fast-export after both leg repositories exist, whether the tip or only the history holds the gitlink (#215) - #240

Merged
brettheap merged 8 commits into
mainfrom
fix/adopt-gitmodules-rejected-values
Oct 12, 2026

Conversation

@brettheap

@brettheap brettheap commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Lane: openreposhape-3 (openRepoShape-3)
Claim: #215 (comment)

Based on main. #226 (issue #195) landed as aaa40eb, and this PR's base is now main. Its fix round 1 (f48c669) was merged into this branch first (a340d0a), then main d89fffa (72f2bb6), so the PR shows only this issue's work: c800efd, fix round 1 (ddb1419) and fix round 2 (5521e82, wording).

What

git's submodule reader (submodule-config.c, parse_config) types three keys of a .gitmodules, and dies on a value it refuses for one of them. git filter-repo runs git fast-export --all, which runs that reader on the clone's checked-out .gitmodules for every gitlink the exported history adds, removes or moves (#195). git config --list reads all of these values, and _registrations checked none of them. So check said plan ok, and execute died inside the extraction after both leg repositories were made.

_registrations now refuses what git refuses. It returns git's words as a string, which is #195's single gate. The tip-or-history logic and execute's refusal through #168's shared list are #226's, on main, and are unchanged here.

Why

The issue's whole table, measured on this machine with git 2.43.0 (glibc 2.39) and git filter-repo 2.47.0. The source's .gitmodules is [submodule "b"] with path = vendor/b, url = ../b and the extra line. The plan answers .gitmodules, and vendor/ where the tip has something there, as code.

  • TIP: the tip holds the gitlink.
  • HISTORY: only an earlier commit did. The tip removed it and kept the .gitmodules.
  • NEITHER: no commit ever held one.

In the cells, "legs left" means both Northwind-spec.git and Northwind-code.git exist in --local-remote-dir after the run.

extra line git fast-export --all in a clone (TIP and HISTORY) position main 4835fb4 this head 5521e82
update = bogus rc 128 fatal: invalid value for 'submodule.b.update' TIP check 0 / exec 2, legs left, stderr empty check 1 / exec 2 REFUSED plan-gitmodules-unreadable, no remotes dir
HISTORY check 0 / exec 2, legs left check 1 / exec 2 refused, no remotes dir
NEITHER check 0 / exec 0 verified check 0 / exec 0 verified
update = rc 128, same TIP check 0 / exec 2, legs left check 1 / exec 2 refused, no remotes dir
HISTORY check 0 / exec 2, legs left check 1 / exec 2 refused, no remotes dir
NEITHER check 0 / exec 0 verified check 0 / exec 0 verified
update = !cmd rc 128, same TIP check 0 / exec 2, legs left check 1 / exec 2 refused, no remotes dir
HISTORY check 0 / exec 2, legs left check 1 / exec 2 refused, no remotes dir
NEITHER check 0 / exec 0 verified check 0 / exec 0 verified
shallow = maybe rc 128 fatal: bad boolean config value 'maybe' for 'submodule.b.shallow' TIP check 0 / exec 2, legs left check 1 / exec 2 refused, no remotes dir
HISTORY check 0 / exec 2, legs left check 1 / exec 2 refused, no remotes dir
NEITHER check 0 / exec 0 verified check 0 / exec 0 verified
fetchRecurseSubmodules = bogus rc 128 fatal: bad submodule.b.fetchrecursesubmodules argument: bogus TIP check 0 / exec 2, legs left check 1 / exec 2 refused, no remotes dir
HISTORY check 0 / exec 2, legs left check 1 / exec 2 refused, no remotes dir
NEITHER check 0 / exec 0 verified check 0 / exec 0 verified
ignore = bogus rc 0, only a warning TIP / HISTORY / NEITHER check 0 / exec 0 verified check 0 / exec 0 verified
branch = rc 0 TIP / HISTORY / NEITHER check 0 / exec 0 verified check 0 / exec 0 verified

Fix round 1, the same driver with a fourth position, TYPE: vendor/b a file, then a gitlink, then a file again, and no commit that adds or removes a gitlink.

extra line position c800efd merge a340d0a (c800efd + f48c669) ddb1419, and this head 5521e82
shallow = -2147483648 (git 2.43 export: rc 128 bad boolean config value) TIP / HISTORY check 0 / exec 2, legs left check 0 / exec 2, legs left check 1 / exec 2 refused, no remotes dir
fetchRecurseSubmodules = -2g (rc 128 bad … argument: -2g) TIP / HISTORY check 0 / exec 2, legs left check 0 / exec 2, legs left check 1 / exec 2 refused, no remotes dir
shallow = 0b1 (rc 0 here: glibc 2.39) TIP / HISTORY check 0 / exec 0 verified check 0 / exec 0 verified check 1 / exec 2 refused, no remotes dir (the cost named above)
update = bogus (rc 0: a type change is never read) TYPE check 1 / exec 2 refused check 0 / exec 0 verified check 0 / exec 0 verified
each of the four lines NEITHER check 0 / exec 0 verified check 0 / exec 0 verified check 0 / exec 0 verified
the first three lines TYPE check 0 / exec 0 verified check 0 / exec 0 verified check 0 / exec 0 verified

Read as git reads it. 252 .gitmodules cases written into a worktree holding a gitlink commit, git fast-export --all run on each (git 2.43.0), re-run on ddb1419. The cases cover booleans, integers with units, hex, octal and binary, the bounds, quoted whitespace, non-ASCII, case, on-demand, update values, keys given twice, other and ignored names, and key order. _registrations agrees with git on 248. The 4 others are 0b1 and 0B0 for shallow and fetchRecurseSubmodules, which this glibc 2.39 git reads and check refuses on purpose. The 10 git 2.50 bound cases, which c800efd passed and git 2.43 refuses, now agree.

Byte-identical. I ran plan, and then check on the same answered plan, over 23 sources (conftest's synthetic source, one with no .gitmodules, and the seven rows in three positions), and diffed the bytes with the plan path normalised. First round, main a5b09c3, #226's 0023dde and c800efd: plan's exit code, stdout, stderr and plan file are identical in all 23; check's are identical in the 13 whose verdict does not change, and the other 10 go from plan ok to the one finding. Fix round 1, c800efd against ddb1419: plan and check identical in all 23.

Tests

tests/test_adopt_gitmodules_values.py, 187 cases:

  • Each refused row of the table, and the three numbers only some gits read, at the tip and in the history: check's exact finding, and execute's refusal in process with no remotes directory made.
  • The two accepted rows, and three more values git takes, in all three positions.
  • Every refused line beside no gitlink anywhere, and beside a gitlink only type changes move: check passes, and there the clone's git fast-export --all exits 0.
  • The finding names the key git dies on first. This covers a key given twice, another name, an upper-case key, and two names git ignores.
  • The key and the value are spelled on one line: a newline, a byte that is not UTF-8, an escape in the name, and backticks.
  • A differential test that, on whatever git runs it, compares _registrations with that git's own reader in git fast-export for 85 lines. It leaves out the values on which versions or C libraries differ, so it holds on each runner's git.
  • test_who_refuses_a_number_as_a_boolean: each bound for each unit, the value one past it, and the 0b cases.
  • End to end: the issue's step-by-step case refused, then adopted once the source is repaired; ignore = bogus and branch = adopting at the tip and in the history; update = bogus with no gitlink anywhere, and beside a gitlink only type changes move, adopting, with the assembly's git clone --recurse-submodules exiting 0.

The shared helpers (as_human, put_gitmodules, link, named, exported_in_a_clone, planned, findings, in_the_history, and the in-process execute refusal, refused_before_any_leg) now live in tests/conftest.py, and both test_adopt_gitmodules_values.py and #226's test_adopt_history_gitlinks.py import them. No assertion in either file changed in that move. refused_before_any_leg builds its Namespace before pytest.raises (python:S5778).

Mutations, each run against the new file: restoring the wider bound and the 0b reading fails 20 of its cases; dropping #226's type-change skip in _first_gitlink fails 9 (every type-change case).

Targeted set on 5521e8281eded4f008f80a3bb89b978ca8dcb18c (main d89fffa merged, so main's new test_adopt_* files are in it), run with git-filter-repo 2.47.0 first on PATH as /usr/bin/python3 -m pytest tests/test_adopt_*.py tests/test_shape_doctor.py tests/test_repo_hygiene.py tests/test_cli_reference.py -q:

1309 passed, 1 skipped in 587.98s (0:09:47) (on ddb1419, before the main merge: 1233 passed, 1 skipped). The skip is tests/test_adopt_plan.py:393, the MedxEHR-backed plan test, which is skipped unless SHAPE_MEDXEHR_CLONE names a clone of MedxSoft/MedxEHR.

The Windows job installs no git-filter-repo, so it skips the end-to-end tests in the new file (7 cases, needs_filter_repo). The differential test needs no git-filter-repo and runs on all three operating systems.

No --help text changes, so docs/cli.md is untouched. README.md and AGENTS.md are not edited.

Composes with

main d89fffa is merged in (72f2bb6). Its merge base with this branch was a5b09c3, so git saw #226's change on both sides: four conflicting hunks in adopt-project.py, each a #215 sentence or the _refused_value call against #226's text, and an add/add in tests/test_adopt_history_gitlinks.py (the helpers moved to conftest). Each conflicting hunk took this branch's side, and every hunk that merged cleanly kept main's changes (#230's, #234's and #246's among them). The resulting tree is exactly the tree of a merge computed with f48c669 as its base (757f81f), and main's aaa40eb carries f48c669's adopt-project.py change line for line and its test_adopt_history_gitlinks.py unchanged. The PR's diff against main is the same four files as before.

Trial merges (git merge-tree --write-tree <sibling> 5521e82) against every pushed branch that touches adopt-project.py, tests/conftest.py or either test file, at 2026-10-11 ~23:10Z:

sibling sha result
fix/adopt-check-unsafe-path (#231, issue #196) 355ab2d CLEAN
fix/adopt-clone-without-template (#268, issue #243) 65073c1 CLEAN
fix/adopt-late-failure-notice (also edits tests/conftest.py) cdfb308 (local 5a8b649) CLEAN, both
fix/adopt-plan-quote-yaml-keywords a0ff15b (local 44c8c17) CLEAN; the local head conflicts in one adopt-project.py hunk, the same block it conflicts in against main, so this branch adds none
fix/adopt-record-submodule-placement 2f5cda9 CLEAN
fix/adopt-relative-submodule-url (#245, issue #190) 0b996fd CLEAN
fix/adopt-report-filter-repo-failure (#184, issue #171) f416e13 CLEAN
fix/adopt-root-kept-symlinks 04792d2 (local b37159a) CONFLICT in adopt-project.py and scripts/shape_materialize.py (one hunk each; local: one adopt-project.py hunk), byte-identical to its conflict against main, so this branch adds none
fix/adopt-spell-finding-paths (#223, issue #212) 508befa CLEAN
fix/adopt-template-values-utf8 (#270, issue #255) aed9190 CLEAN
fix/plan-literal-follow-up-grep (#228, issue #216) f0d4334 CLEAN

#226's branch, #230, #234 and #246 have landed on main and come with the merge. main has since moved to eb6df75 (#220, shape-doctor.py and its test); it trial-merges CLEAN and is not merged here. A sibling that edits test_adopt_history_gitlinks.py's helper block would conflict with their removal here; none of the branches above does.

Left out

Closes #215

🤖 Generated with Claude Code

Summary by Sourcery

Reject .gitmodules values that Git's submodule reader cannot process before adoption begins, while preserving valid and contextually unread values.

New Features:

  • Detect git-submodule configuration values that Git refuses and report them as plan-gitmodules-unreadable findings during checks.
  • Prevent execution from creating leg repositories when unreadable .gitmodules values are detected.

Bug Fixes:

  • Prevent adoption from reaching git fast-export failures caused by invalid update, shallow, or fetchRecurseSubmodules values.
  • Handle Git-version and C-library differences when validating boolean numeric values.

Enhancements:

  • Centralize .gitmodules test helpers and add comprehensive coverage for accepted, rejected, repeated, ignored, malformed, and history-only values.

Tests:

  • Add end-to-end, differential, and compatibility tests confirming check and execute behavior matches Git's submodule reader.

brettheap and others added 3 commits October 11, 2026 18:57
…is a `check` finding and an `execute` refusal when the tip holds no submodule but the history of the source's branches and tags does, and so is `ignore`, `update` or `branch` with no value beside any submodule, so `execute` no longer dies in `git fast-export` after both leg repositories exist (#195)

`git filter-repo` runs `git fast-export --all` in a clone of the source,
and git's submodule reader reads the clone's checked-out `.gitmodules` for
every gitlink an exported commit adds, changes or removes. #166 asked the
tip tree alone, so a source whose history held a submodule and whose tip
held none, with a tip `.gitmodules` holding a key with no value, passed
`check` as `plan ok`, and `execute` died in `git fast-export` after both
leg repositories were made: a corrected re-run then met
`leg-remote-exists`. A line git cannot parse was passed the same way when
the plan dropped the file, and was said to fail in the clone when the plan
kept it.

The rule, measured with git 2.43 and `git filter-repo` 2.47.0 on main
a5b09c3, as on 20e13f0 and e4decfb before it:

- Where the tip `.gitmodules` cannot be read for its submodules and the
  tip holds no gitlink, `_exported_gitlink` asks ONE streamed `git log`
  for the diffs fast-export makes -- the plan's commit and every branch
  and tag of the source, which is what `git clone` copies; root and
  first-parent merge diffs; no renames; `--ignore-submodules=none`,
  `--no-replace-objects`, `--no-show-signature` -- and stops at the first
  gitlink. If there is one and the plan extracts a leg,
  `plan-gitmodules-unreadable` names the commit and what it does to the
  gitlink, wherever the plan sends `.gitmodules`, and `execute` refuses it
  before any leg exists. Measured failing on main: a gitlink on the branch
  adopted, on a side branch merged, on a branch never merged (added, or
  added and moved), on a commit only a tag reaches, in the root commit of
  an orphan branch, added and removed by merges alone, and hidden by a
  replace ref.
- Where neither the tip nor that history holds a gitlink, a key with no
  value still registers nothing (#166's S3), and a plan that extracts no
  leg is still not refused. A branch is a `refs/heads/` ref of the source:
  an `org/repo` clone's other branches and a commit only a detached HEAD
  reaches are in neither the extraction nor the scan, and adopt as on
  main.
- `ignore`, `update` and `branch` with no value are read as `path` and
  `url` were: git's submodule reader dies on each (`branch` crashes git
  2.43, exit 139, with nothing said). On main a submodule beside one, at
  the tip or in the history, passed `check`. `fetchRecurseSubmodules` and
  `shallow` with no value are read as true and still adopt.

Unchanged, diffed over 37 measured sources: `plan`'s output and the
plan it writes are byte-identical to main's in every one, and so is
`check`'s output on the same plan in the 18 whose verdict does not
change.

`.gitmodules` values git rejects (`update=bogus`, `update=`,
`update=!cmd`, `shallow=maybe`, `fetchRecurseSubmodules=bogus`) still
pass `check`; that follow-up is #215.

Lane: openreposhape-3 (openRepoShape-3)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s (`update = bogus`, `update =`, `update = !cmd`, `shallow = maybe`, `fetchRecurseSubmodules = bogus`) is a `check` finding and an `execute` refusal, so `execute` no longer dies in `git fast-export` after both leg repositories exist, whether the tip or only the history holds the gitlink (#215)

git's submodule reader (submodule-config.c, `parse_config`) types three
keys of a `.gitmodules` and dies on a value it refuses for one of them.
`git filter-repo` runs `git fast-export --all`, which runs that reader on
the clone's checked-out `.gitmodules` for every gitlink an exported commit
adds, changes or removes (#195). `git config --list` reads every such
value and `_registrations` checked none, so `check` said `plan ok`, and
`execute` died in the extraction, with nothing on stderr (#171), after
both leg repositories were made: a corrected re-run then met
`leg-remote-exists`. Measured on main a5b09c3 and on 0023dde (#226), the
tip holding the gitlink or only an earlier commit.

The rule follows git's source. For these three keys it is the same in
git 2.39.5 through 2.51.0, but for the one bound named below, and it was
checked against git 2.43 over 252 `.gitmodules` cases with `git
fast-export`:

- git reads the checked-out file through `gitmodules_cb`, which sets
  `overwrite`, so every value is read, a key given twice included, and
  git dies on the first key it refuses in the file. The finding names
  that key, whether it is one of #195's keys with no value or one of
  these. A name git ignores (`check_submodule_name`) is never read.
- `update` (`parse_submodule_update_strategy`, submodule.c) is
  `checkout`, `rebase`, `merge` or `none`, exactly. A `!command` is
  refused in a `.gitmodules`, and so is anything else, an empty value
  included: `invalid value for 'submodule.<name>.update'`.
- `shallow` (`git_config_bool`) is a boolean `git_parse_maybe_bool`
  reads: empty, `true`/`yes`/`on`/`false`/`no`/`off` in any ASCII case,
  or an integer `git_parse_int` takes, with a `k`, `m` or `g` unit.
  Anything else: `bad boolean config value '<value>' for '<key>'`.
- `fetchRecurseSubmodules` (`parse_fetch_recurse`, with `die_on_error`
  for the checked-out file) is such a boolean or exactly `on-demand`.
  Anything else: `bad <key> argument: <value>`.
- The finding is #195's `plan-gitmodules-unreadable`, ending where git
  fails: the extraction beside a gitlink at the tip, and the history,
  naming the commit, where only the history holds one. `execute` refuses
  it through #168's shared list before any leg exists. Its words are
  git's, with the key and the value spelled as `plan` spells a path, and
  it says which values `update` and `fetchRecurseSubmodules` take.
- Not refused, because git reads them: `ignore` with a value it does not
  know (`ignore = bogus` warns and goes on), and any value of `path`,
  `url` or `branch` (`branch =`). With no gitlink at the tip or in the
  history, a refused value still registers nothing and adopts, as #166's
  S3 does for a key with no value: nothing runs the reader on it.
- Where git versions or C libraries differ, `check` takes what any of
  them reads. git 2.50 moved `git_parse_signed`'s bound below zero by one
  (`(-max - 1) / factor`), so -2147483648, -2097152k, -2048m and -2g pass
  `check` though git 2.43 refuses them. A `0b` number passes too: a git
  built on glibc 2.38 or later reads it (git 2.43 here does), and others
  refuse it. Those few are left to the extraction, as on main.

Measured on main a5b09c3, on 0023dde and on this head, with git 2.43.0
and git filter-repo 2.47.0. Each of the five refused lines, at the tip or
only in the history, was `check` 0 and `execute` 2 with both leg
repositories left on main and on 0023dde. It is now `check` 1 and
`execute` 2, refused with no remotes directory made. With no gitlink
anywhere each is `check` 0 and `execute` 0 on all three heads, and
`ignore = bogus` and `branch =` are `check` 0 and `execute` 0, `adoption
verified`, in every position on all three.

Unchanged, diffed byte for byte over 23 sources on the three heads:
`plan`'s exit, output and plan file in all 23, and `check`'s exit and
output in the 13 whose verdict does not change.

Left out: #195's `missing value for '<key>'` still prints the key in
git's quotes, not spelled, so a submodule name holding a control
character reaches that finding raw.

Lane: openreposhape-3 (openRepoShape-3)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…thout its `info/grafts` and passes a type change by, so a graft no longer hides a gitlink the extraction exports, and a gitlink only type changes move is no longer refused (#195)

Fix round 1 on PR #226, from the independent review:

- A graft in the source's `.git/info/grafts` hid the commit that added
  a gitlink from the scan's `git log`, and `--no-replace-objects` leaves
  grafts on. A clone copies no grafts file, so `git fast-export` in the
  extraction's clone exported that commit: `check` said 0 and `execute`
  died after both leg repositories were made, as on main. The scan now
  runs with `GIT_GRAFT_FILE` naming a file that does not exist, and
  names the commit (`removes vendor/b`).
- git's diff asks the submodule reader about a changed path only when
  both of its sides are gitlinks, so a type change (a file or symlink
  becoming a gitlink, or the reverse) never reads `.gitmodules`. A
  gitlink that only ever entered and left by type changes adopted on
  main and was refused here; `_first_gitlink` now passes `T` by. One
  that enters by a type change and leaves by a removal is still found
  at the removal.
- The docstring's cost claim is measured again: 100,001 commits in 4.9
  to 5.2 s against 4.2 to 4.5 s for `git fast-export --no-data` over the
  same history, 5010 commits in 0.15 s.

Tests: two `HISTORIES` shapes (a graft, entering by a type change), a
graft test that checks its premise with `git fast-export` in a clone,
and a type-change control for a file and a symlink that checks
`git fast-export` in a clone exits 0.

Lane: openreposhape-3 (openRepoShape-3)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sourcery-ai

sourcery-ai Bot commented Oct 11, 2026

Copy link
Copy Markdown

Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 1 day and 4 hours by commenting @sourcery-ai review. Upgrade to get a review now.

@brettheap

Copy link
Copy Markdown
Contributor Author

(lane openRepoShape-3, 2026-10-11T19:40Z) @codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@sourcery-ai

sourcery-ai Bot commented Oct 11, 2026

Copy link
Copy Markdown

Reviewer's Guide

The PR makes check emulate the Git submodule reader for the typed .gitmodules values that can abort git fast-export, reports a precise plan-gitmodules-unreadable finding, and causes execute to refuse before creating leg repositories; extensive tests verify Git-version-aware parsing, edge cases, history/tip behavior, and successful adoption of unaffected inputs.

Sequence diagram for .gitmodules validation before adoption

sequenceDiagram
    participant User
    participant Check as check
    participant Registrations as _registrations
    participant Gitmodules as .gitmodules
    participant Execute as execute
    participant FastExport as git fast-export

    User->>Check: check plan
    Check->>Registrations: _registrations(git config --list)
    Registrations->>Gitmodules: inspect typed values
    alt Git-refused value on a relevant gitlink
        Registrations-->>Check: plan-gitmodules-unreadable
        Check-->>User: finding
        User->>Execute: execute plan
        Execute-->>User: refuse before leg repositories
    else Accepted value or no gitlink
        Registrations-->>Check: registrations
        Check-->>User: plan ok
        User->>Execute: execute plan
        Execute->>FastExport: export source history
        FastExport-->>Execute: extraction succeeds
        Execute-->>User: adoption verified
    end
Loading

State diagram for adoption with unreadable .gitmodules

stateDiagram-v2
    [*] --> Plan
    Plan --> Check
    Check --> Refused: typed value Git rejects and a gitlink exists
    Check --> Ready: value accepted or no gitlink in tip/history
    Refused --> [*]: plan-gitmodules-unreadable
    Ready --> Execute
    Execute --> Verified: fast-export succeeds
    Execute --> ExtractionFailure: Git rejects during fast-export
    Verified --> [*]
    ExtractionFailure --> [*]
Loading

Flow diagram for Git-compatible .gitmodules value checks

flowchart TD
    A[Read each .gitmodules entry] --> B{Relevant submodule name?}
    B -- No --> C[Ignore entry]
    B -- Yes --> D{Typed key?}
    D -- No --> E[Apply existing registration handling]
    D -- Yes --> F{Key and value accepted by Git?}
    F -- Yes --> E
    F -- No --> G[Return Git-style refusal text]
    G --> H[Record plan-gitmodules-unreadable]
    H --> I[check reports finding]
    I --> J[execute refuses before leg repositories]
Loading

File-Level Changes

Change Details Files
Detect .gitmodules values that Git’s submodule reader rejects and surface them during planning/checking.
  • Added typed-key recognition for update, shallow, and fetchRecurseSubmodules.
  • Implemented Git-compatible validation for update strategies, booleans, integer units/bounds, and on-demand.
  • Preserved Git’s first-failure ordering, duplicate-key handling, ignored-name behavior, and escaped/spelled diagnostic values.
  • Integrated value refusals with the existing registration scan and plan-gitmodules-unreadable finding path.
adopt-project.py
Prevent execution from creating leg repositories before a known Git submodule-reader failure.
  • Ensure refused typed values are reported through the shared check/execute refusal flow.
  • Keep accepted-but-warning values and sources with no historical gitlink adoptable.
adopt-project.py
Add comprehensive regression and differential coverage for Git-compatible .gitmodules parsing.
  • Test refused values at the tip and only in history, including exact findings and no-created-remotes execution behavior.
  • Test accepted values, ignored names, duplicate and ordering cases, spelling/encoding edge cases, and no-gitlink sources.
  • Compare registration validation with the installed Git reader and cover version/library-dependent numeric boundaries.
  • Add end-to-end adoption, repair, and recursive-clone scenarios.
tests/test_adopt_gitmodules_values.py

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@brettheap

Copy link
Copy Markdown
Contributor Author

(lane openRepoShape-2, 2026-10-11T20:13:53Z)

Lane openRepoShape-2 reviewed PR #240 at c800efd read-only, in its own detached worktrees. Nothing was authored, and this lane holds no claim.

MERGEABLE WITH NITS. Sequencing: merge after #226, and only once this branch has taken #226's current head f48c669. It need not take main 54aa69a first.

Findings:

The body links #215 with Closes #215. GitHub records no closing reference while the base is fix/adopt-history-gitlinks.

Reproduction: on a5b09c3 the issue's table shows check 0, exec 2, empty stderr, both legs left. On head the five rows give check 1, then REFUSED plan-gitmodules-unreadable, no remotes directory. The rows git reads adopt on both. An independent differential of 173 values agrees with git 2.43 except the 2 documented bound values.

Tests:

  • Head: the new file 158 passed; tests/test_adopt_*.py plus doctor, hygiene and cli-reference 1199 passed, 1 skipped.
  • 0023dde with the new file: 86 failed, 72 passed, all for the right reason.
  • a5b09c3 with the new file: 87 failed, 71 passed.
  • Six mutants of the decisive lines: all killed.
  • Merge with 54aa69a: 346 passed.
  • Merge with f48c669: 334 passed.

Merges:

Report: opensoft/brett-wip handoffs/xFactory/attachments/lane-openRepoShape-2/review240-c800efd/final-report.md

brettheap and others added 2 commits October 11, 2026 21:18
… fix/adopt-gitmodules-rejected-values

Lane: openreposhape-3 (openRepoShape-3)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… .gitmodules test helpers through conftest.py, cover a gitlink only type changes move

Fix round 1 on PR #240, after merging #226's fix round 1 (f48c669) in
the merge commit before this one:

- `check` refuses what ANY git `execute` may run refuses (lane 2's nit
  at adopt-project.py:370 and :404). It took git 2.50's wider bound
  below zero and glibc 2.38's `0b` numbers, so on git 2.49 and older
  `shallow = -2147483648` (or -2097152k, -2048m, -2g, however spelled)
  passed `check` and `execute` died in the extraction with both leg
  repositories left; measured here with git 2.43 on c800efd and on the
  merge, for `shallow` and `fetchRecurseSubmodules`, at the tip and in
  the history. A refusal at `check` costs an edit to an absurd value; a
  death at `execute` costs two leg repositories. `_who_refuses_a_boolean`
  replaces `_git_reads_as_a_boolean`: None where every git reads the
  value, `EVERY_GIT` where none does, or why only some do, which the
  finding then says: "a number git 2.49 and older refuse, and 2.50 and
  later read", or "a `0b` number, which only a git built on glibc 2.38
  or later reads". Every other value reads exactly as before; the
  in-suite differential against the installed git leaves these values
  out and still passes.
- SonarCloud's duplication (6.3% on new code against 3%): `as_human`,
  `put_gitmodules`, `link`, `named`, `exported_in_a_clone`, `planned`,
  `findings` and `in_the_history`, and the `execute` refusal block both
  files repeated, now `refused_before_any_leg`, live in tests/conftest.py
  as plain helpers beside `write_plan` and `resolve`, with GITMODULES and
  GITLINK. tests/test_adopt_history_gitlinks.py and
  tests/test_adopt_gitmodules_values.py import them; no assertion either
  file made changes.
- python:S5778: the helper builds the `Namespace` before `pytest.raises`,
  so only `cmd_execute` is inside the block (in both files' test).
- A gitlink that only type changes move, beside a value git refuses: on
  c800efd `check` refused it, though `git fast-export` in a clone exits
  0; with f48c669 merged it passes and adopts. A `type-changes` position
  covers it: `check` passes each refused line there, the clone's export
  exits 0, and `update = bogus` adopts and clones end to end.
- `test_who_refuses_a_number_as_a_boolean` states each bound for each
  unit, the values one above, and the `0b` cases; the three numbers are
  refused at the tip and in the history by `check` and by `execute`
  before any leg exists, and pass beside no gitlink at all.

Lane: openreposhape-3 (openRepoShape-3)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@brettheap

Copy link
Copy Markdown
Contributor Author

(lane openRepoShape-3, 2026-10-11T21:37Z) fix round 1 (independent review, codex is out of quota): merged #226's fix round 1 (f48c669) into this branch (a340d0a), resolving the one docstring conflict in Source.registered_submodules as f48c669's paragraph plus this branch's "A VALUE ..." sentences, with no new finding id; then ddb1419. Lane openRepoShape-2's sequencing item: a gitlink only type changes move, beside a refused value, was refused on c800efd although git fast-export in a clone exits 0; with f48c669 merged it passes and adopts, and a new type-changes position covers it (check passes every refused line there, the clone's export exits 0, and update = bogus adopts and clones end to end). Lane 2's nit at adopt-project.py:370 and :404: check now refuses what ANY git refuses. _who_refuses_a_boolean replaces _git_reads_as_a_boolean, so -2147483648, -2097152k, -2048m and -2g (git 2.49 and older refuse them) and 0b numbers (only glibc 2.38+ builds read them) are plan-gitmodules-unreadable, and the finding says which gits refuse each; measured with git 2.43, shallow = -2147483648 and fetchRecurseSubmodules = -2g went from check 0 / exec 2 with both legs left (c800efd, a340d0a) to check 1 / exec 2 with no remotes dir. The cost, named in the body: on a git that reads those few values, check refuses them anyway. SonarCloud: the helpers both test files repeated (as_human, put_gitmodules, link, named, exported_in_a_clone, planned, findings, in_the_history, and the in-process execute refusal, now refused_before_any_leg) live in tests/conftest.py, with no assertion changed; python:S5778 is fixed there by building the Namespace before pytest.raises. Review finding 3 (the 252-case sweep predating the wording) is re-run on ddb1419: 248 agree with git 2.43, and the 4 others are the deliberate 0b refusals. Mutations: restoring the wide bound fails 20 cases, dropping the type-change skip fails 9. Body refreshed (tables, Composes with at current shas: main 4835fb4 and every sibling CLEAN). Targeted set (tests/test_adopt_*.py + doctor, hygiene, cli-reference) on ddb141916a1a10cbb598b532401f0b677c8770cc: 1233 passed, 1 skipped in 589.91s (the MedxEHR skip). main is not merged here; that waits for #226 to land.

Base automatically changed from fix/adopt-history-gitlinks to main October 11, 2026 23:04
brettheap and others added 2 commits October 11, 2026 23:07
…rejected-values

Lane: openreposhape-3 (openRepoShape-3)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… glibc alone

The verifier's optional nit 1 on fix round 1: C23 makes a base-0
`strtoimax` read `0b`/`0B`, so any C library that implements it reads a
`0b` number, not glibc alone. SOME_LIBRARIES_REFUSE now says "a `0b`
number, which a git reads only where its C library reads C23 binary
numbers (glibc 2.38 or later, for one)", and the comments that said
"only glibc" or "refused by every other" say the same. Wording only:
what `check` refuses is unchanged.

Nit 2 (the frame "git cannot read the source's .gitmodules" over a value
the installed git reads) is left: the frame is #226's shared finding
builder on main, and the parenthesis already says which gits refuse it.

Lane: openreposhape-3 (openRepoShape-3)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@brettheap

Copy link
Copy Markdown
Contributor Author

(lane openRepoShape-3, 2026-10-11T23:19Z) fix round 2 (independent review, codex is out of quota): #226 landed as aaa40eb, so this PR's base is now main and closingIssuesReferences holds #215. Merged main d89fffa into the branch (72f2bb6): four conflicting hunks in adopt-project.py (each a #215 sentence or the _refused_value call against #226's text) and an add/add in tests/test_adopt_history_gitlinks.py (the helper move), each conflicting hunk resolved to this branch's side, every clean hunk keeping main's changes (#230, #234, #246 among them); the resulting tree is exactly the tree of a merge computed with f48c669 as its base (757f81f), and main's aaa40eb carries f48c669's adopt-project.py change line for line and its history test file unchanged, so conftest's helpers serve it as they served f48c669's. Then 5521e82: the verifier's optional nit 1 taken, so the 0b refusal says "a git reads only where its C library reads C23 binary numbers (glibc 2.38 or later, for one)" instead of naming glibc alone; nit 2 (the frame "git cannot read the source's .gitmodules" over a value the installed git reads) left, as that frame is #226's shared builder on main and the parenthesis says which gits refuse it. Re-measured on 5521e82: the issue table and the fix-round-1 table give the same exit codes and remotes as on ddb1419. Trial merges at 23:10Z are CLEAN but fix/adopt-root-kept-symlinks (and lane 3's local heads of it and of fix/adopt-plan-quote-yaml-keywords), whose conflicts are byte-identical to their conflicts against main; main has since moved to eb6df75 (#220), which merges clean and is not merged here. Body updated (base, tables, composition, tests). Targeted set (tests/test_adopt_*.py + doctor, hygiene, cli-reference) on 5521e8281eded4f008f80a3bb89b978ca8dcb18c: 1309 passed, 1 skipped in 587.98s (the MedxEHR skip).

@sonarqubecloud

Copy link
Copy Markdown

@brettheap
brettheap merged commit c2ff2b0 into main Oct 12, 2026
5 checks passed
@brettheap
brettheap deleted the fix/adopt-gitmodules-rejected-values branch October 12, 2026 00:04
brettheap added a commit that referenced this pull request Oct 12, 2026
…opt-record-submodule-placement

Brings in c2ff2b0 (#215) and c3a9645 (#241) on top of d94d155. No rebase.
c2ff2b0 changes adopt-project.py, as this branch does, which is why main
is merged again so soon after d94d155. c3a9645 changes scaffold-project.py
alone.

No conflicts. Checked:

- adopt-project.py: #215 makes a `.gitmodules` value that git's submodule
  reader refuses (`update = bogus`, `shallow = maybe`, ...) #195's
  `plan-gitmodules-unreadable` finding, which `execute` refuses before any
  leg exists. It changes `_registrations`, `Source` and
  `_where_git_fails`, and adds `_refused_value`, `_who_refuses_a_boolean`
  and their constants. This branch's `submodule_record`,
  `_resolution_text` and the record's preamble are unchanged by the merge,
  and so are `_answered_legs`, `_kept_in_a_leg`, `_commit_the_split`,
  `cmd_execute`, `_refuse_what_check_finds`, `_entry_findings`,
  `_submodule_findings`, `submodule_plan_problems`, `Plan`, `y`, `emit`
  and `walk`. So, as for #195, a source #215 refuses gets no leg, no split
  and no record, and the record is still spelled by `double_quoted`.
- No top-level name is bound twice in adopt-project.py,
  scaffold-project.py, scripts/repo_shape.py, scripts/shape_materialize.py,
  shape-doctor.py, validate-manifest.py or update-shape.py. Each merged
  module's names are the base's plus what either side added. No definition
  was changed by both sides.
- The branch's diff against main is still #189 alone: 12 files, +1670
  -157, file by file the same as against d94d155.

Lane: openreposhape-1 (openRepoShape-1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 12, 2026
…, #205, #215, #241 and #249/#250 merge with no conflict, and `_entry_findings` still hands `_leg_findings` the tree and ends with the root-symlink findings

origin/main moved 60588d5 -> b30595c while the targeted runs of 8f624aa
went: 018ce0d (#207 via #222), 67a398a (#272 via #274), d94d155 (#205
via #232), c2ff2b0 (#215 via #240), c3a9645 (#241 via #256) and b30595c
(#249 and #250 via #284). It is merged again because c2ff2b0 changes
adopt-project.py and b30595c changes scripts/shape_materialize.py's
`render`.

Conflicts: none. `git merge-tree --write-tree` gave tree be093a5, and
the merge stages the same tree.

Checked on that tree:
- `_entry_findings` still calls `_leg_findings(plan, tree)` and ends
  with `_root_symlink_findings(plan, source, tree)`.
- No top-level name is bound twice in adopt-project.py,
  scaffold-project.py, scripts/repo_shape.py or
  scripts/shape_materialize.py, none is loaded but bound nowhere (AST),
  and main changed the signature of no function this branch calls.
- Both sides are intact by +/- line sets: each side's change against the
  merge base 60588d5 reappears, line for line, against the other side.
- c2ff2b0 adds helpers to tests/conftest.py (`planned`, `GITLINK`,
  `findings` and others) and no fixture. tests/test_adopt_root_symlinks.py
  imports its conftest names one by one and none of these, so its own
  `planned` and `GITLINK` are unaffected.
- b30595c's `render` spells a double-quoted YAML value with
  `double_quoted`; this branch's walker reads `assembly_root_paths()` and
  never renders.

Lane: openreposhape-1 (openRepoShape-1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 12, 2026
Main brings three commits since d94d155:

- c2ff2b0 (#215 via #240): adopt-project.py refuses a `.gitmodules`
  value that git's submodule reader dies on. It adds helpers to
  tests/conftest.py, adds tests/test_adopt_gitmodules_values.py, and
  moves helpers out of tests/test_adopt_history_gitlinks.py;
- c3a9645 (#241 via #256): scaffold-project.py's `--pin` collisions;
- b30595c (#249, #250 via #284): repo_shape.py's quoted scalars, and
  shape_materialize.py's `render()` spells a value between a YAML
  template's double quotes.

One conflict, in tests/conftest.py. Both sides append after `resolve()`:
main its `.gitmodules` helpers, this branch `spelled()`. Both are kept,
main's block first. Against main, the merged file differs by exactly
this branch's own diff. Against this branch, it differs by main's own
diff, but for where git places two added blank lines.

adopt-project.py merged without a conflict, and each top-level
definition comes whole from one side. c2ff2b0 changes `_registrations`,
`_where_git_fails` and `Source.registered_submodules`, and adds
`_refused_value`, `_who_refuses_a_boolean` and their constants. This
branch changes none of them, and its own `Source.__init__` and
`Source.open` are unchanged. The new refusal comes through
`_entry_findings`. `cmd_execute` calls that in `_refuse_what_check_finds`,
before `_refuse_an_occupied_work_dir` and `_create_leg_remotes`, so the
refusal is raised before any leg exists and no notice is due. Main's
`refused_before_any_leg` asserts that the remotes directory is never
made.

No top-level name is bound twice in adopt-project.py,
scaffold-project.py, scripts/repo_shape.py, scripts/shape_materialize.py
or tests/conftest.py. None of them reads a global it does not bind.

Lane: openreposhape-1 (openRepoShape-1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 12, 2026
…#241, #249/#250, #204, #203 and #238 merge with no conflict, and a `.gitmodules` value git refuses is answered before #190's relative-url check, which reads no url from a file git cannot read

Main since d94d155, the main GitHub's update-branch merged at 8ce5614:
c2ff2b0 (#215 via #240), c3a9645 (#241 via #256), b30595c (#249/#250 via
#284), be2790e (#204 via #227), 6d8d88b (#203 via #236) and ede15ce (#238
via #287).

No conflict hunk. Three files were changed on both sides; `git merge`
auto-merged each, and every hunk of each side lies apart from the other's.
Each side's +/- lines are identical before and after the merge in all
three (`git diff -U0` d94d155..8ce5614 against ede15ce..merge, and
d94d155..ede15ce against 8ce5614..merge). Line numbers are the merged
file's.

adopt-project.py
- main, c2ff2b0 (#215): 384-506, `SUBMODULE_TYPED_KEY_RE` to `_refused_value`
  and `_who_refuses_a_boolean`; 510-515 and 522-524, `_registrations`'
  docstring and its return of `_refused_value`'s words; 791-797,
  `Source.registered_submodules`' docstring; 2181-2185, `_where_git_fails`'
  docstring.
- #190: 36-38 and 51-53, the module docstring; 102-103, the `repo_shape`
  import list; 334-339, `Registrations.url`; 2252-2501, `RELATIVE_URL` to
  `_relative_url_findings`; 2503-2548, `_submodule_findings` and
  `_entry_findings` with `local_remote_dir` and the NOTEs; 2608 and 2630,
  `cmd_check`; 3070-3092, `_repository_urls` and `_remote_urls`;
  3111-3216, `_refuse_what_check_finds` and `_as_resolved_here`; 3443,
  `cmd_execute`.

scripts/repo_shape.py
- main, b30595c (#249/#250): 173-535, the YAML reader's quote boundaries,
  from the comment after `YamlError` to `_split_flow`.
- #190: 1975-1985, `REMOTE_PORT_RE` and `REMOTE_HOST_ALIASES`; 2099-2116,
  `remote_key`'s docstring and body.

tests/test_shape_doctor.py
- #190: 2662-2693, `test_a_port_and_githubs_ssh_host_are_not_part_of_a_remotes_key`.
- main, ede15ce (#238): 5209-5808, the validator-PYTHONPATH tests after
  `test_a_drifted_roots_next_command_runs_when_it_is_pasted`.

Every other file is one side's blob: main's for the 17 main changed, and
8ce5614's for tests/test_adopt_relative_submodule_url.py and
tests/test_adopt_submodules.py. The merge's diff against main is #190's
five files, with the stat it had against d94d155 (1097+, 41-). No
function or class is bound twice at the top level of adopt-project.py,
scaffold-project.py, scripts/repo_shape.py, scripts/shape_materialize.py,
shape-doctor.py, scripts/family.py or the three test files. The helpers
c2ff2b0 moved into tests/conftest.py (`planned`, `findings`, ...) are
plain functions, and tests/test_adopt_relative_submodule_url.py imports
seven named helpers from conftest and defines its own `planned`.

HOW #215 AND #190 MEET. `_submodule_findings` lists
`submodule_plan_problems` first, which holds #215's
`plan-gitmodules-unreadable`, then `_relative_url_findings`. Where git's
submodule reader refuses a value of the source's `.gitmodules`,
`registered_submodules` answers with git's words, a string, and
`_kept_relative_urls` reads no url from a string. So a source with both
problems gets #215's finding alone, and no NOTE either. #190's finding is
not lost: once the value is fixed, `check` reports it, and `execute`
refuses it through the same list. #190 adds lines only after #215's, so it
can hide nothing of #215's. A relative url kept in a leg is always beside a
gitlink kept in a leg, where `_where_git_fails` names a failure, so the
unreadable finding is always printed whenever #190's would be skipped.
Measured on this tree with a source holding `upstream/dependency`,
`url = ../dependency` and `update = bogus`, with `.gitmodules` and
`upstream/` kept in code for testorg:
- `origin` in otherorg: `check` 1 with `plan-gitmodules-unreadable` alone;
  `execute --local-remote-dir` 2, `REFUSED plan-gitmodules-unreadable`,
  with no remotes directory made. With `update = checkout`: `check` 1,
  `plan-submodule-relative-url`.
- `origin` in testorg: the same `check` 1 and `execute` 2, with no NOTE.
  With `update = checkout`: `check` 0 and the NOTE.
- At 8ce5614, before #215, the first source was `plan-submodule-relative-url`.
  The second was `check` 0 with the NOTE, over a file `git fast-export`
  dies on.

Lane: openreposhape-1 (openRepoShape-1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant