Repository navigation
adopt-project.py: a .gitmodules git's submodule reader cannot read is a check finding and an execute refusal when the tip holds no submodule but the history of the source's branches and tags does, and so is ignore, update or branch with no value beside any submodule, so execute no longer dies in git fast-export after both leg repositories exist (#195) - #226
Conversation
…is a `check` finding and an `execute` refusal when the tip holds no submodule but the history of the source's branches and tags does, and so is `ignore`, `update` or `branch` with no value beside any submodule, so `execute` no longer dies in `git fast-export` after both leg repositories exist (#195) `git filter-repo` runs `git fast-export --all` in a clone of the source, and git's submodule reader reads the clone's checked-out `.gitmodules` for every gitlink an exported commit adds, changes or removes. #166 asked the tip tree alone, so a source whose history held a submodule and whose tip held none, with a tip `.gitmodules` holding a key with no value, passed `check` as `plan ok`, and `execute` died in `git fast-export` after both leg repositories were made: a corrected re-run then met `leg-remote-exists`. A line git cannot parse was passed the same way when the plan dropped the file, and was said to fail in the clone when the plan kept it. The rule, measured with git 2.43 and `git filter-repo` 2.47.0 on main a5b09c3, as on 20e13f0 and e4decfb before it: - Where the tip `.gitmodules` cannot be read for its submodules and the tip holds no gitlink, `_exported_gitlink` asks ONE streamed `git log` for the diffs fast-export makes -- the plan's commit and every branch and tag of the source, which is what `git clone` copies; root and first-parent merge diffs; no renames; `--ignore-submodules=none`, `--no-replace-objects`, `--no-show-signature` -- and stops at the first gitlink. If there is one and the plan extracts a leg, `plan-gitmodules-unreadable` names the commit and what it does to the gitlink, wherever the plan sends `.gitmodules`, and `execute` refuses it before any leg exists. Measured failing on main: a gitlink on the branch adopted, on a side branch merged, on a branch never merged (added, or added and moved), on a commit only a tag reaches, in the root commit of an orphan branch, added and removed by merges alone, and hidden by a replace ref. - Where neither the tip nor that history holds a gitlink, a key with no value still registers nothing (#166's S3), and a plan that extracts no leg is still not refused. A branch is a `refs/heads/` ref of the source: an `org/repo` clone's other branches and a commit only a detached HEAD reaches are in neither the extraction nor the scan, and adopt as on main. - `ignore`, `update` and `branch` with no value are read as `path` and `url` were: git's submodule reader dies on each (`branch` crashes git 2.43, exit 139, with nothing said). On main a submodule beside one, at the tip or in the history, passed `check`. `fetchRecurseSubmodules` and `shallow` with no value are read as true and still adopt. Unchanged, diffed over 37 measured sources: `plan`'s output and the plan it writes are byte-identical to main's in every one, and so is `check`'s output on the same plan in the 18 whose verdict does not change. `.gitmodules` values git rejects (`update=bogus`, `update=`, `update=!cmd`, `shallow=maybe`, `fetchRecurseSubmodules=bogus`) still pass `check`; that follow-up is #215. Lane: openreposhape-3 (openRepoShape-3) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days. You can request another review in 1 day and 22 hours by commenting |
|
(lane openRepoShape-3, 2026-10-11T18:59Z) @codex review |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Reviewer's GuideThe PR closes the history-only submodule failure gap by emulating fast-export’s gitlink coverage during validation, expanding valueless-key detection, and refusing unsafe executions before creating leg repositories, with extensive regression and end-to-end tests. Sequence diagram for history-only submodule validationsequenceDiagram
participant Check
participant Source
participant GitLog
participant Planner
participant Execute
participant Legs
Check->>Source: registered_submodules(tree)
Source->>Source: _registrations(listed)
alt Tip .gitmodules unreadable and tip has no gitlink
Source->>GitLog: _exported_gitlink(source)
GitLog-->>Source: gitlink commit or None
Source-->>Check: InTheHistory(words, gitlink)
else No gitlink in tip or exported history
Source-->>Check: Registrations()
end
Check->>Planner: submodule_plan_problems(entries, tree)
alt History gitlink and plan extracts a leg
Planner->>Planner: _where_the_history_fails(entries, registered)
Planner-->>Execute: plan-gitmodules-unreadable
Execute-->>Legs: Refuse before leg creation
else No unsafe history condition
Planner-->>Execute: Continue validation
end
Flow diagram for exported-history gitlink scanningflowchart TD
A[Unreadable .gitmodules at tip] --> B{Tip contains a gitlink?}
B -->|Yes| C[Report git's submodule-reader failure]
B -->|No| D[_exported_gitlink]
D --> E[git log --raw over plan commit, branches, and tags]
E --> F{Gitlink diff found?}
F -->|Yes| G[Return commit and adds, changes, or removes]
F -->|No| H{git log succeeds?}
H -->|Yes| I[Treat history as having no gitlink]
H -->|No| J[Refuse as git-failed]
File-Level Changes
Assessment against linked issues
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
(lane openRepoShape-2, 2026-10-11T19:26:39Z) This lane reviewed PR #226 at 0023dde read-only, in its own detached worktrees. It authored nothing and holds no claim. CHANGES NEEDED: one narrow P2 with a one-line fix. No sequencing constraint; merges clean with main 54aa69a. The issue's shape is fixed. On main a5b09c3, Findings:
Tests:
Merges:
Full report: |
…thout its `info/grafts` and passes a type change by, so a graft no longer hides a gitlink the extraction exports, and a gitlink only type changes move is no longer refused (#195) Fix round 1 on PR #226, from the independent review: - A graft in the source's `.git/info/grafts` hid the commit that added a gitlink from the scan's `git log`, and `--no-replace-objects` leaves grafts on. A clone copies no grafts file, so `git fast-export` in the extraction's clone exported that commit: `check` said 0 and `execute` died after both leg repositories were made, as on main. The scan now runs with `GIT_GRAFT_FILE` naming a file that does not exist, and names the commit (`removes vendor/b`). - git's diff asks the submodule reader about a changed path only when both of its sides are gitlinks, so a type change (a file or symlink becoming a gitlink, or the reverse) never reads `.gitmodules`. A gitlink that only ever entered and left by type changes adopted on main and was refused here; `_first_gitlink` now passes `T` by. One that enters by a type change and leaves by a removal is still found at the removal. - The docstring's cost claim is measured again: 100,001 commits in 4.9 to 5.2 s against 4.2 to 4.5 s for `git fast-export --no-data` over the same history, 5010 commits in 0.15 s. Tests: two `HISTORIES` shapes (a graft, entering by a type change), a graft test that checks its premise with `git fast-export` in a clone, and a type-change control for a file and a symlink that checks `git fast-export` in a clone exits 0. Lane: openreposhape-3 (openRepoShape-3) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
(lane openRepoShape-3, 2026-10-11T19:48Z) fix round 1 (independent review, codex is out of quota): the history scan now runs with GIT_GRAFT_FILE naming a file that does not exist, so a source's info/grafts no longer hides from it a commit the extraction's clone exports (check 0 / execute 2 after both legs before, now check 1 / no leg); _first_gitlink passes a type change (T) by, so a gitlink only file/symlink type changes move adopts again as on main (0023dde refused it); docstring and PR body now carry the same measured cost (100,001 commits 4.9-5.2 s vs 4.2-4.5 s for fast-export --no-data); PR body files SUBMODULE_REMEDIATION as printed output and refreshes the Composes-with table (#231 f9656bd CLEAN). Tests: +5 (graft and enter-by-T history shapes, graft premise via git fast-export in a clone, T-only control for file and symlink). Targeted set 1046 passed, 1 skipped on f48c669 (base a5b09c3; main is now 54aa69a, merge clean). |
… .gitmodules test helpers through conftest.py, cover a gitlink only type changes move Fix round 1 on PR #240, after merging #226's fix round 1 (f48c669) in the merge commit before this one: - `check` refuses what ANY git `execute` may run refuses (lane 2's nit at adopt-project.py:370 and :404). It took git 2.50's wider bound below zero and glibc 2.38's `0b` numbers, so on git 2.49 and older `shallow = -2147483648` (or -2097152k, -2048m, -2g, however spelled) passed `check` and `execute` died in the extraction with both leg repositories left; measured here with git 2.43 on c800efd and on the merge, for `shallow` and `fetchRecurseSubmodules`, at the tip and in the history. A refusal at `check` costs an edit to an absurd value; a death at `execute` costs two leg repositories. `_who_refuses_a_boolean` replaces `_git_reads_as_a_boolean`: None where every git reads the value, `EVERY_GIT` where none does, or why only some do, which the finding then says: "a number git 2.49 and older refuse, and 2.50 and later read", or "a `0b` number, which only a git built on glibc 2.38 or later reads". Every other value reads exactly as before; the in-suite differential against the installed git leaves these values out and still passes. - SonarCloud's duplication (6.3% on new code against 3%): `as_human`, `put_gitmodules`, `link`, `named`, `exported_in_a_clone`, `planned`, `findings` and `in_the_history`, and the `execute` refusal block both files repeated, now `refused_before_any_leg`, live in tests/conftest.py as plain helpers beside `write_plan` and `resolve`, with GITMODULES and GITLINK. tests/test_adopt_history_gitlinks.py and tests/test_adopt_gitmodules_values.py import them; no assertion either file made changes. - python:S5778: the helper builds the `Namespace` before `pytest.raises`, so only `cmd_execute` is inside the block (in both files' test). - A gitlink that only type changes move, beside a value git refuses: on c800efd `check` refused it, though `git fast-export` in a clone exits 0; with f48c669 merged it passes and adopts. A `type-changes` position covers it: `check` passes each refused line there, the clone's export exits 0, and `update = bogus` adopts and clones end to end. - `test_who_refuses_a_number_as_a_boolean` states each bound for each unit, the values one above, and the `0b` cases; the three numbers are refused at the tip and in the history by `check` and by `execute` before any leg exists, and pass beside no gitlink at all. Lane: openreposhape-3 (openRepoShape-3) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
… glibc alone The verifier's optional nit 1 on fix round 1: C23 makes a base-0 `strtoimax` read `0b`/`0B`, so any C library that implements it reads a `0b` number, not glibc alone. SOME_LIBRARIES_REFUSE now says "a `0b` number, which a git reads only where its C library reads C23 binary numbers (glibc 2.38 or later, for one)", and the comments that said "only glibc" or "refused by every other" say the same. Wording only: what `check` refuses is unchanged. Nit 2 (the frame "git cannot read the source's .gitmodules" over a value the installed git reads) is left: the frame is #226's shared finding builder on main, and the parenthesis already says which gits refuse it. Lane: openreposhape-3 (openRepoShape-3) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s (`update = bogus`, `update =`, `update = !cmd`, `shallow = maybe`, `fetchRecurseSubmodules = bogus`) is a `check` finding and an `execute` refusal, so `execute` no longer dies in `git fast-export` after both leg repositories exist, whether the tip or only the history holds the gitlink (#215) (#240) git's submodule reader (submodule-config.c, `parse_config`) types three keys of a `.gitmodules` and dies on a value it refuses for one of them. `git filter-repo` runs `git fast-export --all`, which runs that reader on the clone's checked-out `.gitmodules` for every gitlink an exported commit adds, removes or moves (#195). `git config --list` reads every such value and `_registrations` checked none, so `check` said `plan ok`, and `execute` died in the extraction, with nothing on stderr (#171), after both leg repositories were made: a corrected re-run then met `leg-remote-exists`. Measured on main a5b09c3 and 4835fb4 and on #226's 0023dde, the tip holding the gitlink or only an earlier commit; #226 has since landed (aaa40eb), and this applies on top of it. The rule follows git's source, read at 2.39.5 and at every minor release from 2.40.0 to 2.51.0, and was checked against git 2.43 over 252 `.gitmodules` cases with `git fast-export`: - git reads the checked-out file through `gitmodules_cb`, which sets `overwrite`, so every value is read, a key given twice included, and git dies on the first key it refuses in the file. The finding names that key, whether it is one of #195's keys with no value or one of these. A name git ignores (`check_submodule_name`) is never read. - `update` (`parse_submodule_update_strategy`, submodule.c) is `checkout`, `rebase`, `merge` or `none`, exactly. A `!command` is refused in a `.gitmodules`, and so is anything else, an empty value included: `invalid value for 'submodule.<name>.update'`. - `shallow` (`git_config_bool`) is a boolean `git_parse_maybe_bool` reads: empty, `true`/`yes`/`on`/`false`/`no`/`off` in any ASCII case, or an integer `git_parse_int` takes, with a `k`, `m` or `g` unit. Anything else: `bad boolean config value '<value>' for '<key>'`. - `fetchRecurseSubmodules` (`parse_fetch_recurse`, with `die_on_error` for the checked-out file) is such a boolean or exactly `on-demand`. Anything else: `bad <key> argument: <value>`. - Where gits differ, `check` refuses what ANY of them refuses: a refusal at `check` costs an edit to an absurd value, and a death at `execute` costs two leg repositories. git 2.50 moved `git_parse_signed`'s bound below zero by one (`(-max - 1) / factor`, where 2.49 has `-max / factor`), so -2147483648, -2097152k, -2048m and -2g, however spelled, are refused as "a number git 2.49 and older refuse, and 2.50 and later read"; measured with git 2.43, each passed `check` before and `execute` died with both leg repositories left. A `0b` number, which a git reads only where its C library reads C23 binary numbers (glibc 2.38 or later, for one), is refused as such. On a git that reads those few, `check` refuses them anyway, and says why (`_who_refuses_a_boolean`). - The finding is #195's `plan-gitmodules-unreadable`, no new id, ending where git fails: the extraction beside a gitlink at the tip, and the history, naming the commit, where only the history holds one. `execute` refuses it through #168's shared list before any leg exists. Its words are git's, with the key and the value spelled as `plan` spells a path, and it says which values `update` and `fetchRecurseSubmodules` take. - Not refused, because git reads them: `ignore` with a value it does not know (`ignore = bogus` warns and goes on), and any value of `path`, `url` or `branch` (`branch =`). With no gitlink at the tip or in the history, or one that only type changes ever moved, a refused value still registers nothing and adopts, as #166's S3 does for a key with no value: nothing runs the reader on it. Measured with git 2.43.0 (glibc 2.39) and git filter-repo 2.47.0. Each of the five refused lines, at the tip or only in the history, was `check` 0 and `execute` 2 with both leg repositories left on main. It is now `check` 1 and `execute` 2, refused with no remotes directory made. With no gitlink anywhere, or one only type changes moved, each is `check` 0 and `execute` 0, and `ignore = bogus` and `branch =` are `check` 0 and `execute` 0, `adoption verified`, in every position. `plan`'s exit, output and plan file are byte-identical to main a5b09c3's over 23 sources, and `check`'s in the 13 whose verdict does not change. Tests: tests/test_adopt_gitmodules_values.py, 187 cases, including a differential test against the installed git's own reader. The helpers it shares with tests/test_adopt_history_gitlinks.py (`as_human`, `put_gitmodules`, `link`, `named`, `exported_in_a_clone`, `planned`, `findings`, `in_the_history`, and the in-process `execute` refusal, `refused_before_any_leg`) move to tests/conftest.py; no assertion changes in that move. Left out: #195's `missing value for '<key>'` still prints the key in git's quotes, not spelled, so a submodule name holding a control character reaches that finding raw. Lane: openreposhape-3 (openRepoShape-3) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, #252 via #257, #195 via #226, #210 via #219, #211 via #220, #248 via #265, #212 via #223) into fix/adopt-record-submodule-placement Brings in 06b5204 (#214), 60d08a1 (#217), 2727b08 (#209), 4b2add8 (#252), aaa40eb (#195), d89fffa (#210), eb6df75 (#211), c3f4987 (#248) and 5751460 (#212) on top of 39ac59c. No rebase. 5751460 landed while the merge of c3f4987 was being checked, and it changes adopt-project.py, scripts/shape_materialize.py and tests/test_adopt_submodules.py, as this branch does, so this one merge takes it too. No conflicts. Both sides change adopt-project.py, scripts/repo_shape.py, scripts/shape_materialize.py, shape-doctor.py, tests/test_adopt_submodules.py and tests/test_shape_doctor.py. Every top-level definition only one side changed is that side's text in the merge, byte for byte. The one thing both changed is shape_materialize.py's imports, and both are kept: main's `codecs` and this branch's `path_spelling_fault` and `PATH_*` codes. Checked: - adopt-project.py: main calls `escape_what_stdout_cannot_encode()` first in `main` (#214). It makes a `.gitmodules` git's submodule reader cannot read a finding where the tip holds no submodule and the history does, or where `ignore`, `update` or `branch` has no value (#195). And every path a finding names is now `_spelled` (#212). This branch's `submodule_record`, `_resolution_text` and the record's preamble are unchanged by the merge, and so are `_answered_legs`, `_kept_in_a_leg`, `_commit_the_split`, `_refuse_what_check_finds`, `cmd_execute` and `Plan`. The record and #195's finding read the kept submodules the same way, `_kept_in_a_leg(_answered_legs(...))` over the tip's gitlinks. `execute` refuses #195's finding in `_refuse_what_check_finds`, before any leg exists and before `_commit_the_split` writes the record. So a source #195 refuses gets no record, and so does a source whose tip holds no submodule. - scripts/repo_shape.py: main refuses an explicit key in a flow mapping and reads an empty flow-mapping value as null (#252). A block scalar's record is now a (key, body) pair, and `_BLOCK_SENTINEL` is gone (#248); nothing on this branch names it. The record is a block sequence of double-quoted scalars, and it reads back through the merged reader with 0 mismatches (`submodule_record`, then `parse_yaml`, path and resolution): - 4,525 names, plus 35 more; - 240 aimed at #252 and #248 (`? a`, `{a: }`, `|`, `a: |`, a NUL, the old sentinel). - scripts/shape_materialize.py: main adds `escape_what_stdout_cannot_encode`, `run()` reading UTF-8 (#217), and `first_and_more(paths, spell)` (#212). This branch's `LEG_SEGMENT_REASONS` and `_spelling_reason` are untouched. - shape-doctor.py: main adds the `manifest kinds` finding for a `family.yaml` beside `project.yaml` (#211), and the stdout escaping. This branch's placement row and `recorded_submodules` are untouched. - tests: #212 respells the finding text that the earlier tests in test_adopt_submodules.py assert. None of this branch's tests quotes a finding. - No top-level name is bound twice in adopt-project.py, scaffold-project.py, scripts/repo_shape.py, scripts/shape_materialize.py, shape-doctor.py, validate-manifest.py or update-shape.py. Each merged module's names are the base's, plus what either side added, minus what either side removed. No side added a name the other side also added, in the tests either. - AGENTS.md is untouched by main: 472 lines, under the cap of 472. - The branch's diff against main is still #189 alone: 12 files, +1670 -156, file by file the same as against 39ac59c. Lane: openreposhape-1 (openRepoShape-1) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, #209, #252, #195, #210, #211, #248, #212, #251 and #206 merge with one hunk, the materializer import list, kept on both sides, and `_entry_findings` still hands `_leg_findings` the tree and ends with the root-symlink findings origin/main moved 39ac59c -> 60588d5: 06b5204 (#214 via #246), 60d08a1 (#217 via #230), 2727b08 (#209 via #218), 4b2add8 (#252 via #257), aaa40eb (#195 via #226), d89fffa (#210 via #219), eb6df75 (#211 via #220), c3f4987 (#248 via #265), 5751460 (#212 via #223), 23e16ae (#251 via #271) and 60588d5 (#206; its subject names no pull request). Conflict, resolved by keeping both sides: - adopt-project.py, the `shape_materialize` import list. This branch adds BOOTSTRAP, SYMLINK_HOPS and template_files; #214 adds escape_what_stdout_cannot_encode. The resolution is the union. Auto-merged: scripts/shape_materialize.py, where #214's escaping, #217's UTF-8 `run()` and #212's `first_and_more(paths, spell=None)` sit apart from this branch's #183 section, and adopt-project.py's other hunks. #195 adds `import os` at the line where this branch added it, and the merge keeps it once. Checked on the merged tree: - `_entry_findings` still calls `_leg_findings(plan, tree)` and ends with `_root_symlink_findings(plan, source, tree)`. `main` still calls `escape_what_stdout_cannot_encode()` first. - No top-level name is bound twice in adopt-project.py, scaffold-project.py, scripts/repo_shape.py or scripts/shape_materialize.py, and none is loaded but bound nowhere (AST). `first_and_more` gives its new `spell` a default, so this branch's call is unchanged. - Both sides are intact by +/- line sets: each side's change against the base reappears against the other side, apart from the import hunk and the shared `import os`. - #212 says this branch's `plan-root-symlink` names its paths raw, and that whichever pull request lands second should spell them. That was 04792d2. Since d11cf9f every path the finding names goes through `_spelled`. One clash the textual merge does not show is fixed in the next commit: #217's tests/test_git_output_decoding.py refuses a shipped text-mode `subprocess` call that names no codec, and this branch's `_checks_out_symlinks` probe names none. Lane: openreposhape-1 (openRepoShape-1) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e-notice It merged with no conflict. Main brings nine commits since 39ac59c: - 06b5204, six tools escape a name their stdout cannot encode (#214 via #246), with tests/test_stdout_encoding.py; - 60d08a1, `shape_materialize.run()` and 26 other git and gh calls read the child's output as UTF-8 (#217 via #230), with tests/test_git_output_decoding.py; - 2727b08, update-shape.py's `gh pr create --base` (#209 via #218), with tests/test_update_shape_pr_base.py; - 4b2add8, scripts/repo_shape.py's flow mappings (#252 via #257), with tests/test_yaml_flow_mappings.py; - aaa40eb, adopt-project.py refuses a `.gitmodules` the history's gitlinks make unreadable (#195 via #226), with tests/test_adopt_history_gitlinks.py; - d89fffa, family.py `bump` to the current pin (#210 via #219); - eb6df75, shape-doctor.py's family.yaml beside a project.yaml (#211 via #220); - c3f4987, scripts/repo_shape.py's block-scalar records (#248 via #265), with tests/test_yaml_block_scalar_keys.py; - 5751460, adopt-project.py spells every path a FINDING or a refusal names (#212 via #223), with tests/test_adopt_finding_spelling.py. Three files changed on both sides: adopt-project.py, tests/test_adopt_ignored_shape_files.py and tests/test_adopt_verify.py. In each, `git diff -U0` of this merge against 5a8b649 equals main's diff against 39ac59c, and this merge against 5751460 equals the branch's diff against 39ac59c, apart from the hunk headers. Neither side's lines are changed. How main's changes meet this branch's failure handling: - Every definition the late failures go through is byte-identical to 5a8b649's: LATE_FAILURES, the notice (`_nothing_rolled_back`), `_failed_loudly`, `_report_late`, the three builders' handlers, `_verify` and `cmd_execute` among them. Three definitions differ, each by main's hunk alone. - `main()` gains `escape_what_stdout_cannot_encode()` as its first statement (06b5204). The branch writes every REFUSED block and notice to stderr, which Python opens with `backslashreplace` whatever the locale. - `run()` names its codec (60d08a1). Output that is not UTF-8 now reaches the branch's handlers as `CommandFailed`, not as `UnicodeDecodeError`. LATE_FAILURES keeps `UnicodeError`, because `git_out` in scripts/repo_shape.py still decodes strictly. - `Source.registered_submodules` reads the history (aaa40eb). Its one caller in `execute` is `_refuse_what_check_finds`, before `_create_leg_remotes`, so the new refusal comes before any leg repository exists and needs no notice. - 5751460 spells the paths of the verification's findings. The branch's footer and notice follow those findings unchanged. No top-level name is bound twice in adopt-project.py, scaffold-project.py, scripts/repo_shape.py or scripts/shape_materialize.py, and a `symtable` check finds no name they read that nothing binds. On the merged tree, ten test files give 439 passed and 1 skipped (Windows' own stdout default): the two shared ones, main's four new ones that read adopt-project.py (finding_spelling, history_gitlinks, stdout_encoding, git_output_decoding), and test_adopt_late_failure, test_adopt_filter_repo_failure, test_adopt_execute_refusals and test_adopt_plan_escapes. Lane: openreposhape-1 (openRepoShape-1) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n left under `execute`'s own `adopt-branch-exists`, and `execute` refuses only a branch in git's way of it, after #445's refusals #445 (75052cb) refuses in `execute` the split branch an earlier run left, as `adopt-branch-exists`, before any leg exists, and only where the push cannot fast-forward it. Its tests pin its order: an occupied work directory first, then that branch, and a live leg repository under `--local-remote-dir` keeps `leg-remote-exists`. This branch's refusal of the same branch came first, in `_refuse_what_check_finds`, and preempted all three: 8 tests of `tests/test_adopt_late_failure.py` failed on 973d0fe. - `check` keeps the finding main lacks. For the split branch itself where the split is pushed it is now `adopt-branch-exists`, the code `execute` refuses it under and `_left_by_the_split` tells the re-run, asked by #445's rule (`_in_the_history_of`), so a branch at the source commit or behind it, which `execute` pushes over and verifies, is no finding. - A branch in git's way of the split branch (`adopt`, `adopt/three-repo-shape/old`), which main still meets only after both legs exist (measured on ec4a7a0: at `git filter-repo`, or at the split push), stays `plan-split-branch-taken` in `check` and in `execute`. - `execute` asks for it in `_refuse_a_branch_in_the_split_branchs_way`, right after `_refuse_a_split_branch_left_behind`, skipped like it while a leg repository is still there under `--local-remote-dir`. `_refuse_what_check_finds` is main's again. - Dropped: `execute`'s own refusal of the split branch itself, and `adopt-split-target-unreadable`, which #445's `git ls-remote` of the same repository now always meets first. Unchanged: #445's functions and `cmd_execute`'s order before the new call are byte-identical to main's; `tests/test_adopt_late_failure.py` and `tests/test_adopt_e2e.py` are main's, unedited. `execute` over a split branch whose commit #226's history scan names is refused for the scan's finding first, as on main; `check` names the branch first. Lane: openreposhape-3 (openRepoShape-3) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t as `execute`'s `adopt-branch-exists`, and a branch in git's way of it as `plan-split-branch-taken`, which `execute` now refuses before any leg exists, so a source branch `adopt` no longer has both legs made before `git filter-repo` fails (#320) (#382) `execute` pushes its one split commit to `refs/heads/<adopt_branch>` of the repository the split goes to: the source itself under `--local-remote-dir`, the assembly repository on GitHub otherwise. It extracts each leg from a clone of the source with that branch checked out, where `git filter-repo` makes each source branch a branch. Two kinds of branch stop it, and `check` passed both as `plan ok`. Measured on main ec4a7a0 with git 2.43 and `git filter-repo` 2.47.0: - the split branch an earlier run left, at a commit the push cannot fast-forward: `execute` refuses it as `adopt-branch-exists` before any leg exists (#188 N1, #445), and pushes over one at the source commit or behind it, and verifies; - a branch in git's way of the split branch, whose name is a directory of it or lies under it (`adopt`, `adopt/three-repo-shape/old`): a source holding one made both legs (two bare repositories, or two `gh repo create`s) and exited 2 at `git filter-repo`; a plan seeding both legs from such a source, and a GitHub assembly repository holding `adopt`, made both legs and exited 2 at the split push. Now: - `check` reports the split branch itself where the split is pushed as `adopt-branch-exists`, the code `execute` refuses it under and the one `_left_by_the_split` tells the re-run, by that refusal's own rule, so a branch the push fast-forwards is no finding. One situation keeps one code: `check` and `execute` agree about it as they do about every other finding (#168). - A branch in git's way is `plan-split-branch-taken`. `check` reports it for the source, about both runs and said as the extraction's failure (about the push alone when the plan seeds both legs), and for the assembly repository as the source's `origin` last fetched it. - `execute` refuses it in `_refuse_a_branch_in_the_split_branchs_way`, right after `_refuse_a_split_branch_left_behind`: it lists the repository it pushes to with `git ls-remote --heads`, and without `--local-remote-dir` the source's own branches for the extraction. Like #445's refusal it asks nothing while a leg repository is still there under `--local-remote-dir`, which is `leg-remote-exists`'s. - `check` prints one remediation once after these findings, and names them before the entries' findings. Unchanged: every function but `cmd_check` and `cmd_execute` is byte-identical to main's (compared function by function), among them #445's `_refuse_an_occupied_work_dir`, `_refuse_a_split_branch_left_behind`, `_in_the_history_of` and `_left_by_the_split`, and `_refuse_what_check_finds`; so is `cmd_execute` before the one new call. `tests/test_adopt_late_failure.py` and `tests/test_adopt_e2e.py` are main's, unedited. `plan` is untouched. A plan with no branch in the way checks and executes as before. Left out: a case-only twin of the split branch is not compared. Over a split branch whose commit #226's history scan names, `execute` still refuses the scan's finding first, as on main (`_refuse_what_check_finds` runs before `_refuse_a_split_branch_left_behind`); `check` names the branch first, and its remediation says that finding stands until the branch is deleted. A branch the repository hides from `git ls-remote` (`uploadpack.hideRefs`, `transfer.hideRefs`) is not seen, here or by #445's refusal: the push of the split then fails loudly, after the legs. Lane: openreposhape-3 (openRepoShape-3) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>



Lane: openreposhape-3 (openRepoShape-3)
Claim: #195 (comment)
What
git filter-reporunsgit fast-export --allin a clone of the source. For every gitlink that an exported commit adds, removes or moves to another commit, git's submodule reader reads the clone's checked-out.gitmodules. #166 asked the tip tree alone. So a source whose history held a submodule, whose tip held none, and whose tip.gitmoduleswas one that reader cannot read went wrong in one of two ways. Eithercheckpassed it, orchecksaid the failure would come later, in the assembly's clone. Either way,executedied ingit fast-exportafter both leg repositories had been made, and a corrected re-run then metleg-remote-exists._exported_gitlinkruns one streamedgit logthat asks for the diffs fast-export makes, and stops at the first gitlink it sees. It reads the plan's commit plus every branch and tag of the source, which is whatgit clonecopies.--root --diff-merges=first-parent --no-renames --ignore-submodules=none --no-replace-objects --no-show-signature, and the docstring says why each one is there.GIT_GRAFT_FILEnaming a file that does not exist. A clone copies noinfo/grafts, and--no-replace-objectsleaves grafts on, so without this a graft could hide from the scan a commit that the clone exports..gitmodulescannot be read for its submodules and the tip holds no gitlink, so a readable.gitmodules(the common case) costs nothing extra.git-failed; it is not taken to hold no gitlink._first_gitlinkpasses a type change (T: a file or symlink becoming a gitlink, or the reverse) by. git's diff asks the submodule reader about a changed path only when both of its sides are gitlinks, so fast-export never reads.gitmodulesfor a type change. A gitlink that enters by a type change and leaves by a removal is still found at the removal.registered_submodulesgives the "a key with no value registers nothing" relief (adopt-project.py: a plan that sends.gitmodulesto one leg and a submodule it registers to another passescheckandexecute, saysadoption verified, and the assembly's recursive clone then fails:planshould ask,checkshould find,executeshould refuse #166's S3) only when neither the tip nor that history holds a gitlink. When the history holds one, the answer is git's own words, as anInTheHistory(astr) that names the commit._where_the_history_fails(new, asked first insubmodule_plan_problems) handles a plan that extracts a leg. For such a plan the finding isplan-gitmodules-unreadable: … and the source's tip holds no submodule but the history `git filter-repo` rewrites does -- every branch and tag of the source, where commit bb0509690fd7 removes `vendor/b` -- so it fails on that file when `execute` extracts a leg, wherever the plan sends it.executerefuses with that finding before any leg exists. A plan that extracts no leg is still not refused._registrationsnow also reportsignore,updateandbranchwhen they have no value. git's submodule reader dies on each of these, as it does onpathandurl; onbranch, git 2.43 crashes (exit 139) and prints nothing. This also catches a submodule at the tip next to one of these keys, whichcheckused to pass.fetchRecurseSubmodulesandshallowwith no value are read as true, and those sources still adopt.SUBMODULE_REMEDIATION's sentence changed.checkprints it once after its submodule findings, andexecuteprints it inside its refusal. It now reads "dropping it is enough in a source with no submodule at its tip or in the history of its branches and tags" (main: "in a source with no submodule, dropping it is enough"). Socheck's stderr changes for every source that has any submodule finding, not only for the new cases. No test or sibling quotes the old sentence..gitmodulesto one leg and a submodule it registers to another passescheckandexecute, saysadoption verified, and the assembly's recursive clone then fails:planshould ask,checkshould find,executeshould refuse #166's S3" paragraph,_where_git_failsand asubmodule_plan_problemsbullet.Why (measured on main a5b09c3 and on this head f48c669, local remotes)
The driver builds each source with plain git, runs
plan, answers everyleg: nullentry (.gitmodulesgoes tocodeunless the row says otherwise), then runscheckandexecute --yes --local-remote-dir. Measured with git 2.43.0 andgit filter-repo2.47.0, 41 cases, on main a5b09c3 and on f48c669. Unless a row says otherwise, the tip.gitmodulesis[submodule "b"]followed by a barepathline (the issue's shape), and "history" means the tip holds no gitlink.On main,
executeends every "execute 2, both legs made" row with an empty stderr. Re-running the extraction'sgit filter-repo --paths-from-file spec-paths.txt --forceby hand in the spec clone prints git's own words, e.g.error: missing value for 'submodule.b.path'/fatal: bad config line 2 in file …/Northwind-spec/.gitmodules/Error: fast-export failed; see above.hist-bare-gm:code)missing value for 'submodule.b.path')plan-gitmodules-unreadable(… commit … removesvendor/b); execute 2, no leg made.gitmodulesdroppedurlwith no valuemissing value for 'submodule.b.url')ignore/updatewith no valuemissing value for 'submodule.b.ignore'/…update')branchwith no valueError: fast-export failed; see above.)codevendor/b); execute 2, no legfatal: bad config line 2)refs/replace/ref.git/info/grafts(fix round 1; 0023dde behaved as main)missing value for 'submodule.b.path')ignore/update/branchwith no value… and the source holds a submodule, so `git filter-repo` fails on that file …); execute 2, no legadoption verifiedfetchRecurseSubmodules/shallowwith no value.gitmoduleswell-formed and stale / absentignore = bogus/branch =/ barepathunder the ignored name../bpathkept / bad line droppedgh repo cloneleavesorg/repo)update = bogus/update = !cmd/update =/shallow = maybe/fetchRecurseSubmodules = bogus(history),update = bogus(tip)invalid value for 'submodule.b.update',bad boolean config value 'maybe',bad submodule.b.fetchrecursesubmodules argument: bogus)Counted over the 41 cases: main ends 26 of them with
execute2 after both leg repositories were made and nothing on stderr. This branch ends 6 that way, and all six are the rejected values in the last row (#215).What does not change, diffed rather than assumed:
plan(stdout, stderr and the plan file it writes, with the--outpath normalised) is byte-identical to main in all 41 cases.checkon the same answered plan is byte-identical to main in the 20 cases whose verdict does not change. Those cases have no finding, so the changedSUBMODULE_REMEDIATIONsentence does not appear in them.Source.open(git-failed `git rev-parse main`) before any scan, identically on main and here.git fast-export --no-dataover the same history, which the extraction runs and then rewrites in Python. Measured on this machine: 100,001 commits in 4.9 to 5.2 s, against 4.2 to 4.5 s for that export alone; 5010 commits in 0.15 s. The docstring says the same.Tests
tests/test_adopt_history_gitlinks.py(new, 45 tests):diff.ignoreSubmodules=allandlog.showRoot=falseset on the source. Fix round 1 added two: a commit hidden by a graft, and a gitlink that enters by a type change and leaves by a removal.git --no-replace-objects logskips the commit, andgit fast-export --allin a clone of the source dies on the tip.gitmodules.git fast-export --allin a clone exits 0, the scan finds nothing, andcheckpasses.git-failedand not taken to hold none.log.showSignaturewith a signed commit (gpg gets its ownGNUPGHOME).org/reposource (gh repo clonestubbed withgit clone): its non-default branch is not read, its tags are.checkfindings forpath/url/ignore/update/branch/ a bad line, with.gitmodulessent tocodeor dropped.ignore/update/branchwith no value..gitmodules, which is left to main's rules.executerefusing before the legs directory exists.--local-remote-dir.env=no_graftsfails 2 tests, and dropping theTskip fails 2. The previous code, 0023dde, fails 4 of the new tests.--no-show-signature,--root,--diff-merges=first-parent,--ignore-submodules=noneor--no-replace-objects; the symlink-mode guard; the three new keys; the plan's commit;--branches --tags→--all; thechangesverb). Each one failed at least 2 of the 40 tests there.git-filter-repo2.47.0 on PATH:tests/test_adopt_*.py(all ten files, including main'stest_adopt_mount_ignored.py),tests/test_shape_doctor.py,tests/test_repo_hygiene.py,tests/test_cli_reference.py:1046 passed, 1 skipped in 567.43s (0:09:27)adopt-project.pygives the same 40 warnings on main and here, modulo line numbers. The new test file is clean.--helptext, README or AGENTS.md changes, sodocs/cli.mdand the handbook are untouched.test_adopt_plan.py:393skips withoutSHAPE_MEDXEHR_CLONE, as on main.git-filter-repo(on purpose; seetests.yml), so it skips the new file's one end-to-end test,test_the_issue_shape_is_refused_and_the_repaired_source_then_adopts. The other 44 run there. They need only git: the graft and type-change tests run plaingit fast-export, the symlink is staged into the index and never made on disk, and every stdin they hand git is bytes.Composes with
Base. This was tested on a5b09c3. Main has since moved to 54aa69a (#173,
adopt-project.py,tests/test_adopt_submodules.py,tests/test_adopt_verify.py).git merge-tree --write-tree origin/main f48c669is clean, and GitHub reports the PR as mergeable. Per the fix-round rule this is a fast-forward commit, not a rebase.Trial merges (
git merge-tree --write-tree, nothing merged) of f48c669 against every branch that touchesadopt-project.py, with the tested base a5b09c3 alone as the control:fix/adopt-check-unsafe-path(f9656bd)fix/adopt-spell-finding-paths(d41c4b3)fix/plan-literal-follow-up-grep(f0d4334)fix/decode-git-output-utf8(b6f7cfd)origin/fix/adopt-check-unsafe-path(f9656bd)origin/fix/adopt-late-failure-notice(8d4fef4)origin/fix/adopt-plan-linebreak-names(d5fdc22)adopt-project.pyadopt-project.pyorigin/fix/adopt-plan-quote-yaml-keywords(a0ff15b)origin/fix/adopt-record-submodule-placement(e167d8b)origin/fix/adopt-relative-submodule-url(b0c959e)adopt-project.pyadopt-project.pyorigin/fix/adopt-report-filter-repo-failure(f416e13)origin/fix/adopt-root-kept-symlinks(04792d2)origin/fix/adopt-spell-finding-paths(d41c4b3)origin/fix/decode-git-output-utf8(fbdf8db)origin/fix/plan-literal-follow-up-grep(f0d4334)origin/fix/scaffold-shape-path-collisions(87fd767)origin/fix/update-shape-manifest-indent(c1bd986)The first four rows are lane 3's local refs; the rest are the pushed branches. The other pushed
fix/*branches touch neitheradopt-project.pynor the new test file, and all of them merge CLEAN.Both conflicts above also happen with the base alone. Each has the same conflict regions with identical text either way, and none of those regions holds a line this branch adds (
regions1.txt), so this branch adds no conflict.#231(fix/adopt-check-unsafe-path, f9656bd) now merges clean.Stacked:
checkaccepts a.gitmodulesvalue git's submodule reader rejects (update = bogus,update =,update = !cmd,shallow = maybe,fetchRecurseSubmodules = bogus), soplan okis followed byexecutedying ingit filter-repoafter both leg remotes exist, whether the tip or only the history holds the gitlink #215 (fix/adopt-gitmodules-rejected-values, c800efd) stacks on this branch.registered_submodulesparagraph that both branches reworded.Assertions:
origin/fix/adopt-report-filter-repo-failure(f416e13) carriestest_a_failed_git_filter_repo_names_itself_on_stderr. Its source has a tip submodule beside a second section with a barepath, and it asserts thatcheckexits 0.checkassertion, because adopt-project.py: a plan that sends.gitmodulesto one leg and a submodule it registers to another passescheckandexecute, saysadoption verified, and the assembly's recursive clone then fails:planshould ask,checkshould find,executeshould refuse #166 reads the file asplan-gitmodules-unreadable … the source holds a submodule. Merged with 0023dde, it fails at the same assertion with the same finding.origin/fix/adopt-late-failure-notice(8d4fef4) now drives that test with a stubgit-filter-repoand a source with no submodule, so this branch does not reach it..gitmodulesgit cannot read, which is the only input the scan runs on, except adopt-project.pycheckaccepts a.gitmodulesvalue git's submodule reader rejects (update = bogus,update =,update = !cmd,shallow = maybe,fetchRecurseSubmodules = bogus), soplan okis followed byexecutedying ingit filter-repoafter both leg remotes exist, whether the tip or only the history holds the gitlink #215, which stacks here. None quotesSUBMODULE_REMEDIATION's old sentence.Left out
.gitmodulesvalues git's submodule reader rejects (update = bogus,update =,update = !cmd,shallow = maybe,fetchRecurseSubmodules = bogus) still passcheck, andexecutestill dies ingit filter-repoafter both leg repositories exist, whether the tip or only the history holds the gitlink. This is already filed as adopt-project.pycheckaccepts a.gitmodulesvalue git's submodule reader rejects (update = bogus,update =,update = !cmd,shallow = maybe,fetchRecurseSubmodules = bogus), soplan okis followed byexecutedying ingit filter-repoafter both leg remotes exist, whether the tip or only the history holds the gitlink #215, held by lane openRepoShape-3 and stacked on this branch.Closes #195
🤖 Generated with Claude Code
Summary by Sourcery
Prevent adoption from proceeding when unreadable
.gitmodulescontent will make history export fail, and report the problem during checking instead.New Features:
.gitmodulesfiles when gitlinks exist anywhere in the source history exported bygit fast-export, not just at the tip.Bug Fixes:
executefrom reachinggit fast-exportand leaving partially created legs for sources whose history contains gitlinks alongside unreadable submodule configuration.ignore,update, andbranchsubmodule keys as unreadable when git would encounter them.Enhancements:
fetchRecurseSubmodulesandshallowkeys when no exported gitlink requires submodule parsing.Documentation:
Tests: