Repository navigation
adopt-project.py: a recorded source whose .git this user cannot read is refused source-unreadable, naming that .git, so check and execute no longer call a clone plan made with its .git at mode 000 no longer whole and clone it again (#1099) - #1141
brettheap wants to merge 2 commits into
Conversation
…d is refused `source-unreadable`, naming that `.git`, so `check` and `execute` no longer call a clone `plan` made with its `.git` at mode 000 no longer whole and clone it again (#1099) Measured on main 4ed7c7b as uid 1000, git 2.43.0, Python 3.12.3, with a stub `gh` that records and refuses every call and local remotes. The plan's `source.local_path` is a directory this user can enter; its `.git` is at mode 000: - `cloned: true`: `check` and `execute --yes` exit 2 after "NOTE ... is a clone `plan` made that is no longer whole (git finds no repository there), so its `source.repository`, testorg/Thing, is cloned again from GitHub" and "cloning testorg/Thing", with one `gh repo clone` asked. - `cloned: false`: `REFUSED git-failed: git rev-parse --is-inside-git-dir ... exited 128: fatal: not a git repository`. - A `.git` file (`gitdir:`) at mode 000 does the same, with git's "fatal: error opening '.../.git': Permission denied" where `cloned: false`. Now each is `REFUSED source-unreadable: the `.git` in the plan's `source.local_path`, `<path>/.git`: cannot be read (Permission denied), so <what cannot be told> cannot be told`, exit 2, nothing cloned, `gh` never asked. This is #918's code and wording for a recorded directory this user cannot enter. - `_git_refused` asks the `.git` before git runs, through `_is_a` and `_is_there` (#430). A directory is asked whether its `HEAD` is there, a `stat` that needs only the directory searched, and a regular file is opened. A `.git` that is absent, that holds no `HEAD`, or whose own `stat` is refused is left to git and to #918's guards, as before. - `_recorded_source` refuses it right after asking whether the path is there, for `cloned: true` and `false` alike. `Plan._git_unreadable` words the refusal. - `_source_unreadable` takes the cure for a permission as `permission=`. A `.git` file gets `SOURCE_UNREADABLE_FILE_PERMISSION` (`chmod u+r` on the file). - The `--source` help of `check` and `execute` names the `.git` case, and `docs/cli.md` is regenerated. - `--source` itself is left as it is: the issue names the recorded clone. Tests: `tests/test_adopt_unreadable_clone_git.py`, 18 rows: 14 fail on main and pass here; the 4 readable controls pass on both. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reviewer's GuideImplementationAdd file-specific permission guidance and detect unreadable
Wire the
UIUpdate
DocumentationRegenerate the CLI reference so both command descriptions match the updated source help.
TestsAdd coverage for unreadable
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
RV1141: independent review of #1141 at No MUST and no SHOULD; four NITs, all follow-ups. Measured as uid 1000, git 2.43.0, Python 3.12.3, with a stub Reproduced, then fixed
False-refusal probes (scratch merge;
|
| Case | git's own answer | The PR |
|---|---|---|
.git at 0100, 0300, 0500 |
works | plan ok, 0 gh (no false refusal) |
.git at 0400, 0600 (no search) |
"not a git repository" | source-unreadable, directory cure, 0 gh |
worktree .git file, gitdir readable (cloned false and true) |
works | plan ok, 0 gh |
submodule .git file, readable (both) |
works | plan ok, 0 gh |
submodule .git file at 000 (both) |
"error opening ... Permission denied" | source-unreadable, file cure, 0 gh |
.git a symlink to a readable git dir (both) |
works | plan ok, 0 gh |
.git a symlink, its target dir at 000 (both) |
"not a git repository" | source-unreadable naming <it>/.git, 0 gh |
GIT_DIR set to another repo, .git readable (both, main and merge) |
reads GIT_DIR |
plan ok, as on main |
GIT_DIR set to another repo, .git at 000 (both) |
reads GIT_DIR |
source-unreadable naming .git; main refused too, git-failed ("a git or gh command failed") |
.git emptied (no HEAD) |
"not a git repository" | NOTE, cloned again, 1 gh: unchanged, as intended |
| the clone directory itself at 000 | n/a | #1072's source-unreadable naming the directory, unchanged |
.git/HEAD, .git/objects, .git/refs or .git/config at 000 |
"not a git repository" / "unable to access '.git/config'" | NOTE, cloned again, 1 gh, identical on main (NIT-2) |
worktree gitdir or its common .git at 000; submodule .git/modules/<m> at 000 |
"not a git repository: " | cloned: true: cloned again, 1 gh; cloned: false: git-failed; as on main |
.git a symlink whose target's parent is at 000 |
"not a git repository" | cloned: true: cloned again, 1 gh, as on main (NIT-3) |
No readable or usable .git is now refused. Every remaining clone-again row behaves exactly as on main.
Wording
Each refusal is true and actionable, and matches #1072's family: <named>: cannot be read (Permission denied), so <what> cannot be told. The directory cure "chmod u+rx on the directory" names the .git the line names. The .git file cure is chmod u+r on the file. With cloned: true the wording is "whether plan's clone is still whole, to be read, or no longer whole, to be cloned again,"; with cloned: false it is "whether a repository is there", as #1072's --source refusal says. SOURCE_OVERRIDE_HELP speaks of the recorded local_path, which matches the behaviour. test_cli_reference.py passes on the merge, so docs/cli.md agrees with the help.
Tests (scratch merge 0f3ac25c = main 038dafba9 + 7a2711d0, the gate environment)
| Set | Result |
|---|---|
test_adopt_unreadable_clone_git.py + test_adopt_unenterable_source_and_locale.py |
31 passed |
test_path_guard_hygiene.py (#1111, now on main) + test_cli_reference.py |
72 passed |
test_adopt_unsearchable_source.py, test_agents_md_codes.py, test_repo_hygiene.py, test_adopt_source_clone_record.py |
274 passed |
test_scaffold_leg_path_collisions.py |
104 passed |
test_shape_doctor.py, 8 shards, every id list non-empty |
445 passed, 3 skipped, 448 ids: shards 00 to 07 gave 62, 56, 54, 56, 60, 57, 55 + 3 skipped and 45. Shards 03 and 04 hit this review's 200 s limit and were rerun with no limit by the lane at 13:42-13:46Z on the same merge 0f3ac25c, 56 and 60 passed |
The new file run against main's adopt-project.py (copied into a main worktree): 14 failed, 4 passed, as claimed. They fail for the right reasons:
- the 8 mode rows:
git-failed, or the NOTE plussource-unclonable, wheresource-unreadableis asserted; - the in-process row:
checkread the clone; - the 5 helper rows: no
_git_refused.
The 4 controls pass on both.
Mutants (own, on the merge; 31 rows: the new file + #1072's file; sha256 b2b41532c3a5 checked after every one)
| Mutant | Result |
|---|---|
M1 guard removed (refused = None) |
KILLED, 9 failed |
M2 the .git's own refused stat taken as the .git's |
KILLED, 3 failed (#1072's cannot_enter rows) |
M3 return (exc, a_file) without _absent |
SURVIVED: equivalent. _is_there already answers an absent HEAD False, so _absent is reached only by a race on the file's open |
| M4 every refusal swallowed | KILLED, 9 failed |
M5 a .git file never opened |
KILLED, 4 failed |
M6 the directory asked by os.listdir (read permission) |
KILLED, but only by the in-process row (1 failed) |
| M7 cures swapped | KILLED, 9 failed |
| M8 untold wording swapped | KILLED, 9 failed |
M9 names local_path, not its .git |
KILLED, 9 failed |
M10 guard for cloned: true only |
KILLED, 4 failed |
M11 guard for cloned: false only |
KILLED, 5 failed |
M12 the directory asked for read permission, then HEAD (os.access(R_OK) else listdir) |
SURVIVED (NIT-1) |
Merge-tree
git merge-tree --write-tree of 7a2711d0:
- against
origin/main038dafba9: clean, treec6a2e736. The merged tree differs from main only inadopt-project.py,docs/cli.mdand the new test file. - against each open PR touching
adopt-project.pyordocs/cli.md, all clean:
| PR | Head |
|---|---|
| #1153 | 590ab96e |
| #1143 | 7af5fabc |
| #1125 | 2a53a8e6 |
| #1123 | 5c3908f8 |
| #1122 | c01c4d73 (_source_there) |
| #1121 | 1c517375 |
| #1120 | c55e55e3 |
| #1119 | bba168bd |
| #1118 | 2307036a |
| #1106 | 504a1dd6 |
| #1105 | b28f40de |
Findings
-
NIT-1 (test gap).
_git_refused's docstring says the directory check needs "only search permission". No row pins that, and M12, which demands read permission, survives all 31 rows.- The behaviour is right: 0100 and 0300 measured
plan ok, 0 gh. - Fix: add a row with the
.gitat 0100 (and 0300), withcloned: true, assertingplan okand an emptyghlog.
- The behaviour is right: 0100 and 0300 measured
-
NIT-2 (out of scope, adopt-project.py:
plan --sourceof a repository whose.gitis at mode 000 is refusedgit-failedwith no name for the.git, and a clone whose.git/HEADor whosegitdir:target is at mode 000 is called no longer whole and cloned again, where #1141 refuses a recorded clone's unreadable.gitby name (see #1099) #1151). These are still "no longer whole", with 1ghcall, identical to main:HEAD,objects/,refs/orconfigin.gitat 000;- a worktree's gitdir or common directory at 000;
- a submodule's
.git/modules/<m>at 000.
adopt-project.py:
plan --sourceof a repository whose.gitis at mode 000 is refusedgit-failedwith no name for the.git, and a clone whose.git/HEADor whosegitdir:target is at mode 000 is called no longer whole and cloned again, where #1141 refuses a recorded clone's unreadable.gitby name (see #1099) #1151 namesHEADand the.gitfile's git directory. Addobjects/,refs/andconfig: git's own "is a git directory" asksHEAD,objects/andrefs/, andconfiggives "unable to access '.git/config': Permission denied". -
NIT-3 (docstring).
_git_refusedsays a.gitwhose ownstatis refused "is in apaththis user cannot search". That is false for a.gitsymlink whose target is under a directory this user cannot search. Measured: withcloned: trueit is cloned again, 1 gh, as on main.- Fix: reword the sentence, and list the case in adopt-project.py:
plan --sourceof a repository whose.gitis at mode 000 is refusedgit-failedwith no name for the.git, and a clone whose.git/HEADor whosegitdir:target is at mode 000 is called no longer whole and cloned again, where #1141 refuses a recorded clone's unreadable.gitby name (see #1099) #1151.
- Fix: reword the sentence, and list the case in adopt-project.py:
-
NIT-4 (Sonar, gate passed). The PR adds two Sonar issues:
- S3776:
_recorded_sourcecognitive complexity is now 17 > 15 (adopt-project.py:6652). - S1192: "whether a repository is there" now appears 3 times (
:2090).
Fix: move the two-line guard into a helper (or into
_git_unreadable, returning None), and make the phrase a constant. - S3776:
Lane: openreposhape-4 (openRepoShape-4)
Two conflicts, both from #1180's partial-clone sentence beside this branch's unreadable-.git sentence in SOURCE_OVERRIDE_HELP: - adopt-project.py, SOURCE_OVERRIDE_HELP: the "one this user cannot reach, or one whose `.git` this user cannot read, a clone `plan` made included, as `source-unreadable`, never cloned again." sentence is kept, and #1180's "Either source, where it is a partial clone ..." sentence follows it. - docs/cli.md, the --source help of check and execute: regenerated with scripts/render-cli-reference.py, not edited by hand. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
|
RD1141: independent delta review of #1141 at No MUST, no SHOULD, one NIT. Scope: the merge 1. The delta is unchanged
2. Each conflict hunk keeps both sides
3. docs/cli.md
4. The order of the two refusalsProbe on the head (
Same 16 rows on main The order cannot flip the answer, which is why it is robust: with 5. Gate on a scratch merge of the head with current mainScratch merge
Total 991 passed, 3 skipped, 0 failed. The 3 skips are one reason: "the PowerShell line is printed, and powershell.exe exists, only on a Windows host" ( 6. Merge-trees (
|
| Mutant | Result |
|---|---|
MR1 the branch's .git phrase dropped from SOURCE_OVERRIDE_HELP, docs left |
KILLED, 5 failed / 23 (test_cli_reference.py: help vs file, committed reference vs generator, the generator's own --check, 2 checkout rows) |
| MR2 main's partial-clone sentences dropped from the help, docs left | KILLED, 5 failed / 23, the same rows |
MR3 docs/cli.md's execute paragraph put back to the pre-merge wrap |
KILLED, 5 failed / 23, the same rows |
MR4 the guard removed (refused = None), which is the order of the two rules falling to main's |
KILLED, 25 failed, 18 passed, 8 teardown errors: 9 of the PR's own 18 rows, and all 16 of my probe rows |
MR5 the guard asked after the cloned: false branch has read the clone |
KILLED, 12 failed / 43: the 4 given rows of the PR's file and 8 probe rows |
Findings
- NIT-1 (test gap, follow-up). No row in the repository pins a partial clone whose recorded
.gitis unreadable. The PR's 9 refused-by-name rows andtest_adopt_partial_clone_source.pyeach cover one rule, and the two rules now share--source's help andcheck/execute. The behaviour is right and cannot be flipped by a reorder (section 4), so this is a guard against a later refactor only. Fix: one parametrized row intests/test_adopt_unreadable_clone_git.py(or intest_adopt_partial_clone_source.py) that makesSource("nocheckout"), puts the.gitat mode 000 underneeds_a_mode, and assertsREFUSED source-unreadable, nosource-partial-clone, and 0ghand 0upload-packruns. My probe is at/workspace/projects/openRepoShape-worktrees/_lane4-scratch/tools/run/rd1141/probe/test_rd1141_probe.py(16 + 8 rows) and can be cut down to it.
Not re-reviewed, and not changed by the merge: RV1141's four NITs (the 0100/0300 row, the HEAD/objects/refs/config rows for #1151, the _git_refused docstring on a symlinked .git, and the two Sonar issues, of which SonarCloud's comment of 14:54Z shows 2 new issues, gate passed).
Lane: openreposhape-4 (openRepoShape-4)
…` before `git filter-repo`, so under a Latin-1 locale a source under a directory named with the byte 0xE9 is extracted, where filter-repo died printing that url as U+FFFD, and every branch of the source is still rewritten as before (#1247) (#1251) **Head `abeac66d` is a fix round for RV1251's review of `825923c7`** (#1251 (comment)): one SHOULD and two NITs, set out under "Review round 1" below. The sections after that describe this head. `_extract_leg` (`adopt-project.py`) now unsets the leg clone's `remote.origin.url`, in the clone's own config file (`--file <work>/.git/config`), right after its `checkout -B` and before `git filter-repo` runs. filter-repo then prints no url, so a Latin-1 stdout has no U+FFFD to choke on, and it still turns every remote-tracking branch into a branch first, so the legs are the ones main makes. ## The defect, reproduced on main - **Where:** main `b1f45fa7`, git 2.43.0, Python 3.12.3, git-filter-repo 2.47.0 (`--version` prints `a40bce548d2c`). - **Launcher:** pip's entry point, `#!.../frvenv/bin/python3`, WITHOUT `-E`. So any `PYTHON*` variable would reach filter-repo, and the reproduction sets none: no `PYTHONIOENCODING`, no `PYTHONUTF8`. - **Locale:** Latin-1, built as #1214 and #1106 build it (`localedef -c`, `LOCPATH`, `LC_ALL=en_US.ISO-8859-1`). - **Source:** `make_source_repo` under a directory `caf` plus U+00E9, made as the byte 0xE9. The probe runs `plan`, `check` and `execute --local-remote-dir` as the existing row does, but without its `PYTHONIOENCODING=utf-8:strict`. - **Result:** `plan` exit 0, `check` exit 0 (`plan ok`), and `execute` exit 2: ```text creating the leg repositories bare .../remotes/Northwind-spec.git bare .../remotes/Northwind-code.git REFUSED extracting the spec leg. THE EXACT COMMAND WAS: git filter-repo --paths-from-file .../work/spec-paths.txt --force (in .../work/Northwind-spec) exit 1 --- output --- Traceback (most recent call last): File ".../frvenv/bin/git-filter-repo", line 8, in <module> sys.exit(main()) File ".../git_filter_repo.py", line 4973, in main filter.run() File ".../git_filter_repo.py", line 4878, in run self._migrate_origin_to_heads() File ".../git_filter_repo.py", line 4462, in _migrate_origin_to_heads print(m) UnicodeEncodeError: 'latin-1' codec can't encode character '\ufffd' in position 237: ordinal not in range(256) --- end output --- ``` Both bare leg repositories were left behind. So the defect does not need a `-E` launcher: a person in a Latin-1 locale sets no `PYTHONIOENCODING`, and pip's launcher fails as well. **The existing row, `test_adopt_template_values.py::test_a_utf8_source_directory_still_adopts_with_local_remotes`, under Latin-1 on main:** - With pip's launcher it PASSES (1 passed). Its `PYTHONIOENCODING=utf-8:strict` reaches filter-repo through `execute`'s environment. - With a copy of that launcher whose only change is `-E` on the shebang (a model of the pipx launcher #1247 quotes), it FAILS with the traceback above. That matches W1196's log. - A check that `-E` drops the variable: with `PYTHONIOENCODING=bogus-codec`, pip's launcher dies at start-up, while the `-E` copy prints its version. - On this head the row passes under Latin-1 with both launchers. ## The fix, and the two candidates measured first **Candidate 1 in #1247, `git remote remove origin` before filter-repo, changes what the legs hold.** It is rejected. - `git remote remove` deletes `refs/remotes/origin/*` with the remote. filter-repo then no longer makes the source's other branches into branches, and never rewrites them. - **Measured** with `execute` on main and on candidate 1, over two ASCII sources with fixed dates. For each leg's bare repository the probe compared: every ref, every commit with its tree, parents and full message, and every reachable object. - **The standard source** (`make_source_repo`, one branch): identical. - **A source with a `release` branch** whose commit R changes `specs/001-feature/tasks.md`, no tag reaching it, and on `main` a `git cherry-pick -x R`: the spec leg's tip differs, `0316169e` on main against `82176493` on candidate 1, with the same tree `9ef5b676`. On main its message reads `(cherry picked from commit 407d03a0...)`, R as rewritten. Under candidate 1 it reads `(cherry picked from commit 30e9dd30...)`, R's id in the source. - **It also changes behaviour that `check` describes.** As a mutant, it fails the existing `test_adopt_split_branch.py::test_the_extraction_stops_at_a_source_branch_in_its_way` (both rows, `DID NOT RAISE CommandFailed`). With it, a source branch in git's way no longer stops the extraction. **Candidate 2, `LC_ALL=C.UTF-8` for the child, depends on the host.** - **Measured here:** under `python3 -E`, a pipe's encoding is `utf-8` under `LC_ALL=C.UTF-8`. It is also `utf-8` under `C`, `POSIX` and a locale name that does not exist, because CPython falls back to C and turns on its UTF-8 mode. - **But** it hands the child a locale by name and leans on that interpreter behaviour. It changes the locale of every git command filter-repo starts. And on Windows, `LC_ALL` does not set a pipe's encoding (read, not run). - No `PYTHON*` variable is a cure, because a `-E` launcher drops it. **What this PR does: unset the url only (`git config --file <work>/.git/config --unset remote.origin.url`).** It depends on no locale. It names the clone's own config file, so `GIT_CONFIG` in the environment cannot redirect it to a file of the person's (RV1251's SHOULD-1, below). It goes on from a clone that has no url to unset, and from no other failure (NIT-1, below). - filter-repo's `_migrate_origin_to_heads` (2.47.0) turns `refs/remotes/origin/*` into `refs/heads/*` first. Then it returns when `remote.origin.url` is not in the config, before the NOTICE that printed the url and before its own `git remote rm origin`. - **The legs are byte-identical to main's** in both probes above, the cherry-pick source included (`diff` of the dumps: empty). - **What stays in each work clone** (measured on this head, a plain `execute`): the url-less `[remote "origin"]` with its fetch refspec `+refs/heads/*:refs/remotes/origin/*`, and `branch.main.remote = origin` and `branch.main.merge = refs/heads/main`. Nothing reads any of it. `_push_the_leg` pushes to the leg's url, and `_mount_the_legs` mounts the clone by its path. RV1251 measured that the mounted submodules hash the same as main's. - The change is one `run` in `_extract_leg` (now with a `try`), with a comment. No pinned or template copy of `adopt-project.py` exists, and no document describes this step. ## Review round 1 (RV1251 at `825923c7`) **SHOULD-1, fixed: the unset named no config file.** `GIT_CONFIG` in the environment (a legacy variable git still reads) names the file a bare `git config` writes. - **Reproduced** with `GIT_CONFIG` naming a file that holds `[remote "origin"] url = ...` and the standard source, in `execute`: - main: exit 0, `adoption verified`, the file unchanged. - `825923c7`: exit 2, `REFUSED extracting the spec leg`, both bare legs made and nothing pushed, and the file empty. The first leg's unset took the only key out of it, and filter-repo, reading that empty file as the config, died with a `ValueError` in `get_config_settings`. (RV1251's file ended in an unset exit 5 instead.) - this head: exit 0, `adoption verified`, the file byte for byte as it was. - **Fix:** `run(["git", GIT_CONFIG, "--file", str(work / ".git" / "config"), "--unset", "remote.origin.url"], cwd=work)`. `--local` is no way round it: with `GIT_CONFIG` set, git 2.43 exits 129 (only one config file at a time). - **`work / ".git"` is a directory wherever `execute` reaches the clone:** - the clone is made by one `git clone -q <source> <work>`, and `work` is `work_root / <name>`, where `work_root` is `os.path.realpath(--work-dir)` or a `tempfile.mkdtemp` folder (`_work_root`): an absolute path; - an exported `GIT_DIR` does not move a clone (measured: the clone has its own `.git`); - an exported `GIT_WORK_TREE` does make the clone another shape (the git dir is `work` itself), but `execute` stops on it first, on main and on this head alike, at the legs' `git init --bare` (`REFUSED a git or gh command failed`); - a path whose directory is not there gives exit 255 (`could not lock config file ... No such file or directory`), which is refused. - **Not changed:** filter-repo's own `git config --list` also reads the file `GIT_CONFIG` names (the `ValueError` above came from it), so under `GIT_CONFIG` it sees that file's settings, as on main. On this head the run ends `adoption verified` with the legs main makes. **NIT-1, handled: a clone with no `origin` url.** A global `clone.defaultRemoteName = upstream` gives the clone a remote that is not `origin`, so `git config --unset remote.origin.url` has no key and exits 5 (measured, git 2.43). It can happen. - **Before:** main fails at `mounting the legs` (`No such remote 'origin'`), after both legs are pushed. `825923c7` failed earlier, at the unset (exit 5, an empty output block), with nothing pushed. - **Now:** `_extract_leg` goes on from exit 5 and from no other exit (`GIT_CONFIG_KEY_ABSENT = 5`). The run fails where main fails, not earlier and not worse. Any other exit of the unset (255 for a file git cannot lock, 129 for a usage it refuses) is refused as any failed command is. - **Not mended here:** `clone.defaultRemoteName` itself. `execute` fails on it on main too. It is a follow-up for its own issue. - Exit 5 also covers a config with more than one `remote.origin.url` (measured); that too goes on, as main does. **NIT-2, corrected in this body:** the claim that the fix "depends on no locale and no variable" (it now names its config file, so `GIT_CONFIG` cannot redirect it); the work clone's leftovers, now listed under "What stays in each work clone" (the url-less `[remote "origin"]`, `branch.main.remote` and `branch.main.merge`); and the note that #1141 conflicts, dropped: it merges clean at `0b8fb320`. ## Rows `tests/test_adopt_filter_repo_origin_url.py`, new, five rows: - **`test_filter_repo_starts_with_no_origin_url_and_every_branch`** (any locale). `_extract_leg` itself runs over an ASCII source with a second branch, with `run` spied at the `git filter-repo` call. When filter-repo starts, the clone has no `remote.origin.url` and still has `refs/remotes/origin/main` and `refs/remotes/origin/beside`. filter-repo's output holds neither `Removing 'origin' remote` nor the source path. Afterwards the clone has `refs/heads/beside`, and the leg holds the clone's HEAD. - **`test_execute_adopts_a_source_named_in_latin1_under_latin1`** (Latin-1). It runs `plan`, `check` and `execute --local-remote-dir` for #1247's source (a directory `caf` plus the byte 0xE9) in a Latin-1 locale the test builds as #1106 builds one. No `PYTHON*` variable is set except `PYTHONUTF8=0`. It asserts no traceback, `check` exit 0, `execute` exit 0 and `adoption verified`. - It skips, by name and with `localedef`'s words, where no Latin-1 locale can be built, and on Windows. It also skips where a name that is not UTF-8 cannot be made (`skip_where_a_name_cannot_be_not_utf_8`). - It builds its locale whatever the gate's locale is, so it fails on main under a UTF-8 gate too, which is what CI runs. - **`test_execute_leaves_the_file_git_config_names_alone`** (new, SHOULD-1). `execute` with `GIT_CONFIG` naming a file that holds a `[remote "origin"]` url: `adoption verified`, and the file is byte for byte as it was. The child gets none of the gate's locale variables, so the row runs alike under a UTF-8 and a Latin-1 gate. - **`test_a_clone_with_no_origin_url_goes_on_to_filter_repo`** (new, NIT-1). `_extract_leg` itself under a global `clone.defaultRemoteName = upstream`: the unset exits 5 (recorded as it is raised), the clone's `remote.upstream.url` is the source, and the leg is pushed. - **`test_an_unset_that_fails_for_any_other_reason_stops_the_leg`** (new, NIT-1). A `config.lock` in the clone makes the unset exit 255: `CommandFailed`, before `git filter-repo` and with nothing pushed. It needs no git-filter-repo. - The earlier rows' setup is now two helpers, `extract_the_code_leg` and `adopt_each_step`. The file is ASCII with no backslash escape; each non-ASCII character is built with `chr` or `bytes`. **Counts** (5 rows; the same file in each tree): | code | UTF-8 gate, pip and `-E` | Latin-1 gate, pip and `-E` | |---|---|---| | main (`_extract_leg` as on `dffc4226`; unchanged in `5926ccca`) | 4 failed, 1 passed (the GIT_CONFIG row passes: main leaves the file alone) | 4 failed, 1 passed | | `825923c7` (the previous head) | 2 failed, 3 passed (the GIT_CONFIG row and the exit-5 row fail) | 2 failed, 3 passed | | this head | 5 passed | 5 passed | The two earlier rows fail on main and pass here. The GIT_CONFIG row fails on `825923c7` and passes here. The exit-5 row fails on `825923c7` and on main (no unset to record) and passes here. The lock row passes on `825923c7`, which refuses every exit; it guards the new `except`. ## Mutants Each mutant was applied in a worktree at this head, the five rows plus the two `test_the_extraction_stops_at_a_source_branch_in_its_way` rows were run under UTF-8 and under Latin-1, and `adopt-project.py` was restored with its sha256 checked after each one (`RESTORED sha256 ok`, tree as before). | mutant | UTF-8 | Latin-1 | killed by | |---|---|---|---| | M1: drop the unset | 4 failed, 3 passed | same | the two earlier rows, the exit-5 row and the lock row (no unset to see) | | M2: the unset moved after `git filter-repo` | 2 failed, 5 passed | same | the two earlier rows | | M3: `git remote remove origin` instead | 5 failed, 2 passed | same | the any-locale row, the exit-5 row, the lock row and both split-branch rows | | M4: drop `--file <path>` | 1 failed, 6 passed | same | the GIT_CONFIG row | | M5: `--local` instead of `--file <path>` (exit 129 under `GIT_CONFIG`) | 1 failed, 6 passed | same | the GIT_CONFIG row | | M6: accept every exit of the unset | 1 failed, 6 passed | same | the lock row | | M7: accept none (no `try`) | 1 failed, 6 passed | same | the exit-5 row | | M8: `--file <work>/config`, a path with no file (exit 5, carried on) | 3 failed, 4 passed | same | the any-locale row, the Latin-1 row and the lock row | ## Gates All runs used `/usr/bin/python3 -m pytest -p no:cacheprovider` with one reused basetemp per run kind and `-o tmp_path_retention_policy=failed`, at most 2 pytest runs at a time. The environment had a stub `gh` first on PATH, `GH_TOKEN=bogus`, an empty `GH_CONFIG_DIR`, `GIT_CONFIG_GLOBAL` an empty file and `GIT_CONFIG_NOSYSTEM=1`, and no `GIT_ALLOW_PROTOCOL`, `GIT_SSH_COMMAND`, `GIT_CONFIG`, `LANES_*` or `PYTHON*` variable. filter-repo was pip's launcher (`a40bce548d2c`) unless a row says otherwise. **The tree gated is a scratch merge** of this head and `origin/main` `5926ccca` (built with `merge-tree` and `commit-tree`; its diff against main is this PR's two files). One hygiene row caught a path in the first attempt (`/home/someone/...` in the new test, `test_no_committed_file_names_a_host_absolute_path`); the row now names `https://example.invalid/keep-me.git`, and everything below ran on the final head. **Gate A, UTF-8 (`LC_ALL=C.UTF-8`), one pytest, 23 files:** **1518 passed, 3 skipped** on the scratch merge. - the new file, `test_adopt_template_values.py`, `test_adopt_split_branch.py` and `test_adopt_e2e.py`; - the other four files that name `_extract_leg` or `_push_the_leg`: `test_adopt_filter_repo_failure.py`, `test_adopt_history_gitlinks.py`, `test_adopt_late_failure.py` and `test_adopt_plan_escapes.py`; - `test_adopt_check_throwaway_repos.py`, `test_adopt_partial_clone_source.py`, `test_adopt_pins.py`, `test_adopt_git_rm_bytes.py` and `test_adopt_plan_partial_clone.py` (new on main); - `test_repo_hygiene.py`, `test_path_guard_hygiene.py`, `test_cli_reference.py` and `test_scaffold_leg_path_collisions.py`; - the files that scan every test file: `test_no_auto_maintenance.py`, `test_non_utf8_git_failures.py`, `test_git_output_decoding.py`, `test_setup_project_repo_misdrawn.py`, `test_shape_advisory.py` and `test_openreposhape_package.py`. - **Main `5926ccca`, the same 22 files without the new one:** 1513 passed, 3 skipped. The head's extra 5 are the five rows. - **Skips (`-rs`):** the same 3 on both, none for lack of git-filter-repo: `test_adopt_check_throwaway_repos.py:482` (a disk that keeps case) and `test_no_auto_maintenance.py:222` x2 (`GIT_CONFIG_GLOBAL` set). **`tests/test_shape_doctor.py`, UTF-8, 4 shards from a bash script** (449 ids): **Head (scratch merge):** 118, 110, 118 and 100 passed, with 3 skipped in the last shard (the PowerShell line, Windows only): 446 passed, 3 skipped. RV1251 and W1247 measured 446 passed and 3 skipped on main at `11560eb9` and `b1f45fa7`; it was not re-run on `5926ccca`, because this PR touches nothing the doctor reads. **Latin-1** (the locale built as above, `PYTHONUTF8=0`), the new file and `test_adopt_template_values.py`: | run | main `5926ccca` | this head (scratch merge) | |---|---|---| | new file + `test_adopt_template_values.py`, pip launcher | 4 failed, 69 passed, 5 skipped (template values only) | 4 failed, 74 passed, 5 skipped | | the same, `-E` launcher | 5 failed, 68 passed, 5 skipped (template values only) | 4 failed, 74 passed, 5 skipped | - **The 4 failures are the same rows on main and on this head, under both launchers.** All four already fail on main and are #1214's to judge, not this PR's: `test_plan_refuses_a_flag_no_file_can_hold` at `--elected-by` and `--reference` with U+DCE9, `test_plan_and_check_say_the_same_sentence`, and `test_execute_refuses_a_local_source_directory_that_is_not_utf8` (under Latin-1 the byte 0xE9 is valid text). - The fifth failure on main under `-E` is `test_a_utf8_source_directory_still_adopts_with_local_remotes`, which this head fixes. ## Merge-trees `git merge-tree --write-tree` of this head against: - **main:** clean against `5926ccca`, and the merged `adopt-project.py` keeps the `--file` call. - **#1177 (`5208e874`) and #1225 (`73c9f57e`)**, the only open PRs that touch `_extract_leg`: both clean, each keeps the `--file` call. - **#1141 (`0b8fb320`):** clean. The earlier note that it conflicts is stale and is dropped. - RV1251 merge-tree'd the 27 other open PRs that touch `adopt-project.py` against the previous head: 26 clean, and #1237 conflicts in `tests/test_adopt_check_throwaway_repos.py` with main alone. This round re-ran main, #1177 and #1225 (the only PRs that touch `_extract_leg`) and #1141 (the one the earlier body got wrong). ## Not done, and limits - Windows, another one-byte locale (cp1252 included) and a real pipx install were not run. The `-E` launcher here is a one-line copy of pip's. - **Read from the code, not run:** filter-repo prints nothing else that holds the url. Its other prints of `origin` are under `--sensitive-data-removal`, which `execute` never passes. Closes #1247 Lane: openreposhape-4 (openRepoShape-4) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>



A recorded source whose
.gitthis user cannot read is now refused assource-unreadable, naming that.git.checkandexecuteno longer call a cloneplanmade with its.gitat mode 000 "no longer whole", and no longer clone it again from GitHub. This uses the code and wording #1072 gave a recorded directory this user cannot enter (#918). No refusal code is new.This PR lands only after the v1.2.0 tag.
Reproduced on main
4ed7c7bf5Measured as uid 1000, with git 2.43.0 and Python 3.12.3. The run had a stub
ghfirst on PATH that records and refuses every call,GH_TOKEN=bogus, an emptyGH_CONFIG_DIR,GIT_SSH_COMMAND=falseand an empty global git config. Remotes were local, under--local-remote-dir.The source was a clone with
originathttps://github.com/testorg/Thing.git, soplanrecordedsource.repository: testorg/Thing.plan --source <it>wrote the plan, andcloned:was set by hand where the row needs it. Thenchmod 000 <it>/.git.checkon mainexecute --yeson main.gita directory at 000,cloned: trueplanmade that is no longer whole (git finds no repository there), so itssource.repository, testorg/Thing, is cloned again from GitHub", thencloning testorg/Thing, thensource-unclonable; 1gh repo cloneghcall.gita directory at 000,cloned: falseREFUSED git-failed: git rev-parse --is-inside-git-dir in <it> exited 128: fatal: not a git repository (or any of the parent directories): .git; 0gh.gita file (gitdir:) at 000,cloned: trueghcall.gita file at 000,cloned: falsegit-failed,fatal: error opening '<it>/.git': Permission denied; 0gh.git,cloned: trueandfalseplan ok; 0ghadoption verified; 0ghOn this branch
Each of the four unreadable rows now exits 2 before anything is cloned or made, and
ghis never asked:cloned: false, what cannot be told is "whether a repository is there"..gitthat is a file is givenchmod u+ron the file andchmod u+rxon every directory above it.plan ok,adoption verified, 0ghcalls.What changed
All in
adopt-project.py, plus the regenerateddocs/cli.mdand one new test file._git_refused(path)(new, after_not_a_whole_clone) asks the.gitbefore git runs, through_is_aand_is_there(EveryPath.is_file()/is_dir()/exists()guard in the shipped tools answers False to any OSError on Python 3.14 (3.12 and 3.13 raise PermissionError), so a directory this user cannot search reads as ABSENT: validate-pins.py skips its workflow@<sha>check, update-shape.py sayscopy-missingandupdate-pin-missing("adopt the project") for files that are there, the doctor says DRIFTED (5 copy-missing), and bump-leg.py's remaining guards, and every other tool's, skip or misreport instead of refusing by name #430):HEADis there. Thatstatneeds only the directory searched, and nothing ofHEADitself..gitit cannot read, as it does of an emptied clone. With a directory.git, it also climbs to the parent, which may be another repository.(the OSError, whether the .git is a file), or None. None is returned where nothing is there, where the.githolds noHEAD(git's "no repository", as before), or where the.gitis neither a directory nor a file..git's ownstatis refused. Then the directory itself cannot be searched, and adopt-project.py and family.py: a--sourcethis user cannot enter, a UTF-8 plan checked under the C locale, and a member branch that is not UTF-8 still end in a traceback or a false refusal #918'scwd=guards name it, as before.Plan._recorded_sourceasks_git_refusedright after asking whether the path is there, before it branches onsource.cloned. That coverscloned: trueandfalsealike. The newPlan._git_unreadablewords the refusal. The insertion sits clear of adopt-project.py: under a one-byte locale such as Latin-1,checkreads a plan written in UTF-8 whose source is there, asking the plan's UTF-8 bytes where the spelling as written names nothing, rather than refusingplan-source-gone(#1078) #1106's change to thethereline._source_unreadabletakes the cure for aPermissionErroras a keyword,permission=. It defaults to the directory'sSOURCE_UNREADABLE_PERMISSION, so every other caller is unchanged. A.gitfile is given the newSOURCE_UNREADABLE_FILE_PERMISSION.SOURCE_OVERRIDE_HELP(the--sourcehelp ofcheckandexecute) now says "one this user cannot reach, or one whose.gitthis user cannot read, a cloneplanmade included, assource-unreadable, never cloned again".source-unreadable.docs/cli.mdwas regenerated withscripts/render-cli-reference.py.AGENTS.mdor template sentence names this case.AGENTS.md's codes table is untouched, since no code is added.--sourceitself is unchanged. The issue names the recorded clone (see Out of scope).Rows:
tests/test_adopt_unreadable_clone_git.py(new, 18)4ed7c7bf5test_a_recorded_git_this_user_cannot_read_is_refused_by_name[check,execute x given,cloned x directory,file](8, real mode 000,needs_a_mode)git-failed, orsource-unclonableafterghwas asked)test_a_recorded_clone_whose_git_is_refused_is_not_cloned_again(in-process, thestatof.git/HEADrefused, so it runs as root and on Windows too)checkread the clone, exit 0test_a_git_that_can_be_read_or_is_not_there_is_gits_question[5](the helper)_git_refusedon maintest_a_recorded_git_this_user_can_read_is_read_as_before[given,cloned x directory,file](4 controls)In total: 14 failed and 4 passed on main; 18 passed on this head. The permission rows skip as root and on Windows, as the repository's other permission rows do. The rows'
at_mode_000asserts that the mode bites (os.access(R_OK)is False), and gives the mode back in afinally.Mutants
Each was applied alone to the committed
adopt-project.py(sha256b2b41532c3a5...) and its rows run. The file was then restored, and its sha256 checked equal to the commit's every time.git statuswas clean after all of them. All 8 were killed.refused = None).git(aPermissionErrorreturned where the.gitreads)ghasked anyway (Source.cloneof the record asked, then the same refusal raised)ghlog is not empty) and the in-process row (2ghcalls).gitwhose ownstatis refused taken as the.git'stest_a_recorded_source_this_user_cannot_enter_is_refused_by_name[check-False],[check-True]and[execute-True](the directory must still be named by itself).gitfile given the directory's cure.gitfile never opened.gitwith noHEADrefused..._is_gits_question[a-git-directory-with-no-head]clonedandgivenGate
Run at this head
7a2711d0, 16:35:44Z to 16:46:37Z, with/usr/bin/python3 -m pytest -p no:cacheprovider -o tmp_path_retention_policy=failed. The environment hadEMAILset;GIT_ALLOW_PROTOCOL,GIT_SSH_COMMANDand theLANES_*variables unset; an emptyGIT_CONFIG_GLOBALandGIT_CONFIG_NOSYSTEM=1; test author and committer identities; and a stubghfirst on PATH, withGH_TOKEN=bogusand an emptyGH_CONFIG_DIR.test_adopt_unreadable_clone_git.py,test_adopt_source_clone_record.py,test_adopt_unenterable_source_and_locale.py,test_adopt_unsearchable_source.py, #1091'stest_adopt_root_shape_paths.pyandtest_adopt_root_symlinks.py,test_pin_source_unreadable.py,test_cli_reference.py,test_agents_md_codes.py,test_repo_hygiene.py,test_scaffold_leg_path_collisions.pytest_shape_doctor.py, 8 shards (62, 56, 54, 56, 60, 57, 58, 45 of 448)test_pin_source_unreadable.pywas added from a grep oftests/forsource-unreadable,_source_unreadable,SOURCE_UNREADABLE_,_recorded_sourceandSOURCE_OVERRIDE_HELP. pyflakes onadopt-project.pyand the new test file reports nothing.#1111's path-guard hygiene test (lane 1's rule, #473 16:26Z)
The guard asks the
.gitand itsHEADthroughadopt-project.py's own_is_aand_is_there. It adds no bareis_file,is_dir,exists,is_symlinkoros.pathtest.tests/test_path_guard_hygiene.pyfrom #1111's headb4484647was run on a scratch merge of main4ed7c7bf5, #1111 and this head:_is_aor a reasoned allowlist covers it, and aproject.yamllinked into an unsearchable folder isyaml-unreadable(#430) #1111 + this head: 1 failed, 48 passed. The one failure istest_no_bare_guard_outside_a_stat_helper_or_the_allowlist[adopt-project.py], for three sites:_PlannedRoot._no_directory:self._git().is_dir(here)_PlannedRoot._into_git:self._git().is_dir(here.rpartition("/")[0])_clone_ref_store:(probe / CLONE_GIT_DIR / "reftable").is_dir()_is_aor a reasoned allowlist covers it, and aproject.yamllinked into an unsearchable folder isyaml-unreadable(#430) #1111 alone, without this head: the same three sites, the same 1 failed and 48 passed. They come from adopt-project.py:checknames what the clone's.githolds as git keeps refs, and asks a pin's file through the links the root keeps, so a link into.gitor onto a pin's file is refused before any leg or adopted with the copy beside it (#904) #1091 (4ed7c7bf5, adopt-project.pycheck: a root-kept symlink into the clone's.git, or through a kept link to a declared pin's file, passes, or is refused for a reason that is not true #904), which landed after tests/test_path_guard_hygiene.py and shape-doctor.py: a bare is_file/is_dir/exists guard in a shipped tool fails by file and line unless its_is_aor a reasoned allowlist covers it, and aproject.yamllinked into an unsearchable folder isyaml-unreadable(#430) #1111's head was cut. tests/test_path_guard_hygiene.py and shape-doctor.py: a bare is_file/is_dir/exists guard in a shipped tool fails by file and line unless its_is_aor a reasoned allowlist covers it, and aproject.yamllinked into an unsearchable folder isyaml-unreadable(#430) #1111 needs them allowlisted or fixed before it lands. This PR does not touch them._is_aor a reasoned allowlist covers it, and aproject.yamllinked into an unsearchable folder isyaml-unreadable(#430) #1111 + this head, with only those three sites allowlisted in a scratch copy of the test: 49 passed. This head adds no site.Merge-trees
Each was run as
git merge-tree --write-treeof this head against:origin/main4ed7c7bf55e784b0ffix/1020-bad-id-first-problem2a53a8e676af7e33fix/1061-makefile-follow-ups5c3908f812859bc5fix/1003-adopt-unenterable-source(_source_there)c01c4d736222542ffix/1047-next-make-bootstrap1c517375e294504ffix/1035-source-branch-shownc55e55e346bcc762fix/1083-check-asks-the-split-clone9b998936dcc12164fix/1089-named-link-skip-testsc60066cbtests/test_adopt_root_shape_paths.pyonly, which this head does not touch; #1118 conflicts there withorigin/mainalone tool3-1078-one-byte-locale(_recorded_source'sthereline)504a1dd6e59c5cf4fix/1048-makefile-follow-up-gnu-read-orderb28f40de67d4fec4These are every open pull request that changes
adopt-project.py,docs/cli.mdor the new test file, as listed at 16:33Z.Out of scope
Each was measured on this head, and none is changed here:
plan --source <a repository whose .git is at 000>, and socheck --sourceandexecute --source:REFUSED git-failed: ... fatal: not a git repository, exit 2. Noghis asked and nothing is cloned, but the refusal does not name the.git. The issue names the recorded clone only. A follow-up could give_top_levelthe same guard..gitdirectory whoseHEADis at 000 (cloned: true): still "no longer whole (git finds no repository there)", cloned again, 1ghcall..gitfile whose git directory is at 000 (cloned: true): the same, 1ghcall..gitthe issue measured. Naming every file git might be refused is a wider question, left for a ruling._is_aor a reasoned allowlist covers it, and aproject.yamllinked into an unsearchable folder isyaml-unreadable(#430) #1111 against main: see above (adopt-project.py:checknames what the clone's.githolds as git keeps refs, and asks a pin's file through the links the root keeps, so a link into.gitor onto a pin's file is refused before any leg or adopted with the copy beside it (#904) #1091's three bare guards).git merge-treeof adopt-project.py, shape_materialize.py and tests: a named bad link left out of the copy and pin findings takes no other path's finding with it, and the docstrings and the materializer's refusal name the #979 exception and what helps it (#1089) #1118's headc60066cbwith main alone conflicts intests/test_adopt_root_shape_paths.py. This head does not touch that file.Closes #1099
🤖 Generated with Claude Code
Lane: openreposhape-4 (openRepoShape-4)
Summary by Sourcery
Refuse recorded sources with unreadable
.gitmetadata before repository validation or recloning, and clearly explain how to make them accessible.Bug Fixes:
.gitcannot be read assource-unreadable, naming the inaccessible path and preventing misleading clone recovery or GitHub recloning.Enhancements:
.gitdirectories and files consistently while preserving existing behavior for readable repositories and sources that cannot be entered..gitfiles and directories.Documentation:
.gitmetadata are refused without recloning.Tests:
.gitdirectories and files, readable controls, no-reclone behavior, and helper path-detection cases.