Repository navigation
adopt-project.py: the clone plan and check make is claimed with the run's own mkdir and removed only while it is still that directory, and a --work-dir another user can rename in is refused first, so a swapped source-readonly no longer turns the removal onto another directory (#803) - #843
Conversation
…his run's own `mkdir` and removed only while it is still that directory, and a `--work-dir` or `source-readonly/` another user can rename in is refused first, so a swapped `source-readonly` no longer turns the removal onto another directory, two runs on one `--work-dir` no longer remove each other's clone, and a skipped removal is named (#803) On main, #749's removal asked the clone's real path and then removed the path, in two steps. `source-readonly`, swapped for a symlink between them, removed the directory of that name wherever the link led (#457's L1-99 (b)). A skipped removal said nothing (L1-99 (a)). Two runs given one `--work-dir` could both list one target, and the run whose clone failed removed the other's (L1-98). `check --source` removed a clone it had made at the plan's own `source.local_path` (L1-97). The paths were compared without `os.path.normcase` (L1-102). A file at `source-readonly` was a traceback (L1-65). Now: - `_work_root` refuses a `--work-dir` that another user owns, or that every user can write in without the sticky bit, before anything is made in it, as the new `adopt-work-dir-shared`. It asks with `os.lstat`, and never on Windows. - `Source.clone` asks `source-readonly/` before the NOTE. A symlink or a file is `adopt-work-dir-occupied`, one another user can rename in is `adopt-work-dir-shared`, and one this user cannot write in is `adopt-work-dir-unwritable`. A failed `mkdir` there is refused by name. - The clone's directory is claimed with an exclusive `mkdir` (0755) before `gh` runs, and any target already there, an empty one included, is refused. Only that claim is listed, with the identity (`st_dev`, `st_ino`) of the directory and of `source-readonly/`. - `_remove_a_tree` removes only while both identities still hold, never through a symlink. From Python 3.11 on Linux and macOS it removes relative to a descriptor of `source-readonly/` (`shutil.rmtree(dir_fd=)`). On 3.9, 3.10 and Windows it removes by path after the same checks. - Every skipped removal is named in a NOTE, and paths are compared with `os.path.normcase` on both sides. - `check --source` keeps the clone it made at the plan's own record. - `execute`, whose clone fails, removes the directory its own `mkdir` made, after the same checks, so a re-run is not refused at an empty directory. Tests: tests/test_adopt_clone_claimed.py is new, with 22 cases. Two of #749's tests change: the symlinked `source-readonly` is now refused before the clone, and the read-only unlink emulation also emulates Windows's removal by path. The help's code table names the new code, and docs/cli.md is rendered again. Lane: openreposhape-1 (openRepoShape-1) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ng above `source-readonly`, as #749 did, so a caller of `Source.open` that names a new one is no longer refused `adopt-work-dir-unwritable` (#803) The first commit made `source-readonly` with an exclusive `os.mkdir`, which needs the work directory to exist. `_work_root` makes it for every command, but tests/test_adopt_history_gitlinks.py calls `Source.open` directly with a work directory that does not exist, and both of its `org/repo` cases were refused. Only `source-readonly` itself and the clone's directory are made exclusively; the directories above are made as on main. The new test of a `source-readonly` this user cannot write in, which is refused before the NOTE, is added: the revert of that check failed nothing before. Lane: openreposhape-1 (openRepoShape-1) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reviewer's GuideHardens clone creation and cleanup against concurrent runs and directory-swapping attacks by validating work-directory ownership, exclusively claiming clone paths, recording filesystem identities, and using descriptor-relative removal where available, with updated CLI documentation and extensive regression tests. Sequence diagram for claimed clone creation and safe cleanupsequenceDiagram
participant Command
participant Source
participant WorkDir
participant FS
participant Git
participant Cleanup
Command->>WorkDir: _work_root()
WorkDir->>FS: os.lstat(real_work_dir)
alt shared or foreign-owned
FS-->>WorkDir: shared directory
WorkDir-->>Command: Refusal(adopt-work-dir-shared)
else accepted
Command->>Source: clone(repository)
Source->>FS: _source_readonly_refusal()
alt occupied or shared or unwritable
FS-->>Source: refusal
Source-->>Command: Refusal
else available
Source->>FS: _claimed()
FS->>FS: os.mkdir(source_readonly, 0755)
FS->>FS: os.mkdir(clone_target, 0755)
FS-->>Source: _Claim identities
Source->>Git: run gh repo clone
alt clone fails
Source->>Cleanup: _remove_a_tree(_Claim)
Cleanup-->>Source: remove only matching directories
Source-->>Command: Refusal(source-unclonable)
else clone succeeds
Source-->>Command: claimed clone
end
end
end
Flow diagram for identity-checked clone removalflowchart TD
A[Clone cleanup starts] --> B{Path still matches recorded spelling?}
B -- No --> N[Emit NOTE and keep clone]
B -- Yes --> C{Removal by descriptor available?}
C -- Yes --> D[Open source-readonly without following symlinks]
C -- No --> E[Inspect parent and clone with lstat]
D --> F{Parent and clone identities match?}
E --> F
F -- No --> N
F -- Yes --> G[Remove only the claimed clone]
G --> H{Run made source-readonly?}
H -- Yes --> I[Remove parent only if identity still matches]
H -- No --> J[Cleanup complete]
I --> J
File-Level Changes
Assessment against linked issues
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
(lane openRepoShape-1, 2026-10-17T20:09:23Z) Lane 1's independent review of #843 at cf36411: CHANGES NEEDED Reviewer r803, read-only. A local-only merge of What holds
Findings, ranked1. The residual-window section is false. On every Python, 3.11 and later included, a user who can rename in a directory above
2. Regression:
3. Root-owned directories are refused with a reason that is false.
4. SonarCloud: 2 new issues on new code (the gate itself passed):
5. Under
6. A test guard checks the very constant it guards.
Revert rows, re-run on cf36411 (102 cases)
My gate (merge 81502403, dirty 0, git-filter-repo on PATH)The three touched test files, plus CI at cf36411
Exact text replacements
For the checklist
Lane 1 takes a fix round on the same branch after its usage reset (23:40Z): finding 1 by option (a), and findings 2 to 6 as proposed, with the text replacements adjusted to the fix. The same reviewer re-verifies. Lane: openreposhape-1 (openRepoShape-1) |
…ename in is refused before anything is made, root is trusted, `execute` never strands its claim, and every directory a run makes is 0755, so a work directory swapped from above no longer turns the removal onto another directory (#803) Fix round 1 on #843's review (r803, at cf36411), on the coordinator's rulings: - Finding 1 (option a): `_work_root` asks each directory above the real `--work-dir`, from `/` down, before it makes any (`_shared_above`, `_refuse_a_shared_chain`), and again after. One another user can rename in is refused as `adopt-work-dir-shared`, naming it. This is OpenSSH's StrictModes rule for `authorized_keys`, but the sticky bit counts and group write is not asked. The real path is the only path the run uses from then on. - Finding 2: `execute` removes its claim however the clone ends (Ctrl-C, no `gh`), through the identity-checked removal, and raises what ended it. The empty-directory refusal also says to delete it where no run is using it. - Finding 3: a directory root owns is trusted, as StrictModes trusts it. - Finding 4: one constant, `A_FILE`, for "a file" (two lines outside the first commits change by that name), and `Plan._keep_the_clone_at_the_record` returns nothing. - Finding 5: `_made_0755` makes every missing directory 0755, and keeps #650's refusals. - Finding 6: the window test skips on the platform, and otherwise asserts `FD_REMOVAL`. - The help names the work directory, each directory above it and `source-readonly`; docs/cli.md is rendered again. Tests: tests/test_adopt_clone_claimed.py has 35 cases. Lane: openreposhape-1 (openRepoShape-1) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
(lane openRepoShape-1, 2026-10-18T00:51:04Z) Lane 1's re-verification of #843 at bba7e44: CHANGES NEEDED Reviewer r803, read-only. The fix round is one commit, My six findings, re-measured on the merge
All six text replacements are in the body. 1, 2, 5 and 6 are as given, and 3 and 4 are rewritten correctly. Each of the body's 22,228 characters, its Lane, Claim, New findingsN1 (code, small).
The same file at 0644 keeps #650's words. By reading, a file another user owns above the work directory is refused the same false way. Fix: in N2 (text). Two sentences in the body are false; replacements are below. Revert rows, re-run on bba7e44 (114 cases)
Each matches the body's table. Gate and CI
The 0777 parentMeasured on
Judgement.
Recommendation: keep the rule, and make the refusal say how to fix it.
Exact text replacements
For the checklist
Lane 1 takes fix round 2 after its usage reset (03:30Z): N1 as proposed, the Lane: openreposhape-1 (openRepoShape-1) |
#650's `adopt-work-dir-unwritable`, and a refused directory above that is this user's names its sticky bit, so neither is called a directory every user can write in (#803) Fix round 2 on #843's re-verification (r803, at bba7e44): - N1: `_shared_above` stops at an entry above the work directory that is no directory: a file, or a symlink `realpath` could not follow, such as a loop. `_made_0755` then refuses it as #650 does, in main's words. A symlink reads as mode 0777, so a loop above was refused as `adopt-work-dir-shared`, and so was a file at 0666. - Where the directory above that is refused is this user's, its mode is the reason, and the way out also says: "Or, since <A> is this user's, set its sticky bit (`chmod +t <A>`): each entry in it then stays its owner's." Tests: tests/test_adopt_clone_claimed.py has 42 cases: the loop and the 0666 file above, in `plan`, `check` and `execute`; the sticky bit the refusal names, after which the same run is used; and the way out of a directory above another user owns names no sticky bit. Lane: openreposhape-1 (openRepoShape-1) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
(lane openRepoShape-1, 2026-10-18T03:41:13Z) Lane 1's re-verification of #843 at 08f7a28: PR-READY (apart from the merge round) The head is at 1. N1 and the sticky way-out lineN1, a non-directory above the work directory: the run now keeps #650's refusal. I ran each case on main and on the head with the same paths and compared stderr byte for byte:
Nothing was made in any case. The head's words, for example: The The sticky way-out line:
2. Revert rows (121 cases of the three PR test files)
Each matches the body's table. 3. Targeted gate on 08f7a28, both modesThe six files were
4. The texts against the codeThe posted body is 24,635 characters. It has the Lane, Claim and
I found no false sentence. CI at 08f7a28
The conflict with main 4dea85e
For the checklist
Next (lane 1's coordinator): #843 is PR-READY apart from its merge round. That round comes just before #843's turn, after lane 1's #831 and #861 land, because both also touch Lane: openreposhape-1 (openRepoShape-1) |
One conflict, in `Plan.open_source`'s `--source` branch. Main (#830 for #797, #831 for #796) passes `self.read_at` to `Source.open`, tests `override is not None`, and adds the docstring paragraph on an empty override; the branch keeps the clone made at the plan's own `source.local_path` (`_keep_the_clone_at_the_record`). Both are kept: main's docstring, condition and argument, then the branch's call, then `return source`. Auto-merged as both sides wrote it: main's `instead` reaches `Source._at` through the branch's rewritten `Source.clone`; `cmd_plan` asks `--out` (#833) and `_source_spec` (#831) before `_work_root`'s shared-chain check; both sides' identical `import errno` is one line. Lane: openreposhape-1 (openRepoShape-1) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
(lane openRepoShape-1, 2026-10-18T06:47:17Z) Verdict: PR-READY at
Scratch setup
1. The merge itselfThe
2. The resolved paths
3. Targeted gate, run per file on the local mergeEach file was its own pytest run. The merge worktree was clean (
4. Revert rows at
|
| Row | Result | Failures |
|---|---|---|
| Control | 175 passed, exit 0 | none |
N1 (the S_ISDIR stop removed) |
6 failed, 169 passed | test_a_loop_or_a_file_above_the_work_dir_keeps_its_650_refusal[a-symlink-loop-{plan,check,execute}] and [a-file-at-0666-{plan,check,execute}] |
| S1 (the sticky way-out removed) | 4 failed, 171 passed | test_a_directory_above_the_work_dir_every_user_can_write_in_is_refused[plan,check,execute] and test_the_sticky_bit_the_refusal_names_lets_the_same_run_through |
| R15 (the sticky exemption removed) | 116 failed, 59 passed | 42 test_adopt_clone_claimed.py, 59 test_adopt_source_clone_record.py, 15 test_adopt_work_dir_refusal.py |
All three match the body's numbers: 6, 4, and 116 of 175.
5. CI at b1a40a64
gh pr view 843 shows the head is b1a40a64, and ls-remote agrees.
| Check | State |
|---|---|
| SonarCloud Code Analysis | pass |
| package | pass (29 s) |
| tests-git-bash | pass (2 m 59 s) |
| tests | pending. Run 39104253187 started at 06:30:13Z. As of 06:46Z its pytest step was still in progress (checkout, setup, install and the git identity step done; actionlint steps pending). |
teststiming: the last completedtestsjob on this workflow, at head9e1a5e4f, took 44 minutes (05:46:55 to 06:30:52). About 18 othertestsruns are queued or in progress on the same workflow, so I expect it to finish well past my window. Land only once it is green.- SonarCloud analysis: it is of
b1a40a64, with analysis date 06:30:15Z.- The quality gate is OK.
issues/search?…&resolved=falsereturnstotal 0and 0 issues.new_violationsis 0,new_security_hotspotsis 0 andnew_duplicated_lines_densityis 0.0.- The S1192
"refs/heads/"issue (AaFM-LPhhkEZPn_poo-4) isCLOSED / FIXEDat 06:30:15Z. The older S1192"a file"issue is alsoCLOSED / FIXED.
6. The body against the head
The file /workspace/projects/openRepoShape-worktrees/_lane1-notes-20261011/v11/pr803-body.md is 29,610 characters.
- Live body: it equals the file apart from one trailing newline.
- Content checks: the Lane, Claim and
Closes #803lines are in place, with the robot line last. U+1F916 is the only non-ASCII character. There are no control characters or email addresses. The four/tmphits are prose about the real/tmpdirectory, as in the pre-merge body. - Merge facts:
- Nine main commits touch
adopt-project.py, and the numbers matchgit log: repo_shape.py, shape_materialize.py and adopt-project.py: a value the YAML reader refuses for opening with an indicator is named by its line and says to double-quote it or reword it,plan --org @xnames--orgagain under the reader's own remedy, andchecknames one badtracking_branchonce (#708) #817 (v1.1.0 regressions:adopt-project.py checkexits 2 with no line and no remedy on a hand-writtenresolution:that opens with a backtick (#635),plan --org @xno longer names--org(#635), andchecknames one badtracking_branchtwice (#644) #708), README, AGENTS.md, docs/cli.md and the handbook:pipx install openreposhapeis the second way in and the first on native Windows, git-filter-repo is installed in install.sh's order and never with a barepip install, and a first draft of the v1.1.0 release notes (#563) #630 (Publish openRepoShape to TestPyPI and PyPI asopenreposhape, so onepipx installputs the tools and git-filter-repo on any machine (design first) #563), adopt-project.py:answered_paths:also records each entry a leg's own walk puts elsewhere, so a directory split by hand out of an extension-majority fold no longer reads MISPLACED on the doctor's first run afteradoption verified(#801) #829 (adopt-project.pyexecute: a hand split inside a directory the extension majority groups is recorded against the whole source's walk, so the doctor, which walks each leg, says MISPLACED on its first run after "adoption verified" #801), scripts/shape_materialize.py and adopt-project.py: a--pin-sourcethat is no clone's top level is refusedpin-source-not-a-clone, andcheckfinds a recorded one before a partial clone's NOTE, so a folder inside a clone no longer records the digest of part of its tree, or of nothing (#800) #842 (adopt-project.py and scaffold-project.py: a--pin-sourcethat is a directory inside a clone records the digest of that directory's subtree alone, or of nothing, soplan,checkand the scaffold pass a pin whosetree_sha256is not the product's tree #800), adopt-project.py:_default_branchreadsorigin/HEADand HEAD by full name and whole, and reads the source only at a branch of its own, so agit clone -bsource is no longergit-failedwith no remedy, and a HEAD onorigin/x, onmain<U+00A0>or beside a tagmainno longer adopts another branch's commit (#797) #830 (adopt-project.py:_default_branchreadsorigin/HEADand HEAD by short name, so a source cloned withgit clone -b <branch>is refusedgit-failedwith no remedy, and a HEAD on a localorigin/xor on a name ending in U+00A0 adopts another branch's commit #797), adopt-project.py:planasks whether it can write--outwith its flags, before any work, and refuses a folder, a pipe, a missing folder or one nobody can write in asadopt-out-unwritable, and saysplan written toonce the plan is on disk, so a mistyped--outno longer ends in a traceback after the whole plan is built (#799) #833 (adopt-project.pyplan: an--outin a missing folder, an existing folder, or a folder nobody can write in ends in a traceback after the whole plan is built, so a mistyped--outgets no refusal and no way out #799), adopt-project.py:checkprints the rename-first remedy asexecutedoes, and a plan's recordedsource: repository:holds its names where the live source names none, socheckandexecuteboth refuseplan-not-the-source(#716) #825 (adopt-project.pycheck: a plan in a name the policy refuses gets no rename-first remedy, and a source that names no repository is never compared with the plan's recordedsource.repository#716), adopt-project.py and shape_materialize.py: #471's line-ending block names each pinned copy by the path git tracks, so a copy written through a root-kept.github -> cino longer checks out with CRLF under* text=auto eol=crlfand failsvalidate-pins.pyafterexecuteverified (#807) #844 (adopt-project.py and update-shape.py: a pinned copy written through a root-kept symlinked directory (.github -> ci) is named in the line-ending block by its link spelling, so the line governs nothing, the clone can write the copy CRLF, andvalidate-pins.pyreportsshape-copy-driftafter an adoption that said verified #807) and adopt-project.py: a--source, or a plan'ssource.local_path, that names no repository's top level is refused by name before git reads anything else in it, so a file, a directory inside a repository, an unknown~user, an empty value,owner/.gitand a lost blob no longer end in a traceback, plan a subtree or reachgh(#796) #831 (adopt-project.py--source: a file, a directory inside a repository, an unknown~user/, an empty value andowner/.gitare not refused by name, soplanends in a traceback, plans only a subtree or the current directory, or handsgha name it cannot clone #796). refs/heads/is written once, at line 8224 (BRANCH_REFS).08f7a283wrote it 4 times."a file"is written once, at line 1893 (A_FILE).None if at_the_record else madeis present on24a977c0,08f7a283andfc289dd7.- The 100 and 33 case counts, the 175 total and the 42 new cases are confirmed.
- Nine main commits touch
- No line in the unlisted functions: I mapped every changed line of
git diff fc289dd7 b1a40a64 -- adopt-project.pyto its enclosing function withast. None falls in_source_spec,_top_level,Source.open,Source.of,Source.in_placeorcmd_plan. - Changed in meaning: the empty source, the file, the directory inside a repo and the unwritable
--outare each refused before a work directory under a 0777 directory, as the table above shows. Anorg/reposource still getsadopt-work-dir-shared.checkandexecutestill ask_work_rootfirst. - Gate figures: the
grep -l -E 'adopt-project|adopt_project|\bADOPT\b' tests/*.pylist names 105 files, one of themconftest.py. That leaves 104 test files, and neithertest_cli_reference.pynortest_no_auto_maintenance.pyis among them, so 104 + 2 = 106. I did not re-run the writer's full 106-file gate (7,524 and 7,124 passed). My targeted gate agrees with it on the files I ran. - xfail:
112eeed2is(#735) (#772)and is infc289dd7.test_shape_doctor.pyhas no xfail marker left, and my runs show 0 xfailed.
Cleanup
- I removed all 7 scratch worktrees with
git worktree remove --force. The lane worktree is clean atb1a40a64. - I pushed nothing, posted nothing and committed nothing to a shared branch.
Paths
- Probe tools, logs, stubs and e2e inputs:
/workspace/projects/openRepoShape-worktrees/_lane1-scratch/r803m/. That directory hastools/,sum-A.txt,sum-B.txt,revlog-*.txtande2e/. - Body:
/workspace/projects/openRepoShape-worktrees/_lane1-notes-20261011/v11/pr803-body.md
Next (lane 1's coordinator): the wording fix is applied to the body. #843 is PR-READY at b1a40a64, on condition that CI tests (run 39104253187) finishes green. It takes lane 1's turn after #861, at lane 1's next turn in lane 3's round-robin, gated over the main of that moment. A local merge-tree puts it cleanly over main, #775 and #861's predicted squash (tree a712a8e7), so its update-branch should need no further merge round.
Lane: openreposhape-1 (openRepoShape-1)
|
|
Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days. You can request another review in 5 days and 17 hours by commenting |
… RV883 and run to main c5359df, and the back-port comment names ruleset 25325536 (#883) Round 2 of #883, on lane 3's review RV883 of ede678c (#883 issuecomment-6247451611). Each finding was checked against the code at main c5359df before the text changed, and each held: - `plan written to` is printed after a blank line and before the report, not first: an org/repo source prints its `cloning` line before it. - The pipx command is unchanged since v1.1.0 (`src/` is). Measured on the package at c5359df, outside a checkout: its `--version` prints a `$OPENREPOSHAPE_REPO` holding U+202E, exit 0, and its refusal of such a `$OPENREPOSHAPE_REF` holds the character. The notes say "the bash `openRepoShape`" and carry this as a known limit. - `family.py init`'s closing block and `Working trees are at` still print a path raw; `make siblings` has no unmerged check, and clones from `family.yaml` where git finds no url; `update-shape.py apply --branch` meets `unseen-branch` after its writes; a WHY still shows `?` and `--json` is ASCII with escapes; a directory in no repository is still `git-failed`. Each is now said as the code does it. - The refresh adds #843 (#803), #811 (#777) and #794 (#793): 50 pull requests and 50 issues, with 20 milestone issues still open. - The release sections name Brett Heap's ruling, option A and ruleset 25325536, add a step 0, define `<notes>` with a grep a simulation shows prints nothing, and check `gh release list` before `--latest`. release.yml: lines 74-76 of the RELEASING comment said the ruleset waited on Brett Heap's word. They now name ruleset 25325536. Only comment lines change; the YAML is identical with comments removed. The upgrade check was run again at c5359df on fresh clones of a v1.1.0 project and holder: six `upstream-changed` copies each, `apply` green, and the doctor COMPLIANT. Lane: openreposhape-1 (openRepoShape-1) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…and its round 2, and run to main 33c5128 (#883) Round 2b of #883. It answers RV883's full NIT list on ede678c (lane 3, 2026-10-18T09:29:28Z) and RV883's round 2 on 86baec5 (#883 issuecomment-6248362246). Each finding was checked against the code at main 33c5128 before the text changed: - The #843 limit is two: what `adopt-work-dir-shared` leaves open (group write, Windows, root as the other user, this user's own processes, the default work directory), and what it now refuses that v1.1.0 used (another user's directory, a run as root included, a 0777 directory without the sticky bit, a drive mounted without ownership), with the way out. "What a script sees" names `adopt-work-dir-shared` and `adopt-work-dir-occupied`. - The doctor's NEXT lines, not its `members` row, read no url from an unmerged `.gitmodules`; the `make siblings` limit names that row. `siblings.py` has no unmerged rule; the `validate.yml` templates do. - `validate-family.py`'s `family ok` is scoped to `--no-members` and to a holder whose members are checked out. - The wording points: `update-pr-no-repository` under `--pr`, family.py's `git rm -r --cached` line, `MSYSTEM` set and not empty, `..` and `naming-unclassified`, the keys git refuses with no value, the url the root's config keeps, `contracts: cannot be listed`, install.sh's newest tag and redirect location, the scan test's name, `update-rewrite-symlink`, `os.access` for root, `LC_ALL=C`, `update-validators-red`, `heads/main` in both keys, where `plan written to` came, junctions on Python 3.12, the approval click and the run's approvals, and the back-port route's one run. - #457's L1-17 is narrowed by #825; #853 and #854 join #834 as test limits. The refresh past c5359df adds #870 (#864), #876 (#839), #791 (#787), #859 (#838), #780 (#776), #873 (#857) and #875 (#874): 57 pull requests. It drops the limits for #776, #839 and #874's ref rule, rewrites the #838 clauses, and adds the limits those squashes name (#863, #879, #884, an older git without `GIT_NO_LAZY_FETCH`, family.py's link guard). The open list is the 14 that the 09:42Z milestone read leaves after #875: #675, #707, #739, #755, #764, #834, #840, #853, #854, #856, #863, #878, #879 and #884. The six-and-six shape-copy lists stand: from c5359df to 33c5128 the templates and repo_shape.py changed only in the family root's bootstrap.py and validate-family.py, both listed. No other file changes. Lane: openreposhape-1 (openRepoShape-1) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>



Lane: openreposhape-1 (openRepoShape-1)
Claim: #803 (comment)
A v1.1.1 PR, authored on
mainat24a977c0, with mainfc289dd7merged in since (see "Merge of main"). It lands after the v1.1.0 tag, in lane 3's order, refreshed bygh pr update-branchthen.What
planandcheckmake a clone in the work directory and, since #749, remove it when no plan records it. That clone is now claimed by the run that makes it, and removed only while it is still the directory that run made._work_rootasks each directory above the real--work-dir, from/down, BEFORE it makes any missing one, and again after; then the work directory itself.adopt-work-dir-shared, naming it.authorized_keysto, with two differences: the sticky bit counts, which keeps/tmpopen, and group write is not asked. Root is trusted, as StrictModes trusts it.os.path.realpath, so a symlink in a shared directory that leads into a safe chain is used, as its target.os.lstat, and never on Windows. The default work directory (tempfile.mkdtemp) is not asked.source-readonly/is asked before the NOTE and beforegh(_source_readonly_refusal):adopt-work-dir-occupied, namingsource-readonly(L1-65);adopt-work-dir-shared;adopt-work-dir-unwritable.ghruns (_claimed). It is made with an exclusiveos.mkdir, andgit cloneclones into it. Any target already there is refused asadopt-work-dir-occupied, an empty one included (L1-98). Amkdirofsource-readonlythat fails is refused by name, where it was a traceback (L1-65).source-readonly/and the clone is 0755 whatever the umask (_made_0755); the umask can still narrow it, to 0700 underumask 077. The legs, the assembly and--local-remote-dir's parent are still made with the default mode. adopt-project.py: a--work-dirthat is a file or cannot be made endsplan,checkandexecutein a traceback, and--helpsays nothing of--work-diror of where--orgcomes from #650's refusals of a--work-dirthat cannot be made are unchanged._remove_a_tree). Right after itsmkdir, the run records the identity (st_dev,st_ino) of the clone's directory and ofsource-readonly/. It removes only while both still match, and never through a symlink.source-readonly/, opened without following a symlink (shutil.rmtree(dir_fd=)).os.path.normcaseon both sides (L1-102).--sourceclone thatcheckmakes at the plan's ownsource.local_pathis no longer listed for removal (L1-97).executenow removes something when its clone does not finish. It still lists no clone, and still keeps a clone it made. Before this PR, a failedgh repo cloneleft no target behind, because git removes a directory it created itself. Now the run makes the directory first, and git keeps a directory that existed before the clone. Left there, it would make the nextexecutegiven the same--work-dirrefuse at an empty directory. So however the clone ends short of a clone (ghfailing, Ctrl-C, noghon PATH),executeremoves exactly the directory its ownmkdirmade, andsource-readonly/where it made that too, after the same identity checks, and then raises what ended it.Two lines outside this PR's own code change by one name:
_overand_index_entriesnow spell"a file"as the new constantA_FILE, so the literal is written once (SonarCloud S1192).Each line of the issue: the code and the words
source-readonlyis swapped for a symlink tovictim/, or forvictim/itself, at the start of_remove_a_tree(t111c's injection).checkexits 0,victim/Thingand its file stay, and stderr says:NOTE the clone this run made, `<W>/source-readonly/Thing`, is not removed: what that path leads to now is not the directory this run made, so nothing is removed through it.source-readonlyswapped for a symlink tobigdisk/during the run gets:NOTE the clone this run made, `<W>/source-readonly/Thing`, is not removed: that path now leads to `<bigdisk>/Thing`, so nothing is removed through it.source-readonlyalready there before the run is refused, with noghcall:REFUSED adopt-work-dir-occupied: --work-dir `<W>` already holds `source-readonly`, a symlink to `<bigdisk>`, where this run clones testorg/Thing into a directory of its ownRemediation: Pass a --work-dir that does not hold it, or none: without one, each run makes a new one.REFUSED adopt-work-dir-occupied: --work-dir `<W>` already holds `source-readonly/Thing`, an empty directory, where this run clones testorg/Thing, and a run clones only into a directory it has just made, so that it never takes another run's for its ownRemediation: Pass a --work-dir that does not hold it, or none: without one, each run makes a new one. Or, where no run is using `source-readonly/Thing`, delete it: a run that was killed can leave it empty.Run A's clone then fails (
source-unclonable), and A removes only what its ownmkdirmade.check --source testorg/Thing --work-dir Wexits 0 and keepsW/source-readonly/Thingat the plan's record.os.path.normcaseon both sides, so on Windows aSource-Readonlyon disk no longer skips the removal in silence.source-readonlygets, inplan,checkandexecute:REFUSED adopt-work-dir-occupied: --work-dir `<W>` already holds `source-readonly`, a file, where this run clones testorg/Thing into a directory of its ownRemediation: Pass a --work-dir that does not hold it, or none: without one, each run makes a new one.execute's remedy beginsno leg repository was created and nothing was pushed.mkdirthat the system refuses gets:REFUSED adopt-work-dir-unwritable: --work-dir `<W>` cannot hold `source-readonly`, where this run clones testorg/Thing: making it failed (Permission denied)REFUSED adopt-work-dir-shared: --work-dir `<W>` is under `<A>`, which can be written in by every user, without the sticky bit, so another user could rename `<W>`, or a directory between them, and turn this run's removal onto another directoryRemediation: Pass a --work-dir this user owns, that no other user can write in, and that has no directory above it another user can rename in, or none: without one, each run makes a new one.Where
<A>is this user's, its mode is the reason, and the way out goes on:Or, since `<A>` is this user's, set its sticky bit (`chmod +t <A>`): each entry in it then stays its owner's.which is owned by `<user>`, not by this user or root, so ....--work-dir `<W>` can be written in by every user, without the sticky bit, so another user could rename or replace what this run makes in it, with the same remedy.source-readonlyreadsholds `source-readonly`, which can be written in by every user, without the sticky bit, so another user could rename or replace the clone this run makes in it, with the remedyPass a --work-dir that does not hold it, or none: ....sudorun leaves one, gets main's true refusal again:adopt-work-dir-unwritable.--work-dirthat is a file or cannot be made endsplan,checkandexecutein a traceback, and--helpsays nothing of--work-diror of where--orgcomes from #650's refusal,adopt-work-dir-unwritable.Why (repro on main)
Every new test fails on
mainbut the controls and the six that hold #650's own words for a loop or a file above the work directory (24a977c0;adopt-project.py's clone code is #749's, unchanged sincebf1f099b):_remove_a_tree,checkexits 0 and removesvictim/Thing/precious.txt, both with a symlink and with a directory moved in. The test reads the file back and getsFileNotFoundError.plan) exits 0 and writes its plan. Run A (check, whose stubghwaits on a file and then fails) then removes B's clone.os.pathpatched asntpathanswers, the clone is kept, silently.plan,checkandexecuteeach end inFileExistsError: [Errno 17] File exists: '<W>/source-readonly'.--work-dirat mode 0777, one under a 0777 directory, and asource-readonlyat 0777 or owned by another user, are used as any other. With the work directory swapped from the 0777 directory above it right after the clone'smkdir(adopt-project.py: the cloneplanandcheckmake is claimed with the run's ownmkdirand removed only while it is still that directory, and a--work-diranother user can rename in is refused first, so a swappedsource-readonlyno longer turns the removal onto another directory (#803) #843's review, in process),victimroot/source-readonly/Thingand its file are removed.How
adopt-project.py:_work_rootasks_refuse_a_shared_chain(_shared_above, each directory above the real path, down to the first entry that is no directory) before_made_0755makes anything, and again after, then_shared_with_another_userof the work directory itself.Source.cloneasks_source_readonly_refusal, then the target (os.path.lexists), then prints the NOTE, then_claimed, then runsgh. That is adopt-project.py:--sourceis read by its spelling, anorg/repoin any spelling is cloned by its name, the plan recordssource.cloned, and a clone no plan records is removed, so a recorded clone that is gone or emptied is refused or cloned again for what it is (#652) #749's order: writable, occupied, NOTE, clone. Whereexecutelists nothing, anything that ends the clone removes the claim and is raised again._claimedreturns a_Claim, the only thing onargs.clones._leave_the_clonesasks_why_kept. It compares the real path withnormcase, then calls_remove_a_tree, which raises_NotTheClonewhere an identity does not match.Plan._keep_the_clone_at_the_recordtakes the override's clone off the list.--work-dirhelp of all three commands namesadopt-work-dir-sharedfor the work directory, each directory above it, and itssource-readonly.planandchecksay that asource-readonlythat is a file or a symlink, or that holds the clone's directory already, even empty, isadopt-work-dir-occupied.docs/cli.mdis rendered again.The residual window
Except through group write, and on Windows (both below), no other user can now rename the work directory or anything above it, on any Python. Every directory above the work directory is asked, and so are the work directory and its
source-readonly/. That covers what #843's review measured on the first head, which this PR now refuses:mkdirand the identity read right after it, which the claim then recorded as its own: a second process won 5 of 20 runs on Python 3.12 under load, and the swap made in process won on every Python;.gitlost the victim's files in 17 of 20 runs on the first head, and at the clone's directory in 8 of 20 on main;Where the removal is by descriptor, a rename of
source-readonly/, or of a directory above it, after the checks does not redirect it. That is Python 3.11 and later, on Linux and macOS. The removal is relative to a descriptor of thesource-readonly/this run made, opened without following a symlink and held to its recorded identity. A rename of the clone's own directory insidesource-readonly/after its check still redirects it, on every Python; only a user who can write insource-readonly/can make it, and group write is not asked.On Python 3.9 and 3.10, and on Windows, the removal is by path, after the same identity checks. A rename of
source-readonly, or of a directory above it, between the last check andshutil.rmtree's own open of the path still removes the directory of that name wherever the new path leads. Measured on Python 3.9.23, with the swap made inside the run's own process. On Windows, whereshutil.rmtreeitself walks by path, the window lasts the whole removal: a rename during the walk turns the rest of it onto whatever the path then leads to.What stays open, said plainly:
source-readonly/, the work directory or a directory above it: group write is not asked, on any Python;shutil.rmtreewalks by path: any user whose ACL lets them rename there can open the window;Merge of main
Main
fc289dd7is merged into the branch as a merge commit, not a rebase: the head is nowb1a40a64, whose parents are08f7a283andfc289dd7. Since24a977c0, main has gained nine commits inadopt-project.py: #817 (#708), #630 (#563), #829 (#801), #842 (#800), #830 (#797), #833 (#799), #825 (#716), #844 (#807) and #831 (#796).Plan.open_source's--sourcebranch._default_branchreadsorigin/HEADand HEAD by full name and whole, and reads the source only at a branch of its own, so agit clone -bsource is no longergit-failedwith no remedy, and a HEAD onorigin/x, onmain<U+00A0>or beside a tagmainno longer adopts another branch's commit (#797) #830, and adopt-project.py: a--source, or a plan'ssource.local_path, that names no repository's top level is refused by name before git reads anything else in it, so a file, a directory inside a repository, an unknown~user, an empty value,owner/.gitand a lost blob no longer end in a traceback, plan a subtree or reachgh(#796) #831):Source.openis givenself.read_at;if override:becameif override is not None:; and the docstring gained a paragraph on an empty override.Plan._keep_the_clone_at_the_recordtakes the clone made at the plan's ownsource.local_pathoff the list (L1-97).Source.open,Source.of,Source.in_place,Source.cloneandSource._attake main'sinstead:--tracking-branch, or the plan'ssource.default_branch(adopt-project.py:_default_branchreadsorigin/HEADand HEAD by short name, so a source cloned withgit clone -b <branch>is refusedgit-failedwith no remedy, and a HEAD on a localorigin/xor on a name ending in U+00A0 adopts another branch's commit #797). This PR's rewrittenSource.clonehands it on tocls._at(path, repository, work_root, True, instead)once the claim and the clone are made._recorded_source's oneSource.clonecall, which serves the clone at the record and the one elsewhere, passesinstead=self.read_at, as do its twoSource.in_placecalls; at the record it still lists nothing (None if at_the_record else made, as on24a977c0).cmd_planasks about--out(adopt-project.py:planasks whether it can write--outwith its flags, before any work, and refuses a folder, a pipe, a missing folder or one nobody can write in asadopt-out-unwritable, and saysplan written toonce the plan is on disk, so a mistyped--outno longer ends in a traceback after the whole plan is built (#799) #833) and about what--sourcenames (_source_specand_top_level, adopt-project.py: a--source, or a plan'ssource.local_path, that names no repository's top level is refused by name before git reads anything else in it, so a file, a directory inside a repository, an unknown~user, an empty value,owner/.gitand a lost blob no longer end in a traceback, plan a subtree or reachgh(#796) #831) before_work_root._work_rootthen asks about the directories above the work directory, as on this branch.import errno, and it is one line.docs/cli.mdmerged without a conflict.scripts/render-cli-reference.py --checksays it is what the generator produces.tests/test_adopt_source_clone_record.pyandtests/test_adopt_work_dir_refusal.pyhold main's adopt-project.py--source: a file, a directory inside a repository, an unknown~user/, an empty value andowner/.gitare not refused by name, soplanends in a traceback, plans only a subtree or the current directory, or handsgha name it cannot clone #796 tests beside this PR's changes. They now collect 100 and 33 cases.plan, where two apply.--outthat cannot be written, or a--sourcethat is empty, a file or a directory inside a repository, is now refused before a--work-dirunder a directory every user can write in.b1a40a64, with the work directory under a 0777 directory:--source ''issource-unresolvable, and an--outin a missing folder isadopt-out-unwritable. At08f7a283,--source ''with the same work directory gotadopt-work-dir-shared. Nothing was made in any case. Anorg/reposource atb1a40a64still getsadopt-work-dir-shared.--source: a file, a directory inside a repository, an unknown~user/, an empty value andowner/.gitare not refused by name, soplanends in a traceback, plans only a subtree or the current directory, or handsgha name it cannot clone #796, adopt-project.pyplan: an--outin a missing folder, an existing folder, or a folder nobody can write in ends in a traceback after the whole plan is built, so a mistyped--outgets no refusal and no way out #799).checkandexecuteask_work_rootfirst, as before.--source: a file, a directory inside a repository, an unknown~user/, an empty value andowner/.gitare not refused by name, soplanends in a traceback, plans only a subtree or the current directory, or handsgha name it cannot clone #796's, adopt-project.py:_default_branchreadsorigin/HEADand HEAD by short name, so a source cloned withgit clone -b <branch>is refusedgit-failedwith no remedy, and a HEAD on a localorigin/xor on a name ending in U+00A0 adopts another branch's commit #797's and adopt-project.pyplan: an--outin a missing folder, an existing folder, or a folder nobody can write in ends in a traceback after the whole plan is built, so a mistyped--outgets no refusal and no way out #799's code reads as it does on main.adopt-project.pyas before: no line of_source_spec,_top_level,Source.open,Source.of,Source.in_placeorcmd_plan.adopt-project.pywrites"refs/heads/"once, as main'sBRANCH_REFS;08f7a283wrote it 4 times (SonarCloud S1192). It also writes"a file"once, asA_FILE.Tests
tests/test_adopt_clone_claimed.pyis new, with 42 cases:rmtreethat does not walk by descriptor), and otherwise assertsFD_REMOVALfirst;ghthat waits on a file in steps of 0.1 s, and an empty directory already there;executewhose clone fails, and whose clone is cut short by Ctrl-C or by noghon PATH: the claim is gone, the exception is raised, and the same command again reachesgh;os.pathpatched;mkdirbackstop, both as a file made since and as a refusedmkdir;source-readonlyrefused before the NOTE;source-readonlythat every user can write in, or that another user owns;plan,checkandexecutewith nothing made, the missing work directory included;planandcheckmake is claimed with the run's ownmkdirand removed only while it is still that directory, and a--work-diranother user can rename in is refused first, so a swappedsource-readonlyno longer turns the removal onto another directory (#803) #843's review's swap from the 0777 directory above, reproduced in process: refused before anything is made, the victim intact;plan,checkandexecute: adopt-project.py: a--work-dirthat is a file or cannot be made endsplan,checkandexecutein a traceback, and--helpsays nothing of--work-diror of where--orgcomes from #650'sadopt-work-dir-unwritable, in main's words (these pass onmaintoo, and failed onbba7e445);maintoo: a sticky (1777) directory above, work directory andsource-readonly; a root-owned directory above; a--work-dirgiven through a symlink in a 0777 directory whose target is in a safe chain; root's 1777 work directory (/tmp, emulated); root's 0755 work directory this user cannot write in, which getsadopt-work-dir-unwritable;source-readonlyand the claim underumask 0, and a missing, deep--work-dirmade by aplanunderumask 000: each directory 0755, and used.chown), and emulated throughos.lstatotherwise. Mode tests skip where a mode does not stop this user, and on Windows.--sourceis read by its spelling, anorg/repoin any spelling is cloned by its name, the plan recordssource.cloned, and a clone no plan records is removed, so a recorded clone that is gone or emptied is refused or cloned again for what it is (#652) #749's tests change:test_a_clone_reached_through_a_symlink_is_never_removedpinned the old behaviour, L1-99 (a)'s: the clone was made through a symlinkedsource-readonlyand kept, exit 0. It now expects the refusal, noghcall, and nothing made through the link.test_a_clone_is_removed_where_a_read_only_file_cannot_be_unlinkedgains one line: it emulates Windows, so it now also takes the removal by path, as Windows has no removal by descriptor. On Linux the removal by descriptor takes no handler. POSIX unlinks a read-only file, and a handler there would act through a path again.tests/test_adopt_work_dir_refusal.py's code table names the new code, so its help tests ask for it in each command, whole on one line.Revert table. Each change is reverted alone on
08f7a283, and these runs oftest_adopt_clone_claimed.py,test_adopt_source_clone_record.pyandtest_adopt_work_dir_refusal.py(121 cases) fail:main'sadopt-project.py)adopt-project.py,cf36411e)adopt-project.py,bba7e445)plan,checkandexecute; a work directory another user ownssource-readonlythat is a symlink or a filesource-readonlyanother user can rename insource-readonlymakedirs(exist_ok=True), and an empty target taken)FD_REMOVALforced falseFD_REMOVALnormcaseexecute's removal of its claimghmkdirbackstopmkdirsource-readonlyand the claimumask 0--work-dir, since the suite's own temporary directory is below/workspace/projects, which is at 1777; the sticky-bit control among themsource-readonly, the work directory and a directory above, each another user'stest_cli_reference.py(run with it)execute's removal on any end, not onlyCommandFailedgh--work-dir, since/is root's; the root-owned controls among them_made_0755(Path.mkdir(parents=True)andos.makedirsagain)umask 000source-readonly(cf36411e)test_adopt_history_gitlinks.py::test_an_org_repo_source_is_read_as_the_extraction_clones_it, which callsSource.openwith a work directory that does not exist (both modes, on27ba79b9)N1, S1 and R15 again, at the merged head
b1a40a64. Each was reverted alone, in its own clone, and the same three files were run withgit-filter-repoon PATH. With main's #796 tests they now hold 175 cases, and unchanged they pass all 175.test_adopt_clone_claimed.py, 59 oftest_adopt_source_clone_record.pyand 15 oftest_adopt_work_dir_refusal.py. As before, the suite's temporary directories are below/workspace/projects(1777), so without the sticky-bit exemption the work directories there are refused.Both modes, on
b1a40a64(mainfc289dd7merged), dirty 0, with/usr/bin/python33.12.3 and a stubgh. The run covers 106 files, each run on its own, in 6 shards per mode:grep -l -E 'adopt-project|adopt_project|\bADOPT\b' tests/*.pynames,test_shape_doctor.pyandtest_scaffold_leg_path_collisions.pyamong them (it also namesconftest.py, which is not a test file);test_cli_reference.pyandtest_no_auto_maintenance.py.The results:
git-filter-repoon PATH: 7,524 passed, 17 skipped;<stub>:/usr/local/bin:/usr/bin:/bin, wherecommand -v git-filter-repofinds nothing): 7,124 passed, 417 skipped.The xfailed case of earlier runs,
test_shape_doctor.py::test_make_siblings_dry_run_gives_the_rows_word_for_each_slot[a-file], now passes in both modes: main's #772 (#735) mademake siblingsanswer NOT A CLONE, and dropped the xfail.The suite's temporary directories were under the lane's own scratch directory, below
/workspace/projects, which now has the sticky bit (1777). Before that it was 0777, and this PR refused every--work-dirunder it, the tests' own included.Cross-version probe of the first head (
cf36411e; Python 3.9.23, 3.11.13 and 3.12.3, in process, with a stand-ingh):source-readonlyasshutil.rmtreeis called is closed on 3.11 and 3.12, and removes the victim on 3.9.source-readonlyis refused, a 0777 work directory is refused with nothing made, andexecutewhose clone fails leaves nothing behind, twice over.Windows
os.geteuiddoes not exist on Windows), of the work directory, the directories above it orsource-readonly.os.lstat, which since Python 3.8 does not follow a symlink or a junction, is held to the recorded identity;normcase;--sourceis read by its spelling, anorg/repoin any spelling is cloned by its name, the plan recordssource.cloned, and a clone no plan records is removed, so a recorded clone that is gone or emptied is refused or cloned again for what it is (#652) #749's handler still clears the read-only bit git sets.source-readonlyreads as a directory that leads elsewhere, so it is refused asadopt-work-dir-occupied.#!stub skip there.Not in this PR
_source_spec,Source.open,Source.in_place(adopt-project.py--source: a file, a directory inside a repository, an unknown~user/, an empty value andowner/.gitare not refused by name, soplanends in a traceback, plans only a subtree or the current directory, or handsgha name it cannot clone #796) and the top ofcmd_plan(adopt-project.pyplan: an--outin a missing folder, an existing folder, or a folder nobody can write in ends in a traceback after the whole plan is built, so a mistyped--outgets no refusal and no way out #799). Both are on main now (adopt-project.py: a--source, or a plan'ssource.local_path, that names no repository's top level is refused by name before git reads anything else in it, so a file, a directory inside a repository, an unknown~user, an empty value,owner/.gitand a lost blob no longer end in a traceback, plan a subtree or reachgh(#796) #831, adopt-project.py:planasks whether it can write--outwith its flags, before any work, and refuses a folder, a pipe, a missing folder or one nobody can write in asadopt-out-unwritable, and saysplan written toonce the plan is on disk, so a mistyped--outno longer ends in a traceback after the whole plan is built (#799) #833) and came in with the merge; this PR's diff against main does not touch them.executestill keeps a clone it made, after an early refusal.Known limits
--work-diron a Windows drive (drvfs) reads as mode 0777 and is refused asadopt-work-dir-shared. That is true there: every Linux user of the distribution acts as the same Windows user on that drive.tempfile.mkdtempmakes it 0700 in the system temp folder, and only aTMPDIRthat every user can write in without the sticky bit would open it.--work-dir /tmpand directories root owns. Root is trusted. So--work-dir /tmp(root's, 1777) is used, and stays safe: another user'ssource-readonlythere is still refused. A directory root owns that this user cannot write in, as an earliersudorun leaves one, gets main'sadopt-work-dir-unwritable.--work-dirunder a directory another non-root user owns, or under one every user can write in without the sticky bit, is now refused, though main ran there. That includes a shared/workspaceor project folder at 0777. Where such a directory is this user's, setting its sticky bit (chmod +t) keeps the protection and changes nothing for this user; the refusal says so.adopt-work-dir-shared: on macOS an external exFAT, FAT or NTFS drive, or one with "Ignore ownership on this volume"; on Linux, ntfs-3g by default. The way out is a--work-diron a disk that keeps owners, such as the home directory, or no--work-dir.docker runas root with a bind-mounted host directory owned by uid 1000 does:is owned by user id 1000, not by this user or root. The way out is to run as that directory's owner (docker run --user 1000:1000), to give a--work-dirroot owns, or to give none.Closes #803
🤖 Generated with Claude Code
Summary by Sourcery
Harden clone work-directory ownership and cleanup so concurrent or renamed paths cannot redirect removal onto another directory.
Bug Fixes:
Enhancements:
Documentation:
Tests:
Chores: