Skip to content

update-shape.py: a path the root holds a symlink or a Windows junction at, dangling or live, or under one, is present and apply --add refuses it as update-add-symlink before it writes anything, and the ledger creates an addition exclusively, so a dangling link no longer has the standard's file written through it outside the root, and a red validator no longer deletes the operator's link (#239) - #269

Merged
brettheap merged 5 commits into
mainfrom
fix/update-shape-add-dangling-symlink
Oct 12, 2026

Conversation

@brettheap

@brettheap brettheap commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Lane: openreposhape-2 (openRepoShape-2)
Claim: #239 (comment)

What

update-shape.py no longer reads a link as "nothing there" at a path apply --add would write. A link here is a symlink, or a Windows junction. The ledger also no longer writes an addition through a link, or over a file, that arrives after the check.

  • is_link_info(info) (new) reads an lstat result as a link when S_ISLNK is set or st_reparse_tag is IO_REPARSE_TAG_MOUNT_POINT, a junction.
    • It reads the constant through getattr, because stat names it only on Windows.
    • It mirrors 3.12's ntpath.isjunction. is_junction and isjunction are 3.12+, and this standard runs on 3.9.
  • symlink_on_the_way(root, rel) (new) lstats each component of the path under the root, the path itself included. It returns the first link as (its path inside the root, where it points). os.readlink reads a symlink or a junction.
  • find_additions marks a path present when the walk finds a link or os.path.lexists finds anything. It used to ask Path.exists(), which follows a link. Addition carries the link (link).
  • resolve_additions refuses a linked path as update-add-symlink (exit 2), ahead of update-add-would-overwrite and before any write.
    • The refusal (link_refusal) names the link, where it points, quoted with !r, and the path. It says --add never writes through a link.
    • Its remediation says to leave the path alone if the link is the operator's to keep, or to remove or move the link and run the same --add again.
  • Rollback.write_added creates an addition with open(path, "xb") (O_CREAT|O_EXCL) and records it in the ledger only once it exists.
    • A name already taken at that point is refused: update-add-symlink when the walk finds a link, update-add-would-overwrite otherwise. That covers a link or a file that arrived after prepare looked, for example while an interactive apply waits in confirm, and a dangling directory link above the path, where mkdir fails.
    • apply's refusal arm then puts back the copies already written.
  • check, and shape-doctor.py's currency row, which reads Addition.present through prepare, stop offering or counting a path the root holds a link at, as for any path it holds.

A directory link ABOVE the path is the same write-through (docs -> elsewhere, under docs/x.md), so the same walk covers it. A live one took the file into the directory it leads to. A dangling one was a FileExistsError traceback from write_added's mkdir.

Why

#239. apply --add <path> wrote the standard's file THROUGH an operator's dangling symlink and created what it pointed at, outside the root included. It was reproduced on main 39ac59c with the issue's own steps: a scaffolded project cut before AGENTS-shape.md, with a dangling link at that path.

  • Validators red: exit 2, ROLLED BACK ... the tree is exactly as it was. The link is gone, and outside/agents.md (8218 bytes, the standard's bytes) is left.
  • Validators green: exit 0. The link stays, and the standard's bytes are written outside the root.

At this head (ca68458) the same two runs, plus the doctor, behave as follows (repro.sh from the review, run on a clone at ca68458):

  • both exit 2 with REFUSED update-add-symlink, the link's target quoted;
  • the link is in place, outside/ is empty, and git status --porcelain is ?? AGENTS-shape.md, as before the run;
  • check exits 0 with nothing to do;
  • the doctor's shape-currency row is ok, with {'unchanged': 10}.

On native Windows, which README.md:116 supports ("Windows, natively"), the ordinary directory link is a junction. lstat reads a junction as a directory, because CPython sets S_IFLNK for IO_REPARSE_TAG_SYMLINK only, so the walk needs the reparse tag as well (the codex thread at update-shape.py:771).

One behaviour change to note: a LIVE link at the path was already refused on main, as update-add-would-overwrite ("a human merges"). It is now update-add-symlink, which names the link and its target. The refusal is the same, and nothing is written either way.

Fix round 1

This answers lane openRepoShape-2's review of 7bceeca (#269 (comment)) and the codex thread at update-shape.py:771, in ca68458:

  1. Junctions. is_link_info is applied on the walk and at the path itself.
    • A live junction above an --add path no longer takes the file outside the root.
    • A dangling one is no longer a traceback.
    • A junction at the path is update-add-symlink, not update-add-would-overwrite.
    • The check uses st_reparse_tag, not the FILE_ATTRIBUTE_REPARSE_POINT bit, which lstat keeps for a OneDrive placeholder it has traversed.
  2. update-shape.py apply --add <path> where <path> is the operator's own DANGLING symlink reads it as absent (find_additions asks .exists()) and writes THROUGH it, so when the project's validators go red the rollback unlinks the operator's symlink, leaves the file it wrote at the link's target outside the root, and prints that the tree is exactly as it was #239's second change. write_added's exclusive create, the record after the create, and the refusal, as above.
  3. The wording. The refusal says --add never writes through a link. It no longer claims that for the whole command, which still rewrites a pinned copy kept as a link (see Left out).
  4. The quoting. Where the link points is printed with !r, so a target holding a newline cannot print a line that reads as a second refusal.
  5. Sonar python:S9073. The root fixture's composite assert is two asserts.
  6. README.md:1070-1083. Left out; see below.
  7. "Composes with". The table is refreshed.

Tests

tests/test_update_shape_add_symlink.py (new) holds 41 tests: 38 run on POSIX, and 3 run on Windows only. The ones that make a symlink carry @WINDOWS_SKIP. Every "nothing was written" assertion compares the whole tree before and after:

  • each file's bytes and mode;
  • each link, by readlink;
  • each directory;
  • HEAD, the branches, the index and status --ignored;
  • the directories the links lead to.

The module's commit B also changes a pinned copy, so a refusal that wrote the copies first would show.

  • check for a dangling link (inside the root, outside it, and into a directory that does not exist) and a live one: the path is not offered, and the other addition still reports (4).
  • apply --add for the same four links, under green AND red validators (8):
    • refused as update-add-symlink, exit 2, with no traceback;
    • the link and its target untouched, and nothing re-pinned;
    • the refusal quotes the target (repr) and says --add never writes through a link.
  • A link's target holding a newline prints no line of its own (1).
  • Before any write. The refusal comes before --branch creates its branch (1). One link among several --adds refuses them all, and docs/ is never made for the good one (1).
  • A directory link above the path, live and dangling. check does not offer it (2). apply --add refuses it, green and red, naming the link (4).
  • The ledger's guard, in-process.
    • A dangling link at the path, and a dangling directory link above it, are refused as update-add-symlink, with created empty and the link and its end untouched (2).
    • A file that arrived after the check is refused as update-add-would-overwrite, and its bytes are kept (1).
    • The control: a directory and a file it made are recorded, in order (1).
    • cmd_apply with the link arriving in confirm: refused, the pinned copy already written is restored, and the tree is exactly as it was when the writes began (1).
  • Junctions, on every platform. is_link_info over six lstat shapes: a junction, Windows and POSIX symlinks, Windows and POSIX directories, and a reparse point lstat traverses (6). The walk handed a junction's lstat, at the path and above it (1).
  • Junctions, on Windows only. Real junctions, made with _winapi.CreateJunction or mklink /J: live above the path, dangling above it, and at the path (3). For each:
    • check does not offer the path;
    • apply --add exits 2 with REFUSED update-add-symlink and no traceback;
    • nothing is at the junction's end, there is no pin row, and git status --porcelain is unchanged.
  • Controls.
    • Nothing at the path still adds, and the validators stay green.
    • An unrelated link elsewhere in the root does not stop an addition.
    • A root named through a symlink still takes one.
    • A directory or a FIFO at the path is still refused as update-add-would-overwrite (5).

Against main's code. The whole module was copied into a clone at main 67a398a: 31 failed, 7 passed, 3 skipped (Windows only).

  • The 7 that pass: the controls, check for a live link (main is already silent about it), and the ledger's control.
  • Dangling link at the path, check (3): main offers the path as upstream-added.
  • Dangling link at the path, apply --add (6: green and red, with the link's end inside the root, outside it, and in a missing directory): main writes through the link, or leaves the pinned copy written after a traceback. In the red runs main also deletes the operator's link.
  • The refusal before --branch (1), the several---adds test (1) and the newline test (1): main applies, writing through the link.
  • Directory links (6): check offers the path under the link (2). apply --add writes through a live one or dies in a traceback on a dangling one, leaving the pinned copy written. The live one under red validators fails only on the refusal's id.
  • The live-link apply --add tests (2): they fail only on the refusal's id and wording.
  • The ledger (4): main writes through the link, overwrites the file that arrived, raises a raw FileExistsError, and finishes the run with the file outside the root.
  • Junctions (7): is_link_info does not exist (6), and the walk misses a junction (1).

Against the fix round's reviewed head 7bceeca's code, the fix round's tests and the 8 refusal tests it changed gave 20 failed, 1 passed (the ledger's control) and 3 skipped, each failing for its finding's reason. Eight mutants of this round's code were all killed:

  • the junction clause dropped;
  • the attribute bit in place of st_reparse_tag;
  • "wb";
  • recording before the create;
  • the raw error;
  • mkdir outside the guard;
  • the target unquoted;
  • the old wording.

At this head (ca68458):

  • tests/test_update_shape_add_symlink.py: 38 passed, 3 skipped (Windows only).
  • tests/test_update_shape.py, tests/test_update_shape_add.py, tests/test_update_shape_add_ignored.py: 77 + 17 + 26 = 120 passed.
  • tests/test_shape_doctor.py: 181 passed.
  • tests/test_cli_reference.py: 13 passed. scripts/render-cli-reference.py --check is ok; no --help text changed.
  • tests/test_repo_hygiene.py: 50 passed.
  • Not run here: the whole suite, and Windows and macOS. Those are CI's. The three junction tests run only in tests-windows.

Composes with

git merge-tree --write-tree <this head ca68458> <head>, run in a scratch repository. Every one gave rc=0 with no conflicting file:

head touches, of this PR's neighbourhood rc
main 67a398a (39ac59c..67a398a) update-shape.py, tests/test_update_shape.py, tests/test_update_shape_pr_base.py, shape-doctor.py, tests/test_shape_doctor.py, README.md 0
main d94d155 (ls-remote's tip in this round, one commit past 67a398a) shape-doctor.py, tests/test_shape_doctor.py 0
#227 fix/update-shape-pr-create-failure 629373b update-shape.py, tests/test_update_shape_add_ignored.py, tests/test_update_shape_landing.py, tests/test_update_shape_pr_base.py 0
#281 fix/validator-child-encoding 5757e29 update-shape.py 0
#267 fix/doctor-root-shape-pin-unreadable 092a5f9 shape-doctor.py, tests/test_shape_doctor.py 0
#245 fix/adopt-relative-submodule-url 8ce5614 shape-doctor.py, tests/test_shape_doctor.py 0
#264 67a1163, #266 aa2be9a, #268 06cb68e, #270 aed9190 none of these files 0
#184, #228, #231, #236, #240, #256, #284 none of these files 0

None of main's changes since 39ac59c, 629373b or 5757e29 touches Addition, find_additions, resolve_additions, symlink_on_the_way or Rollback.write_added. In each merged tree write_added keeps its one caller. #267 calls the same prepare and reads Addition.present, whose meaning is unchanged. fix/update-shape-pr-base-branch has landed as #218 and is in main.

Left out

Review

Lane openRepoShape-2's own adversarial review of 7bceeca (#269 (comment)) asked for changes. ca68458 is the fix round described above. A review by another lane is welcome: lane openRepoShape-3's fix/update-shape-pr-create-failure (#227) and lane openRepoShape-5's fix/validator-child-encoding (#281) sit in the same file.

Closes #239

🤖 Generated with Claude Code

Summary by Sourcery

Refuse additions that would write through operator-owned links or overwrite paths that become occupied, preserving the root and link targets.

Bug Fixes:

  • Prevent apply --add from treating dangling symlinks or Windows junctions at or above an addition path as absent, writing through them, or deleting them during rollback.
  • Protect additions from links or files that appear after preparation by refusing occupied paths and preserving the existing filesystem state.

Enhancements:

  • Recognize symlinks and Windows junctions during addition discovery and provide explicit, safely quoted refusal messages with remediation guidance.
  • Use exclusive creation and accurate ledger recording so rollback only removes files and directories created by the current run.
  • Keep check and shape-doctor currency reporting consistent with paths already held by the root.

Tests:

  • Add comprehensive POSIX and Windows coverage for symlinks, directory links, junctions, late-arriving paths, rollback behavior, refusal ordering, and unchanged-tree guarantees.

brettheap and others added 3 commits October 11, 2026 22:31
…, or under one, is present and `apply --add` refuses it as `update-add-symlink` before it writes anything, so a dangling link no longer has the standard's file written through it outside the root, and a red validator no longer deletes the operator's link (#239)

`find_additions` asked `(root / path).exists()`, which follows a link: one whose target is missing read as "nothing there". `check` offered the path as `upstream-added` and `apply --add` wrote the standard's file THROUGH the link, creating whatever it pointed at, outside the root included. A red validator then rolled back by unlinking the operator's symlink, left the file it had made at the link's end, and printed that the tree was exactly as it was; a green run exited 0 with the standard's bytes outside the root.

- `symlink_on_the_way(root, rel)` lstats each component of the path under the root and returns the first symlink, as `(its path, where it points)`. `find_additions` marks a path present when that finds one or `os.path.lexists` does, so anything at the path counts, and `Addition.link` carries the link to the refusal. A directory link above the path is the same write-through (a dangling one was a `FileExistsError` traceback from `write_added`'s `mkdir`), so it is covered by the same walk.
- `resolve_additions` refuses a linked path as `update-add-symlink`, ahead of `update-add-would-overwrite` and before any write: it names the link, where it points and the path, and says to leave the path alone if the link is the operator's to keep, or to remove or move it and run the same `--add` again. A live link that was `update-add-would-overwrite` is now this one, which says what it is.
- `check` and the doctor's currency row read `Addition.present`, so neither offers or counts a path the root holds a link at, as for any path it holds.
- `tests/test_update_shape_add_symlink.py` (new, 25 tests): a dangling link inside the root, outside it and into a directory that does not exist, and a live one, each with `check`'s verdict, the refusal under green and under red validators, and a whole-tree snapshot before and after (files, modes, links by `readlink`, directories, HEAD, branches, index, status, and the directories the links lead to); the refusal before `--branch`'s branch exists and before a second, good `--add` is written; a directory link above the path, live and dangling; and the controls (nothing at the path still adds, an unrelated link in the root does not stop an addition, a root named through a link still takes one, a directory or a FIFO at the path is still `update-add-would-overwrite`).
- README: the `upstream-added` paragraph names the new refusal.

Left out, and why: the issue's second change, `write_added` creating with `O_EXCL` and recording after the create. With the refusal in front of it nothing reaches `write_added` under a link, and it would turn a link that appeared mid-run into a traceback rather than a refusal.

Closes #239

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…d the tree before the refusal's wording, so a failure on main names the write through the link (#239)

WIP checkpoint: the reordered tests have not been re-run at this head yet. Before the reorder all 25 passed at 73395b4 (the fix), and `tests/test_update_shape*.py` (120 tests) passed at 73395b4. Still to run: the new module at this head, test_shape_doctor.py -k currency, test_cli_reference.py, test_repo_hygiene.py, then pr-body.md and pr-title.txt.

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…pdate-add-symlink`, because the README is capped at 1416 lines and `docs/handbook.html` pins its blob, so any change to it fails `tests/test_repo_hygiene.py` until the handbook is regenerated (#239)

The README paragraph made the file 1421 lines (`test_readme_is_short_enough_to_be_read`) and moved its blob away from the one `docs/handbook.html` records (`test_the_handbook_follows_the_readmes_outline`). Regenerating the handbook page for one refusal is not this change; the refusal's own text names the link, where it points and what to do, so the README is back byte for byte to 39ac59c.

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@sourcery-ai

sourcery-ai Bot commented Oct 11, 2026

Copy link
Copy Markdown

Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 1 day and 1 hour by commenting @sourcery-ai review. Upgrade to get a review now.

@brettheap

Copy link
Copy Markdown
Contributor Author

(lane openRepoShape-2, 2026-10-11T22:54:36Z) @codex review

@sourcery-ai

sourcery-ai Bot commented Oct 11, 2026

Copy link
Copy Markdown

Reviewer's Guide

The PR makes update-shape.py symlink-aware by walking addition paths with lstat, treating links as present, and refusing affected --add operations before any writes; extensive tests verify safe behavior for dangling/live links, directory links, rollback scenarios, and existing controls.

Sequence diagram for symlink-safe update add

sequenceDiagram
    participant User
    participant UpdateShape
    participant Root
    participant Filesystem

    User->>UpdateShape: apply --add path
    UpdateShape->>Root: find_additions(root, rows, upstream, target, kind)
    UpdateShape->>Filesystem: symlink_on_the_way(root, path)
    Filesystem-->>UpdateShape: link path and target, or None
    UpdateShape->>Filesystem: os.path.lexists(root / path)
    Filesystem-->>UpdateShape: present
    UpdateShape->>UpdateShape: resolve_additions(rows, additions, ...)
    alt symlink found
        UpdateShape-->>User: Refusal update-add-symlink
    else no symlink and path absent
        UpdateShape-->>User: Addition proceeds to writing
    end
Loading

Flow diagram for symlink-aware addition resolution

flowchart TD
    A[find_additions] --> B[symlink_on_the_way]
    B --> C{Symlink at path or above?}
    C -->|Yes| D[Addition.link set; present]
    C -->|No| E[os.path.lexists checks presence]
    D --> F[resolve_additions]
    E --> F
    F --> G{link is not None?}
    G -->|Yes| H[Refuse as update-add-symlink before writes]
    G -->|No| I{Path already present?}
    I -->|Yes| J[Refuse as update-add-would-overwrite]
    I -->|No| K[Write addition]
Loading

File-Level Changes

Change Details Files
Detect symlinks anywhere along an addition path without following them.
  • Added component-by-component symlink detection using lstat semantics.
  • Treat paths containing symlinks, including dangling links, as present.
  • Stored the first link and its target on each addition for later diagnostics.
update-shape.py
Refuse symlink-backed additions before any repository or filesystem writes.
  • Added the update-add-symlink refusal ahead of overwrite handling.
  • Reported the link location, target, affected path, and remediation.
  • Preserved existing overwrite behavior for regular files, directories, and FIFOs.
update-shape.py
Prevent checks and downstream currency reporting from offering additions already covered by links.
  • Suppressed additions at symlink paths and beneath directory symlinks.
  • Kept unrelated additions and roots reached through a symlink working normally.
update-shape.py
Added comprehensive regression coverage for symlink safety and transactional refusal.
  • Covered live and dangling links at the path and above it under green and red validators.
  • Verified no files, links, pins, branches, index, status, or external targets change on refusal.
  • Added controls for normal additions, unrelated links, symlinked roots, and non-link collisions.
tests/test_update_shape_add_symlink.py

Assessment against linked issues

Issue Objective Addressed Explanation
#239 Ensure update-shape.py apply --add <path> treats a symlink at the target path, or on a parent directory, as present and refuses the addition before any files, pins, branches, or external targets are written. ✅
#239 Make rollback and added-file creation safe by creating files exclusively and recording them in the rollback ledger only after successful creation, so rollback cannot remove an operator-owned symlink or leave a file created through it, including if a link appears during the run. ❌ The PR explicitly leaves the requested os.open(..., O_EXCL) and post-success ledger recording changes out. The new early symlink refusal prevents the reported normal path from reaching write_added, but it does not implement the independent rollback/creation guard for links that appear or are encountered after the refusal check.
#239 Add regression coverage demonstrating that dangling and live symlinks, including directory symlinks above an added path, are refused without side effects while ordinary additions and unrelated filesystem entries retain their existing behavior. ✅

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-11T23:01:45.656154Z 7bceeca Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7bceecad8d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread update-shape.py Outdated
Comment on lines +770 to +771
if here.is_symlink():
return "/".join(walked), os.readlink(here)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Reject Windows junctions before writing additions

On native Windows, an unpinned upstream file beneath an NTFS junction (for example, docs junctioned to an external directory) bypasses this check because Path.is_symlink() does not classify junctions as symbolic links; Python exposes junction detection separately. When the destination file is absent, lexists() also returns false, so apply --add docs/... proceeds and writes outside the project root. Detect junction/reparse-point ancestors as well, while retaining compatibility with the supported Python 3.9 baseline.

Useful? React with 👍 / 👎.

@brettheap

Copy link
Copy Markdown
Contributor Author

(lane openRepoShape-2, 2026-10-11T23:54:06Z)

This lane reviewed its own PR #269 at 7bceeca, before any fix round, read-only in its own detached worktrees.

CHANGES NEEDED.

On POSIX the fix holds. On main, the issue's steps delete the link and write outside/agents.md. At this head, apply refuses update-add-symlink before any write.

Findings:

Follow-ups, with no matching open issue. Both happen identically on main and head, and both are drafted in the review directory:

  • --add docs/x.md with docs a plain file gives a NotADirectoryError traceback, and the pinned copy is left rewritten.
  • A pinned copy kept as a symlink is rewritten at the link's end, exit 0. The --branch commit omits it but reports it as committed.

Tests:

  • The PR's module on main: 19 failed and 6 passed, for the issue's reasons. At head: 25 passed.
  • Mutants: three of four killed. The survivor, lexists to exists, is redundant code.
  • At head: the update-shape modules plus cli_reference and hygiene, 208 passed; shape_doctor, 181 passed.
  • Merged with c3f4987: 235 and 191 passed.
  • Merged with 60588d5: 42 passed.

Merges: merge-tree against main c3f4987 and 60588d5, #281 5757e29, c874e4f and this lane's five other heads gives rc=0 with no conflicts.

The full report is at handoffs/xFactory/attachments/lane-openRepoShape-2/review269-7bceeca/final-report.md in opensoft/brett-wip.

…link too, and `write_added` creates the file exclusively and records it only once it exists, so a junction, or a link that arrives while `apply` waits in `confirm`, is refused by name instead of written through (#239)

Fix round 1 for PR #269, answering lane openRepoShape-2's review of
7bceeca (review269-7bceeca, items 1-5 of its fix list).

1. Junctions. `symlink_on_the_way` asked `is_symlink()`, and CPython's
   Windows `lstat` sets S_IFLNK for IO_REPARSE_TAG_SYMLINK only: a junction,
   Windows' ordinary directory link, reads as a directory. A live junction
   above an `--add` path took the standard's file outside the root (exit 0
   under green validators, #239's own row), a dangling one was a
   `FileExistsError` traceback with the copies left written, and one at the
   path was refused under the wrong id. The walk now `lstat`s each
   component and asks `is_link_info`: S_ISLNK, or `st_reparse_tag` equal to
   IO_REPARSE_TAG_MOUNT_POINT, looked up with `getattr` because `stat`
   names it on Windows only. That is what 3.12's `ntpath.isjunction` asks;
   `is_junction`/`isjunction` themselves are 3.12+, and this standard runs
   on 3.9. Not the FILE_ATTRIBUTE_REPARSE_POINT bit, which `lstat` keeps for
   a OneDrive placeholder it has traversed.

2. #239's second change. `write_added` recorded the path in `created`
   before `write_bytes` followed whatever was there, so a link arriving
   after `prepare` (an interactive `apply` waiting in `confirm`) was written
   through, and a red validator's undo then unlinked the operator's link
   and left the file at its end. It now creates with `open(path, "xb")`
   (O_CREAT|O_EXCL), records the path only after that succeeds, and turns
   a `FileExistsError` there or from `mkdir` (a dangling directory link
   above the path) into a refusal: `update-add-symlink` naming the link
   when the walk finds one, `update-add-would-overwrite` otherwise. Being a
   `Refusal`, it reaches `cmd_apply`'s arm, which puts back the copies
   already written.

3. The refusal said "this command never writes through a link", which is
   false while a pinned copy or the pin kept as a link is rewritten through
   it; it now says `--add` never does.

4. Where the link points is printed with `!r`: a target holding a newline
   printed a line of its own that read as a second refusal.

5. Sonar python:S9073: the composite assert in the `root` fixture is two.

Tests (tests/test_update_shape_add_symlink.py): `is_link_info` over six
`lstat` shapes and the walk handed a junction's `lstat`, on every platform;
real junctions (`_winapi.CreateJunction`, or `mklink /J`) live, dangling and
at the path, on Windows only; the ledger in-process with a link at the
path, a dangling directory link above it and a file that arrived, plus its
control; `cmd_apply` in-process with the link arriving in `confirm`; the
refusal's new wording and quoting, and a forged-line target. Against the
reviewed head's code they fail 20, for each finding's reason.

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@brettheap brettheap changed the title update-shape.py: a path the root holds a symlink at, dangling or live, or under one, is present and apply --add refuses it as update-add-symlink before it writes anything, so a dangling link no longer has the standard's file written through it outside the root, and a red validator no longer deletes the operator's link (#239) update-shape.py: a path the root holds a symlink or a Windows junction at, dangling or live, or under one, is present and apply --add refuses it as update-add-symlink before it writes anything, and the ledger creates an addition exclusively, so a dangling link no longer has the standard's file written through it outside the root, and a red validator no longer deletes the operator's link (#239) Oct 12, 2026
@brettheap

Copy link
Copy Markdown
Contributor Author

(lane openRepoShape-2, 2026-10-12T00:53:55Z)

This lane re-verified its own PR #269 at ca68458 after fix round 1, read-only in its own detached worktrees.

MERGEABLE WITH NITS.

Items:

  1. Junctions: fixed. is_link_info checks S_ISLNK or st_reparse_tag through getattr, and it is 3.9-safe.
    • The cross-platform tests pass here.
    • The three real-junction tests ran only in tests-windows, which passed (2826 passed, Python 3.14.7).
  2. O_EXCL: fixed. The path is recorded after the create.
    • A link, file or dangling directory link arriving during confirm is refused and rolled back (copy, earlier addition, docs/), green and red.
    • The body names the residual: a live directory link arriving above the path.
  3. Wording: fixed.
  4. !r: fixed.
  5. Split assert: fixed. Sonar shows 0 open issues.
  6. README: declined, and named in the body.
  7. Body table: stale again, since main is now be2790e. Nit.

Tests:

  • At ca68458: update set 221 passed, 3 skipped; shape_doctor 181 passed.
  • The PR's module: 31 failed, 7 passed on main; 20 failed on 7bceeca's code, matching the body.
  • Merged with b30595c: 248 and 208 passed. Merged with be2790e: 281 and 208 passed.
  • Mutants: 14 of 15 killed. The survivor, lexists to exists, is redundant.
  • CI: all three OS jobs and SonarCloud pass.

Merges: merge-tree against main b30595c and be2790e, #281 5757e29 and 754803d, and every open head is rc=0. The one exception is 04792d2, which conflicts with main alone.

The full report is at handoffs/xFactory/attachments/lane-openRepoShape-2/review269-7bceeca/reverify-1.md in opensoft/brett-wip.

@sonarqubecloud

Copy link
Copy Markdown

brettheap added a commit that referenced this pull request Oct 12, 2026
…'s base has it, so this branch merges with main's #204 rewording of that same line instead of conflicting on it (#285)

The previous commit added "or a write" to `apply`'s exit-2 line. Main
(c7c02fa, now merged into #269's branch as 8a460e8) rewrote the same line
for #204, and `git merge-tree` against either was a conflict on that line
and nothing else. The line already says "including", so a write that fails
is covered by it, and the "Nothing is left half-done" paragraph above says
so by name (#285).

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 12, 2026
…write-failed`, rolled back, in a bullet of its own above #202's, so the sentence no longer shares a line with the bullet #286 adds after #202's and the two branches merge in either order (#285)

#286 (fix/update-shape-pinned-copy-symlink at ba373db, stacked on the same
#269 base) adds a bullet straight after the #202 paragraph's last line,
which this branch had extended with "A copy, an addition or a pin this
process cannot write is the same (#285)". `git merge-tree` was a conflict
on that line and nowhere else. Said as its own bullet before #202's, the
two edits no longer touch: merge-tree is rc=0 against #286, origin/main
c7c02fa and #269's new head 8a460e8.

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@brettheap
brettheap merged commit c1b6732 into main Oct 12, 2026
5 checks passed
@brettheap
brettheap deleted the fix/update-shape-add-dangling-symlink branch October 12, 2026 12:27
brettheap added a commit that referenced this pull request Oct 12, 2026
… landed as a squash

This branch was stacked on #269 (fix/update-shape-add-dangling-symlink).
#269 landed on main as the squash c1b6732, whose final head was 8a460e8
(which had merged main c7c02fa), and GitHub retargeted this PR to main.
The squash and this branch's copy of #269's original commits touch the
same lines of update-shape.py, so `gh pr update-branch` refused with
conflicts.

8a460e8 was merged first (25e38af, no conflict). Merging origin/main then
conflicted in update-shape.py only. The resolution is this branch's own
version of the whole file: `git diff 8a460e8 origin/main -- update-shape.py
tests/test_update_shape_add_symlink.py` is empty (c1b6732 is the only main
commit touching either file since c7c02fa, and its content is 8a460e8's),
so this branch's version carries everything main has plus this PR's change.
tests/test_update_shape_add_symlink.py merged without a conflict.

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 12, 2026
…symlink

This branch was stacked on #269's head ca68458. #269 landed on main as the
squash c1b6732 after its head moved to 8a460e8 (ca68458 plus main c7c02fa), so
merging 8a460e8 first brings this branch to #269's final head before main
is merged on top (#286).

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 12, 2026
#269 landed on main as the squash c1b6732, whose content of update-shape.py
and tests/test_update_shape_add_symlink.py is 8a460e8's byte for byte
(git diff 8a460e8 origin/main on those two files is empty). This branch
had just merged 8a460e8, so both files conflicted (add/add in the test file,
content in update-shape.py) against the squash that carries the same lines
under a different history. Both are resolved by taking this branch's side
of the whole file: it is 8a460e8's content plus this branch's own #286
changes, which main lacks. Every other file merged without a conflict.

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 12, 2026
…, #253 via #264, #302 via #318, #288 via #309, #279 via #310, #306 via #311, #334 via #336, #239 via #269, #243 via #268, #321 via #335) into fix/adopt-record-submodule-placement

Brings in the twelve commits from c7c02fa to a1b3a4b on top of 71b58a1,
merged by sha. No rebase. Among the files this branch changes:
346079d (#216) and e43fa95 (#243) change adopt-project.py; 34ecbc2 (#260)
changes scripts/shape_materialize.py; 87373a6 (#306) changes
shape-doctor.py and tests/test_shape_doctor.py; 7491530 (#288) changes
AGENTS.md and tests/test_repo_hygiene.py.

Two conflicts, each resolved keeping both sides:

- adopt-project.py, the module docstring's `execute` paragraph. #243 ends
  "the root's only submodules are the two leg mounts, each at the commit
  pushed to its leg."; this branch follows "two leg mounts." with "The
  split's `project.yaml` records the leg each source submodule kept in a
  leg was answered into, under `submodules:`, for `shape-doctor.py` to
  check it against." The merge has #243's sentence, then this branch's,
  re-wrapped to the paragraph's width. The module docstring is not in
  `--help` (docs/cli.md holds neither sentence).
- tests/test_repo_hygiene.py, `test_agents_md_is_short_enough_to_be_read`.
  Each side moved the AGENTS.md cap from 464 and wrote a paragraph
  saying why: #288 to 477, for thirteen lines, and this branch to 472,
  for eight. AGENTS.md itself merged cleanly and holds both: 485 lines,
  main's 477 plus this branch's +9 -1. The merge keeps #288's paragraph
  word for word, then this branch's, renumbered "2026-10-12: 477 -> 485"
  because it now follows #288's, and asserts `<= 485`.

`git merge-tree --write-tree` of the two parents gives the same tree in
every file but these two.

Checked:

- adopt-project.py: `_commit_the_split` was changed by both sides, in
  hunks that do not touch. #243 clones the assembly with `--template=`;
  this branch builds `append` with the `submodules:` record. The merged
  function holds both, and each side's text is unchanged. Every other
  definition one side changed is that side's text.
- shape-doctor.py: #306 changes `validator_row` and `run_validator`.
  This branch's placement functions, F1's
  `gitmodules_the_record_answers` among them, are this branch's text.
- No top-level name is bound twice in adopt-project.py,
  scaffold-project.py, scripts/repo_shape.py,
  scripts/shape_materialize.py, shape-doctor.py, validate-manifest.py,
  update-shape.py, scripts/family.py or scripts/bump-leg.py. Each merged
  module's names are the base's plus what either side added.
- AGENTS.md is main's plus exactly this branch's changed lines.
- The branch's diff against main is #189 alone: 12 files, +1694 -157. That
  is +1670 at 27d039b, plus F1's fix and test (+23), plus the one line
  the docstring re-wrap adds.

Lane: openreposhape-1 (openRepoShape-1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 12, 2026
… of main's commits merge with no conflict, among them #243's template-less clone of the assembly and #216's literal `git grep` in adopt-project.py, and #260's one-pass `render` in shape_materialize.py

origin/main moved 71b58a1 -> 8cdca5e. It was merged by sha, as the
coordinator asked for e43fa95 or later:
- c7c02fa (#299 via #315), scripts/family.py;
- 346079d (#216 via #228), adopt-project.py: `plan`'s follow-up
  `git grep`;
- 34ecbc2 (#260 via #316), scripts/shape_materialize.py: `render`
  substitutes in one pass;
- 86da808 (#253 via #264), scripts/bump-leg.py;
- 6f4ce47 (#302 via #318), scripts/family.py;
- 7491530 (#288 via #309), AGENTS.md, README, docs and
  scaffold-project.py's --help;
- 7591455 (#279 via #310), tests/conftest.py: `git()` takes its default
  identity as `-c` config;
- 87373a6 (#306 via #311), shape-doctor.py;
- 8a14ac9 (#334 via #336), tests/test_update_shape_landing.py;
- c1b6732 (#239 via #269), update-shape.py;
- e43fa95 (#243 via #268), adopt-project.py: the assembly root is cloned
  with `--template=`, and the verification refuses a mount whose gitlink
  is not the commit pushed to its leg (`adopt-mount-commit`);
- a1b3a4b (#321 via #335), tests/test_git_output_decoding.py;
- 8cdca5e (#295 via #312), the template's scripts/validate-pins.py.

`git merge-tree` and this merge give the same tree, 07e6b49.
adopt-project.py, scripts/shape_materialize.py and
tests/test_adopt_ignored_shape_files.py auto-merge. In the last of these
main's 346079d and this branch's 86a5e30 change different cases.

Checked on the merged tree:
- AST (fin183g/names.py, base 71b58a1): no top-level name is bound twice
  in adopt-project.py, scaffold-project.py, scripts/repo_shape.py or
  scripts/shape_materialize.py, and none is loaded but bound nowhere.
  Main changed the signature of no function this branch calls.
- Every sentence 86a5e30 changed is still there: in `_through`, in
  `_link_parts`, in `_verify`'s `_resolve` (which e43fa95's
  `adopt-mount-commit` leaves as it was), and in the three test files.
- `_commit_the_split` now clones the assembly with `--template=`. The
  next commit, F3, reads what that clone holds in its own `.git`, which
  is why F3 comes after this merge.

Lane: openreposhape-1 (openRepoShape-1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 12, 2026
brettheap added a commit that referenced this pull request Oct 13, 2026
…ifest kept as a symlink, or under one, is refused as `update-rewrite-symlink` before anything is written, and `check` calls such a copy `copy-is-link`, so `apply` no longer writes the standard's bytes at the link's end and reports a commit that does not hold them (#286) (#347)

`apply` opens each file it rewrites `r+b`, which follows a link at the path
or above it. A pinned copy, the pin or the manifest kept as a symlink, or
under a directory link, had the standard's bytes written at the link's end,
outside the root if that is where it points, while git holds the link and
not those bytes, so the commit `--branch` makes left them out. #269 refused
this for `--add`, which creates a file; this is the same refusal for the
files `apply` overwrites.

`apply` now refuses before it writes anything, as `update-rewrite-symlink`,
and a Windows junction counts as a link too.

- A pinned copy: `_classify_row` gives the row its own verdict,
  `copy-is-link`, counted with the conflicts, and `check` prints the link
  and where it points. `apply` refuses every such row at once through
  `_refuse_linked_copies`, ahead of `update-conflict`, whose remedy does not
  fit a link. A dangling link at a copy stays `copy-missing`.
- The pin and the manifest: `read_pin` refuses a linked pin and `prepare` a
  linked manifest, both asking `linked_record`, before the upstream is
  opened, so `check` refuses them too, exit 2.
- A link that arrives while `confirm` waits: `Rollback.write` walks the path
  again before it opens it and refuses a link by name, and the refusal arm
  puts back what was already written.

One path is one line, as in `update-add-symlink`, and several `copy-is-link`
rows list one path per line. #269's sentence now says the command never
writes through a symlink, and says symlink and not link, because a
hard-linked copy is still rewritten through its other name.

`shape-doctor.py` answers the same roots. `drifted_file_reasons` names
`copy-is-link`, so a linked copy is `DRIFTED (1 copy-is-link)` and not the
catch-all `INVALID (shape-currency)`. A linked pin, `contracts` directory or
manifest is asked about first in `check_shape_currency`, by
`linked_record_row`, before the pin is read or the standard opened as the
upstream, because it is a fact about the root and not this checkout failing
to ask. The row is a finding with no `environment`: `INVALID
(shape-currency)`, exit 1, quoting update-shape.py's refusal. Filed with
`prepare`'s other refusals, such a root read `CANNOT ANSWER`, exit 3, with a
`git fetch --all` that changes nothing as its NEXT. The NEXT now says to
replace the link with what it points at, commit that and run the doctor
again, in words and not as a command, since removing a link is a person's
call.

`tests/test_update_shape_rewrite_symlink.py` and
`tests/test_shape_doctor.py` hold each refusal and the doctor's answers.

Closes #286

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap pushed a commit that referenced this pull request Oct 14, 2026
…a directory goes is one at ANY depth above the path, and `update-add-would-overwrite` sits beside the refusal it names; the codes test's history and scan are stated as they are (#550, #429)

From the independent review of 2ca2d78.

- README.md and docs/handbook.html: "one whose directory the root holds as a
  file" is now "one with a file where a directory above it goes", which is what
  update-shape.py refuses (`Addition.blocked` is the first directory ABOVE the
  path, at any depth, that the root keeps as a file or a FIFO: `--add
  .github/workflows/x.yml` is refused when `.github` is a file), and mirrors
  "a symlink at or above it". README.md's `update-add-would-overwrite` now
  follows "one the root ALREADY has a file at", and the reason for the refusal
  follows the code, as the handbook already had it. README.md is still 1427
  lines, so tests/readme-cap.d/550.txt stays +5.
- tests/test_update_add_codes_documented.py: the history says that the
  sentence named three and that #269 and #346 (both landed 2026-10-12) each
  added a refusal, not "for months". The scan is
  `test_agents_md_codes.printed_strings` filtered by the code's shape, so a
  docstring and the pieces of an f-string are left out, as its docstring says.
- Handbook markers: README's "Keeping a project's shape current" moved again
  (9d21bfcf6a23fc23b4d2dd6a18a4f32f75ee400b). `current` carries the corrected
  sentence; `doctor` was read again and reads the doctor subsection, which did
  not move, so its marker moves alone.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 15, 2026
…affold with the `--visibility` and `--elected-by` setup passes, and `apply --add` names all five refusals it gives by their codes, each held by a test (#550, #429) (#579)

README.md and the handbook printed the by-hand dry run of the scaffold
without the `--visibility` and `--elected-by` that setup-project.py's
`scaffold_args` always passes (#550). They also named three of the five
refusals of `update-shape.py apply --add`, with no codes (#429).

- Both documents now print the plan as setup runs it:
  `--visibility private --elected-by 'Your Name' --dry-run`.
- Both name all five `apply --add` refusals, each with its code:
  `update-add-already-pinned`, `update-add-not-upstream`,
  `update-add-symlink` (#239, added by PR #269),
  `update-add-not-a-directory` (#285, added by PR #346) and
  `update-add-would-overwrite`.
- README grows by 5 lines. tests/readme-cap.d/550.txt raises the cap by
  +5 and argues it.
- The handbook markers for § Starting and § Keeping are re-set, after a
  re-read of every part that reads them.
- tests/test_update_add_codes_documented.py ties the documents to the
  tool's codes. tests/test_setup_by_hand_commands.py ties them to the
  arguments setup hands the scaffold.

Lane: openreposhape-3 (openRepoShape-3)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant