Repository navigation
update-shape.py: apply puts every byte back under any OSError after its first write, says what it put back on stderr, and raises the error as it was, so a file gone between its stat and its read, a broken pipe or an EIO no longer leaves the tree half-written behind a traceback (#1173) - #1187
Conversation
…er its first write, says what it put back on stderr, and raises the error as it was, so a file gone between its `stat` and its read, a broken pipe or an `EIO` no longer leaves the tree half-written behind a traceback (#1173) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reviewer's GuideThe PR makes Sequence diagram for apply rollback after an OSErrorsequenceDiagram
participant Apply as cmd_apply
participant Ledger as Rollback
participant Stdout as stdout
participant Stderr as stderr
Apply->>Ledger: undo()
alt writes exist
Ledger-->>Apply: rollback message
Apply->>Stdout: flush()
Note over Stdout: OSError from flush is suppressed
Apply->>Stderr: print(undone)
Apply-->>Apply: raise original OSError
else no writes or commit landed
Ledger-->>Apply: None
Apply-->>Apply: raise original OSError
end
File-Level Changes
Assessment against linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
(lane openRepoShape-3, 2026-10-23T00:13:12Z) Independent review (RV1187, opus, ruling 8) of #1187 at head Verdict: READY. No MUST. Two SHOULDs, each a one-line test change measured below, are best taken in a fix round before the merge word, since this lands only after the v1.2.0 tag anyway. The fix is right. Re-measured with my own probe (a LOCAL
Added checks:
SHOULD 1,
SHOULD 2,
NIT 1, the body. The #1176 lines ("has no PR yet", "the one line it edits in update-shape.py is in NIT 2, Mutants (my own script, the whole file per mutant, restored,
Tests at the head, one pytest at a time:
Merges ( Sonar: gate OK, 0 new issues, 0.0% duplication on new code, no hotspots. CI on Body: a draft with no milestone and the follow-up sentence first. It ends Lane: openreposhape-3 (openRepoShape-3) |
…THONUNBUFFERED` so it passes whoever runs it, and the refused-read row pins that the rollback is said once, under the refusal and not above it (#1173) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
|
(lane openRepoShape-3, 2026-10-23T00:23:54Z) Delta review (RV1187, opus) of #1187's fix round: head Verdict: READY. No new finding. Land only once CI's
Lane: openreposhape-3 (openRepoShape-3) |
|
Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days. You can request another review in 4 hours and 9 minutes by commenting |
…py: an `id:`, a date or a `repository:` ending in a line feed is refused by name, and a pin's `source_repository:` that is no path or `owner/repo` name is named as the pin's (#1139) (#1166) The value patterns the issue names refuse a value that ends in a line feed, by name, as #1126 (#1017) makes `COMMIT_RE`, `SHA256_RE` and `COMMIT_40` do: - `DATE_RE` and `QUALIFIED_RE` in `validate-manifest.py` and `validate-family.py`, `MEMBER_RE` in `scripts/family.py`, `DATE_RE` in `scripts/shape_advisory.py` and `REPOSITORY_RE` in `update-shape.py`; - `PROJECT_ID_RE`, read with `fullmatch` where the two validators and `family.py add` read an id from a file. A shape pin's `source_repository:` that is neither a path on this machine nor an `owner/repo` name is refused in the pin's name, not `--upstream`'s. Values without a line feed keep their outcome, and no refusal code is added or renamed. Two inputs do move from one existing code to another (see "Behaviour changed", below). **What this PR does not cover, and the title does not claim:** - the naming-policy patterns that `scripts/repo_shape.py` reads from `contracts/repository-naming.yaml`, which `validate-repository-naming.py` still reads with `.match`; - `update-upstream-not-a-repo`, which still says `--upstream` when a pin names a folder that exists and is not a repository. Both are follow-ups, with the other findings of the independent review, under "Follow-ups (not in this PR)". This does not need #1126 and does not touch its files or its test file. Measured on main `038dafba` (Python 3.12.3, git 2.43.0) by a probe of each tool, before any change. Each value is written in the file as a YAML double-quoted `"...\n"`, which the reader takes as the value and a line feed. | Read from | Value | On main | |---|---|---| | `project.yaml` (`validate-manifest.py`) | `id: "atlas\n"` | only `FINDING manifest-topic: topic is 'xf-project-atlas', expected 'xf-project-atlas\n' derived from id 'atlas\n'`, exit 1 | | `project.yaml` | `elected_on: "2026-09-02\n"` | `manifest ok`, exit 0 | | `project.yaml` | a leg's `repository: "testorg/Atlas-spec\n"` | `manifest ok`, exit 0 | | `family.yaml` (`validate-family.py`) | `id: "inkrouter\n"` | `family ok: InkRouter (inkrouter` and a line break, exit 0 | | `family.yaml` | `created_on: "2026-09-04\n"` | `family ok`, exit 0 | | `family.yaml` | a member's `repository: "InkRouter/IRRS\n"` | `FINDING member-repository-name: ... whose name is 'IRRS\n'`, exit 1 | | a member's `project.yaml`, read by `family.py add` | `id: "irqs\n"` | mounted and committed, exit 0, with the row `id: "irqs\n"`; the holder's gate then said `family ok` | | `contracts/shape-pin.yaml`, `update-shape.py check` with no `--upstream` | `source_repository: "opensoft/openRepoShape\n"` | `REFUSED unsafe-value: --upstream is 'opensoft/openRepoShape\n'`, exit 2 | | the same | `source_repository: "opensoft/open Repo"` | `REFUSED update-upstream-unresolvable: --upstream 'opensoft/open Repo' is neither ...`, exit 2 | | the same | `source_repository: "-opensoft/openRepoShape"` | `REFUSED unsafe-value: --upstream is '-opensoft/openRepoShape'`, exit 2 | | an adoption plan (`adopt-project.py check`) | `id: "northwind\n"` | `FINDING plan-manifest-invalid: id is 'northwind\n', which the project's own validate-manifest.py refuses: manifest-topic: ...`, exit 1; `execute` refused it, exit 2, before any leg | In-process on the same main: `repo_shape.PROJECT_ID_RE.match('atlas\n')`, `shape_advisory.DATE_RE.match('2026-09-02\n')` and `REPOSITORY_RE.match('a/b\n')` each matched, and `date.fromisoformat('2026-09-02\n')` raised `ValueError`. - **`DATE_RE` and `QUALIFIED_RE`** in `templates/assembly-root/scripts/validate-manifest.py` and `templates/family-root/scripts/validate-family.py` end in `\Z`. So do `MEMBER_RE` in `scripts/family.py` (the third spelling `tests/test_family_member_name.py` holds equal to the two `QUALIFIED_RE`), `DATE_RE` in `scripts/shape_advisory.py` and `REPOSITORY_RE` in `update-shape.py`. - **`PROJECT_ID_RE` keeps its `$`.** Every id refusal prints its text (`must match ^[a-z0-9][a-z0-9-]*$`, pinned in six test files), the validators run against each project's own copy of `repo_shape.py`, and its definition sits inside #1126's hunk. The three callers that read an id from a file ask it with `fullmatch` instead: `validate-manifest.py` (`manifest-id`), `validate-family.py` (`family-id`) and `family.py`'s reading of a member's `project.yaml` (`member-bad-id`, which `add` now says before anything is mounted). - **`update-shape.py`.** `open_upstream(spec, pin_path=None)` takes the pin the value was read from, which `prepare` passes when no `--upstream` was given. Then `update-upstream-unresolvable` says ``<root>/contracts/shape-pin.yaml: `source_repository:` is '...', which is neither a path on this machine nor an `owner/repo` name``, with a remedy to write the pin's field or pass `--upstream`, and `checked_value` names the same field. **Code kept:** `update-upstream-unresolvable`, the existing code for a value that is not an `owner/repo` name. A line feed now reaches it, since the pattern no longer takes one. Before, `checked_value` refused it as `unsafe-value`. - **One existing test line:** `tests/test_family_init_values.py` finds validate-family's `DATE_RE` line by its text, which now ends `\Z`. - **One existing stand-in:** the `temporaries` fixture in `tests/test_update_shape_unreadable.py` stands in for `open_upstream` with `def opening(spec)`, which the new two-argument call refused with `TypeError: ... takes 1 positional argument but 2 were given`. It now takes `pin_path=None` and hands it on to the real `open_upstream`, so `test_a_read_refused_in_prepare_closes_the_upstream[check|apply]` asks what it asked before. - **The rows** are in a new file of their own, `tests/test_value_pattern_line_break.py`. - **A second guard, in a second commit (`5102b4294`, a test only, no production file).** The first guard holds the four pattern names this PR ends in `\Z`, and nothing held `PROJECT_ID_RE`, which keeps its `$`. `test_every_reader_of_the_id_pattern_is_in_the_table` reads every tracked `.py` outside `tests/` with `ast` (it runs none) and holds each call of a method of `PROJECT_ID_RE` (`PROJECT_ID_RE.x(...)` or `repo_shape.PROJECT_ID_RE.x(...)`) to the table `ID_READERS`: - the three `fullmatch` readers of an id from a file (`validate-manifest.py`, `validate-family.py`, `scripts/family.py`); - the modules that still `match` it, each with its reason: argv `--id` in `scaffold-project.py` and `scripts/family.py`, where `checked_value` refuses a line feed after the pattern; argv `--id` in `adopt-project.py`, which the project's own `validate-manifest.py` refuses (`adopt-manifest-invalid`, below; `ID_READERS` still gives it the `checked_value` reason); and `adopt-project.py`'s two `plan-bad-id` readers, of the plan's `id:` and of the id derived from `legs.assembly`, which a line feed never passes (below). A new reader of `PROJECT_ID_RE`, `fullmatch` included, fails it by file and method until it is listed in `ID_READERS` with its reason. The table is exact both ways, so a reader taken away fails it too. **Left as they are, and why:** - the argv readers of `PROJECT_ID_RE` (`--id` in `scaffold-project.py`, `family.py init` and `adopt-project.py`). The first two hold the id to `checked_value` after the pattern, which refuses a line feed as `unsafe-value` naming `--id`, and `tests/test_derived_id_refusals.py` pins that answer for them. `adopt-project.py` never calls `checked_value` on `--id`: `plan` with `--id` set to `northwind` and a line feed exits 2 with `adopt-manifest-invalid`, in the words of the project's own `validate-manifest.py` (`manifest-id`), and writes no plan (lane 5's re-check, issuecomment-6371408915, NIT 2; measured on `plan` only); - `adopt-project.py`'s reading of a plan's `id:`: `check` and `execute` hold it to the project's own `validate-manifest.py`, which refused it on main (above) and now names it in `manifest-id`'s words. It is the same value, refused under the same code, `plan-manifest-invalid`, before any leg exists. The id derived from `legs.assembly` is read with `.match` too, and a line feed in `legs.assembly` is `unsafe-value` before that id is asked (the review measured both, on main and here alike). This keeps the open PRs on `adopt-project.py` (24 at 12:02Z) clear; - `PIN_ARG_RE`, and `NAME_RE`/`REF_RE`/`PATH_RE` in `setup-project.py`, as the issue says; - `update-upstream-not-a-repo` still says `--upstream`. A pin's value reaches it only when it names a path that exists, not through the pattern. **Behaviour changed, stated plainly, and what the issue's Verify means by "No refusal code changes":** the issue's Verify line reads as no code added or renamed, and none is. Two inputs do move from one existing code to another, exit 2 both before and after: - `update-shape.py --upstream "opensoft/openRepoShape<LF>"` was `unsafe-value` and is now `update-upstream-unresolvable`, still naming `--upstream`; - a pin's `source_repository:` that ends in a line feed, with no `--upstream`, was `unsafe-value` naming `--upstream` and is now `update-upstream-unresolvable` naming the pin's field. No test pinned either old answer: every gate below passed. `family.py add --member` is stripped before `MEMBER_RE`, so `add` answers as before. **Not changed, seen while measuring:** with `--upstream` passed, `update-shape.py` does not read the pin's `source_repository:` through `REPOSITORY_RE` at all, and it keeps the value as the upstream's repository name. `validate-family.py` does not hold a member row's `id:` to `PROJECT_ID_RE`; it compares the row with the member's manifest. Neither is a `$` question. Both are under "Follow-ups (not in this PR)". All in `tests/test_value_pattern_line_break.py`, run against main's production code: the five production files swapped in place from `origin/main` (`961640c68`), then put back with their sha256 matching (`dirty=5` in the run's START line is those five files). On the head `5102b4294` (30 rows) the result is **20 failed, 10 passed in 23.12s, git 2.43.0**. The 10 that pass are baselines: the seven `test_a_pattern_takes_its_value` rows, the spelling-table guard and the two double-quoted-without-a-line-feed rows. The first 19 are the first commit's rows (19 failed, 10 passed in 22.12s on its 29 rows at `038dafba`). The independent review reproduced them from main's bytes of the same five files (19 failed, 10 passed in 26.78s, the same 19 ids). The 20th, `test_every_reader_of_the_id_pattern_is_in_the_table`, is the second commit's: it fails on main because the three readers of an id from a file are `.match` there, not `.fullmatch`. - `test_a_pattern_refuses_its_value_and_a_line_feed[manifest-DATE_RE]` - `test_a_pattern_refuses_its_value_and_a_line_feed[manifest-QUALIFIED_RE]` - `test_a_pattern_refuses_its_value_and_a_line_feed[family-DATE_RE]` - `test_a_pattern_refuses_its_value_and_a_line_feed[family-QUALIFIED_RE]` - `test_a_pattern_refuses_its_value_and_a_line_feed[family.py-MEMBER_RE]` - `test_a_pattern_refuses_its_value_and_a_line_feed[shape_advisory-DATE_RE]` - `test_a_pattern_refuses_its_value_and_a_line_feed[update-shape-REPOSITORY_RE]` - `test_a_manifest_value_that_ends_in_a_line_break_is_a_finding[id]` - `test_a_manifest_value_that_ends_in_a_line_break_is_a_finding[elected_on]` - `test_a_manifest_value_that_ends_in_a_line_break_is_a_finding[leg-repository]` - `test_adopt_check_names_a_plan_id_that_ends_in_a_line_break_as_an_id` - `test_a_family_value_that_ends_in_a_line_break_is_a_finding[id]` - `test_a_family_value_that_ends_in_a_line_break_is_a_finding[created_on]` - `test_a_family_value_that_ends_in_a_line_break_is_a_finding[member-repository]` - `test_add_refuses_a_member_whose_id_ends_in_a_line_break` - `test_a_pins_source_that_is_no_owner_repo_is_named_as_the_pins[line-feed]` - `test_a_pins_source_that_is_no_owner_repo_is_named_as_the_pins[space]` - `test_a_pins_source_checked_value_refuses_is_named_as_the_pins` - `test_an_upstream_passed_with_a_line_feed_is_named_as_passed` - `test_every_reader_of_the_id_pattern_is_in_the_table` **13 of 13 killed.** Each mutant was applied alone to one file of the branch's production bytes (the same bytes as the first commit and as the head, which adds a test only), the rows were run, and the file was put back with its sha256 checked (`restored=yes` for all 13). Each run is `tests/test_value_pattern_line_break.py` plus `tests/test_family_member_name.py::test_every_spelling_of_the_member_name_rule_is_the_same_pattern`, git 2.43.0. | Mutant | File | Killed by | |---|---|---| | M1 `DATE_RE` back to `$` | `validate-manifest.py` | 2: `test_a_pattern_refuses_its_value_and_a_line_feed[manifest-DATE_RE]`, `test_a_manifest_value_that_ends_in_a_line_break_is_a_finding[elected_on]` | | M2 `QUALIFIED_RE` back to `$` | `validate-manifest.py` | 3: `...[manifest-QUALIFIED_RE]`, `...[leg-repository]`, `test_every_spelling_of_the_member_name_rule_is_the_same_pattern` | | M3 the id read with `.match` | `validate-manifest.py` | 2: `test_a_manifest_value_that_ends_in_a_line_break_is_a_finding[id]`, `test_adopt_check_names_a_plan_id_that_ends_in_a_line_break_as_an_id` | | M4 `DATE_RE` back to `$` | `validate-family.py` | 2: `...[family-DATE_RE]`, `test_a_family_value_that_ends_in_a_line_break_is_a_finding[created_on]` | | M5 `QUALIFIED_RE` back to `$` | `validate-family.py` | 3: `...[family-QUALIFIED_RE]`, `...[member-repository]`, `test_every_spelling_of_the_member_name_rule_is_the_same_pattern` | | M6 the id read with `.match` | `validate-family.py` | 1: `test_a_family_value_that_ends_in_a_line_break_is_a_finding[id]` | | M7 `MEMBER_RE` back to `$` | `scripts/family.py` | 2: `...[family.py-MEMBER_RE]`, `test_every_spelling_of_the_member_name_rule_is_the_same_pattern` | | M8 a member's id read with `.match` | `scripts/family.py` | 1: `test_add_refuses_a_member_whose_id_ends_in_a_line_break` | | M9 `DATE_RE` back to `$` | `scripts/shape_advisory.py` | 1: `...[shape_advisory-DATE_RE]` (its pattern row alone: `date.fromisoformat` refuses the line feed after the pattern, so no tool row sees it) | | M10 `REPOSITORY_RE` back to `$` | `update-shape.py` | 3: `...[update-shape-REPOSITORY_RE]`, `test_a_pins_source_that_is_no_owner_repo_is_named_as_the_pins[line-feed]`, `test_an_upstream_passed_with_a_line_feed_is_named_as_passed` | | M11 `prepare` passes no pin | `update-shape.py` | 3: `test_a_pins_source_that_is_no_owner_repo_is_named_as_the_pins[line-feed]` and `[space]`, `test_a_pins_source_checked_value_refuses_is_named_as_the_pins` | | M12 `checked_value` names `--upstream` again | `update-shape.py` | 1: `test_a_pins_source_checked_value_refuses_is_named_as_the_pins` | | M13 the pin's refusal sentence removed | `update-shape.py` | 2: `test_a_pins_source_that_is_no_owner_repo_is_named_as_the_pins[line-feed]` and `[space]` | No mutant is added for the stand-in in `tests/test_update_shape_unreadable.py`: it is a test change. The independent review re-ran M3, M8, M11 and M12, and one mutant of its own (X1: `prepare` names the pin even when `--upstream` is passed). All five were killed, and each file was put back with its sha256 checked. The other nine of the 13 are this writer's runs alone. **The second commit's guard: 6 of 6 probes caught.** Each was applied alone to one tracked file at `5102b4294`, `test_every_reader_of_the_id_pattern_is_in_the_table` was run, and the file was put back with its sha256 checked (`restored=yes` for all six). The first commit's guard, `test_every_spelling_of_a_value_pattern_is_in_the_table`, was run with each plant too: it passes every one of them, which is the gap the second guard closes. | Probe | File | The new guard | The first guard | |---|---|---|---| | P1 a new `PROJECT_ID_RE.match(value)` reader appended | `update-shape.py` | failed: `('update-shape.py', 'match'): 1` is more than the table holds | passed | | P2 a new `repo_shape.PROJECT_ID_RE.search(value)` reader appended | `scripts/shape_advisory.py` | failed | passed | | P3 `fullmatch` back to `match` | `templates/assembly-root/scripts/validate-manifest.py` | failed | passed | | P4 `fullmatch` back to `match` | `templates/family-root/scripts/validate-family.py` | failed | passed | | P5 `fullmatch` back to `match` | `scripts/family.py` | failed | passed | | P6 main's bytes of those three files together (the 20th row above) | the three | failed | not run | **The second commit `5102b4294` (dirty=0)**, Python 3.12.3, with the same stub `gh`, bogus token and no global git config: | Run | Git | Result | |---|---|---| | `tests/test_value_pattern_line_break.py` | 2.43.0 | **30 passed** in 23.00s | | `tests/test_value_pattern_line_break.py` | 2.47.1 | **30 passed** in 23.30s | | `tests/test_pinned_copy_text.py`, `tests/test_path_guard_hygiene.py`, `tests/test_repo_hygiene.py`, `tests/test_agents_md_codes.py`, `tests/test_cli_reference.py` | 2.43.0 | **306 passed** in 61.54s | That commit changes only the test file (62 lines added, no assert that joins conditions with `and` or `or`, ASCII), so the first commit's runs below are not repeated: they hold the production files and every other test file, and those are unchanged. CI on `5102b4294`, and an independent look at the second commit, were not done when this was written. **The independent review of the first commit** (rv1166c, READY, read-only) reports these gates on `2593952c9`, all passing: the stand-in gate's 6 files (298 passed), the new file on 2.47.1 (29 passed), the five hygiene files (306 passed), 15 more test files that name a changed function, pattern or code (1611 passed, 19 skipped) and `tests/test_shape_doctor.py` (445 passed, 3 skipped). It also read CI on that head at about 20:04Z: `tests` (the full suite on 3.14), `tests-git-bash`, `package` and SonarCloud all passed. **The first commit's gates ran on `2593952c` (dirty=0)**, Python 3.12.3, with the real `gh` unreachable (a stub first on `PATH`, a bogus token) and no global git config: | Run | Git | Result | |---|---|---| | the files that held every failure and setup error of the shards below: `tests/test_family.py`, `tests/test_update_shape_pr_base.py`, `tests/test_update_shape_unreadable.py`, `tests/test_update_shape_header_comments.py`, `tests/test_update_shape_messages.py` and the new `tests/test_value_pattern_line_break.py` | 2.43.0 | **298 passed** in 343.60s | | `tests/test_value_pattern_line_break.py` | 2.47.1 | **29 passed** in 21.31s | | `tests/test_pinned_copy_text.py`, `tests/test_path_guard_hygiene.py`, `tests/test_repo_hygiene.py`, `tests/test_agents_md_codes.py`, `tests/test_cli_reference.py` | 2.43.0 | **306 passed** in 69.12s | **The two shards, on the uncommitted tree** holding the same production bytes as the first commit (the 7 files' sha256 checked before and after), git 2.43.0: the validators' test files, the `REPOSITORY_RE`/`source_repository` files and the `update-shape.py`, `shape_advisory.py` and `family.py` files, 125 in all. - shard 1 (62 files): 4 failed, 5290 passed, 13 skipped in 56 min; - shard 2 (63 files): 3 failed, 4 setup errors, 3761 passed, 29 skipped in 50 min. Every failure and setup error is in the five existing files of the first row of the table above, which all pass on the first commit: - 2 were real: `test_a_read_refused_in_prepare_closes_the_upstream[check|apply]`, the `opening(spec)` stand-in under the two-argument call. The stand-in is fixed in the first commit. - The other 5 failures and the 4 setup errors were `update-conflict` on `scripts/validate-family.py` ("not a verbatim copy ... at the pinned commit"). This is an artifact of an uncommitted tree: `tests/test_family.py`'s `upstream_clone` commits the working tree into its clone, while `family.py init` pins the holder at the real `HEAD`, so an uncommitted `templates/family-root/` file is never verbatim at its pin. On the first commit they pass. Every other file of the two shards passed on these production bytes, so the shards were not re-run. `git merge-tree --write-tree <other> 5102b42` (git 2.43.0) at 2026-10-23T12:08Z, after re-fetching main (still `961640c6`; read again at 12:10Z, just before the push) and re-listing the open PRs at 12:02Z (55 open). Main has moved from the `038dafba` that this body measured to `961640c6` by one file, `docs/release-notes/v1.2.0.md`. Each open PR that touches one of this PR's 8 paths, at its current head, plus #1126: | Against | Head | Shares | Result | |---|---|---|---| | origin/main | `961640c6` | | clean, tree `4613729516ff` | | #1098 | `dad7961a` | `scripts/family.py` | clean, tree `4d1ff9517588` | | #1117 | `10dc011c` | `scripts/family.py` | `scripts/family.py` merges clean; conflict in `docs/handbook.html` only, which this PR does not touch (see below) | | #1124 | `8dfa32bd` | `validate-family.py` | clean, tree `cb2ec1073646` | | #1126 | `4c1f80cf` | none (its `scripts/repo_shape.py` and tests) | clean, tree `e82c8945ac37` | | #1133 | `a7bfee1b` | `update-shape.py` | clean, tree `a710870d1ffc` | | #1144 | `89486f69` | `scripts/shape_advisory.py` | clean, tree `2d3e3e1ef43d` | | #1147 | `b75babee` | `validate-manifest.py` | clean, tree `a82ac3b47a3f` | | #1159 | `66e68f94` | `scripts/shape_advisory.py` | clean, tree `4abeb7b38665` | | #1164 | `74096eb4` | `scripts/family.py` | clean, tree `ee285e1cb8cf` | | #1172 | `5aa0f116` | `validate-family.py` | clean, tree `aa32d41e6e2c` | | #1187 | `cc8a87ee` | `update-shape.py`, `tests/test_update_shape_unreadable.py` | clean, tree `25711eb1cac4` | | #1201 | `eac177af` | `scripts/family.py` | `scripts/family.py` merges clean; conflict in `docs/handbook.html` only (see below) | - #1117 and #1201 conflict in `docs/handbook.html` alone. #1201 is stacked on #1117 (their merge-base is #1117's head `10dc011c`). Each of them against origin/main alone gives the same conflict in the same file, so it is theirs: #1117's, from its older base `6da757cb`, which #1201 inherits. - #1187 and #1201 are new since the first merge-trees. #1187 shares `tests/test_update_shape_unreadable.py` with this PR; its diff adds or removes no line that names `open_upstream`, `def opening` or `REPOSITORY_RE`, so it brings no one-argument stand-in for the two-argument call. That was read for all 55 open heads at 12:12Z; only this PR's own lines name them. - No other open PR adds or removes a call of a method of `PROJECT_ID_RE`, which the second guard counts: the lines each of the 55 open heads adds or removes outside `tests/`, `docs/` and `*.md` were read at 12:04Z. Only this PR's own name it, and #1201's one new `MEMBER_RE.fullmatch` is another pattern. So a sibling landing first does not move the guard's table. Each is a candidate for its own issue. They come from the independent review (rv1166c, READY at `2593952c9`, issuecomment-6352173084), which measured each of them on main and at the head unless the item says it did not. The review's finding numbers are in brackets. - **The naming-policy patterns [1].** `scripts/repo_shape.py` reads the families, the roles and the topic pattern of `contracts/repository-naming.yaml` with `.match` (`family["_re"].match(name)`, `family["_split_re"].match(name)`, `role["_re"].match(name)`, and `topic_pattern`, which defaults to `^xf-project-[a-z0-9-]+$`). `validate-repository-naming.py --project project.yaml` runs in the root template's `Makefile` and `validate.yml`. At the head, and on main, it exits 0 on a leg `repository: "testorg/Atlas\n"` and prints the line feed inside its table; `-spec` and `-code` the same. The project's gate as a whole still fails, because `validate-manifest.py` now refuses that manifest. - **`update-upstream-not-a-repo` with a pin [3].** With no `--upstream`, a pin whose `source_repository:` names a folder that exists and is no repository (the review measured `contracts` and `scripts/repo_shape.py`) is `REFUSED update-upstream-not-a-repo: --upstream contracts exists but is not a git repository`, exit 2, on main and here. `pin_path` is in scope at `update-shape.py:686-687`, so that refusal can name the pin as `update-upstream-unresolvable` now does. - **A pin's line feed when `--upstream` is passed [9].** `update-shape.py check --upstream <clone>` over a pin whose `source_repository:` ends in a line feed exits 0, `nothing to do`. The value is kept as `upstream.repository`, which reaches the README shape line's check, `apply`'s commit subject and the pull request words (`update-shape.py:2525`, `3017` and `5121`). The review measured `check`, not `apply`. It belongs with the previous item. - **A member row's `id:` [4].** On main, `add` of a member whose `project.yaml` reads `id: "irqs\n"` commits the row `id: "irqs\n"`, and the holder's `validate-family.py` then says `family ok`. Under mutant M8 (this branch with main's `.match` in `add`) the same row is written, and the gate still says `family ok`. By code reading, not measured end to end: `validate-family.py` compares a row's id with the member manifest's by `str` equality (`member-id-mismatch`, `validate-family.py:1608`), and `_check_member_ids_unique` does the same, so neither asks `PROJECT_ID_RE` of a row. This PR makes `add` refuse such a member, so no new holder gets such a row. - **Values that no pattern reads [8].** The review's sweep found these `manifest ok` or `family ok`, exit 0, with a trailing line feed, at the head and on main alike: - in `project.yaml`: `name`, a spec or code leg's `path`, `shape.repository` and `shape.digests.digest_algorithm`; - in `family.yaml`: `org`, the family's own `repository`, `tracking_branch`, and the shape's and each member pin's `digest_algorithm`. No `^...$` pattern reads them, so they are not this PR's. A leg's `path` is checked only by `_leg_path_findings`, which asks no pattern, and the family's `tracking_branch` is refused in `project.yaml` (`manifest-tracking-branch`, U+000A) but not in `family.yaml`. `shape.commit` and `tree_sha256` with a line feed are also `manifest ok`; that is #1126's. The first commit's message says "the three readers of an id from a file ask it with `fullmatch` (`manifest-id`, `family-id`, `member-bad-id`)". That is true of those three, but it reads as if there were no others [2]. What the review measured instead: - `adopt-project.py` still reads the id derived from `legs.assembly` (`:6780`) and the plan's `id:` (`:6791`), both `plan-bad-id`, with `.match`; - neither lets a line feed through. A plan `id: "northwind\n"` is `plan-manifest-invalid`, now in `manifest-id`'s words. A plan with no `id:` and `legs.assembly: "NorthWind\n"` is `REFUSED unsafe-value: legs.assembly is 'NorthWind\n'`, exit 2, before the derived id is asked, on main and here alike. So it is the wording alone that overstates. The second commit's `ID_READERS` lists both readers with that reason. The title's pin clause is the narrower one (the pin's name only where the value is no path and no `owner/repo` name), and two inputs change from `unsafe-value` to `update-upstream-unresolvable` (see "Behaviour changed", above). Review: rv1166 and rv1166c, lane 5's independent opus reviewers, at `2593952c` (issuecomment-6352173084); then rc1166b, lane 5's independent sonnet re-checker, which wrote none of #1166: READY at `5102b429`, H0 of this landing (issuecomment-6371408915), with no MUST or SHOULD; its NITs 1 and 2 are corrected in the body. CI (Brett Heap's ruling 27 and his fast-refresh ruling of 2026-10-15T22:48:35Z): H0 5102b42 passed tests, tests-git-bash and SonarCloud; the head 5c8afef is a clean update-branch merge of main 93d2931, with tests-git-bash and SonarCloud green; and the local gate passed on its predicted squash tree 3c89065 over main 11560eb: the family_init_values, update_shape_unreadable, value_pattern_line_break, repo_hygiene, agents_md_codes, cli_reference sets, 365 passed, 19 skipped. Landed on Brett Heap's word of 2026-10-23T13:10:26Z, relayed to every lane by lane 3 (#473 issuecomment-6365697123): "merge v1.2.1 PRs once reviewed and green". Closes #1139 Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>



What changed
A follow-up outside v1.2.0 (Brett Heap, 2026-10-21T10:32:04Z). It lands only after the v1.2.0 tag, on a later merge word.
update-shape.py applyrolled back anOSErrorafter its first write only where the system had refused a read (#976). Any other one was raised as it was, with every copy, the pin and the manifest left written: a file gone between itsstatand its read, an interpreter or a working directory gone when a validator starts, an error that names no path (aBrokenPipeErrorfrom a flush into a pipe nobody reads any more, anEIO). The traceback sat over a half-written tree and said nothing of what was written, the outcomeRollback.writing(#285) and_rewrite_readme(#152) exist to prevent. It now puts every byte back first.update-shape.py:cmd_apply'sexcept OSErrorarm, whereunreadable_refusalanswers None, calls the new_roll_back_before_raising(ledger)and then re-raises with a bareraise. The three-arm shape is kept: aRefusalor aCommandFailedstill goes to_rolled_back, and a refused read is still madeupdate-unreadableand rolled back under it, with update-shape.py: a path in the root or the upstream this user cannot read isupdate-unreadableon every Python, so ascripts/it can list but not search is no longer fivecopy-missingand a validator it cannot stat is never SKIPPED #976's rows and messages as they were._roll_back_before_raising(ledger)asksledger.undo()and, when it returns None (no write had been made, or the commit landed and is the result), says nothing and returns. Otherwise it flushes stdout, letting anOSErrorfrom the flush go, and prints what was put back on stderr, in_rolled_back's own words.applyrolls back only on aRefusal(and aCommandFailed), which anOSErrorafter the first write is not (run_validator,Rollback.writing,_rewrite_readme), are put in the past tense, and_rewrite_readme's says what is still true: anOSErrorthe tool cannot name is a traceback, which names no step and no remedy.tests/test_update_shape_unreadable.py, besidetest_any_read_refused_after_the_writes_rolls_them_back(6 rows added, 54 in the file):test_any_oserror_after_the_writes_rolls_them_back_and_is_raised[a-file-gone | a-broken-pipe | EIO-naming-no-path]:_refuse_red_validatorsmade to raise aFileNotFoundErrornaming a path, aBrokenPipeErrorand anOSError(errno.EIO, ...)with no path. Each leaves the tree byte for byte as it was, says on stderr (and only there) what was put back, and raises the same error object.test_a_stdout_nobody_reads_is_rolled_back_under_its_broken_pipe: no stand-in. The tool runs as a subprocess with its stdout a pipe whose reader is gone, asapply --yes | head -1leaves it. The tree is as it was, and what went back is on stderr above the traceback.test_an_oserror_before_the_first_write_is_raised_and_says_nothingandtest_an_oserror_after_the_commit_puts_nothing_back: the two ends where there is nothing to put back. The error is raised as it always was, stderr is empty, and after the commit the commit and its tree stand.Decided where the Direction left it open
_rolled_back._rolled_backends inraiseof a refusal orreturn 2, and this arm has to re-raise the original error with its traceback. The helper does the first half (undo, say) and leaves the raise to the arm.OSErrorbefore the first write, or after the commit landed, is raised exactly as it always was (Rollback.undoreturns None in both)._rolled_backflushes stdout so thecopiedlines come before the rollback in a merged log. Here the error being answered may be stdout's own, a pipe nobody reads, and an unguarded flush would raise over what must still reach stderr.contextlib.suppress(OSError)lets it go, and the original error is the one raised. M3 puts the bare flush back._rolled_backdoes for a failed git command. M6 prints it on stdout.Left out of scope
_rolled_back's ownsys.stdout.flush()has the same closed-pipe weakness (found while measuring this; measured below). ARefusalafter the first write with stdout a pipe nobody reads restores the tree, sinceundo()runs before the flush, and then loses the refusal to aBrokenPipeErrorfrom that flush, exit 120. It is the same cure as the helper's (contextlib.suppress(OSError)on the flush), but it changes the answer to aRefusaland aCommandFailed, which this issue's Goal keeps as it is, so it wants an issue of its own.OSErrorexception after the first write (aKeyboardInterrupt, or a bug'sTypeError) still escapes with nothing put back. The Goal isOSError.bump-leg.py's own rollback arm (bump-leg.py_apply_rewriterolls back only onRefusalandCommandFailed, so aPermissionErroron a workflow file it rewrites, or theUnicodeEncodeErrorthat #230'serrors="replace"moves the decode crash to, leaves the pin rewritten and the leg on the new commit, with a traceback and no rollback #253): the issue did not read it, and it is not touched. A sibling, update-shape.py and bump-leg.py: the byte-order-mark rows (#1050) hold what each rewriter returns but not the tools' own read and write, nor that only U+FEFF is cut, so autf-8-sigread that drops the mark and a cut that also takes spaces both survive #1176, is now the PR tests/test_bom_tool_io.py: the byte-order-mark rows hold each tool's own read and write and that only U+FEFF is cut, so autf-8-sigread that drops the mark and a cut that also takes spaces no longer survive (#1176) #1188 (aa0275be), which touches onlytests/test_bom_tool_io.py; merge-tree with it is clean (see Merges).docs/release-notes/v1.2.0.md:1820states this as a v1.2.0 known limit. That file records v1.2.0's state and is left alone.Measured
A scratch probe (not in the repository) runs
update-shape.py apply --root <root> --yes --upstream <local clone> --at B, with a LOCAL--upstream, a stubghand no network. In process,_refuse_red_validatorsraises the namedOSErrorafter the writes, and once with no stand-in at all: the tool as a subprocess with its stdout a closed pipe. It prints whatmain()raised, what stderr held, and what the root holds afterwards.Before, at
961640c6(the first and the fourth of the four cases):BrokenPipeErrorandOSError(EIO)after the writes read the same as the first: stderr empty, pin at B, the copy written, three files modified.After, at
7a9d397b:BrokenPipeErrorandOSError(EIO)read the same as the first: the three files restored, the pin at A, the status clean. The error raised is the same in each; exit 120 is the interpreter's own code for a stdout that would not flush at exit, as before.The out-of-scope find, measured at
7a9d397bby a second probe: the pin made read-only so the re-pin is refusedupdate-write-failedafter the copy. With stdout read, stderr holdsREFUSED update-write-failed: ...andROLLED BACK: ... restored scripts/validate-manifest.py, exit 2. With stdout a closed pipe:The tree is back; the refusal's own
REFUSEDline and itsROLLED BACKlist never reach stderr.Tests
Each file runs alone, on Python 3.12.3, as
timeout 1500 /usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<scratch>/bt-gate -o tmp_path_retention_policy=failed <file>. A stubghthat exits 97 is first on PATH, withGH_TOKEN=bogus, an emptyGH_CONFIG_DIR,GIT_SSH_COMMAND=false,GIT_CONFIG_GLOBALa scratch file that sets onlygc.autoandmaintenance.auto, andEMAILset. Every run exited 0.The guards the brief names come first, then the update-shape files, then the other files that name
update-shape.pyand read whatapplyraises or prints.tests/test_shape_doctor.pyis not run:shape-doctor.pyis not changed.tests/test_update_shape_unreadable.pytests/test_repo_hygiene.pytests/test_path_guard_hygiene.pytests/test_no_auto_maintenance.pytests/test_agents_md_codes.pytests/test_cli_reference.pytests/test_update_add_codes_documented.pytests/test_update_shape_partial_rollback.pytests/test_update_shape_messages.pytests/test_update_shape_apply_lines_and_headers.pytests/test_update_shape.pytests/test_update_shape_landing.pytests/test_update_shape_rewrite_symlink.pytests/test_update_shape_add_symlink.pytests/test_update_shape_add_not_a_directory.pytests/test_update_shape_add_ignored.pytests/test_update_shape_add.pytests/test_update_shape_branch_any_directory.pytests/test_update_shape_hardlink.pytests/test_update_shape_header_comments.pytests/test_update_shape_kept_helper.pytests/test_update_shape_merged_away.pytests/test_update_shape_next_quoting.pytests/test_update_shape_paste_lines.pytests/test_update_shape_pr_base.pytests/test_doctor_unreadable_paths.pytests/test_doctor_unreadable_pin.pytests/test_bom_reads.pytests/test_bom_rewrites.pytests/test_stdout_encoding.pytests/test_windows_paths.pytests/test_validator_child_encoding.pytests/test_pin_rewrite_layout.pytests/test_openreposhape_package.pytests/test_commit_trailers.pytests/test_refusal_default_remediation.pytests/test_chain_remedy_callers.pytests/test_pinned_copy_text.pyMutants
Each mutant breaks one behaviour of
update-shape.pyonce and runstests/test_update_shape_unreadable.py -k "oserror or nobody_reads"(the 6 new rows). The file is then restored from a saved copy and checked withcmp(exit 0 each time). Run twice, by the writer before the commit and again at the head, with the same verdicts.if refusal is None: raise, no rollbacktest_any_oserror_after_the_writes_rolls_them_back_and_is_raised[a-file-gone],[a-broken-pipe],[EIO-naming-no-path],test_a_stdout_nobody_reads_is_rolled_back_under_its_broken_pipeprint(undone, file=sys.stderr)removedsys.stdout.flush()test_a_stdout_nobody_reads_is_rolled_back_under_its_broken_pipe...is_raisedrows,test_a_stdout_nobody_reads_...,test_an_oserror_before_the_first_write_is_raised_and_says_nothing,test_an_oserror_after_the_commit_puts_nothing_backif undone is None: returnremoved, so nothing-to-put-back still printstest_an_oserror_before_the_first_write_is_raised_and_says_nothing,test_an_oserror_after_the_commit_puts_nothing_backprint(undone)in place ofprint(undone, file=sys.stderr)...is_raisedrows andtest_a_stdout_nobody_reads_...Merges
git merge-tree --write-treeof this head (cc8a87ee) withorigin/main(961640c6, frozen until the v1.2.0 tag) and with each open PR that touchesupdate-shape.pyortests/test_update_shape_unreadable.py, at the heads GitHub lists now. The open PRs were re-listed live after the fix commit (48 open); no PR other than these touches either file.All are clean. #1166 edits the same test file, in other rows, and merges without conflict. #1176 is #1188: it touches only
tests/test_bom_tool_io.py, no file this change touches.Fix round
RV1187's review (ruling 8, READY, no MUST) of
7a9d397basked for two one-line test changes and a body fix. One new commit,cc8a87ee, touchestests/test_update_shape_unreadable.pyalone;update-shape.pyis byte for byte as it was.PYTHONUNBUFFERED. With it exported as 1, stdout is unbuffered, so thecopiedprint breaks the pipe first andRollback.writinganswersREFUSED update-write-failed: ... Broken pipeinstead of the rollback list. The child's env is now{**os.environ, "LANES_LANE": "", "PYTHONUNBUFFERED": ""}, astests/test_bootstrap_refusal_exit.pyblanks it. At7a9d397bthe row FAILED withPYTHONUNBUFFERED=1exported (stderr beganREFUSED update-write-failed); at this head it passes with it exported as 1 and with it unset (2 passed each, with the refused-read row).test_any_read_refused_after_the_writes_rolls_them_backnow also asserts(out + err).count("ROLLED BACK") == 1. Mutant M8 (the_roll_back_before_raising(ledger)call moved aboveif refusal is None) SURVIVED all 54 rows at7a9d397b(54 passed) and is KILLED now, by that row alone, withupdate-shape.pyrestored and checked bycmp(exit 0).utf-8-sigread that drops the mark and a cut that also takes spaces both survive #1176 lines are corrected above and in Merges: update-shape.py and bump-leg.py: the byte-order-mark rows (#1050) hold what each rewriter returns but not the tools' own read and write, nor that only U+FEFF is cut, so autf-8-sigread that drops the mark and a cut that also takes spaces both survive #1176 is tests/test_bom_tool_io.py: the byte-order-mark rows hold each tool's own read and write and that only U+FEFF is cut, so autf-8-sigread that drops the mark and a cut that also takes spaces no longer survive (#1176) #1188 (aa0275be), which touches onlytests/test_bom_tool_io.py, and merge-tree with it is clean.update-shape.py:5276, left as it is. RV1187 marked it optional and "fine as is" (when stderr itself is the dead pipe, the print of the rollback list raises its ownBrokenPipeErrorover the original; the tree is already back and the exit is 120, as at main)._rolled_back's flush above,landing_failed's own stdout flush, which after the commit loses the refusal and the NEXT line the same way, andRollback.writing, whosecopiedprint underPYTHONUNBUFFERED=1reports a broken stdout pipe asREFUSED update-write-failed.Re-run at
cc8a87ee, one pytest at a time, each under the stub-ghhost rules:tests/test_update_shape_unreadable.pywithtests/test_update_shape.pyandtests/test_update_shape_add_ignored.py, in one pytesttests/test_path_guard_hygiene.pytests/test_repo_hygiene.pytests/test_no_auto_maintenance.pyMutants M1 to M6 and M8, each run against the whole of
tests/test_update_shape_unreadable.py(54 rows), the file restored from a saved copy and checked withcmp(exit 0 each):if refusal is None: raise, no rollback...is_raisedrows andtest_a_stdout_nobody_reads_...print(undone, file=sys.stderr)removedsys.stdout.flush()test_a_stdout_nobody_reads_......is_raisedrows,test_a_stdout_nobody_reads_...,test_an_oserror_before_the_first_write_...,test_an_oserror_after_the_commit_...if undone is None: returnremovedtest_an_oserror_before_the_first_write_...,test_an_oserror_after_the_commit_...print(undone)on stdout...is_raisedrows andtest_a_stdout_nobody_reads_..._roll_back_before_raising(ledger)call aboveif refusal is None(SURVIVED at7a9d397b)test_any_read_refused_after_the_writes_rolls_them_backCloses #1173
Lane: openreposhape-3 (openRepoShape-3)
🤖 Generated with Claude Code