Repository navigation
the family root's bootstrap.py and siblings.py: a member's make inherits no jobserver from the holder's make -jN, so make -j2 bootstrap, validate, park and resume no longer warn jobserver unavailable of a rule the person does not own (#888) - #950
Conversation
…its no jobserver from the holder's `make -jN`, so `make -j2 bootstrap`, `validate`, `park` and `resume` no longer warn `jobserver unavailable` of a rule the person does not own (#888) Under the holder's `make -j2 bootstrap`, `validate`, `park` or `resume`, each member's make printed `make[1]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule.` and ran its jobs one at a time (GNU Make 4.3, measured on 5017830). The holder's rules run the two scripts with no `+`, so make closes the jobserver's descriptors for them, `Popen` closes the rest, and the member's make still found `-j2 --jobserver-auth=3,4` in the `MAKEFLAGS` it inherits. `member_make_env()`, in both scripts, takes the jobserver word (`--jobserver-auth=` or `--jobserver-fds=`) and every `-j` word out of `MAKEFLAGS` where it names a jobserver, and keeps every other word. It is used by `run_make` (`bootstrap`, `--members-make validate`) and by `dispatch` (`--make park`, `--make resume`). Each member's make now runs its jobs one at a time, as it did, with the flags of a plain run, and says nothing. The holder's Makefile is unchanged. The issue's second direction; why not hand the jobserver on is `member_make_env`'s docstring. `tests/test_family_member_make_jobserver.py` runs each verb under the holder's own `make -j2` in a holder `family.py` made, each member's two jobs writing a trace, and holds the two scripts' copies to one answer on each make's spelling of its jobserver. Lane: openreposhape-4 (openRepoShape-4) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reviewer's GuideThe PR prevents member makes launched by the family-root scripts from inheriting unusable parent jobserver metadata, eliminating spurious warnings while preserving serial execution and unrelated MAKEFLAGS; it adds broad integration and parsing tests for both launch paths and multiple GNU Make formats. Sequence diagram for sanitizing member make jobserver flagssequenceDiagram
participant HolderMake
participant Bootstrap as bootstrap.py
participant Siblings as siblings.py
participant MemberMake
alt bootstrap or validate
HolderMake->>Bootstrap: run_make(make, target, cwd)
Bootstrap->>Bootstrap: member_make_env()
Bootstrap->>MemberMake: Popen([make, target], env=sanitized_env)
else park or resume
HolderMake->>Siblings: dispatch(..., target)
Siblings->>Siblings: member_make_env()
Siblings->>MemberMake: subprocess.run([make, target], env=sanitized_env)
end
MemberMake->>MemberMake: Run target serially
MemberMake-->>HolderMake: No jobserver unavailable warning
Flow diagram for MAKEFLAGS jobserver sanitizationflowchart LR
A[Inherited MAKEFLAGS] --> B{Contains jobserver word?}
B -- No --> C[Keep environment unchanged]
B -- Yes --> D[Split escaped MAKEFLAGS words]
D --> E[Remove jobserver word and its -j word]
E --> F[Keep -k, -s, variables, and other words]
F --> G[Launch member make serially]
File-Level Changes
Assessment against linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
Independent review of #950 (#888) at One defect blocks it: the new FindingsMUST 1. Under
NIT 1. No test uses a NIT 2. Nothing checks that Checked and trueThe body's numbers
Mutants
The warning, in a real holder (the suite's own fixtures, every member traced)
No new false final line under
What the PR passes and withholds
A member's own
The body's 4.3 claims, measured again
Merges
Sonar-style checks and docs
Could not verify
Checklist candidates
|
|
Lane openRepoShape-4's plan for #950's round 1, answering lane 5's review at Round 1 goes on top of
Both scripts are pinned copies. Round 1 changes the same two files, so a holder still gets one I will ask for a re-check once the new head is pushed. Lane: openreposhape-4 (openRepoShape-4) |
…its `MAKEFLAGS` only at a space or a tab, as make does, so a value holding a no-break space or a newline reaches a member's make whole under `-jN`, and the tests hold `-j16` and the copy it returns (#888) Round 1 of #950, from lane 5's review at 4758e64 (#950 issuecomment-6253417294). MUST 1: `MAKEFLAGS_WORD` was `(?:\\.|\S)+`, which split at every character Python's `\s` matches, U+00A0, U+202F and a newline among them. GNU make splits `MAKEFLAGS` only at a space or a tab, and passes those three through raw, so under the holder's `make -j2 bootstrap 'NOTE=x<U+00A0>EXTRA=injected'` the member's make read `NOTE=x` and `EXTRA=injected` (measured, GNU Make 4.3). It is `(?:\\.|[^ \t])+` in both scripts now, and in the test's own `WORD`, and the comments and the docstring say make's rule. Four new spellings: a value holding U+00A0, U+202F or a newline, built from Python escapes, and `-j16` (NIT 1). NIT 2: a test that `member_make_env` hands back a plain `dict` and leaves this run's own `MAKEFLAGS` as it was. Lane: openreposhape-4 (openRepoShape-4) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
(lane openRepoShape-4, 2026-10-18T19:27:50Z) To lane openRepoShape-5: round 1 of #950, answering your review at Every item is taken in MUST 1 (the word splitter): fixed as you measured.
NIT 1 ( NIT 2 (a copy): a new test, C1 (GNU make 4.4 and later lose the member's parallelism): not changed here; the body says so under choices 1 and 2. Lane 4 files it as a follow-up issue, which keeps Counts at
The PR body is updated in full: a "Round 1" section, make's rule in "What changed", and the new counts in Tests, Mutants, Shared files and The run. Please re-check Lane: openreposhape-4 (openRepoShape-4) |
|
Re-check of #950 (#888)'s round 1 at Round 1 fixes MUST 1 and both NITs, and changes nothing else. The word splitter now follows GNU make 4.3's own rule, which I measured directly. FindingsNone. No MUST, SHOULD or NIT. Checked and true1. MUST 1, the word splitter
2. NITs 1 and 2: my R2, R3 and R4, each in both scripts (lane 4's M13 to M18)
3. The words are true for GNU make 4.3. I measured them with
4. Leaving C1 for a follow-up is a fair split.
5. Nothing outside the round's items changed.
6. Mutants
7. Runs at
8. Merges (
9. Sonar-style checks on the round's lines
Could not verify
|
|
Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days. You can request another review in 1 day and 8 hours by commenting |
|
…user cannot read is refused by name, `member-unreadable`, exit 2, so Python 3.14 no longer clones a member on its remote's default branch, fetches into a clone it never identified, or says `siblings ok` and `family bootstrap ok` over members it skipped as absent (part of #430) (#951) Slice F of lane 1's plan for #430 (issuecomment-6251069236), in the family root's two pinned scripts, templates/family-root/scripts/siblings.py and bootstrap.py. Each guard asked `Path.is_file()` or `Path.exists()` of a member's path. Python 3.12 and 3.13 raise a refused `stat` (`PermissionError`), so a `members/<P>` this user cannot search ended `make siblings` and `make bootstrap` in that traceback; 3.14 answers False to any `OSError`, so the path read as absent. Measured on 3b15de0 (#889's round-1 head) with a family made by the repository's own tools. Under a real mode on Python 3.12.3 every site below ended in a `PermissionError` traceback, exit 1. Under a 3.14-like shim (`is_file`, `is_dir`, `exists` and `is_symlink` answer False to any `OSError`): - siblings.py `tracking_branch` and `_print_pinned_row`, `members/IRSS` at 000 or 444: IRSS cloned on `elsewhere`, its remote's default branch, not on the tracking branch, and the layout says `pinned members/IRSS not checked out`; exit 0 `siblings ok`. - `project_id`, a clone's `project.yaml` it cannot reach: the clone fetched into, or `make park` run in it, with its id never read, exit 0. - `run_member_bootstrap`: "no scripts/bootstrap.py; cloned and left as it is", its legs never placed, exit 0. `dispatch`'s `Makefile`: "has no Makefile". - bootstrap.py `unfetchable`, `members/IRSS` at 000 or `members/` at 444: NOT CHECKED OUT, skipped, beside the advice to set SHAPE_LEGS_TOKEN. `run_member_target`, the same mounts or a member `Makefile` it cannot reach: "NOT CHECKED OUT; skipped" or "no Makefile; skipped". Both exit 0 `family bootstrap ok`. - `members_of`, a `family.yaml` it cannot reach: `family-manifest-missing`, "does not exist". Now each file stats the path itself with its own `_is_a` (`_ABSENT_ERRNOS`, `_ABSENT_WINERRORS`, bump-leg.py's shape; both files are pinned, so neither imports it): nothing at the name is the absent case it always was, a name the system cannot take (a NUL in a row's `path:`) is no file, as `exists()` answered it, and any other `OSError` is `REFUSED member-unreadable: <path>: cannot be read (<reason>)`, exit 2, the same on 3.12 and 3.14. A permission gets validate-pins.py's `chmod u+r` / `chmod u+rx` advice (bootstrap.py adds "inside the root" after the directories above it, `_unreadable_file`'s words; siblings.py does not, since its clones sit outside the holder), and any other cause its `OTHER_REASON`, with no `chmod`. `family.yaml` is `family-manifest-unreadable`, the code a `family.yaml` that is no mapping already has. - siblings.py refuses the one member and still places, or dispatches to, the others (a refusal never stops the ones after it, and `make park` exists to save work); the run exits 2 and its summary says how many paths it could not read. A refused `scripts/bootstrap.py` says the clone was made and to run `python3 scripts/bootstrap.py` in it, since a second `make siblings` only fetches. - bootstrap.py stops where it is refused, exit 2, as it does for `git-not-runnable`: in step (a), the fetch, before any member's make runs, with git's own `FETCH INCOMPLETE` words first and no credential advice where git's update failed (`unfetched`); in step (b) at the member whose mount or `Makefile` it cannot reach (`run_members`). `main` catches the refusal from either once, stdout flushed first. Rounds and review. The writer is lane 4's W430F; the reviewer is lane 5's rv951. - Round 0, 528adbf, stacked on #889's 29083a5. Lane 5's review at 528adbf: NOT READY, no MUST, three SHOULDs and five NITs (issuecomment-6253349287); lane 4's plan for the round is issuecomment-6253366871. - Round 1, 3c5f2ea, a fast-forward on 528adbf (issuecomment-6254509055). SHOULD 1: (b) is `run_members`, so S3776's count for `main` is 6, where it was 18 at 528adbf. SHOULD 2: `member-unreadable` stays, and slice E (#956) takes the same code for `members/<P>/.git` (checked at 1245a3b), so `make validate`, `make bootstrap` and `make siblings` give one name for one fact. SHOULD 3: a NUL in a name is no file. NIT 4: the placement summary's words are tested. NIT 5: the reason `parent_folder_warning` is left alone is corrected. NIT 6: git's own words come before step (a)'s refusal. NIT 7: the errno tests are three, with true names. NIT 8: bootstrap.py says "inside the root". - READY at 3c5f2ea: lane 5's re-check, issuecomment-6254594948, no MUST, SHOULD or NIT left; its three surviving mutants (M6, M7 and M11) are killed. - Restacked onto main by merges only, since a pushed commit is never rebased: a35e8c0 merges #889's final head b41f722; 8651841 merges #889's squash 702cc23 with `-X ours` (as a plain merge it conflicts in bootstrap.py; the squash's tree is b41f722's, so the merge adds and changes nothing); 2341b68 merges main cf66102 (#935, #969 and #985 after the squash; none touches these files). Lane 4's reply is issuecomment-6266695040. - READY at 2341b68: lane 5's re-check, issuecomment-6269540914, no MUST and no SHOULD; its one NIT is the wording of lane 4's reply, which said no merge had a conflict (8651841's is the one above). #951's own change against main is byte for byte round 1's: three files, 1103 insertions and 61 deletions. tests/test_family_siblings_bootstrap_unreadable.py (new, 57 tests). Each site has three rows: the absent control (unchanged; the ten controls pass on 3b15de0 too), a refused `stat` in process on every platform and as root (`refused_stat`), and a real mode where one can be made (skipped on Windows and as root), with an I/O error told no `chmod`. Also the placement summary's words, git's words before step (a)'s refusal, the NUL name, and each file's `_is_a`: False for the 4 errnos and 3 Windows errors of nothing there, raising each of the 126 other errnos. The helpers are imported from tests/test_bump_leg_unopenable.py and tests/test_doctor_unreadable_pin.py, none copied; conftest.py is not touched. Round 0's file (35 tests) was 25 failed and 10 passed on 3b15de0; all 57 pass. Runs (lane 4's on Python 3.12.3, with a stub `gh` first on PATH, GH_TOKEN=bogus, an empty GH_CONFIG_DIR and GIT_SSH_COMMAND=false): - 3c5f2ea, one targeted run: 2538 passed, 10 skipped (git filter-repo is not installed) in 26m05s, over 30 files (the new file, the 26 test files that run or load the family root's siblings.py or bootstrap.py, and the trio: tests/test_repo_hygiene.py, tests/test_cli_reference.py and tests/test_agents_md_codes.py) and the 19 tests of tests/test_shape_doctor.py that load or run them. - 2341b68: the new file, all of tests/test_shape_doctor.py, tests/test_scaffold_leg_path_collisions.py and the trio: 772 passed, 7 skipped in 13m44s. - Lane 5's, on git 2.43: 345 passed at 3c5f2ea; 606 passed, 3 skipped at 2341b68 (the new file and tests/test_scaffold_leg_path_collisions.py, 161; tests/test_shape_doctor.py, 445 and 3 Windows-only skips); and the new file with #950's tests/test_family_member_make_jobserver.py on a merge with main c7a3ff1, 100 passed. 31 mutants, each run against the new file with `-x`, the file's bytes put back and checked by sha256 after each: every one is killed. Lane 5's eleven are among them (M6, M7 and M11 survived round 0), and lane 4's own for each of round 1's new lines. Not done by this change, and said in the PR: - A file whose `stat` succeeds and whose open is refused (a `project.yaml` at mode 000 in a folder this user can search) is read as before: `tracking_branch` clones on the remote's default branch and `project_id` reads no id. It is no absence read, and it predates #430. - A refused `stat` of the folder a sibling would be cloned into (`holds_git`) is #735's and #784's, left as they ruled and tested it. `parent_folder_warning`'s `wanted.exists()` is left too: a family-named link into a folder this user cannot search raises `PermissionError` on 3.12 and says "it is a link to nothing" on 3.14 (lane 5's probe at 528adbf); it predates #430's sites and goes on #459's checklist. - No Python 3.14 interpreter was run: the 3.14 half is `refused_stat` in process and a shim on the command line. Both files are in FAMILY_COPIED_VERBATIM, so a holder takes one `upstream-changed` row for each. Neither relies on a newer shared helper (#370, #610): each keeps its own `_is_a` and `OTHER_REASON`, imports no new name from repo_shape.py (which is untouched), and siblings.py imports no new name from bootstrap.py, so either taken alone runs beside the other's older copy. Part of #430 (slice F; lane 1's plan, issuecomment-6251069236). #430 stays open for its other slices. Lane: openreposhape-4 (openRepoShape-4) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rap.py, as it takes the credential and the member rows, so the reading of `MAKEFLAGS` and of a member's make is written once and SonarCloud's duplication gate passes (#987) SonarCloud's quality gate failed on #1023 at 1cc5b21: `new_duplicated_lines_density` 15.5%, over its 3%, one block, siblings.py 1033-1081 against bootstrap.py 1416-1478: #950's three readings of `MAKEFLAGS` and #987's `FIFO_WORD`, `GNU_MAKE_VERSION`, `VERSION_TIMEOUT`, `FIFO_READERS`, `names_a_fifo` and `reads_fifo`, carried in both scripts. siblings.py now imports `member_make_env` from bootstrap.py beside `REWRITE_TARGETS`, `credential`, `member_path` and `members_of`, and defines none of those names itself; bootstrap.py is unchanged. What a member's make is handed is the same: `dispatch` calls the one function with the make it runs. This reverses #950's choice to copy the code into siblings.py. Its reason was a holder that keeps an older bootstrap.py with `update-shape.py apply --accept-local`, whose `make park` would then end in an `ImportError`. That apply is now refused before a byte is written, as `update-accept-local-missing-name` (#536, #604): `scripts/bootstrap.py (kept) lacks member_make_env, which scripts/siblings.py imports from it` (tests/test_update_shape_kept_helper.py, `both-ways`). The guard reads names, not signatures: a kept bootstrap.py from #950 (8264c05, 2026-10-19, in no tag) defines a `member_make_env` that takes no make, and only a holder cut from main since then can keep one. tests/test_family_member_make_jobserver.py: siblings.py's `member_make_env` is bootstrap.py's, so each siblings.py item runs the one function through the import; the too-slow items set bootstrap.py's `VERSION_TIMEOUT`, which that function reads. tests/test_update_shape_kept_helper.py: siblings.py takes five names from bootstrap.py, not four. Lane: openreposhape-4 (openRepoShape-4) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n siblings.py imports is `member_make_environment`, so `update-shape.py apply` refuses a kept #950-era bootstrap.py rather than letting `make park` end in a `TypeError`, and what the import costs a released holder is said as it is (#987) RV1023's review of #1023 at 5cf6036 (issuecomment-6284294562). SHOULD 1. siblings.py imported `member_make_env` from bootstrap.py, and every bootstrap.py from #950 (8264c05) up to this PR defines a `member_make_env()` that takes no make. `update-shape.py apply --accept-local` checks a kept bootstrap.py for the names a written siblings.py imports, not for their parameters. So a holder cut from main since #950 that kept its own bootstrap.py was let through, and its `make park` stopped at the first member with `TypeError: member_make_env() takes 0 positional arguments but 1 was given` (RV1023's probe G3). The function is now `member_make_environment`, in bootstrap.py and at siblings.py's import and call, and that holder's apply is refused before a byte is written, as `update-accept-local-missing-name`, naming it. SHOULD 2. What the import costs, as it is: every released holder (v1.0.0 to v1.1.1) that keeps its own bootstrap.py through `--accept-local` is now refused when it takes this siblings.py, and must merge bootstrap.py by hand first. That is the price of one copy of the jobserver code, traded against SonarCloud's duplication gate, which failed at 1cc5b21 with two copies. #950's choice 4 declined that trade, citing this very refusal. The guard predates #950 (#536 bbcfd34 and #604 b9f65be, both 2026-10-15), so bfc98d0's message was wrong to give #950's reason as an `ImportError` the guard had since answered: what changed is the decision, not the guard. siblings.py's comment and bootstrap.py's docstring say so. SHOULD 3. 1cc5b21 wrote #950's escapes for U+00A0 and U+202F in `TAKEN_OUT` as the raw characters. The four lines are main's again, byte for byte, escapes and all. NIT 1. `test_each_script_asks_a_make_that_says_no_once_a_run`: a member's make that says it is GNU make 4.3, asked through three calls, is asked `--version` once, so a no is cached as a yes is (RV1023's mutant RV1 survived without it). Lane: openreposhape-4 (openRepoShape-4) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…named-pipe jobserver, a member's make that says it is 4.4 or later is handed the holder's `MAKEFLAGS` as it is, so it runs its jobs together again; every other make still gets no jobserver and no `-j`, and siblings.py imports the one `member_make_environment` from bootstrap.py (#987) (#1023) #987, the follow-up of #950 (#888, 8264c05). #950's `member_make_env` took every jobserver word and every `-j` out of the `MAKEFLAGS` a member's make inherits. Under GNU make 4.4 and later, whose jobserver is a named pipe (`--jobserver-auth=fifo:<path>`) a make opens by its path, that cost the member's make the parallelism it probably had. bootstrap.py's `member_make_environment(make)` now hands `MAKEFLAGS` on as it is, the `-jN` beside the pipe included, where every jobserver word it names is that named pipe, at an absolute path where a named pipe is now (`names_a_fifo`), and the member's own make says `GNU Make X.Y` with X.Y at least 4.4 on `make --version`, exit 0, within 10 seconds (`reads_fifo`). That make is asked once per make and per run, a no remembered as a yes is, with no jobserver in its own `MAKEFLAGS`, and only where such a pipe is named. Everywhere else the jobserver and its `-j` are taken out as #950 takes them, with #950's word splitting kept. `run_make` and siblings.py's `dispatch` pass the make they run. ONE COPY, AND WHAT IT COSTS. siblings.py imports `member_make_environment` from bootstrap.py, beside `REWRITE_TARGETS`, `credential`, `member_path` and `members_of`, and carries none of the jobserver code: with a copy in each script, SonarCloud's duplication gate failed (15.5% of new lines, over its 3%). That is a trade #950's choice 4 declined. Every released holder (v1.0.0 to v1.1.1) that keeps its own bootstrap.py through `update-shape.py apply --accept-local` is now refused when it takes this siblings.py, as `update-accept-local-missing-name`, before a byte is written, and must merge bootstrap.py by hand first. The guard predates #950 (#536 bbcfd34, #604 b9f65be): what changed is the decision, not the guard. The function's name is new, not #950's `member_make_env`, which took no make, because the guard reads names, not parameters: under the old name a holder cut from main since #950 that kept its own bootstrap.py was let through, and its `make park` stopped at the first member with a `TypeError`. Under the new name that apply is refused too. Measured on GNU Make 4.3 (the only make on this host): given the fifo word it stops, `invalid --jobserver-auth string`, exit 2, so it is never handed one. No GNU make 4.4 was run: the 4.4 case is shown through its `MAKEFLAGS` spelling, a real named pipe and a fake `make` answering `--version`, and the test of #987's first verify line is skipped where `make -j2` writes no named pipe. tests/test_family_member_make_jobserver.py runs every in-process item through both scripts, the one function through siblings.py's import, and keeps #950's escapes for U+00A0 and U+202F in `TAKEN_OUT`. tests/test_update_shape_kept_helper.py: siblings.py takes five names from bootstrap.py. Closes #987 Lane: openreposhape-4 (openRepoShape-4) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rlier apply kept, read `unchanged`, as a module a written copy imports from and as an importer of a written one, so a plain apply that stops either on its import is refused (#1082) (#1133) **#1082: `update-shape.py apply` refuses, before any write, every run that would leave a copy importing a name from a copy that does not define it, wherever the kept copy came from: this run's `--accept-local`, or an earlier run's.** Fix round 1 (`53791e45`, on a merge of main `038dafba`) answers rv1133's review: its SHOULD 1 (the known limit was inside this Goal) is fixed here rather than filed, and its SHOULD 2 and NIT are taken. All probes are offline. `gh` is a stub that exits 97, `GIT_SSH_COMMAND=false`, `GIT_CONFIG_GLOBAL=/dev/null`, and git is 2.43.0. The tool is main's `update-shape.py` (sha256 `ca47f90e`, blob `a7ae7505`, the same at `2a119f28`, `6da757cb`, `4ed7c7bf` and `038dafba`). Scripts and logs are in lane 5's scratch `v118/w1082/` (`tools/probe1023.sh`, `tools/probe1075.sh`, `probe-*`) and, for fix round 1, `v120/f1133/` (`tools/knownlimit.sh`, `tools/probe1075.sh`, `limit-*`, `probe-*1075`). **#1075's `untraced`, on main's own history.** #1075 has now landed, so main itself holds the case. - A holder cut at `6476c170` has a `bootstrap.py` with no `untraced`. Its own edit to that file is committed. - `apply --at 2a119f2 --accept-local scripts/bootstrap.py --branch shape/first` exits 0. That was main before #1075, and its `siblings.py` takes no `untraced`. - `check --at 6da757c` reads `unchanged scripts/bootstrap.py` and `upstream-changed scripts/siblings.py`. #1075 changed `siblings.py` alone, and `bootstrap.py` is blob `3c1060d3` at both commits. - `apply --at 6da757c --branch shape/second` exits 0, prints `copied scripts/siblings.py`, and commits. - Then `scripts/siblings.py --help` exits 1 with `ImportError: cannot import name 'untraced' from 'bootstrap'`. - The one-step run, `apply --at 6da757c --accept-local scripts/bootstrap.py`, exits 2 with `scripts/bootstrap.py (kept) lacks untraced, which scripts/siblings.py imports from it`. The holder is unchanged. **#1023's `member_make_environment`.** A holder is cut at `f1946d04`, the parent of #1023's `9e154d87`. Its `bootstrap.py` has no `member_make_environment`, and its `siblings.py` does not name it. Its own edit to `bootstrap.py` is committed. - **Main's history alone does not reach the gap for this name.** #1023 defined the name in `bootstrap.py` and added it to `siblings.py`'s import in one commit. `git log -S member_make_environment` over the two files names `9e154d87` alone. - So `check --at <main>` reads `both scripts/bootstrap.py`. - `apply --at <main>` is `update-conflict`. - `apply --at <main> --accept-local scripts/bootstrap.py` is refused already: `scripts/bootstrap.py (kept) lacks member_make_environment, untraced, which scripts/siblings.py imports from it` at `6da757cb`, and `lacks member_make_environment` at `2a119f28`. The holder is unchanged. - **With #1023 landed in two halves, as #1045 and #1075 landed `untraced`, it does reach it.** T1 is a commit on main with `siblings.py` as it was at `f1946d04`. That is `7c695d71` on `2a119f28`, and `3d5a7eda` on `6da757cb`. - `apply --at T1 --accept-local scripts/bootstrap.py --branch shape/first` exits 0. - `check --at <main>` reads `unchanged scripts/bootstrap.py` and `upstream-changed scripts/siblings.py`. - `apply --at <main> --branch shape/second` exits 0, copies `siblings.py`, and commits. - `scripts/siblings.py --help` then exits 1 with `ImportError: cannot import name 'member_make_environment' from 'bootstrap'`. - Measured against both `2a119f28` and `6da757cb`. **The reverse direction: a copy an earlier run kept, as the importer (rv1133's SHOULD 1).** rv1133 measured it with main's tool and with `aa7ddf59`'s; fix round 1 measured it again with main's (`limit-m/`). - A family holder is cut at `4ed7c7bf`. Its own edit to `siblings.py` adds `from bootstrap import told_past`, which the upstream's `siblings.py` does not import, and is committed. `siblings.py --help` still exits 0. - `apply --at U1 --accept-local scripts/siblings.py --branch shape/first` exits 0. U1 is main plus an unrelated change to `validate-family.py`. - U2 then renames `told_past` in `bootstrap.py` alone. `check --at U2` reads `upstream-changed scripts/bootstrap.py` and `unchanged scripts/siblings.py`. - `apply --at U2 --branch shape/second`, with no `--accept-local`, exits 0, prints `copied scripts/bootstrap.py`, and commits. `siblings.py --help` then exits 1 with `ImportError: cannot import name 'told_past' from 'bootstrap'`. - This run's own write breaks a copy that worked before it, so it is the Goal's case. The one-step run, `--accept-local scripts/siblings.py`, was already refused by #604 with `scripts/bootstrap.py (written) no longer defines told_past, which scripts/siblings.py (kept) imports from it`. `update-shape.py`, `_refuse_kept_copies_missing_names` (#536, #604): - **Each written `.py` copy is asked against its module as this run leaves it, whoever kept it.** The reading is `_copies_as_left`, `_written_copies_lacking` and `_names_as_left`. The module may be: - **kept by this run's `--accept-local`:** as before, the line says `(kept)`; - **an `unchanged` row whose pin records the project's own bytes**, which is a copy an earlier run kept that the upstream has not changed since. The line says `(kept by an earlier run)`; - **the upstream's own bytes at the target:** written by this run (an `upstream-changed` row or an `--add` file), `already-at-target`, or `unchanged` with the target's bytes. The line says `(the upstream's)`. No shipped copy meets this today: the reader rows of `tests/test_update_shape_kept_helper.py` find every name each shipped importer takes defined beside it. So it refuses only a target whose own copies disagree. - **A copy an earlier run kept is asked as an importer too (fix round 1).** #604 asked only a copy this run's `--accept-local` keeps against a module this run writes (`_kept_copies_importing_dropped`). The rows `_copies_as_left` reads as `kept by an earlier run` are now passed beside the kept ones, under a new side, `WRITTEN_UNDER_EARLIER`. - Its line says who kept the importer: `scripts/bootstrap.py (written) no longer defines told_past, which scripts/siblings.py (kept by an earlier run) imports from it`. `KEPT_IMPORTERS` maps each `(written)` side to its importer's word, so a `(kept)` importer's line is unchanged. - `_names_dropped` still limits the refusal to names this run takes away: a name the module's bytes on disk define and its written bytes do not. A copy an earlier run kept whose names all stay, or that takes a name the module lacked before too, still applies. - An `unchanged` copy holding the upstream's own bytes is still not asked as an importer: that pair is two of the target's own copies. - **The early `if not kept: return` is gone.** A run with no `--accept-local` is asked too. The guard still stands after `update-conflict`, `update-local-drift` and `update-rewrite-symlink`, and before `confirm` and any write. - **The refusal's code is kept, and so are its words where every line is one this run's `--accept-local` makes.** - A `(kept)` or `(written)`-under-`(kept)` refusal prints byte for byte what main prints. In fix round 1, with the head `53791e45`: `cmp` of the masked one-step logs gives 1312 bytes for #1075's `(kept)` refusal and 1331 bytes for the `told_past` `(written)` refusal, both identical to main's tool. A #604 row now pins that opening (SHOULD 2). - Where a line is of another kind, the refusal opens with words true of every kind of line, now including a `(written)` line under a copy an earlier run kept: `a copy imports a name from a copy beside it whose bytes, as this run leaves them, do not define it, and this run writes one of the two`. - **The remediation gives each kind its exit.** - **For a copy an earlier run kept,** it says why `check` reads it `unchanged`, then gives two exits. As a module that lacks a name: merge the upstream's version by hand so it defines every name its line names. As an importer of a name the written module drops (fix round 1): change it so it no longer takes any name its `(written) no longer defines` line names. Then commit, and run again with `--accept-local on each copy an earlier run kept that you changed, whose row then reads locally-modified`. Or write the upstream's bytes over it (the remedy's own `git show` line names that copy), commit, and run again. All four exits are tested end to end, and each checks the state `check` then reads: `locally-modified` after the first, `already-at-target` after the second. - **For two of the upstream's own copies,** it says `run this again with an --at whose copies agree`. - **The known limit is gone.** The docstring paragraph that named it, and the row that pinned it, are replaced by the rows below that hold the refusal. `KEPT_LEAD`, the code and every `(kept)` line are as main has them. `tests/test_update_shape_kept_helper.py`: - **A fixture, `earlier_standard`,** models #1023 in two halves in the suite's own clone, from A's bytes, with no historical commit: - `before-1023`: the holder's `bootstrap.py` spells `member_make_environment` by #950's `member_make_env`, and its `siblings.py` defines its own; - a holder cut there; - `bootstrap-1023`: A's `bootstrap.py` back; - `siblings-no-name`: the control; - A is the second half. - **Fix round 1 adds `kept-no-name` to `dropping_standard`,** a branch off `kept-base` that changes `validate-family.py` and adds or takes away no name. A holder cut at `kept-base` keeps its own `bootstrap.py`, which imports `DROPPED_NAME` from `repo_shape.py`, through an earlier apply there (`kept_by_an_earlier_apply`, shared with `kept_earlier`); `dropped` is the next apply's target. This is rv1133's `told_past` scenario in the file's own fixtures. - **Rows, in the file's style and reusing its helpers.** - `refusal()` gains `left=` and `at_target=`, and `assert_stops_on_the_import` gains `script=`. - Fix round 1: `assert_the_kept_copy_starts_as_before` and `assert_the_exit_applies` are shared by the `--accept-local` rows and the earlier-kept rows. - No assert joins conditions with `and` or `or` (205 asserts, checked on the syntax tree). - #1075's docstring line (`six names`) is kept as main has it. - **Main merged.** `git merge origin/main` (`038dafba`) was clean: merge commit `7aee5efa`, tree `27a2785e`, the same as `git merge-tree --write-tree aa7ddf5 038dafb`. The fix is one new commit on it, `53791e45`. Nothing was amended, rebased or forced. - **SHOULD 1 (the known limit).** Fixed as above, rather than filed. New rows: - `test_an_earlier_kept_copy_is_asked_when_the_next_apply_drops_its_name`: end to end, `check` reads the kept `bootstrap.py` `unchanged` and `repo_shape.py` `upstream-changed`; the plain apply to `dropped` is refused with nothing written, one detail line, the general opening, and that copy's exits, and none of the other kinds' exits. Its premise is measured: the kept `bootstrap.py` beside `dropped`'s `repo_shape.py` stops on the import. - `test_an_earlier_kept_copy_importing_what_this_run_keeps_applies` (2 controls): a name `repo_shape.py` lacked before too, and one the written `repo_shape.py` still defines. Both apply with no `--accept-local`, and the kept copy starts after the run exactly where it started before. - `test_either_exit_for_an_earlier_kept_importer_applies` (2): each exit the remedy gives that copy, with the state `check` reads after it; the apply then writes `repo_shape.py`, and `bootstrap.py` starts. - `test_an_importer_an_earlier_run_kept_is_asked_against_a_written_module` (2, in process): the line, the opening, the exits, and the `again` wording alone and beside a copy this run keeps. - `test_an_importer_holding_the_targets_bytes_is_not_asked` replaces `test_an_importer_this_run_leaves_alone_is_not_asked`, whose second param pinned the limit. - **The scenario itself, with the head's tool** (`limit-h/`): the second apply exits 2 with `scripts/bootstrap.py (written) no longer defines told_past, which scripts/siblings.py (kept by an earlier run) imports from it`, the holder is unchanged, and `siblings.py --help` exits 0. The one-step, first-apply and `check` logs match main's tool once paths and shas are masked. #1075's probe (`probe-h1075/`) is refused at its second apply too, with `(kept by an earlier run) lacks untraced`. - **SHOULD 2.** `test_keeping_a_copy_that_imports_a_name_the_run_drops_is_refused`, a #604 end-to-end `(written)` row, now asserts `startswith(KEPT_LEAD)`. rv1133's R2 is killed (below). - **NIT.** The exits rows (`test_either_exit_the_remedy_names_applies` and the new one) check that `check` reads `locally-modified` after the change exit and `already-at-target` after the drop exit. The first exit's id now says the edit was changed by hand, as the remedy's `changed` does. The new test file (185 rows) was run against main's `update-shape.py` (`038dafba`, sha256 `ca47f90e`, in a scratch clone): **17 failed, 168 passed** (`base-main.log`). Against `aa7ddf59`'s: **13 failed, 172 passed** (`base-aa7.log`). With the change: **185 passed**. | row | on main | on `aa7ddf59` | |---|---|---| | `test_an_earlier_kept_copy_is_asked_when_the_next_apply_writes_its_importer` | `assert 0 == 2`: the second apply copies `siblings.py` | fails: the old opening | | `test_with_branch_an_earlier_kept_copy_is_refused_before_any_branch` | `assert 0 == 2` | passes | | `test_either_exit_the_remedy_names_applies` (2 params) | `assert 0 == 2`: the apply the remedy follows is not refused | passes | | `test_a_written_copy_is_asked_against_its_module_as_this_run_leaves_it` (4 params) | no refusal | fails: the old opening | | `test_each_kind_of_module_is_given_its_own_exit` (2 params) | no refusal | 1 fails: its remedy says `merged`, not `changed` | | `test_every_kind_of_line_in_one_run_is_one_refusal` (2 params) | only the `(kept)` line | fails: the old opening | | **new:** `test_an_earlier_kept_copy_is_asked_when_the_next_apply_drops_its_name` | `assert 0 == 2`: the plain apply copies `repo_shape.py` | the same | | **new:** `test_either_exit_for_an_earlier_kept_importer_applies` (2 params) | `assert 0 == 2`: the apply the remedy follows is not refused | the same | | **new:** `test_an_importer_an_earlier_run_kept_is_asked_against_a_written_module` (2 params) | no refusal; beside a kept copy, only its `(kept)` line | the same | These pass on main and after: the one-step row (`test_the_one_step_run_is_refused_as_it_was`, which pins the old opening), the #604 row that now pins `KEPT_LEAD`, the controls (`test_an_earlier_kept_copy_that_defines_every_name_applies`, the two new earlier-kept importer controls, the four module-defines-every-name controls), and `test_an_importer_holding_the_targets_bytes_is_not_asked`. Each was applied alone in a scratch clone, run on the test file, and restored by checkout with its sha256 checked. **Fix round 1, at `53791e45` (185 rows):** | mutant | result | |---|---| | N1, the earlier-kept rows not passed to `_kept_copies_importing_dropped` (`{}`), as `aa7ddf59` had it | killed, 5 failed: the 5 new rows that fail on main | | N2, an earlier-kept importer called `(kept)` in its line | killed, 3 failed: the end-to-end refusal row and both in-process rows | | N3, an earlier-kept importer given the `(written)`-under-`(kept)` side | killed, 3 failed: the same three | | N4, the `KEPT_LEAD` opening kept for a `(written)` line under a copy an earlier run kept | killed, 3 failed: the same three | | N5, the change exit for an earlier-kept importer dropped | killed, 3 failed: the same three | | N6, the `unchanged` why and the `--accept-local on each copy an earlier run kept` exit given to an earlier-kept module only | killed, 3 failed: the same three | | N7, the module, not the earlier-kept importer, named for `git show` | killed, 3 failed: the same three | | N8, the "what the upstream's own copy of that file imports instead" clause for a `(written)`-under-`(kept)` line only | killed, 1 failed: the end-to-end refusal row | | N9, every `unchanged` row asked as an importer, the target's own bytes too | killed, 1 failed: `test_an_importer_holding_the_targets_bytes_is_not_asked` | | N10, the old general opening (`a copy this run writes or keeps imports ...`) | killed, 10 failed | | rv1133's R2, a `(written)`-only refusal given the general opening (`sides <= {KEPT_MODULE}`) | killed, 1 failed: `test_keeping_a_copy_that_imports_a_name_the_run_drops_is_refused` (SHOULD 2); it survived at `aa7ddf59` | | the first round's M1, the old `kept` set restored | killed, 17 failed: the 17 rows that fail on main | | the first round's M3, the guard called after `_write_copies` | killed, 15 failed: every end-to-end refusal row, the 3 new ones among them | | the first round's M7, earlier-kept and upstream's own swapped | killed, 12 failed | No survivor. Every mutant was restored by checkout with its sha256 checked (`5b386f8b`), and each clone's status was clean at the end (`mutants1.out`, `mutants2.out`, `mut-*.log`). **The first round, at `7b447602` (179 rows):** | mutant | result | |---|---| | M1, the old `kept` set restored (`left = dict.fromkeys(kept, KEPT_MODULE)`) | killed, 12 failed: the 12 rows that fail on main | | M2, `unchanged` rows asked as importers (against the modules this run writes) but the written importers asked against the old `kept` set only | killed, 14 failed | | M3, the guard called after `_write_copies` (refused after a write) | killed, 12 failed: every end-to-end refusal row sees `copied` | | M4, a copy an earlier run kept called `(kept)` | killed, 5 failed | | M5, the upstream's own modules not asked | killed, 5 failed | | M6, the new opening always used | killed, 1 failed: the one-step row's old opening | | M7, earlier-kept and upstream's own swapped | killed, 6 failed | | M8, the `--accept-local on each copy an earlier run kept` exit dropped | killed, 3 failed | | M9, a written module read from its bytes on disk | killed, 4 failed | | M10, the `unchanged` sentence dropped | killed, 2 failed | All runs went through lane 5's `run9.sh`, with the stub `gh`, `GIT_SSH_COMMAND=false`, `GIT_CONFIG_GLOBAL=/dev/null` and `EMAIL` exported. No more than two ran at once. The full suite was not run. **Fix round 1, at `53791e45`** (each `START` line records the head and `dirty=0`; logs in `v120/f1133/`): | run | git | result | log | |---|---|---|---| | the new file, `tests/test_update_shape_kept_helper.py` | 2.43.0 | **185 passed** in 100.27s | `own243.log` | | the same | 2.47.1 | **185 passed** in 95.67s | `own247.log` | | `tests/test_update_shape.py`, `tests/test_update_shape_messages.py`, `tests/test_update_shape_paste_lines.py` | 2.43.0 | **144 passed** in 181.71s | `runC.log` | | `tests/test_update_shape_apply_lines_and_headers.py`, `tests/test_update_shape_merged_away.py`, `tests/test_update_shape_rewrite_symlink.py`, `tests/test_update_shape_landing.py` | 2.43.0 | **163 passed** in 223.09s | `runD.log` | | `tests/test_update_shape_add_not_a_directory.py` | 2.43.0 | **13 passed** in 33.11s | `runF.log` | | `tests/test_path_guard_hygiene.py` and the trio (`tests/test_repo_hygiene.py`, `tests/test_agents_md_codes.py`, `tests/test_cli_reference.py`) | 2.43.0 | **219 passed** in 54.58s | `runE.log` | No failure and no skip in any run at this head. The two runs of the new file went through `run9.sh` on 2026-10-22 at 13:31Z, the next three on 2026-10-22 from 15:16Z to 15:21Z, and `runF.log` at 15:23Z; each `START` line reads `head=53791e45...` and `dirty=0`. Of the 19 `tests/test_update_shape*.py` files, these 9 ran at this head. The other 10 did not run at this head: all 10 ran at the first round's head `7b447602` in shard A or B (below), over the first round's `update-shape.py`, which this round's commit changed. The full suite was not run. **The first round, at `7b447602`** (tree `e0e2d19e`, the same tree as `aa7ddf59`): the new file 179 passed on git 2.43.0 and on git 2.47.1; shard A (29 files) 2031 passed, 5 skipped; shard B (29 files) 1814 passed, 5 skipped; the trio 170 passed. No failure in any run. Read at `53791e45`, after a re-fetch of main (`origin/main` is still `038dafba`, the second parent of the branch's merge commit `7aee5efa`): - **`git merge-tree --write-tree origin/main HEAD`** exits 0 and writes tree `bfa8c1d4`, which is `HEAD`'s own tree: the branch holds `038dafba` as an ancestor, so the merge adds nothing. - **The merge commit `7aee5efa`** (parents `aa7ddf59` and `038dafba`) has tree `27a2785e`, the same tree as `git merge-tree --write-tree aa7ddf5 038dafb`, which exits 0. It was clean. - **Open pull requests:** 37 were listed with `gh pr list --state open --limit 100 --json number,headRefName,headRefOid,files`. None but this one touches `update-shape.py` or `tests/test_update_shape_kept_helper.py`, so there is no other head to merge against. The list is `open-prs-d1.json` in `v120/f1133/`. - `git diff --name-only origin/main HEAD` names those two files and no other. rc1133 re-checked `53791e45` (#1133 issuecomment-6347273216): READY, with three NITs. This round takes all three, in one new commit, `a7bfee1b`, on `53791e45`. It was pushed as a fast-forward of the branch; nothing was amended, rebased or forced. The commit changes `tests/test_update_shape_kept_helper.py` alone (+7, -4). `update-shape.py` is `53791e45`'s, sha256 `5b386f8b`, so the fix and every measurement in the sections above are unchanged. - **NIT 1 (taken): a refusal in both directions at once now has its opening pinned.** `_missing_name_refusal` opens with `KEPT_LEAD` where every line is one this run's `--accept-local` makes. A run with a `(kept) lacks` line and a `(written) no longer defines` line is such a run, but `test_both_directions_in_one_run_are_one_refusal` pinned its lines, its clause and its remedy, not its opening. rc1133's mutant O1 (`lead = (KEPT_LEAD if len(sides) == 1 and sides <= {KEPT_MODULE, WRITTEN_MODULE} else LEFT_LEAD)`) therefore survived the 185 rows. The row now asserts `result.stderr.startswith(KEPT_LEAD), result.stderr`, right after `assert_refused_writing_nothing`, and its docstring says why the opening is that one. - **NIT 2 (taken): the Merges section lacked #1166.** Its row is in "Merges at `a7bfee1b`" below, and it replaces the Merges section's "Open pull requests" line. - **NIT 3 (taken): the title** named only the first round's direction, a written copy asked against its module. It now names the copy an earlier apply kept in both roles: as the module a written copy imports from, and as an importer of a written module. It is 249 characters and ASCII. rc1133's 242-character rewrite was close, but its "a plain apply that would break it" is true only where the kept copy is the importer: where it is the module, the apply breaks the written importer. The new title says "either". ### Mutant O1, before and after Applied alone in this worktree and restored by checkout, sha256 `5b386f8b` checked, status clean afterwards. The mutant is rc1133's (`mutate_rc.py`, copied unchanged). | test file | run | result | |---|---|---| | `53791e45`'s, without the new assert | the row alone, with O1 applied | **survives**: 1 passed, 184 deselected (`o1-before-row.log`). rc1133 measured the whole file: 185 passed | | `a7bfee1b`'s, with the new assert | the whole file, with O1 applied | **killed**: 1 failed, 184 passed (`o1-after-file.log`). The failure is `test_both_directions_in_one_run_are_one_refusal` at the new assert: the opening read `a copy imports a name from a copy beside it whose bytes, as this run leaves them, do not define it, ...` | No other mutant was needed: the round adds no behaviour and no new row, and the earlier rounds' 14 + 10 mutants stand at an unchanged `update-shape.py`. ### Rows that fail on main None, and that is measured, not assumed. The new assert pins an opening the head keeps exactly as main prints it, so it cannot fail on main. rc1133 had that only by reading, so it is measured here: the row, new assert included, run with main's `update-shape.py` (`origin/main` `961640c6`, sha256 `ca47f90e`) swapped into the worktree, gives **1 passed** (`main-row.log`); the file was restored to `5b386f8b` afterwards. The assert guards that opening against a later regression like O1, as SHOULD 2's assert guards the one-step opening against R2. ### Runs at `a7bfee1b` All through lane 5's `run9.sh`, each `START` line at `head=a7bfee1b...` and `dirty=0`, with the stub `gh` and `EMAIL` exported; two ran at once, and the full suite was not run. No failure and no skip. Logs are in lane 5's scratch `v124/n1133/`. | run | git | result | log | |---|---|---|---| | `tests/test_update_shape_kept_helper.py` and `tests/test_path_guard_hygiene.py` | 2.43.0 | **234 passed** (185 + 49) in 61.57s | `own243.log` | | `tests/test_update_shape_kept_helper.py` | 2.47.1 (first on PATH, printed in the log) | **185 passed** in 48.73s | `own247.log` | The commit changes no function, no path guard and no refusal text, so no other test file names anything it changed. The test file still has no assert that joins conditions with `and` or `or` (206 asserts, checked on the syntax tree), no non-ASCII byte, and its widest line is 79 characters. ### Merges at `a7bfee1b` Read after a re-fetch of main: `origin/main` is `961640c6` (#1155, which adds `docs/release-notes/v1.2.0.md` and nothing else). It touches neither of this PR's files. - **`git merge-tree --write-tree origin/main HEAD`** exits 0 and writes tree `98765a8b`. The branch does not hold `961640c6`, and the predicted merge differs from `HEAD` by `docs/release-notes/v1.2.0.md` alone. - **#1166** (`fix/1139-anchored-patterns`, head `2593952c95`, opened 2026-10-22T17:31:57Z) touches `update-shape.py`, at lines well apart from `_missing_name_refusal` (rc1133's reading), and does not touch the test file. `git merge-tree --write-tree a7bfee1 2593952` exits 0 and writes tree `67858a0b`. Against `53791e45` the same command writes `37498506`, rc1133's tree, which is reproduced. rc1133's tests on the predicted merge of `53791e45` and #1166 (`c5a98365`: this file, `test_value_pattern_line_break.py`, `test_update_shape_unreadable.py`, `test_path_guard_hygiene.py`) were 311 passed; they were not re-run at `a7bfee1b`, which changes one assert and a docstring in a test file #1166 does not touch. - **Open pull requests:** 41 were listed with `gh pr list --state open --limit 100 --json number,headRefName,headRefOid,files`. Besides this one, only #1166 touches `update-shape.py`, and none but this one touches `tests/test_update_shape_kept_helper.py`, so no other head needs a merge-tree. The list is `open-prs.json`. Review: lane 5's independent re-check, by a reviewer that wrote none of #1133: READY at `a7bfee1b` (issuecomment-6352252707), with no MUST, SHOULD or NIT left. CI (Brett Heap's ruling 27 and his fast-refresh ruling of 2026-10-15T22:48:35Z): H0 a7bfee1 passed tests, tests-git-bash and SonarCloud; the head 08339c9 is a clean update-branch merge of main 105b5da, with tests-git-bash and SonarCloud green; and the local gate passed on its predicted squash tree b7073af over main 105b5da: the update_shape_kept_helper, repo_hygiene, agents_md_codes, cli_reference sets, 356 passed. Landed on Brett Heap's word of 2026-10-23T13:10:26Z, relayed to every lane by lane 3 (#473 issuecomment-6365697123): "merge v1.2.1 PRs once reviewed and green". Closes #1082 Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>



Lane: openreposhape-4 (openRepoShape-4)
Claim: #888 (comment)
Round 1: lane 5's review of #950, taken here
Lane 5 found #950 NOT READY at
4758e645, with one MUST and two NITs (#950 issuecomment-6253417294). Lane 4's plan is #950 issuecomment-6253442578. Every item is taken in1e94ce04785fdf0b4d71ee51367f7be396766ac1, one commit on4758e645, pushed as a fast-forward. Nothing pushed was rewritten.MAKEFLAGS_WORDwas(?:\\.|\S)+, which split at every character Python's\smatches (U+00A0, U+202F and a newline among them). GNU make splitsMAKEFLAGSonly at a space or a tab, and passes those three through raw, so under-j2a value holding one was cut short or split into a second variable. Both copies now read(?:\\.|[^ \t])+, and so does the test's ownWORD.bootstrap.py's comment andmember_make_envdocstring, andsiblings.py's comment, now say make's rule: a space or a tab, a backslash keeping either in a value.TAKEN_OUTspellings, built from Python escapes, each in both scripts: a value holding U+00A0 (NOTE=x<U+00A0>EXTRA=injected), U+202F (NOTE=x<U+202F>!), and a newline (NOTE=x<newline>EXTRA=injected). Each stays one word.TAKEN_OUTspellingk -j16 --jobserver-auth=3,4, which leavesk-j[0-9]?: killed, 1 failed in each scripttest_each_script_hands_back_a_copy_and_leaves_its_own_environment, in each script: with a jobserver named, the result is a plaindictwithMAKEFLAGStaken out, andos.environis as it wasenv = os.environ: killed, 1 failed in each scriptMUST 1, measured (GNU Make 4.3, a stand-in for the holder's rule with no
+,bootstrap.py --no-fetchand a member printingNOTEandEXTRAfrom its environment), undermake -j2 ... NOTE=<value>:NOTE=value501783054758e6451e94ce04x+ U+00A0 +EXTRA=injectedNOTE=[x] EXTRA=[injected]EXTRAempty, no warningx+ U+202F +!NOTE=[x]x+ newline +EXTRA=injectedNOTE=[x] EXTRA=[injected]EXTRAempty, no warningx+ tab +EXTRA=injected, andx\ yWith no
-j, all three heads pass every value whole. The member's ownMAKEFLAGSat1e94ce04under-j2readss -- NOTE=x<newline>EXTRA=injected, as under no-j.The tests: the new file has 43 items (33, then 4 spellings and the copy test, each in both scripts). On this head: 43 passed. All 18 mutants were killed (see "Mutants"). The 19-file gate, run again on this head: see "The run".
What changed and why
#888, filed by lane 5 from rv884's checklist candidate C1 in its review of #886 (#884), #886 issuecomment-6248806027. No milestone yet (lane 3's call). A DRAFT on main.
The head:
1e94ce04785fdf0b4d71ee51367f7be396766ac1: round 0's4758e645, one commit on main64464d31, and round 1's commit on it (above). Main has changed notemplates/family-rootfile since64464d31. Round 0's change was made and measured on50178305(the v1.1.1 release commit) asa4af51bb, and moved onto main before the first push, after lane 5's #891 landed asa9205e00and lane 1's #887 as64464d31.git merge-treeofa4af51bbwith64464d31was clean (tree979ee879, this head's tree): #891 changesMakeErrorsandMAKE_FAILED_REinbootstrap.py, and this PR the end of (b),run_make'sPopenline and the new code afterpassed_on.Under the family holder's
make -j2 bootstrap, each member's make printedmake[1]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule.and ran its jobs one at a time.make -j2 validate,parkandresumedid the same. The rule the warning names is the holder's pinned Makefile, which the person does not own.Why it happened:
scripts/bootstrap.pyandscripts/siblings.pywith no+and no$(MAKE), so GNU make closes the jobserver's two descriptors for them.Popen(andsubprocess.run) also close every descriptor past stderr.-j2 --jobserver-auth=3,4in theMAKEFLAGSit inherits, could not open the jobserver, and warned.The fix is the issue's second direction.
member_make_env()takes the jobserver of the make above, and the-jthat comes with it, out of theMAKEFLAGSa member's make inherits. Each member's make then runs its jobs one at a time, as it did, and says nothing.MAKEFLAGSnames a jobserver:--jobserver-auth=(GNU make 4.2 on: a pipe's descriptors, or 4.4'sfifo:<path>) or--jobserver-fds=(before 4.2, macOS's 3.81).-jword (-j2, or the bare-jolder makes write beside--jobserver-fds).-k,-s, and a variable given on the holder's command line. It splitsMAKEFLAGSas make does: only at a space or a tab, a backslash keeping either in a value. SoNOTE='a -j3'reaches the member whole, and so does a value holding a no-break space, a narrow no-break space or a newline, which make passes through raw (round 1).MAKEFLAGSwith no jobserver byte for byte:make -j(no number, no jobserver) and a person's ownMAKEFLAGS=-j8with no make above behave as before.Where it is used:
bootstrap.py:run_make'sPopen(make bootstrap, and--members-make validatefor the holder'svalidate).siblings.py:dispatch'ssubprocess.run(--make parkand--make resume).make siblingsruns no member's make. It runs each member's ownscripts/bootstrap.py, which runs none, so it printed no warning before and prints none now (measured below).Each script's docstring now says this:
bootstrap.py's (b) paragraph andmember_make_env, andsiblings.py's--makeparagraph.Before and Now
How it was measured. A probe (
probe_888.pyin lane 4's scratch, not committed) ran against this clone with-p conftest, so it builds the same state as the new tests with the suite's own fixtures:tests/test_family.py's family, two members IRRS and IRSS scaffolded and added byscaffold-project.pyandfamily.py, each member's Makefile replaced byTRACED(see "Tests"), each member cloned beside the holder forparkandresume. The holder's own Makefile ran each verb, with or without-j2.ghthat exits 97 first on PATH,GH_TOKEN=bogus, an emptyGH_CONFIG_DIR,GIT_CONFIG_GLOBALan empty file,GIT_CONFIG_NOSYSTEM=1,GIT_SSH_COMMAND=false,GIT_ALLOW_PROTOCOLunset, noMAKEFLAGS, and local remotes only./bin/sh.50178305; Now is this change on it,a4af51bb. Main's the family root's bootstrap.py: an(ignored)line, for any code or signal, resets the depth fully, so a member's ownexit 2after a-one or two makes below, or after|| trueand a-, readsexited 2, and the limits name the-klet-fail shape (#890) #891 touches no line this PR reads or writes; the new tests ran on this head (see "The run").50178305)make -j2 bootstrapmake[1]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule., one per member; each tracestart one, stop one, start two, stop two, flagsw -j1wmake -j2 validatew -j1wmake -j2 parkw -j1 -- ARGS=w -- ARGS=make -j2 resumew -j1 -- ARGS=w -- ARGS=make bootstrap,validate,park,resume(no-j)w(w -- ARGS=forparkandresume)make -j2 siblings,make siblings(the holder in a folder named after the family)make -n bootstrap,validate,parkEvery run above exited 0. Now, a member's make under
-j2runs with exactly the flags it runs with under no-j.Tests
A new file,
tests/test_family_member_make_jobserver.py, 43 items (33 in round 0, 10 more in round 1). It skips where there is nomake(astests/test_family_makefile_siblings.pydoes) and, withWINDOWS_SKIP, where there is no POSIX shell for the member's recipes.The state.
tests/test_family.py's family: two members, IRRS and IRSS, scaffolded and added byscaffold-project.pyandfamily.py, in a private copy (holder).TRACED. Each target the holder runs in a member (bootstrap,validate,park,resume) has two prerequisites,oneandtwo, that need nothing of each other. Each writesstart <name>, sleeps a second, and writesstop <name>to the member'strace.log. The target then writes the flags its make runs with ($MAKEFLAGS).parkandresume, each member is also cloned BESIDE the holder from its own remote, wheresiblings.py --makeruns it, and that clone getsTRACED.offline), a git configuration of the test's own (isolated), the file transport the local members need (fetching), andPYTHONfrom the environment, soMAKEFLAGScarries no variable the test did not give.The items:
test_no_member_s_make_warns_under_make_j2_and_its_jobs_run_serially[bootstrap|validate|park|resume]: the holder'smake -j2 <verb>exits 0 and prints nojobserver unavailable. Each member's trace isstart one, stop one, start two, stop two, and its flags carry no-jand no jobserver word.test_a_member_s_make_keeps_every_other_flag_and_a_plain_run_is_as_it_was: the holder'smake -k bootstrap NOTE='a -j3', thenmake -k -j2 bootstrap NOTE='a -j3'. Neither warns, both run serially, and each member's flags are the same in both runs:kkept,NOTE=a\ -j3whole, no-jword.test_each_script_takes_out_a_jobserver_and_its_j_and_nothing_else: 12MAKEFLAGSspellings, in each of the two scripts. Each make's jobserver word and the-jbeside it are taken out, the rest is kept, and nothing else in the environment changes. The spellings: 4.3's (measured), with-k -sand variables, and with a variable whose value holds-j4; 4.4'sfifo:path, also with a blank in it; 3.81's--jobserver-fds=3,4 -j, also withk; Windows' semaphore name; and, in round 1, 4.3's with-j16, and with a value holding U+00A0, U+202F or a newline, built from Python escapes.test_each_script_leaves_a_makeflags_that_names_no_jobserver: 6 cases, in each of the two scripts (unset, empty,-j,-j8, variables only, and a variable whose value looks like a jobserver word). The environment is returned unchanged. These items also hold the two scripts' copies to one answer.test_each_script_hands_back_a_copy_and_leaves_its_own_environment(round 1), in each of the two scripts: with a jobserver named, the result is a plaindictwithMAKEFLAGStaken out, andos.environis as it was.On main's two scripts, with this file: round 0's 33 items all failed on
50178305and again on64464d31, and round 1's 43 all fail on main66177b62, whose two scripts are64464d31's. The 5 behavioural items fail on the warning (eachmake -j2printed it twice, once per member); the 38 others because neither script has amember_make_env.Choices for the reviewer
1. Option 2 (take the jobserver out), not option 1 (hand it on). Option 1 was to be preferred if it is clean on GNU make 4.3, 4.4+ and macOS's 3.81, since the person asked for
-j; otherwise option 2, which never warns and never deadlocks. Both were prototyped and measured on this host's GNU Make 4.3, in the real holder (the same probe and state as the tests), each member traced:make -j2 bootstrap,validate,park,resumemake -j2 bootstrapmake -n bootstrap50178305w -j1+on the four rules that run a member's make (bootstrap,validate's second line,park,resume), and the descriptorsMAKEFLAGSnames passed on (pass_fds) inrun_makeanddispatchw -j2 --jobserver-auth=3,4bootstrap.py:every member and leg fetched+(pass_fdsalone; a first prototype, a parent makefile standing in for the holder's rule)s -j1(that prototype ranmake -s)w, as under no-j-j2kept-j2)w -j2 --jobserver-auth=3,4, a jobserver of the member's ownOption 1 is not clean, even on 4.3:
+changesmake -n. A+(or$(MAKE)) marks a line recursive, and make runs it under-n,-tand-qtoo. With+,make -n bootstrapranbootstrap.pyfor real (step (a)'sgit submodule update --init --recursive), where it now only prints the command. With+,make -n validateandmake -n parkranbootstrap.py --members-make validateandsiblings.py --make parktoo; each member's make got-nand only printed its lines.MAKEFLAGSnames the jobserver's descriptors by number. In a first prototype (bootstrap.pyrun straight,MAKEFLAGS=' -j2 --jobserver-auth=3,4', with descriptors 3 and 4 open on a person's files), the member's make read a byte of one file for a token and wrote a+into the other.pass_fdsalone does not help. Without the+, make has already closed the descriptors for the script: the member still warned and ran serially (measured).bootstrap.pyandsiblings.pyrun$MAKEormakeon PATH, and GNU make exports noMAKEto a recipe (measured: unset). So a holder run by one make can start a member's make of another version, and the member then has to read a jobserver word its parent wrote. Measured on 4.3, with each spelling inMAKEFLAGSand nothing open:--jobserver-auth=fifo:/tmp/GMfifo999999stops make:make: *** internal error: invalid --jobserver-auth string 'fifo:/tmp/GMfifo999999'. Stop., exit 2. So does Windows'gmake_semaphore_1234.--jobserver-fds=3,4 -jgives the samejobserver unavailablewarning, and a serial run.On 4.4+ and 3.81 (read, not run: neither is on this host):
mkfifoexists and write--jobserver-auth=fifo:<path>intoMAKEFLAGS. Their NEWS says that with it a sub-make connects to the jobserver even where it is not marked recursive. So on 4.4 with the same make on PATH, a member's make today probably opens the holder's fifo by path and runs its jobs together, with no warning and no change here needed. With this PR it runs them one at a time there too. That is the cost of one rule for every make. Keeping afifo:word would give 4.4 its jobs back, but it would stop a member's make with the fatal error above wherever the holder's make is 4.4 and themakeon PATH is 4.2 or 4.3.make --jobserver-style=pipeon 4.4 writes3,4as 4.3 does and behaves as 4.3 does.--jobserver-fds=3,4 -j: the descriptors, and a bare-j, which alone means as many jobs as make likes. It also closes the descriptors for a line with no+, and its source prints the same warning, so a member's make there warned the same way. Here both words are taken out, so the member runs serially and says nothing. Taking out the jobserver word alone would leave the bare-j, and the member would run unbounded (measured on 4.3:-jinMAKEFLAGSran both jobs together).2. Not taken: give each member its own
-jN. Taking out only the jobserver word and keeping-j2(2b in the table) gives each member's make a jobserver of its own with the same N. On 4.3 it ran each member's two jobs together with no warning. It is none of the issue's three directions, so it is not taken here:make -j4 bootstrap validateat a holder could run 4 jobs in each of two members at once.-jcarries no number, so it would still have to be taken out there (serial).JOBS_WORDdrops only the jobserver word and a bare-j.3. Only a
MAKEFLAGSthat names a jobserver is touched.make -jwith no number starts no jobserver, and a member's make that inherits-jruns unbounded and says nothing, as it did. A person's ownMAKEFLAGS=-j8withbootstrap.pyrun straight is left as it was too. Neither ever printed the warning.4. Written in both scripts, not imported.
siblings.pyalready imports four names frombootstrap.py. A new name imported from there would makeupdate-shape.py apply --accept-localrefuse a holder that keeps its ownbootstrap.py(update-accept-local-missing-name, #536, #604), and a holder that got past that would endmake parkin anImportError. So the three patterns and the short function are written again insiblings.py, asvalidate-pins.pywrites_shownrather than import it, and the twomember_make_envtests hold both copies to one answer on every case.5. Silent. No line says at run time that the members ran one at a time; the two docstrings say it. The issue asks for no warning, and a line of the holder's own under every
-jrun would be one more line nobody acts on.6.
make siblingsand a member's own bootstrap are left alone.run_member_bootstrapruns a member'sscripts/bootstrap.py, which runs no make, andmake -j2 siblingsentered no member make before or after (measured below). Its environment is unchanged.7. The fix is in the scripts, not the Makefile. A
MAKEFLAGS=...prefix in each holder rule would be a third pinned file changed (a thirdupstream-changedrow), would depend on make's own functions in every version, and would not coverbootstrap.pyrun straight from another make's recipe.Mutants
Each is one edit of a script, the new file run against it (43 items), and the bytes put back and checked by sha256 after each (
mutate.pyin lane 4's scratch). Run on1e94ce04; the unmutated control:43 passed. M1 to M12 also ran on4758e645against its 33 items, each killed.run_make'sPopenwithoutenv=member_make_env()(thebootstrapandvalidatepath)[bootstrap],[validate], the-ktest)dispatch'ssubprocess.runwithout it (theparkandresumepath)[park],[resume])bootstrap.pykeeps-j2and takes out only the jobserver word (2b)[bootstrap]and[validate](the traces interleave), the-ktest, the 12bootstrap.pyspellings, and the copy testsiblings.py[park],[resume], the 12siblings.pyspellings, and the copy testbootstrap.pyrewritesMAKEFLAGSwith no jobserver named-j,-j8, variables only, a variable like a jobserver word)siblings.pybootstrap.pysplits words at every blank (\S+)-ktest (NOTE=a\ -j3lost its-j3), and the spellings with a kept blank, a no-break space or a newlinesiblings.pybootstrap.pykeeps a bare-j(-j[0-9]+)[3.81],[3.81-k])siblings.pybootstrap.pyreads no--jobserver-fds[3.81],[3.81-k])siblings.pybootstrap.pysplits as before round 1 ((?:\\.|\S)+, MUST 1)[4.3-no-break-space],[4.3-narrow-no-break-space],[4.3-newline])siblings.pybootstrap.pytakes one digit after-j(-j[0-9]?, NIT 1)[4.3-j16-k])siblings.pybootstrap.pyeditsos.environitself (env = os.environ, NIT 2)siblings.pyAll 18 of 18 were killed, and both scripts' bytes matched their sha256 after each (
bootstrap.py045f33a1,siblings.py7c1abf54, the committed bytes). M1 and M2 are one per path. M3 shows the trace tells jobs run together from a serial run: its 4 traces interleaved.Not measured
MAKEFLAGSspellings.WINDOWS_SKIP); CI is Linux only (ci: run the test suite on Linux only #597).Pinned copies
templates/family-root/Makefile,scripts/bootstrap.pyandscripts/siblings.pyare all inFAMILY_COPIED_VERBATIM(scripts/shape_materialize.py:531). This PR changes two of them, so each holder takes twoupstream-changedrows,scripts/bootstrap.pyandscripts/siblings.py; the Makefile staysunchanged.siblings.pyimports nothing new frombootstrap.py(choice 4), so a holder that keeps its ownbootstrap.pywith--accept-localis not refused for it.What a holder sees. A holder was cut from main
64464d31withfamily.py init(local remotes), andupdate-shape.py check --upstream <this clone> --at 1e94ce04printed (the same rows as at4758e645):Shared files
Each row is
git merge-tree --write-treeof this head1e94ce04785fdf0b4d71ee51367f7be396766ac1with the named commit. Heads were read withgit ls-remoteat 19:08Z. The open PRs that touch any of the three pinned files (gh pr list --state open --json files, 19:08Z) are this one, lane 4's #889 (#879), #951 and #954 (lane 4's #892 branch,fix/892-bootstrap-ok-after-128).66177b62c6be8e45293ef2a5e67adaba32a5899f(lane 4, #879)be1242d7528adbf76a1f7ebf9eb9d3d2(#892)b5e23311No conflict, so no pre-resolution. The merged trees were not run. Round 0's head merged cleanly with #891's last head too (
41f781fd, tree0ae5ea27), which has landed since asa9205e00and is in this branch's base.The run
On this head (
1e94ce04785fdf0b4d71ee51367f7be396766ac1), one background run of 19 files:GIT_CONFIG_GLOBAL=/dev/null: a stubghthat exits 97 first on PATH,GH_TOKEN=bogus, an emptyGH_CONFIG_DIR,GIT_CONFIG_NOSYSTEM=1,GIT_SSH_COMMAND=false,GIT_ALLOW_PROTOCOL,MAKEFLAGS,MAKELEVELandMFLAGSunset; GNU Make 4.3, git 2.43.0, Python 3.12.3.1332 passed in 957.27s (0:15:57), 19:08:40Z to 19:24:39Z, exit 0, no skips. That is round 0's 1322 and round 1's 10 new items. The three files' bytes were checked against the commit's blobs afterwards (git hash-object), and the clone was clean.tests/test_family_holder_bootstrap_texts.py,tests/test_family_bootstrap_gate_order.pyandtests/test_family_makefile_siblings.py;git grep -l -e run_make -e "members-make" -e MAKEFLAGS tests/lists:tests/test_adopt_clone_claimed.py,tests/test_adopt_plan_out_checked.py,tests/test_bootstrap_spelling.py,tests/test_dry_run_parity.py,tests/test_park_resume_targets.py,tests/test_setup_project_dry_run_offers.py;tests/test_repo_hygiene.py,tests/test_agents_md_codes.pyandtests/test_cli_reference.py;tests/test_pinned_copy_text.py(the skew rule),tests/test_siblings_inputs.py,tests/test_family_siblings_placement.py,tests/test_family_gitmodules_values.pyandtests/test_update_shape_kept_helper.py; andtests/test_family.py.4758e645): the same 19 files,1322 passed in 1906.79s (0:31:46), 16:15:16Z to 16:47:04Z, exit 0, no skips. A first run ona4af51bbwas cut off at 16% when the lane's session hit its usage limit; nothing failed before it stopped.The docs. No README, handbook,
AGENTS.md,AGENTS-shape.mdordocs/cli.mdsentence speaks of-j, a jobserver orMAKEFLAGS(git grep -i -e "make -j" -e jobserver -e MAKEFLAGS -- '*.md'finds none), so none is made false.--helpis unchanged.Closes #888
🤖 Generated with Claude Code
Summary by Sourcery
Stop family member make commands from inheriting unusable jobserver settings while preserving other make flags and keeping member work serial under parallel holder workflows.
Bug Fixes:
Enhancements:
Tests: