Skip to content

adopt-project.py: plan's source line names the source's branch as _shown names a path, so a bidi control, a line boundary or a byte that is not UTF-8 in it is spelled and no longer reaches the terminal raw (#1035) - #1120

Draft
brettheap wants to merge 2 commits into
mainfrom
fix/1035-source-branch-shown

Conversation

@brettheap

@brettheap brettheap commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Closes #1035

What was wrong

plan's source line printed the source's branch as it is: ({source.branch}). A branch is a name its author chose, and git lets one hold U+202E and the other bidirectional controls, the line boundaries U+0085 and U+2028, and a zero-width space. A source on ma<U+202E>in, planned with --tracking-branch main, put the U+202E in the terminal's bytes, where it reorders what the line shows after it.

What changed

adopt-project.py, in _print_plan_report: the line names _shown(source.branch), the helper plan already uses for every other name it prints (the table, the follow-ups, the split message's listing, #199 and #900), and as check spells this branch in plan-stale and plan-tracking-branch-missing. No new escaper.

  • A printable branch prints exactly as before: (main) stays (main).
  • Any other branch is spelled between backticks: (`ma\u202ein`).
  • The line for a source plan cloned is the same line, so it gets the same spelling.

Tests

tests/test_adopt_plan_source_branch_shown.py (new, 19 rows):

  • Seven end-to-end rows make the branch in a real local source and run plan with the real gh unreachable, reading the bytes it prints: U+202E, U+2066, U+0085, U+2028, U+200B, an ordinary feat/x-1 (must print unchanged), and a branch that is not UTF-8 under a strict stdout (skipped where no filesystem can name the byte). Each asserts the raw character is not in stdout, the whole source line, and that the plan file still records the branch as it is.
  • Twelve in-process rows call _print_plan_report with the branch set on the Source: seven ASCII controls (NUL, BEL, BS, CR, LF, ESC, DEL), which git refuses in any ref name so no source can hold one, and five characters for the cloned-source form.

On main's adopt-project.py: 18 failed, 1 passed (the ordinary-branch row, which guards against over-escaping). With this change: 19 passed.

Mutants, each run on the new file with the file restored and its sha256 checked after: raw branch 18 failed; _spelled for _shown 1 failed; repr 19 failed; _shown for ASCII branches only 11 failed; strips only U+202E 2 failed; _shown on the first character only 18 failed.

Gate, targeted (Linux, the landing gate's environment, a stub gh), at this head over two shards: 1708 passed, 1 skipped for the new file, the adopt tests that read plan's report, branch names or locale bytes, test_scaffold_leg_path_collisions.py, test_repo_hygiene.py and test_cli_reference.py; 445 passed, 3 skipped for test_shape_doctor.py. No failure.

Known follow-up, outside this change

plan written to {out} (in cmd_plan) prints the --out path as typed, so a bidi control in it reaches the terminal raw. It is the caller's own typed path and not repository data, and the issue names the source line only, so it is left as it is. Lane 4 files the issue for it.

The plan table's padding (#1034) is untouched.

Open PRs on adopt-project.py: this head merges cleanly with #1105 and #1106. #1091 conflicts with main itself in lines 695-860, away from the hunk here, and not because of this change.

🤖 Generated with Claude Code
Lane: openreposhape-4 (openRepoShape-4)

Summary by Sourcery

Safely spell source branch names in plan reports to prevent terminal-confusing output without changing ordinary branch display.

Bug Fixes:

  • Sanitize source branch names in plan reports so bidirectional controls, line separators, zero-width characters, ASCII controls, and non-UTF-8 bytes cannot reach the terminal raw while preserving the branch in the generated plan.

Enhancements:

  • Keep printable branch names unchanged while applying the existing safe-name spelling consistently to cloned and local sources.

Tests:

  • Add end-to-end and in-process coverage for unsafe, printable, control-character, cloned-source, and non-UTF-8 branch names.

Brett Heap and others added 2 commits October 21, 2026 13:09
… `_shown` names a path, so a bidi control, a line boundary or a byte that is not UTF-8 in it is spelled and no longer reaches the terminal raw (#1035)

The branch was printed as it is, `({source.branch})`. It is `_shown` now: a
printable branch reads as it did, and any other is spelled between backticks as
`check` spells it in `plan-stale` and `plan-tracking-branch-missing`.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…r and zero-width characters are written as ASCII escapes in the rows' source, not as the characters (#1035)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@sourcery-ai

sourcery-ai Bot commented Oct 21, 2026

Copy link
Copy Markdown

Reviewer's Guide

The PR updates plan's source report to render branch names with the existing _shown escaping logic, preventing terminal-affecting characters and invalid UTF-8 from being emitted raw while preserving ordinary branch output. A focused test suite covers real Git/CLI paths, synthetic invalid ref names, cloned sources, and plan-file fidelity.

File-Level Changes

Change Details Files
Sanitize source branch names in the plan report before writing them to terminal output.
  • Changed the source summary to pass source.branch through the existing _shown formatter.
  • Preserved unchanged output for printable branch names while escaping bidi controls, line boundaries, zero-width characters, ASCII controls, and invalid UTF-8 bytes.
  • Applied the same branch spelling to cloned-source report lines.
adopt-project.py
Added regression coverage for safe branch rendering across real CLI execution and direct report generation.
  • Added end-to-end tests for bidi controls, Unicode line boundaries, zero-width space, ordinary branches, and non-UTF-8 branch bytes.
  • Added in-process tests for ASCII controls that Git cannot store in ref names and for cloned sources.
  • Verified the rendered source line remains printable and that the plan file retains the original branch name.
tests/test_adopt_plan_source_branch_shown.py

Assessment against linked issues

Issue Objective Addressed Explanation
#1035 Prevent plan’s source line from printing branch names containing bidi controls or other unprintable characters raw, while leaving printable branch names unchanged. ✅
#1035 Check other plan output for values printed raw and identify any remaining cases. ✅

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sonarqubecloud

Copy link
Copy Markdown

@brettheap

Copy link
Copy Markdown
Contributor Author

RV1120: independent review of #1120 at c55e55e3: READY

Independent adversarial review for lane 4. No MUST, no SHOULD; four NITs for follow-ups, below.

Merge with current main

The fix

  • _print_plan_report prints _shown(source.branch), the helper the table, follow-ups and questions already use. The cloned form shares the line. Nothing else changed.
  • _shown, every code point. _shown("ma" + chr(cp) + "in") for all 0x110000 code points (Python 3.12.3, Unicode 15.0): no output holds a character str.isprintable() rejects, and no unprintable input comes back raw. Every Cc, Cf, Cs, Co, Cn, Zl, Zp, and Zs other than the ASCII space is spelled: bidi embeddings, overrides and isolates, LRM/RLM/ALM, U+200B-U+200D, U+2060, U+FEFF, U+00AD, U+180E, the tag characters, U+0085, U+2028/U+2029, U+00A0.
  • End to end (a real source on the branch, plan --tracking-branch main, real gh unreachable): U+200D, U+2029 and U+00A0 are spelled; feat/caf\u00e9 and \u6a5f\u80fd/x print unchanged; a branch named `main` is spelled `\x60main\x60`.
  • A branch that is not UTF-8 (caf<0xE9>), by PYTHONIOENCODING:
stdout main PR
utf-8:strict (caf\udce9) (`caf\udce9`)
utf-8:surrogateescape raw byte 0xE9 (`caf\udce9`)
utf-8:replace (caf?) (`caf\udce9`)
utf-8:backslashreplace (caf\udce9) (`caf\udce9`)
ascii:strict (caf\udce9) (`caf\udce9`)
latin-1:surrogateescape raw byte 0xE9 (`caf\udce9`)

The test file

  • ASCII only: LC_ALL=C grep -nP '[^\x00-\x7F]' finds nothing in it or in the PR's added lines. c55e55e33 replaces exactly the four raw characters with \u202e, \u2066, \u2028, \u200b.
  • On the merge: 19 passed, 0 skipped (the non-UTF-8 row ran).
  • With main's adopt-project.py: 18 failed, 1 passed (the ordinary-branch row). Each fails for the right reason: the raw character is in the output (the five end-to-end rows, the seven controls, the five cloned rows), and the non-UTF-8 row lacks the backticks.

Gate (Linux, the landing gate's environment, stub gh), on 9480fc65f

Files Result
the new file, test_adopt_source_clone_record, test_adopt_plan_escapes, test_adopt_plan_out_checked 205 passed
test_adopt_branch_names, test_adopt_tracking_branch, test_adopt_c_locale_bytes, test_adopt_literal_names, test_adopt_git_reads, test_tracking_branch_ref, test_non_utf8_git_failures 317 passed
test_scaffold_leg_path_collisions, test_repo_hygiene, test_cli_reference 227 passed
test_shape_doctor (two shards) 445 passed, 3 skipped (Windows-only PowerShell rows)

Mutants (mine; each on the new file, the file restored and its sha256 checked after each)

Mutant at the source line Result
A: raw branch for a cloned source 5 failed
B: raw branch in place 13 failed
E: _spelled for every non-ASCII branch 0 failed (survives)
F: backticks stripped from the spelling 18 failed
G: only categories C* spelled 2 failed (U+2028 rows)
J: encode('ascii', 'backslashreplace') 18 failed
L: a surrogate replaced with ? 1 failed
M: the plan file records _shown(branch) 5 failed
N: isprintable() alone, no backtick rule 0 failed (survives)

E and N leave correct code's behaviour unpinned (N2 below); the PR's code is right on both.

Findings

  • N1 (NIT), _shown itself. str.isprintable() passes the 267 Default_Ignorable code points that are not Cf: 263 Mn, among them U+034F, U+17B4/U+17B5, U+180B-U+180D, U+180F, U+FE00-U+FE0F and U+E0100-U+E01EF, and the four Lo fillers U+115F, U+1160, U+3164 and U+FFA0. Git takes them in a branch, and U+034F, U+3164 and U+FE0F reach plan's stdout raw (measured), so ma<U+034F>in reads as main. The unsafe-value refusal's repr prints U+3164 raw on stderr the same way. They reorder nothing, and this is the helper everywhere, not this line. Follow-up: treat Default_Ignorable_Code_Point as unprintable in _shown and escape it in _spelled_character, as plan and check both print through them.
  • N2 (NIT), the tests. Add a printable non-ASCII branch (feat/caf\u00e9) to test_plan_names_an_ordinary_branch_as_it_did, and a row for a branch named `main`. Mutants E and N survive without them.
  • N3 (NIT), the tests. The non-UTF-8 row runs only under utf-8:strict, where main wrote no raw byte. Parametrize it over utf-8:surrogateescape too, the stdout on which main wrote 0xE9 raw.
  • N4 (NIT), outside adopt-project.py: plan's source line prints the source's branch raw, so a bidi control in a branch name reaches the terminal #1035. cloning {repository} (read only) into {path} prints the caller's --work-dir raw for an org/repo source; print it _shown, with adopt-project.py: plan written to prints the caller's --out path raw, so a bidi control in it reaches the terminal, where the other plan lines spell paths with _shown #1115's --out.

Lane: openreposhape-4 (openRepoShape-4)

This was referenced Oct 21, 2026
brettheap pushed a commit that referenced this pull request Oct 23, 2026
…a value that holds three default-ignorable characters is refused and shown with each one escaped, the compiled `DEFAULT_IGNORABLE` is held to the table, and the skew rule knows the four new names (#1128)

RV1171's review of #1171 (issuecomment-6356224053), the SHOULD and two of
the NITs.

- tests/test_adopt_shown_default_ignorables.py (46 rows, now 48):
  - S1: `m` U+034F `a` U+3164 `in` U+FE0F through `checked_value`'s
    `unsafe-value` refusal and through `_shown`: each is written as its
    escape and none is raw. Every other row held one character to a value,
    so a `visible_repr` that escaped only the first match passed all of
    them.
  - N1: the compiled `DEFAULT_IGNORABLE` matches exactly the table's code
    points over all 0x110000. The sweeps take their expected value from the
    pattern, so a pattern one past each range passed them.
- tests/test_pinned_copy_text.py: `DEFAULT_IGNORABLE_RANGES`,
  `DEFAULT_IGNORABLE`, `code_point_escaped` and `visible_repr` join
  `NEWER_THAN_A_KEPT_LIBRARY`, so the skew rule refuses a pinned script that
  imports one.

Stacked on #1120 (see #1035). Test-only.

Lane: openreposhape-4 (openRepoShape-4)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@brettheap brettheap added this to the v1.2.1 milestone Oct 23, 2026
brettheap added a commit that referenced this pull request Oct 23, 2026
…py` where GNU make at the root reads a kept `GNUmakefile` or `makefile`, and its NOTE names that file and what it takes for `make bootstrap` to run there (#1047) (#1121)

GNU make reads the first it finds of `GNUmakefile`, `makefile` and `Makefile`, and only that one. Where the source keeps `GNUmakefile` or `makefile` in the root, `make bootstrap` at the root of the clone reads the source's file, not the shape's `Makefile`, and exits 2 with "No rule to make target 'bootstrap'". `execute`'s NEXT printed it there anyway.

This PR also covers the third box of #1048 (see #1048): the `_next_steps` NOTE that said the shape's targets "arrive only once the follow-up merges the two", which is untrue where GNU make reads a kept `GNUmakefile` or `makefile`. #1105 covers that issue's other two boxes, and leaves this one to this PR.

## Measured

GNU Make 4.3 on Linux, a disk that keeps case. One directory per set of files, each file with a `which` target:

| files | `make which` reads | `make -n bootstrap` |
|---|---|---|
| `GNUmakefile` | `GNUmakefile` | exit 2, no rule |
| `makefile` | `makefile` | exit 2 |
| `Makefile` (with a `bootstrap` rule) | `Makefile` | exit 0 |
| `GNUmakefile` + `makefile` | `GNUmakefile` | exit 2 |
| `GNUmakefile` + `Makefile` | `GNUmakefile` | exit 2 |
| `makefile` + `Makefile` | `makefile` | exit 2 |
| all three | `GNUmakefile` | exit 2 |

A `GNUmakefile` that holds `include Makefile` runs the shape's rule (exit 0), and so does `make -f Makefile bootstrap`.

What NEXT printed on main, from a real `plan`, `check` and `execute --local-remote-dir`, with the printed command run in a recursive clone of the split ("S" is the source's own `scripts/bootstrap.py`, kept with `scripts/`):

| the root keeps | NEXT on main | in the clone |
|---|---|---|
| none | `make bootstrap` | exit 0 |
| `Makefile` | `python3 scripts/bootstrap.py`, #480's NOTE | exit 0 |
| `makefile`, `GNUmakefile`, or both | `make bootstrap` | **exit 2, no rule** |
| `Makefile` + `makefile`, `Makefile` + `GNUmakefile`, all three | `python3 scripts/bootstrap.py`, and a NOTE saying the targets arrive once the two `Makefile`s are merged | the command exits 0; `make bootstrap` exits 2, and still does after `cat shape/Makefile >> Makefile` |
| S + `GNUmakefile`, S + `makefile`, S + `Makefile` + `GNUmakefile` | no command; the NOTE says to merge, "and then run `make bootstrap`" | `make bootstrap` exits 2 |

## What changed

`adopt-project.py`, `_next_steps`:
- It takes `surviving=`, the paths the plan's entries send to the root (`paths_for["root"]`, passed by `cmd_execute`).
- It asks them for a kept `GNUmakefile` or `makefile` in GNU make's order, with `_root_keeps`, as `_makefile_follow_up` asks them. So NEXT names the same file as the plan's `CONTRACTS_DIR` follow-up.
- Where it finds one, the command is `python3 scripts/bootstrap.py`. The NOTE names the file in the follow-up's words, and says `make bootstrap` and the shape's other make targets run there only once that file has them, or includes a Makefile that does:
  - the shape's `Makefile` is the root's: "NOTE the shape's Makefile is the root's `Makefile`, but GNU make at the root reads the source's `GNUmakefile`, the first of `GNUmakefile`, `makefile` and `Makefile` it finds, so `make bootstrap` and the shape's other make targets run there only once `GNUmakefile` has them, or includes a Makefile that does; `python3 scripts/bootstrap.py` is what the shape's `make bootstrap` runs."
  - the source also keeps its own `Makefile`: "NOTE the root keeps its own Makefile and the shape's is shape/Makefile, and GNU make at the root reads ..." (the same clause, and the same ending).
  - the source's own script went aside too: the merges first, as before, "and then run `python3 scripts/bootstrap.py`; GNU make at the root reads ...".
- A root that keeps neither name gets the block it got before, byte for byte. That includes `MAKEFILE`, which GNU make never reads where case is kept.

## Tests

`tests/test_adopt_next_make_reads_first.py`, new, 40 tests (38, and the 2 of the fix round below):
- 13 rows word for word: none, `Makefile`, `makefile`, `GNUmakefile`, `makefile`+`GNUmakefile`, `Makefile`+`makefile`, `Makefile`+`GNUmakefile`, all three, S, S+`Makefile`, S+`GNUmakefile`, S+`makefile`, S+`Makefile`+`GNUmakefile`.
- 3 rows that keep `make bootstrap`: `MAKEFILE`; longer names (`makefile.am`, `Makefile.in`, `GNUmakefile.bak`); and the names below the root.
- 13 rows in which NEXT names the same file as `_makefile_follow_up`, for the same root paths.
- 9 end to end rows, one for each row that keeps `GNUmakefile` or `makefile`. Each is a real `execute`, and its clone is made by pasting NEXT's own clone line. In that clone:
  - NEXT's lines are the row's;
  - `make which` prints the file the NOTE names;
  - `make -n bootstrap` has no rule;
  - the printed command exits 0 and ends `bootstrap ok`. For S rows, it does so once the shape's script is merged, as the NOTE says;
  - where a `Makefile` is kept, the follow-up's merge alone still leaves no rule;
  - an `include Makefile` in the file GNU make reads makes `make -n bootstrap` run the shape's recipe.

On main's `adopt-project.py`, the first 38 all fail. The 9 end to end rows fail on what NEXT prints. The unit rows that guard the unchanged cases fail there only for want of the `surviving=` keyword. M1 below shows the same rows passing and every row that keeps `GNUmakefile` or `makefile` failing. On this head, all 40 pass.

Mutants of the change, each restored and checked by sha256. Each run is the new file plus the five `next` tests of `test_adopt_e2e.py` (43 tests). Every mutant is killed:

| mutant | failed |
|---|---|
| M1 `first` never found | 27 |
| M2 `makefile` asked before `GNUmakefile` | 6 |
| M3 the command ignores `first` | 6 |
| M4 the branch for a `Makefile` kept beside `first` dropped | 6 |
| M5 the branch for a script gone aside ignores `first` | 9 |
| M6 the lookup folds case | 7, two of them existing `test_adopt_e2e.py` rows |
| M7 `cmd_execute` passes no paths | 9, the end to end rows |

Gate on this head, named files only, one basetemp per run. Each run had `EMAIL` set, `GIT_CONFIG_GLOBAL` an empty file, `GIT_CONFIG_NOSYSTEM=1`, a stub `gh` first on PATH, `GH_TOKEN=bogus` and an empty `GH_CONFIG_DIR`:
- the new file, `test_adopt_make_reads_first`, `test_adopt_e2e`: 98 passed, with the new file at 38 tests (at `c04bb710`; the fix round's gate is below).
- `test_adopt_github_exits`, `test_adopt_printed_sentences`, `test_repo_hygiene`, `test_readme_windows_quoting_sentence`, `test_cli_reference`, `test_agents_md_codes`: 291 passed.
- `test_scaffold_leg_path_collisions`: 104 passed.
- `test_adopt_contracts_dir`, `test_adopt_verify`, `test_adopt_pins`, `test_adopt_root_manifest`, `test_adopt_ignored_shape_files`, `test_adopt_nested_attributes`, `test_adopt_shape_pin_and_work_dir`: 378 passed.
- `test_shape_doctor.py` in 4 shards: 112 + 112 + 112 + 109 = 445 passed, and 3 skipped (the PowerShell line, which is printed only on a Windows host).

## Fix round 1 (RV1121's SHOULD-1)

RV1121 found the code right and one test missing: nothing held `cmd_execute` to handing `_next_steps(surviving=)` only the root's paths. Its mutant X4, which passes `paths_for["root"] + paths_for["code"] + paths_for["drop"]`, kept all 43 tests green. The case is a real one: the `root-project-makefile` rule sends a kept `GNUmakefile` to the root, and its reason says that moving a Makefile that only builds the code to the code leg "is a one-line edit of the plan" (`contracts/path-classification.yaml`, lines 236-237).

Two end to end rows, `test_next_keeps_make_bootstrap_where_the_plan_sends_the_makefile_elsewhere[code]` and `[drop]`. Each keeps `GNUmakefile` in the source, edits the plan to send it to `leg: code` (then `leg: drop`), runs a real `execute`, and pastes NEXT's own clone line. Each asserts that NEXT's lines after the clone are `    make bootstrap` and nothing else, that the clone's root has no `GNUmakefile`, and that `make -n bootstrap` there exits 0 and runs `python3 scripts/bootstrap.py`.

The `execute`, paste and clone steps of the nine existing end to end rows moved into one helper, `executed_and_cloned`, which the two new rows share, so no block of the test is written twice. Those rows assert what they asserted. The file now holds 40 tests (38 and 2).

- **X4, on this head's `adopt-project.py` with the one line changed:** the new file and the five `next` rows of `test_adopt_e2e.py` (45 selected): 2 failed, 43 passed. Both new rows fail on NEXT's lines, which are `python3 scripts/bootstrap.py` and the NOTE "the shape's Makefile is the root's `Makefile`, but GNU make at the root reads the source's `GNUmakefile`, ...", naming a file the root does not have. The file was restored and its sha256 read `60bad59a...` again.
- **On main's `adopt-project.py`:** the two rows pass, since main printed `make bootstrap` there. They hold the unchanged case against the new lookup, and X4 is the mutant that tells them apart.
- **On this head `1c5173758`**, same environment as above: the new file 40 passed, `test_adopt_e2e.py` 27 passed, `test_repo_hygiene.py` 100 passed.
- **Merge:** `git merge-tree --write-tree` of this head with main at `823624d93`: clean (tree `73ddafb2`).

RV1121's four NITs are not in this PR; the lane files them.

## Merge

- `git merge-tree --write-tree` of this head with main at `6da757cbe`, where the branch starts: clean (tree `21955cc7`). Main has since moved to `7cd66902a`, which changes `scaffold-project.py`, its test and the docs, and none of this branch's files. The merge with it is clean too (tree `ec4e6c8e`).
- With #1105's head `5a4de7f76`: clean (tree `3394d19d`). #1105's `adopt-project.py` change is in `_predict_collisions`, apart from `_next_steps`. On that merge, built in a `git clone --shared` checkout, `test_adopt_next_make_reads_first`, #1105's `test_adopt_collision_make_reads` and `test_pinned_copy_text` gave 152 passed.
- With the other open pull requests that change `adopt-project.py` (#1120 `c55e55e3`, #1119 `bdcf9952`, #1118 `c60066cb`, #1106 `504a1dd6`, #1091 `160de118`): each clean.

## Not in this PR

- Once #1105 lands, the lookup "which of `GNUmakefile` and `makefile` the root keeps first" is written three times: in `_makefile_follow_up`, `_predict_collisions` and `_next_steps`. Each is the same two lines over `GNU_MAKE_READS` and `_root_keeps`. They are kept apart here so this PR and #1105 touch disjoint lines; one helper could serve all three after both land.
- `templates/assembly-root/AGENTS-shape.md` line 19 ("`git clone --recurse-submodules` this repository and run `make bootstrap`") has the same gap in an adopted root that keeps `GNUmakefile` or `makefile`. It is pinned, and #1105 changes that file, so it is left.
- On a disk that ignores case (#1049), a kept `makefile` is reported as a `Makefile` collision, and NEXT's NOTE says "the root keeps its own Makefile". The new clause is true there too, since GNU make reads that file and merging into it brings the targets. The spelling is #1049's.
- A kept directory named `GNUmakefile/` or `makefile/` is named as the source's file. GNU make stops on it with "Is a directory". `_makefile_follow_up` does the same.

Closes #1047

Lane: openreposhape-4 (openRepoShape-4)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Nov 8, 2026
…s message spells a path byte that is not UTF-8 as `_shown` does, an escape, and no longer says it names it with a `?` in its place, which #900 replaced (#1018) (#1104)

`env_commit`'s docstring in `scripts/shape_materialize.py` says what the split commit's message does with a `root` or `drop` path that holds a byte that is not UTF-8, as the code does since #900. It said the message names such a path "with a `?` in place of that byte, a path that does not exist". That was true before #900, and it is not true now. A reader of the docstring is told about a `caf?.md` the tool no longer writes.

Found by lane 1's writer on #900 and filed as #1018. Lane 5 took it outside v1.2.0, under Brett Heap's ruling of 2026-10-21T10:32:04Z.

Measured on main `2a119f281` and again on `16f9a0e81` (main after #1087), with a small script that calls: `adopt-project.py`'s own `_split_message` and `_shown`, then `env_commit` and a real `git commit`, read back with `cat -A`.

- The issue's claim held: the docstring (about line 2103 when filed, line 2268 on `2a119f281`, line 2349 on this branch) still said the message names the byte with a `?`.
- The code does not. `_shown("caf\udce9.md")` is `` `caf\udce9.md` ``, and the split message for a `root` and a `drop` entry named that holds no lone surrogate. Through `env_commit` and `git cat-file commit HEAD`, `cat -A`:

  ```
  DROPPED (in no leg and no longer here):$
    `caf\udce9.md`$
  STAYS IN THE ASSEMBLY ROOT:$
    README.md$
    `caf\udce9.md`$
  ```
- What stays true of `env_commit` itself: a message that holds the lone surrogate, handed to it directly, is still written as `?`. `cat -A` reads `Name: caf?.md$`. So the `errors="replace"` paragraph is kept, and what changes is who it describes.
- #1087 (main `16f9a0e81`) changed `scripts/shape_materialize.py` by 128 lines, none of them in `env_commit`, `COMMIT_COMMAND` or the message, and changed nothing in `adopt-project.py` (0 diff lines). The rebase onto it was clean and the docstring needed no other change.

One file, `env_commit`'s docstring, in two paragraphs: the `errors="replace"` paragraph (round 1, 11 lines added and 7 removed) and the sentence above it (round 2, below; 3 added and 2 removed). Together 14 added and 9 removed against main `6da757cb`. The `errors="replace"` paragraph now says:

- encoding, a character UTF-8 cannot carry is written as `?`, and nothing raises after both legs are pushed, as before;
- `adopt-project.py` refuses a lone surrogate in a path it extracts (`entry_path_problem`), and a `root` or `drop` entry that holds one is listed in the split message but never raw: the message names each path as `plan`'s report does (`_shown`, #900), so the byte 0xE9 reads as an escape with no surrogate in it, `` `caf\\udce9.md` ``, the path the source holds;
- written raw it would be a `?`, `caf?.md`, a path that does not exist, which is what `env_commit` still writes for a lone surrogate it is handed.

The backslash is doubled in the source, as the neighbouring docstrings write `\\udcc3`. A single `\u` in a non-raw docstring would be an escape that makes a lone surrogate. Measured: the docstring holds no character from U+D800 to U+DFFF and holds the six characters `\udce9`. The longest docstring line is 78 characters.

**The code is unchanged.** A script compares the AST of `scripts/shape_materialize.py` with every docstring blanked: the branch's file is identical to `origin/main`'s after the rebase onto `16f9a0e81` (round 1), and the merge of this head into `origin/main` `6da757cb` is identical to `6da757cb`'s file (round 2).

**No pin moves.** `scripts/shape_materialize.py` is not a pinned copy. `COPIED_VERBATIM`, `COPIED_FROM_SHAPE`, `FAMILY_COPIED_VERBATIM` and `FAMILY_COPIED_FROM_SHAPE` do not list it, and `templates/assembly-root/scripts/validate-pins.py` says of it "which no project carries". So no `upstream-changed` row, no digest, no assembly copy and no `tests/test_pinned_copy_text.py` entry changes.

**No other text pins these words.** Searched in `tests/`, `README.md`, `docs/`, `docs/handbook.html`, `AGENTS.md` and `templates/`: no test reads `env_commit`'s docstring (`__doc__`, `getdoc`), and none of them holds "in place of that byte" or "a path that does not exist" for this. The other places that name the `?` are about this function's behaviour or about history, and are still true: `_split_message`'s docstring says `caf\\udce9.md` "was `caf?.md`" (before #900), `_refused_by_policy` already says the message "lists it as `_shown` does (#900)", and `tests/test_adopt_spelled_names.py` quotes the message as it was on `c7a3ff16`. The `?` in `argv_text`, `trailer_line` and the commit-trailer text is about a `--trailer`, not the split message.

No rows and no mutants: the change is a docstring, the code is the same AST, and no test pins the docstring. So there is no row to fail on main and nothing to mutate.

All with `$Q/v110/run9.sh`, `EMAIL=service@opensoft.dev`, `GIT_SSH_COMMAND=false`, `GIT_CONFIG_GLOBAL=/dev/null`, the stub `gh`, a fake token, git 2.43.0 (`/usr/bin/git`).

Two shards, run before the rebase, at the `2a119f281` base with this one-file change in the tree. Together they are every `tests/*.py` that names `shape_materialize` or `env_commit` (86 files, including `tests/test_adopt_spelled_names.py` and `tests/test_adopt_entry_paths.py`, which run `env_commit` and the #900 message) and `tests/test_agents_md_codes.py`:

| run | files | result |
|---|---|---|
| shard A | 43 | 2873 passed, 4 skipped in 1707.51s, rc=0 |
| shard B | 44 | 4611 passed, 31 skipped in 2462.01s, rc=0 |

After the rebase onto `16f9a0e81`, on the committed head `e32b2703d733f6c60c1b2cba2d62d164d93d7b9a` (`dirty=0`): the trio, `tests/test_repo_hygiene.py`, `tests/test_agents_md_codes.py` and `tests/test_cli_reference.py`, **170 passed in 55.67s**, rc=0. The docstring-only change needed no new shards: the shards' counts are runs at the `2a119f281` base, and #1087 did not touch `env_commit`.

Round 2, on the committed head `de89aa81b1533e32f1007f991aeae05cd915b36d` (`dirty=0`): the same trio, **170 passed in 61.77s**, rc=0. The round 2 commit is one docstring sentence, so it needed no new shards either.

git 2.47.1: no run. The change adds no test file of its own, so there is none to run on it.

`git merge-tree --write-tree HEAD <other>` with HEAD at `de89aa81b1533e32f1007f991aeae05cd915b36d`, after a re-fetch of `origin main` with `-c gc.auto=0 -c maintenance.auto=false` (round 2, about 14:03Z on 2026-10-21):

| against | result |
|---|---|
| `origin/main` `6da757cbe9a4eb66d60c34f38dcc2b332d175e8b` | clean, rc 0, tree `15c11b6c959fc4c7540105d5d62e3f961c8e27fd` |
| #1105 `fix/1048-makefile-follow-up-gnu-read-order` `5a4de7f763b5a3c32adce9614a3a539508416eb4`, an open PR that touches `scripts/shape_materialize.py` | clean, rc 0, tree `db05aead67052643942e35a9689d9dd4a0873fda` |
| #1118 `fix/1089-named-link-skip-tests` `c60066cbeec18b8b6c926e254ed4bee6b9a486c2`, the other open PR that touches `scripts/shape_materialize.py` | clean, rc 0, tree `a40df3b63c32c2a3d8f4c40587b094940c8d6567` |

Read with `gh pr list --state open --json number,headRefName,headRefOid,files`: 16 open PRs, three of them (this one, #1105 and #1118) touch `scripts/shape_materialize.py`, the only file this change touches. The other 13 (#1120, #1119, #1117, #1111, #1109, #1107, #1106, #1102, #1101, #1098, #1097, #1091, #1088) touch no file this change touches.

The same command with HEAD at the reviewed head `e32b2703` gives rv1104's trees: `de31d857` against `6da757cb` and `195ffe19` against #1105. The round 1 merge against #1090 is moot: #1090 has landed in `6da757cb`.

Held as a draft until the v1.2.0 tag; it is now on the v1.2.1 milestone.

- **NIT 1, `scripts/shape_materialize.py` lines 2340-2343, a line round 1 did not change.** The paragraph above the reworded one said the adoption's split message "names each entry path as the source spells it". Since #900 that is true of a printable path alone: `_split_message` lists any other through `_shown`, as the escape the next paragraph describes. It now says "as the source spells it (a printable one; see below)". One new commit on top of `e32b2703`, `de89aa81b1533e32f1007f991aeae05cd915b36d`, 3 lines added and 2 removed, docstring only, pushed as a fast-forward (no rebase, the PR is open).
  - Measured: the AST with docstrings blanked is identical to `e32b2703`'s and, merged into `6da757cb`, to `6da757cb`'s. The docstring holds no lone surrogate, still holds the six characters `\udce9`, and its longest line is 78 characters.
  - Searched `tests/`, `README.md`, `docs/`, the handbook mirror, `AGENTS.md`, `templates/` and the scripts for "names each entry path", "as the source spells" and the new words: only `tests/test_adopt_entry_paths.py` line 233 matches, a test's docstring about `execute` handing each path on as the source spells it, which is another sentence and pins nothing here.
- **NIT 2, this body.** It named main as `16f9a0e81` and #1090 as the one open PR that touches the file. #1090 has landed. Main is `6da757cb`, and the merge-trees above are redone against it, against #1105 and against #1118, a draft that opened since and also touches the file.

Review: rv1104-1107, lane 5's independent opus reviewer, which wrote none of #1104: READY at `de89aa81`, H0 of this landing (issuecomment-6318111413), with no finding; NIT 1 and NIT 2 of the round before are fixed. CI (Brett Heap's ruling 27 and his fast-refresh ruling of 2026-10-15T22:48:35Z): H0 de89aa8 passed tests, tests-git-bash and SonarCloud; the head 92285d4 is a clean update-branch merge of main b66d8f0, with tests-git-bash and SonarCloud green; and the local gate passed on its predicted squash tree 6bd2128 over main b66d8f0: the repo_hygiene, agents_md_codes, cli_reference sets, 171 passed. Landed on Brett Heap's word of 2026-10-23T13:10:26Z, relayed to every lane by lane 3 (#473 issuecomment-6365697123): "merge v1.2.1 PRs once reviewed and green".

Closes #1018

Lane: openreposhape-5 (openRepoShape-5)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Nov 8, 2026
…eeps a `GNUmakefile` or a `makefile`, which GNU make reads first, the Makefile collision follow-up no longer says the shape's make targets arrive with the merge, and the pinned file says `make` reads the first of the three and the block reaches it only once that file holds it or includes a Makefile that does (#1048) (#1105)

Two sentences said a kept `Makefile` is what holds the shape's block back. GNU make reads the first of `GNUmakefile`, `makefile` and `Makefile` it finds, and only that one, so where the root also keeps `GNUmakefile` or `makefile`, `make` reads that file, and reaches the merged `Makefile` only where that file includes it.

**Measured** (a real `execute`, then `make -n` in the clone, for each kept name; r1046's NIT-B, repeated; the include rows are RV1105's SHOULD-1):

| the source keeps | file `make` reads | after `cat shape/Makefile >> Makefile`, `make -n bootstrap` |
|---|---|---|
| `Makefile` | `Makefile` | exit 0 |
| `Makefile` + `makefile` | `makefile` | exit 2, "No rule to make target 'bootstrap'" |
| `Makefile` + `GNUmakefile` | `GNUmakefile` | exit 2, same |
| all three | `GNUmakefile` | exit 2, same |
| `Makefile` + a `makefile` or `GNUmakefile` that ends in `include Makefile` | that file | exit 0, runs `python3 scripts/bootstrap.py` (exit 2 before the merge) |
| `makefile` or `GNUmakefile` alone | that file | no merge to make: the shape's `Makefile` is the root's, and `make contracts-dir` has no rule until that file holds the block or includes `Makefile` |

**What changed**
- `scripts/shape_materialize.py`, `collision_follow_up`: takes `reads_first`, the file GNU make reads before the shape's `Makefile`. Where it is set, the Makefile's follow-up says GNU make reads that file, the first of the three it finds, and that the shape's make targets reach `make` there only once that file has them, or includes a Makefile that does; until then `python3 scripts/bootstrap.py` is what the shape's `make bootstrap` runs. Where `scripts/bootstrap.py` went aside too, it says neither command runs the shape's bootstrap until the script is merged, and then `python3 scripts/bootstrap.py` runs it. Where the root keeps only the source's `Makefile`, the sentence is what it was, byte for byte. `GNU_MAKE_READS` joins `SHAPE_MAKE_TARGETS`.
- `adopt-project.py`, `_predict_collisions`: reads the file from the plan's root paths as `_makefile_follow_up` does, so the `CONTRACTS_DIR` line and the collision line of one plan name the same file.
- `templates/assembly-root/AGENTS-shape.md`: says `make contracts-dir` prints the block wherever `make` reads the shape's Makefile, names the three files in the order the code reads them, and says that where a `GNUmakefile` or `makefile` is kept, beside a `Makefile` or instead of one, `make` reads it, and not the shape's Makefile unless that file includes it. It is one verbatim copy: the digest in every project's `contracts/shape-pin.yaml` is rendered from its bytes at adopt and scaffold time, and no digest or `tree_sha256` of it is stored in this repository, so there is no pin to move here; each project re-pins through `update-shape.py` as for any template change.
- `tests/test_pinned_copy_text.py`: `ADOPTED_SENTENCES` holds the new text.
- `tests/test_adopt_collision_make_reads.py` (new, 31 rows): the follow-up per kept set, the two lines of one plan naming one file, no collision line where no `Makefile` is kept, the two name tuples equal, the pinned file naming every file the code reads first, the script-aside tail where `make` reads another file, and, in a real adoption's clone, `make -n` held to both sentences for each kept name and for a kept `GNUmakefile` or `makefile` that ends in `include Makefile`.

**Fix round 1** (RV1105's review, `b28f40de0` on top of `5a4de7f76`)
- **SHOULD-1:** the follow-up said the targets reach `make` "only once `X` holds them, or includes a Makefile that does, and not with that merge alone". The last clause is untrue where `X` already ends in `include Makefile`: the merge alone brings them. The clause is gone, and the docstring paragraph, the `SHAPE_MAKE_TARGETS` comment and the `ADOPTED_SENTENCES` comment now say the merge alone brings the targets only where that file includes `Makefile`.
- **NIT-1 (and RV1121's NIT-3):** the follow-up now uses the words of #1121's NOTE: "has them", and "is what the shape's `make bootstrap` runs". `_next_steps` is not touched.
- **NIT-2:** the script-aside tail, where `reads_first` is set, said "neither `make bootstrap` nor `python3 scripts/bootstrap.py` runs the shape's bootstrap before both merges are made", which reads as if `make bootstrap` runs it after them. It now says neither runs it until the script is merged, and then `python3 scripts/bootstrap.py` runs it. I did not keep "before both merges": where `X` is given the targets itself, as the sentence before invites, `make bootstrap` runs the shape's bootstrap once the script is merged, with no `Makefile` merge at all.
- **NIT-3:** the template's "and never the shape's Makefile" is now ", and not the shape's Makefile unless that file includes it". `ADOPTED_SENTENCES` changes with it. No other copy or regex in `tests/` pins that sentence (`git grep` for "never the shape" and "reads the first of them").
- **NIT-4:** the `collision_follow_up` docstring no longer says `execute` reports through it. It says `plan` predicts the follow-ups and stores them in `follow_ups:`, the split commit carries those, `execute` names its collisions by path only (its `beside` lines and the "WRITTEN BESIDE" list), and `Materialized.follow_ups()` has no caller. The method stays.
- NITs 5 to 8 are left to the lane.

**Rows of this round:** `tests/test_adopt_collision_make_reads.py` and `tests/test_pinned_copy_text.py`, 118 rows. With this round's test files on `5a4de7f76`'s code, 17 fail and 101 pass; on `b28f40de0` all 118 pass. The 17 fail for the right reason:

| rows | why they fail on `5a4de7f76` |
|---|---|
| 4 collision rows (`Makefile+makefile`, `Makefile+GNUmakefile`, all three, `Makefile+GNUmakefile+scripts`) | the line holds "not with that merge alone" |
| 2 new script-tail rows (`GNUmakefile`, `makefile`) | the line holds "before both merges are made" |
| 2 new include rows in a real clone (`GNUmakefile`, `makefile`) | `make -n bootstrap` exits 2 before the merge and 0 after it alone (both asserted first, and passing), then the split message holds "not with that merge alone" |
| 7 clone rows, the pinned-file row and `test_a_scaffolded_root_is_told_its_makefile_exports_none` | the pinned file still says "and never the shape's Makefile" |

**Mutants of this round**, each run against the 118 rows on `b28f40de0`, each file restored and its sha256 checked:

| id | mutant | rows failing | result |
|---|---|---|---|
| MA | ", and not with that merge alone" put back | 6 | killed |
| MB | the script-aside tail put back ("before both merges are made") | 3 | killed |
| MC | the template's "and never the shape's Makefile" put back | 9 | killed |
| MD1 | include rows: `not in message` inverted to `in message` | 2 | killed |
| MD2 | include rows: after the merge, `returncode == 0` inverted | 2 | killed |
| ME | "holds them" put back | 6 | killed |
| MF | "the shape's `bootstrap` target runs" put back | 5 | killed |

**Gate** on `b28f40de0`:

| files | passed | failed | skipped |
|---|---|---|---|
| `test_adopt_collision_make_reads`, `test_pinned_copy_text`, `test_adopt_make_reads_first`, `test_adopt_contracts_dir`, `test_adopt_printed_sentences`, `test_adopt_e2e`, `test_repo_hygiene`, `test_scaffold_leg_path_collisions` | 470 | 0 | 0 |
| `test_shape_doctor`, 8 shards (62 + 56 + 54 + 56 + 60 + 57 + 58 + 45 collected) | 445 | 0 | 3 (PowerShell rows, Windows host only) |
| **total** | **915** | **0** | **3** |

**Merge-tree:** `b28f40de0` against main `4ed7c7bf5` is clean, and so is it against each open PR that touches these files: #1104, #1106, #1118, #1119, #1120, #1121 (`1c5173758`), #1122, #1123 and #1125.

**Round 0** (`5a4de7f76`, main `6da757cbe` merged in): on main's code (plus the constant alone), 12 of the then 27 rows failed and 15 passed; the 30 test files that run the changed code or read the template gave 2109 passed, 0 failed and 4 skipped (3 PowerShell rows in `test_shape_doctor.py`, 1 MedxEHR row in `test_adopt_plan.py` that needs a clone `SHAPE_MEDXEHR_CLONE` names); nine mutants each failed 1 to 11 of the then 114 rows.

**Not here**
- The third box of #1048, the `_next_steps` NOTE in `adopt-project.py`, is covered by the change #1047 makes to it: see #1047 and #1121.
- `templates/assembly-root/AGENTS-shape.md` line 19 and the five-commands table say `make bootstrap` without this caveat. They say nothing about a kept file, so they are outside #1048's two sentences.

Closes #1048

Lane: openreposhape-4 (openRepoShape-4)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

adopt-project.py: plan's source line prints the source's branch raw, so a bidi control in a branch name reaches the terminal

1 participant