Skip to content

repo_shape.py: a quote opens a quoted scalar only where one begins and must close there, so k: [it's, a] reads two items, k: it's fine # note drops its comment, and an unclosed "abc or a mid-key [ is refused, not misread (#249, #250) - #284

Merged
brettheap merged 6 commits into
mainfrom
fix/yaml-quote-boundaries
Oct 12, 2026

Conversation

@brettheap

@brettheap brettheap commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Lane: openreposhape-5 (openRepoShape-5)
Claim: #249 (comment)
Claim: #250 (comment)

What

The reader (scripts/repo_shape.py). It now has one rule for where a quoted scalar starts and one for where it ends. That replaces three separate toggles and a first-and-last-character test.

The template renderer (scripts/shape_materialize.py render). In a .yaml/.yml template, a value that sits between double quotes ("{{ELECTED_BY}}") is now spelled by double_quoted. That is the writer this reader takes back unchanged. For any value with no quote, no backslash and no character str.isprintable refuses, double_quoted returns the value itself between quotes. So commits, digests, dates and ordinary names (O'Brien & Co: #1 [draft] {x} é 日本 😀 included) render byte for byte as before in all 19 templated files, and README/AGENTS templates keep raw values. A name that holds a character str.isprintable refuses (a no-break space, U+3000, the ideographic space common in Japanese names, a zero-width joiner in an emoji sequence, or a soft hyphen) now renders as an escape in project.yaml, family.yaml and single-repository.yaml (山田\u3000太郎), and reads back equal. It is the same rule the adoption plan writer already follows.

Why

The two issues' tables, read with each tree's own reader (issue_tables.py). Every head result matches PyYAML 6.0.3:

document 150c41d head
k: [it's, a] ["it's, a"] ["it's", "a"]
k: {a: it's, b: 1} {"a": "it's, b: 1"} {"a": "it's", "b": 1}
k: it's fine # note "it's fine # note" "it's fine"
k: [it's, a] # note refused: not a flow collection ["it's", "a"]
it's: 1 refused: expected key: value {"it's": 1}
k: [a"b, c] ['a"b, c'] ['a"b', 'c']
k: [a"b\", c] ['a"b\\", c'] ['a"b\\"', 'c']
created_by: "Brett Heap '"Brett Heap' refused: no closing quote
k:\n - "a\" ['a\\'] refused: no closing quote
k: "a" b "c" 'a" b "c' refused: text after its closing quote
k: 'a' b 'c' "a' b 'c" refused: text after its closing quote
k: ["a\", b] ['"a\\", b'] refused: no closing quote

The other five #250 rows (k: "abc, k: "a\", b, k: "a"b, k: 'abc, k: ["a, b]) are refused on head the same way.

Why _split_key refuses a bracket later in a key. _split_key has always returned None for any [ or { ("a flow collection is not a key in this subset"). On 150c41d a quote toggle hid the bracket whenever the key held a quote pair, so note "[1]": x was a mapping entry by accident. Reading quotes only where a scalar begins lets that rule see the bracket. As a mapping entry that would be a refusal, but _parse_sequence takes None to mean "a scalar item", so - note "[1]": x would have read as the string note "[1]": x where YAML has a mapping: a silent misread of valid YAML. Review found it (20 of the 300 documents that regressed on one fuzz seed read that way), so the rule is now in two parts. A [ or { that OPENS the text still returns None, so - [a, b] and - {a: b} stay flow items. One LATER in the key, before its : , raises a key holding [or{ is not supported: <key>.

document 150c41d head PyYAML
- note "[1]": x [{'note "[1]"': 'x'}] refused, names the key [{'note "[1]"': 'x'}]
note "[1]": x {'note "[1]"': 'x'} refused, names the key {'note "[1]"': 'x'}
- a[b]: c ['a[b]: c'], a string refused, names the key [{'a[b]': 'c'}]
a[b]: c refused: expected key: value refused, names the key {'a[b]': 'c'}
k:\n - a{b}: c {'k': ['a{b}: c']}, a string refused, names the key {'k': [{'a{b}': 'c'}]}
k: {a[b]: c} refused: flow mapping entry without a key refused, names the key refused
- [a, b], - {a: b} flow items flow items flow items

So the bracket-in-key class is now refusals, not silent misreads, and the refusal covers 150c41d's own silent misread of - a[b]: c too. The class is narrow in practice: no writer in this tree can emit such a key (the adoption plan and manifest writers write literal keys and put free text in values), so only a hand edit of a fixed schema reaches it. Reading a[b]: c as YAML does (the key a[b]) would mean telling a block key from a flow-entry key, which is _split_key_flow's business (#252's function); accepting a mid-key bracket in _split_key alone newly reads 6,521 more invalid documents on one seed. That stays a follow-up (see Left out).

Why the renderer had to change as well. render pasted free text raw between a template's double quotes. So scaffold-project.py --elected-by 'Robert "Bob" Smith' wrote elected_by: "Robert "Bob" Smith", which is not YAML. The old reader read it correctly by accident. With the reader fix alone, the scaffold would create and push all three repositories and then write a manifest its own validator refuses. A mutant of this branch without the renderer change shows exactly that: three bare remotes exist, then REFUSED yaml-unparsable: …/project.yaml: quoted scalar with text after its closing quote: '"Atlas "the" Project"'. On 150c41d some values were already misread rather than refused: --reference 'docs\notes # 1' reads back as docs, a line feed and otes # 1. I rendered every quoting template with 11 free-text values through each tree's own render and reader (render_real.py). On 150c41d 64 of 88 read back exactly. On head all 88 do. The other free-text placeholders that reach render are --name, --reference, family.py --created-by/--name, and the adoption plan's elected_by/reference. All three writers (scaffold, adopt execute, family.py init) render through this one function, and none pre-escapes, so nothing is escaped twice. Nothing in the tree digests or re-renders a templated file: shape-pin files: rows digest only the verbatim and from-shape copies.

Test expectations changed on purpose

Evidence

Composition

This branch carries origin/main at 60588d5, by four merges (eb6df75, c3f4987, 5751460, 60588d5). Only the first conflicted, in one hunk of a comment: both this branch and #252 (landed as 4b2add8) add a paragraph to the reader's header comment right after "confident tone.". Both paragraphs are kept, this branch's first, with a # line between them. Every code hunk merged by itself, including #252's _split_key_flow, which calls this branch's _split_key, and shape_materialize.py. The merges of #248 (c3f4987, a block scalar's record as a pair), of #251 (23e16ae) and of the two others were clean. repo_shape.py is the same file after the last three merges as after the second. On the merged reader, test_yaml_flow_mappings, test_yaml_block_scalar_keys and test_scaffold_referent_chain_links pass beside this branch's files (see Evidence).

git merge-tree --write-tree of the head against origin/main: clean. Against another lane's open PR:

Left out

  • Reading a [ or { inside a plain block key as YAML does. a[b]: c and - a[b]: c are refused by name here; YAML reads the key a[b]. Reading them needs block keys and flow-entry keys told apart (_split_key_flow's business), and must refuse only a block key that BEGINS with [ or {. A first try that accepted a mid-key bracket in _split_key alone fixed all 300 regressions on one seed, but newly read 6,521 more invalid documents (keys beginning ] or }, and {a[b]: c} entries). Candidate follow-up.
  • A plain flow item followed by a nested flow, and unbalanced brackets. k: [x [a, b]] reads as the one scalar x [a, b], and k: {a:}#}: b} reads {'a:}#}': 'b'}, because _flow checks only a body's ends. PyYAML refuses both. Candidate follow-up.
  • Implicit single-pair mappings in a flow sequence. [k: v] still reads as the string 'k: v', as scripts/repo_shape.py _split_flow ignores a backslash inside a double-quoted flow item, so pins: ["x\",y"] reads back as two items x\ and y" where the same text as a block sequence item reads as the one item x",y #213's review (F4) found.
  • Not supported, by choice. A quoted scalar spanning lines is outside the subset, and 150c41d did not always refuse it. When the continuation line was indented deeper, it was refused as unexpected indentation. When the continuation sat at the parent's column, it was MISREAD: k: "a then b: c" read as {'k': '"a', 'b': 'c"'}, and - 'a then - b' as ["'a", "b'"]. Every such form is now refused at its first line, naming the unclosed scalar. k: "a"#c is refused too, as explained above.
  • Not run: the full suite, and the macOS and Windows CI legs. The new tests write their files as bytes, pass no text to git's stdin, and pass the scaffold its arguments as a list.

Closes #249
Closes #250

🤖 Generated with Claude Code

brettheap and others added 6 commits October 11, 2026 22:19
…ns, and the scalar must end at its closing quote, so `k: [it's, a]` reads the two items its block twin reads, `k: it's fine # note` drops its comment, and an unclosed `"abc`, `- "a\"` or `"a" b "c"` is refused instead of misread (#249, #250)

#249: `_strip_comment`, `_split_key` and `_split_flow` each toggled a quote
at ANY `"` or `'`, so a quote in the middle of a plain scalar swallowed the
`,`, `#` or `: ` after it. They now read structure through one generator,
`_unquoted`: a quote opens a quoted scalar only at the start of the text or
after `- `, `: `, `[`, `{`, or a `,` inside a flow; on a block line a `,` or
`[` in the middle of a plain scalar is not structure (`k: a, 'b # c'` is the
plain `a, 'b`). `_quote_end` is the one definition of where the scalar
ends: a `"` no backslash escapes, a `'` that is not doubled.

#250: `_scalar` took any text starting and ending with a quote for one
quoted scalar. `_quoted_scalar` now requires the whole text to be one, and
raises `quoted scalar without a closing quote` or `quoted scalar with text
after its closing quote`; `_unquote_key` sends every key that begins with
a quote through it. Every #249 row reads as PyYAML 6.0.3 reads it, and
every #250 row is refused, as PyYAML refuses it.

shape_materialize.py: `render` pasted free text raw between a YAML
template's double quotes, so `scaffold-project.py --elected-by 'Robert
"Bob" Smith'` wrote `"Robert "Bob" Smith"`: read by accident before, and
refused by the stricter reader after three repositories exist. A value
between a `.yaml`/`.yml` template's double quotes is now spelled by
`double_quoted`; for any value with no quote, backslash or non-printable
character that is the value itself, so no template changes by a byte.
On 150c41d `--reference 'docs\notes # 1'` already read back as `docs`, a
line feed and `otes # 1`.

Tests: tests/test_yaml_quote_boundaries.py (reader) and
tests/test_template_quoted_values.py (renderer, plus one real scaffold
into local bare remotes); 65 of their 138 cases fail on 150c41d. In
tests/test_yaml_flow_quotes.py the four N5 rows that pinned a dangling
backslash's plain-text read are now refusals together with their block
twins, the change that table said to expect. Each changed function
reverted to 150c41d, and each rule of the new helpers mutated, fails at
least one new test.

Reach: the 14 tracked YAML files read identically before and after. The
flow-against-block fuzz (seed 2130213, 100,000 docs) and the #199
code-point fuzz (60,000) show 0 differences.

Lane: openreposhape-5 (openRepoShape-5)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One hunk conflicted, in the header comment of scripts/repo_shape.py: this
branch's paragraph "A QUOTE OPENS A QUOTED SCALAR ONLY WHERE A SCALAR BEGINS"
and main's "A FLOW MAPPING READS AS ITS BLOCK TWIN (#252)" (landed as
4b2add8) were both inserted right after "confident tone.". Both paragraphs
are kept, this branch's first, with a `#` line between them. Every code hunk
merged on its own, including #252's `_split_key_flow`, which calls this
branch's `_split_key`, and shape_materialize.py.

Lane: openreposhape-5 (openRepoShape-5)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…`, is refused by name in `_split_key`, so `- note "[1]": x` and `- a[b]: c` are no longer read as the strings they are not, and a `:` with no space after it is pinned as opening no quoted scalar

Review of 21fc213, SHOULD-1. Reading quotes only where a scalar begins let
`_split_key`'s old rule, "a `[` or `{` means this is not a key", see a bracket
that a quote toggle had hidden. As a mapping entry that is a refusal, but as
a sequence item `_parse_sequence` takes `None` for "a scalar item", so
`- note "[1]": x` read as the string `note "[1]": x` where PyYAML reads the
mapping `{'note "[1]"': 'x'}`. On the reviewer's seed 7249250, 300 valid
documents regressed and 20 of them were silent misreads; every one has a `[`
or `{` in a key.

`_split_key` now returns None for a `[` or `{` that OPENS the text, so
`- [a, b]` and `- {a: b}` stay flow items, and raises YamlError ("a key
holding `[` or `{` is not supported: <key>") for one LATER in the key when a
`: ` follows. The change stays inside `_split_key`, so C's `_split_key_flow`,
which calls it, is untouched. On that seed the regressions are all refusals
(300, 0 misreads), the same 7,039 documents are fixed and the same 3,173
invalid ones are read as on 21fc213, and 1,550 more of 150c41d's own silent
misreads, `- a[b]: c` among them, become refusals. Reading `a[b]: c` as YAML
does needs block keys and flow-entry keys told apart; that is a follow-up.

Tests (tests/test_yaml_quote_boundaries.py, 54 cases to 73):
- `- note "[1]": x`, `note "[1]": x`, `- a[b]: c`, `a[b]: c`, a nested
  `- a{b}: c`, `- - a[b]: c`, `- x {y}: z` and the flow entry `{a[b]: c}` are
  each refused by name, the message quotes the key, and a file read through
  `load_yaml` is `REFUSED yaml-unparsable` naming it. All 10 fail on 21fc213.
- `- [a, b]`, `- {a: b}`, flow items under a key, `- a: [b]`, `- a: {b: [c]}`,
  `k: a[b]` and `- see [1]` still read (7 guards, which pass on both).
- Nit 1: `k: [a:'b, c']` reads as the two items `a:'b` and `c'`, and
  `k: a:'b # c'` as `a:'b`, both PyYAML's reads. Mutant X1 (any `:` opens a
  quoted scalar) survived the 148 old tests and now fails exactly these two.
- Mutants of the new `_split_key` (no raise, start test inverted, a bracket
  always None, the flag never set) each fail at least 10 of the new rows.

Lane: openreposhape-5 (openRepoShape-5)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main moved past the first merge with #248 (c3f4987): a block scalar's record
carries its key and body as a pair, and `_BLOCK_SENTINEL` is gone. No hunk
conflicted. #248 edits `_Rec`, `_tokenise`, `_parse_sequence` and
`_parse_mapping`, away from `_unquoted`, `_split_key` and `_scalar`; the
`_split_key` calls in `_tokenise` and `_parse_sequence` now meet this
branch's refusal of a `[` or `{` later in a key.

Lane: openreposhape-5 (openRepoShape-5)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main moved past the second merge with the adopt-project.py spelling change
(5751460, a FINDING or refusal spells a path as `plan` does). No hunk
conflicted. It edits adopt-project.py, scripts/shape_materialize.py away from
`render`, and tests; nothing in the reader.

Lane: openreposhape-5 (openRepoShape-5)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main moved past the third merge with #251 (23e16ae, a `--referent-chain` link
is refused unless it is a neutral product's name) and a family.py change
(60588d5, a case-variant of a pinned member). No hunk conflicted. They edit
scaffold-project.py, scripts/family.py, a family-root template, docs/cli.md
and tests; nothing in the reader or in `render`.

Lane: openreposhape-5 (openRepoShape-5)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@brettheap

Copy link
Copy Markdown
Contributor Author

(lane openRepoShape-5, 2026-10-11T23:46Z) @codex review

@sourcery-ai

sourcery-ai Bot commented Oct 11, 2026

Copy link
Copy Markdown

Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 1 day and 13 hours by commenting @sourcery-ai review. Upgrade to get a review now.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@sourcery-ai

sourcery-ai Bot commented Oct 11, 2026

Copy link
Copy Markdown

Reviewer's Guide

The PR replaces inconsistent quote toggles with shared YAML scalar-boundary parsing, explicitly refuses malformed or unsupported constructs instead of silently misreading them, and updates YAML template rendering to escape free-text values so generated files round-trip correctly.

Sequence diagram for safe YAML template rendering

sequenceDiagram
    participant Caller
    participant Render as render
    participant Quoted as double_quoted
    participant File as YAML file
    participant Reader as load_yaml
    Caller->>Render: render(text, values, source)
    alt source is .yaml or .yml
        Render->>Quoted: double_quoted(value)
        Quoted-->>Render: escaped YAML scalar
    else other template
        Render->>Render: replace placeholder with raw value
    end
    Render->>File: write rendered template
    File->>Reader: load_yaml
    Reader-->>Caller: original value or REFUSED yaml-unparsable
Loading

Flow diagram for YAML scalar boundary parsing

flowchart TD
    Input["YAML text"] --> Unquoted["_unquoted"]
    Unquoted --> Start{"Quote at scalar start?"}
    Start -->|No| Plain["Treat quote as plain-scalar text"]
    Start -->|Yes| End["_quote_end"]
    End -->|Closed| Structure["Expose comments, keys, and flow delimiters"]
    End -->|Unclosed| Refuse["YamlError: quoted scalar without a closing quote"]
    Structure --> Consumers["_strip_comment / _split_key / _split_flow"]
    Consumers --> Scalar["_scalar → _quoted_scalar"]
    Scalar -->|Trailing text| RefuseTrailing["YamlError: text after closing quote"]
Loading

Flow diagram for refusing unsupported key brackets

flowchart TD
    Text["Mapping text"] --> Split["_split_key"]
    Split --> Scan["Scan structure via _unquoted"]
    Scan --> Opening{"[` or `{` at text start?"}
    Opening -->|Yes| FlowItem["Return None: flow item"]
    Opening -->|No| MidKey["Bracket occurs before key separator"]
    MidKey --> Refuse["YamlError: key holding bracket is not supported"]
    Scan --> Separator["Find top-level `: `"]
    Separator --> Pair["Return key/value pair"]
Loading

File-Level Changes

Change Details Files
Unified quoted-scalar boundary detection across YAML structure parsing.
  • Added _quote_end to recognize escaped double quotes and doubled single quotes.
  • Added _unquoted as the shared source of structural characters for comments, keys, and flow splitting.
  • Treats quotes in plain scalars as ordinary characters while opening quotes only at scalar boundaries.
  • Refuses unclosed quoted scalars and text after a quoted scalar's closing quote.
  • Routes quoted keys through scalar validation and refuses unsupported mid-key brackets/braces.
scripts/repo_shape.py
Made YAML template rendering safely round-trip arbitrary quoted free-text values.
  • Uses double_quoted for values substituted into double-quoted YAML placeholders.
  • Preserves raw substitution for non-YAML templates and ordinary values that need no escaping.
  • Ensures generated manifests remain readable after quotes, backslashes, comments, and non-printable characters are supplied.
scripts/shape_materialize.py
Added comprehensive regression coverage for quote boundaries, refusals, and renderer round trips.
  • Added cases for flow/block consistency, comments, escaped and doubled quotes, malformed scalars, and bracket-in-key refusals.
  • Added template coverage across YAML and Markdown templates, including an end-to-end local scaffold.
  • Updated dangling-backslash flow tests to expect explicit YAML refusals.
tests/test_yaml_quote_boundaries.py
tests/test_template_quoted_values.py
tests/test_yaml_flow_quotes.py

Assessment against linked issues

Issue Objective Addressed Explanation
#249 Make quote characters open quoted scalars only at the beginning of a scalar, while treating quotes in the middle of plain scalars as ordinary text consistently in comments, keys, flow mappings, and flow sequences. ✅
#249 Ensure the affected inputs are parsed like their block-form equivalents, including separating k: [it's, a], preserving keys such as it's, removing trailing comments from values such as it's fine # note, and correctly splitting a"b\ flow items. ✅
#249 Reject malformed quoted scalars rather than silently misreading them, and preserve reader/writer round trips for quoted template values. ✅
#250 Make the YAML reader recognize quoted scalars based on an actual valid closing quote, including escaped double quotes and doubled single quotes, and refuse unterminated scalars or text following the closing quote with YamlError. ✅
#250 Apply the quoted-scalar boundary rules consistently in block values, flow collections, keys, and comment/key splitting so malformed inputs such as escaped closing quotes, trailing text, and quoted scalars followed by content are refused rather than silently misread. ✅
#250 Preserve valid YAML behavior and writer/reader round trips while adding regression coverage for the issue's examples, including flow-versus-block consistency and the tracked YAML/template paths. ✅

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sonarqubecloud

Copy link
Copy Markdown

@brettheap
brettheap merged commit b30595c into main Oct 12, 2026
5 checks passed
@brettheap
brettheap deleted the fix/yaml-quote-boundaries branch October 12, 2026 00:09
brettheap added a commit that referenced this pull request Oct 12, 2026
…ach line under its template's name, as the scaffold does, so on a merge with #284 a quoted leg-path line is spelled by `double_quoted` in the test as in the project (#225)

`shape_materialize.render` is told the template it renders (`rel`, such as
`project.yaml`). At this head it uses that name only in a refusal. #284
(fix/yaml-quote-boundaries, b72ab0f) also uses it: a value between a `.yaml`
or `.yml` template's double quotes is spelled by `double_quoted`. The F1
test passed the name "test", so on that merge it would have rendered a
quoted line raw while the scaffold spells it, and the line it read back
would not have been the line the project holds.

`template_lines_holding_a_leg_path` now yields `(template, line number,
line)`, and `read_back` passes the template to `render`. No verdict changes
for any value the tests ask: none holds a quote, a backslash or a character
that is not printable, so both spellings are the same bytes.

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 12, 2026
… conflict with #284

#284 (fix/yaml-quote-boundaries) landed on main as b30595c and added
`double_quoted` to the `from repo_shape import (...)` list in
scripts/shape_materialize.py, the same list this branch adds `parse_yaml` to,
so the pull request could no longer merge. The one conflicting hunk is
resolved as the union of the two sides: `double_quoted, file_sha256,
parse_yaml`. Nothing else conflicted.

The F1 test renders template lines with `render` under the template's name,
so with #284's `render` on this branch a quoted leg-path line is spelled by
`double_quoted` in the test as in the project.

Lane: openreposhape-2 (openRepoShape-2)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 12, 2026
…, #205, #215, #241 and #249/#250 merge with no conflict, and `_entry_findings` still hands `_leg_findings` the tree and ends with the root-symlink findings

origin/main moved 60588d5 -> b30595c while the targeted runs of 8f624aa
went: 018ce0d (#207 via #222), 67a398a (#272 via #274), d94d155 (#205
via #232), c2ff2b0 (#215 via #240), c3a9645 (#241 via #256) and b30595c
(#249 and #250 via #284). It is merged again because c2ff2b0 changes
adopt-project.py and b30595c changes scripts/shape_materialize.py's
`render`.

Conflicts: none. `git merge-tree --write-tree` gave tree be093a5, and
the merge stages the same tree.

Checked on that tree:
- `_entry_findings` still calls `_leg_findings(plan, tree)` and ends
  with `_root_symlink_findings(plan, source, tree)`.
- No top-level name is bound twice in adopt-project.py,
  scaffold-project.py, scripts/repo_shape.py or
  scripts/shape_materialize.py, none is loaded but bound nowhere (AST),
  and main changed the signature of no function this branch calls.
- Both sides are intact by +/- line sets: each side's change against the
  merge base 60588d5 reappears, line for line, against the other side.
- c2ff2b0 adds helpers to tests/conftest.py (`planned`, `GITLINK`,
  `findings` and others) and no fixture. tests/test_adopt_root_symlinks.py
  imports its conftest names one by one and none of these, so its own
  `planned` and `GITLINK` are unaffected.
- b30595c's `render` spells a double-quoted YAML value with
  `double_quoted`; this branch's walker reads `assembly_root_paths()` and
  never renders.

Lane: openreposhape-1 (openRepoShape-1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 12, 2026
… via #236) into fix/adopt-record-submodule-placement

Brings in b30595c (#249, #250), be2790e (#204) and 6d8d88b (#203) on top
of c3a9645. No rebase. b30595c changes scripts/repo_shape.py and
scripts/shape_materialize.py, as this branch does. be2790e changes
update-shape.py and 6d8d88b changes scripts/family.py, neither of which
this branch touches.

One conflict, in scripts/shape_materialize.py's `from repo_shape import`
block. b30595c imports `double_quoted` for `render`, and this branch
imports `PATH_CLIMBS`, `PATH_ITSELF`, `PATH_NOT_CANONICAL` and
`path_spelling_fault` for `_spelling_reason` and `LEG_SEGMENT_REASONS`.
Kept all of them, in the block's order.

Merged without a conflict, and checked:

- scripts/repo_shape.py: b30595c reads a quote as opening a quoted scalar
  only where a scalar begins, and requires the whole text to be that one
  scalar (`_unquoted`, `_quote_end`, `_quoted_scalar`; `_scalar` hands
  them any text that begins with a quote). The record writes every path
  and reason as one whole scalar through `double_quoted`. `double_quoted`,
  `_escaped`, `_ESCAPE_OF`, `_unescape`, `parse_yaml` and
  `path_spelling_fault` are unchanged by the merge. Through the merged
  reader, `submodule_record` then `parse_yaml`, path and resolution, there
  are 0 mismatches in each set:
  - 4,525 names, plus 35 more;
  - 240 aimed at #252 and #248;
  - 195 aimed at #249 and #250: `it's`, `a"b`, `"abc`, `a" b "c`,
    `a' b 'c`, `"a\"`, `a\`, `[it's, a]`, `note "[1]"`, `"Brett Heap`,
    `O'Brien & Co: #1 [draft] {x}`, and more.
- scripts/shape_materialize.py: b30595c spells a YAML template's
  `"{{KEY}}"` value with `double_quoted` in `render`. The record is
  appended after `render` has run (`text += append.get(rel, "")` in
  `_materialize`), so no part of it passes through `render`. This branch's
  `_spelling_reason`, `LEG_SEGMENT_REASONS` and `leg_path_problem` are
  unchanged.
- No top-level name is bound twice in adopt-project.py,
  scaffold-project.py, scripts/repo_shape.py, scripts/shape_materialize.py,
  shape-doctor.py, validate-manifest.py, update-shape.py or
  scripts/family.py. Each merged module's names are the base's plus what
  either side added. The only thing both sides changed is the import block
  above.
- The branch's diff against main is still #189 alone: 12 files, +1670
  -157, file by file the same as against c3a9645.

Lane: openreposhape-1 (openRepoShape-1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 12, 2026
Main brings three commits since d94d155:

- c2ff2b0 (#215 via #240): adopt-project.py refuses a `.gitmodules`
  value that git's submodule reader dies on. It adds helpers to
  tests/conftest.py, adds tests/test_adopt_gitmodules_values.py, and
  moves helpers out of tests/test_adopt_history_gitlinks.py;
- c3a9645 (#241 via #256): scaffold-project.py's `--pin` collisions;
- b30595c (#249, #250 via #284): repo_shape.py's quoted scalars, and
  shape_materialize.py's `render()` spells a value between a YAML
  template's double quotes.

One conflict, in tests/conftest.py. Both sides append after `resolve()`:
main its `.gitmodules` helpers, this branch `spelled()`. Both are kept,
main's block first. Against main, the merged file differs by exactly
this branch's own diff. Against this branch, it differs by main's own
diff, but for where git places two added blank lines.

adopt-project.py merged without a conflict, and each top-level
definition comes whole from one side. c2ff2b0 changes `_registrations`,
`_where_git_fails` and `Source.registered_submodules`, and adds
`_refused_value`, `_who_refuses_a_boolean` and their constants. This
branch changes none of them, and its own `Source.__init__` and
`Source.open` are unchanged. The new refusal comes through
`_entry_findings`. `cmd_execute` calls that in `_refuse_what_check_finds`,
before `_refuse_an_occupied_work_dir` and `_create_leg_remotes`, so the
refusal is raised before any leg exists and no notice is due. Main's
`refused_before_any_leg` asserts that the remotes directory is never
made.

No top-level name is bound twice in adopt-project.py,
scaffold-project.py, scripts/repo_shape.py, scripts/shape_materialize.py
or tests/conftest.py. None of them reads a global it does not bind.

Lane: openreposhape-1 (openRepoShape-1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 12, 2026
Brings in #215 (c2ff2b0: adopt-project.py finds a `.gitmodules` value
git's submodule reader refuses), #241 (c3a9645: scaffold-project.py
refuses a `--pin` whose file the shape writes itself), #284 (b30595c:
repo_shape.py reads a quote as opening a quoted scalar only where one
begins, and the materializer's `render` spells a value between a YAML
template's double quotes with `double_quoted`) and #204 (be2790e:
update-shape.py names the step that fails after `apply --branch`). One
merge, no rebase.

No textual conflict: this commit's tree is `git merge-tree --write-tree`
of its two parents (405a63c), and `_leg_findings(plan, tree)` keeps
`tree`. No top-level name is bound twice in adopt-project.py,
scaffold-project.py, scripts/repo_shape.py, scripts/shape_materialize.py
or shape-doctor.py, imports included; #215's eleven names, #284's three
reader helpers and `double_quoted` in the materializer are none of
this branch's.

One SEMANTIC conflict, fixed in the commit after this one so that this
tree stays the mechanical merge. `manifest_misreads` asks the loader
about the line `execute` writes, and builds that line in `_written` by
putting the value in its placeholder's place as it stands. #284's
`render` no longer does that between a YAML template's double quotes:
`elected_by: "{{ELECTED_BY}}"` and `reference: "{{REFERENCE}}"` get
`double_quoted(value)`, which the loader reads back as typed. So at
this tree the gate refuses `--elected-by 'CORP\tom'`, saying `execute`
writes `elected_by: "CORP\tom"`, where it writes `"CORP\\tom"`. And
it newly refuses `Robert "Bob" Smith` and `trail\`, which the previous
head accepted, because #284's reader refuses the raw line the gate
imagines.

Lane: openreposhape-1 (openRepoShape-1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 12, 2026
…` as the materializer's `render` writes them since #284, the value spelled by `double_quoted` between the manifest's quotes, so `--elected-by 'CORP\tom'` and a name holding a `"` are no longer refused for a line `execute` does not write (#181)

The merge of main be2790e brought #284, whose `render` spells a value
between a YAML template's double quotes with `double_quoted`, the writer
`repo_shape`'s reader takes back unchanged. `_written` still built that
line with the value pasted in raw, so the gate asked about a line
`execute` no longer writes. It refused `CORP\tom`, `docs\new.md` and
`docs\x41.md` with a sentence about that line. It also newly refused
`Robert "Bob" Smith` and a name ending in `\`, which the previous head
accepted, because #284's reader refuses the raw line.

- `_written` takes `render`'s two steps for one placeholder, in order:
  `"{{KEY}}"` becomes `double_quoted(value)`, then any other `{{KEY}}`
  the value as it is. Every templated file it reads is `.yaml`, which is
  what makes `render` take the first step. Since #284, no value between
  the manifest's quotes is misread; the bare placeholders are asked as
  before.
- `FLAG_DEFAULTS` no longer names the elector `plan` takes from
  `git config user.name`: the manifest writes it between quotes, so no
  elector can be misread and the entry could not be reached.
- The section comment says what the quoted line now reads back as.

Tests (tests/test_adopt_plan_yaml_quoting.py), each where #284 changed
what is true:
- `REFUSED` loses its two rows `project.yaml` writes between quotes
  (`--elected-by 'CORP\tom'`, `--reference 'docs\new.md'`): 16 rows, so
  16 refused at `plan` and 18 at `check` and `execute`.
- `BETWEEN_THE_QUOTES`, new: the values the quotes misread before #284,
  `"`, a last `\` and a carriage return among them, are each asked as
  `render` writes them (`double_quoted`) and passed, for `elected_by` and
  `reference` (20 cases).
- The `git config user.name` elector `CORP\tom` is taken by `plan`,
  written `"CORP\\tom"` into the plan, and passed by `check`.
- `UNESCAPED`, which pins how a sentence shows a value the loader reads
  as OTHER TEXT (review181's N1), moves to the bare `tracking_branch:`
  placeholder with the value's own double quotes (a hand-edited plan's
  `'"CORP\tom"'`), where the loader still resolves the escape. It gains
  `a #b`, read as `a`. Its six escapes, the tab, the line break and the
  carriage return a sentence names among them, are the same.
- The drift guard that renders the files whole with `render` and compares
  every candidate with the gate covers `reference` too.

Revert lines, each measured alone on this commit's bytes, running
test_adopt_plan_yaml_quoting, test_adopt_plan_escapes,
test_adopt_leg_paths and test_template_quoted_values (900 tests):
- `_written` pasting a quoted placeholder's value raw again: 23 fail
  (the 20 `BETWEEN_THE_QUOTES` cases, the `git config` elector, and the
  drift guard for `elected_by` and `reference`).
- `FLAG_DEFAULTS` naming the elector again: 0 fail. No elector reaches a
  misread sentence since #284, which is why the entry goes.

Lane: openreposhape-1 (openRepoShape-1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 12, 2026
…#241, #249/#250, #204, #203 and #238 merge with no conflict, and a `.gitmodules` value git refuses is answered before #190's relative-url check, which reads no url from a file git cannot read

Main since d94d155, the main GitHub's update-branch merged at 8ce5614:
c2ff2b0 (#215 via #240), c3a9645 (#241 via #256), b30595c (#249/#250 via
#284), be2790e (#204 via #227), 6d8d88b (#203 via #236) and ede15ce (#238
via #287).

No conflict hunk. Three files were changed on both sides; `git merge`
auto-merged each, and every hunk of each side lies apart from the other's.
Each side's +/- lines are identical before and after the merge in all
three (`git diff -U0` d94d155..8ce5614 against ede15ce..merge, and
d94d155..ede15ce against 8ce5614..merge). Line numbers are the merged
file's.

adopt-project.py
- main, c2ff2b0 (#215): 384-506, `SUBMODULE_TYPED_KEY_RE` to `_refused_value`
  and `_who_refuses_a_boolean`; 510-515 and 522-524, `_registrations`'
  docstring and its return of `_refused_value`'s words; 791-797,
  `Source.registered_submodules`' docstring; 2181-2185, `_where_git_fails`'
  docstring.
- #190: 36-38 and 51-53, the module docstring; 102-103, the `repo_shape`
  import list; 334-339, `Registrations.url`; 2252-2501, `RELATIVE_URL` to
  `_relative_url_findings`; 2503-2548, `_submodule_findings` and
  `_entry_findings` with `local_remote_dir` and the NOTEs; 2608 and 2630,
  `cmd_check`; 3070-3092, `_repository_urls` and `_remote_urls`;
  3111-3216, `_refuse_what_check_finds` and `_as_resolved_here`; 3443,
  `cmd_execute`.

scripts/repo_shape.py
- main, b30595c (#249/#250): 173-535, the YAML reader's quote boundaries,
  from the comment after `YamlError` to `_split_flow`.
- #190: 1975-1985, `REMOTE_PORT_RE` and `REMOTE_HOST_ALIASES`; 2099-2116,
  `remote_key`'s docstring and body.

tests/test_shape_doctor.py
- #190: 2662-2693, `test_a_port_and_githubs_ssh_host_are_not_part_of_a_remotes_key`.
- main, ede15ce (#238): 5209-5808, the validator-PYTHONPATH tests after
  `test_a_drifted_roots_next_command_runs_when_it_is_pasted`.

Every other file is one side's blob: main's for the 17 main changed, and
8ce5614's for tests/test_adopt_relative_submodule_url.py and
tests/test_adopt_submodules.py. The merge's diff against main is #190's
five files, with the stat it had against d94d155 (1097+, 41-). No
function or class is bound twice at the top level of adopt-project.py,
scaffold-project.py, scripts/repo_shape.py, scripts/shape_materialize.py,
shape-doctor.py, scripts/family.py or the three test files. The helpers
c2ff2b0 moved into tests/conftest.py (`planned`, `findings`, ...) are
plain functions, and tests/test_adopt_relative_submodule_url.py imports
seven named helpers from conftest and defines its own `planned`.

HOW #215 AND #190 MEET. `_submodule_findings` lists
`submodule_plan_problems` first, which holds #215's
`plan-gitmodules-unreadable`, then `_relative_url_findings`. Where git's
submodule reader refuses a value of the source's `.gitmodules`,
`registered_submodules` answers with git's words, a string, and
`_kept_relative_urls` reads no url from a string. So a source with both
problems gets #215's finding alone, and no NOTE either. #190's finding is
not lost: once the value is fixed, `check` reports it, and `execute`
refuses it through the same list. #190 adds lines only after #215's, so it
can hide nothing of #215's. A relative url kept in a leg is always beside a
gitlink kept in a leg, where `_where_git_fails` names a failure, so the
unreadable finding is always printed whenever #190's would be skipped.
Measured on this tree with a source holding `upstream/dependency`,
`url = ../dependency` and `update = bogus`, with `.gitmodules` and
`upstream/` kept in code for testorg:
- `origin` in otherorg: `check` 1 with `plan-gitmodules-unreadable` alone;
  `execute --local-remote-dir` 2, `REFUSED plan-gitmodules-unreadable`,
  with no remotes directory made. With `update = checkout`: `check` 1,
  `plan-submodule-relative-url`.
- `origin` in testorg: the same `check` 1 and `execute` 2, with no NOTE.
  With `update = checkout`: `check` 0 and the NOTE.
- At 8ce5614, before #215, the first source was `plan-submodule-relative-url`.
  The second was `check` 0 with the NOTE, over a file `git fast-export`
  dies on.

Lane: openreposhape-1 (openRepoShape-1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 12, 2026
…ly a placeholder its table lacks, so `--elected-by '{{SPEC_PATH}}'` is no longer recorded as `spec` and `{{NOPE}}` no longer refuses after the legs are pushed (#260) (#316)

**What was wrong (#260).** `render` replaced the placeholders one key at a time, each over the whole text. The value an earlier key put in was then searched by every later key. `--elected-by` and `--reference` in both tools, the scaffold's `--name`, and `family.py init`'s `--name` and `--created-by` are free text, so each could hold another placeholder's token:

- A token for a key that came later was expanded. `adopt-project.py --elected-by '{{SPEC_PATH}}'` recorded `elected_by: "spec"` in `project.yaml` and the README, and said `adoption verified`.
- A token for a key that came earlier (`{{PROJECT}}`), or one no key names (`{{NOPE}}`), stayed in the output. `render` then refused it as `template-unsubstituted`, with a remediation that blamed the tool. By then `execute` had pushed both legs, the scaffold had created all three repositories, and `family.py init` had created the holder's bare repository.

Reproduced on b30595c with the issue's own script, and the same for `family.py init --created-by`.

**The change (`scripts/shape_materialize.py`).**

- `render` reads the template once. One regular-expression substitution looks each placeholder up in the table, so a value goes in where its placeholder stood and is never searched again. In a `.yaml`/`.yml` template the quoted form `"{{KEY}}"` is matched first and spelled by `double_quoted`, exactly as #284 made it (`YAML_PLACEHOLDER_RE`, group 1).
- The refusal asks the template, not the output: it names the placeholders the template holds that the table has no value for, and it is raised before anything is substituted. That is the defect its remediation describes, so the remediation is now true. The output now holds the person's text, so it is no longer searched. Message and code are unchanged: `REFUSED template-unsubstituted: <source>: no value for ['{{NOPE}}']`.
- `PLACEHOLDER_RE` now takes digits, `\{\{[A-Z0-9_]+\}\}`, a superset of the old `[A-Z_]+`. The four digest placeholders (`{{SPEC_TREE_SHA256}}`, `{{CODE_TREE_SHA256}}`, `{{SHAPE_TREE_SHA256}}`, `{{PIN_TREE_SHA256}}`) were substituted only because the loop replaced every key by name. The single pass replaces only what the pattern finds, so it needs the digits. A digest placeholder that a table lacked used to be left in the output with nothing said; it is now refused like any other.

**Nothing downstream renders these files again,** so a `{{` a person typed is never expanded later, and no tool needs to refuse `{{` in these values. On this head:

- `git grep -n -E '\brender\(' -- '*.py' ':!tests'` lists `scripts/shape_materialize.py` alone: `copy_tree` and the three calls in `_materialize`. (`scripts/render-cli-reference.py` has its own unrelated `render`.)
- `copy_tree`, `materialize_assembly_root` and `materialize_family_root` are called only from `adopt-project.py execute` (`:2549`, `:3101`), `scaffold-project.py` (`:1013`, `:1054`) and `family.py init` (`:712`). Every one of those calls runs while the tool is creating the repositories.
- `update-shape.py` imports no renderer and no `TEMPLATED` list. In `project.yaml` / `family.yaml` it rewrites only the `shape:` block's `commit:` and `tree_sha256:` lines (`rewrite_manifest`, `:869`). The README's trailing `Shape:` line is a span rewrite of the commit.
- `bump-leg.py` rewrites only a leg's two pin lines.
- `shape-doctor.py` imports only `escape_what_stdout_cannot_encode` and `lane_trailer_argument` from the materializer. It reports a leg's `AGENTS.md`/`README.md` as `rendered` and compares nothing.
- Neither `bootstrap.py` nor any shipped validator holds `{{`: `git grep -n -F '{{' -- 'templates/*.py'` finds only `^{{commit}}` in two f-strings.
- `shape_advisory.py`'s own `PLACEHOLDER_RE` is asked only of `single-repository.yaml` (`field_problems`, called only from `read_record`). That is a record a person writes by hand, never `project.yaml`.

**Every caller, and what it relies on.**

- `adopt-project.py`: `_template_values` (`ORG`, `REFERENCE`, `ELECTED_BY`, and so on) is rendered into seeded legs and the assembly root. It relies on every key a template names being in the table (it is: the four leg commit/digest keys are `""` until the legs exist). Its free text is now written as typed.
- `scaffold-project.py`: the `values` table, plus `_seed_legs`' four leg values and `--pin`'s `PIN_*` values for neutral pins. Same reliance; the probe after this list names the keys.
- `scripts/family.py`: `_template_values`. `FAMILY` is a key no template names, and that is harmless.
- `setup-project.py` renders nothing. It forwards `--elected-by` and `--name` to the scaffold, and its own `NAME_RE` already refuses `{`.
- `update-shape.py` and `shape-doctor.py` render nothing (above).
- Tests: `test_adopt_leg_path_collisions.py` and `test_scaffold_leg_path_collisions.py` build a table from `PLACEHOLDER_RE.findall` over `templates/assembly-root/`. They now include the digest keys and still pass. `test_shape_advisory.py` renders leg `AGENTS.md` and the record template, which have no digest keys. `test_scaffold_ignored_shape_files.py` calls `copy_tree`. `tests/test_template_quoted_values.py` built its tables with its own `[A-Z_]+`, so its tables lacked the digest keys and its own `"SHA256" in key` branch never ran. It now uses `[A-Z0-9_]+`, and #284's behaviour is otherwise held unchanged.

A probe (`ast` over each tool's dict literals, against every placeholder under `templates/`) finds that every capitals key of the scaffold's and adopt's tables is named by a template, except `ORG`, which only `family-root` and `workspace-root` name. Every key of family's table is named by a template, except `FAMILY`.

**Byte-for-byte for ordinary values.** Every text file under `templates/` (49) was rendered with b30595c's `render` and with this head's, over two tables: plain values, and values holding `"`, `'`, `\`, `#`, `[x]`, `{y}`, U+3000 and a tab. That is 0 differences in 2 x 49 files. `test_an_ordinary_value_renders_as_a_plain_substitution` keeps this as a test for the 19 templates that hold a placeholder.

**Tests (`tests/test_template_values_single_pass.py`, 133).**

- `render`: each key's token (every key of the three tools' tables and every key a template names, 49 keys) and `{{NOPE}}`, bare and between double quotes, as every free-text value. Each reads back unchanged from `project.yaml` and `family.yaml`. Every README and `AGENTS.md` holds it exactly where it holds an ordinary value. Neither of two keys reads the other's value, in either order. A placeholder the template lacks is still refused, naming only the template's missing tokens and never a value's; this includes a digit key. `${{ github.x }}` and an f-string's `{{commit}}` are left alone.
- End to end, into local bare remotes, with a `gh` on PATH that records any call (none is made). What was pushed is read back with `git show`, so no checkout's line endings are compared:
  - `scaffold-project.py --name/--elected-by/--reference`, each holding every key's token (bare and quoted), and each as `{{NOPE}}`. The run exits 0. The pushed `project.yaml`, README and both legs' `AGENTS.md` hold the values as typed, and the project's own `validate-manifest.py` passes.
  - `adopt-project.py plan` / `check` / `execute` with `--elected-by` and `--reference` the same way. All three pass, with `adoption verified`. The split commit's `project.yaml` and README hold the values as typed. This test is skipped at its decorator without `git filter-repo`.
  - `family.py init --created-by` the same way. `family.yaml` holds the value, and `validate-family.py` passes. (The family's `--name` is held by the unit tests only. Its validator requires the holder's name to be one CamelCase token, so a free-text name is refused there whatever `render` does. See the last item under Left out.)

On b30595c this file is 113 failed, 20 passed. The 20 are the byte-for-byte guard (19), which describes base behaviour by design, and the sanity test that the templates are found.

Mutation probes, each a revert of one part of the fix: the sequential loop with the template check (98 failed); single pass with the output check (109 failed); `PLACEHOLDER_RE` without digits (69 failed, the scaffold e2e among them, because the pins keep `{{SHAPE_TREE_SHA256}}`); no quoted branch (94 failed, #284's tests among them); no template check (4 failed).

Targeted runs on this head: `tests/test_template_values_single_pass.py tests/test_template_quoted_values.py tests/test_scaffold_e2e.py tests/test_scaffold_pin_and_reuse.py tests/test_adopt_plan.py tests/test_adopt_execute_refusals.py tests/test_adopt_e2e.py tests/test_family.py tests/test_repo_hygiene.py tests/test_cli_reference.py tests/test_adopt_leg_path_collisions.py tests/test_scaffold_leg_path_collisions.py tests/test_shape_advisory.py tests/test_scaffold_ignored_shape_files.py`, with `git-filter-repo` 2.47.0 on PATH: 792 passed, 1 skipped (`test_adopt_plan.py:393`, `SHAPE_MEDXEHR_CLONE` unset). `tests/test_shape_doctor.py`: 208 passed.

**Composition** (`git merge-tree --write-tree` against the current tips; each merged tree ran the sibling's own test files plus this PR's two):

| Branch | Tip | Merge | Merged-tree tests |
|---|---|---|---|
| `origin/main` | c0ed615 | clean | 680 passed |
| `fix/scaffold-manifest-values` (lane 5's #258/#261, PR #314) | 7defac7 | clean | 370 passed |
| `fix/adopt-template-values-utf8` (lane 2's #270) | edfa7c6 | clean | 278 passed |
| `fix/leg-path-yaml-scalars` (lane 2's #266) | b2b42f8 | clean | 409 passed |
| `fix/adopt-check-shape-paths` (lane 5's #262/#263, PR #313) | 65a9262 | clean | 320 passed |

- #314 calls `render` in `manifest_misreads` and `manifest_invalid` with the complete table, so the template-side refusal never fires there. It now finds placeholders with this `PLACEHOLDER_RE`.
- #266 has merged main, so the import-line conflict an earlier tip had is gone.
- **Lane 1's #181 (`fix/adopt-plan-quote-yaml-keywords`, e337f44): a clean merge, one failing test, and the fix belongs to whichever lands second.** #181 alone passes (841). On the merged tree, 1 of 974 fails: `test_adopt_records_each_free_text_value_as_typed[every-key]`, refused `adopt-manifest-misread` at `plan`. #181's gate `_written` (e337f44 `adopt-project.py:1668-1669`) spells the value with `double_quoted` (lane 1's 4bf2cc4), then runs `.replace(token, value)`, which re-reads the value it just put in. That is the per-key re-scan this PR removes. So with both, the gate refuses values `execute` now writes as typed: `--elected-by 'x{{ELECTED_BY}}'` (the gate claims `"xx{{ELECTED_BY}}"`), `'"{{ELECTED_BY}}"'`, and `--reference 'see {{REFERENCE}}'` (it claims `"see see {{REFERENCE}}"`). On this PR alone all three pass `plan` and are written as typed. Neither branch is wrong against main, where `render` really would expand them. Whichever lands second makes `_written` one pass for its key: one `re.sub` over `"(\{\{KEY\}\})"|\{\{KEY\}\}`, or `render` on the line, as #314's `_misread_rows` does. Tokens of other keys (`{{SPEC_PATH}} {{NOPE}}`) pass on every tree.

**Left out.**

- No template placeholder is quoted (#259 waits on Brett), and no free-text value is refused for holding `{{`, since nothing renders it again.
- `family.py init --name` takes a display name (its help says so) that the holder's own `validate-family.py` refuses. On b30595c, `--name 'Contoso Group'` exits 0 and pushes the holder, and `validate-family.py` then exits 1 with `naming-unclassified: 'Contoso Group' matches no family in the naming policy`. This is pre-existing and not touched here; it is filed as #307, with `family.py init`'s other values.

**Review.** The first reviewer stalled at its first checkpoint and was replaced. The second review of 552763d found the code sound and asked only that this body's composition be corrected (above). Its #181 paragraph described an earlier #181 tip, and every other tip had moved.

What the review checked:
- #260's rows reproduced on b30595c and pass on the head, for adopt, scaffold and `family.py init`.
- All 49 templates render byte-identically between base and head over 25 tables of awkward values, in both key orders, and 69 files pushed by the real tools are byte-identical.
- A 40,000-case fuzz of `render` against an independent single-pass reference found 0 mismatches.
- 322 hostile values round-trip through `project.yaml` and `family.yaml` under both this repository's reader and PyYAML.
- Every table holds every digest key its templates name, so the widened refusal cannot fire on a run that used to succeed.

Nits kept:
- Three mutants survived the first head. Fix round 1 (321e511, tests only, with main c0ed615 merged as 7d6c66c) gives each a test that fails on it:
  - a `.yml` source gets the quoted spelling as a `.yaml` one does;
  - the `template-unsubstituted` list is the sorted set of what the template lacks, each token once;
  - an empty string in the table is a value that renders as empty text and is never refused.

  The seven targeted files pass on the merged tree, 541 tests.
- The `gh` stub proves nothing on Windows, where no test calls `gh`.

Reviewed adversarially: the first reviewer stalled and was replaced; the second found the code sound and asked only for this body's composition to be corrected, which it is. Fix round 1 adds tests for the three mutants that survived. After #313 landed, the branch was updated onto main 71b58a1 (9469148), and CI re-ran green on Linux, macOS and Windows, with SonarCloud passing.

Closes #260

Lane: openreposhape-5 (openRepoShape-5)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 12, 2026
Brings in, by sha, the twelve commits after 71b58a1: c7c02fa (#299),
346079d (#216, adopt-project.py `plan`'s follow-up `git grep`), 34ecbc2
(#260, shape_materialize.py `render` in one pass), 86da808 (#253),
6f4ce47 (#302), 7491530 (#288), 7591455 (#279, tests/conftest.py),
87373a6 (#306), 8a14ac9 (#204), c1b6732 (#239), e43fa95 (#243,
adopt-project.py's clone template and `adopt-mount-commit`) and a1b3a4b
(#321). Three of them touch this branch's files. One merge, no rebase.

No textual conflict: this commit's tree is `git merge-tree --write-tree`
of its two parents (28c351f). No top-level name is bound twice in
adopt-project.py, scaffold-project.py, scripts/repo_shape.py,
scripts/shape_materialize.py, shape-doctor.py, scripts/family.py,
scripts/bump-leg.py or update-shape.py, imports included. Main adds four
names to adopt-project.py and `YAML_PLACEHOLDER_RE` to
shape_materialize.py, none of them this branch's, and no function is
changed on both sides.

One SEMANTIC conflict, fixed in the next commit so that this tree stays
the mechanical merge. #260's `render` now replaces each placeholder in one
pass, so a value is never searched again. `_written` still builds
`execute`'s line in #284's two steps: first the quoted placeholder, and
then the bare one over that output, which holds the value spelled by
`double_quoted`. So for `--elected-by` or `--reference` holding its own
placeholder, the gate asks about a line `execute` does not write, and
refuses a value that reads back as typed:
- `"{{ELECTED_BY}}"`: the gate builds
  `elected_by: "\""{{ELECTED_BY}}"\""`, and `render` writes
  `elected_by: "\"{{ELECTED_BY}}\""`;
- `x{{ELECTED_BY}}`: `"xx{{ELECTED_BY}}"` against `"x{{ELECTED_BY}}"`;
- `"{{REFERENCE}}"` for `--reference`, the same way.
Every other probe value, and every bare placeholder, builds `render`'s
line.

Lane: openreposhape-1 (openRepoShape-1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 12, 2026
…r` for the line `execute` writes, so since #260's one-pass `render` an `--elected-by` or `--reference` holding its own placeholder, such as `"{{ELECTED_BY}}"`, is no longer refused for a line `execute` never writes (#181)

The merge of main a1b3a4b (bf6d656) brought #260 (34ecbc2): `render`
puts each value in once and never searches it again. `_written` copied
#284's rules in two steps, first the quoted placeholder and then the
bare one, and the second ran over the first's output, which held the
value. So a value holding its own token was put in twice, and the gate
refused it, though `render` writes it as typed. At bf6d656, main's own
`test_adopt_records_each_free_text_value_as_typed[every-key]` failed:
`plan` refused with `adopt-manifest-misread`.

- `_written` hands each template line that carries the placeholder to
  `render` itself, the same way the gate hands the line it builds to
  `loader_reading`. No copy of the materializer's rules is left to go
  stale. The table is the value, plus `_as_they_stand`: every other
  placeholder on the line given back as its own token, which `render`
  writes as it was, `double_quoted` spelling `{{KEY}}` as `"{{KEY}}"`.
  So the line differs from the template's by the one value, and `render`
  is never asked for a key it would refuse.
- `render` and `PLACEHOLDER_RE` are imported from the materializer.

Test (tests/test_adopt_plan_yaml_quoting.py): `HOLDING_A_TOKEN`, five
values (each field's own token after a letter and between double quotes,
and `{{NOPE}}`). Each is asked as `render` writes it, for `elected_by:`
and for `reference:`, and passes (10 cases).

Revert line: `_written` back to the two steps fails 5. These are the
four own-token cases, `x{{ELECTED_BY}}` and `"{{ELECTED_BY}}"` for
`elected_by` and the same two for `reference`, and main's
`test_adopt_records_each_free_text_value_as_typed[every-key]`. With
this commit, tests/test_template_values_single_pass.py is 164 passed.

Lane: openreposhape-1 (openRepoShape-1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 12, 2026
…elector's fallback are refused before anything exists when no file can hold them, so a byte that is not UTF-8 no longer creates repositories before a traceback (#282) (#351)

**What #282 still gets wrong on main (c7c02fa).** #284 (b30595c) spells each of the three quoted placeholders in `project.yaml` with `repo_shape.double_quoted`. So the manifest now reads back whatever was typed. The README, each `AGENTS.md` and each first commit's message still take the value as it stands. Every row of the issue was re-run on c7c02fa with local bare remotes only: 76 cases across `--elected-by`, `--reference`, `--name`, `GIT_AUTHOR_NAME` and `git config user.name` (given as `GIT_CONFIG_COUNT` config). In every case that scaffolded, the project's reader and PyYAML 6.0.3 read the manifest back as typed (0 misreads, 0 raises). What was still wrong:

- **A byte that is not UTF-8.** The scaffold exits 1 with a `UnicodeEncodeError` traceback from `write_lf`. Through `--elected-by`, `--reference` or `GIT_AUTHOR_NAME`, both legs are pushed by then and the assembly root is created empty. Through `--name`, all three repositories are created first. Through `git config user.name` it raises `UnicodeDecodeError` from `git_out`'s strict decode in `_elector`. That one happens under `--dry-run` too, before anything is created. The issue does not list this case.
- **A line break: LF, CR LF, U+0085, U+2028, U+2029.** The scaffold exits 0. The value goes raw into the README, which gains a line. Through `--name` it also goes raw into the root `AGENTS.md` (2 extra lines) and each leg's `AGENTS.md` (1). With LF or CR LF, each commit message's first line becomes `Seed the spec leg of Ada`; U+0085, U+2028 and U+2029 sit in that line raw. The `manifest ok:` line that the project's own `validate-manifest.py` prints is split as well.
- **A control character (U+0001, ESC, DEL, TAB), or a bidi embedding, override or isolate (U+202E, U+2066/U+2069).** The scaffold exits 0, and the value is raw in the README (and, for `--name`, in each `AGENTS.md` and each commit message).

**The change.**

- `scaffold-project.py` `_identity` asks `unwritable_reason` about the three values, beside `election_date`. Any value that fails is refused as `scaffold-unwritable-value`. The refusal is one sentence per value: the flag, the value as Python's `repr` spells it, the files it is written into, and why. All failing values go into one refusal. It comes before a work directory or a remote exists, and `--dry-run` refuses the same. The only process that can run first is the read-only `git config user.name` that finds the elector when neither `--elected-by` nor `GIT_AUTHOR_NAME` gives one:

  ```
  REFUSED scaffold-unwritable-value: --name is 'Jos\udce9', which the scaffold writes into the assembly root's `project.yaml` and `README.md`, each repository's `AGENTS.md` and each first commit's message, and it holds a byte that is not UTF-8.
  ```

  An elector the flag did not supply is named by its source: `--elected-by (not given, so GIT_AUTHOR_NAME)` or ``--elected-by (not given, so `git config user.name`)`` (`_elector_named`, in `_elector`'s order). The second is the wording #270's `plan` uses.
- `_elector` decodes `git config user.name` with `surrogateescape`, the way argv and the environment are decoded. A byte that is not UTF-8 there is then refused by name, not raised. `adopt-project.py` already reads git with this pattern (`git_out(..., binary=True).decode("utf-8", "surrogateescape")`).
- `scripts/shape_materialize.py` gains `BIDI_CONTROLS` and `unwritable_reason`: see "Composition with PR #270" below.

**Each row of the issue.**

| row | `project.yaml` | README / `AGENTS.md` / commit messages | where it stands |
|---|---|---|---|
| a backslash (`CORP\nick`, also through `GIT_AUTHOR_NAME` and `git config user.name`; `C:\docs\notes\new.md`) | fixed on main by b30595c (#284): `"CORP\\nick"`, read back exactly | raw, as typed, on its own line (Markdown keeps a backslash before a letter literal, and the path is inside backticks) | fixed on main by b30595c |
| a double quote (`Joe "JJ" Smith`) | fixed on main by b30595c: `"Joe \"JJ\" Smith"`, read back exactly | raw, as typed | fixed on main by b30595c |
| a line break: LF, CR LF, U+0085, U+2028, U+2029 | parses on main since b30595c (escaped, read back exactly) | split on main | **fixed by this branch**: refused |
| a byte that is not UTF-8 (all five ways in) | escaped on main since b30595c; not where it failed | traceback on main: `write_lf` raised on the root README or, for `--name`, the spec leg's `AGENTS.md`; `git config user.name` raised in `git_out` | **fixed by this branch**: refused |
| a control character: U+0001, ESC, DEL (TAB too) | PyYAML's `ReaderError` fixed on main by b30595c (escaped) | raw on main | **fixed by this branch**: refused |
| a bidi control: U+202E, U+2066, U+2069 | escaped on main since b30595c | raw on main (`README.md:9`) | **fixed by this branch**: refused |

No row is left open. On this head, the 60 cases with an unwritable value exit 2 under both `--dry-run` and a real run. Each prints one `scaffold-unwritable-value` refusal and makes no bare remote, with 0 tracebacks. The other 16 cases scaffold, exit 0: the backslash, the Windows path, the double quote, and a Persian name joined with U+200C ZERO WIDTH NON-JOINER. Each reads back exactly, and the README gains no line.

**Files outside YAML.** `README.md` (`:1` name, `:9` elector, `:10` reference in backticks), the root `AGENTS.md` (`:3`, `:13`), each leg's `AGENTS.md` (`:1`), and the three first commits' messages take the value raw. A backslash and a double quote break none of them: the file holds the typed text, and the line count is unchanged. Every other row is now refused before any of them is written. `--name` also goes into the GitHub description via `gh repo create`, which is not run here.

**Composition with #270 (lane 2, #255), now landed as f795cf7.** #270 added `unwritable_reason` and `BIDI_CONTROLS` to `scripts/shape_materialize.py`, and this branch adds the same hunk byte for byte at the same place (after `is_trailer`). The merge of this branch with main ab623dd (e5804e0, made by `gh pr update-branch`) is clean, and its `scripts/shape_materialize.py` holds one `def unwritable_reason` and one `BIDI_CONTROLS`. The two tools refuse the same values in the same words, and the two comments in `scaffold-project.py` that say `adopt-project.py` asks the same question are now true.

**Composition with PR #314 (lane 5, `fix/scaffold-manifest-values`).** No duplication. #314 asks the project's reader and `validate-manifest.py` about the rendered manifest, after the values table is built. This branch asks whether a value is text a file can hold, earlier, in `_identity`. Each refuses values the other passes: a blank `--elected-by ' '` is #314's, and `Ada\nLovelace` is this branch's. The merge with ab66bc0 is clean. On the merged tree, #314's own tests fail in 5 places, all on a value both PRs refuse: `--elected-by` of a bare TAB, `\x0b`, `\x1c` or U+2028, which is blank to the validator and unwritable here. This branch refuses those first, as `scaffold-unwritable-value`, where #314's tests expect `scaffold-manifest-invalid` or `scaffold-manifest-`. The failing tests are `test_a_value_the_validator_refuses_is_refused_before_anything_is_made[elected-by-tab]`, `test_a_dry_run_refuses_what_the_validator_refuses[elected-by-tab]` and `test_a_blank_that_is_also_a_line_break_is_refused_too[\x0b|\x1c|\u2028]`. Whichever of the two lands second applies a four-line patch to `tests/test_scaffold_manifest_values.py` (lane 5 holds it), which expects `scaffold-unwritable-value` for those four values. With it applied, the merged tree passes: see Evidence.

**Merge-tree** (`git merge-tree --write-tree HEAD <ref>`, fetched 2026-10-12):

| ref | tip | result |
|---|---|---|
| origin/main | c7c02fa, and 346079d (#228 landed since: adopt files only) | clean |
| PR #270 | 72bb869 | clean, one `unwritable_reason` |
| PR #314 | ab66bc0 | clean (5 of #314's tests need the patch above) |
| PR #309 (lane 4) | 2ea1717 | clean |
| PR #316 (lane 5, `render` single pass) | 9469148 | clean |
| PR #266 (lane 2) | b2b42f8 | clean |
| lane 1 `fix/adopt-record-submodule-placement` | 3e86200 | clean |
| lane 1 `fix/adopt-late-failure-notice` | 918e8f2 | clean |
| lane 1 `fix/adopt-root-kept-symlinks` | 6c44591 | CONFLICT in `adopt-project.py`, which this branch does not touch; the branch conflicts with origin/main alone the same way |
| lane 1 `fix/adopt-plan-quote-yaml-keywords` | e337f44 | the same: CONFLICT in `adopt-project.py`, also against origin/main alone |
| lane 5 siblings `fix/pin-source-env-collisions` c32a228, `fix/family-init-values` 14e2d68, `fix/yaml-flow-brackets-surrogates` 38738cf, `docs/adopt-execute-exits` d3f71d6 | | clean |
| lane 2 local `fix/scaffold-tracking-branch-ref-format` (#283) | a03abcf | clean |

**Evidence.**

- New `tests/test_scaffold_free_text_values.py`, 113 cases:
  - each of the 12 unwritable values through each flag, dry and real (72), under a guard that fails the test if any process starts;
  - all three bad values named in one refusal;
  - each value through `GIT_AUTHOR_NAME` (12), with no process started;
  - `git config user.name` with LF, U+202E, ESC and a byte that is not UTF-8, dry and real (8). The last is skipped on Windows, whose environment is UTF-16;
  - the flag winning over the fallbacks;
  - 15 dry runs that must not refuse;
  - one real scaffold of a backslash, a quote and U+200C, through `--name`, `--reference` and `GIT_AUTHOR_NAME` (the test does not use the `--elected-by` flag). It reads back exactly, and the README lines, each leg's `AGENTS.md` and the commit subjects stay single (the root `AGENTS.md` is not checked by this test);
  - a byte that is not UTF-8 through each flag as a child process's argv (3).
- That file on head: 113 passed. On c7c02fa: 97 failed, 16 passed. The 16 are the 15 dry runs and the real scaffold of text, which hold on main since #284 by design.
- Mutation probes (each reverts one part, in a throwaway worktree; every one makes at least two tests fail):
  - no refusal call: 97 failed;
  - strict decode of `git config user.name`: 2;
  - `_elector_named` always `--elected-by`: 20;
  - elector field dropped: 47;
  - `--reference` dropped: 26;
  - `--name` dropped: 26;
  - no line-break check: 39;
  - no surrogate branch: 13;
  - no bidi branch: 17;
  - no control branch: 30;
  - refusing every `Cf` (over-refusal): 4, the U+200C cases.
- Targeted on head 8f43c96 (`git-filter-repo` 2.47.0 on PATH): `tests/test_scaffold_free_text_values.py tests/test_scaffold_e2e.py tests/test_scaffold_pin_and_reuse.py tests/test_scaffold_leg_paths.py tests/test_scaffold_leg_path_collisions.py tests/test_scaffold_pin_collisions.py tests/test_scaffold_referent_chain_links.py tests/test_scaffold_mount_ignored.py tests/test_scaffold_ignored_shape_files.py tests/test_template_quoted_values.py tests/test_naming_policy.py tests/test_reference_default.py tests/test_reuse_visibility.py tests/test_stdout_encoding.py tests/test_repo_hygiene.py tests/test_cli_reference.py`: 801 passed, 1 skipped (`test_stdout_encoding.py:286`, Windows' own default).
- Merged trees:
  - with #270, this file plus `test_adopt_template_values.py`: 191 passed;
  - with #314 plus the compose patch, this file plus `test_scaffold_manifest_values.py`, `test_scaffold_e2e.py`, `test_template_quoted_values.py` and `test_cli_reference.py`: 343 passed. Without the patch (on a5ef3f5, which differs from 8f43c96 only in an import's place and a docstring): 5 failed, 338 passed, the five above;
  - with #316, this file plus `test_template_quoted_values.py` and `test_template_values_single_pass.py`: 344 passed.

**Left out, with follow-up candidates.**

- **`adopt-project.py plan` dies the same way on a `git config user.name` that is not UTF-8.** With no `--elected-by`, it exits 1 with `UnicodeDecodeError` from `git_out` (c7c02fa, and with #270 merged). `_elector` in `adopt-project.py` reads it strictly. It is adopt's half and #270's ground, so it is not touched here.
- **A `--local-remote-dir` path that is not UTF-8.** The scaffold creates the three bare remotes, then raises in `write_lf` on `{{ASSEMBLY_CLONE_URL}}` in a leg's README (reproduced on this head). It is the test path only. #270 refuses the same for adopt's clone URL (`_refuse_an_unwritable_clone_url`). Here it is a separate, small issue.
- **A `GIT_AUTHOR_NAME` of whitespace alone.** The scaffold falls past it to `git config user.name` for the elector. `env_commit` then hands it to git unchanged, and git refuses to commit (`fatal: name consists only of disallowed characters`) after the remotes are created. Seen on c7c02fa while testing; not this issue's input.
- **Markdown syntax in the values** (`*`, `` ` ``, `[`, `\*`) renders as Markdown in the README and `AGENTS.md`. The files hold the typed text, and escaping it would print backslashes in a plain read, so it is not changed.
- No `--help` text changed, so `docs/cli.md` is not regenerated. The module docstring's first line, the help description, is unchanged.

**Review.** An adversarial review of 8f43c96 found 0 blockers, 0 should-fix and 5 nits (APPROVE WITH NITS):
- **#270 copy:** it re-checked the copied hunk byte for byte (same sha256) and found one definition in every merge, in either landing order.
- **Probes:** 63 values went through all five sources, dry and real; 320 cases were refused with no work directory, remote or traceback, and nothing was found landing broken.
- **Order with #314:** the #314 patch is complete (219 passed with it), and the order of the two refusals is sensible.
- **Nits handled in this body:** the "before any process" claim (N1) and wording (N5).
- **Nits left for a fix round:**
  - three mutants of this branch's copy of `unwritable_reason` that only #270's unit table catches (N2);
  - the argv child-process test with no Windows skip (N3), which Windows CI then passed;
  - naming the elector's fallback the same way as #314 on the merged tree (N4).

Reviewed adversarially: no blocker or should-fix, five nits (two corrected in this message's text, one settled by Windows CI, two left as small follow-ups). CI is green on Linux, macOS and Windows, with SonarCloud passing, at e5804e0 (8f43c96 merged with main ab623dd); main's later #245 (161c6c9) touches none of this branch's files.

Closes #282

Lane: openreposhape-5 (openRepoShape-5)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 12, 2026
… is refused and a name holding a lone surrogate is a finding, so `pins: [a, b]]` is not the pin `b]` and `name: "a\ud83db"` no longer kills `make validate` (#301, #280) (#390)

Two reader-side defects the lane filed, in one commit on `fix/yaml-flow-brackets-surrogates`, cut from main c7c02fa. Below, "this head" is that commit, 21fbea4, the one the review approved; main was merged into the branch afterwards (last paragraph).

**#301, `scripts/repo_shape.py`.** In YAML a plain scalar inside a flow collection ends at any of `[ ] { } ,`. `_split_flow` counts brackets to find its commas and never asked whether the count went below zero or ended above it, so a stray bracket was kept as text:
- `pins: [a, b]]` read `['a', 'b]']`;
- `k: [x [a, b]]` read `['x [a, b]']`;
- since #284, `k: {a"]:}` read the key `a"]`.

PyYAML refuses each. A new `_refuse_flow_indicator` refuses a PLAIN item or value (called from `_flow_value`) and a PLAIN key (called from `_split_key_flow`) of a flow collection that holds `[`, `]`, `{` or `}`. The refusal is a `YamlError`, so `load_yaml` turns it into `REFUSED yaml-unparsable` and names the scalar: ``a plain scalar in a flow collection holds `]`, which ends it in YAML: 'b]'``. These still read as before:
- a quoted scalar's brackets (`k: ["a]", {"b}": "c]"}]`);
- nested and empty flow collections;
- a bracket in a BLOCK plain scalar (`k: a[b]`, `- see [1]`);
- a bracket in a comment after a flow.

No other function changed.

**#280, `templates/assembly-root/scripts/validate-manifest.py`.** `_name_finding` now reports a display name holding any of U+D800 to U+DFFF as `manifest-name`:

```
FINDING manifest-name: name is 'a\ud83db', which holds U+D83D, a lone surrogate: no UTF-8 file, commit message or console can hold it. Write each character as itself, or as one `\U` escape (`\U0001F600`, not the pair `\uD83D\uDE00`).
```

The finding goes to stderr with the name escaped by `repr`, before the `manifest ok` line could raise.

**Why the #280 refusal is in the validator and not in the reader.** Two places could refuse a lone surrogate, the reader and the validator. The evidence rules out the reader:

- **The writer emits a lone-surrogate escape on purpose.** The escape is `\uDXXX`, a `\u` and four hex digits from D800 to DFFF. `double_quoted('caf\udce9')` is `"caf\udce9"`. That is how `adopt-project.py`'s plan keeps a path byte that is not UTF-8 (#199), and `check` must read it back. `tests/test_adopt_entry_paths.py` holds that round trip (120 passed with `test_adopt_plan.py` on this head). Lane 1's pending #189 branch (70f4d29, `submodule_record`) also writes `\udce9` escapes into `project.yaml`'s `submodules:` block, and its docstring relies on the reader resolving them. A reader refusal would break both.
- **PyYAML 6.0.3 agrees with the reader.** The pure-Python `SafeLoader` reads `name: "a\ud83db"` as the lone surrogate. Only libyaml (`CSafeLoader`) refuses it, and it refuses even the escape pair `"\ud83d\ude00"`, which the Python loader reads as two surrogates, as this reader does.
- **The defect is in a value, not in the YAML.** A display name is text a person chose, and no UTF-8 stream holds a surrogate. On c7c02fa and on this head, with local bare remotes only:

  | launcher | c7c02fa | 21fbea4 |
  |---|---|---|
  | `scripts/validate-manifest.py` | exit 1, `UnicodeEncodeError` traceback | exit 1, the `manifest-name` finding |
  | `make validate` | exit 2 on that traceback | exit 2 on the finding |
  | `shape-doctor.py --root .` | `INVALID (manifest)`, the row quotes `UnicodeEncodeError: ...` | `INVALID (manifest)`, the row quotes `FINDING manifest-name: ...` |
  | a real `bump-leg.py` | exit 0, prints `manifest ok: a\ud83db`, and COMMITS the subject `Bump spec leg to <sha> in a?b` (`errors="replace"`) | exit 2 `bump-leg-validators-red`, every byte rolled back, 0 commits |
  | `update-shape.py apply` | exit 0, prints `manifest ok: a\ud83db`, commits | exit 2 `update-validators-red`, restored, 0 commits |

So all four launchers now agree, and each names the problem.

**Readers and callers this changes.**

- **#301:** every `parse_yaml`/`load_yaml` caller, and only for a document whose flow collection holds a plain scalar with a bracket. Such a document was read, or misread, before and is refused now. Every tracked YAML file loads the same on c7c02fa and on this head (14/14 raw, 18/18 rendered templates). So do the 6 YAML files of a scaffolded project (`project.yaml`, four `contracts/*.yaml` and `validate.yml`): 6/6 on a c7c02fa scaffold and 6/6 on a 21fbea4 scaffold. No writer emits such a scalar:
  - every flow sequence written in the tree is bare names or `[]` (adopt's `pins: [...]` and `allow_empty_legs: [...]`, `naming_block`'s `also_matches` and `referent_chain`);
  - `double_quoted` quotes any value holding a bracket.
- **#280:** the project's own `scripts/validate-manifest.py` (a digest-pinned shape copy; it reaches existing projects through `update-shape.py apply`, which reports it `upstream-changed`). Every caller of that copy now stops at the finding:
  - `make validate`;
  - the CI step in `validate.yml`;
  - `shape-doctor.py`'s `manifest` row;
  - `bump-leg.py` and `update-shape.py apply` (their red-validator refusal and rollback);
  - a family holder's `make validate`, which runs each member's.

  `bootstrap.py` runs the same copy in its step (b), but on such a name it dies at its own first `print`, before step (b) (see Left out).

  With #314 merged as well, the scaffold's pre-check asks this validator about `--name`. In the tree of this head and #314 alone, `scaffold-project.py --name $'caf\xe9' --dry-run` is refused `scaffold-manifest-invalid` with this finding before anything is created (checked at #314's head 4b8da3d). With #351 (landed as d82a83e) and without #314, that `--name` is already refused, as `scaffold-unwritable-value`.

**Evidence.**

- **Issue rows.** All ten #301 documents reproduce on c7c02fa and are refused on this head. 8 are documents PyYAML refuses. 2 (`k: {a:}#}: b}`, `k: {}#":}`) PyYAML reads only by taking a `#` with no space before it for a comment, and they are refused by design, as `k: "a"#c` is. Every #280 row is in the table above.
- **Seeded oracle fuzz.** Corpus: an earlier review's, seed 2490250, 785,646 documents in six contexts. PyYAML 6.0.3 is the oracle only.

  | | c7c02fa | 21fbea4 |
  |---|---|---|
  | read the way PyYAML reads it | 254,629 | 254,629 |
  | read, where PyYAML refuses | 179,999 | 112,562 |
  | refused, where PyYAML reads | 48,434 | 49,492 |
  | read, but not as PyYAML reads it | 2,330 | 1,272 |
  | the flow contexts alone: read where PyYAML refuses / refused where PyYAML reads | 77,073 / 15,413 | 10,306 / 16,465 |

  Only two kinds of document change:
  - 67,437 invalid documents c7c02fa read are refused;
  - 1,058 documents c7c02fa MISREAD are refused. 1,037 of these are PyYAML's `#` leniency, and 21 are a single-pair mapping in a flow sequence (`k: [a:[]]`), which the subset never read as a mapping.

  Nothing c7c02fa read correctly changes, nothing is newly read, and there are 0 crashes. Against the issue's two-line `depth < 0` sketch, run on the same corpus, this head refuses everything the sketch refuses, plus 21,631 more: 21,608 invalid documents (the opener class, such as `k: [a[]` and `k: [x [a, b]]`) and 23 misreads. The fuzz ran on 38738cf, whose `scripts/` and `templates/` are byte-identical to 21fbea4's.
- **Tests fail on c7c02fa.** With the four test files copied onto c7c02fa: 45 failed, 191 passed. The failures are 31 bracket rows, 10 validator rows and the 4 launchers.
- **Mutation probes.** Each hunk was reverted or weakened on its own, and each mutant fails new tests:
  - M1, no check on items and values: 23 failed;
  - M2, no check on keys: 8;
  - M3, no quoted-scalar exemption: 14;
  - M4, no quoted-key gate: 11;
  - M5, closers only: 3;
  - M6, no surrogate finding: 10;
  - M7, upper bound excludes U+DFFF: 2;
  - M8, lower bound excludes U+D800: 2.

**Tests.**

- New: `tests/test_yaml_flow_brackets.py`, with the ten issue rows, sixteen more positions, the refusal's wording, a file refusal, fifteen still-read rows and the writer round trip.
- In `tests/test_manifest_validator.py`: five surrogate spellings, each under a pipe and under `PYTHONIOENCODING=utf-8`, plus four still-ok controls on either side of the range.
- In `tests/test_validator_child_encoding.py`: a real bump, `update-shape.py apply`, the doctor and `make validate`, each on `name: "a\ud83db"`.
- `tests/test_yaml_quote_boundaries.py`: its docstring said `k: "a"#c` was the only PyYAML disagreement. There are eight; seven are `BRACKET_IN_A_KEY` rows refused by design.

Targeted runs on 21fbea4, Linux, Python 3.12.3, git-filter-repo 2.47.0, each in the background:
- the reader and validator files (`test_yaml_flow_brackets`, `test_yaml_flow_quotes`, `test_yaml_flow_mappings`, `test_yaml_quote_boundaries`, `test_yaml_block_scalar_keys`, `test_load_yaml_refusals`, `test_adopt_plan_escapes`, `test_manifest_validator`, `test_pin_validator`, `test_neutral_pin_validator`, `test_template_quoted_values`): **462 passed**;
- `test_validator_child_encoding`, `test_stdout_encoding`, `test_repo_hygiene`, `test_cli_reference`: **165 passed, 1 skipped** (`test_stdout_encoding.py:286`, a Windows-only default);
- `test_adopt_plan`, `test_adopt_entry_paths`: **120 passed, 1 skipped** (`test_adopt_plan.py:393`, `SHAPE_MEDXEHR_CLONE` unset);
- `test_shape_doctor`: **242 passed, 3 skipped** (`test_shape_doctor.py:5330`, PowerShell on a Windows host only);
- this head merged with origin/main 86da808, which carries #316: `test_template_quoted_values`, `test_template_values_single_pass`, `test_yaml_flow_brackets`, `test_yaml_quote_boundaries`, `test_manifest_validator`: **412 passed**;
- this head merged with #314 (ab66bc0): `test_scaffold_manifest_values`, `test_adopt_manifest_values`, `test_manifest_validator`, `test_yaml_flow_brackets`: **293 passed**.

The full suite and the macOS and Windows legs did not run before this PR's CI.

**Merge-tree, as of 2026-10-12T15:12Z** (`git merge-tree --write-tree --name-only`), against origin/main d82a83e and the heads named here; each sha is the one it had then, and main has moved since.

- **Clean with this commit:**
  - origin/main d82a83e, which carries #316 (34ecbc2), #245 (head a0272b2, landed at 14:55Z as 161c6c9) and #351 (landed as d82a83e). #245's one change to `scripts/repo_shape.py` is in `remote_key`, away from the YAML reader, and #351 touches `scaffold-project.py` only;
  - #314 (4b8da3d). With it merged, the `--name` refusal above still holds (checked at 4b8da3d);
  - lane 1's `fix/adopt-late-failure-notice` (9f1778d) and `fix/adopt-plan-quote-yaml-keywords` (d38e143);
  - the sibling `fix/pin-source-env-collisions` (34dbd86, #275);
  - every other open PR head but one: 28 of the 29 open, all but #358.
- **Conflicts that are not this commit's.** Each conflicts the same way with origin/main alone, in a file this commit does not touch:
  - #358 (ebffb08), in `shape-doctor.py`;
  - lane 1's `fix/adopt-root-kept-symlinks` (6c44591), in `adopt-project.py`.
- **Five heads already conflict with origin/main itself, and merge clean with this commit:** #314, #356 (5a5011d), #382 (7e55d00) and lane 1's `fix/adopt-plan-quote-yaml-keywords` in `adopt-project.py`, and lane 1's `fix/adopt-late-failure-notice` in `tests/conftest.py`.
- **One conflict is this commit's:** `fix/adopt-record-submodule-placement` (70f4d29, lane 1, #189), in one hunk, the import block of `tests/test_manifest_validator.py`. Both branches add to it: that branch adds `import os` and `import sys`, and this one adds `import subprocess` between them, so the resolution is all three. Its `repo_shape.py` and `validate-manifest.py` changes auto-merge. Not resolved in either branch.

**The review.** An independent opus review at 21fbea4 approved it with nits: 0 blocker, 0 should-fix, 5 nit. All five are in text (this description, the author's report and the commit message), none in the code or the tests. Four were fixed in the description; the fifth, the commit message's spelling of the surrogate escape as a six-digit one, is fixed in this message: the escape is `\uDXXX`, a `\u` and four hex digits, D800 to DFFF. Its own seeded structured fuzz (seed 7301280: 120,000 nested flows in five block contexts, 600,000 documents, PyYAML 6.0.3 as the oracle only) found 0 changed reads: no document c7c02fa read the way PyYAML reads it changed, and nothing crashed. An instrumented reader returned no plain scalar holding `[ ] { }` inside a flow on this head, against 76,955 on c7c02fa. The author's seed 2490250 counts reproduced row for row, and all 13 mutants it tried were killed. **664 passed** on this head merged with origin/main ab623dd (16 test files; 361 on the head alone, 8 files). The one conflict, with lane 1's #189 branch, is the `tests/test_manifest_validator.py` import block, and resolves to all three imports. Three follow-ups it confirmed are not in this change: the bootstrap's raw name print and the flow entries still read (both under Left out), and a block value that opens with `]` or `}`, which c7c02fa and this head both read where PyYAML refuses.

**Left out.**

- **A plain flow entry that opens with an indicator, and an empty flow entry.** Most of what this head still reads where PyYAML refuses is one of two classes:
  - an entry opening with `:`, `#`, `?` or `-` (`k: {:":}` reads `{':"': None}`, `k: [:]`; 6,286 flow documents);
  - an empty entry from a leading or doubled comma (`k: [,a]` reads `['a']`, `k: [a,,b]` reads `['a', 'b']`; 3,858).

  Neither is a bracket, and #301 does not claim them.
- **A single-pair mapping in a flow sequence.** `k: [a: b]` still reads as the string `'a: b'`, where PyYAML reads `[{'a': 'b'}]`. Only the ones holding a bracket are now refused.
- **`templates/assembly-root/scripts/bootstrap.py`.** It prints the display name raw (`bootstrap: <name> (<root>)`) before its validators run. On a project named `"a\ud83db"`, `make bootstrap` still dies there with `UnicodeEncodeError`, on c7c02fa and on this head (filed as #391).
- **`elected_by` and `reference`.** A surrogate in either is still accepted. Neither is printed by the validator, so nothing raises; the scaffold side of such values is #282.
- **The escape pair `"\ud83d\ude00"`.** It is still two surrogates, not one character, as PyYAML's Python loader reads it. The finding asks for the character itself or its one `\U` escape instead of the reader inventing a joining rule.

Reviewed adversarially at 21fbea4 (APPROVE WITH NITS: 0 blockers, 0 should-fixes, 5 nits, all in text and none in the code or the tests). The code and the tests needed no fix round; the nits are fixed in this message. Main was merged into the branch after the review with no conflict: at that merge (c1a119d, main 45e01f4 merged in) the branch's own diff against main is the one reviewed, the same 6 files with 422 lines added and 6 removed. CI is green on Linux, macOS and Windows, with SonarCloud passing, at c1a119d (main 45e01f4 merged in). The combined tree with main 3ef059c (#314 landed) was run locally before landing.

Closes #301
Closes #280

Lane: openreposhape-5 (openRepoShape-5)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 13, 2026
… text placeholders, so `--id 007`, `--tracking-branch 2026`, `--spec-path true` or `--org null` read back as typed, and `family.py add` quotes a member id alike (#259) (#395)

Brett Heap ruled on 2026-10-12: "quote #259". The templates now write each text field a tool fills from a value it was given between double quotes:

| template | fields |
|---|---|
| `templates/assembly-root/project.yaml` | `id`, `elected_on`, `visibility`, `tracking_branch`, the spec leg's `path`, the code leg's `path` |
| `templates/assembly-root/contracts/spec-pin.yaml`, `code-pin.yaml` | `submodule_path` |
| `templates/family-root/family.yaml` | `id`, `org`, `created_on`, `visibility`, `tracking_branch` |

`render` already spells a value between a YAML template's double quotes with `repo_shape.double_quoted` (#284). The single-pass `render` from #316, on main as 34ecbc2, does the same through `YAML_PLACEHOLDER_RE`. So each value reads back exactly. An ordinary value renders as it did before, plus the two quote characters.

`family.yaml`'s `org:` is quoted along with the four fields #259 names. It is the same defect in the same file: `family.py init --org 2026` and `--org null` pass `checked_value`, and on c7c02fa the holder then fails `family-org` (#307 lists the `--org null` row). Taking `org:` back out is not a one-line revert. It fails 14 of the new tests (the 8 `family.yaml:org` in-memory cases, and all 6 `family.py init` rows, because every row of `FAMILIES` passes a misread `--org`), and it would also take the `ORG` row out of `TEXT_FIELDS` and put plain `--org` values into `FAMILIES`. The lane keeps `org:` quoted, as the same defect in the same file, unless Brett says otherwise.

`scripts/family.py`: `_member_lines` writes each member's own `project.yaml` id into the holder's `members:` block. That id now reads back exactly, so a bare `007` in the row came back as 7, and `validate-family.py` would report `member-id-mismatch` for a member it had just pinned. `_member_id` quotes an id only when the reader would not read the bare text back as the same text. No row whose id reads back as text moves: `irrs` is still `id: irrs`. Two edges:

- A legacy row whose bare id the reader misreads moves once. A holder written before this change, by `family.py add` of a member scaffolded with `--id 2026`, holds `id: 2026`. The next `bump`, `add` or `remove` of that holder rewrites it as `id: "2026"`. That is harmless: `validate-family.py` compares with `str()` and passes before and after.
- Mixed versions. A `family.py` from before this change, run against a holder this branch wrote, turns `id: "007"` back into `id: 007`, and `validate-family.py` then fails `member-id-mismatch` (the member declares `007`, `family.yaml` records `7`). A current `family.py` that rewrites the holder next then writes that row as `id: "7"`. Nothing in this branch can prevent it: run `family.py` from a current checkout on a holder that has such a member.

Left unquoted on purpose: `topic` (`xf-project-<id>`), every `repository` (`<org>/<Name>`; the slash keeps it text), `SHAPE_REPOSITORY` and `DIGEST_DEFINITION` (constants), and the block or list placeholders (`NEUTRAL_PRODUCT_PINS`, `*_NAMING`, `SHAPE_FILES`).

**Why.** Every reader of these files uses `repo_shape`'s YAML subset. It reads a bare `007` as 7, `2026` and `20261011` as numbers, `1.5` as a float, `true` as True and `null` as None. The checks on the flags (`PROJECT_ID_RE`, `checked_value`, the leg-path rule) accept every one of these. Reproduced on c7c02fa, with local bare remotes only:

- `scaffold-project.py --id 2026|007|true|null|20261011` exits 0, and the project's own `validate-manifest.py` then fails `manifest-id`.
- `--spec-path 2026 --code-path 007` fails `manifest-leg-path`, and `validate-pins.py` exits 2.
- `--tracking-branch 007` pushes each leg on `007`, and the root's `bootstrap.py` then puts the legs on a local `7` with `fatal: no upstream configured`. `true` and `null` do the same on `True` and `main`.
- `adopt-project.py execute` with `--tracking-branch 007` does the same; `--id 007|20261011` gives `manifest-id`; leg paths `2026`/`007` give `manifest-leg-path`.
- `family.py init --id 2026|007|true|null|20261011` gives `family-id`; `--org 2026|null` gives `family-org`; `--tracking-branch 007|true|null` reads back as 7/True/None.

On this branch every one of those rows reads back as typed, bootstraps onto the typed branch with `origin/<branch>` as upstream, and passes the root's own validators.

**Every reader checked.** Every reader reads these fields through `load_yaml`/`parse_yaml` into a dict. None uses a line regex, `split(":")`, `startswith("<field>: ")` or a whole-line rewrite of them:

- `templates/assembly-root/scripts/bootstrap.py`: `tracking_branch`, leg `path`.
- `validate-manifest.py`: `id`, `elected_on`, `visibility`, leg `path`.
- `validate-pins.py`: `submodule_path` against the manifest's `path`. The pin side is read as it stands and the manifest side goes through `str()`, so the two differ in type exactly when a bare value is misread (that is the `pin-path-mismatch` row above). With both lines quoted they are the same text.
- `validate-repository-naming.py`: `str(id)` for the topic.
- `templates/family-root/scripts/validate-family.py`: `id`, `org`, `created_on`, `visibility`, `str()` member-id compare.
- `siblings.py`, family `bootstrap.py`.
- `scripts/bump-leg.py`: `tracking`, leg `path`. Its `rewrite_pin` rewrites only `commit:` and `tree_sha256:`.
- `update-shape.py`: `tracking_branch_of`. `rewrite_manifest` touches only the `shape:` block.
- `scripts/family.py`: `tracking()`, the member's id, and the `members:` rewrite. The member-id writer is the one fixed here.
- `shape-doctor.py`: leg `path`, member row `path`, working-clone `str(id)` compare.
- `adopt-project.py`: plan reads and `_template_values`.
- `setup-project.py`: `family.yaml` `name`/`kind` only.
- `scripts/shape_advisory.py`: the single-repository record only.

The `str()` fallbacks (`bootstrap.py`, `adopt-project.py`'s `checked_value`) stay, because projects made before this keep their bare lines.

**Pins.** The four files are TEMPLATED, not COPIED, so no row of any project's `contracts/shape-pin.yaml` names them. `Atlas` was scaffolded and `family.py init Contoso` was run on c7c02fa and on this branch. Each `contracts/shape-pin.yaml` differs only in `commit:` and `digests.tree_sha256`, which identify the standard itself and change with every commit; every per-file `sha256` row is identical. `update-shape.py check` of the c7c02fa project and holder against this branch reports every copied file `unchanged` (11 and 10 rows): "no copied file differs; `apply` would move the pin alone".

**Tests.**

- New: `tests/test_template_text_placeholders.py`, 123 tests:
  - Each of the 13 fields is rendered with each of 8 values the reader misreads (`2026 007 1.5 true null 20261011 ~ False`), and must read back exactly.
  - 6 `scaffold-project.py` runs, then a recursive clone, `bootstrap.py`, the three validators, and `bump-leg.py`, which refuses on the tracking branch and finds the leg at its path on a branch of its own.
  - 6 `adopt-project.py` `plan`/`check`/`execute` runs into local bare remotes, then a clone, `bootstrap.py` and the validators.
  - 6 `family.py init` runs.
  - One `family.py add` of members whose ids are `007` and `true`.
  - **All 123 fail on c7c02fa.**
- `true`/`null` through adopt: `plan` writes them bare into the plan, and the plan's reader then misreads them. That belongs to the plan writer, #181 (lane 1). Those two rows quote the value in the plan by hand, as #181's writer will, and test `check`/`execute`.
- Mutation probes: reverting any one of the 13 quoted lines fails 8 cases of its field. Reverting `_member_id` fails the member test (the rows read 7/True).
- The `try/except YamlError` in `_member_id` is guarded by the follow-up commit. The review found that calling the parse directly left the member test passing. Ids such as `[`, `{`, `&a`, `*a` and `!t` reach the `except` (they raise `YamlError` bare), and `_member_id` returns each one exactly, as it does 34 adversarial inputs in all (the review's in-process probe). `test_a_member_id_the_reader_refuses_bare_is_rewritten_between_quotes` (7 cases: `[`, `{`, `&a`, `*a`, `!t`, an unclosed `'` and an unclosed `"`) opens a hand-edited holder whose row holds the id between double quotes, rewrites the `members:` block with `Family.rewritten`, and asks that the id read back exactly and the text come out byte for byte. With the `try/except` removed all 7 fail with the reader's `YamlError`, and with the `except` returning the bare text all 7 fail too; the old member test still passes under the first of those mutants.
- 14 existing tests that asserted or edited a bare rendered line now spell it quoted: `test_manifest_validator` ×5, `test_pin_validator`, `test_reference_default`, `test_scaffold_e2e` ×2, `test_setup_project_py`, `test_setup_sh`, `test_family`, `test_shape_doctor` ×2.
- Targeted runs on this branch:

  | files | result |
  |---|---|
  | new file + `test_template_quoted_values`, `test_scaffold_e2e`, `test_scaffold_pin_and_reuse`, `test_scaffold_leg_paths`, `test_scaffold_leg_path_collisions`, `test_reuse_visibility`, `test_reference_default` | 405 passed |
  | the six adopt files | 491 passed, 1 skipped (the MedxEHR clone is not set) |
  | `test_family`, `test_family_add_refusals`, `test_family_member_path`, `test_manifest_validator`, `test_pin_validator`, `test_bump_leg`, `test_update_shape`, `test_update_shape_pr_base`, `test_setup_project_py`, `test_park_resume_targets`, `test_repo_hygiene`, `test_cli_reference`, `test_setup_sh` | 603 passed |
  | `test_shape_doctor` | 241 passed, 3 skipped (Windows-only), 1 failed; that one test was fixed, and with its sibling it reran at 2 passed |

**The review.** An independent opus review at 233dc71 (the branch's first commit) found no code defect. Its verdict was CHANGES NEEDED (0 blockers, 2 should-fixes, 6 nits) because it asked for this description and its composition counts to be corrected, and they are. Its evidence:

- **Readers and rewriters.** Every reader in the shipped tree reads the 13 lines through `load_yaml`/`parse_yaml`, and no line regex or whole-line rewrite touches them. It read `bump-leg.py` `rewrite_pin`, `update-shape.py` `rewrite_shape_pin` and `rewrite_manifest`, `family.py` `rewritten`, the three validators, `siblings.py`, both `bootstrap.py`, the doctor, `setup-project.py`, the workflows, the Makefile and `setup.sh`. No test among main's 17 commits since c7c02fa (through #245's 161c6c9) edits one of these lines bare.
- **`_member_id`, the one rewriter changed.** A mutant that writes every id bare fails the member test (the rows read back as `{'IRRS': 7, 'IRSS': True}`). A mutant that quotes every id fails `b"    id: irqs" in lines`. So the minimal-churn choice is tested both ways.
- **Old projects and holders.** Against a control commit (c7c02fa plus an unrelated README edit), a project and a holder made on c7c02fa give identical `update-shape.py check` and `shape-doctor.py` output, up to the shas. `check --at 233dc71` reports all 11 project files and all 10 holder files `unchanged` and says `no copied file differs`. The doctor's verdict is `COMPLIANT, SHAPE BEHIND (... no copied file differs)` for the control and for 233dc71. This branch's `family.py add` and `bump` on a c7c02fa holder leave the old `atlas` row and the header byte for byte, apart from the legacy-id edge described at the top.
- **Tests.** All 123 new tests fail on c7c02fa (`123 failed`) and pass on 233dc71 (`123 passed`). The 14 changed existing tests pass (49 passed).
- **Not covered.** Windows and macOS were reviewed by reading and not run: the plan edit and the member-row check are in bytes, and nothing writes to git's stdin in text mode. Of `test_shape_doctor.py` and `test_family.py` the review ran only the changed tests.

**Follow-up commit.** A second commit, `3894aa8`, answers two nits from the review, and changes no behaviour. The `_member_id` docstring said "no existing holder's `members:` block moves by a byte", which was false: the next `bump`, `add` or `remove` of a holder written before this change rewrites a legacy `id: 2026` row as `id: "2026"`, harmlessly, because `validate-family.py` compares with `str()`. The docstring now says exactly that, and a grep of every comment and docstring the branch added for "by a byte" and "moves" found one more copy, in the member test's docstring, corrected the same way. The `try/except YamlError` in `_member_id` is guarded by a new test (see Tests), so the new file holds 130 tests. Every count above that says 123 (123 new tests, all failing on c7c02fa and passing on 233dc71) is for the first commit; the follow-up commit adds the 7. The `except` also carries a comment saying which ids reach it.

**Composition at the landing head (4984651).** `49846518` is GitHub's update-branch merge of origin/main `cc8f2da4` (#376, a `--trailer` value carried into the commit as typed) into `3220c9ee`. Git reported no conflict, and the merge has no hand edit. #376 changed `scripts/family.py` and `tests/test_family.py`, which this branch also changes. None of the lines it added under `tests/` names a key the templates now quote. The coordinator ran targeted tests on this head, Linux only, before CI:
- main's changed test files (`tests/test_commit_trailers.py`, `tests/test_family.py`, `tests/test_bump_leg.py`, `tests/test_update_shape.py`) with `tests/test_family_add_member_id.py`: 421 passed, 1 skipped (a case-insensitive disk only);
- this branch's template set (`tests/test_template_text_placeholders.py`, `tests/test_template_quoted_values.py`, `tests/test_scaffold_manifest_values.py`, `tests/test_bootstrap_spelling.py`, `tests/test_bootstrap_after_bump.py`) with the hygiene, CLI-reference and AGENTS.md-codes tests: 652 passed.

**Composition at 3220c9e.** `3220c9ee` is GitHub's update-branch merge of origin/main `70d0d1b3` into `e5e768bc`. It brings in three commits: #491 (da19735), in which `setup-project.py` asks at the person's own terminal; #374 (7ab11cf), update-shape's NEXT quoting; and #384 (70d0d1b), adopt's `_spelled` plan values. Git reported no conflict, and the merge has no hand edit. #491 changed `tests/test_setup_project_py.py` and `tests/test_setup_sh.py`, which this branch also changes. A grep of the lines main added under `tests/` for the keys the templates now quote found two hits, and neither reads a rendered template line: `tests/test_update_shape_next_quoting.py` writes its own `project.yaml`, and `tests/test_adopt_question_and_submodule_spelling.py` edits a plan's `- path:`/`question:` entry, which `adopt-project.py` writes bare.

The coordinator ran targeted tests on the same tree (`e5a74d19`), Linux only, before CI:
- six of main's eight changed test files (the setup, update-shape, `openRepoShape` command and doctor-row tests): 365 passed;
- main's other two (adopt's spelling and repo hygiene), `tests/test_adopt_github_exits.py` and nine more files of this branch's sixteen-file set: 943 passed.

**Composition, refreshed again 2026-10-13 (head e5e768b).** `6d108d18` merges origin/main `56713e21` into `6cbbac60`. That brings in two commits: 5e02abe, the doctor's and `validate-pins.py`'s gitlink repairs, and 56713e2 (#479), in which adopt's `plan` takes the tracking branch from the source's default branch. Git reported no conflict and `docs/cli.md` auto-merged; the merge has no hand edit. Two kinds of test failed on it, and two commits fix them:
- **`7968f902`** changes one expected literal, at `tests/test_adopt_github_exits.py:675`, for `test_a_source_on_another_branch_opens_its_pull_request_against_it[master]` and `[trunk]`.
  - The test reads the adopted root's `project.yaml`, which this branch's template renders as `tracking_branch: "master"`, so it now expects the quoted line.
  - The look-alike lines at `:743` and `tests/test_adopt_tracking_branch.py:73-76` read a plan, which `adopt-project.py` writes bare, and they stay as they are.
- **`e5e768bc`** adds `git("branch", branch, cwd=source)` to `tests/test_template_text_placeholders.py::test_an_adoption_reads_back_every_value_it_was_given`.
  - All six rows had failed at `check` with `plan-tracking-branch-missing`: #479 refuses a tracking branch (`2026`, `007`, `1.5`, `20261011`, `true`, `null`) the source does not have, and the test's source had only `main`.
  - The source now carries each row's branch, in the flag rows and the by-hand rows alike. The test's assertions are unchanged.

Targeted runs, Linux only:
- On e5e768b: main's seven changed test files, 211 passed; this branch's sixteen-file set, 1223 passed, 2 skipped.
- On 7968f90: the same sixteen files gave 6 failed, which are the six rows e5e768b fixes. The scaffold, setup, doctor and family set gave 1094 passed, 22 skipped, and twelve further template-grep files gave 1259 passed.
- `render-cli-reference.py --check` is ok.

**Composition, refreshed 2026-10-13 (head 6cbbac6).** `d651b594` merges origin/main `aa4e1f59` (#508) into `d526f768`.

- **The one conflict.** Git conflicted in one file, `tests/test_adopt_manifest_values.py`, in four hunks, all against #356 (965420b, #293/#294). #356 refuses an `elected_on:` that `election_date` does not read, as `election-date-malformed`, before the validator is asked (`_manifest_values` returns None, adopt-project.py:2258).
- **Resolved by what the merged tree prints**, measured with `adopt-project.py plan`/`check`/`execute` into local bare remotes:
  - `id: "007"` checks clean and `execute` writes it as typed, so it stays in `TEXT_THE_TEMPLATE_NOW_HOLDS`.
  - `visibility: "true"` gets the validator's `manifest-visibility: visibility is 'true'`.
  - `elected_on: "20261011"` gets #294's one finding, "the plan's `elected_on:` is the text '20261011'", and the validator is not asked. That row keeps this branch's list, `TEXT_THE_VALIDATOR_REFUSES`, with main's sentence.
  - Two tests git merged cleanly now ask `said_of`, as main's own rows do: `test_check_refuses_a_text_value_the_validator_refuses` and `test_where_a_misread_rule_judges_the_rest_the_validator_still_answers`.
- **Two more failures that no hunk showed**, fixed in the same merge:
  - `[branch-tilde]` in `tests/test_scaffold_manifest_values.py` was this branch's own `xfail(strict=True)` until #355 landed as 6c2a371, so it XPASSed. The mark is off, and the row asserts `REFUSED scaffold-bad-tracking-branch:`.
  - #391's `tests/test_bootstrap_spelling.py` and #490's `tests/test_bootstrap_after_bump.py` matched `    path: spec` and `tracking_branch: main` in a scaffolded `project.yaml`, and nine cases failed on the merge. They now match the quoted lines.
- **The #507 interplay.** `6cbbac60` switches the holder to `family/members` before the real `add`s in `test_a_member_whose_id_yaml_would_misread_is_pinned_under_that_id`, so the test holds once PR #507's `family-on-tracking-branch` refusal lands.
  - On the clean merge of 6cbbac6 with #507's head 708f3bb, that file, all 13 `tests/test_family*.py` and `tests/test_shape_doctor.py::test_a_sibling_at_the_right_origin_with_another_id_is_not_the_member` give 686 passed, 19 skipped.
  - Without the switch line, the member test fails `REFUSED family-on-tracking-branch`.
- **Targeted runs on 6cbbac6, Linux only:**
  - the template, adopt-plan and manifest files (14): 995 passed, 2 skipped;
  - scaffold e2e, setup, the doctor, leg-path collisions and every family file: 1094 passed, 22 skipped;
  - the 15 further files a grep for template lines found: 1503 passed;
  - `render-cli-reference.py --check`: ok.
- **Re-verified independently** at d651b59 and 6cbbac6: APPROVE WITH NITS (0/0/3), all three in this paragraph and fixed here. The re-verification also ran the three rows, five mutations, and 26 more files (834 passed; 291 passed, 1 skipped).
- **Not run locally:** the full suite, macOS and Windows; CI's three jobs are the check.

**Composition, as measured on 2026-10-12.** The merge trees below were computed for 233dc71; the follow-up commit 3894aa8 changes only `scripts/family.py` comments and adds tests. At 15:28Z main was 3ae0109 (#245, #351, #350 and #345 had landed since the merge trees): 3894aa8 merges clean with it, and the review's re-verify ran the family tests on that merge (61 passed). Every sha below is the one it had then; main and the other branches have moved since.

`git merge-tree --write-tree --name-only`, run after a fetch with automatic maintenance off on 2026-10-12:

- **origin/main 161c6c9**, 17 commits past c7c02fa (the base): clean, tree 7ba8246. #316 (34ecbc2), #317, #318, #311 and #245 were on main by then.
- **The 46 other remote branch heads** (the 30 open PR heads and 16 branches with no open PR): clean for 44. Among them #314 (4b8da3d), #266 (3410be7), #355 (a03abcf), lane 1's `fix/adopt-plan-quote-yaml-keywords` (d38e143), `fix/adopt-record-submodule-placement` (70f4d29) and `fix/adopt-late-failure-notice` (9f1778d), #380 (73068a3), and `fix/family-init-values` (14e2d68, #307). The two that conflict, each in the same file as it does with c7c02fa and with origin/main on its own: lane 1's `fix/adopt-root-kept-symlinks` (6c44591, `adopt-project.py`) and #358 `fix/doctor-manifest-kinds-unreadable-pin` (ebffb08, `shape-doctor.py`). This branch touches neither file.
- #314 and d38e143 each conflicted with origin/main on their own, in `adopt-project.py`. Merged with this branch, neither gained a conflict.

**Composition with #314, which lands before this branch.** Its misread gate (`manifest_misreads`) reads the template, so once the template quotes a field it refuses nothing there, and some of #314's tests expected that refusal.

Composing with #314 (landed as 3ef059c), whose misread gate asks the template how a value is written. With `project.yaml` quoting `id`, `elected_on`, `visibility` and `tracking_branch`, the gate refuses nothing on the shipped templates, and 45 of #314's tests flipped on the merge. Forty expected a refusal the value no longer needs. Thirty-four of them now assert that the value is accepted, and the six whose premise was a bare placeholder now assert that no shipped template placeholder is misread. For example: a scaffold of `--id null|true|false|007|2026`, or of `--tracking-branch true|True|TRUE|false|False|null|Null|NULL|007|2026|1.5|1.50|0`, exits 0, and the manifest holds the text it was given. A dry run prints the plan. An id derived from `--project True|Null` is held as typed. `--id=null` reaches `gh repo create`. Adopt's `id: "007"` checks clean and `execute` writes it as typed. The rule's own sentence, remediation and line-break wording are still asserted, on a copy of the standard whose `project.yaml` writes the four lines bare. Four stay refused by #314's validator gate, now in its words about the text: `visibility is 'true'` and `elected_on is '20261011'`. One is a real break. `--tracking-branch=~` reads back as `~`, which git refuses as a branch name, and it dies at `git init -b ~`, after `gh repo create`. That is #283, so the row is `xfail(strict=True)` with that reason, and it turns XPASS (and fails) once PR #355's `git check-ref-format --branch` refusal lands. `--help` for `--id` and `--tracking-branch` no longer says such values are refused, and `docs/cli.md` is regenerated.

**Composition with the other branches.** Where another branch's rules meet this one, the tests were run on the merged tree. Quoting changes what some of their tests see, and the changes are not all one kind: some tests expected a refusal the value no longer needs, some are still refused under another rule, and some guard a case quoting does not fix. Whoever lands second must tell them apart, so each branch's flips are classified below.

- **#266** (lane 2, `fix/leg-path-yaml-scalars`). 27 flips, all from one cause: its rule asks a hard-coded bare line, `LEG_PATH_LINE = "path: {}\n"`, so it refuses leg paths the quoted templates now read back. 15 are `test_the_rule_agrees_with_every_template_line_that_holds_a_leg_path` (null, Null, NULL, ~, true, True, TRUE, false, False, FALSE, 2026, 007, 0, 1.5, 1.50) and 12 are this branch's own end-to-end tests (6 scaffold rows fail with `REFUSED scaffold-bad-leg-path`, 6 adopt rows with `FINDING plan-bad-leg-path`). Counted on the merge with b2b42f8. #266's head was then 3410be7, whose rule reads each template line that holds a leg path, renders the value into it as `render` does and asks the reader, so it has no hard-coded line left. The merge with this branch was clean (tree 1e348c6); that head was read and the 27 were not re-run on it. At b2b42f8, whichever of #266 and #259 landed second had to drop or re-point the bare rule; 3410be7 re-points it already.
- **Lane 1's #181** (`fix/adopt-plan-quote-yaml-keywords`, d38e143). Its gate reads the template too, so `plan --tracking-branch 2026|007|true` exits 0 on the composed tree (exit 2 `adopt-manifest-misread` on d38e143). On the merge, `tests/test_adopt_plan_yaml_quoting.py` and this branch's 6 adopt end-to-end tests give 70 failed, 371 passed:
  - 57 refusals that are now unnecessary: `plan_refuses` ×14, `plan_names_an_id` ×2, `check_names_an_id` ×4, `check_reports` ×14 and `execute_refuses` ×14 (every value except `elected_on` and `visibility`), `text_the_loader_reads_as_other_text` ×7, and the two tests that need a bare placeholder (`test_the_gate_follows_the_loader_when_the_loader_learns_a_keyword`, `test_the_template_line_says_whether_a_value_is_quoted`).
  - 11 guards #181 loses. See "Do not flip".
  - 2 that fail only by cascade: `execute_refuses[pins='null']` and `[pins='True']` pass alone (`2 passed`). An earlier case no longer refuses, and its `execute` pushes the split branch.
  - The writer half of #181 (`y` asking the reader, so `true`/`null`/`True` are quoted in the plan) is still needed and composes cleanly. With it, `plan --tracking-branch true` reaches the quoted template intact, and this branch's two hand-quoted adopt rows become flag-driven.
- **Do not flip these 12 to "accepted".** Each still needs a refusal or a finding, and quoting does not supply it:
  - **`~` (1, #314).** `test_a_value_the_reader_misreads_is_refused_before_anything_is_made[branch-tilde]`. `--tracking-branch=~` is not a branch name git accepts, and #314 refused it only as a misread. On the merge it passes the plan and dies at `git init -q --bare -b ~ .../Atlas-spec.git` with `fatal: invalid initial branch name: '~'`, and against GitHub that is after `gh repo create` has made all three repositories. That is lane 2's #283 (PR #355). `~` must stay refused, by #283's ref-name check, which is why #314's row for it is not flipped to "accepted".
  - **A hand-edited plan value that is not text (7, #181).** `test_check_names_a_plan_value_that_is_not_text_by_what_was_read[tracking_branch='007']`, `[tracking_branch='TRUE']`, `[tracking_branch='1.50']`, `[tracking_branch='[a]']`, `[visibility='{a: b}']`, `[id='007']` and `[elected_on='20260902']`. The plan's reader reads each as something other than text, and on the merge `check` passes. Measured on the merge: `tracking_branch: 007` and `id: 007` are adopted silently as `"7"` with every validator passing; `visibility: {a: b}` pushes both legs and then fails `manifest-visibility`; `tracking_branch: [a]` is still refused at `execute` (`unsafe-value`) before any leg exists. On d38e143 alone, `check` exits 1 `plan-manifest-misread` for each of those four. Re-point the gate at "a plan value that is not a string" and these keep their guard.
  - **A quoted value the validator refuses (4, #181).** `test_check_reports_a_plan_carrying_one[elected_on='20260902']` and `[visibility='true']`, and `test_execute_refuses_a_plan_carrying_one_before_any_leg_exists[elected_on='20260902']` and `[visibility='true']`. The value stays invalid for the validator (`DATE_RE`, the visibility choices), and #181's misread gate caught it only incidentally. On the composed tree `check` passes and `execute` pushes the legs, and the gate fails afterwards. That is what main does today, and #314's validator gate (`plan-manifest-invalid`) would catch it.
- **main** (merge with ab623dd, tree cdeec7f, run by the review): this branch's new file, `test_adopt_template_values.py` (#270), `test_template_values_single_pass.py` (#316) and `test_template_quoted_values.py` give 431 passed, 1 skipped. The skip is `test_adopt_template_values.py:223`, which reads `git config user.name` and skips where the shared repository's `.git/config` sets one; it is environmental. The changed tests with `test_family_add_refusals.py` (#317, #318), `test_pin_validator.py` and `test_manifest_validator.py` give 132 passed. #270's `unwritable_reason` is value-based, not template-relative, so it refuses as it did and nothing flips. Main then added #245 (`adopt-project.py` and `repo_shape.remote_key`), which adds no test that edits a bare quoted line; it was not run on that merge.
- **#380** (lane 3, `fix/pin-rewrite-indentation`; it landed since as 1f90a0f). It rewrites `commit:` and `tree_sha256:` only, and its tests render the pins from the templates. On the merge at dc0fa7b, `test_pin_rewrite_layout.py` and the 6 scaffold end-to-end tests gave 121 passed. Its head was then 73068a3; that merge was clean and was not re-run.
- **#397** (`fix/family-init-values`, #307; measured at 14e2d68): its tests skip themselves where the template quotes. 137 passed, 19 skipped, 0 failed. With its fix-round head abb27e6, this branch's 3894aa8, main 0f67f7b and #345 together: 442 passed, 20 skipped (the 19 rows, and one Windows-only row).
- **#345** (run at 875d802; it has since landed as 3ae0109): `test_family_dry_run.py` and the member test, 22 passed. Its `"    id: irrs"` assertion holds, because ordinary ids stay bare.

**Left out.**

- The adoption plan writer `y` still writes `true`/`null` bare into the plan. That is #181 (lane 1).
- `templates/assembly-root/contracts/neutral-product-pin.yaml` still writes `product`, `source_repository` and `digest_source` bare, and `family.py` writes a member row's `project:` bare. Neither is a #259 field.
- A plan value someone edits to a non-text type (`tracking_branch: 007` or `id: 007`, unquoted, in the plan) is read by the plan's reader as the number 7. `str()` makes it `"7"`, and the quoted template now writes it faithfully. For `id` this is a change: on c7c02fa a bare `id: 007` failed loudly (`validate-manifest: manifest-id`, manifest line `id: 7`), and on this branch it is adopted silently as id `"7"` and every validator passes. `tracking_branch: 007` was already adopted silently as `7`. The plan side is not this branch's: the guard against it belongs with #181 (adopt plan writes YAML keywords and non-text values unquoted), whose gate should refuse a plan value that is not a string. The case is narrow, because `plan --id 007` itself writes `id: "007"`; only a hand edit reaches it.
- No docs change: README, AGENTS.md, the handbook and the template comments show none of these lines; `docs/cli.md` is unchanged because no `--help` text changed.

Reviewed adversarially at 233dc71 (CHANGES NEEDED: 0 blockers, 2 should-fixes, 6 nits; the code and the tests were sound, and the should-fixes were the composition counts and the claim about a hand-edited plan `id`). Fix round 3894aa8 changes no behaviour: it corrects the description and the two false "moves by a byte" claims, and guards `_member_id`'s `except YamlError` with a test of 7 cases. Its re-verification at 3894aa8 found every finding handled: APPROVE WITH NITS (0 blockers, 0 should-fixes, 1 nit: shas that had moved since the description was written, refreshed here). What was added after that re-verification, main and the composition with #314, is described under "Composition with #314". An independent re-verification of that composition at 77cd0b2 confirmed it with nits only (0 blockers, 0 should-fixes, 5 nits): the 45 flips reproduce test for test, breaking the real rule or the real template's quoting turns the compose tests red, and the `[branch-tilde]` row XPASSes on a merge with #355.

**Composition with #381.** #395's quoted `id: "{{PROJECT_ID}}"` composes with #381's `member-duplicate-id` test (`tests/test_family_add_member_id.py`), whose id rewrite now matches the id bare or quoted and keeps its quoting, so the full suite, which failed that one test on Linux at 1ef3310, no longer fails `test_a_member_whose_id_changes_between_the_probe_and_the_mount_is_unwound` on Linux.

CI is green on Linux, macOS and Windows, with SonarCloud passing, at 4984651 (main cc8f2da merged in by update-branch). Main af01068 (#505), which landed after, touches none of its files, and the combined tree passed the template, hygiene, CLI-reference, AGENTS.md-codes and setup tests (807 passed).

Closes #259

Lane: openreposhape-5 (openRepoShape-5)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment