Repository navigation
scaffold-project.py and adopt-project.py ask the project's YAML reader and validate-manifest.py about the manifest's values before anything exists, so --id null, --tracking-branch true and a blank --elected-by are refused (#258, #261) - #314
Merged
Merged
Conversation
… new project's manifest are put through its own YAML reader and its own `validate-manifest.py` before anything is created, so `--id null` and `--tracking-branch true` no longer scaffold a project that reads them back as None and True, and a blank `--elected-by`, `--elected-on 2026-W41-6` or `--org test+org` no longer ends a scaffold or `adoption verified` over a project whose first bootstrap fails (#258) (#261) `project.yaml` writes `id:` and `tracking_branch:` bare, so `scaffold-project.py --id null` and `--id true` made three repositories whose first bootstrap failed `manifest-id`, and `--tracking-branch true`, `007` or `null` pushed every repository on the branch as typed while bootstrap read the manifest back as `True`, `7` or `main` and put each leg on a local branch the remotes do not have, with no upstream, exit 0. Both tools also accepted what the project's own `validate-manifest.py` refuses: a blank `--elected-by`, `--reference` or `--name`, `--elected-on 2026-W41-6` or `20261011` (which `date.fromisoformat` reads from Python 3.11), `--org test+org`, and a plan edited to `visibility: secret` or `elected_by: null` (written as `elected_by: ""`). A `--project` of 247 to 250 characters died in `Path.exists` with `File name too long` under `--local-remote-dir`, and `plan` admitted 251 to 255 that `check` refuses. `shape_materialize` now asks the two programs the project will use. A value is refused, never retyped, as #225, #181 and #251 refuse: - `manifest_misreads` renders each template line that carries a value and reads it with the project's reader (`parse_yaml`); the template says which placeholders are bare and which quoted, so quoting them (#259) later makes the refusal go away with no edit. The scaffold asks it (`scaffold-manifest-misread`); adopt's is #181's. - `manifest_invalid` renders `project.yaml` from the whole table and runs the validator the template ships, loaded from that file and not re-implemented, with no bytecode written into the template tree. The scaffold (`scaffold-manifest-invalid`), `adopt-project.py plan` (`adopt-manifest- invalid`) and `check` (`plan-manifest-invalid`, so `execute` refuses it through the one list, before a leg) say what it says, naming the value. - `election_date` admits only what the validator's `DATE_RE` admits, on every Python. - `refuse_unstorable_names`: a name is a directory's too, at most 255 bytes, `.git` included under `--local-remote-dir`. Refused rather than quoted in the templates: one decision for every tool that renders them is #259's, and numeric branch names (`2026`, `1.5`) that worked by accident are refused here as `adopt-project.py plan` refuses them on #181. Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…licy does not classify is `check`'s naming finding and no crash, and no second finding about a manifest nobody could write (#261) The question asked of a plan's values renders the manifest from them, which needs names that classify. `_manifest_values` returns None for those, and `check` reports them in its own words; this holds that. Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… as `plan-bad-leg-path` and again as the validator's `manifest-leg-path`, so a plan with `spec_path: ../spec` is one finding as it was (#261) The question asked of a plan's values ran the validator over the whole manifest, which also refuses a `path:` that is absolute, has a `..`, is `.` or is empty. `_leg_findings` already reports each of those as `plan-bad-leg-path` in the same list, so `tests/test_adopt_leg_paths.py` (twelve rows of `test_check_reports_a_non_canonical_path_as_a_finding`) saw two findings for one value. `manifest_invalid` takes the codes a caller reports by a rule of its own and `check` and `execute` pass the two leg-path codes. The scaffold and `plan` judge a leg path before the manifest is rendered and ignore nothing, so a rule they did not apply would still refuse. The two tests that copy the template tree now leave out `__pycache__`, which a stale one in the source tree would otherwise put in the copy. Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lan's values that are text, or no value, and leave a value the reader read as a number, a boolean or a list to the misread rule, so a plan's `id: 007` is not echoed as `'7'` (#261) `execute` writes `str` of what the plan's reader read, so `id: 007` (the number 7) or `elected_on: 20260902` reached the validator as text the plan never said, and the finding echoed it (`id is '7'`). A value read as another kind is a misread, which `adopt-project.py` on #181 judges in words that name what was read and never echo the retyped text, and a merge of the two put both findings on one value and failed that branch's own test of the echo. The values asked about are the ones the plan holds as text, and as no value at all (`elected_by: null` is written blank, which is the finding wanted). The plan edits that make a bad `elected_on` are quoted in the tests, so they are text on every tree. Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the validator's finding and not for the code of the first finding in the list, so a value a second rule also judges does not move it (#261) `execute` refuses under the code of the first finding `check` prints (#168). A plan's `elected_on` of `"20261011"` is also a misread on #181, which lists its finding first, so the refusal wore that code on a merge of the two. What the test is about is that this finding is in the refusal. Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One conflict, at the tail of `_entry_findings` in adopt-project.py: main's `_unsafe_path_findings` (#196) adds a remediation and this branch's `_manifest_findings` (#258) adds a findings group and a remediation. Kept both: `findings + value_findings + submodule_findings` and `unsafe_remediation + value_remediation + remediation`. Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…isread` says what holds of every value it refuses, that the project's reader reads back something other than what was typed, and no longer says `--tracking-branch 2026` or `1.5` fails the first bootstrap or lands the legs on a branch the remotes lack (#258) The review of ca21b57 found the text false for exactly the values this refusal newly takes from people. On main both scaffold, bootstrap with exit 0, and get the upstream origin/2026 or origin/1.5, because every reader of `tracking_branch` does `str()` and `str(2026)` is `2026` again. The refusal stays, as #181 and #266 refuse the same class and quoting the templates is #259's to decide; its reason is now the true one. It names the failures that do hold (an id that is not text fails validate-manifest.py, `007` as a branch reads back as `7`), says that a number which survives `str()` works only because of the tools that read it and is still a number to the project's own reader, and gives the way forward: a value the reader takes for text. Tests: the wording is asserted for `--tracking-branch 2026`, `1.5` and `007` and `--id 2026` and `null` (five rows that fail on the old text), and the claim about `str()` is asked of the project's own reader. Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…is missing, broken or raising is the named refusal `manifest-validator-unusable` and not a traceback, `check` leaves a value a misread rule judges to that rule once one is in the list, a name too long is shown at both ends, a placeholder is what `PLACEHOLDER_RE` finds, and the other branch of `_line_shown` is tested (#258) (#261) Four findings of the adversarial review of ca21b57, each small. SHOULD-3, the double finding with #181. `_manifest_findings` takes `misreads_elsewhere` (default False). Set, the placeholders `shape_materialize.misread_placeholders` names, which are the ones `manifest_misreads` refuses, are not asked of the validator, so `elected_on: "20261011"`, `visibility: "true"` and `id: "007"` get one finding from a list that also holds the misread rule's, and not a pair that says the retyped text (`id is 7`). It is False here, because the review's reading (always leave such a value to the misread rule) makes this branch alone accept `id: "007"` at `check`, where it is refused today: six tests of this branch fail when the parameter is made unconditional. Whoever lands second passes it True where the two lists meet; nothing is lost in either order. NIT-7. `manifest_validator` loads the template's `validate-manifest.py`, and `manifest_invalid` asks it. A missing, unreadable or broken file, one that is missing what is asked of it (`DATE_RE`, `_findings`), or an exception from the question was a traceback and exit 1 (a validator without `DATE_RE` was an `AttributeError` in `election_date`). Each is now `REFUSED manifest-validator-unusable`, naming the file and saying the copy of the standard is the defect, before anything is created. A failed load is not cached. NIT-3. `scaffold-name-too-long` cut each name at its first 24 characters, which lost `-spec` and `-code`, so a 251-character project read as two identical pairs of sentences. A name is shown at both ends. NIT-5. `_PLACEHOLDER_KEY_RE` was a second copy of `PLACEHOLDER_RE`'s alphabet, which #260 widens to digits. `_placeholders_of` reads a line with `PLACEHOLDER_RE` itself, so the two cannot drift. NIT-4. The non-printable branch of `_line_shown` was guarded by no test (a mutant that always used backticks survived 84 tests); it is now, for a vertical tab, a file separator and a line separator. Tests: each of the above fails when its change is reverted (ten mutants, all killed, in the report). Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… (`id: "007"`, `visibility: "true"`, `elected_on: "20261011"`) is asserted to be ONE finding in the list `check` prints, and `check` and `execute` are asked only that they refuse it, so a merge with #181 that does not pass `misreads_elsewhere` fails here and one that does edits no test (#261) The first cut pinned the validator's own finding for `elected_on: "20261011"` in the rows every plan edit is run through, and the unwired call site in a test of its own. A throwaway merge with #181 (e337f44) that passed `misreads_elsewhere=True` then failed three tests of this branch for the reason the parameter exists: the value is now the misread rule's finding. Now the three values are rows of one list, asked at three levels: `_manifest_findings` (this rule alone: one finding in the validator's words; `misreads_elsewhere`: none), `check` and `execute` (refuse the value and name it, whichever rule says so), and `_entry_findings` (exactly one finding names the value, not zero and not two). On this branch alone all of them hold. In the throwaway merge with #181 they hold with the call site passing `misreads_elsewhere=True`, and the third fails for each of the three values, two findings and not one, with the plain keep-both resolution: the test is the reminder to whoever lands second. `elected-on-basic-format` leaves PLAN_EDITS, the rows that name the validator's words in `check` and `execute`; the same value is a row of the new list. Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
(lane openRepoShape-5, 2026-10-12T01:43Z) @codex review |
|
Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days. You can request another review in 4 days and 21 hours by commenting |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Reviewer's GuideThe PR adds a shared, template-driven preflight layer that asks the project's actual YAML reader and validate-manifest.py whether generated values are representable, then applies consistent refusals and findings in scaffold and adopt before any plan, directory, remote, or leg repository is created; it also adds filesystem-name checks, CLI documentation, and extensive regression coverage. Sequence diagram for scaffold manifest preflightsequenceDiagram
participant User
participant Scaffold as scaffold-project.py
participant Materializer as shape_materialize.py
participant Reader as Project YAML reader
participant Validator as Template validate-manifest.py
participant Filesystem
participant Remote
User->>Scaffold: run with manifest values
Scaffold->>Materializer: refuse_manifest_values(values)
Materializer->>Reader: manifest_misreads(values)
Reader-->>Materializer: parsed template values
Materializer->>Validator: manifest_invalid(values)
Validator-->>Materializer: validator findings
alt value cannot be represented
Materializer-->>Scaffold: Refusal(scaffold-manifest-misread or invalid)
Scaffold-->>User: refuse before creation
else values accepted
Materializer-->>Scaffold: preflight passed
Scaffold->>Filesystem: create work directory
Scaffold->>Remote: create repositories
end
Sequence diagram for adopt plan and execute validationsequenceDiagram
participant User
participant Adopt as adopt-project.py
participant Materializer as shape_materialize.py
participant Validator as Template validate-manifest.py
participant Plan
participant Repositories
User->>Adopt: plan or execute
Adopt->>Plan: _flag_plan(args, names, pins)
Adopt->>Materializer: _manifest_values(plan, naming)
Materializer->>Validator: manifest_invalid(project.yaml)
Validator-->>Materializer: FINDING entries
alt plan values invalid
Materializer-->>Adopt: manifest_value_problems
alt plan command
Adopt-->>User: refuse adopt-manifest-invalid
else check or execute
Adopt-->>User: plan-manifest-invalid FINDING
end
else values accepted
Materializer-->>Adopt: valid manifest values
Adopt->>Repositories: create or update leg repositories
end
Flow diagram for manifest and name preflight decisionsflowchart TD
Start[Input values and project names] --> Names{Names fit filesystem limits?}
Names -->|No| NameRefusal[Refuse scaffold-name-too-long or adopt-name-too-long]
Names -->|Yes| Render[Render template manifest]
Render --> Read{YAML reader returns identical text?}
Read -->|No| Misread[Refuse manifest-misread]
Read -->|Yes| Validate[Run template validate-manifest.py]
Validate -->|Validator unavailable| ValidatorRefusal[Refuse manifest-validator-unusable]
Validate -->|Findings| Invalid[Refuse or report manifest-invalid]
Validate -->|Pass| Create[Proceed with plan, directories, or repositories]
File-Level Changes
Assessment against linked issues
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
This was referenced Oct 12, 2026
Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t the manifest with a fixed, well-formed commit and digest and run no `git`, so a checkout that cannot name its revision is asked the question and not skipped in silence, and SonarCloud's `pythonsecurity:S8705` on the `git rev-parse HEAD` this branch added to `_manifest_values` has nothing left to flag (#261) `_manifest_values` rendered the manifest FOR VALIDATION with the standard's real commit, read with `git rev-parse HEAD`, and (through `_template_values`) the real digest of its tree, read with `git ls-tree -r`. The validator the template ships (`validate-manifest.py`, `_shape_findings`) holds `shape.commit` to `COMMIT_RE` and `shape.digests.tree_sha256` to `SHA256_RE` and compares neither with anything, `manifest_misreads` is asked only about the placeholders a tool names to a person (`MANIFEST_FLAGS`, `MANIFEST_PLAN_KEYS`, neither of which holds either one), and `manifest_invalid` passes the values through. So the two reads bought nothing, and where `git` failed (a `git archive` copy, a downloaded zip) the function returned None and `plan` and `check` skipped the whole question without a word: `plan --elected-by ' '` exited 0 and wrote the plan (NIT-9 of the review). Now the table is built by `_template_table`, which is handed its commit and digest and runs nothing, and `_manifest_values` hands it two module constants, `_VALIDATION_SHAPE_COMMIT` and `_VALIDATION_SHAPE_DIGEST`: lowercase hex of the right length with a letter in it, which the project's own YAML reader reads as text whether the template quotes it or not. The `Refusal` fallback is gone, so `plan` and `check` always ask. No new function of this branch reaches `run`, `git_out`, `tree_digest` or `subprocess` any more (the call-graph of the functions this branch adds or changes, before and after, is in the report). `execute` is untouched where it matters: it still reads `git rev-parse HEAD` and calls `_template_values`, which is `_template_table` with the real commit and `tree_digest(SHAPE_ROOT, commit)`, so what it writes into `project.yaml` and the two leg pins is what it wrote before. `test_execute_still_renders_ the_real_commit_and_digest` stops `execute` where the legs would be built and asserts the table it holds carries the standard's real HEAD and the real tree digest and neither constant. tests/test_adopt_manifest_values.py: the test that asserted the old behaviour (`test_a_checkout_that_cannot_name_its_revision_skips_the_question`) is replaced by 16, of which 14 fail on the previous head and pass now: - `plan` refuses a blank `--elected-by` or `--reference` and an `--org` of `test+org`, and `check` finds each of the seven plan edits, run from a copy of the tool tree with NO `.git`, with the same REFUSED and FINDING lines as the same tool in a checkout; a good plan still plans and checks `plan ok` there; - with `git_out` and `tree_digest` raising, `_manifest_findings` still finds the value, and with every way of starting a process (`subprocess.run`, `Popen`, `check_output`, `check_call`, `call`, `os.system`) raising it still answers; - the constants are 40 and 64 lowercase hex with a letter, read as text by the project's reader quoted or bare, and are what `_manifest_values` holds; - the validator gives the same sentences for any well-formed commit and digest, refuses `not-a-commit` for its form, and neither value is a placeholder either tool names to a person. Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…IFEST_FLAGS` names `_template_table` as what builds the table, the two test files lose a lane-private token and eight composite assertions, and `scripts/validate-manifest.py` is spelled once in `shape_materialize` (#261) Review round 3 of this branch found four small things and no defect in the code. None of this changes what any tool does. - adopt-project.py: the comment above `MANIFEST_FLAGS` said the plan is read "the way `execute` reads it (`_template_values`)". `_template_values` is now the wrapper that reads the real commit and digest; the table is built by `_template_table`, which both it and `_manifest_values` call. The comment says that. - tests/test_adopt_manifest_values.py: a section comment and a `mktemp` prefix named review-round identifiers that mean nothing in the repository. The section comment now says what the section tests, and the temporary directory is `no-git-standard`. - SonarCloud `python:S9073` (split a composite assertion): `a and b` is now `assert a` and `assert b`, with the same operands and the same failure message, at five places in tests/test_adopt_manifest_values.py and three in tests/test_scaffold_manifest_values.py. - SonarCloud `python:S1192` in scripts/shape_materialize.py: the literal "scripts/validate-manifest.py" was spelled in `COPIED_VERBATIM`, in `EXECUTABLE` and in `MANIFEST_VALIDATOR`. `MANIFEST_VALIDATOR` is now defined once, beside `BOOTSTRAP` and the other path names those two lists share, and both lists use it. Its value is unchanged (checked against the previous module), and `run()` is not touched. Tests (venv with git-filter-repo 2.47.0 on PATH): tests/test_adopt_manifest_values.py, tests/test_scaffold_manifest_values.py and tests/test_scaffold_leg_path_collisions.py 289 passed; tests/test_manifest_validator.py, tests/test_repo_hygiene.py and tests/test_cli_reference.py 102 passed; tests/test_update_shape.py and tests/test_family.py, which use the moved constant, 176 passed. Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This was referenced Oct 12, 2026
brettheap
added a commit
that referenced
this pull request
Oct 12, 2026
…ly a placeholder its table lacks, so `--elected-by '{{SPEC_PATH}}'` is no longer recorded as `spec` and `{{NOPE}}` no longer refuses after the legs are pushed (#260) (#316)
**What was wrong (#260).** `render` replaced the placeholders one key at a time, each over the whole text. The value an earlier key put in was then searched by every later key. `--elected-by` and `--reference` in both tools, the scaffold's `--name`, and `family.py init`'s `--name` and `--created-by` are free text, so each could hold another placeholder's token:
- A token for a key that came later was expanded. `adopt-project.py --elected-by '{{SPEC_PATH}}'` recorded `elected_by: "spec"` in `project.yaml` and the README, and said `adoption verified`.
- A token for a key that came earlier (`{{PROJECT}}`), or one no key names (`{{NOPE}}`), stayed in the output. `render` then refused it as `template-unsubstituted`, with a remediation that blamed the tool. By then `execute` had pushed both legs, the scaffold had created all three repositories, and `family.py init` had created the holder's bare repository.
Reproduced on b30595c with the issue's own script, and the same for `family.py init --created-by`.
**The change (`scripts/shape_materialize.py`).**
- `render` reads the template once. One regular-expression substitution looks each placeholder up in the table, so a value goes in where its placeholder stood and is never searched again. In a `.yaml`/`.yml` template the quoted form `"{{KEY}}"` is matched first and spelled by `double_quoted`, exactly as #284 made it (`YAML_PLACEHOLDER_RE`, group 1).
- The refusal asks the template, not the output: it names the placeholders the template holds that the table has no value for, and it is raised before anything is substituted. That is the defect its remediation describes, so the remediation is now true. The output now holds the person's text, so it is no longer searched. Message and code are unchanged: `REFUSED template-unsubstituted: <source>: no value for ['{{NOPE}}']`.
- `PLACEHOLDER_RE` now takes digits, `\{\{[A-Z0-9_]+\}\}`, a superset of the old `[A-Z_]+`. The four digest placeholders (`{{SPEC_TREE_SHA256}}`, `{{CODE_TREE_SHA256}}`, `{{SHAPE_TREE_SHA256}}`, `{{PIN_TREE_SHA256}}`) were substituted only because the loop replaced every key by name. The single pass replaces only what the pattern finds, so it needs the digits. A digest placeholder that a table lacked used to be left in the output with nothing said; it is now refused like any other.
**Nothing downstream renders these files again,** so a `{{` a person typed is never expanded later, and no tool needs to refuse `{{` in these values. On this head:
- `git grep -n -E '\brender\(' -- '*.py' ':!tests'` lists `scripts/shape_materialize.py` alone: `copy_tree` and the three calls in `_materialize`. (`scripts/render-cli-reference.py` has its own unrelated `render`.)
- `copy_tree`, `materialize_assembly_root` and `materialize_family_root` are called only from `adopt-project.py execute` (`:2549`, `:3101`), `scaffold-project.py` (`:1013`, `:1054`) and `family.py init` (`:712`). Every one of those calls runs while the tool is creating the repositories.
- `update-shape.py` imports no renderer and no `TEMPLATED` list. In `project.yaml` / `family.yaml` it rewrites only the `shape:` block's `commit:` and `tree_sha256:` lines (`rewrite_manifest`, `:869`). The README's trailing `Shape:` line is a span rewrite of the commit.
- `bump-leg.py` rewrites only a leg's two pin lines.
- `shape-doctor.py` imports only `escape_what_stdout_cannot_encode` and `lane_trailer_argument` from the materializer. It reports a leg's `AGENTS.md`/`README.md` as `rendered` and compares nothing.
- Neither `bootstrap.py` nor any shipped validator holds `{{`: `git grep -n -F '{{' -- 'templates/*.py'` finds only `^{{commit}}` in two f-strings.
- `shape_advisory.py`'s own `PLACEHOLDER_RE` is asked only of `single-repository.yaml` (`field_problems`, called only from `read_record`). That is a record a person writes by hand, never `project.yaml`.
**Every caller, and what it relies on.**
- `adopt-project.py`: `_template_values` (`ORG`, `REFERENCE`, `ELECTED_BY`, and so on) is rendered into seeded legs and the assembly root. It relies on every key a template names being in the table (it is: the four leg commit/digest keys are `""` until the legs exist). Its free text is now written as typed.
- `scaffold-project.py`: the `values` table, plus `_seed_legs`' four leg values and `--pin`'s `PIN_*` values for neutral pins. Same reliance; the probe after this list names the keys.
- `scripts/family.py`: `_template_values`. `FAMILY` is a key no template names, and that is harmless.
- `setup-project.py` renders nothing. It forwards `--elected-by` and `--name` to the scaffold, and its own `NAME_RE` already refuses `{`.
- `update-shape.py` and `shape-doctor.py` render nothing (above).
- Tests: `test_adopt_leg_path_collisions.py` and `test_scaffold_leg_path_collisions.py` build a table from `PLACEHOLDER_RE.findall` over `templates/assembly-root/`. They now include the digest keys and still pass. `test_shape_advisory.py` renders leg `AGENTS.md` and the record template, which have no digest keys. `test_scaffold_ignored_shape_files.py` calls `copy_tree`. `tests/test_template_quoted_values.py` built its tables with its own `[A-Z_]+`, so its tables lacked the digest keys and its own `"SHA256" in key` branch never ran. It now uses `[A-Z0-9_]+`, and #284's behaviour is otherwise held unchanged.
A probe (`ast` over each tool's dict literals, against every placeholder under `templates/`) finds that every capitals key of the scaffold's and adopt's tables is named by a template, except `ORG`, which only `family-root` and `workspace-root` name. Every key of family's table is named by a template, except `FAMILY`.
**Byte-for-byte for ordinary values.** Every text file under `templates/` (49) was rendered with b30595c's `render` and with this head's, over two tables: plain values, and values holding `"`, `'`, `\`, `#`, `[x]`, `{y}`, U+3000 and a tab. That is 0 differences in 2 x 49 files. `test_an_ordinary_value_renders_as_a_plain_substitution` keeps this as a test for the 19 templates that hold a placeholder.
**Tests (`tests/test_template_values_single_pass.py`, 133).**
- `render`: each key's token (every key of the three tools' tables and every key a template names, 49 keys) and `{{NOPE}}`, bare and between double quotes, as every free-text value. Each reads back unchanged from `project.yaml` and `family.yaml`. Every README and `AGENTS.md` holds it exactly where it holds an ordinary value. Neither of two keys reads the other's value, in either order. A placeholder the template lacks is still refused, naming only the template's missing tokens and never a value's; this includes a digit key. `${{ github.x }}` and an f-string's `{{commit}}` are left alone.
- End to end, into local bare remotes, with a `gh` on PATH that records any call (none is made). What was pushed is read back with `git show`, so no checkout's line endings are compared:
- `scaffold-project.py --name/--elected-by/--reference`, each holding every key's token (bare and quoted), and each as `{{NOPE}}`. The run exits 0. The pushed `project.yaml`, README and both legs' `AGENTS.md` hold the values as typed, and the project's own `validate-manifest.py` passes.
- `adopt-project.py plan` / `check` / `execute` with `--elected-by` and `--reference` the same way. All three pass, with `adoption verified`. The split commit's `project.yaml` and README hold the values as typed. This test is skipped at its decorator without `git filter-repo`.
- `family.py init --created-by` the same way. `family.yaml` holds the value, and `validate-family.py` passes. (The family's `--name` is held by the unit tests only. Its validator requires the holder's name to be one CamelCase token, so a free-text name is refused there whatever `render` does. See the last item under Left out.)
On b30595c this file is 113 failed, 20 passed. The 20 are the byte-for-byte guard (19), which describes base behaviour by design, and the sanity test that the templates are found.
Mutation probes, each a revert of one part of the fix: the sequential loop with the template check (98 failed); single pass with the output check (109 failed); `PLACEHOLDER_RE` without digits (69 failed, the scaffold e2e among them, because the pins keep `{{SHAPE_TREE_SHA256}}`); no quoted branch (94 failed, #284's tests among them); no template check (4 failed).
Targeted runs on this head: `tests/test_template_values_single_pass.py tests/test_template_quoted_values.py tests/test_scaffold_e2e.py tests/test_scaffold_pin_and_reuse.py tests/test_adopt_plan.py tests/test_adopt_execute_refusals.py tests/test_adopt_e2e.py tests/test_family.py tests/test_repo_hygiene.py tests/test_cli_reference.py tests/test_adopt_leg_path_collisions.py tests/test_scaffold_leg_path_collisions.py tests/test_shape_advisory.py tests/test_scaffold_ignored_shape_files.py`, with `git-filter-repo` 2.47.0 on PATH: 792 passed, 1 skipped (`test_adopt_plan.py:393`, `SHAPE_MEDXEHR_CLONE` unset). `tests/test_shape_doctor.py`: 208 passed.
**Composition** (`git merge-tree --write-tree` against the current tips; each merged tree ran the sibling's own test files plus this PR's two):
| Branch | Tip | Merge | Merged-tree tests |
|---|---|---|---|
| `origin/main` | c0ed615 | clean | 680 passed |
| `fix/scaffold-manifest-values` (lane 5's #258/#261, PR #314) | 7defac7 | clean | 370 passed |
| `fix/adopt-template-values-utf8` (lane 2's #270) | edfa7c6 | clean | 278 passed |
| `fix/leg-path-yaml-scalars` (lane 2's #266) | b2b42f8 | clean | 409 passed |
| `fix/adopt-check-shape-paths` (lane 5's #262/#263, PR #313) | 65a9262 | clean | 320 passed |
- #314 calls `render` in `manifest_misreads` and `manifest_invalid` with the complete table, so the template-side refusal never fires there. It now finds placeholders with this `PLACEHOLDER_RE`.
- #266 has merged main, so the import-line conflict an earlier tip had is gone.
- **Lane 1's #181 (`fix/adopt-plan-quote-yaml-keywords`, e337f44): a clean merge, one failing test, and the fix belongs to whichever lands second.** #181 alone passes (841). On the merged tree, 1 of 974 fails: `test_adopt_records_each_free_text_value_as_typed[every-key]`, refused `adopt-manifest-misread` at `plan`. #181's gate `_written` (e337f44 `adopt-project.py:1668-1669`) spells the value with `double_quoted` (lane 1's 4bf2cc4), then runs `.replace(token, value)`, which re-reads the value it just put in. That is the per-key re-scan this PR removes. So with both, the gate refuses values `execute` now writes as typed: `--elected-by 'x{{ELECTED_BY}}'` (the gate claims `"xx{{ELECTED_BY}}"`), `'"{{ELECTED_BY}}"'`, and `--reference 'see {{REFERENCE}}'` (it claims `"see see {{REFERENCE}}"`). On this PR alone all three pass `plan` and are written as typed. Neither branch is wrong against main, where `render` really would expand them. Whichever lands second makes `_written` one pass for its key: one `re.sub` over `"(\{\{KEY\}\})"|\{\{KEY\}\}`, or `render` on the line, as #314's `_misread_rows` does. Tokens of other keys (`{{SPEC_PATH}} {{NOPE}}`) pass on every tree.
**Left out.**
- No template placeholder is quoted (#259 waits on Brett), and no free-text value is refused for holding `{{`, since nothing renders it again.
- `family.py init --name` takes a display name (its help says so) that the holder's own `validate-family.py` refuses. On b30595c, `--name 'Contoso Group'` exits 0 and pushes the holder, and `validate-family.py` then exits 1 with `naming-unclassified: 'Contoso Group' matches no family in the naming policy`. This is pre-existing and not touched here; it is filed as #307, with `family.py init`'s other values.
**Review.** The first reviewer stalled at its first checkpoint and was replaced. The second review of 552763d found the code sound and asked only that this body's composition be corrected (above). Its #181 paragraph described an earlier #181 tip, and every other tip had moved.
What the review checked:
- #260's rows reproduced on b30595c and pass on the head, for adopt, scaffold and `family.py init`.
- All 49 templates render byte-identically between base and head over 25 tables of awkward values, in both key orders, and 69 files pushed by the real tools are byte-identical.
- A 40,000-case fuzz of `render` against an independent single-pass reference found 0 mismatches.
- 322 hostile values round-trip through `project.yaml` and `family.yaml` under both this repository's reader and PyYAML.
- Every table holds every digest key its templates name, so the widened refusal cannot fire on a run that used to succeed.
Nits kept:
- Three mutants survived the first head. Fix round 1 (321e511, tests only, with main c0ed615 merged as 7d6c66c) gives each a test that fails on it:
- a `.yml` source gets the quoted spelling as a `.yaml` one does;
- the `template-unsubstituted` list is the sorted set of what the template lacks, each token once;
- an empty string in the table is a value that renders as empty text and is never refused.
The seven targeted files pass on the merged tree, 541 tests.
- The `gh` stub proves nothing on Windows, where no test calls `gh`.
Reviewed adversarially: the first reviewer stalled and was replaced; the second found the code sound and asked only for this body's composition to be corrected, which it is. Fix round 1 adds tests for the three mutants that survived. After #313 landed, the branch was updated onto main 71b58a1 (9469148), and CI re-ran green on Linux, macOS and Windows, with SonarCloud passing.
Closes #260
Lane: openreposhape-5 (openRepoShape-5)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 12, 2026
Contributor
Author
|
(lane openRepoShape-2, 2026-10-12T12:36Z) Composition note for #314 from lane openRepoShape-2's #355 (#283,
The fix is two lines in - pytest.param("--tracking-branch", "~", "no value at all", id="branch-tilde"),
@@
- "on", "0x1F", "1e3", ".inf"]
+ "on", "0x1F", "1e3"]With both edits, the merged tree passes #314's file (104) and #355's (157): 261 passed. Whichever of #314 and #355 lands second carries the edit. If #355 lands first, these are the two lines #314 needs. Neither PR is waiting on the other. Lane: openreposhape-2 (openRepoShape-2) |
brettheap
added a commit
that referenced
this pull request
Oct 12, 2026
…f-format, with the planned test edits to #314's file Main now holds #314, which asks the project's YAML reader about the manifest's values before anything exists. Its tests/test_scaffold_manifest_values.py lists `--tracking-branch ~` among the values the reader misreads (`branch-tilde`) and `.inf` among the values a tracking branch may hold (TEXT_BRANCHES). This PR asks git whether the branch is a name it will create, and git refuses both `~` and `.inf`, so on the merged tree those two rows failed (2 failed in `-k "branch-tilde or inf"`). The merge was clean (no textual conflict). The two hunks of tools/pr314-test-edits-for-composition-v2.patch were applied to the merge's working tree: the `branch-tilde` row leaves MISREAD_VALUES, and `.inf` leaves TEXT_BRANCHES and its comment. `~` is still refused, by this PR's `scaffold-bad-tracking-branch`, which tests/test_tracking_branch_ref.py pins. Nothing else was edited by hand: scaffold-project.py and shape_materialize.py merged without a conflict. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
brettheap
pushed a commit
that referenced
this pull request
Oct 12, 2026
… `plan` asks `check`'s manifest question of the plan it is about to write, whose `pins:` now holds each pin as the mapping `plan` writes since #273, and #314's key-for-key test pins `openGlass@<commit>` from a local stand-in, where this branch refuses a name alone #314 asks the project's YAML reader and `validate-manifest.py` about the manifest's values before anything exists, in both tools. One conflict, `adopt-project.py`, one hunk: this branch added `_refuse_an_unrecordable_pin_source` after `_checked_plan_inputs`, and #314 added its section "The project must be able to hold what `execute` writes into it (#261)" (`MANIFEST_FLAGS`, `_flag_plan`, `_manifest_values`, `_manifest_findings`, `_refuse_unholdable_values`) at the same place. Both are kept, this branch's function first; neither calls the other. `scaffold-project.py`, `scripts/shape_materialize.py` and `docs/cli.md` merged with no conflict (`render-cli-reference.py --check` ok). Where the two meet, as lane 5 composed them before (implT's recipe, `pr314-composition.diff`): - `_flag_plan` built `"pins": list(pins)`. `pins` here is `declared_pin_values`' `{product: PIN_* values}`, so that list held the products, where `plan` writes each pin as a mapping of `PIN_FIELDS`. It now builds those mappings, so the plan it asks about is the plan it writes; `_manifest_values` still reads products through `Plan.pins`. The two annotations say `dict`. - `tests/test_adopt_manifest_values.py::test_plan_asks_what_check_asks_of_the_same_keys` planned `--pin openGlass`, a name alone, which `plan` refuses since #273 as `pin-malformed`. It now plans `--pin openGlass@<commit>` with `--pin-source` naming a stand-in repository the test makes, so it reads no network. Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap
added a commit
that referenced
this pull request
Oct 12, 2026
… is refused and a name holding a lone surrogate is a finding, so `pins: [a, b]]` is not the pin `b]` and `name: "a\ud83db"` no longer kills `make validate` (#301, #280) (#390) Two reader-side defects the lane filed, in one commit on `fix/yaml-flow-brackets-surrogates`, cut from main c7c02fa. Below, "this head" is that commit, 21fbea4, the one the review approved; main was merged into the branch afterwards (last paragraph). **#301, `scripts/repo_shape.py`.** In YAML a plain scalar inside a flow collection ends at any of `[ ] { } ,`. `_split_flow` counts brackets to find its commas and never asked whether the count went below zero or ended above it, so a stray bracket was kept as text: - `pins: [a, b]]` read `['a', 'b]']`; - `k: [x [a, b]]` read `['x [a, b]']`; - since #284, `k: {a"]:}` read the key `a"]`. PyYAML refuses each. A new `_refuse_flow_indicator` refuses a PLAIN item or value (called from `_flow_value`) and a PLAIN key (called from `_split_key_flow`) of a flow collection that holds `[`, `]`, `{` or `}`. The refusal is a `YamlError`, so `load_yaml` turns it into `REFUSED yaml-unparsable` and names the scalar: ``a plain scalar in a flow collection holds `]`, which ends it in YAML: 'b]'``. These still read as before: - a quoted scalar's brackets (`k: ["a]", {"b}": "c]"}]`); - nested and empty flow collections; - a bracket in a BLOCK plain scalar (`k: a[b]`, `- see [1]`); - a bracket in a comment after a flow. No other function changed. **#280, `templates/assembly-root/scripts/validate-manifest.py`.** `_name_finding` now reports a display name holding any of U+D800 to U+DFFF as `manifest-name`: ``` FINDING manifest-name: name is 'a\ud83db', which holds U+D83D, a lone surrogate: no UTF-8 file, commit message or console can hold it. Write each character as itself, or as one `\U` escape (`\U0001F600`, not the pair `\uD83D\uDE00`). ``` The finding goes to stderr with the name escaped by `repr`, before the `manifest ok` line could raise. **Why the #280 refusal is in the validator and not in the reader.** Two places could refuse a lone surrogate, the reader and the validator. The evidence rules out the reader: - **The writer emits a lone-surrogate escape on purpose.** The escape is `\uDXXX`, a `\u` and four hex digits from D800 to DFFF. `double_quoted('caf\udce9')` is `"caf\udce9"`. That is how `adopt-project.py`'s plan keeps a path byte that is not UTF-8 (#199), and `check` must read it back. `tests/test_adopt_entry_paths.py` holds that round trip (120 passed with `test_adopt_plan.py` on this head). Lane 1's pending #189 branch (70f4d29, `submodule_record`) also writes `\udce9` escapes into `project.yaml`'s `submodules:` block, and its docstring relies on the reader resolving them. A reader refusal would break both. - **PyYAML 6.0.3 agrees with the reader.** The pure-Python `SafeLoader` reads `name: "a\ud83db"` as the lone surrogate. Only libyaml (`CSafeLoader`) refuses it, and it refuses even the escape pair `"\ud83d\ude00"`, which the Python loader reads as two surrogates, as this reader does. - **The defect is in a value, not in the YAML.** A display name is text a person chose, and no UTF-8 stream holds a surrogate. On c7c02fa and on this head, with local bare remotes only: | launcher | c7c02fa | 21fbea4 | |---|---|---| | `scripts/validate-manifest.py` | exit 1, `UnicodeEncodeError` traceback | exit 1, the `manifest-name` finding | | `make validate` | exit 2 on that traceback | exit 2 on the finding | | `shape-doctor.py --root .` | `INVALID (manifest)`, the row quotes `UnicodeEncodeError: ...` | `INVALID (manifest)`, the row quotes `FINDING manifest-name: ...` | | a real `bump-leg.py` | exit 0, prints `manifest ok: a\ud83db`, and COMMITS the subject `Bump spec leg to <sha> in a?b` (`errors="replace"`) | exit 2 `bump-leg-validators-red`, every byte rolled back, 0 commits | | `update-shape.py apply` | exit 0, prints `manifest ok: a\ud83db`, commits | exit 2 `update-validators-red`, restored, 0 commits | So all four launchers now agree, and each names the problem. **Readers and callers this changes.** - **#301:** every `parse_yaml`/`load_yaml` caller, and only for a document whose flow collection holds a plain scalar with a bracket. Such a document was read, or misread, before and is refused now. Every tracked YAML file loads the same on c7c02fa and on this head (14/14 raw, 18/18 rendered templates). So do the 6 YAML files of a scaffolded project (`project.yaml`, four `contracts/*.yaml` and `validate.yml`): 6/6 on a c7c02fa scaffold and 6/6 on a 21fbea4 scaffold. No writer emits such a scalar: - every flow sequence written in the tree is bare names or `[]` (adopt's `pins: [...]` and `allow_empty_legs: [...]`, `naming_block`'s `also_matches` and `referent_chain`); - `double_quoted` quotes any value holding a bracket. - **#280:** the project's own `scripts/validate-manifest.py` (a digest-pinned shape copy; it reaches existing projects through `update-shape.py apply`, which reports it `upstream-changed`). Every caller of that copy now stops at the finding: - `make validate`; - the CI step in `validate.yml`; - `shape-doctor.py`'s `manifest` row; - `bump-leg.py` and `update-shape.py apply` (their red-validator refusal and rollback); - a family holder's `make validate`, which runs each member's. `bootstrap.py` runs the same copy in its step (b), but on such a name it dies at its own first `print`, before step (b) (see Left out). With #314 merged as well, the scaffold's pre-check asks this validator about `--name`. In the tree of this head and #314 alone, `scaffold-project.py --name $'caf\xe9' --dry-run` is refused `scaffold-manifest-invalid` with this finding before anything is created (checked at #314's head 4b8da3d). With #351 (landed as d82a83e) and without #314, that `--name` is already refused, as `scaffold-unwritable-value`. **Evidence.** - **Issue rows.** All ten #301 documents reproduce on c7c02fa and are refused on this head. 8 are documents PyYAML refuses. 2 (`k: {a:}#}: b}`, `k: {}#":}`) PyYAML reads only by taking a `#` with no space before it for a comment, and they are refused by design, as `k: "a"#c` is. Every #280 row is in the table above. - **Seeded oracle fuzz.** Corpus: an earlier review's, seed 2490250, 785,646 documents in six contexts. PyYAML 6.0.3 is the oracle only. | | c7c02fa | 21fbea4 | |---|---|---| | read the way PyYAML reads it | 254,629 | 254,629 | | read, where PyYAML refuses | 179,999 | 112,562 | | refused, where PyYAML reads | 48,434 | 49,492 | | read, but not as PyYAML reads it | 2,330 | 1,272 | | the flow contexts alone: read where PyYAML refuses / refused where PyYAML reads | 77,073 / 15,413 | 10,306 / 16,465 | Only two kinds of document change: - 67,437 invalid documents c7c02fa read are refused; - 1,058 documents c7c02fa MISREAD are refused. 1,037 of these are PyYAML's `#` leniency, and 21 are a single-pair mapping in a flow sequence (`k: [a:[]]`), which the subset never read as a mapping. Nothing c7c02fa read correctly changes, nothing is newly read, and there are 0 crashes. Against the issue's two-line `depth < 0` sketch, run on the same corpus, this head refuses everything the sketch refuses, plus 21,631 more: 21,608 invalid documents (the opener class, such as `k: [a[]` and `k: [x [a, b]]`) and 23 misreads. The fuzz ran on 38738cf, whose `scripts/` and `templates/` are byte-identical to 21fbea4's. - **Tests fail on c7c02fa.** With the four test files copied onto c7c02fa: 45 failed, 191 passed. The failures are 31 bracket rows, 10 validator rows and the 4 launchers. - **Mutation probes.** Each hunk was reverted or weakened on its own, and each mutant fails new tests: - M1, no check on items and values: 23 failed; - M2, no check on keys: 8; - M3, no quoted-scalar exemption: 14; - M4, no quoted-key gate: 11; - M5, closers only: 3; - M6, no surrogate finding: 10; - M7, upper bound excludes U+DFFF: 2; - M8, lower bound excludes U+D800: 2. **Tests.** - New: `tests/test_yaml_flow_brackets.py`, with the ten issue rows, sixteen more positions, the refusal's wording, a file refusal, fifteen still-read rows and the writer round trip. - In `tests/test_manifest_validator.py`: five surrogate spellings, each under a pipe and under `PYTHONIOENCODING=utf-8`, plus four still-ok controls on either side of the range. - In `tests/test_validator_child_encoding.py`: a real bump, `update-shape.py apply`, the doctor and `make validate`, each on `name: "a\ud83db"`. - `tests/test_yaml_quote_boundaries.py`: its docstring said `k: "a"#c` was the only PyYAML disagreement. There are eight; seven are `BRACKET_IN_A_KEY` rows refused by design. Targeted runs on 21fbea4, Linux, Python 3.12.3, git-filter-repo 2.47.0, each in the background: - the reader and validator files (`test_yaml_flow_brackets`, `test_yaml_flow_quotes`, `test_yaml_flow_mappings`, `test_yaml_quote_boundaries`, `test_yaml_block_scalar_keys`, `test_load_yaml_refusals`, `test_adopt_plan_escapes`, `test_manifest_validator`, `test_pin_validator`, `test_neutral_pin_validator`, `test_template_quoted_values`): **462 passed**; - `test_validator_child_encoding`, `test_stdout_encoding`, `test_repo_hygiene`, `test_cli_reference`: **165 passed, 1 skipped** (`test_stdout_encoding.py:286`, a Windows-only default); - `test_adopt_plan`, `test_adopt_entry_paths`: **120 passed, 1 skipped** (`test_adopt_plan.py:393`, `SHAPE_MEDXEHR_CLONE` unset); - `test_shape_doctor`: **242 passed, 3 skipped** (`test_shape_doctor.py:5330`, PowerShell on a Windows host only); - this head merged with origin/main 86da808, which carries #316: `test_template_quoted_values`, `test_template_values_single_pass`, `test_yaml_flow_brackets`, `test_yaml_quote_boundaries`, `test_manifest_validator`: **412 passed**; - this head merged with #314 (ab66bc0): `test_scaffold_manifest_values`, `test_adopt_manifest_values`, `test_manifest_validator`, `test_yaml_flow_brackets`: **293 passed**. The full suite and the macOS and Windows legs did not run before this PR's CI. **Merge-tree, as of 2026-10-12T15:12Z** (`git merge-tree --write-tree --name-only`), against origin/main d82a83e and the heads named here; each sha is the one it had then, and main has moved since. - **Clean with this commit:** - origin/main d82a83e, which carries #316 (34ecbc2), #245 (head a0272b2, landed at 14:55Z as 161c6c9) and #351 (landed as d82a83e). #245's one change to `scripts/repo_shape.py` is in `remote_key`, away from the YAML reader, and #351 touches `scaffold-project.py` only; - #314 (4b8da3d). With it merged, the `--name` refusal above still holds (checked at 4b8da3d); - lane 1's `fix/adopt-late-failure-notice` (9f1778d) and `fix/adopt-plan-quote-yaml-keywords` (d38e143); - the sibling `fix/pin-source-env-collisions` (34dbd86, #275); - every other open PR head but one: 28 of the 29 open, all but #358. - **Conflicts that are not this commit's.** Each conflicts the same way with origin/main alone, in a file this commit does not touch: - #358 (ebffb08), in `shape-doctor.py`; - lane 1's `fix/adopt-root-kept-symlinks` (6c44591), in `adopt-project.py`. - **Five heads already conflict with origin/main itself, and merge clean with this commit:** #314, #356 (5a5011d), #382 (7e55d00) and lane 1's `fix/adopt-plan-quote-yaml-keywords` in `adopt-project.py`, and lane 1's `fix/adopt-late-failure-notice` in `tests/conftest.py`. - **One conflict is this commit's:** `fix/adopt-record-submodule-placement` (70f4d29, lane 1, #189), in one hunk, the import block of `tests/test_manifest_validator.py`. Both branches add to it: that branch adds `import os` and `import sys`, and this one adds `import subprocess` between them, so the resolution is all three. Its `repo_shape.py` and `validate-manifest.py` changes auto-merge. Not resolved in either branch. **The review.** An independent opus review at 21fbea4 approved it with nits: 0 blocker, 0 should-fix, 5 nit. All five are in text (this description, the author's report and the commit message), none in the code or the tests. Four were fixed in the description; the fifth, the commit message's spelling of the surrogate escape as a six-digit one, is fixed in this message: the escape is `\uDXXX`, a `\u` and four hex digits, D800 to DFFF. Its own seeded structured fuzz (seed 7301280: 120,000 nested flows in five block contexts, 600,000 documents, PyYAML 6.0.3 as the oracle only) found 0 changed reads: no document c7c02fa read the way PyYAML reads it changed, and nothing crashed. An instrumented reader returned no plain scalar holding `[ ] { }` inside a flow on this head, against 76,955 on c7c02fa. The author's seed 2490250 counts reproduced row for row, and all 13 mutants it tried were killed. **664 passed** on this head merged with origin/main ab623dd (16 test files; 361 on the head alone, 8 files). The one conflict, with lane 1's #189 branch, is the `tests/test_manifest_validator.py` import block, and resolves to all three imports. Three follow-ups it confirmed are not in this change: the bootstrap's raw name print and the flow entries still read (both under Left out), and a block value that opens with `]` or `}`, which c7c02fa and this head both read where PyYAML refuses. **Left out.** - **A plain flow entry that opens with an indicator, and an empty flow entry.** Most of what this head still reads where PyYAML refuses is one of two classes: - an entry opening with `:`, `#`, `?` or `-` (`k: {:":}` reads `{':"': None}`, `k: [:]`; 6,286 flow documents); - an empty entry from a leading or doubled comma (`k: [,a]` reads `['a']`, `k: [a,,b]` reads `['a', 'b']`; 3,858). Neither is a bracket, and #301 does not claim them. - **A single-pair mapping in a flow sequence.** `k: [a: b]` still reads as the string `'a: b'`, where PyYAML reads `[{'a': 'b'}]`. Only the ones holding a bracket are now refused. - **`templates/assembly-root/scripts/bootstrap.py`.** It prints the display name raw (`bootstrap: <name> (<root>)`) before its validators run. On a project named `"a\ud83db"`, `make bootstrap` still dies there with `UnicodeEncodeError`, on c7c02fa and on this head (filed as #391). - **`elected_by` and `reference`.** A surrogate in either is still accepted. Neither is printed by the validator, so nothing raises; the scaffold side of such values is #282. - **The escape pair `"\ud83d\ude00"`.** It is still two surrogates, not one character, as PyYAML's Python loader reads it. The finding asks for the character itself or its one `\U` escape instead of the reader inventing a joining rule. Reviewed adversarially at 21fbea4 (APPROVE WITH NITS: 0 blockers, 0 should-fixes, 5 nits, all in text and none in the code or the tests). The code and the tests needed no fix round; the nits are fixed in this message. Main was merged into the branch after the review with no conflict: at that merge (c1a119d, main 45e01f4 merged in) the branch's own diff against main is the one reviewed, the same 6 files with 422 lines added and 6 removed. CI is green on Linux, macOS and Windows, with SonarCloud passing, at c1a119d (main 45e01f4 merged in). The combined tree with main 3ef059c (#314 landed) was run locally before landing. Closes #301 Closes #280 Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap
added a commit
that referenced
this pull request
Oct 12, 2026
…landed, with one guard in _manifest_values and #314's 5 elected_on: rows retargeted to election-date-malformed #314 (scaffold-project.py and adopt-project.py ask validate-manifest.py about the manifest's values) landed on main as 3ef059c. Its adopt-project.py is a5438a5's merged with 92e29d9, the composition pr-body-v3.md planned. adopt-project.py conflicted in 2 hunks of _entry_findings, both kept: - the docstring lists #261's validator findings, then #190's relative urls, and last this PR's plan values (#293, #294); - the return is findings + split + unwritable_findings + value_findings + moved + field_findings, the remediations in that order, and notes. Main's auto-merged paragraph said the relative urls of #190 come "last"; it now says the plan values come after them. The guard: _manifest_values returns None when _elected_on_findings(plan) has a finding, right after its _check_names refusal. Without it, a plan with no reference: had check exit 2 on election_date's own refusal, raised from _template_table through default_reference and naming --elected-on, before this PR's finding was listed; with reference: kept, the date was two findings, the validator's and this PR's. In plan the guard is a no-op: _checked_plan_inputs asks election_date of --elected-on first. #314's 5 rows for an elected_on: that election_date now refuses get this PR's one election-date-malformed finding, so they are retargeted to it in tests/test_adopt_manifest_values.py (said_of gives each row's finding, REFUSED prefix and remediation), and none is dropped: - elected-on-iso-week in test_check_finds_a_value_the_validator_refuses, test_execute_refuses_it_before_any_leg_exists and test_check_finds_it_from_a_tree_with_no_git_as_in_a_checkout; - elected-on-basic-format in test_execute_refuses_a_text_value_the_reader_misreads_before_any_leg and test_a_text_value_the_reader_misreads_is_one_finding_on_this_rule_alone, where _manifest_findings is now ([], []) and _plan_field_findings holds the one finding. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 12, 2026
brettheap
pushed a commit
that referenced
this pull request
Oct 12, 2026
…plan-manifest-invalid` joins `_entry_findings` after #255's values, and this branch's `plan-root-symlink` stays after the source's own submodules origin/main moved 92e29d9 -> dd8d0ea (twelve commits), merged by sha: - f0a8bf6 (#330 via #381), scripts/family.py; - 87d2eb7 (#333 via #379), update-shape.py; - cd1fc79 (#327 via #375), shape-doctor.py; - 45e01f4 (#348 via #396), tests/test_agents_md_codes.py: every code-shaped name AGENTS.md spells is a code a tool prints; - 3ef059c (#258 and #261 via #314), adopt-project.py, scaffold-project.py and scripts/shape_materialize.py: the project's YAML reader and its `validate-manifest.py` are asked about the manifest's values before anything exists; `check` reports `plan-manifest-invalid` through `_manifest_findings` in `_entry_findings`; `_repository_urls` refuses `adopt-name-too-long`; - f7f4b14 (#366 via #399), scripts/family.py; - cb7063f (#301 and #280 via #390), scripts/repo_shape.py and the template's validate-manifest.py: a bracket or brace in a plain flow scalar is refused; - 5678f68 (#278, #297 and #298 via #357), shape-doctor.py; - 53bee5b (#289 via #352), scripts/bump-leg.py; - f5cb616 (#369 via #398), scripts/shape_materialize.py: the docstrings of `force_stage` and `stage_written`; - a92e60b (#326 via #400), scripts/family.py; - dd8d0ea (#292 via #353), adopt-project.py: `_elector` keeps a `user.name` byte that is not UTF-8, `_remote_repository` reads an origin URL that is not UTF-8 as none, and `adopt-no-org` is asked before `checked_value`. scripts/shape_materialize.py, scripts/repo_shape.py and every other file merge with no conflict. adopt-project.py conflicts in two hunks of `_entry_findings`, each resolved by keeping both sides: 1. The docstring. Main's order, with #261's manifest findings after #255's values and before #190's relative urls; this branch's symlinks stay after the source's own submodules. 2. The return. Main's `_manifest_findings(plan)` call, and the findings in main's order: each entry's path (#196), coverage, the legs and what the root keeps where the shape writes, the source's own submodules (#166), this branch's `_root_symlink_findings`, #255's values, #261's manifest findings, and #190's relative urls last. The remediation and the NOTEs are main's. How each of main's commits that reach this branch's code meets it: - 3ef059c: `_manifest_findings` asks the validator the template ships about the values `execute` writes; it reads no symlink and no path the shape writes. Its materializer half adds the validator's helpers, `refuse_unstorable_names` and `MANIFEST_VALIDATOR`, and holds `election_date` to the validator's `DATE_RE`; it leaves `place`, `_materialize`, `copy_tree`, `template_files` and `_refuse_unwritable` as they were. In `execute`, its `adopt-name-too-long` (in `_repository_urls`) comes before #255's clone-URL refusal, `_refuse_what_check_finds` and this branch's `core.symlinks` probe, all before any leg. Its new paragraph of the module docstring follows the one 6b53a6d corrected; both merge with no conflict. - f5cb616: docstrings only. - dd8d0ea: `_remote_repository`, `_checked_plan_inputs`' organisation and `_elector`, none of them this branch's. - cb7063f: the plans `check` and `execute` read go through this reader. The plans `plan` writes for this branch's tests hold their entries as block mappings, each path `double_quoted` where it needs to be, and the rehearsal below read every one of them. Checked on the merged tree: - AST (fin183g/names.py, base 92e29d9): no top-level name is bound twice in adopt-project.py, scaffold-project.py or scripts/repo_shape.py, and none is loaded but bound nowhere. scripts/shape_materialize.py binds `YamlError` and `parse_yaml` twice, as main dd8d0ea does (1f90a0f's import and 3ef059c's, the same objects); the merge adds none. - Both sides intact (fin183g/sides_intact.py): the only lines that differ from either side's own change are those of the two hunks. - A rehearsal of this merge (tree d21f8b4, the same as this one) passed tests/test_adopt_root_symlinks.py with main's test_adopt_{manifest_values,git_reads,relative_submodule_url}.py, test_scaffold_manifest_values.py, test_agents_md_codes.py and test_yaml_flow_brackets.py: `440 passed`. Lane: openreposhape-1 (openRepoShape-1) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap
pushed a commit
that referenced
this pull request
Oct 12, 2026
Merged by sha: origin/main at 19:37Z. Past 92e29d9 it brings twelve commits. These four reach this branch's files or the reader its gate asks: - 3ef059c (#314, for #258 and #261): `plan-manifest-invalid`, asked of the project's own validate-manifest.py, through `_manifest_findings`. - cb7063f: the reader refuses a bracket in a plain flow scalar. - f5cb616: docstrings in scripts/shape_materialize.py. - dd8d0ea: `_elector` keeps a `user.name` byte that is not UTF-8, for #255. adopt-project.py conflicted in two hunks, both in `_entry_findings`. Each keeps both sides: 1. The docstring. The order is this branch's misread values (#181), the submodules, #255's unwritable values, then main's validator (#261), and the relative urls (#190) last. 2. The return. - Findings: unsafe, coverage, legs and shape, misread, submodules, unwritable, validator, relative url. - Remediations: the same order. - `_manifest_findings(plan)` is called as main calls it. Everything else merged clean. That includes `_checked_plan_inputs` and `_template_values`, which main split into `_template_table`. At this merge a value both gates judge is two findings: `id: "007"`, `id: "null"` and `elected_on: "20260902"`. Main's own `test_a_text_value_the_reader_misreads_is_one_finding_in_the_list_check_prints` fails for `id-007` and `elected-on-basic-format`, as its docstring says it will on a merge that does not compose the two. A `visibility` outside the three choices that is not text (`true`, `[secret]`) is no finding at all. The next commit composes them. Lane: openreposhape-1 (openRepoShape-1) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap
pushed a commit
that referenced
this pull request
Oct 12, 2026
… before its date guard, so `check` and `execute` refuse a plan with a bad `elected_on:` and a bad organisation or id for the same first problem (#294) The guard this PR added to `_manifest_values` (return None when `_elected_on_findings` has a finding, so #314's validator is not asked about a date this PR already reports) returned before the function's `checked_value("org", ...)` and before `_template_table` read `plan.project_id`. Those two are where `check` meets a bad organisation and a bad id; `execute` meets the organisation earlier, in `_repository_urls`, and the id here, through `_refuse_what_check_finds`. So with `elected_on: someday` beside them: - `org: "-evil"`: `check` exited 1 listing only the date, and `execute` exited 2 `REFUSED unsafe-value`; - `id: "North Wind"`: `check` exited 2 `REFUSED plan-bad-id` (from the topic line it prints later), and `execute` exited 2 `REFUSED election-date-malformed`. On main 580d7df both commands name `unsafe-value`, and `plan-bad-id`, for those plans. The organisation is now refused, and the id read, before the guard, so both commands name the same first problem again, as on main, and a date-only plan is still one finding in the plan key's own words, not `plan-manifest-invalid` beside it. The docstrings of `_manifest_values` and `_manifest_findings` now also say they return nothing for such a date. tests/test_adopt_execute_refusals.py: `test_a_bad_date_does_not_change_what_check_and_execute_refuse_first`, for the organisation and the id, with `reference:` kept and removed: `check` exits 2 on the code `execute` refuses with, and no leg is created. All 4 fail on 88c430c's code and pass here. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
14 of 17 tasks
brettheap
added a commit
that referenced
this pull request
Oct 12, 2026
… lines and clones nothing, and render-cli-reference.py refuses a tree that is not a checkout, so asking a download for help is no longer a clone of GitHub (#341) (#452) From anywhere but a checkout of the standard (a downloaded `setup.sh`, `curl … | bash -s -- --help`, a `git archive` extraction), `setup.sh --help` went down section 0 like any other run. It cloned the public upstream before printing a word, died with git's exit 128 offline, and otherwise printed the CLONE's `setup-project.py` usage. `scripts/render-cli-reference.py` inherited all of it in such a tree: every rendering, `--check` included, reached GitHub and then documented upstream's flags under that tree's `setup.sh` heading. The clone attempts were reproduced on 87d2eb7, offline (`GIT_ALLOW_PROTOCOL=file`); what the generator renders when that clone succeeds is read from the code, not reproduced: | | base 87d2eb7 | this branch | |---|---|---| | lone `setup.sh`, `bash setup.sh --help` | `(0) self-bootstrap`, `git clone --quiet --depth 1 -- https://github.com/opensoft/openRepoShape.git <tmp>`, `fatal: transport … not allowed`, exit 128 | the shim's own text, exit 0, no clone (`GIT_TRACE`), no temporary directory | | same, `OPENREPOSHAPE_REPO=<checkout>` | a local clone, then the clone's `usage: setup-project.py …` | the shim's own text, no clone | | `bash -s -- -h < setup.sh` | the clone, exit 128 | the shim's own text, exit 0 | | `render-cli-reference.py --check` in a copy with no `.git` | tried the clone (stopped only by `GIT_ALLOW_PROTOCOL`, surfacing as `setup.sh --help exited 128`) | refuses, naming the tree, exit 2, no clone | **What changed.** **`setup.sh`** - Section 0's one scan of the command line now also notes `-h` and `--help` before the first `--`. - The shim answers them first: before the `$OPENREPOSHAPE_REPO` guard, before `mktemp`, before `git clone`. That is where `setup-project.py` answers them: before it looks at anything. - The text says: - this is the bash front door to `setup-project.py`; - any other run from here first clones the standard into a temporary directory, from `$OPENREPOSHAPE_REPO` or, when that is unset or empty, `https://github.com/opensoft/openRepoShape.git`; - the one flag list is `python3 setup-project.py --help`, which answers with no network from a checkout or from that one file downloaded on its own, and `docs/cli.md` in the repository. - It lists no flag, so #50's one `--help` still holds. - The default is now one constant, `DEFAULT_SHAPE_REPO` (the name `setup-project.py` gives it), read by both the clone and the help. The help never prints the variable's value. - Inside a checkout nothing changes: the shim still execs `setup-project.py` before section 0 reads a word. - The header comment no longer says the shim "parses nothing". It now says the shim parses nothing of the RUN, and names the three things section 0 reads. The comment above the scan no longer says the scan never decides what the run does: the first two reads only decide whether a temporary directory is removed and whether a ref is checked out, and the third decides whether there is a run at all. **`scripts/render-cli-reference.py`** - `require_a_checkout()` asks `setup.sh`'s own `is_shape_checkout` question: `git -C <tree> rev-parse --show-toplevel`, and `scaffold-project.py` plus `contracts/repository-naming.yaml` at that top level. - It also asks that the top level IS the tree (`os.path.samefile`). A copy nested inside another checkout passes the shim's probe, and its `setup.sh --help` would print that checkout's usage. - It is called in `main()`, before the no-bash logic, and at the top of `render()`, which `tests/test_cli_reference.py` calls directly. The refusal names the tree and says to run it in a `git clone`, exit 2. - It reads git's answer the way the shim's `$(...)` does: `os.fsdecode(...).rstrip("\r\n")`, so a checkout whose path ends in a space, or is not UTF-8, is a checkout here too. - Every child now runs under `GIT_ALLOW_PROTOCOL=file`, so a regression fails closed. - Inside a checkout neither measure changes a byte of a rendered help block. `--check` was `[ok]` before, and `[ok]` after regenerating. **Docs** - `README.md`: "`python3 setup-project.py --help` and, from a checkout, `setup.sh --help` list every flag, this one included. From a download, `setup.sh --help` clones nothing and lists none: it names `setup-project.py --help` … and `docs/cli.md`". That is +3 lines against main; the README cap in `tests/test_repo_hygiene.py` goes 1419 -> 1422, with its dated paragraph. `openRepoShape --help` is not the subject any more, because it names eight of `setup-project.py`'s fourteen flags besides `--help`. § What setup.sh does no longer gives the shim a line count ("about seventy lines of code"; it was 93 before this change and is 122 now), here, in the handbook and in `setup-project.py`'s docstring. - `docs/handbook.html`: the page's copy of that sentence and of the shim's description, the footer's history clause, and the new README blob (`5465d135…`). - `openRepoShape --help`: its one sentence "`./setup.sh --help` prints setup-project.py's usage, which lists them all" was false from a download. It now names `python3 setup-project.py --help`, and `./setup.sh --help` "in a checkout of this standard". - `docs/cli.md` is regenerated: a note under `setup.sh` saying what `--help` does outside a checkout, and that `openRepoShape` sentence. - `tests/test_setup_sh.py`: `test_no_test_here_depends_on_gh_being_authenticated`'s docstring is made true. **Decisions** (the coordinator's; Brett may reverse). 1. **The shim's answer outside a checkout is option (a), with no flag list.** It is a short text of its own, exit 0, with no clone, no network and no temporary directory. Inside a checkout `--help` is still `setup-project.py`'s. - `-h` is treated like `--help`, because `setup-project.py` reads both. - Only arguments before `--` count. `setup.sh <Project> -- --help` is still a run, whose scaffold is asked for ITS help. - **Known false positives** (written into the shim's comment): runs `setup-project.py` would refuse anyway, such as `--org --help` or `--wat --help`, get the text and exit 0 instead of exit 2. Nothing is created either way. - **One known false negative**: `--visibility -- --help`, where `--` is a value, still clones as before. Fixing it would need the value-flag table in bash, which is the second parser #50 removed. 2. **Whose help.** The shim's text is its own, so it names no clone URL or ref. No path that prints a usage is left unattributed. `setup.sh <P> -- --help` prints the clone's scaffold usage under the `(0) self-bootstrap` clone line and `checkout: <tmp>`, which already say whose it is. 3. **The generator gets both measures**: the checkout refusal and `GIT_ALLOW_PROTOCOL=file`. 4. **Docs**: README, the handbook's reading of it, the shim's header, the test docstring and `openRepoShape --help`'s sentence, with `docs/cli.md` regenerated. 5. **The #50 guard is changed on purpose.** `tests/test_setup_project_py.py::test_setup_sh_parses_no_flags_of_its_own` refused any `--help` in the shim's code. It now excuses the spelling in exactly two places: the lines of `shim_help()`, each of which must be one `say "..."` with no `$(`, `${` or backtick in it, and section 0's one `-h|--help)` label. Outside those two, neither `--help` nor a bare `-h` may occur in the code. It still refuses `-y)`, `-h)`, `--yes)` and a second `--help)` label, and since fix round 1 it also refuses a quoted `"--help")`, a `[[ $1 == --help ]]` reader anywhere else (the developer path, the scan), and a reader hidden in a `say` line. **Evidence.** The runs and the merge-tree below were taken at e2df9a8, the branch's head before the last merge of main (the last paragraph). - **Targeted runs on e2df9a8** (d15a888, the fix round, merged with `origin/main` 53bee5b; the branch's first commit 968db85 is the writer's 16d1639 re-authored with the same tree 0d3243f), with a stub `gh` first on PATH, `GH_TOKEN` bogus, an empty `GH_CONFIG_DIR`, `GIT_ALLOW_PROTOCOL=file` and a blanked git identity: - `tests/test_setup_sh.py` and `tests/test_setup_project_py.py`: **132 passed**, 0 skipped; - `tests/test_cli_reference.py`, `tests/test_repo_hygiene.py`, `tests/test_agents_md_codes.py` and `tests/test_openreposhape_command.py`: **164 passed**, 0 skipped; - all six files together: **296 passed**, 0 skipped, 0 failed. Before the merge, on a throwaway merge of d15a888 and main 5678f68, the same six files gave 296 passed as well. review341 ran them on 968db85 merged with main 3ef059c: 132 + 162 passed. - **The new tests against base 87d2eb7's code**: 15 failed. These are every new behaviour test (the shim's help ×4 spellings, its text, no echo, `bash -s`), the five generator refusal tests, `GIT_ALLOW_PROTOCOL` and the shim-probe tie. The 9 that pass are the "unchanged" guards (`-h`/`--help` inside a checkout, `-- --help` passthrough ×2) and pre-existing tests. - **Mutation proof**: 18 single-hunk mutations in a throwaway worktree, each caught by a named test. They cover: - the help exit; - `-h`; - the `--` stop; - help after the env guard; - the default not named; - a flag line added; - the default not shared; - a second `--help)` label; - `main`'s guard, `render`'s guard, and both; - the top-level-is-the-tree check; - the files check; - `GIT_ALLOW_PROTOCOL`; - the files drifting from the shim; - the Tool note; - the `openRepoShape` sentence; - the README sentence. - **The new tests**: - `tests/test_setup_sh.py` (`WINDOWS_SKIP` with the rest of the file): - `--help`, `-h`, after a run's arguments, and beside `--keep-shape-checkout --shape-ref`: exit 0, the shim's text, no `(0) self-bootstrap`, no usage, no `git clone` in `GIT_TRACE`, nothing in `TMPDIR`; - the text names the variable, its default, `setup-project.py --help` and `docs/cli.md`, and no flag but `--help`; - a forged `$OPENREPOSHAPE_REPO` is never echoed; - `bash -s -- --help`; - `<Project> -- --help` and `-- -h` still clone (locally) and reach the scaffold's usage. This is also the positive control that the trace sees a clone; - inside a checkout, `-h` and `--help` are `setup-project.py --help` byte for byte, on stdout and on stderr (stderr was not compared before the fix round; by probe it was already identical). - `tests/test_cli_reference.py`: - a `git ls-files` copy with no `.git`, one inside an unrelated work tree, and one inside a work tree that looks like a checkout of the standard: `--check` and a plain write both exit 2, naming the tree, with nothing written and no clone; - `render()` itself refuses; - the checkout refusal comes before the no-bash one; - `child_env()` pins `GIT_ALLOW_PROTOCOL=file`; - the generator's files are the ones the shim's probe reads; - (fix round 1) `--check` in a real `git init` copy whose path ends in a space, and in one whose path is not UTF-8, exits 0 with `[ok]`; each is skipped where the file system cannot hold the name (both on Windows; the second wherever creating the directory fails, which is expected on macOS, where no runner was available here). **Merge-tree, as of e2df9a8 (re-run after merging `origin/main` 53bee5b into the branch).** - At e2df9a8, `origin/main` (53bee5b) is an ancestor of the head: the branch carries a merge commit, so it was mergeable with main by construction. `render-cli-reference.py --check` was `[ok]` on it. - **#352 landed as 53bee5b (#289) and conflicted in `docs/handbook.html`'s footer, as predicted.** The merge keeps both "added (#n)" clauses in landing order after "(#288)": (#289), then (#341), and keeps this branch's `README.md blob 5465d13`, which is `git hash-object README.md` again: main does not touch README.md and its own blob is `e8da9381…`. - #314 has landed (3ef059c). The other PRs then open that share a file with this branch, re-checked against d15a888 before the merge (heads fetched at that time) and #403 again against e2df9a8: - clean against this branch: #394 (`tests/test_repo_hygiene.py`, not the README cap; it conflicts with main alone in `tests/test_manifest_validator.py`, a file this branch does not touch), #395 (one-line `visibility` assertion edits in `tests/test_setup_sh.py` and `tests/test_setup_project_py.py`, independent of this branch; its hunks in `docs/cli.md` are at :246-279), #397, #400 and #401 (each regenerates other sections of `docs/cli.md`); - **#403 (head 5b1da66) conflicted in the same footer hunk**: it merged clean with main alone and conflicted with this branch. It has since landed as dba5fb0 (#383, #403), and the merge of main into this branch (the last paragraph) keeps every clause in landing order after "(#289)", that is (#383) and then (#341), and names the blob of the README that lands, `5465d135ca7263010ee354424041f90c4553d2e7`. Main does not change README.md, so that is the blob this branch already named. - `docs/cli.md`: #395, #397, #400 and #401 all merged clean with d15a888 plus main 5678f68, and the generator's `--check` was `[ok]` on each merged tree (#395's head 1ef3310, #397's abb27e6, #400's 7062a47, #401's 409f10b). Should one ever conflict, regenerating resolves it. **Left out** (candidate follow-ups, not filed here). - `setup.sh --preflight` from a download still clones before checking the machine, though `setup-project.py` answers `--preflight` before its own self-bootstrap. The preflight is Python, so the shim cannot answer it itself; it needs its own decision. - From a download, `setup.sh --help` still needs Python 3.9 and git first: the interpreter and git checks run before the checkout probe, as they always did. - Windows does not run the shim. The generator refusal tests do run there: they reach the refusal before any bash. The `tests-windows` job was their first run on that platform. **Fix round 1 (review341's CHANGES NEEDED 0/1/5).** One commit (d15a888) on top of 968db85, then a merge of `origin/main` 53bee5b (e2df9a8), no force: - **S1 (should).** The #50 guard excused the spelling `--help` everywhere in the shim, so a second reader in any form but an unquoted label passed it. It now excuses the spelling in `shim_help()`'s `say "..."` lines and the one `-h|--help)` label, and nowhere else (decision 5). Mutation proof in a throwaway worktree: the guard passes on the unmutated shim and fails on each of review341's three variants (a quoted `"--help")` label, a `[[ "${1:-}" == --help ]]` reader in the developer path, a `[[ "$arg" == --help ]]` reader inside the scan) and on four more (a reader as its own line in `shim_help()`, one chained after a `say`, one substituted into a `say`, a bare `-h` reader). The first four and the bare `-h` reader fail only the guard, and the help tests around it stay green; the chained and substituted `say` variants also break the help's own output. - **N1.** The inside-a-checkout test compares stderr as well as stdout. With the developer-path reader above, it now fails and `test_help_exits_zero` still passes. - **N2.** The generator reads git's top level with `os.fsdecode(...).rstrip("\r\n")`, so a checkout at a path ending in a space, or not UTF-8, is accepted again, as the shim accepts it. A test for each; on 968db85's generator both fail with exit 2 "not a checkout". - **N3.** README and the handbook name `python3 setup-project.py --help` as the command that lists every flag; `openRepoShape --help` names eight of fourteen. The handbook's copy, its footer clause and its README blob follow. README stays 1422 lines, so its cap does not move. - **N4.** "About seventy lines of code" is dropped from README, the handbook and `setup-project.py`'s docstring: the shim is 122 lines of code, 93 before this change, and a count in prose goes stale again. - **N5.** The comment above section 0's scan says the third read decides whether there is a run at all. - `docs/cli.md` is unchanged: no `--help` text moved, and `--check` is `[ok]`. Reviewed adversarially at 968db85, the branch's first commit (CHANGES NEEDED: 0 blockers, 1 should-fix, 5 nits). The shim and the generator behaved as this description says. The should-fix was the #50 guard, which a second `--help` reader slipped past; the nits were a test that compared stdout and not stderr, two real checkouts the generator refused (a path ending in a space, a path that is not UTF-8), and three pieces of stale doc or comment text (the README and handbook sentence that said `openRepoShape --help` lists every flag, the shim's line count, and the shim's own comment). Fix round 1 (d15a888, merged with main 53bee5b at e2df9a8) resolved them, and its re-verification of e2df9a8 read APPROVE WITH NITS (0 blockers, 0 should-fixes, 1 nit: § What setup.sh does does not say that the shim answers `--help` itself outside a checkout, left as it was because README says it a few lines above). Main 75052cb was then merged in at 4c28890: the one conflict, `docs/handbook.html`'s footer, keeps both "added" clauses (#383's, then #341's) and names the merged README's blob (5465d13), and the merged `docs/cli.md` matches its generator. CI is green on Linux, macOS and Windows, with SonarCloud passing, at 4c28890 (main 75052cb merged in). Closes #341 Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap
pushed a commit
that referenced
this pull request
Oct 12, 2026
…says #314 landed first and unifying the two is on #459's checklist The comment said the assembly root's questions (`manifest_misreads`, `manifest_invalid`) "are not on the branch this was cut from" and that "whichever of the two changes lands second unifies them". #314 landed first, as 3ef059c, so they are on main, and this change lands second without unifying them: that is an item on tracking issue #459's checklist. The comment now says that, and keeps the reason this file has its own smallest version (the assembly root's questions are written for `project.yaml` and its `validate-manifest.py`) and what the two differ in (the template, the validator's entry point and the sentence that names the file). A comment only: no code changes. Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
brettheap
added a commit
that referenced
this pull request
Oct 12, 2026
…it check-ref-format --branch` before anything exists, so `~`, `a..b`, `x.lock` and `a/` are refused under `--dry-run` and no longer die at the first `git init -b` after `gh repo create` has made the repositories (#283) (#355) `--tracking-branch` of `scaffold-project.py` and `scripts/family.py init` was held only to `checked_value`'s alphabet, which keeps an option and a shell word off a command line. The alphabet is wider than what git accepts as a branch: `~`, `a~1`, `a..b`, `x.lock`, `a/`, `/a`, `a//b`, `.a` and `a.` all pass it, and `git init -b` refuses each. `--dry-run` printed a plan and exited 0. A real run against GitHub had `gh repo create` make the repositories before the first `git init -b` died, and the re-run was then refused `scaffold-remote-exists` until the operator deleted them by hand. Both tools now ask git, before any work directory, remote or repository exists, and under `--dry-run` too. A name git refuses is a named refusal, `scaffold-bad-tracking-branch` or `family-bad-tracking-branch`, exit 2, no traceback, and no `gh repo create` is run. The question is two functions in `scripts/shape_materialize.py`: `branch_problem` returns the sentence when `git check-ref-format --branch` does not accept the value exactly as written, and `refuse_bad_branch` raises the caller's own refusal code with it. - Git is asked, not copied, so a rule git adds or relaxes is ours when the person upgrades. - It is asked with no repository in play: `git --git-dir=<an empty temporary directory> check-ref-format --branch <name>`, run from that directory. The verdict is the same from every directory, including one inside a repository git cannot open and one removed under the person, where the default `main` must still scaffold. - A git that cannot answer for any name says so. After a refusal, git is asked about `main`; if it refuses that too (a malformed global config), the name "could not be checked" and the remedy is about git, not the name. Git's words are localised, so they are never read. - A leading `-` is refused without asking, a name git would expand rather than echo is refused as not the name written, and a git that cannot run is a refusal, not a traceback. In `scaffold-project.py` the call sits after `_declaration` and `_check_declared_names` and before `_shape_revision`, the work directory and every remote, because the refusals made earlier start no process and tests hold that. In `family.py` it follows `checked_value`, before `init` runs `gh repo create` for the holder. The helper lives in `shape_materialize.py`, not `repo_shape.py`, as `refuse_bad_leg_paths` does: `repo_shape.py` is a shape copy every project digest-pins. No template file and no `--help` text changes. Two rows of #314's `tests/test_scaffold_manifest_values.py`, `~` and `.inf`, name values git refuses, so this change removes them there. Left out: `adopt-project.py` takes the flag the same way and has the same defect, at three sites, so it is its own change. A leading `+` and `update-shape.py`'s own `--branch` check are follow-ups too. Closes #283 Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap
pushed a commit
that referenced
this pull request
Oct 12, 2026
…s cite #343, #401 and #456, drop the known limits main no longer has (#290, #391, #225, #294), list all 30 open milestone issues, state the Windows park-and-resume exception beside the 1.0 promise, and give a pin procedure that pastes into macOS's zsh (#472) A fact-check of cbe44fe found nothing wrong at the drafting sha, but main had moved. Three PRs landed and fourteen milestone issues were filed in the forty-five minutes after the draft, and several sentences were true only with a qualifier the notes had dropped. This commit refreshes the notes against 437a360 and takes the checker's corrections. Refresh: - Section 5 adds #343 (`update-pin-row` and `shape-pin-row` for a shape pin whose `files:` is not a list), #401 (`plan --help` for `--spec-path` and `--code-path`) and #456 (`make bootstrap` escapes what the console cannot print). - Section 6 drops #290 and #391, which those PRs closed. - Section 6 also drops #225 and #294. Both are still open, but offline probes at 437a360 show #314 fixed them: - `scaffold-project.py --spec-path null --dry-run` refuses `scaffold-manifest-invalid` before anything exists; - `adopt-project.py check` on a plan with `elected_on: someday` and no `reference:` refuses `election-date-malformed`. - #181 now names the input that still reaches it, a source file named `null`. The probe showed `plan` writing `path: null` and `check` then finding the path uncovered. - New unfiled checklist items join section 6: - #458's `ImportError` item, now that #456 is on main; - #457's L2-6; - the `CONTRACTS_DIR` item; - four adoption inputs that cannot be tested on Windows. - Section 7 is rebuilt from the milestone, which now holds 30 open issues. #340 is closed, so its line is gone. Corrections: - **Section 1** states the promise as ruled, then says that on Windows park and resume run under WSL2, because native Windows parks nothing (README). - **The Windows CI sentence** says the adoption suites skip there until #471 lands, with a comment to drop the qualifier then. - **The pin procedure** ends its push line without a `# comment`. An interactive zsh without `interactivecomments` would hand the comment's words to `git push` as refspecs, the class #277 reports. The pull request step is in prose after the block. The section also says that the shape pin always records a commit, and that `apply` counts the files it re-copies before it asks. - **The YAML reader** sentence names `yaml-unparsable` beside `yaml-unreadable`, and claims no more than the reader does. - **The doctor** sentence keeps AGENTS.md's qualifier, "into the repository", and says that outside a checkout it clones the standard first. - **#303** gives both outcomes. **#389** is described as the gap that #305's refusal covers. - **The hermetic-tests** sentence claims only what its PRs and AGENTS.md establish: no test reaches the real `gh`, and the suite scaffolds into local bare repositories. Lane: openreposhape-4 (openRepoShape-4) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap
added a commit
that referenced
this pull request
Oct 12, 2026
…ut family.yaml before creating anything, `--dry-run` too, so `--id null`, `--tracking-branch true`, a blank `--created-by` and `--created-on 20261011` are refused (#307) (#397) `family.py init` rendered `templates/family-root/family.yaml` from `--id`, `--org`, `--name`, `--created-by`, `--created-on` and `--tracking-branch` and pushed it. The holder reads that file with its own YAML reader and its own `scripts/validate-family.py`, and `init` asked neither anything. Every row of #307 reproduces on c7c02fa (the issue's own script, with a stub `gh`): `--dry-run` and the real run both exit 0 for all 15 rows, and - `--created-by ' '`, `--created-on 20261011` / `2026-W41-6` / `tomorrow`, `--id null|true|false`, `--org null`, `--name 'Contoso Group'` leave a holder whose `validate-family.py` exits 1 from its first commit. `init` cannot be run again over it (`family-remote-exists`, no `--force`). - `--tracking-branch true`, `007`, `null`, `2026` exit 0 with `validate-family.py` exit 0 (it has no rule about `tracking_branch`), the holder pushed on that branch, `family.yaml` reading back as `True`, `7`, `None`, `2026`, and `update-shape.py apply --branch` printing `gh pr create --base True` / `--base 7` / `--base main` (for `null`) / `--base 2026`. **What changed.** `scripts/family.py` asks two questions about the manifest `init` is about to write, right after the substitution table is built: before the plan is printed, before the `--dry-run` return, before `_create_holder_remote` (which is where `gh repo create` runs against GitHub). Both refuse, name the flag, and say nothing was created. - `family_manifest_misreads` renders each line of the TEMPLATE that carries one of the values, reads it back with the holder's own reader (`repo_shape.parse_yaml`) and refuses a value that does not come back as the same text: `family-manifest-misread`. This is what catches `--id null`, `--org null`, `--created-on 20261011` and the silent `--tracking-branch true|007|null|2026`. A line the refusal quotes is shown the way Python spells it where it holds a line break or an escape, so a value cannot forge a second line of the refusal. - `family_manifest_invalid` renders the whole `family.yaml` and runs the validator the template ships over it, loaded from `templates/family-root/scripts/validate-family.py` (`Report` and `_check_envelope`), never re-implemented: `family-manifest-invalid`, in the validator's own words. This is what catches a blank `--created-by`, `--created-on 2026-W41-6` / `tomorrow` and `--name 'Contoso Group'`, and whatever the validator learns next. A misread is reported first and alone, because the validator's finding about it (`id is None`) is its consequence. A validator that is missing, broken, raising or exiting, whose `Report` has no `findings`, or that answers with a finding that is not text, is `family-validator-unusable`, not a traceback and not a zero. - A value no flag supplied is named by where it came from (`--id (not given, so --family lowercased)`). - `init --help` and `docs/cli.md` (regenerated, `render-cli-reference.py --check` ok) say what is held to what. No template is edited. A refused value is refused, not quoted or corrected. **Fix round 1 (the review of 14e2d68).** The first review found one broken test elsewhere in the suite and four claims no test guarded. This round is one commit on top of 14e2d68 (tests, plus one change to `scripts/family.py` for the validator that cannot be asked); the first commit is not amended. - **`tests/test_stdout_encoding.py` (the blocker).** The `family-init-dry-run` row ran `init --name <non-ASCII> --dry-run` and asserted the name is printed in the plan. The holder's own `validate-family.py` refuses that name (`naming-unclassified`: a family holder is one CamelCase token), so `init` now refuses it before the plan prints. That is this PR working as intended, so the TEST changed and the refusal did not: the row drops `--name`. The plan still prints the non-ASCII `--created-by` on its `created by` line, so the constrained stdout still has a value to escape (`created by 日本 ...` under `cp1252`, the characters themselves under UTF-8), and the row still fails if `family.py main` stops calling `escape_what_stdout_cannot_encode`. On the first head the old row failed (`1 failed`); the row as it is now passes on this head, and on this head merged with origin/main. - **A validator that cannot be asked.** The in-process call now sits in one `try` that also catches `SystemExit` and reads `report.findings` there, and a finding that is not text is refused by name. Three cases used to escape the refusal: a `Report` with no `findings` (an `AttributeError` traceback), a finding that is not a string (a `TypeError` from the regular expression) and a `_check_envelope` that calls `sys.exit(0)` (`init` exited 0, having printed and created nothing). Each is now `family-validator-unusable`, exit 2, nothing created; one test per case over a deliberately broken copy of the template, and one run at the command line for the exit. - **`_line_shown`.** A value with a line break, an ESC and a would-be forged line (`x\nFINDING forged: y\x1b[2J`, and a CR, U+2028 and U+0085 variant) is one escaped sentence: asserted on `family_manifest_misreads` itself, through `main` and what it prints (in process, so no argv crosses to Windows), and at the command line (skipped on Windows before any call, since argv is UTF-16 there; the in-process rows hold it on every OS). - **Two more guarded hunks.** The `sys.path` restore (also when the validator fails to load) and the comment-line skip each had a surviving mutant; each now has a test that kills it. The bytecode flag's restore is guarded by the same tests. - **The bytecode test** asked the REAL template's `__pycache__` before and after, so any stale `validate-family*.pyc` there failed it whatever the code did. It runs over a copy of the template made without a `__pycache__`, with the flag cleared first so that it is `family_validator` that must set it, and a control shows that importing the same file that way does write a `.pyc`. **How this composes with #259 (Brett ruled to quote the placeholders).** The misread check never says which lines are bare: it renders the real template line and reads it back. So once `fix/template-quoted-placeholders` (PR #395, first head 233dc71) quotes `id:`, `org:`, `created_on:`, `visibility:` and `tracking_branch:`, `--id null`, `--org null` and `--tracking-branch true|007|null|2026` read back exactly, the first question refuses nothing, and no edit is needed here. What the validator refuses stays refused: `--created-on 20261011` becomes `family-manifest-invalid` (`family-created-on`) instead of a misread, and a blank `--created-by` and `--name 'Contoso Group'` are unchanged. Proved three ways, all in `tests/test_family_init_values.py`: - the same values through a copy of the template with the placeholder forced quoted: accepted, and `parse_yaml(render(...))` reads the exact text back; - an end-to-end run of a forged one-commit checkout of `scripts/`, `templates/` and `contracts/` whose `family.yaml` is forced quoted: `init` exits 0, the holder's own `validate-family.py` exits 0, `family.yaml` reads back as typed, and the branch pushed is the typed one (a second forged checkout forced bare: refused, nothing created); - the tests that read the real template's own quoting to say what to expect: run here with the real template as it is, and on a tree with #259's branch merged in (below), so the file passes whichever of this and #259 lands first. **Once #259 is on main, 19 rows skip for good and one sentence goes unreached.** With the template quoted, the 16 end-to-end rows of `test_init_asks_the_holders_own_programs_before_anything_is_created` whose quoted answer is "the run is fine" and the 3 of `test_an_id_the_family_name_would_derive_is_named_by_where_it_came_from` skip with a message that says why (the 8 `test_a_quoted_value_is_held_as_typed` rows un-skip and pass), and nothing fails. `MANIFEST_MISREAD_REMEDIATION`'s "family.yaml writes a value like these unquoted" is then text no real template reaches; a template that writes one bare (the forged bare checkout the tests use) still does. Nothing is changed here for it: whichever of this and #259 lands second can drop the dead rows and reword that sentence in the same PR. **Same helpers as #314 (#261)?** #314's `manifest_misreads` / `manifest_invalid` are hard-wired to the assembly root (`project.yaml`, its templated list, `validate-manifest.py`'s `_findings`) and were not on main when this PR was written, so this PR writes the smallest version the family needs, in `scripts/family.py` (not `shape_materialize.py`, which #266, #270, #314 and #316 all edit). They differ only in the template, the validator's entry point and the sentence that names the file. #314 landed first (3ef059c), and this PR leaves both sets of helpers as they are: **unifying them is left to a follow-up on #459's checklist (#459 (comment) **Evidence, refreshed when this PR was opened (head abb27e6).** Main had then moved to 28bdc84: #345 landed as 3ae0109, #350 as 0f67f7b, and lane 3's `single-repository.yaml` as 28bdc84. The review's re-verify of abb27e6 ran on the head merged with 28bdc84: clean, with the six init, encoding, CLI-reference and family files at 305 passed / 9 skipped, and `test_family.py` with #345's `test_family_dry_run.py` at 126 passed. Merge-tree was also clean against #259's 3894aa8 (PR #395) and #314's 4b8da3d. The results below were taken before that; main has moved since. - Reproduction, before and after, with the issue's script: on c7c02fa every row exits 0; on this branch all 13 failing rows now exit 2 on `--dry-run` and on the real run, with no holder, no remote directory and nothing pushed; the two controls (`--tracking-branch main`, `--name ContosoGroup`) are unchanged. `--tracking-branch 2026` is refused too, as #314 does for the scaffold: it worked only because every reader calls `str()` on the number. - `tests/test_family_init_values.py`: 125 tests, 103 of them fail on c7c02fa; on this branch 117 pass and 8 skip (the 8 rows that need the real template to be quoted, which they run against the day it is). On 14e2d68, the first head, the 4 tests of this round that guard a code change are the ones that fail (the three validator cases and the exit at the command line); the rest guard what 14e2d68 already did. - Mutations: fifteen of the change in the first round (drop the call, move it after the `--dry-run` return, move it after the remote is created, empty either question, ignore the template's quoting, ignore the template's validator, drop the not-given naming, drop a finding no flag owns, write bytecode into the template, let a validator's `SystemExit` through at load, ask the validator first, drop the unusable-validator checks), and eight more in this round, each killed by a new test: `_line_shown` always in backticks (9 fail), the `sys.path` restore dropped (2), the bytecode flag's restore dropped (2), the comment skip dropped (1), `findings` read outside the `try` (1), the non-text check dropped (1), `SystemExit` let through the run (2, one a zero at the command line), and `main` without `escape_what_stdout_cannot_encode` (the `family-init-dry-run` row of `test_stdout_encoding.py`). The write-bytecode mutation is now killed by the new bytecode test. - Targeted runs on the head of this branch, abb27e6 (Linux, Python 3.12.3, git 2.43.0, git-filter-repo 2.47.0): `test_family_init_values.py test_family.py test_stdout_encoding.py test_cli_reference.py`: 276 passed, 9 skipped (the 8 rows that need a quoted template, and one Windows-only row). - Merged with origin/main at 161c6c9 (which has #316, #317, #318, #245 and #349): `test_family_init_values.py test_family.py test_family_add_refusals.py test_family_add_excludes.py test_family_member_path.py test_stdout_encoding.py test_cli_reference.py` 407 passed, 9 skipped; `test_shape_doctor.py` 255 passed, 3 skipped (the PowerShell row, which needs a Windows host); `test_update_shape_add.py test_update_shape_pr_base.py test_template_values_single_pass.py test_scaffold_ignored_shape_files.py` 223 passed (with the git identity blanked, because the standard's own checkout sets a `user.name` that scaffold and adopt tests read). Main then moved to 0f67f7b (#350 landed, which changes the validator this reads, and a scaffold PR): `test_family_init_values.py test_stdout_encoding.py test_cli_reference.py test_validate_family_duplicates.py` on this branch merged with it: 197 passed, 9 skipped. - Merge-tree (`git merge-tree --write-tree`), all clean: origin/main at 0f67f7b; #345 (abaaf3c, which landed as 3ae0109); #314 (4b8da3d); and `fix/template-quoted-placeholders` (3894aa8, #259). This branch + origin/main + #345 + #259's branch together are clean too, and `test_family_init_values.py test_family.py test_stdout_encoding.py test_cli_reference.py test_family_dry_run.py test_validate_family_duplicates.py test_template_text_placeholders.py` there: 442 passed, 20 skipped (the 19 rows above, which skip because #259 quotes the template, and one Windows-only row). #345 changes `cmd_bump`, `cmd_remove` and `build_parser`, not `cmd_init`; #350 (landed) added member-name checks to the validator and left `_check_envelope` and `Report`, the two names this change asks, as they are. #314 at 4b8da3d did not merge with origin/main by itself then (`adopt-project.py`, from what landed after its last merge), so it was not composed with the rest, but it was clean against this branch alone. #309, #312, #316, #317, #318 and #350, which the first round merged against or named, have landed. - `--created-by '{{ORG}}'`: on c7c02fa `init` created the remote and then died with `template-unsubstituted` once the worktree was rendered. Merged with main, #316's one-pass `render` writes the value as typed: `init` exits 0, `family.yaml` records `created_by: "{{ORG}}"` and the holder's own `validate-family.py` is green. This change does not touch it and does not refuse it. (On this branch alone, without #316, the pre-flight render refuses it before anything exists, with the same `template-unsubstituted` text.) **Left out.** - `AGENTS.md`, `README.md`, `docs/handbook.html`: no prose change (#309 has landed since; `init --help` carries the sentence). - A calendar check of `--created-on` (`2026-13-45` passes the validator's `DATE_RE`): the validator's rule, not copied here. - A branch git itself refuses (`a..b`, `x.lock`): #283. - `--org test+org` passes `validate-family.py`; not claimed. - `--created-by` with an ESC or OSC is printed raw by the plan's `created by` line on c7c02fa and here: not part of this change. - No run against GitHub (the `gh repo create` ordering is proved in-process, with every step that would reach GitHub replaced by one that fails the test), none on macOS or Windows. Reviewed adversarially at 14e2d68 (CHANGES NEEDED: 1 blocker, 1 should-fix, 6 nits). The blocker was a test elsewhere in the suite that the new refusal turned red, `tests/test_stdout_encoding.py`'s `family-init-dry-run` row; the should-fix was the line-escaping hunk (`_line_shown`), which no test guarded. Fix round 1 (abb27e6) changed that test and not the refusal, guarded `_line_shown` and the two other surviving mutants, closed the three ways a broken validator escaped `family-validator-unusable`, and rewrote the bytecode test over a copy of the template. Its re-verification at abb27e6 found every finding closed: APPROVE WITH NITS (0 blockers, 0 should-fixes, 2 nits, both about this description and the commit trailer and not about the code: the trailer names Opus 5.5 below, and the description's merge-tree and composition paragraphs are refreshed above). CI is green on Linux, macOS and Windows, with SonarCloud passing, at d216ba8 (main 8782333 merged in). Closes #307 Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap
added a commit
that referenced
this pull request
Oct 13, 2026
… exists, a source symlink with an empty target and the source's own `project.yaml` kept in the root, and `plan` no longer offers `root` for that file (#303, #305) (#387) Two ways `check` said `plan ok` to a source `execute` could not adopt, both reproduced on c7c02fa with the issues' own scripts. **#303, a symlink whose target is empty.** A `120000` entry whose blob is the empty blob passed `check`, which never reads a link's target. - With symlinks on (git's default where the disk can hold one), `execute` created both leg repositories, then failed in its first clone of the source: exit 2, nothing on stderr, two empty legs left behind. Linux refuses a symlink to nothing. This happened whatever the plan answered: `docs/empty-link` sent to spec, and a top-level `empty-link` answered `root` or `drop`. - With `core.symlinks=false`, it said `adoption verified`. The spec leg held the empty link, a clone of that leg exited 128, and the assembly's recursive clone exited 1. The top-level link answered `root` or `code` verified the same way. Only `drop` gave an assembly that clones. **#305, the source's own `project.yaml` answered `root`.** `plan` asked about it under `ambiguous-top-level-yaml`, whose question offers `root`. Answered `root`, `check` said `plan ok`. `execute` wrote the shape's manifest beside the source's file as `shape/project.yaml`, said "nothing was overwritten" and `adoption verified`. `shape-doctor.py` then said NOT A SHAPE ROOT (exit 2) and pointed at `adopt-project.py plan`. `make bootstrap` failed `naming (exit 2), manifest (exit 1), pins (exit 2)`. Answered `code` or `spec`, the doctor said COMPLIANT and `bootstrap ok`. **What changed.** - **`plan-empty-symlink` (#303).** `_empty_symlink_findings` reports each `120000` row of size 0 in `Source.tree()`. The size `ls-tree -l` gives a symlink is the length of its target, so no target is read; a SHA-256 repository reports size 0 too. - It is listed right after #196's `plan-unsafe-path`, through `_source_path_findings`, on the line `_entry_findings` already had. So `check` prints it, and `execute` refuses it through `_refuse_what_check_finds` before any leg exists (#168). - `EMPTY_SYMLINK_REMEDIATION` is said once after the findings, in `check` and in `execute`'s refusal. The fix is in the source: give the link a target, or remove it, commit, and re-run `plan`. - `plan` notes each such link in `check`'s words, and still writes the plan. - **`ambiguous-project-yaml` (#305).** A new rule in `contracts/path-classification.yaml`, above `ambiguous-top-level-yaml`, for the file `project.yaml`. Its question asks spec, code or drop, and says `root` is not an answer for this file. `release.yaml` keeps the old question. - **`plan-root-blocks-manifest` (#305).** #313's machinery is extended, not copied: `_shape_blocks`, which `_root_shape_findings` runs over each disk, now refuses the root holding `project.yaml` outright, with no copy question after it. It fires for a kept row at that path (file or symlink), or a kept directory there, on a disk that keeps case or one that ignores it. The finding's remediation is #313's `_fix_the_kept_path`. - A submodule there stays `plan-submodule-root`'s, said once. - `_kept_there` is the at-or-under lookup the copy branch already did, factored out for both. - `execute` refuses it before any leg exists, through the same list. - **Files.** `adopt-project.py` (+205/-13), `contracts/path-classification.yaml` (+28), the two new test files (399 and 366 lines) and `tests/test_adopt_root_shape_paths.py` (+6/-2). **Decisions.** - **#303 is refused whatever the plan answers, and whatever `core.symlinks` says.** - Where symlinks are made, no answer adopts. The run fails at the first clone of the source, which checks the whole tree out before anything is taken out. Measured with the link in `docs/` (sent to spec), at the top level answered `root` and `drop`, and with every path rooted or dropped and both legs seeded: there the seeded legs were pushed and the assembly's clone exited 128. - `check` cannot know where `execute` will run. A check that passed `drop` with symlinks off would pass a plan the next Linux machine cannot adopt, and the root's own `validate` workflow runs on `ubuntu-latest`. - The remediation costs one commit in the source. - **#305 covers the manifest only.** The issue asked whether other rendered root paths need the same rule. Measured on c7c02fa, with the source's own file kept in the root at each of the 18 `assembly_root_paths()`: - `check` passes all 18. - These verify over a broken assembly: `contracts/spec-pin.yaml` and `contracts/code-pin.yaml` (DRIFTED, pins fail); `scripts/validate-pins.py`, `scripts/validate-manifest.py`, `scripts/repo_shape.py` and `contracts/repository-naming.yaml` (INVALID); `scripts/validate-repository-naming.py` (COMPLIANT, naming fails); `scripts/bootstrap.py` (COMPLIANT, and `make bootstrap` runs the source's own script). - `contracts/shape-pin.yaml` fails verification after both legs exist. - README.md, AGENTS.md, CLAUDE.md, .gitignore, .gitattributes, AGENTS-shape.md and the workflow come out COMPLIANT with the copy beside them. `Makefile` is rooted by default, and the follow-up merge is its designed exit. - `project.yaml` is the one path `plan`'s own question offered `root` for. The others are reached only by hand-editing a `contracts/` or `scripts/` answer to `root`. They are left for a follow-up issue (under Left out), not folded in here. - **A symlink at `project.yaml` is refused too.** On c7c02fa: - a link to the source's own file verified and was NOT A SHAPE ROOT; - a link to nothing had the manifest written through it, and came out COMPLIANT on Linux. A clone that makes no symlinks holds that link as a file naming its target, so it is refused anyway, and #183 refuses a link to nothing at a shape path regardless; - a directory `project.yaml/` verified and was NOT A SHAPE ROOT. - **A case twin `Project.yaml` kept in the root is refused, with the case aside.** It adopts COMPLIANT on Linux. On a disk that ignores case, `place` writes the manifest beside it. This is the trade #172 and #313 already make. **Evidence.** In this section and in the two below it, "this head" is 69c54c9, the branch's own commit and the one the review approved; main and #314 were merged into the branch afterwards (last paragraphs). - **Reproductions, on this head** (exits as printed): - #303 run 1: `check exit 1`, one `plan-empty-symlink` finding; `execute exit 2`, `REFUSED plan-empty-symlink`, `leg repositories left:` empty. - #303 run 2 (`core.symlinks=false`): the same finding and refusal; no leg, no verification. - Top-level `empty-link`, symlinks off and answered root, code or drop, and symlinks on and answered root or drop: each `check exit 1`, `execute exit 2`, no leg. - #305 root: `check exit 1`, one `plan-root-blocks-manifest` finding; `execute exit 2`, `leg repositories:` empty. - #305 code, spec and drop: `plan ok`, `adoption verified`, the root's `project.yaml` begins `schema_version: 1`, doctor COMPLIANT, `bootstrap ok`. - The plan's entry for `project.yaml` now reads `rule: ambiguous-project-yaml`, with no `(root)` in its question. - `project.yaml` as a link to a file, a link to nothing, a directory, and the twin `Project.yaml`: each `check exit 1` and `REFUSED plan-root-blocks-manifest`. - No row needs real GitHub or a cloud tool. - **New tests.** `tests/test_adopt_empty_symlinks.py` (28) and `tests/test_adopt_root_manifest.py` (27). - The symlinks are made with `git update-index --cacheinfo`, so no disk symlink is needed. The one test that checks a link out (a link given a target, end to end) is skipped on Windows by its marker, before it runs. - On c7c02fa: 37 failed, 18 passed. The 18 are the controls that must pass on both sides: a symlink with a target, the remediations adopting, `project.yaml` sent to a leg or dropped, a submodule there, other shape paths keeping their copy, and `release.yaml`'s question. - **One existing test changed.** `test_a_file_kept_at_a_path_the_shape_writes_is_not_a_finding` in `tests/test_adopt_root_shape_paths.py` held that a kept file at any shape path is no finding. It now holds that for every path but the manifest, which is `plan-root-blocks-manifest`. - **Mutation probes:** 14 of 14 hunks reverted one at a time, each killed by at least one new test. - **Targeted runs.** The adopt list and the doctor ran on a local state of this branch (7a22a21) whose files were this head's but for `tests/test_adopt_root_shape_paths.py`. That file was re-run on this head, so the adopt list is two runs, not one sum. - The 15 adopt files (`test_adopt_empty_symlinks`, `_root_manifest`, `_root_shape_paths`, `_leg_path_collisions`, `_execute_refusals`, `_plan`, `_e2e`, `_submodules`, `_verify`, `_mount_ignored`, `_ignored_shape_files`, `_entry_paths`, `_finding_spelling`, `_history_gitlinks`, `_spec_only`) on 7a22a21: 837 passed, 1 failed, 1 skipped (a MedxEHR clone the plan test needs). The one failure is the #313 test this head changes. - `tests/test_adopt_root_shape_paths.py` alone, on this head: 120 passed. - `tests/test_shape_doctor.py`, as its own stream: 242 passed, 3 skipped (PowerShell lines only a Windows host prints). - `tests/test_scaffold_leg_path_collisions.py`, `tests/test_repo_hygiene.py` and `tests/test_cli_reference.py`: 167 passed. - This head merged with origin/main at 87373a6, which brings `tests/conftest.py`'s new `git()` identity: the two new files, `tests/test_adopt_root_shape_paths.py` and the three above, 342 passed. **The review.** An independent opus review at 69c54c9 approved with nits: 0 blocker, 0 should, 8 nit. It reproduced every row of both issues on c7c02fa and on this head, and killed 6 of 6 mutants: two of the author's, re-run, and four of its own. Its targeted runs on this head merged with origin/main ab623dd (`git merge-tree` clean, with #349's session `gh` guard in place): - the new files, `tests/test_adopt_root_shape_paths.py`, `tests/test_scaffold_leg_path_collisions.py`, `tests/test_adopt_execute_refusals.py`, `tests/test_repo_hygiene.py` and `tests/test_cli_reference.py`: 379 passed; - the other adopt files, in two streams: 303 passed, 1 skipped (the MedxEHR clone), and 323 passed; - the clone-template, literal-names, template-values, plan-escapes, stdout-encoding and Windows-path tests, which cover #268 and #270: 266 passed, 2 skipped (an operator `user.name` set; Windows only); - `tests/test_shape_doctor.py`: 254 passed, 3 skipped (PowerShell lines only a Windows host prints). c7c02fa's own `tests/test_adopt_root_shape_paths.py` against this head's code gave 1 failed (the `project.yaml` row, `:191`) and 119 passed, so no other #313 behaviour changed. - **A cross-lane composition, with lane 1's #183.** On the tree merged with lane 1's `fix/adopt-root-kept-symlinks` (#183, 6c44591), `tests/test_adopt_root_symlinks.py::test_a_link_with_an_empty_target_is_fixed_in_the_source` fails: `AssertionError: assert 'plan-empty-symlink' == 'plan-root-symlink'`. An empty link at or above a shape path is reported by both findings, and `execute` refuses with the first, which is this branch's. The two remediations agree (fix it in the source). Whichever of the two lands second removes the duplicate report. The same file also fails 3 other tests on origin/main with #183 and without this branch (`[a-link-at-the-copy-itself]` and `test_the_materializer_refuses_before_it_writes`, two cases), because #313's `plan-root-blocks-shape-copy` and `materialize-copy-blocked` answer first; those are not this branch's. - **Two known nits, left for a later round.** - An empty link AT `project.yaml` (or at a #313 copy path) gets `plan-root-blocks-manifest`, whose fix offers `drop`, and `plan-empty-symlink`, which says `drop` is no fix. - `plan` notes every empty link (25 links print 25 NOTEs), where `check` names 20 and counts the rest. - The other nits are smaller. A symlink target that begins with a NUL byte passes `check` and fails like #303, and the issue's `_verify` bullet, are follow-up issues; the case twin `Project.yaml` is under Left out. The new tests' `SYMLINKS_OFF` replaces the Windows job's own `GIT_CONFIG_COUNT` settings (`core.autocrlf=false`, `core.longpaths=true`) for those child processes, which is harmless while none of them checks anything out. **Merge-tree, as of 2026-10-12T15:14Z.** `git merge-tree --write-tree --name-only` of this head (69c54c9), against heads fetched then; every sha below is the one it had then, and main has moved since (see the merge paragraph below). It is a textual merge only; the semantic meeting with #183 was tested by the review (the first item above). - origin/main: clean at 0f67f7b. That includes #316 (34ecbc2), #268 (e43fa95), #270 (f795cf7) and #245 (161c6c9), which have landed since this branch was cut. It was clean at 161c6c9 and at ab623dd, the review's merge, too. - Open PRs touching `adopt-project.py`: #184 (f416e13), #353 (dcbc37b), #356 (5a5011d), #371 (34b9500), #373 (5ac03de), #377 (c35c896), #378 (2da8691), #382 (7e55d00) and #384 (a96297e) are all clean. #356 and #382 conflict with origin/main alone (2 and 5 blocks, from #245's landing), not with this head. - #314 (4b8da3d) was NOT clean: one block, in `_entry_findings`' docstring, where each side adds a paragraph after the same one (this head's "THEN THE SOURCE'S EMPTY SYMLINKS (#303)" and #314's "A VALUE NO FILE CAN HOLD"). Keep both. It had been clean at f45edf4, and it had 3 blocks against origin/main alone since #245 landed, so it needed a merge of main in any case. - Lane 1's branches without a PR: `fix/adopt-late-failure-notice` (9f1778d), `fix/adopt-record-submodule-placement` (70f4d29) and `fix/adopt-plan-quote-yaml-keywords` (d38e143) were clean. `fix/adopt-plan-quote-yaml-keywords` was 3 blocks against this head at e337f44, and conflicts with origin/main alone in 4. - `fix/adopt-root-kept-symlinks` (6c44591, #183) conflicts in 3 blocks of `adopt-project.py`: the module docstring's `check` paragraph, `_entry_findings`' docstring and its return line. They are the same 3 that c7c02fa itself has with that branch, and this branch's one added docstring line sits inside the first. It has 5 blocks against origin/main alone. - Lane 5's `fix/adopt-pin-writes-pin-file` (d806e8f): clean. `docs/adopt-execute-exits` is #371, above. - When #183 lands, an empty symlink kept at or above a shape path is reported twice, by `plan-empty-symlink` and by #183's `plan-root-symlink` (the first review item above). A dangling link kept at `project.yaml` is reported twice too, by `plan-root-blocks-manifest` and `plan-root-symlink`, and an empty one three times. Whichever lands second should keep one. **Brought onto main past #314 (bbbb541).** Since this branch was cut from c7c02fa, main took #245 (relative `.gitmodules` urls), #270 (`plan-unwritable-value`), #371 (`execute` verifies before it exits on a refused `gh pr create`), #377 (`plan` asks about every path `execute` will not extract) and #314 (`plan-manifest-invalid`, 3ef059c), all in `adopt-project.py`. The merge of origin/main 3ef059c has ONE conflict, in `_entry_findings`' docstring, where this branch adds "THEN THE SOURCE'S EMPTY SYMLINKS (#303)" and #314 adds "A VALUE NO FILE CAN HOLD IS SAID BEFORE THE VALIDATOR'S WORD ON IT (#255 before #261)" after the same paragraph: both are kept, this branch's first, because it describes the head of the list and #314's its tail. Nothing in code conflicted. `check` and `execute` read one list, and it composes as each feature says: a source's unsafe paths (#196), its empty symlinks (#303), coverage, `_leg_findings` with `plan-root-blocks-manifest` (#305), #166's split, #270's `plan-unwritable-value`, #314's `plan-manifest-invalid`, then #245's relative urls last. So an empty symlink beside an invalid value is still refused wearing `plan-empty-symlink`, with no leg created; probed on the merged tree with an empty link plus `visibility: secret`, plus a line break in `visibility`, and with `project.yaml` answered `root` plus `visibility: secret`, each listing the findings in that order. No test expectation or order changed: the merged tree was rehearsed before #314 landed (69c54c9 + 45e01f4 + c6f441b) and is the same tree object as the pushed merge, and `git diff 3ef059c bbbb541` is this PR's five files and nothing else (+1004/-15). Targeted runs on bbbb541: the new files, `test_adopt_root_shape_paths`, `test_adopt_manifest_values`, `test_scaffold_leg_path_collisions`, `test_agents_md_codes` and `test_cli_reference` 418 passed; `test_shape_doctor` and #377's, #245's and #270's adopt tests 595 passed, 4 skipped (PowerShell lines only a Windows host prints; one operator `user.name`); the other adopt files 234 passed, 1 skipped (the MedxEHR clone) and the guards, literal-names, clone-template, plan-escapes, gitmodules, leg-paths, stdout and Windows-path tests 1101 passed, 1 skipped. **Left out.** - `_verify` and the materializer do not themselves refuse a root `project.yaml` that is not the shape's manifest. This is the issue's second bullet. Once `execute` refuses the plan before any leg, through the same list `check` prints, nothing reaches them with one. It is a follow-up issue. - A top-level file named `project.yaml` only once case is ignored (`Project.yaml`) is still asked `ambiguous-top-level-yaml`'s question, which offers `root`. `check` refuses that answer with the case aside, before any leg. - #171 (a failed clone exits 2 silently) is lane 1's PR #184. This PR only keeps the empty-symlink run from reaching that clone. - Follow-up issues proposed by this PR (two since filed): - the source's own file kept in the root at one of the other shape paths the root's tools read (the pins, the validators, `scripts/repo_shape.py`, the naming policy, `scripts/bootstrap.py`, `contracts/shape-pin.yaml`), each measured above; - a symlink whose target begins with a NUL byte (`\0spec.md`, size 8), which `plan-empty-symlink`'s size test does not see: it fails like #303 (filed as #388); - `_verify` asking whether the split's root `project.yaml` is the shape's manifest (filed as #389); - an entry git cannot check out where `execute` runs, other than an empty symlink or a NUL-led one (a target longer than the platform allows, say), which is still found only by the first clone, after both legs exist. #303 leaves this as a note. Reviewed adversarially at 69c54c9 (APPROVE WITH NITS: 0 blockers, 0 should-fixes, 8 nits). The review reproduced every row of both issues on c7c02fa and on that head, and killed 6 of 6 mutants. No fix round followed: the branch's own commit is as reviewed, and what was added after it is the merge described above. CI is green on Linux, macOS and Windows, with SonarCloud passing, at fce86fa (main 7a8b981 merged in). Its merge with main a73b4e9 passed the targeted set (the three adopt test files, `tests/test_shape_doctor.py` and `tests/test_scaffold_leg_path_collisions.py`): 597 passed, 3 skipped. Closes #303 Closes #305 Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap
added a commit
that referenced
this pull request
Oct 13, 2026
…d `open_glass` share one `SHAPE_PIN_SOURCE_<PRODUCT>`, so `--pin` refuses the pair and `validate-pins.py` notes a declared one and reads the variable for neither (#275) (#409) `pin_source_env_name` turns every character outside `[A-Z0-9_]` into `_`, so `open-glass`, `open_glass` and `open.glass` are three legal product names, with three pin files (`contracts/<product lowercased>-pin.yaml`), and one variable, `SHAPE_PIN_SOURCE_OPEN_GLASS`. On ab623dd (this branch's main parent; the same rows reproduce on c7c02fa), with the issue's own probe against local bare remotes and a `gh` that exits 97: | step | ab623dd | this head (6494714) | |---|---|---| | `scaffold-project.py --pin open-glass@<A> --pin open_glass@<B>` | exit 0, three repositories | exit 2 `scaffold-pin-collides`, nothing created | | `validate-pins.py`, no source, on a project that already declares the pair | exit 0, both skips offer `SHAPE_PIN_SOURCE_OPEN_GLASS` | exit 0, one `note neutral-pin-source-collides`; the skips offer `--pin-source <product>=<path>` and a sibling checkout only | | `SHAPE_PIN_SOURCE_OPEN_GLASS=<open-glass>` | `neutral-pin-source-unreadable` for `open_glass`, exit 1 | exit 0, the note; the variable is read for neither pin, so each pin is read from its own source or skipped | | `SHAPE_PIN_SOURCE_OPEN_GLASS=<open_glass>` | `neutral-pin-source-unreadable` for `open-glass`, exit 1 | the same | | `--pin-source open-glass=... --pin-source open_glass=...` | exit 0 | both pins recompute, exit 0, the note | **What changed.** Every variable name is kept: an operator may have set one and the docs name them. What is done instead: - `scripts/repo_shape.py`: `pin_source_env_collisions(products)` is the one pure answer, `{variable: [products]}` for each variable two or more distinct products share. Distinct means distinct PIN FILES (`pin_product_key`, `lower()`, which is what `contracts/<product lowercased>-pin.yaml` is named by), so a product typed twice or in two cases is #241's file collision and never this one. `pin_source_env_shared` is the set of those keys. - `scaffold-project.py`: `_refuse_colliding_pins` (left where it is) names each variable's `--pin` arguments as typed after #241's file sentences, in the same `scaffold-pin-collides` refusal. Dry run and real run say the same no, before any process starts and ahead of the leg-path refusal. #241's wording and remediation are unchanged when only a file collides. A NEW pair is still refused, so the situation is not created anew. - `templates/assembly-root/scripts/validate-pins.py`: one NOTE per variable, `neutral-pin-source-collides`, naming every product and the variable, the keyed `--pin-source <product>=<path>` that reads each pin, and that which declaration stays is the human's decision. It is never a finding and never moves the exit code. A product whose variable is shared does not read it (rule 2 of `resolve_neutral_pin_source`), and its skip hint offers the keyed `--pin-source <product>=<path>` and not the variable or the bare form. A variable one product has to itself reads and is offered exactly as before. - `scripts/repo_shape.py`, the two chain readers: `link_pins_from_trees` does not read a variable two links of one chain share, and the `declared-unverified` hint stops naming it. Only a hand-written chain can have two such links, since scaffold takes every link but the first from the naming policy's alphanumeric `open<Product>` form. Each reader of `pin_source_env_name`, decided: | site | decision | |---|---| | `scaffold-project.py --pin` | refuse (above) | | `validate-pins.py` rule 2 and the skip hint | a note, and the variable is neither read nor offered for a shared product | | `repo_shape.py` `resolve_link_source` (via `link_pins_from_trees`) | skip the variable when another link of the chain shares it | | `repo_shape.py` `_chain_link_findings` hint | omit the variable for a shared link | | `adopt-project.py --pin`, `plan`, `check`, `execute` | not in this PR, see below | **Fix round 1 (from the review of 34dbd86).** **The design change: a declared pair is a NOTE and not a finding. This is the lane coordinator's decision, and Brett may reverse it.** The first round made `neutral-pin-source-collides` a FINDING. Main through ab623dd scaffolds such a project (`--pin open-glass@A --pin open_glass@B` exits 0), and `update-shape.py apply` runs the NEW `validate-pins.py` over the rewritten root, so for a project already declaring the pair the finding ended every apply as `REFUSED update-validators-red`, rolled back, for this update and every later one, until a person deleted a true declaration of descent. AGENTS.md tells an agent never to edit "a pinned file, a digest or a manifest to make a row go green", and the finding's remediation was exactly that, so the person was locked out. The coordinator chose the review's option (b): it is a NOTE now. The guard stays (the variable is read for neither pin), each pin still verifies from its own source, scaffold still refuses a new pair, and `validate-pins.py` says the rest in one line. To put the finding back, restore 34dbd86's `report.finding(...)` call in `_report_pin_source_collisions` (it is not a word swap for `report.advise`: `advise` takes one string, `finding` a code and a detail), give the finding the remediation 34dbd86 had, and turn the tests that expect the note (and `test_a_project_declaring_a_colliding_pair_takes_a_shape_update`) back into exit 1; the first round's versions of all of them are on 34dbd86. - **Proved on a real old project.** Main (ab623dd) scaffolded the pair into local bare remotes; `update-shape.py apply` to the first round's head 34dbd86 exited 2 `REFUSED update-validators-red` and rolled back (`git branch` shows only `main`); to this head it exits 0, the project's validator passes and prints the note, and the doctor's `pins` row is `ok` before and after. The doctor prints `passes` for a validator that exits 0 and does not echo its stdout, so the note itself is visible in `update-shape.py apply`'s output and in `validate-pins.py`'s, and the verdict moves for no reason but the update (`COMPLIANT, SHAPE BEHIND` to `COMPLIANT`). It is now a test: `test_a_project_declaring_a_colliding_pair_takes_a_shape_update`, over local stand-ins beside the project, so no `gh` is asked. - **Collisions by the variable.** Pairs were decided by `casefold()`, but the pin file is `lower()`. `openß`/`openss`, `openſ`/`opens`, `openfi`/`openfi` and `openς`/`openσ` (the letters are U+00DF, U+017F, U+FB01 and U+03C2 beside U+03C3) share a variable and have two pin files, and were never found. They are now, from the pin-file fold (`pin_product_key`). A pool of 31 names is checked pairwise against `neutral_pin_path` and `pin_source_env_name` themselves, and no single product is ever a collision. They are reachable only from a hand-written `project.yaml`. - **Windows.** The guard's only test needed the `#!` `gh` stub and was skipped on Windows. `test_a_shared_variable_answers_for_neither_pin_with_no_gh` puts a checkout of each pin beside the project, names one in the variable, and needs no `gh`: on ab623dd it is `neutral-pin-source-unreadable`, here each pin reads its own sibling. The hint and the non-ASCII rows are tested in-process, with no skip and no filesystem. - **The hint.** For a product two pins share, the skip hint offers the keyed `--pin-source <product>=<path>`: the bare form answers for every pin with no more specific source, so with two pins it gives the other the finding this issue is about. - **Trailers and title.** The trailers of the earlier commits (a30091f and 34dbd86 name Sonnet 5.5) are settled in this message, which carries Opus 5.5. The PR title now names `repo_shape.py`. - `scaffold-project.py` is not touched this round: `fix/adopt-pin-writes-pin-file` moves that code, and the change would only conflict with it. **Left out.** - `adopt-project.py` is untouched. Its half arrives with #273's PR, which moves scaffold's pin machinery into a shared module and routes adopt through `pin_source_env_collisions`. Once it lands, the one place that still keys by `casefold()` (`pin_variable_collisions` in `scripts/shape_materialize.py`) should read `pin_product_key`; it cannot differ for an ASCII `--pin`, which is all `PIN_ARG_RE` takes. - `scaffold-project.py`'s own `_colliding_pin_variables` also keys its argument lists by `casefold()` and one docstring there still says the project's validator reports the pair "as a finding"; both go with the code when #273 moves it. #312's `PinSources` keys `--pin-source` by `casefold()` too, so `openß=` and `openss=` are one key there; that is #312's. - A product in `neutral_product_pins:` and a chain link in `naming.referent_chain` that share a variable are read by two validators in one process, and neither sees both sets. Not detected here. - The doctor does not print a passing validator's stdout, so its `pins` row does not carry the note. Not changed: it would add a file to this PR. - No `--help` text changed, so `docs/cli.md` is untouched. `README.md` is at its line cap and its blob is pinned by `docs/handbook.html`, so the one README sentence this wants needs a regenerated handbook and waits for its own change. - `validate-pins.py` and `repo_shape.py` ship into projects, so every project's `shape-pin.yaml` reads `upstream-changed` for both until `update-shape.py apply`, which now succeeds for a project that declares such a pair. **Evidence.** All on the head (6494714), with git-filter-repo 2.47.0 on PATH, a `gh` that cannot be reached (a stub first on PATH, `GH_TOKEN` bogus, an empty `GH_CONFIG_DIR`), and the repository-local `user.name` blanked at command-line scope, as the standard's checkout sets one that two scaffold tests read. - `tests/test_pin_source_env_collisions.py` is 137 tests and passes on this head. Laid over 34dbd86, 50 fail and 87 pass: the note and update tests, the non-ASCII rows, the hint, the sibling guard, and `pin_product_key` itself (which 34dbd86 does not have); the scaffold refusals, the chain readers and the controls hold on both. - Targeted, run on ad6f569, whose tree differs from this head in `tests/test_pin_source_env_collisions.py` only (the last edits made one comparison read the committed blob and wrapped one line): the new file, `test_pin_validator`, `test_neutral_pin_validator`, `test_cli_reference`, `test_manifest_validator`, `test_naming_policy` and `test_repo_hygiene` **445 passed**; the scaffold pin, collision, referent-chain, leg-path and e2e files **245 passed**; `tests/test_update_shape*.py` **218 passed, 3 skipped** (junctions, NTFS only); `tests/test_shape_doctor.py` **254 passed, 3 skipped** (the PowerShell line, a Windows host only). - On this head itself: the new file, `test_repo_hygiene`, `test_cli_reference`, `test_pin_validator` and `test_neutral_pin_validator` **244 passed**. - This head merged with origin/main a983e6a, in a throwaway worktree: the new file, `test_pin_validator`, `test_neutral_pin_validator`, the scaffold pin, collision, referent-chain and e2e files, the whole doctor file and `test_cli_reference` **590 passed, 3 skipped** (the PowerShell line); the new file alone, in its final form, **137 passed**. origin/main had moved twice since; the merge-tree below is against it. - Twelve mutation probes, each hunk reverted alone: all caught. The note-to-finding one is caught by the update test (a committed mutant, since the test clones the repository's HEAD) and the guards by the sibling test, run with every test that needs a `#!` stub deselected as a Windows runner would; the hint one by an in-process test. - Nothing ran on macOS or Windows before this PR's CI. **Merge-tree, as computed during the fix round on 2026-10-12** (every sha and head is the one it had then; main and the other branches have moved since). `git merge-tree --write-tree` of this head (6494714) against origin/main 1f90a0f is clean, and so is the merge of origin/main a983e6a, which the merged-tree runs above used. #312, #309, #270, #268, #228, #245 and #351 have landed. The rest were computed on ad6f569 (the same tree but for one test file) against a983e6a. Open PRs touching this branch's files, all clean against the head: #314, #355 and #373 (`scaffold-project.py`), #343 (`validate-pins.py`), #390 (`repo_shape.py`) and #394 (`repo_shape.py`, which conflicts with origin/main by itself). The other 25 open PRs and the lane-5 branches `fix/adopt-check-link-and-manifest`, `fix/scaffold-free-text-values`, `fix/family-init-values`, `docs/adopt-execute-exits`, `fix/yaml-flow-brackets-surrogates` and `fix/adopt-pin-writes-pin-file` (#273's, 7a94b6b, also clean for this head) are clean too. #356, #382 and `fix/adopt-record-submodule-placement` conflict with origin/main and not with this head; #358, `fix/adopt-late-failure-notice` and `fix/adopt-plan-quote-yaml-keywords` conflict with both, with origin/main by themselves, so they are not this branch's. Neither are the five PRs below #200 that origin/main has not landed (#94, #96, #98, #100 and #161), which conflict with it too. The PR set is read from the refs under `refs/pull/` that origin/main's history has no `(#n)` subject for, not from GitHub, so a closed one may be in it. **One conflict, with `fix/link-source-values` (#338, 40762ca) in `scripts/repo_shape.py`, four hunks.** Not resolved in either branch; take both sides: (1) `resolve_link_source`'s signature, `default_source=None, env_shared: bool = False, *, tried: list | None = None`; (2) its docstring, both paragraphs; (3) `link_pins_from_trees`'s docstring, both paragraphs; (4) its call, `default_source, pin_product_key(name) in shared, tried=asked)` followed by #338's `if tried is not None: tried[name] = asked`. The auto-merged `resolve_link_source` body already composes: a shared variable is never an `env` candidate and so never appears in #338's `tried`. **Known limits after the re-verification (APPROVE WITH NITS, 0 blockers, 0 should-fixes, 3 nits, at 6494714).** For the four non-ASCII pairs this change now detects (`ß`/`ss`, `ſ`/`s`, `fi`/`fi`, `ς`/`σ`), the keyed `--pin-source <product>=<path>` form does NOT separate the two pins: #312's source lookup folds both keys into one, so one pin can get a false `unreadable` finding. A sibling checkout does separate them. Only a hand-written `project.yaml` can declare such a pair. The doctor's `pins` row still prints only "passes" for a validator that passes, so it does not carry the collision note; that needs `shape-doctor.py`, and is left for a follow-up. The wording for restoring the finding is under "Fix round 1" above: it is 34dbd86's `report.finding(...)` call, not a word swap. Reviewed adversarially at 34dbd86 (CHANGES NEEDED: 0 blockers, 2 should-fixes, 5 nits). The first should-fix was the finding itself, which locked a project that already declares the pair out of every `update-shape.py apply`; the second was a description that had gone stale. Fix round 1 (6494714) made the pair a NOTE, which is the coordinator's decision and one Brett may reverse (the way back is above), keyed collisions by the pin file's own fold, gave the guard a test that runs on Windows, and made a shared product's skip hint offer the keyed `--pin-source`. Its re-verification at 6494714 found every finding handled: APPROVE WITH NITS (0 blockers, 0 should-fixes, 3 nits: the keyed form and the non-ASCII pairs, the doctor's silence about the note, and the wording for restoring the finding, each stated above). CI is green on Linux, macOS and Windows, with SonarCloud passing, at b70a07d (main a73b4e9 merged in). Its merge with main 5a25387 passed the targeted set (the six pin and scaffold test files, `tests/test_validate_pins_unreadable_workflow.py`, `tests/test_ruleset_hint_tracking_branch.py` and `tests/test_shape_doctor.py`): 850 passed, 3 skipped. Closes #275 Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap
added a commit
that referenced
this pull request
Oct 13, 2026
… exists, a plan `follow_ups:` that is not a list of text and an `elected_on:` that `election_date` does not read, with or without a `reference:`, so `follow_ups: 5` is no longer a traceback and `elected_on: someday` is no longer refused under a flag neither command takes (#293, #294) (#356) `check` did not ask about a hand-edited plan's `follow_ups:` or `elected_on:`. `follow_ups: 5` ended `check` and `execute` in a `TypeError` traceback, and `follow_ups: abc` passed both, `execute` writing the follow-ups `a`, `b` and `c`. A date `election_date` refuses was refused only on its way to the manifest's reference: with no `reference:`, `check` stopped at exit 2 on `REFUSED election-date-malformed` naming `--elected-on`, a flag neither command takes, and said to re-run with it. Before #314 that refusal came after both legs were created, and a plan that kept its `reference:` adopted with the word in its manifest. `_plan_follow_ups` is now the one read of `follow_ups:` for `_plan_values`, `_print_what_will_happen` and `_commit_the_split`, where it is read inside #445's `committing the split` step. `_follow_ups_findings` reports `plan-bad-follow-ups` for a value that is not a list, saying what it holds, and once for each item that is not text. An empty list and a bare key stay valid. `_elected_on_findings` asks `election_date`, the rule `plan` asks of `--elected-on`, about `str` of the plan's value, with or without a `reference:`, so `plan`, `check` and `execute` cannot disagree about one date. A date it refuses is a FINDING `election-date-malformed` in the plan key's own words, and the remediation says to edit `elected_on:` in the plan file. Only an absent or null `elected_on:` is "no date": `""`, `false`, `0`, `[]` and `{}` are findings. Any other refusal of `election_date` is raised as it is. Both findings come last in `_entry_findings`, through `_plan_field_findings`, the date first as `plan` writes the keys, so `execute` refuses them through `_refuse_what_check_finds`, before #445's checks of what an earlier run left and before any leg exists. The refusal says no leg repository was created, and no notice of what a run left follows it. `_manifest_values` returns None for a plan whose `elected_on:` is such a finding, so #314's validator is not asked about a manifest whose reference that date chooses: the date is one finding, in this rule's words, and not `plan-manifest-invalid` beside it or a `REFUSED` raised from `default_reference` before the list is printed. It refuses the plan's `org:` and reads its `id:` before that guard, so a bad date beside a bad organisation or id does not make `check` and `execute` name different problems first. #314's rows for `"2026-W41-6"` and `"20261011"` in `tests/test_adopt_manifest_values.py` now expect `election-date-malformed`. `tests/test_adopt_execute_refusals.py` gains 54 tests: `check` and `execute` for each bad value, controls, a stricter `election_date` stood in by monkeypatch, both findings in one plan, a bad date beside a bad `org:` or `id:`, and an end-to-end run per issue that refuses with no leg and no notice, then adopts from the corrected plan in the same work directory. Closes #293 Closes #294 Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap
added a commit
that referenced
this pull request
Oct 13, 2026
… is `PRODUCT=PATH` only when its key names a chain link, and a link source that cannot be entered is a WARNING with the system's reason, not a traceback (#338) (#453) **What was wrong.** Two faults, one in the parser and one in the lookup. - **The parser.** `_link_sources_from_args` split every `--link-source` value at its first `=`. A link tree under a directory named `a=b`, passed bare, became a source keyed to a link `a` that no chain names. Nothing asked for it, so the link stayed `declared-unverified`, exit 0. The WARNING then offered `--link-source openXdox=<path>` to the person who had passed exactly that. A mistyped key failed the same silent way. - **The lookup.** `repo_shape.link_manifest_path` asked `Path.is_dir()` and then `is_file()`. Before Python 3.14 those swallow only ENOENT, ENOTDIR, EBADF and ELOOP. So each of these ended the naming gate in a traceback, exit 1: - a keyed source under a parent this user cannot enter; - `SHAPE_PIN_SOURCE_<LINK>` pointing there; - a checkout `../openXdox` at mode 000, which every `--project` run asks for; - a 300-character name. **What changed.** **How a value is read** (`LinkSources` in `scripts/validate-repository-naming.py`). This mirrors #312's `PinSources`, with a chain link where that file has a declared pin: - The text before the first `=` is a LINK only when it casefolds to a link of a chain this run reads. Those are the entries of `--referent-chain`, and of each leg's `naming.referent_chain:` under `--project`. Both are read before any leg is classified. - The keyed reading wins even over a directory spelled like the whole value. `./` reads that directory. - Any other value is a bare PATH, taken whole. **What a lookup says** (`scripts/repo_shape.py`): - `link_manifest_lookup` asks `stat()` inside a guard, as `_digest_from_source` does in #312. It returns the system's reason when a source cannot answer, or says which of four plain cases it is (`no such file or directory`, `it holds no project.yaml`, `its project.yaml is not a file`, `it is neither a directory nor a file`), and it never raises. It replaces `link_manifest_path`, which is gone: nothing calls it. `link_declarations` gives the reason a manifest could not be read, and `declared_neutral_pins`, which it left with no caller, is removed (fix round 1). - `resolve_link_source` and `link_pins_from_trees` take a keyword-only `tried`. It records each source asked, in order, with its reason. Nothing in `repo_shape` prints. - `link_pins_from_trees` also takes a keyword-only `with_reasons` (fix round 2). Asked, it returns a `LinkPins`, a dict equal to the plain mapping that also remembers why a tree it reached could not be read. Only the scaffold asks. **What the validator prints** (stderr, after each name's WARNINGs): - `WARNING link-source-unreadable: <path> (<how>) could not answer for <link>: <reason>`. This covers a `--link-source` asked for a link that holds a pin, a `SHAPE_PIN_SOURCE_<LINK>` likewise, and a checkout beside the project that exists and still cannot answer. A keyed value that also names a directory gets a second line saying how to ask for that directory with `./`. A bare value holding `=` says it was read as a PATH. - `NOTE --link-source <typed>: not used — <why>.` for a value no such link asked for. The why is one of five: a later value replaced it; no name reads that link's own declaration (a keyed value); no name records a chain at all; no name reads any chain link's own declaration (a bare value, where chains exist and none is read); or every link that is read had a more specific source. - Three kinds of source are never named: a variable or a checkout asked only for a chain's LAST link (no one ever reads that link's own declaration); a variable or a checkout for a chain no classification reads (fix round 1: a spec leg's chain, a name with no referent, a chain whose shape fails; a `--link-source` value for such a chain is the NOTE); and a checkout that is absent or holds no manifest. - `--quiet` drops both, with the WARNINGs. So `make bootstrap` and the doctor's `naming` row print nothing new. `--help`, the module docstring and `docs/cli.md` (regenerated) say all of this. **Fix round 1 (lane 5's review of 40762ca: 0 blockers, 2 should, 4 nits).** Second commit, 4b21263. Each review finding, in order: - **A link manifest whose `neutral_product_pins:` is not a list of names is a tree that cannot answer** (SHOULD-1). `link_declarations` iterated `data.get(key) or []` as it stood, against the `link_pins_from_trees` docstring this change wrote ("nothing a source does raises out of here"). `5` ended the naming gate, the scaffold and the project's `validate-manifest.py` in `TypeError: 'int' object is not iterable`. A scalar `openDox` was read as the characters `D`, `e`, `n`, `o`, `p`, `x`, so the chain broke, and under `--project` that is a FINDING. A mapping was read as its keys, so `openDox: yes` DECLARED `openDox`. Each is now `(None, why)`, as `not a mapping` is: declared-unverified, with the reason named ("`neutral_product_pins` is not a list", "holds an entry that is not text"). An absent or null value, an empty list, and a blank entry (`null`, `''`) in a list, declare nothing, as ever. The fix is in the shared function, so the scaffold and `validate-manifest.py` are reached too. **Three behaviours change on purpose here, and the tests of fix round 2 pin each:** - **A falsy value is no longer an empty list.** `neutral_product_pins: 0`, `false` or `''` was read as no pin at all (`or []`), so a chain through the tree was BROKEN, and under `--project` that is a FINDING. It is now a tree that cannot answer: the link is declared-unverified, a WARNING, exit 0. - **An entry that is not text is no longer read as text.** `[openInk, 0]` and `[openInk, false]` dropped the entry as a blank, and `[openInk, true]` read it as the text `True`, so a chain through the list VERIFIED. Each is now declared-unverified, with a WARNING saying the list "holds an entry that is not text". That is stricter than the link tree's own `validate-manifest.py`, which takes `str(pin)` of an entry (`_declared_pins`, `templates/assembly-root/scripts/validate-manifest.py:85`): a chain is better left unverified than verified through a name nobody wrote. - **The scaffold no longer refuses `chain-not-declared` over a pins value that is a scalar.** With `--pin-source` a tree whose `neutral_product_pins:` is the scalar `openInk`, the scaffold used to refuse (`REFUSED chain-not-declared: ... openXdox declares I, e, k, n, o, p, not openInk`, exit 2, from the characters). It is now the declared-unverified WARNING and the scaffold goes on, exit 0, which is its documented rule for a tree that cannot answer: "a warning, never a refusal, because the check is offline". The warning says the tree was reached and could not be read (fix round 2, below). - **The reach wording is corrected** (SHOULD-2): "Who else this reaches" below, and the docstring of the test that made the claim. What stops crashing is the naming and manifest steps. `make validate`, `validate.yml` and `make bootstrap` still end in a traceback on the sibling row before Python 3.14, at a file this change does not touch (below). On 3.14 the sibling is passed over without a word there instead, which is #430. - **The mutation gaps** (nit). Six of the review's ten mutations survived. Five now die, each by a new test: a `project.yaml` that is a directory; a device or a FIFO as the source; the order A, B, A; a value holding `=` with no chain at all; and every failure of `SHAPE_PIN_SOURCE_<LINK>` for a link that holds a pin (a path that is not there, a directory with no manifest, a file that is no manifest), not only a permission. The sixth, the `typed in self.used` test in `unread_lines`, could not be killed because it was dead: a value is read as one link's keyed source or is the one bare PATH, so every lookup asks it of the same path and it is used or unread, never both. That was asserted over 293 tests and the test is removed. - **A chain no classification reads** (nit). `classify` consults a link's own declaration only for a name that claims a referent, with a recorded chain whose shape holds. For any other chain (one recorded on a spec leg; one on a name with no `open<Product>` to reach; one that does not begin at a held pin or does not end at a referent of the name) a source was still asked, so a path that was not there was reported as `link-source-unreadable` and a good tree was counted used, each for a question nobody put. `NamingPolicy.reads_link_declarations` now says when `resolve_referent` reads a chain, and `LinkSources.link_pins` asks it first: such a chain asks nothing, and a value for it is the NOTE that no name reads that link's declaration. A test runs `resolve_referent` with a mapping that records whether it was read, over every way a chain is and is not, and holds the new method to it. - **Dead code and stale sentences** (nit). `declared_neutral_pins` is removed: `git grep` over the tree, every pushed branch and every sibling worktree finds no caller. `link_manifest_lookup`'s docstring no longer says "replaces" over a name that is gone. This description no longer says `tests/test_naming_policy.py` takes `SEARCH_DENIED` from #312: it defines a mark of its own, with the same condition and its own reason text. **Fix round 2 (lane 5's re-review of 4b21263: 0 blockers, 0 should, 3 nits).** Third commit, 2ac911c. Each finding, in order: - **The three behaviours the body did not state are stated and tested** (nit 1). Round 1's not-a-list fix changed three behaviours on purpose, each listed under SHOULD-1 above, and the re-review found two of them had no test. R1 (`if declared is None` read as `if not declared`, which makes `0`, `false` and `''` declare nothing again) and R2 (`if pin is None or pin == ""` read as `if not pin`, which drops a `0` or `false` entry as a blank) both passed all 210 cases of `tests/test_naming_policy.py` at 4b21263. Six rows join `NOT_A_LIST_OF_NAMES`, which runs at the CLI and in process: `0`, `false` and `''` as the value, and `0`, `false` and `true` among the names. R1 now fails 9 cases and R2 fails 6. - **The scaffold's warning is true for a tree it reached** (nit 1, the third behaviour). Over a `--pin-source` whose `neutral_product_pins:` is not a list, round 1 turned the `chain-not-declared` refusal into a declared-unverified WARNING that says the tree "is not reachable here (a sibling checkout, SHAPE_PIN_SOURCE_<LINK>, or --link-source <LINK>=<path> would let it be read)". `--pin-source` did reach the tree, and only its declaration could not be read, so that blamed a path that was right. Now `link_pins_from_trees` takes a keyword-only `with_reasons`; asked, it returns a `LinkPins`, a dict equal to the plain mapping that also keeps `unreadable`, `{link: why}` for each link whose source answered and whose manifest could not be read. `_chain_link_findings` words such a link "which was reached but could not be read as a declaration (<why>)", in a branch of its own, and only the scaffold asks. The new text is asserted for each of the eleven not-a-list rows, with `scaffold-project.py --dry-run` run as a subprocess. - **Every other path's text is the bytes it was.** The sentence for a link no source reached is the same code in the same lines, and four tests hold it: the generic warning for a plain mapping, for an empty `LinkPins` and for an absent tree; the scaffold's own old warning for a clone that holds no `project.yaml`; the naming validator's two lines (its warning, and the `link-source-unreadable` line that names the reason); and `validate-manifest.py`'s. The naming validator and `validate-manifest.py` do not ask for reasons, so they print what they printed. - **One more scaffold path changes with it, on purpose.** A tree the scaffold reached and could not read as a declaration for any other reason (a manifest that does not parse, or is not a mapping) used to say "not reachable here" too, and was no truer. It now says what it is, with the system's reason, and a test for each of the two (a manifest that is not a mapping, one that does not parse) holds it, at the scaffold's command line and in process: narrowing the wording to the not-a-list reasons fails both. - `_declaration` still returns its four values, so nothing else in `scaffold-project.py` moved. - **A qualified chain link in the spy test** (nit 2). `_chain_read_cases` had no case whose chain link is qualified, so `reads_link_declarations` could take the chain as typed and not through `_chain_links` (R3) and pass every test. With `--referent-chain testorg/openXdox,openDox` that turns a verified chain into a declared-unverified one. `("codexDox", {"openXdox"}, ("testorg/openXdox", "openDox"), True)` is a case now, and R3 fails it. - **The description** (nit 3): the ten corrections of the re-review are applied (the reason a lookup returns, the fifth NOTE reason, the third kind of silent source, the SHOULD-1 bullet, "before Python 3.14", the `validate-pins.py` bullet, the 3.14.7 rows, the merge-tree section, the filed numbers, and the project's own `validate-manifest.py`). The mutation proofs: all three survivors were reproduced first at 4b21263 (210 passed each) and each dies at 2ac911c. Six mutations of the new code each die too: the warning ignoring the reasons (12 cases fail), the lookup recording none (12), the scaffold not asking (11), `with_reasons` on by default (3), an absent tree called reached (3), and the reached sentence used for a tree nothing reached (17). **Where it departs from #312, and why.** - **An explicit source that cannot answer stays a WARNING, exit 0.** #312 made an unreadable `--pin-source` a finding. Three reasons for the difference: - Exit 1 here means "a name is wrong", and this module has always said an unread link tree is a warning. - `SHAPE_PIN_SOURCE_<PRODUCT>` also answers `validate-pins.py`'s digest question, from a checkout that need not hold a manifest. - The lookup falls through to the next source, which may still verify the link. - **The fall-through is kept**, as it always was. With `--pin-source`, a keyed value is final. - **The lines go to stderr, spelled `WARNING`/`NOTE` as this file spells them.** #312's notes are ` note` lines on stdout, where that file's report lives. **Who else this reaches.** The guard is in `repo_shape.py`, so the same `link_pins_from_trees` lookup stops crashing for these callers too: - **The project's own `validate-manifest.py`.** It reaches the lookup through `SHAPE_PIN_SOURCE_<LINK>` or a sibling checkout, and now gives its usual declared-unverified WARNING instead of the traceback. - **`scaffold-project.py --referent-chain`.** A `SHAPE_PIN_SOURCE_<LINK>` under a locked parent no longer kills the plan, and `--pin-source` verifies the link. Neither of these prints the reason a source could not be entered; the scaffold says it only for a tree it reached and could not read (fix round 2, above). A `--pin-source` the scaffold cannot enter is still a traceback: that is #339 (`pin_digest_from_source`), and this PR does not touch it. **What this does not reach, and what the doctor says.** - **The project's gates still end in a traceback on the sibling row.** The NAMING and MANIFEST steps of `make validate`, `validate.yml` and `make bootstrap` stop crashing for a checkout `../openXdox` at mode 000. Those targets also run `validate-pins.py`, whose `resolve_neutral_pin_source` asks `(sibling / ".git").exists()` with no guard (`templates/assembly-root/scripts/validate-pins.py:463` on main), so before 3.14 a sibling at mode 000 still ends them in a `PermissionError`. That file is #312's, and `fix/pin-source-env-collisions` (PR #409, since landed), `fix/adopt-pin-writes-pin-file` and open PRs #343 and #419 touched it when this was written, so this PR does not edit it. The same line is listed for Python 3.14 by #430 (lane 2), where the helper that would cure both halves is asked for; the pre-3.14 traceback reproduction is added there as a comment, not filed apart. A `SHAPE_PIN_SOURCE_<LINK>` under a locked parent is different: `validate-pins.py` makes that a `neutral-pin-source-unreadable` FINDING (#312), and that stays. - **The doctor's `naming` row** runs the STANDARD's copy of the validator, so it stops crashing at once. **Its `manifest` row** runs the project's OWN `validate-manifest.py` on the project's own `repo_shape.py`, so it changes only once the project has applied this change. Over an existing project the doctor says `INVALID (manifest, pins)` until `update-shape.py apply` lands the two `upstream-changed` rows below. - **The verdict.** For `SHAPE_PIN_SOURCE_<LINK>` under a locked parent, on a project that carries these copies, `INVALID (naming, manifest, pins)` becomes `INVALID (pins)`, and `pins` is #312's finding for that locked variable, which is right. For a sibling at mode 000 the `pins` that remains is the traceback above, not a finding. The two copied scripts this change edits (`scripts/repo_shape.py` and `scripts/validate-repository-naming.py`) are copied into every project and pinned there. A project takes the change through `update-shape.py` (two `upstream-changed` rows), and a family holder takes the `repo_shape.py` row. Neither file has a second copy in this repository. **Evidence.** - **Issue rows, main ab623dd** (`repro338.sh`, a `gh` stub first on PATH). Under 3.10.21 and 3.12.3, rows 4-7 (keyed under a mode-000 parent, `SHAPE_PIN_SOURCE_OPENXDOX` there, a sibling at mode 000, a 300-character name) are a traceback, exit 1. Rows 2-3 (`a=b`, the typo) are a silent declared-unverified, exit 0. Under 3.14.7 row 1 verifies, and rows 2-7 are the silent declared-unverified WARNING, exit 0. - **Issue rows, this PR.** Under 3.10.21, 3.11.16, 3.12.3, 3.12.14, 3.13.15, 3.14.7 and 3.15.0rc2 (first commit), all 7 rows exit 0 with no traceback, and the output is byte-identical across versions. `a=b` verifies. The other five rows each name their source with its reason. Re-run on the fix-round head, 4b21263, under 3.10.21, 3.12.3 and 3.14.7: byte-identical to the first commit's run. Re-run on the fix-round-2 head, 2ac911c, under the same three: all 7 rows exit 0 with no traceback, the output is byte-identical across the three versions, and it is identical to the re-review's runs of the previous head over main. - **Reach on main.** Main gives a `PermissionError` traceback, exit 1, for `scaffold --dry-run`, the project's `validate-manifest.py`, `validate-repository-naming.py --project` and its `--quiet` form, each with `SHAPE_PIN_SOURCE_OPENXDOX` under a locked parent. This PR gives exit 0 for all four. `validate-pins.py` on the sibling row is the exception said above. - **New tests.** 17 cases (11 functions) in the first commit; 16 fail on ab623dd. The fix round adds 31 cases in `tests/test_naming_policy.py`, and 17 of them fail on 40762ca: the five shapes of pins that are not a list (each at the CLI, in-process and, for the scalar, under `--project`), and the chains nobody reads. The other 14 guard what a mutation could break (below) or the side the fix must not move (an absent, null or blank pin list still answers). - **Targeted run on 4b21263**, files `test_naming_policy`, `test_manifest_validator`, `test_scaffold_referent_chain_links`, `test_cli_reference`, `test_repo_hygiene`, `test_neutral_pin_validator`, `test_scaffold_e2e`, `test_shape_doctor`, `test_git_output_decoding` and `test_pin_validator`: **765 passed, 3 skipped**. The 3 skips are the PowerShell-only rows. (40762ca: 734 passed, 3 skipped.) `render-cli-reference.py --check` is ok. - **Mutation probes, first commit.** Reverting the stat guard, the parse, keyed-over-directory, the NOTE lines, the chain's-end restriction, the sibling-absence filter, the manifest reason, the `--quiet` drop, the "read as a PATH" sentence, the `./` line, the project-chain read, the WARNING lines or the env reporting each fails at least one new test. - **Targeted runs on 2ac911c (fix round 2).** Under 3.12.3, `test_naming_policy` (238 cases: 210 and 28 new), `test_cli_reference`, `test_manifest_validator`, `test_scaffold_referent_chain_links`, `test_repo_hygiene` and `test_setup_project_py`: **468 passed**; and `test_scaffold_e2e`, `test_scaffold_pin_and_reuse`, `test_scaffold_leg_path_collisions`, `test_scaffold_pin_collisions`, `test_neutral_pin_validator`, `test_git_output_decoding` and `test_pin_validator`: **312 passed**. Under 3.10.21 and under 3.14.7, `test_naming_policy`, `test_manifest_validator` and `test_scaffold_referent_chain_links`: **321 passed** each. `render-cli-reference.py --check` is ok, and `ast.parse(feature_version=(3, 9))` passes on the three changed files. - **Mutation probes, fix round.** The review's ten, re-run on 4b21263: MW1, MW2, MW3, MW5 and MW11 now die (they survived), and MW4, MW7, MW12 and MW13 still die; MW6 is the dead test, removed. (Fix round 2 kills the three the re-review found, R1 to R3, and six more: see above.) Nine more for the new code (the list check, the entry check, the null and blank tolerances, `reads_link_declarations` always true, without the shape check, without the held pins, and the validator ignoring it): eight die. The ninth, `reads_link_declarations` ignoring whether the name has a referent, survives as an equivalent mutant: with no referent `_chain_shape_problem`'s last question always reports a problem, so the explicit early return is redundant, and is kept because `resolve_referent` has the same first line. The spy test holds the two together. - **Windows.** The mode-000 rows skip on Windows and as root, using a `SEARCH_DENIED` mark this file defines, with the same condition as #312's. The new FIFO and `/dev/null` rows skip where those do not exist. `a=b` and the 300-character name run everywhere. On Windows the long-name row asserts the line, not POSIX's reason. None of it had been run on Windows or macOS when this was written; the last paragraph gives the CI result. **Merge-tree.** Recomputed at 19:04Z with `git merge-tree --write-tree` of 2ac911c against `origin/main` as it was then, 53bee5b, and against every one of the other 37 pushed branches that touches a file this PR changes (a scan of each head against its merge-base with main): - **Clean:** origin/main 53bee5b (it carries #314 and #390); #394 `fix/adopt-record-submodule-placement` 563ea16 (`scripts/repo_shape.py` and `validate-manifest.py`) and #406 `fix/validate-family-unnamed-mounts` 369af2e (`scripts/repo_shape.py`); #397 abb27e6, #400 7062a47, #401 409f10b, #416 7cca975, #452 e2df9a8 and #395 1ef3310 (`docs/cli.md`); #355 6fdbe3d, #373 741cbf2 and #395 (`scaffold-project.py`). - **Conflict, four hunks each, all in `scripts/repo_shape.py`:** `fix/pin-source-env-collisions` (PR #409) 6494714, and `fix/adopt-pin-writes-pin-file` 421e4e5 (no PR). Both also touch `scaffold-project.py`, and that merges clean with this PR's one edit there. Fix round 2's new hunks merge with neither: #409's change to the "not reachable here" sentence (the `offered` text, when a variable is shared) and the reached branch sit in different lines of `_chain_link_findings`. The four hunks, each taking both sides: 1. the `resolve_link_source` signature: `default_source=None, env_shared: bool = False, *, tried: list | None = None`; 2. its docstring: both paragraphs; 3. the `link_pins_from_trees` docstring and the lines after it: both paragraphs, then `found`, this PR's `unreadable` and #409's `links` and `shared`; 4. the call: `resolve_link_source(name, root, keyed_sources, default_source, pin_product_key(name) in shared, tried=asked)` (the same spelling on both branches), followed by `if tried is not None: tried[name] = asked`. Resolved that way in a throwaway and committed there, the merge with 6494714 passed **496 tests** (`test_naming_policy`, `test_pin_source_env_collisions`, `test_neutral_pin_validator`, `test_scaffold_referent_chain_links`, `test_manifest_validator`, `test_cli_reference`), and the merge with 421e4e5 passed **559 tests** (the same files and `test_pin_source_unreadable`). `render-cli-reference.py --check` is ok on both, pyflakes is clean, and the grammar parses as 3.9's. Neither branch carried the resolution when this was measured. #409 has since landed, and the merge of main into this branch (the last paragraph) takes those four hunks both ways. The earlier counts (468 and 434 from the first fix round, 430 from the re-review) are those runs' own, at their heads. **Left out.** Each of these was found by lane 5's reviews of this change, and each is filed or folded: - **A bare `--link-source` answers for every link with no more specific source** (#441). In a chain of three or more links, one bare tree is read as two links' declarations, and a chain that holds is reported broken, a FINDING for a name with no direct pin on its referent. Pre-existing, the same on main; `validate-pins.py`'s `--pin-source` has the same documented rule and was not examined. - **An empty path after a key (`--link-source openXdox=`) is `Path("")`, the current directory** (#442). Run from a project root, the project's own `project.yaml` is read as the link's declaration. Pre-existing; `validate-pins.py`'s `--pin-source openXdox=` has the same shape (there a finding naming `.`). - **`validate-pins.py`'s `(sibling / ".git").exists()` is unguarded** (the sibling row above), folded into lane 2's #430, which already covers the Python 3.14 half: the pre-3.14 traceback and its reproduction are added there as a comment. #312's file. - **The project's OWN manifest** with `neutral_product_pins: 5` ends in a `TypeError` in two gates before either can say anything about it (#443): `_pins_from_project` in the naming gate (`scripts/validate-repository-naming.py:302` on main), and `_declared_pins` in the project's own `validate-manifest.py` (`templates/assembly-root/scripts/validate-manifest.py:85`), which ends it before its own `manifest-neutral-product-pins` finding (`_neutral_product_pins_type_finding`, `:524` on main) can print. Reproduced on 3ef059c. That is the project's own declaration, read by the project's own gates, and not a link source, so it is outside this change; "read by the project's own gates" covers two crashing gates. Also not done here: - **Python 3.9 (the floor) was not run** because it is not installed here. The grammar of the three changed files was checked with `ast.parse(feature_version=(3, 9))`. Windows and macOS were not run here; CI ran them (the last paragraph). Reviewed adversarially at 40762ca, the branch's first commit (CHANGES NEEDED: 0 blockers, 2 should-fixes, 4 nits). The head stopped every crash in the issue's table, and the should-fixes were a docstring claim the code did not keep for a `neutral_product_pins:` that is not a list, and reach claims that overstated what the project's gate does on the sibling row. Fix round 1 (4b21263) resolved every finding, and the re-review of 4b21263 read APPROVE WITH NITS (0 blockers, 0 should-fixes, 3 nits: three behaviour changes this description had not stated, two of them untested; a spy-test case for a qualified chain link; and description corrections). Fix round 2 (2ac911c) resolved those three, and its re-verification of 2ac911c read APPROVE WITH NITS (0 blockers, 0 should-fixes, 2 nits: no test of the scaffold's wider wording for a manifest that is not a mapping or does not parse, now pinned by two tests that each fail when the wording is narrowed, and a cross-reference in this description that pointed the wrong way, now corrected). Main (9736424, #409's squash) was merged into the branch as ced8fac. `scripts/repo_shape.py` conflicted in four hunks; each takes both sides, as the review of 2ac911c resolved it: `resolve_link_source(..., default_source=None, env_shared: bool = False, *, tried=None)`, both docstrings kept, `link_pins_from_trees` opening with `found`, `unreadable`, `links` and `shared`, and the call passing `pin_product_key(name) in shared` with `tried=asked`. Nothing else conflicted. After the merge, `tests/test_pin_source_env_collisions.py` and the targeted set passed (667 and 312), and `render-cli-reference.py --check` printed `[ok]`. CI is green on Linux, macOS and Windows, with SonarCloud passing, at 938a131 (main 48585e6 merged in). Its merge with main 994be39 passed the targeted set (the naming, manifest, scaffold, pin and setup test files): 983 passed. Closes #338 Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap
added a commit
that referenced
this pull request
Oct 13, 2026
… writes its pin file as the scaffold does, so an adopted root passes `validate-pins.py` (#273); a `--pin-source` that cannot answer is `pin-source-unreadable` (#339) (#508) **Brought up to date after #409 squash-landed** by merging origin/main (9736424, #409's squash) as f9ed46f. Its `scaffold-project.py` conflict was resolved to this branch's side once `git diff b70a07d 9736424 -- scaffold-project.py` (#409's final head against its squash) came out empty. So the file is the one tested on this branch plus main's own later changes to it, and holds nothing of #409's; details below. This branch carries #275's adopt half. It was built on `fix/pin-source-env-collisions`, PR #409 (#275's scaffold half and its fix round), merged at 6494714 as 583f647, over the earlier 5690235 (34dbd86). After 6494714, #409 only merged main, five times, up to its final head b70a07d, and none of those merges has a hand edit (`git show --remerge-diff` is empty for each). Adopt's refusal of two products that share a source variable goes through #409's function, so this branch lands after #409. It was never rebased. `git diff b70a07d 9736424` is empty for each file #409 touched: `scaffold-project.py`, `scripts/repo_shape.py`, `templates/assembly-root/scripts/validate-pins.py` and `tests/test_pin_source_env_collisions.py`. What f9ed46f resolves, four files with one hunk each: - **`scaffold-project.py`.** - The conflict hunk is main's block from `PIN_COLLISION_REMEDIATION` through `_pin_collision_remediation`, which #409's squash edits: it adds `PIN_VARIABLE_REMEDIATION`, `_colliding_pin_variables` and `_pin_collision_remediation` and changes `_refuse_colliding_pins`. This branch moved all of that to `scripts/shape_materialize.py`. The hunk takes this branch's side. (The merge commit's message says "re-adds"; this is the exact account.) - #409's auto-merged import of `PIN_SOURCE_ENV_PREFIX, pin_source_env_collisions` is dropped, because nothing in the file uses either name any more. - It is not `git checkout --ours` of the whole file. Main changed the file beyond #409 since this branch last merged it: #355's `--tracking-branch` check (6c2a371) and a73b4e9's `RULESET_HINT`. Those lines stay. - The result is the file `git merge-tree` gives for this branch with main before the squash (518010b). Its diff from the file tested here has the same patch-id as main's own 3ef059c..518010b change to it. - **`templates/assembly-root/scripts/validate-pins.py`.** Main's side, which is #409's final blob, with main's #359 workflow reader in it. This branch had no change of its own to the file beyond #409. - **`adopt-project.py`, `_commit_the_split`.** - Main: #445 wraps the function in its late-failure `try`, and #394 builds an `append` with the plan's submodule record. - This branch: its only edits there were a comment and `neutral_pins=_recorded_pins(plan)` on the materializer call. - Both now sit on main's call inside the `try`: `append=append, neutral_pins=_recorded_pins(plan)`. - **`docs/cli.md`.** It is regenerated with `scripts/render-cli-reference.py`, and `--check` is ok. It differs from main's only in the `--pin`, `--pin-owner` and `--pin-source` help of `adopt-project.py plan`. `scripts/repo_shape.py`, `templates/assembly-root/scripts/validate-pins.py` and `tests/test_pin_source_env_collisions.py` are main's blobs, which are #409's: this branch adds nothing to them. Nothing of #409 is lost. Its scaffold behaviour is `shape_materialize.refuse_colliding_pins` and `pin_variable_collisions`, keyed by #409's `pin_product_key`. **73d65c6, after the merge.** - **The change.** `_read_pins` reads `plan.get("pins")` where it read `plan.data.get("pins")`. - **Why.** #445's `tests/test_adopt_late_failure.py` hands `_commit_the_split` a `SplitPlan` as its plan, a `dict` that adds `entries`. On f9ed46f, three of its tests stopped at "committing the split" with `AttributeError: 'SplitPlan' object has no attribute 'data'` (3 failed, 83 passed in that file). - **Nothing else changes.** `Plan.get` with no default returns `plan.data.get(key)` exactly, so a loaded plan reads as before. `adopt-project.py plan --pin` took a NAME only (`--pin openGlass`) and refused a trailing `@<commit>`. `plan`, `check` and `execute` each exited 0, and `execute` printed `adoption verified`. The manifest then listed the name in `neutral_product_pins:` with no `contracts/<product>-pin.yaml` beside it, and on its first run the adopted root's own `validate-pins.py` refused it as `neutral-pin-missing`. #273's table, reproduced on c7c02fa with the issue's own commands (local bare remotes, and a stub `gh` that exits 97): | `--pin` | c7c02fa | this branch | |---|---|---| | `openGlass` | `execute` exit 0, `- openGlass`; clone's `validate-pins.py` exit 2 `neutral-pin-missing` | `plan` exit 2 `pin-malformed`, with the scaffold's remediation (`--pin openGlass@<40 hex>`); no plan is written | | `opensoft/openGlass` | exit 0, `- opensoft/openGlass`; exit 2 `neutral-pin-missing` for `contracts/opensoft/openglass-pin.yaml` | `plan` exit 2 `pin-malformed` | | `spec` | exit 0; `validate-pins.py` exit 0, reading the leg's `contracts/spec-pin.yaml` as the neutral pin | `spec@<sha>` is `adopt-pin-collides` at `plan` | | `shape` | exit 0; `contracts/shape-pin.yaml` read as the neutral pin | `shape@<sha>` is `adopt-pin-collides` at `plan` | | `openGlass@<sha>`, `opensoft/openGlass@<sha>` | `plan` exit 2 `adopt-pin-malformed` | `plan`, `check` and `execute` exit 0, and `gh` is called 0 times. `contracts/openglass-pin.yaml` is written and the manifest lists `- openGlass`. In the clone, `validate-manifest.py` exits 0, and `validate-pins.py` exits 0 with `openGlass: pin commit … digest recomputes` | Brett ruled for decision 1's first option ("go with your recommendation on #273"): adopt takes the scaffold's pin and writes the scaffold's pin file. That reverses the NAME-only `--pin` from #6, and the comment that described it is replaced with the new contract. - **One implementation.** The scaffold's pin code moved into `scripts/shape_materialize.py`, next to `neutral_pin_path` and the materializer that writes the file. What moved: - `PIN_ARG_RE`, `parse_pin`, `pin_digest_from_source` and `pin_digest_from_gh`; - `declared_pin_values`, `declared_pin_products` and `PIN_COLLISION_REMEDIATION` (in #309's wording); - from #275's branch, `PIN_VARIABLE_REMEDIATION` and the variable check; - `_refuse_colliding_pins`, split into the pure `pin_collisions` and `pin_variable_collisions`, and `refuse_colliding_pins(refusal=...)`, which raises one refusal for both. The code did not move verbatim. `_refuse_colliding_pins` became the three functions above; `pin_digest_from_source` was rewritten for #339 (below); and fix round 1 changed three of them (below): `pin_variable_collisions` keys each pin by #275's `pin_product_key`, `pin_digest_from_source` spells a path that is not printable, and `pin_digest_from_gh` refuses a missing `gh`. The scaffold's behaviour is otherwise unchanged. `scaffold-project.py` imports what it calls under its old names. Lane 5's review ran 34 scaffold cases (every pin refusal, the `gh` stub's four modes, dry and real successes) on 34dbd86 and on this branch at 7a94b6b. rc, stdout, stderr, the `gh` log and the files made were identical, apart from the shape's own commit and digest and #339's row. Its pin tests and #275's pass as they are. `tests/test_git_output_decoding.py`'s exemption for `pin_digest_from_gh` follows the function to its new file. - **`plan`.** `--pin`, `--pin-owner` and `--pin-source` mean what they mean for the scaffold. `plan` reads the digest from `--pin-source` or `gh api`, and records product, repository, commit, digest and `digest_source` in the plan, so `check` and `execute` read no network for a pin. `plan` refuses: - a name with no `@<commit>`, as `pin-malformed`. That includes an empty `--pin ''`, which adopt used to drop and now refuses, as the scaffold always has; - `spec`, `code` and `shape`, in any case, as `adopt-pin-collides` (#273 decision 4), in the scaffold's sentence and remediation (#241, #309); - one product given twice, the same way, whether it is a repeat or two names that share one file (decision 5); - two products that one `SHAPE_PIN_SOURCE_<PRODUCT>` names, such as `open-glass` and `open_glass`, the same way, through #275's check; - a leg path over a pin's file, as `adopt-leg-path-collides`; - a `--pin-source` path that `check` would refuse in `digest_source:` (fix round 1), as `adopt-unwritable-value`. It is asked only when a `--pin` records the path; - with no `gh` on PATH and no `--pin-source`, `gh-not-found` (fix round 1). - **`check` and `execute`.** These are new `check` findings, and `execute` refuses each one before any leg exists. They come from `_leg_findings`, the list `check` and `_refuse_what_check_finds` share: - `plan-pin-unrecorded`: a name alone (an old plan's `pins: [openGlass]`, or a hand edit), or an entry missing its commit or digest; - `plan-pin-malformed`; - `plan-pin-collides`, which covers two pins at one file, a pin at a file the shape writes, and two products that share one source variable; - `plan-root-holds-pin`: a source path kept in the root at the pin's file. The materializer would write the shape's pin beside it under `shape/`, and `validate-pins.py` would read the source's file instead. How each finding ends: - `plan-pin-unrecorded` and `plan-pin-malformed` end with the `plan --pin <product>@<40 hex commit>` that records the pin whole. - `plan-pin-collides` ends with re-running `plan` with each product once, or, for two products that share a source variable, with one product of each such group. - `plan-root-holds-pin` ends with the kept path's own remedy: send that entry to a leg, `drop` it, or rename it in the source and re-run `plan`. A leg moved over a pin's file in the plan is `plan-leg-path-collides`. - **Also part of #386: its `pins:` row.** A hand-edited `pins:` that is not a list (a name such as `pins: openGlass`, a number, a mapping, a quoted string) is now `plan-pin-malformed` in `check` (`` `pins:` is 'openGlass', not a list of pins ``), and `execute` refuses it before any leg exists. - Where: `_read_pins` in `adopt-project.py`; the test row is `plan-pin-malformed-not-a-list` in `tests/test_adopt_pins.py`. - Before: on main, `check` said `plan ok` for the name, the mapping and the string, and gave a traceback for the number (reproduced by lane 5's review on cb7063f). #386 records that `execute` then wrote the name as eight one-letter pins. - Not fixed here: #386's other keys, `allow_empty_legs:`, `paths:`, `legs:` and `source:`, so #386 stays open. Lane 2's #435 takes all of #386, `pins:` included, and its `pins:` rule keeps only text items where this branch writes mappings; whichever lands second reconciles them (#435 (comment), and "Where the meaning meets" below). - **`execute`.** It writes each pin through `materialize_assembly_root(neutral_pins=...)`. The manifest lists the product with the owner stripped (decision 3), so the entry and the file agree. - **Docs.** `--help` is updated and `docs/cli.md` regenerated (`render-cli-reference.py --check` ok). `pin_digest_from_source` caught only `Refusal`. A directory that is no clone is git's own failure, so it was always `pin-source-unreadable`, exit 2. A path `subprocess.run` cannot enter never reaches git: it is a `FileNotFoundError`, `NotADirectoryError` or `PermissionError` out of `cwd=`, and nothing caught it. `scaffold-project.py --pin ... --pin-source <that>` ended `--dry-run` and the real run alike in a traceback, exit 1. Since the move above, `adopt-project.py plan --pin` reads its digest through the same function and ended the same way. Both reach it through `declared_pin_values`, and `check` and `execute` read no source, so that call is the only reader. The issue's table, run on ab623dd for the scaffold and on d806e8f for `adopt-project.py plan` (adopt has no `--pin-source` on ab623dd), then on this head. `gh` is a stub that logs and exits 97, with `GH_TOKEN` bogus and an empty `GH_CONFIG_DIR`; it was called 0 times in every run: | `--pin-source` | before | this branch | |---|---|---| | a directory that is not a clone | `pin-source-unreadable`, exit 2 | unchanged | | a path that does not exist | `FileNotFoundError` traceback, exit 1 | `pin-source-unreadable`, exit 2: `<path> could not answer for opensoft/openGlass @ <40 hex>: no such directory` | | a file | `NotADirectoryError`, exit 1 | exit 2, `... it is a file, not a directory` | | under a parent this user cannot enter | `PermissionError`, exit 1 | exit 2, `... Permission denied` | | a symlink loop (not in the issue) | `RuntimeError` out of `Path.resolve()` on 3.12, exit 1 | exit 2, `... Too many levels of symbolic links` | The real run and `--dry-run` give the same stdout and stderr, and nothing exists afterwards (no remote directory, no work directory; for adopt, no plan file). What changed, in `scripts/shape_materialize.py` only: - `pin_digest_from_source` catches `OSError` as well as `Refusal` and raises the existing `pin-source-unreadable` with its remediation unchanged. The message names the path and the reason. - `stat()` judges the source inside the guard, as #312 does in `validate-pins.py`, and the sentences are that file's: a missing path, or one beneath a file, is "no such directory"; a file is "it is a file, not a directory"; any other `OSError`, from `stat()` or from git's `cwd=`, is the system's own reason (`strerror`). It is not `exists()`/`is_dir()`, which before 3.14 swallow only a missing path. A source `stat()` can read but `cwd=` cannot enter, a directory with no search bit, fails in `tree_digest` and the same guard answers it. - `declared_pin_values` resolves the path through a guarded `_resolved_pin_source`. `Path.resolve()` raises a `RuntimeError` on a symlink loop before 3.13 (an `OSError` since), before the digest was asked for. As typed, `stat()` meets the loop and says so. - A source that answers is read exactly as before: a relative path resolves against the working directory, a symlink to a clone is followed, `digest_source` reads `local-clone (<path as typed>)`, and the digest is `tree_digest`'s. `tests/test_pin_source_unreadable.py` (46 tests in the #339 commit; fix round 1 adds 17 for S1 and moves one, below) runs each case through `scaffold-project.py --dry-run`, its real run and `adopt-project.py plan`: missing, a file, beneath a file, relative and missing, a parent and a source with no search bit (skipped on Windows and as root, by a mark, so before any fixture builds a directory), a 300-character name, a symlink loop (skipped on Windows) and a directory that is no clone. Each asserts exit 2, the refusal naming the path and the reason, its remediation, no traceback and nothing created. It also holds `--dry-run` and the real run to the same stdout and stderr, checks the path is printed escaped under an ASCII stderr and as a surrogate when it is not UTF-8, and checks that a relative source and a symlink to a clone read as before. The session `gh` guard fails any test whose run called `gh`. origin/main ab623dd reached this branch through implQ's branch: 34dbd86 is a30091f merged with ab623dd. One file conflicted there, `templates/assembly-root/scripts/validate-pins.py`, in seven hunks, because #312 (8cdca5e) replaced `keyed_sources` and `default_source` with `PinSources` and #275 added `env_shared` and `shared`. #312's blob is the same at its PR head, at the squash and at ab623dd, so it landed the file implQ resolved against. The resolution keeps #312's side in all seven and adds #275's on top: | # | hunk | resolution | |---|---|---| | 1 | `resolve_neutral_pin_source` signature | #312's `sources: PinSources` and 3-tuple return, plus #275's `env_shared: bool = False` | | 2 | its docstring tail | #312's `Returns (path, how, typed)` text, plus #275's "Rule 2 is skipped when `env_shared`" paragraph | | 3 | `_recompute_neutral_pin_digest` signature | #312's `sources`, plus #275's `shared: frozenset[str]` after `report` | | 4 | the `resolve_neutral_pin_source(...)` call inside it | #312's 3-tuple unpack, plus `env_shared = product.casefold() in shared` and the extra argument | | 5 | `_check_one_neutral_product_pin` signature | #312's `sources`, plus #275's `shared` | | 6 | its call of `_recompute_neutral_pin_digest` | `sources, report, shared` | | 7 | `_check_neutral_product_pins` docstring and body | #312's `PinSources(...)` and `_note_unused_pin_sources`, plus #275's paragraph, `_report_pin_source_collisions(products, report)` and `shared = frozenset(pin_source_env_shared(products))` | This branch's own merge of 34dbd86 (5690235) had no conflict, and neither had its merge of #409's fix round, 6494714 (583f647). That fix round made `neutral-pin-source-collides` a note and keyed "one product" by `repo_shape.pin_product_key`. At 421e4e5, `templates/assembly-root/scripts/validate-pins.py`, `scripts/repo_shape.py` and `tests/test_pin_source_env_collisions.py` were byte-identical to the tree `git merge-tree` gives for 6494714 with origin/main 3ef059c (blobs 5604007, 091966b, 5220787), so this branch added nothing to those three files beyond #409. The 34dbd86 merge brought in #335, #312, #317, #270 (f795cf7), #344 and #349 with no conflict. Earlier merges of origin/main (87373a6, for #309's reworded constant, and e43fa95) are unchanged. Fix round 1 merged origin/main three more times: - **cd1fc79 (2bcfa69).** It brought #245, #351, #350, #345, #372, #371, #354, #380, #377 and the rest since ab623dd, with no conflict. - **45e01f4 (4098b61).** It brought #396's `tests/test_agents_md_codes.py`. That test finds `scaffold-pin-collides` as a string constant of `scripts/shape_materialize.py`, which `SHIPPED` lists. - **3ef059c (421e4e5), #314.** One conflict, one hunk in `adopt-project.py`. This branch's `_refuse_an_unrecordable_pin_source` and #314's section "The project must be able to hold what `execute` writes into it (#261)" were added at the same place; both are kept. Where the two meet, lane 5's earlier recipe applies: - `_flag_plan` builds each pin as the `PIN_FIELDS` mapping `plan` writes, not `list(pins)` of products. So the plan `plan` asks the validator about is the plan it writes. - #314's `test_plan_asks_what_check_asks_of_the_same_keys` plans `--pin openGlass@<commit> --pin-source <stand-in>` where it planned the name alone, which `plan` now refuses. Then f9ed46f, after #409 squash-landed, and 73d65c6 after it (top of this description). On the merged tree exactly one test failed: `tests/test_adopt_template_values.py::test_check_finds_and_execute_refuses_a_hand_edited_value[pins-surrogate]`. It plans, then hand-edits the plan to `pins: ["openX\ud800"]`, a name-only pin with a lone surrogate. Before this branch the refusal's headline was `REFUSED plan-unwritable-value`. A name-only pin in a plan is now `plan-pin-unrecorded` (#273), so `check` has a second, independent finding for that very plan, and the headline is the first finding's: ``` REFUSED plan-pin-unrecorded: 2 finding(s) ... FINDING plan-pin-unrecorded: `pins:` entry 1, 'openX\ud800', is a name with no commit and no digest ... FINDING plan-unwritable-value: a `pins:` item is 'openX\ud800', which `execute` writes into the assembly root's `project.yaml`, and it holds a lone surrogate ... ``` Nothing about #270's contract changed. The lone surrogate is still found as `plan-unwritable-value`, listed under the refusal, and `execute` still refuses before any leg exists. `check` builds `_leg_findings`, where `_pin_findings` sits, before it calls `_value_findings`, so the pin finding comes first. d806e8f changes one assertion for that row, `pins` only: the headline is `REFUSED plan-pin-unrecorded: ` where it was `PLAN_REFUSED`. The other 15 rows and the finding assertions, including the `plan-unwritable-value` line, are untouched. Lane 5's review of 7a94b6b found 3 should-fix and 5 nits. Each is answered here: - **#409's `pin_product_key` (review N1).** The merge of #409's fix round could not carry it, because this branch had moved the function that used it out of `scaffold-project.py`. `pin_variable_collisions` now keys each pin by that function, where it casefolded. `openß` and `openss` are two pin files and one `SHAPE_PIN_SOURCE_OPENSS`, and a casefold named each of the two pins twice. No `--pin` or recorded `pins:` entry `PIN_ARG_RE` admits can spell either, so the test asks the function directly. `refuse_colliding_pins`' docstring now says, in #409's words, that the root's `validate-pins.py` only notes such a pair. - **S1, a `--pin-source` path that is not printable.** `pin_digest_from_source` names the path as typed unless it is not printable (`str.isprintable`), and then as Python's `repr` spells it (`shown_source`). That covers the refusal's head and git's own reason, which names the directory it ran in. Printed raw, an ESC reached the terminal, and a line break or a carriage return began a line of the path's choosing, a forged `REFUSED` line among them, from the scaffold and from adopt. The traceback #339 replaced had shown a `repr`. A printable path is named byte for byte as before. - **S2, no `gh` on PATH.** `pin_digest_from_gh` refuses as `gh-not-found`, exit 2, as `repo_shape.tree_digest_from_gh` names it, and offers `--pin-source`. It was a `FileNotFoundError` traceback, exit 1, from the scaffold and, new with this branch, from `adopt-project.py plan --pin`. - **S3, a hand-edited `commit:` or `tree_sha256:`.** The lane's decision is to accept it and document it (option (c)); no behaviour changed. A pair that disagrees passes `check` and `execute` (`adoption verified`), because neither reads a network for a pin, by design. The adopted root's own `validate-pins.py` then exits 1 with `neutral-pin-digest-mismatch` wherever it can read the product. `check` now says so once, after the pins it will write: "each pin as `plan` read it, not read again here: the root's own `validate-pins.py` recomputes its digest wherever it can read the product". Options (a), `execute` recomputes before it creates the legs, and (b), `check` recomputes from a clone still on the machine, are filed as #449. - **N2, a `--pin-source` that `plan` cannot record.** `plan` asks `check`'s question (`one_printable_line`) of the path it records in each pin's `digest_source:`, before any digest is read. It refuses one that fails as `adopt-unwritable-value`, in #270's words (`unwritable_reason`), or as "an invisible formatting character" where those name nothing. Before, it was planned, exit 0, and printed raw in the report. `check` then refused it as `plan-pin-malformed` and said to re-run the same `plan`. The report's `pin` line and `check`'s `write` line now spell the digest's source with `_shown`. The scaffold half is pre-existing (a line break in the path is written into the pin file; a byte that is not UTF-8 dies after all three repositories exist) and is filed as #450. - **N3, six hunks no test guarded.** Each now has a test that fails when it alone is reverted: - the two name guards (a `pins:` name holding an ESC and a forged line); - the lower-casing of the commit and the digest (an upper-cased hand edit); - the `digest_source` printability check; - the not-text check (`commit: 1234567` unquoted); - the gitlink filter (a submodule kept where the pin is written is `plan-submodule-root` alone). - **N4.** This body. - **N5, #339's third decision.** A `--pin-source` that nothing reads is still silent, now in adopt too. This PR does not decide it. #451, filed, lays out refuse, note and leave, so closing #339 loses nothing. Lane 5's second review, of 421e4e5, found no blocking item, 2 should-fix items and 3 nits. The two should-fix items are this body: the landing after #409 (above) and #386 (above). The code needed no change for them. f63d1ed adds two tests and no code: - **n1. `plan` asks nothing of a `--pin-source` no `--pin` records.** With the `args.pin and` guard dropped, such a path holding an ESC or a zero-width joiner was refused in a sentence naming no file ("`execute` writes into , and ..."), and every test still passed. `test_a_pin_source_no_pin_records_is_not_asked_about` holds the plan to exit 0 and `pins: []`, with the path named nowhere in the plan or the report. The ESC row is POSIX-only by the file's `POSIX_NAMES` mark; the zero-width joiner row runs everywhere. It holds #451's answer today, silence, so a decision there changes this test on purpose. - **n2. A printable `--pin-source` is printed as it is.** With `_spelled` in place of `_shown` in `plan`'s `pin` line, every test still passed. `test_a_printable_pin_source_is_printed_as_it_is` holds `plan`'s `pin` line, and `check`'s `write` line for the plan `plan` wrote, to their exact text, ending `(local-clone (<path>))` with no backtick. - **n3** needs a code change and is left out (below). **Fix round 2, on f63d1ed.** Each run had a `gh` stub first on PATH (it logs and exits 97), `GH_TOKEN` bogus, an empty `GH_CONFIG_DIR`, `git-filter-repo` 2.47.0 next on PATH, and the identity blanked with `GIT_CONFIG_COUNT`; the session `gh` guard came first inside each run. - **The pin set: 812 passed, 2 skipped.** It covers adopt pins, adopt plan, #339's file, #275's tests, the pin and neutral-pin validators, git output decoding, adopt template values, CLI reference, scaffold pin and reuse, scaffold pin collisions, #314's two files and #396's file. The skips are `test_adopt_plan.py:403` (no MedxEHR clone) and `test_adopt_template_values.py:223` (this host reads `git config user.name` from the repository, not from `GIT_CONFIG_GLOBAL`). That is fix round 1's 809 passed, plus the three new rows. - **Shape doctor and scaffold leg-path collisions: 369 passed, 3 skipped** (the PowerShell line exists only on a Windows host). - **Mutation probes**, each one line changed in a throwaway worktree, over `tests/test_adopt_pins.py` and `tests/test_pin_source_unreadable.py` less the five end-to-end adoptions: - the `args.pin and` guard dropped: 2 failed, both n1 rows; - `plan`'s `pin` line `_spelled`: 1 failed, the n2 test; - `check`'s `write` line `_spelled`: 2 failed, the n2 test and `test_an_upper_cased_hand_edit_is_written_lower_cased`; - `plan`'s `pin` line raw: 1 failed, the no-break-space test that already guarded that hunk. **On the merged tree, 73d65c6** (the merge f9ed46f and the `_read_pins` commit), with the same harness: - **The pin set** above, with #445's `test_adopt_late_failure.py`, `test_adopt_git_reads.py` and `test_adopt_filter_repo_failure.py` and #387's `test_adopt_empty_symlinks.py` and `test_adopt_root_manifest.py`: **988 passed, 2 skipped**. The skips are the two above. - **Shape doctor, scaffold leg-path collisions and the adopt suites:** **967 passed, 3 skipped** (the PowerShell line). The adopt suites are e2e, execute refusals, verify, root shape paths (#387's too), leg-path collisions, clone template, ignored shape files and submodules. - **On the merge alone, f9ed46f** (the same tree as before the `_read_pins` commit): `tests/test_adopt_late_failure.py` gave 3 failed, 83 passed, the three `AttributeError` rows above. - `render-cli-reference.py --check` is ok on the merged tree. pyflakes reports the same four warnings on `adopt-project.py`, `scaffold-project.py` and `scripts/shape_materialize.py` as before the merge. **Fix round 1, on 421e4e5.** Each run had `git-filter-repo` 2.47.0 on PATH and the identity blanked with `GIT_CONFIG_COUNT`. A `gh` stub that logs and exits 97 was first on PATH, with `GH_TOKEN` bogus and an empty `GH_CONFIG_DIR`, and the session `gh` guard came first inside each run. - **The pin set:** 809 passed, 2 skipped (the same files and skips as above). - **Doctor and scaffold:** 782 passed, 3 skipped (the PowerShell line exists only on a Windows host). It covers shape doctor, scaffold leg-path collisions, scaffold e2e, referent chain links, naming policy, repo hygiene and #351's free-text values. - **The adopt suites:** 882 passed. It covers e2e, execute refusals, verify, spec-only, leg paths, root shape paths, plan escapes, leg-path collisions, clone template and submodules. - **Before the #314 merge,** on 74ce4d9's tree: the first two sets gave 577 passed, 2 skipped and 782 passed, 3 skipped. On 2bcfa69's tree (the merges alone, before the fix) they gave 542 passed, 2 skipped and 782 passed, 3 skipped. After the 45e01f4 merge, `tests/test_agents_md_codes.py`, repo hygiene and CLI reference: 110 passed. - **Lane 5's second review** also ran 421e4e5 merged with main cb7063f and with 53bee5b: the pin set, the manifest-value set, and the doctor set with #357's `tests/test_doctor_validator_rows.py`. All passed, with only the skips above. Proof that fix round 1's tests test something: - **On 2bcfa69's tree,** with this head's two test files laid over it, 31 of the new rows fail: - S1: 17; - N2 and its report spelling: 7; - S2: 3; - the `pin_product_key` port: 2; - `check`'s S3 line: 1; - the not-UTF-8 `adopt plan` row: 1. The N3 rows pass there, as they must: they guard hunks this branch already had. - **Mutation probes.** Each hunk was reverted alone, in a throwaway worktree, and both test files were run: 18 of 18 were caught. - **The port:** the casefold keying (2 fail); the casefold lookup alone (2). - **S1:** the path named raw (13); git's reason named raw (2); every path `repr`'d, printable or not (27). - **S2:** no `except` (3). - **N2:** no `plan` check (11); only #270's question asked (1); the formatting-character reason said otherwise (1); the report's `pin` line raw (1); `check`'s `write` line raw (1). - **S3:** the line dropped (1). - **N3:** the `_pin_fix` name guard (1); the `_read_pins` path guard (1); the lower-casing (1); the `digest_source` check (1); the not-text check (1); the gitlink filter (1). - **The issue rows reproduced.** S1, S2 and N2 were reproduced on 2bcfa69's tree and shown gone on the fix (a probe driving both tools; `gh` was called 0 times). **Before fix round 1.** Runs were on 7a94b6b, with `git-filter-repo` 2.47.0 first on PATH and the repository-local `user.name` blanked with `GIT_CONFIG_COUNT` (a workstation whose standard checkout sets one fails `test_the_scaffold_refuses_with_no_elector` on main itself; CI sets none). 7a94b6b differs from ffd1297 in `tests/test_pin_source_unreadable.py` only, so the three suites that do not read that file ran on ffd1297: - the new file: 46 passed; - adopt pins, adopt plan, git output decoding: 172 passed, 1 skipped (the MedxEHR clone is absent); - scaffold pin and reuse, pin collisions, e2e, referent chain links, neutral pin and pin validators, #275's tests, naming policy, repo hygiene, CLI reference, scaffold leg-path collisions: 571 passed; - shape doctor: 254 passed, 3 skipped (the PowerShell line exists only on a Windows host); - the new file with repo hygiene, git output decoding and CLI reference, once more after the last edit: 176 passed. Earlier on 5690235 and d806e8f, before the #339 commit: the adopt suites (e2e, execute refusals, verify, spec-only, leg paths, root shape paths, plan escapes, leg-path collisions, clone template) 740 passed, and `tests/test_adopt_template_values.py` with #273's other adopt tests 489 passed, 2 skipped. They were not re-run on 7a94b6b: the #339 diff touches `pin_digest_from_source` and the call in `declared_pin_values`, which those suites reach through `test_adopt_pins.py` only. Nothing was run on macOS or Windows. Proof the new tests test something: - **On d806e8f.** 38 of the 46 fail, each on the traceback and its exit 1. The 8 that pass are the unchanged behaviour: the not-a-clone refusal (4 rows) and the valid-source rows (4). - **Mutation probes.** Ten hunks reverted one at a time, each failing at least one new test: the `OSError` catch (16 fail), the catch narrowed to `OSError` only (21), the missing-path clause (9), the clause narrowed to `FileNotFoundError` (3), the `S_ISDIR` judgement (3), the loop guard (4), no resolve at all (3), the old `.resolve()` call (4), the reason as `str(exc)` (12), and the file sentence as the OS's (3). - #273's own 52 tests still fail on c7c02fa (51 fail or error; the one that passes checks the test's own table), and its 18 mutation probes were each caught. **Re-verified after the merge.** An independent re-verification of f9ed46f and 73d65c6 found the code correct: - #409 is carried whole. A 30,000-case fuzz of `--pin` lists through #409's final refusal and this branch's port differed in 0 cases. - `scaffold-project.py` equals the tested file plus main's own changes that are not #409's (same patch-id). - `_commit_the_split` keeps #445's `try` and #394's `append`. - `_read_pins` reads a loaded plan as before. Its findings were all in this description (the #506 bullet and three stale facts), now fixed. Runs on the head and on its merge with main 48585e6: 988 passed, 2 skipped; and 967 passed, 3 skipped. `git merge-tree --write-tree` was run at 01:48Z on 2026-10-13, comparing hunks per file. Each PR head, fetched from `refs/pull/<n>/head` with no `gh`, was merged with 73d65c6 and with origin/main 9736424 alone. Two sets were asked about: - every PR this description listed before that main does not now contain; - every PR numbered above #468 whose head main does not contain: #473, #474, #490, #491, #493, #494, #495, #496, #497, #503, #504 and #505. **Open state, read with `gh` by the coordinator at 02:00Z and refreshed at 02:25Z after a re-verification on main 48585e6.** Every pull request named below is open. #184, #94, #96, #98, #100 and #161 are closed, and #465 landed as 48585e6. `fix/adopt-plan-quote-yaml-keywords` has no pull request. #506 (`fix/scaffold-reuse-seeded-legs`) opened after this list was made, and the re-verification reads it. - **Landed since the last draft of this list:** #343, #346, #355, #358, #373, #378, #382, #387, #394, #397, #401, #404, #409, #419, #452, #456, #465, #466 and #484. #394's `_commit_the_split` conflict and #401's `docs/cli.md` conflict are both resolved in f9ed46f, above. - **Clean, 20:** #266, #347, #356, #374, #376, #384, #395, #453, #467, #468, #473, #474, #490, #491, #493, #494, #495, #497, #503 and #505. #453, #468 and #491 were updated onto main since the first read and now merge clean. #507 (lane 5's #486-#488) shares only the generated `docs/cli.md` with this branch and merges clean with it. - **Conflicting with main alone, and this branch adds no hunk:** - #402: `shape-doctor.py` 5; - #496: `shape-doctor.py` 10 and `tests/test_shape_doctor.py` 1. **Conflicts this branch adds:** - **#435** (3efcc7a, lane 2's `fix/adopt-plan-field-types`, for #386). - It adds 2 hunks in `adopt-project.py`: 5 here, against 3 with main alone. They are `render_plan`'s `pins:` line and `Plan.pins`. - Keep this branch's `_pin_lines` and `_pin_products`. The rest is under "Where the meaning meets". - **#504** (37ef29a, `fix/adopt-root-kept-symlinks`). - It adds 1 hunk in `adopt-project.py`: 2 here, against 1 with main alone. It is the materializer call in `_commit_the_split`. - Keep both arguments and both comments: `append=append, neutral_pins=_recorded_pins(plan), spell=_spelled`. - **#506** (119d8c4, `fix/scaffold-reuse-seeded-legs`). - It adds 1 hunk in `scaffold-project.py`. With main alone, #506 conflicts only in `docs/handbook.html`. - #506 inserts `LEG_TEMPLATES` right after `declared_pin_products`, which this branch moved out, and rewrites `_reusable_remotes` further down. Keeping both sides breaks the file with an unterminated string literal. - Keep only #506's `LEG_TEMPLATES` block. Drop #506's `declared_pin_values`/`declared_pin_products`, which moved here, and this branch's three-line `_reusable_remotes` opener, which #506 rewrites below. Resolved that way, pyflakes is clean and #506's scaffold tests pass (88). - **`fix/adopt-plan-quote-yaml-keywords`** (ed9968f; a branch with no pull request). - It adds 2 hunks in `adopt-project.py`: 3 here, against 1 with main alone. - One is `render_plan`'s `pins:` line. Keep `_pin_lines`, which already sends product, repository and `digest_source` through `y`. - The other is `_refuse_unholdable_values`' signature. Keep `pins: dict`, with that branch's `_validator_judged` and `_named_as_read` above it. - `fix/adopt-late-failure-notice` is no longer on origin. #445, the late-failure notice, is on main and merged here. **Where the meaning meets:** - **#435 and #386.** The coordinator has said this on #435: #435 (comment) - #435's `_plan_key_findings` (`plan-bad-pins`) keeps only text items in `pins:`, and its `PINS_REMEDIATION` tells a person to write `pins: [openGlass]`. - This branch writes each pin as a mapping, and it makes a name alone `plan-pin-unrecorded`. - Read from #435's code, not run: on a tree with both, the plan `plan --pin` writes is `plan-bad-pins` in `check`. - Whichever lands second reconciles them, with three changes: - `_plan_key_findings` takes this branch's mappings, or leaves `pins:` items to `_read_pins`; - a `pins:` that is not a list gives one finding, not both `plan-bad-pins` and `plan-pin-malformed`; - `PINS_REMEDIATION` says the mapping `plan` writes. - Until that is done, `tests/test_adopt_pins.py::test_a_plan_with_a_whole_pin_checks_ok` fails. - **#314.** It landed, and it is merged here (above). - **`fix/adopt-plan-quote-yaml-keywords`.** Its pin tests are written against the name-only `--pin`. They need the `name@commit` form, and their expected `pins: [...]` line becomes `_pin_lines`'s mapping. - **Two unwritable values are refused in two runs (lane 5's second review, n3).** `plan --elected-by 'A<LF>B' --pin openGlass@<sha> --pin-source <a path holding an ESC>` refuses the elector alone, exit 2. Re-run with the elector retyped, it refuses the source; #270's refusal names every unwritable value at once. Reproduced on 421e4e5, with `gh` called 0 times. One refusal would need a code change, not made here: - `_refuse_an_unrecordable_pin_source` would split into a function that returns the sentence; - `_checked_plan_inputs` would ask it before #270's `unwritable_values` raise, and raise one `adopt-unwritable-value` that names both, with both remediations; - a test would add the two-value row. N2's question is `one_printable_line`, not `unwritable_reason`, so it cannot simply join #270's list. - **Docs.** The README, AGENTS.md and `docs/handbook.html` are unchanged, because none of them calls adopt's `--pin` a name. A sentence saying adopt now takes the scaffold's `--pin` is a follow-up. It costs an AGENTS.md line-cap raise, and a README edit forces the handbook's README blob to change. - **A product spelled `null`, `true` or `false`.** The scaffold already accepts `--pin null@<sha>`: it exits 0 and builds a root whose `validate-pins.py` exits 1 `neutral-pin-entry-malformed`. Adopt's `plan` writes `product: null` through `y`, and `check` then finds `plan-pin-malformed` and refuses it before any leg exists. The one fix for both tools is in `parse_pin`, and it would change the scaffold's behaviour. - **Wording shared with the scaffold.** `digest_source` is written bare into the pin file, so a `--pin-source` path holding ` #` is cut short there. `templates/assembly-root/contracts/neutral-product-pin.yaml` and `validate-pins.py`'s `neutral-pin-missing` remediation name only the scaffold as the writer. Two comments in `scripts/repo_shape.py` still name the scaffold's `pin_digest_from_gh`, in a digest-pinned shipped copy. - **Filed as follow-ups:** - #449: S3, a hand-edited `commit:` or `tree_sha256:` that disagrees, with options (a) and (b); - #450: the scaffold half of N2, a `--pin-source` path no file can hold; - #451: #339's third decision (N5), a `--pin-source` that nothing reads. - **`validate-pins.py` prints a `--pin-source` path raw** in `neutral-pin-source-unreadable`, the twin of S1. It is a shipped template that #409 and other open PRs edit, so it is left for its own change. No issue is filed for it yet. - **Windows and macOS** were not run here; CI ran both (below). Two assertions that depend on what Windows answers for a path (a name past the filesystem's limit, a path beneath a file) check only the refusal and its remediation there. Fix round 1's rows that need a control character or a byte that is not UTF-8 in a path, or a symlink to `git`, are skipped on Windows by a mark, and so is fix round 2's ESC row. The rows for a no-break space and a zero-width joiner run everywhere, and so does the in-process `gh-not-found` test. Reviewed independently by lane 5 at 7a94b6b (3 should-fix, 5 nits: fix round 1) and at 421e4e5 (0 blockers, 2 should-fix, 3 nits: fix round 2). After #409's squash, the re-verification of f9ed46f and 73d65c6 found the code correct; its 1 should-fix and 3 nits were in this description and are fixed. GitHub's update-branch then merged main twice (33bae3f, 2170526), with no hand edit. CI's tests, tests-macos and tests-windows and SonarCloud all passed at 2170526 (main cac6bb2 merged in). On that head merged with main 57c85e9 (#490, #474 and #468, none of which touches this branch's files), the pin and adopt set with `tests/test_update_shape_landing.py` gave 1112 passed, 2 skipped, and the doctor, adopt end-to-end, unfinished-merge and bootstrap set gave 1056 passed, 3 skipped (the doctor's PowerShell line, which exists only on a Windows host). Closes #273 Closes #339 Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap
added a commit
that referenced
this pull request
Oct 13, 2026
… text placeholders, so `--id 007`, `--tracking-branch 2026`, `--spec-path true` or `--org null` read back as typed, and `family.py add` quotes a member id alike (#259) (#395) Brett Heap ruled on 2026-10-12: "quote #259". The templates now write each text field a tool fills from a value it was given between double quotes: | template | fields | |---|---| | `templates/assembly-root/project.yaml` | `id`, `elected_on`, `visibility`, `tracking_branch`, the spec leg's `path`, the code leg's `path` | | `templates/assembly-root/contracts/spec-pin.yaml`, `code-pin.yaml` | `submodule_path` | | `templates/family-root/family.yaml` | `id`, `org`, `created_on`, `visibility`, `tracking_branch` | `render` already spells a value between a YAML template's double quotes with `repo_shape.double_quoted` (#284). The single-pass `render` from #316, on main as 34ecbc2, does the same through `YAML_PLACEHOLDER_RE`. So each value reads back exactly. An ordinary value renders as it did before, plus the two quote characters. `family.yaml`'s `org:` is quoted along with the four fields #259 names. It is the same defect in the same file: `family.py init --org 2026` and `--org null` pass `checked_value`, and on c7c02fa the holder then fails `family-org` (#307 lists the `--org null` row). Taking `org:` back out is not a one-line revert. It fails 14 of the new tests (the 8 `family.yaml:org` in-memory cases, and all 6 `family.py init` rows, because every row of `FAMILIES` passes a misread `--org`), and it would also take the `ORG` row out of `TEXT_FIELDS` and put plain `--org` values into `FAMILIES`. The lane keeps `org:` quoted, as the same defect in the same file, unless Brett says otherwise. `scripts/family.py`: `_member_lines` writes each member's own `project.yaml` id into the holder's `members:` block. That id now reads back exactly, so a bare `007` in the row came back as 7, and `validate-family.py` would report `member-id-mismatch` for a member it had just pinned. `_member_id` quotes an id only when the reader would not read the bare text back as the same text. No row whose id reads back as text moves: `irrs` is still `id: irrs`. Two edges: - A legacy row whose bare id the reader misreads moves once. A holder written before this change, by `family.py add` of a member scaffolded with `--id 2026`, holds `id: 2026`. The next `bump`, `add` or `remove` of that holder rewrites it as `id: "2026"`. That is harmless: `validate-family.py` compares with `str()` and passes before and after. - Mixed versions. A `family.py` from before this change, run against a holder this branch wrote, turns `id: "007"` back into `id: 007`, and `validate-family.py` then fails `member-id-mismatch` (the member declares `007`, `family.yaml` records `7`). A current `family.py` that rewrites the holder next then writes that row as `id: "7"`. Nothing in this branch can prevent it: run `family.py` from a current checkout on a holder that has such a member. Left unquoted on purpose: `topic` (`xf-project-<id>`), every `repository` (`<org>/<Name>`; the slash keeps it text), `SHAPE_REPOSITORY` and `DIGEST_DEFINITION` (constants), and the block or list placeholders (`NEUTRAL_PRODUCT_PINS`, `*_NAMING`, `SHAPE_FILES`). **Why.** Every reader of these files uses `repo_shape`'s YAML subset. It reads a bare `007` as 7, `2026` and `20261011` as numbers, `1.5` as a float, `true` as True and `null` as None. The checks on the flags (`PROJECT_ID_RE`, `checked_value`, the leg-path rule) accept every one of these. Reproduced on c7c02fa, with local bare remotes only: - `scaffold-project.py --id 2026|007|true|null|20261011` exits 0, and the project's own `validate-manifest.py` then fails `manifest-id`. - `--spec-path 2026 --code-path 007` fails `manifest-leg-path`, and `validate-pins.py` exits 2. - `--tracking-branch 007` pushes each leg on `007`, and the root's `bootstrap.py` then puts the legs on a local `7` with `fatal: no upstream configured`. `true` and `null` do the same on `True` and `main`. - `adopt-project.py execute` with `--tracking-branch 007` does the same; `--id 007|20261011` gives `manifest-id`; leg paths `2026`/`007` give `manifest-leg-path`. - `family.py init --id 2026|007|true|null|20261011` gives `family-id`; `--org 2026|null` gives `family-org`; `--tracking-branch 007|true|null` reads back as 7/True/None. On this branch every one of those rows reads back as typed, bootstraps onto the typed branch with `origin/<branch>` as upstream, and passes the root's own validators. **Every reader checked.** Every reader reads these fields through `load_yaml`/`parse_yaml` into a dict. None uses a line regex, `split(":")`, `startswith("<field>: ")` or a whole-line rewrite of them: - `templates/assembly-root/scripts/bootstrap.py`: `tracking_branch`, leg `path`. - `validate-manifest.py`: `id`, `elected_on`, `visibility`, leg `path`. - `validate-pins.py`: `submodule_path` against the manifest's `path`. The pin side is read as it stands and the manifest side goes through `str()`, so the two differ in type exactly when a bare value is misread (that is the `pin-path-mismatch` row above). With both lines quoted they are the same text. - `validate-repository-naming.py`: `str(id)` for the topic. - `templates/family-root/scripts/validate-family.py`: `id`, `org`, `created_on`, `visibility`, `str()` member-id compare. - `siblings.py`, family `bootstrap.py`. - `scripts/bump-leg.py`: `tracking`, leg `path`. Its `rewrite_pin` rewrites only `commit:` and `tree_sha256:`. - `update-shape.py`: `tracking_branch_of`. `rewrite_manifest` touches only the `shape:` block. - `scripts/family.py`: `tracking()`, the member's id, and the `members:` rewrite. The member-id writer is the one fixed here. - `shape-doctor.py`: leg `path`, member row `path`, working-clone `str(id)` compare. - `adopt-project.py`: plan reads and `_template_values`. - `setup-project.py`: `family.yaml` `name`/`kind` only. - `scripts/shape_advisory.py`: the single-repository record only. The `str()` fallbacks (`bootstrap.py`, `adopt-project.py`'s `checked_value`) stay, because projects made before this keep their bare lines. **Pins.** The four files are TEMPLATED, not COPIED, so no row of any project's `contracts/shape-pin.yaml` names them. `Atlas` was scaffolded and `family.py init Contoso` was run on c7c02fa and on this branch. Each `contracts/shape-pin.yaml` differs only in `commit:` and `digests.tree_sha256`, which identify the standard itself and change with every commit; every per-file `sha256` row is identical. `update-shape.py check` of the c7c02fa project and holder against this branch reports every copied file `unchanged` (11 and 10 rows): "no copied file differs; `apply` would move the pin alone". **Tests.** - New: `tests/test_template_text_placeholders.py`, 123 tests: - Each of the 13 fields is rendered with each of 8 values the reader misreads (`2026 007 1.5 true null 20261011 ~ False`), and must read back exactly. - 6 `scaffold-project.py` runs, then a recursive clone, `bootstrap.py`, the three validators, and `bump-leg.py`, which refuses on the tracking branch and finds the leg at its path on a branch of its own. - 6 `adopt-project.py` `plan`/`check`/`execute` runs into local bare remotes, then a clone, `bootstrap.py` and the validators. - 6 `family.py init` runs. - One `family.py add` of members whose ids are `007` and `true`. - **All 123 fail on c7c02fa.** - `true`/`null` through adopt: `plan` writes them bare into the plan, and the plan's reader then misreads them. That belongs to the plan writer, #181 (lane 1). Those two rows quote the value in the plan by hand, as #181's writer will, and test `check`/`execute`. - Mutation probes: reverting any one of the 13 quoted lines fails 8 cases of its field. Reverting `_member_id` fails the member test (the rows read 7/True). - The `try/except YamlError` in `_member_id` is guarded by the follow-up commit. The review found that calling the parse directly left the member test passing. Ids such as `[`, `{`, `&a`, `*a` and `!t` reach the `except` (they raise `YamlError` bare), and `_member_id` returns each one exactly, as it does 34 adversarial inputs in all (the review's in-process probe). `test_a_member_id_the_reader_refuses_bare_is_rewritten_between_quotes` (7 cases: `[`, `{`, `&a`, `*a`, `!t`, an unclosed `'` and an unclosed `"`) opens a hand-edited holder whose row holds the id between double quotes, rewrites the `members:` block with `Family.rewritten`, and asks that the id read back exactly and the text come out byte for byte. With the `try/except` removed all 7 fail with the reader's `YamlError`, and with the `except` returning the bare text all 7 fail too; the old member test still passes under the first of those mutants. - 14 existing tests that asserted or edited a bare rendered line now spell it quoted: `test_manifest_validator` ×5, `test_pin_validator`, `test_reference_default`, `test_scaffold_e2e` ×2, `test_setup_project_py`, `test_setup_sh`, `test_family`, `test_shape_doctor` ×2. - Targeted runs on this branch: | files | result | |---|---| | new file + `test_template_quoted_values`, `test_scaffold_e2e`, `test_scaffold_pin_and_reuse`, `test_scaffold_leg_paths`, `test_scaffold_leg_path_collisions`, `test_reuse_visibility`, `test_reference_default` | 405 passed | | the six adopt files | 491 passed, 1 skipped (the MedxEHR clone is not set) | | `test_family`, `test_family_add_refusals`, `test_family_member_path`, `test_manifest_validator`, `test_pin_validator`, `test_bump_leg`, `test_update_shape`, `test_update_shape_pr_base`, `test_setup_project_py`, `test_park_resume_targets`, `test_repo_hygiene`, `test_cli_reference`, `test_setup_sh` | 603 passed | | `test_shape_doctor` | 241 passed, 3 skipped (Windows-only), 1 failed; that one test was fixed, and with its sibling it reran at 2 passed | **The review.** An independent opus review at 233dc71 (the branch's first commit) found no code defect. Its verdict was CHANGES NEEDED (0 blockers, 2 should-fixes, 6 nits) because it asked for this description and its composition counts to be corrected, and they are. Its evidence: - **Readers and rewriters.** Every reader in the shipped tree reads the 13 lines through `load_yaml`/`parse_yaml`, and no line regex or whole-line rewrite touches them. It read `bump-leg.py` `rewrite_pin`, `update-shape.py` `rewrite_shape_pin` and `rewrite_manifest`, `family.py` `rewritten`, the three validators, `siblings.py`, both `bootstrap.py`, the doctor, `setup-project.py`, the workflows, the Makefile and `setup.sh`. No test among main's 17 commits since c7c02fa (through #245's 161c6c9) edits one of these lines bare. - **`_member_id`, the one rewriter changed.** A mutant that writes every id bare fails the member test (the rows read back as `{'IRRS': 7, 'IRSS': True}`). A mutant that quotes every id fails `b" id: irqs" in lines`. So the minimal-churn choice is tested both ways. - **Old projects and holders.** Against a control commit (c7c02fa plus an unrelated README edit), a project and a holder made on c7c02fa give identical `update-shape.py check` and `shape-doctor.py` output, up to the shas. `check --at 233dc71` reports all 11 project files and all 10 holder files `unchanged` and says `no copied file differs`. The doctor's verdict is `COMPLIANT, SHAPE BEHIND (... no copied file differs)` for the control and for 233dc71. This branch's `family.py add` and `bump` on a c7c02fa holder leave the old `atlas` row and the header byte for byte, apart from the legacy-id edge described at the top. - **Tests.** All 123 new tests fail on c7c02fa (`123 failed`) and pass on 233dc71 (`123 passed`). The 14 changed existing tests pass (49 passed). - **Not covered.** Windows and macOS were reviewed by reading and not run: the plan edit and the member-row check are in bytes, and nothing writes to git's stdin in text mode. Of `test_shape_doctor.py` and `test_family.py` the review ran only the changed tests. **Follow-up commit.** A second commit, `3894aa8`, answers two nits from the review, and changes no behaviour. The `_member_id` docstring said "no existing holder's `members:` block moves by a byte", which was false: the next `bump`, `add` or `remove` of a holder written before this change rewrites a legacy `id: 2026` row as `id: "2026"`, harmlessly, because `validate-family.py` compares with `str()`. The docstring now says exactly that, and a grep of every comment and docstring the branch added for "by a byte" and "moves" found one more copy, in the member test's docstring, corrected the same way. The `try/except YamlError` in `_member_id` is guarded by a new test (see Tests), so the new file holds 130 tests. Every count above that says 123 (123 new tests, all failing on c7c02fa and passing on 233dc71) is for the first commit; the follow-up commit adds the 7. The `except` also carries a comment saying which ids reach it. **Composition at the landing head (4984651).** `49846518` is GitHub's update-branch merge of origin/main `cc8f2da4` (#376, a `--trailer` value carried into the commit as typed) into `3220c9ee`. Git reported no conflict, and the merge has no hand edit. #376 changed `scripts/family.py` and `tests/test_family.py`, which this branch also changes. None of the lines it added under `tests/` names a key the templates now quote. The coordinator ran targeted tests on this head, Linux only, before CI: - main's changed test files (`tests/test_commit_trailers.py`, `tests/test_family.py`, `tests/test_bump_leg.py`, `tests/test_update_shape.py`) with `tests/test_family_add_member_id.py`: 421 passed, 1 skipped (a case-insensitive disk only); - this branch's template set (`tests/test_template_text_placeholders.py`, `tests/test_template_quoted_values.py`, `tests/test_scaffold_manifest_values.py`, `tests/test_bootstrap_spelling.py`, `tests/test_bootstrap_after_bump.py`) with the hygiene, CLI-reference and AGENTS.md-codes tests: 652 passed. **Composition at 3220c9e.** `3220c9ee` is GitHub's update-branch merge of origin/main `70d0d1b3` into `e5e768bc`. It brings in three commits: #491 (da19735), in which `setup-project.py` asks at the person's own terminal; #374 (7ab11cf), update-shape's NEXT quoting; and #384 (70d0d1b), adopt's `_spelled` plan values. Git reported no conflict, and the merge has no hand edit. #491 changed `tests/test_setup_project_py.py` and `tests/test_setup_sh.py`, which this branch also changes. A grep of the lines main added under `tests/` for the keys the templates now quote found two hits, and neither reads a rendered template line: `tests/test_update_shape_next_quoting.py` writes its own `project.yaml`, and `tests/test_adopt_question_and_submodule_spelling.py` edits a plan's `- path:`/`question:` entry, which `adopt-project.py` writes bare. The coordinator ran targeted tests on the same tree (`e5a74d19`), Linux only, before CI: - six of main's eight changed test files (the setup, update-shape, `openRepoShape` command and doctor-row tests): 365 passed; - main's other two (adopt's spelling and repo hygiene), `tests/test_adopt_github_exits.py` and nine more files of this branch's sixteen-file set: 943 passed. **Composition, refreshed again 2026-10-13 (head e5e768b).** `6d108d18` merges origin/main `56713e21` into `6cbbac60`. That brings in two commits: 5e02abe, the doctor's and `validate-pins.py`'s gitlink repairs, and 56713e2 (#479), in which adopt's `plan` takes the tracking branch from the source's default branch. Git reported no conflict and `docs/cli.md` auto-merged; the merge has no hand edit. Two kinds of test failed on it, and two commits fix them: - **`7968f902`** changes one expected literal, at `tests/test_adopt_github_exits.py:675`, for `test_a_source_on_another_branch_opens_its_pull_request_against_it[master]` and `[trunk]`. - The test reads the adopted root's `project.yaml`, which this branch's template renders as `tracking_branch: "master"`, so it now expects the quoted line. - The look-alike lines at `:743` and `tests/test_adopt_tracking_branch.py:73-76` read a plan, which `adopt-project.py` writes bare, and they stay as they are. - **`e5e768bc`** adds `git("branch", branch, cwd=source)` to `tests/test_template_text_placeholders.py::test_an_adoption_reads_back_every_value_it_was_given`. - All six rows had failed at `check` with `plan-tracking-branch-missing`: #479 refuses a tracking branch (`2026`, `007`, `1.5`, `20261011`, `true`, `null`) the source does not have, and the test's source had only `main`. - The source now carries each row's branch, in the flag rows and the by-hand rows alike. The test's assertions are unchanged. Targeted runs, Linux only: - On e5e768b: main's seven changed test files, 211 passed; this branch's sixteen-file set, 1223 passed, 2 skipped. - On 7968f90: the same sixteen files gave 6 failed, which are the six rows e5e768b fixes. The scaffold, setup, doctor and family set gave 1094 passed, 22 skipped, and twelve further template-grep files gave 1259 passed. - `render-cli-reference.py --check` is ok. **Composition, refreshed 2026-10-13 (head 6cbbac6).** `d651b594` merges origin/main `aa4e1f59` (#508) into `d526f768`. - **The one conflict.** Git conflicted in one file, `tests/test_adopt_manifest_values.py`, in four hunks, all against #356 (965420b, #293/#294). #356 refuses an `elected_on:` that `election_date` does not read, as `election-date-malformed`, before the validator is asked (`_manifest_values` returns None, adopt-project.py:2258). - **Resolved by what the merged tree prints**, measured with `adopt-project.py plan`/`check`/`execute` into local bare remotes: - `id: "007"` checks clean and `execute` writes it as typed, so it stays in `TEXT_THE_TEMPLATE_NOW_HOLDS`. - `visibility: "true"` gets the validator's `manifest-visibility: visibility is 'true'`. - `elected_on: "20261011"` gets #294's one finding, "the plan's `elected_on:` is the text '20261011'", and the validator is not asked. That row keeps this branch's list, `TEXT_THE_VALIDATOR_REFUSES`, with main's sentence. - Two tests git merged cleanly now ask `said_of`, as main's own rows do: `test_check_refuses_a_text_value_the_validator_refuses` and `test_where_a_misread_rule_judges_the_rest_the_validator_still_answers`. - **Two more failures that no hunk showed**, fixed in the same merge: - `[branch-tilde]` in `tests/test_scaffold_manifest_values.py` was this branch's own `xfail(strict=True)` until #355 landed as 6c2a371, so it XPASSed. The mark is off, and the row asserts `REFUSED scaffold-bad-tracking-branch:`. - #391's `tests/test_bootstrap_spelling.py` and #490's `tests/test_bootstrap_after_bump.py` matched ` path: spec` and `tracking_branch: main` in a scaffolded `project.yaml`, and nine cases failed on the merge. They now match the quoted lines. - **The #507 interplay.** `6cbbac60` switches the holder to `family/members` before the real `add`s in `test_a_member_whose_id_yaml_would_misread_is_pinned_under_that_id`, so the test holds once PR #507's `family-on-tracking-branch` refusal lands. - On the clean merge of 6cbbac6 with #507's head 708f3bb, that file, all 13 `tests/test_family*.py` and `tests/test_shape_doctor.py::test_a_sibling_at_the_right_origin_with_another_id_is_not_the_member` give 686 passed, 19 skipped. - Without the switch line, the member test fails `REFUSED family-on-tracking-branch`. - **Targeted runs on 6cbbac6, Linux only:** - the template, adopt-plan and manifest files (14): 995 passed, 2 skipped; - scaffold e2e, setup, the doctor, leg-path collisions and every family file: 1094 passed, 22 skipped; - the 15 further files a grep for template lines found: 1503 passed; - `render-cli-reference.py --check`: ok. - **Re-verified independently** at d651b59 and 6cbbac6: APPROVE WITH NITS (0/0/3), all three in this paragraph and fixed here. The re-verification also ran the three rows, five mutations, and 26 more files (834 passed; 291 passed, 1 skipped). - **Not run locally:** the full suite, macOS and Windows; CI's three jobs are the check. **Composition, as measured on 2026-10-12.** The merge trees below were computed for 233dc71; the follow-up commit 3894aa8 changes only `scripts/family.py` comments and adds tests. At 15:28Z main was 3ae0109 (#245, #351, #350 and #345 had landed since the merge trees): 3894aa8 merges clean with it, and the review's re-verify ran the family tests on that merge (61 passed). Every sha below is the one it had then; main and the other branches have moved since. `git merge-tree --write-tree --name-only`, run after a fetch with automatic maintenance off on 2026-10-12: - **origin/main 161c6c9**, 17 commits past c7c02fa (the base): clean, tree 7ba8246. #316 (34ecbc2), #317, #318, #311 and #245 were on main by then. - **The 46 other remote branch heads** (the 30 open PR heads and 16 branches with no open PR): clean for 44. Among them #314 (4b8da3d), #266 (3410be7), #355 (a03abcf), lane 1's `fix/adopt-plan-quote-yaml-keywords` (d38e143), `fix/adopt-record-submodule-placement` (70f4d29) and `fix/adopt-late-failure-notice` (9f1778d), #380 (73068a3), and `fix/family-init-values` (14e2d68, #307). The two that conflict, each in the same file as it does with c7c02fa and with origin/main on its own: lane 1's `fix/adopt-root-kept-symlinks` (6c44591, `adopt-project.py`) and #358 `fix/doctor-manifest-kinds-unreadable-pin` (ebffb08, `shape-doctor.py`). This branch touches neither file. - #314 and d38e143 each conflicted with origin/main on their own, in `adopt-project.py`. Merged with this branch, neither gained a conflict. **Composition with #314, which lands before this branch.** Its misread gate (`manifest_misreads`) reads the template, so once the template quotes a field it refuses nothing there, and some of #314's tests expected that refusal. Composing with #314 (landed as 3ef059c), whose misread gate asks the template how a value is written. With `project.yaml` quoting `id`, `elected_on`, `visibility` and `tracking_branch`, the gate refuses nothing on the shipped templates, and 45 of #314's tests flipped on the merge. Forty expected a refusal the value no longer needs. Thirty-four of them now assert that the value is accepted, and the six whose premise was a bare placeholder now assert that no shipped template placeholder is misread. For example: a scaffold of `--id null|true|false|007|2026`, or of `--tracking-branch true|True|TRUE|false|False|null|Null|NULL|007|2026|1.5|1.50|0`, exits 0, and the manifest holds the text it was given. A dry run prints the plan. An id derived from `--project True|Null` is held as typed. `--id=null` reaches `gh repo create`. Adopt's `id: "007"` checks clean and `execute` writes it as typed. The rule's own sentence, remediation and line-break wording are still asserted, on a copy of the standard whose `project.yaml` writes the four lines bare. Four stay refused by #314's validator gate, now in its words about the text: `visibility is 'true'` and `elected_on is '20261011'`. One is a real break. `--tracking-branch=~` reads back as `~`, which git refuses as a branch name, and it dies at `git init -b ~`, after `gh repo create`. That is #283, so the row is `xfail(strict=True)` with that reason, and it turns XPASS (and fails) once PR #355's `git check-ref-format --branch` refusal lands. `--help` for `--id` and `--tracking-branch` no longer says such values are refused, and `docs/cli.md` is regenerated. **Composition with the other branches.** Where another branch's rules meet this one, the tests were run on the merged tree. Quoting changes what some of their tests see, and the changes are not all one kind: some tests expected a refusal the value no longer needs, some are still refused under another rule, and some guard a case quoting does not fix. Whoever lands second must tell them apart, so each branch's flips are classified below. - **#266** (lane 2, `fix/leg-path-yaml-scalars`). 27 flips, all from one cause: its rule asks a hard-coded bare line, `LEG_PATH_LINE = "path: {}\n"`, so it refuses leg paths the quoted templates now read back. 15 are `test_the_rule_agrees_with_every_template_line_that_holds_a_leg_path` (null, Null, NULL, ~, true, True, TRUE, false, False, FALSE, 2026, 007, 0, 1.5, 1.50) and 12 are this branch's own end-to-end tests (6 scaffold rows fail with `REFUSED scaffold-bad-leg-path`, 6 adopt rows with `FINDING plan-bad-leg-path`). Counted on the merge with b2b42f8. #266's head was then 3410be7, whose rule reads each template line that holds a leg path, renders the value into it as `render` does and asks the reader, so it has no hard-coded line left. The merge with this branch was clean (tree 1e348c6); that head was read and the 27 were not re-run on it. At b2b42f8, whichever of #266 and #259 landed second had to drop or re-point the bare rule; 3410be7 re-points it already. - **Lane 1's #181** (`fix/adopt-plan-quote-yaml-keywords`, d38e143). Its gate reads the template too, so `plan --tracking-branch 2026|007|true` exits 0 on the composed tree (exit 2 `adopt-manifest-misread` on d38e143). On the merge, `tests/test_adopt_plan_yaml_quoting.py` and this branch's 6 adopt end-to-end tests give 70 failed, 371 passed: - 57 refusals that are now unnecessary: `plan_refuses` ×14, `plan_names_an_id` ×2, `check_names_an_id` ×4, `check_reports` ×14 and `execute_refuses` ×14 (every value except `elected_on` and `visibility`), `text_the_loader_reads_as_other_text` ×7, and the two tests that need a bare placeholder (`test_the_gate_follows_the_loader_when_the_loader_learns_a_keyword`, `test_the_template_line_says_whether_a_value_is_quoted`). - 11 guards #181 loses. See "Do not flip". - 2 that fail only by cascade: `execute_refuses[pins='null']` and `[pins='True']` pass alone (`2 passed`). An earlier case no longer refuses, and its `execute` pushes the split branch. - The writer half of #181 (`y` asking the reader, so `true`/`null`/`True` are quoted in the plan) is still needed and composes cleanly. With it, `plan --tracking-branch true` reaches the quoted template intact, and this branch's two hand-quoted adopt rows become flag-driven. - **Do not flip these 12 to "accepted".** Each still needs a refusal or a finding, and quoting does not supply it: - **`~` (1, #314).** `test_a_value_the_reader_misreads_is_refused_before_anything_is_made[branch-tilde]`. `--tracking-branch=~` is not a branch name git accepts, and #314 refused it only as a misread. On the merge it passes the plan and dies at `git init -q --bare -b ~ .../Atlas-spec.git` with `fatal: invalid initial branch name: '~'`, and against GitHub that is after `gh repo create` has made all three repositories. That is lane 2's #283 (PR #355). `~` must stay refused, by #283's ref-name check, which is why #314's row for it is not flipped to "accepted". - **A hand-edited plan value that is not text (7, #181).** `test_check_names_a_plan_value_that_is_not_text_by_what_was_read[tracking_branch='007']`, `[tracking_branch='TRUE']`, `[tracking_branch='1.50']`, `[tracking_branch='[a]']`, `[visibility='{a: b}']`, `[id='007']` and `[elected_on='20260902']`. The plan's reader reads each as something other than text, and on the merge `check` passes. Measured on the merge: `tracking_branch: 007` and `id: 007` are adopted silently as `"7"` with every validator passing; `visibility: {a: b}` pushes both legs and then fails `manifest-visibility`; `tracking_branch: [a]` is still refused at `execute` (`unsafe-value`) before any leg exists. On d38e143 alone, `check` exits 1 `plan-manifest-misread` for each of those four. Re-point the gate at "a plan value that is not a string" and these keep their guard. - **A quoted value the validator refuses (4, #181).** `test_check_reports_a_plan_carrying_one[elected_on='20260902']` and `[visibility='true']`, and `test_execute_refuses_a_plan_carrying_one_before_any_leg_exists[elected_on='20260902']` and `[visibility='true']`. The value stays invalid for the validator (`DATE_RE`, the visibility choices), and #181's misread gate caught it only incidentally. On the composed tree `check` passes and `execute` pushes the legs, and the gate fails afterwards. That is what main does today, and #314's validator gate (`plan-manifest-invalid`) would catch it. - **main** (merge with ab623dd, tree cdeec7f, run by the review): this branch's new file, `test_adopt_template_values.py` (#270), `test_template_values_single_pass.py` (#316) and `test_template_quoted_values.py` give 431 passed, 1 skipped. The skip is `test_adopt_template_values.py:223`, which reads `git config user.name` and skips where the shared repository's `.git/config` sets one; it is environmental. The changed tests with `test_family_add_refusals.py` (#317, #318), `test_pin_validator.py` and `test_manifest_validator.py` give 132 passed. #270's `unwritable_reason` is value-based, not template-relative, so it refuses as it did and nothing flips. Main then added #245 (`adopt-project.py` and `repo_shape.remote_key`), which adds no test that edits a bare quoted line; it was not run on that merge. - **#380** (lane 3, `fix/pin-rewrite-indentation`; it landed since as 1f90a0f). It rewrites `commit:` and `tree_sha256:` only, and its tests render the pins from the templates. On the merge at dc0fa7b, `test_pin_rewrite_layout.py` and the 6 scaffold end-to-end tests gave 121 passed. Its head was then 73068a3; that merge was clean and was not re-run. - **#397** (`fix/family-init-values`, #307; measured at 14e2d68): its tests skip themselves where the template quotes. 137 passed, 19 skipped, 0 failed. With its fix-round head abb27e6, this branch's 3894aa8, main 0f67f7b and #345 together: 442 passed, 20 skipped (the 19 rows, and one Windows-only row). - **#345** (run at 875d802; it has since landed as 3ae0109): `test_family_dry_run.py` and the member test, 22 passed. Its `" id: irrs"` assertion holds, because ordinary ids stay bare. **Left out.** - The adoption plan writer `y` still writes `true`/`null` bare into the plan. That is #181 (lane 1). - `templates/assembly-root/contracts/neutral-product-pin.yaml` still writes `product`, `source_repository` and `digest_source` bare, and `family.py` writes a member row's `project:` bare. Neither is a #259 field. - A plan value someone edits to a non-text type (`tracking_branch: 007` or `id: 007`, unquoted, in the plan) is read by the plan's reader as the number 7. `str()` makes it `"7"`, and the quoted template now writes it faithfully. For `id` this is a change: on c7c02fa a bare `id: 007` failed loudly (`validate-manifest: manifest-id`, manifest line `id: 7`), and on this branch it is adopted silently as id `"7"` and every validator passes. `tracking_branch: 007` was already adopted silently as `7`. The plan side is not this branch's: the guard against it belongs with #181 (adopt plan writes YAML keywords and non-text values unquoted), whose gate should refuse a plan value that is not a string. The case is narrow, because `plan --id 007` itself writes `id: "007"`; only a hand edit reaches it. - No docs change: README, AGENTS.md, the handbook and the template comments show none of these lines; `docs/cli.md` is unchanged because no `--help` text changed. Reviewed adversarially at 233dc71 (CHANGES NEEDED: 0 blockers, 2 should-fixes, 6 nits; the code and the tests were sound, and the should-fixes were the composition counts and the claim about a hand-edited plan `id`). Fix round 3894aa8 changes no behaviour: it corrects the description and the two false "moves by a byte" claims, and guards `_member_id`'s `except YamlError` with a test of 7 cases. Its re-verification at 3894aa8 found every finding handled: APPROVE WITH NITS (0 blockers, 0 should-fixes, 1 nit: shas that had moved since the description was written, refreshed here). What was added after that re-verification, main and the composition with #314, is described under "Composition with #314". An independent re-verification of that composition at 77cd0b2 confirmed it with nits only (0 blockers, 0 should-fixes, 5 nits): the 45 flips reproduce test for test, breaking the real rule or the real template's quoting turns the compose tests red, and the `[branch-tilde]` row XPASSes on a merge with #355. **Composition with #381.** #395's quoted `id: "{{PROJECT_ID}}"` composes with #381's `member-duplicate-id` test (`tests/test_family_add_member_id.py`), whose id rewrite now matches the id bare or quoted and keeps its quoting, so the full suite, which failed that one test on Linux at 1ef3310, no longer fails `test_a_member_whose_id_changes_between_the_probe_and_the_mount_is_unwound` on Linux. CI is green on Linux, macOS and Windows, with SonarCloud passing, at 4984651 (main cc8f2da merged in by update-branch). Main af01068 (#505), which landed after, touches none of its files, and the combined tree passed the template, hygiene, CLI-reference, AGENTS.md-codes and setup tests (807 passed). Closes #259 Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 14, 2026
brettheap
added a commit
that referenced
this pull request
Oct 14, 2026
… keyword, so `--spec-path null`, `--id null` and `--tracking-branch true` adopt as typed, and `plan`, `check` and `execute` refuse, before anything exists, a value the project's own loader would read back as something else from the line `execute` writes (#518) `adopt-project.py`'s plan writer, `y()`, now writes a string bare only where the plan's own YAML loader (`scripts/repo_shape.py`) reads it back as that same string, after `key: ` and as a `- ` item. Anything else is quoted with `double_quoted`. So `null`, `true` and their other spellings are quoted. `yes`, `on` and `.inf` stay bare, because this loader reads them as text. On `main` the plan read such a value back as no value or a boolean: - `--spec-path null` mounted the leg at `spec`; - `--tracking-branch null` put both legs on `main`; - `--id null` made the id `northwind`; - `--tracking-branch true`, `--spec-path true`, `--id true`, `--pin null@<commit>` and `--elected-by null` were refused at `check` about a value nobody typed. Each of these now adopts and bootstraps as typed. An ordinary adoption is byte for byte `main`'s, for three sets of flags. The same rule guards what `execute` writes into the project's own YAML. `plan` (`adopt-manifest-misread`, before the plan is written), `check` (`plan-manifest-misread`) and `execute` (before any leg exists) refuse a value the project's own loader would read back as something else from the line `execute` writes. The gate builds that line with the materializer's own `render`, from the template or from `_template_table`, and asks the loader. It keeps no list of words. It asks the project's own `validate-manifest.py` first. It names a value only where that validator makes no finding about it and `execute` writes it in every mode, so the two never both name one value. On `main`'s shipped templates it names nothing, because #259 and #514 quote the lines it asks about; #516 quotes a pin file's own `product:`. It stays as a backstop, as #314's scaffold twin does: a line written bare again is refused with no edit to the rule. An explicit null leg path in a plan (`spec_path: null`, `~` or nothing) is now `plan-bad-leg-path` instead of the default `spec`. The leg-path not-text sentence asks for the path meant, not "so quote it". The gate composes with #513 and #504. Its finding comes after the mount paths, before #504's `plan-root-symlink`, and #513's `plan-not-the-source` follows both. A plan with all three causes, probed with the pins' list written bare, gets one finding each at `check`. `plan` and `execute` refuse #513's cause first. #511 adds no finding: on a source that keeps its own `.gitattributes`, #181's values adopt with #511's line-ending block appended, and a plan this gate refuses is refused before that file is touched. What this leaves to `main`: - A hand-edited plan value its loader reads as a number or a boolean (`id: 007`, `tracking_branch: true` beside a source branch `True`) is written as `str` of it, as on `main`. - The scaffold's twin of the gate (#314) is a second function. Making them one is a follow-up. tests/test_adopt_plan_yaml_quoting.py adds 524 tests, all through the real loader: - the round trip of every keyword spelling, number, date, indicator and escape candidate, every string over two small alphabets, and 4000 seeded strings; - 20 values read back as given through `plan`, `check` and `execute`; - the gate's refusals, asked in this process over copies of the templates and of the pins' list written bare again. The revert table, measured at 9efe95a, names the tests each change's revert fails. Two guards fail none alone and 15 together, which the PR discloses. At 7d6e13c the targeted sets pass with and without `git-filter-repo`, the rest skipped: 1928 and 1833 tests in 14 files, 3869 and 3686 in 65 more, 963 and 953 in 11, 364 in 10, 248 and 223 in five more of `main`'s test files that changed since 89c6a6c, and 37 in #511's new one. Reviews: - Lane 2's P2 is fixed by the gate. - review181's adversarial pass found 7 NITs. - rv181f's re-verifies found F2, F2b, G1 (a BLOCKER), G2 and H1, each fixed. - rv181g found B1 (a BLOCKER, fixed in 9efe95a), and M1, the `with_origin` test fix that #513's merge needed (6c9e585). - Codex found no major issue at 9efe95a, and SonarCloud's nine new smells there are fixed. `main` was merged in thirty-one times up to 7d6e13c (`main` 5ed3e5d), never rebased, each conflict resolved by keeping both sides. CI is green on Linux, macOS and Windows at 7d6e13c, which has `main` 5ed3e5d merged in. Closes #181 Lane: openreposhape-1 (openRepoShape-1) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lane: openreposhape-5 (openRepoShape-5)
Claim: #258 (comment)
Claim: #261 (comment)
What
Both tools fill
templates/assembly-root/from values a person typed or edited into a plan, and thenew project then reads the result with two programs of its own: its YAML reader (
repo_shape) andits
scripts/validate-manifest.py. Neither tool asked either of them. Now each is asked, before awork directory or a remote exists and under
--dry-runalike, and a value the project cannot holdis refused with the value named and the reason given.
scaffold-project.py writes
--idand--tracking-branchinto project.yaml unquoted, so--id nullor--id truescaffolds a project whose first bootstrap failsmanifest-id, and--tracking-branch true,007ornullputs the clone's legs onTrue,7ormainwith no upstream: #181's defect on the scaffold side, beside #225 #258, the scaffold.id:andtracking_branch:are written bare.--id nulland--id truescaffolded three repositories whose first bootstrap failed
manifest-id;--tracking-branch true,007ornullpushed every repository on the branch as typed while bootstrap read the manifestback as
True,7ormainand put each leg on a local branch the remotes do not have, with noupstream, exit 0. Now
scaffold-manifest-misreadnames the value and what the reader makes of it.adopt-project.py and scaffold-project.py accept values the new project's own validate-manifest.py refuses (a blank
--elected-byor--reference,--elected-on 2026-W41-6or20261011on Python 3.11+,--org test+org, a plan'svisibility: secret), soadoption verifiedor a scaffold exit 0 is followed by a failed first bootstrap; andplanadmits a 251-255-character--projectthatcheckrefuses #261, both tools. A blank--elected-by,--referenceor--name,--elected-on 2026-W41-6or
20261011(read bydate.fromisoformatfrom Python 3.11),--org test+org, and a plan editedto
visibility: secretorelected_by: nullwere accepted, andadoption verifiedor a scaffoldexit 0 followed. Now the scaffold says
scaffold-manifest-invalid,adopt-project.py plansaysadopt-manifest-invalid,checkmakes aplan-manifest-invalidFINDING of each, andexecuterefuses it through
check's own list (adopt-project.pyexecuterefuses onlyplan-unresolvedandplan-stale, so a plancheckrejects as uncovered or with a bad leg creates and pushes both legs before it fails verification #168), before a leg repository exists. The sentence quotesthe validator's own finding, so the words are its words.
Names that cannot be directories. A
--projectof 247 to 250 characters died inPath.existswith
OSError: File name too longand a traceback under--local-remote-dir(scaffold from 247,adopt
execute);planadmitted 251 to 255 thatcheckrefused asunsafe-value: legs.spec.scaffold-name-too-longandadopt-name-too-longrefuse the first before anything exists, andplannow holds the three derived names tochecked_valueascheckdoes.A validator that cannot be asked. A copy of the standard whose
templates/assembly-root/scripts/validate-manifest.pyis missing, does not load, lacks what is askedof it (
DATE_RE,_findings) or raises, ismanifest-validator-unusableand not a traceback: it names thefile, says the defect is the copy's and that nothing was created. A failed load is not cached.
scripts/shape_materialize.py(the one module both tools import, in no copy list):manifest_misreadsrenders each template line that carries a value and reads it with
parse_yaml;manifest_invalidrenders
project.yamlfrom the whole substitution table and runs the validator the template ships,loaded from that file (no bytecode written into the template tree,
sys.pathput back);refuse_manifest_valuesandmanifest_value_problemscompose them;misread_placeholdersnames theplaceholders
manifest_misreadswould refuse;election_dateadmits only what the validator's ownDATE_REadmits, on every Python;refuse_unstorable_namesholds a name to 255 bytes,.gitincluded under
--local-remote-dir, and shows a name too long at both ends. A placeholder is whatPLACEHOLDER_REfinds in a line, not a second copy of its alphabet.scaffold-project.py:_refuse_unholdable_manifestafter the substitution table is built and beforethe plan is printed, the name rule before anything asks
gh api;--helpfor--id,--elected-byand
--tracking-branchsays the rule.adopt-project.py:_flag_plan,_manifest_values(the tableexecuterenders from, asked of_template_valuesand not copied),_manifest_findingsin_entry_findings,_refuse_unholdable_valuesin
cmd_plan, the name rule in_repository_urls.docs/cli.md: regenerated (four help paragraphs).tests/test_scaffold_manifest_values.py(106 tests),tests/test_adopt_manifest_values.py(79).Decision: refuse, not quote (the evidence)
The templates write
id:,elected_on:,visibility:,tracking_branch:and the leg paths bare, andthe project's reader reads
null,trueand007as None, True and 7. Either the tool refuses whatthe project would misread, or the template quotes the placeholder (#259, which Brett has not decided).
This refuses, because:
--spec-path null,true,~,2026,1.5), so it creates all three repositories and the new project's own bootstrap failsmanifest-leg-path#225 (--spec-path 2026: "refused rather than quotedin the templates"), adopt-project.py plan writes
null,trueand other YAML keywords unquoted, soplan --spec-path nullsilently mounts the leg at the defaultspec#181 (adopt-manifest-misread), scaffold-project.py: a leg path the project's own YAML reader would read back as another type (null,~,true,False,2026,1.5) is refused before anything is created, under--dry-runalike, through the one definition adopt-project.py asks, so--spec-path 2026no longer creates three repositories whose first bootstrap failsmanifest-leg-path, andadopt-project.py executeno longer saysadoption verifiedbefore that bootstrap fails (#225) #266, scaffold-project.py--referent-chainwrites each middle link into project.yaml as typed (naming_block,scripts/shape_materialize.py:1010-1011) and checks none, so a link holding#scaffolds all three repositories, exits 0, and pushes a manifest the project's own validators refuse asyaml-unparsable, and a link holding'or]is recorded as a different chain #251 (a chain link that is not a name). Ascaffold that quoted while
adopt-project.py planrefuses--tracking-branch 007would disagree withit about whether
007is a branch.project.yamland leg pins for every tool that rendersthem (the scaffold and adopt
execute), and templates/assembly-root/project.yaml and the two leg pins write eight text placeholders bare (id,elected_on,visibility,tracking_branch, the two legpaths andsubmodule_paths), so a checked value YAML reads as another type (007,2026,true,null) is misread by the project or refused by the tool; quoting them reads every one back exactly and lets numeric branch names adopt #259 asks for that to be decided once. (family.py initrenders
templates/family-root/, not these lines; it has the same defects in its own file and is family.py init exits 0 and pushes a holder its own validate-family.py refuses for--id null,--created-by ' ',--created-on 20261011or--name 'Contoso Group', and puts--tracking-branch trueon a branch its manifest reads back as True #307.)a placeholder is quoted those values read back and nothing here refuses them, with no edit.
test_quoting_the_template_would_make_the_refusal_go_awayholds that.it is refused, with the reading named.
A behaviour change for input that used to work.
--tracking-branch 2026and1.5worked by accident(
str(2026)gives2026back, and every reader oftracking_branchdoesstr():bootstrap.py,update-shape.py,bump-leg.py); they are refused asadopt-project.py planrefuses them on #181. #259'squoting would allow them again. The refusal's remediation says so in the true terms: it says only what holds
of every value refused (the project's reader reads back something other than what was typed), names the
failures that do hold (an id that is not text fails
validate-manifest.py;007as a branch reads back as7), says that2026or1.5as a branch work today only because the tools that read it turn the numberback into text and are still numbers to the project's reader, and gives the way forward, a value the reader
takes for text (
atlas,release-2026,v2). It no longer says the first bootstrap would fail, which isfalse of exactly those two values.
Reusing the validator's rule, not copying it
The rule is the validator's own:
manifest_validator()loadstemplates/assembly-root/scripts/validate-manifest.pyfrom the template it ships, and
election_dateasks itsDATE_RE. The alternative, moving the predicatesinto
scripts/repo_shape.py, is a shape copy: every project in the estate would meet anupstream-changedrow on its next
update-shape.py checkfor a question no project asks of itself (the argumentshape_materializealready makes for the leg-path rule and the commit trailers). Loading the file leaves thevalidator the one place the rule is written, and any rule it learns is asked here with no edit
(
test_the_election_date_asks_the_validator_not_a_copy_of_its_rule).Evidence
Reproduction scripts: the issues' own loops (
repro258.sh,repro261.sh), local remotes only.Each row is in the logs named in the report; every row of the two issues, and the extra rows lane 2 added
to #258 (
--id 2026and007,--project NullandTrue,--tracking-branch TRUE,Nulland1.50), is atest that fails on c3f4987 (130 of the 170 new tests fail there: the rest assert what must
still work, such as
release-1andv1.2.3, or hold the API).Targeted runs on this head, all with the venv
git-filter-repo2.47.0 on PATH:SHAPE_MEDXEHR_CLONEunset)tests/test_shape_doctor.py, its own streamMutation probes: each hunk reverted in a throwaway worktree, at least one new test fails for each of the
eighteen of the first round and the ten of the second (the date rule, the scaffold gate, the misread scan, the
validator question, the bytecode and
sys.pathguards, both name rules, adoptplan, the one listcheckandexecuteread, the grouping by value, the attribution, the leg-path and non-text filters, the list-legsguard,the no-revision guard, the unclassifiable-names guard; and the remediation wording,
misreads_elsewhere, theplaceholder pattern,
_line_shown, the name tails and the three validator guards).Reach: every tracked YAML file (14) loaded with
repo_shape.load_yamlon c3f4987 and on this head, parsedresults identical (nothing here touches the reader; 7 are unrendered templates the reader refuses on both).
Writers of the constructs:
scaffold-project.pyandadopt-project.py execute(assembly root); the familyholder's writer is #307's.
Nothing ran on Windows or macOS before this PR's CI: the new tests pass bytes and never
text=Trueto git, writeplans with
newline="\n", and the ones that copy the standard (a_copy_of_the_standard) copy a tree the waytests/test_family.pydoes.Composition (
git merge-tree --write-tree, at head 7defac7, fetched just before)_entry_findings, resolved by keeping both (findings + value_findings + submodule_findingsandunsafe_remediation + value_remediation + remediation)test_template_values_single_pass.py,test_template_quoted_values.pytest_adopt_root_shape_paths.pytest_leg_path_yaml_scalars.py,test_adopt_leg_paths.pyadopt-project.py_entry_findings: its docstring and its last statementmisreads_elsewhere=True(below): 1050 passed, 1 skipped: the two new files,test_adopt_plan_yaml_quoting.py,test_adopt_leg_paths.py,test_adopt_plan.py,test_adopt_execute_refusals.py,test_cli_reference.pyWith #181, and who does what. Both branches add a findings group to
_entry_findings. Keeping both,findings + misread + value_findings + submodule_findings, gives two findings for a plan value that is TEXT and thatthe template writes bare, which the reader reads back as a number or a boolean (
elected_on: "20261011",visibility: "true",id: "007"): #181'splan-manifest-misreadand this branch'splan-manifest-invalid, thesecond saying the retyped
id is 7._manifest_findings(plan, misreads_elsewhere=True)leaves such a value to themisread rule, by the placeholders
shape_materialize.manifest_misreadsrefuses, so it is judged once. It is Falsehere because with nothing else asking the misread question the validator's finding is the only one: always
leaving such a value out makes this branch alone accept
id: "007"atcheck, which it refuses, and 13 testsof this branch fail with it on. Whoever lands second passes
misreads_elsewhere=Trueat the one call in_entry_findings; nothing is lost in either order, andtest_a_text_value_the_reader_misreads_is_one_finding_in_the_list_check_printsfails for each of the three values ona merge that has not (two findings), and passes on one that has. Run on a throwaway merge with #181's e337f44:
the three pairs are three single findings, and the other 16 rows of the 19-row plan and
checkdriver are unchanged. Re-pointing#181's own gate at
shape_materialize.manifest_misreads(one definition for both tools, which #258 asks) isthe same person's job and is not done here: lane 1's branch is not this PR's.
#266fires first at the scaffold (its leg-path rule, before the manifest gate), at adoptplanand atcheck, sothe two messages about a leg path never disagree once it lands.
#260makes a value holding a{{KEY}}acceptable as typed (see the first left-out item).
Left out, and why
adopt-project.py's misread side is adopt-project.py plan writesnull,trueand other YAML keywords unquoted, soplan --spec-path nullsilently mounts the leg at the defaultspec#181's:--tracking-branch 007atplan, a plan edited to a non-textvalue (
id: 007read as the number 7), and a value carrying a line break (elected_on: "2026-09-02\nfoo: bar").This branch already refuses
--id trueand--id 007atplan, andid: "007",visibility: "true"and
elected_on: "20261011"atcheckandexecute, through the validator'smanifest-id,manifest-visibilityandmanifest-elected-on.--spec-path 2026): scaffold-project.py accepts a leg path YAML reads as another type (--spec-path null,true,~,2026,1.5), so it creates all three repositories and the new project's own bootstrap failsmanifest-leg-path#225's, now scaffold-project.py: a leg path the project's own YAML reader would read back as another type (null,~,true,False,2026,1.5) is refused before anything is created, under--dry-runalike, through the one definition adopt-project.py asks, so--spec-path 2026no longer creates three repositories whose first bootstrap failsmanifest-leg-path, andadopt-project.py executeno longer saysadoption verifiedbefore that bootstrap fails (#225) #266.planrefuses one through the validator'smanifest-leg-path(it passes no
ignore), andcheckignores that code because it reports a leg path in the leg-path rule's own words(
ignore=LEG_PATH_CODES), so a hand-editedspec_path: "2026"still checksplan okon this branch as it did onmain. scaffold-project.py: a leg path the project's own YAML reader would read back as another type (
null,~,true,False,2026,1.5) is refused before anything is created, under--dry-runalike, through the one definition adopt-project.py asks, so--spec-path 2026no longer creates three repositories whose first bootstrap failsmanifest-leg-path, andadopt-project.py executeno longer saysadoption verifiedbefore that bootstrap fails (#225) #266 judges it first everywhere.scripts/family.py init: the same defects in the holder; verified here (--created-by ' ',--created-on 20261011and--id nullexit 0 and the holder'svalidate-family.pyis red;--tracking-branch trueis silent). family.py init exits 0 and pushes a holder its own validate-family.py refuses for--id null,--created-by ' ',--created-on 20261011or--name 'Contoso Group', and puts--tracking-branch trueon a branch its manifest reads back as True #307.--pin null@<sha>and a--pinproduct the reader misreads (the scaffold writesneutral_product_pinsand a pinfile product bare): not asked; adopt-project.py plan writes
null,trueand other YAML keywords unquoted, soplan --spec-path nullsilently mounts the leg at the defaultspec#181 does adopt's.--org: the scaffold'sgh repo createstill fails for a name longer than GitHub's 100 characters, at the firstleg, before anything exists; that is GitHub's limit and not the validator's.
--pinwithout--pin-sourceis reached after_declarationhas read the forge withgh api(a read); the leg-path and name rules come before it (the review's NIT-8).Remaining nits of the adversarial review that are not fixed here, one line each:
planandcheckdisagree about--spec-path 2026on this branch alone (see the second item above), andthe scaffold's sentence names no flag for
--spec-path true; scaffold-project.py: a leg path the project's own YAML reader would read back as another type (null,~,true,False,2026,1.5) is refused before anything is created, under--dry-runalike, through the one definition adopt-project.py asks, so--spec-path 2026no longer creates three repositories whose first bootstrap failsmanifest-leg-path, andadopt-project.py executeno longer saysadoption verifiedbefore that bootstrap fails (#225) #266's rule fires first everywhere once it lands.checkexits 2 with one REFUSED and no findings for a badorg:(unsafe-value) or an unparseableelected_on:in a plan with noreference:(election-date-malformed, naming--elected-onabout a plan key);both are better verdicts than main's
plan ok, but refusals and not FINDINGs.--elected-by '{{SPEC_PATH}}'is recorded aselected_by: "spec") or, after the legs are pushed, refused astemplate-unsubstitutedwith "a defect in the substitution table of the tool you ran" #260 lands, a value that holds a{{KEY}}is refused astemplate-unsubstituted(a defect of the tool),under
--dry-runtoo; main accepted the dry run and wrote the wrong text or died after the push. With adopt-project.py and scaffold-project.py substitute the template placeholders one after another, so a free-text value holding another placeholder's token is expanded (--elected-by '{{SPEC_PATH}}'is recorded aselected_by: "spec") or, after the legs are pushed, refused astemplate-unsubstitutedwith "a defect in the substitution table of the tool you ran" #260 both areaccepted as typed.
--pinand no--pin-source, a misread--id nullis reached only after the forge read (above)._reading_of'slen(parsed) == 1andkey in valuesin
_sentences_of.Review
An adversarial review of ca21b57 approved with nits. It had three should-fixes, none a code defect:
--tracking-branch 2026and1.5._entry_findings, and a stale body.null,trueand other YAML keywords unquoted, soplan --spec-path nullsilently mounts the leg at the defaultspec#181 lands.Fix round 1 (7defac7) handles all three and four nits. Its re-verification found no blocker or should-fix. It confirmed that taking the review's #181 suggestion unconditionally would drop this branch's only finding for
id: "007",visibility: "true"andelected_on: "20261011". Themisreads_elsewhereflag is therefore the handover: whichever of this PR and #181 lands second passesTrueat the one call in_entry_findings, and the guard test fails if it does not. A seeded run of 1324 edited plans on the flagged merge with #181 left no value with no finding and none with two.Two new nits are left:
_findingsreturns something other than a list still ends in a traceback._entry_findingshas no comment pointing at the handover.Fix round 2 (SonarCloud)
Tests on all three OSes passed at 7defac7, but SonarCloud's quality gate failed on its security rating for new code:
pythonsecurity:S8705atshape_materialize.run(). That is the agentic argument-injection family #119 marked a false positive. The only new call intorun()wasgit rev-parse HEADin this tool's own checkout, with constant arguments. Fix round 2 (cf3e8d3) removes it, and a second git call with it. SonarCloud still flags the same S8705 on cf3e8d3, so that call was not its cause. The finding sits wholly insiderun(), and PR #316 carries the same one with no new caller ofrun(). It stays open for a disposition (below).planandcheckask the project's own validator about the manifest with a fixed, well-formed placeholder commit and tree digest. The question starts no process; the commands still read the source withgit. The validator holds both to their form and compares them with nothing._template_valuesalso rantree_digest(SHAPE_ROOT, commit)(git ls-tree -rover the whole standard) on everyplanandcheck. The table body moved to_template_table, which runs nothing.executestill builds its table with the real commit and digest, which a new test asserts.--elected-byfrom a tree with no.gitexactly as in a checkout, which fixes the review's NIT-9. There are 16 new tests: 14 fail on the previousadopt-project.py.execute's 29-key table and its 50 pushed files are identical to 7defac7's. An audit hook saw 0 process starts in the manifest question. 14 of the 16 tests fail on 7defac7, and all nine mutants of the round are caught.Fix round 3 (nits)
Fix round 3 (ab66bc0) changes no behaviour:
_template_values.python:S9073).MANIFEST_VALIDATOR,python:S1192).Targeted runs: 289, 102 and 176 passed, and the suite collects (3499).
Closes #258
Closes #261
🤖 Generated with Claude Code
Summary by Sourcery
Refuse manifest values and repository names that the generated project or filesystem cannot safely hold before scaffolding or adoption creates anything.
New Features:
Bug Fixes:
Enhancements:
Documentation:
Tests: