Skip to content

shape-doctor.py: fifteen NEXT builders end the command with # <comment>, which a stock interactive zsh (macOS's default shell) does not read as a comment, so pasting the line hands the comment's words to the command as arguments, runs whatever follows a ;, and runs the make bootstrap the legs and members comments put in backticks, then exits 127 #277

Closed

Description

@brettheap

Lane: openreposhape-4 (openRepoShape-4)

Found by this lane's adversarial review of the #238 fix branch (head 59724c9; its out-of-scope item O2, and, for the last paragraph of "What it should do", its nit N2). The review reproduced the zsh half in a pty; this issue re-verified it on main eb6df75 (d89fffa plus #220; the NEXT builders listed below were counted on eb6df75) with a fresh pty run, and found by the same run that the tail is not only ignored but executed in part. It is pre-existing and not caused by any open branch. Not claimed.

What is wrong. The doctor's NEXT lines are built as <command> # <comment> and printed on one line (shape-doctor.py:3835-3840, and as next in --json). # starts a comment in sh -c, in bash (interactive too) and in zsh -f -i -c (all three checked), which is how every test and every agent tool call reads it, but in an INTERACTIVE zsh without setopt interactivecomments (zsh's default, macOS's login shell, the one a person pastes into) and, by the shells' documentation (not run here), in interactive csh/tcsh it is an ordinary character. The pasted line then runs the text after it as shell code:

# $C: a scaffolded root with `topic: xf-project-wrong` in project.yaml. $PTY: python's pty module driving the shell, as a person pastes.
python3 $U/shape-doctor.py --root $C --json     # the manifest row's next:
#   python3 $C/scripts/validate-manifest.py --root $C   # read its output in full; it names what it refused
sh -c "$NEXT"                                   # exit 1, FINDING manifest-topic: topic is 'xf-project-wrong', ...
$PTY bash --norc --noprofile -i                 # the same finding, pasted-exit=1
$PTY zsh -f -i                                  # zsh 5.9; `[[ -o interactivecomments ]]` is off here
#   validate-manifest.py: error: unrecognized arguments: # read its output in full
#   zsh: command not found: it
#   pasted-exit=127

The argument error is the first command; ; ends it, it names what it refused is a second command that does not exist, and the status the person sees is that one's (127), not the validator's. A backtick pair in the tail is command substitution, and two builders put one there: legs (:1400) and members (:2957) both say "what make bootstrap runs". Pasting the legs NEXT into zsh -f -i, in a directory holding a Makefile whose bootstrap target creates a file (the root's own bootstrap script was absent, so the python half failed with exit 2):

# $MK: a scratch directory whose Makefile has a `bootstrap` target that runs `touch MARKER`.
PASTE_CWD=$MK $PTY zsh -f -i            ->  ls $MK  ->  MARKER  Makefile      # `make bootstrap` ran, in the current directory
PASTE_CWD=$MK $PTY bash --norc --noprofile -i   ->  no MARKER
cd $MK && sh -c "$NEXT"                  ->  no MARKER

So a pasted legs or members NEXT can run make bootstrap in whatever directory the person is standing in (and, standing in the root itself, the script the line was already running, a second time). Putting the comment on a line of its own does not help: the same text pasted as its own line into zsh -f -i prints zsh: command not found: # and still runs the backticks (MARKER was created again).

Which builders. Fifteen sites in shape-doctor.py end a NEXT with a # tail (:870, :887, :1247, :1313, :1328, :1400, :1596, :2053, :2259, :2577, :2586, :2591, :2957, :3276, :3473); the naming row's and a few others carry none. What an interactive zsh makes of the tails' own text, read off the source: a ; that starts a second command at :870, :887, :1247, :2053, :2577, :2591 and :3473; backticks (command substitution) at :1400, :2957, :2577 (`files:` and a whole validate-pins.py command) and :2586 (`apply --add AGENTS-shape.md`); an apostrophe at :2577 ("repository's") and :2586 ("human's"), which leaves the shell at a continuation prompt until the quote is closed (tried with that tail text on an echo in zsh -f -i: the pasted line never completed and the shell had to be killed); parentheses at :1596, which zsh tries as a glob (the same kind of stand-in line: zsh: no matches found: (placeholders):, nothing ran). The tests read the tail the way sh does: shlex.split(line, comments=True) (tests/test_shape_doctor.py:1355) and powershell_tokens (:1320) drop it, RENDERED_NOTE (:1485) and :4168 assert its text, so nothing in the suite can see the difference.

Why. The comment is documentation of the command and was put on the command's line because a terminal shows both at once. AGENTS.md tells an agent to "run the one the row names" and does not say which shell (AGENTS.md:208-214), and the doctor's own history (the comment at shape-doctor.py:880, "A COMMAND THAT RUNS WHEN IT IS PASTED", and #96, #100, #102) states the principle this breaks: the line is a command, not prose wearing a command's clothes.

What it should do. A NEXT is the command and only the command. The decisions are the author's:

  1. Where the sentence goes. Into the row's own text (reason) or a separate field of --json (for instance why), printed on a line below the NEXT that is not itself a command line, so the NEXT can be pasted whole and the explanation is still read. The comment-on-its-own-line variant is NOT a remedy (shown above: # is still a word, and backticks still run); a tail is safe only if it holds no shell metacharacter at all, which nothing guards.
  2. Whether --json's next stays the bare command (it should) and what the text report prints under it.
  3. The tests: next_argv-style readers (comments=True, powershell_tokens) can stay for the quoting tests but a test must paste the NEXT into a shell that does not read # as a comment, so the next regression is seen. zsh -f -i in a pty is the reproduction used here; a runner with no zsh skips it, and the same property can be asserted without a shell: the printed line contains no # outside a quoted value and no backtick.
  4. The related portability point (shape-doctor.py validator_row falls back to the standard's TEMPLATE validator when a root carries no copy of its own, and prints that path as the NEXT command without the PYTHONPATH it runs it under, so pasting the manifest, pins or family row's NEXT dies with ModuleNotFoundError: No module named 'repo_shape' and exit 1, the code a validator uses for a finding #238 review N2): on main no NEXT carries a NAME=value prefix. shape-doctor.py validator_row falls back to the standard's TEMPLATE validator when a root carries no copy of its own, and prints that path as the NEXT command without the PYTHONPATH it runs it under, so pasting the manifest, pins or family row's NEXT dies with ModuleNotFoundError: No module named 'repo_shape' and exit 1, the code a validator uses for a finding #238's fix (origin/fix/doctor-template-validator-next, head 88c81e0, not on main, no PR yet, waiting for shape-doctor.py: a project.yaml or family.yaml that is there and does not load is INVALID (manifest) or INVALID (family), exit 1, quoting load_yaml's own refusal and naming the project's own validator, instead of NOT A SHAPE ROOT, exit 2, with an adopt-project.py plan NEXT and a Triad row calling an assembly root a single repository (#205) #232) prints PYTHONPATH=<q> python3 <template> ... for the standard's template copy (validator_command, shape-doctor.py:1064 on that head). That spelling works in sh, bash and zsh (checked), but by documentation fails in csh/tcsh (PYTHONPATH=...: Command not found.) and in fish before 3.1 (not verified: none of the three is installed here); env PYTHONPATH=<q> python3 <template> ... works in all of them for four more characters (checked in sh, bash and zsh). Decide it with this issue so the first NAME=value NEXT is spelled once.

Not checked: macOS's own /bin/zsh (the default of interactivecomments is the same in zsh's documentation; the run above is zsh 5.9 on Linux), csh/tcsh/fish (not installed), PowerShell (where # is a comment, so the nt lines are unaffected), and the NEXT lines other tools print (update-shape.py, family.py, adopt-project.py and AGENTS.md's own code blocks also carry # ... tails; not audited here).

How to test. A test that takes every next from a doctor --json run over fixtures that make each of those rows red (manifest, pins, legs, shape currency, placement, machine), asserts the line has no shell comment and no backtick or ; outside a quoted value, and, where zsh is on PATH, pastes it into zsh -f -i in a pty (pty.openpty plus subprocess.Popen, TERM=dumb) and asserts the command's own exit code and output. Before the change the manifest case pastes unrecognized arguments: # and exits 127; the legs case creates the file.

Related: #238 (its fix round and review: N2 and O2), #96, #100, #102 (the quoting rules these lines already follow), and #276 (the legs and members rows' bootstrap.py NEXT), whose lines are two of the fifteen.

Activity

  1. brettheap commented on Oct 12, 2026

    @brettheap
    ContributorAuthor

    CLAIMED — lane openRepoShape-4, session 3431c370-df80-4c11-a041-41bce42d8b38@Eagle, 2026-10-12T11:06:59Z, for #277

    Logged in lanes/log/openRepoShape-4.md at 1f6b081f05859f83bf7fa7669845909e6af291e6.

    The three reads (lane-collision-protocol Rule 1):

    1. existing `CLAIMED —` comments on opensoft/openRepoShape#277:
       none
    2. `gh pr list --repo opensoft/openRepoShape --state all --search "#277"`, kept where the row or its body names `#277`:
    267	shape-doctor.py: a root's own `contracts/shape-pin.yaml` that does not read as a pin (a merge conflict, a non-UTF-8 byte, a `commit:` that is not 40 hex, a deleted file) is `INVALID`, exit 1, with the git command that puts back the last pin a tool wrote as its NEXT, instead of `CANNOT ANSWER`, exit 3, sending the reader to `git fetch --all` in the standard (#237)	fix/doctor-root-shape-pin-unreadable	OPEN
    3. `git ls-remote --heads git@github.com:opensoft/openRepoShape` branches naming `277` as a whole token:
       none
    
  2. added 5 commits that reference this issue on Oct 12, 2026
  3. added this to the v1.0.0 milestone on Oct 12, 2026
  4. added 10 commits that reference this issue on Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions