Skip to content

scripts/shape_advisory.py repository_names and shape-doctor.py origin_url and origin_name read git remote get-url origin through git_out and catch Refusal and OSError only, so an origin URL that is not UTF-8 makes read answer not-answered (a single repository loses the Triad advisory, a leg clone its pointer to the assembly root) and ends shape-doctor.py --root in a UnicodeDecodeError traceback, exit 1, where an ASCII origin gives NOT A SHAPE ROOT, exit 2 #385

Closed

Description

@brettheap

(lane openRepoShape-2, 2026-10-12T14:57Z)

Found by this lane's filer while measuring #292's fix (PR #353), whose "Left out" section says that shape_advisory.repository_names and shape-doctor.py read the same origin URL and are a follow-up. Reproduced on origin/main ab623dd, and on #353's head dcbc37b (ab623dd with #353 merged), which changes adopt-project.py and one test file only, so scripts/shape_advisory.py and shape-doctor.py are main's. Nothing is created or changed in any run below. Not claimed.

What is wrong. Three functions ask git for origin's URL through repo_shape.git_out and catch (Refusal, OSError) only. git_out returns proc.stdout.decode("utf-8").strip() (scripts/repo_shape.py:843), and a URL that is not UTF-8 raises UnicodeDecodeError, a ValueError, past all three:

  • repository_names (scripts/shape_advisory.py:430-446; the read :440, the handler :441), which _read calls (:718) before the leg and workspace routes;
  • origin_name (shape-doctor.py:3485; the read :3538), reached from repo_local_origin_name (:3556, :3570), which the naming row (check_not_a_root_naming, :3587) and the way-in row (check_the_way_in, :3877) both call;
  • origin_url (shape-doctor.py:3068; the read :3094), reached from mounted_from (:3157) and working_clone (:3263), which member_state (:3271) calls for each row of a family holder's members row.

What each caller then does is different, and the first is silent on purpose. read() (:670) catches every Exception (:695) and answers not-answered (NOT_ANSWERED, :139), which says nothing: the ratified rule is that the advisory changes no exit status, so what it "did not expect" ends there, silent. A URL that is not UTF-8 is not unexpected. The same function already answers a repository with no origin by naming nothing, and it never needed the URL for the leg route that reads the repository's own AGENTS.md (_leg_by_agents_md, :477), which runs after names is read and so never runs. The doctor has no such net: shape-doctor.py --root ends in a traceback, exit 1.

So a repository whose origin holds a byte that is not UTF-8 (a clone of a local path under a directory named in a legacy code page, a mount or a share; nobody has done anything wrong, and git stores the bytes untouched):

# $U: a checkout of openRepoShape at ab623dd. Every repository is local; no network.
mkdir -p "$R/single/src" && git -C "$R/single" init -q -b main
echo 'print(1)' > "$R/single/src/app.py"; echo '# r' > "$R/single/README.md"
git -C "$R/single" add -A && git -C "$R/single" commit -qm init
cp -a "$R/single" "$R/ascii"; git -C "$R/ascii" remote add origin "$R/cafe/repo.git"
cp -a "$R/single" "$R/latin"; git -C "$R/latin" remote add origin "$R/caf"$'\xe9'"/repo.git"     # a path holding the byte 0xe9
python3 "$U/scaffold-project.py" --org testorg --project Northwind --elected-by 'T H' --elected-on 2026-09-02 \
        --local-remote-dir "$R/remotes" --work-dir "$R/work"
git clone -q "$R/remotes/Northwind-spec.git" "$R/leg"                  # a real spec leg: its AGENTS.md says so
git -C "$R/leg" remote set-url origin "$R/caf"$'\xe9'"/Northwind-spec.git"
advisory() { python3 -I -c 'import sys; sys.path.insert(0, sys.argv[1] + "/scripts"); import shape_advisory as sa; r = sa.read(sys.argv[2]); print(r.state, "|", r.why)' "$U" "$1"; }
advisory "$R/ascii"; advisory "$R/latin"; advisory "$R/leg"
#   advise | a repository that has not elected the Triad and has recorded nothing
#   not-answered | the advisory could not be read here (UnicodeDecodeError), so it says nothing
#   not-answered | the advisory could not be read here (UnicodeDecodeError), so it says nothing
python3 "$U/shape-doctor.py" --root "$R/latin"
#   exit 1; Traceback: shape-doctor.py line 4368, in main / 4054, in run_checks / 3587, in check_not_a_root_naming /
#   3570, in repo_local_origin_name / 3538, in origin_name / repo_shape.py line 843, in git_out
#   UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 111: invalid continuation byte
the repository the run result on ab623dd
single repository, ASCII origin (control) shape_advisory.read advise
the same, origin holds \xe9 read not-answered; repository_names raises UnicodeDecodeError (0xe9 in position 111)
the same shape-doctor.py --root exit 1, the traceback above
single repository, ASCII origin (control) shape-doctor.py --root exit 2, advisory Triad ... and NOT A SHAPE ROOT (exit 2)
the same, origin holds \xe9 adopt-project.py plan --org testorg main: exit 1, UnicodeDecodeError from _remote_repository :836 (#292), before the advisory is asked. dcbc37b (#353): exit 0, no triad line (ASCII control: one triad line)
a directory named Northwind-spec, ASCII origin (control) read leg-clone, by name
the same, origin holds \xe9 read not-answered
a scaffolded spec leg cloned, ASCII origin (control) read; shape-doctor.py --root leg-clone by its AGENTS.md; the Triad row says it is a clone of the spec leg of a Triad, exit 2
the same clone, origin set to a path holding \xe9 read; shape-doctor.py --root not-answered; exit 1, the traceback above
any repository, origin holds \xe9 origin_url, origin_name, repo_local_origin_name called directly each raises UnicodeDecodeError

Why. One decode, three handlers that name Refusal and OSError and not the ValueError a strict decode raises. The advisory's own net hides it in read (by design, for what nobody foresaw) and turns the foreseeable into the same silence as the unforeseeable; the doctor has no net. #292 and PR #353 fix the same decode for adopt-project.py's _remote_repository and _elector by catching it at the call sites in that file, and leave these for a follow-up.

What it should do.

  1. repository_names treats an origin it cannot decode as unnamed, returning the directory's name alone as it does for a missing origin (url = ""), which is the answer adopt-project.py: _elector keeps a user.name byte that is not UTF-8 for #255's question, _remote_repository reads such an origin URL as no org/repo, and plan asks adopt-no-org before checked_value, so a Latin-1 name is refused as adopt-unwritable-value naming --elected-by, and plan, check and execute go on past such an origin, where both ended in a UnicodeDecodeError traceback (#292) #353 gives _remote_repository. read then answers as it does for any repository: advise for a single one, leg-clone through the leg's own AGENTS.md (the first route, which needs no URL) or the directory's name, workspace, recorded-single. Whether the URL's last segment may still be used when only an earlier directory is not UTF-8 (the URL in the rows above ends in Northwind-spec.git) is the author's choice; a name that itself holds such a byte must not reach classify or be printed.
  2. origin_url and origin_name in shape-doctor.py catch the same error. origin_url returns "", which its own docstring already defines as "a directory this report cannot identify", and origin_name returns None, so the naming, way-in and members rows classify on what they have and shape-doctor.py --root ends the verdict it ends for the same repository with no origin (exit 2, NOT A SHAPE ROOT, for a single repository).
  3. No change to git_out. scripts/repo_shape.py is a pinned copy (COPIED_FROM_SHAPE): a change there moves the sha256 every scaffolded and adopted project's contracts/shape-pin.yaml records for it. The three call sites are in scripts/shape_advisory.py and shape-doctor.py, neither of which a project pins; catch UnicodeDecodeError beside Refusal there, or call with binary=True and decode there.
  4. read keeps its net. The ratified rule stands; what changes is that this input no longer reaches it.

How to test. In tests/test_shape_advisory.py, beside test_a_workspace_path_no_machine_can_hold_fails_nothing (:547, the precedent: an input no machine can hold costs exactly its own entry and every other row keeps its exit), with its helpers (repository :81, quiet_env :74, doctor_json :98, adopt_plan :141, rendered_leg_agents :118, no_traceback :130, and ROW = "triad-advisory"). Write the URL into the repository's .git/config as bytes (config.write_bytes(config.read_bytes() + b'[remote "origin"]\n\turl = ' + url + b"\n"), as with_origin does in #353's tests/test_adopt_git_reads.py), so the byte reaches git as it is on every platform, and skip where git remote get-url origin does not give it back as written. Use b"../caf\xe9/repo.git".

Not in this issue. adopt-project.py's _remote_repository and _elector (#292, PR #353). setup-project.py's remote_url (:1184) reads the same URL through its own run, which decodes with errors="replace": run here, it returns the URL with U+FFFD and does not raise. templates/family-root/scripts/siblings.py asks for origin through git_text (:208, :358); it is a pinned copy and was not run. Whether other git_out callers need the same guard is a separate question, as #292 says.

Related: #292 and PR #353 (the same decode for adopt-project.py; its "Left out" names this); #163 and PR #164 (the Triad advisory, its ratified silences and NOT_ANSWERED); #255 and PR #270 (a non-UTF-8 byte arriving from argv, refused as adopt-unwritable-value; this one arrives from git).

Lane: openreposhape-2 (openRepoShape-2)

Activity

  1. brettheap commented on Oct 12, 2026

    @brettheap
    ContributorAuthor

    CLAIMED — lane openRepoShape-2, session 0279452a-ea8e-486f-bf99-698cb0a9bb2f@Eagle, 2026-10-12T15:32:30Z, for #385

    Logged in lanes/log/openRepoShape-2.md at 637f671fa4ee0077d638784e0eb8472f146e8e65.

    The three reads (lane-collision-protocol Rule 1):

    1. existing `CLAIMED —` comments on opensoft/openRepoShape#385:
       none
    2. `gh pr list --repo opensoft/openRepoShape --state all --search "#385"`, kept where the row or its body names `#385`:
       none
    3. `git ls-remote --heads git@github.com:opensoft/openRepoShape` branches naming `385` as a whole token:
       none
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions