bootstrap.py and repo_shape.py: a leg on a branch whose name is not UTF-8 makes every later make bootstrap end in a UnicodeDecodeError, and a character the filesystem encoding lacks still ends step (a) in a traceback #710

repo_shape.git_out should hand a branch name that is not UTF-8 back to its callers, so that a leg on such a branch does not end every later make bootstrap in a traceback. The riders about names step (a) carries or cannot write ride with it.

From lane 4's checklist triage of 2026-10-16 (T2), approved by Brett Heap at 2026-10-16T13:36:07Z ("approve all triage recommendations"). One pull request covers the lines below, and it is the SAME pull request as #709, so that templates/assembly-root/scripts/bootstrap.py is re-pinned once and not twice.

Parent: #459. Milestone: v1.1.1 (a follow-up filed after Brett Heap's v1.1.1 word of 2026-10-16T13:35:28Z).

Authored after the v1.1.0 work; lands after the v1.1.0 tag unless the release owner (lane 3) says otherwise.

Lands after:

• lane 5's bootstrap.py (the family root's and the assembly root's): make bootstrap with no git on PATH ends in a FileNotFoundError traceback, exit 1, where the validators refuse git-not-runnable, exit 2 #694 (filed, no pull request yet). It edits the same _try_git and git_out sites in templates/assembly-root/scripts/bootstrap.py that L4-125 goes through (recorded_gitlink, _try_git, the git run), to catch an OSError where git is not on PATH. Whichever lands second re-gates against the other.
• lane 2's Pinned shape copies: one bump, so AGENTS-shape.md is true of every root, the scaffold's chain WARNING offers only its own sources and spells --pin-source, the root's not-used note spells it too, and THE LOCKSTEP RULE's bump runs (#610) #634 (ready; it edits scripts/repo_shape.py, a pinned copy).
• bootstrap.py (the assembly root's): a leg moved off a branch has an ignored file the pin tracks overwritten and a merge in progress ended, with no word, and a tracking branch ahead of the pin with no commit of its own is left where it is #709, in the same pull request.
• lane 1's adopt-project.py _default_branch reads the source's branch name through git_out, which decodes strictly, so a source whose HEAD (or origin/HEAD) names a branch that is not UTF-8 ends plan, check and execute in a UnicodeDecodeError traceback, exit 1, from Source.open, with no plan written and no refusal naming the branch #425 (v1.1.0, claimed): adopt-project.py's _default_branch reads the source's branch name through the same strict git_out, so a source whose HEAD names a branch that is not UTF-8 ends plan in the same UnicodeDecodeError. adopt-project.py _default_branch reads the source's branch name through git_out, which decodes strictly, so a source whose HEAD (or origin/HEAD) names a branch that is not UTF-8 ends plan, check and execute in a UnicodeDecodeError traceback, exit 1, from Source.open, with no plan written and no refusal naming the branch #425 asks that git_out NOT change, because it is pinned. This issue does change it, so one of the two pull requests decides the decode rule and the other follows; if this one lands first, adopt-project.py _default_branch reads the source's branch name through git_out, which decodes strictly, so a source whose HEAD (or origin/HEAD) names a branch that is not UTF-8 ends plan, check and execute in a UnicodeDecodeError traceback, exit 1, from Source.open, with no plan written and no refusal naming the branch #425's traceback goes with it.

Effort: S: one decode rule in git_out. L4-124 waits for its ruling. Platform: all (L4-124 was measured under LC_ALL=C PYTHONUTF8=0, because the review host has no Latin-1 locale).

Files: scripts/repo_shape.py (git_out, :1048 on c6122e8) and templates/assembly-root/scripts/bootstrap.py; both are pinned shape copies, re-pinned once, in the same pull request as #709.

Lane openRepoShape-4 claims it at filing; see its CLAIMED comment.

Why this is filed. Not an ordinary repository: the branch name has to be one that is not UTF-8, which bootstrap.py --branch $'a\xffb' (which #644 deliberately keeps), a manifest's "\udcXX" escape or a hand edit makes. But once it exists, EVERY later make bootstrap in that root ends in a traceback until the branch is renamed by hand, with nothing naming it. Line numbers are at c6122e8 (#639); main has moved since.

Lines

• L4-125 (P3): a leg on a branch whose name is not UTF-8 makes EVERY later bootstrap.py end in a traceback. The decode is repo_shape.git_out's strict proc.stdout.decode("utf-8") (scripts/repo_shape.py:1048 on c6122e8; :923 when measured), reached via place_at_pin, _head_of (bootstrap.py:211) and _try_git (:120). Such a branch can be created by bootstrap.py --branch $'a\xffb', which bootstrap.py and validate-manifest.py: a branch holding a NUL or a lone surrogate is a finding and step (a) does not hand it to git, so tracking_branch: "a\ud83db" or a leg's branch: "a\x00b" no longer ends make bootstrap in a subprocess traceback under a manifest ok (#455) #644 deliberately keeps, by a manifest's "\udcXX" escape (L4-126), or by hand. Measured on main: run 1 exit 0 branch a\udcffb created; run 2 exit 1, UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 12. The head is the same (R455 O3). (source)

Rides along (backlog; tick it here if the same change covers it)

• L4-124 (NEEDS A RULING): a character the filesystem encoding lacks still ends bootstrap's step (a) in a traceback. The same on main and on the head. Measured with LC_ALL=C PYTHONUTF8=0, because this host has no Latin-1 locale (locale -a: C, C.utf8, POSIX) and sys.getfilesystemencoding() is then ascii, the same mechanism: tracking_branch: "feature/日本" gives the validator 0 manifest ok, then bootstrap 1, UnicodeEncodeError: 'ascii' codec can't encode characters in position 19-20: ordinal not in range(128). Under the default UTF-8 locale the same value creates feature/日本. assembly-root bootstrap.py: a lone surrogate or NUL in tracking_branch (or a leg's branch:) ends make bootstrap in a subprocess traceback in step (a), and validate-manifest.py says manifest ok, so no tool names it #455's own "Survive it" asks that "a branch the platform cannot hand to git" be treated as one git refuses, and bootstrap.py and validate-manifest.py: a branch holding a NUL or a lone surrogate is a finding and step (a) does not hand it to git, so tracking_branch: "a\ud83db" or a leg's branch: "a\x00b" no longer ends make bootstrap in a subprocess traceback under a manifest ok (#455) #644 scoped it out on purpose (the _argument_fault docstring). Where: bootstrap.py:125 (_argument_fault asks NUL and lone surrogates only). Shape of a fix: fault wherever _fsencoded_length(char) == 0. It needs its own remediation (the name is good; run under a UTF-8 locale or PYTHONUTF8=1), which changes what a person is told, so it needs a ruling of its own (R455 O1). (source)
• L4-126 (a question; R455 recommends no change here, and the fix at L4-125): step (a) CARRIES a manifest value holding U+DC80 to U+DCFF that step (b) then names as a finding. tracking_branch: "a\udcffb", or a leg's branch: "\udc80": step (a) creates and checks out refs/heads/a\xffb (resp. refs/heads/\x80) in each leg, because _argument_fault asks only whether os.fsencode can write the surrogate as one byte, and then the validator names the value (U+DCFF / U+DC80). Following the finding (editing back to main) and running again ends in UnicodeDecodeError (measured: bootstrap 1, make 2). That is L4-125's decode, the same on main, where the validator said nothing. Two options: (a) _checkout_all_tracking_branches (bootstrap.py:840) asks the validator's any-surrogate rule of a value from the manifest and keeps the os.fsencode rule for --branch alone, about 6 lines and one test, so step (a) never acts on a value step (b) refuses; (b) leave it and fix the root cause in L4-125. R455 recommends (b): the input is hand-written and exotic, and (a) only hides one route into L4-125 (R455 F4). (source)

What is expected

• git_out has ONE decode rule for what git prints, and a later make bootstrap on a leg whose branch is a\xffb ends branch ... at ... and not a traceback. The rule is the pull request's to choose, but it must keep the name's bytes, since the name goes back to git. A test makes the branch with git branch $'a\xffb' (POSIX only; skipped where the filesystem refuses the name, as the repository's other such tests are) and runs bootstrap twice.
• L4-126 is answered by that fix, option (b), unless Brett Heap rules for option (a). The pull request says so and ticks it.
• L4-124 is NOT in the first change. It waits for a ruling from Brett Heap on what a person is told when the platform's encoding cannot write a good branch name, and stays open here until then.

Not a duplicate: lane 5's #694 is a missing git, not a name git cannot decode. No issue numbered #650 to #700 cites L4-124 to L4-126 (read at 13:17Z); #425 is the adopt-project.py reader of the same decode.

Each line above is a line of #459's checklist comment (issuecomment-6214587074). Tick it there too when this closes.

Lane: openreposhape-4 (openRepoShape-4)