[brettheap]

What a person now sees

When a bump or remove rollback cannot check a member out, it prints the line to type from _head_by_hand in scripts/family.py. For a member whose branch is not UTF-8 (read from git with surrogateescape) the line was

git -C members/Thing checkout -q 'caf\udce9' --

The stdout stream writes the surrogate as those six characters, and typed they name no branch: git answers fatal: invalid reference: caf\udce9 (reproduced by the new tests on the unfixed code). It is now

git -C members/Thing checkout -q "$(printf 'caf\351')" --

and git is put back on refs/heads/caf\xe9, read as bytes, in sh, dash, bash and (review round) zsh.

The change, scripts/family.py

• New _branch_by_hand(branch) (line 7753): a name that is UTF-8 text keeps shlex.quote, byte for byte. A name that holds a surrogate escape is spelled "$(printf '<escapes>')". Each byte that is not printable ASCII (below 0x20, DEL, above 0x7E) is a three-digit octal escape, e.g. \351. Each printable ASCII byte passes through, except ', \, % and (review round) !, which are escaped as \047, \134, \045 and \041 too. So the printf format reproduces the exact bytes in any POSIX shell, and the line is pure ASCII whatever stream prints it.
• _head_by_hand (line 7781) calls it where it called shlex.quote(branch) (line 7787). Nothing else in the file changed. _head_by_hand uses shlex.quote (POSIX), not the host quoter shell_quoted, so tests/test_readme_windows_quoting_sentence.py is not involved (it passes unchanged).

Where the issue left it open, these are decided as lane 3 ruled:

• All three special bytes are spelled in octal, not %% and \\, so one rule covers every byte and a ' never ends the quotation.
• $(...) strips trailing newlines, which loses nothing: a ref name cannot hold a newline.
• The issue's other option (say the name is not UTF-8 and give the commit) was not taken.

Tests, tests/test_family_head_by_hand_bytes.py (new, 45 rows at first, 61 after the review round)

• A real repository with a branch named by the bytes caf\xe9, a second branch checked out, the printed line (as the stream writes it, backslashreplace) run through sh -c, then git symbolic-ref HEAD read as bytes must be refs/heads/caf\xe9. A second row has a name full of syntax git does hold (', %s, %d, ", $HOME, backticks, braces, ;&|<>()!#) beside a non-UTF-8 byte.
• A row for a name holding ', \ and % beside a non-UTF-8 byte, and a row of every byte 1 to 255: printf %s <word> in sh, dash, bash and zsh writes the name's exact bytes (git cannot hold \ or a control byte, so the word is asked for itself).
• The exact text for the issue's branch, the word is ASCII, never holds udce9, every byte the format or its quotes would read is an octal escape, and DEL, newline and 0xFF are escapes.
• UTF-8 names (including ', %, \, a space, $HOME, and e-acute and CJK text) give shlex.quote's word and the unchanged line; no branch still prints the commit; the path is still shlex.quoted.
• On main _member_head still raises on a branch that is not UTF-8, so _head_by_hand is tested directly, not the rollback end to end. tests/test_family_member_branch_not_utf8.py is not touched: it exists only on the adopt-project.py and family.py: a source this user cannot enter is source-unreadable, a UTF-8 plan reads its source under the C locale, and a member on a branch that is not UTF-8 is put back on it (#918) #1072 branch.
• Shell-running rows are skipped on Windows (WINDOWS_SKIP); dash, bash and zsh rows skip where absent.

Left out of scope

• The path argument of the same line, shlex.quote(path), has the same fault for a member mounted at a path that is not UTF-8. The issue names the branch only, so it is left alone.
• The rollback end to end for a branch that is not UTF-8: it waits on adopt-project.py and family.py: a source this user cannot enter is source-unreadable, a UTF-8 plan reads its source under the C locale, and a member on a branch that is not UTF-8 is put back on it (#918) #1072.
• No "Needs a ruling" or "Documented limit" section in family.py: a rollback's by-hand checkout line spells a branch that is not UTF-8 as Python's \udce9, which typed names no branch #1079.

Test commands (stub gh first on PATH, GH_TOKEN=bogus, empty GH_CONFIG_DIR, EMAIL set, one file at a time)

• pytest tests/test_family_head_by_hand_bytes.py: 45 passed in 1.04s
• pytest tests/test_family_bump_remove_rollback.py: 31 passed in 34.20s
• pytest tests/test_family_py_unreadable.py: 75 passed in 36.10s
• pytest tests/test_readme_windows_quoting_sentence.py: 44 passed in 16.58s
• pytest tests/test_repo_hygiene.py: 100 passed in 2.49s
• pytest tests/test_no_auto_maintenance.py: 60 passed in 11.43s
• python3 -m pyflakes scripts/family.py tests/test_family_head_by_hand_bytes.py: no output; the same on scripts/family.py at origin/main.

Mutants (family.py restored after each and checked with cmp)

1. shlex.quote(branch) back for every name in _head_by_hand: KILLED, 8 failed. The real-repository rows fail with fatal: invalid reference: caf\udce9; test_the_line_for_the_issues_branch_is_spelled_exactly and test_a_name_that_is_not_utf8_never_prints_the_surrogate fail too.
2. The % escape dropped (not in "'\\"): KILLED, 16 failed: the apostrophe-percent-and-more real-repository rows, test_each_byte_the_format_or_its_quotes_would_read_is_an_octal_escape, and test_the_shell_hands_git_the_exact_bytes rows apostrophe-percent-and-more, apostrophe-backslash-percent, percent-conversions and every-byte, each in all three shells.
3. The ' escape dropped: KILLED, 13 failed.
4. The \ escape dropped: KILLED, 4 failed: test_each_byte_the_format_or_its_quotes_would_read_is_an_octal_escape and the apostrophe-backslash-percent row in each shell (a name holding \n, \t, \x41 and \0, which printf would read as escapes).
5. DEL passed through (byte <= 0x7f): KILLED, 1 failed, test_the_word_holds_printable_ascii_only_whatever_the_bytes (it survived the shell rows, which is why that test exists).
6. The UTF-8 branch removed (every name in the printf form): KILLED, 43 failed.

Merges (git merge-tree --write-tree, head 1bae5e1656bfa2cd542f4ed2ef5b9fa24558ca84, cut from origin/main bccae28052f8aa786ba3907d6aabd880cc29062d)

• origin/main bccae2805: clean.
• tests: five adopt tests fail when the thing they guard is removed: bytes on stdin, a stub gh that keeps its #! line, a cleared GIT_ALLOW_PROTOCOL, each guard of two pairs reached alone, and three name shapes (#916) #1071 24273dd4: clean (its merge tree equals origin/main's, so tests: five adopt tests fail when the thing they guard is removed: bytes on stdin, a stub gh that keeps its #! line, a cleared GIT_ALLOW_PROTOCOL, each guard of two pairs reached alone, and three name shapes (#916) #1071 is already in main).
• adopt-project.py and family.py: a source this user cannot enter is source-unreadable, a UTF-8 plan reads its source under the C locale, and a member on a branch that is not UTF-8 is put back on it (#918) #1072 2ffd8235 (also edits family.py; does not touch _head_by_hand): clean. The new test file also passes on the merged tree (with adopt-project.py and family.py: a source this user cannot enter is source-unreadable, a UTF-8 plan reads its source under the C locale, and a member on a branch that is not UTF-8 is put back on it (#918) #1072's own tests/test_family_member_branch_not_utf8.py, its two end-to-end rows error there and at adopt-project.py and family.py: a source this user cannot enter is source-unreadable, a UTF-8 plan reads its source under the C locale, and a member on a branch that is not UTF-8 is put back on it (#918) #1072's own head alike, because the check was run on a git archive of the tree, which is not a git repository).
• adopt-project.py and shape-doctor.py: a .gitmodules with no section asks nothing and says how to remove it, a submodule beside one is named, so is a file's answered_paths: row with a /, and an identical drop counts as drop (#914) #1073 b58a4e2c: clean.

After the rebase onto bccae2805 (tests only, no overlap) the new file and tests/test_repo_hygiene.py were run again: 45 passed and 100 passed. The other four files were run on the first base, 5e539a878, and scripts/family.py is byte-identical on both.

Review round (RV1084, issuecomment-6303644023; head 6ab9d3263dffe3e2cc19270c28117ac90da81d6f, one commit on top, pushed as a fast-forward)

• SHOULD-1, tests/test_family_head_by_hand_bytes.py:42: SHELLS gains zsh (macOS's default shell). shell_or_skip still skips a shell that is not installed; here its 10 rows run and none skip (-v -rs: 61 passed, 0 skipped).
• SHOULD-2, tests/test_family_head_by_hand_bytes.py:268: test_the_module_reads_a_name_as_this_test_does is now test_surrogateescape_reads_the_issues_branch_as_caf_and_u_dce9. Its docstring says it checks Python's own surrogateescape decode, and that the module's own read of such a branch is adopt-project.py and family.py: a source this user cannot enter is source-unreadable, a UTF-8 plan reads its source under the C locale, and a member on a branch that is not UTF-8 is put back on it (#918) #1072's _member_head, which this file does not test.
• NIT-1, scripts/family.py:7775: the printf word spells ! as \041 too, so the line never depends on a shell's history scanner. Only a name that is not UTF-8 uses the printf word, so a UTF-8 name is shlex.quoted as before (new rows fix!ec and cafe-acute!, byte for byte). New row test_the_bang_is_an_octal_escape_so_no_history_scanner_reads_the_line (exact text, no ! in the word or the line), and a bang row in each of the four shells.
• NIT-2, the commit trailer: no change.
• Mutant, ! passed through (not in "'\\%"): KILLED, 1 failed, test_the_bang_is_an_octal_escape_so_no_history_scanner_reads_the_line. A non-interactive -c shell does not history-expand, so only the exact-text row can see it. Re-run at the new head: % escape dropped KILLED (21 failed), DEL passed through KILLED (1), shlex.quote back at the call site KILLED (11).
• Tests at the new head, one file at a time under the stub gh: test_family_head_by_hand_bytes.py 61 passed; test_family_bump_remove_rollback.py 31 passed; test_readme_windows_quoting_sentence.py 44 passed; test_repo_hygiene.py 100 passed; pyflakes on scripts/family.py and the test file: no output.
• Merges at the new head (git merge-tree --write-tree, after a fresh fetch): origin/main c13c3985f clean, adopt-project.py and family.py: a source this user cannot enter is source-unreadable, a UTF-8 plan reads its source under the C locale, and a member on a branch that is not UTF-8 is put back on it (#918) #1072 2ffd8235 clean, adopt-project.py and shape-doctor.py: a .gitmodules with no section asks nothing and says how to remove it, a submodule beside one is named, so is a file's answered_paths: row with a /, and an identical drop counts as drop (#914) #1073 b58a4e2c clean. The Merges section above is the state at 1bae5e165.

Follow-up

• family.py: a failed bump's by-hand lines quote a leg mounted at a path that is not UTF-8 as Python's \udcXX, so the typed git -C <member>/<leg> and submodule update -- <leg> name no path #1086 takes the leg path, the same by-hand lines' path argument for a leg mounted at a path that is not UTF-8. This PR leaves it alone.

Closes #1079

Lane: openreposhape-3 (openRepoShape-3)

🤖 Generated with Claude Code

Summary by Sourcery

Make failed rollback checkout instructions reproduce non-UTF-8 branch names correctly when typed in a POSIX shell.

Bug Fixes:

• Preserve the exact byte representation of non-UTF-8 branch names in the POSIX shell command printed for failed rollbacks.

Enhancements:

• Retain the existing quoting behavior for UTF-8 branch names while safely rendering arbitrary branch bytes as printable shell input.

Tests:

• Add comprehensive coverage for non-UTF-8, shell-sensitive, arbitrary-byte, UTF-8, and unchanged rollback command cases across supported POSIX shells.

[sourcery-ai]

Reviewer's Guide

The PR fixes rollback instructions for branches containing non-UTF-8 bytes by rendering those names as ASCII-only POSIX shell commands that reconstruct the original bytes with printf, while preserving existing UTF-8 quoting and adding broad shell-level regression tests.

Sequence diagram for byte-preserving rollback checkout instructions

sequenceDiagram
    participant Rollback as Rollback flow
    participant Family as scripts/family.py
    participant Shell as POSIX shell
    participant Git as git

    Rollback->>Family: _head_by_hand(path, at, branch)
    Family->>Family: _branch_by_hand(branch)
    alt branch is valid UTF-8
        Family-->>Rollback: shlex.quote(branch)
    else branch contains surrogateescaped bytes
        Family-->>Rollback: "$(printf '<octal escapes>')"
    end
    Rollback->>Shell: Execute checkout instruction
    Shell->>Shell: printf reconstructs exact branch bytes
    Shell->>Git: checkout -q <branch bytes> --
    Git-->>Shell: checkout branch

Loading

Flow diagram for branch spelling in _head_by_hand

flowchart TD
    A[_head_by_hand receives branch] --> B{branch.encode utf-8 succeeds?}
    B -->|Yes| C[shlex.quote branch]
    B -->|No| D[Encode with surrogateescape]
    D --> E[Keep printable ASCII except apostrophe backslash percent]
    E --> F[Render all other bytes as three-digit octal escapes]
    F --> G[Wrap as printf command substitution]
    C --> H[Build git checkout instruction]
    G --> H
    H --> I[ASCII-safe instruction reconstructs exact branch bytes in POSIX shell]

Loading

File-Level Changes

Change	Details	Files
Add byte-preserving POSIX-shell formatting for non-UTF-8 branch names in manually printed checkout commands.	Detect surrogateescaped branch names by attempting UTF-8 encoding. Encode non-printable bytes and shell/printf-sensitive characters as three-digit octal escapes inside a $(printf ...) substitution. Keep UTF-8 branch names on the existing shlex.quote path and leave commit and path handling unchanged.	scripts/family.py
Add comprehensive regression coverage for exact branch-byte preservation and unchanged quoting behavior.	Test generated commands against real repositories and sh, dash, and bash, including names containing non-UTF-8 bytes and shell/printf syntax. Verify ASCII-only output, exact issue spelling, octal escaping, arbitrary byte handling, and preservation of UTF-8, no-branch, and path behavior. Skip shell-dependent cases on unsupported platforms or when shells are unavailable.	tests/test_family_head_by_hand_bytes.py

Assessment against linked issues

Issue	Objective	Addressed	Explanation
#1079	Ensure the rollback’s by-hand checkout command represents a branch name containing non-UTF-8 bytes in a form a person can type that causes the shell and Git to receive the original branch bytes.	✅

Possibly linked issues

• family.py: a rollback's by-hand checkout line spells a branch that is not UTF-8 as Python's \udce9, which typed names no branch #1079: The PR fixes the issue by emitting a shell word that preserves exact branch bytes instead of printing unusable surrogate escapes.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[brettheap]

(lane openRepoShape-3, 2026-10-20T22:57:11Z) Independent review (RV1084) of 1bae5e1: READY. No MUST.

Probe: 453 real branches (every byte value git holds in a ref name, as x<b>\xe9 and x<b>y, plus 23 odd names such as caf\xe91, a)b\xe9, $(id)\xe9, ODD_REF, %b\xe9, !x\xe9, 0x80-0xFF) made in a real repository; the _head_by_hand line, written with backslashreplace, run with <sh> -c in sh, dash 0.5.12, bash 5.2.21 and zsh 5.9; HEAD read back as bytes. 453/453 in each shell, under LC_ALL unset, C and C.UTF-8. The 87 UTF-8 names print exactly as origin/main's line. Interactive bash and zsh do not history-expand a ! inside the printf word (a double-quoted control did expand).

Tests (one at a time, stub gh): test_family_head_by_hand_bytes.py 45 passed; test_family_bump_remove_rollback.py 31 passed; test_readme_windows_quoting_sentence.py 44 passed. pyflakes clean.
Mutants: 1 (shlex.quote back) KILLED 8; 2 (% escape dropped) KILLED 16; 5 (DEL passed through) KILLED 1; extra (outer double quotes dropped) KILLED 3 by the every-byte shell rows. family.py restored and checked with cmp each time.
Merges (merge-tree): origin/main c13c398 clean (bootstrap.py only; family.py and the new test are the same blobs); #1072 2ffd823 clean, and the new file passes on that merge (45 passed); #1073 b58a4e2 clean. On the #1072 merge, _member_head reads caf\xe9 as 'caf\udce9' and the printed line puts HEAD back in dash, bash and zsh.

SHOULD:

1. tests/test_family_head_by_hand_bytes.py:40: add "zsh" to SHELLS (macOS's default shell; the suite already drives zsh; all 453 probe names pass in zsh; shell_or_skip covers its absence).
2. tests/test_family_head_by_hand_bytes.py:245: test_the_module_reads_a_name_as_this_test_does never calls the module; no change to family.py can fail it. Rename it to what it checks (Python's surrogateescape decode) or make it call _member_head once adopt-project.py and family.py: a source this user cannot enter is source-unreadable, a UTF-8 plan reads its source under the C locale, and a member on a branch that is not UTF-8 is put back on it (#918) #1072 lands.

NIT:

1. scripts/family.py:7774: escaping ! as \041 would make the line independent of each shell's history scanner. Optional; no fault found.
2. The commit trailer names Sonnet 5.5; set the attribution you want in the squash message.

Follow-up, not this PR: a leg path read with surrogateescape (family.py:7991) reaches shlex.quote at 8277, 8279 and, through _head_by_hand, 7787. The printf form cannot be reused as is: $(...) strips a trailing newline, which a path can hold, and dash has no $'...'. Draft title: "family.py: a failed bump's by-hand lines quote a leg mounted at a path that is not UTF-8 as Python's \udcXX, so the typed git -C <member>/<leg> and submodule update -- <leg> name no path".

PR body: closingIssuesReferences is [1079]; it ends Closes #1079, the Lane line and the robot line, then Sourcery's appended summary (as on #1071), which has no closing keyword; nothing above U+007F but the robot.

Lane 3: a fix round (F1084) takes SHOULD-1, SHOULD-2 and NIT-1 (! as \041), then a delta review. The leg-path case is filed as a follow-up issue, outside this PR.

Lane: openreposhape-3 (openRepoShape-3)

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[brettheap]

(lane openRepoShape-3, 2026-10-20T23:08:32Z) Independent delta review (RV1084d) of the fix round, 1bae5e1..6ab9d32 (one commit, 2 files, +42/-14): READY. No MUST, no SHOULD.

SHOULD-1: SHELLS now holds zsh (tests/test_family_head_by_hand_bytes.py:42). With -v -rs: 61 passed, 0 skipped; all 10 zsh rows ran and passed (2 real-repository rows, 8 shell-bytes rows including bang-zsh).
SHOULD-2: the test is now test_surrogateescape_reads_the_issues_branch_as_caf_and_u_dce9 (:268) and its docstring is true. Checked: at this head _member_head on a real repository sitting on caf\xe9 raises UnicodeDecodeError, as the docstring says.
NIT-1: scripts/family.py:7775 escapes ! as \041; !x!\xe9 prints as "$(printf '\041x\041\351')". UTF-8 names are still shlex.quote's word and line byte for byte (9 of 9, including fix!ec, cafe-acute!, !x, a!b!!c).
Mutant, ! passed through again ("'\%"): KILLED, 1 failed (test_the_bang_is_an_octal_escape_so_no_history_scanner_reads_the_line). family.py restored and checked with cmp; tree clean.

Tests at 6ab9d32 (one file at a time, stub gh exit 97, GH_TOKEN=bogus, empty GH_CONFIG_DIR, GIT_SSH_COMMAND=false, EMAIL set): test_family_head_by_hand_bytes.py 61 passed, 0 skipped; test_family_bump_remove_rollback.py 31 passed. pyflakes clean.
Merges (merge-tree, fresh fetch): origin/main c13c398 clean (fec87ddf); #1072 2ffd823 clean (0f8f4e07); #1073 b58a4e2 clean (a35b505f).
Body: closingIssuesReferences is [1079]; it ends Closes #1079, the Lane line and the robot line; no other closing keyword anywhere; #1086 appears only under Follow-up; nothing above U+007F but the robot.

NIT (optional, body only): the Tests section's second and fifth bullets still say "in sh, dash and bash" and "dash and bash rows skip where absent"; adding zsh would match the Review round section. The round-1 trailer NIT stands: set the attribution you want in the squash message.

Lane 3: the body NIT is fixed now (zsh in both Tests bullets). #1084 lands at 6ab9d32 through its pilot, chained after #1072 and #1073, one HOLD at a time.

Lane: openreposhape-3 (openRepoShape-3)

[sourcery-ai]

Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 17 hours and 49 minutes by commenting @sourcery-ai review. Upgrade to get a review now.