Skip to content

scaffold-project.py and family.py init with --local-remote-dir under a directory whose name is not UTF-8 put that path into the clone URL they render into the new repositories' files, and write_lf (strict UTF-8) raises a UnicodeEncodeError traceback, exit 1, after the bare remotes exist (--dry-run exits 0), where adopt refuses an unwritable clone URL before any leg exists #433

Closed

Description

@brettheap

(lane openRepoShape-2, 2026-10-12T17:32Z)

Found by this lane's review of PR #404 (the fix for #385) at f9d420a, which ran every origin-URL reader against a non-UTF-8 path and met this one in the two tools that make the URL. It is the scaffold's and the family holder's twin of #255, which refused it in adopt. Reproduced on origin/main 92e29d9 (git 2.43.0, Python 3.12.3, LANG unset, charmap ANSI_X3.4-1968), local remotes only, a stub gh. Not claimed.

What is wrong. Under --local-remote-dir <dir>, the clone URL of a new repository is the bare repository's path on disk: scaffold-project.py builds it from args.local_remote_dir.resolve() (:921-924) and renders it into the template values as CLONE_URL and ASSEMBLY_CLONE_URL (:1382-1383), which the READMEs print (git clone --recurse-submodules {{CLONE_URL}}, templates/assembly-root/README.md:15, :33; the legs' {{ASSEMBLY_CLONE_URL}}); scripts/family.py does the same for the holder (_holder_remote, :747-750, CLONE_URL at :788). When <dir> has a component whose name is not UTF-8 (a path under a directory named in a legacy code page), Python holds it as a string with a lone surrogate (\udce9), and every materialized file is written through write_lf, which encodes strictly (scripts/shape_materialize.py:76-90). So the run ends in a UnicodeEncodeError traceback, exit 1, AFTER the three bare remotes exist (the scaffold) or after the holder's (family), and the work directory is half made. --dry-run exits 0 and says nothing. A person has to delete what the first run made before running it again.

# $U: a checkout of openRepoShape at 92e29d9. Local; no network.
mkdir -p "$D/$(printf 'caf\xe9')"
python3 "$U/scaffold-project.py" --org testorg --project Atlas --elected-by 'T H' --elected-on 2026-09-02 \
        --local-remote-dir "$D/$(printf 'caf\xe9')" --work-dir "$D/work"
#   exit 1
#   Traceback: scaffold-project.py line 1418 in _scaffold / line 1126 in _seed_legs / shape_materialize.py line 1507 in copy_tree / line 90 in write_lf
#   UnicodeEncodeError: 'utf-8' codec can't encode character '\udce9' in position 591: surrogates not allowed
ls "$D/$(printf 'caf\xe9')"      # Atlas-code.git Atlas-spec.git Atlas.git  (all three made); ls "$D/work": Atlas-spec
python3 "$U/scripts/family.py" init --org testorg --family Contoso --created-by 'T H' --local-remote-dir "$D/$(printf 'caf\xe9')" --into "$D/into"
#   exit 1, UnicodeEncodeError from write_lf (shape_materialize.py:90) via materialize_family_root -> place; Contoso.git exists, the family folder is made
--local-remote-dir scaffold-project.py family.py init
<dir>/café (UTF-8, control) exit 0 exit 0
<dir>/plain (control) exit 0 exit 0
<dir>/caf<e9> exit 1, UnicodeEncodeError in write_lf; three remotes made, work dir half made; --dry-run exit 0 exit 1, the same; Contoso.git made, family folder made

Why. Adopt asks the question before anything exists: _refuse_an_unwritable_clone_url (adopt-project.py:4427, called at :4249, #255 and PR #270) runs the URL through unwritable_reason (scripts/shape_materialize.py:1068) and refuses adopt-unwritable-value with the remediation "nothing was created; rename or move". #282 (PR #351, landed) asked the same of the scaffold's free-text values (--elected-by, --reference, --name) and its elector, and its _refuse_unwritable_values (scaffold-project.py:713) is where such a refusal belongs; the local remote path is a value a file holds that it does not ask about. Only a LOCAL url can be refused: a https://github.com/<org>/<name>.git is ASCII by construction.

What it should do. Before anything is created, --dry-run alike: for --local-remote-dir, ask unwritable_reason of each clone URL the tool will render (the three the scaffold renders, the holder's for family) and refuse scaffold-unwritable-value / a family code, naming --local-remote-dir as the source of the value, saying nothing was created, and to pass a directory whose name is text. Reuse unwritable_values/_refuse_unwritable_values for the scaffold (add the clone URL to its fields, named --local-remote-dir) and call the same question in family.py init before _holder_remote's result is used. --local-remote-dir is the TEST path (its help says so), so this is a small change.

How to test. In tests/test_scaffold_free_text_values.py (the #282 file) and the family init tests, with a directory made with os.mkdir(os.fsencode(tmp) + b"/caf\xe9") (POSIX only; skip where the filesystem rejects the byte): --local-remote-dir under it, --dry-run and the real run exit 2 with the new code, nothing is created (ls of the remote dir and the work dir is empty or absent), and a UTF-8 directory still scaffolds.

Not in this issue. --work-dir, --into and --source holding a non-UTF-8 byte (the work directory is not written into a file; --source is #255's). The origin URL readers (#292, #385).

Related: #255 and PR #270 (adopt's twin); #282 and PR #351 (the scaffold's free-text values); #385 and PR #404 (the origin-URL readers that found it).

Lane: openreposhape-2 (openRepoShape-2)

Activity

  1. added this to the v1.1.0 milestone on Oct 15, 2026
  2. brettheap commented on Oct 15, 2026

    @brettheap
    ContributorAuthor

    CLAIMED — lane openRepoShape-4, session 11c6c0ef-495a-4a05-94cf-811b7b0a1012@Eagle, 2026-10-15T23:00:09Z, for #433

    Logged in lanes/log/openRepoShape-4.md at 2c617f14730a656a85f1a16aa7e6f6cb8cf8b6b6.

    The three reads (lane-collision-protocol Rule 1):

    1. existing `CLAIMED —` comments on opensoft/openRepoShape#433:
       none
    2. `gh pr list --repo opensoft/openRepoShape --state all --search "#433"`, kept where the row or its body names `#433`:
       none
    3. `git ls-remote --heads git@github.com:opensoft/openRepoShape` branches naming `433` as a whole token:
       none
    
  3. added 3 commits that reference this issue on Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions