shape_materialize.py commit_trailers: on a git without commit --trailer, a non-ASCII --trailer travels in the stdin commit message and each undecodable byte becomes ? under LC_ALL=C PYTHONUTF8=0, so a trailer of José is committed as Jos?? #322

Lane: openreposhape-3 (openRepoShape-3)

What happens. Without --trailer support commit_trailers appends the trailer lines to the message text (body + "\n" + "\n".join(lines)), and the commit helpers write it on stdin with encoding="utf-8", errors="replace". Under a C locale with UTF-8 mode off, Python decodes the argument Jos<C3><A9> as Jos\udcc3\udca9 (surrogateescape), and a lone surrogate encodes as ? under replace. Measured on be2790e with git_takes_trailer_option forced to False (git 2.43.0 here has the option, so an older git was emulated, not run):

LC_ALL=C PYTHONUTF8=0   ->  b'Bump the pin\n\nCo-Authored-By: Jos??\n'
PYTHONUTF8=1            ->  b'Bump the pin\n\nCo-Authored-By: Jos\xc3\xa9\n'

Per #230, in a UTF-8 locale a trailer argument whose bytes are not valid UTF-8 is replaced the same way. Before #230 both raised UnicodeEncodeError. With a git that has --trailer the value goes as an argument and is kept (not measured).

Where. scripts/shape_materialize.py:commit_trailers (the no-option branch, :971-972); the stdin writers scripts/family.py:_commit_one (:380), scripts/bump-leg.py:commit_once (:503), update-shape.py:commit_on_branch (:1778).

Why it matters. The trailers are Lane: and Co-Authored-By: lines, the commit's record of who produced it; a name altered silently is worse than one refused. Rare: it needs a git without the option and a C locale. Direction (from #230): the commit helpers encode stdin with surrogateescape while decoding git's output as UTF-8.

Source. PR #230, ## Left out, first bullet; coordinator's running list item 6.