Skip to content

fix: decode ZCode hook input as UTF-8 on Windows - #151

Merged
Grivn merged 1 commit into
masterfrom
codex/issue-142-zcode-utf8
Oct 3, 2026
Merged

Grivn merged 1 commit into
masterfrom
codex/issue-142-zcode-utf8

Conversation

@Grivn

@Grivn Grivn commented Oct 3, 2026

Copy link
Copy Markdown
Member

Windows PowerShell can decode ZCode's UTF-8 hook payload as CP936, corrupting Chinese text and losing stop_hook_active. The Stop hook then repeats its blocking reminder. Decode stdin explicitly as UTF-8, preserve the source BOM required for Chinese completion tokens, and accept the documented Chinese evaluations in both PowerShell and shell hooks. The reminder now requests a [mnemon] completion marker for other languages.

Closes #142.

Actual input Original asset Fixed asset
UTF-8 {"stop_hook_active":true,"last_assistant_message":"中"} with CP936 console input Incorrect block Silent exit
无需写入持久记忆 or 已写入记忆 Repeated block Silent exit
Ordinary response without an evaluation block One reminder with completion-marker guidance

The opt-in runtime suite installs hooks with the built CLI in an isolated home containing spaces and Chinese characters, verifies the installed bytes, and runs both the original asset and the fixed hooks. Windows runs require powershell.exe 5.1. The regular deterministic CI gate remains unchanged; the Manual Integration workflow adds a targeted zcode-windows selection.

Validation:

  • go build -o mnemon . and make test pass.
  • Real CLI smoke, setup/reinstall/eject, and Memory CLI E2E pass (258/258 checks).
  • Portable PowerShell 7 regression tests pass, including the original-asset controls. A separate three-code-page check improves from 19/27 to 27/27 expected outcomes.
  • Native Windows PowerShell 5.1 run passes at this commit: 34 cases (30 fixed-hook cases and four original-asset controls), with console input code pages 65001 and 936. The runner's system ANSI code page is 1252; CP936 is selected explicitly at the input boundary.

After updating Mnemon, rerun mnemon setup --target zcode --global --yes and start a new ZCode session to refresh installed hooks. The verification covers actual installed hook processes; it does not exercise the ZCode GUI.

Preserve the active-stop guard across console code pages and ship the Chinese completion tokens with a UTF-8 BOM for Windows PowerShell 5.1. Keep shell completion behavior aligned and ask for a language-neutral evaluation marker.

Add deployed-byte checks and an opt-in Windows process suite with the old asset as a regression control. Validated with go build, make test, isolated CLI smoke, and real PowerShell 7 UTF-8/CP936 before-and-after probes; Windows PowerShell 5.1 runs through Manual Integration.
@Grivn
Grivn merged commit a8b8503 into master Oct 3, 2026
4 checks passed
@Grivn
Grivn deleted the codex/issue-142-zcode-utf8 branch October 11, 2026 12:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ZCode stop.ps1 on Windows misdecodes UTF-8 payload under GBK locale; guard skipped, blocks every turn

1 participant