diff --git a/CHANGELOG.md b/CHANGELOG.md index efe2a4e170..137dc052e1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -375,6 +375,20 @@ already in use` and no way forward short of finding the file in a temporary key. ([#285](https://github.com/quickbeard/codev-code/pull/285)) +### TUI + +- **CoDev warns when the terminal cannot display Unicode** — in a legacy + Windows console window (classic conhost, e.g. a plain "Windows PowerShell" + window) using a raster font, everything outside the OEM code page — + Vietnamese diacritics, CJK, most box drawing — is drawn as `?` no matter + what the app writes, so prompts and responses looked corrupted and the + model got the blame. The TUI now detects that console configuration at + startup and shows a warning toast recommending Windows Terminal or a + TrueType console font such as Consolas. It cannot fix the rendering — only + the console's own font setting decides what it can draw — but the failure + now names itself instead of corrupting silently. + ([#287](https://github.com/quickbeard/codev-code/pull/287)) + ### Desktop / Web UI - **Switching between Code and Work brings the project back too** — the switch diff --git a/DIVERGENCES.md b/DIVERGENCES.md index 27b1f1c287..5f74029c13 100644 --- a/DIVERGENCES.md +++ b/DIVERGENCES.md @@ -106,6 +106,7 @@ from the squash message first, or use a plain merge. | `packages/tui/src/ui/dialog-prompt.tsx`, `packages/tui/src/config/keybind.ts`, `packages/tui/src/app.tsx` | Ctrl+V paste in dialog prompts. keybind.ts: added `"dialog.prompt.paste"` definition (`ctrl+v`, `preventDefault: false`, no CommandMap entry — dotted keys map to themselves). dialog-prompt.tsx: a `dialog.prompt.paste` command in the existing priority-1 binding layer reads `useClipboard().read?.()` and inserts `text/plain` content (newlines stripped, matching the single-line `InputRenderable` paste — dialog prompts are single-line fields and some upstream consumers store the value untrimmed); the command list is gathered alongside `dialog.prompt.submit`. app.tsx: `TuiInput.clipboard?: ClipboardService` passed to `` so tests inject a deterministic clipboard. Covered by the ctrl+v test in the fork-only `packages/tui/test/custom-provider.view.test.tsx`. | Many Windows terminals deliver ctrl+v as a key event instead of a bracketed paste, and upstream's clipboard-reading `prompt.paste` command targets only the main session prompt — so dialog fields (API keys in /connect, the custom-provider flow) could not paste at all (upstream issue anomalyco/opencode#39837). Bracketed paste still goes through the textarea's built-in default handler; the binding layer's `!props.busy` guard disables paste while busy. | Keep the keybind definition, the paste command + gathered binding list, and the `clipboard` input plumbing; take upstream's surrounding changes. Sent upstream as anomalyco/opencode#39844 (identical code) — once it lands in a synced release this row collapses to nothing; if upstream ships a different dialog paste fix, prefer theirs and drop this row. | | `packages/tui/src/clipboard.ts`, `packages/tui/src/component/prompt/index.tsx` | `read()`'s `win32`/WSL branch reads the text clipboard with `powershell.exe … [Console]::Out.Write([string](Get-Clipboard -Raw))` (UTF-8 `OutputEncoding`) right after the existing image probe, instead of falling through to `clipboardy`. | `clipboardy`'s Windows backend is the only one with no PATH-resolved fallback — it spawns `../fallbacks/windows/clipboard_x86_64.exe`, resolved from `import.meta.url`. Inside our compiled single-file binary that resolves under the bunfs root (`B:/~BUN/root/`) and the `.exe` is not embedded (verified: `codev-code-windows-x64@1.18.4-6` ships only `bin/codev.exe`), so every text read failed with ENOENT and was swallowed by the `.catch`. `clipboard.read()` returning `undefined` silently broke ctrl+v in the main prompt (`prompt.paste`), in dialog prompts (`dialog.prompt.paste`, see the row above), and in the empty-bracketed-paste fallback. Copy was unaffected (`copyCommand` already shells out to PowerShell), as was image paste. `Console.Out.Write` bypasses PowerShell's formatting pipeline so no trailing newline is appended. Paired with a `packages/tui/src/component/prompt/index.tsx` change: the `prompt.paste` command now toasts when the clipboard read yields nothing, instead of returning silently — that silence is why a completely dead paste path went unreported for so long. | Keep the PowerShell text read as long as we ship a compiled binary. If upstream drops `clipboardy` or vendors the Windows helper next to the executable, prefer theirs and drop this row. | | `packages/tui/src/terminal-win32.ts`, `packages/opencode/src/cli/cmd/debug/index.ts` | terminal-win32.ts: added a `win32ConsoleMode()` export returning the console input mode read through the guard's own kernel32 handle (undefined under the same conditions that stop `win32InstallCtrlCGuard` installing). debug/index.ts: registered the fork-only `ConsoleCommand` (`codev debug console`, `packages/opencode/src/cli/cmd/debug/console.ts`). | Windows users whose console keeps ENABLE_PROCESSED_INPUT set get ctrl+c as a console break rather than a keypress, which strands the cmd.exe shims in a hub launch chain at "Terminate batch job (Y/N)?" and looks like a frozen terminal. `codev debug console` reads the mode, installs the real guard, re-reads to prove the bit actually clears, then unhooks and re-reads — so an affected machine reports itself instead of needing a maintainer to bisect it. `codevhub doctor` relays the JSON form as its `console-ctrl-c` check. | Keep the `win32ConsoleMode` export and the one-line command registration; take upstream's surrounding changes. Both collapse to nothing if upstream ever ships its own console diagnostic. | +| `packages/tui/src/terminal-win32.ts`, `packages/tui/src/app.tsx` | terminal-win32.ts: added a `win32RasterFontConsole()` export (kernel32 `GetCurrentConsoleFontEx` via the existing dlopen table): true only when stdout is a win32 TTY, none of `WT_SESSION`/`TERM_PROGRAM`/`TERM` are set (conhost sets none of them; Windows Terminal, VS Code, mintty, ssh each set one), and the current console font lacks `TMPF_TRUETYPE`. app.tsx: an `onMount` next to the other startup toasts shows a one-shot warning toast when it returns true, telling the user to use Windows Terminal or a TrueType console font. | Legacy conhost with a raster font can only draw OEM-code-page characters: the display path best-fit converts everything else to "?" at draw time (chcp/SetConsoleOutputCP docs), so Vietnamese/CJK users see corrupted prompts and responses even though the buffer and the data are correct. The app cannot fix the rendering, so it warns instead of corrupting silently. | Keep the `win32RasterFontConsole` export and the warning-toast `onMount`; take upstream's surrounding changes. Drop both if upstream ships its own legacy-console/raster-font warning. | | `packages/opencode/script/build.ts` | Platform packages' `package.json` name `codev-code-` (dist dir names unchanged — the bun compile target derives from them); compiled binary emitted as `bin/codev` (was `bin/opencode`; smoke-test path follows); win32 targets additionally stage ripgrep at `bin/rg.exe` (`RG_VERSION` — `"latest"` resolves the newest GitHub release per build via the releases API, an exact version pins its release tag; asset zip extracted in-memory with `@zip.js/zip.js` so no unzip/powershell host dependency, magic-byte-checked against proxy block pages; one release lookup, and x64/x64-baseline share one download per arch) | npm publishing under the fork's names; the binary inside platform packages carries the fork's name (postinstall.mjs copies it into the `codev` bin shim). The `Script.release` upload block stays upstream-shaped and unused — the fork's `.github/workflows/release.yml` packages and uploads GitHub-release artifacts itself. Bundling rg makes a bare `npm i -g codev-code` self-sufficient on Windows, where fff is disabled so file search depends entirely on rg and corporate networks block the runtime GitHub download (see the Windows file-search row for the resolution side). `RG_VERSION` defaults to `"latest"`: the build machine needs GitHub access anyway, and floating leaves no version to keep in sync — binary.ts's pinned `VERSION` governs only the runtime download fallback (the mirror hosts just that version) and is independent of the bundled copy. | Keep our `name:` line, `bin/codev` paths, and the ripgrep bundling block; take upstream's surrounding changes. | | `packages/opencode/script/publish.ts` | `PUBLISH_NAME = "codev-code"` const; published-name→dir map; main pkg name + bin `codev` → `./bin/codev.exe` (postinstall stub file renamed from `${pkg.name}.exe`, matching the renamed platform binary); embedded README; no `repository` field in the generated package.json (npm renders it as the Repository link and derives a `#readme` Homepage from it — both kept off the npm page); env-guarded exit before the Docker/AUR/Homebrew section; `npm publish` retried up to 5 times with linear backoff, treating a version that turns up on the registry after a failed attempt as published; stale `*.tgz` deleted before `bun pm pack`; the dist manifest scan skips the generated main package directory | npm-only publishing under the fork's names (unscoped, public on npmjs); the installed command is `codev`; upstream's other channels target anomalyco infra; the npm page should not link the fork's GitHub repo. Releases run from a laptop rather than CI, where the 13 concurrent 40-60MB uploads regularly time one package out (`npm error FETCH_ERROR`) and abort the release mid-way; resuming such a release must not make the main package depend on itself. `bun pm pack` writes the tarball into the directory it packs, so the retry path it enables would otherwise bundle the previous run's tarball into the next one, doubling every package (~55MB → ~112MB) and making the timeouts far worse. | Keep our names, description, bin mapping, omitted `repository`, retry loop + early exit; take upstream's npm-section changes above the exit. If upstream adds its own publish retry, drop ours for theirs. | | `packages/opencode/bin/opencode` | `base` prefix `opencode-` → `codev-code-` | Runtime platform-binary lookup must match the published names. | Keep our prefix. | diff --git a/packages/tui/src/app.tsx b/packages/tui/src/app.tsx index a073042533..e3f14c7951 100644 --- a/packages/tui/src/app.tsx +++ b/packages/tui/src/app.tsx @@ -86,7 +86,7 @@ import type { EventSource } from "./context/sdk" import { DialogVariant } from "./component/dialog-variant" import { createTuiAttention } from "./attention" import * as TuiAudio from "./audio" -import { win32DisableProcessedInput, win32FlushInputBuffer } from "./terminal-win32" +import { win32DisableProcessedInput, win32FlushInputBuffer, win32RasterFontConsole } from "./terminal-win32" import { destroyRenderer } from "./util/renderer" import { cliErrorMessage, errorFormat } from "./util/error" @@ -552,6 +552,21 @@ function App(props: { onSnapshot?: () => Promise; pluginHost: TuiPlugi void enableAutoApprove() }) + // Legacy conhost with a raster font best-fit converts everything outside the + // OEM code page to "?" at draw time, so non-ASCII text (Vietnamese, CJK) + // looks corrupted no matter what the app writes. Warn once at startup. + onMount(() => { + if (!win32RasterFontConsole()) return + toast.show({ + variant: "warning", + title: "This console cannot display Unicode", + message: + 'The current console font is a raster font, so accented and non-Latin characters will show as "?". ' + + "Run CoDev Code in Windows Terminal, or switch the console font to a TrueType font such as Consolas.", + duration: 15000, + }) + }) + onMount(() => { batch(() => { if (args.agent) local.agent.set(args.agent) diff --git a/packages/tui/src/terminal-win32.ts b/packages/tui/src/terminal-win32.ts index 84d236505e..b1bf294369 100644 --- a/packages/tui/src/terminal-win32.ts +++ b/packages/tui/src/terminal-win32.ts @@ -2,7 +2,13 @@ import { dlopen, ptr } from "bun:ffi" import type { ReadStream } from "node:tty" const STD_INPUT_HANDLE = -10 +const STD_OUTPUT_HANDLE = -11 const ENABLE_PROCESSED_INPUT = 0x0001 +const TMPF_TRUETYPE = 0x04 +// CONSOLE_FONT_INFOEX: cbSize u32, nFont u32, dwFontSize 2×i16, FontFamily u32, +// FontWeight u32, FaceName wchar[32]. FontFamily sits at byte offset 12. +const CONSOLE_FONT_INFOEX_SIZE = 84 +const FONT_FAMILY_OFFSET = 12 const kernel = () => dlopen("kernel32.dll", { @@ -10,6 +16,7 @@ const kernel = () => GetConsoleMode: { args: ["ptr", "ptr"], returns: "i32" }, SetConsoleMode: { args: ["ptr", "u32"], returns: "i32" }, FlushConsoleInputBuffer: { args: ["ptr"], returns: "i32" }, + GetCurrentConsoleFontEx: { args: ["ptr", "i32", "ptr"], returns: "i32" }, }) let k32: ReturnType | undefined @@ -68,6 +75,31 @@ export function win32ConsoleMode() { return buf[0]! } +/** + * True when stdout is a legacy console (conhost) window rendering with a + * raster font. + * + * Raster fonts can only draw characters from the OEM code page, so even a + * correctly written Unicode buffer is best-fit converted at draw time — every + * character outside the code page shows as "?" (Vietnamese diacritics, CJK, + * box drawing). The env guards skip hosts that render fine regardless of the + * reported font: Windows Terminal sets WT_SESSION, and other modern hosts + * (VS Code, mintty, ssh) set TERM_PROGRAM or TERM, which conhost never does. + */ +export function win32RasterFontConsole() { + if (process.platform !== "win32") return false + if (!process.stdout.isTTY) return false + if (process.env.WT_SESSION || process.env.TERM_PROGRAM || process.env.TERM) return false + if (!load()) return false + + const info = new Uint8Array(CONSOLE_FONT_INFOEX_SIZE) + const view = new DataView(info.buffer) + view.setUint32(0, CONSOLE_FONT_INFOEX_SIZE, true) + const handle = k32!.symbols.GetStdHandle(STD_OUTPUT_HANDLE) + if (k32!.symbols.GetCurrentConsoleFontEx(handle, 0, ptr(info)) === 0) return false + return (view.getUint32(FONT_FAMILY_OFFSET, true) & TMPF_TRUETYPE) === 0 +} + let unhook: (() => void) | undefined /**