Skip to content

JNI: NewStringUTF (modified UTF-8) mangles astral-plane chars — emoji alphabet text is corrupted in transit #72

Open

Description

@willwade

Follow-up from the #71 review loop (review M3, pre-existing).

jni_bridge.cpp uses env->NewStringUTF(...) for engine→Kotlin strings (e.g. output callback text at jni_bridge.cpp:189 and friends). JNI's NewStringUTF expects modified UTF-8 (CESU-8): 4-byte UTF-8 sequences — every astral-plane codepoint, i.e. most emoji — are invalid and get mangled. Since c003214 the app ships an Emoji alphabet, so astral chars are a live output path: the engine emits correct UTF-8, the Kotlin side receives garbage.

(The UTF-16-level coalescing contract added in #71 is correct at the Kotlin layer — ImeOutputCoalescerTest pins it — but the runtime path through JNI corrupts the string before Kotlin sees it.)

Fix pattern: decode UTF-8 → UTF-16 manually (or via String::fromUTF8 helpers) and construct with env->NewString(utf16, len). Audit every NewStringUTF in jni_bridge.cpp; also check the reverse direction (GetStringUTFChars — same modified-UTF-8 caveat) for Kotlin→engine strings (training import, seedBuffer).

Split out of #71 to keep that PR scoped to the batching fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions