feat(text): Add complex text shaping and rasterization

[OmarAglan]

• Split from feat(text): Shape complex single-line UI text #3231
• Depends on chore(lib): Define Uniscribe itemization control #3287 and feat(text): Add Unicode paragraph direction detection #3291.

Adds Uniscribe-based measurement and rasterization to FontCharsClass. The implementation itemizes a paragraph, preserves its bidirectional state and directional overrides, selects fonts at cluster boundaries, and draws the resulting runs onto a shared baseline.

Measurement and rasterization apply the same size limits. The implementation checks allocation dimensions, synchronizes GDI drawing before bitmap access, releases partial results on failure, and unloads Uniscribe during renderer shutdown. Existing UI callers continue using the legacy renderer until the separate integration PR.

Validation

• Clean VC6 Release builds of Generals and Zero Hour.
• Local x86 checks: 24 direction cases and 20 layout/pixel comparisons against native whole-paragraph Uniscribe output, with zero failures.
• Local probes for font/cluster boundaries, sizing, failure cleanup, repeated rasterization, and loader unload/reload.
• git diff --check.

The native harnesses are local validation tools and are not included in this PR.

Implementation developed with AI assistance.

[qodo-free-for-open-source-projects]

PR Summary by Qodo

Add Uniscribe complex text shaping and rasterization

✨ Enhancement 🕐 40+ Minutes

AI Description

• Adds Uniscribe shaping, bidirectional layout, font fallback, measurement, and rasterization.
• Enforces shared size limits, allocation checks, GDI synchronization, and failure cleanup.
• Generates supplementary Unicode direction data and unloads Uniscribe during shutdown.

Diagram

graph TD
A["UTF-16 Text"] --> B["Complex Detection"] --> C["Paragraph Direction"] --> D["Run Itemization"] --> E["Font Clusters"] --> F["Uniscribe Layout"] --> G["GDI Bitmap"] --> H["16-bit Raster"]

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. DirectWrite shaping and rasterization

• ➕ Provides a newer text stack with integrated fallback and broad script support.
• ➕ Could reduce manual GDI and Uniscribe run management.
• ➖ Conflicts with the legacy VC6 and GDI-oriented renderer environment.
• ➖ Would require larger renderer and platform compatibility changes.

2. HarfBuzz and FreeType pipeline

• ➕ Offers a portable shaping and rasterization path across supported platforms.
• ➕ Avoids dependence on legacy Windows Uniscribe behavior.
• ➖ Introduces substantial dependencies and build-system integration work.
• ➖ Requires font discovery, fallback, metrics, and raster compatibility to be redesigned.

Recommendation: Retain the Uniscribe-based approach for this PR. It fits the existing Windows GDI font pipeline, preserves VC6 compatibility, and limits integration risk while still handling mixed directionality and cluster-safe font selection. DirectWrite or HarfBuzz would be preferable only as part of a broader renderer modernization.

Files changed (10) +838 / -10

Enhancement (5) +471 / -10

complextext.hImplement Uniscribe run layout and drawing +228/-0

Implement Uniscribe run layout and drawing

• Introduces RAII-managed complex text layouts that itemize paragraphs, preserve bidi state, split fonts at cluster boundaries, shape runs, calculate shared metrics, and reorder them visually. Draws runs on a common baseline and flushes GDI before bitmap access or cleanup.

Core/Libraries/Source/WWVegas/WW3D2/complextext.h

render2dsentence.cppAdd complex text measurement and rasterization +147/-8

Add complex text measurement and rasterization

• Adds Windows-only complex-text detection and a FontCharsClass path for measuring or rasterizing shaped paragraphs with consistent size and overflow limits. Rasterization uses a 24-bit DIB, converts intensity to the existing packed font format, and cleans up partial GDI resources on failure; pixel conversion is shared with legacy glyph storage.

Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp

render2dsentence.hExpose complex text APIs on FontCharsClass +4/-1

Expose complex text APIs on FontCharsClass

• Declares complex-text detection, measurement, and optional raster generation. Removes the unused cached CharAscent member because complex layouts derive ascent per shaped run.

Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.h

unicodebidi.hDetermine UTF-16 paragraph direction +84/-0

Determine UTF-16 paragraph direction

• Implements first-strong paragraph-level detection with directional-isolate handling. Uses Windows classification for BMP characters and a binary-searched Unicode table for supplementary code points.

Core/Libraries/Source/WWVegas/WW3D2/unicodebidi.h

Usp10Loader.hDefine Uniscribe itemization control state +8/-1

Define Uniscribe itemization control state

• Replaces the ScriptControl forward declaration with the compatible bitfield structure required to pass non-null itemization control data for full bidirectional analysis.

Core/Libraries/Source/WWVegas/WWLib/Usp10Loader.h

Documentation (1) +39 / -0

unicode-license.txtInclude the Unicode data license +39/-0

Include the Unicode data license

• Adds Unicode License V3 covering the generated bidirectional class data included by the renderer.

Core/Libraries/Source/WWVegas/WW3D2/unicode-license.txt

Other (4) +328 / -0

CMakeLists.txtRegister complex text and bidi support files +3/-0

Register complex text and bidi support files

• Adds the complex layout header, Unicode direction helper, and generated supplementary bidi table to the WW3D2 source list.

Core/Libraries/Source/WWVegas/WW3D2/CMakeLists.txt

supplementarybidi.inlAdd generated supplementary bidi direction ranges +265/-0

Add generated supplementary bidi direction ranges

• Provides Unicode 17.0.0 ranges for supplementary right-to-left and neutral code points, with omitted ranges defaulting to left-to-right. The table supports first-strong paragraph direction detection for complete UTF-16 characters.

Core/Libraries/Source/WWVegas/WW3D2/supplementarybidi.inl

ww3d.cppUnload Uniscribe during renderer shutdown +6/-0

Unload Uniscribe during renderer shutdown

• Releases the lazily loaded Uniscribe module when WW3D shuts down, allowing clean resource teardown and subsequent reloads.

Core/Libraries/Source/WWVegas/WW3D2/ww3d.cpp

generate_supplementary_bidi.pyGenerate and verify supplementary bidi ranges +54/-0

Generate and verify supplementary bidi ranges

• Adds a reproducible Unicode 17.0.0 data generator with source version and SHA-256 validation. It emits compact non-left-to-right ranges and supports checking the committed table for drift.

scripts/generate_supplementary_bidi.py

[qodo-free-for-open-source-projects]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)

Great, no issues found!

Qodo reviewed your code and found no material issues that require review

Tip of the day

💡 Did you know, you can ask Qodo to dismiss a finding you disagree with, with your reason on record

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

[greptile-apps]

Greptile Summary

This PR adds Windows Uniscribe-based complex-text detection, paragraph itemization, bidirectional run ordering, font-aware shaping, and rasterization to FontCharsClass.

• Computes paragraph direction across BMP and supplementary UTF-16 characters.
• Shapes font runs and draws them on a shared baseline into an ARGB4-compatible raster.
• Adds checked Unicode bidi data generation and unloads the Uniscribe loader during renderer shutdown.
• Preserves the legacy rendering path until a subsequent integration change.
• Two repository prologue/comment requirements remain to be addressed.

Confidence Score: 4/5

The implementation has no established behavioral defect, but the explicit repository requirements for current-year comments and GPL source prologues must be satisfied before merging.

The shaping, allocation, cleanup, and loader-lifecycle paths appear internally consistent; the remaining findings are limited to two concrete repository-rule violations in generated and generator-file prologues.

Files Needing Attention: Core/Libraries/Source/WWVegas/WW3D2/supplementarybidi.inl, scripts/generate_supplementary_bidi.py

Important Files Changed

Filename	Overview
Core/Libraries/Source/WWVegas/WW3D2/complextext.h	Implements itemization, cluster-aware font runs, Uniscribe shaping, visual ordering, baseline alignment, and cleanup.
Core/Libraries/Source/WWVegas/WW3D2/render2dsentence.cpp	Exposes complex-text detection, measurement, and checked bitmap rasterization through FontCharsClass.
Core/Libraries/Source/WWVegas/WW3D2/unicodebidi.h	Determines the first-strong paragraph level over BMP and supplementary UTF-16 characters.
scripts/generate_supplementary_bidi.py	Reproducibly generates the supplementary bidi table but lacks the required GPL prologue.
Core/Libraries/Source/WWVegas/WW3D2/supplementarybidi.inl	Adds generated Unicode 17 bidi ranges but violates the date-comment and GPL-prologue requirements.
Core/Libraries/Source/WWVegas/WWLib/Usp10Loader.h	Extends the dynamically loaded Uniscribe API declarations needed for itemization, layout, and rasterization.
Core/Libraries/Source/WWVegas/WW3D2/ww3d.cpp	Unloads the synchronized Uniscribe loader during renderer shutdown.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[UTF-16 text] --> B[Determine first-strong paragraph level]
    B --> C[ScriptItemize paragraph]
    C --> D[ScriptBreak cluster boundaries]
    D --> E[Split runs by item and selected font]
    E --> F[ScriptStringAnalyse each run]
    F --> G[ScriptLayout visual run order]
    G --> H{Measurement only?}
    H -- Yes --> I[Return width and height]
    H -- No --> J[Draw runs on shared baseline]
    J --> K[GdiFlush]
    K --> L[Convert bitmap intensity to font raster]

Loading

Prompt To Fix All With AI

### Issue 1
Core/Libraries/Source/WWVegas/WW3D2/supplementarybidi.inl:3
**Outdated Copyright Year**

This newly generated comment says `Copyright 2025 Unicode, Inc.`, which violates the repository directive that newly created code comments must not reference dates before 2026. The generator emits the same outdated year, so it and the generated file must be updated before merging.

### Issue 2
scripts/generate_supplementary_bidi.py:1
**Missing GPL Prologue**

This new Python source file begins with a module docstring instead of the GPL license header required for new source files. Add the required prologue here and to the generated `supplementarybidi.inl` file, ideally through the generator, before merging.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

_{Reviews (1): Last reviewed commit: "feat(text): Add complex text shaping and..." | Re-trigger Greptile}