Vertex work on the GPU, lifter fixes for the game-window freezes, CEF window docs

README.md

@@ -10,13 +10,15 @@ server streams your game files to your own browser only.
 
 ## What works
 
-Against a local NosCore stack, in Chromium:
+Against a local NosCore stack, in Chromium (Chrome or Edge 137 or later): the browser build needs WebAssembly
+JavaScript Promise Integration (`-sJSPI` in `recomp/wasm/compile_all.py`), which is how the game yields to the browser
+once per frame. Browsers without it, such as Safari 26 and earlier, cannot start it.
 
 - login, server and channel selection, character selection, entering the world;
 - walking, chat, the game windows (inventory, skills, ...), GM teleports between maps;
 - Direct3D 9 rendering on WebGL2 (fixed-function texture stages, lighting, fog, texture-coordinate generation),
   GDI text through the browser's font engine, Miles audio through WebAudio;
-- about 50-60 frames per second in town.
+- about 60 frames per second in town at 1080p on an integrated GPU.
 
 Every one of the client's 19,436 functions is recompiled (90.4% of the code section's bytes; the rest is data the
 compiler keeps there: VMTs, RTTI, jump tables, strings). What is still open is listed in
@@ -38,7 +40,7 @@ NosCore.exe ──IDR + Ghidra──▶ function list + names (symbols/<md5>/)
   semantics.
 - **Portable host** (`recomp/port`): the Windows the client expects, implemented on browser primitives: WebGL2 for
   Direct3D 9, OffscreenCanvas for GDI text, WebAudio for Miles, SharedArrayBuffer rings for input and sockets (the
-  game loop never yields to the browser).
+  game loop yields to the browser only at Present, once per frame, when it draws straight into the page's canvas).
 - **Native host** (`recomp/runtime`, MSVC): the same generated C as a Windows executable, useful for debugging.
 
 The full story, every gotcha found so far, and how to continue are in [docs/STATUS.md](docs/STATUS.md).

docs/STATUS.md

@@ -29,7 +29,7 @@ python recomp\web\serve.py --game "C:\Program Files (x86)\Nostale" --user <accou
 ```
 
 - **Worker:** the client runs in a Web Worker and its loop never yields, so everything it talks to is synchronous: input arrives through a SharedArrayBuffer ring, network through sync XHR, frames leave as ImageBitmaps.
-- **Rendering:** vertices are transformed on the CPU (`render.c` `fetch`); WebGL2 on an OffscreenCanvas (`render_gl.inc`) rasterises, evaluates two texture stages, alpha-tests, blends and depth-tests. Each distinct texture-stage, alpha-test and fog setup gets its own fragment program with that setup as constants: a software rasteriser (SwiftShader in headless Chrome) runs every branch of a shader that reads it from uniforms, which made it about six times slower. Textures upload once per Lock (object +56 is the dirty flag) and are mipmapped. `fill_caps` and the `Check*` methods describe only what this renderer does (two stages, eight lights, culling, scissor; no stencil, user clip planes, shaders, offscreen render targets, cube or volume textures), and creation calls refuse what the caps leave out. Triangle strips are flattened into lists with every odd triangle's first two vertices swapped, so culling sees D3D's winding. Host writes to the backbuffer (unlocking it, a GDI DC, StretchRect/UpdateSurface) reach the screen through `r_write_back`. Under Node the software rasteriser in `render.c` is still used. D3D9 samples pixels at integer coordinates: the GL path offsets by half a pixel, the software path samples at x, y.
+- **Rendering:** on a GPU, draws from vertex buffers are transformed, lit and fogged by the vertex shader (`render_gl.inc` `gl_buffer_draw`): each vertex and index buffer is uploaded once and again after a writable Lock, and the matrices, material and lights go as uniforms. Everything else (pretransformed vertices, DrawPrimitiveUP, dynamic or locked buffers), and every draw under a software rasteriser such as SwiftShader, where this path made small maps about 15% slower, is transformed on the CPU (`render.c` `fetch`). WebGL2 on an OffscreenCanvas rasterises, evaluates two texture stages, alpha-tests, blends and depth-tests. Each distinct texture-stage, alpha-test and fog setup gets its own fragment program with that setup as constants: a software rasteriser (SwiftShader in headless Chrome) runs every branch of a shader that reads it from uniforms, which made it about six times slower. Textures upload once per Lock (object +56 is the dirty flag) and are mipmapped. `fill_caps` and the `Check*` methods describe only what this renderer does (two stages, eight lights, culling, scissor; no stencil, user clip planes, shaders, offscreen render targets, cube or volume textures), and creation calls refuse what the caps leave out. Triangle strips are flattened into lists with every odd triangle's first two vertices swapped, so culling sees D3D's winding. Host writes to the backbuffer (unlocking it, a GDI DC, StretchRect/UpdateSurface) reach the screen through `r_write_back`. Under Node the software rasteriser in `render.c` is still used. D3D9 samples pixels at integer coordinates: the GL path offsets by half a pixel, the software path samples at x, y.
 - **Text:** `hle_text.c` tracks DCs, fonts and selected bitmaps; glyphs are rasterised by the worker's OffscreenCanvas 2D context. The client builds its own glyph atlas from `TextOutA` per character.
 - **Game files:** served by `serve.py` with HTTP Range support and mounted lazily, in lower case, at `/c:/nostale`.
 - **Network:** Winsock is relayed through `serve.py` (`/net/connect|send|recv|close`). Sockets the page stops polling are closed after 20 s; otherwise a reloaded page keeps the account logged in on the world server (`failc` AlreadyConnected). `--remap FROM=TO` redirects a port when :4000 is taken locally.
@@ -49,7 +49,7 @@ python recomp\web\serve.py --game "C:\Program Files (x86)\Nostale" --user <accou
 **Open (2026-10-28):**
 1. Gameplay not exercised yet: combat and skills, NPC dialogs and shops, trading, every UI window. Extend `drive.mjs`; each new feature may hit a missing API or rendering feature (the per-setup logs and the F9 frame dump show which).
 2. Speed is fine for testing; the next step would be vertex transform and lighting in a vertex shader (7-8 ms CPU per busy frame).
-3. CEF (NosMall, the in-game browser) is not loaded. Its windows (`TCEFWindowParent`) still open with an empty body: the Game Guide on F11, which the client also opens when the first NPC dialogue ends.
+3. CEF (NosMall, the in-game browser) is not loaded. Its windows (`TCEFWindowParent`) still open with an empty body: the Game Guide on F11, and the tutorial's `web` steps (`tutorial.dat`, such as script 1 step 20 `t01_basicmove` right after the first NPC dialogue).
 4. Android: Chrome on Android should run it once memory is reserved lazily and `serve.py` is reachable over HTTPS.
 
 ### WebAssembly (portable host), as of 2026-10-27

recomp/build.ps1

@@ -11,9 +11,11 @@ $obj = Join-Path $Out 'obj'
 New-Item -ItemType Directory -Force $obj | Out-Null
 $vcvars = 'C:\Program Files\Microsoft Visual Studio\18\Professional\VC\Auxiliary\Build\vcvars32.bat'
 
+# Generated code inlines helpers from the runtime header, so a newer header makes every object stale.
+$header = (Get-Item (Join-Path $runtime 'rt.h')).LastWriteTime
 $stale = Get-ChildItem $Gen -Filter *.c | Where-Object {
     $o = Join-Path $obj ($_.BaseName + '.obj')
-    $Full -or -not (Test-Path $o) -or (Get-Item $o).LastWriteTime -lt $_.LastWriteTime
+    $Full -or -not (Test-Path $o) -or (Get-Item $o).LastWriteTime -lt $_.LastWriteTime -or (Get-Item $o).LastWriteTime -lt $header
 }
 $list = Join-Path $Out 'stale.rsp'
 $stale | ForEach-Object { '"' + $_.FullName + '"' } | Set-Content -Encoding ascii $list

recomp/lift.py

@@ -881,6 +881,7 @@ def memval(op):
 
 
 PROLOGUE_REGS = {"ebp", "ebx", "esi", "edi"}
+SELF_SAVES = {f"{d}, {r}" for d in ("ebx", "esi", "edi") for r in ("eax", "edx", "ecx")}
 
 
 def reloc_code_seeds(L, fns):
@@ -940,11 +941,13 @@ def entry_shape(t):
     for t in sorted(L.reloc_targets):
         if t in stored or covered(t) or not entry_shape(t):
             continue
-        # An aligned pointer to a Delphi prologue is an entry however long the routine is before its ret.
+        # An aligned pointer to a Delphi prologue is an entry however long the routine is before its ret. A method
+        # that keeps Self opens with 'push ebx; mov ebx, eax'.
         head = L.disasm_range(t, t + 2)
         if t % 4 == 0 and len(head) == 2 and head[0].mnemonic == "push" and head[0].op_str in PROLOGUE_REGS and (
                 head[1].mnemonic == "push" and head[1].op_str in PROLOGUE_REGS
                 or (head[1].mnemonic, head[1].op_str) == ("mov", "ebp, esp")
+                or head[1].mnemonic == "mov" and head[1].op_str in SELF_SAVES
                 or head[1].mnemonic in ("add", "sub") and head[1].op_str.startswith("esp, ")):
             out.append(t)
             continue
@@ -1093,6 +1096,20 @@ def owned(x):
             "lds", "into", "sldt", "lcall", "ljmp", "retf", "iretd", "int1", "salc", "aam", "aad"}
 
 
+def skip_padding(L, start, end):
+    """Past the alignment filler after a function: single nop, int3 or zero bytes and whole 'mov eax, eax' (8B C0).
+    Skipping a lone 8B would start decoding inside the next function's first 'mov eax, [eax + n]'."""
+    while start < end:
+        b = L.code[start - L.code_lo]
+        if b in (0x00, 0x90, 0xCC):
+            start += 1
+        elif b == 0x8B and start + 1 < end and L.code[start + 1 - L.code_lo] == 0xC0:
+            start += 2
+        else:
+            break
+    return start
+
+
 def absorb_gaps(L, fns):
     """Unowned runs between functions that decode as clean code (no junk opcodes, ending in ret or jmp) are code
     only reached in ways static analysis misses. Each joins the function before it, with a label at its start."""
@@ -1102,8 +1119,7 @@ def absorb_gaps(L, fns):
     added = []
     for (lo1, hi1, k1), (lo2, _, _) in zip(spans, spans[1:]):
         start, end = hi1, lo2
-        while start < end and L.code[start - L.code_lo] in (0x00, 0x90, 0xCC, 0x8B, 0xC0):
-            start += 1
+        start = skip_padding(L, start, end)
         if end - start < 4:
             continue
         if sum(1 for a in range(start, end - 3) if a in L.reloc_slots) * 4 >= (end - start) // 2:

recomp/port/hle_d3d9.c

@@ -382,7 +382,10 @@ static void present_frame(void) {
     }
     r_prof_frame();
     if (++frames % 300 == 1) port_log("Present frame %u", frames);
-    if (!r_gl_present()) web_present(backbuffer);
+    if (!r_gl_present()) {
+        web_present(backbuffer);
+        web_frame_wait();
+    }
     if (shot_dir && (frames == 1 || frames % 600 == 0)) {
         char path[400];
         snprintf(path, sizeof path, "%s/frame_%06u.bmp", shot_dir, frames);
@@ -653,6 +656,7 @@ static void b_Lock(CPU* c) {
     if (!out || off > M32(THIS + 36) || (size && off + size > M32(THIS + 36))) { RET(D3DERR_INVALIDCALL); return; }
     M32(out) = M32(THIS + 28) + off;
     M32(THIS + 64)++;   /* buffer locks nest */
+    if (!(ARG(4) & 0x10)) M32(THIS + 56) = 1;   /* not D3DLOCK_READONLY: the GPU copy is stale */
     RET(0);
 }
 static void b_Unlock(CPU* c) {

recomp/port/port.h

@@ -125,3 +125,4 @@ void hle_d3d9_register(void);
 void hle_cursor_override(const uint8_t* rgba, int w, int h, int hx, int hy);
 void hle_cursor_restore(void);
 void web_present(uint32_t surface);
+void web_frame_wait(void);

recomp/port/render.c

@@ -730,6 +730,7 @@ static void log_state_combo(void) {
 }
 
 static void gl_draw_list(uint32_t prim, Vtx* v, uint32_t n);
+static int gl_buffer_draw(uint32_t prim, uint32_t n, uint32_t first, int32_t base_vertex, int indexed);
 
 /* Frame dump (F9 in the page): every draw of the next frame with its state, to debug a scene that renders wrong. */
 static int dump_state;   /* 1: armed, dump from the next Present; 2: dumping this frame */
@@ -787,7 +788,8 @@ static int reserve(Vtx** buf, uint32_t* cap, uint32_t n) {
 static void draw_impl_draw(uint32_t prim, uint32_t start, uint32_t count) {
     layout_begin();
     uint32_t n = vertex_count(prim, count);
-    if (!R.vb || !n || !R.vb_stride || !reserve(&vbuf, &vbuf_cap, n)) return;
+    if (!R.vb || !n || !R.vb_stride) return;
+    if (gl_buffer_draw(prim, n, start, 0, 0) || !reserve(&vbuf, &vbuf_cap, n)) return;
     uint32_t base = M32(R.vb + 28) + R.vb_offset;
     for (uint32_t k = 0; k < n; k++) fetch(base + (start + k) * R.vb_stride, &vbuf[k]);
     draw_list(prim, count, vbuf, n);
@@ -824,7 +826,8 @@ static void draw_indexed_cached(uint32_t prim, uint32_t count, uint32_t n, uint3
 static void draw_impl_draw_indexed(uint32_t prim, int32_t base_vertex, uint32_t start_index, uint32_t count) {
     layout_begin();
     uint32_t n = vertex_count(prim, count);
-    if (!R.vb || !R.ib || !n || !reserve(&vbuf, &vbuf_cap, n)) return;
+    if (!R.vb || !R.ib || !n) return;
+    if (gl_buffer_draw(prim, n, start_index, base_vertex, 1) || !reserve(&vbuf, &vbuf_cap, n)) return;
     int idx32 = M32(R.ib + 20) == 102;
     uint32_t ib = M32(R.ib + 28) + start_index * (idx32 ? 4 : 2);
     draw_indexed_cached(prim, count, n, M32(R.vb + 28) + R.vb_offset, R.vb_stride, ib, idx32, base_vertex);