Skip to content

fix(runtime): consolidate patch correctness and retire unsafe shortcuts - #202

Open
f0rr0 wants to merge 13 commits into
mainfrom
f0rr0/consolidate-patch-correctness
Open

f0rr0 wants to merge 13 commits into
mainfrom
f0rr0/consolidate-patch-correctness

Conversation

@f0rr0

@f0rr0 f0rr0 commented Sep 7, 2026

Copy link
Copy Markdown
Owner

Latest update — 41c74d8 (2026-09-08)

Not merge-ready. The Windows/MSVC exception-recovery blocker remains. The new execution-stack guard is a tested review candidate, not enabled in consumer packages.

This update pushes the completed resource-budget cleanup and the actual stack-safety candidate code:

  • Centralize existing transport/link-time budgets, preserve their values, and document ownership, tradeoffs, PostgreSQL/PGlite comparisons and the distinction between linear and execution stacks. No new environment flag or larger production stack.
  • Add four candidate patches and focused regression probes: Wasmer VM accounting/EH guard, fixed Wasmer API query, PostgreSQL early check, and Rust import registration. Each passes clean application against its documented base.
  • Keep those patches outside production series and dependency selection. Published Wasmer 7.2.1 lacks the new API; Windows/browser contracts and recovery margins are not qualified. This PR does not pretend to deliver the new guard to downstream consumers yet.
  • Remove ordinary-host-call stack snapshots from the candidate. Its successful fixed query has no allocation, lock, arbitrary callback or host-stack switch. Missing accounting and the 64 KiB engine floor fail terminally; PostgreSQL retains a separate 512 KiB early-recovery reserve and the engine retains its 1 MiB default.
  • Preserve code and evidence without importing binaries, producer trees, the benchmark framework or historical result archives.

Candidate evidence, not exact-head release qualification

  • Wasmer VM: 62/62 debug and 62/62 release tests; baremetal compile check.
  • 1,000-cycle scalar ABI/nested-stack probe; 100 serialized-AOT deep throw/catch and rethrow cycles, followed by expected terminal stack failure.
  • Real PostgreSQL memory/directory recovery: repeated SQLSTATE 54001, subsequent queries, PL exception catches, savepoint rollback and directory reopen.
  • 144 successful benchmark children against the retained previous guarded direct-binding candidate, with identical guest/AOT and SDK dependency versions. RTT improved 2.6% memory and 3.6% in the directory follow-up. The initial +8.2% directory temporary-INSERT warning did not reproduce at that magnitude: the eight-pair follow-up measured +1.7%; COPY measured −2.0%. Small overheads remain uncertain; no blanket non-regression claim.
  • PR preparation: pinned Moon repo:docs-policy sdk-contracts:all passed (9 tasks, 5 cached), profile-budget test passed, both pq_flush ABI tests passed, regression-probe Rust formatting passed, all four candidate patches passed git apply --check, and staged whitespace checks passed.

Before activation: settle/consume the upstream VM/API contract (coordinate Wasmer #6913), qualify recovery margins and error cleanup across supported targets, resolve the Windows/browser contract, then integrate pins/imports/artifact invalidation together. No release build or exhaustive qualification was rerun merely to publish this review update.


Summary

Previously published checkpoint: 0f4a996. Not merge-ready: Windows/MSVC Wasm exception recovery is blocked in the consumed Wasmer runtime. The known formatting and Postmaster source-root fixes are pushed. No Windows target/test is removed or waived, and the repeat optimization remains held outside this PR.

Consolidate the completed patch-correctness work on current main, without importing the research harness, historical result archives, old binaries or producer trees.

  • Recover Native trusted-session, startup/cwd cleanup, host process-boundary and COPY/cancellation fixes while preserving main's ABI 10 and public SDK APIs.
  • Replace WASIX host-longjmp symptom handling with live guest-local recovery and typed outcomes. Integrate the matching Rust/browser host ABI, bounded protocol/tool output, entropy and fail-closed lifetime/transport changes.
  • Remove 13 rejected embedded WASIX patches, including LIKE, int4 comparator shortcuts, semaphore reset and scalar atomics. Preserve the seek, parser, WAL-sync and JSONB optimizations already merged on main.
  • Use the fixed strict-memory AOT profile llvm-opta-ro_ftable throughout production, caches, manifests and all eight carriers; reject incompatible legacy overrides and artifacts.
  • Clean the ordered Native (22), embedded WASIX (31), Postmaster PostgreSQL (7), and browser-host (27) patch series, with patch-local motivation and focused checks. The Postmaster runtime has separate ordered Wasmer (9) and libc (8) series. Do not claim the inherited runtime bundle is fully qualified or upstream-ready.
  • Align generated-header installation/fingerprints, ABI export policies, Moon cache inputs and the existing core-only fast-path packaging check.

This is intentionally a substantial correctness integration, not a small isolated performance patch: guest/host ABI and Native process ownership must land together. It excludes optional compiler/runtime experiments and the obsolete pre-N-API Node filesystem path.

Follow-up implemented at 51789da

  • Preserve PostgreSQL's Native fsync=on default instead of silently adding -F; explicit fsync=off remains available. Fresh Linux ICU direct and installed-broker checks verify both settings, identity/login policy, session reset and nondefault-database settings.
  • Fix the ICU collation import failure without permitting subprocess execution, and move the signal-mask declaration to its type-owning header to fix Windows runtime/extension compilation.
  • Replace the Postmaster monoliths with 8 Wasmer + 7 libc logical patches. Strict application produces exactly the previous source trees. Separate Wasmer0009 disables unsafe nonvolatile-memory assumptions in all four compiler routes and invalidates old artifact identities.
  • Postmaster source/shell units and both freshly compiled product/CLI strict-memory identity tests pass. The corrected-profile hosted Postmaster producer/qualification at 51789da also passed (job); the subsequent libc0008 full carrier remains subject to the new head's CI.
  • SDK header mirrors, Clippy and the build graph failures are corrected. The repository-pinned Moon toolchain runs locally; SDK contracts and graph tests pass.

Additional follow-up at 8e45be7

  • Initialize the WASIX host-selected role as the real PostgreSQL session principal during startup. Enforce catalog login/admission, role/database defaults and login triggers; prevent RESET ROLE/DISCARD ALL from restoring bootstrap-superuser rights. Delete the late SET ROLE workarounds from Rust direct/proxy and browser direct/worker paths.
  • Preserve typed startup SQLSTATE/details for Rust SDK callers and the exact original error bytes for proxy clients, using the existing decoder.
  • Make libc sigsetjmp evaluate both arguments once while keeping setjmp in the live caller. C/C++ O0/O2 host checks and actual C/C++ Wasm O2 execution pass; the old macro fails the negative control. The new helper/header was tested over the retained EH sysroot on a freshly built strict Wasmer runner, not mislabelled a full sealed-carrier rebuild.
  • Add inherited memcmp alignment, byte-order and protected-page boundary checks; no new speedup is claimed. Update ordered-series source locks and expose the newly established POSIX signal-mask limitation truthfully.
  • Reuse the active AOT-enabled serializer executable instead of rebuilding a shared bare xtask path which another worktree can replace. No new public environment flag or research framework.

Hosted failure fixes prepared at 94d674d

  • Initialize and consistently propagate the Postmaster libc source root before capability validation. A no-build behavioral test executes the real script prefix/inventory with default/custom roots and verifies missing-libc rejection.
  • Correct the single Biome formatting failure. The full 263-file JS formatting path list now passes locally using the exact pinned Biome version.
  • These fixes plus profile/experiment, timeout and Windows-blocker documentation are pushed at 0f4a996. All material producers finished before the push: iOS extensions and Linux/macOS AOT passed on 8e45be7; Windows serialized/validated/installed AOT successfully but crashed during SQL-error recovery in its Rust regression suite. Neither the failed parent run nor its successful individual jobs are represented as exact-head green qualification.

Final focused validation

  • Fresh portable guest/support/initdb, standard and ICU seeds, strict AOT packaging/artifact checks pass. Strict PostgreSQL replay: 22 Native / 31 WASIX / 7 Postmaster, no fuzz/rejects.
  • Rust on those fresh assets: 211 library tests, 19 runtime tests, 7 proxy tests, 7 PostgreSQL regression tests pass with extensions enabled. This is a different feature scope from the initial tools-enabled test count below.
  • The final tools-enabled configuration additionally passes 223 library tests, 7 public-API tests and 5 doctests.
  • Real Chrome memory/IndexedDB recovery and COPY floor plus configured-role direct→worker→direct reopen, defaults, login triggers, reset and privilege/NOLOGIN rejection pass. TypeScript 343 tests / 36 files, binding and example typechecks pass.
  • SDK contracts, graph unit tests, WASIX source/artifact guards, Postmaster canonical source/shell units and source lock verification pass. Rust formatting and Clippy pass for WASIX with/without extensions and xtask with/without the serializer feature.
  • Final benchmark: 220 independent children, all correctness and actual durability/version oracles pass. Prior corrected candidate and final identity candidate retain matching RTT; final/strict-main RTT is +3.5% memory / +2.3% directory. Historical narrow-directory ~10% warning did not repeat, but final/prior wide temporary INSERT warns +5.1% memory / +4.2% directory. No blanket non-regression claim.
  • Stock PostgreSQL 18.4 context still shows final narrow INSERT 1.41x memory / 1.64x directory, wide INSERT about 1.60x, and mixed OLTP 1.35x / 1.51x. Two stock-server process replicates and differing transports make this descriptive context, not paired cross-engine proof. The checked-in report contains the full nine-case table, scale, settings and provenance. Equal affinity was not exclusive CPU isolation; unrelated shared-host activity remained.

Initial 34f173f validation

  • Strict PostgreSQL 18.4 patch replay: 22 Native / 30 WASIX / 7 Postmaster, no fuzz or rejected hunks.

  • Fresh Linux standard Native library: ABI conformance, configured identity/login policy, six startup rejection cases, signal/mask/timer ownership, cancellation and COPY timeout/backpressure, five cwd cases and broad C smoke passed.

  • Rust WASIX: 215 unit tests, 5 doctests, 7 public API tests passed.

  • Browser TypeScript: 343 tests / 36 files and typecheck passed; six pinned-host source-contract tests passed.

  • Real isolated Chrome: memory and IndexedDB both passed repeated syntax/division error recovery, PL/pgSQL exception catch, set-based INSERT, 1 MB COPY callback abort/drain/reuse; IndexedDB close/reopen preserved all 1,000 rows.

  • Fresh portable guest/support/initdb built; standard and ICU seed initialization/profile probes passed. Strict AOT compiled, packaged and passed artifact validation; all eight actual generated carrier identities were checked.

  • Shared protocol schema/generated views, compiled C bridge ABI, 64 MiB aggregate tool-output policy, SJLJ artifact guard, patch-review freshness, Postmaster source lock, focused libc/runtime checks, Rust formatting and documentation policy passed.

  • assets verify-committed passed using the existing core-only build flag; full extension catalog validation remains the default outside that explicitly scoped path.

  • Fresh Rust WASIX on the new portable guest and strict AOT: 18/18 runtime-smoke and 7/7 PostgreSQL regression tests passed. These exercise memory/directory storage, sync/async callback error and panic recovery/reuse, prepared parameters, INSERT/savepoints, rollback/cancellation, directory ownership/reopen, backup/restore and recovery after an abruptly killed child. These are focused cases, not exhaustive crash qualification. Rust COPY-specific coverage was not available in these suites; actual browser COPY coverage is listed above.

Boundaries and follow-up

  • Windows/MSVC is a merge blocker. Job 101905963703 reaches an explicit panic in wasmer-vm 7.2.1's MSVC throw() implementation during savepoints_error_recovery_and_indexed_updates, exit 0xe06d7363. Official v7.4.0 and current upstream main still contain the unsupported MSVC exception stubs; an ordinary dependency upgrade is not a fix. Genuine engine exception support or a separately qualified compatible guest is required. Skipping tests, catching the panic or silently falling back to Native would not preserve the guest recovery/isolation contract.

  • The retained repeat experiment is held, not promoted: 184 correctness/settings-checked children show wide temporary INSERT 41.5% / 33.5% faster and logged wide 29.0% / 20.7% faster (memory/directory). A separate 72-child targeted+A/A recheck resolved the preconstructed-directory warning but retained a prepared-directory warning of approximately 6–7% against both baseline controls, predominantly in transaction body. Shared-host interference remains a limitation. Exact candidate artifacts and original baselines remain retained; no experimental framework is staged, and neither durability nor fill-chunk tuning is a demonstrated remedy for the warning.

  • SQL statement_timeout is not a reliable CPU-bound deadline in current Rust WASIX. Actual baseline/candidate probes expose inherited libc timer-field and Rust unit/one-shot/delivery gaps. Native timeout/reuse and injected interrupt tests do not prove Wasm delivery. Public Rust/usage docs now explicitly distinguish accepting the GUC or timing out a caller future from stopping guest execution; issue Follow up patch consolidation: signal semantics, performance profiling, and qualification #201 records the coordinated remedy.

  • No release-ready or blanket non-regression claim. The initial 324-child strict-profile comparison and final 220-child identity screen are separate evidence sets; the later one does not erase the earlier warnings. All accepted directory reports actually verify fsync/synchronous_commit/full_page_writes=on. No Native durability-change timing is inferred from the WASIX screen.

  • Inherited Postmaster POSIX signal masks are not implemented. Pinned pthread_sigmask returns success/no-op, sigpending returns EINVAL, and handler sa_mask is ignored. The capability inventory now says unsupported. The macro fix does not repair delivery masking; correct remediation needs coordinated Wasmer/libc mask/pending delivery semantics and PostgreSQL stress tests. This is a high-priority known limitation, not an observed corruption or new-regression claim.

  • Four completed prior/final ABBA profile captures collected 45,283 samples on unchanged strict guest/AOT payloads, using existing public Wasmer function extents for diagnostic address maps. The wide workload's repeat(chr(...),96) makes 96 one-byte copies per row; imported-copy self samples account for 24–27%, while direct statistics-counter samples are about 0.45%. This supports a bounded general repeat experiment, not causal attribution of the +5.1% warning. No new optimization is promoted. The user-authorized temporary profiling setting was restored to its original perf_event_paranoid=4 immediately after capture; no persistent setting changed.

  • Hosted Native builds and the portable WASIX, Postmaster and iOS-extension producers passed at parent 51789da. The later 8e45be7 head also passed Native/extension and portable WASIX producers plus Linux/macOS AOT, but exposed the known formatting/root failures and Windows exception blocker. That parent run was cancelled by the push only after material producers finished, with app/E2E work remaining. 0f4a996 must qualify in its own run 34177975360; parent results are not relabelled as final-head results.

  • Native now preserves fsync=on by default; durable-write latency can increase. Functional reopen tests still do not establish crash durability. No WASIX durability relaxation is introduced.

  • Guest and hosts/AOT must be rebuilt together. Old profile manifests cannot be relabelled; terminal guest failures cannot be treated as recoverable SQL errors.

  • Browser host runtime code was built before mail-header normalization. Every patch diff is byte-identical before/after that metadata-only step; the original build marker is retained and a local before/after equivalence receipt records both identities. This is not labelled an exact-release artifact.

  • Local Moon affected-project resolution, SDK contracts and graph unit tests pass with the repository-pinned toolchain.

  • Broad platform/ICU, extension, hostile-extension/resource, crash and release qualification remain separate work.

The complete prioritized remainder is issue #201: inherited Postmaster signal-mask semantics, a profile-supported repeat experiment and unresolved wide-INSERT warning, remaining platform/runtime qualification and upstream review. Optional compiler/ICU/ownership ideas are conditional research, not mandatory features. Rejected CRC/lazy-globals/unsafe shortcuts are not automatically revived.

See docs/internal/PATCH_CONSOLIDATION_20260907.md for the consumer implications and explicit retirement table. The original research state is preserved in a separate local recovery commit and is not staged here.

@vercel

vercel Bot commented Sep 7, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
oliphaunt-docs Ready Ready Preview Sep 8, 2026 8:56pm UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant