Harden Foundations memory and concurrency contracts

[tdecroyere]

Goal

Harden the low-level Foundations runtime around explicit memory, lifetime, and concurrency contracts. This keeps the existing arena/stack design intact while fixing correctness bugs that could otherwise lead to out-of-bounds allocations, races, stale handles, corrupted free lists, or unsafe access to uncommitted memory.

MemoryArena

• remove public SystemPopMemory and keep rollback internal to StackMemoryArena;
• keep SystemClearMemoryArena / SystemFreeMemoryArena as explicit exclusive lifetime operations;
• page-align the allocator header, keep it permanently committed, and make page metadata describe data pages only;
• calculate page indexes relative to the data region and remove floating-point page calculations;
• make system page-size initialization thread-safe without cross-translation-unit initialization-order dependence;
• replace the shared push check-then-add race with a bounded CAS reservation loop;
• fix StackExtraStorage capacity checks to use the actual working arena;
• serialize shared commit/decommit bookkeeping before mutating PageInfo, committed-state bits, or page counts;
• make platform reserve/commit/decommit accounting atomic and only update counters after successful OS operations;
• account for still-committed pages when an arena reservation is released;
• reject size/alignment/multiplication overflow instead of allowing wrapped reservations or array sizes;
• propagate allocation failures through zero/array/string helpers instead of dereferencing null storage;
• make SystemCommitMemory report success/failure and prevent committed pushes from returning a pointer when the platform commit fails;
• make SystemClearMemoryArena directly decommit committed data pages and reset all page tracking instead of relying on conservative range bookkeeping;
• preserve intentionally conservative per-page min/max tracking for normal sparse commit/decommit operations to avoid heavier fragmentation metadata;
• align typed array/struct allocations to alignof(T) while keeping raw byte pushes on the Foundations default alignment;
• make alignment padding explicit arena capacity consumption and include it in allocated-byte accounting;
• document typed pushes as raw storage only: no constructors, destructors, copy constructors, or other hidden object-lifetime behavior;
• keep raw buffer copy/duplicate/concat helpers byte-copy based and document that element types must be safe for byte-wise copying;
• make wide-character duplication follow the same logical-length + hidden null-terminator convention as character strings;
• make POSIX decommit replace committed pages with fixed anonymous PROT_NONE mappings so physical backing is released while the virtual address range remains reserved;
• release thread-local stack arena backing storage automatically when a thread exits, including any lazily-created StackExtraStorage, through a small internal TLS wrapper;
• preserve the existing advanced nested StackMemoryArena lifetime model and its regression coverage.

DataPool

• replace the unsafe ABA-prone lock-free free list with a short synchronized index allocator/recycler;
• bound concurrent index allocation so the pool cannot advance past capacity;
• make concurrent removal of the same handle recycle the slot exactly once;
• validate generations with atomic loads and skip the reserved invalid generation value on wrap;
• propagate backing-memory commit failures before publishing/writing a new item;
• document that lookups are thread-safe but returned raw pointers do not pin item lifetime against later remove/reuse.

Dictionary

• replace the unsafe concurrent linked-list/free-list mutation scheme with deterministic synchronized dictionary operations;
• initialize hash/value/next before publishing an entry into its bucket;
• bound entry allocation under contention and remove timing-dependent retry loops;
• prevent removed entries from being recycled while another dictionary operation is traversing them;
• fix ReadOnlySpan<T> hashing so Length is converted to bytes with sizeof(T);
• make missing SystemGetDictionaryValue return nullptr as documented while preserving the zero-value operator[] convenience;
• document raw pointer/reference lifetime limitations after lookup.

Span and string buffers

• keep Span / ReadOnlySpan lightweight, non-owning pointer-and-length views;
• make const-buffer construction and slicing const-correct;
• define character span Length as logical characters excluding the null terminator;
• fix SystemDuplicateBuffer<char> to preserve that logical length;
• make SystemDuplicateBuffer<wchar_t> follow the same null-terminated backing-storage convention;
• make the shader serializer write its null terminator explicitly instead of depending on the previous incorrect string length.

Regression coverage

Adds focused tests for:

• arena size/push overflow and exact VM accounting after free;
• clear after partial page commitment, including clean reuse/decommit of the same page;
• typed alignof(T) allocation after a deliberately misaligned bump position;
• concurrent typed alignment under CAS contention;
• concurrent push capacity and shared-page commit;
• StackExtraStorage ancestor allocations;
• short-lived thread cleanup of both primary stack arena storage and nested StackExtraStorage backing;
• commit result validation;
• DataPool capacity, duplicate concurrent remove, and thousands of concurrent remove/reuse cycles;
• Dictionary capacity, concurrent remove/reuse cycles, missing lookup semantics, and non-char span hashing;
• Span/string logical length and null termination for both char and wchar_t duplication.

Intentional follow-ups outside this PR's current Foundations scope: platform-specific descriptor/query free lists in the graphics backends.