issues: the all-CPU build waits on the frame allocator's lock, for the 2 MiB windows

issues/a-served-directory-costs-each-process-a-two-mebibyte-window.md

@@ -4,7 +4,7 @@ kind: tooling
 opened: 2026-09-27
 ---
 
-# A served directory costs each process a 2 MiB window, and nobody measured it
+# A served directory costs each process a 2 MiB window
 
 Every process that names a file under a served directory connects to that
 directory's server once and lends it a `toyos::fs::WINDOW_BYTES` region
@@ -13,7 +13,33 @@ touched, held for the process's life, and mapped into the server too. A
 desktop process that touches `/home`, `/config`, `/state` and `/apps` holds
 8 MiB of windows; fsd maps up to `MAX_SERVED` of them. Whether the window's
 size is paid for by the reads that fill it, and what the desktop's processes
-hold in windows together, has not been measured.
+hold in windows together, has not been measured. The size's stated reason,
+that 2 MiB is the size shared memory comes in, is gone: a region is whole
+4 KiB pages.
+
+**Creating it is most of what the all-CPU build's children add in CPU**
+(2026-11-11, [evidence](https://github.com/ToyOSOrg/evidence/tree/a22307591cfdefcf9fa1fef2ae5d1de79b926e9e/spawnperf-diag); the attribution is in
+`issues/spawning-a-process-costs-toyos-up-to-147-times-linuxs.md`). A build
+child makes four `SHM_CREATE`s beside its four directory connections, by
+inference from the SDK their windows. Each is 512 frames drawn one at a time
+under the frame allocator's one lock and zeroed: 95 µs a window on the T14 and
+133 µs on the AMD bench with one child at a time, where a child's four are 17
+and 19 % of its CPU. With one child a CPU, `SHM_CREATE` is 59 % (T14) and 72 %
+(AMD) of the growth in the children's CPU; of that, 47 % (T14) and 83 % (AMD)
+is spin on that lock
+(`issues/per-cpu-scheduler-and-pmm-state-shares-lines-every-cpu-writes.md`),
+and the T14's other 231 ms is not split. How much of the build's wall-clock
+loss against Linux this is has not been measured; that it is most of it is an
+inference.
+
+**Decided** (the orchestrator, 2026-11-11), third of three: the window's size
+waits for the file-call work. Once a served file's reads, writes and
+attributes go through its paged object (PR #1258 and the work after it), the
+window carries paths, small answers and the writes a holder's admission
+refuses, and is cut to what that needs. Before that, its creation costs what
+its region's first-touch draw makes it
+(`issues/a-shared-memory-region-draws-and-zeroes-every-frame-when-it-is-made.md`).
 
 **Exit**: the resident memory of a desktop boot's processes with and without
-their windows, measured, and the window's size justified from it or cut.
+their windows, measured, and the window's size justified from it or cut once
+the file-call work carries reads and writes on the paged object.

issues/a-shared-memory-region-draws-and-zeroes-every-frame-when-it-is-made.md

@@ -0,0 +1,40 @@
+---
+status: assigned
+kind: defect
+opened: 2026-11-11
+---
+
+# A shared-memory region draws and zeroes every frame when it is made
+
+`SharedMemObject::create` (`kernel/src/object/shm.rs`) draws every frame of a
+region and zeroes it before the call returns, so a region costs its whole size
+in time and memory however little of it is touched, where a process's private
+memory takes a frame at each page's first touch. On the spawn path it is a
+build child's `SHM_CREATE`: 95 µs a call on the T14 and 133 µs on the AMD bench
+with one child at a time, a child's four 17 and 19 % of its CPU, and most of
+what the all-CPU build's children add in CPU; by inference from the SDK, those
+four are the 2 MiB windows of its directory connections
+(`issues/a-served-directory-costs-each-process-a-two-mebibyte-window.md`,
+[evidence](https://github.com/ToyOSOrg/evidence/tree/a22307591cfdefcf9fa1fef2ae5d1de79b926e9e/spawnperf-diag)).
+
+**Decided** (the orchestrator, 2026-11-11), second of three after the batched
+draw (`issues/per-cpu-scheduler-and-pmm-state-shares-lines-every-cpu-writes.md`):
+a region's frames are drawn at its pages' first touch. The whole region is
+promised when it is made, so it is refused then and never at a touch. Two
+kinds stay drawn whole when they are made: a device's run, as a device needs,
+and an inbox's rings, which the kernel writes in interrupt context, where no
+frame may be drawn. One draw serves both kernel-made zeroed frames for a
+touched hole, `pmm::alloc_promised`: a region's page, drawn by
+`SharedMemObject::page()` into the region's own slot for that page, and a
+touched hole of a MEMORY volume's paged object (mapped paged objects, 7b in
+`issues/process-memory-in-4-kib-pages-still-copies-each-programs-text.md`),
+drawn into the pager's index under its lock. A region's slot is never revoked
+and a pager's page is, so the two keep their own stores. The module header's
+"a fault on a mapping of one never allocates" changes with it.
+
+**Assigned**: the spawn-path work, after the batched draw lands.
+
+**Exit**: a region's creation draws no frame and a first touch draws one,
+promised whole at creation and refused only there, but for a device's run and
+an inbox's rings, which are drawn whole when made; and a build child's
+`SHM_CREATE` cost read again.

issues/per-cpu-scheduler-and-pmm-state-shares-lines-every-cpu-writes.md

@@ -52,6 +52,33 @@ benches.
 A fourth, `IN_PASS`, was measured with them and now sits in its CPU's own
 record (`kernel/src/cpu_local.rs`), a line no other CPU writes.
 
-**Exit**: that attribution, posted, and read again after the change; and
-each of the three either written per CPU on a line no other CPU writes, or
-kept with its refusal reason at its site.
+**The frame allocator's lock is what the all-CPU build waits on**
+(2026-11-11, [evidence](https://github.com/ToyOSOrg/evidence/tree/a22307591cfdefcf9fa1fef2ae5d1de79b926e9e/spawnperf-diag); the attribution is in
+`issues/spawning-a-process-costs-toyos-up-to-147-times-linuxs.md`). Under the
+build's 256 children, one a CPU, it is 99 % of every contended lock wait on
+both benches: 418,000 contended acquisitions and 243 ms of wait a round on the
+T14 (`j8`), 1.05 of its 8 CPUs for the whole round, and 605,000 and 2,348 ms on
+the AMD bench (`j16`), 7.1 of its 16. 82 % of the spin is in the children's
+`SHM_CREATE`s, which draw each frame of what are, by inference from the SDK,
+their 2 MiB directory windows under its own hold (`SharedMemObject::create`),
+and 13 to 14 % in their faults, which draw each frame of a populated run under
+its own hold (`owned_frame`). The next contended lock waits under a hundredth
+as long. On the T14 the spin is a third of the children's growth in CPU, so
+the lock bounds less of the build there than on the AMD bench, where it is 72 %.
+
+**Decided** (the orchestrator, 2026-11-11), first of three: **a draw of many
+frames takes the lock once.** A promised batch serves a shared-memory region's
+frames and a fault's whole run, the way frees already go back, a region's drop
+through `pmm::free_all` and a space's teardown through `free_drawn`: one
+acquisition a window or a run instead of one a frame, with promises and ledgers
+moving as for as many single draws. Per-CPU frame caches in front of the lock
+are held until the batched build is measured; they would take single draws off
+it too, but every promise must stay backed by a frame free now, so a draw that
+finds the shared pool empty while another CPU's cache holds frames has to reach
+them. The second, a region's frames drawn at first touch, and the third, the
+window's size, are in their own records (the spawn record names them).
+
+**Exit**: that attribution, posted, and read again after the change; the
+all-CPU build's lock waits read again with the batched draw; and each of the
+three either written per CPU on a line no other CPU writes, or kept with its
+refusal reason at its site.

issues/spawning-a-process-costs-toyos-up-to-147-times-linuxs.md

@@ -62,7 +62,6 @@ spawner now reads a served image whole, through the pager, into fresh memory
 (`issues/a-large-spawn-on-the-amd-bench-is-a-seventh-slower-than-mains.md`),
 and shared text is stage 5 of
 `issues/process-memory-in-4-kib-pages-still-copies-each-programs-text.md`.
-The parallel build's loss where the serial build wins is unmeasured.
 `issues/idle-cpus-enter-the-c-states-cst-names.md` records the T14's `j1`
 under deep idle.
 
@@ -97,9 +96,56 @@ behind: `posix_spawn_wait.std` and `process.spawn_wait` are unresolved against
 Linux's 60% spread, and the rest are ahead. The all-CPU build is behind on both, 2.22x and 2.57x, where the
 one-at-a-time build is ahead on both.
 
+**Where the all-CPU build's time goes, 2026-11-11** ([evidence](https://github.com/ToyOSOrg/evidence/tree/a22307591cfdefcf9fa1fef2ae5d1de79b926e9e/spawnperf-diag)): one
+diagnosis boot a bench at main 34cb4c918 with measurement commits, the build's
+256 children at `j1` and at one a CPU, three rounds each, a child's CPU split by
+system call, fault, lock spin and shootdown wait. The probes slow the round
+(the AMD bench's `j16` round took 331 ms against 313 to 316 ms without them),
+so the shares are of a slightly slowed machine. A child does the same work at
+every width (53 system calls, 90 faults), and its CPU grows from 2.28 to 5.14 ms
+on the T14 (`j8`) and from 2.83 to 15.5 ms on the AMD bench (`j16`). Of that
+growth over the 256 children:
+
+| | T14 `j1` -> `j8` | AMD `j1` -> `j16` |
+|---|---|---|
+| children's CPU, ms a round | 583 -> 1,316 | 724 -> 3,977 |
+| `SHM_CREATE`, of the growth | 432 ms, 59 % (201 ms spin) | 2,342 ms, 72 % (1,934 ms spin) |
+| faults, of the growth | 91 ms, 12 % (33 ms spin) | 456 ms, 14 % (324 ms spin) |
+| every other system call | 28 ms | 57 ms |
+| user time and exit, the rest | 182 ms | 400 ms |
+| lock spin across the machine, a CPU's worth | 1.06 of 8 | 7.1 of 16 |
+| of the contended lock waits, the frame allocator's one lock | 99 % | 99.7 % |
+| shootdown waits, issued or served | none | none |
+
+The four `SHM_CREATE`s a child makes are, by inference from the SDK, the
+2 MiB windows of its four directory connections, each 512 frames drawn one at a
+time under the frame allocator's one lock and zeroed: 95 µs a window on the T14
+and 133 µs on the AMD bench even at `j1`, where a child's four are 17 and 19 % of
+its CPU. At `j<cpus>` the draws queue on that lock. Lock spin across the machine
+grows by 244 ms on the T14, a third of the children's growth, and by 2,354 ms
+on the AMD bench, 72 %; the rest of the growth, user time and exit among it,
+is not lock spin. The windows are
+`issues/a-served-directory-costs-each-process-a-two-mebibyte-window.md`, the
+lock `issues/per-cpu-scheduler-and-pmm-state-shares-lines-every-cpu-writes.md`.
+The next contended lock, the process table's in `process_data`, waits 1.2 ms a
+round on the T14 and 3.7 ms on the AMD bench; the kernel heap's 0.7 and 2.7 ms.
+The rest's growth is not split further: the lock spin outside the children's
+system calls and faults, which bounds the exit path's, is 10 and 77 ms.
+
+Decided, in this order (the orchestrator, 2026-11-11): a draw of many frames
+takes the allocator's lock once, with per-CPU frame caches held until that is
+measured (the frame allocator's record); a shared-memory region's frames are
+drawn at its pages' first touch, promised when it is made, and a device's run
+stays drawn whole
+(`issues/a-shared-memory-region-draws-and-zeroes-every-frame-when-it-is-made.md`);
+and the window's size waits for the file-call work (the window's record). The
+rest's growth is read again after the first two.
+
 **Exit**: these rows within Linux's on both benches in a labelled
 current-state comparison, or each remaining difference measured to its parts
 and justified.
 
 **Owner**: process creation (`kernel/src/process.rs`, the SDK's spawn and
-`sdk/libc/src/spawn.rs`) and, for the parallel build, the scheduler (`kernel/src/sched/`).
+`sdk/libc/src/spawn.rs`) and, for the parallel build, the frame allocator
+(`kernel/src/mm/pmm.rs`), the shared-memory object (`kernel/src/object/shm.rs`)
+and the file-call window (`sdk/toyos/src/fs.rs`).