diff --git a/issues/a-served-directory-costs-each-process-a-two-mebibyte-window.md b/issues/a-served-directory-costs-each-process-a-two-mebibyte-window.md index 83d6310ddf..65b0b6efb9 100644 --- a/issues/a-served-directory-costs-each-process-a-two-mebibyte-window.md +++ b/issues/a-served-directory-costs-each-process-a-two-mebibyte-window.md @@ -4,7 +4,7 @@ kind: tooling opened: 2026-09-27 --- -# A served directory costs each process a 2 MiB window, and nobody measured it +# A served directory costs each process a 2 MiB window Every process that names a file under a served directory connects to that directory's server once and lends it a `toyos::fs::WINDOW_BYTES` region @@ -13,7 +13,33 @@ touched, held for the process's life, and mapped into the server too. A desktop process that touches `/home`, `/config`, `/state` and `/apps` holds 8 MiB of windows; fsd maps up to `MAX_SERVED` of them. Whether the window's size is paid for by the reads that fill it, and what the desktop's processes -hold in windows together, has not been measured. +hold in windows together, has not been measured. The size's stated reason, +that 2 MiB is the size shared memory comes in, is gone: a region is whole +4 KiB pages. + +**Creating it is most of what the all-CPU build's children add in CPU** +(2026-11-11, [evidence](https://github.com/ToyOSOrg/evidence/tree/a22307591cfdefcf9fa1fef2ae5d1de79b926e9e/spawnperf-diag); the attribution is in +`issues/spawning-a-process-costs-toyos-up-to-147-times-linuxs.md`). A build +child makes four `SHM_CREATE`s beside its four directory connections, by +inference from the SDK their windows. Each is 512 frames drawn one at a time +under the frame allocator's one lock and zeroed: 95 µs a window on the T14 and +133 µs on the AMD bench with one child at a time, where a child's four are 17 +and 19 % of its CPU. With one child a CPU, `SHM_CREATE` is 59 % (T14) and 72 % +(AMD) of the growth in the children's CPU; of that, 47 % (T14) and 83 % (AMD) +is spin on that lock +(`issues/per-cpu-scheduler-and-pmm-state-shares-lines-every-cpu-writes.md`), +and the T14's other 231 ms is not split. How much of the build's wall-clock +loss against Linux this is has not been measured; that it is most of it is an +inference. + +**Decided** (the orchestrator, 2026-11-11), third of three: the window's size +waits for the file-call work. Once a served file's reads, writes and +attributes go through its paged object (PR #1258 and the work after it), the +window carries paths, small answers and the writes a holder's admission +refuses, and is cut to what that needs. Before that, its creation costs what +its region's first-touch draw makes it +(`issues/a-shared-memory-region-draws-and-zeroes-every-frame-when-it-is-made.md`). **Exit**: the resident memory of a desktop boot's processes with and without -their windows, measured, and the window's size justified from it or cut. +their windows, measured, and the window's size justified from it or cut once +the file-call work carries reads and writes on the paged object. diff --git a/issues/a-shared-memory-region-draws-and-zeroes-every-frame-when-it-is-made.md b/issues/a-shared-memory-region-draws-and-zeroes-every-frame-when-it-is-made.md new file mode 100644 index 0000000000..bfc1d1bc10 --- /dev/null +++ b/issues/a-shared-memory-region-draws-and-zeroes-every-frame-when-it-is-made.md @@ -0,0 +1,40 @@ +--- +status: assigned +kind: defect +opened: 2026-11-11 +--- + +# A shared-memory region draws and zeroes every frame when it is made + +`SharedMemObject::create` (`kernel/src/object/shm.rs`) draws every frame of a +region and zeroes it before the call returns, so a region costs its whole size +in time and memory however little of it is touched, where a process's private +memory takes a frame at each page's first touch. On the spawn path it is a +build child's `SHM_CREATE`: 95 µs a call on the T14 and 133 µs on the AMD bench +with one child at a time, a child's four 17 and 19 % of its CPU, and most of +what the all-CPU build's children add in CPU; by inference from the SDK, those +four are the 2 MiB windows of its directory connections +(`issues/a-served-directory-costs-each-process-a-two-mebibyte-window.md`, +[evidence](https://github.com/ToyOSOrg/evidence/tree/a22307591cfdefcf9fa1fef2ae5d1de79b926e9e/spawnperf-diag)). + +**Decided** (the orchestrator, 2026-11-11), second of three after the batched +draw (`issues/per-cpu-scheduler-and-pmm-state-shares-lines-every-cpu-writes.md`): +a region's frames are drawn at its pages' first touch. The whole region is +promised when it is made, so it is refused then and never at a touch. Two +kinds stay drawn whole when they are made: a device's run, as a device needs, +and an inbox's rings, which the kernel writes in interrupt context, where no +frame may be drawn. One draw serves both kernel-made zeroed frames for a +touched hole, `pmm::alloc_promised`: a region's page, drawn by +`SharedMemObject::page()` into the region's own slot for that page, and a +touched hole of a MEMORY volume's paged object (mapped paged objects, 7b in +`issues/process-memory-in-4-kib-pages-still-copies-each-programs-text.md`), +drawn into the pager's index under its lock. A region's slot is never revoked +and a pager's page is, so the two keep their own stores. The module header's +"a fault on a mapping of one never allocates" changes with it. + +**Assigned**: the spawn-path work, after the batched draw lands. + +**Exit**: a region's creation draws no frame and a first touch draws one, +promised whole at creation and refused only there, but for a device's run and +an inbox's rings, which are drawn whole when made; and a build child's +`SHM_CREATE` cost read again. diff --git a/issues/per-cpu-scheduler-and-pmm-state-shares-lines-every-cpu-writes.md b/issues/per-cpu-scheduler-and-pmm-state-shares-lines-every-cpu-writes.md index 274b1de62c..95879749e9 100644 --- a/issues/per-cpu-scheduler-and-pmm-state-shares-lines-every-cpu-writes.md +++ b/issues/per-cpu-scheduler-and-pmm-state-shares-lines-every-cpu-writes.md @@ -52,6 +52,33 @@ benches. A fourth, `IN_PASS`, was measured with them and now sits in its CPU's own record (`kernel/src/cpu_local.rs`), a line no other CPU writes. -**Exit**: that attribution, posted, and read again after the change; and -each of the three either written per CPU on a line no other CPU writes, or -kept with its refusal reason at its site. +**The frame allocator's lock is what the all-CPU build waits on** +(2026-11-11, [evidence](https://github.com/ToyOSOrg/evidence/tree/a22307591cfdefcf9fa1fef2ae5d1de79b926e9e/spawnperf-diag); the attribution is in +`issues/spawning-a-process-costs-toyos-up-to-147-times-linuxs.md`). Under the +build's 256 children, one a CPU, it is 99 % of every contended lock wait on +both benches: 418,000 contended acquisitions and 243 ms of wait a round on the +T14 (`j8`), 1.05 of its 8 CPUs for the whole round, and 605,000 and 2,348 ms on +the AMD bench (`j16`), 7.1 of its 16. 82 % of the spin is in the children's +`SHM_CREATE`s, which draw each frame of what are, by inference from the SDK, +their 2 MiB directory windows under its own hold (`SharedMemObject::create`), +and 13 to 14 % in their faults, which draw each frame of a populated run under +its own hold (`owned_frame`). The next contended lock waits under a hundredth +as long. On the T14 the spin is a third of the children's growth in CPU, so +the lock bounds less of the build there than on the AMD bench, where it is 72 %. + +**Decided** (the orchestrator, 2026-11-11), first of three: **a draw of many +frames takes the lock once.** A promised batch serves a shared-memory region's +frames and a fault's whole run, the way frees already go back, a region's drop +through `pmm::free_all` and a space's teardown through `free_drawn`: one +acquisition a window or a run instead of one a frame, with promises and ledgers +moving as for as many single draws. Per-CPU frame caches in front of the lock +are held until the batched build is measured; they would take single draws off +it too, but every promise must stay backed by a frame free now, so a draw that +finds the shared pool empty while another CPU's cache holds frames has to reach +them. The second, a region's frames drawn at first touch, and the third, the +window's size, are in their own records (the spawn record names them). + +**Exit**: that attribution, posted, and read again after the change; the +all-CPU build's lock waits read again with the batched draw; and each of the +three either written per CPU on a line no other CPU writes, or kept with its +refusal reason at its site. diff --git a/issues/spawning-a-process-costs-toyos-up-to-147-times-linuxs.md b/issues/spawning-a-process-costs-toyos-up-to-147-times-linuxs.md index cc7aa1a133..2edc41bfd0 100644 --- a/issues/spawning-a-process-costs-toyos-up-to-147-times-linuxs.md +++ b/issues/spawning-a-process-costs-toyos-up-to-147-times-linuxs.md @@ -62,7 +62,6 @@ spawner now reads a served image whole, through the pager, into fresh memory (`issues/a-large-spawn-on-the-amd-bench-is-a-seventh-slower-than-mains.md`), and shared text is stage 5 of `issues/process-memory-in-4-kib-pages-still-copies-each-programs-text.md`. -The parallel build's loss where the serial build wins is unmeasured. `issues/idle-cpus-enter-the-c-states-cst-names.md` records the T14's `j1` under deep idle. @@ -97,9 +96,56 @@ behind: `posix_spawn_wait.std` and `process.spawn_wait` are unresolved against Linux's 60% spread, and the rest are ahead. The all-CPU build is behind on both, 2.22x and 2.57x, where the one-at-a-time build is ahead on both. +**Where the all-CPU build's time goes, 2026-11-11** ([evidence](https://github.com/ToyOSOrg/evidence/tree/a22307591cfdefcf9fa1fef2ae5d1de79b926e9e/spawnperf-diag)): one +diagnosis boot a bench at main 34cb4c918 with measurement commits, the build's +256 children at `j1` and at one a CPU, three rounds each, a child's CPU split by +system call, fault, lock spin and shootdown wait. The probes slow the round +(the AMD bench's `j16` round took 331 ms against 313 to 316 ms without them), +so the shares are of a slightly slowed machine. A child does the same work at +every width (53 system calls, 90 faults), and its CPU grows from 2.28 to 5.14 ms +on the T14 (`j8`) and from 2.83 to 15.5 ms on the AMD bench (`j16`). Of that +growth over the 256 children: + +| | T14 `j1` -> `j8` | AMD `j1` -> `j16` | +|---|---|---| +| children's CPU, ms a round | 583 -> 1,316 | 724 -> 3,977 | +| `SHM_CREATE`, of the growth | 432 ms, 59 % (201 ms spin) | 2,342 ms, 72 % (1,934 ms spin) | +| faults, of the growth | 91 ms, 12 % (33 ms spin) | 456 ms, 14 % (324 ms spin) | +| every other system call | 28 ms | 57 ms | +| user time and exit, the rest | 182 ms | 400 ms | +| lock spin across the machine, a CPU's worth | 1.06 of 8 | 7.1 of 16 | +| of the contended lock waits, the frame allocator's one lock | 99 % | 99.7 % | +| shootdown waits, issued or served | none | none | + +The four `SHM_CREATE`s a child makes are, by inference from the SDK, the +2 MiB windows of its four directory connections, each 512 frames drawn one at a +time under the frame allocator's one lock and zeroed: 95 µs a window on the T14 +and 133 µs on the AMD bench even at `j1`, where a child's four are 17 and 19 % of +its CPU. At `j` the draws queue on that lock. Lock spin across the machine +grows by 244 ms on the T14, a third of the children's growth, and by 2,354 ms +on the AMD bench, 72 %; the rest of the growth, user time and exit among it, +is not lock spin. The windows are +`issues/a-served-directory-costs-each-process-a-two-mebibyte-window.md`, the +lock `issues/per-cpu-scheduler-and-pmm-state-shares-lines-every-cpu-writes.md`. +The next contended lock, the process table's in `process_data`, waits 1.2 ms a +round on the T14 and 3.7 ms on the AMD bench; the kernel heap's 0.7 and 2.7 ms. +The rest's growth is not split further: the lock spin outside the children's +system calls and faults, which bounds the exit path's, is 10 and 77 ms. + +Decided, in this order (the orchestrator, 2026-11-11): a draw of many frames +takes the allocator's lock once, with per-CPU frame caches held until that is +measured (the frame allocator's record); a shared-memory region's frames are +drawn at its pages' first touch, promised when it is made, and a device's run +stays drawn whole +(`issues/a-shared-memory-region-draws-and-zeroes-every-frame-when-it-is-made.md`); +and the window's size waits for the file-call work (the window's record). The +rest's growth is read again after the first two. + **Exit**: these rows within Linux's on both benches in a labelled current-state comparison, or each remaining difference measured to its parts and justified. **Owner**: process creation (`kernel/src/process.rs`, the SDK's spawn and -`sdk/libc/src/spawn.rs`) and, for the parallel build, the scheduler (`kernel/src/sched/`). +`sdk/libc/src/spawn.rs`) and, for the parallel build, the frame allocator +(`kernel/src/mm/pmm.rs`), the shared-memory object (`kernel/src/object/shm.rs`) +and the file-call window (`sdk/toyos/src/fs.rs`).