[Japabu]

Problem

The all-CPU build (work.build_shaped.j<cpus>) is 2.22x Linux on the T14 and 2.57x on the AMD bench while the
one-at-a-time build is ahead on both, and the spawn record called the loss unmeasured.

Result

One diagnosis boot a bench (branch spawnperf-split, c2f345b: main 34cb4c9 with MEASUREMENT ONLY commits, never
to land) split each child's CPU by system call, fault, lock spin and shootdown wait, at j1 and one child a CPU,
three rounds each, and logged every contended sync::Lock wait by site.

• A child's CPU grows from 2.28 to 5.14 ms (T14 j8) and from 2.83 to 15.5 ms (AMD j16) for the same 53
  system calls and 90 faults.
• SHM_CREATE is 59 % and 72 % of that growth, faults 12 % and 14 %, every other system call under 2 %.
• The frame allocator's one lock is 99 % of every contended lock wait: its waits are 1.05 of 8 CPUs (T14) and 7.1
  of 16 (AMD) for the whole round. The next lock (the process table in process_data) waits under a hundredth as long.
• No CPU waited on a TLB shootdown it issued or served one during the rounds: the children are single-threaded.
• The four SHM_CREATEs are, by inference from the SDK, the 2 MiB windows of the child's four directory
  connections, each 512 frames drawn one at a time under that lock and zeroed: 95 us (T14) and 133 us (AMD) a
  window even at j1, where a child's four are 17 and 19 % of its CPU.

The orchestrator decided the follow-ups (2026-11-11), in this order, and the records carry them as decisions:

1. A draw of many frames takes the allocator's lock once; per-CPU frame caches are held until that is measured
   (the frame allocator's record).
2. A shared-memory region's frames are drawn at first touch, promised when it is made; a device's run and an inbox's
   rings (which the kernel writes in interrupt context) stay drawn whole. One draw, pmm::alloc_promised, serves a
   region's hole (SharedMemObject::page(), into the region's slot) and a MEMORY volume's hole (map7b's N4, into the
   pager's index under its lock): two stores, since a pager's page is revocable and a region's is not. A record of its
   own, a-shared-memory-region-draws-and-zeroes-every-frame-when-it-is-made.md (defect, assigned), since the
   window's record is tooling.
3. The window's size waits for the file-call work (An open file is written and stated through its paged object, one write at a time, against credit #1258 and after) (the window's record).

The spawn record carries the attribution and names the frame allocator, the shared-memory object and the window as
the parallel build's owners. Lock spin is a third of the children's growth on the T14 and 72 % on the AMD bench, so
the lock bounds the AMD build far more than the T14's.

Evidence

https://github.com/ToyOSOrg/evidence/tree/a22307591cfdefcf9fa1fef2ae5d1de79b926e9e/spawnperf-diag, which the records
link.

The scripts regenerate the tables from the published log lines byte for byte (attrib.py, split.py, exit 0).
The instrumentation adds a probe to every system call once on; the j16 round's wall reads 331 ms against boot 2's
313 to 316 ms without it, so the shares are of a slightly slowed machine.

Checks

Records only. No code changes. Whichever of this and #1268 lands second updates the frame allocator's record to
say the batched draw is built, leaving its measurement owed.

🤖 Generated with Claude Code

[Japabu]

Review of #1265 at a11cab8 (base main 45935f4), records only

Architecture

The records' direction is sound:

• Draw many frames under one hold of the allocator's lock first, and build per-CPU caches only if the batched build still waits on the lock.
• Draw a shared-memory region's frames at first touch, promised at creation, while a device's run stays eager.
• Size the window after the file-call work.

None of this needs a ruling from the owner. The manual already says pages are populated lazily ("Reservation, commitment and population are separate … pages are populated lazily"), so today's eager shared memory is the divergence. The shm.rs module header ("a fault on a mapping of one never allocates") changes when the lazy draw is built. The change is +88/−5 lines of markdown and no code. A consistency review is the right check; no build is needed.

Evidence checked

I ran python3 -I attrib.py t14=… amd=… and split.py for each bench on the published lines. Each exited 0.

• split-t14.txt and split-amd.txt are byte-identical.
• attrib.md is identical except for the order of two T14 sites tied at 14 µs (scheduler.rs:342 and process.rs:761). The tie is broken by set iteration order, which string hashing randomizes on every run.

I checked every figure in the three records against those outputs. These all match:

• 53 system calls and 90 faults a child (13,568 calls and 23,039 faults over 256 children).
• A child's CPU: 2.28 → 5.14 ms on the T14 and 2.83 → 15.5 ms on the AMD bench.
• The growth table: 432/201, 2,342/1,934, 91/33, 456/324, 28, 57, 182 and 400 ms.
• The frame allocator's lock: 99 % and 99.7 % of contended lock waits; 418k and 605k contended acquisitions; 243 and 2,348 ms of spin.
• The spin's split: 82 % in SHM_CREATE and 13–14 % in faults.
• The process table at 1.2 and 3.7 ms; the kernel heap at 0.7 and 2.7 ms; spin outside the children at 10 and 77 ms.
• 95 and 133 µs per SHM_CREATE.

The code facts also hold: SharedMemObject::create takes the lock once per frame and zeroes each frame; owned_frame takes it once per frame; frees are batched; WINDOW_BYTES is 2 MiB, and its comment's reason is stale, as the record says. The exceptions follow.

Findings

BLOCKER 1. The decisions are presented as open proposals.
per-cpu-scheduler-and-pmm-state-shares-lines-every-cpu-writes.md says "Proposed, not built: a draw of many frames takes the lock once". a-served-directory-costs-each-process-a-two-mebibyte-window.md says "Proposed, not built, either of:". The orchestrator's decisions are:

1. Batched draws under one hold come next.
2. A region's frames are drawn at first touch, promised at creation, and device runs stay eager.
3. Shrinking the window waits for the file-call work (An open file is written and stated through its paged object, one write at a time, against credit #1258 and what follows it).

Record these as decisions, in that order, with their real decider: the orchestrator, 2026-11-11, not the owner unless the owner made them. Make the exits match. The window record's exit says "read again after the change" and should name which change.

The decided lazy-draw work also needs a home whose kind fits it. The window record is kind: tooling (a measurement owed). Either promote it or give the kernel change a record of its own. The spawn record's Owner line should then name the shared-memory object (kernel/src/object/shm.rs) as well. The PR body's "two proposals" changes with this; that correction is the orchestrator's.

BLOCKER 2. Figures that are false as written.

• Window record, "with one child a CPU 59 % (T14) and 72 % (AMD) of the growth in the children's CPU, most of it spun on that lock". On the T14, SHM_CREATE spun 201.1 ms of its 431.9 ms growth, which is 47 %. On the AMD bench it is 1,933.5 of 2,341.5 ms, 83 %. The claim is false for the T14. State it per bench and label the T14's other 231 ms as not split.
• Window record, "Each is 512 frames … 95 µs … 17 and 19 % of a child's CPU", and the spawn record, "95 µs a window … 17 and 19 % of a child's CPU there". 95 µs is one window (96.9 ms over 1,024 calls). 17 % is all four windows together (96.9 of 582.9 ms). As written, each window reads as 17 %.
• Frame allocator record, "1.06 CPUs spinning for the whole round". That is the spin across the whole machine (246.3 ms over a 232.7 ms round). The allocator lock's own wait is 243.3 ms, or 1.05 CPUs. This one is a nit; fix it with the others.

BLOCKER 3. Inferences without a label.

• Frame allocator record: "the children's SHM_CREATEs, which draw each frame of a 2 MiB window". The evidence does not carry the size; the exit line logs none. The other two records say "by inference from the SDK"; say it here too.
• Window record heading: "Its creation is most of what the all-CPU build loses". What was measured is a share of the children's CPU growth from j1 to j<cpus>. That is not the build's wall-clock loss against Linux. Either state the growth share or label the step from it to wall time as an inference.

SUGGESTION. Lock spin across the whole machine is about a third of the children's CPU growth on the T14 (244 of 733 ms) but most of it on the AMD bench. The records' "the build waits on the frame allocator's lock" fits the AMD bench far better. One sentence would keep the batched draw's expected gain on the T14 from being overstated. The growth in user time and exit (+43 % and +85 %) is not lock spin either.

SUGGESTION. The spawn record does not carry the probe's overhead (the j16 round took 331 ms against 313–316 ms without it). One clause would keep that caveat beside the shares.

SUGGESTION. Break attrib.py's sorting ties by site name so that "byte for byte" holds on every run.

SEND BACK.

[Japabu]

Review round 2 of #1265 at 9cbcbf6 (records only, focused)

Architecture

The direction is unchanged and sound. The batched draw comes first, then a region's frames at first touch, and then the window's size after the file-call work. The lazy draw now has its own defect record. That record's status: assigned with kind: defect is a valid pairing under issues/README.md, and its body names the assignee. The change is +97/−47 lines of markdown and no code, so a consistency review is the check.

Round-1 blockers

• B1 resolved.

  • Each record says "Decided (the orchestrator, 2026-11-11)". The order (first, second and third of three) is the same in all four records and the PR body.
  • The window record's exit names the file-call work.
  • The lazy draw has its own record.
  • The spawn record's Owner line names kernel/src/object/shm.rs.

• B2 resolved. I regenerated attrib.md from the published lines and checked each corrected figure against it:

  Figure	Computation
  59 % and 72 %	431.9/733.3 and 2,341.5/3,253.0
  47 % and 83 %	201.1/431.9 and 1,933.5/2,341.5
  231 ms	431.9 − 201.1
  95 and 133 µs a window	96.9 and 136.6 ms over 1,024 calls
  17 and 19 % for a child's four	96.9/582.9 and 136.6/724.4
  1.05 CPUs	243.3/232.7
  7.1 CPUs	2,348.2/330.7
  244 ms, a third	(246.3 − 1.9)/733.3
  2,354 ms, 72 %	(2,355.9 − 2.0)/3,253.0
  82 %	201.1/246.3 and 1,933.5/2,355.9
  13–14 %	32.9/246.3 and 324.1/2,355.9

• B3 resolved. The allocator record now says "by inference from the SDK". The window record's heading is now a share of the children's CPU growth, and the step from that to wall-clock time is labelled as an inference.

• Suggestions taken:

  • The T14's lock spin is stated as a third of its children's growth.
  • The probes' overhead is stated.
  • attrib.py breaks ties by name. I ran python3 -I attrib.py … 12 times, each run with a fresh string-hash seed, and all 12 outputs were byte-identical to attrib.md. split.py was identical on two runs for each bench.

Findings

BLOCKER 1. issues/a-shared-memory-region-draws-and-zeroes-every-frame-when-it-is-made.md:12-15: an unlabelled inference. The line reads: "A served directory's 2 MiB window is the largest such region on the spawn path: 95 µs a window … a child's four 17 and 19 % … most of what the all-CPU build's children add in CPU".

• Problem: what was measured is SHM_CREATE. That those calls are the windows is the inference from the SDK which the other three records now label. This new record repeats it as a measured fact, which is the same gap as round-1 B3.
• Fix: add "by inference from the SDK", or attribute the figures to SHM_CREATE.

BLOCKER 2 (conditional on the orchestrator's answer). The same record, :23-27 and :32: the scope of the recorded decision.

• (a) Text that is not in the PR body's decision. The "Decided (the orchestrator)" paragraph also says: "It is one mechanism with the kernel's zeroed frame for a touched hole of a MEMORY volume's paged object … and is built on 7b's object mappings (A memory object's range is mapped where the caller says, and an immutable one is shared and executable: /system text is ROOT's own pages #1259)". The PR body's decision 2 contains neither. The implementer's handoff calls the meaning of "one mechanism" an open question for the orchestrator, and calls the base a plan. Whatever part of this is not the orchestrator's decision moves out of the Decided paragraph. The build order can simply go: it belongs to the next PR.
• (b) A later decision. After this commit, the implementer's handoff records that the orchestrator accepted the first-touch design with inbox rings drawn whole, because their kernel words are reached in interrupt context. If that decision stands, the Decided paragraph and the exit ("a region's creation draws no frame") must name the rings; otherwise the exit is false for them.

The orchestrator settles both points. This blocks only where the record misstates the decision.

SUGGESTION. The records link the evidence at 36117dda5, whose attrib.py still orders ties by hash. Linking a22307591 instead makes "byte for byte" true of the script they cite. The tables are identical either way.

SUGGESTION. Once #1268 lands, the allocator record's first decision is built. Whichever of #1265 and #1268 lands second should say so in that record and leave only the measurement as owed. git merge-tree of the two heads exits 0.

SEND BACK for BLOCKER 1, a one-phrase fix, and for BLOCKER 2 if any part of it misstates the decision.

[Japabu]

Review round 3 of #1265 at 3e79317 (records only, focused on round 2's points)

Architecture

The direction is unchanged. This round is +22/−16 lines of markdown in four records. As a records-only change it needs a consistency review, not a build.

Round-2 points

• BLOCKER 1 resolved. issues/a-shared-memory-region-draws-and-zeroes-every-frame-when-it-is-made.md:12-17 gives the figures as "a build child's SHM_CREATE: 95 µs a call …". It labels "by inference from the SDK, those four are the 2 MiB windows". The window record also attributes them to SHM_CREATE with the same label.
• BLOCKER 2 resolved. I checked the record against the orchestrator's own notes. Those notes record three decisions: one draw function shared with map7b's N4, two stores, and rings drawn whole. They also record that "built on A memory object's range is mapped where the caller says, and an immutable one is shared and executable: /system text is ROOT's own pages #1259" is not a decision.
  • One draw (:26-31). pmm::alloc_promised serves a region's page, drawn by SharedMemObject::page() into the region's slot, and a touched hole of a MEMORY volume's paged object, drawn into the pager's index.
  • Two stores (:31-32). The reason given is true: a pager's page is revocable and a region's slot is not.
  • Rings drawn whole (:23-26, and in the exit at :37-40). The reason "where no frame may be drawn" holds. The allocator's lock is a plain sync::Lock, and only a Masked lock may be taken in a handler (kernel/src/sync.rs:120-133).
  • "Built on A memory object's range is mapped where the caller says, and an immutable one is shared and executable: /system text is ROOT's own pages #1259" is gone. No record in the PR mentions A memory object's range is mapped where the caller says, and an immutable one is shared and executable: /system text is ROOT's own pages #1259.
  • The PR body agrees. Its decision 2 states the same three points.
• Suggestion taken. All four records link evidence a22307591 ("pr-1265: the attribution script sorts deterministically").

Findings

None.

Note on landing order

#1250's and #1268's new records-only commits each add an Owed on metal: bullet at the same place where this PR adds its paragraphs:

• 8be9cc4 adds one to the spawn record.
• 82555fb adds one to the allocator record.
• Both conflict with this head: git merge-tree exits 1.

Whichever lands second resolves the conflicts. As the PR body says, the second of this PR and #1268 also records the batched draw as built.

LAND

[Japabu]

Orchestrator: review r3 LAND at 3e79317 (records only). Queued first of the spawnperf three; #1250 and #1268 resolve their record conflicts with it when they land (their owed-on-metal bullets).