diff --git a/issues/a-short-memset-on-x86-64-costs-a-rep-stosb-start-up.md b/issues/a-short-memset-on-x86-64-costs-a-rep-stosb-start-up.md index 1ed1b3caf05..295d960a85f 100644 --- a/issues/a-short-memset-on-x86-64-costs-a-rep-stosb-start-up.md +++ b/issues/a-short-memset-on-x86-64-costs-a-rep-stosb-start-up.md @@ -24,4 +24,4 @@ any program makes pays the instruction's start-up. - That wipe costs more than `rep stosb` on long records. reject at 16 KiB reads 0.98 against ring on the T14 (0.76 with the `memset`) and 1.11–1.14 on the AMD bench (1.03). - Evidence for those readings: the r4 judge logs linked above. -**Exit.** `fill` takes a short-length path, or selects by the fast short `rep stosb` feature where the CPU has it. The crate's workarounds then go: the refusal returns to `fill(0)`, and `block.rs`'s header drops its reason. `memcpy`'s `rep movsb` may share the cost; it is unmeasured. The T14 reads a 0 B and a 64 B `memset` within a few nanoseconds of a store loop's. The kernel's counterpart is `issues/kernel-forward-copies-and-fills-are-one-rep-movsb-or-stosb-on-every-cpu.md`. +**Exit.** `fill` takes a short-length path, or selects by the fast short `rep stosb` feature where the CPU has it. The crate's workarounds then go: the refusal returns to `fill(0)`, and `block.rs`'s header drops its reason. `memcpy`'s `rep movsb` may share it: `mem.memcpy.64` reads 0.61 of Linux's on the AMD bench (`issues/a-small-memcpy-on-the-amd-bench-runs-at-as-little-as-six-tenths-of-linuxs.md`). The T14 reads a 0 B and a 64 B `memset` within a few nanoseconds of a store loop's. The kernel's counterpart is `issues/kernel-forward-copies-and-fills-are-one-rep-movsb-or-stosb-on-every-cpu.md`. diff --git a/issues/a-small-memcpy-on-the-amd-bench-runs-at-as-little-as-six-tenths-of-linuxs.md b/issues/a-small-memcpy-on-the-amd-bench-runs-at-as-little-as-six-tenths-of-linuxs.md new file mode 100644 index 00000000000..d4c2897dd73 --- /dev/null +++ b/issues/a-small-memcpy-on-the-amd-bench-runs-at-as-little-as-six-tenths-of-linuxs.md @@ -0,0 +1,44 @@ +--- +status: open +kind: defect +opened: 2026-10-28 +--- + +# A small memcpy on the AMD bench runs at as little as six tenths of Linux's + +`osbench mem`'s `mem.memcpy.` copies a buffer of that size with each +system's own `memcpy`. On ToyOS that is `sdk/libc`'s, whose `copy_forward` is +one `rep movsb` at every length (`sdk/libc/src/arch/x86_64.rs`). On Linux it +is the static musl build's. + +From the 2026-10-28 scoreboard (evidence branch `scoreboard-2026-10-28`): +ToyOS at 069cbbeb8, four boots off the internal NVMe, against five runs of +each bench's Linux as installed, interleaved. A cell is the p50 with its +run-to-run range. + +| row | AMD ToyOS | AMD Linux | T14 ToyOS | T14 Linux | +|---|---|---|---|---| +| `mem.memcpy.64` | 3,942 MB/s (0.7%) | 6,492 MB/s (1.7%) | 18,507 MB/s (6.9%) | 6,226 MB/s (1.2%) | +| `mem.memcpy.256` | 14,493 MB/s (0.7%) | 16,158 MB/s (1.7%) | 29,720 MB/s (11.1%) | 24,932 MB/s (1.5%) | +| `mem.memcpy.16384` | 75,330 MB/s (1.2%) | 73,707 MB/s (3.0%) | 185,364 MB/s (0.3%) | 201,387 MB/s (17.9%) | +| `mem.memcpy.262144` | 25,696 MB/s (45.1%) | 36,472 MB/s (3.4%) | 55,722 MB/s (0.0%) | 55,634 MB/s (0.1%) | +| `mem.memcpy.67108864` | 6,140 MB/s (0.5%) | 6,302 MB/s (4.5%) | 9,257 MB/s (0.6%) | 9,677 MB/s (1.2%) | + +On the AMD (Ryzen 7 4800U) ToyOS loses 64 bytes at 0.61 of Linux's, 256 at +0.90 and 256 KiB at 0.70, each resolved. Zen 2 lacks ERMS +(`issues/kernel-forward-copies-and-fills-are-one-rep-movsb-or-stosb-on-every-cpu.md`), +and a `rep movsb` start-up is the cost +`issues/a-short-memset-on-x86-64-costs-a-rep-stosb-start-up.md` records for +`memset`. This row is that record's `memcpy` measurement. On the T14 +(i5-1135G7) ToyOS wins 64 bytes threefold. It loses 16 KiB (1.09x) and +64 MiB (1.05x), both resolved, and matches at 256 KiB. The sweep asks for +huge pages, which Linux gives and ToyOS does not; the 64 MiB row's 0.95 of +Linux's equals `mem.bw.copy`'s under the same difference +(`issues/user-memory-hot-paths-are-slower-than-2-mib-pages.md`). The 256 KiB row's 45% +ToyOS range on the AMD is unexplained. + +**Exit**: `mem.memcpy.*` within Linux's on both benches in a labelled +current-state comparison, or each remaining difference measured and +justified against the vendors' guidance. + +**Owner**: userland's copy (`sdk/libc/src/memory.rs` and its `arch`). diff --git a/issues/a-small-message-round-trip-over-a-native-connection-costs-twice-linuxs-on-the-t14.md b/issues/a-small-message-round-trip-over-a-native-connection-costs-twice-linuxs-on-the-t14.md new file mode 100644 index 00000000000..cd61c5a15ac --- /dev/null +++ b/issues/a-small-message-round-trip-over-a-native-connection-costs-twice-linuxs-on-the-t14.md @@ -0,0 +1,40 @@ +--- +status: open +kind: defect +opened: 2026-10-28 +--- + +# A small message's round trip over a native connection costs twice Linux's on the T14 + +`osbench ipc`'s `ipc.reqrep.8` sends 8 bytes to a server process and waits +for its 8-byte answer, over ToyOS's native connection and over a Unix stream +socket on Linux. `ipc.reqrep_handle.8` also moves a handle with each request. + +From the 2026-10-28 scoreboard (evidence branch `scoreboard-2026-10-28`): +ToyOS at 069cbbeb8, four boots off the internal NVMe, against five runs of +each bench's Linux as installed, interleaved. A cell is the p50 with its +run-to-run range. The AMD's Linux runs 1 to 3 timed through the TSC and 4 +and 5 through the HPET, at 1.2 µs a timestamp. + +| row | T14 ToyOS | T14 Linux | AMD ToyOS | AMD Linux | +|---|---|---|---|---| +| `ipc.reqrep.8` | 7,184 ns (1.9%) | 3,515 ns (22.1%) | 7,324 ns (25.3%) | 13,829 ns (58.1%) | +| `ipc.reqrep_handle.8` | 8,789 ns (2.5%) | 7,260 ns (3.7%) | 8,887 ns (23.4%) | 21,077 ns (42.7%) | + +On the T14 ToyOS loses both rows, 2.04x and 1.21x, and both resolve. On the +AMD it does not lose: +- `reqrep_handle.8` resolves ToyOS better (0.42x). ToyOS's slowest boot, + 9,959 ns, beats every TSC Linux run, 20,238 to 21,099 ns. +- `reqrep.8` is unresolved. Linux's runs spread within the TSC runs + themselves, 7,684, 14,983 and 15,719 ns, and the HPET runs read 13,829 and + 10,551, so the median is an HPET run's. + +The bulk-copy rows (`ipc.reqrep.4096`, `65536`) are +`issues/user-memory-hot-paths-are-slower-than-2-mib-pages.md`'s. What the +T14's 8-byte round trip spends its time on is unmeasured. + +**Exit**: `ipc.reqrep.8` and `ipc.reqrep_handle.8` within Linux's on both +benches in a labelled current-state comparison, or each remaining difference +measured to its parts and justified. + +**Owner**: native IPC (`sdk/toyos/src/ipc.rs` and the kernel's message path). diff --git a/issues/a-tlb-shootdown-reaches-only-the-cpus-that-ran-the-space.md b/issues/a-tlb-shootdown-reaches-only-the-cpus-that-ran-the-space.md index e75186e5b49..e36fbdd2985 100644 --- a/issues/a-tlb-shootdown-reaches-only-the-cpus-that-ran-the-space.md +++ b/issues/a-tlb-shootdown-reaches-only-the-cpus-that-ran-the-space.md @@ -31,5 +31,10 @@ another space runs there, and faults touching the page again after the initiator unmapped it meanwhile. The TCG model has no PCID (`build/arch.rs`), so only metal can show a switched-out CPU's entries. +In the 2026-10-28 scoreboard (evidence branch `scoreboard-2026-10-28`) the +T14 lost `mem.munmap_part.4k` at 1.22x Linux's (1,558 against 1,280 ns) and +`mem.munmap_touched_remote.4k` at 1.32x (3,028 against 2,290 ns), each +resolved; the AMD won both. + **Exit**: that case, on each proving machine with PCID, red with the flush on entry removed; and the T14's munmap and `vm_protect` figures against Linux. diff --git a/issues/a-tmpfs-metadata-call-costs-toyos-up-to-six-times-linuxs.md b/issues/a-tmpfs-metadata-call-costs-toyos-up-to-six-times-linuxs.md new file mode 100644 index 00000000000..30f31aa391b --- /dev/null +++ b/issues/a-tmpfs-metadata-call-costs-toyos-up-to-six-times-linuxs.md @@ -0,0 +1,54 @@ +--- +status: open +kind: defect +opened: 2026-10-28 +--- + +# A tmpfs metadata call costs ToyOS up to six times Linux's + +`osbench files` runs the same operations under each system's memory +filesystem: ToyOS's `/tmp`, which the kernel serves through syscalls today, +and Linux's `/dev/shm`, tmpfs through the VFS. + +From the 2026-10-28 scoreboard (evidence branch `scoreboard-2026-10-28`): +ToyOS at 069cbbeb8, four boots off the internal NVMe, against five runs of +each bench's Linux as installed, interleaved. A cell is the p50 with its +run-to-run range. The AMD's Linux runs 1 to 3 timed through the TSC and 4 +and 5 through the HPET, at 1.2 µs a timestamp. + +| row | T14 ToyOS | T14 Linux | AMD ToyOS | AMD Linux | +|---|---|---|---|---| +| `fs.mem.open_close` | 7,291 ns (7.7%) | 1,114 ns (8.1%) | 6,157 ns (0.8%) | 2,946 ns (48.4%) | +| `fs.mem.stat` | 3,115 ns (5.3%) | 584 ns (2.6%) | 2,616 ns (1.1%) | 1,513 ns (90.7%) | +| `fs.mem.rename` | 7,374 ns (6.3%) | 1,624 ns (2.3%) | 6,282 ns (1.1%) | 4,538 ns (35.7%) | +| `fs.mem.mkdir_rmdir` | 9,011 ns (9.1%) | 2,213 ns (1.5%) | 7,705 ns (3.1%) | 6,702 ns (23.3%) | +| `fs.mem.tree.100` | 1,505,195 ns (4.1%) | 548,925 ns (1.1%) | 1,384,280 ns (1.3%) | 1,457,995 ns (2.6%) | +| `fs.mem.create_write_unlink` | 12,832 ns (4.7%) | 4,715 ns (2.1%) | 10,801 ns (0.6%) | 12,743 ns (13.7%) | +| `fs.mem.write_4k_at` | 1,237 ns (4.0%) | 625 ns (5.0%) | 1,563 ns (7.7%) | 1,823 ns (61.0%) | +| `fs.mem.read_4k_at` | 1,222 ns (2.3%) | 710 ns (4.5%) | 1,422 ns (3.5%) | 1,403 ns (70.8%) | + +On the T14 ToyOS loses all eight, 1.7x to 6.5x, each resolved. On the AMD it +loses `open_close` (2.1x) and `rename` (1.4x), both resolved, wins +`tree.100`, `create_write_unlink` and `write_4k_at`, and is unresolved on the +other three. There the HPET explains Linux's spread: its TSC runs read +`open_close` at 2.9 µs and its HPET runs at 4.3 µs. The listing +(`fs.mem.readdir.1000`) is `issues/a-listing-still-scans-the-whole-mount.md`'s, +and the sequential copies are +`issues/user-memory-hot-paths-are-slower-than-2-mib-pages.md`'s. + +**The kernel's tmpfs is not where this is fixed.** Step 9 of +`issues/the-kernel-is-small-interrupts-post-and-threads-wait.md`, under the +owner's rulings of 2026-09-25 and 2026-10-03, deletes `tmpfs.rs`, +`fs_rename.rs`, `file_cache.rs` and `vfs.rs`, and moves `/tmp` to +userland per session. An `fs.mem` call then becomes a file server request, +which costs 3 to 12 times a kernel file call today +(`issues/a-file-server-request-costs-three-to-twelve-times-a-kernel-file-call.md`). +So the move makes this gap larger, unless the file server's round trip +shrinks first. + +**Exit**: once `/tmp` is served from userland, these eight rows within +Linux's on both benches in a labelled current-state comparison, or each +remaining difference measured to its parts and justified. + +**Owner**: whatever serves `/tmp` through step 9's migration. Until then, +nothing in the kernel's tmpfs is owed this work. diff --git a/issues/a-two-thread-ping-pong-costs-toyos-up-to-twice-linuxs-on-the-t14.md b/issues/a-two-thread-ping-pong-costs-toyos-up-to-twice-linuxs-on-the-t14.md new file mode 100644 index 00000000000..dc3a01c71b1 --- /dev/null +++ b/issues/a-two-thread-ping-pong-costs-toyos-up-to-twice-linuxs-on-the-t14.md @@ -0,0 +1,47 @@ +--- +status: open +kind: defect +opened: 2026-10-28 +--- + +# A two-thread ping-pong costs ToyOS up to twice Linux's on the T14 + +`osbench sched`'s `pingpong.*` passes a turn between two threads, by futex +wait and wake (`futex`) and by a pipe's write and read (`pipe`). It runs on +one CPU (`same_cpu`) and across cores (`cross_core`). Each sample is one +round trip as the pinger sees it. + +From the 2026-10-28 scoreboard (evidence branch `scoreboard-2026-10-28`): +ToyOS at 069cbbeb8, four boots off the internal NVMe, against five runs of +each bench's Linux as installed, interleaved. A cell is the p50 with its +run-to-run range. The AMD's Linux runs 1 to 3 timed through the TSC and 4 +and 5 through the HPET, at 1.2 µs a timestamp. + +| row | T14 ToyOS | T14 Linux | AMD ToyOS | AMD Linux | +|---|---|---|---|---| +| `pingpong.futex.same_cpu` | 4,580 ns (10.1%) | 2,279 ns (4.4%) | 5,015 ns (31.3%) | 6,248 ns (27.1%) | +| `pingpong.futex.cross_core` | 4,574 ns (2.5%) | 3,341 ns (0.9%) | 5,662 ns (26.7%) | 9,864 ns (43.5%) | +| `pingpong.pipe.same_cpu` | 6,186 ns (1.3%) | 4,504 ns (54.4%) | 5,656 ns (1.2%) | 7,739 ns (24.9%) | +| `pingpong.pipe.cross_core` | 6,802 ns (7.1%) | 3,712 ns (6.0%) | 5,746 ns (30.5%) | 15,021 ns (57.8%) | + +On the T14 ToyOS loses all four, 1.37x to 2.01x, each resolved. On the AMD +ToyOS is faster in all four, and against Linux's TSC runs alone too: the +futex rows read 6,141 to 6,248 ns (`same_cpu`) and 9,648 to 9,864 ns +(`cross_core`) there, both above ToyOS's medians. The T14's Linux timed every +run through the TSC. Where its round trip's time goes, the wait, the wake or the +switch, is unmeasured. + +#1074 has since changed the pipe on main and reads `pingpong.pipe.*` lower +on its head than its base +(`issues/a-64-kib-pipe-write-is-two-percent-below-the-two-mebibyte-ring.md`), +so the `pipe` rows' current state against Linux is owed a new reading. +The scheduler's pair-wake records bear on both kinds: +`issues/a-wake-whose-waker-parks-next-is-forwarded-to-an-idle-cpu.md` and +`issues/an-ordinary-wake-posted-on-its-own-cpu-waits-for-that-cpus-next-pass.md`. + +**Exit**: `pingpong.*` within Linux's on both benches in a labelled +current-state comparison, or each remaining difference measured to its parts +and justified. + +**Owner**: the kernel's futex and wake path (`kernel/src/sched/futex.rs`, +`kernel/src/sched/`), and the pipe (`kernel/src/pipe.rs`) for its rows. diff --git a/issues/no-published-run-compares-toyos-with-linux-on-one-machine.md b/issues/no-published-run-compares-toyos-with-linux-on-one-machine.md index 723a6c4f1cd..c07acd6f323 100644 --- a/issues/no-published-run-compares-toyos-with-linux-on-one-machine.md +++ b/issues/no-published-run-compares-toyos-with-linux-on-one-machine.md @@ -82,7 +82,8 @@ comparison until it can: AX200, are measured under Linux alone. - **USB, NVMe and cpuidle identity**: their owners serve no program that reads them. -- **Loopback** (`net.*`): netstack refuses 127/8 as a destination. +- **Loopback** (`net.*`): netstack refuses 127/8 as a destination + (`issues/toyos-has-no-loopback-so-osbench-net-measures-nothing.md`). **Exit**: both systems' outputs on the T14 and the AMD bench published in evidence, Linux's run before the T14's wipe. Linux's outputs are a goal diff --git a/issues/spawning-a-process-costs-toyos-up-to-four-times-linuxs-on-the-t14.md b/issues/spawning-a-process-costs-toyos-up-to-four-times-linuxs-on-the-t14.md new file mode 100644 index 00000000000..40f2d220f3a --- /dev/null +++ b/issues/spawning-a-process-costs-toyos-up-to-four-times-linuxs-on-the-t14.md @@ -0,0 +1,75 @@ +--- +status: open +kind: defect +opened: 2026-10-28 +--- + +# Spawning a process costs ToyOS up to four times Linux's on the T14 + +`osbench`'s spawn rows spawn a program, wait for it, and time the whole. +`proc.spawn_wait.*` uses std's `Command`, `proc.posix_spawn_wait.*` C's +`posix_spawn` and `waitpid`, and `process.spawn_wait` spawns osbench itself. +`tiny` is a minimal program written to `/tmp`, `large` the same with 32 MiB +of data it never reads, and `std` the osbench binary. `thread.spawn_join.*` +creates and joins an empty thread. `work.build_shaped.j` compresses 256 +files, one child process each, as many at once as there are CPUs. + +From the 2026-10-28 scoreboard (evidence branch `scoreboard-2026-10-28`): +ToyOS at 069cbbeb8, four boots off the internal NVMe, against five runs of +each bench's Linux as installed, interleaved. A cell is the p50 with its +run-to-run range. The AMD's Linux runs 1 to 3 timed through the TSC and 4 +and 5 through the HPET, at 1.2 µs a timestamp. + +| row | T14 ToyOS | T14 Linux | AMD ToyOS | AMD Linux | +|---|---|---|---|---| +| `proc.spawn_wait.tiny` | 52,548 ns (19.8%) | 52,094 ns (1.2%) | 69,724 ns (11.8%) | 266,663 ns (64.7%) | +| `proc.spawn_wait.large` | 202,300 ns (13.0%) | 54,906 ns (3.0%) | 170,284 ns (18.2%) | 273,444 ns (10.2%) | +| `proc.posix_spawn_wait.tiny` | 55,722 ns (10.1%) | 43,698 ns (2.4%) | 75,015 ns (4.2%) | 92,704 ns (150.0%) | +| `proc.posix_spawn_wait.std` | 451,798 ns (1.7%) | 169,630 ns (2.1%) | 504,838 ns (6.8%) | 327,668 ns (158.1%) | +| `process.spawn_wait` | 452,918 ns (4.7%) | 179,133 ns (6.0%) | 534,832 ns (3.8%) | 911,540 ns (64.2%) | +| `thread.spawn_join` | 28,144 ns (1.0%) | 25,040 ns (2.6%) | 41,094 ns (4.4%) | 72,607 ns (34.6%) | +| `thread.spawn_join.stack64k` | 28,562 ns (0.9%) | 23,441 ns (1.1%) | 40,322 ns (3.3%) | 65,844 ns (22.2%) | + +| row | ToyOS | Linux | +|---|---|---| +| T14 `work.build_shaped.j8` | 166,245,528 ns (2.4%) | 99,148,404 ns (1.2%) | +| AMD `work.build_shaped.j16` | 270,675,582 ns (1.0%) | 116,739,823 ns (13.4%) | + +**On the T14** ToyOS loses every spawn and thread row but one. The +like-for-like pair is std's `Command` with the two programs: `tiny` reads +1.01x, unresolved, and `large` 3.68x, resolved. Under `Command`, then, what +costs ToyOS on that bench is what `large` adds, its 32 MiB in `/tmp` +(8,192 pages). `posix_spawn` of the osbench binary reads 2.66x and +`process.spawn_wait` 2.53x. + +**On the AMD** ToyOS resolves better on `spawn_wait.large`, +`posix_spawn_wait.tiny`, `spawn_wait.tiny` and every thread row, and stays +better against Linux's TSC runs alone (`large`: Linux 259 to 273 µs, ToyOS +155 to 186 µs). It reads better, unresolved, on `process.spawn_wait`, and +1.54x worse, unresolved, on `posix_spawn_wait.std`. +Linux's spreads there are its own: bimodal within the TSC runs +(`posix_spawn_wait.std` run 2 reads 740 µs), and a 1.2 µs timestamp is under +1% of a spawn. + +**The all-CPU build loses on both benches**, 1.68x on the T14 (`j8`) and +2.32x on the AMD (`j16`), where ToyOS's one-at-a-time build (`j1`) beats +Linux's on both. + +What `large` exercises is recorded. A spawn from a `/tmp` file snapshots it, +taking a reference to each resident page under the file cache's lock and 16 +bytes a page (`issues/user-memory-hot-paths-are-slower-than-2-mib-pages.md`, +"Spawn"), and shared text is stage 5 of +`issues/process-memory-in-4-kib-pages-still-copies-each-programs-text.md`. +This record adds the measured cost of that mechanism against Linux, the +osbench binary's spawn on the T14, the thread rows, and the parallel build, +whose loss where the serial build wins is unmeasured. +`issues/idle-cpus-enter-the-c-states-cst-names.md` records the T14's `j1` +under deep idle. + +**Exit**: these rows within Linux's on both benches in a labelled +current-state comparison, or each remaining difference measured to its parts +and justified. + +**Owner**: process creation (`kernel/src/process.rs`, `kernel/src/elf/`, the +SDK's spawn and `sdk/libc/src/spawn.rs`) and, for the parallel build, the +scheduler (`kernel/src/sched/`). diff --git a/issues/std-yield-now-on-toyos-yields-nothing.md b/issues/std-yield-now-on-toyos-yields-nothing.md new file mode 100644 index 00000000000..9b44422dcad --- /dev/null +++ b/issues/std-yield-now-on-toyos-yields-nothing.md @@ -0,0 +1,30 @@ +--- +status: open +kind: defect +opened: 2026-10-28 +--- + +# std's `yield_now` on ToyOS yields nothing + +ToyOS has no yield syscall, and its std's `thread::yield_now` is a spin-loop +hint (`sdk/std/sys/thread.rs`). A Rust program that yields to let another +runnable thread go keeps its CPU instead, and spin-wait loops written against +`yield_now` behave unlike they do on every other platform. `osbench`'s +`time.yield` therefore reports `unsupported` on ToyOS rather than time a hint +against Linux's `sched_yield`. + +From the 2026-10-28 scoreboard (evidence branch `scoreboard-2026-10-28`): +ToyOS at 069cbbeb8, four boots off the internal NVMe, against five runs of +each bench's Linux as installed, interleaved. A cell is the p50 with its +run-to-run range. + +| row | T14 Linux | AMD Linux | ToyOS | +|---|---|---|---| +| `time.yield` | 190,677 ps (0.6%) | 552,167 ps (1.9%) | unsupported, all 4 boots on both benches | + +**Exit**: a yield that gives the CPU to another runnable thread of the +caller's band, std's `yield_now` on it, and `time.yield` measured on +ToyOS within Linux's on both benches. + +**Owner**: the scheduler's syscall surface (`kernel/src/sched/`, `sdk/abi`) +and ToyOS's std (`sdk/std/sys/thread.rs`). diff --git a/issues/toyos-durable-writes-on-the-t14s-nvme-run-six-times-slower-in-some-boots.md b/issues/toyos-durable-writes-on-the-t14s-nvme-run-six-times-slower-in-some-boots.md new file mode 100644 index 00000000000..dac95da6200 --- /dev/null +++ b/issues/toyos-durable-writes-on-the-t14s-nvme-run-six-times-slower-in-some-boots.md @@ -0,0 +1,44 @@ +--- +status: open +kind: defect +opened: 2026-10-28 +--- + +# ToyOS's durable writes on the T14's NVMe run six times slower in some boots + +On the T14 (SK hynix BC501, volatile write cache present, so ToyOS's NVMe +driver issues Flush), five boots of PR #1097's storage code on 2026-10-28, +all off the internal NVMe with DATA on it, read these p50s: + +| boot | `fsync_clean` | `write_4k_fsync` | `append_4k_fsync` | `create_fsync_unlink` | `seq_write_fsync` | +|---|---|---|---|---|---| +| c8f3fbfb0 (16:08) | 3.02 ms | 6.13 ms | 6.18 ms | 6.87 ms | 162 MB/s | +| 069cbbeb8 (16:47) | 0.46 ms | 1.10 ms | 1.01 ms | 1.17 ms | 347 MB/s | +| 069cbbeb8 (17:31) | 3.42 ms | 6.65 ms | 6.85 ms | 5.05 ms | 149 MB/s | +| 069cbbeb8 (17:39) | 0.50 ms | 1.08 ms | 1.04 ms | 7.10 ms | 167 MB/s | +| 069cbbeb8 (17:48) | 0.58 ms | 1.08 ms | 1.09 ms | 1.23 ms | 311 MB/s | + +The bench's Linux, on ext4 on the same NVMe, read these rows in the runs +interleaved with each set of boots. The c8f3fbfb0 boot had its own +validation's two Linux runs: `write_4k_fsync` 0.81 to 1.01 ms, +`append_4k_fsync` 0.80 to 1.04 ms, `create_fsync_unlink` 0.86 to 0.93 ms, and +`seq_write_fsync` 587 to 598 MB/s. The four 069cbbeb8 boots were interleaved +with five Linux runs: `write_4k_fsync` 0.99 to 1.04 ms, `append_4k_fsync` 0.83 +to 1.25 ms, `create_fsync_unlink` 0.94 to 0.98 ms, and `seq_write_fsync` 486 to +600 MB/s. Two of the five ToyOS boots ran slow through all of `persist`. +A third turned slow partway, between `append_4k_fsync` and +`create_fsync_unlink`. No change to the NVMe driver, the block service, the +file server or the DATA format lies between the two heads. The boots' NVMe +and DATA log lines do not tell the slow ones apart, and the cause is +unknown. A fast boot's clean fsync, a superblock write and a Flush, takes +0.46 to 0.58 ms where Linux's takes 23 µs +(`issues/a-clean-fsync-on-data-writes-the-superblock-and-flushes-the-disk.md`). +The figures are evidence branch `scoreboard-2026-10-28`'s and PR #1097's. + +**Exit**: repeated boots on the T14 read every durable `persist` row +(`write_4k_fsync`, `append_4k_fsync`, `create_fsync_unlink` and +`seq_write_fsync`) within Linux's, in every boot and for the whole of each +boot's run. Or the slow mode is explained by a measured cause. + +**Owner**: the NVMe driver (`system/nvme/`) and the DATA volume's commit +(`lib/bcachefs/src/fs.rs`). diff --git a/issues/toyos-durable-writes-on-the-t14s-nvme-took-1-to-6-ms-across-two-boots.md b/issues/toyos-durable-writes-on-the-t14s-nvme-took-1-to-6-ms-across-two-boots.md deleted file mode 100644 index fb0341082f4..00000000000 --- a/issues/toyos-durable-writes-on-the-t14s-nvme-took-1-to-6-ms-across-two-boots.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -status: open -kind: defect -opened: 2026-10-28 ---- - -# ToyOS's durable writes on the T14's NVMe took 1 to 6 ms across two boots - -On the T14 (SK hynix BC501, volatile write cache present, so ToyOS's NVMe -driver issues Flush), two boots of PR #1097, both off the internal NVMe with -DATA on it, read these p50s. Linux is two runs on ext4 on the same NVMe in -each validation: - -| row | ToyOS, c8f3fbfb0 | ToyOS, 069cbbeb8 | Linux (both validations) | -|---|---|---|---| -| `persist.fsync_clean` | 3,019,602 ns | 464,505 ns | 22,970 to 23,012 ns | -| `persist.write_4k_fsync` | 6,125,947 ns | 1,099,737 ns | 811,313 to 1,035,997 ns | -| `persist.append_4k_fsync` | 6,182,133 ns | 1,012,768 ns | 796,449 to 1,200,332 ns | -| `persist.seq_write_fsync` | 162 MB/s | 347 MB/s | 527 to 598 MB/s | - -No change to the NVMe driver, the block service, the file server or the -DATA format lies between the two heads; the second rebased onto #1095's -boot path, and both booted off the NVMe. Why one boot's durable writes cost -six times the other's is unknown. The clean fsync's remaining 20x, a -superblock write and a Flush where Linux's clean fsync takes 23 µs, is -`issues/a-clean-fsync-on-data-writes-the-superblock-and-flushes-the-disk.md`. - -**Exit**: repeated boots on the T14 read ToyOS's `persist.write_4k_fsync` -within Linux's, or the boots that do not are explained by a measured cause. - -**Owner**: the NVMe driver (`system/nvme/`) and the DATA volume's commit -(`lib/bcachefs/src/fs.rs`). diff --git a/issues/toyos-has-no-loopback-so-osbench-net-measures-nothing.md b/issues/toyos-has-no-loopback-so-osbench-net-measures-nothing.md new file mode 100644 index 00000000000..e302991bafa --- /dev/null +++ b/issues/toyos-has-no-loopback-so-osbench-net-measures-nothing.md @@ -0,0 +1,53 @@ +--- +status: open +kind: defect +opened: 2026-10-28 +--- + +# ToyOS has no loopback, so `osbench net` measures nothing + +netstack treats 127/8 as a destination that never exists (`is_martian`, +`system/netstack/wire/src/addr.rs`), and so does each of its paths: +- **TCP connect.** `shard::connect` asks the route first, and the route + refuses a martian (`system/netstack/ip/src/route.rs`, + `Counter::RouteInvalidDestination`) before it would match the host's own + address. +- **TCP listen.** `shard::listen` refuses an address the host does not hold, + 127.0.0.1 among them, as `NotLocal`, answered `ERR_INVALID_INPUT` + (`system/netstack/src/serve.rs`). So `TcpListener::bind("127.0.0.1:0")`, + a loopback program's first call, fails. +- **UDP.** `unusable` refuses a loopback destination and a send to the + host's own address by name (`system/netstack/udp/src/lib.rs`, + `Counter::SendLoopback`, `SendToSelf`). +- **Tests.** The unit tests hold these refusals: the route's + `s_ip_rte_006_invalid_and_local_destinations` (127.0.0.1 is + `RouteInvalidDestination`, `system/netstack/ip/tests/addr.rs`) and UDP's + `s_udp_us_003_only_an_assigned_address_binds`. + +**The connect's refusal is misreported.** `serve.rs`'s `connect_refused` +answers any route refusal `ERR_NOT_CONNECTED`, whose contract is "a machine +with no address or no route yet … which clears when a lease lands". For +127/8 it never clears. A program connecting to localhost is told it is +waiting for a lease, not that the destination is refused. + +`osbench net` measures TCP and UDP between two local processes over +127.0.0.1, and reports all five rows unsupported on ToyOS without trying. From +the 2026-10-28 scoreboard (evidence branch `scoreboard-2026-10-28`), five +runs of each bench's Linux: + +| row | T14 ToyOS | T14 Linux | AMD ToyOS | AMD Linux | +|---|---|---|---|---| +| `net.tcp.connect_accept_close` | unsupported | 20,913 ns (1.0%) | unsupported | 49,002 ns (59.0%) | +| `net.tcp.reqrep.64` | unsupported | 8,935 ns (8.5%) | unsupported | 27,241 ns (30.8%) | +| `net.tcp.stream.65536` | unsupported | 6,719 MB/s (2.3%) | unsupported | 2,894 MB/s (25.1%) | +| `net.udp.reqrep.64` | unsupported | 7,287 ns (2.3%) | unsupported | 22,642 ns (78.6%) | + +A program that talks to itself or a sibling over 127.0.0.1, as many servers +and tests do, cannot run unchanged on ToyOS. + +**Exit**: netstack carries a loopback path, and a connect it still refuses is +answered by what it is, not as a missing lease. `osbench net` attempts its +rows on ToyOS, and they are within Linux's on both benches in a labelled +current-state comparison. + +**Owner**: netstack (`system/netstack/`). diff --git a/issues/user-memory-hot-paths-are-slower-than-2-mib-pages.md b/issues/user-memory-hot-paths-are-slower-than-2-mib-pages.md index 497c7d6c12d..958c3edfa36 100644 --- a/issues/user-memory-hot-paths-are-slower-than-2-mib-pages.md +++ b/issues/user-memory-hot-paths-are-slower-than-2-mib-pages.md @@ -82,6 +82,12 @@ The copies (`fs.tmp`, `ipc`, `pipe`) point at the user-copy path above. "With runs" is `a75abd3f6` against `b48c7dd9d` on the T14, one run each; `6d826e313` gave 500, 521 and 132,434 for the first three rows. +`mem.bw.copy` streams two 64 MiB buffers it asks to be huge. In the +2026-10-28 scoreboard (evidence branch `scoreboard-2026-10-28`) Linux gave +131,072 KiB of huge pages and ToyOS none; ToyOS read 0.95 of Linux's on the +T14 (6,651 against 6,992 MB/s) and 0.98 on the AMD (8,352 against 8,550), +each resolved worse. + `mem.realloc_grow` no longer copies. std and libc map a block of 256 KiB or more on its own (`sdk/libc/src/mapped.rs`) and grow it by `vm_remap`, which moves pages. On the T14 at 204243e7f, `1m_256m` took 45.9 us, its last