Skip to content

osbench's disk groups as a metal row, and the forced journal write recorded as measured and declined - #1215

Merged
Japabu merged 3 commits into
mainfrom
osbench-disk-row
Nov 11, 2026
Merged

Japabu merged 3 commits into
mainfrom
osbench-disk-row

Conversation

@Japabu

@Japabu Japabu commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

What this is

Two things kept from the closed PR #1202, which carried Force Unit Access through the block path and was measured and declined.

  • osbench_disk, a metal row: osbench persist without the rest of the suite, for a change to the disk's path. persist alone writes the bench's NVMe (the durable rows and fs.disk.*). files rides along as the memory filesystem's rows and nvme for its place in the list; nvme is read-only and reports its rows unsupported on ToyOS. It reuses benchcase.toml, the suite's judge, and the suite's bound as a ceiling. Only a pass that names it runs it. A journal entry is one flush and a forced write: FUA through the block path #1202's and nvme: the controller's own idle power states, by APST #1131's A/B ran on it: 24 boots, every verdict passed.
  • Records, so the measurement is not repeated blind:
    • issues/a-forced-journal-write-is-no-faster-than-a-write-and-a-flush.md (rejected): what was measured, on which revisions, the evidence, the one result that points the other way, and the one open question.
    • issues/the-t14s-slow-durable-writes-ended-during-80-s-of-idle-in-four-boots-of-four.md (finding, assigned to nvme: the controller's own idle power states, by APST #1131's measurement): seen in the same boots; its exit is a count against the rate at which the mode ends by itself.
    • issues/storage-is-layers-and-a-role-is-a-filesystem.md no longer calls the forced write assigned work; it points at the rejected record. The slow-write record points at the finding.

The result the record holds

Median of four boots' p50 (range over boots), main with the row (c9d6eff6f) against the forced write (8a8ff27b9):

AMD row write and Flush forced
persist.write_4k_fsync 2.36 ms (2.24-2.46) 4.00 ms (4.00-4.01)
persist.append_4k_fsync 2.41 ms (2.32-2.51) 4.00 ms (4.00-4.00)
persist.create_fsync_unlink 2.62 ms (2.54-2.93) 4.47 ms (4.25-4.64)
  • Durable rows: 55% to 71% slower on the AMD bench. On the T14, forced writes in the drive's fast mode (read off nvme: the controller's own idle power states, by APST #1131's image 24f5de778, which also set APST) took 0.93-1.08 ms against 0.97 and 1.12 ms. No faster at best, 71% slower at worst.
  • One bulk row points the other way: fs.disk.seq_write.1048576 was 24% faster forced on the T14 in every boot (460-498 against 362-394 MB/s), and 13% slower on the AMD bench (399-406 against 458-472 MB/s). The ruling stands: a mixed bulk result does not pay for that loss on every durable row of one bench.
  • Whether the AMD's 4.00 ms is the drive's or ours was not established.

Evidence: https://github.com/ToyOSOrg/evidence/blob/7efbe1295aa54370b213719b4d4f460e7a103554/ab/README.md. Two gaps in that README, corrected in the records rather than there: it classes the T14's round 3 of main as fast though the boot began slow, and it does not mention the T14's sequential-write row.

Checks

  • cargo run -- --ci host at 72ce4d25e (before review r1): exit 1, 109 of 110 steps green. The one red is llvm::tests::the_caller_s_environment_reaches_neither_the_build_nor_the_key, red on every macOS host (issues/the-llvm-environment-test-breaks-git-on-macos.md). Log: https://github.com/ToyOSOrg/evidence/blob/1dfcfeb85c6a0991ffa7ee2324c66f9904c89268/README.md
  • Head 6526e1c48, rebased onto main 2ec4ad950: since 72ce4d25e only comments and records changed. The non-comment lines of tests/toyos.rs's diff against main are identical before and after, and the harness compiles (cargo test --test toyos-build --no-run, exit 0). The host run was not repeated.

No guest or metal run is owed: the change is a harness registration and two records. The row itself ran on both benches as above, at c9d6eff6f, whose registration is the one here.

🤖 Generated with Claude Code

@Japabu

Japabu commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

Review r1 of #1215 at 72ce4d25e (base ad5db5a65)

Verdict: SEND BACK. Five blockers: two in the row's comments, three in the records. None needs a build or a boot to fix.

Read: both commits, the full diff, tests/toyos.rs and tests/common/metal.rs around the row, the osbench groups it names, issues/README.md, the closed #1202 and the open #1131, and the evidence at 7efbe129 (ab/README.md, ab/verdict.md, ab/judge.py, metal/rules.txt, all 24 ab/rows/*-r?-?.txt) and 1dfcfeb8 (host/ci-host.log). Nothing was built or run.

Architecture and scope

  • No product code changes. Four files: tests/toyos.rs (+26), tests/guest/osbench/README.md (+3), two new issue files (+87). Production growth is zero.
  • Nothing of A journal entry is one flush and a forced write: FUA through the block path #1202 leaked. At the head there is no fua, write_durable, DPOFUA or "Force Unit Access" outside issues/. The manual has none either.
  • The row is a sound kind of registration. It is named_only, so no unfiltered or nightly pass takes it and the word osbench does not select it. It reuses benchcase.toml, whose three job symlinks exist. Its judge asserts no timing: DATA mounted on the bench's NVMe, ACPI mode before the first job, each job exit 0 and group=<g> done. Its 525 s is a list ceiling, not a timing bound. Both benches are declared at the registration.
  • It has a real user. nvme: the controller's own idle power states, by APST #1131 is stacked on it with a staged measurement.
  • The registration measured is the one here. c9d6eff6f and 463ba715d have the same patch id.
  • It merges with current main (2ec4ad950, after A child's arguments and environment are memory its parent lays out #1200): git merge-tree writes a tree with no conflict. Main's changes since the base touch a doc comment in tests/common/metal.rs and none of the row's judges, constants or image config.

BLOCKERS

  1. tests/toyos.rs:1300-1304 and :1883 — the row's claim is false for two of its three groups. The comments say "the disk's figures, on the bench's NVMe" and "The groups that write the disk".

    • files runs on the memory filesystem: files::suite(&files::memory()) (tests/guest/osbench/src/main.rs:129-131), /tmp, medium=memory.
    • nvme is read-only by its own header and on ToyOS reports both benches unsupported (tests/guest/osbench/src/nvme.rs:1-24). Every A/B boot's log holds exactly those two unsupported lines.
    • Only persist (the durable rows and fs.disk.*) writes the bench's NVMe. ab/judge.py read nothing else from a ToyOS boot.
    • Consequence: a reader takes fs.* memory rows and an unsupported group for disk figures, and the row has no single claim.
    • Fix, either: run persist alone, or keep the groups and say what each is there for (for example files as the memory-filesystem counterpart of fs.disk.*). The second changes no image, which matters to nvme: the controller's own idle power states, by APST #1131's staged request.
    • tests/guest/osbench/README.md:237 sits under "Both write the bench's NVMe" and should follow.
  2. tests/toyos.rs:1886-1887 — "No boot has timed the disk's groups apart from the rest of the suite" is false at landing. Eight boots of this registration (arm M) and sixteen more of the same job list ran it alone. Their rows carry timestamps: the last disk row lands 18 s after the NVMe comes up on the T14 (t14-r1-M) and 14 s on the AMD bench (amd-r1-M). Reusing the suite's bound as a ceiling is fine. Either say only that, or derive the row's own from its groups' spans.

  3. issues/a-forced-journal-write-is-no-faster-than-a-write-and-a-flush.md does not claim exactly what was measured. The AMD table and the T14 figures match ab/verdict.md to the digit; these do not:

    • :42 "no faster at best", and :36 "decided every row". On the T14, fs.disk.seq_write.1048576 was faster forced in every boot: 472, 498, 496, 460 MB/s against main's 394, 386, 362, 394. That is 24% better and resolved by the A/B's own rule, in slow and fast boots alike. On the AMD bench the same row was 13% slower, also resolved. The record should scope its claim to the durable rows and state this row, since a record that exists to stop a blind retry must hold the one result that points the other way.
    • :42 "66% slower at worst". 66% is append_4k_fsync. The record's own table gives 69% (write_4k_fsync) and 71% (create_fsync_unlink); the evidence says "+55% to +71%".
    • :38 "a third arm's boots" names no revision. It is 24f5de778, nvme: the controller's own idle power states, by APST #1131 on the forced write, which also set APST. The fast-mode comparison is forced-with-APST against write-and-Flush without.
    • :20 the Linux column is the median of three runs, not of four boots.
  4. issues/the-t14s-slow-durable-writes-ended-during-80-s-of-idle-in-four-boots-of-four.md:22-24 — "no such change ... the other two began fast" is refuted by t14-r3-M. That boot is main with the row: no after-idle row, no APST.

    • Its persist.write_4k_fsync: n=100, min 1.10 ms, p50 2.20 ms, p90 6.41 ms, p99 6.83 ms, mean 3.76 ms. About half its samples ran at the slow mode's speed.
    • Every durable row after it is fast (append_4k_fsync p50 1.12 ms, p90 1.15 ms). Compare t14-r2-M, fast throughout: p90 0.98 ms.
    • The evidence calls it fast only because its rule is p50 over 3 ms.
    • So seven of the eight other boots met the slow mode, six stayed in it through persist, and one left it inside its first durable row with neither candidate cause present. The four-of-four observation stands, but the control sentence is wrong.
    • :29-35 "Either could be what ended the mode" omits the third case, that the mode ends unprompted. One boot of the exit's arm ending there would not show idle did it; the exit needs the count against this base rate. issues/toyos-durable-writes-on-the-t14s-nvme-run-six-times-slower-in-some-boots.md already holds a boot that changed mode mid-run.
  5. issues/storage-is-layers-and-a-role-is-a-filesystem.md:37-44 on main still carries A journal entry is one flush and a forced write: FUA through the block path #1202's plan and is not in this PR. It says "FUA is missing end to end" and "Assigned with nvme: the controller's own idle power states, by APST #1131, whose APST wake cost the second Flush is part of". After this lands, the tracker would hold a track that calls the forced write assigned work beside a rejected record that says nothing is owed. The two-Flush cost stays true; the FUA sentences should go or point at the rejected record. It is the only record on main that names the plan.

SUGGESTIONS

  • The finding's status: open: its body says nvme: the controller's own idle power states, by APST #1131's measurement carries the separating arm, which reads as assigned. A line in the slow-write defect record pointing at the finding would let its reader find it.
  • For the orchestrator, the PR description: "alone on the bench's NVMe" and "66% slower at worst" need the corrections of blockers 1 and 3.
  • The evidence's ab/README.md has the same two gaps (r3-M classed fast, the T14 sequential-write row unmentioned). It is append-only and need not change; the records are where the correction belongs.

Records against issues/README.md

  • rejected with status: none, and its body owes nothing: correct. The ruling is attributed to the orchestrator, which is what A journal entry is one flush and a forced write: FUA through the block path #1202's closing comment says.
  • finding with status: open, no severity, an exit that is a measurement: correct in form.
  • Both name only the tracker's standing bench names. Revisions c9d6eff6f, 8a8ff27b9 and 24f5de778 resolve.
  • Checked and accurate: the journal's flush, entry, flush (lib/bcachefs/src/engine/mount/mod.rs, write_entries); the AMD sample (min 3.71, p50 4.01, p90 4.25 is amd-r2-F); the after-idle row's eight samples each after 10 s, with 80 s between its neighbours' timestamps in every P boot; the T14 ranges; Linux's 0.79 to 0.92 ms; that nvme: the controller's own idle power states, by APST #1131's staged measurement carries an arm with the idle row and no APST.

Evidence

  • Host log, read in full for failures. Revision line 72ce4d25e. 1 of 110 step(s) red: the build system: cargo test --lib, exit 101. Its one failing test is llvm::tests::the_caller_s_environment_reaches_neither_the_build_nor_the_key, with the SDKROOT=/no/such.sdk git failure that issues/the-llvm-environment-test-breaks-git-on-macos.md records. Every other FAILED line sits inside a control step, where a failure is the expected verdict. the harness's own checks is green, including the metal selection tests. I accept the red as independent.
  • No guest or metal run is owed, as the PR says. The registration ran on both benches at an identical patch, all 24 boots verdict: passed.
  • Identifiers. Neither evidence commit holds a home path, MAC, IP address, serial, hostname or model string; paths and models are masked.

SEND BACK

Japabu and others added 3 commits November 11, 2026 10:29
`osbench_disk` runs `files`, `nvme` and `persist` alone on the bench's
NVMe, so a change to the disk's path or its power states is measured
without the whole suite's boot. PR #1202's and PR #1131's A/B ran on it.
Only a pass that names it runs it: the nightly `osbench` boot measures the
same groups.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…low mode ending in idle

PR #1202 wrote the bcachefs journal's entries with Force Unit Access, one
Flush fewer per commit. Its A/B on both benches found it no faster at best
(the T14, fast mode) and 66% slower at worst (the AMD), so it is closed and
nothing of it lands. The record says what was measured, on which
revisions, and the one open question, so it is not retried blind.

The same boots showed the T14's slow durable writes ending during the
after-idle row's 80 s of idle in four boots of four. That arm also set
APST, so which of the two ends the mode is not separated; recorded as a
finding with the measurement that separates them as its exit.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…rds hold what was measured

1. The row's comments called all three groups the disk's figures. Only
   `persist` writes the bench's NVMe; `files` is the memory filesystem's
   rows and `nvme` is read-only and unsupported on ToyOS. The comments and
   the README say so. No group is dropped, so the image is as it was.
2. The bound's comment said no boot had timed the groups apart from the
   suite. Twenty-four have. It now says only that the suite's bound is
   reused as a ceiling.
3. The rejected record omitted the one result that points the other way:
   on the T14 `fs.disk.seq_write.1048576` was 24% faster forced in every
   boot, and 13% slower on the AMD bench. It is stated, with why the
   ruling stands. "No faster at best" is scoped to the durable rows, the
   worst figure is 71%, the third arm is named with its revision and its
   APST, and the Linux column is labelled as three runs.
4. The T14 finding's control was wrong: one boot with neither idle nor
   APST left the slow mode by itself. The record counts it, names the
   third case, and makes its exit a count against that base rate. It is
   `assigned` to PR #1131's measurement, and the slow-write record points
   at it.
5. The storage track still called the forced write assigned work. It now
   points at the rejected record; the two-Flush cost stays.

Comments and records only.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@Japabu

Japabu commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

Orchestrator verification at 6526e1c (rebased onto main 2ec4ad9), after review r1's SEND BACK at 72ce4d2 (five blockers, comments and records).

No code changed: the non-comment lines this branch adds to tests/toyos.rs are identical before and after (same hash of the diff against main with comment lines removed); the other files are the osbench README and four issue records.

  1. The row's comments say what each group is for: only persist writes the bench's NVMe, files is the memory-filesystem counterpart, nvme is read-only and unsupported on ToyOS. The README line follows.
  2. The bound's comment says only that the suite's bound is reused as a ceiling.
  3. The rejected record scopes "no faster at best" to the durable rows, gives 71% as the worst, names the third arm (24f5de7, which also set APST), labels the Linux column as three runs, and holds the row that points the other way (fs.disk.seq_write.1048576: 24% faster forced on the T14 in every boot, 13% slower on the AMD bench) with the reason the ruling stands.
  4. The T14 finding counts round 3 of main as leaving the slow mode by itself, states the unprompted case, makes its exit a count against that one-in-seven rate, and is assigned; the slow-write record points at it.
  5. issues/storage-is-layers-and-a-role-is-a-filesystem.md no longer lists the forced write as assigned work and points at the rejected record.

Queueing.

@Japabu
Japabu marked this pull request as ready for review November 11, 2026 08:38
@Japabu
Japabu added this pull request to the merge queue Nov 11, 2026
Merged via the queue into main with commit 9a4e2ac Nov 11, 2026
10 checks passed
@Japabu
Japabu deleted the osbench-disk-row branch November 11, 2026 08:58
Japabu added a commit that referenced this pull request Nov 11, 2026
…both benches

Four osbench boots a bench interleaved with five Linux runs, the Linux
reference taken in the same session on the same machine. The T14 reads 70
rows behind Linux, 52 ahead, 7 at parity and 5 unresolved; the AMD bench 35,
76, 8 and 15. The home record carries every row behind, ranked, with the
record that owns it.

Four rows behind had no record and get one: a clock read on the T14, a wake
onto a busy CPU on both benches, the T14's 4- and 64-waiter storm drains,
and the one-group all-CPU total. The yield record's row is measured now
(3.41x on the T14, 1.13x on the AMD bench), so it is a defect under a slug
that says so. The spawn, memory-filesystem, file-server, small-message,
ping-pong, memcpy and user-memory records gain this session's table.

The T14's durable-write record is left to #1215, which rewrites it; this
session read its slow mode in four boots of four.

Evidence: https://github.com/ToyOSOrg/evidence/tree/d44ebdae1f9f584d7b826e1498dbfd791a8d2339

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant