diff --git a/issues/no-published-run-compares-toyos-with-linux-on-one-machine.md b/issues/no-published-run-compares-toyos-with-linux-on-one-machine.md index 24e214b8c1c..4ce5bfc1134 100644 --- a/issues/no-published-run-compares-toyos-with-linux-on-one-machine.md +++ b/issues/no-published-run-compares-toyos-with-linux-on-one-machine.md @@ -34,6 +34,13 @@ its groups: - a short CPU crunch, one thread then all, that stops at TjMax less 15 C where the package's temperature can be read: `crunch.*`. +`tests/guest/osbench/README.md` is how a comparison is run: both halves from +one revision, interleaved runs, Linux as installed for the baseline, and +Linux under ToyOS's frequency request as a labelled experiment. `persist` is +not yet one disk: ToyOS writes the boot stick's LOG volume, and moves to the +bench's NVMe once ToyOS designates its DATA partition itself. The README +says why the AMD bench's Linux keeps time on the HPET. + `internet.*`, HTTPS to Cloudflare's speed test, is exploratory evidence of a path neither system owns, run only when a pass names its row: no automated test depends on the public internet. diff --git a/issues/osbench-times-cpus-leaving-idle-and-not-their-settled-clock.md b/issues/osbench-times-cpus-leaving-idle-and-not-their-settled-clock.md index 1bcde5520b8..ff18a6dd5d1 100644 --- a/issues/osbench-times-cpus-leaving-idle-and-not-their-settled-clock.md +++ b/issues/osbench-times-cpus-leaving-idle-and-not-their-settled-clock.md @@ -28,9 +28,19 @@ counter reading. Since round 2 every benchmark but the wake and fairness groups' (whose CPUs idle or are all busy by design) runs after `clock::settle`, which spins the -measuring CPU until three probes of a dependent `add` chain agree within 1%, -and every result line prints that clock as `mhz_settled` and the clock just -after as `mhz`; each group also prints `cpu.clock`, 200 probes from idle. +measuring CPU until every reading of a dependent `add` chain for 50 ms +agrees within 3% (`osbench_rules::settle`), or for 0.5 s, and every result +line prints that clock as `mhz_settled`, the clock a second settle reaches +after the row as `mhz_after`, and the clock just after as `mhz`; each group +also prints `cpu.clock`, 200 probes from idle. A reading is the difference +of a short add chain's time and a long one's, so the cost of reading the +time is no part of it. Three probes in a row, the rule until 2026-10-27, +took the AMD bench's `schedutil` step for the settled clock: its Linux runs +of 2026-10-24 settled at 1,357 MHz. On 2026-10-28 every settle of the AMD's +Linux ran out its time under a 1% rule (84 of 84) where ToyOS's on the same +CPU never did (0 of 92). That Linux timed through the HPET, whose 1.2 µs a +read sat inside a 25 µs probe, reading the clock some 5% low and moving it +by the read's jitter. **Exit**: a T14 run on each system shows `mhz_settled` near the all-CPU rate per CPU within the run-to-run spread, or names the benchmark whose CPU did diff --git a/issues/user-memory-hot-paths-are-slower-than-2-mib-pages.md b/issues/user-memory-hot-paths-are-slower-than-2-mib-pages.md index c363a28f498..497c7d6c12d 100644 --- a/issues/user-memory-hot-paths-are-slower-than-2-mib-pages.md +++ b/issues/user-memory-hot-paths-are-slower-than-2-mib-pages.md @@ -38,6 +38,8 @@ did; the T14's numbers are below: (`UserTables::leaf_table`); cause not found. Accepted as a measured tradeoff by the coordinator on PR #1055, under the owner's delegation of implementation tradeoffs (2026-10-23), against about 380 ns saved per page touched in order. + That row now faults every 4 KiB (PR #1079, so both systems fault the + same bytes), so no osbench row measures a lone fault any more. - **Spawn.** `proc.spawn_wait_loop.100` rose 2.3% at `a75abd3f6` and is within 1.5% at `6d826e313`. - **Frees.** A clear walks the 64 futex buckets once per table of leaves that diff --git a/tests/guest/osbench/README.md b/tests/guest/osbench/README.md new file mode 100644 index 00000000000..a08e43896d3 --- /dev/null +++ b/tests/guest/osbench/README.md @@ -0,0 +1,151 @@ +# osbench: ToyOS and Linux on one bench + +`osbench` is one program built for both systems (`src/main.rs`). A comparison +is one bench's runs of each, read together by `osbench-compare`. What follows +is what makes those runs comparable. A run taken any other way measures the +setup, not the system. + +## Build both halves from one revision + +The ToyOS half is the `osbench` metal row's image (`tests/images/benchcase.toml`): + +```text +cargo test --test toyos-build -- --metal --metal-readback osbench +``` + +The Linux half is static (`x86_64-unknown-linux-musl`, linked by the +toolchain's own LLD per `.cargo/config.toml`), so a bench runs it without a +toolchain: + +```text +rustup target add --toolchain nightly-2026-07-22 x86_64-unknown-linux-musl +cargo +nightly-2026-07-22 build -p osbench --target x86_64-unknown-linux-musl --profile toyos +``` + +That nightly is the build's stage-0 compiler (`build/sysroot.rs`'s +`STAGE0_CARGO`), the upstream one nearest ToyOS's own. With the `toyos` +profile osbench's own code is generated alike for both; std's platform layer +and the C library beneath it are each system's own. + +## Run + +- **At least four runs of each system, interleaved**: Linux, ToyOS, Linux, + ToyOS. A ToyOS run is one boot and one readback directory. A Linux run is + `osbench all osbench`, + with `--metal-host` for a bench other than the T14. +- **Compare** every run of the bench in one call: + `cargo run -p osbench-compare -- `. It + refuses runs of one system that disagree on their setup (`SETUP`) and + states each system's, and what each run's boot chose (`PER_RUN`, the + clocksource) run by run. Each row's verdict says whether the runs resolve a + difference. Each p is its row's alone: with no difference anywhere, about + 2.9% of rows resolve by chance, a few in a table this size. +- **The clocks are told beside each verdict.** No program reads the + package's limit state or temperature on ToyOS, so the clock is what both + systems report. A clock reading is the difference between a short add + chain's time and a long one's, so what reading the time costs is no part + of it; a settle holds once every reading for 50 ms agrees within 3%. A + settled row carries the clock settled before it + (`mhz_settled`) and after it (`mhz_after`). The verdict counts per system + the runs where the second is under nine tenths of the first. A power or + heat limit reached inside the row is the likely cause, an inference: the + AMD bench's package limit held a long single-thread row at 1.4 to 1.9 GHz + after a 4.29 GHz start. A row that began already limited is not marked + that way, since both its settles read the limited clock. So the verdict + also says where the systems' median settled clocks lie more than a tenth + apart. A settle that ran out its 0.5 s says so (`unsettled`), and is + counted rather than judged. + +## Linux as installed, and Linux under ToyOS's requests + +**The baseline is Linux as the bench boots it**: its own governor, its own +clocksource, its own checks. What it does on that machine is ordinary OS +behaviour and part of what is compared. A comparison names its +configuration in its setup lines, and `osbench-compare` refuses to mix two. +What a boot chooses for itself, the clocksource, belongs to Linux as booted: +each run states its own, and runs that differ are compared together. + +**A matched request is a controlled experiment, labelled as one.** It asks +the CPU's frequency control for what ToyOS's kernel asks of the same CPU, so +the clock is the hardware's answer to one request on both systems. It +answers a different question from the baseline, and never replaces it: + +- **The T14** (i5-1135G7, `intel_pstate` with HWP) has no experiment: its + baseline already makes ToyOS's request but for the floor. ToyOS asks for + HWP's lowest to highest performance at EPP 0x80 (`kernel/pure/cpu.rs`). + The baseline's `IA32_HWP_REQUEST` read `0x80002a04` on every CPU + (2026-10-28): EPP 0x80 (`epp=balance_performance*8`), highest 0x2a, and + lowest 4 where ToyOS asks 1. The floor reaches a CPU waking from idle, not + a settled row. Check `governor=powersave*8`, `epp=balance_performance*8` + and `intel_no_turbo=0`. +- **The AMD bench** (Ryzen 7 4800U, `acpi-cpufreq`). ToyOS writes no P-state + or boost register, so the firmware's P0 and Core Performance Boost stand. + Linux's `performance` governor asks for that. The experiment sets it + before each Linux run, since every ToyOS boot between them boots Linux + back to `schedutil`, and gives `schedutil` back after the last: + + ```text + echo performance | sudo tee /sys/devices/system/cpu/cpufreq/policy*/scaling_governor + echo schedutil | sudo tee /sys/devices/system/cpu/cpufreq/policy*/scaling_governor + ``` + + Check `governor=performance*16` and `boost=1`. + +When ToyOS's request changes, the experiment changes with it. + +**The groups.** `fair.apart` sets each workload in a group of its own: a +ToyOS Job, and on Linux a scope's cgroup, which is a group only where the +user's manager delegates the `cpu` controller. Check +`fair.apart.units=3xcgroup`. With `1xcgroup` Linux's `apart` is +thread-fair, and the pair compares nothing. + +**The clock.** Every timestamp is read from the TSC on ToyOS. The AMD +bench's Linux chooses its clocksource at each boot. Its watchdog moves it +off the TSC when, early in boot, the TSC and the HPET disagree. On the HPET +each `Instant::now` is a syscall's cost, 1.21 µs against the T14's 14 ns, +carried by every per-operation timing of that run. In 30 boots' kernel logs +(2026-10-28) the watchdog fired in 17: +- in 12, Linux's refined calibration read 1799.54 to 1799.61 MHz, some + 1,630 ppm above the TSC's 1796.62 MHz; +- in 5, with a right calibration, the TSC ran some 1.7 ms ahead of the HPET + over half a second, a step whose cause is unknown. + +In a 13th boot the calibration read 1799.565 MHz and the watchdog let it +stand, so that Linux kept a TSC clock some 1,630 ppm fast for the whole +boot. Its `clocksource=tsc` reads the same as a right one's. The 12 others +kept the TSC on a right calibration. Each run's clocksource is part of the +Linux it measured, so the compare states it per run. + +At runtime neither fault shows: across 16 CPUs and idle gaps up to 1 s, no +CPU's TSC read behind another's and no step exceeded the 1.2 µs of a read. +A Linux forced onto the TSC (`tsc=reliable`) has those checks switched off, +so it is no baseline. + +## The disk is not yet one disk + +`persist` writes each system's persistent filesystem. On Linux that is the +runner's home, ext4 on the bench's NVMe. On ToyOS it is still the boot +stick's LOG volume, FAT32 over USB 2.0, because ToyOS's DATA is the bench's +partition only once ToyOS has designated it, and ToyOS cannot yet do that +itself. So `persist` compares two media. Its lines name theirs, and +`osbench-compare` refuses a verdict on any row whose media differ. + +## What is not a comparison + +The table also holds rows only one system can measure. They show the +system's figure beside the other's reason: `unsupported` and `unmeasured` +rows, Linux's `.slack1ns` variants and `fair.session`, and ToyOS's +`tail.diag`. They are records, not verdicts. diff --git a/tests/guest/osbench/compare/src/lib.rs b/tests/guest/osbench/compare/src/lib.rs index 58c197eebe3..feea0194dac 100644 --- a/tests/guest/osbench/compare/src/lib.rs +++ b/tests/guest/osbench/compare/src/lib.rs @@ -11,6 +11,29 @@ //! uncorrected for the table's other rows. Better or worse is judged only //! for a unit whose better direction is known ([`better`]); for any other //! the verdict says which way ToyOS differs. +//! +//! Each row's clocks are told beside its verdict, since no program reads a +//! power or heat limit on both systems: +//! +//! - in how many runs the clock settled after the row was under nine tenths +//! of the one settled before it ([`clock_fell`]), which a limit reached +//! inside the row most likely explains; +//! - where the two systems' median clocks settled before the row lie more +//! than a tenth apart ([`clocks_apart`]), which also catches a row that +//! started already limited, as a fall cannot; +//! - in how many runs a settle ran out its time (`unsettled`), whose clocks +//! say nothing either way. +//! +//! A row whose lines name their medium (`medium=`, the memory or the kind of +//! disk under a filesystem) is compared only where every run's medium is one: +//! a row written to an NVMe disk under one system and a USB stick under the +//! other has a verdict of what the two media are, not of the systems. +//! +//! A system's runs on one machine are one configuration ([`SETUP`]): runs that +//! disagree on it are refused, since their median would mix two, and each +//! machine's table states what each system's runs were taken under. What a +//! system chooses afresh at each boot ([`PER_RUN`]) is part of that system as +//! booted: it is stated run by run and never refused. use std::collections::BTreeMap; use std::fmt::Write as _; @@ -26,9 +49,28 @@ pub enum Os { Toyos, } +impl Os { + pub fn name(self) -> &'static str { + match self { + Os::Linux => "Linux", + Os::Toyos => "ToyOS", + } + } +} + #[derive(Clone, Debug, PartialEq, Eq)] pub enum Outcome { - Measured { unit: String, p50: u64 }, + /// `clock_fell` as [`clock_fell`] judges the line's two clocks, unless + /// either is `unsettled`; `settled` the clock settled before it; `medium` + /// as the line names it. + Measured { + unit: String, + p50: u64, + clock_fell: bool, + settled: Option, + unsettled: bool, + medium: Option, + }, /// Unmeasured, unsupported or refused, with the run's reason. Not { kind: String, why: String }, } @@ -49,7 +91,7 @@ impl Run { /// One run's lines. A benchmark a run reports in more than one group, as /// every group's `cpu.clock`, is named `@`, and the `k`th of -/// one name in a group, as `fair.flat`'s jobs, `#k`. A run with no +/// one name in a group `#k`. A run with no /// osbench line, two systems' lines or one group started twice is refused. pub fn parse(text: &str) -> Result { let mut os = None; @@ -89,7 +131,16 @@ pub fn parse(text: &str) -> Result { let fields: BTreeMap<&str, &str> = words.filter_map(|w| w.split_once('=')).collect(); let outcome = if let Some(unit) = fields.get("unit") { let p50 = fields.get("p50").and_then(|v| v.parse().ok()).ok_or_else(|| format!("{bench}: no p50"))?; - Outcome::Measured { unit: unit.to_string(), p50 } + let mhz = |key: &str| fields.get(key).and_then(|v| v.parse().ok()); + let unsettled = fields.contains_key("unsettled"); + Outcome::Measured { + unit: unit.to_string(), + p50, + clock_fell: !unsettled && clock_fell(mhz("mhz_settled"), mhz("mhz_after")), + settled: mhz("mhz_settled"), + unsettled, + medium: fields.get("medium").map(|m| m.to_string()), + } } else if let Some((kind, why)) = ["unmeasured", "unsupported", "refused"].iter().find_map(|k| fields.get(k).map(|why| (*k, *why))) { @@ -114,6 +165,19 @@ pub fn parse(text: &str) -> Result { Ok(Run { os: os.ok_or("no osbench line")?, conds, benches }) } +/// Whether the clock fell across a measurement: the clock settled after it +/// under nine tenths of the clock settled before it. A line that settled no +/// clock says nothing either way. +pub fn clock_fell(settled: Option, after: Option) -> bool { + matches!((settled, after), (Some(settled), Some(after)) if after * 10 < settled * 9) +} + +/// Whether two systems' median settled clocks lie more than a tenth of the +/// higher apart. +pub fn clocks_apart(a: f64, b: f64) -> bool { + (a - b).abs() * 10.0 > a.max(b) +} + pub fn median(values: &[u64]) -> f64 { let mut v = values.to_vec(); v.sort_unstable(); @@ -212,6 +276,63 @@ pub fn better(unit: &str) -> Option { /// varies by the run. const PATHS: [(&str, &str, usize); 2] = [("lan.", "lan.peer_link", 2), ("internet.", "internet.link", 1)]; +/// The conditions every run of one system on one machine must share: the +/// kernel, the CPU's frequency request, and what serves each filesystem the +/// `fs` and `persist` rows use. +pub const SETUP: [&str; 13] = [ + "kernel", + "cpufreq_driver", + "governor", + "epp", + "min_khz", + "max_khz", + "boost", + "intel_no_turbo", + "platform_profile", + "smt_active", + "fs.mem.served_by", + "fs.served.served_by", + "persist.served_by", +]; + +/// The conditions a system chooses at each boot, stated per run: the clock +/// every timestamp is read from, which Linux's watchdog can move off the TSC +/// in one boot and not the next. +pub const PER_RUN: [&str; 1] = ["clocksource"]; + +/// `os`'s [`PER_RUN`] conditions, each as `key=,...` in the +/// runs' order, `absent` for a run that names none. +fn per_run(runs: &[&Run], os: Os) -> Vec { + let runs: Vec<&&Run> = runs.iter().filter(|r| r.os == os).collect(); + PER_RUN + .iter() + .filter(|key| runs.iter().any(|r| r.conds.contains_key(**key))) + .map(|key| { + let said: Vec<&str> = runs.iter().map(|r| r.conds.get(*key).map_or("absent", String::as_str)).collect(); + format!("{key}={}", said.join(",")) + }) + .collect() +} + +/// `os`'s setup on these runs, as `key=value` in [`SETUP`]'s order, or the +/// first key its runs disagree on. +fn setup(runs: &[&Run], os: Os) -> Result, String> { + let runs: Vec<&&Run> = runs.iter().filter(|r| r.os == os).collect(); + let mut said = Vec::new(); + for key in SETUP { + let values: std::collections::BTreeSet> = runs.iter().map(|r| r.conds.get(key)).collect(); + match values.iter().collect::>()[..] { + [] | [None] => {} + [Some(value)] => said.push(format!("{key}={value}")), + _ => { + let seen: Vec<&str> = values.iter().map(|v| v.map_or("absent", String::as_str)).collect(); + return Err(format!("the {} runs disagree on {key}: {}", os.name(), seen.join(" / "))); + } + } + } + Ok(said) +} + fn path(said: &str, words: usize) -> String { said.split(['_', ' ']).take(words).collect::>().join(" ") } @@ -239,6 +360,16 @@ pub fn compare(runs: &[Run]) -> Result { write!(out, " {cond}: {seen}.").unwrap(); } writeln!(out, "\n").unwrap(); + for os in [Os::Linux, Os::Toyos] { + let said = setup(&runs, os).map_err(|why| format!("{machine}: {why}"))?; + if !said.is_empty() { + writeln!(out, "{} setup: {}.\n", os.name(), said.join(", ")).unwrap(); + } + let each = per_run(&runs, os); + if !each.is_empty() { + writeln!(out, "{} per run: {}.\n", os.name(), each.join(", ")).unwrap(); + } + } writeln!(out, "| bench | unit | Linux p50s | Linux median | Linux range | ToyOS p50s | ToyOS median | ToyOS range | ToyOS/Linux | p | verdict |").unwrap(); writeln!(out, "|---|---|---|---|---|---|---|---|---|---|---|").unwrap(); let names: std::collections::BTreeSet<&String> = runs.iter().flat_map(|r| r.benches.keys()).collect(); @@ -262,14 +393,22 @@ pub fn compare(runs: &[Run]) -> Result { fn compare_one(runs: &[&Run], name: &str) -> String { let mut unit = None; let mut values: BTreeMap> = BTreeMap::new(); + let mut fell: BTreeMap = BTreeMap::new(); + let mut unsettled_runs: BTreeMap = BTreeMap::new(); + let mut settled: BTreeMap> = BTreeMap::new(); + let mut media: BTreeMap>> = BTreeMap::new(); let mut not: BTreeMap> = BTreeMap::new(); for run in runs { match run.benches.get(name) { - Some(Outcome::Measured { unit: u, p50 }) => { + Some(Outcome::Measured { unit: u, p50, clock_fell, settled: mhz, unsettled, medium }) => { if unit.get_or_insert(u.as_str()) != u { return "| | | | | | | | | units differ between runs".into(); } values.entry(run.os).or_default().push(*p50); + *fell.entry(run.os).or_default() += usize::from(*clock_fell); + *unsettled_runs.entry(run.os).or_default() += usize::from(*unsettled); + settled.entry(run.os).or_default().extend(mhz.filter(|_| !*unsettled)); + media.entry(run.os).or_default().insert(medium.clone()); } Some(Outcome::Not { kind, why }) => not.entry(run.os).or_default().push(format!("{kind}: {why}")), None => not.entry(run.os).or_default().push("absent".into()), @@ -287,7 +426,17 @@ fn compare_one(runs: &[&Run], name: &str) -> String { }; let [lp, lm, lr] = side(Os::Linux); let [tp, tm, tr] = side(Os::Toyos); + // A side that names no medium is a medium of its own beside one that does. + let all_media: std::collections::BTreeSet<&Option> = media.values().flatten().collect(); + let differ = all_media.len() > 1 && all_media.iter().any(|m| m.is_some()); let (ratio, p, verdict) = match (values.get(&Os::Linux), values.get(&Os::Toyos)) { + (Some(_), Some(_)) if differ => { + let named = |os: Os| { + let said = media.get(&os).into_iter().flatten().map(|m| m.as_deref().unwrap_or("unnamed")); + said.collect::>().join("+") + }; + ("-".into(), "-".into(), format!("the media differ: Linux {}, ToyOS {}", named(Os::Linux), named(Os::Toyos))) + } (Some(l), Some(t)) => { let ratio = median(t) / median(l); let exact = mann_whitney(t, l); @@ -309,13 +458,28 @@ fn compare_one(runs: &[&Run], name: &str) -> String { _ => ("-".into(), "-".into(), String::new()), }; let mut verdict = verdict; + let mut note = |said: String| { + let sep = if verdict.is_empty() { "" } else { "; " }; + verdict = format!("{verdict}{sep}{said}"); + }; + if let (Some(l), Some(t)) = (settled.get(&Os::Linux).filter(|v| !v.is_empty()), settled.get(&Os::Toyos).filter(|v| !v.is_empty())) { + let (l, t) = (median(l), median(t)); + if clocks_apart(l, t) { + note(format!("settled clocks apart: Linux {l:.0} MHz, ToyOS {t:.0} MHz")); + } + } + for (counts, said) in [(&fell, "the clock fell"), (&unsettled_runs, "the clock did not settle")] { + for (os, n) in counts.iter().filter(|(_, n)| **n > 0) { + let of = values.get(os).map_or(0, Vec::len); + note(format!("{said} in {n} of {of} {} run(s)", os.name())); + } + } for (os, reasons) in ¬ { let mut distinct = reasons.clone(); distinct.sort(); distinct.dedup(); - let who = if *os == Os::Linux { "Linux" } else { "ToyOS" }; let sep = if verdict.is_empty() { "" } else { "; " }; - verdict = format!("{verdict}{sep}{who} {} run(s) {}", reasons.len(), distinct.join(" / ")); + verdict = format!("{verdict}{sep}{} {} run(s) {}", os.name(), reasons.len(), distinct.join(" / ")); } format!("{} | {lp} | {lm} | {lr} | {tp} | {tm} | {tr} | {ratio} | {p} | {verdict}", unit.unwrap_or_default()) } @@ -345,7 +509,10 @@ mod tests { ); assert_eq!(r.os, Os::Toyos); assert_eq!(r.machine(), Some("Bench_CPU")); - assert_eq!(r.benches["lan.wired.tcp.rtt.64"], Outcome::Measured { unit: "ns".into(), p50: 812 }); + assert_eq!( + r.benches["lan.wired.tcp.rtt.64"], + Outcome::Measured { unit: "ns".into(), p50: 812, clock_fell: false, settled: None, unsettled: false, medium: None } + ); assert_eq!( r.benches["lan.wifi.tcp.upload"], Outcome::Not { kind: "unsupported".into(), why: "ToyOS_has_no_Wi-Fi_driver".into() } @@ -360,8 +527,8 @@ mod tests { let groups = ["sched", "fair"].map(|g| format!("osbench v=2 os=linux group={g} start\nosbench v=2 os=linux bench=cpu.clock unit=MHz n=1 min=1 p50=4000 max=1 mean=1\n")).concat(); let run = parse(&groups).unwrap(); assert_eq!(run.benches.keys().collect::>(), ["cpu.clock@fair", "cpu.clock@sched"]); - let jobs = "osbench v=2 os=linux bench=fair.flat unit=permille n=5 min=1 p50=63 max=1 mean=1 job=build\n".repeat(2); - assert_eq!(parse(&jobs).unwrap().benches.keys().collect::>(), ["fair.flat#1", "fair.flat#2"]); + let jobs = "osbench v=2 os=linux bench=wake.storm unit=ns n=5 min=1 p50=63 max=1 mean=1\n".repeat(2); + assert_eq!(parse(&jobs).unwrap().benches.keys().collect::>(), ["wake.storm#1", "wake.storm#2"]); let mixed = "osbench v=2 os=linux cond cpus=8\nosbench v=2 os=toyos cond cpus=8\n"; assert!(parse(mixed).unwrap_err().contains("two systems")); assert!(parse("nothing here\n").is_err()); @@ -486,6 +653,86 @@ mod tests { assert!(compare(&runs).unwrap().contains("| the runs disagree on lan.peer_link |")); } + #[test] + fn a_row_whose_clock_fell_in_a_run_says_so_per_system() { + assert!(clock_fell(Some(4289), Some(1884))); + assert!(!clock_fell(Some(4289), Some(3861)), "a tenth down is not a fall"); + assert!(clock_fell(Some(4289), Some(3860))); + assert!(!clock_fell(None, Some(1396)), "no settled clock, no judgement"); + let deflate = |os: &str, p50: u64, after: u64| { + run(os, &[&format!("bench=crunch.single.deflate unit=ns n=63 min=1 p50={p50} max=1 mean=1 mhz_settled=4289 mhz_after={after} mhz=1300")]) + }; + let mut runs: Vec = [(65, 4200), (66, 4210), (64, 4190), (65, 4200)].iter().map(|&(p, m)| deflate("linux", p, m)).collect(); + runs.extend([(63, 1884), (66, 1396), (63, 4200), (64, 4280)].iter().map(|&(p, m)| deflate("toyos", p, m))); + let table = compare(&runs).unwrap(); + let row = table.lines().find(|l| l.starts_with("| crunch.single.deflate")).unwrap(); + assert!(row.ends_with("not resolved; the clock fell in 2 of 4 ToyOS run(s) |"), "{row}"); + } + + #[test] + fn clocks_settled_apart_and_settles_cut_short_are_told() { + assert!(clocks_apart(4289.0, 1884.0)); + assert!(!clocks_apart(4186.0, 3788.0), "9.5% apart is within a tenth"); + assert!(clocks_apart(4186.0, 3767.0)); + let at = |os: &str, v: u64, settled: u64, unsettled: &str| { + run(os, &[&format!("bench=fs.mem.stat unit=ns n=2000 min=1 p50={v} max=1 mean=1 mhz_settled={settled} mhz_after=1000{unsettled} mhz=1")]) + }; + let mut runs: Vec = [10, 11, 12, 13].iter().map(|v| at("linux", *v, 4200, "")).collect(); + runs.extend([20, 21, 22].iter().map(|v| at("toyos", *v, 1884, ""))); + runs.push(at("toyos", 23, 4200, " unsettled=after")); + let row = compare(&runs).unwrap().lines().find(|l| l.starts_with("| fs.mem.stat")).unwrap().to_string(); + assert!(row.contains("settled clocks apart: Linux 4200 MHz, ToyOS 1884 MHz"), "{row}"); + assert!(row.contains("the clock fell in 4 of 4 Linux run(s)"), "{row}"); + assert!(row.contains("the clock fell in 3 of 4 ToyOS run(s); the clock did not settle in 1 of 4 ToyOS run(s) |"), "{row}"); + } + + #[test] + fn a_row_on_two_media_is_no_comparison_of_the_systems() { + let fsync = |os: &str, v: u64, medium: &str| { + run(os, &[&format!("bench=persist.fsync_clean unit=ns n=200 min=1 p50={v} max=1 mean=1 medium={medium}")]) + }; + let mut runs: Vec = [67, 68, 66, 67].iter().map(|v| fsync("linux", *v, "nvme")).collect(); + runs.extend([1687, 1690, 1700, 1695].iter().map(|v| fsync("toyos", *v, "usb"))); + let row = compare(&runs).unwrap().lines().find(|l| l.starts_with("| persist.fsync_clean")).unwrap().to_string(); + assert!(row.ends_with("| - | - | the media differ: Linux nvme, ToyOS usb |"), "{row}"); + let mut runs: Vec = [67, 68, 66, 67].iter().map(|v| fsync("linux", *v, "nvme")).collect(); + runs.extend([70, 71, 72, 73].iter().map(|v| fsync("toyos", *v, "nvme"))); + assert!(compare(&runs).unwrap().contains("| ToyOS worse |"), "one medium is a comparison"); + let mut runs: Vec = [67, 68, 66].iter().map(|v| fsync("linux", *v, "nvme")).collect(); + runs.push(fsync("linux", 69, "sata")); + runs.extend([70, 71, 72, 73].iter().map(|v| fsync("toyos", *v, "nvme"))); + assert!(compare(&runs).unwrap().contains("the media differ: Linux nvme+sata, ToyOS nvme"), "nor one system's runs on two"); + let mut runs: Vec = [67, 68, 66, 67].iter().map(|v| fsync("linux", *v, "nvme")).collect(); + runs.extend([70, 71, 72, 73].iter().map(|v| run("toyos", &[&format!("bench=persist.fsync_clean unit=ns n=200 min=1 p50={v} max=1 mean=1")]))); + assert!(compare(&runs).unwrap().contains("the media differ: Linux nvme, ToyOS unnamed"), "nor a named medium and none"); + } + + #[test] + fn a_systems_runs_on_two_setups_are_refused_and_one_setup_is_stated() { + let under = |os: &str, governor: &str| { + run(os, &[&format!("cond governor={governor}"), "cond clocksource=tsc", "bench=time.now unit=ps n=15 min=1 p50=14 max=1 mean=1"]) + }; + let mixed = [under("linux", "performance*16"), under("linux", "schedutil*16"), under("toyos", "absent")]; + let why = compare(&mixed).unwrap_err(); + assert!(why.contains("the Linux runs disagree on governor: performance*16 / schedutil*16"), "{why}"); + let one = [under("linux", "performance*16"), under("linux", "performance*16")]; + let table = compare(&one).unwrap(); + assert!(table.contains("Linux setup: governor=performance*16."), "{table}"); + let absent = [under("linux", "performance*16"), run("linux", &["bench=time.now unit=ps n=15 min=1 p50=14 max=1 mean=1"])]; + assert!(compare(&absent).unwrap_err().contains("disagree on governor: absent / performance*16")); + } + + #[test] + fn a_clocksource_each_boot_chose_is_stated_per_run_and_never_refused() { + let booted = |clocksource: &str| { + run("linux", &[&format!("cond clocksource={clocksource}"), "bench=time.now unit=ps n=15 min=1 p50=14 max=1 mean=1"]) + }; + let runs = [booted("hpet"), booted("tsc"), booted("hpet"), run("toyos", &["bench=time.now unit=ps n=15 min=1 p50=9 max=1 mean=1"])]; + let table = compare(&runs).unwrap(); + assert!(table.contains("Linux per run: clocksource=hpet,tsc,hpet."), "{table}"); + assert!(!table.contains("ToyOS per run"), "{table}"); + } + #[test] fn internet_runs_that_left_by_different_kinds_of_link_are_not_compared() { let left = |os: &str, said: &str| { diff --git a/tests/guest/osbench/rules/Cargo.toml b/tests/guest/osbench/rules/Cargo.toml index 017682bdf1a..b36bb67dccf 100644 --- a/tests/guest/osbench/rules/Cargo.toml +++ b/tests/guest/osbench/rules/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "osbench-rules" -description = "osbench's pure rules, host-tested: when a munmap pair's x2APIC readings admit its figure, where the CPU crunch stops for heat, an energy counter's span across its wrap, and whether two counters readings describe their row." +description = "osbench's pure rules, host-tested: when a munmap pair's x2APIC readings admit its figure, where the CPU crunch stops for heat, an energy counter's span across its wrap, whether two counters readings describe their row, and when a CPU's clock has settled." version = "0.1.0" edition = "2021" license = "MIT OR Apache-2.0" diff --git a/tests/guest/osbench/rules/src/lib.rs b/tests/guest/osbench/rules/src/lib.rs index 1b94183a951..eaa1737141b 100644 --- a/tests/guest/osbench/rules/src/lib.rs +++ b/tests/guest/osbench/rules/src/lib.rs @@ -1,8 +1,9 @@ //! The judgements osbench makes over what it read, apart from the reading, so //! a host tests them: whether a munmap pair's figure stands ([`pair`]), where //! the CPU crunch stops for heat ([`heat`]), how far an energy counter -//! advanced across its wrap ([`energy`]), and whether two counters readings -//! describe the row they were read around ([`window`]). +//! advanced across its wrap ([`energy`]), whether two counters readings +//! describe the row they were read around ([`window`]), and when a CPU's +//! clock has settled ([`settle`]). pub mod pair { //! A munmap pair is an initiator that unmaps and a sibling that touched @@ -394,3 +395,109 @@ pub mod window { } } } + +pub mod settle { + //! A clock has settled once every reading over [`STEADY_NS`] agrees + //! within 3%. A frequency governor can hold a CPU that just turned busy + //! at a lower clock for tens of milliseconds before it steps up, so a + //! span much shorter than that takes the step for the settled clock. The + //! smallest steps measured are the AMD bench's 1.65 to 1.85 GHz and the + //! T14's 3.79 to 4.19 GHz, 11% and 9.5%; a boosting CPU's readings + //! agreed within 1% wherever reading the time was cheap. + + /// How long the readings must agree. + pub const STEADY_NS: u64 = 50_000_000; + + /// The clock in MHz that `cycles` more adds in the long chain than in + /// the short one say, timed at `long_ns` and `short_ns`: `None` where the + /// long chain took no longer, as when the short one was preempted, which + /// would otherwise read as a clock of many THz. + pub fn probe_mhz(cycles: u64, short_ns: u64, long_ns: u64) -> Option { + long_ns.checked_sub(short_ns).filter(|&ns| ns > 0).map(|ns| cycles * 1000 / ns) + } + + /// The readings since the last one that disagreed. + #[derive(Clone, Copy, Debug, PartialEq, Eq)] + pub struct Plateau { + since_ns: u64, + lo: u64, + hi: u64, + } + + impl Plateau { + pub fn new(at_ns: u64, mhz: u64) -> Self { + Plateau { since_ns: at_ns, lo: mhz, hi: mhz } + } + + /// The reading `mhz` taken at `at_ns`: whether every reading for + /// [`STEADY_NS`] up to it agrees within 3%. A reading that does not + /// agree with those before it starts the span again from itself. + pub fn settled_with(&mut self, at_ns: u64, mhz: u64) -> bool { + let (lo, hi) = (self.lo.min(mhz), self.hi.max(mhz)); + if (hi - lo) * 100 > hi * 3 { + *self = Plateau::new(at_ns, mhz); + } else { + (self.lo, self.hi) = (lo, hi); + } + at_ns - self.since_ns >= STEADY_NS + } + } + + #[cfg(test)] + mod tests { + use super::*; + + const MS: u64 = 1_000_000; + + /// The first time a probe every quarter millisecond from `at` settles. + fn settles(readings: impl Fn(u64) -> u64) -> Option<(u64, u64)> { + let mut plateau = Plateau::new(0, readings(0)); + (1..4000).map(|i| i * MS / 4).find_map(|at| plateau.settled_with(at, readings(at)).then(|| (at, readings(at)))) + } + + #[test] + fn a_governors_30_ms_step_is_not_the_settled_clock() { + let ramp = |at: u64| if at < 30 * MS { 1362 } else { 4033 }; + assert_eq!(settles(ramp), Some((80 * MS, 4033))); + } + + #[test] + fn the_smallest_measured_steps_are_not_the_settled_clock() { + for (from, to) in [(1650, 1850), (3789, 4189)] { + let step = |at: u64| if at < 30 * MS { from } else { to }; + assert_eq!(settles(step), Some((80 * MS, to)), "{from} to {to} MHz"); + } + } + + #[test] + fn readings_within_a_percent_or_two_hold_the_span() { + let wobble = |at: u64| if (at / MS).is_multiple_of(2) { 4241 } else { 4170 }; + assert_eq!(settles(wobble), Some((50 * MS, wobble(50 * MS)))); + } + + #[test] + fn a_reading_that_disagrees_starts_the_span_again() { + let mut plateau = Plateau::new(0, 4000); + assert!(!plateau.settled_with(40 * MS, 4000)); + assert!(!plateau.settled_with(41 * MS, 3700), "7.5% below"); + assert!(!plateau.settled_with(90 * MS, 3700), "49 ms since the disagreement"); + assert!(plateau.settled_with(91 * MS, 3700)); + } + + #[test] + fn a_probe_reads_the_difference_and_a_preempted_short_chain_reads_nothing() { + // 300,000 cycles over 70.7 µs, ToyOS's 4,241 MHz on the AMD bench. + assert_eq!(probe_mhz(300_000, 23_600, 94_340), Some(4240)); + // The time read's cost, in both chains alike, cancels. + assert_eq!(probe_mhz(300_000, 23_600 + 1_210, 94_340 + 1_210), Some(4240)); + assert_eq!(probe_mhz(300_000, 500_000, 94_340), None, "short chain preempted past the long"); + assert_eq!(probe_mhz(300_000, 94_340, 94_340), None); + } + + #[test] + fn a_slow_drift_settles_only_once_it_stops() { + let drift = |at: u64| 3000 + (at / MS).min(200) * 5; + assert_eq!(settles(drift).map(|(_, mhz)| mhz), Some(4000)); + } + } +} diff --git a/tests/guest/osbench/src/clock.rs b/tests/guest/osbench/src/clock.rs index e6a7fe2eeae..c317f321ee5 100644 --- a/tests/guest/osbench/src/clock.rs +++ b/tests/guest/osbench/src/clock.rs @@ -1,62 +1,101 @@ //! The clock a CPU is running at, read the same way on both systems: //! `arch::add_chain`'s adds wait a cycle each, so cycles are the number of -//! adds. +//! adds. A probe times a short chain and a long one and takes the +//! difference, so the cost of reading the time, which both timings carry, is +//! no part of the clock it reads: 1.2 µs a read on a Linux timing through the +//! HPET, a twentieth of a 100,000-cycle chain. -use std::sync::atomic::{AtomicU64, Ordering}; +use std::sync::Mutex; use std::time::{Duration, Instant}; +use osbench_rules::settle::{probe_mhz, Plateau}; + use crate::report::result; use crate::ns_since; -/// What the last [`settle`] settled at, for the next result line to print. -static SETTLED: AtomicU64 = AtomicU64::new(0); +/// What the last [`settle`] reached, for the next result line to print. +static SETTLED: Mutex> = Mutex::new(None); + +/// The clock a settle reached, and whether it held: `capped` is a settle +/// that ran out its [`SETTLE_LIMIT`] and took the reading it had. +#[derive(Clone, Copy)] +pub struct Settled { + pub mhz: u64, + pub capped: bool, +} -/// Turns per probe: 100,000 cycles, 25 to 50 µs on either bench. +/// Turns in a probe's short chain, 100,000 cycles; its long one is +/// [`LONG`] times that, so the difference is 300,000 cycles: 70 µs at the +/// AMD bench's 4.24 GHz, 230 µs at 1.3 GHz. const TURNS: u64 = 1_000; +const LONG: u64 = 4; const ADDS: u64 = crate::arch::ADDS_PER_TURN; +/// The cycles a probe's difference spans. +const CYCLES: u64 = ADDS * TURNS * (LONG - 1); -fn burst() -> u64 { +fn chain(turns: u64) -> u64 { let t = Instant::now(); - crate::arch::add_chain(TURNS); - ns_since(t).max(1) + crate::arch::add_chain(turns); + ns_since(t) +} + +/// One probe for `cpu.clock`'s distribution, in MHz: `None` where the short +/// chain was preempted past the long one. +fn burst() -> Option { + let short = chain(TURNS); + probe_mhz(CYCLES, short, chain(TURNS * LONG)) } -/// The clock the calling CPU runs at now, in MHz: the fastest of five probes, -/// so an interrupt inside one does not read as a slow clock. +/// The clock the calling CPU runs at now, in MHz: the fastest of five of +/// each chain, taken in turn, so an interrupt inside one reads as neither a +/// slow clock nor, in the short chain, a fast one. A clock that rose so far +/// between the two that the fastest long chain took no longer than the +/// fastest short one is read again. pub fn mhz() -> u64 { - let ns = (0..5).map(|_| burst()).min().expect("five probes"); - ADDS * TURNS * 1000 / ns + loop { + let (mut short, mut long) = (u64::MAX, u64::MAX); + for _ in 0..5 { + short = short.min(chain(TURNS)); + long = long.min(chain(TURNS * LONG)); + } + if let Some(mhz) = probe_mhz(CYCLES, short, long) { + return mhz; + } + } } +/// The longest a settle spins before it takes the clock it has. +const SETTLE_LIMIT: Duration = Duration::from_millis(500); + /// A steady-state warm-up of the calling CPU, for a benchmark to run after: -/// probes back to back for at least 10 ms and until three in a row agree -/// within 1%, or for 200 ms. The clock it reached is printed beside the next -/// result as `mhz_settled`, and `mhz` beside it is the clock just after. +/// [`mhz`] back to back until `osbench_rules::settle` calls the clock +/// settled, or for [`SETTLE_LIMIT`]. The clock it reached is printed beside +/// the next result as `mhz_settled`, and `mhz` beside it is the clock just +/// after. pub fn settle() { let start = Instant::now(); - let mut last = [0u64; 3]; + let at = || u64::try_from(start.elapsed().as_nanos()).expect("a settle under 584 years"); + let mut plateau = Plateau::new(at(), mhz()); loop { - last.rotate_left(1); - last[2] = ADDS * TURNS * 1000 / burst(); - let (lo, hi) = (last.iter().min().copied().unwrap_or(0), last.iter().max().copied().unwrap_or(0)); - let steady = lo > 0 && (hi - lo) * 100 <= hi; - let spent = start.elapsed(); - if (steady && spent >= Duration::from_millis(10)) || spent >= Duration::from_millis(200) { - SETTLED.store(last[2], Ordering::Relaxed); + let reading = mhz(); + let held = plateau.settled_with(at(), reading); + if held || start.elapsed() >= SETTLE_LIMIT { + *SETTLED.lock().expect("no settle panics holding it") = Some(Settled { mhz: reading, capped: !held }); return; } } } -/// The clock the last [`settle`] reached, once: zero when none has run since. -pub fn take_settled() -> u64 { - SETTLED.swap(0, Ordering::Relaxed) +/// The clock the last [`settle`] reached, once: `None` when none has run +/// since. +pub fn take_settled() -> Option { + SETTLED.lock().expect("no settle panics holding it").take() } /// `cpu.clock`: 200 probes back to back after one, MHz. The first probes of a /// CPU leaving idle show its ramp; the rest, the clock it settles at. pub fn distribution() { burst(); - let mut mhz: Vec = (0..200).map(|_| ADDS * TURNS * 1000 / burst()).collect(); - result("cpu.clock", "MHz", &mut mhz, &[("adds_per_probe", (ADDS * TURNS).to_string())]); + let mut mhz: Vec = (0..200).filter_map(|_| burst()).collect(); + result("cpu.clock", "MHz", &mut mhz, &[("cycles_per_probe", CYCLES.to_string())]); } diff --git a/tests/guest/osbench/src/crunch.rs b/tests/guest/osbench/src/crunch.rs index 0fdffa4ed27..c9e60a4f168 100644 --- a/tests/guest/osbench/src/crunch.rs +++ b/tests/guest/osbench/src/crunch.rs @@ -218,6 +218,9 @@ fn run_phase(phase: &str, threads: usize, inputs: &Arc, heat: Option<&He continue; } } + // The clock before the slice, beside the one after it: a slice + // long enough to meet the package's power limit says so. + crate::clock::settle(); let around = crate::around::begin(); let (mut ns, elapsed, placed) = slice(workload, threads, inputs, &stopped); around.end(&bench, &placed); diff --git a/tests/guest/osbench/src/fair.rs b/tests/guest/osbench/src/fair.rs index d022791d86d..bfd442beb12 100644 --- a/tests/guest/osbench/src/fair.rs +++ b/tests/guest/osbench/src/fair.rs @@ -14,17 +14,17 @@ //! machine's CPUs. //! //! Each system runs it once with all three workloads in this program's own -//! group, and once with a group per workload. On ToyOS the group is the Job -//! object its threads share CPU time under: `onejob` leaves the three in this -//! program's, and `perjob` starts each workload in a Job of its own -//! (`SysCap::new_job`, which needs `Rights::JOBS`). On Linux `flat` leaves -//! them in this process's session and cgroup, where the fair unit is the -//! thread; `session` gives each a session of its own, whose autogroup is the -//! unit where autogroup applies; and `scope` puts each in a transient -//! `systemd-run --user --scope` with equal `CPUWeight`, whose cgroup is the -//! unit where the user's manager has the `cpu` controller. A `units` line says -//! which unit applied: `3xcgroup` is a cgroup per workload, `1xcgroup` one -//! cgroup for all three. +//! group, `shared`, and once with a group per workload, `apart`; a line per +//! job, `fair..`. On ToyOS the group is the Job object its +//! threads share CPU time under, and `apart` starts each workload in a Job of +//! its own (`SysCap::new_job`, which needs `Rights::JOBS`). On Linux `shared` +//! leaves them in this process's session and cgroup, where the fair unit is +//! the thread, and `apart` puts each in a transient `systemd-run --user +//! --scope` with equal `CPUWeight`, whose cgroup is the unit where the user's +//! manager has the `cpu` controller; Linux also runs `session`, a session +//! each, whose autogroup is the unit where autogroup applies. A `units` line +//! says which unit applied: `3xcgroup` is a cgroup per workload, `1xcgroup` +//! one cgroup for all three. use std::process::{Command, Stdio}; use std::thread; @@ -196,8 +196,8 @@ fn units(_: Grouping, keys: &[String]) -> String { #[cfg(target_os = "toyos")] fn units(grouping: Grouping, _: &[String]) -> String { match grouping { - Grouping::Plain => "1xjob", - Grouping::Job => "3xjob", + Grouping::Shared => "1xjob", + Grouping::Apart => "3xjob", } .into() } @@ -213,38 +213,30 @@ fn stop(children: impl IntoIterator) /// How the coordinator starts each job. #[derive(Clone, Copy)] pub enum Grouping { - /// Every job in this program's group: ToyOS's `onejob`, Linux's `flat`. - Plain, - /// Each workload in a ToyOS Job of its own. - #[cfg(target_os = "toyos")] - Job, + /// Every job in this program's group. + Shared, + /// Each workload in a group of its own: a ToyOS Job, a Linux scope's cgroup. + Apart, + /// Each workload in a Linux session, whose autogroup is the unit. #[cfg(target_os = "linux")] Session, - #[cfg(target_os = "linux")] - Scope, } impl Grouping { fn name(self) -> &'static str { match self { - #[cfg(target_os = "toyos")] - Grouping::Plain => "onejob", - #[cfg(target_os = "toyos")] - Grouping::Job => "perjob", - #[cfg(target_os = "linux")] - Grouping::Plain => "flat", + Grouping::Shared => "shared", + Grouping::Apart => "apart", #[cfg(target_os = "linux")] Grouping::Session => "session", - #[cfg(target_os = "linux")] - Grouping::Scope => "scope", } } fn command(self, exe: &std::path::Path) -> Result { match self { - Grouping::Plain => Ok(Command::new(exe)), + Grouping::Shared => Ok(Command::new(exe)), #[cfg(target_os = "toyos")] - Grouping::Job => { + Grouping::Apart => { use std::os::toyos::process::CommandExt; let cap = crate::sys::syscap().ok_or("no capability was endowed, so no job can be made")?; let job = cap.new_job().map_err(|e| format!("a new job was refused: {e:?}"))?; @@ -267,7 +259,7 @@ impl Grouping { Ok(command) } #[cfg(target_os = "linux")] - Grouping::Scope => { + Grouping::Apart => { let mut command = Command::new("systemd-run"); command.args(["--user", "--scope", "--quiet", "--collect", "-p", "CPUWeight=100", "--"]).arg(exe); Ok(command) @@ -348,11 +340,11 @@ pub fn contend(grouping: Grouping) { let mut permille: Vec = counts.iter().zip(&totals).map(|(c, t)| c * 1000 / t).collect(); permille.sort_unstable(); let cpus_milli = permille[permille.len() / 2] * cpus; - let mut extra = vec![("job", kind.to_string()), ("cpus_milli_p50", cpus_milli.to_string())]; + let mut extra = vec![("cpus_milli_p50", cpus_milli.to_string())]; if *kind == "helpers" { extra.push(("helpers_spawned", helpers.clone())); } - result(&bench, "permille", &mut permille, &extra); + result(&format!("{bench}.{kind}"), "permille", &mut permille, &extra); } let mut rate: Vec = totals.iter().map(|t| t * 1_000_000_000 / windows.width).collect(); result(&format!("{bench}.total"), "chunks/s", &mut rate, &[]); @@ -360,10 +352,10 @@ pub fn contend(grouping: Grouping) { #[cfg(target_os = "linux")] pub fn groupings() -> [Grouping; 3] { - [Grouping::Plain, Grouping::Session, Grouping::Scope] + [Grouping::Shared, Grouping::Apart, Grouping::Session] } #[cfg(target_os = "toyos")] pub fn groupings() -> [Grouping; 2] { - [Grouping::Plain, Grouping::Job] + [Grouping::Shared, Grouping::Apart] } diff --git a/tests/guest/osbench/src/files.rs b/tests/guest/osbench/src/files.rs index a820e7712c5..401a9c57066 100644 --- a/tests/guest/osbench/src/files.rs +++ b/tests/guest/osbench/src/files.rs @@ -1,12 +1,12 @@ -//! Files, through std's `fs` on both systems, under a directory each one -//! keeps in memory: the same operations on Linux's VFS over tmpfs and on -//! ToyOS's two kinds of file service, the kernel's own tmpfs (syscalls) and a -//! file server process (IPC per request). Persistent storage is `storage`'s. +//! Files, through std's `fs` on both systems: the same operations under a +//! filesystem each keeps in memory (`mem`), and on ToyOS under `/home` +//! (`served`). Persistent storage is `storage`'s. //! -//! The `` in each name says where it ran: `shm` is Linux's -//! `/dev/shm`; `tmp` is ToyOS's `/tmp`, which the kernel serves; `served` is -//! ToyOS's `/home`, which the DATA file server serves, and which a boot with -//! no DATA partition keeps in memory (the file server says so in the log). +//! `mem` is Linux's `/dev/shm` through the VFS, and ToyOS's `/tmp`, which the +//! kernel serves through syscalls. `served` is ToyOS's DATA file server, +//! through IPC per request: on the volume the bench's DATA partition holds +//! where it holds one of ours, and in memory where it does not (the file +//! server says which in the log). use std::fs::{self, File, OpenOptions}; use std::io::{Read, Seek, SeekFrom, Write}; @@ -17,22 +17,47 @@ use crate::report::{condition, refuse}; const MIB: usize = 1 << 20; -/// A directory to measure under, and what serves it. +/// A directory to measure under, what serves it, and the medium beneath, +/// where the program can tell: `memory`, or the disk's kind. pub struct Target { pub name: &'static str, - pub dir: &'static str, - pub served_by: &'static str, + pub dir: String, + pub served_by: String, + pub medium: Option, } -pub fn memory_targets() -> Vec { - if cfg!(target_os = "toyos") { - vec![ - Target { name: "tmp", dir: "/tmp", served_by: "the kernel's tmpfs, through syscalls" }, - Target { name: "served", dir: "/home", served_by: "the DATA file server, through IPC" }, - ] +impl Target { + /// What every line measured under the target carries. + pub fn fields(&self) -> Vec<(&'static str, String)> { + let mut fields = vec![("served_by", self.served_by.clone())]; + fields.extend(self.medium.clone().map(|m| ("medium", m))); + fields + } +} + +pub fn memory() -> Target { + let (dir, served_by) = if cfg!(target_os = "toyos") { + ("/tmp", "the kernel's tmpfs, through syscalls") } else { - vec![Target { name: "shm", dir: "/dev/shm", served_by: "tmpfs, through the VFS" }] + ("/dev/shm", "tmpfs, through the VFS") + }; + Target { name: "mem", dir: dir.into(), served_by: served_by.into(), medium: Some("memory".into()) } +} + +/// Every directory `files` measures under. +pub fn targets() -> Vec { + let mut targets = vec![memory()]; + if cfg!(target_os = "toyos") { + // Memory or the DATA partition's volume: the program cannot tell which. + let served_by = "the DATA file server, through IPC".into(); + targets.push(Target { name: "served", dir: "/home".into(), served_by, medium: None }); } + targets +} + +/// The persistent filesystem's directory, or `None` where the system has none. +pub fn disk() -> Option { + crate::sys::persistent_dir().map(|(dir, served_by, medium)| Target { name: "disk", dir, served_by, medium: Some(medium) }) } /// A scratch directory of this process's under `dir`, removed whole on drop. @@ -77,32 +102,34 @@ fn page_at(i: usize, pages: usize) -> u64 { (x % pages as u64) * 4096 } -/// Every benchmark under `target`. Each one's comment is its warm-up and -/// repetitions. +/// Every benchmark under `target`, each line naming what served it and its +/// medium. Each one's comment is its warm-up and repetitions. pub fn suite(target: &Target) { - condition(&format!("fs.{}.dir", target.name), target.dir); - condition(&format!("fs.{}.served_by", target.name), target.served_by); + condition(&format!("fs.{}.served_by", target.name), &target.served_by); #[cfg(target_os = "linux")] - condition(&format!("fs.{}.mount", target.name), crate::sys::mount_options(target.dir)); - let scratch = match Scratch::new(target.dir) { + condition(&format!("fs.{}.mount", target.name), crate::sys::mount_options(&target.dir)); + let scratch = match Scratch::new(&target.dir) { Ok(s) => s, Err(e) => return refuse(&format!("fs.{}", target.name), format!("no scratch directory under {}: {e}", target.dir)), }; let name = |op: &str| format!("fs.{}.{op}", target.name); + let with = |fields: &[(&'static str, String)]| -> Vec<(&'static str, String)> { + fields.iter().cloned().chain(target.fields()).collect() + }; // Opening an existing 4 KiB file for reading and closing it: 2,000 after 100. let small = scratch.at("small"); if let Err(e) = fill(&small, 4096) { return refuse(&name("setup"), e); } - each(&name("open_close"), 100, 2000, &[], |_| io("open", File::open(&small)).map(drop)); + each(&name("open_close"), 100, 2000, &with(&[]), |_| io("open", File::open(&small)).map(drop)); // `metadata` of the same file: 2,000 after 100. - each(&name("stat"), 100, 2000, &[], |_| io("stat", fs::metadata(&small)).map(drop)); + each(&name("stat"), 100, 2000, &with(&[]), |_| io("stat", fs::metadata(&small)).map(drop)); // A new file, 100 bytes written to it, closed and removed: 1,000 after 50. let fresh = scratch.at("fresh"); - each(&name("create_write_unlink"), 50, 1000, &[("bytes", "100".into())], |_| { + each(&name("create_write_unlink"), 50, 1000, &with(&[("bytes", "100".into())]), |_| { let mut f = io("create", File::create(&fresh))?; io("write", f.write_all(&[7u8; 100]))?; drop(f); @@ -120,11 +147,11 @@ pub fn suite(target: &Target) { match OpenOptions::new().read(true).write(true).open(&big) { Ok(mut f) => { let mut buf = vec![0u8; 4096]; - each(&name("read_4k_at"), 100, 2000, &[("file_bytes", (16 * MIB).to_string())], |i| { + each(&name("read_4k_at"), 100, 2000, &with(&[("file_bytes", (16 * MIB).to_string())]), |i| { io("seek", f.seek(SeekFrom::Start(page_at(i, pages))))?; io("read", f.read_exact(&mut buf)) }); - each(&name("write_4k_at"), 100, 2000, &[("file_bytes", (16 * MIB).to_string())], |i| { + each(&name("write_4k_at"), 100, 2000, &with(&[("file_bytes", (16 * MIB).to_string())]), |i| { io("seek", f.seek(SeekFrom::Start(page_at(i, pages))))?; io("write", f.write_all(&buf)) }); @@ -143,7 +170,7 @@ pub fn suite(target: &Target) { Ok(()) })(); match made { - Ok(()) => each(&name("readdir.1000"), 5, 50, &[("entries", "1000".into())], |_| { + Ok(()) => each(&name("readdir.1000"), 5, 50, &with(&[("entries", "1000".into())]), |_| { let mut n = 0; for entry in io("read_dir", fs::read_dir(&listed))? { std::hint::black_box(io("entry", entry)?.file_name()); @@ -158,7 +185,7 @@ pub fn suite(target: &Target) { // A file renamed within its directory, back and forth: 2,000 after 100. let (a, b) = (scratch.at("a"), scratch.at("b")); match File::create(&a) { - Ok(_) => each(&name("rename"), 100, 2000, &[], |i| { + Ok(_) => each(&name("rename"), 100, 2000, &with(&[]), |i| { let (from, to) = if i % 2 == 0 { (&a, &b) } else { (&b, &a) }; io("rename", fs::rename(from, to)) }), @@ -167,7 +194,7 @@ pub fn suite(target: &Target) { // An empty directory made and removed: 1,000 after 50. let sub = scratch.at("sub"); - each(&name("mkdir_rmdir"), 50, 1000, &[], |_| { + each(&name("mkdir_rmdir"), 50, 1000, &with(&[]), |_| { io("mkdir", fs::create_dir(&sub))?; io("rmdir", fs::remove_dir(&sub)) }); @@ -175,7 +202,7 @@ pub fn suite(target: &Target) { // A directory of 100 files of 64 bytes made, then every file and the // directory removed: ns per whole tree, 20 after 2. let tree = scratch.at("tree"); - each(&name("tree.100"), 2, 20, &[("files", "100".into())], |_| { + each(&name("tree.100"), 2, 20, &with(&[("files", "100".into())]), |_| { io("mkdir", fs::create_dir(&tree))?; for i in 0..100 { io("write", fs::write(tree.join(format!("f{i:03}")), [3u8; 64]))?; @@ -189,17 +216,18 @@ pub fn suite(target: &Target) { // 64 MiB written to a new file and read back whole, in 4 KiB and 1 MiB // calls: MB/s, five rounds after one. Last, as the largest: a server that // cannot hold it takes no other benchmark's number with it. - sequential(&name("seq"), &scratch.at("seq"), 64 * MIB, 5); + sequential(&name("seq"), &scratch.at("seq"), 64 * MIB, 5, &with(&[])); } /// `len` bytes written to `path` and read back, in 4 KiB and in 1 MiB calls: /// `_write.` and `_read.`, MB/s, `rounds` after -/// one. A write round creates the file anew; the read rounds read the last -/// one written. -pub fn sequential(prefix: &str, path: &Path, len: usize, rounds: usize) { +/// one, each line with `fields` beside its own. A write round creates the +/// file anew; the read rounds read the last one written. +fn sequential(prefix: &str, path: &Path, len: usize, rounds: usize, fields: &[(&'static str, String)]) { let block: Vec = (0..MIB).map(|i| (i % 253) as u8).collect(); for call in [4096usize, MIB] { - let extra = [("call_bytes", call.to_string()), ("file_bytes", len.to_string())]; + let extra: Vec<_> = + [("call_bytes", call.to_string()), ("file_bytes", len.to_string())].into_iter().chain(fields.iter().cloned()).collect(); throughput(&format!("{prefix}_write.{call}"), rounds, len as u64, &extra, || { let mut f = io("create", File::create(path))?; for chunk in 0..len / call { diff --git a/tests/guest/osbench/src/main.rs b/tests/guest/osbench/src/main.rs index bd1beaa1fad..c95ac65b47e 100644 --- a/tests/guest/osbench/src/main.rs +++ b/tests/guest/osbench/src/main.rs @@ -1,11 +1,12 @@ //! `osbench`: the same system workloads, built from one source for ToyOS //! and for Linux, so one machine's two systems can be compared. //! -//! `osbench ` runs a group, or `all` of them in [`GROUPS`]' order; the -//! groups in [`BY_NAME`] need another party and run only by name. A ToyOS job -//! list passes no arguments, so there the group is the name the program was -//! started under, `osbench_`. Every other command is one -//! this program gives its own children. +//! `osbench ` runs a group, or `all` of them in [`GROUPS`]' order, each +//! in a process of its own as a ToyOS job list runs them; the groups in +//! [`BY_NAME`] need another party and run only by name. A ToyOS job list +//! passes no arguments, so there the group is the name the program was started +//! under, `osbench_`. Every other command is one this program gives its +//! own children. //! //! Every result is one line of `key=value` words (`report`): a distribution //! with its sample count, minimum, p50, p90, p99, p99.9, maximum and mean, @@ -16,25 +17,12 @@ //! (`unsupported`), not a failure. //! //! Where a facility differs the code does too, in `sys` and said at the -//! benchmark: native IPC, persistent storage's medium, the tail group's -//! diagnostics. Everything else is the same source on both. +//! benchmark: native IPC, the directories each filesystem is reached at, the +//! tail group's diagnostics. Everything else is the same source on both. //! -//! The Linux build is static (`x86_64-unknown-linux-musl`, linked by the -//! toolchain's own LLD per `.cargo/config.toml`), so a bench runs it without a -//! toolchain: -//! -//! ```text -//! rustup target add --toolchain nightly-2026-07-22 x86_64-unknown-linux-musl -//! cargo +nightly-2026-07-22 build -p osbench --target x86_64-unknown-linux-musl --profile toyos -//! ``` -//! -//! That nightly is the build's stage-0 compiler (`build/sysroot.rs`'s -//! `STAGE0_CARGO`, which this follows), the upstream one nearest ToyOS's own. //! On Linux `nvme` and `energy` read what only root may (the raw namespace, -//! the package's energy counter) and say so unprivileged. -//! With the `toyos` profile every guest binary is built with, osbench's own -//! code is generated alike for both; std's platform layer and the C library -//! beneath it (musl's on Linux) are each system's own. +//! the package's energy counter) and say so unprivileged. `README.md` is how +//! the Linux half is built and how a comparison is run. mod arch; mod around; @@ -138,16 +126,16 @@ fn fair() { } fn files() { - for target in files::memory_targets() { + for target in files::targets() { files::suite(&target); } } /// Every group the machine alone runs, in the order `all` and the ToyOS job /// list run them: `boot` first, since it reads how long the boot took to -/// reach it; `energy` while the machine is still quiet; `crunch`, which heats -/// it, after every other measurement; and `persist` last, since it writes the -/// volume a ToyOS boot's log is on. +/// reach it; `energy` while the machine is still quiet; and `crunch`, which +/// heats it, last: ToyOS reads no temperature to cool down by, so a group +/// after it would start at a clock its Linux run does not. const GROUPS: [(&str, fn()); 17] = [ ("boot", boot::suite), ("identity", identity::suite), @@ -164,8 +152,8 @@ const GROUPS: [(&str, fn()); 17] = [ ("work", work::suite), ("tail", tail::suite), ("nvme", nvme::suite), - ("crunch", crunch::suite), ("persist", storage::suite), + ("crunch", crunch::suite), ]; /// The groups that need another party: `lan` the `osbench-peer` on the other @@ -183,6 +171,25 @@ fn group(name: &str, run: fn()) { report::group(name, &format!("done secs={}", t.elapsed().as_secs())); } +/// Every group in [`GROUPS`], each a child of its own, so no group runs in a +/// process an earlier one shaped: exits 1 if any did not measure. +fn all() { + let exe = std::env::current_exe().expect("a path to this program"); + let mut failed = Vec::new(); + for (name, _) in GROUPS { + let status = std::process::Command::new(&exe).arg(name).status(); + match status { + Ok(status) if status.success() => {} + Ok(status) => failed.push(format!("{name} ({status})")), + Err(e) => failed.push(format!("{name} (would not start: {e})")), + } + } + if !failed.is_empty() { + eprintln!("osbench: groups that did not measure: {}", failed.join(", ")); + std::process::exit(1); + } +} + fn main() { let args: Vec = std::env::args().collect(); let invoked = args.first().and_then(|a| a.rsplit('/').next()).unwrap_or_default(); @@ -192,7 +199,7 @@ fn main() { }; let rest = args.get(2..).unwrap_or_default(); match command { - "all" => GROUPS.iter().for_each(|(name, run)| group(name, *run)), + "all" => return all(), "nop" => return, "spin" => return wake::spin(), "fair-job" => return fair::job(rest), diff --git a/tests/guest/osbench/src/mem.rs b/tests/guest/osbench/src/mem.rs index eb5cb9838d5..74bb0a57266 100644 --- a/tests/guest/osbench/src/mem.rs +++ b/tests/guest/osbench/src/mem.rs @@ -5,9 +5,10 @@ //! //! Both systems fault each anonymous page in on its first touch: ToyOS in //! 4 KiB pages always, Linux in 4 KiB pages where huge pages are refused and -//! in 2 MiB pages where they are asked for and given. `map_touch_unmap` is the -//! whole cost. The allocator is each system's own: ToyOS std's dlmalloc -//! under one lock, Linux's static musl malloc. +//! in 2 MiB pages where they are asked for and given. `first_touch` faults in +//! the same bytes either way, and `map_touch_unmap` is the whole cost. The +//! allocator is each system's own: ToyOS std's dlmalloc under one lock, +//! Linux's static musl malloc. use std::alloc::{alloc, dealloc, realloc, Layout}; use std::sync::atomic::{AtomicBool, AtomicU64, Ordering}; @@ -38,37 +39,40 @@ pub fn suite() { realloc_grow(); } -/// A fresh 64 MiB mapping written once per page: ns per page touched, each -/// round's mean, and the whole map-touch-unmap per page; 4 KiB pages with -/// huge pages refused, and 2 MiB pages with them asked for. Ten rounds after one. +/// A fresh 64 MiB mapping written once per 4 KiB, so every byte of it is +/// faulted in whatever page size backs it: ns per MiB touched, each round's, +/// and the whole map-touch-unmap per MiB; with huge pages refused (`4k`) and +/// asked for (`2m`). Ten rounds after one. fn first_touch() { const LEN: usize = 64 * MIB; - for (name, step, map) in [("4k", PAGE, sys::map_anon_4k as fn(usize) -> *mut u8), ("2m", HUGE, sys::map_anon_huge)] { - let pages = (LEN / step) as u64; + const MIBS: u64 = (LEN / MIB) as u64; + for (name, map) in [("4k", sys::map_anon_4k as fn(usize) -> *mut u8), ("2m", sys::map_anon_huge)] { let round = || { let t = Instant::now(); let ptr = map(LEN); let mapped = ns_since(t); let before = sys::counts(); let t = Instant::now(); - for page in 0..LEN / step { + for page in 0..LEN / PAGE { // SAFETY: inside the mapping just made. - unsafe { ptr.add(page * step).write_volatile(1) }; + unsafe { ptr.add(page * PAGE).write_volatile(1) }; } let touched = ns_since(t); let faults = sys::counts().faults.zip(before.faults).map(|(b, a)| b - a); + let huge = sys::huge_backing(); let t = Instant::now(); // SAFETY: the whole mapping, unused from here. unsafe { sys::unmap(ptr, LEN) }; - (touched / pages, (mapped + touched + ns_since(t)) / pages, faults) + (touched / MIBS, (mapped + touched + ns_since(t)) / MIBS, faults, huge) }; round(); crate::clock::settle(); - let rounds: Vec<(u64, u64, Option)> = (0..10).map(|_| round()).collect(); + let rounds: Vec<(u64, u64, Option, String)> = (0..10).map(|_| round()).collect(); let faults = rounds.iter().filter_map(|r| r.2).max().map_or("unread".to_string(), |f| f.to_string()); - let extra = [("page_bytes", step.to_string()), ("faults_per_round_max", faults), ("huge_pages", sys::huge_backing())]; - result(&format!("mem.first_touch.{name}"), "ns/page", &mut rounds.iter().map(|r| r.0).collect::>(), &extra); - result(&format!("mem.map_touch_unmap.{name}"), "ns/page", &mut rounds.iter().map(|r| r.1).collect::>(), &extra); + let huge = rounds.last().map(|r| r.3.clone()).unwrap_or_default(); + let extra = [("mapping_bytes", LEN.to_string()), ("faults_per_round_max", faults), ("huge_pages_last_round", huge)]; + result(&format!("mem.first_touch.{name}"), "ns/MiB", &mut rounds.iter().map(|r| r.0).collect::>(), &extra); + result(&format!("mem.map_touch_unmap.{name}"), "ns/MiB", &mut rounds.iter().map(|r| r.1).collect::>(), &extra); } } diff --git a/tests/guest/osbench/src/procs.rs b/tests/guest/osbench/src/procs.rs index 29d870202d4..7fd728918a6 100644 --- a/tests/guest/osbench/src/procs.rs +++ b/tests/guest/osbench/src/procs.rs @@ -145,8 +145,8 @@ fn c_spawn_wait(path: &CString, argv: &[&CString]) -> Result<(), sys::Refused> { } pub fn suite() { - let memory = crate::files::memory_targets().remove(0).dir; - let dir = match Scratch::new(memory) { + let memory = crate::files::memory().dir; + let dir = match Scratch::new(&memory) { Ok(d) => d, Err(e) => return refuse("proc", format!("no scratch directory under {memory}: {e}")), }; diff --git a/tests/guest/osbench/src/report.rs b/tests/guest/osbench/src/report.rs index 3d45bba87df..bd9bf2ee20a 100644 --- a/tests/guest/osbench/src/report.rs +++ b/tests/guest/osbench/src/report.rs @@ -60,9 +60,12 @@ pub fn any_refused() -> bool { /// One measured distribution, by nearest rank. A percentile the sample count /// cannot resolve prints `-`: p99 needs 100 samples and p99.9 a thousand. /// `mhz` is the reporting CPU's clock read just after the measurement -/// (`clock::mhz`), and `mhz_settled` the clock the benchmark's warm-up -/// reached before it (`clock::settle`), so every number says the clock it -/// was taken at. +/// (`clock::mhz`). Where the benchmark's warm-up settled the clock first, +/// `mhz_settled` is the clock it reached, and `mhz_after` the clock a settle +/// just after the measurement reaches: a CPU that only idled meanwhile is +/// back at its clock, and one a power or heat limit holds is not. +/// `unsettled` names a settle that ran out its limit and took the clock it +/// had. pub fn result(bench: &str, unit: &str, samples: &mut [u64], extra: &[(&str, String)]) { if samples.is_empty() { refuse(bench, "no samples"); @@ -85,11 +88,18 @@ pub fn result(bench: &str, unit: &str, samples: &mut [u64], extra: &[(&str, Stri for (key, value) in extra { line.push_str(&format!(" {key}={}", word(value))); } - match crate::clock::take_settled() { - 0 => {} - settled => line.push_str(&format!(" mhz_settled={settled}")), + let after = crate::clock::mhz(); + if let Some(before) = crate::clock::take_settled() { + crate::clock::settle(); + let settled_after = crate::clock::take_settled().expect("the settle just run"); + line.push_str(&format!(" mhz_settled={} mhz_after={}", before.mhz, settled_after.mhz)); + let unsettled: Vec<&str> = + [("before", before.capped), ("after", settled_after.capped)].iter().filter(|(_, c)| *c).map(|(w, _)| *w).collect(); + if !unsettled.is_empty() { + line.push_str(&format!(" unsettled={}", unsettled.join("+"))); + } } - line.push_str(&format!(" mhz={}", crate::clock::mhz())); + line.push_str(&format!(" mhz={after}")); emit(&line); } diff --git a/tests/guest/osbench/src/storage.rs b/tests/guest/osbench/src/storage.rs index c44b91133ae..af1d628a39e 100644 --- a/tests/guest/osbench/src/storage.rs +++ b/tests/guest/osbench/src/storage.rs @@ -1,34 +1,34 @@ -//! Persistent storage, each system on the medium it has, so **not like for -//! like**: what these say is what a durable write costs on the storage a -//! measured boot has, never which kernel is faster. +//! Persistent storage, each system on the medium a measured boot has, so +//! **not like for like** until ToyOS's DATA is the bench's partition: what +//! these say is what a durable write costs there, each line naming it. //! //! On a ToyOS benchcase boot that is the boot USB stick's `TOYOS-LOG` -//! partition, FAT32, through the LOG file server and usbd: the boot has no -//! DATA partition, so `/home` and `/state` are memory (`files`'s `served`), -//! and the stick's other volumes are read-only. On Linux it is a directory in -//! the runner's home, on whatever disk and filesystem hold it, which the -//! conditions name. Every file here is removed at the end; the most alive at -//! once is under 10 MiB, inside what the log volume leaves free. +//! partition, FAT32, through the LOG file server and usbd: the stick's other +//! volumes are read-only, and DATA is the bench's partition only once ToyOS +//! has designated it. On Linux it is a directory in the runner's home, on +//! whatever disk and filesystem hold it. Every file here is removed at the +//! end; the most alive at once is under 10 MiB, inside what the log volume +//! leaves free. use std::fs::{self, File, OpenOptions}; use std::io::{Read, Seek, SeekFrom, Write}; -use crate::files::Scratch; +use crate::files::{self, Scratch}; use crate::measure::{each, io, throughput}; use crate::report::{condition, refuse}; -use crate::sys; const MIB: usize = 1 << 20; pub fn suite() { - let Some((dir, medium)) = sys::persistent_dir() else { + let Some(disk) = files::disk() else { return refuse("persist", "no persistent directory to write in"); }; - condition("persist.medium", medium); - let scratch = match Scratch::new(&dir) { + let scratch = match Scratch::new(&disk.dir) { Ok(s) => s, Err(e) => return refuse("persist", format!("no scratch directory: {e}")), }; + condition("persist.served_by", &disk.served_by); + let served = disk.fields(); let block = [0x6bu8; 4096]; // 4 KiB written over the start of a 1 MiB file, then `sync_all`: 100 after 5. @@ -39,20 +39,20 @@ pub fn suite() { })(); match made { Ok(mut f) => { - each("persist.write_4k_fsync", 5, 100, &[], |_| { + each("persist.write_4k_fsync", 5, 100, &served, |_| { io("seek", f.seek(SeekFrom::Start(0)))?; io("write", f.write_all(&block))?; io("fsync", f.sync_all()) }); // `sync_all` of the same file with nothing new in it: 200 after 10. - each("persist.fsync_clean", 10, 200, &[], |_| io("fsync", f.sync_all())); + each("persist.fsync_clean", 10, 200, &served, |_| io("fsync", f.sync_all())); } Err(e) => refuse("persist.write_4k_fsync", e), } // 4 KiB appended, then `sync_data`: 100 after 5, so the file grows by 420 KiB. match OpenOptions::new().create(true).append(true).open(scratch.at("log")) { - Ok(mut f) => each("persist.append_4k_fsync", 5, 100, &[], |_| { + Ok(mut f) => each("persist.append_4k_fsync", 5, 100, &served, |_| { io("write", f.write_all(&block))?; io("fsync", f.sync_data()) }), @@ -62,7 +62,7 @@ pub fn suite() { // A new file, 4 KiB written and `sync_all`ed, closed and removed: 50 // after 5. The directory is not synced. let fresh = scratch.at("fresh"); - each("persist.create_fsync_unlink", 5, 50, &[], |_| { + each("persist.create_fsync_unlink", 5, 50, &served, |_| { let mut f = io("create", File::create(&fresh))?; io("write", f.write_all(&block))?; io("fsync", f.sync_all())?; @@ -75,7 +75,8 @@ pub fn suite() { // calls, which both systems answer from their caches. let seq = scratch.at("seq"); let chunk = vec![0x2du8; MIB]; - let extra = [("file_bytes", (8 * MIB).to_string()), ("call_bytes", MIB.to_string())]; + let extra: Vec<_> = + [("file_bytes", (8 * MIB).to_string()), ("call_bytes", MIB.to_string())].into_iter().chain(served.iter().cloned()).collect(); throughput("persist.seq_write_fsync", 3, 8 * MIB as u64, &extra, || { let mut f = io("create", File::create(&seq))?; for _ in 0..8 { diff --git a/tests/guest/osbench/src/sys/linux.rs b/tests/guest/osbench/src/sys/linux.rs index 38c17ec5722..91c38cec919 100644 --- a/tests/guest/osbench/src/sys/linux.rs +++ b/tests/guest/osbench/src/sys/linux.rs @@ -225,7 +225,8 @@ fn busy_permille(span: Duration) -> Option { (total > 0).then(|| (total - (idle1 - idle0)) * 1000 / total) } -/// What a Linux number depends on, read and never changed. +/// What a Linux number depends on, read and never changed. `README.md` says +/// which of them a comparison sets. pub fn conditions() { condition("kernel", read("/proc/sys/kernel/osrelease").unwrap_or_default()); let distro = fs::read_to_string("/etc/os-release").ok().and_then(|text| { @@ -238,6 +239,7 @@ pub fn conditions() { condition("cpufreq_driver", per_cpu("cpufreq/scaling_driver")); condition("governor", per_cpu("cpufreq/scaling_governor")); condition("epp", per_cpu("cpufreq/energy_performance_preference")); + condition("min_khz", per_cpu("cpufreq/scaling_min_freq")); condition("max_khz", per_cpu("cpufreq/scaling_max_freq")); condition( "intel_no_turbo", @@ -245,6 +247,11 @@ pub fn conditions() { ); condition("boost", read("/sys/devices/system/cpu/cpufreq/boost").unwrap_or_else(|| "absent".into())); condition("amd_pstate", read("/sys/devices/system/cpu/amd_pstate/status").unwrap_or_else(|| "absent".into())); + condition("platform_profile", read("/sys/firmware/acpi/platform_profile").unwrap_or_else(|| "absent".into())); + // What every timestamp here is read from: one the vDSO cannot read costs + // each `Instant::now` a syscall. + let clocksource = read("/sys/devices/system/clocksource/clocksource0/current_clocksource"); + condition("clocksource", clocksource.unwrap_or_else(|| "absent".into())); for supply in fs::read_dir("/sys/class/power_supply").into_iter().flatten().flatten() { let at = supply.path(); let get = |f: &str| read(&at.join(f).to_string_lossy()); @@ -624,9 +631,11 @@ fn tcp_info(stream: &std::net::TcpStream) -> Result { ]) } -/// The runner's home, and the filesystem and disk model holding it; the -/// path itself names the user and is never printed. -pub fn persistent_dir() -> Option<(String, String)> { +/// The runner's home, the filesystem and disk model holding it, as `ext4, +/// NVMe disk, model `, and the disk's kind as a medium: `nvme`, `usb`, +/// `sata`, `mmc`, `virtio`, `other`, or `none` for no block device. The path +/// itself names the user and is never printed. +pub fn persistent_dir() -> Option<(String, String, String)> { let home = std::env::var("HOME").ok()?; let (fstype, source, _) = mount_of(&home)?; // btrfs reports an anonymous `0:NN` that names no block device; its @@ -639,19 +648,22 @@ pub fn persistent_dir() -> Option<(String, String)> { sysfs(format!("{}:{}", libc::major(rdev), libc::minor(rdev))) }); let Some(device) = device else { - return Some((home, format!("{fstype}, on no block device"))); + return Some((home, format!("{fstype}, on no block device"), "none".into())); }; let (disk, mapped) = backing_disk(device); let model = read(&disk.join("device/model").to_string_lossy()).unwrap_or_else(|| "unknown".into()); let name = disk.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default(); - let kind = [("nvme", "NVMe"), ("sd", "SCSI, SATA or USB"), ("mmcblk", "MMC"), ("vd", "virtio")] - .iter() + // A SCSI disk is a USB one where sysfs puts it under a USB device. + let usb = disk.to_string_lossy().contains("/usb"); + let sd = if usb { ("USB", "usb") } else { ("SCSI or SATA", "sata") }; + let (kind, medium) = [("nvme", ("NVMe", "nvme")), ("sd", sd), ("mmcblk", ("MMC", "mmc")), ("vd", ("virtio", "virtio"))] + .into_iter() .find(|(prefix, _)| name.starts_with(prefix)) - .map_or("other", |(_, kind)| kind); + .map_or(("other", "other"), |(_, kind)| kind); let through = if mapped { " through device-mapper" } else { "" }; // The device's own name is never printed: an installer or a user chose // a mapper's (`luks-`, a volume group named for the host). - Some((home, format!("{fstype}{through}, {kind} disk, model {model}"))) + Some((home, format!("{fstype}{through}, {kind} disk, model {model}"), medium.into())) } /// The `major:minor` of the filesystem holding `path`, off mountinfo's third diff --git a/tests/guest/osbench/src/sys/toyos.rs b/tests/guest/osbench/src/sys/toyos.rs index b986caf73f0..0eb9c4db882 100644 --- a/tests/guest/osbench/src/sys/toyos.rs +++ b/tests/guest/osbench/src/sys/toyos.rs @@ -389,9 +389,11 @@ pub fn map_anon_4k(len: usize) -> *mut u8 { map_anon(len) } -/// The one writable persistent volume a benchcase boot has, and what it is. -pub fn persistent_dir() -> Option<(String, String)> { - Some(("/log".into(), "boot USB stick TOYOS-LOG partition, FAT32, LOG file server over usbd mass storage".into())) +/// The one writable volume a benchcase boot certainly has on a disk, what +/// serves it, and its medium. +pub fn persistent_dir() -> Option<(String, String, String)> { + let served_by = "boot USB stick TOYOS-LOG partition, FAT32, LOG file server over usbd mass storage"; + Some(("/log".into(), served_by.into(), "usb".into())) } /// What a transfer row reports of the stack beneath it, as netstack's `inspect` answers it: diff --git a/tests/guest/osbench/src/system.rs b/tests/guest/osbench/src/system.rs index cbbffa3fb5f..79d35246766 100644 --- a/tests/guest/osbench/src/system.rs +++ b/tests/guest/osbench/src/system.rs @@ -12,10 +12,9 @@ use crate::{chunk, ns_since}; const MIB: usize = 1 << 20; -/// What one chunk of the fairness work costs one thread, 200 after one: the -/// same instructions on both systems, so a difference between them is the -/// clock that thread ran at. It is a 2 ms single-threaded burst, so it reads -/// the clock a CPU leaving idle has, not the clock it settles at. +/// What one chunk of the fairness work costs one thread, 200 after a settle: +/// the same instructions on both systems, so a difference between them is the +/// clock that thread ran at. pub fn chunk_rate() { chunk(); crate::clock::settle(); diff --git a/tests/guest/osbench/src/time.rs b/tests/guest/osbench/src/time.rs index 99e1d7d426d..42be598d1af 100644 --- a/tests/guest/osbench/src/time.rs +++ b/tests/guest/osbench/src/time.rs @@ -7,11 +7,6 @@ use std::time::{Duration, Instant}; use crate::report::result; use crate::ns_since; -/// What std's `yield_now` is on each system: Linux's is `sched_yield`; -/// ToyOS's enters no kernel (`sdk/std/sys/thread.rs`: there is no yield -/// syscall, and it is a spin-loop hint). -const YIELD: &str = if cfg!(target_os = "toyos") { "spin_hint_no_syscall" } else { "sched_yield" }; - pub fn suite() { clock_read(); sleeps(""); @@ -72,9 +67,14 @@ fn sleeps(suffix: &str) { } } -/// `thread::yield_now` with nothing else to run: 15 rounds of 100,000 calls -/// after one, picoseconds per call. +/// `thread::yield_now` with nothing else to run, `sched_yield` on Linux: 15 +/// rounds of 100,000 calls after one, picoseconds per call. ToyOS has no +/// yield to measure: its std's `yield_now` is a spin-loop hint +/// (`sdk/std/sys/thread.rs`). fn yields() { + if cfg!(target_os = "toyos") { + return crate::report::unsupported("time.yield", "no yield syscall: std's yield_now is a spin-loop hint"); + } const CALLS: u64 = 100_000; let round = || { let t = Instant::now(); @@ -86,5 +86,5 @@ fn yields() { round(); crate::clock::settle(); let mut ps: Vec = (0..15).map(|_| round()).collect(); - result("time.yield", "ps", &mut ps, &[("yield_is", YIELD.into()), ("calls_per_sample", CALLS.to_string())]); + result("time.yield", "ps", &mut ps, &[("calls_per_sample", CALLS.to_string())]); } diff --git a/tests/guest/osbench/src/work.rs b/tests/guest/osbench/src/work.rs index 87d71f290d5..783aeb1429d 100644 --- a/tests/guest/osbench/src/work.rs +++ b/tests/guest/osbench/src/work.rs @@ -52,8 +52,8 @@ pub fn cc(args: &[String]) { } pub fn suite() { - let memory = crate::files::memory_targets().remove(0).dir; - let dir = match Scratch::new(memory) { + let memory = crate::files::memory().dir; + let dir = match Scratch::new(&memory) { Ok(d) => d, Err(e) => return refuse("work", format!("no scratch directory under {memory}: {e}")), }; diff --git a/tests/images/benchcase.toml b/tests/images/benchcase.toml index f810120f758..580062e5459 100644 --- a/tests/images/benchcase.toml +++ b/tests/images/benchcase.toml @@ -4,9 +4,9 @@ # Only what a job list needs to run and to reach the log on the boot stick, the # ACPI server, and netstack for `osbench lan` and `osbench internet`, each its # own metal row (`tests/toyos.rs`): no compositor or sound, so little but the -# measurement asks for a CPU. DATA is memory here, the stick having no DATA -# partition, and the log volume is the one persistent volume a job can write -# (`osbench persist`). +# measurement asks for a CPU. The stick has no DATA partition, so DATA is the +# bench's TOYOS-DATA partition where ToyOS has designated it, and memory where +# it has not; the log volume is what `osbench persist` writes. # # The runner's capability carries `jobs`, for the fairness run that starts each # workload as a job of its own; `counters`, `trace` and `roster`, so `osbench @@ -40,13 +40,13 @@ serves = ["netstack"] devices = ["pci:02:00:00/8086:15fc"] # The order `osbench all` runs the groups in. The bound is the metal row's, its -# 370 s on the 60 s every list has (`tests/toyos.rs`), restated for a boot of +# 465 s on the 60 s every list has (`tests/toyos.rs`), restated for a boot of # this config built without the row; a derived metal config states its own. [programs.test-runner] receives = ["power", "netstack"] syscap = ["jobs", "counters", "trace", "roster", "inventory", "dup"] args = [ - "--bound-ms=430000", + "--bound-ms=525000", "osbench_boot", "osbench_identity", "osbench_energy", @@ -62,8 +62,8 @@ args = [ "osbench_work", "osbench_tail", "osbench_nvme", - "osbench_crunch", "osbench_persist", + "osbench_crunch", "reboot", ] diff --git a/tests/toyos.rs b/tests/toyos.rs index d5173fa98a1..416a2e487b2 100644 --- a/tests/toyos.rs +++ b/tests/toyos.rs @@ -1624,7 +1624,7 @@ const TESTCASES: &[metal::Arm] = &[metal::once( /// `osbench`'s groups, each a job named by its symlink, in `osbench all`'s /// order: `boot` first, which reads how long the boot took to reach it, and -/// `persist` last, which writes the log volume. +/// `crunch` last, which heats the machine. const OSBENCH_GROUPS: [&str; 17] = [ "osbench_boot", "osbench_identity", @@ -1641,8 +1641,8 @@ const OSBENCH_GROUPS: [&str; 17] = [ "osbench_work", "osbench_tail", "osbench_nvme", - "osbench_crunch", "osbench_persist", + "osbench_crunch", ]; /// The suite runs past every boot's bound on purpose: `tail` alone waits out @@ -1651,10 +1651,12 @@ const OSBENCH_GROUPS: [&str; 17] = [ /// served 64 MiB write stalled and `files` took 80 s against 13. The groups /// since add at most 50 s by their own spans: `crunch`'s one-thread 16 s /// (ToyOS reads no temperature, so its all-core phase does not run), and -/// under 30 s of memcpy sizes, pipe sizes and 1 ms timers. So a list bound of -/// 430 s leaves that stall 136 s. +/// under 30 s of memcpy sizes, pipe sizes and 1 ms timers. Each of the some +/// ninety benchmarks that settle the clock settles it before and after +/// itself, 50 ms where the clock holds and 0.5 s at most (`clock::settle`): +/// under 95 s. So a list bound of 525 s leaves that stall 136 s. const BENCHCASE: &[metal::Arm] = &[metal::Arm { - adds_ms: 370_000, + adds_ms: 465_000, ..metal::once("benchcase", "tests/images/benchcase.toml", &[], &OSBENCH_GROUPS) }];