From 513c99670c3e338af951196f94a1af711dcc7985 Mon Sep 17 00:00:00 2001 From: japabu Date: Wed, 28 Oct 2026 18:29:41 +0100 Subject: [PATCH 1/6] osbench energy: the package's power on both systems, off the same registers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The kernel's counters carry the package's energy and its unit register (Intel MSR_PKG_ENERGY_STATUS and MSR_RAPL_POWER_UNIT, SDM Vol. 3B §17.10 and Vol. 4 Table 2-20; AMD PKG_ENERGY_STAT and RAPL_PWR_UNIT, PPR 55922), the package's C2, C3, C6 and C7 residency where Table 2-20 gives them, and AMD's per-core CORE_ENERGY_STAT, all under Rights::TRACE, the energy being a side channel (CVE-2020-8694). The package's are read on the boot processor alone, for its package, as the firmware calls are. Admission follows the existing rule for model-specific counters: Intel's by the DisplayModels whose SDM sections name Table 2-20, less 06_AFH, whose Table 2-57 restates the residency in other units, and for package residency 06_66H, whose Table 2-42 gives 3F8H no rate; AMD's by Family 17h, whose PPRs in hand agree, with CPUID_Fn80000007_EDX[RAPL]. Family 19h's PPR 55901 changes the counters' widths and scope, so it is not read. No hypervisor's emulation is read for any of them. osbench's energy group becomes a group by name with its own metal row (osbench_energy, the benchcase image with that one job), since idle power is the whole machine's. On ToyOS it reads SYS_COUNTERS, on Linux the same registers through /dev/cpu//msr as root, decoded by one rule (osbench_rules::energy). It waits for the machine to go quiet, then reads the package each second across 60 s idle, 30 s of a 10 ms sleeper and 30 s each of one and of every CPU spinning: power, residency and energy per chunk, with each CPU's APERF/MPERF and core residency across each load. Linux states its panel, backlight, radios and links as setup, which compare refuses to mix. The counters metal judge holds the energy counters' presence to the bring-up line and prints the package's power. Co-Authored-By: Claude Opus 5.5 --- ...ompares-toyos-with-linux-on-one-machine.md | 13 +- .../the-kernel-owns-cpu-performance-state.md | 15 +- kernel/pure/cpu.rs | 189 +++++++++-- kernel/src/arch/aarch64/counters.rs | 13 +- kernel/src/arch/x86_64/counters.rs | 45 ++- kernel/src/counters.rs | 26 +- lib/inspect/src/kernel.rs | 18 +- sdk/abi/src/counters.rs | 71 ++++- tests/guest/osbench/README.md | 53 ++++ tests/guest/osbench/compare/src/lib.rs | 13 +- tests/guest/osbench/rules/Cargo.toml | 2 +- tests/guest/osbench/rules/src/lib.rs | 66 +++- tests/guest/osbench/src/arch/x86_64.rs | 29 ++ tests/guest/osbench/src/energy.rs | 298 ++++++++++++++---- tests/guest/osbench/src/main.rs | 18 +- tests/guest/osbench/src/sys/linux.rs | 139 ++++++++ tests/guest/osbench/src/sys/mod.rs | 40 ++- tests/guest/osbench/src/sys/toyos.rs | 73 +++++ tests/images/benchcase.toml | 1 - tests/toyos.rs | 61 +++- 20 files changed, 1023 insertions(+), 160 deletions(-) diff --git a/issues/no-published-run-compares-toyos-with-linux-on-one-machine.md b/issues/no-published-run-compares-toyos-with-linux-on-one-machine.md index 4079fe32210..b710a7fd1c1 100644 --- a/issues/no-published-run-compares-toyos-with-linux-on-one-machine.md +++ b/issues/no-published-run-compares-toyos-with-linux-on-one-machine.md @@ -21,7 +21,9 @@ its groups: sides' IDs match or either side's changed across it; - lateness past a 1 ms timer: `time.timer_1ms.late`; - NVMe reads, read-only: `nvme.read.*`; -- an idle minute's package energy: `energy.idle_60s.package`; +- the package's power, idle and under defined loads, with its C-state + residency where Intel documents it: `energy.*`, run only by name, a ToyOS + boot of its own (`osbench_energy`) and as root under Linux; - the machine's PCI, USB, NVMe and cpuidle identity: `identity`, with no serial number or network identifier; - TCP connect, round trip and throughput between the two benches: @@ -68,12 +70,13 @@ comparison until it can: - **NVMe reads**: a program reaches a disk's blocks only through a grant the supervisor mints for a file server's role or the idle slot, and the bench's NVMe holds another system. -- **Package energy and temperature**: no counter carries either +- **Package temperature**: no counter carries it (`toyos_abi::counters::Counter`), so the ToyOS crunch runs its one-thread phase alone and reports every all-core row `unmeasured`, with nothing to - stop it at. The kernel's sampler of `IA32_PACKAGE_THERM_STATUS`, - `MSR_TEMPERATURE_TARGET` and `MSR_PKG_ENERGY_STATUS` in - `issues/the-kernel-owns-cpu-performance-state.md` is what closes both. The + stop it at. The sampler's `IA32_PACKAGE_THERM_STATUS` and + `MSR_TEMPERATURE_TARGET` in + `issues/the-kernel-owns-cpu-performance-state.md` is what closes it; the + package's energy is a counter now (`Counter::PackageEnergy`). The first ToyOS crunch on the T14 held 3,788 MHz all-core where Linux fell to 1.7-1.9 GHz, which `issues/the-t14s-thermal-modes-do-not-move-its-package-power-limits.md` diff --git a/issues/the-kernel-owns-cpu-performance-state.md b/issues/the-kernel-owns-cpu-performance-state.md index 06fd0d3626e..e4a60f5cbcb 100644 --- a/issues/the-kernel-owns-cpu-performance-state.md +++ b/issues/the-kernel-owns-cpu-performance-state.md @@ -29,11 +29,16 @@ performance bias, on every CPU that has all of them writing that register whole. *Exit*: the ruling, and the bit declared or recorded as firmware's. - **The sampler**: a program that reads the envelope every 60 s and at a - span's start and end, with package power from `MSR_PKG_ENERGY_STATUS` and - temperature from `IA32_PACKAGE_THERM_STATUS` and `MSR_TEMPERATURE_TARGET`, - the bar's validity conditions. The energy counter is a side channel - (CVE-2020-8694), under `Rights::TRACE` like the rest. *Exit*: one valid - span on the T14. + span's start and end, with package power and temperature from + `IA32_PACKAGE_THERM_STATUS` and `MSR_TEMPERATURE_TARGET`, the bar's + validity conditions. The package's energy is read already, under + `Rights::TRACE` as a side channel (CVE-2020-8694): `Counter::PackageEnergy` + and its unit, its C2 to C7 residency, AMD's core energy, all on the boot + processor for its package but the last (`kernel/pure/cpu.rs`). A machine of + more than one package counts only the boot processor's; Intel's PP0 and PP1 + planes, and the T14's PC8 to PC10, which SDM Vol. 4 does not give Tiger + Lake, are unread. Temperature is still to build. *Exit*: one valid span on + the T14. Not covered: hybrid Intel and AMD CPPC, `issues/the-perf-state-declaration-refuses-hybrid-intel-and-amd-cppc.md`; diff --git a/kernel/pure/cpu.rs b/kernel/pure/cpu.rs index 10b61f5377f..6414ce1ceb3 100644 --- a/kernel/pure/cpu.rs +++ b/kernel/pure/cpu.rs @@ -180,6 +180,11 @@ pub struct CounterFacts { pub cpuid_1_ecx: u32, /// 0 unless CPUID.0 reaches leaf 6. pub cpuid_6_ecx: u32, + /// 0 unless CPUID.8000_0000H reaches leaf 8000_0007H. + pub cpuid_8000_0007_edx: u32, + /// The CPU reads its package's counters: the boot processor, so a + /// machine's package counters are one CPU's. + pub reads_package: bool, } /// The counters a CPU has: a counter it lacks is `#GP` to read, and the @@ -192,41 +197,92 @@ pub struct Counters { pub smi: bool, /// `MSR_CORE_C6_RESIDENCY` and `MSR_CORE_C7_RESIDENCY` (3FDH, 3FEH). pub core_residency: bool, + /// The package's energy counter and its units, by whose registers. + pub package_energy: Option, + /// `MSR_PKG_C2_RESIDENCY`, `_C3_`, `_C6_` and `_C7_` (60DH, 3F8H, 3F9H, + /// 3FAH). + pub package_residency: bool, + /// AMD's `CORE_ENERGY_STAT` (C001_029AH). + pub core_energy: bool, } +/// Whose registers carry a package's energy and its units, each counting +/// `1/2^ESU` J, ESU bits 12:8 of the unit register, in bits 31:0 of the +/// energy register. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Rapl { + /// `MSR_RAPL_POWER_UNIT` (606H) and `MSR_PKG_ENERGY_STATUS` (611H): SDM + /// Vol. 3B §17.10.1 and §17.10.3, Table 2-20. + Intel, + /// `RAPL_PWR_UNIT` (C001_0299H) and `PKG_ENERGY_STAT` (C001_029BH): PPR + /// 55922, as 55772 and OSRR 56255 give them for Family 17h's other models. + Amd, +} + +/// `CPUID_Fn80000007_EDX[RAPL]`, "Running average power limit" (PPR 55922). +const CPUID_8000_0007_EDX_RAPL: u32 = 1 << 14; + pub fn counters(facts: &CounterFacts) -> Counters { let documented = matches!(facts.vendor, Vendor::Intel | Vendor::Amd); + let id = Ident::new(facts.vendor, facts.signature); + // ToyOS's rule for every model-specific counter: under a hypervisor the + // register is its emulation, not the hardware's count it is read for. + let hardware = facts.cpuid_1_ecx & CPUID_1_ECX_HYPERVISOR == 0; + let table_2_20 = hardware && in_table_2_20(&id); + let amd_rapl = hardware && amd_rapl(&id, facts.cpuid_8000_0007_edx); + let package_energy = if table_2_20 { + Some(Rapl::Intel) + } else if amd_rapl { + Some(Rapl::Amd) + } else { + None + }; Counters { // SDM Vol. 1 CPUID.06H:ECX[0]; PPR 55922 `CPUID_Fn00000006_ECX[EffFreq]`. aperf_mperf: documented && facts.cpuid_6_ecx & CPUID_6_ECX_APERFMPERF != 0, - // ToyOS's rule: under a hypervisor 34H is its emulation, not the - // firmware's count it is read for. - smi: counts_smis(&Ident::new(facts.vendor, facts.signature)) - && facts.cpuid_1_ecx & CPUID_1_ECX_HYPERVISOR == 0, - // The same rule: a hypervisor's emulation is not the core's count. - core_residency: counts_core_residency(&Ident::new(facts.vendor, facts.signature)) - && facts.cpuid_1_ecx & CPUID_1_ECX_HYPERVISOR == 0, + smi: hardware && counts_smis(&id), + core_residency: table_2_20, + package_energy: package_energy.filter(|_| facts.reads_package), + // Table 2-42 restates 3F8H for 06_66H without a rate. + package_residency: table_2_20 && id.model != 0x66 && facts.reads_package, + core_energy: amd_rapl, } } +/// Family 17h's PPRs in hand (55922 for Model 60h, 55772 for Model 20h, +/// OSRR 56255 for Models 00h-2Fh) each enumerate RAPL by +/// [`CPUID_8000_0007_EDX_RAPL`] and define C001_0299H, C001_029AH and +/// C001_029BH alike: 32-bit energy counters in `1/2^ESU` J. PPR 55901, for +/// Family 19h Model 11h, makes the core's counter 64 bits wide and the +/// package's an L3 complex's, so no other family is read by this rule. +fn amd_rapl(id: &Ident, cpuid_8000_0007_edx: u32) -> bool { + id.vendor == Vendor::Amd && id.family == 0x17 && cpuid_8000_0007_edx & CPUID_8000_0007_EDX_RAPL != 0 +} + /// The family 6 DisplayModels SDM Vol. 4 (325462-093) names as supporting -/// Table 2-20, each in its section's opening (§2.11-§2.16 and §2.17), which -/// defines `MSR_CORE_C6_RESIDENCY` (3FDH) and `MSR_CORE_C7_RESIDENCY` (3FEH) -/// as counting at the TSC's rate, the unit [`Counter::CoreC6`] promises. -/// Of the tables that supersede it for these models, only Table 2-57 names -/// either address, for 06_AFH, and counts both in units of the P1 clock, -/// so 06_AFH is left out. AMD documents no core residency counter in the -/// PPRs in hand. +/// Table 2-20, each in its section's opening (§2.11-§2.16 and §2.17). Table +/// 2-20 defines `MSR_CORE_C6_RESIDENCY` and `_C7_` (3FDH, 3FEH) and +/// `MSR_PKG_C2_RESIDENCY`, `_C3_`, `_C6_` and `_C7_` (60DH, 3F8H, 3F9H, +/// 3FAH) as counting at the TSC's rate, the unit [`Counter::CoreC6`] and +/// [`Counter::PackageC2`] promise, and `MSR_RAPL_POWER_UNIT` (606H) and +/// `MSR_PKG_ENERGY_STATUS` (611H) by §17.10. Of the tables that supersede it +/// for these models, Table 2-57 counts 3FDH and 3FEH in units of the P1 +/// clock and gives 3F9H and 3FAH no rate, so 06_AFH is left out, its energy +/// with them; Table 2-42 gives 06_66H's 3F8H no rate; and Tables 2-30, 2-32, +/// 2-36 and 2-50 restate 606H's energy unit as §17.10.1 gives it. Table 2-8, +/// which gives 606H another unit, is no listed model's. AMD documents no +/// residency counter in the PPRs in hand. /// /// [`Counter::CoreC6`]: toyos_abi::counters::Counter::CoreC6 -const CORE_RESIDENCY_MODELS: [u32; 36] = [ +/// [`Counter::PackageC2`]: toyos_abi::counters::Counter::PackageC2 +const TABLE_2_20_MODELS: [u32; 36] = [ 0x2a, 0x2d, 0x3a, 0x3e, 0x3c, 0x45, 0x46, 0x3f, 0x3d, 0x47, 0x4f, 0x56, 0x4e, 0x5e, 0x55, 0x8e, 0x9e, 0x66, 0xa5, 0xa6, 0x7e, 0x8c, 0x8d, 0x6a, 0x6c, 0x97, 0x9a, 0xba, 0xb7, 0xbf, 0x8f, 0xcf, 0xaa, 0xad, 0xae, 0xbd, ]; -fn counts_core_residency(id: &Ident) -> bool { - id.vendor == Vendor::Intel && id.family == 6 && CORE_RESIDENCY_MODELS.contains(&id.model) +fn in_table_2_20(id: &Ident) -> bool { + id.vendor == Vendor::Intel && id.family == 6 && TABLE_2_20_MODELS.contains(&id.model) } /// The Intel family 6 DisplayModels a section of SDM Vol. 4 names as @@ -366,40 +422,111 @@ mod tests { const T14_SIGNATURE: u32 = 0x0008_06c1; fn intel(signature: u32) -> CounterFacts { - CounterFacts { vendor: Vendor::Intel, signature, cpuid_1_ecx: 0, cpuid_6_ecx: CPUID_6_ECX_APERFMPERF } + CounterFacts { + vendor: Vendor::Intel, + signature, + cpuid_1_ecx: 0, + cpuid_6_ecx: CPUID_6_ECX_APERFMPERF, + cpuid_8000_0007_edx: 1 << 8, + reads_package: true, + } } + const NONE: Counters = Counters { + aperf_mperf: false, + smi: false, + core_residency: false, + package_energy: None, + package_residency: false, + core_energy: false, + }; + /// 06_8CH, which §2.17 names as supporting Table 2-20. #[test] - fn the_t14_counts_smis_and_has_aperf_and_mperf() { - assert_eq!(counters(&intel(T14_SIGNATURE)), Counters { aperf_mperf: true, smi: true, core_residency: true }); + fn the_t14_counts_smis_residency_and_its_packages_energy() { + let all = Counters { + aperf_mperf: true, + smi: true, + core_residency: true, + package_energy: Some(Rapl::Intel), + package_residency: true, + core_energy: false, + }; + assert_eq!(counters(&intel(T14_SIGNATURE)), all); + let ap = CounterFacts { reads_package: false, ..intel(T14_SIGNATURE) }; + assert_eq!(counters(&ap), Counters { package_energy: None, package_residency: false, ..all }); } /// 06_AFH: Table 2-57 counts its core residency at P1, not the TSC. #[test] - fn sierra_forest_counts_no_core_residency_in_tsc_ticks() { - assert!(!counters(&intel(0x000a_06f0)).core_residency); - assert!(counters(&intel(0x000a_06f0)).smi, "its 34H stands"); + fn sierra_forest_counts_no_residency_and_no_energy() { + let c = counters(&intel(0x000a_06f0)); + assert!(!c.core_residency && !c.package_residency && c.package_energy.is_none()); + assert!(c.smi, "its 34H stands"); + } + + /// 06_66H: Table 2-42 gives its 3F8H no rate, and supersedes nothing else + /// the rule reads. + #[test] + fn cannon_lake_counts_no_package_residency_and_the_rest() { + let c = counters(&intel(model(0x66))); + assert!(c.core_residency && c.package_energy == Some(Rapl::Intel) && !c.package_residency); } - /// The AMD bench, Family 17h Model 60h: PPR 55922 gives it `EffFreq` and - /// no `MSR0000_0034`. + /// The AMD bench, Family 17h Model 60h: PPR 55922 gives it `EffFreq`, + /// `CPUID_Fn80000007_EDX[RAPL]` and no `MSR0000_0034`; without the RAPL + /// bit it has no energy counter. #[test] - fn the_amd_bench_has_aperf_and_mperf_and_no_smi_count() { - let renoir = CounterFacts { vendor: Vendor::Amd, signature: 0x0086_0f01, cpuid_1_ecx: 0, cpuid_6_ecx: 1 }; - assert_eq!(counters(&renoir), Counters { aperf_mperf: true, smi: false, core_residency: false }); + fn the_amd_bench_has_aperf_and_mperf_its_energy_and_no_smi_count() { + let renoir = CounterFacts { + vendor: Vendor::Amd, + signature: 0x0086_0f01, + cpuid_1_ecx: 0, + cpuid_6_ecx: 1, + cpuid_8000_0007_edx: 0x6799, + reads_package: true, + }; + let all = Counters { aperf_mperf: true, package_energy: Some(Rapl::Amd), core_energy: true, ..NONE }; + assert_eq!(counters(&renoir), all); + assert_eq!(counters(&CounterFacts { reads_package: false, ..renoir }), Counters { package_energy: None, ..all }); + assert_eq!(counters(&CounterFacts { cpuid_8000_0007_edx: 0x2799, ..renoir }), Counters { aperf_mperf: true, ..NONE }); + } + + /// PPR 55901 changes the registers' widths and scope for Family 19h. + #[test] + fn only_amd_family_17h_reads_rapl() { + let zen4 = CounterFacts { + vendor: Vendor::Amd, + signature: 0x00a1_0f11, + cpuid_1_ecx: 0, + cpuid_6_ecx: 1, + cpuid_8000_0007_edx: 0x6799, + reads_package: true, + }; + assert_eq!(counters(&zen4), Counters { aperf_mperf: true, ..NONE }); + let hygon = CounterFacts { vendor: Vendor::Other(*b"HygonGenuine"), signature: 0x0090_0f01, ..zen4 }; + assert_eq!(counters(&hygon), NONE); } #[test] fn a_vendor_neither_intel_nor_amd_has_neither() { let other = CounterFacts { vendor: Vendor::Other(*b"CentaurHauls"), ..intel(T14_SIGNATURE) }; - assert_eq!(counters(&other), Counters { aperf_mperf: false, smi: false, core_residency: false }); + assert_eq!(counters(&other), NONE); } #[test] - fn a_guest_counts_no_smis() { + fn a_guest_counts_nothing_model_specific() { let guest = CounterFacts { cpuid_1_ecx: CPUID_1_ECX_HYPERVISOR, ..intel(T14_SIGNATURE) }; - assert_eq!(counters(&guest), Counters { aperf_mperf: true, smi: false, core_residency: false }); + assert_eq!(counters(&guest), Counters { aperf_mperf: true, ..NONE }); + let amd = CounterFacts { + vendor: Vendor::Amd, + signature: 0x0086_0f01, + cpuid_1_ecx: CPUID_1_ECX_HYPERVISOR, + cpuid_6_ecx: 1, + cpuid_8000_0007_edx: 0x6799, + reads_package: true, + }; + assert_eq!(counters(&amd), Counters { aperf_mperf: true, ..NONE }); } /// Family 6 signatures with the given DisplayModel, stepping 1. diff --git a/kernel/src/arch/aarch64/counters.rs b/kernel/src/arch/aarch64/counters.rs index 7fb20966ec2..2b0e3fa13e1 100644 --- a/kernel/src/arch/aarch64/counters.rs +++ b/kernel/src/arch/aarch64/counters.rs @@ -7,5 +7,16 @@ use crate::counters::Hardware; pub fn bring_up() {} pub fn read() -> Hardware { - Hardware { smi: None, aperf: None, mperf: None, envelope: None, firmware: None, core_c6: None, core_c7: None } + Hardware { + smi: None, + aperf: None, + mperf: None, + envelope: None, + firmware: None, + core_c6: None, + core_c7: None, + package: None, + package_residency: None, + core_energy: None, + } } diff --git a/kernel/src/arch/x86_64/counters.rs b/kernel/src/arch/x86_64/counters.rs index 47908c331ab..7108b107ee4 100644 --- a/kernel/src/arch/x86_64/counters.rs +++ b/kernel/src/arch/x86_64/counters.rs @@ -6,18 +6,38 @@ use core::sync::atomic::{AtomicU8, Ordering::Relaxed}; use kernel::cpu::CounterFacts; +use kernel::cpu::Rapl; + use super::cpu; -use crate::counters::{Firmware, Hardware}; +use crate::counters::{Firmware, Hardware, Package}; const MSR_SMI_COUNT: u32 = 0x34; const MSR_CORE_C6_RESIDENCY: u32 = 0x3FD; const MSR_CORE_C7_RESIDENCY: u32 = 0x3FE; const IA32_MPERF: u32 = 0xE7; const IA32_APERF: u32 = 0xE8; +// SDM Vol. 4 Table 2-20. +const MSR_RAPL_POWER_UNIT: u32 = 0x606; +const MSR_PKG_ENERGY_STATUS: u32 = 0x611; +const MSR_PKG_C2_RESIDENCY: u32 = 0x60D; +const MSR_PKG_C3_RESIDENCY: u32 = 0x3F8; +const MSR_PKG_C6_RESIDENCY: u32 = 0x3F9; +const MSR_PKG_C7_RESIDENCY: u32 = 0x3FA; +// PPR 55922 `RAPL_PWR_UNIT`, `CORE_ENERGY_STAT` and `PKG_ENERGY_STAT`. +const AMD_RAPL_PWR_UNIT: u32 = 0xC001_0299; +const AMD_CORE_ENERGY_STAT: u32 = 0xC001_029A; +const AMD_PKG_ENERGY_STAT: u32 = 0xC001_029B; const SMI: u8 = 1 << 0; const APERF_MPERF: u8 = 1 << 1; const CORE_RESIDENCY: u8 = 1 << 2; +const INTEL_RAPL: u8 = 1 << 3; +const AMD_RAPL: u8 = 1 << 4; +const PACKAGE_RESIDENCY: u8 = 1 << 5; +const CORE_ENERGY: u8 = 1 << 6; +/// Bits 31:0 of every energy register: bits 63:32 are reserved (SDM Vol. 3B +/// Figure 17-37; PPR 55922). +const ENERGY: u64 = 0xFFFF_FFFF; /// A CPU's verdict, written by that CPU before it reads a counter. pub struct Local { @@ -37,15 +57,26 @@ fn admitted() -> &'static AtomicU8 { pub fn bring_up() { let (signature, _, cpuid_1_ecx, _) = cpu::cpuid(1, 0); let (_, cpuid_6_ecx) = cpu::leaf_6(); + let cpuid_8000_0007_edx = + if cpu::cpuid(0x8000_0000, 0).0 >= 0x8000_0007 { cpu::cpuid(0x8000_0007, 0).3 } else { 0 }; let verdict = kernel::cpu::counters(&CounterFacts { vendor: cpu::vendor(), signature, cpuid_1_ecx, cpuid_6_ecx, + cpuid_8000_0007_edx, + reads_package: super::percpu::cpu_id() == 0, }); let bits = if verdict.smi { SMI } else { 0 } | if verdict.aperf_mperf { APERF_MPERF } else { 0 } - | if verdict.core_residency { CORE_RESIDENCY } else { 0 }; + | if verdict.core_residency { CORE_RESIDENCY } else { 0 } + | match verdict.package_energy { + Some(Rapl::Intel) => INTEL_RAPL, + Some(Rapl::Amd) => AMD_RAPL, + None => 0, + } + | if verdict.package_residency { PACKAGE_RESIDENCY } else { 0 } + | if verdict.core_energy { CORE_ENERGY } else { 0 }; admitted().store(bits, Relaxed); } @@ -62,5 +93,15 @@ pub fn read() -> Hardware { firmware: super::smi_cmd::counted().map(|(calls, nanos)| Firmware { calls, nanos }), core_c6: msr(CORE_RESIDENCY, MSR_CORE_C6_RESIDENCY), core_c7: msr(CORE_RESIDENCY, MSR_CORE_C7_RESIDENCY), + package: msr(INTEL_RAPL, MSR_PKG_ENERGY_STATUS) + .map(|energy| Package { energy: energy & ENERGY, power_unit: cpu::rdmsr(MSR_RAPL_POWER_UNIT) }) + .or_else(|| { + msr(AMD_RAPL, AMD_PKG_ENERGY_STAT) + .map(|energy| Package { energy: energy & ENERGY, power_unit: cpu::rdmsr(AMD_RAPL_PWR_UNIT) }) + }), + package_residency: (admitted & PACKAGE_RESIDENCY != 0).then(|| { + [MSR_PKG_C2_RESIDENCY, MSR_PKG_C3_RESIDENCY, MSR_PKG_C6_RESIDENCY, MSR_PKG_C7_RESIDENCY].map(cpu::rdmsr) + }), + core_energy: msr(CORE_ENERGY, AMD_CORE_ENERGY_STAT).map(|energy| energy & ENERGY), } } diff --git a/kernel/src/counters.rs b/kernel/src/counters.rs index 3009b37279e..6932a6ceeb3 100644 --- a/kernel/src/counters.rs +++ b/kernel/src/counters.rs @@ -57,6 +57,17 @@ pub struct Hardware { /// `MSR_CORE_C6_RESIDENCY` and `MSR_CORE_C7_RESIDENCY`, where admitted. pub core_c6: Option, pub core_c7: Option, + pub package: Option, + /// The package's C2, C3, C6 and C7 residency, where admitted. + pub package_residency: Option<[u64; 4]>, + pub core_energy: Option, +} + +/// The package's energy counter and its unit register, where this CPU reads +/// them for its package. +pub struct Package { + pub energy: u64, + pub power_unit: u64, } /// The commands this CPU wrote to the firmware, where it is the one that @@ -109,7 +120,7 @@ pub fn bring_up() { let me = percpu::cpu_id() as usize; let words = sample(me); block(me).publish(words); - let [smi, aperf, mperf, request, pkg, epb, c6, c7] = [ + let [smi, aperf, mperf, request, pkg, epb, c6, c7, energy, pc, core_energy] = [ Counter::Smi, Counter::Aperf, Counter::Mperf, @@ -118,11 +129,15 @@ pub fn bring_up() { Counter::EnergyPerfBias, Counter::CoreC6, Counter::CoreC7, + Counter::PackageEnergy, + Counter::PackageC2, + Counter::CoreEnergy, ] .map(|c| held(&words, c)); log!( "counters: cpu{me} reads smi={smi} aperf={aperf} mperf={mperf} hwp_request={request} \ - hwp_request_pkg={pkg} energy_perf_bias={epb} core_c6={c6} core_c7={c7}" + hwp_request_pkg={pkg} energy_perf_bias={epb} core_c6={c6} core_c7={c7} \ + package_energy={energy} package_residency={pc} core_energy={core_energy}" ); } @@ -151,6 +166,13 @@ fn sample(me: usize) -> [u64; WORDS] { Counter::DeepIdles => Some(crate::cpu_local::of(me as u32).idle.deep()), Counter::CoreC6 => hardware.core_c6, Counter::CoreC7 => hardware.core_c7, + Counter::PackageEnergy => hardware.package.as_ref().map(|p| p.energy), + Counter::RaplPowerUnit => hardware.package.as_ref().map(|p| p.power_unit), + Counter::PackageC2 => hardware.package_residency.map(|r| r[0]), + Counter::PackageC3 => hardware.package_residency.map(|r| r[1]), + Counter::PackageC6 => hardware.package_residency.map(|r| r[2]), + Counter::PackageC7 => hardware.package_residency.map(|r| r[3]), + Counter::CoreEnergy => hardware.core_energy, }; if let Some(value) = value { words[1] |= 1 << counter as usize; diff --git a/lib/inspect/src/kernel.rs b/lib/inspect/src/kernel.rs index ee1c72f0473..61f3d072c61 100644 --- a/lib/inspect/src/kernel.rs +++ b/lib/inspect/src/kernel.rs @@ -63,11 +63,19 @@ mod tests { Record { cpu, hardware_id: cpu * 2, stale, values } } - /// The T14's shape: every counter on every CPU, and the firmware's - /// calls on the boot processor, which makes them. + /// Every counter on every CPU, and the firmware's calls on the boot + /// processor, which makes them. fn whole(cpu: u32) -> Record { let (calls, nanos) = if cpu == 0 { (Some(1), Some(15_968)) } else { (None, None) }; - record(cpu, false, [Some(10), Some(4817), Some(100), Some(200), Some(3), Some(0x8000_2a04), Some(0x8000_ff01), Some(6), calls, nanos, Some(40), Some(1_000), Some(3_000)]) + record( + cpu, + false, + [ + Some(10), Some(4817), Some(100), Some(200), Some(3), Some(0x8000_2a04), Some(0x8000_ff01), Some(6), calls, + nanos, Some(40), Some(1_000), Some(3_000), Some(77_000), Some(0xa_0e03), Some(5), Some(6), Some(7), Some(8), + Some(900), + ], + ) } #[test] @@ -88,7 +96,9 @@ mod tests { /// `TRACE`: only what the record carries, and no zero in place of the rest. #[test] fn an_absent_counter_has_no_path() { - let got = render(&[record(0, false, [Some(10), None, None, None, None, None, None, None, None, None, None, None, None])]).unwrap(); + let mut values = [None; Counter::COUNT]; + values[Counter::Stamp as usize] = Some(10); + let got = render(&[record(0, false, values)]).unwrap(); let paths: Vec<&str> = got.keys().map(String::as_str).collect(); assert_eq!(paths, vec!["kernel.cpu.0.hardware_id", "kernel.cpu.0.stale", "kernel.cpu.0.stamp"]); } diff --git a/sdk/abi/src/counters.rs b/sdk/abi/src/counters.rs index e37e29bf70c..a73cd9270ae 100644 --- a/sdk/abi/src/counters.rs +++ b/sdk/abi/src/counters.rs @@ -12,9 +12,10 @@ //! absent and never zero; a record with no stamp is a CPU whose answer could //! not be read whole, and says nothing about it but its index. //! -//! Every counter counts up from boot, so the difference of two reads is exact; -//! the power envelope's registers ([`Counter::HwpRequest`], -//! [`Counter::HwpRequestPkg`] and [`Counter::EnergyPerfBias`]) count nothing +//! Every counter counts up, so the difference of two reads is exact, modulo +//! its width where it is narrower than 64 bits; the power envelope's +//! registers ([`Counter::HwpRequest`], [`Counter::HwpRequestPkg`] and +//! [`Counter::EnergyPerfBias`]) and [`Counter::RaplPowerUnit`] count nothing //! and are what the CPU held when it read them. //! //! [`SYS_COUNTERS`]: crate::syscall::SYS_COUNTERS @@ -65,10 +66,29 @@ pub enum Counter { CoreC6, /// The same for C7 (`MSR_CORE_C7_RESIDENCY`). CoreC7, + /// The energy the CPU's package consumed, 32 bits wide, in the unit + /// [`Counter::RaplPowerUnit`] gives (`MSR_PKG_ENERGY_STATUS`, AMD's + /// `PKG_ENERGY_STAT`). Held by the boot processor, for its package. + PackageEnergy, + /// The units of the package's power, energy and time counters + /// (`MSR_RAPL_POWER_UNIT`, AMD's `RAPL_PWR_UNIT`), beside the energy. + RaplPowerUnit, + /// Time the package was in its processor-specific C2 states, in TSC ticks + /// (`MSR_PKG_C2_RESIDENCY`). Held by the boot processor, for its package. + PackageC2, + /// The same for C3 (`MSR_PKG_C3_RESIDENCY`). + PackageC3, + /// The same for C6 (`MSR_PKG_C6_RESIDENCY`). + PackageC6, + /// The same for C7 (`MSR_PKG_C7_RESIDENCY`). + PackageC7, + /// The energy the CPU's core consumed, 32 bits wide, in the package's + /// unit (AMD's `CORE_ENERGY_STAT`): SMT siblings read their one core's. + CoreEnergy, } impl Counter { - pub const COUNT: usize = 13; + pub const COUNT: usize = 20; pub const ALL: [Counter; Self::COUNT] = [ Self::Stamp, Self::Smi, @@ -83,6 +103,13 @@ impl Counter { Self::DeepIdles, Self::CoreC6, Self::CoreC7, + Self::PackageEnergy, + Self::RaplPowerUnit, + Self::PackageC2, + Self::PackageC3, + Self::PackageC6, + Self::PackageC7, + Self::CoreEnergy, ]; pub const fn name(self) -> &'static str { @@ -100,10 +127,18 @@ impl Counter { Self::DeepIdles => "deep_idles", Self::CoreC6 => "core_c6", Self::CoreC7 => "core_c7", + Self::PackageEnergy => "package_energy", + Self::RaplPowerUnit => "rapl_power_unit", + Self::PackageC2 => "package_c2", + Self::PackageC3 => "package_c3", + Self::PackageC6 => "package_c6", + Self::PackageC7 => "package_c7", + Self::CoreEnergy => "core_energy", } } - /// The rights a `SysCap` carries for this counter to be answered. + /// The rights a `SysCap` carries for this counter to be answered. The + /// energy counters are a side channel (CVE-2020-8694). pub const fn needs(self) -> Rights { match self { Self::Stamp | Self::Smi | Self::FirmwareCalls | Self::FirmwareNanos => Rights::COUNTERS, @@ -115,7 +150,14 @@ impl Counter { | Self::EnergyPerfBias | Self::DeepIdles | Self::CoreC6 - | Self::CoreC7 => Rights::COUNTERS.union(Rights::TRACE), + | Self::CoreC7 + | Self::PackageEnergy + | Self::RaplPowerUnit + | Self::PackageC2 + | Self::PackageC3 + | Self::PackageC6 + | Self::PackageC7 + | Self::CoreEnergy => Rights::COUNTERS.union(Rights::TRACE), } } } @@ -217,8 +259,8 @@ mod tests { #[test] fn every_shape_of_record_reads_back_as_written() { for r in [ - record(false, [Some(1), Some(u64::from(u32::MAX)), Some(3), Some(u64::MAX), Some(0), Some(0x8000_2a04), Some(6), Some(0), Some(2), Some(31_936), Some(7_000_000), Some(5), None]), - record(true, [Some(9), None, None, None, Some(4), None, Some(0x8000_ff01), None, Some(0), None, Some(12), None, Some(9)]), + record(false, [Some(1), Some(u64::from(u32::MAX)), Some(3), Some(u64::MAX), Some(0), Some(0x8000_2a04), Some(6), Some(0), Some(2), Some(31_936), Some(7_000_000), Some(5), None, Some(u64::from(u32::MAX)), Some(0xa_0e03), Some(8), None, Some(0), Some(u64::MAX), None]), + record(true, [Some(9), None, None, None, Some(4), None, Some(0x8000_ff01), None, Some(0), None, Some(12), None, Some(9), None, None, None, Some(3), None, None, Some(77)]), record(true, [None; Counter::COUNT]), ] { assert_eq!(Record::decode(&r.encode()), Ok(r)); @@ -228,7 +270,10 @@ mod tests { /// Absent is not zero: a zero that is there reads back there. #[test] fn a_present_zero_is_not_an_absent_counter() { - let r = record(false, [Some(5), Some(0), None, None, None, None, None, None, None, None, None, None, None]); + let mut values = [None; Counter::COUNT]; + values[Counter::Stamp as usize] = Some(5); + values[Counter::Smi as usize] = Some(0); + let r = record(false, values); assert_eq!(Record::decode(&r.encode()).unwrap().get(Counter::Smi), Some(0)); assert_eq!(Record::decode(&r.encode()).unwrap().get(Counter::Aperf), None); } @@ -245,13 +290,15 @@ mod tests { #[test] fn a_word_under_an_absent_counter_is_refused() { - let mut raw = record(false, [Some(1), None, None, None, None, None, None, None, None, None, None, None, None]).encode(); + let mut values = [None; Counter::COUNT]; + values[Counter::Stamp as usize] = Some(1); + let mut raw = record(false, values).encode(); raw.0[16 + 8 * Counter::Mperf as usize] = 1; assert_eq!(Record::decode(&raw), Err(Undecodable::Absent(Counter::Mperf))); } - /// The counters that time programs and the power envelope are the trace - /// right's, and the four that are neither, the CPU's stamp and what its + /// The counters that time programs, the power envelope and the energy + /// counters are the trace right's, and the four that are neither, the CPU's stamp and what its /// firmware took of it, are readable on `COUNTERS` alone. #[test] fn what_times_a_program_or_is_power_needs_the_trace_right() { diff --git a/tests/guest/osbench/README.md b/tests/guest/osbench/README.md index d5eb80df4cd..def2ba8d04b 100644 --- a/tests/guest/osbench/README.md +++ b/tests/guest/osbench/README.md @@ -134,6 +134,59 @@ CPU's TSC read behind another's and no step exceeded the 1.2 µs of a read. A Linux forced onto the TSC (`tsc=reliable`) has those checks switched off, so it is no baseline. +## Energy: its own boot, its own Linux run + +`energy` is a group by name, outside `all`: idle power is the whole +machine's, so nothing else runs beside it. A ToyOS run is a boot of the +`osbench_energy` metal row, the benchcase image with that one job: + +```text +cargo test --test toyos-build -- --metal --metal-readback osbench_energy +``` + +A Linux run reads the same registers through the `msr` driver, as root: + +```text +sudo modprobe msr +sudo osbench energy /tmp/energy.txt; rfkill unblock all' + ``` + + `energy.radios` and `energy.links` say what each was. On the T14 both + systems keep the I219's link up. +- **The machine on AC, and the same stick or none in it.** + ## Both write the bench's NVMe `persist` and `fs.disk` run on each system's persistent filesystem on the diff --git a/tests/guest/osbench/compare/src/lib.rs b/tests/guest/osbench/compare/src/lib.rs index 4ed7516f55f..18ff087f94e 100644 --- a/tests/guest/osbench/compare/src/lib.rs +++ b/tests/guest/osbench/compare/src/lib.rs @@ -277,9 +277,10 @@ pub fn better(unit: &str) -> Option { const PATHS: [(&str, &str, usize); 2] = [("lan.", "lan.peer_link", 2), ("internet.", "internet.link", 1)]; /// The conditions every run of one system on one machine must share: the -/// kernel, the CPU's frequency request, and what serves each filesystem the -/// `fs` and `persist` rows use. -pub const SETUP: [&str; 13] = [ +/// kernel, the CPU's frequency request, what serves each filesystem the `fs` +/// and `persist` rows use, and the panel, its light, the radios and the links +/// the `energy` rows draw beside. +pub const SETUP: [&str; 17] = [ "kernel", "cpufreq_driver", "governor", @@ -293,6 +294,10 @@ pub const SETUP: [&str; 13] = [ "fs.mem.served_by", "fs.disk.served_by", "persist.served_by", + "energy.display", + "energy.backlight", + "energy.radios", + "energy.links", ]; /// The conditions a system chooses at each boot, stated per run: the clock @@ -615,7 +620,7 @@ mod tests { #[test] fn a_unit_of_no_known_direction_is_told_only_which_way_it_differs() { - let power = |os: &str, v: u64| run(os, &[&format!("bench=energy.idle_60s.package unit=mW n=1 min=1 p50={v} max=9 mean=5")]); + let power = |os: &str, v: u64| run(os, &[&format!("bench=energy.idle.package unit=mW n=1 min=1 p50={v} max=9 mean=5")]); let mut runs: Vec = [900, 910, 920, 930].iter().map(|v| power("linux", *v)).collect(); runs.extend([1500, 1510, 1520, 1530].iter().map(|v| power("toyos", *v))); let table = compare(&runs).unwrap(); diff --git a/tests/guest/osbench/rules/Cargo.toml b/tests/guest/osbench/rules/Cargo.toml index b36bb67dccf..40d34a97d06 100644 --- a/tests/guest/osbench/rules/Cargo.toml +++ b/tests/guest/osbench/rules/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "osbench-rules" -description = "osbench's pure rules, host-tested: when a munmap pair's x2APIC readings admit its figure, where the CPU crunch stops for heat, an energy counter's span across its wrap, whether two counters readings describe their row, and when a CPU's clock has settled." +description = "osbench's pure rules, host-tested: when a munmap pair's x2APIC readings admit its figure, where the CPU crunch stops for heat, the package energy counter's span, unit and power, when a machine is quiet, whether two counters readings describe their row, and when a CPU's clock has settled." version = "0.1.0" edition = "2021" license = "MIT OR Apache-2.0" diff --git a/tests/guest/osbench/rules/src/lib.rs b/tests/guest/osbench/rules/src/lib.rs index 5aa66442715..fa7e652d490 100644 --- a/tests/guest/osbench/rules/src/lib.rs +++ b/tests/guest/osbench/rules/src/lib.rs @@ -1,7 +1,7 @@ //! The judgements osbench makes over what it read, apart from the reading, so //! a host tests them: whether a munmap pair's figure stands ([`pair`]), where -//! the CPU crunch stops for heat ([`heat`]), how far an energy counter -//! advanced across its wrap ([`energy`]), whether two counters readings +//! the CPU crunch stops for heat ([`heat`]), what an energy counter +//! read and when the machine is quiet ([`energy`]), whether two counters readings //! describe the row they were read around ([`window`]), and when a CPU's //! clock has settled ([`settle`]). @@ -211,8 +211,13 @@ pub mod heat { } pub mod energy { - //! An energy counter that counts up from zero to its range and wraps to - //! zero, as Linux's powercap `energy_uj` does at `max_energy_range_uj`. + //! The package's energy counter as SDM Vol. 3B §17.10.1 and §17.10.3 and + //! PPR 55922's RAPL registers give it: 32 bits counting up and wrapping, + //! each step `1/2^ESU` J, ESU bits 12:8 of the unit register; and when + //! the machine is quiet enough for an idle reading to be its own. + + /// The counter's width: bits 31:0. + pub const RANGE: u64 = u32::MAX as u64; /// What the counter advanced from `before` to `after`, where it wrapped at /// most once between them; `None` for a reading outside `0..=range`. @@ -223,9 +228,32 @@ pub mod energy { Some(if after >= before { after - before } else { range - before + 1 + after }) } + /// `steps` of the counter whose unit register reads `power_unit`, in µJ. + pub fn microjoules(steps: u64, power_unit: u64) -> u64 { + let esu = (power_unit >> 8) & 0x1f; + u64::try_from((u128::from(steps) * 1_000_000) >> esu).expect("32-bit steps of at most 1 J") + } + + /// `uj` over `ns`, in mW. + pub fn milliwatts(uj: u64, ns: u64) -> u64 { + u64::try_from(u128::from(uj) * 1_000_000 / u128::from(ns.max(1))).unwrap_or(u64::MAX) + } + + /// The most CPU time the machine may spend in a second for that second to + /// be quiet: a hundredth of one CPU. + pub const QUIET_NS: u64 = 10_000_000; + /// The quiet seconds in a row an idle reading waits for. + pub const QUIET_SECONDS: usize = 3; + + /// Whether the last [`QUIET_SECONDS`] of `busy_ns`, each a second's CPU + /// time, were each quiet. + pub fn quiet(busy_ns: &[u64]) -> bool { + busy_ns.len() >= QUIET_SECONDS && busy_ns[busy_ns.len() - QUIET_SECONDS..].iter().all(|&ns| ns < QUIET_NS) + } + #[cfg(test)] mod tests { - use super::span; + use super::*; #[test] fn a_counter_that_did_not_wrap_spans_the_difference() { @@ -236,6 +264,7 @@ pub mod energy { fn a_wrap_counts_the_steps_to_the_range_and_on_from_zero() { assert_eq!(span(990, 9, 999), Some(19)); assert_eq!(span(999, 0, 999), Some(1)); + assert_eq!(span(RANGE, 4, RANGE), Some(5)); } #[test] @@ -243,6 +272,33 @@ pub mod energy { assert_eq!(span(1001, 5, 1000), None); assert_eq!(span(5, 1001, 1000), None); } + + /// §17.10.1's default ESU, 10000b, is 15.3 µJ a step; the T14's + /// datasheet default (631122, `PACKAGE_POWER_SKU_UNIT` 000A0E03h) + /// is 0EH, 61 µJ. No other field of the register moves it. + #[test] + fn a_step_is_two_to_the_minus_esu_joules() { + assert_eq!(microjoules(1 << 16, 0x000a_1003), 1_000_000); + assert_eq!(microjoules(1 << 14, 0x000a_0e03), 1_000_000); + assert_eq!(microjoules(1000, 0x000a_0e03), 61_035); + assert_eq!(microjoules(1 << 14, 0xffff_e0ff | 0x0e00), 1_000_000); + assert_eq!(microjoules(RANGE, 0x1f00), 1_999_999); + } + + #[test] + fn power_is_energy_over_time() { + assert_eq!(milliwatts(1_500_000, 1_000_000_000), 1_500); + assert_eq!(milliwatts(3_160, 1_000_000), 3_160); + assert_eq!(milliwatts(5, 0), 5_000_000, "no division by zero"); + } + + #[test] + fn quiet_is_every_one_of_the_last_seconds_under_the_bar() { + assert!(!quiet(&[0, 0]), "too few seconds"); + assert!(quiet(&[50_000_000, 0, 9_999_999, 0])); + assert!(!quiet(&[0, 0, QUIET_NS, 0, 0])); + assert!(quiet(&[0, 0, QUIET_NS, 0, 0, 0])); + } } } diff --git a/tests/guest/osbench/src/arch/x86_64.rs b/tests/guest/osbench/src/arch/x86_64.rs index 4628303457d..884bca9066c 100644 --- a/tests/guest/osbench/src/arch/x86_64.rs +++ b/tests/guest/osbench/src/arch/x86_64.rs @@ -6,6 +6,8 @@ pub use cpuid::{cpu_id, cpu_id_is}; #[cfg(target_os = "toyos")] pub use cpuid::{core_of, cpu_model, current_cpu, threads_per_core}; +#[cfg(target_os = "linux")] +pub use cpuid::power_registers; /// `turns` turns of 100 dependent register adds, each waiting one cycle for /// the one before it on every x86-64 core either bench has (Intel's and AMD's @@ -68,6 +70,33 @@ mod cpuid { use core::arch::x86_64::__cpuid_count; use std::sync::OnceLock; + /// The registers the energy group reads on Linux, the ones ToyOS's kernel + /// reads for the same counters (`kernel/pure/cpu.rs`), by CPUID.0's + /// vendor: Intel's from SDM Vol. 4 Table 2-20, AMD's from PPR 55922. A + /// register a CPU lacks reads as absent. + #[cfg(target_os = "linux")] + pub fn power_registers() -> Option { + let id = __cpuid_count(0, 0); + let vendor: Vec = [id.ebx, id.edx, id.ecx].iter().flat_map(|r| r.to_le_bytes()).collect(); + match &vendor[..] { + b"GenuineIntel" => Some(crate::sys::PowerRegisters { + energy: 0x611, + power_unit: 0x606, + package_residency: [Some(0x60d), Some(0x3f8), Some(0x3f9), Some(0x3fa)], + core_residency: [Some(0x3fd), Some(0x3fe)], + core_energy: None, + }), + b"AuthenticAMD" => Some(crate::sys::PowerRegisters { + energy: 0xc001_029b, + power_unit: 0xc001_0299, + package_residency: [None; 4], + core_residency: [None; 2], + core_energy: Some(0xc001_029a), + }), + _ => None, + } + } + struct Topology { /// CPUID leaf 0BH enumerates topology (SDM Vol. 2A: a subleaf 0 whose /// EBX[15:0] is 0 enumerates nothing). diff --git a/tests/guest/osbench/src/energy.rs b/tests/guest/osbench/src/energy.rs index 31e03cdf156..c682032f04e 100644 --- a/tests/guest/osbench/src/energy.rs +++ b/tests/guest/osbench/src/energy.rs @@ -1,79 +1,253 @@ -//! An idle minute's package energy: this program asleep for 60 s and the -//! CPU package's energy counter read each second, mW per second. Linux reads -//! its powercap `package-0` zone, whose counter takes root to read; ToyOS's -//! kernel counts no package energy for a program to read. +//! The package's power, idle and under defined loads, read off the same +//! registers on both systems (`sys::power`): its energy counter and unit +//! (SDM Vol. 3B §17.10, PPR 55922's RAPL), its C-state residency where Intel +//! documents it, and each CPU's APERF/MPERF, core residency and, on AMD, core +//! energy. ToyOS reads them through `SYS_COUNTERS` under `Rights::TRACE`, and +//! Linux through `/dev/cpu//msr` as root. +//! +//! **First the machine is let go quiet** ([`osbench_rules::energy::quiet`]), +//! for at most [`QUIET_BOUND`]: a ToyOS boot starts the group while its +//! services are still starting, the ACPI server's idle states among them. +//! +//! Then each load runs for its span, and the package is read once a second: +//! each row is that second's power, its residency and, under a spin, its +//! energy per [`crate::chunk`]. Each CPU's counters are read at the load's +//! two ends, so Linux sends no IPI inside a second; ToyOS's every read kicks +//! every CPU. No figure is better higher or lower by itself: a load that ran +//! more work may draw more. -const BENCH: &str = "energy.idle_60s.package"; +use std::sync::atomic::{AtomicBool, AtomicU64, Ordering::Relaxed}; +use std::time::{Duration, Instant}; + +use osbench_rules::energy::{self, RANGE}; + +use crate::report::{note, result}; +#[cfg(target_os = "linux")] +use crate::sys; +use crate::sys::{power, CpuPower, Package, Refused}; + +/// The longest the group waits for the machine to go quiet. +const QUIET_BOUND: Duration = Duration::from_secs(120); + +/// The residency rows, in [`Package::residency`]'s order. +const STATES: [&str; 4] = ["pc2", "pc3", "pc6", "pc7"]; + +#[derive(Clone, Copy)] +enum Load { + Idle, + /// One thread sleeping this long and waking, over and over. + Wake(Duration), + /// This many threads each running [`crate::chunk`] back to back. + Spin(usize), +} pub fn suite() { - imp::suite(); + #[cfg(target_os = "linux")] + { + power::conditions(); + // Its own CPU is the one whose registers the package is read from. + sys::pin_self(0); + } + let loads = [ + ("energy.idle", 60, Load::Idle), + ("energy.wake_10ms", 30, Load::Wake(Duration::from_millis(10))), + ("energy.spin.t1", 30, Load::Spin(1)), + ("energy.spin.all", 30, Load::Spin(crate::cpus())), + ]; + if let Err(why) = power::package() { + for (bench, ..) in loads { + why.clone().report(&format!("{bench}.package")); + } + return; + } + wait_quiet(); + for (bench, seconds, load) in loads { + let mut measured = Ok(()); + crate::around::ran_during(bench, || measured = row(bench, seconds, load)); + if let Err(why) = measured { + why.report(&format!("{bench}.package")); + } + } } -#[cfg(target_os = "toyos")] -mod imp { - pub fn suite() { - crate::report::unsupported( - super::BENCH, - "no counter carries the package's energy: toyos_abi::counters::Counter has none", - ); +/// Seconds of the machine's CPU time, each read a second apart, until the +/// last few were each quiet or [`QUIET_BOUND`] has passed. +fn wait_quiet() { + let start = Instant::now(); + let mut seconds = Vec::new(); + let mut before = match power::busy_ns() { + Ok(ns) => ns, + Err(why) => return note("energy.quiet", &[("unread", why.why())]), + }; + while !energy::quiet(&seconds) && start.elapsed() < QUIET_BOUND { + std::thread::sleep(Duration::from_secs(1)); + let now = match power::busy_ns() { + Ok(now) => now, + Err(why) => return note("energy.quiet", &[("unread", why.why())]), + }; + // A ToyOS process that ended leaves the roster with its time. + seconds.push(now.saturating_sub(before)); + before = now; } + let last_us: Vec = seconds.iter().rev().take(energy::QUIET_SECONDS).rev().map(|ns| (ns / 1000).to_string()).collect(); + note( + "energy.quiet", + &[ + ("quiet", energy::quiet(&seconds).to_string()), + ("waited_s", start.elapsed().as_secs().to_string()), + ("last_busy_us", last_us.join(",")), + ], + ); } -#[cfg(target_os = "linux")] -mod imp { - use std::fs; - use std::path::{Path, PathBuf}; - use std::time::{Duration, Instant}; +/// One thread's chunk count, on a line of its own. +#[repr(align(128))] +#[derive(Default)] +struct Slot(AtomicU64); - use crate::report::{condition, refuse, result, unmeasured}; +/// Run `load` for `seconds`, reading the package each second, and report. +fn row(bench: &str, seconds: u64, load: Load) -> Result<(), Refused> { + if let Load::Spin(_) = load { + crate::clock::settle(); + } + let threads = match load { + Load::Idle => 0, + Load::Wake(_) => 1, + Load::Spin(n) => n, + }; + let stop = AtomicBool::new(false); + let counts: Vec = (0..threads).map(|_| Slot::default()).collect(); + let first = power::cpus()?; + let mut samples = Samples::default(); + let sampled = std::thread::scope(|s| { + for slot in &counts { + let stop = &stop; + s.spawn(move || { + #[cfg(target_os = "linux")] + sys::unpin(sys::tid()); + while !stop.load(Relaxed) { + match load { + Load::Wake(period) => std::thread::sleep(period), + _ => { + crate::chunk(); + } + } + slot.0.fetch_add(1, Relaxed); + } + }); + } + let sampled = sample(seconds, &counts, matches!(load, Load::Spin(_)), &mut samples); + stop.store(true, Relaxed); + sampled + }); + sampled?; + let last = power::cpus()?; + let unit = samples.power_unit; + let mut fields = vec![("seconds", seconds.to_string())]; + let done: u64 = counts.iter().map(|c| c.0.load(Relaxed)).sum(); + match load { + Load::Idle => {} + Load::Wake(_) => fields.push(("wakes", done.to_string())), + Load::Spin(n) => { + fields.push(("threads", n.to_string())); + fields.push(("chunks_per_s", (done / seconds.max(1)).to_string())); + } + } + if let Some(mw) = unit.and_then(|unit| cores_mw(&first, &last, unit, samples.span_ns)) { + fields.push(("cores_mw", mw.to_string())); + } + result(&format!("{bench}.package"), "mW", &mut samples.mw, &fields); + for (state, permille) in STATES.iter().zip(&mut samples.residency) { + if !permille.is_empty() { + result(&format!("{bench}.{state}"), "permille", permille, &[]); + } + } + if !samples.nj_per_chunk.is_empty() { + result(&format!("{bench}.per_chunk"), "nJ", &mut samples.nj_per_chunk, &[]); + } + cpus(bench, &first, &last); + Ok(()) +} - const SECONDS: usize = 60; +#[derive(Default)] +struct Samples { + mw: Vec, + residency: [Vec; 4], + nj_per_chunk: Vec, + power_unit: Option, + span_ns: u64, +} - fn read_u64(path: &Path) -> std::io::Result { - fs::read_to_string(path)?.trim().parse().map_err(std::io::Error::other) +/// A reading of the package each second for `seconds`, into `out`, with the +/// energy per chunk where `per_chunk`. +fn sample(seconds: u64, counts: &[Slot], per_chunk: bool, out: &mut Samples) -> Result<(), Refused> { + let chunks = || counts.iter().map(|c| c.0.load(Relaxed)).sum::(); + let mut before: Package = power::package()?; + let (start, mut at, mut done) = (Instant::now(), Instant::now(), chunks()); + out.power_unit = before.power_unit; + for second in 1..=seconds { + // The span is the measurement: each read is due a second after the start, not after the last read. + std::thread::sleep((start + Duration::from_secs(second)).saturating_duration_since(Instant::now())); + let after = power::package()?; + let (now, done_now) = (Instant::now(), chunks()); + let ns = u64::try_from((now - at).as_nanos()).expect("a second under 584 years"); + if let (Some(e0), Some(e1), Some(unit)) = (before.energy, after.energy, after.power_unit) { + let steps = energy::span(e0, e1, RANGE) + .ok_or_else(|| Refused::Failed(format!("the energy counter read {e1:#x}, past 32 bits")))?; + let uj = energy::microjoules(steps, unit); + out.mw.push(energy::milliwatts(uj, ns)); + if per_chunk && done_now > done { + out.nj_per_chunk.push(uj * 1000 / (done_now - done)); + } + } + let ticks = after.tsc.wrapping_sub(before.tsc); + for (i, rows) in out.residency.iter_mut().enumerate() { + if let (Some(r0), Some(r1)) = (before.residency[i], after.residency[i]) { + rows.push(osbench_rules::window::permille(r1.wrapping_sub(r0), ticks)); + } + } + (before, at, done) = (after, now, done_now); } + out.span_ns = u64::try_from(start.elapsed().as_nanos()).expect("a row under 584 years"); + Ok(()) +} - /// The powercap zone whose name is `package-0`. - fn zone() -> Option { - let mut zones: Vec = fs::read_dir("/sys/class/powercap").ok()?.flatten().map(|e| e.path()).collect(); - zones.sort(); - zones.into_iter().find(|z| fs::read_to_string(z.join("name")).is_ok_and(|n| n.trim() == "package-0")) +/// The cores' power over the row where each core counts its own energy: +/// each core once, SMT siblings reading the same counter. +fn cores_mw(first: &[CpuPower], last: &[CpuPower], unit: u64, span_ns: u64) -> Option { + let mut seen = std::collections::BTreeSet::new(); + let mut steps = 0; + for b in last { + let a = first.iter().find(|a| a.cpu == b.cpu)?; + if seen.insert(b.core) { + steps += energy::span(a.core_energy?, b.core_energy?, RANGE)?; + } } + (!seen.is_empty()).then(|| energy::milliwatts(energy::microjoules(steps, unit), span_ns)) +} - pub fn suite() { - let Some(zone) = zone() else { - return unmeasured(super::BENCH, "no powercap zone is named package-0", &[]); - }; - let energy = zone.join("energy_uj"); - let range = match read_u64(&zone.join("max_energy_range_uj")) { - Ok(range) => range, - Err(e) => return refuse(super::BENCH, format!("max_energy_range_uj: {e}")), - }; - let mut before = match read_u64(&energy) { - Ok(uj) => uj, - Err(e) if e.kind() == std::io::ErrorKind::PermissionDenied => { - return unmeasured(super::BENCH, &format!("energy_uj was refused ({e}): `sudo osbench energy` reads it"), &[]); - } - Err(e) => return refuse(super::BENCH, format!("energy_uj: {e}")), - }; - condition("energy.zone", zone.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default()); - let start = Instant::now(); - let mut at = start; - let mut mw = Vec::with_capacity(SECONDS); - for second in 1..=SECONDS { - // The minute is the measurement: each read is due a second after the start, not after the last read. - std::thread::sleep((start + Duration::from_secs(second as u64)).saturating_duration_since(Instant::now())); - let after = match read_u64(&energy) { - Ok(uj) => uj, - Err(e) => return refuse(super::BENCH, format!("energy_uj: {e}")), - }; - let now = Instant::now(); - let Some(uj) = osbench_rules::energy::span(before, after, range) else { - return refuse(super::BENCH, format!("energy_uj read {after}, past its range {range}")); - }; - mw.push(uj * 1000 / u64::try_from((now - at).as_micros()).unwrap_or(u64::MAX).max(1)); - (before, at) = (after, now); - } - result(super::BENCH, "mW", &mut mw, &[("seconds", SECONDS.to_string())]); +/// One line a CPU: its APERF/MPERF ratio and its core's C6 and C7 residency +/// over the row, each against its own TSC, which the residency counts at +/// (SDM Vol. 4 Table 2-20). +fn cpus(bench: &str, first: &[CpuPower], last: &[CpuPower]) { + let delta = |x: Option, y: Option| x.zip(y).map(|(x, y)| y.wrapping_sub(x)); + let shown = |v: Option| v.map_or("-".to_string(), |v| v.to_string()); + for b in last { + let Some(a) = first.iter().find(|a| a.cpu == b.cpu) else { continue }; + let ticks = b.tsc.wrapping_sub(a.tsc); + let permille = |x, y| shown(delta(x, y).map(|d| osbench_rules::window::permille(d, ticks))); + let ratio = delta(a.aperf, b.aperf) + .zip(delta(a.mperf, b.mperf)) + .and_then(|(aperf, mperf)| osbench_rules::window::aperf_mperf_permille(aperf, mperf)); + note( + &format!("{bench}.cpus"), + &[ + ("cpu", b.cpu.to_string()), + ("core", b.core.to_string()), + ("aperf_mperf_permille", shown(ratio)), + ("core_c6_permille", permille(a.core_c6, b.core_c6)), + ("core_c7_permille", permille(a.core_c7, b.core_c7)), + ], + ); } } diff --git a/tests/guest/osbench/src/main.rs b/tests/guest/osbench/src/main.rs index 9e88b2f3654..e2ecd32319f 100644 --- a/tests/guest/osbench/src/main.rs +++ b/tests/guest/osbench/src/main.rs @@ -21,7 +21,7 @@ //! tail group's diagnostics. Everything else is the same source on both. //! //! On Linux `nvme` and `energy` read what only root may (the raw namespace, -//! the package's energy counter) and say so unprivileged. `README.md` is how +//! the model-specific registers) and say so unprivileged. `README.md` is how //! the Linux half is built and how a comparison is run. mod arch; @@ -131,13 +131,12 @@ fn files() { /// Every group the machine alone runs, in the order `all` and the ToyOS job /// list run them: `boot` first, since it reads how long the boot took to -/// reach it; `energy` while the machine is still quiet; and `crunch`, which -/// heats it, last: ToyOS reads no temperature to cool down by, so a group -/// after it would start at a clock its Linux run does not. -const GROUPS: [(&str, fn()); 17] = [ +/// reach it; and `crunch`, which heats it, last: ToyOS reads no temperature +/// to cool down by, so a group after it would start at a clock its Linux run +/// does not. +const GROUPS: [(&str, fn()); 16] = [ ("boot", boot::suite), ("identity", identity::suite), - ("energy", energy::suite), ("system", system), ("sched", sched), ("fair", fair), @@ -154,9 +153,10 @@ const GROUPS: [(&str, fn()); 17] = [ ("crunch", crunch::suite), ]; -/// The groups that need another party: `lan` the `osbench-peer` on the other -/// bench, `internet` the public internet. -const BY_NAME: [(&str, fn()); 2] = [("lan", lan::suite), ("internet", internet::suite)]; +/// The groups that run only by name: `lan` needs the `osbench-peer` on the +/// other bench, `internet` the public internet, and `energy` a machine +/// running nothing else, in a ToyOS boot of its own and as root on Linux. +const BY_NAME: [(&str, fn()); 3] = [("lan", lan::suite), ("internet", internet::suite), ("energy", energy::suite)]; /// A group between its first and last lines, after the machine's conditions /// and the clock its CPU starts at. diff --git a/tests/guest/osbench/src/sys/linux.rs b/tests/guest/osbench/src/sys/linux.rs index 45188760ccc..2d7d873c20c 100644 --- a/tests/guest/osbench/src/sys/linux.rs +++ b/tests/guest/osbench/src/sys/linux.rs @@ -278,6 +278,145 @@ pub fn conditions() { condition("busy_permille_1s", busy_permille(Duration::from_secs(1)).map_or("unknown".into(), |p| p.to_string())); } +/// Model-specific register addresses, by [`crate::arch`]'s choice for the +/// CPU's vendor. +pub struct PowerRegisters { + pub energy: u32, + pub power_unit: u32, + pub package_residency: [Option; 4], + pub core_residency: [Option; 2], + pub core_energy: Option, +} + +/// The power counters through the `msr` driver's `/dev/cpu//msr`, which +/// takes root: the package's on CPU 0, which reads them locally where the +/// caller runs there, and each CPU's on that CPU, an IPI each. +pub mod power { + use std::fs::File; + use std::os::unix::fs::FileExt; + + use super::super::{CpuPower, Package, Refused}; + use super::PowerRegisters; + + const IA32_TIME_STAMP_COUNTER: u32 = 0x10; + const IA32_MPERF: u32 = 0xe7; + const IA32_APERF: u32 = 0xe8; + + fn registers() -> Result { + #[cfg(target_arch = "x86_64")] + let registers = crate::arch::power_registers(); + #[cfg(not(target_arch = "x86_64"))] + let registers = None; + registers.ok_or_else(|| Refused::Absent("no CPU vendor whose power registers osbench reads".into())) + } + + fn device(cpu: u32) -> Result { + let path = format!("/dev/cpu/{cpu}/msr"); + File::open(&path).map_err(|e| match e.kind() { + std::io::ErrorKind::NotFound => Refused::Withheld(format!("{path}: {e}: `sudo modprobe msr`")), + std::io::ErrorKind::PermissionDenied => Refused::Withheld(format!("{path}: {e}: `sudo osbench energy` reads it")), + _ => Refused::Failed(format!("{path}: {e}")), + }) + } + + /// The register, or `None` where the CPU lacks it: the driver answers a + /// `#GP` with `EIO`. + fn msr(dev: &File, cpu: u32, at: u32) -> Result, Refused> { + let mut word = [0u8; 8]; + match dev.read_at(&mut word, u64::from(at)) { + Ok(8) => Ok(Some(u64::from_le_bytes(word))), + Err(e) if e.raw_os_error() == Some(libc::EIO) => Ok(None), + other => Err(Refused::Failed(format!("cpu{cpu}'s MSR {at:#x}: {other:?}"))), + } + } + + fn each(dev: &File, cpu: u32, at: Option) -> Result, Refused> { + at.map_or(Ok(None), |at| msr(dev, cpu, at)) + } + + pub fn package() -> Result { + let r = registers()?; + let dev = device(0)?; + let tsc = msr(&dev, 0, IA32_TIME_STAMP_COUNTER)?.ok_or_else(|| Refused::Failed("cpu0 read no TSC".into()))?; + let mut residency = [None; 4]; + for (slot, at) in residency.iter_mut().zip(r.package_residency) { + *slot = each(&dev, 0, at)?; + } + Ok(Package { tsc, energy: msr(&dev, 0, r.energy)?, power_unit: msr(&dev, 0, r.power_unit)?, residency }) + } + + pub fn cpus() -> Result, Refused> { + let r = registers()?; + super::allowed_cpus() + .into_iter() + .map(|cpu| { + let cpu = cpu as u32; + let dev = device(cpu)?; + Ok(CpuPower { + cpu, + core: super::core_of(cpu), + tsc: msr(&dev, cpu, IA32_TIME_STAMP_COUNTER)? + .ok_or_else(|| Refused::Failed(format!("cpu{cpu} read no TSC")))?, + aperf: msr(&dev, cpu, IA32_APERF)?, + mperf: msr(&dev, cpu, IA32_MPERF)?, + core_c6: each(&dev, cpu, r.core_residency[0])?, + core_c7: each(&dev, cpu, r.core_residency[1])?, + core_energy: each(&dev, cpu, r.core_energy)?, + }) + }) + .collect() + } + + /// The CPU time the whole machine has been busy for, in nanoseconds, off + /// `/proc/stat` at its tick's resolution: everything but idle and iowait, + /// this sleeping process included. + pub fn busy_ns() -> Result { + let stat = std::fs::read_to_string("/proc/stat").map_err(|e| Refused::Failed(format!("/proc/stat: {e}")))?; + let fields: Vec = + stat.lines().next().unwrap_or_default().split_whitespace().skip(1).filter_map(|f| f.parse().ok()).collect(); + let (Some(idle), Some(iowait)) = (fields.get(3), fields.get(4)) else { + return Err(Refused::Failed("/proc/stat's first line has no idle and iowait".into())); + }; + // SAFETY: sysconf takes a name and reads nothing else. + let hz = u64::try_from(unsafe { libc::sysconf(libc::_SC_CLK_TCK) }).unwrap_or(100).max(1); + Ok((fields.iter().sum::() - idle - iowait) * 1_000_000_000 / hz) + } + + /// The panel and its light, the radios and the links, which the package's + /// draw depends on: each connected DRM connector's DPMS, each backlight's + /// level, each rfkill switch, and each network link's state. + pub fn conditions() { + use crate::report::condition; + let listed = |dir: &str, said: &dyn Fn(&std::path::Path) -> Option| -> String { + let mut seen: Vec = std::fs::read_dir(dir) + .into_iter() + .flatten() + .flatten() + .filter_map(|e| said(&e.path()).map(|s| format!("{}:{s}", e.file_name().to_string_lossy()))) + .collect(); + seen.sort(); + if seen.is_empty() { "none".into() } else { seen.join(",") } + }; + let get = |at: &std::path::Path, f: &str| super::read(&at.join(f).to_string_lossy()); + condition( + "energy.display", + listed("/sys/class/drm", &|at| (get(at, "status")? == "connected").then(|| get(at, "dpms").unwrap_or_default())), + ); + condition( + "energy.backlight", + listed("/sys/class/backlight", &|at| Some(format!("{}/{}", get(at, "actual_brightness")?, get(at, "max_brightness")?))), + ); + condition( + "energy.radios", + listed("/sys/class/rfkill", &|at| { + let blocked = get(at, "soft")? == "1" || get(at, "hard")? == "1"; + Some(format!("{}_{}", get(at, "type")?, if blocked { "blocked" } else { "unblocked" })) + }), + ); + condition("energy.links", listed("/sys/class/net", &|at| if at.ends_with("lo") { None } else { get(at, "operstate") })); + } +} + /// The cgroup whose `cpu` controller governs this process: the deepest one /// on its path whose parent enables `cpu` for its children, or `/` when /// none does. diff --git a/tests/guest/osbench/src/sys/mod.rs b/tests/guest/osbench/src/sys/mod.rs index 55cc08e31c5..5861093b198 100644 --- a/tests/guest/osbench/src/sys/mod.rs +++ b/tests/guest/osbench/src/sys/mod.rs @@ -1,7 +1,7 @@ //! What std does not offer portably, as each OS offers it: the cheapest //! syscall, a futex, a clock every process shares, anonymous mappings, IPC -//! between processes, a process's own accounting and the machine's -//! conditions. Both modules export the same names; Linux adds affinity, +//! between processes, a process's own accounting, the machine's +//! conditions and its power counters (`power`). Both modules export the same names; Linux adds affinity, //! which ToyOS has none of, and the readers of its cgroup and autogroup, and //! ToyOS the diagnostics the tail group reads (`diag`). @@ -41,19 +41,22 @@ impl Counts { } /// Why a call did not do its work: a facility the system does not offer, by -/// the one answer that says so, or any other failure. +/// the one answer that says so; one it offers that this run may not use, such +/// as what only root reads; or any other failure. #[derive(Clone)] pub enum Refused { Absent(String), + Withheld(String), Failed(String), } impl Refused { - /// Report `bench` as unsupported for an absence, and refused for anything - /// else. + /// Report `bench` as unsupported for an absence, unmeasured for what this + /// run was not granted, and refused for anything else. pub fn report(self, bench: &str) { match self { Refused::Absent(why) => crate::report::unsupported(bench, why), + Refused::Withheld(why) => crate::report::unmeasured(bench, &why, &[]), Refused::Failed(why) => crate::report::refuse(bench, why), } } @@ -62,11 +65,36 @@ impl Refused { /// and so is failing now. pub fn why(self) -> String { match self { - Refused::Absent(why) | Refused::Failed(why) => why, + Refused::Absent(why) | Refused::Withheld(why) | Refused::Failed(why) => why, } } } +/// The package's power counters as its registers hold them, read on one CPU: +/// that CPU's TSC beside them, the energy counter and its unit register +/// (`osbench_rules::energy`), and its C2, C3, C6 and C7 residency where the +/// CPU documents them, each `None` where it does not. +pub struct Package { + pub tsc: u64, + pub energy: Option, + pub power_unit: Option, + pub residency: [Option; 4], +} + +/// One CPU's power counters as its registers hold them: its TSC beside them, +/// APERF and MPERF, its core's C6 and C7 residency and its core's energy, in +/// the package's unit, each `None` where the CPU does not have it. +pub struct CpuPower { + pub cpu: u32, + pub core: u32, + pub tsc: u64, + pub aperf: Option, + pub mperf: Option, + pub core_c6: Option, + pub core_c7: Option, + pub core_energy: Option, +} + /// A whole anonymous read-write mapping, unmapped on drop. pub struct Mapping { ptr: *mut u8, diff --git a/tests/guest/osbench/src/sys/toyos.rs b/tests/guest/osbench/src/sys/toyos.rs index 207ed14dc7f..e8ef328c6fc 100644 --- a/tests/guest/osbench/src/sys/toyos.rs +++ b/tests/guest/osbench/src/sys/toyos.rs @@ -244,6 +244,11 @@ pub mod diag { /// Idle periods the kernel chose a declared deep state for, counted /// as each ends. pub deep_idles: Option, + /// The package's counters, on the CPU that reads them. + pub package_energy: Option, + pub rapl_power_unit: Option, + pub package_residency: [Option; 4], + pub core_energy: Option, } /// The capability's diagnostics for the rows that read the counters @@ -281,6 +286,11 @@ pub mod diag { core_c6: r.get(Counter::CoreC6), core_c7: r.get(Counter::CoreC7), deep_idles: r.get(Counter::DeepIdles), + package_energy: r.get(Counter::PackageEnergy), + rapl_power_unit: r.get(Counter::RaplPowerUnit), + package_residency: [Counter::PackageC2, Counter::PackageC3, Counter::PackageC6, Counter::PackageC7] + .map(|c| r.get(c)), + core_energy: r.get(Counter::CoreEnergy), }) }) .collect() @@ -369,6 +379,69 @@ pub mod diag { } } +/// The power counters, through the runner's capability: `SYS_COUNTERS` +/// under `Rights::TRACE`, the package's held by the boot processor. Every read +/// kicks every CPU (`toyos_abi::counters`), the package's too. +pub mod power { + use super::super::{CpuPower, Package, Refused}; + use super::diag::{self, CpuCounts}; + + fn read() -> Result, Refused> { + let d = diag::shared().ok_or_else(|| Refused::Withheld("this boot endows osbench no capability".into()))?; + d.counters().map_err(Refused::Failed) + } + + pub fn package() -> Result { + let cpus = read()?; + let held = cpus.into_iter().find(|c| c.package_energy.is_some() || c.package_residency.iter().any(Option::is_some)); + let Some(c) = held else { + return Err(Refused::Absent( + "no CPU's record carries the package's counters: the kernel's `counters: cpu0 reads` line says whether it \ + reads them, or the capability holds no TRACE" + .into(), + )); + }; + if c.stale { + return Err(Refused::Failed(format!("cpu{}, which reads the package, answered stale", c.cpu))); + } + Ok(Package { + tsc: c.stamp.ok_or_else(|| Refused::Failed(format!("cpu{}'s record carries no stamp", c.cpu)))?, + energy: c.package_energy, + power_unit: c.rapl_power_unit, + residency: c.package_residency, + }) + } + + pub fn cpus() -> Result, Refused> { + read()? + .into_iter() + .map(|c| { + if c.stale { + return Err(Refused::Failed(format!("cpu{} answered stale", c.cpu))); + } + Ok(CpuPower { + cpu: c.cpu, + core: crate::arch::core_of(c.apic), + tsc: c.stamp.ok_or_else(|| Refused::Failed(format!("cpu{}'s record carries no stamp", c.cpu)))?, + aperf: c.aperf, + mperf: c.mperf, + core_c6: c.core_c6, + core_c7: c.core_c7, + core_energy: c.core_energy, + }) + }) + .collect() + } + + /// The CPU time every process but this one has run for, in nanoseconds, + /// by the roster. + pub fn busy_ns() -> Result { + let d = diag::shared().ok_or_else(|| Refused::Withheld("this boot endows osbench no capability".into()))?; + let me = std::process::id(); + Ok(d.roster().iter().filter(|((pid, _), _)| *pid != me).map(|(_, (_, ns))| ns).sum()) + } +} + pub fn protect(ptr: *mut u8, len: usize, writable: bool) -> Result<(), super::Refused> { let prot = if writable { VmProt::ReadWrite } else { VmProt::Read }; // SAFETY: the caller's mapping; a protection change moves no memory. diff --git a/tests/images/benchcase.toml b/tests/images/benchcase.toml index e172d1a076a..061fcaccbb9 100644 --- a/tests/images/benchcase.toml +++ b/tests/images/benchcase.toml @@ -49,7 +49,6 @@ args = [ "--bound-ms=525000", "osbench_boot", "osbench_identity", - "osbench_energy", "osbench_system", "osbench_sched", "osbench_fair", diff --git a/tests/toyos.rs b/tests/toyos.rs index 117b7aac09c..d1339863334 100644 --- a/tests/toyos.rs +++ b/tests/toyos.rs @@ -1158,6 +1158,16 @@ const METAL: &[(&str, metal::Metal)] = &[ }, }, ), + ( + // `osbench energy`: the package's power, idle and under defined loads, + // compared with the same program as root under the bench's Linux. A + // boot of its own, since idle power is the whole machine's: nothing + // else may run in it. Evidence and not a verdict, as `osbench` is; the + // row holds that the idle row measured, which it does only where the + // kernel reads the package's energy. The T14 and the AMD bench. + "osbench_energy", + metal::Metal { arms: OSBENCH_ENERGY, judge: |b| osbench_energy_measured(b[0]) }, + ), ( // `osbench lan` against the `osbench-peer` the other bench's Linux runs, // compared with the same program under this bench's Linux on the same @@ -1697,10 +1707,9 @@ const TESTCASES: &[metal::Arm] = &[metal::once( /// `osbench`'s groups, each a job named by its symlink, in `osbench all`'s /// order: `boot` first, which reads how long the boot took to reach it, and /// `crunch` last, which heats the machine. -const OSBENCH_GROUPS: [&str; 17] = [ +const OSBENCH_GROUPS: [&str; 16] = [ "osbench_boot", "osbench_identity", - "osbench_energy", "osbench_system", "osbench_sched", "osbench_fair", @@ -1732,6 +1741,14 @@ const BENCHCASE: &[metal::Arm] = &[metal::Arm { ..metal::once("benchcase", "tests/images/benchcase.toml", &[], &OSBENCH_GROUPS) }]; +/// `energy` waits up to 120 s for the machine to go quiet, then measures 60 s +/// idle and three 30 s loads, settling the clock before and after each spin: +/// under 160 s more. 300 s leaves its quiet wait 20 s over its own bound. +const OSBENCH_ENERGY: &[metal::Arm] = &[metal::Arm { + adds_ms: 300_000, + ..metal::once("energybenchcase", "tests/images/benchcase.toml", &[], &["osbench_energy"]) +}]; + /// The two groups that need another party share one boot when one pass names /// both. Each adds its own worst span; the 60 s every list has is the /// boot's margin, not either group's. @@ -1820,6 +1837,13 @@ fn since_boot(line: &str) -> Option { line.strip_prefix('[')?.split_whitespace().nth(2)?.parse().ok() } +/// `osbench energy` ran in ACPI mode, and its idle row measured the package. +fn osbench_energy_measured(boot: &metal::Readback) -> Result<(), String> { + osbench_ran(boot, &["osbench_energy"])?; + boot.log().must_say("bench=energy.idle.package unit=mW ")?; + Ok(()) +} + /// A timing boot was in ACPI mode before its first job and stayed there, as a /// shipped image and the bench's Linux are. In legacy mode the T14's firmware /// stops every CPU every 2.2 s, and its time would be measured as ToyOS's @@ -2840,11 +2864,12 @@ fn acpi_press_is_one_sci() -> Result<(), String> { Ok(()) } -/// `osbench identity` and `osbench fair` on tests/images/benchcase.toml with -/// only those two jobs: each exits 0, which it does only once every row it -/// ran measured or said why not, its capability rows among them. +/// `osbench identity`, `osbench fair` and `osbench energy` on +/// tests/images/benchcase.toml with only those jobs: each exits 0, which it +/// does only once every row it ran measured or said why not, its capability +/// rows among them. A guest reads no package energy, so `energy` says why. fn osbench_capability_groups_exit_0() -> Result<(), String> { - const JOBS: [&str; 2] = ["osbench_identity", "osbench_fair"]; + const JOBS: [&str; 3] = ["osbench_identity", "osbench_fair", "osbench_energy"]; let committed = compile::repo_root().join("tests/images/benchcase.toml"); let text = fs::read_to_string(&committed).map_err(|e| format!("{}: {e}", committed.display()))?; let derived = toyos_metal::image::derive(&text, 300_000, &JOBS, &[]).map_err(|why| why.to_string())?; @@ -2858,7 +2883,7 @@ fn osbench_capability_groups_exit_0() -> Result<(), String> { ); let mut console = format!("{}\n", qemu.boot_log()); let last = format!("===TEST_END {} ", JOBS[JOBS.len() - 1]); - await_guest(&mut qemu, &mut console, "osbench's two groups to end", |said| said.contains(&last))?; + await_guest(&mut qemu, &mut console, "osbench's groups to end", |said| said.contains(&last))?; let said = serial::Serial::named("osbench boot", console); for job in JOBS { said.must_say(&format!("===TEST_END {job} exit=0==="))?; @@ -6660,7 +6685,8 @@ const SMI_SPAN_NS: u64 = 4_444_000_000; /// /// Read and not held: each CPU's idle busy fraction, which an SMI in the idle /// second raises on every CPU alike by the time it held them, its spinning -/// clock, what one round cost its reader, and how late each CPU's kick handler +/// clock, the package's power across the idle second and the spin, what one +/// round cost its reader, and how late each CPU's kick handler /// ran under the `loaded` phase, which /// `issues/toyos-beats-linuxs-latency-on-the-t14.md` sets its bar for. fn counters_on_metal(back: &metal::Readback) -> Result<(), String> { @@ -6719,7 +6745,9 @@ fn counters_on_metal(back: &metal::Readback) -> Result<(), String> { } for (cpu, counters) in read { let reads = kernel.must_say(&format!("counters: cpu{cpu} reads "))?; - for counter in ["smi", "aperf", "mperf", "hwp_request", "hwp_request_pkg", "energy_perf_bias"] { + for counter in + ["smi", "aperf", "mperf", "hwp_request", "hwp_request_pkg", "energy_perf_bias", "package_energy", "core_energy"] + { if !reads.contains(&format!("{counter}={}", counters.contains_key(counter))) { return Err(format!("{name}: cpu{cpu} carries {counters:?} and its bring-up said {reads:?}")); } @@ -6742,7 +6770,8 @@ fn counters_on_metal(back: &metal::Readback) -> Result<(), String> { } } for cpu in 0..cpus { - for (name, &value) in &idle1[&cpu] { + // The energy counters are 32 bits wide and wrap (`toyos_abi::counters`). + for (name, &value) in idle1[&cpu].iter().filter(|(name, _)| !name.ends_with("_energy")) { if idle0[&cpu][name] > value || value > spin[&cpu][name] { return Err(format!("cpu{cpu}'s {name} went backwards: {idle0:?} {idle1:?} {spin:?}")); } @@ -6810,6 +6839,18 @@ fn counters_on_metal(back: &metal::Readback) -> Result<(), String> { ); } eprintln!(" [counters] the spin's read, a whole round, took its reader {} ns", clock["spin"].1); + if let (Some(&unit), Some(_)) = (idle0[&0].get("rapl_power_unit"), idle0[&0].get("package_energy")) { + let mw = |a: &Read, b: &Read, from: u64, to: u64| { + let steps = (b[&0]["package_energy"].wrapping_sub(a[&0]["package_energy"])) & 0xffff_ffff; + let uj = (u128::from(steps) * 1_000_000) >> ((unit >> 8) & 0x1f); + uj * 1_000_000 / u128::from((to - from).max(1)) + }; + eprintln!( + " [counters] the package drew {} mW across the idle second and {} mW across the spin (a reading)", + mw(idle0, idle1, at0, at1), + mw(idle1, spin, at1, at2), + ); + } if aperf_per_mperf.iter().any(|&r| r < 1.0) { return Err(format!("APERF ran {aperf_per_mperf:.3?} of MPERF across the spin, some cpu's busy clock below the rate MPERF counts at")); } From c523c9471e02d8fe4747010f4979fe34b385cb4d Mon Sep 17 00:00:00 2001 From: japabu Date: Wed, 28 Oct 2026 22:35:16 +0100 Subject: [PATCH 2/6] osbench energy: Linux reads every CPU, and one tick of /proc/stat is quiet Round 1 on both benches found two defects in the Linux half. The group pinned its main thread to CPU 0 before it counted the CPUs, and available_parallelism honours the mask, so `spin.all` ran one thread. Its workers' unpin restored the pinned mask too, and the per-CPU reads read CPU 0 alone. The package is now read on the reader's own CPU, the package's registers being the same on every CPU of it, and nothing is pinned. Linux's /proc/stat counts busy time in ticks of 10 ms, so a second with one tick in it never passed a bar of strictly under 10 ms: the AMD bench's Linux waited out the 120 s bound. A quiet second is now at most one tick. Each load also notes its per-second power in order (`.series`): the T14's ToyOS idle minute read 5.1 to 18.1 W while its later 10 ms sleeper read 2.9 W at the same PC2 residency, and the order says when. Co-Authored-By: Claude Opus 5.5 --- tests/guest/osbench/rules/src/lib.rs | 10 +++++----- tests/guest/osbench/src/energy.rs | 16 +++++----------- tests/guest/osbench/src/sys/linux.rs | 13 +++++++------ 3 files changed, 17 insertions(+), 22 deletions(-) diff --git a/tests/guest/osbench/rules/src/lib.rs b/tests/guest/osbench/rules/src/lib.rs index fa7e652d490..0b425e60000 100644 --- a/tests/guest/osbench/rules/src/lib.rs +++ b/tests/guest/osbench/rules/src/lib.rs @@ -240,7 +240,7 @@ pub mod energy { } /// The most CPU time the machine may spend in a second for that second to - /// be quiet: a hundredth of one CPU. + /// be quiet: a hundredth of one CPU, one tick of Linux's `/proc/stat`. pub const QUIET_NS: u64 = 10_000_000; /// The quiet seconds in a row an idle reading waits for. pub const QUIET_SECONDS: usize = 3; @@ -248,7 +248,7 @@ pub mod energy { /// Whether the last [`QUIET_SECONDS`] of `busy_ns`, each a second's CPU /// time, were each quiet. pub fn quiet(busy_ns: &[u64]) -> bool { - busy_ns.len() >= QUIET_SECONDS && busy_ns[busy_ns.len() - QUIET_SECONDS..].iter().all(|&ns| ns < QUIET_NS) + busy_ns.len() >= QUIET_SECONDS && busy_ns[busy_ns.len() - QUIET_SECONDS..].iter().all(|&ns| ns <= QUIET_NS) } #[cfg(test)] @@ -295,9 +295,9 @@ pub mod energy { #[test] fn quiet_is_every_one_of_the_last_seconds_under_the_bar() { assert!(!quiet(&[0, 0]), "too few seconds"); - assert!(quiet(&[50_000_000, 0, 9_999_999, 0])); - assert!(!quiet(&[0, 0, QUIET_NS, 0, 0])); - assert!(quiet(&[0, 0, QUIET_NS, 0, 0, 0])); + assert!(quiet(&[50_000_000, 0, QUIET_NS, 0]), "one tick of /proc/stat is quiet"); + assert!(!quiet(&[0, 0, QUIET_NS + 1, 0, 0])); + assert!(quiet(&[0, 0, QUIET_NS + 1, 0, 0, 0])); } } } diff --git a/tests/guest/osbench/src/energy.rs b/tests/guest/osbench/src/energy.rs index c682032f04e..63282edda01 100644 --- a/tests/guest/osbench/src/energy.rs +++ b/tests/guest/osbench/src/energy.rs @@ -12,8 +12,8 @@ //! Then each load runs for its span, and the package is read once a second: //! each row is that second's power, its residency and, under a spin, its //! energy per [`crate::chunk`]. Each CPU's counters are read at the load's -//! two ends, so Linux sends no IPI inside a second; ToyOS's every read kicks -//! every CPU. No figure is better higher or lower by itself: a load that ran +//! two ends, so Linux reads the package on the reader's own CPU inside a +//! second; ToyOS's every read kicks every CPU. No figure is better higher or lower by itself: a load that ran //! more work may draw more. use std::sync::atomic::{AtomicBool, AtomicU64, Ordering::Relaxed}; @@ -22,8 +22,6 @@ use std::time::{Duration, Instant}; use osbench_rules::energy::{self, RANGE}; use crate::report::{note, result}; -#[cfg(target_os = "linux")] -use crate::sys; use crate::sys::{power, CpuPower, Package, Refused}; /// The longest the group waits for the machine to go quiet. @@ -43,11 +41,7 @@ enum Load { pub fn suite() { #[cfg(target_os = "linux")] - { - power::conditions(); - // Its own CPU is the one whose registers the package is read from. - sys::pin_self(0); - } + power::conditions(); let loads = [ ("energy.idle", 60, Load::Idle), ("energy.wake_10ms", 30, Load::Wake(Duration::from_millis(10))), @@ -123,8 +117,6 @@ fn row(bench: &str, seconds: u64, load: Load) -> Result<(), Refused> { for slot in &counts { let stop = &stop; s.spawn(move || { - #[cfg(target_os = "linux")] - sys::unpin(sys::tid()); while !stop.load(Relaxed) { match load { Load::Wake(period) => std::thread::sleep(period), @@ -156,6 +148,8 @@ fn row(bench: &str, seconds: u64, load: Load) -> Result<(), Refused> { if let Some(mw) = unit.and_then(|unit| cores_mw(&first, &last, unit, samples.span_ns)) { fields.push(("cores_mw", mw.to_string())); } + let series: Vec = samples.mw.iter().map(u64::to_string).collect(); + note(&format!("{bench}.series"), &[("mw", series.join(","))]); result(&format!("{bench}.package"), "mW", &mut samples.mw, &fields); for (state, permille) in STATES.iter().zip(&mut samples.residency) { if !permille.is_empty() { diff --git a/tests/guest/osbench/src/sys/linux.rs b/tests/guest/osbench/src/sys/linux.rs index 2d7d873c20c..04a5048910c 100644 --- a/tests/guest/osbench/src/sys/linux.rs +++ b/tests/guest/osbench/src/sys/linux.rs @@ -289,8 +289,8 @@ pub struct PowerRegisters { } /// The power counters through the `msr` driver's `/dev/cpu//msr`, which -/// takes root: the package's on CPU 0, which reads them locally where the -/// caller runs there, and each CPU's on that CPU, an IPI each. +/// takes root: the package's on the caller's own CPU, read there without an +/// IPI unless the scheduler moved it, and each CPU's on that CPU, an IPI each. pub mod power { use std::fs::File; use std::os::unix::fs::FileExt; @@ -336,13 +336,14 @@ pub mod power { pub fn package() -> Result { let r = registers()?; - let dev = device(0)?; - let tsc = msr(&dev, 0, IA32_TIME_STAMP_COUNTER)?.ok_or_else(|| Refused::Failed("cpu0 read no TSC".into()))?; + let cpu = super::current_cpu(); + let dev = device(cpu)?; + let tsc = msr(&dev, cpu, IA32_TIME_STAMP_COUNTER)?.ok_or_else(|| Refused::Failed(format!("cpu{cpu} read no TSC")))?; let mut residency = [None; 4]; for (slot, at) in residency.iter_mut().zip(r.package_residency) { - *slot = each(&dev, 0, at)?; + *slot = each(&dev, cpu, at)?; } - Ok(Package { tsc, energy: msr(&dev, 0, r.energy)?, power_unit: msr(&dev, 0, r.power_unit)?, residency }) + Ok(Package { tsc, energy: msr(&dev, cpu, r.energy)?, power_unit: msr(&dev, cpu, r.power_unit)?, residency }) } pub fn cpus() -> Result, Refused> { From 31be41287be337daa5465508847ba1c461df1f7b Mon Sep 17 00:00:00 2001 From: japabu Date: Sat, 31 Oct 2026 17:34:15 +0100 Subject: [PATCH 3/6] issues: the T14's idle package stays in PC2 The scoreboard's T14 rows put ToyOS's idle package at 7137 mW against Linux's 1648 mW, with the package in PC2 994 permille of the minute. Phase 2's scouts (evidence pr-1100 r5) ruled out the nine driverless functions, the deepest core state, and the cores and graphics planes as the consumer; the display under the firmware's GOP, the NVMe without APST and the I219 remain. Recorded as a defect held by the power task. Co-Authored-By: Claude Opus 5.5 --- issues/the-t14s-idle-package-stays-in-pc2.md | 45 ++++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100644 issues/the-t14s-idle-package-stays-in-pc2.md diff --git a/issues/the-t14s-idle-package-stays-in-pc2.md b/issues/the-t14s-idle-package-stays-in-pc2.md new file mode 100644 index 00000000000..5c2735723e2 --- /dev/null +++ b/issues/the-t14s-idle-package-stays-in-pc2.md @@ -0,0 +1,45 @@ +--- +status: assigned +kind: defect +opened: 2026-10-31 +--- + +# The T14's idle package stays in PC2 + +Held by the power task that follows #1100. + +On the T14, `osbench_energy` gives an idle package of 7137 mW under ToyOS +against 1648 mW under Linux, 4.3×. Under a 10 ms sleeper it is 2863 against +1219 mW. ToyOS's package spends 994 ‰ of the idle minute in PC2 and none +deeper, against Linux's 102 ‰ in PC2. Every ToyOS core waits in C7. Power +belongs in the idle states and in device power management: none of this gap +may be closed by slowing work. The AMD bench is at 1.15×. + +Intel's datasheet (631121 Vol. 1 §3.2.5, Table 3-5) holds the package in PC2 +when its cores are in C6 or deeper and its graphics in RC6, but an LTR, a +near timer or a device memory access refuses any deeper state. Table 3-8 +caps the package at PC3 while a processor PCIe link is in L0, and at PC8 +until it reaches L1.2. + +Ruled out, by measurement: + +- **The driverless functions.** Parking all nine in D3hot changed nothing + under ToyOS, and unbinding them under Linux cost Linux nothing. +- **The deepest core state.** Holding every core at an 800 µs wake latency + is inside the idle seconds' scatter. +- **The cores and graphics as consumers.** PP0 is about 0.3 W and PP1 under + 0.03 W, so about 6.8 W lies outside both planes. On Linux the equivalent + is about 1.5 W. + +Still to separate: + +- **the display engine** scanning out under the firmware's GOP programming, + with no DMC firmware. Intel's Tiger Lake display reference (Vol. 12, "Display + C5 and C6") requires DMC for DC5 and DC6. +- **the NVMe** in its operational state, with no APST + (`issues/diskserver-drives-nvme-apst.md`), and its link's power state. +- **the I219** under netstack, through its LTR. + +**Exit**: the T14's `energy.idle.package` row is unresolved against Linux's +in `osbench_energy`, or the remaining gap is a cause named and recorded at its +owner. From 55922d7a9743ed05587c75f97e808344810c1045 Mon Sep 17 00:00:00 2001 From: japabu Date: Sat, 31 Oct 2026 18:17:32 +0100 Subject: [PATCH 4/6] review r1: both systems read every CPU each second, and the AMD gap is filed - osbench energy reads one round a second on both systems, the package's counters and every CPU's (`power::round`): ToyOS's SYS_COUNTERS round kicks each CPU, and now Linux's reads each CPU's MSRs, an IPI to each but its own, so the two methods cost alike. A row's per-CPU notes come off its first and last rounds. - The README and the module header say that, and the quiet bar is "at most 10 ms"; the quiet gate measures CPU time alone, the core C6/C7 notes say whether the cores reached their deepest state. - Linux's power registers come from `crate::arch`, AArch64's answering none; AMD's only on Family 17h with RAPL, as the kernel admits them. - `Rights::TRACE` names the package and core energy it grants. - The `counters` judge decodes its reading by `osbench_rules::energy`. - issues/the-amd-benchs-idle-package-draws-15-percent-over-linuxs.md files the AMD bench's resolved idle deficit; the T14 issue says which statistic its figures are. Co-Authored-By: Claude Opus 5.5 --- Cargo.lock | 1 + Cargo.toml | 3 ++ ...le-package-draws-15-percent-over-linuxs.md | 29 +++++++++++++++ issues/the-t14s-idle-package-stays-in-pc2.md | 7 +++- sdk/abi/src/handle.rs | 6 ++- tests/guest/osbench/README.md | 13 ++++--- tests/guest/osbench/src/arch/aarch64.rs | 9 ++++- tests/guest/osbench/src/arch/x86_64.rs | 22 ++++++++--- tests/guest/osbench/src/energy.rs | 37 +++++++++++-------- tests/guest/osbench/src/sys/linux.rs | 23 ++++++------ tests/guest/osbench/src/sys/mod.rs | 7 ++++ tests/guest/osbench/src/sys/toyos.rs | 18 +++++---- tests/toyos.rs | 7 ++-- 13 files changed, 128 insertions(+), 54 deletions(-) create mode 100644 issues/the-amd-benchs-idle-package-draws-15-percent-over-linuxs.md diff --git a/Cargo.lock b/Cargo.lock index f6abd5dccfc..43b64db3cda 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -3580,6 +3580,7 @@ dependencies = [ "kernel", "libc", "libm", + "osbench-rules", "pem", "rustls", "serde", diff --git a/Cargo.toml b/Cargo.toml index 7026709d242..87f423b3fa2 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -319,6 +319,9 @@ toyos-metal = { path = "tests/metal" } toyos-quiesce = { path = "lib/quiesce" } # The parity table `crypto_parity` prints and the judge reads, under one rule. toyos-parity = { path = "lib/parity" } +# The package energy counter's decode, so the `counters` judge's reading of +# it is osbench's rule and not a second one. +osbench-rules = { path = "tests/guest/osbench/rules" } [patch.crates-io] loom = { git = "https://github.com/ToyOSOrg/loom", branch = "toyos" } diff --git a/issues/the-amd-benchs-idle-package-draws-15-percent-over-linuxs.md b/issues/the-amd-benchs-idle-package-draws-15-percent-over-linuxs.md new file mode 100644 index 00000000000..2e309f7cd3f --- /dev/null +++ b/issues/the-amd-benchs-idle-package-draws-15-percent-over-linuxs.md @@ -0,0 +1,29 @@ +--- +status: assigned +kind: defect +opened: 2026-10-31 +--- + +# The AMD bench's idle package draws 15 % over Linux's + +Held by the power task that follows #1100. + +On the AMD bench (Family 17h Model 60h), `osbench_energy` resolves the idle +package against ToyOS. Over four runs a side, every ToyOS run lies above +every Linux run (p = 0.029): + +- The median of the runs' per-second p50s is 1688 mW against 1472 mW. +- The median of the runs' means is 1688 mW against 1501 mW. + +The cores are not the consumer. Their own energy counters give ToyOS 0 mW +over the idle minute and Linux 11 mW, so the gap of about 200 mW lies +outside the cores: in the SoC, memory, I/O or devices. Its 10 ms sleeper, +1700-1710 against 1391-1705 mW, is not resolved. Work may not be slowed to +close the gap. + +The bench documents no package C-state residency (PPR 55922), so ToyOS's +rows cannot show which state the package reaches. + +**Exit**: the AMD bench's `energy.idle.package` row is unresolved against +Linux's in `osbench_energy`, or the remaining gap is a cause named and +recorded at its owner. diff --git a/issues/the-t14s-idle-package-stays-in-pc2.md b/issues/the-t14s-idle-package-stays-in-pc2.md index 5c2735723e2..a220a951cad 100644 --- a/issues/the-t14s-idle-package-stays-in-pc2.md +++ b/issues/the-t14s-idle-package-stays-in-pc2.md @@ -9,11 +9,14 @@ opened: 2026-10-31 Held by the power task that follows #1100. On the T14, `osbench_energy` gives an idle package of 7137 mW under ToyOS -against 1648 mW under Linux, 4.3×. Under a 10 ms sleeper it is 2863 against +against 1648 mW under Linux, 4.3×: the median, over four runs a side, of +each run's per-second p50. The medians of the runs' means are 8272 against +2167 mW, 3.8×. Under a 10 ms sleeper it is 2863 against 1219 mW. ToyOS's package spends 994 ‰ of the idle minute in PC2 and none deeper, against Linux's 102 ‰ in PC2. Every ToyOS core waits in C7. Power belongs in the idle states and in device power management: none of this gap -may be closed by slowing work. The AMD bench is at 1.15×. +may be closed by slowing work. The AMD bench's smaller gap is +`issues/the-amd-benchs-idle-package-draws-15-percent-over-linuxs.md`. Intel's datasheet (631121 Vol. 1 §3.2.5, Table 3-5) holds the package in PC2 when its cores are in C6 or deeper and its graphics in RC6, but an LTR, a diff --git a/sdk/abi/src/handle.rs b/sdk/abi/src/handle.rs index 86cfb1978a8..70eb761896e 100644 --- a/sdk/abi/src/handle.rs +++ b/sdk/abi/src/handle.rs @@ -227,8 +227,10 @@ impl Rights { /// /// Beside [`COUNTERS`](Self::COUNTERS), [`SYS_COUNTERS`] answers each /// CPU's `APERF` and `MPERF` — its frequency and busy fraction, the - /// Hertzbleed power channel — the kicks it took, the wake-ups that time - /// keystrokes, and the power envelope it runs under. [`SYS_TRACE_READ`] + /// Hertzbleed power channel — the package's energy and C-state + /// residency and each AMD core's energy, the RAPL power channel + /// (CVE-2020-8694), the kicks it took, the wake-ups that time keystrokes, + /// and the power envelope it runs under. [`SYS_TRACE_READ`] /// answers, on this alone, when every thread in the machine was woken, ran /// and parked, and on which CPU. Admin tools hold it, `inspect`, `trace` /// and `test-runner`; no diff --git a/tests/guest/osbench/README.md b/tests/guest/osbench/README.md index def2ba8d04b..6b488884ead 100644 --- a/tests/guest/osbench/README.md +++ b/tests/guest/osbench/README.md @@ -155,9 +155,10 @@ Both read the package's energy counter in its own unit, its C2, C3, C6 and C7 residency where Intel documents them (not PC8 to PC10 on the T14, which SDM Vol. 4 gives no Tiger Lake), and each CPU's APERF/MPERF and core residency, and on AMD each core's energy (`cores_mw`). The group first waits, -up to 120 s, for three seconds in a row in which the machine ran under 10 ms -of CPU time (`energy.quiet`): a ToyOS boot reaches it while its services, -the ACPI server's idle states among them, are still starting. +up to 120 s, for three seconds in a row in which the machine ran at most +10 ms of CPU time (`energy.quiet`): a ToyOS boot reaches it while its +services are still starting. The wait measures CPU time alone; each row's +core C6/C7 note says whether the cores had reached their deepest idle state. The rows: `energy.idle` (60 s), `energy.wake_10ms` (one thread sleeping 10 ms over and over, 30 s), `energy.spin.t1` and `energy.spin.all` (one thread and @@ -165,9 +166,9 @@ one per CPU running the compute chunk, 30 s each), each a second's power (`.package`, mW), residency (`.pc2` to `.pc7`, permille) and under a spin a chunk's energy (`.per_chunk`, nJ). No unit is better higher or lower by itself: a spin that ran faster may draw more, and work is not slowed for -power. ToyOS's every read kicks every CPU once a second; Linux's reads the -package on CPU 0, where the reader is pinned, and each CPU only at a load's -two ends. +power. Each second both systems read the package and every CPU's counters +in one round, which reaches every CPU: ToyOS's kicks each, and Linux's `msr` +driver interrupts each but the reader's own. **The same devices, on both**: diff --git a/tests/guest/osbench/src/arch/aarch64.rs b/tests/guest/osbench/src/arch/aarch64.rs index 898274f513e..4edd8f4095c 100644 --- a/tests/guest/osbench/src/arch/aarch64.rs +++ b/tests/guest/osbench/src/arch/aarch64.rs @@ -1,6 +1,11 @@ //! What osbench needs written for AArch64 in particular, on Linux: a chain of -//! dependent adds to time the clock with, and the bytes of a program that -//! exits. ToyOS on AArch64 is refused by the selector. +//! dependent adds to time the clock with, the bytes of a program that exits, +//! and no power registers. ToyOS on AArch64 is refused by the selector. + +/// The energy group reads no AArch64 power counter. +pub fn power_registers() -> Option { + None +} /// `turns` turns of 100 dependent register adds, each waiting on the one /// before it. Inline assembly, so the chain is the one written whatever the diff --git a/tests/guest/osbench/src/arch/x86_64.rs b/tests/guest/osbench/src/arch/x86_64.rs index 884bca9066c..0fa5a20715a 100644 --- a/tests/guest/osbench/src/arch/x86_64.rs +++ b/tests/guest/osbench/src/arch/x86_64.rs @@ -70,10 +70,12 @@ mod cpuid { use core::arch::x86_64::__cpuid_count; use std::sync::OnceLock; - /// The registers the energy group reads on Linux, the ones ToyOS's kernel - /// reads for the same counters (`kernel/pure/cpu.rs`), by CPUID.0's - /// vendor: Intel's from SDM Vol. 4 Table 2-20, AMD's from PPR 55922. A - /// register a CPU lacks reads as absent. + /// The registers the energy group reads on Linux for the counters ToyOS's + /// kernel reads (`kernel/pure/cpu.rs`), by CPUID.0's vendor: Intel's from + /// SDM Vol. 4 Table 2-20, AMD's from PPR 55922 on Family 17h with + /// `CPUID_Fn80000007_EDX[RAPL]`, where they are 32-bit counters; another + /// AMD family is refused, as the kernel refuses it. A register a CPU + /// lacks reads as absent. #[cfg(target_os = "linux")] pub fn power_registers() -> Option { let id = __cpuid_count(0, 0); @@ -86,7 +88,7 @@ mod cpuid { core_residency: [Some(0x3fd), Some(0x3fe)], core_energy: None, }), - b"AuthenticAMD" => Some(crate::sys::PowerRegisters { + b"AuthenticAMD" if amd_family_17h_rapl() => Some(crate::sys::PowerRegisters { energy: 0xc001_029b, power_unit: 0xc001_0299, package_residency: [None; 4], @@ -97,6 +99,16 @@ mod cpuid { } } + /// Family 17h (CPUID.1:EAX base family 0FH plus extended family 08H) with + /// RAPL, bit 14 of CPUID_Fn80000007_EDX (PPR 55922). + #[cfg(target_os = "linux")] + fn amd_family_17h_rapl() -> bool { + let eax = __cpuid_count(1, 0).eax; + let family = ((eax >> 8) & 0xf) + ((eax >> 20) & 0xff); + let rapl = __cpuid_count(0x8000_0000, 0).eax >= 0x8000_0007 && __cpuid_count(0x8000_0007, 0).edx & (1 << 14) != 0; + family == 0x17 && rapl + } + struct Topology { /// CPUID leaf 0BH enumerates topology (SDM Vol. 2A: a subleaf 0 whose /// EBX[15:0] is 0 enumerates nothing). diff --git a/tests/guest/osbench/src/energy.rs b/tests/guest/osbench/src/energy.rs index 63282edda01..08f0db1d560 100644 --- a/tests/guest/osbench/src/energy.rs +++ b/tests/guest/osbench/src/energy.rs @@ -7,14 +7,15 @@ //! //! **First the machine is let go quiet** ([`osbench_rules::energy::quiet`]), //! for at most [`QUIET_BOUND`]: a ToyOS boot starts the group while its -//! services are still starting, the ACPI server's idle states among them. +//! services are still starting. The gate measures CPU time alone; whether +//! the cores had reached their deepest idle state is each row's core C6/C7 +//! note. //! -//! Then each load runs for its span, and the package is read once a second: -//! each row is that second's power, its residency and, under a spin, its -//! energy per [`crate::chunk`]. Each CPU's counters are read at the load's -//! two ends, so Linux reads the package on the reader's own CPU inside a -//! second; ToyOS's every read kicks every CPU. No figure is better higher or lower by itself: a load that ran -//! more work may draw more. +//! Then each load runs for its span, read once a second by a round of every +//! CPU's counters and the package's (`power::round`), which reaches every CPU +//! on both systems: each row is that second's power, its residency and, +//! under a spin, its energy per [`crate::chunk`]. No figure is better higher +//! or lower by itself: a load that ran more work may draw more. use std::sync::atomic::{AtomicBool, AtomicU64, Ordering::Relaxed}; use std::time::{Duration, Instant}; @@ -22,12 +23,12 @@ use std::time::{Duration, Instant}; use osbench_rules::energy::{self, RANGE}; use crate::report::{note, result}; -use crate::sys::{power, CpuPower, Package, Refused}; +use crate::sys::{power, CpuPower, Refused, Round}; /// The longest the group waits for the machine to go quiet. const QUIET_BOUND: Duration = Duration::from_secs(120); -/// The residency rows, in [`Package::residency`]'s order. +/// The residency rows, in [`crate::sys::Package::residency`]'s order. const STATES: [&str; 4] = ["pc2", "pc3", "pc6", "pc7"]; #[derive(Clone, Copy)] @@ -48,7 +49,7 @@ pub fn suite() { ("energy.spin.t1", 30, Load::Spin(1)), ("energy.spin.all", 30, Load::Spin(crate::cpus())), ]; - if let Err(why) = power::package() { + if let Err(why) = power::round() { for (bench, ..) in loads { why.clone().report(&format!("{bench}.package")); } @@ -111,7 +112,6 @@ fn row(bench: &str, seconds: u64, load: Load) -> Result<(), Refused> { }; let stop = AtomicBool::new(false); let counts: Vec = (0..threads).map(|_| Slot::default()).collect(); - let first = power::cpus()?; let mut samples = Samples::default(); let sampled = std::thread::scope(|s| { for slot in &counts { @@ -133,7 +133,7 @@ fn row(bench: &str, seconds: u64, load: Load) -> Result<(), Refused> { sampled }); sampled?; - let last = power::cpus()?; + let (first, last) = (&samples.first_cpus, &samples.last_cpus); let unit = samples.power_unit; let mut fields = vec![("seconds", seconds.to_string())]; let done: u64 = counts.iter().map(|c| c.0.load(Relaxed)).sum(); @@ -145,7 +145,7 @@ fn row(bench: &str, seconds: u64, load: Load) -> Result<(), Refused> { fields.push(("chunks_per_s", (done / seconds.max(1)).to_string())); } } - if let Some(mw) = unit.and_then(|unit| cores_mw(&first, &last, unit, samples.span_ns)) { + if let Some(mw) = unit.and_then(|unit| cores_mw(first, last, unit, samples.span_ns)) { fields.push(("cores_mw", mw.to_string())); } let series: Vec = samples.mw.iter().map(u64::to_string).collect(); @@ -159,7 +159,7 @@ fn row(bench: &str, seconds: u64, load: Load) -> Result<(), Refused> { if !samples.nj_per_chunk.is_empty() { result(&format!("{bench}.per_chunk"), "nJ", &mut samples.nj_per_chunk, &[]); } - cpus(bench, &first, &last); + cpus(bench, first, last); Ok(()) } @@ -170,19 +170,24 @@ struct Samples { nj_per_chunk: Vec, power_unit: Option, span_ns: u64, + /// Each CPU's counters at the row's first and last rounds. + first_cpus: Vec, + last_cpus: Vec, } /// A reading of the package each second for `seconds`, into `out`, with the /// energy per chunk where `per_chunk`. fn sample(seconds: u64, counts: &[Slot], per_chunk: bool, out: &mut Samples) -> Result<(), Refused> { let chunks = || counts.iter().map(|c| c.0.load(Relaxed)).sum::(); - let mut before: Package = power::package()?; + let Round { package: mut before, cpus } = power::round()?; + out.first_cpus = cpus; let (start, mut at, mut done) = (Instant::now(), Instant::now(), chunks()); out.power_unit = before.power_unit; for second in 1..=seconds { // The span is the measurement: each read is due a second after the start, not after the last read. std::thread::sleep((start + Duration::from_secs(second)).saturating_duration_since(Instant::now())); - let after = power::package()?; + let Round { package: after, cpus } = power::round()?; + out.last_cpus = cpus; let (now, done_now) = (Instant::now(), chunks()); let ns = u64::try_from((now - at).as_nanos()).expect("a second under 584 years"); if let (Some(e0), Some(e1), Some(unit)) = (before.energy, after.energy, after.power_unit) { diff --git a/tests/guest/osbench/src/sys/linux.rs b/tests/guest/osbench/src/sys/linux.rs index 04a5048910c..2297415bf23 100644 --- a/tests/guest/osbench/src/sys/linux.rs +++ b/tests/guest/osbench/src/sys/linux.rs @@ -289,13 +289,13 @@ pub struct PowerRegisters { } /// The power counters through the `msr` driver's `/dev/cpu//msr`, which -/// takes root: the package's on the caller's own CPU, read there without an -/// IPI unless the scheduler moved it, and each CPU's on that CPU, an IPI each. +/// takes root: the package's on whichever CPU the caller runs on, and each +/// CPU's on that CPU, an IPI to each but the caller's. pub mod power { use std::fs::File; use std::os::unix::fs::FileExt; - use super::super::{CpuPower, Package, Refused}; + use super::super::{CpuPower, Package, Refused, Round}; use super::PowerRegisters; const IA32_TIME_STAMP_COUNTER: u32 = 0x10; @@ -303,11 +303,7 @@ pub mod power { const IA32_APERF: u32 = 0xe8; fn registers() -> Result { - #[cfg(target_arch = "x86_64")] - let registers = crate::arch::power_registers(); - #[cfg(not(target_arch = "x86_64"))] - let registers = None; - registers.ok_or_else(|| Refused::Absent("no CPU vendor whose power registers osbench reads".into())) + crate::arch::power_registers().ok_or_else(|| Refused::Absent("no CPU vendor whose power registers osbench reads".into())) } fn device(cpu: u32) -> Result { @@ -334,8 +330,14 @@ pub mod power { at.map_or(Ok(None), |at| msr(dev, cpu, at)) } - pub fn package() -> Result { + /// The package's counters and then every CPU's, as ToyOS's round reads + /// them: a read of every CPU each time. + pub fn round() -> Result { let r = registers()?; + Ok(Round { package: package(&r)?, cpus: cpus(&r)? }) + } + + fn package(r: &PowerRegisters) -> Result { let cpu = super::current_cpu(); let dev = device(cpu)?; let tsc = msr(&dev, cpu, IA32_TIME_STAMP_COUNTER)?.ok_or_else(|| Refused::Failed(format!("cpu{cpu} read no TSC")))?; @@ -346,8 +348,7 @@ pub mod power { Ok(Package { tsc, energy: msr(&dev, cpu, r.energy)?, power_unit: msr(&dev, cpu, r.power_unit)?, residency }) } - pub fn cpus() -> Result, Refused> { - let r = registers()?; + fn cpus(r: &PowerRegisters) -> Result, Refused> { super::allowed_cpus() .into_iter() .map(|cpu| { diff --git a/tests/guest/osbench/src/sys/mod.rs b/tests/guest/osbench/src/sys/mod.rs index 5861093b198..7f707c3d571 100644 --- a/tests/guest/osbench/src/sys/mod.rs +++ b/tests/guest/osbench/src/sys/mod.rs @@ -95,6 +95,13 @@ pub struct CpuPower { pub core_energy: Option, } +/// One reading of the power counters, the package's and every CPU's +/// together: on either system it reaches every CPU. +pub struct Round { + pub package: Package, + pub cpus: Vec, +} + /// A whole anonymous read-write mapping, unmapped on drop. pub struct Mapping { ptr: *mut u8, diff --git a/tests/guest/osbench/src/sys/toyos.rs b/tests/guest/osbench/src/sys/toyos.rs index e8ef328c6fc..860ae28ac1a 100644 --- a/tests/guest/osbench/src/sys/toyos.rs +++ b/tests/guest/osbench/src/sys/toyos.rs @@ -383,7 +383,7 @@ pub mod diag { /// under `Rights::TRACE`, the package's held by the boot processor. Every read /// kicks every CPU (`toyos_abi::counters`), the package's too. pub mod power { - use super::super::{CpuPower, Package, Refused}; + use super::super::{CpuPower, Package, Refused, Round}; use super::diag::{self, CpuCounts}; fn read() -> Result, Refused> { @@ -391,9 +391,14 @@ pub mod power { d.counters().map_err(Refused::Failed) } - pub fn package() -> Result { - let cpus = read()?; - let held = cpus.into_iter().find(|c| c.package_energy.is_some() || c.package_residency.iter().any(Option::is_some)); + /// One `SYS_COUNTERS` round, which kicks every CPU. + pub fn round() -> Result { + let read = read()?; + Ok(Round { package: package(&read)?, cpus: cpus(read)? }) + } + + fn package(cpus: &[CpuCounts]) -> Result { + let held = cpus.iter().find(|c| c.package_energy.is_some() || c.package_residency.iter().any(Option::is_some)); let Some(c) = held else { return Err(Refused::Absent( "no CPU's record carries the package's counters: the kernel's `counters: cpu0 reads` line says whether it \ @@ -412,9 +417,8 @@ pub mod power { }) } - pub fn cpus() -> Result, Refused> { - read()? - .into_iter() + fn cpus(read: Vec) -> Result, Refused> { + read.into_iter() .map(|c| { if c.stale { return Err(Refused::Failed(format!("cpu{} answered stale", c.cpu))); diff --git a/tests/toyos.rs b/tests/toyos.rs index d1339863334..699e8d77486 100644 --- a/tests/toyos.rs +++ b/tests/toyos.rs @@ -6840,10 +6840,11 @@ fn counters_on_metal(back: &metal::Readback) -> Result<(), String> { } eprintln!(" [counters] the spin's read, a whole round, took its reader {} ns", clock["spin"].1); if let (Some(&unit), Some(_)) = (idle0[&0].get("rapl_power_unit"), idle0[&0].get("package_energy")) { + use osbench_rules::energy; let mw = |a: &Read, b: &Read, from: u64, to: u64| { - let steps = (b[&0]["package_energy"].wrapping_sub(a[&0]["package_energy"])) & 0xffff_ffff; - let uj = (u128::from(steps) * 1_000_000) >> ((unit >> 8) & 0x1f); - uj * 1_000_000 / u128::from((to - from).max(1)) + energy::span(a[&0]["package_energy"], b[&0]["package_energy"], energy::RANGE) + .map(|steps| energy::milliwatts(energy::microjoules(steps, unit), to - from)) + .map_or("unreadable".to_string(), |mw| mw.to_string()) }; eprintln!( " [counters] the package drew {} mW across the idle second and {} mW across the spin (a reading)", From cf0fc945fc55b334171c831e123f0b505cbf8df4 Mon Sep 17 00:00:00 2001 From: japabu Date: Sat, 31 Oct 2026 18:44:01 +0100 Subject: [PATCH 5/6] issues: the T14's two causes, each reproduced on Linux Phase 3 (evidence pr-1100 r7): Linux with NVMe APST off rises from 1622 to 5127 mW idle and goes no deeper than PC3; Linux with no graphics driver stays in PC2 at 2796 mW, its 10 ms sleeper matching ToyOS's within 1 %; and on ToyOS, stopping the GOP scan-out's plane fetches releases PC2 for PC3. The I219 changed nothing on either system. APST, the larger share, comes first; the display's power programming is an owner decision. Co-Authored-By: Claude Opus 5.5 --- issues/the-t14s-idle-package-stays-in-pc2.md | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/issues/the-t14s-idle-package-stays-in-pc2.md b/issues/the-t14s-idle-package-stays-in-pc2.md index a220a951cad..618ad04d776 100644 --- a/issues/the-t14s-idle-package-stays-in-pc2.md +++ b/issues/the-t14s-idle-package-stays-in-pc2.md @@ -33,15 +33,21 @@ Ruled out, by measurement: - **The cores and graphics as consumers.** PP0 is about 0.3 W and PP1 under 0.03 W, so about 6.8 W lies outside both planes. On Linux the equivalent is about 1.5 W. +- **The I219.** Parking it under ToyOS, or unbinding it under Linux, + changed nothing. -Still to separate: +The two causes, each reproduced on Linux by putting ToyOS's state there: -- **the display engine** scanning out under the firmware's GOP programming, - with no DMC firmware. Intel's Tiger Lake display reference (Vol. 12, "Display - C5 and C6") requires DMC for DC5 and DC6. -- **the NVMe** in its operational state, with no APST - (`issues/diskserver-drives-nvme-apst.md`), and its link's power state. -- **the I219** under netstack, through its LTR. +- **The NVMe without APST** (`issues/diskserver-drives-nvme-apst.md`). + - With APST off, Linux's idle rises from 1622 to 5127 mW (per-second p50), and its package goes no deeper than PC3. + - The rise is large when nothing wakes and smaller under a 10 ms sleeper, which is ToyOS's own shape. + - The link's ASPM, L1 substates and LTR are as firmware left them, the same on both systems. +- **The display as the GOP left it.** + - With no graphics driver (`nomodeset`), Linux idles at 2796 mW and stays in PC2 976 ‰. Its 10 ms sleeper draws 2864 mW, against ToyOS's 2831. + - Under ToyOS, stopping the scan-out's plane fetches (Intel's PLANE_CTL Plane Enable) releases PC2 for PC3. + - Linux's driver reaches PC8 with the panel lit. The fix is ToyOS's own display programming, with the panel lit; turning it off is not the fix. Per Intel's Tiger Lake display reference (Vol. 12), that means watermarks, SAGV, DMC firmware for DC5/DC6, and PSR for PC9/PC10. A driver for it is an owner decision. + +APST is the larger share and comes first. **Exit**: the T14's `energy.idle.package` row is unresolved against Linux's in `osbench_energy`, or the remaining gap is a cause named and recorded at its From ed35ba93742345448c144483477850df452d7067 Mon Sep 17 00:00:00 2001 From: japabu Date: Sat, 31 Oct 2026 19:01:01 +0100 Subject: [PATCH 6/6] review r2: the AMD issue's runs predate the matched read, and the offline-judge selection defect is filed The AMD idle deficit was measured while Linux read only the package each second and ToyOS's read kicked every CPU, which inflates that row; the issue says so and gives the cores' range across the four runs (ToyOS 0-1 mW, Linux 11-20 mW). The harness's offline judge cannot take the `counters` row without `counters_silent` and so without a patch: filed as tooling. Co-Authored-By: Claude Opus 5.5 --- ...nters-metal-row-without-counters-silent.md | 24 +++++++++++++++++++ ...le-package-draws-15-percent-over-linuxs.md | 12 +++++++--- 2 files changed, 33 insertions(+), 3 deletions(-) create mode 100644 issues/no-filter-word-takes-the-counters-metal-row-without-counters-silent.md diff --git a/issues/no-filter-word-takes-the-counters-metal-row-without-counters-silent.md b/issues/no-filter-word-takes-the-counters-metal-row-without-counters-silent.md new file mode 100644 index 00000000000..8671df77c73 --- /dev/null +++ b/issues/no-filter-word-takes-the-counters-metal-row-without-counters-silent.md @@ -0,0 +1,24 @@ +--- +status: open +kind: tooling +opened: 2026-10-31 +--- + +# No filter word takes the counters metal row without counters_silent + +`tests/common/metal.rs`'s `select` takes every row and shared member whose +name contains a filter word. Every substring of `counters`, the row's whole +name, is also part of the shared member `counters_silent`, which rides +`shared-debug`. An offline run (`--metal --metal-readback counters`) +judges only once every selected boot has a readback. So judging the +`counters` row over a `testcases` readback alone stages `shared-debug` +instead, and nothing is judged. + +#1100's review had to drop `counters_silent` from the selection with a +temporary patch to judge round 1's T14 readback (evidence pr-1100 r8, +`judge-select.patch`). Any row whose name is part of another row's or +member's name has the same problem. + +**Exit**: a filter word can select one row alone, for example an exact-name +form, and the `counters` row is judged offline over a `testcases` readback +with no patch. diff --git a/issues/the-amd-benchs-idle-package-draws-15-percent-over-linuxs.md b/issues/the-amd-benchs-idle-package-draws-15-percent-over-linuxs.md index 2e309f7cd3f..cd0cf0d8e45 100644 --- a/issues/the-amd-benchs-idle-package-draws-15-percent-over-linuxs.md +++ b/issues/the-amd-benchs-idle-package-draws-15-percent-over-linuxs.md @@ -15,9 +15,15 @@ every Linux run (p = 0.029): - The median of the runs' per-second p50s is 1688 mW against 1472 mW. - The median of the runs' means is 1688 mW against 1501 mW. -The cores are not the consumer. Their own energy counters give ToyOS 0 mW -over the idle minute and Linux 11 mW, so the gap of about 200 mW lies -outside the cores: in the SoC, memory, I/O or devices. Its 10 ms sleeper, +These runs predate the matched read. In them, Linux read only the package +each second, while ToyOS's read kicked every CPU. That asymmetry adds to +ToyOS's figure in exactly this row, by an amount not yet measured; a run of +the matched read settles it. + +The cores are not the consumer. Their own energy counters give ToyOS 0-1 mW +over the idle minute across the four runs, against Linux's 11-20 mW, so the +gap of about 200 mW lies outside the cores: in the SoC, memory, I/O or +devices. Its 10 ms sleeper, 1700-1710 against 1391-1705 mW, is not resolved. Work may not be slowed to close the gap.