From 27bf960ec656cc584456fa0ab2225b1d70423bc9 Mon Sep 17 00:00:00 2001 From: japabu Date: Wed, 28 Oct 2026 09:07:46 +0100 Subject: [PATCH 1/5] counters: every reading is taken after its read was called MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A counters read within JOIN (10 ms) of the last round's issue used to be served that round, so a reader could get a reading taken up to 10 ms before it asked, and two reads 5 ms apart one reading twice. #1073's group-start diagnostic lost 13 of its 17 clock windows to it. Now a read joins only a round issued after its call began. Inside JOIN of the last issue it rests until JOIN has passed and then issues its own; readers resting together wake at the same instant, the first issues and the rest join. The wait is the round rate limit with a known end, not a delay standing in for an event, and toyos_abi::counters says a read may wait up to 10 ms. The kick rate per CPU is unchanged: one per JOIN at most. Consumers follow: - osbench_rules::window is strict: a window lies between the row less its reads' cost and the row; JOIN_NS is gone. - counters_read requires the read straight after an answered one to give every CPU a new stamp, where it used to loop until a stamp moved. - acpi_mediated's Counted, which read until the boot processor's stamp was one it had not seen, is a plain function. Counter::DeepIdleNanos is deleted with its kernel accounting: the kernel credits a period's time only when it ends, so its time is no fraction of a window, and no consumer read it. DeepIdles stays. Closes issues/a-counters-read-within-10-ms-of-another-returns-the-older-reading-as-new.md. The idle track records D1-D3's findings, with mem's 3789 open and the fixed counters as the instrument to read it next. The AMD deflate record compares MPERF against MPERF on the same machine (APM Vol. 2 §17.3: MPERF counts only in C0), not against the TSC. Co-Authored-By: Claude Opus 5.5 --- ...nother-returns-the-older-reading-as-new.md | 31 ----------- .../idle-cpus-enter-the-c-states-cst-names.md | 40 ++++++++++----- ...nt-slow-in-some-boots-at-the-same-clock.md | 7 +-- kernel/src/counters.rs | 47 +++++++++++------ kernel/src/idle.rs | 13 ++--- lib/inspect/src/kernel.rs | 4 +- sdk/abi/src/counters.rs | 18 +++---- tests/guest/osbench/rules/src/lib.rs | 30 +++++------ .../guest/rust-tests/src/bin/acpi_mediated.rs | 51 +++++++------------ .../guest/rust-tests/src/bin/counters_read.rs | 13 ++--- 10 files changed, 109 insertions(+), 145 deletions(-) delete mode 100644 issues/a-counters-read-within-10-ms-of-another-returns-the-older-reading-as-new.md diff --git a/issues/a-counters-read-within-10-ms-of-another-returns-the-older-reading-as-new.md b/issues/a-counters-read-within-10-ms-of-another-returns-the-older-reading-as-new.md deleted file mode 100644 index 08f4f672f1c..00000000000 --- a/issues/a-counters-read-within-10-ms-of-another-returns-the-older-reading-as-new.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -status: assigned -kind: defect -opened: 2026-10-28 ---- - -# A counters read within 10 ms of another returns the older reading as new - -`SYS_COUNTERS` reads every CPU in a round: the reader kicks each CPU and -waits for its answer (`kernel/src/counters.rs`). A read issued within -`JOIN` (10 ms) of the last round's issue waits on that round and is served -its values. Nothing in the reply says so. The reader gets a reading taken up -to 10 ms before it asked, and two reads 5 ms apart can get one reading twice. - -#1073's group-start diagnostic read the counters around each group's 5 ms -`cpu.clock` window. 13 of its 17 windows came back with before equal to -after on every CPU (`ToyOSOrg/evidence` branch `pr-1073`, `r6/`). osbench's -shipped instruments now take each window from the readings' own counter -stamps and refuse a CPU whose readings are not the row's -(`osbench_rules::window`). That makes the defect visible, not absent: a -caller that does not compare stamps still reads an old reading as current. - -Owner: the idle track's follow-up -(`issues/idle-cpus-enter-the-c-states-cst-names.md`), in -`kernel/src/counters.rs` and the `toyos_abi::counters` record. - -**Exit**: a read never returns a reading older than its own call as -current. Either it waits for a round issued after it, or its record says it -was served an earlier round and when that round was issued; the ABI test -and a kernel host test hold whichever is chosen. osbench's window rule -stays as the reader's check. diff --git a/issues/idle-cpus-enter-the-c-states-cst-names.md b/issues/idle-cpus-enter-the-c-states-cst-names.md index 6cad4707afd..40496d12553 100644 --- a/issues/idle-cpus-enter-the-c-states-cst-names.md +++ b/issues/idle-cpus-enter-the-c-states-cst-names.md @@ -79,11 +79,7 @@ each against its control with the deep states off, all else equal): `r4/` in the evidence), and acpiserver ran for the whole of each of those clock windows: a measured busy core beside the clock's thread. At the starts of net, procs, mem, tail and nvme still at 3789, the roster names - nothing beside osbench for more than 0.9 ms of the 5.5 ms window. Mem - stays unexplained. Net, procs, tail and nvme reached 4189 under the - always-deepest control, so the history holding idle CPUs shallow after - the group before is the open hypothesis there; a per-CPU state read - inside the window would test it. + nothing beside osbench for more than 0.9 ms of the 5.5 ms window. - Group starts at 4189 across the eight T14 boots with the deep states: 8 and 8 (r2), 8 and 10 (r3), 10 (B4), 5 (B5), 10 (B6) and 10 (B7), counted as a maximum above 4100 MHz. Only sched, files, ipc and work reached it @@ -96,15 +92,31 @@ each against its control with the deep states off, all else equal): underran. **#1066's exit is unmet** where this track stands: the T14's `cpu.clock` reaches -ratio 42 at 5 to 10 of 17 group starts against Linux's 13. Of the groups -below it, boot, identity, energy and system are held by acpiserver's -measured start-up work; the rest are the open question above. Three -diagnostic boots (#1073's `r6/`) read the counters around each group's -clock, the same with the history switched off, and the scheduler's diary -across it. The counters read came back as one reading twice in 13 of 17 -windows (`issues/a-counters-read-within-10-ms-of-another-returns-the-older-reading-as-new.md`), -and switching the history off left the count inside the spread. The -finding goes into this record with the track's next change. +ratio 42 at 5 to 10 of 17 group starts against Linux's 13. Three diagnostic +T14 boots at #1073's head read each group's clock window (`r6/` in the +evidence): + +- With the counters read around each window (D1), 8 group starts reached + 4189; with the history also admitting every state (D2), 10. Both are + inside the spread: the history does not hold the count down, and no + idle-policy change is earned. +- D1's counters showed nothing. In 13 of 17 windows the two reads, 5 ms + apart, shared one round (since fixed: every reading is taken after its + read was called), and the windows that did differ were read before + osbench took a window from the readings' own stamps. +- The scheduler's diary across each window (D3), a boot in which boot, + identity, energy, system and mem were at 3789: in the first four + acpiserver ran beside osbench, 5.5 ms of the window by the roster. In mem + the diary and the roster agree that no thread but osbench's ran on any + CPU, and mem stays at 3789 under always-deepest too, so neither thread + work nor the idle choice holds it. The diary counts a thread running from + before the window only from its first event inside it. Net, procs, tail + and nvme reached 4189 in this boot. + +So boot, identity, energy and system are held by acpiserver's measured +start-up work. Mem's 3789 is open: interrupts, which no thread view sees, +or the package's power or thermal state after procs. The counters read +around mem's clock window is the instrument to read it with next. **Exit**: on the T14, since a KVM guest's idle is its host's: the idle minute's package energy, timer lateness no worse than `hlt`'s by more than diff --git a/issues/the-amd-benchs-deflate-runs-6-percent-slow-in-some-boots-at-the-same-clock.md b/issues/the-amd-benchs-deflate-runs-6-percent-slow-in-some-boots-at-the-same-clock.md index 079e7d34f96..480ef9c20b5 100644 --- a/issues/the-amd-benchs-deflate-runs-6-percent-slow-in-some-boots-at-the-same-clock.md +++ b/issues/the-amd-benchs-deflate-runs-6-percent-slow-in-some-boots-at-the-same-clock.md @@ -18,9 +18,10 @@ than one value across twelve boots (`ToyOSOrg/evidence` branch `pr-1073`, The thread's APERF/MPERF ratio, the one figure of the pair AMD defines (APM Vol. 2 §17.3), differed by under 1 % across the boots. The slow boots include one with every idle CPU in C1 and the rest with the deep states. -In the slow A\* boot and the slow B4 boot, no other CPU's MPERF advanced by -a thousandth of its TSC's count across the row, and MPERF counts only in -C0 (§17.3). +In the slow A\* boot and the slow B4 boot, every other CPU's MPERF advanced +under a thousandth of the deflate thread's own across the row (0 against +999, each in thousandths of the same window), and MPERF counts only in C0 +(§17.3). So it is not the idle states, the clock, the placement, or a running neighbour. diff --git a/kernel/src/counters.rs b/kernel/src/counters.rs index 393c057cefe..93e0fe8b0d3 100644 --- a/kernel/src/counters.rs +++ b/kernel/src/counters.rs @@ -7,9 +7,12 @@ //! the round's [`ANSWER`] bound has passed. A CPU answers from its kick //! handler ([`serve_here`]) wherever it was; one whose interrupts stay closed //! past the bound is reported stale with the last block it published, never -//! waited for without end. A read within [`JOIN`] of the last round's issue -//! waits on that round, so no reader makes a CPU take more than one kick per -//! [`JOIN`]. +//! waited for without end. +//! +//! **Every reading is taken after its read was called.** A read joins a round +//! issued after its call began, or issues its own; a round is issued at most +//! once per [`JOIN`], so a read inside one waits until it has passed, and no +//! reader makes a CPU take more than one kick per [`JOIN`]. //! //! A model-specific counter is read only where the CPU's own verdict at //! [`bring_up`] admitted it, and the power envelope only where the CPU-state @@ -40,7 +43,7 @@ const ANSWER: Budget = Budget::of( /// The most often a round is issued. const JOIN: Cadence = Cadence::every( Duration::from_millis(10), - "a reader within it of the last round's issue waits on that round, so a kick per CPU per 10 ms at most", + "a reader within it of the last round's issue waits until it has passed, so a kick per CPU per 10 ms at most", ); /// The counters a CPU's model-specific registers hold, where its bring-up @@ -145,8 +148,7 @@ fn sample(me: usize) -> [u64; WORDS] { Counter::EnergyPerfBias => hardware.envelope.as_ref().map(|e| e.energy_perf_bias), Counter::FirmwareCalls => hardware.firmware.as_ref().map(|f| f.calls), Counter::FirmwareNanos => hardware.firmware.as_ref().map(|f| f.nanos), - Counter::DeepIdles => Some(crate::cpu_local::of(me as u32).idle.deep().0), - Counter::DeepIdleNanos => Some(crate::cpu_local::of(me as u32).idle.deep().1), + Counter::DeepIdles => Some(crate::cpu_local::of(me as u32).idle.deep()), Counter::CoreC6 => hardware.core_c6, Counter::CoreC7 => hardware.core_c7, }; @@ -195,23 +197,36 @@ pub fn read(rights: Rights, out: &mut UserBytesMut) -> Result (generation, issued, false), - _ => { - let generation = ROUNDS.issue(Reach::Narrow); - *last = Some((generation, now)); - (generation, now, true) + let parkable = Parkable::at_entry(); + let called = crate::clock::now(); + let (generation, issued, fresh) = loop { + let due = { + let mut last = LAST.lock(); + let now = crate::clock::now(); + match *last { + // Strictly after: every answer to it is taken after its issue, and so after this call. + Some((generation, issued)) if issued > called => break (generation, issued, false), + Some((_, issued)) if now < issued + JOIN.duration() => issued + JOIN.duration(), + _ => { + let generation = ROUNDS.issue(Reach::Narrow); + *last = Some((generation, now)); + break (generation, now, true); + } } + }; + // A rate limit with a known end, not a delay standing in for an event: the next + // round may be issued at `due` and no sooner. Readers resting here wake together; + // the first issues it, and the rest join. + let handle = crate::sched::driver::current_handle().expect("a syscall runs on a thread"); + if watch::wait_until(&parkable, handle.watch(), 0, WaitClass::Other, Deadline::at(due), || false).is_err() { + return Err(SyscallError::Gone); } }; // This CPU's kick, if it has one, waits behind this syscall. answer(fresh); let answered = || (0..cpus).all(|cpu| ROUNDS.served(cpu, generation)); let deadline = Deadline::at(issued + ANSWER.duration()); - if watch::wait_until(&Parkable::at_entry(), &ANSWERED, 0, WaitClass::Other, deadline, answered).is_err() { + if watch::wait_until(&parkable, &ANSWERED, 0, WaitClass::Other, deadline, answered).is_err() { return Err(SyscallError::Gone); } for cpu in 0..cpus { diff --git a/kernel/src/idle.rs b/kernel/src/idle.rs index 64202231109..25ac31c5418 100644 --- a/kernel/src/idle.rs +++ b/kernel/src/idle.rs @@ -41,9 +41,8 @@ pub struct Local { /// declaration it learned it under. history: AtomicU64, learned_under: AtomicU64, - /// Idle periods spent in a declared state, and their nanoseconds. + /// Idle periods spent in a declared state. deep: AtomicU64, - deep_nanos: AtomicU64, } impl Local { @@ -54,13 +53,12 @@ impl Local { history: AtomicU64::new(0), learned_under: AtomicU64::new(0), deep: AtomicU64::new(0), - deep_nanos: AtomicU64::new(0), } } - /// Idle periods this CPU spent in a declared state, and their nanoseconds. - pub fn deep(&self) -> (u64, u64) { - (self.deep.load(Ordering::Relaxed), self.deep_nanos.load(Ordering::Relaxed)) + /// Idle periods this CPU spent in a declared state. + pub fn deep(&self) -> u64 { + self.deep.load(Ordering::Relaxed) } } @@ -176,7 +174,6 @@ pub fn wait(plan: &idle::Plan) { // Falling back to C1: what this CPU publishes is what it waits in. crate::sched::driver::publish_idle_exit(0); } - let entered = crate::clock::nanos_since_boot(); match (state, &d) { (Some(state), Some(d)) => crate::arch::idle::enter(d.entries[state]), _ => crate::arch::hw::halt(), @@ -184,10 +181,8 @@ pub fn wait(plan: &idle::Plan) { // Awake: no hold has a reason to wake this CPU until it waits again. crate::sched::driver::publish_idle_exit(0); let woke = crate::clock::nanos_since_boot(); - let slept = woke.saturating_sub(entered); if state.is_some() { local.deep.fetch_add(1, Ordering::Relaxed); - local.deep_nanos.fetch_add(slept, Ordering::Relaxed); } let Some(d) = d else { return }; let costs = &d.costs[..d.len]; diff --git a/lib/inspect/src/kernel.rs b/lib/inspect/src/kernel.rs index bdcb28e246b..ee1c72f0473 100644 --- a/lib/inspect/src/kernel.rs +++ b/lib/inspect/src/kernel.rs @@ -67,7 +67,7 @@ mod tests { /// calls on the boot processor, which makes them. fn whole(cpu: u32) -> Record { let (calls, nanos) = if cpu == 0 { (Some(1), Some(15_968)) } else { (None, None) }; - record(cpu, false, [Some(10), Some(4817), Some(100), Some(200), Some(3), Some(0x8000_2a04), Some(0x8000_ff01), Some(6), calls, nanos, Some(40), Some(2_000_000), Some(1_000), Some(3_000)]) + record(cpu, false, [Some(10), Some(4817), Some(100), Some(200), Some(3), Some(0x8000_2a04), Some(0x8000_ff01), Some(6), calls, nanos, Some(40), Some(1_000), Some(3_000)]) } #[test] @@ -88,7 +88,7 @@ mod tests { /// `TRACE`: only what the record carries, and no zero in place of the rest. #[test] fn an_absent_counter_has_no_path() { - let got = render(&[record(0, false, [Some(10), None, None, None, None, None, None, None, None, None, None, None, None, None])]).unwrap(); + let got = render(&[record(0, false, [Some(10), None, None, None, None, None, None, None, None, None, None, None, None])]).unwrap(); let paths: Vec<&str> = got.keys().map(String::as_str).collect(); assert_eq!(paths, vec!["kernel.cpu.0.hardware_id", "kernel.cpu.0.stale", "kernel.cpu.0.stamp"]); } diff --git a/sdk/abi/src/counters.rs b/sdk/abi/src/counters.rs index 8a883d5edca..c6642967cc8 100644 --- a/sdk/abi/src/counters.rs +++ b/sdk/abi/src/counters.rs @@ -3,6 +3,9 @@ //! //! **A CPU's hardware counters are readable only on that CPU**, so a read asks //! every CPU to copy its counters out and waits for the answers under a bound. +//! Every answer is taken after the read was called. Reads close together +//! share one round of asking, and a round is asked at most every 10 ms, so a +//! read may wait up to 10 ms before its round begins. //! A CPU that has not answered by then is [`Record::stale`]: its values are the //! last it gave, and its [`Counter::Stamp`] says when it gave them. A counter //! the CPU does not have, or that the caller's capability may not read, is @@ -53,8 +56,6 @@ pub enum Counter { /// Idle periods the CPU waited in a state deeper than C1 its firmware /// declared. DeepIdles, - /// Nanoseconds those periods lasted, each from the choice to the wake. - DeepIdleNanos, /// Time the CPU's core was in its processor-specific C6 states, in TSC /// ticks (`MSR_CORE_C6_RESIDENCY`): the hardware's own count, beside the /// kernel's choices. @@ -64,7 +65,7 @@ pub enum Counter { } impl Counter { - pub const COUNT: usize = 14; + pub const COUNT: usize = 13; pub const ALL: [Counter; Self::COUNT] = [ Self::Stamp, Self::Smi, @@ -77,7 +78,6 @@ impl Counter { Self::FirmwareCalls, Self::FirmwareNanos, Self::DeepIdles, - Self::DeepIdleNanos, Self::CoreC6, Self::CoreC7, ]; @@ -95,7 +95,6 @@ impl Counter { Self::FirmwareCalls => "firmware_calls", Self::FirmwareNanos => "firmware_nanos", Self::DeepIdles => "deep_idles", - Self::DeepIdleNanos => "deep_idle_nanos", Self::CoreC6 => "core_c6", Self::CoreC7 => "core_c7", } @@ -112,7 +111,6 @@ impl Counter { | Self::HwpRequestPkg | Self::EnergyPerfBias | Self::DeepIdles - | Self::DeepIdleNanos | Self::CoreC6 | Self::CoreC7 => Rights::COUNTERS.union(Rights::TRACE), } @@ -216,8 +214,8 @@ mod tests { #[test] fn every_shape_of_record_reads_back_as_written() { for r in [ - record(false, [Some(1), Some(u64::from(u32::MAX)), Some(3), Some(u64::MAX), Some(0), Some(0x8000_2a04), Some(6), Some(0), Some(2), Some(31_936), Some(0), Some(7_000_000), Some(5), None]), - record(true, [Some(9), None, None, None, Some(4), None, Some(0x8000_ff01), None, Some(0), None, Some(12), None, None, Some(9)]), + record(false, [Some(1), Some(u64::from(u32::MAX)), Some(3), Some(u64::MAX), Some(0), Some(0x8000_2a04), Some(6), Some(0), Some(2), Some(31_936), Some(7_000_000), Some(5), None]), + record(true, [Some(9), None, None, None, Some(4), None, Some(0x8000_ff01), None, Some(0), None, Some(12), None, Some(9)]), record(true, [None; Counter::COUNT]), ] { assert_eq!(Record::decode(&r.encode()), Ok(r)); @@ -227,7 +225,7 @@ mod tests { /// Absent is not zero: a zero that is there reads back there. #[test] fn a_present_zero_is_not_an_absent_counter() { - let r = record(false, [Some(5), Some(0), None, None, None, None, None, None, None, None, None, None, None, None]); + let r = record(false, [Some(5), Some(0), None, None, None, None, None, None, None, None, None, None, None]); assert_eq!(Record::decode(&r.encode()).unwrap().get(Counter::Smi), Some(0)); assert_eq!(Record::decode(&r.encode()).unwrap().get(Counter::Aperf), None); } @@ -244,7 +242,7 @@ mod tests { #[test] fn a_word_under_an_absent_counter_is_refused() { - let mut raw = record(false, [Some(1), None, None, None, None, None, None, None, None, None, None, None, None, None]).encode(); + let mut raw = record(false, [Some(1), None, None, None, None, None, None, None, None, None, None, None, None]).encode(); raw.0[16 + 8 * Counter::Mperf as usize] = 1; assert_eq!(Record::decode(&raw), Err(Undecodable::Absent(Counter::Mperf))); } diff --git a/tests/guest/osbench/rules/src/lib.rs b/tests/guest/osbench/rules/src/lib.rs index 1b94183a951..7f9326f0db5 100644 --- a/tests/guest/osbench/rules/src/lib.rs +++ b/tests/guest/osbench/rules/src/lib.rs @@ -248,19 +248,13 @@ pub mod energy { pub mod window { //! Whether one CPU's two counters readings describe the row they were //! read around. Each reading carries the CPU's own counter as it - //! answered, so the readings' window is the difference of the two, - //! whatever the kernel served. + //! answered, so the readings' window is the difference of the two. //! //! The row is the span from the first read's call to the second read's - //! return. A CPU answers each read inside its call, so its window lies - //! within the row and is shorter by at most the two calls' cost, except - //! that a read issued within [`JOIN_NS`] of another round is served that - //! round, issued up to [`JOIN_NS`] before the call. A window outside - //! those bounds is not the row's. - - /// The kernel's counters join interval (`kernel/src/counters.rs`'s - /// `JOIN`): how long before its call a read may be served a round. - pub const JOIN_NS: u64 = 10_000_000; + //! return. A CPU answers each read inside its call + //! (`toyos_abi::counters`), so its window lies within the row and is + //! shorter by at most the two calls' cost. A window outside those bounds + //! is not the row's. /// The row's span and what its two reads cost, in nanoseconds. #[derive(Clone, Copy, Debug, PartialEq, Eq)] @@ -319,8 +313,7 @@ pub mod window { } let window_ns = to_ns(ticks); let shortest = span.row_ns.saturating_sub(span.reads_ns); - let longest = span.row_ns.saturating_add(JOIN_NS); - if !(shortest..=longest).contains(&window_ns) { + if !(shortest..=span.row_ns).contains(&window_ns) { return Err(Refusal::NotTheRow { window_ns, row_ns: span.row_ns, reads_ns: span.reads_ns }); } Ok((ticks, window_ns)) @@ -357,17 +350,18 @@ pub mod window { } #[test] - fn a_before_served_from_a_joined_round_stands_within_the_join() { - assert_eq!(of(false, Some(0), Some(2_020_000_000), ROW, ns), Ok((2_020_000_000, 1_010_000_000))); + fn a_window_longer_than_the_row_is_refused() { + assert_eq!(of(false, Some(0), Some(2_000_000_000), ROW, ns), Ok((2_000_000_000, 1_000_000_000)), "the row exactly"); + // A `before` taken before the row's first read was called. assert_eq!( - of(false, Some(0), Some(2_020_000_002), ROW, ns), - Err(Refusal::NotTheRow { window_ns: 1_010_000_001, row_ns: 1_000_000_000, reads_ns: 200_000 }) + of(false, Some(0), Some(2_000_000_002), ROW, ns), + Err(Refusal::NotTheRow { window_ns: 1_000_000_001, row_ns: 1_000_000_000, reads_ns: 200_000 }) ); } #[test] fn a_window_shorter_than_the_row_less_its_reads_is_refused() { - // An `after` served a round issued inside the row. + // An `after` taken before the row's second read was called. assert_eq!( of(false, Some(0), Some(1_900_000_000), ROW, ns), Err(Refusal::NotTheRow { window_ns: 950_000_000, row_ns: 1_000_000_000, reads_ns: 200_000 }) diff --git a/tests/guest/rust-tests/src/bin/acpi_mediated.rs b/tests/guest/rust-tests/src/bin/acpi_mediated.rs index 39d609d6a68..8bf8537c045 100644 --- a/tests/guest/rust-tests/src/bin/acpi_mediated.rs +++ b/tests/guest/rust-tests/src/bin/acpi_mediated.rs @@ -359,35 +359,21 @@ const STORM: usize = 10_000; /// processor, as `acpi_release`'s. const STARTS: usize = 64; -/// What the boot processor counts of the kernel's writes to `SMI_CMD`, read -/// in a round of the counters issued after the last one read here returned: a -/// read within the kernel's joining window of a round answers with that -/// round, which may be older than what the caller just did, so this reads -/// until the boot processor's stamp is one it has not seen. Nothing else on -/// this boot reads the counters, so a round not seen is one this asked for. -struct Counted<'a> { - cap: &'a SysCap, - seen: Option, -} - -impl Counted<'_> { - fn firmware_calls(&mut self) -> u64 { - loop { - let mut raw = vec![RawRecord::EMPTY; syscall::cpu_count() as usize]; - let n = self.cap.counters(&mut raw).expect("acpi: the estate's capability reads the counters"); - let records: Vec = raw[..n].iter().map(|r| Record::decode(r).expect("acpi: a record that decodes")).collect(); - let stamp = records[0].get(Counter::Stamp); - if records.iter().any(|r| r.stale) || stamp == self.seen { - std::thread::yield_now(); - continue; - } - self.seen = stamp; - // No other CPU holds the counter: none writes the port. - for r in &records[1..] { - assert_eq!(r.get(Counter::FirmwareCalls), None, "acpi: cpu{} counts firmware calls, and makes none", r.cpu); - } - return records[0].get(Counter::FirmwareCalls).expect("acpi: the boot processor counts its writes to SMI_CMD"); +/// What the boot processor counts of the kernel's writes to `SMI_CMD`, in a +/// round every CPU answered: every answer is taken after its read was called. +fn firmware_calls(cap: &SysCap) -> u64 { + loop { + let mut raw = vec![RawRecord::EMPTY; syscall::cpu_count() as usize]; + let n = cap.counters(&mut raw).expect("acpi: the estate's capability reads the counters"); + let records: Vec = raw[..n].iter().map(|r| Record::decode(r).expect("acpi: a record that decodes")).collect(); + if records.iter().any(|r| r.stale) { + continue; + } + // No other CPU holds the counter: none writes the port. + for r in &records[1..] { + assert_eq!(r.get(Counter::FirmwareCalls), None, "acpi: cpu{} counts firmware calls, and makes none", r.cpu); } + return records[0].get(Counter::FirmwareCalls).expect("acpi: the boot processor counts its writes to SMI_CMD"); } } @@ -441,7 +427,6 @@ fn firmware(holder: &Holder, info: &AcpiInfo, cap: &SysCap) { let sci_en = || holder.read(Space::SystemIo, control, Width::Word).expect("acpi: PM1a_CNT reads") & 1; let last_written = || holder.read(Space::SystemIo, smi_cmd, Width::Byte).expect("acpi: SMI_CMD reads"); let write = |at: u64, width: Width, value: u64| holder.write(Space::SystemIo, at, width, value); - let mut counted = Counted { cap, seen: None }; let hex = |bytes: &[u64]| bytes.iter().map(|byte| format!("{byte:#04x}")).collect::>().join(", "); // Whether this chipset interrupts its firmware on a write to the port: // the ICH9's `SMI_EN` at PMBASE + 30h, `APMC_EN` its bit 5, PMBASE being @@ -450,7 +435,7 @@ fn firmware(holder: &Holder, info: &AcpiInfo, cap: &SysCap) { // The kernel's own commands: refused, and the chipset saw neither, which // leaves ACPI mode on the second. - let before = (counted.firmware_calls(), last_written()); + let before = (firmware_calls(cap), last_written()); assert!(named.iter().all(|&byte| byte != 0 && !CROSSED.contains(&byte) && !STORMED.contains(&byte)), "acpi: this FADT names {}", hex(&named)); for kept in named { assert_eq!(write(smi_cmd, Width::Byte, kept), Err(Refused::KernelCommand), "acpi: {kept:#04x}, which the FADT names, was written to SMI_CMD"); @@ -460,12 +445,12 @@ fn firmware(holder: &Holder, info: &AcpiInfo, cap: &SysCap) { for (at, width) in [(smi_cmd, Width::Word), (smi_cmd, Width::DWord), (smi_cmd - 1, Width::Word), (smi_cmd - 3, Width::DWord)] { assert_eq!(write(at, width, CROSSED[0]), Err(Refused::CommandSpan), "acpi: a {width:?} at {at:#x}"); } - assert_eq!((counted.firmware_calls(), last_written()), before, "acpi: a refused command was written or counted"); + assert_eq!((firmware_calls(cap), last_written()), before, "acpi: a refused command was written or counted"); // A call: the chipset has the byte, and the boot processor counted one write. let mut asked_from = vec![call_off_the_boot_processor(holder, smi_cmd, CROSSED[0])]; assert_eq!(last_written(), CROSSED[0], "acpi: SMI_CMD does not hold the byte the kernel was asked to write"); - assert_eq!(counted.firmware_calls(), before.0 + 1, "acpi: one firmware call"); + assert_eq!(firmware_calls(cap), before.0 + 1, "acpi: one firmware call"); for value in &CROSSED[1..] { asked_from.push(call_off_the_boot_processor(holder, smi_cmd, *value)); assert_eq!(last_written(), *value); @@ -489,7 +474,7 @@ fn firmware(holder: &Holder, info: &AcpiInfo, cap: &SysCap) { }); let refused_at = stormed.unwrap_or_else(|| panic!("acpi: {STORM} firmware calls in a row were made, and none refused")); assert_ne!(last_written(), STORMED[refused_at % 2], "acpi: a call refused CommandRate reached SMI_CMD"); - assert_eq!(counted.firmware_calls(), before.0 + made as u64, "acpi: the boot processor counted another number of writes than were made"); + assert_eq!(firmware_calls(cap), before.0 + made as u64, "acpi: the boot processor counted another number of writes than were made"); println!( "acpi: firmware calls of {} were asked from the CPUs of x2APIC id {asked_from:?}; this chipset's SMI_EN reads {smi_en:#010x}, APMC_EN {}", hex(&CROSSED), diff --git a/tests/guest/rust-tests/src/bin/counters_read.rs b/tests/guest/rust-tests/src/bin/counters_read.rs index d757594abd9..d8d77547f76 100644 --- a/tests/guest/rust-tests/src/bin/counters_read.rs +++ b/tests/guest/rust-tests/src/bin/counters_read.rs @@ -12,8 +12,9 @@ //! - A round every CPU answered, read until there is one: each CPU's record //! is in its place, carries its stamp, and names a hardware id no other //! does — a reader that filled every block itself would name its own on -//! each. The next round: every stamp moved on, and every CPU but at most -//! the reader's own took a kick in between, which a kick counted as anything +//! each. The read straight after: every stamp moved on, since every answer +//! is taken after its read was called, and every CPU but at most the +//! reader's own took a kick in between, which a kick counted as anything //! else would not show. //! - The rights: on `COUNTERS` alone the counters that time programs are //! absent and the rest as before; without it, and on a handle that is no @@ -57,13 +58,7 @@ fn main() { assert_eq!(cap().counters(&mut []), Ok(cpus), "the empty buffer's count"); let first = answered(cap()); - let second = loop { - let next = answered(cap()); - // Equal stamps are the same round, joined; a later one is the next. - if next.iter().zip(&first).any(|(n, f)| n.get(Counter::Stamp) != f.get(Counter::Stamp)) { - break next; - } - }; + let second = answered(cap()); for (cpu, (f, s)) in first.iter().zip(&second).enumerate() { assert!(s.get(Counter::Stamp) > f.get(Counter::Stamp), "cpu{cpu}'s stamp did not move: {f:?} then {s:?}"); assert_eq!(f.hardware_id, s.hardware_id, "cpu{cpu} named two hardware ids"); From 74c60af4d9545fb2acf0c854b1b0c2bb48f76ed6 Mon Sep 17 00:00:00 2001 From: japabu Date: Wed, 28 Oct 2026 09:47:47 +0100 Subject: [PATCH 2/5] idle track: the exit costs as absolute deltas, a tradeoff for the architect MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The wake-storm and sleep-lateness costs were recorded as accepted on ratios alone. The track now gives them in microseconds, controlled: two A* boots (every idle CPU in C1) against two B3 boots at f6510defd, one session a bench, with B7 at 18fe4e150 beside them. On the T14 a 10 ms sleep lands 10 us later and a 4- or 16-thread storm drains 52 to 91 us later; on the AMD a 1 ms sleep lands 27 us sooner and nothing else moves (its storm reads 107 or 306 us either way). Their consequences for interactive, audio (soundserver's hold, half of a 2.9 ms period, admits the T14's 1048 us exit) and normal workloads are stated as inference and labelled a tradeoff to bring to the architect, not accepted. cpu.clock's ratio 42 at group starts is a diagnostic of the turbo grant, not the product target; the exit is worded so, with single-thread rows at Linux's on the same T14 in its place, and no idle predictor is tuned before the fixed counters have read mem's window. acpiserver's processor _OSC commit: a commit that fails, is refused or masks bits gates nothing, because _CST is read either way and each state is checked on its own terms (cst.rs: FFH bus-master avoidance, SW-coordinated _CSD and type-3 SystemIO withheld; SystemIO not used on Intel). The bits are statements of what OSPM does (302223-007 §2.1), and the firmware's answer to them is the _CST it returns. osc.rs states that invariant; no code changes. Co-Authored-By: Claude Opus 5.5 --- .../idle-cpus-enter-the-c-states-cst-names.md | 56 +++++++++++++------ system/acpiserver/src/osc.rs | 4 +- 2 files changed, 42 insertions(+), 18 deletions(-) diff --git a/issues/idle-cpus-enter-the-c-states-cst-names.md b/issues/idle-cpus-enter-the-c-states-cst-names.md index 40496d12553..96bf60c6423 100644 --- a/issues/idle-cpus-enter-the-c-states-cst-names.md +++ b/issues/idle-cpus-enter-the-c-states-cst-names.md @@ -34,23 +34,42 @@ faster than a `hlt` control. Its `work.build_shaped.j1` p50 read 1176 ms against the control's 967, with its own 5 runs spread from 817 to 1594 ms, the control on a different `main`: unresolved. -**The exit cost of deep states, accepted** (orchestrator, 2026-10-28, under -the owner's direction of 2026-10-27), as #1073's first boots read it at -`59dcd1f03`, p50 against main's `ea6414e53` and against Linux 6.8 on the same -bench (Linux's 50 µs timer slack included): +**The exit cost of deep states is a tradeoff to bring to the architect, not +accepted.** It was recorded as accepted on ratios alone (1.5 to 3.4 times), +which accepts no material compromise. In absolute terms, p50 in µs with every +idle CPU in C1 (two A\* boots) against the deep states (two B3 boots), one +build (`f6510defd`) and one session a bench, B7 at `18fe4e150` beside them, +and Linux 6.8 on the same bench (its 50 µs timer slack included; #1073's +first boots): -| row | T14 main | T14 deep idle | T14 Linux | AMD main | AMD deep idle | AMD Linux | -|---|---|---|---|---|---|---| -| `time.sleep_late.1ms` | 17.7 µs | 22.9 µs | 71.5-71.8 µs | 32.5 µs | 5.4 µs | 69.7-70.8 µs | -| `time.sleep_late.10ms` | 17.8 µs | 29.5 µs | 213 µs | 32.6 µs | 32.3 µs | 342-344 µs | -| `wake.storm.drain.w4` | 37 µs | 127 µs | 121-123 µs | 105 µs | 307 µs | 77-78 µs | -| `wake.storm.drain.w16` | 77 µs | 144 µs | 370-1041 µs | 333 µs | 329 µs | 113-200 µs | +| row | T14 C1 | T14 deep | T14 delta | T14 Linux | AMD C1 | AMD deep | AMD delta | AMD Linux | +|---|---|---|---|---|---|---|---|---| +| `time.sleep_late.1ms` | 17.7, 17.8 | 22.1, 14.6 (B7 17.2) | −3 to +4 | 71.5-71.8 | 32.2, 32.5 | 5.6, 5.6 (B7 5.4) | −27 | 69.7-70.8 | +| `time.sleep_late.10ms` | 17.8, 17.8 | 28.3, 28.3 (B7 27.4) | +10 | 213 | 32.3, 32.6 | 32.6, 33.3 (B7 32.8) | +0.3 to +1 | 342-344 | +| `wake.storm.drain.w4` | 38.5, 37.6 | 126.1, 128.6 (B7 127.9) | +88 to +91 | 121-123 | 306, 107 | 112, 306 (B7 107) | none: 107 or 306 either way | 77-78 | +| `wake.storm.drain.w16` | 74.1, 77.6 | 161.1, 145.5 (B7 130.1) | +52 to +87 | 370-1041 | 304, 328 | 321, 331 (B7 350) | within the spread | 113-200 | A sleep long enough for a deep state is woken early by its exit and still lands later the deeper the state was (none at 100 µs, C2 at 1 ms, C10 at 10 ms), which an early wake that fell short would make hundreds of µs: consistent with the cost of the caches a deep state powers off, not measured -as that. A burst of wakes to CPUs in deep states pays their exits. +as that. A burst of wakes to CPUs in deep states pays their exits. The cost +is the T14's; on the AMD the deep state lands a 1 ms sleep 27 µs sooner and +moves nothing else. + +What it costs, as inference, not measured per workload: + +- **Interactive**: an input event that wakes an idle CPU on the T14 lands up + to about 10 µs later, and one that fans out to 4 to 16 sleeping threads + about 50 to 90 µs later, against a 60 Hz frame's 16.7 ms. +- **Audio**: while a stream plays, soundserver holds the exit to half its + 2.9 ms period, about 1.45 ms, which both T14 states (253 and 1048 µs) and + the AMD's (400 µs) are inside: so a period's wake may still pay up to + 1048 µs of exit on the T14, a third of the period. Measured, its worst IRQ + lateness is not moved and no window underran (below). +- **Normal workloads**: a server or build that wakes many sleeping threads + in bursts pays the storm's 50 to 90 µs per burst on the T14, latency + rather than CPU time; throughput rows did not lose (below). **Where the exit stands** (#1073's second boots at `f8749dc69`, two boots each against its control with the deep states off, all else equal): @@ -91,8 +110,9 @@ each against its control with the deep states off, all else equal): to 564 µs and AMD 0 to 2661. The deep states move neither, and no window underran. -**#1066's exit is unmet** where this track stands: the T14's `cpu.clock` reaches -ratio 42 at 5 to 10 of 17 group starts against Linux's 13. Three diagnostic +**`cpu.clock` at group starts** is a diagnostic of the turbo grant, not a +product target: the T14's reaches ratio 42 at 5 to 10 of 17 group starts +against Linux's 13, and #1066 set Linux's count as its exit. Three diagnostic T14 boots at #1073's head read each group's clock window (`r6/` in the evidence): @@ -116,11 +136,13 @@ evidence): So boot, identity, energy and system are held by acpiserver's measured start-up work. Mem's 3789 is open: interrupts, which no thread view sees, or the package's power or thermal state after procs. The counters read -around mem's clock window is the instrument to read it with next. +around mem's clock window is the instrument to read it with next, and no +idle predictor is tuned before it has been read. **Exit**: on the T14, since a KVM guest's idle is its host's: the idle minute's package energy, timer lateness no worse than `hlt`'s by more than -the chosen state's exit latency, and osbench's `cpu.clock` maximum at ratio -42 (4185 to 4190 MHz, as Linux's reads it on the same T14) in every group -where Linux's reaches 42. +the chosen state's exit latency, the exit costs above decided by the +architect, and single-thread rows (`crunch.single`) at Linux's on the same +T14. `cpu.clock` at each group start is read beside them to explain a +shortfall, not held to ratio 42 for its own sake. **Mutation**: always the deepest state. **Oracle**: Linux's figures. diff --git a/system/acpiserver/src/osc.rs b/system/acpiserver/src/osc.rs index 9f8f1a03ce5..0ee4bd2c1b7 100644 --- a/system/acpiserver/src/osc.rs +++ b/system/acpiserver/src/osc.rs @@ -11,7 +11,9 @@ //! and what the firmware does with a statement is in the objects it then //! returns (`_CST` among them, [`crate::cst`]). A query some firmware answers //! with the same bit whatever was asked, nothing included, would otherwise -//! decide what ToyOS says it does. +//! decide what ToyOS says it does. So a processor commit that fails, is +//! refused or masks bits gates nothing: `_CST` is read either way, and each +//! state it returns is checked on its own terms. //! //! **What ToyOS claims is one declaration a UUID** ([`PLATFORM`], //! [`PROCESSOR`]), each bit set only where ToyOS does what the bit promises, From 7c9cf910f0d58fcc571d4053fd06bf44d40daa34 Mon Sep 17 00:00:00 2001 From: japabu Date: Wed, 28 Oct 2026 10:06:40 +0100 Subject: [PATCH 3/5] review r1: no osbench row starts straight after a rate-limited wait B2: with every reading now taken after its call, an osbench counters read within 10 ms of its previous one sleeps out the kernel's round interval. The crunch loop reads after one workload and before the next a few prints apart, and tail's diary run reads straight after the 60 s run's last read, so on ToyOS alone a measured slice would begin straight after a sleep of up to 10 ms. Since #1079 the comparison measures the systems, not the setup. around::read now gives this process's last reading again where a fresh read would fall inside the interval: one reading bounds both rows beside it, nothing sleeps, and the window rule still holds, since a row's span runs from its first reading's call. The interval is one constant, toyos_abi::counters::INTERVAL_NS, which the kernel's round rate (JOIN, renamed INTERVAL: joining no longer depends on it) and osbench both name. B1: around.rs no longer justifies printing periods by the period time this branch deleted. The track's deltas are the same-build pairs (w16 +68 to +87 with B7 +52, 10 ms sleep +10.5 to +10.6, AMD 10 ms 0 to +1), the history clause is gone, and the crunch.single criterion is dropped from the exit: it was not the architect's wording. Co-Authored-By: Claude Opus 5.5 --- .../idle-cpus-enter-the-c-states-cst-names.md | 20 +++++++++---------- kernel/src/counters.rs | 14 ++++++------- sdk/abi/src/counters.rs | 7 +++++-- tests/guest/osbench/src/around.rs | 19 +++++++++++++++--- 4 files changed, 37 insertions(+), 23 deletions(-) diff --git a/issues/idle-cpus-enter-the-c-states-cst-names.md b/issues/idle-cpus-enter-the-c-states-cst-names.md index 96bf60c6423..941b5b223dc 100644 --- a/issues/idle-cpus-enter-the-c-states-cst-names.md +++ b/issues/idle-cpus-enter-the-c-states-cst-names.md @@ -45,9 +45,9 @@ first boots): | row | T14 C1 | T14 deep | T14 delta | T14 Linux | AMD C1 | AMD deep | AMD delta | AMD Linux | |---|---|---|---|---|---|---|---|---| | `time.sleep_late.1ms` | 17.7, 17.8 | 22.1, 14.6 (B7 17.2) | −3 to +4 | 71.5-71.8 | 32.2, 32.5 | 5.6, 5.6 (B7 5.4) | −27 | 69.7-70.8 | -| `time.sleep_late.10ms` | 17.8, 17.8 | 28.3, 28.3 (B7 27.4) | +10 | 213 | 32.3, 32.6 | 32.6, 33.3 (B7 32.8) | +0.3 to +1 | 342-344 | +| `time.sleep_late.10ms` | 17.8, 17.8 | 28.3, 28.3 (B7 27.4) | +10.5 to +10.6 | 213 | 32.3, 32.6 | 32.6, 33.3 (B7 32.8) | 0 to +1 | 342-344 | | `wake.storm.drain.w4` | 38.5, 37.6 | 126.1, 128.6 (B7 127.9) | +88 to +91 | 121-123 | 306, 107 | 112, 306 (B7 107) | none: 107 or 306 either way | 77-78 | -| `wake.storm.drain.w16` | 74.1, 77.6 | 161.1, 145.5 (B7 130.1) | +52 to +87 | 370-1041 | 304, 328 | 321, 331 (B7 350) | within the spread | 113-200 | +| `wake.storm.drain.w16` | 74.1, 77.6 | 161.1, 145.5 (B7 130.1) | +68 to +87 (B7 +52) | 370-1041 | 304, 328 | 321, 331 (B7 350) | within the spread | 113-200 | A sleep long enough for a deep state is woken early by its exit and still lands later the deeper the state was (none at 100 µs, C2 at 1 ms, C10 at @@ -61,14 +61,14 @@ What it costs, as inference, not measured per workload: - **Interactive**: an input event that wakes an idle CPU on the T14 lands up to about 10 µs later, and one that fans out to 4 to 16 sleeping threads - about 50 to 90 µs later, against a 60 Hz frame's 16.7 ms. + about 70 to 90 µs later, against a 60 Hz frame's 16.7 ms. - **Audio**: while a stream plays, soundserver holds the exit to half its 2.9 ms period, about 1.45 ms, which both T14 states (253 and 1048 µs) and the AMD's (400 µs) are inside: so a period's wake may still pay up to 1048 µs of exit on the T14, a third of the period. Measured, its worst IRQ lateness is not moved and no window underran (below). - **Normal workloads**: a server or build that wakes many sleeping threads - in bursts pays the storm's 50 to 90 µs per burst on the T14, latency + in bursts pays the storm's 70 to 90 µs per burst on the T14, latency rather than CPU time; throughput rows did not lose (below). **Where the exit stands** (#1073's second boots at `f8749dc69`, two boots @@ -121,9 +121,8 @@ evidence): inside the spread: the history does not hold the count down, and no idle-policy change is earned. - D1's counters showed nothing. In 13 of 17 windows the two reads, 5 ms - apart, shared one round (since fixed: every reading is taken after its - read was called), and the windows that did differ were read before - osbench took a window from the readings' own stamps. + apart, were served one round, and the windows that did differ were read + before osbench took a window from the readings' own stamps. - The scheduler's diary across each window (D3), a boot in which boot, identity, energy, system and mem were at 3789: in the first four acpiserver ran beside osbench, 5.5 ms of the window by the roster. In mem @@ -141,8 +140,7 @@ idle predictor is tuned before it has been read. **Exit**: on the T14, since a KVM guest's idle is its host's: the idle minute's package energy, timer lateness no worse than `hlt`'s by more than -the chosen state's exit latency, the exit costs above decided by the -architect, and single-thread rows (`crunch.single`) at Linux's on the same -T14. `cpu.clock` at each group start is read beside them to explain a -shortfall, not held to ratio 42 for its own sake. +the chosen state's exit latency, and the exit costs above decided by the +architect. `cpu.clock` at each group start is read beside them as a +diagnostic, not held to ratio 42 for its own sake. **Mutation**: always the deepest state. **Oracle**: Linux's figures. diff --git a/kernel/src/counters.rs b/kernel/src/counters.rs index 93e0fe8b0d3..3009b37279e 100644 --- a/kernel/src/counters.rs +++ b/kernel/src/counters.rs @@ -10,9 +10,9 @@ //! waited for without end. //! //! **Every reading is taken after its read was called.** A read joins a round -//! issued after its call began, or issues its own; a round is issued at most -//! once per [`JOIN`], so a read inside one waits until it has passed, and no -//! reader makes a CPU take more than one kick per [`JOIN`]. +//! issued after its call began, or issues its own; rounds are at least +//! [`INTERVAL`] apart, so a read inside one waits until it has passed, and no +//! reader makes a CPU take more than one kick per [`INTERVAL`]. //! //! A model-specific counter is read only where the CPU's own verdict at //! [`bring_up`] admitted it, and the power envelope only where the CPU-state @@ -40,9 +40,9 @@ const ANSWER: Budget = Budget::of( "every CPU that has not answered is read stale, with the block it last published", ); -/// The most often a round is issued. -const JOIN: Cadence = Cadence::every( - Duration::from_millis(10), +/// The least time between two rounds. +const INTERVAL: Cadence = Cadence::every( + Duration::from_nanos(toyos_abi::counters::INTERVAL_NS), "a reader within it of the last round's issue waits until it has passed, so a kick per CPU per 10 ms at most", ); @@ -206,7 +206,7 @@ pub fn read(rights: Rights, out: &mut UserBytesMut) -> Result called => break (generation, issued, false), - Some((_, issued)) if now < issued + JOIN.duration() => issued + JOIN.duration(), + Some((_, issued)) if now < issued + INTERVAL.duration() => issued + INTERVAL.duration(), _ => { let generation = ROUNDS.issue(Reach::Narrow); *last = Some((generation, now)); diff --git a/sdk/abi/src/counters.rs b/sdk/abi/src/counters.rs index c6642967cc8..e37e29bf70c 100644 --- a/sdk/abi/src/counters.rs +++ b/sdk/abi/src/counters.rs @@ -4,8 +4,8 @@ //! **A CPU's hardware counters are readable only on that CPU**, so a read asks //! every CPU to copy its counters out and waits for the answers under a bound. //! Every answer is taken after the read was called. Reads close together -//! share one round of asking, and a round is asked at most every 10 ms, so a -//! read may wait up to 10 ms before its round begins. +//! share one round of asking, and rounds are at least [`INTERVAL_NS`] apart, +//! so a read may wait up to 10 ms before its round begins. //! A CPU that has not answered by then is [`Record::stale`]: its values are the //! last it gave, and its [`Counter::Stamp`] says when it gave them. A counter //! the CPU does not have, or that the caller's capability may not read, is @@ -21,6 +21,9 @@ use crate::handle::Rights; +/// The least time between two rounds of asking every CPU, in nanoseconds. +pub const INTERVAL_NS: u64 = 10_000_000; + /// One counter a CPU keeps, by its index in [`Record::values`]. #[derive(Clone, Copy, PartialEq, Eq, Debug)] pub enum Counter { diff --git a/tests/guest/osbench/src/around.rs b/tests/guest/osbench/src/around.rs index 431d6c4f65f..6f3ecbb3413 100644 --- a/tests/guest/osbench/src/around.rs +++ b/tests/guest/osbench/src/around.rs @@ -45,17 +45,31 @@ impl Around { /// One counters read, with when its call began and what it cost. #[cfg(target_os = "toyos")] +#[derive(Clone)] pub struct Reading { pub cpus: Vec, called: std::time::Instant, cost_ns: u64, } +/// The counters now, or this process's last reading where a read now would +/// fall within the kernel's interval between rounds +/// (`toyos_abi::counters::INTERVAL_NS`) of it: that read would sleep out the +/// interval, and osbench would then be the one system that sleeps right +/// before a measured row. One reading then bounds both rows beside it. #[cfg(target_os = "toyos")] pub fn read(d: &crate::sys::diag::Diag) -> Result { + static LAST: std::sync::Mutex> = std::sync::Mutex::new(None); + let mut last = LAST.lock().expect("no reader panics holding the last reading"); + let recent = |r: &&Reading| crate::ns_since(r.called) < r.cost_ns + toyos_abi::counters::INTERVAL_NS; + if let Some(reading) = last.as_ref().filter(recent) { + return Ok(reading.clone()); + } let called = std::time::Instant::now(); let cpus = d.counters()?; - Ok(Reading { cpus, called, cost_ns: crate::ns_since(called) }) + let reading = Reading { cpus, called, cost_ns: crate::ns_since(called) }; + *last = Some(reading.clone()); + Ok(reading) } /// One line a CPU: what it did between `before` and `after`. Every figure @@ -64,8 +78,7 @@ pub fn read(d: &crate::sys::diag::Diag) -> Result { /// ([`osbench_rules::window`]). Core residency counts at the TSC's rate (SDM /// Vol. 4 Table 2-20), the window's own counter; APERF/MPERF is printed as /// the ratio, the one figure of the pair either vendor defines. Deep idle is -/// counted as the periods that ended in the window: the kernel credits a -/// period's time when it ends, so its time is no fraction of the window. +/// counted as the periods that ended in the window. #[cfg(target_os = "toyos")] pub fn cpus(bench: &str, before: &Reading, after: &Reading) { use osbench_rules::window; From da105aba7424e92c257c5e97b4638782f2fc43d6 Mon Sep 17 00:00:00 2001 From: japabu Date: Wed, 28 Oct 2026 11:17:49 +0100 Subject: [PATCH 4/5] issues: the counters row's idle second can hold the EC's second boot query #1084's T14 testcases boot at 7c9cf910f judged FAIL counters: the EC's query 0x4f was taken for the first time 0.87 s into counters_metal's idle second. counters_metal starts the second at the first query, and here the first was 0x40 (12.520 s), not 0x4f, which every earlier testcases readback on hand took first. Independent of #1084: the read before the second took 22 us, and the line is the embedded controller's, which the branch does not touch. The same boot's counters_read and the shared-debug counters_silent passed. Co-Authored-By: Claude Opus 5.5 --- ...cond-can-hold-the-ecs-second-boot-query.md | 30 +++++++++++++++++++ 1 file changed, 30 insertions(+) create mode 100644 issues/the-counters-rows-idle-second-can-hold-the-ecs-second-boot-query.md diff --git a/issues/the-counters-rows-idle-second-can-hold-the-ecs-second-boot-query.md b/issues/the-counters-rows-idle-second-can-hold-the-ecs-second-boot-query.md new file mode 100644 index 00000000000..912fd4cf826 --- /dev/null +++ b/issues/the-counters-rows-idle-second-can-hold-the-ecs-second-boot-query.md @@ -0,0 +1,30 @@ +--- +status: open +kind: defect +opened: 2026-10-28 +--- + +# The `counters` row's idle second can hold the EC's second boot query + +`counters_metal` starts its idle second once acpiserver has said it took +the embedded controller's first query (`acpi_said` in +`tests/guest/rust-tests/src/bin/counters_metal.rs`), and the row reds any +line stamped inside that second. On the T14 the controller raises more than +one query near boot. In every testcases readback on hand the first was 0x4f, +at 13 to 22 s, so the second began after it. In #1084's T14 testcases boot at +`7c9cf910f` (`ToyOSOrg/evidence` branch `pr-1084`, `r3/`) the first was 0x40 +at 12.520 s. The second began at 12.672 s, and 0x4f was taken for the first +time at 13.538 s, inside it: `FAIL counters: the idle second ... holds lines`. +Query 0x40 also comes first in other T14 images' boots (benchcase, +cryptocase, netbenchcase readbacks). + +The machine was not quiet, which the row is right to refuse. But the row's +premise, that the first query ends the controller's boot queries, does not +hold, so its verdict depends on which query the firmware raises first. + +Owner: the `counters` row (`counters_on_metal` in `tests/toyos.rs`, and +`counters_metal`). + +**Exit**: the idle second begins only once the controller's boot queries +are over, by a rule that holds for the `r3/` boot (0x40 then 0x4f), and that +readback judges green under it. From be8e971f8b9d89d1a478a484c86b2d9f6282d1da Mon Sep 17 00:00:00 2001 From: japabu Date: Wed, 28 Oct 2026 11:22:08 +0100 Subject: [PATCH 5/5] issues: the counters-row record states the earlier boots as they read Two of the 32 earlier testcases boots also took 0x40 first; what kept them green was that their idle second began after both queries. Co-Authored-By: Claude Opus 5.5 --- ...cond-can-hold-the-ecs-second-boot-query.md | 25 +++++++++++-------- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/issues/the-counters-rows-idle-second-can-hold-the-ecs-second-boot-query.md b/issues/the-counters-rows-idle-second-can-hold-the-ecs-second-boot-query.md index 912fd4cf826..d27412694f3 100644 --- a/issues/the-counters-rows-idle-second-can-hold-the-ecs-second-boot-query.md +++ b/issues/the-counters-rows-idle-second-can-hold-the-ecs-second-boot-query.md @@ -9,22 +9,25 @@ opened: 2026-10-28 `counters_metal` starts its idle second once acpiserver has said it took the embedded controller's first query (`acpi_said` in `tests/guest/rust-tests/src/bin/counters_metal.rs`), and the row reds any -line stamped inside that second. On the T14 the controller raises more than -one query near boot. In every testcases readback on hand the first was 0x4f, -at 13 to 22 s, so the second began after it. In #1084's T14 testcases boot at -`7c9cf910f` (`ToyOSOrg/evidence` branch `pr-1084`, `r3/`) the first was 0x40 -at 12.520 s. The second began at 12.672 s, and 0x4f was taken for the first -time at 13.538 s, inside it: `FAIL counters: the idle second ... holds lines`. -Query 0x40 also comes first in other T14 images' boots (benchcase, -cryptocase, netbenchcase readbacks). +line stamped inside that second. On the T14 the controller raises 0x4f near +boot, and in some boots 0x40 within about a second of it, in either order +(`r3/prior-testcases.txt` in `ToyOSOrg/evidence` branch `pr-1084`). In the +32 earlier testcases boots listed there, the idle second began after every +query of the boot's first 23 s. #1084's T14 testcases boot at `7c9cf910f` +carried two jobs (`r3/`). Its first query was 0x40 at 12.520 s, its idle +second began 152 ms later at 12.672 s, and 0x4f was taken for the first +time at 13.538 s, inside it: +`FAIL counters: the idle second ... holds lines`. The machine was not quiet, which the row is right to refuse. But the row's premise, that the first query ends the controller's boot queries, does not -hold, so its verdict depends on which query the firmware raises first. +hold. Its verdict depends on how long the boot's other work delays the +second, which passing only in large images hides. Owner: the `counters` row (`counters_on_metal` in `tests/toyos.rs`, and `counters_metal`). **Exit**: the idle second begins only once the controller's boot queries -are over, by a rule that holds for the `r3/` boot (0x40 then 0x4f), and that -readback judges green under it. +are over, by a rule that does not depend on the image's other jobs, and +the `r3/` boot is refused as not quiet before its idle second, rather than +red inside it.