The kernel declares each CPU's HWP request, and the counters round reads the power envelope back under TRACE

issues/kernel/the-kernel-owns-cpu-performance-state.md

@@ -0,0 +1,43 @@
+---
+status: open
+kind: track
+opened: 2026-09-28
+---
+
+# The kernel owns CPU performance state
+
+The self-hosting bar is measured under a fixed power envelope that must be
+read back for a whole build span. The kernel declares the envelope from the
+one CPU-state declaration (`kernel/src/arch/x86_64/control_regs.rs`) and
+`SYS_COUNTERS` reads it back per CPU, under `Rights::TRACE`.
+
+**A register reaches a read only once boot has proven it**: the kernel has no
+`rdmsr` fault fixup, so a register a read is the first to touch is a kernel
+`#GP` a holder of the right can cause. Each register is enumerated by CPUID
+or read at boot by the declaration's own check before a read can reach it.
+
+Declared and read back: the HWP request, its package's and the energy/
+performance bias, on every CPU that has all of them
+(`toyos_cpuvuln::hwp`). Still to build:
+
+- **RAPL**: PL1, PL2 and their windows through `MSR_PKG_POWER_LIMIT` and the
+  MMIO mirror in the host bridge's MCHBAR, at the bar's values; a limit
+  firmware locked (bit 63) is refused by name. Neither register is
+  enumerated by CPUID. *Exit*: the T14 reads both back at the bar's values.
+- **Turbo**: `IA32_MISC_ENABLE` bit 38 is firmware's. Its other bits are
+  model-specific, so declaring the one bit needs the owner's ruling on
+  writing that register whole. *Exit*: the ruling, and the bit declared or
+  recorded as firmware's.
+- **The sampler**: a program that reads the envelope every 60 s and at a
+  span's start and end, with package power from `MSR_PKG_ENERGY_STATUS` and
+  temperature from `IA32_PACKAGE_THERM_STATUS` and `MSR_TEMPERATURE_TARGET`,
+  the bar's validity conditions. The energy counter is a side channel
+  (CVE-2020-8694), under `Rights::TRACE` like the rest. *Exit*: one valid
+  span on the T14.
+
+Not covered: hybrid Intel and AMD CPPC,
+`issues/kernel/the-perf-state-declaration-refuses-hybrid-intel-and-amd-cppc.md`;
+AArch64 declares nothing. No QEMU CPU enumerates HWP, so every write and read
+of these registers runs only on the T14.
+
+Owner: the orchestrator, which holds the T14.

issues/kernel/the-perf-state-declaration-refuses-hybrid-intel-and-amd-cppc.md

@@ -0,0 +1,31 @@
+---
+status: open
+kind: defect
+opened: 2026-09-29
+---
+
+# The perf-state declaration refuses hybrid Intel and AMD CPPC
+
+Every Intel client CPU since Alder Lake is hybrid, and AMD CPUs name their
+performance controls through CPPC, not HWP. On both the kernel declares no
+performance request, firmware's values stand, and `SYS_COUNTERS` carries no
+power envelope.
+
+The refusals are `toyos_cpuvuln::hwp`'s (`toyos-cpuvuln/src/hwp.rs`):
+- `HwpRefusal::Hybrid`, for CPUID.(7,0):EDX[15]: the declared minimum is a
+  ratio (`MSR_PLATFORM_INFO[47:40]`), and a hybrid CPU's HWP scale is not its
+  ratio scale.
+- `HwpRefusal::NoHwp`, for CPUID.6:EAX[7] clear, which is where an AMD CPU
+  with CPPC is refused; its controls are not read.
+- `HwpRefusal::NotIntel`, for HWP on any vendor but Intel, because the
+  minimum comes from `MSR_PLATFORM_INFO`, which is Intel's.
+
+`control_regs::hwp_declared` logs the refusal once and declares nothing on
+any CPU.
+
+Owner: the orchestrator. **Exit**: on a hybrid Intel CPU each core type's
+request is declared on that core's own HWP scale; on an AMD CPU with CPPC,
+`MSR_AMD_CPPC_ENABLE` and `MSR_AMD_CPPC_REQ` are declared from its
+`MSR_AMD_CPPC_CAP1` and asserted on every CPU; and on one machine of each
+kind the `counters` metal row holds the envelope it reads back to the
+declaration.

issues/kernel/the-scheduler-and-the-clocks-never-talk.md

@@ -21,6 +21,8 @@ Two couplings every production scheduler has and ours lacks:
 
 Both wait for an AML interpreter
 (`issues/kernel/toyos-runs-the-machine-in-acpi-mode-and-interprets-its-aml.md`),
-because today the kernel neither reads nor sets P-states or C-states on metal.
+because the kernel reads no C-state on metal and chooses no P-state: it
+declares HWP's bounds (`kernel/src/arch/x86_64/control_regs.rs`), and the CPU
+chooses within them.
 Filed so the placement track (see `placement-is-blind-to-caches-and-topology.md`)
 leaves room for an idle-state input when it designs wake placement.

kernel/src/arch/aarch64/counters.rs

@@ -1,10 +1,11 @@
 //! No model-specific counter is read on this architecture: the activity
-//! monitors' cycle counters (`FEAT_AMUv1`) are unread.
+//! monitors' cycle counters (`FEAT_AMUv1`) are unread, and no performance
+//! request is declared.
 
 use crate::counters::Hardware;
 
 pub fn bring_up() {}
 
 pub fn read() -> Hardware {
-    Hardware { smi: None, aperf: None, mperf: None }
+    Hardware { smi: None, aperf: None, mperf: None, envelope: None }
 }

kernel/src/arch/x86_64/boot.rs

@@ -119,8 +119,7 @@ pub fn clock(args: &KernelArgs) {
 /// and the TSC is time since power-on.
 pub fn report_counter_origin() {
     const IA32_TSC_ADJUST: u32 = 0x3b;
-    // Leaf 7 exists when the maximum leaf reaches it.
-    if super::cpu::cpuid(0, 0).0 < 7 || super::cpu::cpuid(7, 0).1 & (1 << 1) == 0 {
+    if super::cpu::leaf_7().1 & (1 << 1) == 0 {
         log!("boot: IA32_TSC_ADJUST not on this CPU");
         return;
     }

kernel/src/arch/x86_64/control_regs.rs

@@ -1,11 +1,22 @@
-//! What `CR0`, `CR4` and `IA32_EFER` hold on every CPU in this machine. One
-//! declaration, applied by the BSP and every AP and checked on each; nothing
-//! else may write any of the three. Each register is written whole: `CR0`
-//! and `EFER` are constants, `CR4` is required bits plus whatever optional
-//! bits this CPU offers. `EFER.NXE` lets bit 63 of a paging entry mean *not
-//! executable* ([`Prot`](crate::mm::policy::Prot)).
-
-use core::sync::atomic::{AtomicU64, Ordering};
+//! What `CR0`, `CR4`, `IA32_EFER` and the performance request hold on every
+//! CPU in this machine. One declaration, applied by the BSP and every AP and
+//! checked on each; nothing else may write any of them. Each register is
+//! written whole: `CR0` and `EFER` are constants, `CR4` is required bits plus
+//! whatever optional bits this CPU offers. `EFER.NXE` lets bit 63 of a paging
+//! entry mean *not executable* ([`Prot`](crate::mm::policy::Prot)).
+//!
+//! The performance request is HWP's: `IA32_HWP_INTERRUPT` where it exists,
+//! `IA32_PM_ENABLE`, `IA32_HWP_REQUEST` as `toyos_cpuvuln::hwp_request` derives
+//! it from this CPU's own registers, `IA32_HWP_REQUEST_PKG` and
+//! `IA32_ENERGY_PERF_BIAS`, on a machine whose CPUs have every one of them,
+//! and none on one that does not: refused by name once, firmware's values
+//! standing. `IA32_MISC_ENABLE`'s turbo bit is not declared: that register's
+//! other bits are model-specific and firmware's, and writing it whole would
+//! decide them.
+
+use core::sync::atomic::{AtomicU64, AtomicU8, Ordering};
+
+use toyos_cpuvuln::HwpFacts;
 
 use super::cpu;
 use crate::log;
@@ -47,6 +58,33 @@ mod cr4 {
     pub const SMAP: u64 = 1 << 21;
 }
 
+/// The performance request's registers, SDM Vol. 4.
+mod hwp {
+    pub const PLATFORM_INFO: u32 = 0xCE;
+    pub const ENERGY_PERF_BIAS: u32 = 0x1B0;
+    pub const PM_ENABLE: u32 = 0x770;
+    pub const HWP_CAPABILITIES: u32 = 0x771;
+    pub const HWP_REQUEST_PKG: u32 = 0x772;
+    pub const HWP_INTERRUPT: u32 = 0x773;
+    pub const HWP_REQUEST: u32 = 0x774;
+}
+
+/// `IA32_PM_ENABLE` on every CPU: HWP on, which only a reset turns off.
+const PM_ENABLE: u64 = 1;
+
+/// `IA32_HWP_INTERRUPT` on every CPU that has it: nothing in this kernel takes
+/// an HWP notification.
+const HWP_INTERRUPT: u64 = 0;
+
+/// `IA32_HWP_REQUEST_PKG` on every CPU: the widest range at the request's own
+/// preference. Inert, since no CPU's request sets package control, and
+/// declared so that firmware does not decide it either.
+const HWP_REQUEST_PKG: u64 = 0x01 | 0xff << 8 | toyos_cpuvuln::HWP_EPP << 24;
+
+/// `IA32_ENERGY_PERF_BIAS` on every CPU: Linux's `ENERGY_PERF_BIAS_NORMAL`
+/// (`arch/x86/include/asm/msr-index.h:858`).
+const ENERGY_PERF_BIAS: u64 = 6;
+
 /// `CR0` on every CPU: `TS` stays clear because lazy FP switching would leak
 /// a register file across `#NM`; `AM` stays clear because Ring 3 has no `#AC` path.
 pub const CR0: u64 = cr0::PE | cr0::MP | cr0::ET | cr0::NE | cr0::WP | cr0::PG;
@@ -106,8 +144,45 @@ pub fn init_cr0(cpu_id: u32) {
     bench::report(cpu_id, before);
 }
 
-/// Puts this CPU's `CR4` and `EFER` into the declaration and checks all
-/// three against it. Must run after [`init_cr0`] and before `arch::syscall::init`, which needs `SCE` set.
+/// Whether the machine's declaration carries the performance request: the
+/// BSP's verdict, which every AP must reach too.
+static HWP: AtomicU8 = AtomicU8::new(HWP_UNDECIDED);
+const HWP_UNDECIDED: u8 = 0;
+const HWP_DECLARED: u8 = 1;
+const HWP_REFUSED: u8 = 2;
+
+/// Whether this CPU gets a performance request, from CPUID alone, and whether
+/// it has `IA32_HWP_INTERRUPT`: the machine's answer, a CPU that disagrees
+/// named.
+fn hwp_declared(cpu_id: u32) -> Option<toyos_cpuvuln::Hwp> {
+    let (cpuid_6_eax, cpuid_6_ecx) = cpu::leaf_6();
+    let verdict = toyos_cpuvuln::hwp(&HwpFacts {
+        vendor: cpu::vendor(),
+        signature: cpu::cpuid(1, 0).0,
+        cpuid_6_eax,
+        cpuid_6_ecx,
+        cpuid_7_0_edx: cpu::leaf_7().3,
+    });
+    let mine = if verdict.is_ok() { HWP_DECLARED } else { HWP_REFUSED };
+    match HWP.compare_exchange(HWP_UNDECIDED, mine, Ordering::Release, Ordering::Acquire) {
+        Ok(_) => {
+            if let Err(refusal) = verdict {
+                log!("control_regs: no performance request is declared: {}", refusal.reason());
+            }
+        }
+        Err(machine) => assert!(
+            machine == mine,
+            "control_regs: cpu{cpu_id} {} a performance request and the BSP {} one",
+            if mine == HWP_DECLARED { "can hold" } else { "cannot hold" },
+            if machine == HWP_DECLARED { "declared" } else { "refused" },
+        ),
+    }
+    verdict.ok()
+}
+
+/// Puts this CPU's `CR4` and `EFER` into the declaration and checks both
+/// against it. Must run after [`init_cr0`] and before `arch::syscall::init`,
+/// which needs `SCE` set.
 pub fn init(cpu_id: u32) {
     let declared = declaration(cpu_id);
     if !skipped(cpu_id) {
@@ -128,6 +203,47 @@ pub fn init(cpu_id: u32) {
     self_check(cpu_id, declared);
 }
 
+/// Puts this CPU's performance request into the declaration and checks it.
+/// Must run after [`init`] and after this CPU's IDT is loaded, so a fault in
+/// its `wrmsr` or `rdmsr` panics by name.
+///
+/// The interrupt is cleared and HWP enabled before any other HWP register is
+/// touched, the request's input included: SDM Vol. 3B §17.4.2 (253669-093US),
+/// "Additional MSRs associated with HWP may only be accessed after HWP is
+/// enabled, with the exception of IA32_HWP_INTERRUPT and MSR_PPERF", and
+/// `intel_pstate_hwp_enable`'s order.
+pub fn init_performance(cpu_id: u32) {
+    let Some(hwp) = hwp_declared(cpu_id) else { return };
+    // SAFETY: `hwp_declared` admitted this CPU only where CPUID enumerates HWP,
+    // and `IA32_HWP_INTERRUPT` where it enumerates that too; both values hold
+    // only defined bits.
+    unsafe {
+        if hwp.notify {
+            cpu::wrmsr(hwp::HWP_INTERRUPT, HWP_INTERRUPT);
+        }
+        cpu::wrmsr(hwp::PM_ENABLE, PM_ENABLE);
+    }
+    let request = toyos_cpuvuln::hwp_request(cpu::rdmsr(hwp::HWP_CAPABILITIES), cpu::rdmsr(hwp::PLATFORM_INFO));
+    // SAFETY: HWP is enabled, so its registers are live; CPUID enumerated EPP,
+    // the package request and EPB, and every value holds only defined bits.
+    unsafe {
+        cpu::wrmsr(hwp::HWP_REQUEST, request);
+        cpu::wrmsr(hwp::HWP_REQUEST_PKG, HWP_REQUEST_PKG);
+        cpu::wrmsr(hwp::ENERGY_PERF_BIAS, ENERGY_PERF_BIAS);
+    }
+    hwp_check(cpu_id, request, hwp.notify);
+}
+
+/// The power envelope this CPU runs under, where the performance request is
+/// declared: read only after this CPU's [`init_performance`], which enabled HWP.
+pub fn envelope() -> Option<crate::counters::Envelope> {
+    (HWP.load(Ordering::Acquire) == HWP_DECLARED).then(|| crate::counters::Envelope {
+        hwp_request: cpu::rdmsr(hwp::HWP_REQUEST),
+        hwp_request_pkg: cpu::rdmsr(hwp::HWP_REQUEST_PKG),
+        energy_perf_bias: cpu::rdmsr(hwp::ENERGY_PERF_BIAS),
+    })
+}
+
 /// Whether the declaration carries `PCIDE`, and therefore whether `INVPCID` is this machine's flush.
 pub fn pcid_active() -> bool {
     DECLARED_CR4.load(Ordering::Acquire) & cr4::PCIDE != 0
@@ -205,11 +321,8 @@ fn supported() -> u64 {
         [(0, cr4::FSGSBASE), (7, cr4::SMEP), (20, cr4::SMAP)];
     const CPUID_7_ECX: [(u32, u64); 1] = [(2, cr4::UMIP)];
 
-    let (max_leaf, _, _, _) = cpu::cpuid(0, 0);
     let (_, _, ecx1, edx1) = cpu::cpuid(1, 0);
-    // A leaf above the maximum answers with the highest basic leaf's data
-    // instead of faulting, so an unguarded read here would misreport bits.
-    let (_, ebx7, ecx7, _) = if max_leaf >= 7 { cpu::cpuid(7, 0) } else { (0, 0, 0, 0) };
+    let (_, ebx7, ecx7, _) = cpu::leaf_7();
 
     let mut have = 0;
     for (bit, flag) in CPUID_1_EDX {
@@ -272,6 +385,42 @@ fn self_check(cpu_id: u32, declared_cr4: u64) {
     CHECKED.fetch_add(1, Ordering::Relaxed);
 }
 
+/// [`self_check`] for the performance request, logged first for the same
+/// reason; the line carries the request's two inputs, so a reader can
+/// recompute it.
+fn hwp_check(cpu_id: u32, request: u64, notify: bool) {
+    let pm_enable = cpu::rdmsr(hwp::PM_ENABLE);
+    let live = cpu::rdmsr(hwp::HWP_REQUEST);
+    let pkg = cpu::rdmsr(hwp::HWP_REQUEST_PKG);
+    let epb = cpu::rdmsr(hwp::ENERGY_PERF_BIAS);
+    let interrupt = notify.then(|| cpu::rdmsr(hwp::HWP_INTERRUPT));
+    log!(
+        "control_regs: cpu{} pm_enable={} hwp_request={:#010x} hwp_request_pkg={:#010x} epb={} \
+         hwp_interrupt={:x?} hwp_capabilities={:#010x} platform_info={:#018x}",
+        cpu_id,
+        pm_enable,
+        live,
+        pkg,
+        epb,
+        interrupt,
+        cpu::rdmsr(hwp::HWP_CAPABILITIES),
+        cpu::rdmsr(hwp::PLATFORM_INFO),
+    );
+    for (name, holds, declared) in [
+        ("pm_enable", Some(pm_enable), PM_ENABLE),
+        ("hwp_request", Some(live), request),
+        ("hwp_request_pkg", Some(pkg), HWP_REQUEST_PKG),
+        ("epb", Some(epb), ENERGY_PERF_BIAS),
+        ("hwp_interrupt", interrupt, HWP_INTERRUPT),
+    ] {
+        let Some(holds) = holds else { continue };
+        assert!(
+            holds == declared,
+            "control_regs: cpu{cpu_id} holds {name}={holds:#x}, the declaration is {declared:#x}",
+        );
+    }
+}
+
 /// How many CPUs hold the declaration, said once after the last of them has
 /// been checked. A divergent CPU panics inside [`self_check`], so what this
 /// line adds is the *count*: a CPU that never reached [`init`] at all is

kernel/src/arch/x86_64/counters.rs

@@ -1,9 +1,10 @@
 //! Which of this CPU's model-specific counters it reads: decided on the CPU
-//! itself at bring-up, by `toyos_cpuvuln::counters` over its own CPUID.
+//! itself at bring-up, by `toyos_cpuvuln::counters` over its own CPUID, and
+//! the power envelope where `control_regs` declared it.
 
 use core::sync::atomic::{AtomicU8, Ordering::Relaxed};
 
-use toyos_cpuvuln::{CounterFacts, Vendor};
+use toyos_cpuvuln::CounterFacts;
 
 use super::{cpu, percpu};
 use crate::counters::Hardware;
@@ -20,13 +21,10 @@ const APERF_MPERF: u8 = 1 << 1;
 static ADMITTED: [AtomicU8; MAX_CPUS] = [const { AtomicU8::new(0) }; MAX_CPUS];
 
 pub fn bring_up() {
-    let (max_leaf, ebx, ecx, edx) = cpu::cpuid(0, 0);
-    let id: [u8; 12] = core::array::from_fn(|i| [ebx, edx, ecx][i / 4].to_le_bytes()[i % 4]);
     let (signature, _, cpuid_1_ecx, _) = cpu::cpuid(1, 0);
-    // As `init_scattered_cpuid_features` reads it: a level past the range's own is no level.
-    let cpuid_6_ecx = if (6..=0xFFFF).contains(&max_leaf) { cpu::cpuid(6, 0).2 } else { 0 };
+    let (_, cpuid_6_ecx) = cpu::leaf_6();
     let verdict = toyos_cpuvuln::counters(&CounterFacts {
-        vendor: Vendor::from_id(&id),
+        vendor: cpu::vendor(),
         signature,
         cpuid_1_ecx,
         cpuid_6_ecx,
@@ -44,5 +42,6 @@ pub fn read() -> Hardware {
         smi: msr(SMI, MSR_SMI_COUNT).map(|count| count & 0xFFFF_FFFF),
         aperf: msr(APERF_MPERF, IA32_APERF),
         mperf: msr(APERF_MPERF, IA32_MPERF),
+        envelope: super::control_regs::envelope(),
     }
 }