[Japabu]

Draft, LAND by review. The host checks, the models and guest run 43062760497 are green; the metal A/B is owed (below), the benches being down.

Problem

On both metal benches a same-CPU futex ping-pong spends about half its round trip in the run queue. The segment probes of the wakeperf diagnosis (evidence below) read 1,837 of 3,998 probed ns on the T14 and 1,645 of 3,336 on the AMD bench: every park dissolves its job's group and every wake forms it again, each through two EEVDF queues. The finer probes put, per park and wake:

	T14	AMD
EEVDF joins, leaves and charges, both levels	~245 ns	~240 ns
the group map's insert and remove, with the job reference the group drops	~130 ns	~55 ns
the pick (refresh, two picks, take, dispatch)	~90 ns	~65 ns
the parked map's remove at the wake	~100 ns	~55 ns

A finer probe serialises the pipeline around each segment, so these overstate each part; their ranking is what this change acts on.

What changes

Each run-queue operation costs less, and none decides differently.

• A division by a power of two is a shift (eevdf::div, eevdf::div_rem, used by Queue::rebase, issue, enter, lag_of and charge). Every thread weighs 2^16, and so does a group holding all its job's runnable threads, so most of the queue's 128-bit divisions, each a __divti3 call on x86-64, go.
• A lone entity leaves without the general path (Queue::exit): alone it is the whole of the queue's sum, so it takes no lag and leaves the mean at its own virtual time, which is where the general path rebases to. The single-thread group of every ping-pong takes this path at both levels.
• A group holding all its job's runnable threads weighs the job's weight without a division (Job::group_weight).
• The groups a queue holds are a vector in job order rather than a BTreeMap (RunQueue::members, member, enter, leave, refresh, ready, pop_surplus, groups): nothing is allocated, freed or walked through nodes, and every walk keeps its order. A pass already walks every group in refresh, so the vector's linear insert costs a pass no more.
• A task carries its mask's reach (affinity::Reach, CpuMask::reach, Task::reach, set where Task::refresh_affinity and TaskBuilder::build load the mask), and the queue counts pinned and unrestricted tasks by it (RunQueue::count_in, count_out, takers, holds_movable) instead of comparing a 128-byte mask at every entry and pick, which on x86-64 is a memcmp call.

Not changed: the parked map (CpuSched's BTreeMap), which is the wake's largest single item but not the run queue; a hash table there needs a seeded hasher in the core. The job reference a group takes and drops (an atomic each way) stays: the group needs its job's runnable count while its only thread is running, and nothing else holds that count for it.

For the branches that rebase on this one (stopgate #1241, handles3): the functions changed are those named above, in kernel/pure/sched/{eevdf,queue,job,task,affinity}.rs. Nothing in kernel/src/ changes, and cpu.rs does not.

Why no decision changes

kernel/pure/sched/queue/reference.rs is main's run queue and its EEVDF queue at 67713cb, test-only, less the protocol port's escape hatches, the comments and what the test does not call. queue/differential.rs drives both over 300 generated schedules of 2,000 operations: up to four jobs whose threads spread over up to three queues, so group weights are not all powers of two, real-time threads, masks of one, two and every CPU, masks that move while their task is out, inserts with carried standing, picks, charges, yields, put-backs, blocks, surplus answers, removals and refreshes. Every answer, every query of every queue (lengths, takers, weight, groups in order, keys in order, decision instant) and every standing a leaving task takes away must be the same. The reference goes with the test when a change means to decide differently.

Negative controls, each a mutation applied to this head, run and restored (log in the evidence):

mutation	result
div_rem's non-power-of-two remainder halved	red
a lone exit's unfinished request one unit short	red
a group's member stored out of job order	red
a two-CPU mask counted unrestricted	red
the whole-job shortcut taken with one thread elsewhere	red
a mask's reach not refreshed with it	red
the power-of-two division rounded towards zero	red
div_rem's power-of-two remainder halved	green, an equivalent mutant: fine(ns) is a multiple of every power-of-two weight up to 2^48, so that remainder is always zero

Checks

The code the A/B measures is 8750f77's. 9e31612 after it adds eevdf's division test, renames a local index in RunQueue::{enter, leave}, rewords the reference's header and the profile record: no behaviour changes, so the unit tests and clippy ran again there and the mutations and models, run at 8750f77, stand. Logs: https://github.com/ToyOSOrg/evidence/tree/3ddc686338d20b13528b597a7437e3b027479728/checks, and the later ones linked in their rows.

check	why	result
cargo test -p kernel --lib, and with --features sched-check	the touched modules' unit tests, as CI runs them	exit 0 at 9e31612: 458 and 461 passed (log)
cargo clippy -p kernel --lib --tests --features sched-check, and with sched-check,protocol-port and the kernel's controls; -p kernel for x86_64-unknown-none (default, and boot-actuators,test-actuators) and aarch64-unknown-none-softfloat; the adopted lints, -D warnings	the shapes of build/clippy.rs that compile these files	exit 0 each, at 9e31612
the mutations above	the differential test's negative controls	7 red, 1 equivalent (at 8750f77)
div_and_div_rem_are_floor_division, under the mutation the differential test cannot see (the power-of-two remainder halved)	the shift's own control	red (log)
cargo run -- --ci host models	kernel-sim and kernel-loom drive this run queue	run 16:52 to 17:06 UTC, after 8750f77's commit at 16:50:55 UTC, on that tree: exit 0, 79 steps green: kernel-loom's 29 model files and kernel-sim's unit, policy, scenarios, topology and deadline_claim_race tests, and every model's control red as required (log)
guest rows 43062760497: std_threads_count_the_cpus_and_give_back_ended_threads, affinity_pins_hold_on_every_cpu, vector_state_survives_a_switch	threads, sleeps and pins on a real two-CPU scheduler; a smoke, since the differential test and the models hold the decisions	queued

No aarch64 guest row: nothing here is architecture code or ordering, and the pure core's arithmetic and containers are what the host tests drive.

Host microbenchmark, indicative only (Apple M-series under other agents' builds, the kernel's toyos profile, git archive trees of main and 8750f77, runs in the order below, each figure the best of 7 repetitions; scratch code, not committed, in the evidence). Every reading, ns a park and wake:

run	tree	run-queue operations	whole pass, nothing else parked	whole pass, 64 others parked
1	main	122.0	265.9	694.2
2	head	95.7	240.0	254.5
3	head	70.3	295.2	248.6
4	main	117.6	912.2	285.3
5	head	70.7	212.6	209.8
6	main	117.0	260.6	256.8

The run-queue operations read 117 to 122 on main and 70 to 96 at head in every run. The whole pass is noisier: main's three runs read 266, 912 and 261 with nothing else parked, the head's 240, 295 and 213. The host is not a target: its divider and memory system differ, and the metal A/B below is the measurement.

Owed on metal

Both metal benches are down, so metal is no longer a landing condition here. The speed-up is not yet measured on metal. Recorded as owed:

• One A/B of main from before and after this change on the T14 and the AMD bench, with a Linux reference timed through the TSC (issues/a-yield-…, a-two-thread-ping-pong-…, a-small-message-round-trip-…): main must read time.yield and pingpong.futex.same_cpu faster by more than 3% on both, and no scheduler, IPC, wake or spawn row slower.
• Lands on guest run 43062760497 (green at 8750f77). The head after it changes tests, records, one test-only header and a local variable's name in RunQueue::{enter, leave}.

Issue records

issues/a-yield-costs-…, issues/a-two-thread-ping-pong-… and issues/a-small-message-round-trip-… gain where their time goes, from the diagnosis. issues/the-guest-profile-keeps-its-checks-which-cost-under-five-percent.md records the checks' measured cost and that they stay.

Evidence

Branch pr-1260, https://github.com/ToyOSOrg/evidence/tree/3ddc686338d20b13528b597a7437e3b027479728: diagnosis/ holds the probes' readings, the tables and the tools behind the figures above (scout branch wakeperf-scout, pushed, never lands); checks/ holds this PR's check, mutation and microbenchmark logs.

Risks

• The reference is 800 lines of test-only code that freezes main's run queue. It earns its place while the run queue's cost is being worked on (the pass's own periodic work is next); it is deleted with the test by the first change that means to decide differently.
• The vector of groups is linear in the groups a CPU holds per insert and remove. refresh already walks them all at every pick, so a pass is linear in them either way.

🤖 Generated with Claude Code

[Japabu]

Review of #1260 at 86e6c6a (code 8750f77)

Architecture. Ownership and representation stay as they were. Each change is local and exact: a floor shift replaces a power-of-two divide, a lone entity leaves without the general path, the whole-job weight needs no divide, a sorted Vec replaces a BTreeMap that holds a handful of groups, and a cached mask class avoids comparing 128-byte masks. The real cost, a job's group dissolving at every park and forming again at every wake, stays. That is correct under the owner's no-decision-change condition, and the C2 records track it. The manual needs no edit because no decision changes. Net growth: production +78 lines across affinity, eevdf, job, queue and task; tests +1,104 (reference.rs 795, differential.rs 309); issues +65. #1241 and #1261 both merge cleanly with this head (git merge-tree). #1241's task.rs hunks touch the state word and stop bits, not the affinity fields or TaskInner's construction, and #1261 touches none of these files.

1. No decision changes: verified by reading the code

• div and div_rem (eevdf.rs:130-147). For d = 2^k > 0, an arithmetic right shift is ⌊n/d⌋ for every i128 (negatives and i128::MIN included), and n & (d-1) lies in [0, d). These equal div_euclid and rem_euclid, neither side can overflow, and negative lag rounds the same way. Every call site has d > 0:

  • rebase is guarded by total > 0.
  • issue, enter and charge use weights of at least 1, asserted in enter.
  • lag_of returns early when total == 0.

  In charge, main's truncating / and % equal the floor forms because total = fine(ns) + rem ≥ 0.

• Lone exit (eevdf.rs:470-478). sum = Σ w·(v − base) is maintained exactly: by enter, charge, exit and rebase, while issue and set_slice leave v alone. With one entity, total = w, so the general path either rebases base to v or finds base == v already. Its lag is then ⌊w·0/w⌋ = 0, sum and total go to 0, and request_left = (vd − v)·w. Removing a lone root returns NIL without touching the node, and seq is kept. The short path writes the same state. None of the general path's checked products can overflow in this case, so no panic is lost.

• group_weight (job.rs:76). (2^16·all/all).max(1) = 2^16 exactly, and the product stays below 2^48. The case here == all == 0 used to divide by zero; it is now unreachable, because enter passes 1 after job.enter() and a member group always holds at least one entity.

• members. binary_search_by_key on unique sorted keys gives the found index or the insertion point. insert and remove keep the vector sorted, so every walk (refresh, ready and its callers, pop_surplus, groups) runs in BTreeMap key order.

• Reach. affinity is a private field of TaskInner, written only in build (task.rs:1027) and refresh_affinity (task.rs:865), and both set reach from the same mask. Several covers NONE exactly as main's predicates did.

2. The differential test

• Does it compare everything a decision depends on? Yes. After every step it compares every public query of RunQueue. That includes keys in walk order, which exposes tree shape, and decision, which exposes both requests and overtaken_in. It also compares every operation's answer and each leaving task's Left and group_lag. Hidden entity state is compared only through its effects, which is the right bar for this claim.
• Is the generator's coverage adequate? Yes.
  • Masks change while their task is out of every queue.
  • Groups join and dissolve, carry group lag, and go through readmit with carried lag.
  • Group weights vary through splits over up to three CPUs. Job and thread weights are constants in this kernel; eevdf's own tests cover arbitrary weights.
  • Preemption is covered: would_preempt_for runs after every insert onto a busy CPU, and decision is compared after every step.
• Gaps, none material.
  • A fair thread gaining RT while running (the RT branch of put_back) is never generated, because RtState is fixed per spec. That branch's only changed callee is leave, which other operations exercise.
  • A mask changing while its task runs is not generated. It goes through the same refresh_affinity.
• The 8th mutation is equivalent: confirmed. Power-of-two weights are at most MAX_WEIGHT = 2^40, asserted at every enter. fine(ns) is ns·2^48. rem is zeroed at every enter, the only place a weight changes (through readmit). By induction the power-of-two remainder is always 0.
• Mutation log. It records HEAD 3861e0efc, whose tree is identical to 8750f77's (git diff --stat is empty).

3. The frozen reference's lifecycle

It is recorded where it is enforced: in reference.rs's header ("A change that means to decide differently deletes both") and in the PR's Risks. The test's red is the trigger, so it needs no issue record. Apart from comments, I diffed the reference against main 67713cb. It is faithful except that the escape hatches and unused items are gone and Job::group_weight became a local function with the same behaviour.

4. The records

• The C2 figures match diagnosis/tables/fine-split.txt. Per park and wake:
  • EEVDF: 251 and 240.
  • Group map plus drop: 128 and 56.
  • Parked map: 102 and 56.
  • Pick: 89 and 65.
• The checks record's figures match *-compare.md: median speedup 1.028 and 1.048, quartiles 1.00/1.07 and 1.00/1.09, over 47 rows.

Findings

BLOCKER: none.

SUGGESTION

• kernel/pure/sched/task.rs:799 — reach is a second field that two sites must keep in step with affinity. M6 shows that today only a test catches a stale one. A single owned value built from the mask, such as { mask, reach } with one constructor, would make the mismatch unrepresentable.
• kernel/pure/sched/queue.rs:192 — in leave, at is the Place parameter and is shadowed by a usize index. enter's Ok(at)/Err(at) are also indices, while at is a Place everywhere else in the module. Naming the index i would remove the ambiguity.
• kernel/pure/sched/eevdf.rs:130 — once the reference goes, nothing will hold div and div_rem to div_euclid and rem_euclid directly. A unit test of about 15 lines would outlive it: random and extreme i128 numerators, against power-of-two and other positive divisors.
• Keeping the reference. The PR justifies keeping it because "the pass's own periodic work is next". The differential test drives only RunQueue, so item 2 benefits only if it changes RunQueue itself (for example refresh). If item 2 stays in cpu.rs, keeping the 800 lines is speculative. Deleting them after this PR's evidence, and keeping the test above, would cost less. This is the orchestrator's call.
• issues/the-guest-profile-keeps-its-checks-which-cost-under-five-percent.md:9 — say who declined, as the other rejected records do ("Declined by the owner on …"). The brief records keeping the checks as an orchestrator decision, and issues/README.md presents a rejected record as an answer the owner gave.
• kernel/pure/sched/queue/reference.rs:2 — "unchanged but for the protocol port's escape hatches" is true of behaviour, not of the text. "Main's run queue at 67713cb, less the escape hatches and what the test does not call" would be exact.

PR description, for the orchestrator

• "(in 308,328 of the views compared)" appears in no published log. Publish the count or drop it; M1b's red already shows that non-power-of-two weights reach the test.
• The models log carries no revision. Its timestamps (16:52–17:06Z) follow 8750f77's commit (16:50:55Z), and the only later commit changes markdown alone, so it stands. Saying so in the PR would help.
• The host whole-pass figure, "257 to 266 against 210 to 255 in the runs the host left quiet", leaves out three of six base readings and one of six head readings after the fact. It is labelled indicative, but it would be better to show every reading.

Landing preconditions

• Guest run 43062760497. It is still queued at 8750f77. Its rows exercise the scheduler this PR changes, so under the testing policy it must be green before landing.
• Metal A/B rule. The rule is sound for the predicted rows:
  • It was fixed before any boot.
  • The pairs run BH, HB, HB, BH, which cancels drift and order effects.
  • The arms differ by exactly this PR's code: git diff b90157108 03ee87fdb is identical to git diff 67713cbc1 8750f770b.
  • Requiring all four pairs to agree has a one-direction null probability of 1/16, and the 3% floor screens out layout-sized shifts.
• Two weaknesses in the rule.
  • Multiplicity. "No row slower" is tested over about 94 row-and-bench pairs, so a noisy row can be flagged by chance. Say beforehand what a flagged row that was not predicted triggers: report the spread of its four B boots, and count it as a regression only if it lies outside that spread or on a path the change touches.
  • Bimodal rows. pingpong.futex.cross_core read 257 ns on main's 4-boot median (t14-compare.md) against 4,563 on the plain boot, and wake.periodic.idle on the AMD bench read 34,937 against 5,999. Their p50 flips with the mode, not with the change. The first is inside the predicted pingpong.* set. Count such rows neither way, or report them separately.
• What the A/B must show:
  1. Every one of the 16 boots passes its verdict.
  2. No row on a scheduler, IPC, wake, yield or spawn path is counted slower on either bench.
  3. pingpong.futex.same_cpu and time.yield, the rows the diagnosis attributes to the run queue, are counted faster on both benches. If they do not clear 3%, the PR body and the three C2 records' "The run queue's operations cost less and decide as before #1260 makes the run queue's operations cheaper" must state the measured result instead of a gain. A null result alone does not reject an exact change that removes work by construction.

LAND, pending a green guest run 43062760497 and an A/B meeting the three conditions above.

[Japabu]

Orchestrator verification, landing under the benches-down rule (metal unavailable; review + host + guest, metal owed):

• Review LAND. Head 47091b4 vs guest-tested 8750f77: the decided floor-division test (test module only), the frozen reference's header (test-only), records, and one non-test change — the local at renamed i in RunQueue::enter/leave, behaviour-identical (read by me with git diff).
• Guest run 43062760497 at 8750f77: success.
• Owed on metal: the before/after A/B on both benches with a TSC-timed Linux reference, recorded in the yield, ping-pong and small-message records; the speed-up is not yet measured on metal.

[Japabu]

Orchestrator: rebased onto main d17f22c at 817938d; git range-diff shows every commit identical except the record commit, whose only change is keeping #1253's owed bullets beside #1260's paragraph (verified). Guest run 43062760497 still covers the code. Re-queued.