[Japabu]

Problem

SYS_FUTEX_WAIT translated its word under the space's lock, and then wait_until's predicate translated it
again, under the same lock and with an Arc clone of the current space, on every check: before arming,
after arming, after each wake and at the deadline. wakeperf's segments put that translation at 60 to 63 ns,
taken two to four times per wake round trip, inside body_end:futex_wait of about 185 to 205 ns.

Resulting behavior

• One translation per call (Space::translate -> Translation), which also reads the space's new count
  of leaf changes (Usage::shot_down) in the same hold of the lock.
• Every change to a present leaf is counted: AddressSpace::shoot_down increments the count first, under
  the lock, before the TLB shootdown, the futex revoke and the free. build's unwinding and discard, which
  called the tables' shootdown directly, now go through it, so their shootdowns also count in the test
  kernel's SPACE_REACHED (which only munmap deltas read).
• The predicate is a load of the word through the direct map and Translation::holds: an acquire fence
  and a load of the count. Unchanged, the address still maps the frame and the frame is not freed. Moved,
  one hold of the lock asks whether the address still maps the frame — the check every call made before —
  and takes the new count if it does. So a change elsewhere in the space (another range's munmap,
  mprotect) neither ends the wait early nor turns its timeout into a wake.
• futex::watch_of and scheduler::futex_wake take the physical address instead of a direct-map pointer.

What keeps the key valid

Unchanged from before: an unmap ends the waits on the frames it frees (futex::revoke_frames, under the
bucket locks) after it counts the change, and a waiter arms under its bucket's lock before it checks. Either
the revoke finds the waiter armed and ends it, or the arm came after the revoke, so the waiter's check
happens after the count moved and the lock-held look finds the address no longer mapping the frame. Reading
a frame freed meanwhile through the direct map is what the previous code also did between its check and its
read; at worst it is a spurious return, which the ABI allows. No loom model is added: the ordering is the
bucket lock's and the space lock's, which their own models cover; the count is only read and written under
or after them.

Not done

• A lock-free page-table walk for the one translation. Unsafe on AArch64 as things stand: its shootdown is a
  broadcast TLBI with no IPI, so nothing waits for a walker before tables are freed; it would need deferred
  table freeing.
• No park counter is added (the brief asked to check benchrows first; this change adds none).

Checks

Targeted per the testing policy. kernel/src has no host tests.

check	result
at 2583d7de8: cargo clippy -p kernel with build/clippy.rs's adopted lints, -D warnings, x86_64-unknown-none: default, boot-actuators,test-actuators, the instrument set	exit 0 each
the same, aarch64-unknown-none-softfloat: default, boot-actuators,test-actuators	exit 0 each
at bb5ae5397 (review r1: one SAFETY comment): cargo check -p kernel --target x86_64-unknown-none	exit 0

Guest rows:

• The first dispatch, 43058887998, ended at No test matches filter "futex_wake_counts" before building a
  guest: six of its eight names are shared-boot members, which guest rows do not select. It established
  nothing.
• 43064655857 at 2583d7de8, green: std_threads_count, libc_children, affinity_pins, vector_state,
  and on AArch64 (mm::space sits beneath it) virt_smp, virt_unmap_touch. The head differs from
  2583d7de8 in one SAFETY comment and an issue record only.

Owed on metal

Both benches are down, so this lands on its review, host checks and guest rows. The speed-up is not yet
measured on metal. issues/a-two-thread-ping-pong-costs-toyos-up-to-twice-linuxs-on-the-t14.md records
what is owed:

• the T14's whole testcases, testcases-watchdog and shared-debug boots of main after this PR, every
  member passing, among them futex_wake_counts (its orphaned_by_unmap arm is the revoke race),
  kill_ends_every_wait, exit_wait_storm, std_threading, std_threads, shm_release_reclaims and
  tlb_shootdown_waits (which reads SPACE_REACHED, which discard and build's unwinding now also count);
• an A/B of main before and after this PR, on the T14 and the AMD bench (osbench sched, time, files,
  ipc; four boots a side, rotated; exact two-sided Mann-Whitney over per-boot p50s), and Linux's
  osbench sched on each bench. Claimed: pingpong.futex.same_cpu and pingpong.futex.cross_core lower or
  not separated, the latter only from boots whose round trips parked; no other row 5% or more worse unless
  this change explains it.

Not taken (review r1's optional suggestion 2): a shared-boot member asserting that a timed wait beside a
sibling's mmap/munmap churn answers 1. The ABI allows a spurious 0, so it would hold a quality of the answer
rather than a contract.

Risks

• A process with heavy mmap/munmap churn pays the lock-held look once per check after each change, which
  is what every check cost before; between changes, a check is a load.

🤖 Generated with Claude Code

[Japabu]

Review r1 of #1253 at 2583d7d

Architecture. A wait now translates its word once and validates that translation against a per-space count of leaf changes, kept under the space's own lock. The count lives in Usage, the space's existing lock-free face, and Translation keeps the space alive for the length of the wait. No lock, park counter or new waiting mechanism is added. +87/-55 in 5 files. It merges cleanly with main. #1211 changes process.rs and scheduler.rs elsewhere (kernel stacks, an import), and none of the futex or space code.

Correctness is sound.

• Coverage of the count.
  • Every change to a present leaf goes through AddressSpace::shoot_down: protect, discard, clear (reached from munmap and release_object), mremap's move, and build's unwinding. The last two are newly counted.
  • Held -> Mapped and Empty -> Mapped change no present leaf.
  • Drop runs only once no Translation holds the space.
• ABA cases.
  • Frame freed and reused. If the same VA now maps a frame with the same number, the slow path answers what a fresh translation would, so waiting on is correct. If the address maps anything else, the wait returns Changed, a spurious return the ABI allows.
  • Mapping changed and changed back (protect None then RW, or a discard whose pinned frame is kept in place). The count moved, the slow path finds the same frame, and the wait goes on correctly.
  • COW break. ToyOS has none. A Copy range takes its private frame at the first touch of any kind, and futex_word's Read reach is such a touch.
• Lost wakes: none new.
  • Every path that frees a frame counts the change before revoke_frames, and arming and revoking take the same bucket lock.
  • A waiter armed first is revoked. One armed afterwards sees the count through that lock on its post-arm check, and the slow path finds the address gone.
  • A pre-arm check that reads a stale count only goes on to arm and check again.
• Memory ordering of the count. It is incremented under the space lock, Relaxed. Readers read it under that lock, or after fence(Acquire), whose only job is to tie the word's value to the count it is validated against. The key's validity rests on the bucket and space locks, which existing models cover, so I agree no loom model is needed.
• The draft's defect is fixed. An unrelated change sends holds to the slow path once, which refreshes at, so a timeout is no longer reported as a wake.
• A read after a revoke. Every check after a revoke now reads the freed frame through the direct map before holds; the old code checked the translation first. This is a harmless read of RAM: the only frames outside the direct map are Every kernel stack sits above unmapped memory, and an overflow names its stack #1211's permanent bring-up guards, and those are never user frames.

Blockers

1. Guest evidence does not exist.
   • Run 43058887998 ended at No test matches filter "futex_wake_counts" (exit 1, 16:32:30) before building a guest.
   • Six of the eight names are shared-boot members. Guest rows select only MACHINE_TESTS and SCREEN_TESTS, so those six run only in the metal boots.
   • The PR's guest list is therefore false; correcting the body is the orchestrator's.
   • mm::space is memory management shared beneath AArch64, so dispatch virt_unmap_touch virt_smp, the two valid names.

Suggestions

1. kernel/src/scheduler.rs:262: the SAFETY comment says the direct map "maps every frame". Main now has holes in it (Every kernel stack sits above unmapped memory, and an overflow names its stack #1211's bring-up guards). Saying "every frame a user page can hold" keeps the stated reason true after the merge.

2. Nothing tests the fix to the draft. A shared-boot member could:

   • make a 50 ms timed futex_wait on one thread;
   • mmap and munmap another range in a loop on a sibling thread until the wait returns;
   • assert the answer is 1.

   That would catch a later change to holds that brings the bug back. QEMU can run it, because it asserts the answer, not the time. It is optional, since the ABI tolerates a spurious answer.

Evidence

• logs/f1/diff.patch matches this head's diff against f79df95 line for line. The clippy runs cover x86-64 default, actuators and instruments, and AArch64 default and actuators, each exit 0. The change touches no feature-gated code beyond the actuators' reached counter, which those runs cover.

What metal must show

• T14 Part 1 green at this head. In particular:
  • futex_wake_counts, whose orphaned_by_unmap arm is the revoke race;
  • kill_ends_every_wait, exit_wait_storm, std_threading, std_threads, shm_release_reclaims;
  • shared-debug's tlb_shootdown_waits, which reads SPACE_REACHED, a counter discard and build's unwinding now also increment.
• A/B: record how the futex ping-pong moved; no control row worse by 5% or more. On the A/B rule itself, see A clock read takes RDTSCP and holds nothing after it, and std's Instant is the page's nanoseconds #1240's review.

SEND BACK. This is for the evidence only; no code change is requested. It can land once the virt_unmap_touch virt_smp dispatch is green and the metal results above are recorded.

[Japabu]

Response to review r1, head bb5ae53.

• Guest evidence: the first dispatch named shared-boot members and established nothing (the body now says so). The orchestrator's 43064655857 carries virt_smp and virt_unmap_touch with four x86-64 rows.
• Suggestion 1 taken: the SAFETY comment says the direct map maps every frame a user page can hold (cargo check -p kernel exit 0). The queued metal boots at 2583d7d stand; this commit changes only that comment.
• Suggestion 2 not taken: the ABI allows a spurious 0, so the test would hold a quality of the answer rather than a contract, and it would change the testcases image already queued for this head.

🤖 Generated with Claude Code

[Japabu]

Orchestrator: verified bb5ae53 against review r1. git diff 2583d7de8 bb5ae5397 changes only the SAFETY comment in kernel/src/scheduler.rs ("every frame a user page can hold", which is accurate after #1211's guard pages). No code changed, so the queued T14 boots at 2583d7d stay valid. Landing needs guest run 43064655857 (std_threads_count libc_children affinity_pins vector_state virt_smp virt_unmap_touch) green, the T14 testcases/shared-debug boots green (especially futex_wake_counts, kill_ends_every_wait, the std threading tests and tlb_shootdown_waits), and the A/B's futex ping-pong recorded.

[Japabu]

Orchestrator verification, landing under the benches-down rule (metal unavailable; review + host + guest, metal owed):

• Review LAND. Head 1ec47d7 vs guest-tested 2583d7d: one SAFETY comment in kernel/src/scheduler.rs (no executable line) + one record.
• Guest run 43064655857 at 2583d7d: success.
• Owed on metal recorded in issues/a-two-thread-ping-pong-costs-toyos-up-to-twice-linuxs-on-the-t14.md; the speed-up is not yet measured on metal.