[Japabu]

Problem

The mio fork's ToyOS selector arms a watch for each registration at the start of every select, from a copy of the list it takes under the selector's lock, and then parks in inbox_submit (issues/mios-selector-arms-no-registration-made-while-another-thread-selects.md). A registration that another thread made while that select was parked was not in the copy, so its handle had no watch until the select returned for some other reason. With no timeout and nothing else ready, the select waited for good while the new source's event happened. Upstream's Registry::try_clone exists for exactly that other thread. The registering thread cannot arm the watch itself: a ring has one submitter, so its submit while the select is parked is refused Busy.

What this does

Fork commits 3e070f6 and ed29321 on ToyOSOrg/mio toyos, on top of 4108eff. Both lockfiles that name the fork pin ed29321 (Cargo.lock, tests/guest/rust-tests/Cargo.lock).

• A change wakes the select. The selector owns an event, and every select watches it beside the registrations, under a token no registration's generation takes. A register or reregister made between a select taking its copy and draining what it waited for signals the event, once, under the lock the copy is taken with. The parked select wakes and arms its registrations again from a fresh copy. It takes the signal under that copy's lock, so any later signal is a change the copy does not hold. A deregister needs no wake: the drain already drops a completion whose registration is gone.
• A select returns only with an event or at its timeout. A wake that leaves nothing to report arms the registrations again and waits out what is left of the timeout. That covers a change, and a completion for a registration that has since changed. A select with nothing registered now waits too. It used to return at once whatever its timeout, which upstream's selectors do not do. A timeout past what an Instant holds waits for good, and a timeout's nanoseconds no longer wrap.
• A refused registration watch is reported, and ends its registration (ed29321, review r1). The kernel refuses a watch it will never fire: a handle closed while it was watched, one held without the right to wait on it, or an interest its object has no watch for. The drain used to drop that refusal silently. Under the new loop it would have been armed again and refused again, spinning until the timeout. Now it is reported once as an error (event::is_error, which was false for every event before) in the directions the registration reports, so whoever reads or writes the source meets the refusal. No later select arms it again. A refused watch on the selector's own event fails the select by name.
• One event type. The selector's change event and a waker's event are one type with one constructor, so one SAFETY argument covers both. Its take is as loud as its signal.
• New guest test mio_registers_while_parked. The main thread polls with no timeout. A second thread waits until the kernel's roster shows the main thread blocked, past the flag it raises just before polling. It then registers a Waker through a cloned registry and wakes it. The poll must return that waker's token. If it returns without it, or never returns, the second thread ends the process by name past a 10 s ceiling, so the shared boot is not wedged. The poll is parked with nothing else registered: a Poll holds one Waker (upstream's debug assertion), and the testcases boot runs no netstack, so a waker is the one source it can make ready.
• Records.
  • The issue record is closed. Its owed metal run of mio_waker_coalesces (A poll belongs to the handle it was made through, and a ring to its one submitter #1261) is part of the testcases boot of main that issues/nothing-fails-when-a-devices-release-or-close-stops-answering-its-polls.md already owes whole, with "every other shared member that polls passing". That boot now also holds mio_registers_while_parked.
  • New: issues/mios-selector-fails-every-select-past-63-registrations.md, found while reading and not fixed here. The ring has 64 submission slots and each select queues every watch in one submit, so past 63 registrations, the watch on the change event among the 64, every select fails. A TCP stream registered for both directions takes two registrations, so a tokio runtime with its waker and one listener fails at its 31st stream.
  • New: issues/a-fork-registered-in-local-toml-dirties-the-nested-lockfiles-and-stops-the-build.md, met while building this and recorded, not fixed. A fork registered in .cargo/local.toml, as the implementer rules say, writes [[patch.unused]] into the rust/ and sdk/libc lockfiles, and the build then refuses rust/src/bootstrap as dirty.

Why a guest test

The selector is ToyOS-only code over the kernel's inbox. What it must establish is the kernel's parked submit waking for the selector's own event, and two threads ordered by the kernel's own report of one of them blocked. No host build has that. QEMU establishes order and completion, which is the whole claim. The test is a member of the testcases shared boot, x86-64 only, as every member is. It runs on the guest row testcases_members_exit_0 (QEMU x86-64 q35, KVM on the guest runner) and on the T14's testcases boot.

Checks

Base: main 3c5c55584. Head: 924b72173, on a macOS arm64 host unless it says otherwise.

• Guest row testcases_members_exit_0 at 924b72173: run 43163983980. Queued; its verdict is added here.
• Negative control of the mechanism (review r1, B1): branch mio-registration-wakes-selector-control-b1 at 95301017f. It is the head with the fork at ed29321 vendored under tests/guest/rust-tests/mio-control, in which SelectorInner::change signals nothing, and the guest crate takes mio from that copy. The empty poll still waits. Vendoring keeps the fork's toyos branch the only branch on the fork. Run 43163987153, dispatched beside the head's, was red as predicted: 1 of testcases's 242 member(s) did not exit 0: test_rs_mio_registers_while_parked, exit 1, with a waker registered and woken while the poll was parked was not its event within 10s 10 s after the member started. Every other member exited 0. So the change signal is what delivers the late registration; the empty-poll wait alone does not. The control branch was deleted after the run, and its two patches are in the evidence.
• The guest test crate: mio_registers_while_parked and mio_waker_coalesces built as build_toyos_bin builds them (cargo build --target x86_64-unknown-toyos --profile toyos, sysroot 0dd1ef87d639f652, the toolchain's clang). Exit 0, with no warnings, at 924b72173, with mio fetched from the fork at ed29321. The fork was type-checked for the target before the push, also exit 0.
• Host, at 924b72173: cargo test --lib -- sourcegate passes, 9 tests, over the new and changed records. cargo test -p toyos-net-node passes; its host tests take mio from the fork at ed29321, and their unix selector is unchanged.
• Round 1, at f9e3135d2: cargo run -- --ci host exit 1, 1 of 123 steps red. That one is the_caller_s_environment_reaches_neither_the_build_nor_the_key, issues/the-llvm-environment-test-breaks-git-on-macos.md, red on every macOS host, in a file this branch does not touch. cargo run -- --build-only exit 0. What changed since is the fork pin and three records; the two host checks above cover what those can change.
• Round 1 guest runs: 43151307989 at f48024a32, green with 242 members. The whole-change control 43151340044, with the fork pinned back to 4108eff, was red on test_rs_mio_registers_while_parked alone, through the base selector's empty-poll early return.

Evidence on pr-1276: round 2's checks and the control's mutation, the .cargo/local.toml reproduction, and round 1's host logs and guest runs.

Risks

• A change costs at most one signal per select. It is one event_signal while a select is parked, and one event_take at the next copy. A register made while no select waits costs nothing more.
• An empty wait now parks. A caller that polled an empty Poll with no timeout to return at once now waits. That is upstream's meaning, and tokio always registers its waker first.
• A refused registration is dropped after it is reported. A caller that goes on using the source meets the refusal in its own read or write, as with an error event on Linux. A reregister makes it a registration again.

Owed on metal

Nothing new. The testcases boot of main is already owed on the T14 (see above), and mio_registers_while_parked is a member of it.

🤖 Generated with Claude Code

[Japabu]

Review r1 of #1276 at f9e3135

Architecture

The selector is the right owner, and an event it signals itself (the self-pipe pattern) is the right mechanism. It needs no kernel change and respects the one-submitter ring. selecting and signalled keep a registration made when no select is waiting to no syscall at all. An empty poll now waits for its timeout, which matches upstream's contract.

The fork rules are met:

• 3e070f6 fast-forwards toyos from 4108eff, and its message gives the rationale.
• Both lockfiles that name the fork are pinned. sdk/libc/Cargo.lock and the tls-* lockfiles have no mio.
• toyos-abi is still a dependency by version.

Net change: fork selector.rs +118/−30; ToyOS +64 test lines, +22 for the new record, −36 for the closed one. The branch merges cleanly with main at 302fb36, and Endowments::take and SYSCAP_LABEL are unchanged there.

Races I traced, with no lost or duplicated wake found:

• change() signals and to_watch() takes, both under the selector lock, and signalled mirrors the event's state.
• The CHANGED watch is armed after the take, and the kernel's arm looks at readiness first, so a signal between the take and the arm fires the new watch at once.
• Each submit's watch on a handle replaces the earlier one (Polls::admit), so CHANGED polls do not pile up across iterations.
• Completions are written only inside submit, so none appear between a drain and the next copy.
• A registration made between the drain and the next copy is in that copy.

Evidence checked:

• Run 43151307989 at f48024a: 242 members.
• Control 43151340044 at e29225f: red only on test_rs_mio_registers_while_parked at the assertion.
• Both host logs: the recorded macOS red only.

Findings

B1. Evidence: no run shows the test red on the recorded defect itself.
The control (mio back at 4108eff) goes red because the base selector returns at once from an empty select. The PR says this honestly, but it is the other behaviour change in the same fork commit. So no red so far separates "the registration signal wakes a parked select" from "an empty select now waits".

The test's own red path for the defect, a poll that never returns followed by the second thread's exit(1) past CEILING, has never run. The new loop also re-arms from a fresh copy on any empty return. By reading, nothing else returns a parked submit here: min_complete 1, no deadline, no kill, no dropped completion. Only a run confirms that the signal is what delivers the late registration.

Cheapest sufficient check: one testcases_members_exit_0 row whose guest crate builds the fork at 3e070f6 with change()'s event_signal removed. Use either a disposable control branch whose [patch] points at a copy carrying that one-line mutation, or a disposable fork ref, whichever fits the fork rules. Expected: mio_registers_while_parked exits 1 by name after 10 s, and every other member exits 0. It costs about one runner slot, about 15 minutes, the same as the control already run.

B2. forks/mio src/sys/toyos/selector.rs:264-269, 302-303 (3e070f6): a refused registration watch now turns select into a busy loop.
A refusal completion (result < 1) is dropped without a word: a handle closed without a deregister, a handle without WAIT, or an interest the object has no watch for. A ToyOS TcpStream has no Drop that deregisters its two bells.

Before, that made select return at once with nothing, which was wrong but visible. Now events is empty, so the loop re-arms, the kernel refuses again at once, and the select spins at 100 % CPU until its timeout. With None, it spins for good unless another source fires. The loop's comment ("arm them again, and wait out the timeout") and the PR body ("waits out what is left of the timeout") are false for this case.

The spin is not new in kind, but the new loop now hides it inside a wait that claims to wait. Fix: report the refused registration, for example as an event for which is_error answers true (the ToyOS event::is_error is hard-wired false today), or fail the select by name as the change already does for CHANGED. If that is out of scope, record it with an exit in issues/ and correct the comment and the PR body.

B3. issues/mios-selector-fails-every-select-past-63-registrations.md:16: the stated consequence is wrong by about half.
"A tokio runtime on ToyOS cannot serve a 64th socket" is incorrect. A ToyOS TcpStream registered for both directions takes two registrations, its receive bell and its bell (src/net/tcp/toyos_stream.rs register). Beside the runtime's waker and a listener, the 31st stream already takes the total past 63. State the consequence in registrations, or correct the count.

S1. OwnedEvent repeats Waker's wait/signal pair, its event_create call and its unsafe construction. Waker could hold an OwnedEvent, so one constructor carries the one SAFETY argument.

S2. selector.rs:191. to_watch ignores event_take's result (let _), while change() propagates event_signal's error, and both comments say the call is never refused. A take that did fail would leave the event signalled with signalled == false, and every later select would spin without a word. Make both calls loud (expect), consistent with "never refused".

Notes for the orchestrator

These are not findings against this branch.

• The .cargo/local.toml fork registration that dirties the nested lockfiles has published evidence (evidence a3ad7b1, pr-1276/local-toml/) but no record in issues/.
• The closed record's metal run of mio_waker_coalesces is folded into nothing-fails-when-…'s "every other shared member that polls passing". That works, because 3e070f6 contains 4108eff.

SEND BACK (B1, B2, B3)