Skip to content

A poll belongs to the handle it was made through, and a ring to its one submitter - #1261

Merged
Japabu merged 5 commits into
mainfrom
handles3-poll-via
Nov 12, 2026
Merged

Japabu merged 5 commits into
mainfrom
handles3-poll-via

Conversation

@Japabu

@Japabu Japabu commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

A ring kept its polls by handle, but an object's watch kept its entries by nothing. A close could therefore end every ring's poll on the object or none, and close_ends_polls chose which wrong answer each object kind got:

  • closing a duplicate of a pipe end, an event's wait side or an acceptor ended the other handle's polls in every ring;
  • closing the handle a Process, SysCap, Job or claim watch was made through left that watch armed until its object posted;
  • two threads could submit to one ring and race its last completion slot, and a ring's close woke neither of them;
  • mio's selector reported an event for a handle after its deregister, and toyos::Poller was Sync over a two-step tail move.

Resulting behaviour (the design approved in the design poll; decisions settled 2026-11-11)

  watch entry  =  (ring, Via { table, handle })        HandleEntry.polled = a poll was made through it here
  register     =  one hold of the table: handle still there?  add entries  :  end the poll
  leave table  =  remove (close, endow) / committed transfer (send)  ->  Watch::end_rings(via)
  • A poll belongs to the handle it was made through. Invariant P2: an entry is on a watch only while its handle is in its table. Closing or moving that handle answers its polls NotFound. Closing another handle to the same object answers nothing. A refused move ends nothing. close_ends_polls, its per-kind table, the three list locks every close took, the acceptor's cancel at its last handle, and the kernel/CLAUDE.md caveat that pointed at it are all deleted. cancel_rings stays for a source that ends under live handles, such as a claim's release.
  • A ring has one submitter, by type. inbox_submit claims the ring with one exchange and holds Submitting until it returns. A second concurrent submit is refused with a new ABI error, Busy (SyscallError 11; std ResourceBusy, libc EBUSY). The completions are reachable only through Submitting, so the room an answer was read for is the room it is written into. An answer no longer posts the ring's watch, and Kept::looking is gone. Closing a ring wakes its submitter, which returns NotFound.
  • toyos::Poller is Send and no longer Sync.
  • mio (ToyOSOrg/mio toyos at 4108eff, a fast-forward of the pinned fe42def): a registration carries a generation. The drain drops a completion whose registration is gone or was made again, under the lock deregister takes. No cancel op is added to the ABI.
  • The one-shot, the polls, delivery, the park predicate, registration and teardown move to kernel/pure/sched/poll.rs. kernel-loom already compiles that path, and its by-path copies of inbox/once.rs and inbox/polls.rs are gone.
  • ring_park_herd is now 256 threads, each parked in submit on a ring of its own and woken by an event each round. It remains mask_windows' load (decision 1).
  • Manual: architecture/kernel.md's "One way to wait" states both rules.

Records

  • Closed: a-close-of-one-handle-ends-every-rings-poll-on-its-object and its duplicate closing-a-pipe-end-ends-every-holders-polls-on-it, closed together; a-watch-outlives-the-close-of-the-process-handle-it-was-made-through; mio-deregister-fd-leaves-a-pending-poll-live; closing-a-ring-leaves-its-submitters-parked; two-submitters-of-one-ring-race-its-last-completion-slot; toyos-poller-is-sync-and-a-watch-moves-the-tail-in-two-steps.
  • New: mios-selector-arms-no-registration-made-while-another-thread-selects (decision 3; found by reading, not fixed here).
  • Narrowed and renamed: a-process-lengthens-…-on-one-ring becomes a-claims-holder-lengthens-its-handlers-interrupts-off-walk-by-the-polls-it-arms. Its first bullet, N threads on one ring, can no longer happen. The small-kernel track's stage 6 step 2 says what the herd now is.
  • Updated: an-irq-watchs-freeing-cancel-compiles-in-a-handler (end_polls frees as cancel_polls does), a-second-poller-on-a-windows-handle-stalled-the-prompt (its two related records are fixed), and nothing-fails-when-a-devices-release-or-close-stops-answering-its-polls. That last record names the new close path, WatchRef::end_polls's Claim arm, and its exit is restated for it (review round 2).
  • Not addressed: a-second-poller-on-a-windows-handle-stalled-the-prompt (cause unknown) and the-two-watch-poll-model-counts-a-dropped-watchs-answer-as-its-completion.

Checks

The owner's testing policy applies: the touched crates' tests and clippy shapes, the loom models and controls this change touches, and the image. Runs 3 and 4 were at f79df9574 plus this change's tree. Run 4 reran everything that edits after run 3 could change: one model rewritten, redundant clones removed, and the pure tests' ring counter. Those edits cannot change run 3's other results.

check why exit
cargo test -p kernel --lib --features sched-check (run 4) the pure scheduler with sched::poll's new tests and watch_list's end_rings test; 462 before the edits 0
cargo test -p kernel-loom --tests --no-run (run 3), then --test loom_watch --test inbox_answer --test poll_once (run 4) every model compiles; the touched models pass: 15, 14 and 2 0
controls, as build/ci.rs runs them: poll-fire-load-store (poll_once; loom_watch's four, the new close model among them, run 4), post-is-an-answer, commit-ignores-notify, register-outside-table, admit-keeps-the-replaced, teardown-posts-nothing (run 3) each reds exactly its named tests, with the message its CI row names 101 each, as required
cargo test -p kernel-sim --no-run the simulator's Ring gains via 0
cargo test -p toyos-build --lib (run 3) the control table against the manifests, the source gates, the issue-record gates 101, one red: llvm::tests::the_caller_s_environment_reaches_neither_the_build_nor_the_key, which is issues/the-llvm-environment-test-breaks-git-on-macos.md. The same red on PR #1239's tree, with none of this change's build edits (run 4). 383 passed
cargo test --test toyos-checks (run 4) the harness's own checks: a guest test added and one deleted 0
cargo test -p toyos --lib poller the SDK's poller 0
clippy in CI's shapes: kernel host library (run 4), x86-64 default and with boot-actuators,test-actuators, kernel-loom/kernel-sim/toyos-build --all-targets (run 4), toyos --all-targets, toyos-abi lint on what changed 0 each
cargo run -- --build-only (run 3) the kernel, std with Busy, libc, every image program 0
metal staging at f897ae424 (round 1) compiles every guest binary, the new test and the herd among them, and mio at 4108eff: the build's dep-info names checkouts/mio-…/4108eff/src/sys/toyos/selector.rs 2: staged, machine untouched

I did not run aarch64 locally: the change has no arch code. The shared watch beneath both architectures goes to the guest runner's virt_jobs_at_el2.

Guest. This change's guest claims are poll_belongs_to_its_handle and the reshaped ring_park_herd. Both are shared-boot members on metal, not QEMU rows, so no guest row runs them. With the benches down they are owed on metal (below). QEMU rows that exercise polls through the SDK, libc, netstack and the compositor, and one AArch64 row for the shared watch, are dispatched as one batch. Run 43064052517 at dd1d57622, 5 passed of 5 (session_lifecycle, stream_holders, libc_children, virt_jobs_at_el2, netstack_holders_exported_to_children; log). The later rounds change nothing these rows build.

Evidence (ToyOSOrg/evidence pr-1261 at f29103ba): index, run 3 (summary), run 4 (summary), metal staging and request.

At the current head 39f9da686, rebased onto #1246: guest run 43108631616 covers it at that head. It runs the same five rows, virt_jobs_at_el2 among them for the shared watch beneath both architectures. Round 1's one code change, the guest tests' mio pin, reaches only mio_waker_coalesces, a metal member. The result follows when it arrives.

Owed on metal (the benches are down; metal is no longer a landing condition). Each item is recorded in a record this PR updates:

  • Functional boots (issues/nothing-fails-when-a-devices-release-or-close-stops-answering-its-polls.md):
    • Rows: the testcases, testcases-watchdog and shared-debug boots of main, whole, on the T14.
    • Main must show poll_belongs_to_its_handle's four progress lines and : ok, pipe_other_end_gone passing, and every other shared member that polls passing.
  • mio (issues/mios-selector-arms-no-registration-made-while-another-thread-selects.md): mio_waker_coalesces in main's testcases boot on the T14, passing, built from mio 4108eff.
  • The herd (issues/a-claims-holder-lengthens-its-handlers-interrupts-off-walk-by-the-polls-it-arms.md): mask_windows, the windowscase boot of main on the T14. Main must show it passing and test_rs_ring_park_herd passed, which also shows the job admits the 256 rings.
  • The poll path's cost is not measured on metal. The design poll named osbench's wake, pipe and IPC rows as the check (recorded in the devices' poll record). The comparison is the T14's benchcase boot of main from before this change against main from after it, two boots a side, in the order B, H, H, B. A row is slower only if all its head p50s lie above all its base p50s over four boots a side. The first two a side are a screen, and a flagged row gets a second round. Each row is reported as its after/before ratio beside both sides' ranges.

The images staged for the queued boots are deleted. The request (r2) stays as the record of what was asked.

Review round 1 (f897ae424)

  • Blocker 1, fixed. tests/guest/rust-tests keeps its own lockfile, which still pinned mio at fe42def. So my earlier claim that staging compiled 4108eff was false: the first staging built the old selector, and the fix shipped in no ToyOS binary. That lockfile now pins 4108eff. The re-staged build's dep-info names 4108eff's selector, and mio_waker_coalesces was relinked after it (dep-info). Every lockfile that names the fork now pins 4108eff.
  • Blocker 2, fixed. system/netstack/src/watches.rs and sdk/toyos/tests/transport_loom.rs now cite kernel/pure/sched/poll.rs. issues/parallel-tests-red-under-other-suites.md no longer names the deleted gate: what it held is structural now.
  • Re-staged for the T14 together with the hot-path A/B, before the benches went down (the images are now deleted) (evidence, index). Comment, record and lockfile edits change no host check, so the host evidence above stands.

Review round 2 (8e8b85256)

  • Rebased onto main 45935f4b9. Main deleted issues/parallel-tests-red-under-other-suites.md, and the deletion is kept, so round 1's edit to it is gone, along with that sentence of its commit message. git range-diff shows the main commit unchanged. No code changed, so the images staged at f897ae424 stay valid.
  • Record restated: issues/nothing-fails-when-a-devices-release-or-close-stops-answering-its-polls.md now names the close path this change made: ops::end_polls, WatchRef::end_polls's Claim arm, then DeviceClaim::end_polls, then pcidev/isa::end_polls. It also names the release that still answers live polls, tear_down_departed. Its exit asks for a test of each path, red with that call removed. The audio claim it named no longer exists.
  • Evidence: the empty lock-check log is replaced by the check rerun with its exit (r2/lock-check-rust-tests.log). It resolves mio at 4108eff, exit 0.
  • A/B rule restated before any boot ran; it is now part of the owed list above.

Rebase onto #1246 (06fcb81ea)

Risks

🤖 Generated with Claude Code

@Japabu

Japabu commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

Review of #1261 at dd1d57622

Architecture

This matches design-polls.md and the three 2026-11-11 decisions. A registration is now (table, handle) on both sides. Invariant P2 is enforced at the one owner of "in or out of a table", which is HandleTable::remove and a committed transfer, so a close no longer has to choose per kind. With that, close_ends_polls, its per-kind table and the acceptor's cancel go. One submitter per ring, enforced by the Submitting type, removes Kept::looking, the wake-every-submitter on each answer, and the model of two submitters.

The decision logic now lives in kernel/pure under host tests and loom. Production size is roughly flat: poll.rs replaces once.rs and polls.rs, the handle table grows by 78 lines, and ops and inbox shrink. Tests grow by four loom models, six host tests and one guest test, and the herd is reshaped. The deleted test and model covered scenarios that can no longer happen.

Races traced

  • Close vs post: the Once compare-exchange decides between them. If the post wins, the look resolves a retired handle and answers NotFound; if the end wins, the answer is NotFound without a look. Either way the poll is answered once.
  • Close vs registration: while_held and remove both run under the process table lock. A RawHandle's generation means "still in the table" implies the same entry and therefore the same object.
  • Send-away vs wake: the entries are taken out, committed, retired and ended in one hold. A refused move restores polled. A look after the commit finds no handle and answers NotFound.
  • Replacing watch vs post: admit withdraws and removes the old poll under the ring's lock. Only the submitter changes the list, so the expect in looked_at holds as a real invariant.
  • Teardown vs park: the state is taken before the post, so awake reads None.
  • Spawn endowment: it moves handles with remove (kernel/src/loader/start.rs:186-190), so endowing ends their polls; A child's handles are a list its parent names: labels and namespaces leave the kernel #1252 keeps that.
  • Lock order: a claim's watch is now reached from a close as well as from a registration, as table → claim Held → IrqWatch. Before, a close never reached a claim (unreachable!). I found no path that takes these locks in the reverse order.

The two departures from the design

Both are sound.

  • With one entry per handle in Polls, a split Once cannot produce two answers to a replaced watch: deliver answers only kept polls, once each. admit-keeps-the-replaced is the planted mistake that matters there.
  • The close-vs-post model now spans two watches because one watch's list lock serializes a post and a close on that watch. A real poll stands on up to three watches. Run 4's poll-fire-load-store reds it with "a post and a close both took the poll", and each new control reds its own model with its message (I read the run 3 and run 4 logs).

Findings

BLOCKER

  1. tests/guest/rust-tests/Cargo.lock:466-468 — this lockfile still pins mio at fe42def, so no ToyOS binary carries the mio fix.

    • Problem: tests/guest/rust-tests is excluded from the workspace (Cargo.toml:196) and keeps its own lockfile. It is the only ToyOS consumer of mio, through mio_waker_coalesces. The root Cargo.lock bump reaches only toyos-net-node's host dev-dependency, where src/sys/toyos is not compiled.
    • Consequence: the fork fix ships in no ToyOS binary, and mio-deregister-fd-leaves-a-pending-poll-live is closed with its fix unused. The fork rule that every changed consumed fork commit lands with every lockfile naming it is broken. The check-table row "metal staging … compiles … mio at 4108eff" is false.
    • Evidence: the staged build's dep-info, wt-b/tests/guest/rust-tests/target/x86_64-unknown-toyos/toyos/deps/mio-a8e64fdb4dc81efd.d, was written at 18:59:41, during staging and after the 18:57:27 commit. It lists ~/.cargo/git/checkouts/mio-c4440d7c13afb477/fe42def/src/sys/toyos/selector.rs.
    • Fix: run cargo update -p mio in tests/guest/rust-tests, compile it for ToyOS, re-stage the metal images before the T14 B run (staging touches no machine), and correct the PR text. mio_waker_coalesces then drives the rewritten drain's registration, generation and waker path in the shared boot.
  2. Stale references to deleted code.

    • system/netstack/src/watches.rs:6, :10 and sdk/toyos/tests/transport_loom.rs:296 cite kernel/src/inbox/polls.rs, which this PR deletes; the code now lives in kernel/pure/sched/poll.rs.
    • issues/parallel-tests-red-under-other-suites.md:482-486 says sourcegate's a_capability_closing_ends_no_log_poll keeps close_ends_polls's SysCap arm at false. This PR deletes both. That property is now structural: a close ends only the entries made through its own handle.

    Comment and record edits only, so nothing needs rerunning.

Judgements requested

Landing conditions

  • Guest run 43064052517 was still queued when I reviewed. All five rows must pass. The fixes above do not touch what those rows build.
  • Metal, after re-staging:
    • testcases, testcases-watchdog and shared-debug green as whole boots, including poll_belongs_to_its_handle: ok with its four progress lines, pipe_other_end_gone, mio_waker_coalesces (then on 4108eff) and every other poll user;
    • windowscase with mask_windows passing and test_rs_ring_park_herd passed ("256 threads parked in submit, each on its own ring, woken 64 times"). Each of the 256 rings charges inbox::heap() to the job. By my reading that is about 180 KiB per ring (an estimate), roughly 45 MiB more than the old one-ring herd, so this boot is also the check that the job admits them.

SUGGESTION

Verdict

SEND BACK for blockers 1 and 2, both small. After the fixes and a re-stage: LAND pending the guest run and metal results above.

Japabu added a commit that referenced this pull request Nov 11, 2026
… names the deleted poll files

`tests/guest/rust-tests` is outside the workspace and keeps its own
lockfile, which still pinned mio at `fe42def`. It is the one ToyOS consumer of
mio (`mio_waker_coalesces`), so the selector's generation drop shipped in no
ToyOS binary. Its lockfile now pins `4108eff`, as the root lockfile does;
`cargo metadata --locked` agrees with both.

`system/netstack/src/watches.rs` and `sdk/toyos/tests/transport_loom.rs`
cited `kernel/src/inbox/polls.rs`, now `kernel/pure/sched/poll.rs`.
`issues/parallel-tests-red-under-other-suites.md` named the source gate and
the `close_ends_polls` arm this change deleted. What the gate held is now
structural: a close ends only the polls made through the handle it closes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

Review round 2 of #1261 at f897ae424

Architecture

Round 1 changes one lockfile line, two comments and one record (+7/−9). The design assessment from round 1 stands.

Round 1's blockers

  • Blocker 1: resolved.
    • tests/guest/rust-tests/Cargo.lock:468 and the root Cargo.lock:1764 both pin 4108eff. No other lockfile names the fork.
    • fe42def...4108eff is one commit ahead and changes only src/sys/toyos/selector.rs, so the lock's dependency list correctly stays the same. The fork's toyos head is 4108eff.
    • The re-staged build's dep-info (r1/mio-a8e64fdb4dc81efd.d) lists checkouts/mio-…/4108eff/src/sys/toyos/selector.rs. It was written at 19:43:08 local time (17:43 UTC in r1/stage-head.log, exit 2 at f897ae424), after the commit at 19:42:50.
    • The PR body now admits the earlier false claim.
    • One gap in the evidence: r1/lock-check-rust-tests.log is 0 bytes, yet the INDEX calls it cargo metadata --locked, exit 0. The claim does not need it, because the reading, the fork comparison and the dep-info establish it. The INDEX should stop citing an empty log.
  • Blocker 2: resolved.

Findings

BLOCKER

  1. issues/nothing-fails-when-a-devices-release-or-close-stops-answering-its-polls.md:16-17, :23, :26-27: this assigned record names WatchRef::cancel_polls's Irq arm (kernel/src/object/ops.rs:263) as the close site for a claim, and its exit and evidence are written about that arm. It belongs to the same class as round 1's blocker 2, and round 1 missed it.
    • Problem: this PR deletes WatchRef::cancel_polls. At base, a claim's close reached no watch at all (Claim(_) => unreachable!). With this PR it does:

      • HandleTable::remove
      • ops::end_polls
      • WatchRef::end_polls's Claim arm (ops.rs:156)
      • DeviceClaim::end_polls (kernel/src/object/device.rs:137)
      • pcidev::end_polls or isa::end_polls

      No test closes a polled claim. poll_belongs_to_its_handle covers pipes, events and processes. So this PR's new path is exactly what the record calls nothing failing.

    • Consequence: the small-kernel track's stage 6 step 5 reads this record's exit, and that exit names code that no longer exists.

    • Fix: name the close site as WatchRef::end_polls's Claim arm, and restate the evidence and the exit for it. For example: a poll made through a claim and closed with it answers NotFound, and the test reds with that arm made {}. Add the record to the PR's "Updated" list. This is a record edit only, and nothing reruns.

Hot-path A/B rule (asked)

Not adequate as a verdict at 2 boots a side, though B,H,H,B is right. The order keeps a monotone drift from putting both head boots above both base boots.

  • The problem is chance. If head changes nothing, the chance that both head p50s lie above both base p50s is 1/C(4,2) = 1/6 for each row.
  • About three rows would be flagged even for an identical image. The rule's patterns match 18 rows on ToyOS: 5 ipc.*, 2 pingpong.pipe.*, and 11 wake.* (periodic.idle, periodic.loaded, onto_busy, and storm.{call,drain} at w1, w4, w16 and w64). If the rows were independent, "no row slower" would happen only about 4% of the time ((5/6)^18). Rows within one boot are correlated, which raises that figure, but a flagged row is still weak evidence.
  • The repo's own comparison could not resolve these either. tests/guest/osbench/compare resolves a difference only at p ≤ 0.05 by the exact Mann-Whitney test. At 2 v 2 its smallest two-sided p is 1/3. The README says overlapping runs are "unresolved, not equal".

Restate the rule before the T14 runs; the images need no re-staging:

  • the four queued boots are a screen;
  • a row whose two head p50s both lie above both base p50s is a candidate, and about 3 are expected by chance;
  • the candidates get a second B,H,H,B round;
  • a row is slower only if all 4 head p50s lie above all 4 base p50s (exact p = 2/70 two-sided, which the compare tool calls resolved);
  • each row is reported as its head/base median ratio beside both arms' boot-to-boot ranges;
  • a row with no flag is reported as "no difference resolved", not as "no cost".

Round 1 did not make this A/B a landing condition, and it still is not one. A single flag under the current rule must not by itself send the PR back.

Evidence

  • Guest run 43064052517: completed/success at headSha dd1d57622, 5 passed of 5 (session_lifecycle, stream_holders, libc_children, virt_jobs_at_el2, netstack_holders_exported_to_children). Round 1 changes nothing those rows build. Only mio_waker_coalesces, a metal member, links mio. The guest condition is met.

Landing conditions: metal, from the T14 images staged at f897ae424

  • testcases, testcases-watchdog and shared-debug green as whole boots. They must include poll_belongs_to_its_handle: ok with its four progress lines, pipe_other_end_gone, mio_waker_coalesces (now built from 4108eff) and every other poll user.
  • windowscase with mask_windows passing and test_rs_ring_park_herd passed ("256 threads parked in submit, each on its own ring, woken 64 times"). That also shows the job admits the 256 rings.
  • Rebase onto main, keeping the deletion of issues/parallel-tests-red-under-other-suites.md.

Verdict

SEND BACK for the record edit, which needs no rerun. After it: LAND pending the metal results above. The A/B is evidence, not a gate.

Japabu added a commit that referenced this pull request Nov 11, 2026
… names the deleted poll files

`tests/guest/rust-tests` is outside the workspace and keeps its own
lockfile, which still pinned mio at `fe42def`. It is the one ToyOS consumer of
mio (`mio_waker_coalesces`), so the selector's generation drop shipped in no
ToyOS binary. Its lockfile now pins `4108eff`, as the root lockfile does;
`cargo metadata --locked` agrees with both.

`system/netstack/src/watches.rs` and `sdk/toyos/tests/transport_loom.rs`
cited `kernel/src/inbox/polls.rs`, now `kernel/pure/sched/poll.rs`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu added a commit that referenced this pull request Nov 11, 2026
…s change made

`issues/nothing-fails-when-a-devices-release-or-close-stops-answering-its-polls.md`
named `WatchRef::cancel_polls`'s `Irq` arm as a claim's close site. This
change deleted that arm. A claim's close or move now ends the polls made
through its handle, by `ops::end_polls` and `WatchRef::end_polls`'s `Claim`
arm. The record now says so. It also says which release still answers a live
poll: `tear_down_departed`, because a release after the last close finds none.
Its exit is restated for both: a test for each path, red with its call
removed. The record's audio claim no longer exists and leaves it.

Rebased onto main 45935f4, which deleted
`issues/parallel-tests-red-under-other-suites.md`; the deletion is kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

Orchestrator: verified 8e8b852 against follow-up r2. The rebase onto 45935f4 keeps main's deletion of issues/parallel-tests-red-under-other-suites.md. git range-diff shows commit 1 identical and commit 2 changed only by dropping that record's hunk and the sentence about it, so no code moved and the images staged at f897ae4 stand. Commit 3 edits only issues/nothing-fails-when-a-devices-release-or-close-stops-answering-its-polls.md, restated for the new close path. The A/B rule was restated as a screen before any boot. Guest 43064052517: 5 of 5. Lands on the T14 boots (testcases, testcases-watchdog, shared-debug, windowscase) passing with the criteria in r2.

Japabu added a commit that referenced this pull request Nov 11, 2026
…e poll-path A/B, recorded while the benches are down

The benches are down, so metal is no longer a landing condition. What only
metal shows of this change is recorded as owed, each in a record this change
already updates:
- `issues/nothing-fails-when-a-devices-release-or-close-stops-answering-its-polls.md`:
  the T14's `testcases`, `testcases-watchdog` and `shared-debug` boots of
  main, with `poll_belongs_to_its_handle` and `pipe_other_end_gone`; and the
  osbench A/B of main from before and after this change, over the wake, pipe
  and IPC rows, by the rule stated before any boot ran.
- `issues/mios-selector-arms-no-registration-made-while-another-thread-selects.md`:
  `mio_waker_coalesces`, built from mio `4108eff`.
- `issues/a-claims-holder-lengthens-its-handlers-interrupts-off-walk-by-the-polls-it-arms.md`:
  the `windowscase` boot, with `mask_windows` and the reshaped herd.

Record edits only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu and others added 5 commits November 12, 2026 00:01
…ne submitter

A ring kept its polls by handle, but an object's watch kept its entries by
nothing. So a close could end every ring's poll on the object or none, and
`close_ends_polls` chose which of the two wrong answers each object kind got.
Closing a duplicate of a pipe end, an event's wait side or an acceptor ended
the other handle's polls in every ring. Closing the handle a `Process`,
`SysCap`, `Job` or claim watch was made through left that watch armed until
its object posted. Two threads could submit to one ring, race its last
completion slot, and park on a ring whose close woke neither.

A poll and its watch entries now carry `Via { table, handle }`: the handle
table it was looked up in, and the handle there. A handle's entry carries one
bit, `polled`. The registration adds the entries in one hold of the table
that reads the handle still there (`poll::register`, through
`HandleTable::while_held`); a handle that left first ends the poll. Every way
out of the table ends the entries made through that handle and no other's:
`remove`, which is a close and an endowment, and a committed `transfer`,
which is a send. A refused move ends nothing. `Watch::end_rings(via)` takes
the matching entries out a few at a time and fires and frees them with the
list lock let go. `close_ends_polls`, its per-kind table and the three list
locks every close took are gone. So is the acceptor's cancel at its last
handle, which no entry can outlive any more. `cancel_rings` stays for a
source that ends under live handles: a claim's release, a function leaving.

A ring has one submitter by type. `inbox_submit` claims the ring with one
exchange and holds a `Submitting` until it returns. A second concurrent submit
is refused with a new ABI error, `Busy`, and touches nothing. The completions
are reached only through `Submitting`, so the room an answer was read for is
the room it is written into. An answer no longer posts the ring's watch, and
`Kept::looking` is gone with the look's hiding of a poll from a second
submitter. The ring's teardown posts its watch after taking the state, so a
submitter parked on it reads the ring gone and returns `NotFound`. `Poller`
loses `unsafe impl Sync`.

The one-shot, the polls, delivery, the park predicate, the registration and
the teardown move from `kernel/src/inbox/` to `kernel/pure/sched/poll.rs`,
which `kernel-loom` already compiles. They get host tests of the transition
table, replacement by key, the room read and written, and the park predicate
over a ring gone. Four new loom models, each with a control that must red it:
a close racing a post (the split `Once`), a close racing a registration
(`register-outside-table`), a replacing watch racing a post
(`admit-keeps-the-replaced`), and a teardown racing a park
(`teardown-posts-nothing`). The two-submitter model and the two
`inbox_answer` cases that staged a second submitter's watch during a look are
deleted with what they modelled.

The mio fork's selector gives each registration a generation, which its
watch's token carries, and its drain drops a completion whose registration is
gone or was made again. So no event is reported for a handle after its
deregister returned. The fork is pinned at that commit.

`poll_belongs_to_its_handle` replaces `inbox_cancel_wakes`, whose claim this
inverts. It shows that a duplicate's close answers nothing on a pipe end and on
an event's wait side, and that the watched handle's own close answers its
parked submitter `NotFound`. It shows that a child watched through a duplicate
is answered at that duplicate's close, while the child lives, and that a
watched handle sent away is answered. And it shows that a second submit is
refused `Busy` and a ring closed under an unbounded submit returns it.
`ring_park_herd` parks its 256 threads each on a ring of its own and wakes
them by events: one ring can no longer hold more than one.

Closes `issues/a-close-of-one-handle-ends-every-rings-poll-on-its-object.md`
and its duplicate `issues/closing-a-pipe-end-ends-every-holders-polls-on-it.md`,
`issues/a-watch-outlives-the-close-of-the-process-handle-it-was-made-through.md`,
`issues/mio-deregister-fd-leaves-a-pending-poll-live.md`,
`issues/closing-a-ring-leaves-its-submitters-parked.md`,
`issues/two-submitters-of-one-ring-race-its-last-completion-slot.md` and
`issues/toyos-poller-is-sync-and-a-watch-moves-the-tail-in-two-steps.md`.
The interrupts-off walk record is narrowed to a claim's handler and renamed.
The mio selector's missed registration during a parked select gets its own
record.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… names the deleted poll files

`tests/guest/rust-tests` is outside the workspace and keeps its own
lockfile, which still pinned mio at `fe42def`. It is the one ToyOS consumer of
mio (`mio_waker_coalesces`), so the selector's generation drop shipped in no
ToyOS binary. Its lockfile now pins `4108eff`, as the root lockfile does;
`cargo metadata --locked` agrees with both.

`system/netstack/src/watches.rs` and `sdk/toyos/tests/transport_loom.rs`
cited `kernel/src/inbox/polls.rs`, now `kernel/pure/sched/poll.rs`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s change made

`issues/nothing-fails-when-a-devices-release-or-close-stops-answering-its-polls.md`
named `WatchRef::cancel_polls`'s `Irq` arm as a claim's close site. This
change deleted that arm. A claim's close or move now ends the polls made
through its handle, by `ops::end_polls` and `WatchRef::end_polls`'s `Claim`
arm. The record now says so. It also says which release still answers a live
poll: `tear_down_departed`, because a release after the last close finds none.
Its exit is restated for both: a test for each path, red with its call
removed. The record's audio claim no longer exists and leaves it.

Rebased onto main 45935f4, which deleted
`issues/parallel-tests-red-under-other-suites.md`; the deletion is kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e poll-path A/B, recorded while the benches are down

The benches are down, so metal is no longer a landing condition. What only
metal shows of this change is recorded as owed, each in a record this change
already updates:
- `issues/nothing-fails-when-a-devices-release-or-close-stops-answering-its-polls.md`:
  the T14's `testcases`, `testcases-watchdog` and `shared-debug` boots of
  main, with `poll_belongs_to_its_handle` and `pipe_other_end_gone`; and the
  osbench A/B of main from before and after this change, over the wake, pipe
  and IPC rows, by the rule stated before any boot ran.
- `issues/mios-selector-arms-no-registration-made-while-another-thread-selects.md`:
  `mio_waker_coalesces`, built from mio `4108eff`.
- `issues/a-claims-holder-lengthens-its-handlers-interrupts-off-walk-by-the-polls-it-arms.md`:
  the `windowscase` boot, with `mask_windows` and the reshaped herd.

Record edits only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…o longer uses

#1241 deleted the watch's gate model and this change deleted the
two-submitter model. Those were `loom_watch`'s two uses of `CPU1`, so after
both the import is unused, and `cargo clippy -p kernel-loom --all-targets`
refuses it. The rebase's other resolutions are in the commits it replays:
`resolve` and `ThisProcess` look up through #1246's `with_handles`, and the
watch imports `poll::Via` beside main's `Arc` and `CellLock`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

Rebase review of #1261 at 39f9da686 (onto main 06fcb81ea)

Architecture

The design is unchanged from the LAND at 8e8b85256. A poll still belongs to (table, handle). The one table lock it relies on is now #1246's per-process Handles lock instead of ProcessData's.

The resolutions

  • kernel/src/inbox/mod.rs. resolve (:563) reads the object and handles.id() in one with_handles hold. ThisProcess::while_held (:574) runs HandleTable::while_held in that same kind of hold. Registration, close and send still serialize on the one table lock:

    • registration: polls::register → while_held → add;
    • close: sys_close → with_handles → ops::close → remove → left → end_polls;
    • send: with_handles (ipc.rs:492, :507) → move_handles → transfer's commit → end_polls.

    Every thread reaches its process's one Arc<Handles>, so Via::table is the same id for all of them. "A poll ends with its handle" still holds.

  • The lock order is unchanged. It is table → watch list, where it was ProcessData → watch list. A poll belongs to the handle it was made through, and a ring to its one submitter #1261 takes no lock before the table lock:

    • arm releases the ring's state lock before register;
    • deliver calls look/resolve between its ring.polls closures.

    Under the table lock, A poll belongs to the handle it was made through, and a ring to its one submitter #1261's additions take only watch-list locks, and end_rings fires with the list lock released. Handles.table is the same crate::sync::Lock type as ProcessData's.

  • kernel/pure/sched/watch.rs:43. {Arc, CellLock} is main's import after The machine's stop keeps its gate and waits for no thread #1241 removed Gate (and with it fence, AtomicU32 and Ordering), plus A poll belongs to the handle it was made through, and a ring to its one submitter #1261's poll::Via. gate-fence-off is gone from both manifests and from CONTROLS, as on main.

  • 39f9da686. The commit message is accurate. CPU1's only use at 8e8b85256 was the gate model The machine's stop keeps its gate and waits for no thread #1241 deleted, and its only use on main is the two-submitter model this change deletes. Control: with fe1128e76's loom_watch.rs, cargo clippy -p kernel-loom --all-targets -- -D warnings fails with unused import: CPU1. At 39f9da686 it is clean.

  • git range-diff 45935f4b9..8e8b85256 origin/main..39f9da686.

Evidence (mine, at 39f9da686, in the worktree's own target dir)

  • cargo test -p kernel --lib --features sched-check: exit 0, 469 passed. The 12 kernel clippy shapes: all exit 0.
  • Models: loom_watch 14, inbox_answer 14 and poll_once 2 passed, and cargo test -p kernel-sim is green. The loom_watch set is 8e8b85256's minus a_transition_racing_an_opening_gate_is_never_missed, which The machine's stop keeps its gate and waits for no thread #1241 deleted.
  • Controls, invoked as run_control invokes them and judged by judge_control's rule: each red, with every verdict line present and no empty run. They are post-is-an-answer, poll-fire-load-store (both poll_once and loom_watch, including "a post and a close both took the poll"), commit-ignores-notify, notify-flag-load-only, register-outside-table, admit-keeps-the-replaced, teardown-posts-nothing and fault-posted-before-it-is-set.
  • cargo clippy -p kernel-loom --all-targets, with and without those features: clean. every_model_control_is_run: pass.
  • My first run shared a target dir with the A receive reports what each handle names, and a typed handle is made of a received one only where the kernel said so #1239 worktree, and every bare-target shape failed on SyscallError::Busy. Cargo had reused the other tree's toyos-abi, which has the same relative path and older mtimes. The rerun in this tree's own target dir, reported above, is clean. Keep worktrees' target dirs apart.
  • Guest 43108631616 is still queued, so I have not read it.

Findings

LAND

@Japabu

Japabu commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator Author

Orchestrator: landing under the benches-down rule. Rebase review LAND at 39f9da6; guest run 43108631616 at that head: success (five rows incl. aarch64 virt_jobs_at_el2). Owed on metal recorded; the PR's own shared-boot members (poll_belongs_to_its_handle, pipe_other_end_gone, …) will also run on the shared-boot guest row (#1271) on main. Whichever of #1227 and #1261 lands second adds SyscallError::Busy => Self::Busy to failure.rs — #1261 lands first.

@Japabu
Japabu marked this pull request as ready for review November 12, 2026 00:54
@Japabu
Japabu added this pull request to the merge queue Nov 12, 2026
Merged via the queue into main with commit a7ca8ad Nov 12, 2026
11 checks passed
@Japabu
Japabu deleted the handles3-poll-via branch November 12, 2026 01:13
Japabu added a commit that referenced this pull request Nov 12, 2026
…leave the kernel

The kernel kept, for every process, the labels its parent put on each
endowment, compared three of them (`job`, `self`, `quit`), answered them back
through `SYS_ENDOWMENTS`, and held a namespace object that mapped service
names to connectors behind three more calls. None of that is the kernel's:
it is what one program calls a handle it gave another.

Now a spawn moves a list of handles and nothing else. The child's table
starts with its handle to itself (`RawHandle::SELF`), its notice
(`RawHandle::NOTICE`), then the endowments in the order the spawn listed them
(`RawHandle::endowed(i)`); the kernel installs them so and asserts it. What
the parent calls each is a table it writes in the child's startup block,
which the child's SDK checks once, whole, before it looks one up: a table
that is no list of distinct, non-empty UTF-8 names ends the child, said on
its `fd:2` where the table still names one.

- `SpawnArgs::job` replaces the `job` label: a borrowed handle carrying
  `WRITE`, resolved once like the place and the pager, so the commit's
  re-check that the moved handle still named the job goes.
- `SYS_CONNECT` (102, `SYS_NAMESPACE_OPEN`'s number) connects through a
  connector carrying `WRITE`, which a connector is now minted with. One
  without it may only be passed on.
- `SYS_ENDOWMENTS` (99), `SYS_NAMESPACE_BUILD` (101), `SYS_NAMESPACE_NAMES`
  (170), the namespace object, `SpawnEndow`, `EndowEntry`, the label blob
  and its per-process charge go; the freed numbers stay unassigned.
  `MAX_SPAWN_ENDOWMENTS` is 96: the 30 a spawn carried plus the 64 a
  namespace held, so everything one table and one namespace expressed still
  fits. The names are bounded by the startup block alone.
- `ObjectKind::Namespace` goes and its word 8 stays unused, as `ObjectKind`
  says a word that goes does: `SysCap` through `DeviceSet` keep 9 to 19,
  `from_word` looks a word up rather than indexing by it, the SDK's census
  finds a kind's count by its place in `ALL`, and the SDK's receive test
  takes word 8 and the word past the last kind as ones no kind has.
- The supervisor, whose parent is the kernel, starts holding its system
  capability, the file server's image, ROOT's and the boot names at the four
  handles `toyos_abi::boot::supervisor` states.

In userland a service is a connector named `svc:<name>` (a row's
`receives`, its view's `fs:` directories, minted grants) or
`provide:<name>` (one a launching client sent). `endow::service` looks one
up and connects; a helper is given a duplicate of each its parent holds,
narrowed by `CommandExt::services_only`. Everything else is taken once by
name. The supervisor gives a provided connector to a row that does not
receive it without `WRITE`; swap's connector arrives with `WRITE` alone, as
its one holder only connects. The supervisor adopts its own directory
capabilities (`std::os::toyos::fs::adopt_services`), which no child
inherits: so a storage row, whose view is empty, now starts holding no file
grant, where it inherited the supervisor's whole tree.

A connector a peer sent, as a session's launcher and controls are, is typed
only where the kernel reported a connector (`ReceivedHandle::typed`), and an
endowed one on its parent's word. std's `adopt_services` is `unsafe`, as
`adopt_namespace` was: the handles it takes are its caller's word.

`SYS_NAMESPACE_OPEN`'s name was the last caller of `SyscallContext::user_str`,
since #1238 reads a spawn's path as bytes bounded by `MAX_PATH`. So
`user_str` and `MAX_USER_STR` go. `KernelStack::heap` charged every thread
two 64 KiB strings and a page for what a call copies in. It now charges, by
their types, what a spawn copies in and holds where it may be preempted:
- its path, name and build-id at their bounds;
- its ranges;
- its endowments;
- the mappings of the device claims it moves, in a vector sized once at the
  first claim so it never grows past that.
It claims nothing for another call.

A block session holds a connector and no service name, so diskserver's two
messages for a session's server that broke the protocol or went silent
name the session by its partition GUID. #1261's `poll_belongs_to_its_handle`
connects through its connector, where it opened a one-entry namespace.

Kernel: 55,475 lines in 210 files at main a7ca8ad to 55,117 in 209.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu added a commit that referenced this pull request Nov 12, 2026
The `testcases` boot owed on metal for #1252 now names the four shared-boot
members whose rebased forms have only been compiled: #1261's
`poll_belongs_to_its_handle`, which connects through its connector where it
opened a one-entry namespace, and `messages_are_whole`, `moved_rights` and
`mutual_kill`. No guest row runs them until #1271's shared-boot row lands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu added a commit that referenced this pull request Nov 12, 2026
…r's slots, as no-holders ones are

Main brings #1273: a pager's object slots are made as they are wanted
(kernel::slots::Slots), pager_create takes no object count, and the
objects whose last holder went are a list through the slots, not a walk
over every slot. #1258 had kept its dirtied objects the old way, a flag
an entry carries, a count, and a walk in notices.

- Slots holds a const number of mark lists, each apart from the others;
  a removal unmarks a slot on every one. The pager's table has two:
  NO_HOLDERS, #1273's, and DIRTIED. owe_dirtied marks and unmarks the
  second, has_notices asks both, notices takes from both, and a release
  unmarks both. The only walk over every slot left is the pager's end.
- An entry keeps #1258's writers and record beside #1273's fields, and
  add makes it in the slot make_slot charged for. holder_gone counts the
  holder down and say_no_holders marks the object only once no holder's
  write is under way, as #1258 had it.
- pager_create, the SDK's Pager::create and pagerprobe take no object
  count.
- Busy came from both sides. #1261 on main made it general, an inbox's
  submitter among its uses, mapped to EBUSY and ResourceBusy; #1258 used
  it for a write to a file a program runs from, mapped to ETXTBSY and
  ExecutableFileBusy. It keeps main's mapping, and its doc names both
  uses: a holder's write to a sealed object now answers EBUSY.
- The file server's tests take #1273's keeps argument (None where the
  count is not what a test is about) and #1258's record.
- A slot grows from #1273's 72 bytes to 216: #1258's object state,
  record and writers, and the second mark. The /tmp record's per-file
  cost says so, measured as #1273 measured it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu added a commit that referenced this pull request Nov 12, 2026
…he last holder is refused

The seal refused a holder's write as `Busy`, which main's #1261 had made
an inbox submitter's word, so one kind said two things and libc mapped
them to one errno. The seal now answers `TextBusy` (12): libc gives
ETXTBSY and std `ExecutableFileBusy`, and `Busy` keeps its one meaning.

A holder's write that begins once the object's last holder went is
refused as `Gone`: the pager may already have been told no holder is
left, and the write would dirty what it was told no one holds. The
no-holders notice is re-checked against writes under way as well as
holders when it is taken.

`owe_dirtied` goes; its two callers mark and unmark the dirtied list
directly. The slot table's test marks slot 3 on both lists, so a
removal that unmarks only one list fails it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu added a commit that referenced this pull request Nov 12, 2026
…leave the kernel

The kernel kept, for every process, the labels its parent put on each
endowment, compared three of them (`job`, `self`, `quit`), answered them back
through `SYS_ENDOWMENTS`, and held a namespace object that mapped service
names to connectors behind three more calls. None of that is the kernel's:
it is what one program calls a handle it gave another.

Now a spawn moves a list of handles and nothing else. The child's table
starts with its handle to itself (`RawHandle::SELF`), its notice
(`RawHandle::NOTICE`), then the endowments in the order the spawn listed them
(`RawHandle::endowed(i)`); the kernel installs them so and asserts it. What
the parent calls each is a table it writes in the child's startup block,
which the child's SDK checks once, whole, before it looks one up: a table
that is no list of distinct, non-empty UTF-8 names ends the child, said on
its `fd:2` where the table still names one.

- `SpawnArgs::job` replaces the `job` label: a borrowed handle carrying
  `WRITE`, resolved once like the place and the pager, so the commit's
  re-check that the moved handle still named the job goes.
- `SYS_CONNECT` (102, `SYS_NAMESPACE_OPEN`'s number) connects through a
  connector carrying `WRITE`, which a connector is now minted with. One
  without it may only be passed on.
- `SYS_ENDOWMENTS` (99), `SYS_NAMESPACE_BUILD` (101), `SYS_NAMESPACE_NAMES`
  (170), the namespace object, `SpawnEndow`, `EndowEntry`, the label blob
  and its per-process charge go; the freed numbers stay unassigned.
  `MAX_SPAWN_ENDOWMENTS` is 96: the 30 a spawn carried plus the 64 a
  namespace held, so everything one table and one namespace expressed still
  fits. The names are bounded by the startup block alone.
- `ObjectKind::Namespace` goes and its word 8 stays unused, as `ObjectKind`
  says a word that goes does: `SysCap` through `DeviceSet` keep 9 to 19,
  `from_word` looks a word up rather than indexing by it, the SDK's census
  finds a kind's count by its place in `ALL`, and the SDK's receive test
  takes word 8 and the word past the last kind as ones no kind has.
- The supervisor, whose parent is the kernel, starts holding its system
  capability, the file server's image, ROOT's and the boot names at the four
  handles `toyos_abi::boot::supervisor` states.

In userland a service is a connector named `svc:<name>` (a row's
`receives`, its view's `fs:` directories, minted grants) or
`provide:<name>` (one a launching client sent). `endow::service` looks one
up and connects; a helper is given a duplicate of each its parent holds,
narrowed by `CommandExt::services_only`. Everything else is taken once by
name. The supervisor gives a provided connector to a row that does not
receive it without `WRITE`; swap's connector arrives with `WRITE` alone, as
its one holder only connects. The supervisor adopts its own directory
capabilities (`std::os::toyos::fs::adopt_services`), which no child
inherits: so a storage row, whose view is empty, now starts holding no file
grant, where it inherited the supervisor's whole tree.

A connector a peer sent, as a session's launcher and controls are, is typed
only where the kernel reported a connector (`ReceivedHandle::typed`), and an
endowed one on its parent's word. std's `adopt_services` is `unsafe`, as
`adopt_namespace` was: the handles it takes are its caller's word.

`SYS_NAMESPACE_OPEN`'s name was the last caller of `SyscallContext::user_str`,
since #1238 reads a spawn's path as bytes bounded by `MAX_PATH`. So
`user_str` and `MAX_USER_STR` go. `KernelStack::heap` charged every thread
two 64 KiB strings and a page for what a call copies in. It now charges, by
their types, what a spawn copies in and holds where it may be preempted:
- its path, name and build-id at their bounds;
- its ranges;
- its endowments;
- the mappings of the device claims it moves, in a vector sized once at the
  first claim so it never grows past that.
It claims nothing for another call.

A block session holds a connector and no service name, so diskserver's two
messages for a session's server that broke the protocol or went silent
name the session by its partition GUID. #1261's `poll_belongs_to_its_handle`
connects through its connector, where it opened a one-entry namespace.

Kernel: 56,188 lines in 212 files at main d1cefba to 55,830 in 211.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu added a commit that referenced this pull request Nov 12, 2026
The `testcases` boot owed on metal for #1252 now names the four shared-boot
members whose rebased forms have only been compiled: #1261's
`poll_belongs_to_its_handle`, which connects through its connector where it
opened a one-entry namespace, and `messages_are_whole`, `moved_rights` and
`mutual_kill`. No guest row runs them until #1271's shared-boot row lands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant