[Japabu]

Problem

Every request through a directory capability took the process's one turn on that capability (a Turnstile) for its whole round trip, though the wire carries 16 tagged requests. A thread asking beside another thread of its own process waited for the other's answers. In scratch timers on the T14, fstat on /tmp beside such a thread read 21.4 to 21.6 µs p50, against 7.6 alone (2.8x). Beside another process it read 8.4 µs (issues/a-call-beside-another-thread-of-its-process-is-unmeasured-on-metal.md, the record of the measurement now owed).

Change

• Requests in flight at once.
  • A request holds one of 16 slots: its 128 KiB share of the window, the place its reply is filed, and the low bits of its tag.
  • A request larger than a slot holds the whole window. A caller waiting for the whole window goes ahead of any slot asked for after it.
  • One connection per process and capability stays, so the server's per-connection fairness is unchanged.
• Whoever waits receives, a turn per connection.
  • A caller waiting for its reply takes its connection's receive turn if nobody on that connection holds it.
  • It takes replies off its connection and files each other caller's reply in that caller's slot, until its own comes.
  • It then wakes a caller still waiting on the same connection, which takes the turn if it is still free. A request given back unsent does the same when nobody on its connection receives.
  • A failed receive, or a reply naming no request in flight, fails every request in flight on that connection.
• One lock. Every slot's request and the turn are read and changed only under the table's lock, held for nothing else. A caller sleeps on its slot's own futex word, which is moved after every change that concerns it.
• A reply's wait is bounded.
  • A server that answers nothing for REPLY_BOUND_NS while a reply is owed is let go as one that ended.
  • That is Unanswered, a new client-local error (ETIMEDOUT, TimedOut), and every caller waiting on that connection fails with it.
  • The bound is the block protocol's silence bound, 365 s, plus 60 s of room for the server's own work and turns. The SDK keeps its own copy of the block bound, BLOCK_SILENCE_NS, since toyos-blockring depends on it. The file server, which depends on both and whose replies wait on block requests, asserts the copy equals blockring's SILENCE_BOUND_NS.
  • A file server's reply has no derived worst case. That gap is recorded with severity: [hang] in issues/a-file-server-reply-has-no-derived-worst-case.md.
  • The hello's receive is bounded the same way.
• Connections by generation.
  • A connection that ends is retired. Its requests in flight finish or fail with it, and the next request connects again.
  • A request takes its slot before its connection. A failed request retires its connection before it gives its slot back, so the slot's next request goes out on a newer connection and window.
  • Each connection lends its own window, so a server still writing a request it was let go with writes that window, never a newer connection's.
  • A file id is checked against the request's own connection, so an id from an older connection is Stale.
  • A link to follow comes back with its target, copied out of the request's share before the share is given back.
• One shared Dir. Dir's calls take &self, so a capability is one Arc<Dir>. The turnstile and its comments go. Callers that bound a Dir mutably no longer do: pkg, the supervisor, osbench and five rust-tests binaries.
• Manual. The files page now says two things:
  • a process's threads share its connection to each capability, and their requests are in flight at once, but for one that needs the whole window or one beyond sixteen;
  • a server silent past its bound while it owes a reply is let go as if it had crashed, as a block server is.
• osbench rows. osbench gains stat.beside_a_thread and open_close.beside_a_thread: each measured while another thread of the process asks stat as fast as it is answered.

Checks

Host, at the head 04ced8d, on #1258's 66de9f9 (main merged, with #1235's block silence bound). Logs: d146278e/r3; earlier rounds' at r2 and r1.

check	result
cargo test -p toyos --lib fs::, files::, sync::	rc 0
clippy's shapes for toyos (all targets, and fd-table), warnings denied	rc 0
cargo test -p toyos-blockring --lib and its clippy	rc 0
cargo test --manifest-path system/fileserver/Cargo.toml (it compiles the bound's assertion) and its clippy	rc 0, 132 + 3; clippy shows main's four findings
cargo test --manifest-path kernel/loom/Cargo.toml --test fs_flight (9 models)	rc 0, 40 s
cargo clippy --manifest-path kernel/loom/Cargo.toml --tests -- -D warnings	rc 0
the default and testcases images, at round 1; this round changes no ToyOS-only code	rc 0

The loom models. They run the SDK's own in-flight file, sdk/toyos/src/fs/flight.rs, against loom's lock and atomics. The shim's futex sleeps under a lock that each wake takes after its store, and it returns only once its own word is woken, as the kernel's futex does. So a wake the protocol leaves out is a loom deadlock. The nine models:

• two callers answered in reverse order;
• one caller asking twice, its second request in the slot it gave back, beside another;
• a connection ending with requests in flight;
• a connection ending beside a failed send and a request on a fresh connection, which may take the failed send's slot;
• two callers and a failed send on one connection, the turn handed to the failed send going on;
• two connections, each receiving for itself while the other's server waits on the first caller;
• two callers on one connection beside a caller leading another, the turn handed on staying on its connection;
• a reply naming no request;
• a whole-window request and a slot request, each answered.

The bound is 3 preemptions for the three-thread models and 2 for the slot-reuse and four-thread ones, which run past minutes at 3.

Controls. Each is a patch applied, run model by model, and restored.

• One turn per Dir (round 1's flight.rs): the two-connection model deadlocks.
• The filing wake deleted: deadlocks in the reply, slot-reuse, connection-end, failed-send and hand-on models.
• The leader's hand-on wake deleted: deadlocks in the reply, slot-reuse, failed-send and hand-on models.
• The give's hand-on wake deleted: deadlocks in the failed-send model.
• The hand-on ignoring its connection: deadlocks in the hand-on model alone.
• fail ignoring the connection: the fresh-connection model's request fails with the old connection.

The first version's lock-free hand-on. Review found a stale failure filed into a reused slot, and a store-buffering lost receiver that loom cannot see. Both are gone with the lock. An earlier version of this description said the first model "found a real lost receiver". That was wrong: the red came from loom's partial modification order, which C++ forbids, and the plain-store control built on it is dropped.

Not built here. osbench's ToyOS half and the five rust-tests binaries build only in the harness. Each change there drops a mut from a Dir binding now used through &self. Every rust-tests binary type-checks against the ToyOS sysroot at the head (Rebased onto main, below); osbench's ToyOS half is not checked here. The testcases shared-boot guest row builds them all.

Rebased onto main. First onto ee15a2e (#1258 merged, beside #1252). #1252 made a directory capability a connector the process holds, where it had been a name in a namespace; the rebase keeps that with this PR's shared connection. connect makes the Dir and its first connection, reconnect connects through the held connector, the SDK's capability table keeps Arc<Dir>, and the supervisor's sync, pkg and three guest tests drop a mut (r8). Then onto 29c5f2d (#1227, #1264, #1280). #1227 replaced libc's and std's tables with one Failure table, so Error::Unanswered is that table's TimedOut (libc's ETIMEDOUT, std's TimedOut); its match is exhaustive, so the compile settles that the arm is there. Commits 1, 3 and 4 are unchanged by that rebase (r9).

Checks at the head 4227aa4, each rc 0 (r9): the SDK's tests and two clippy shapes, the nine loom models, the file server's tests, the testcases image (its sysroot, libc and std, and the supervisor), every rust-tests binary type-checked against that sysroot, and cargo check -p pkg --all-targets on the host, where pkg's one change compiles without a warning. Two runs in r8 ended rc 101, neither because of this PR. cargo test -p supervisor cannot build on the host, since the supervisor is a ToyOS program; the image is its compile. The clippy run on the supervisor and pkg stopped at a redundant_clone in system/pkg/src/archive.rs:177, a line this PR does not touch and the same on main.

Guest. Run 43165795439 at d88c74f, on main ee15a2e, kept fetchable as branch fileops-tags-hold-d88c74fa1. This PR's change there differs from its change at the head 4227aa4 only in where Unanswered's ETIMEDOUT and TimedOut are said, the Failure arm; the rest of the difference is main's. The run covers nvme_disk_boots, the_data_partition_beside_the_stick_is_designated, the_log_lands_on_the_boot_stick, virt_user_mode (aarch64; the SDK transport is shared) and testcases_members_exit_0, the testcases shared boot as a guest row (#1271), whose members put several requests on one connection (a_served_files_writes_land_whole's server writes) and run the export, dup, writable and generation members. Every program's file calls go through this client, so the rows establish the single-thread path, connect and generation. Before the rebase, 43152649297 passed the same five at 2e7dd79 (log); 43117897753 passed the four rows at an earlier head; 43143742563 and 43147796257 were cancelled while queued.

Metal. None; the benches are down. The speed-up is not measured. The record owes osbench_disk on the T14, main before and after, A, B, B, A: stat.beside_a_thread and open_close.beside_a_thread within 1.2x of stat and open_close on both volumes.

Risks

• Two threads' requests on one connection run only in guests. The protocol rests on the loom models; the integration on testcases_members_exit_0's multi-threaded members: all 242 exited 0 in run 43152649297 before the rebase, under QEMU on two vCPUs, and run 43165795439 runs them at d88c74f. Nothing has run it on metal.
• The reply bound rests on no figure of the file server's own. A request may wait on several block requests in turn, and on other connections' requests. A server that keeps its block contract can therefore stay silent past the bound and be let go (issues/a-file-server-reply-has-no-derived-worst-case.md).
• Not every wait is bounded. A blocking send to a hung server whose queue is full of CLOSEs waits without bound, as it did before this change.
• Many large requests. A request over 128 KiB waits for every slot, as a caller of 16 small ones may wait for it. Large reads and writes from several threads at once take turns, as all did before.

🤖 Generated with Claude Code

[Japabu]

Review of #1272 at b38ca25 (on fileops-step1 230a2ad)

Architecture

The shape is right. It is what the record's exit asks for:

• one connection per process and capability, so the server's turn per connection and its fairness stay as they are;
• slots that divide the window;
• each thread waiting for its own tag;
• no kernel addition.

Whoever-waits-receives (leader and follower) is the standard way to demultiplex one stream without a receiver thread. With the turnstile gone, Dir becomes &self, and owner.rs loses the turn's comments and its Capability wrapper. The manual's files page carries the decision. The client must also connect again under requests still in flight, so Link by generation, retire and the copied Follow target each earn their place.

The synchronisation is not the simplest. A slot has five states. A separate turn word is handed on by compare-exchange, a Lock sits beside both, and the code reasons about orderings across all of it. Both protocol defects below live in that lock-free part.

The simpler representation keeps everything else:

• Put slot state, the receiver and the hand-on in the existing table Lock, and keep a per-slot futex word only for sleeping.
• Hold that lock to file a reply, to withdraw a slot whose send failed, and to pick the next receiver.
• The lock is held for nanoseconds against round trips of microseconds, so it costs nothing measurable, and B1 and B2 become impossible.

Size. The change is +995/−417, net +578:

• SDK production about +280: flight.rs +258, fs.rs +137, owner.rs −18, and sync.rs −97, which includes the turnstile's test;
• loom shim and models +261, osbench +22;
• manual +5, record +10.

Required

B1. sdk/toyos/src/fs/flight.rs:202-210 with fs.rs:1287-1302 and flight.rs:225 — a request whose send fails gives its slot back while the leader may still file a failure into it.

• The path. call_with marks the slot WAITING (sent) before conn.send. If the send fails, it never answers: the ticket drops and give swaps the slot to IDLE.

• The race. fail checks WAITING and the link, then files, as two steps. In the gap:

  • the slot is given back;
  • a request on the fresh connection takes the slot and marks it WAITING;
  • fail then files the old connection's error into it.

• Shown by loom on this head's own flight.rs. I wrote a scratch model with three callers:

  • a leader whose connection ends;
  • a caller whose send fails;
  • a caller on a fresh link whose server answers.

  At preemption bound 3 it reds: left: Err(Gone), right: Ok(33). Tag 33 is slot 1, the slot the failed sender gave back.

• Consequence in fs.rs.

  • The fresh request returns Gone and retires the healthy new connection, so every file the process opened on it goes Stale.
  • Its real reply then arrives naming no request in flight, and that fails every other request on the connection.
  • The same race comes with a server's end, when the send and the leader's receive both fail.

• False comments. The SAFETY notes at flight.rs:47-49 ("while its state says the request waits") and :254-255 ("nothing files a slot that no longer waits") are false at the head.

• Fix. Either:

  • do the filing and the withdrawal under the table lock (above); or
  • have the filer claim the slot by compare-exchange before it writes the cell, and the owner withdraw its slot by compare-exchange, waiting out a claim it loses.

  Add the model I describe in B3, which must go green.

B2. flight.rs:159-170 with :166-167 and :214-221 — under Rust's memory model the hand-on can lose a receiver, and loom cannot look.

• The interleaving. A caller handed the turn does two things: a relaxed CAS of LEAD to WAITING, then a CAS of the turn that fails. The leader letting go also does two: a Release store of the turn, then the hand-on CAS, whose failure ordering is Relaxed. This is store buffering:
  • both failing CASes may read the older value, so the caller sees the turn held and the leader sees LEAD;
  • the caller then sleeps on WAITING with the turn free and nobody to wake it;
  • its reply waits on the connection until some other request of the process arrives.
• One way to reach it. The caller is handed the turn while it is between sent and its send, and a third caller then takes the turn.
• Why loom cannot see it. In loom a compare_exchange reads the newest store even when it fails (src/rt/atomic.rs, match_rmw_to_stores).
• Measured on an aarch64 host. The scratch program is outside the tree.
  • The head's exact operations: 0 of 10⁷ trials.
  • The textbook store-and-load control: 48%.
  • A relaxed cas followed by a plain load: 48%.
  • So what holds today is the hardware's ordering of the leader's stlr before casal (ARMv8 keeps a store-release in order before a later load-acquire). The source's orderings do not require it.
• Fix. Use SeqCst on those four operations, failure orderings included, or the lock from B1.

B3. kernel/loom/src/lib.rs:286-350 — the models cannot detect a missing wake, and the claim that they do is false.

• The shim. It sleeps every waiter on one global condvar and notify_alls on any wake. Every give ends with wake_all(&freed), so a waiter whose own word nobody woke is woken anyway.
• The kernel. Its futex wakes only on a post to the waiter's own word: scheduler::futex_wait waits on futex::watch_of(phys_addr).
• Probes at the head. Each was applied, run and restored.
  • With the wake deleted from hand_on (:217), all four models are green.
  • With the wake deleted from file (:249), all four models are green.
  • Each deletion is a permanent hang on ToyOS.
• With a per-word shim. A wait returns only when its own word is woken, counted per address under the shim's lock, in about 20 lines. The head passes all four models. Both probes red with deadlocks: the hand-on probe in the connection-end model, the filing probe in the reply model.
• Text to correct. The shim's doc (:290-292, "A waiter nobody wakes is then loom's deadlock"), tests/fs_flight.rs:8-10 and the PR body make the false claim.
• Coverage the models lack.
  • No caller makes a second request, so no model has slot reuse or a handed caller meeting a newer leader.
  • No send fails.
  • No second link exists.
  • The whole-window model never receives.
• The filing control. "Filing by plain store" reds through loom's partial modification order. The PR body itself says C++ forbids the read it reports, so this is not a negative control: no defect stands behind its red. The swap's comment (flight.rs:241-243) gives a reason a plain store already satisfies by coherence. Name the real reason or drop it. "Found a real lost receiver" in the commit message and in the brief overstates it.
• Fix.
  • Make the shim per word, and correct the three texts.
  • Add the stale-filing model from B1 and a model in which one caller makes two requests beside another.
  • Keep "no hand-on" as a control, and drop the filing control.

B4. flight.rs:128, :170, fs.rs:894, sync.rs:64 — no new wait has a failure timeout, and nothing records that.

• The waits:

  • the slot wait;
  • the wait for the turn or a reply;
  • connecting, which is held across the hello's round trip;
  • and beneath them, the receive.

  All are unbounded (futex_wait(.., None) and rx.recv).

• Not new in kind. The turnstile and the receive had no bound either. Root's rule still applies to the waits this PR writes, and their interface can carry an error.

• Where one bound would go. At the receive. On expiry the leader fails its link, which files every waiter on it, so the futex waits inherit the bound.

• Fix. Bound it, or record the gap with owner and exit. In the same record, or another, record the hazard the PR names under Risks: a live old server can write a share the client has lent again.

B5. issues/a-process-s-threads-take-turns-on-a-directory-capability-for-whole-round-trips.md — the tree now refutes the record's slug and its first paragraph.

• Threads no longer take turns for whole round trips, and Turnstile is deleted. Its "Owner" line still says the turnstile stays.

• The new "Current state" paragraph contradicts the paragraph above it.

• What is owed now is a measurement.

• Fix. Under issues/README.md:

  • rename the slug to what is owed;
  • rewrite the body to the present state;
  • set kind: tooling.

  No file in the tree cites the slug but the record itself.

Optional

• S1. docs/src/architecture/files.md:23-24: "none waits for another's round trip" is untrue in two cases, a request larger than a share and a 17th request. Qualify it.
• S2. The log ring's controls are features that build/ci.rs CONTROLS reds on every host run. Making this model's surviving control one too keeps the models' teeth.
• S3. hand_on hands to the lowest waiting slot whatever its link. A request on a retired but live link can starve behind a newer link's traffic. Handing on within the leader's link first, or a turn per link, removes that.
• S4. A panic inside the turn can only come from take_filed's expect. It leaves turn at 1 and every other caller of that Dir asleep with no bound. A guard that lets the turn go and fails the link would make the failure loud instead of a hang.

Evidence

My runs at b38ca25. All in a detached worktree, now removed with its target/.

• SDK tests, rc 0:
  • cargo test -p toyos --lib fs::: 13 passed;
  • files::: 11 passed;
  • sync::: 1 passed.
• cargo test --manifest-path kernel/loom/Cargo.toml --test fs_flight: 4 passed at bound 3 in 16.7 s.
• Controls at bound 3:
  • no hand-on: the reply model and the connection-end model each red as a deadlock;
  • filing by plain store: the reply model reds as a deadlock.
• Clippy with the adopted lints and warnings denied, rc 0 for each shape:
  • -p toyos --all-targets;
  • -p toyos --lib --features fd-table;
  • -p kernel-loom --all-targets;
  • -p osbench for both Linux musl targets.
• The author's 66cc3ba5 evidence.
  • The checks ran at 1ee3e6d. 1ee3e6da9..230a2ad3d touches no file under sdk/toyos.
  • controls/no-hand-on.log shows a red in the connection-end model, then loom's abort. It never shows the reply model the README names. My rerun shows that model reds too.
• Not compiled by me. The five rust-tests binaries and osbench's ToyOS half. Each changes only a binding's mut, and every Dir method is now &self, so inspection suffices.

The guest run, and whether a dedicated test is warranted

At this head, or at a revision whose fs.rs, fs/flight.rs, files/owner.rs and sync.rs are the fix's:

The four rows. nvme_disk_boots, the_data_partition_beside_the_stick_is_designated, the_log_lands_on_the_boot_stick and virt_user_mode boot and pass. Every program's file calls go through this client, so they establish the single-thread path, connect and generation.

The testcases shared boot on several vCPUs must show:

• a_served_files_writes_land_whole and a_served_files_read_never_tears_against_its_write passing on both /home (DATA) and /tmp. Their two writers and reader send requests to the server on one capability whenever a holder's write or read is refused, which is the only place the boot puts several requests in flight on one connection.
• served_file_offset, an_export_shares_its_descriptions, stat_identity and app_view passing: export, adopt, dup, writable and the generation, all changed here.
• No hang within the row's bound.

What the run cannot show. That requests overlapped (nothing counts them), or either race above. It establishes order and contents only.

A dedicated guest test is not warranted. The protocol's interleavings are a host matter: loom over the real file, once B3's shim and models are in. A guest test cannot force them, and it would pass at this head despite B1. What the guest adds is the real futex and IPC, which these members already drive.

SEND BACK

[Japabu]

Round 1 response

Fixed at ab93cc9, which also rebases onto #1258's head 695146e. Evidence:
20fe3e43/r1.

B1 and B2, fixed by the lock, as the orchestrator decided. Every slot's request (its link, tag and state: idle, waiting, or filed) and the receive turn now live in the table and change only under its lock. The lock is held to read or change them and for nothing else. A caller sleeps on its slot's own futex word, which is moved after every change that concerns it.

• Filing and failure. Filing a reply and failing a connection both check, under the lock, that the request is waiting on that link. A slot given back is idle. So a failure can no longer reach a slot that a fresh connection's request has taken.
• Giving back an unsent request. That goes under the lock too. If nobody receives, it offers the turn to another waiting caller. This covers a caller handed the turn whose send then fails.
• What went. The lock-free hand-on, LEAD, the turn word and the cells are all gone. flight.rs has no unsafe left.
• S3 taken. A leader hands the turn to a caller on its own connection first.

B3, taken.

• The futex shim. It now counts wakes per word under its lock, and a wait returns only once its own word is woken. The shim's doc, the models' header and the body are corrected.
• Models. There are now six, adding: a caller asking twice beside another (a slot used again); your stale-filing model, a connection ending beside a failed send and a fresh connection's request, which is green; and a whole-window request that receives.
• Controls. Each is a patch run model by model, then restored:
  • the filing wake deleted: deadlocks in the reply, slot-reuse and connection-end models;
  • the leader's hand-on wake deleted: deadlocks in the reply, slot-reuse and fresh-connection models;
  • the give's hand-on wake deleted: deadlocks in the fresh-connection model;
  • fail ignoring the link: the fresh-connection model reds on its own assertion.
• The plain-store control is dropped, along with the "found a real lost receiver" claim. This round's commit message and the body say so.
• Bounds. The models run at preemption bound 3, except the slot-reuse and four-thread models, which run at 2 because at 3 they run past minutes. All six take 16 s.

B4, bounded.

• next_reply waits for readability for at most REPLY_BOUND_NS, five minutes, as the disk service's own client waits for its answers. It then receives.
• On expiry the leader fails its connection with Unanswered, a new client-local error mapped to ETIMEDOUT and TimedOut. Every caller waiting on that connection is failed with it, so the futex waits and the slot wait end within the same bound. The hello's receive uses the same bound, so connecting is bounded too.
• The reused-window hazard is removed rather than recorded. Each connection now lends a window of its own, kept by its Link. A server still writing a request it was let go with writes that connection's window, never a newer one's.

B5, done. The record is now issues/a-call-beside-another-thread-of-its-process-is-unmeasured-on-metal.md, with kind: tooling, its body at the present state, and its owed metal measurement.

S1 taken. The manual names both exceptions: a request that needs the whole window, and one beyond the sixteen a connection carries.

S2 declined for now. The controls stay checked patches in the evidence, as this assignment's others are. Making one a CONTROLS feature would mean the SDK declaring a cfg for the loom crate.

S4. With the lock, nothing in the turn can panic. flight.rs has neither an expect nor an unwrap.

Checks at ab93cc9, all rc 0.

• the SDK's fs::, files:: and sync:: tests;
• toyos's two clippy shapes;
• the six loom models (17 s) and the loom crate's clippy;
• the default and testcases images, which compile pkg, the supervisor, ToyOS's libc and std's file errors.

Guest. Run 43107126235 at the head carries the four rows. 43099728928, at b38ca25, was cancelled.

[Japabu]

Review of #1272 round 2 at ab93cc9 (on fileops-step1 695146e)

Architecture

The locked core is the right representation.

• Slot state and the turn now change under one lock, and each caller sleeps on its own word. That makes round 1's B1 and B2 impossible by construction.
• flight.rs has no unsafe and nothing that reasons about orderings.
• Giving each connection its own window is also the right ownership.

Two representation choices still leave defects (N1, N2).

• The receive turn belongs to the Dir, but replies arrive per connection.
• A request picks its connection before it picks its slot.

Each fix is a few lines and adds no new state.

Size. Net +645 (+1099/−454):

• SDK production about +294: fs.rs +140, flight.rs +267, owner.rs −18, sync.rs −97, libc and std +2;
• loom shim and models +319;
• osbench +22;
• manual +6, record +4.

Round 1

• B1, resolved. file and fail check Waiting and the link under the lock. give makes the slot idle under the same lock. The fresh-connection model is green, and fail ignoring the link reds it on its own assertion.
• B2, resolved. The lock-free hand-on is gone.
• B3, resolved.
  • The shim's futex now wakes only the word woken.
  • Six models run the real flight.rs through #[path]. World::ask follows call_with's order: take, sent, send, answer, give.
  • I reran all four controls model by model, and every red matches the PR's table.
  • The three wake deletions red as loom deadlocks: the thread whose wake was dropped is blocked and the main thread is blocked joining it.
  • fail ignoring the link reds on "a fresh connection's request failed with the old one".
  • The plain-store control and the overclaim are gone.
• B4, not resolved. See N1 and N3.
• B5, resolved. The slug, kind: tooling and the present-state body are fixed. Nothing cites the old slug.

Required (blockers)

N1. sdk/toyos/src/fs/flight.rs:157-161, :182-185, :204 and :260-266: the receive turn is one per Dir, not one per connection, so a caller on a retired connection waits with no bound while another connection's caller holds the turn.

• Two live connections are possible. A Dir can have requests in flight on a retired but live connection and on the current one. A connection is retired without failing its other requests when:

  • a send is refused, for example for kernel room (call_with, fs.rs:1278 and :1286-1288);
  • a reply carries handles nobody asked for (none_carried, one or opened);
  • a request goes out on a connection fetched before its retirement (N2).

• How the wait loses its bound.

  • answer takes the turn only while the Dir-wide receiving flag is false.
  • A leader receives only on its own connection, and only next_reply in a leader is bounded.
  • So while a caller on the current connection leads, a caller on the retired connection sleeps on its word with no bound, its reply already queued and unread.
  • When the leader finishes, waiting(Some(link)) prefers its own connection (S3). Under steady traffic, the retired connection's callers wait as long as that traffic lasts.

• Shown by loom on this head's flight.rs. The scratch model is outside the tree:

  • a caller on connection 1 is answered;
  • a caller on connection 2 waits for a reply that its server sends only after the first caller returns.

  At bound 3 it deadlocks: the main thread joins the first caller, the first caller sleeps on its word, and the second leads connection 2.

• Round 1's S3 was wrong to offer two equivalent fixes. "Own connection first" drains a retired connection only while one of its own callers leads. A turn per connection removes the starvation.

• Fix.

  • Make receiving the mask of leading slots. A caller leads when no leading slot's request is on its own connection.
  • A leader, and an unsent request's give, hand the turn on only within their own connection.
  • In scratch this changes about 15 lines. With it, the new model and the six pass at their bounds: 7 passed in 19 s.
  • Add the model. It reds at this head, so it is its own control.

N2. sdk/toyos/src/fs.rs:922-926: a request takes its connection before its slot, so it can go out on a connection already retired, into a share that the let-go server still writes.

• The order. ticket calls connection() first and then flight.take, which may wait for a slot.
• The trace.
  1. Thread T gets Arc<L1> and waits in take behind 16 requests.
  2. The server is slow past the bound. The leader fails all 16 with Unanswered.
  3. Each failed caller retires L1, then gives back its slot when its ticket drops.
  4. T takes slot j, writes its data into L1's window at j, and sends on L1.
  5. The server, which never stopped working, handles the queued old request j and writes its answer into that share before it reads T's request.
  6. T's WRITE puts the old request's bytes into T's file. T receives the old tag and fails Malformed, but the write has already landed.
• The same race without the wait. T can also be preempted between connection() and take.
• Contracts this breaks.
  • It breaks the module's contract on L1 (fs.rs:36-38): no two spans overlap, and at most 16 requests are unanswered.
  • It makes the Link doc (:798-800) promise less than readers will assume. The doc covers only a newer connection's window, not later requests on the same retired connection.
• Fix.
  • Take the slot first, then the connection, and give the slot back if connecting fails.
  • Every failed request retires its connection before it gives its slot back. So the next request in that slot goes out on a newer connection and window.

N3. sdk/toyos/src/fs.rs:1467-1473: the five-minute bound rests on a false premise and is shorter than one block request's own bound.

• "Each answered or refused within the disk service's minute" is false.
  • That minute is ANSWER_BOUND_NS (lib/blockring/src/wire.rs:77). It bounds only the answer to a listing or an open.
  • At this branch's base, a block session waits for its completion with no bound (system/diskserver/src/session.rs:296, wait_for(u64::MAX)).
  • On main since A block session never waits on its server's word alone #1235, the block protocol allows a server to be silent for SILENCE_BOUND_NS = 365 s (lib/blockring/src/lib.rs:69). That is more than 300 s.
  • A USB disk is recorded as able to wait past even that bound (issues/a-usb-disk-session-can-wait-past-the-block-silence-bound-behind-other-sessions.md, on main).
• "Five minutes is past any such sum" is unsupported.
  • The server is one loop, and one request can be many block requests. For example, an fsync writes back every owed page inside the loop (system/fileserver/src/pages.rs:826-842).
  • Nothing sums those block requests or the turns behind other connections.
• Consequence.
  • A disk that stays within its own contract, such as an NVMe controller reset of up to 365 s, makes every process owed a reply let the server go.
  • Each such process retires its connection, and every open file of that capability answers Stale from then on.
  • Before this PR those callers waited and continued.
• Failing every caller on the connection at expiry is right. The server serves a connection's requests one at a time, so silence on the connection is silence for all of them. Their file ids go stale with the connection anyway.
• Fix.
  • Derive the bound from what a reply waits on: at least the block silence bound once the branch rests on A block session never waits on its server's word alone #1235, plus stated room for the server's own turns.
  • Say at the constant that a file server's reply has no derived worst case. Record that gap in issues/ with owner and exit (severity: [hang]).
  • The manual's files page says that a crash invalidates a server's open files (docs/src/architecture/files.md:85-88). Add that a server silent past its bound while it owes a reply is let go as if it had crashed, as the storage page says for a block server.

N4. sdk/toyos/src/fs.rs:888: "lent the window again" is false. Each connection is now lent a window of its own (attach, :903-907). Delete the phrase or replace it with "lent a window of its own".

Optional

• For the orchestrator, corrections to the PR body.
  • "Every wait in the client ends within the bound" is false at this head because of N1.
  • It is also false for a blocking send to a hung server whose 256-message queue has filled with CLOSEs, which take no slot. That case predates this PR.
  • Under Risks, "each within the disk service's minute" is false (N3).

The questions asked

• Lost wakes: none. Every sleeper loads its word before taking the lock, and every change bumps the word after the lock is released.
• Lost turns: none within one connection, but one across connections (N1). The three named cases are correct by inspection:
  • A leader that times out. fail files every request on its connection, its own included. It hands on, call_with retires the connection, and every failed caller wakes.
  • A reply that arrives after its caller timed out. Only a later leader on that connection can receive it. file refuses the reply because the slot is idle or its link or tag differs, and fails only that connection's stragglers with Malformed. The late write stays in that connection's window once N2 is fixed.
  • A whole-window request whose connection fails. It holds every slot, so no other request waits. Its give frees every slot and wakes freed.
• The new error is mapped consistently.
  • Unanswered = 0x83 is local: code gives None, and from_code never yields it.
  • It crosses the C interface through from_byte, and both tests cover it.
  • libc maps it to ETIMEDOUT and std to TimedOut.
  • toyos-abi is rightly untouched, since this is the userland protocol's error.
• Per-connection windows. A newer connection's window is safe. The same retired connection is not (N2).
• Compiling osbench and the rust-tests binaries.
  • Inspection suffices; no build check is needed now.
  • Each edit drops a mut or changes &mut Dir to &Dir.
  • None of these files keeps a &mut use of a Dir, follow_target, or a copy of Refused. I checked with grep.
  • If a compile is wanted anyway, the harness's --metal-readback staging compiles every rust-tests binary without touching a machine.
  • Separately, no guest row today runs the members that put several requests on one connection: shared boots run only on metal until The shared boots run as guest rows, each member judged by its own exit #1271. That gap belongs in Risks, as it is now.

Evidence

My runs at ab93cc9. All ran in a detached worktree, now removed with its targets.

• cargo test -p toyos --lib with fs::, files:: and sync::: 13, 11 and 1 passed, rc 0.
• cargo test --manifest-path kernel/loom/Cargo.toml --test fs_flight: 6 passed in 16.2 s.
• The four controls from the author's evidence, each patch run model by model and then restored: the same reds as the PR's table, for the reasons given above.
• Clippy with the adopted lints and -D warnings, rc 0 on each shape:
  • -p toyos --all-targets;
  • -p toyos --lib --features fd-table;
  • -p kernel-loom --all-targets.
• cargo run -- --build-only --boot-config tests/images/testcases.toml: rc 0 in 160 s. This compiles the SDK, the supervisor, the file server and pagerprobe for ToyOS. The std sysroot is keyed on sdk/std and sdk/toyos/src.
• Scratch, outside the tree:
  • N1's model deadlocks at this head;
  • with the per-connection-turn variant of flight.rs, that model and the six pass.

The author's 20fe3e43/r1 evidence. It was taken at this head's tree, and my runs reproduce it.

Guest run 43107126235. Still queued, so not read.

SEND BACK

[Japabu]

Round 2 response

Fixed at b92c7d2. It now sits on #1258's merge of main (c809599), which brings #1235. Evidence: https://github.com/ToyOSOrg/evidence/tree/7700bf338c950a2a636d036e27cb913366f0a3b0/r2

N1, a turn per connection.

• receiving is now the mask of the slots whose callers lead. A caller leads when nobody leads its own connection.
• A leader, and the give of an unsent request, hand on only within their connection.
• Your model, each_connection_receives_for_itself, is in at bound 3. It deadlocks against round 1's flight.rs (r2/controls/turn-per-dir).
• With hand-on now per connection, the give's wake reds in no earlier model. A new model, a_failed_send_hands_its_turn_on, covers it: two callers and a failed send on one connection. That control now reds there.

N2, the slot before the connection.

• ticket takes the slot, then the connection, and gives the slot back if connecting fails.
• call_with retires the connection on every failure before the ticket drops, so a slot's next request goes out on a newer connection and window.
• The reason is stated at ticket.

N3, the derived bound.

• REPLY_BOUND_NS is the block silence bound, 365 s, plus 60 s of room for the server's own work and turns.
• The SDK cannot name SILENCE_BOUND_NS, because toyos-blockring depends on toyos. So the derivation is stated at the constant, and blockring asserts toyos::fs::REPLY_BOUND_NS > SILENCE_BOUND_NS, which holds the copy honest.
• The constant says a file server's reply has no derived worst case. That gap is recorded in issues/a-file-server-reply-has-no-derived-worst-case.md, with severity: [hang], an owner and an exit.
• The manual's files page says a server silent past its bound while it owes a reply is let go as if it had crashed, as a block server is.

N4. The phrase now reads "lent a window of its own".

Body. "Every wait in the client ends within the bound" is gone, and so is the disk-minute risk. The Risks now name both the reply bound's missing figure and the full-queue blocking send.

Checks, all rc 0:

• the SDK's fs::, files:: and sync:: tests;
• toyos's two clippy shapes;
• toyos-blockring's tests and clippy;
• the eight loom models (55 s) and the loom crate's clippy.

Controls. Each is run model by model:

• one turn per Dir: red;
• filing wake deleted: red in four models;
• leader's hand-on wake deleted: red in three;
• give's hand-on wake deleted: red in the failed-send model;
• fail ignoring the link: red.

This round changes no ToyOS-only code, so the images built at round 1 stand.

Guest. Run 43112790990 at b92c7d2 carries the four rows. 43107126235 is cancelled.

[Japabu]

Review of #1272 round 3 at b92c7d2 (on fileops-step1 c809599)

Architecture

The receive turn now lives where replies arrive.

• receiving is the mask of leading slots, and receives(link) derives a connection's turn from those slots' links. This adds no state.
• Hand-on, in both lead and give, is scoped to one link.
• Taking the slot before the connection closes N2 by ordering alone.

The bound is derived from the block protocol's silence bound.

• The gap is recorded and the manual sentence is added.
• The compile-time tie is incomplete (R1).

Size.

• This round: +135/−41. Production is +27 (fs.rs +17, flight.rs +6, blockring +4), models +42, record +24, manual +1.
• Whole PR: production is about +321 net:
  • fs.rs +157 and flight.rs +273;
  • owner.rs −18 and sync.rs −97;
  • libc and std +2, blockring +4.
• The loom shim and models add +361 and osbench +22.
• Manual +7; records +53/−25.

Round 2

• N1, resolved.
  • A slot's receiving bit is set only by its own caller in answer and cleared only by that caller in lead. Both happen under the lock, and the clear is in the same section that picks waiting(link).
  • While a slot leads, its link cannot change: only its own sent writes it. So receives(link) means exactly "a caller leads link".
  • The model each_connection_receives_for_itself deadlocks against round 1's flight.rs (e217642) and passes at the head.
• N2, resolved. See "Slot first" below.
• N3, partly resolved.
  • The bound, the record (severity: [hang], exit, owner) and the manual sentence are all in.
  • The manual sentence matches the storage page's rule for a block client.
  • The compile-time check does not tie the SDK's copy to its source (R1).
• N4, resolved. The doc now reads "lent a window of its own".
• Round 2's notes on the PR body are applied. "Every wait is bounded" and "the disk service's minute" are gone, and the unbounded blocking send is listed under Risks.

Required (blocker)

R1. lib/blockring/src/lib.rs:71-73 and sdk/toyos/src/fs.rs:1476-1490: the check ties only the order of the two bounds, not the SDK's copy of the block bound, so the stated room can shrink unseen.

• What the check covers.
  • REPLY_BOUND_NS is BLOCK_SILENCE_NS + 60 s, and BLOCK_SILENCE_NS is a private copy of toyos_blockring::SILENCE_BOUND_NS.
  • The assertion checks only REPLY_BOUND_NS > SILENCE_BOUND_NS.
• Consequence.
  • SILENCE_BOUND_NS can be raised to anything under 425 s and the build stays green. The room then shrinks toward nothing.
  • The docs at both SDK constants become false: "365 s", "60 s of room", "The block protocol's bound".
  • The decision was "at least the block silence bound plus stated room". The check enforces only the first half.
  • Root CLAUDE.md asks for shared declarations rather than private copies. Where the dependency direction forces a copy, the copy must be checked against its source.
• Placement.
  • The assertion makes the block protocol crate name the file protocol's client.
  • Blockring's header (lib.rs:43, "Pure: alloc, toyos-blockhold and toyos::transport") no longer lists everything the crate uses.
• Fix.
  • Assert that the copy equals its source, for example toyos::fs::BLOCK_SILENCE_NS == toyos_blockring::SILENCE_BOUND_NS, with the copy made public.
  • The natural home is the file server's library, for example system/fileserver/src/bounds.rs. It depends on both crates, is a host-tested workspace member, and is the component whose replies wait on block requests.
  • If the check stays in blockring, add toyos::fs to its header.
  • Either way, the parenthetical at REPLY_BOUND_NS can shrink to saying the copy is checked there.

Optional

S1. kernel/loom/tests/fs_flight.rs: no model reds when hand-on ignores the connection.

• The gap. I deleted && self.requests[i].link == link from waiting (flight.rs:271), and all eight models still pass.
• A model that catches it. One caller asks on connection 2 and two callers ask on connection 1. Connection 2's server answers only after both connection-1 callers have returned.
  • At the head it passes: 0.9 s at bound 1, 45 s at bound 2.
  • Under the patch it deadlocks in 1.8 s.
• Why it matters. It controls the decided rule that hand-on stays within a connection, for about a second of suite time.
• Why it is optional. The filter is plain on inspection.

The questions asked

No lost turn or lost wake across connections.

• A caller sleeps only after it has seen a leader on its own link. When that leader leaves, it wakes a slot still Waiting on the same link. The woken caller then does one of these:
  • leads;
  • finds a newer leader on its link, which takes over the duty to hand on;
  • finds its own request filed. Only a leader on that link files it, and a fail covers every Waiting request on the link.
  • has its send refused. give then hands on within the link.
• Hand-on never crosses connections, so a leader on another link cannot swallow the wake.
• My controls agree (table below). Round 1's per-Dir turn deadlocks the two-connection model, and so does a global turn with per-link hand-on.
• Every sleeper loads its word before taking the lock, and every change moves the word after releasing it, as in round 2.

A retired connection's callers are always drained.

• They lead their own link, independent of the current connection's traffic.
• Each leader's receive is bounded by REPLY_BOUND_NS. On expiry or end, fail files every Waiting request on that link.
• An observation, not a finding:
  • A request that took its slot before a failure can call connection() between the leader's fail and the retire in call_with. It then goes out on the failed connection.
  • It leads that connection and waits at most one more bound, or sees Gone at once.
  • Its slot was free when it was taken, so its span overlaps no failed request's span.

Slot first: the window-overlap trace is closed.

• A request that gets no reply fails in call_with. call_with retires the connection before it returns (fs.rs:1295-1297), and the ticket's drop gives the slot back afterwards (:1339-1342).
• The table lock orders any take of that slot after the give. So the taker's connection() runs after the retire and returns a newer link or connects again.
• Round 2's step 4, T sending on L1 in slot j, can no longer happen.
• Replies that arrive but fail checking (none_carried, one, opened) also retire before the drop, and the server is done with that span anyway.
• ticket's new error path gives back an idle slot, so it hands on nothing.
• No connecting and table lock is held across the other, so there is no new lock order.

The compile-time check ties order only, not the copy (R1).

The 60 s of room is stated, not derived. That is what it claims to be:

• the constant's doc and the record both say no figure sums the server's block waits or its turns;
• the record's claims about fsync and room-making write-back match system/fileserver/src/pages.rs.

This meets the decision ("stated room"). No finding.

Evidence

My runs at b92c7d2. All ran in a detached worktree with its own target directory, now removed.

• SDK tests. cargo test -p toyos --lib with fs::, files:: and sync::: 13, 11 and 1 passed, rc 0.
• Blockring. cargo test -p toyos-blockring --lib: 46 passed, rc 0. This compiles the assertion.
• Clippy with the adopted lints and -D warnings, rc 0 on each shape:
  • -p toyos --all-targets;
  • -p toyos --lib --features fd-table;
  • -p toyos-blockring --all-targets;
  • kernel/loom --all-targets.
• Loom. cargo test --manifest-path kernel/loom/Cargo.toml --test fs_flight: 8 passed in 46.7 s, rc 0.
• Controls. Each patch ran model by model and was then restored; the worktree was clean afterwards. Every red is a loom deadlock unless noted.

control	models that red
filing wake deleted	reply, slot-reuse, connection-end, failed-send
leader's hand-on wake deleted	reply, slot-reuse, failed-send
give's hand-on wake deleted	failed-send
fail ignores the link	fresh-connection, on its own assertion
round 1's flight.rs (e217642)	two-connection
receives ignores the link (mine)	two-connection, fresh-connection
waiting ignores the link (mine)	none (S1)

The rows that the author also ran match the PR's table.

The author's evidence (7700bf33/r2).

• checks/tree.txt records e217642 plus a working diff. Its production files match the head; it lacks the eighth model and the record.
• The 8-model log has no tree record.
• My runs at the head reproduce both.

ToyOS build.

• The base moved to c809599 since round 1's images.
• No user of this PR's removed API remains at the head: grep finds no follow_target, Turnstile or mut Dir binding.
• So the base move cannot break a ToyOS-only caller through this PR's API.

Guest run 43112790990. Still queued, so not read.

SEND BACK

[Japabu]

Round 3 response

Fixed at 04ced8d, rebased onto #1258's round 6 (66de9f9). Evidence: https://github.com/ToyOSOrg/evidence/tree/d146278ec430a6d50d07ba7f1526121be5f33d9e/r3

R1.

• toyos::fs::BLOCK_SILENCE_NS is now public.
• system/fileserver/src/bounds.rs asserts that it equals toyos_blockring::SILENCE_BOUND_NS. The file server depends on both crates, and its replies are what wait on block requests.
• The assertion is gone from blockring, which no longer names the file client.
• The note at REPLY_BOUND_NS is shortened to a link to the constant.

S1, taken. The new model is a_turn_handed_on_stays_on_its_connection: two callers on connection 1, beside a caller leading connection 2 whose server answers last. It runs at bound 2.

• Its control has waiting ignore the connection. It reds as a deadlock in that model and in no other.
• The other models stay green under it, so this model alone holds the rule.

Checks at the head, all rc 0 but the file server's clippy:

• the SDK's tests and both clippy shapes;
• toyos-blockring's tests and clippy;
• the file server's tests (132 + 3), which compile the assertion; its clippy shows main's four findings;
• the nine loom models (40 s) and the loom crate's clippy.

Controls, model by model:

• the three wake deletions, each red as a deadlock, in the models the body lists;
• the hand-on ignoring its connection, red as a deadlock.

flight.rs is unchanged since round 2, so that round's turn-per-Dir and fail-ignores-link controls stand.

Guest. Run 43117897753 at 04ced8d carries the four rows. 43112790990 is cancelled.

[Japabu]

Review r4 of #1272 at 04ced8d (on #1258's 66de9f9)

Scope. Two things:

• 04ced8d, which fixes round 3's R1 and takes its S1;
• the rebase of the three earlier commits from c809599 onto 66de9f9.

Architectural assessment

The block silence bound has one source.

• That source is blockring's SILENCE_BOUND_NS. The SDK, which blockring depends on, keeps a public copy, and the file server asserts the two are equal.
• That is the right owner: the file server depends on both crates, and its replies are what wait on block requests.
• Blockring no longer names the file client. Its header ("Pure: alloc, toyos-blockhold and toyos::transport") is true again, and git grep toyos::fs lib/blockring finds nothing.

Size this round.

• Production +12/−11: fs.rs ±7, bounds.rs +5, blockring −4.
• Model +27/−4.

The rebase changed nothing else

• git range-diff c8095992a..b92c7d2ff 66de9f97b..e2ed962a1 marks all three commits =.
• git diff c8095992a b92c7d2ff and git diff 66de9f97b e2ed962a1 are byte-identical.
• 66de9f9 changes only lib/bcachefs/src/engine/mount/data.rs and lib/bcachefs/src/engine/tests/data.rs, and this PR touches neither.

Round 3

R1, resolved.

• system/fileserver/src/bounds.rs:37-40 asserts toyos::fs::BLOCK_SILENCE_NS == toyos_blockring::SILENCE_BOUND_NS.

• The copy is public (sdk/toyos/src/fs.rs:1488-1490), and its doc says the file server holds it equal.

• REPLY_BOUND_NS is BLOCK_SILENCE_NS + 60 s (:1486). So raising the block bound now forces the copy and the reply bound up with it, and the 60 s of room cannot shrink unseen.

• The old order-only assertion now follows from the equality, and it went with blockring's mention of toyos::fs.

• The doc at REPLY_BOUND_NS no longer restates "365 s" or the long parenthetical; it links the copy.

• Controls (mine). Each was applied, checked with cargo check --manifest-path system/fileserver/Cargo.toml --lib, and restored:

  control	result
  the copy moved to 366 s	E0080, "assertion failed: toyos::fs::BLOCK_SILENCE_NS == toyos_blockring::SILENCE_BOUND_NS", at bounds.rs:40
  the source moved to 400 s	the same

• Where the assertion is compiled. --ci host runs cargo test --manifest-path for each program with host tests, which compiles the file server's library. Every guest build of the file server compiles it too.

S1, taken. kernel/loom/tests/fs_flight.rs:249-269, a_turn_handed_on_stays_on_its_connection.

• The shape is the one round 3 described. One caller asks on connection 2, and two callers ask on connection 1. Connection 1's server answers both before connection 2's server answers.
• It runs at bound 2, as the four-thread models do. The module header now names it.
• At the head: it passes within the nine-model suite.
• Control: waiting ignores the link. That is round 3's patch and the author's hand-on-any-connection.patch.
  • This model deadlocks. Loom reports the main thread, the connection-2 caller and one connection-1 caller blocked, and the other connection-1 caller terminated. That is the turn handed to the other connection's leader.
  • All eight other models pass under it, as the PR body says.
• Control: the leader's hand-on wake deleted (flight.rs:190). This model deadlocks too, which agrees with the PR's table.

Findings

None required. No suggestions.

Evidence

My runs at 04ced8d, in a detached worktree with its own target directory, with build/clippy.rs's adopted lints and -D warnings unless noted. After the controls were restored, git status was clean.

check	result
cargo test -p toyos --lib	rc 0, 192 passed
cargo clippy -p toyos --all-targets	rc 0
cargo clippy -p toyos --lib --features fd-table	rc 0
cargo test -p toyos-blockring --lib	rc 0, 46 passed
cargo clippy -p toyos-blockring --all-targets	rc 0
cargo test --manifest-path system/fileserver/Cargo.toml	rc 0, 132 + 3 passed
cargo clippy --manifest-path system/fileserver/Cargo.toml --all-targets	rc 101 with main's four findings only: listing.rs:94, pages/tests.rs:792 and :879, writeback.rs:91
cargo test --manifest-path kernel/loom/Cargo.toml --test fs_flight	rc 0, 9 passed in 37.2 s
cargo clippy --manifest-path kernel/loom/Cargo.toml --tests -- -D warnings, and -p kernel-loom --all-targets with the adopted lints	rc 0, rc 0
control, waiting ignores the link, each model alone	a_turn_handed_on_stays_on_its_connection rc 101 (loom deadlock); the other eight rc 0
control, the leader's hand-on wake deleted, the new model	rc 101 (loom deadlock)
control, the copy or the source of the bound moved	rc 101 (E0080 at bounds.rs:40), each

Guest run 43117897753 at 04ced8d: still queued (last updated 23:11:34Z), so there is no result.

The author's evidence. d146278e/r3/checks/tree.txt records 04ced8d. The controls summary matches the PR body's list and my two runs.

LAND. It still lands only after #1258, its base. #1258's round-7 blocker is a comment line in a file this PR does not touch, so fixing it would not change this review.

[Japabu]

Rebased onto #1258's merge of main: ba7b635

#1258 merged main d1cefba, with #1273, at a2e4dfd. This branch is rebased onto it. Main's changes under this branch met it in three places:

• adopt now takes a Connection, as main's adopt_exported and served_file_offset pass it. The rebased code drops the mut.
• Received handles are typed on main. The ticket's one answers a crate::ipc::ReceivedHandle. export and stream type it, as main's Dir did, and image and an open's holder turn it into an OwnedHandle.
• system/fileserver/src/bounds.rs keeps main's handshake Policy beside the block-bound assertion.

flight.rs, the loom shim and the models are unchanged.

Checks at ba7b635, all rc 0 but the file server's clippy:

• the SDK's tests and clippy;
• toyos-blockring's tests and clippy;
• the nine loom models (41 s) and the loom crate's clippy;
• the file server's tests (132 + 3); its clippy shows main's four findings;
• the default and testcases images.

Evidence: https://github.com/ToyOSOrg/evidence/tree/05bd1f86dde74c9220af61535ac9888748738a4f/r4

Guest. No run is dispatched here. 43117897753 ran at the old base. With main and #1273 now beneath this branch, it needs a run at this head before it lands; the orchestrator decides when.

[Japabu]

Review r5 of #1272 at ba7b635 (on #1258's a2e4dfd)

Scope.

• The rebase from cb11db4 (on 4240a3d) to ba7b635 (on a2e4dfd), read with git range-diff 4240a3d79..cb11db45c a2e4dfdac..ba7b635ec.
• Each file's added and removed lines, compared before and after.
• Main's A receive reports what each handle names, and a typed handle is made of a received one only where the kernel said so #1239 changes to the same calls, git diff 4240a3d79 a2e4dfdac -- sdk/toyos/src/fs.rs sdk/toyos/src/files/owner.rs, as the reference.
• I ran no tests or builds, as the reviewer's role requires. The results below are the author's r4 logs.

Architectural assessment

The rebase changed only the three places the author names.

• Commits 2 and 3 are =.
• Commits 1 and 4 differ only in those three places.
• The PR's added and removed lines are identical before and after in sdk/toyos/src/fs/flight.rs, kernel/loom/tests/fs_flight.rs, the loom shim in kernel/loom/src/lib.rs, sdk/toyos/src/sync.rs and system/fileserver/src/bounds.rs.
• Main's loom changes beneath this branch (the poll and shootdown models and their features) touch neither the shim nor fs_flight.

Each resolution keeps both sides' guarantees.

• adopt (files/owner.rs:390-391, served_file_offset.rs).
  • The signature is main's: adopt_exported(connection: Connection), which A receive reports what each handle names, and a typed handle is made of a received one only where the kernel said so #1239 types from the endowment.
  • This PR's own change there is still only the dropped mut.
• Reply handles (fs.rs:1012, :1102, :1160, :1165, :1169-1174, :1326-1336).
  • one answers a ReceivedHandle.
  • export and stream make their Connection and Pipe with typed(), and a mismatch answers Malformed.
  • image and an open's holder become an OwnedHandle.
  • Each conversion is the one main's Dir makes at the same call, so A receive reports what each handle names, and a typed handle is made of a received one only where the kernel said so #1239's rule holds: a typed handle is made only where the kernel reported its kind.
  • This PR's rules hold too:
    • a wrong count still retires the request's own connection (:1331-1333);
    • a wrong kind, as on main, answers Malformed and leaves the connection;
    • every early ? drops the ticket, which gives the slot back (:1339-1343).
• bounds.rs. The hunk that adds the assertion is byte-identical. Only its context moved, to main's HANDSHAKES: Policy::quartered(32), which stays.

Findings

None required. No suggestions.

For the next rebase. #1258's B2 changes held_write in sdk/toyos/src/files/owner.rs, which this PR also edits, so the next rebase should be range-diffed again.

Evidence

The author's r4 logs (05bd1f86). tree.txt records ba7b635. All are rc 0 except the file server's clippy:

• cargo test -p toyos --lib filtered to fs:: (13), files:: (11) and sync:: (1), the modules the PR touches;
• clippy -p toyos --all-targets and --features fd-table;
• toyos-blockring's tests and clippy;
• the nine fs_flight loom models, 41 s, and the loom crate's clippy;
• the file server, 132 + 3;
• the default and testcases images;
• the file server's clippy: rc 101, main's four findings.

Guest.

• Run 43143742563 at ba7b635 is still queued.
• 43117897753, at the old base, never reported.
• So no guest row has a result for this PR at any head. Every program's file calls go through the changed client, so that run's four rows are a landing condition, after An open file is written and stated through its paged object, one write at a time, against credit #1258 lands.

Metal: none. It is not a landing condition while the benches are down.

LAND, once #1258 has landed and run 43143742563's rows pass.

[Japabu]

Rebased onto #1258's round-8 head 3581821 (main 3c5c555 merged there). The range-diff is = for all four commits (r5/range-diff.txt).

Checks at the head 6bf86dd, all rc 0: the SDK's tests (214) and two clippy shapes, nine loom models, the file server's tests, and the testcases image, which builds libc's and std's arms from both branches (r5).

Guest: run 43147796257 at 6bf86dd runs the four rows and testcases_members_exit_0. 43143742563 was cancelled while still queued.

[Japabu]

Rebased onto #1258 cb8701b (round 9: the stale holder assertion in a_served_file_reads_through_its_paged_object). Commits 2 to 4 are =; the first differs only in that member's context line. Every rust-tests binary type-checks against the ToyOS sysroot (r6).

Guest: 43152649297 at 2e7dd79 runs the four rows and testcases_members_exit_0. 43147796257 was cancelled while queued.

[Japabu]

Rebased onto main ee15a2e (#1258 merged, beside #1252). #1252 made a directory capability a connector the process holds. The conflicts were Dir::connect/reconnect, the SDK's capability table, the supervisor's sync and three guest tests. Each keeps main's connector and this PR's shared connection, and commit 4 is = (range-diff and checks). The code changed, so 43152649297 no longer stands for the head. Run 43165795439 at d88c74f runs the four rows and testcases_members_exit_0.

Error::Unanswered's libc and std arms move into #1227's Failure table, whichever of the two lands second.

[Japabu]

Review of #1272's rebase onto #1252 at d88c74f

Scope: the conflict resolutions with #1252 only. The design (per-connection receive turn, slot before connection, the 425 s reply bound) is settled and not reopened.

Architecture

The resolutions are the natural composition. #1252 owns where the connection comes from (Source::Held, a connector borrowed for the process's life). #1272 owns how one connection is shared (link, connecting, flight). Each kept its part. Dir::connect is main's signature with #1272's lazy connection(). reconnect is main's body with #1272's &self -> Link shape. Dir::named and the SDK's capability() are main's, holding Arc<Dir> as #1272 did. Nothing new was invented to make them fit.

What I verified

• Range-diff (cb8701bee..2e7dd79aa against ee15a2e5b..d88c74fa1). Commit 4 is =. Commits 1–3 differ only in these places:

  • Source::Named became Source::Held, and names.open became port::connect;
  • NotFound → NotHeld became PermissionDenied → NotHeld;
  • Dir::connect(names, …) became Dir::named(…) or Dir::connect(held.as_handle());
  • the imports and the CAPABILITY_PREFIX doc follow main;
  • every mut drop is unchanged.

  The commit messages are unchanged and name no namespace.

• A child's handles are a list its parent names: labels and namespaces leave the kernel #1252's semantics kept. Every hunk A child's handles are a list its parent names: labels and namespaces leave the kernel #1252 made in fs.rs and owner.rs is present at the head: the header, the imports, Source::Held, connect, named, reconnect and capability(). For each of the ten files both sides touched, git diff ee15a2e5b d88c74fa1 shows only A process's threads have their requests to a directory in flight at once #1272's lines, so main's changes there survive. Nothing named Turnstile, Namespace or follow_target remains. The deleted issue is unchanged on main since the old base, so no modify/delete hunk was lost.

• Every way sys_connect fails, and the word for each (kernel/src/syscall/ipc.rs:68, handle.rs:53). Each maps the same as on main:

  • Rights (no WRITE) → PermissionDenied → NotHeld, main's "one it may only pass on".
  • Acceptor closed → Gone → Error::Gone. The supervisor keeps a boot service's acceptor across a restart (system/supervisor/src/main.rs:26-34). So a reconnect through the same connector queues for the restarted server, and its hello is bounded by REPLY_BOUND_NS. A service with no restart owed answers Gone on every later attempt, and path_call_then's single retry stops.
  • Queue full, table full or charge → ResourceExhausted → OutOfResources.
  • WrongType → InvalidArgument → the SDK panics in Error::kernel. BadHandle and Stale are a handle fault. Both are main's existing treatment of a parent-endowed handle and are unchanged here.

• Two threads reconnecting at once (fs.rs:876-900). Both find link empty. One takes connecting, and the other waits there and then finds the new Link. If the connect fails, each waiter tries for itself and gets its own word. Lock order is always connecting then link, and retire takes only link, so the locks cannot deadlock. A late failure on the old link calls retire. Its ptr_eq check leaves the new link installed. A Fid from the old generation is Stale on the new link. generation moves only after a successful hello.

• Callers. room, sync and every other Dir method take &self (fs.rs:1140, :1145). Each dropped mut (pkg :75, supervisor :2071, osbench and the five rust-tests binaries) is therefore sound by inspection. The supervisor's sync passes a BorrowedHandle<'static> from its 'static files.

• Evidence r8 at d88c74f, with a clean tree before and after. All of the following passed (rc 0):

  • cargo test -p toyos --lib: 224 passed;
  • clippy on toyos, all targets, and with fd-table, with the adopted lints;
  • the nine loom models;
  • the file server: 132 + 3 passed;
  • the testcases image;
  • cargo check --bins of rust-tests against its sysroot.

Findings

B1 — PR body, "Rebased onto main ee15a2e": one build claim is false, and the linked evidence holds two failed runs the body does not mention.

• pkg was not built. The body says the testcases image built "libc, std, the supervisor and pkg". r8/build-testcases.log compiles supervisor but never pkg, and tests/images/testcases.toml has no pkg row. r8's only other pkg run, cargo clippy -p supervisor -p pkg, stopped at pkg's library. So pkg's changed line (system/pkg/src/main.rs:75) was compiled by nothing at the head.
• Two failed runs. r8/summary.txt records supervisor rc=101 and clippy-supervisor-pkg rc=101, linked from the body as "checks" with no explanation. My reading is that neither comes from this change:
  • all 44 errors in supervisor.log are std::os::toyos and ToyOS Command extensions that do not exist on the host, and none is at :2071;
  • the clippy failure is redundant_clone at system/pkg/src/archive.rs:177. That line is identical on main, in a package that build/clippy.rs already excludes from its host shapes.
• Consequence. A reader of the record would take pkg as compiled and would see two reds nobody investigated.
• Fix: the description only; the code needs no change. Strike pkg from the image's list. Say that pkg's one change rests on inspection: room and sync take &self, and the closure at :94 borrows. Alternatively, cite a row in 43165795439 whose image builds pkg. Name the two rc 101 runs and why each is independent. While editing, "(the rebase's evidence above)" in "Not built here" points to a paragraph that comes below it.

No other findings. The resolutions keep #1252's connector and words and #1272's shared connection, restart path and receive turn, and no check was lost.

SEND BACK (B1 is a correction to the PR body, which the orchestrator owns; nothing in the branch needs to change).

[Japabu]

Rebased onto main 29c5f2d (#1227, #1264, #1280); head 4227aa4.

• Error::Unanswered moves into One table reads every refusal, each runtime keeps one from it to its word, and _SC_ARG_MAX is what a caller may pass #1227's Failure table as TimedOut, which libc answers as ETIMEDOUT and std as TimedOut. One table reads every refusal, each runtime keeps one from it to its word, and _SC_ARG_MAX is what a caller may pass #1227 removed libc's and std's own arms. The match is exhaustive, so the compile settles that the arm is there. Commits 1, 3 and 4 are = (r9).
• Checks at the head, each rc 0: the SDK (227) and two clippy shapes, nine loom models, the file server, the testcases image (a fresh sysroot), every rust-tests binary type-checked, and cargo check -p pkg --all-targets on the host.
• B1 (body): pkg is struck from the image's list, and pkg's change is compiled by that host check. The two rc-101 runs in r8 are named, with why neither depends on this PR. The pointer now reads "Rebased onto main, below".
• d88c74f, run 43165795439's commit, stays fetchable as fileops-tags-hold-d88c74fa1.

[Japabu]

Orchestrator verification at 4227aa4, the rebase onto main 29c5f2d (#1227). I read the range-diff against d88c74f. Commits 1, 3 and 4 are =. Commit 2 changes only Error::Unanswered's mapping, which moves from libc's and std's old tables into #1227's Failure table as Self::TimedOut (libc ETIMEDOUT, std TimedOut, the same words as before). The match is exhaustive and the author's fresh-sysroot testcases build compiled it.

The rebase review's B1 is answered in the body:

• pkg is compiled by cargo check -p pkg --all-targets;
• the two rc-101 runs are named and shown independent;
• the pointer is fixed.

Landing run 43165795439 at d88c74f, base ee15a2e:

• all four rows pass;
• testcases_members_exit_0 shows 241 of 243 members exiting 0;
• the two that don't, abuse_listener_hijack and adopted_services_stay, are A child's handles are a list its parent names: labels and namespaces leave the kernel #1252's defects that A sent handle of another kind is InvalidArgument whatever its rights, and adopted_services_stay checks only its own copy of toyos #1280 fixed on main 7814a21. They are independent of this PR.

Queueing.