diff --git a/issues/getsockname-answers-an-address-nobody-asked-netstack-for.md b/issues/getsockname-answers-an-address-nobody-asked-netstack-for.md deleted file mode 100644 index c5855f4c709..00000000000 --- a/issues/getsockname-answers-an-address-nobody-asked-netstack-for.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -status: open -kind: defect -opened: 2026-09-08 ---- - -# `getsockname` answers an address nobody asked netd for - -`sdk/libc/src/socket.rs`'s `getsockname()` fills the caller's `sockaddr` -with `[10, 0, 2, 15]` and the socket's local port. The address is a literal in -that function; netd is not asked, and the SDK has nothing to ask it with — -`toyos::net` carries a stream's connect, a listen, an accept, the datagram -calls and `dns_lookup`, and no call that answers "what is this machine's -address": a socket's `toyos::net::socket::local` is the address its bind -named. - -It was true by coincidence until now: netd carried the same literal, so the -shim and the stack agreed. netd takes its address from DHCP as of the change -that filed this, so the two agree only on a machine whose server happens to -lease `10.0.2.15` — QEMU's user-mode backend does, and the bench's router does -not. Every C program that asks what address it is bound to is told the wrong -one there. - -What it costs to fix is a message type on netd's protocol and one plumbed -answer; what it costs to leave is that the one caller of `getsockname` in a -POSIX program is the one that then advertises an address nothing can reach. -Nothing in the tree reads it today, which is why this is a finding and not a -defect. - -**Now a defect, 2026-11-01.** std carries the same literal: -`sdk/std/sys/net/connection.rs`'s `TcpStream::socket_addr` answers -`10.0.2.15` with the stream's local port. Since #1122, osbench's -`sys::link_of` names a connection's link by its local address, so on any -network that leases another address `osbench_lan` refuses the group. The -T14 session for the ramp issue read `bench=lan refused=no_netstack_link_holds_10.0.2.15` -on a lease from the bench's router. QEMU's guest test passes only because -slirp leases that address. The fix gives a stream's local address with -its port: netstack's connect and accept answers carry it, and the SDK's -`stream::endpoints`, std and libc return it. diff --git a/issues/no-published-run-compares-toyos-with-linux-on-one-machine.md b/issues/no-published-run-compares-toyos-with-linux-on-one-machine.md index b710a7fd1c1..3b1d8bae098 100644 --- a/issues/no-published-run-compares-toyos-with-linux-on-one-machine.md +++ b/issues/no-published-run-compares-toyos-with-linux-on-one-machine.md @@ -49,8 +49,9 @@ test depends on the public internet. The first paired session (`ToyOSOrg/evidence` branch `pr-1065`, `r3/`) put ToyOS's T14 LAN download and 64 B round trip at Linux's and its upload at -a third of it (`issues/tcp-opens-a-connections-window-slower-than-linux-in-its-first-300-ms.md`, -where what remains of that is now recorded). +a third of it. That gap was closed by #1069, #1102, #1112 and #1114, and +the T14 parity session of #1147 read the upload and its first 300 ms at +Linux's ([evidence](https://github.com/ToyOSOrg/evidence/tree/1c9f4c83c362a1b30746f1e096ad412a2d5f25f9/r2/session)). The cryptographic rows are not osbench's. AES-GCM, ChaCha20-Poly1305, SHA-256 and the rest were measured under Linux on both benches against ring, diff --git a/issues/tcp-opens-a-connections-window-slower-than-linux-in-its-first-300-ms.md b/issues/tcp-opens-a-connections-window-slower-than-linux-in-its-first-300-ms.md deleted file mode 100644 index a37e0ac54c9..00000000000 --- a/issues/tcp-opens-a-connections-window-slower-than-linux-in-its-first-300-ms.md +++ /dev/null @@ -1,383 +0,0 @@ ---- -status: open -kind: defect -opened: 2026-10-24 ---- - -# [tcp] opens a connection's window slower than Linux in its first 300 ms - -On the T14's wired path to the AMD bench's Wi-Fi peer, a paced ToyOS TCP -sender (#1114) delivers 6.07 MB in a new connection's first 100 ms and -29.2 MB by 300 ms. In the same session the bench's Linux delivered -6.10–6.38 MB and 28.6–29.6 MB. The upload row reads 114 MB/s against -Linux's 113. Only the 100 ms mark is short, by 30 KB in one boot (the -parity boot, at the end of this record). Before #1114, ToyOS delivered -2.0–3.3 MB and 23.8–25.6 MB. Received TCP is at parity on that path. - -This record began as the upload at a third of Linux's (41 MB/s against 113). -What follows is that history, the causes found and removed, and the residual. - -**Measured** on 2026-10-24 with `osbench lan` at `d99ef5a35`. The path is -the T14's I219 at 1000 Mb/s, through the access point, to the AMD bench's -Linux `osbench-peer` on its Wi-Fi. Each cell is one run's p50 of five 3 s -rounds: - -| | ToyOS (3 runs) | Linux (4 runs) | -|---|---|---| -| `lan.wired.tcp.upload`, MB/s | 41, 48, 31 | 113, 113, 114, 113 | -| `lan.wired.tcp.download`, MB/s | 112, 112, 109 | 112, 110, 109, 112 | - -- No ToyOS round reached a Linux one: ToyOS's rounds ran 31 to 59 MB/s and - Linux's 112 to 114. The exact rank test over the runs gives p = 1/35. -- The same runs' HTTPS upload to the public internet took 1.40 times - Linux's time. - -Evidence: `ToyOSOrg/evidence` branch `pr-1065`, `r3/`. - -The paired session at `aeba59fb0` read 62, 63, 65 and 64 MB/s against -Linux's 113, 110, 111 and 112 (p = 0.029): [evidence `aeba59fb0/`](https://github.com/ToyOSOrg/evidence/tree/37288c84c175a647145fe83fd51d945806cdb4ff/r1/aeba59fb0). - -**Not the limit on the 2026-10-27 and -28 heads: the transmit ring's -depth.** At 16 descriptors the ring holds -15 frames, 184 µs of a gigabit wire, and 64 MB/s is exactly 15 segments -every 339 µs, so a ring drained before the pass that refills it was the -candidate. Pull request #1069 read it at 256 descriptors on 2026-10-27, -four ToyOS runs interleaved with four Linux ones: ToyOS 36, 36, 39 and -44 MB/s, Linux 112 in all four. The rounds inside each ToyOS run ranged from -27–35 to 46–67 MB/s, and downloads stayed at parity (110–113). On that -head the deeper ring moved nothing; whether depth limits a sender whose -window grows as it should is not read by it -([evidence `af239c8bb/`](https://github.com/ToyOSOrg/evidence/tree/37288c84c175a647145fe83fd51d945806cdb4ff/r1/af239c8bb)). - -**Ruled out: loss.** Pull request #1069's counters read on 2026-10-28, four -runs on the 16-descriptor ring and two on 256, against Linux's 111–112: no -retransmission timeout and no recovery in any run, one 1448-byte segment -resent across all six, no frame missed by the card. Linux resent none. The -256-descriptor ring's two runs read the same 36 MB/s with a ninth to a half -of the 16-descriptor ring's 60,665–66,044 full-ring waits -([evidence `132d5ef75/`](https://github.com/ToyOSOrg/evidence/tree/f06c0e92fa4c7172fe5c0946c540c34b434179dc/r2/132d5ef75)). - -**Found on the way: slow start grew once per pass.** [tcp]'s cwnd grew by -one segment for the first ACK after a stop at cwnd, and not for the rest: -netstack hands the stack every ACK one pass received before the pass's -transmit opportunity, so a pass's ACKs grew the window as one. #1069 makes -each ACK count (RFC 5681 §3.1), and the window grow only after an -opportunity cwnd ended, not one the card's room ended (RFC 9438 §5.8). -That is a correction the RFCs require, held by [tcp]'s host tests -`s_cc_016` and `s_cc_017`. Its effect on the T14's upload is read by one -session that alternated it with a control, below; the other readings come -from separate sessions and credit it nothing. - -**Read at `afe0c7dac`, 2026-10-28**, a head that also grew the window while -the card held the sender, so neither its rate nor its stop samples are the -per-ACK fix's alone; the corrected head, `d184223b0`, is read below. Upload 58, 59, 58 and 59 MB/s against -Linux's 110–112 ([evidence `afe0c7dac/`](https://github.com/ToyOSOrg/evidence/tree/695218c8513e785bf0465e2f13e0575f5723a2e3/r3/afe0c7dac)). Each ToyOS row's 15 s was -held by cwnd for 1.5–2.1 s, at stops whose mean cwnd was 192–228 KB and -mean smoothed round trip 4.5–5.9 ms; by the peer's window for 87–189 ms; -by the sender itself for about 1 ms. Every resent segment was a loss -probe, and nearly every one came back D-SACKed. Linux ended its rounds at a -mean cwnd of 1,706–2,208 segments and a round trip of 9.7–11.7 ms: its -window is never the limit, and its own queue fills the path. - -**Two hypotheses, which the `d184223b0` session tested**: - -- The card's refill limits a sender whose window no longer does: at - `afe0c7dac` the card was full 100,053–106,424 times a row, though that - head's cwnd times are not creditable. If so, the 256-descriptor ring - reads faster than 16 at `d184223b0` and its full-ring waits fall. -- The deeper ring read no better on 2026-10-27 and -28 because slow start, - growing once per pass, held those rows at cwnd. Nothing measured that: - `af239c8bb` had no counters, and `132d5ef75` charged Nagle holds to - congestion. - -**Read at `d184223b0`, 2026-10-28** -([evidence `d184223b0/`](https://github.com/ToyOSOrg/evidence/tree/bc4188c1070d283e38f7a96019437b99697fe4c5/r4/d184223b0)), four runs on 16 descriptors and two on 256, -against Linux's 112–113: - -- On 16 descriptors ToyOS read 58, 61, 59 and 57 MB/s. -- The ring's depth is not the limit. 256 descriptors read 60 and 57 MB/s. - The card was still full 77,036–79,567 times a row (100,246–106,959 on 16). - cwnd held the sender 4.6–4.9 s of a row (2.1–2.4 s on 16), at a mean of - 488–506 KB and 8.0 ms (310–354 KB and 4.1–5.6 ms on 16): on the deeper - ring cwnd was larger and the round trip longer, and the rate was the - same. That the deeper ring's own queue lengthened the round trip is an - inference from those three readings, not a measurement. -- No loss: one segment resent, D-SACKed, across the six runs, and no - probe that ended a slow start. - -So the first hypothesis did not hold: with the corrected slow start the -deeper ring still reads no better. The second is not settled by this -session, which has no unfixed control. What remains open -is how fast the card drains what it is given: even at 256 descriptors it -is full about 5,000 times a second. That lead is the I219's transmit -configuration (interrupt moderation, descriptor write-back, how often -netstack learns of completions) and is taken up apart from this record's -pull request. - -**The per-ACK fix, measured: 2026-10-28, interleaved with its control** -([evidence `3599f5bb3/`](https://github.com/ToyOSOrg/evidence/tree/d03d7394ed2d34deb1f0c9244bd85f78d7551537/r5/3599f5bb3)). A was `3599f5bb3` with only the per-ACK -fix reverted, every counter kept; B was `3599f5bb3`; four of each, A B A B, -with Linux at the session's start and end (109 and 110 MB/s). - -- B read 59, 56, 57 and 57 MB/s; A read 52, 31, 31 and 40. B was faster in - every pair, and over the eight runs the exact rank test's one-sided p is - 1/70. -- cwnd held each 15 s row 2.2–2.5 s under B, 5.5–8.9 s under A, at a mean - cwnd of 335–354 KB against 266–301 KB. -- Pooled across the four runs of each, A's rounds spanned 30–60 MB/s and - B's 55–66. - -So in one session the fix raises the rate and narrows the spread of its -rounds, pooled across runs, from 30 MB/s to 11. Heads without it read 31–65 MB/s across -earlier sessions while Linux held 110–114. The fix is consistent with most -of that, but not with `aeba59fb0`'s 62–65, above B's reading here. That -part of the spread across sessions stays unexplained. - -**The card sends slowly, read 2026-10-28** ([evidence `c54355277/`](https://github.com/ToyOSOrg/evidence/tree/5db2f7f5ed4e54c12fd74c8aa2368de4b4a6590a/r1/c54355277)). -Pull request #1075 counted, at each full ring, the frames `GPTC` had on the -wire that netstack had not seen written back: none, in either of two runs, -while the ring sat full 10.2–10.4 s of each 15 s row. The part sent what it -held at about 42,000 frames a second, 23.8 µs a frame against 12.3 µs of -wire. - -**Measured 2026-10-28** ([evidence `f9563de4f/`](https://github.com/ToyOSOrg/evidence/tree/d6fbb4b149683056857dd8554c2c77cfbdd465ef/r2/f9563de4f)): -`TCTL`'s `MULR` and `RRTHRESH` set, against the bring-up's whole write -clearing both, raises the upload. One session alternated the two, four of -each, with Linux at both ends (111 and 110 MB/s): - -- The bring-up left `TCTL` at `0x3003f0fa` with both fields and `0x0003f0fa` - without. -- Upload p50 87, 86, 82 and 85 MB/s with them, against 58, 56, 55 and 56 - without: faster in every pair, and over the eight runs the exact rank - test's one-sided p is 1/70. -- The wire carried 870,000–912,000 frames a row with them, against - 618,000–626,000 without. No completion was unreported at any full answer. -- The ring still sat full 8.3–8.7 s of each 15 s row (10.3–10.6 s without), - and cwnd held the sender 4.1–4.5 s (2.3–2.6 s without). - -**Derived from the datasheet, not separated by the measurement**: which of -the two fields the gain is. 82574 §10.2.6.1 has `MULR` set the number of -read requests for transmit data: "When set to 0b, the 82574 submits only -one request at a time, When set to 1b, the 82574 might submit up to four -concurrent requests". It gives `MULR` an initial value of `1b` and -`RRTHRESH` `01b`. - -**The T14's reset does not give §10.2.6.1's initial values.** §10.2.6.1 -says `MULR` "is loaded from the NVM word 0x24/0x14", and §4.2 that a -software reset "re-reads the per-function NVM fields" and resets the bits -normally read from the NVM to "their default hardware values". On the T14, -`TCTL` read `0x3003f0f8` both before and after the bring-up's reset -([evidence `19fbf239c/`](https://github.com/ToyOSOrg/evidence/tree/4cbc624ffec885040aa268ad47743ed5df94e51e/r4/19fbf239c)), `CT` and `COLD` included, where those -values would read about `0x30000008`. Either the reset kept the register or -this MAC's own reset value is that one: no reading separates the two, and -the 82574's table does not describe the PCH's MAC. Either way the value the -bring-up met was not one the driver can count on, so it now writes every -`TCTL` field itself, `MULR` and `RRTHRESH` at §10.2.6.1's initial values. - -**The rest of the transmit setup the datasheet requires, read 2026-10-28** -([evidence `800e35c69/`](https://github.com/ToyOSOrg/evidence/tree/a911b388eac3990bde439b95091b41a00335748c/r1/800e35c69)): no measured rate effect. Pull request #1078 sets -`TXDCTL` bit 22, which 82574 §10.2.6.11 says "Must be set to 1b for proper -operation", leaves `TIDV` alone, sets `TARC0`'s `ENABLE` and writes the -receive filter registers. One session alternated main's driver (A), that -change (B) and that change with transmit prefetch (C, `PTHRESH` 16, -`HTHRESH` 1), three of each, with Linux at both ends (110 and 111 MB/s): - -- Upload p50 A 86, 86 and 85 MB/s, B 85, 86 and 86, C 88, 87 and 87. B reads - no different from A, and lands as the datasheet's requirement, not a gain. -- C read higher than B in all three rounds, by about 1.5 %, which three runs - a side cannot resolve: the exact rank test's least p is 1/20 one-sided and - 1/10 two-sided. The prefetch tuning did not land. -- The ring sat full 7.8–8.1 s of each 15 s row in all three arms, and no - completion was unreported at a full answer. - -What is left, about 1.3 times short of Linux with the ring full about half -of each row whatever the descriptor fetch does, is the next lead: what the -part waits on for each frame once it holds it. - -**The part keeps line rate while it holds frames; the gap is each -connection's first second, read 2026-10-28** ([evidence `d1ea41856/`](https://github.com/ToyOSOrg/evidence/tree/9f309c28654d89c37e8fb0defc9fd2b1c61cc91b/r1/d1ea41856)). -On the full ring the I219 sent 83,600 frames a second, line rate. One session -read what the peer had received after its first byte, a mean over each row's -five connections, at `d1ea41856` (A) and A with a card-held stop counted as -cwnd's (B), two of each, with Linux at both ends: - -| received by | 100 ms | 300 ms | 1 s | 2 s | upload p50 | -|---|---|---|---|---|---| -| Linux (2 runs) | 5.9–6.4 MB | 27.9–29.0 MB | 109–110 MB | 224–225 MB | 113, 113 | -| A (2 runs) | 0.32 MB | 2.0–2.1 MB | 38–41 MB | 149–153 MB | 88, 89 | -| B (2 runs) | 0.32–0.35 MB | 2.1–2.3 MB | 46–47 MB | 162–163 MB | 93, 92 | - -- From 1 s to 2 s ToyOS carried 112–117 MB, Linux 115. Nothing was lost. -- The peer acknowledged about 50 KB, 34 segments, an ACK. Each ACK that grew - cwnd grew it by one SMSS (RFC 5681 §3.1's min(N, SMSS)), so slow start grew - a few percent a round trip instead of doubling. -- Under A, 75–76 % of ACKs grew nothing: the card's room had ended the - opportunity before them with more than a segment of cwnd left. - -Pull request #1102 replaces the initial slow start with HyStart++ -(RFC 9406, which RFC 9438 §4.10 recommends for CUBIC), whose L = 8 for a -sender that does not pace caps the growth an ACK gives at eight segments, and -counts a card-held window as in use when at least half of it is in flight -(RFC 7661 §4.3's validated window, which RFC 9438 §4.6 admits). - -**#1102's A/B, 2026-10-28** ([evidence](https://github.com/ToyOSOrg/evidence/tree/8f70147d28011645de1fd458f3091297287b57b9/r2/ab)). A was the fix -reverted, B the fix; three of each, A B A B A B, with Linux at both ends: - -| | upload p50, MB/s | received by 100 ms | 300 ms | 1 s | 2 s | -|---|---|---|---|---|---| -| Linux (2 runs) | 113, 113 | 6.4–6.5 MB | 29.7 MB | 110–111 MB | 226–227 MB | -| A (3 runs) | 87, 87, 87 | 0.33–0.34 MB | 2.0–2.1 MB | 34–38 MB | 144–149 MB | -| B (3 runs) | 112, 112, 112 | 2.1–3.3 MB | 23.9–25.4 MB | 106–107 MB | 222–223 MB | - -- B was faster in every pair: one-sided p = 1/20. No segment was resent in - any B run, and no loss probe was sent. -- B's rounds ran 111–115 MB/s. Linux's 113 in this session is above them, - inside Linux's 109–114 across this record's sessions. -- What is left is the first 300 ms. In every B row HyStart++ entered - Conservative Slow Start 590–702 times, about 120–140 a connection, and - resumed every time, never exiting. CSS grows by a quarter while it lasts. - That this oscillation is the residual is an inference: no run separates it. - -The clock does not separate the two systems. Each upload row's own -readings, `mhz` just after it and `mhz_settled` before it, were about -1.6 GHz under ToyOS (settled 1.6, 1.8 or 3.8 GHz). Under Linux they were -1.5 GHz (settled 1.5 or 4.2 GHz), and Linux still moved 113 MB/s. At most, -the clock matters through what ToyOS spends per byte sent. - -**#1112's session, 2026-10-29** -([evidence](https://github.com/ToyOSOrg/evidence/tree/96c693b58d242d8ec236e0805b34b714306ec4c2/r1/t14)). -B was #1102's head, C per-segment RTT samples (#1112), D C with L = ∞ and -no pacing, a diagnostic arm that does not land. Three of each, B C D B C D -B C D, with Linux at both ends: - -| | upload p50, MB/s | received by 100 ms | 300 ms | 1 s | -|---|---|---|---|---| -| Linux (2 runs) | 113, 114 | 5.9–6.0 MB | 27.2–29.0 MB | 108–110 MB | -| B (3 runs) | 112, 112, 112 | 2.1–2.4 MB | 24.0–24.7 MB | 105.5–105.8 MB | -| C (3 runs) | 112, 112, 112 | 2.2–3.2 MB | 24.3–25.6 MB | 105.2–106.6 MB | -| D (3 runs) | 114, 113, 114 | 5.2–6.4 MB | 28.3–29.7 MB | 109.9–111.2 MB | - -- L = 8 is what holds the ramp. Without it D reached Linux's marks, and - nothing was resent. -- CSS entries were 2.0–2.7 times the rounds that entered CSS, in D as in B - and C. Most of them are flaps within a round (s_cc_023), and removing them - was not needed for parity. -- Finer RTT samples (C) did not change the ramp. - -RFC 9406 §4.3 gives L = ∞ only to a paced sender. Pull request #1114 -paces [tcp] (RFC 9002 §7.7) and removes the cap. - -**#1114's A/B, 2026-10-31** -([evidence](https://github.com/ToyOSOrg/evidence/tree/58dcdecab592116996c0e3cc84e4e3c8b7aa50c7/r4/t14)). -C was #1112 after its first review, P #1114's pacing with N = 2 in slow -start, Q P with N = 1.25 throughout. Three of each, C P Q C P Q C P Q, with -Linux at both ends, on trees from before netstack's drivers became their -own processes (#1106): - -| | upload p50, MB/s | received by 100 ms | 300 ms | 1 s | -|---|---|---|---|---| -| Linux (2 runs) | 112, 112 | 5.7 MB | 26.1–26.2 MB | 105.7–106.4 MB | -| C (3 runs) | 112, 112, 112 | 2.0–3.1 MB | 23.8–25.4 MB | 105.4–106.8 MB | -| P (3 runs) | 114, 114, 114 | 6.3–6.6 MB | 29.3–29.9 MB | 110.9–111.1 MB | -| Q (3 runs) | 114, 114, 114 | 5.9–6.3 MB | 29.1–29.5 MB | 110.2–111.0 MB | - -- P beat C at 300 ms in every pair, and resent nothing. -- P beat Q at 100 ms in every pair, by 0.1–0.5 MB, so N = 2 in slow start - stays. -- In C every RTT sample but two came from a segment's departure. -- That session's bucket could hold up to a millisecond at the rate, which on - a short round trip is more than the window. #1114's pacer, as of - 3a0744d34, holds at most half the window. The parity read waits on a boot - of that head, on current main. - -**The parity boot, 2026-11-01** -([evidence](https://github.com/ToyOSOrg/evidence/tree/8f08a8e9399817fc669e395dc717ee236a18ed35/r7/t14)). -One boot of #1114 at 3a0744d34, on main after #1106, with Linux before and -after it: - -| | upload p50, MB/s | received by 100 ms | 300 ms | 1 s | 2 s | -|---|---|---|---|---|---| -| Linux (2 runs) | 113, 113 | 6.10, 6.38 MB | 28.63, 29.56 MB | 109.1, 110.5 MB | 224.6, 226.1 MB | -| ToyOS | 114 | 6.07 MB | 29.19 MB | 110.7 MB | 227.0 MB | - -- The rule set before the run asked for the 100 ms and 300 ms marks at or - above the lower Linux run's, p50 within 1 MB/s, and nothing resent. The - 300 ms mark, p50 and loss passed. The 100 ms mark missed by 30 KB, 0.5 %, - so the rule is not met and this record stays open. -- That gap is smaller than the spread of the marks across runs: Linux's two - runs differ by 280 KB, and the A/B's three P runs by 260 KB. One boot does - not separate it from noise. -- Nothing was resent, and no RTO, recovery or loss probe occurred. Every RTT - sample came from a departure (33,319, none from the echo). CSS entered in - 134 rounds, 420 times, and never exited. -- The pacer held 73 ms over the row, against 33–41 ms with the A/B's - bucket. The half-window bound holds more on the short round trip, as - expected. - -**The 100 ms miss is not resolved by the bench, 2026-11-01.** Three readings -say the parity boot's 30 KB is below what the session form can tell apart: - -- The rule cannot pass reliably at parity. With one ToyOS run between two Linux - runs and all three drawn alike, ToyOS is the lowest of the three one time in - three. A third of sessions would miss at exact parity. -- The instrument's step is a read. The peer counts at a mark what it had - before the read that passed it, and it reads up to 64 KiB at a time - (`osbench-lan`'s `RampCount`). So each round's mark can sit up to 64 KiB - low, and how far depends on how the sender's segments meet the peer's - reads. -- The marks spread more than the miss, and where they spread is the session. - Linux's two runs differed by 280 KB at 100 ms in the parity boot. Across - this record's sessions Linux read 5.7–6.5 MB. The paced sender read 6.3–6.6 - MB in its A/B, above both Linux runs (5.7 MB) in all three pairs, and 6.07 MB - in the parity boot. - -A model of the ramp gives no candidate either. On [tcp]'s host network, a -1 Gb/s sender into a card at 78 frames a millisecond (113 MB/s) at round trips -of 2, 4 and 8 ms delivers 10.26, 8.27 and 3.49 MB by 100 ms. - -- Two changes move those by under 0.1 MB: the 1 ms bucket without the - half-window bound, and a window stop that grows cwnd at half of cwnd in - flight, as a card stop does (RFC 9438 §4.6 admits RFC 7661 there). -- Removing pacing costs 0.07, 0.48 and 1.1 MB. - -That model's receiver is [tcp]'s, not the peer's Linux. On the bench, both -senders are held by the peer's window first: ToyOS for 66 ms a connection in -the parity boot, Linux for 92–148 ms. So the first 100 ms reads mostly the -receiver's window growth, which both senders share. - -`osbench` now reports each round's marks beside their mean -(`peer_received_by_ms_bytes`), so a session gives five connections a -run and the run's own spread. - -**The exit's first session read no ToyOS row, 2026-11-01** -([evidence](https://github.com/ToyOSOrg/evidence/tree/35f09268e09714a8547496bdd766bcd3ca998325/r1)). This was Linux, ToyOS at `4f52c3056`, Linux, all -exiting 0. ToyOS's `osbench_lan` found the peer and then refused the group: -`no netstack link holds 10.0.2.15`. std's `TcpStream::local_addr` answers -QEMU's user-network address whatever the lease is, and since #1122 osbench -names a connection's link by that address. The T14's lease is not it. So the -session does not decide, and the next one waits on -`issues/getsockname-answers-an-address-nobody-asked-netstack-for.md`. - -Its Linux runs do give the per-connection spread the rule needs: - -- p50 113 and 113 MB/s. -- At 100 ms the rounds ran 5.82–6.63 MB and 5.13–6.38 MB, a standard - deviation of 478 KB over the ten. The parity boot's 30 KB miss is a - sixteenth of that. -- At 300 ms they ran 28.8–30.0 MB and 27.7–29.7 MB. -- The two runs do not differ: the two-sided rank test gives p = 0.69 at - 100 ms and 0.42 at 300 ms. So the rule's exchangeability check passes. - -**Exit**: in one T14 session, Linux, ToyOS, Linux, `osbench_lan`'s -`lan.wired.tcp.upload` reads: - -- p50 at or above the lower Linux run's, less 1 MB/s; -- nothing resent beyond a loss probe; -- at 100 ms and at 300 ms, ToyOS's five rounds not ranked below Linux's ten: - the exact one-sided rank test gives p > 0.05, so parity fails one session - in twenty. - -If Linux's two runs differ at either mark at p ≤ 0.05 by the exact two-sided -rank test, the connections are not exchangeable across runs. Then the session -does not decide, and an interleaving of Linux, ToyOS, Linux, ToyOS, Linux -compares runs instead. Or the cause is named and owned by its own issue. diff --git a/issues/toyos-runs-in-any-open-source-hypervisor.md b/issues/toyos-runs-in-any-open-source-hypervisor.md index 665c899b084..d4ed6e01437 100644 --- a/issues/toyos-runs-in-any-open-source-hypervisor.md +++ b/issues/toyos-runs-in-any-open-source-hypervisor.md @@ -39,8 +39,6 @@ and, filed before this track: - `issues/a-machine-without-an-iommu-refuses-every-claim.md` — bhyve; crosvm for every device it emulates; and VirtualBox, QEMU and Cloud Hypervisor by default (`issues/no-claim-gets-an-address-space-under-virtio-iommu.md`). -- `issues/getsockname-answers-an-address-nobody-asked-netstack-for.md` — - QEMU's user-mode network address in `sdk/libc`. - `issues/pci-is-enumerated-on-segment-group-0-alone.md` — Cloud Hypervisor's `num_pci_segments`. diff --git a/sdk/libc/src/socket.rs b/sdk/libc/src/socket.rs index c791c8ed2a6..062881fe63f 100644 --- a/sdk/libc/src/socket.rs +++ b/sdk/libc/src/socket.rs @@ -825,23 +825,23 @@ pub unsafe extern "C" fn getsockname( addr: *mut Sockaddr, addrlen: *mut SocklenT, ) -> i32 { - let local_port = match sock_from_fd(fd) { + let (ip, port) = match sock_from_fd(fd) { Some(Sock::Stream(held)) => match stream::endpoints(held.handle()) { - Ok((_, local_port)) => local_port, + Ok((_, local)) => local, Err(refused) => { set_errno(crate::errcode::of_stream(refused)); return -1; } }, Some(Sock::Listener(held) | Sock::Datagram(held)) => match socket::local(held.handle()) { - Ok((_, port)) => port, + Ok(local) => local, Err(e) => { set_errno(net_err_to_errno(e)); return -1; } }, + // Neither bound nor connected: the unspecified address, and no port. Some(Sock::Entry(mut socket)) => match *socket.slot() { - Some(_) => 0, + Some(_) => ([0; 4], 0), None => { set_errno(EBADF); return -1; } }, None => { set_errno(EBADF); return -1; } }; - // Local address: 10.0.2.15 (QEMU default) - fill_sockaddr(addr, addrlen, [10, 0, 2, 15], local_port); + fill_sockaddr(addr, addrlen, ip, port); 0 } diff --git a/sdk/std/sys/net/connection.rs b/sdk/std/sys/net/connection.rs index 3bef6d9d3de..a8859e1a509 100644 --- a/sdk/std/sys/net/connection.rs +++ b/sdk/std/sys/net/connection.rs @@ -215,8 +215,8 @@ impl TcpStream { } pub fn socket_addr(&self) -> io::Result { - let (_, local_port) = stream::endpoints(self.handle()).map_err(stream_refused)?; - Ok(SocketAddr::V4(SocketAddrV4::new(Ipv4Addr::new(10, 0, 2, 15), local_port))) + let (_, (addr, port)) = stream::endpoints(self.handle()).map_err(stream_refused)?; + Ok(SocketAddr::V4(SocketAddrV4::new(Ipv4Addr::from(addr), port))) } pub fn shutdown(&self, how: Shutdown) -> io::Result<()> { diff --git a/sdk/toyos/src/net.rs b/sdk/toyos/src/net.rs index c77a679c7a6..c49ab8e9e9a 100644 --- a/sdk/toyos/src/net.rs +++ b/sdk/toyos/src/net.rs @@ -155,6 +155,13 @@ pub enum NetError { Io, } +/// Whether `addr` can be a host's own: neither unspecified, multicast nor broadcast, the +/// check an answer naming this machine's end of a stream or a connected socket passes. +fn unicast(addr: [u8; 4]) -> bool { + let addr = core::net::Ipv4Addr::from(addr); + !addr.is_unspecified() && !addr.is_multicast() && !addr.is_broadcast() +} + impl NetError { pub fn from_error_code(code: u32) -> Self { match code { @@ -185,8 +192,9 @@ ipc_payload! { } /// The answer to [`MsgType::TcpConnect`], with the stream's bells - /// (`net::stream::BELLS`). + /// (`net::stream::BELLS`): the address and port the stream left from. pub struct TcpConnectResponse { + pub local_addr: [u8; 4], pub local_port: u16, pub _pad: u16, } @@ -221,6 +229,7 @@ ipc_payload! { pub remote_addr: [u8; 4], pub remote_port: u16, pub local_port: u16, + pub local_addr: [u8; 4], pub nodelay: u32, } @@ -539,19 +548,23 @@ mod tests { )); } - /// The accept's answer as a client decodes it: the peer, the port, and - /// `nodelay` last, any non-zero word on, and an answer without it refused. + /// The accept's answer as a client decodes it: the peer, the local port + /// and address, and `nodelay` last, any non-zero word on, and an answer + /// without it refused. #[test] fn an_accepts_answer_ends_in_its_streams_nodelay() { - let mut wire = [192, 0, 2, 7, 0x39, 0x30, 0x16, 0, 0, 0, 0, 0]; + let mut wire = [192, 0, 2, 7, 0x39, 0x30, 0x16, 0, 198, 51, 100, 9, 0, 0, 0, 0]; let answer: TcpAcceptResponse = crate::ipc::decode_payload(&wire).unwrap(); - assert_eq!((answer.remote_addr, answer.remote_port, answer.local_port), ([192, 0, 2, 7], 12345, 22)); + assert_eq!( + (answer.remote_addr, answer.remote_port, answer.local_port, answer.local_addr), + ([192, 0, 2, 7], 12345, 22, [198, 51, 100, 9]) + ); assert_eq!(answer.nodelay, 0); - wire[8..].copy_from_slice(&[0, 0, 0, 0x80]); + wire[12..].copy_from_slice(&[0, 0, 0, 0x80]); let answer: TcpAcceptResponse = crate::ipc::decode_payload(&wire).unwrap(); assert_ne!(answer.nodelay, 0); assert!(matches!( - crate::ipc::decode_payload::(&wire[..8]), + crate::ipc::decode_payload::(&wire[..12]), Err(IpcError::Malformed) )); } diff --git a/sdk/toyos/src/net/socket.rs b/sdk/toyos/src/net/socket.rs index eae1046b436..065028e54d4 100644 --- a/sdk/toyos/src/net/socket.rs +++ b/sdk/toyos/src/net/socket.rs @@ -92,6 +92,16 @@ crate::ipc_payload! { pub value: u32, } + /// [`PEER`]'s answer: the peer a connect named, and the address the socket sends there from, + /// which the connect fixed. + #[derive(Debug, PartialEq, Eq)] + pub struct Peer { + pub addr: [u8; 4], + pub port: u16, + pub _pad: u16, + pub local_addr: [u8; 4], + } + /// An exported end's [`ADOPT`]: what kind of socket it holds, and the /// address and port it is at. pub struct Adopt { @@ -371,10 +381,16 @@ pub fn is_socket(handle: BorrowedHandle<'_>) -> bool { kind(handle).is_some() } -/// The address the socket `handle` reaches is at, as its listen or bind named -/// it, and the port it holds. +/// The address the socket `handle` reaches is at and the port it holds: a +/// connected datagram socket's is the one its connect fixed, as netstack answers +/// it, and any other's is the one its listen or bind named. pub fn local(handle: BorrowedHandle<'_>) -> Result<([u8; 4], u16), NetError> { - TABLE.lock().get(handle.raw()).map(|end| (end.addr, end.port)).ok_or(NetError::NotASocket) + let end = TABLE.lock().get(handle.raw()).ok_or(NetError::NotASocket)?; + let connected = match end.kind { + Kind::Datagram => connected(handle)?.map(|peer| peer.local_addr), + Kind::Listener => None, + }; + Ok((connected.unwrap_or(end.addr), end.port)) } /// `handle`, which reaches one of this process's sockets, as a descriptor's object: let go by @@ -459,15 +475,21 @@ pub fn connect(handle: BorrowedHandle<'_>, addr: [u8; 4], port: u16) -> Result<( /// The peer the socket `handle` reaches is connected to, by any of its holders. pub fn peer(handle: BorrowedHandle<'_>) -> Result, NetError> { + Ok(connected(handle)?.map(|peer| (peer.addr, peer.port))) +} + +/// netstack's [`PEER`] answer for the datagram socket `handle` reaches, `None` while no +/// holder connected it. One naming a local address no socket sends from is refused. +fn connected(handle: BorrowedHandle<'_>) -> Result, NetError> { call_on(handle, Kind::Datagram, |slot, end| { let _turn = TURNS[slot].lock(); ipc::signal(end.connection, PEER).map_err(hangup)?; let mut rx = FrameRx::::new(); match answer(&end.connection, &mut rx) { - Ok(frame) => { - let peer: Datagram = ipc::decode_payload(rx.payload(frame.payload_len)).map_err(|_| NetError::Protocol(PEER))?; - Ok(Some((peer.addr, peer.port))) - } + Ok(frame) => match ipc::decode_payload::(rx.payload(frame.payload_len)) { + Ok(peer) if super::unicast(peer.local_addr) => Ok(Some(peer)), + _ => Err(NetError::Protocol(PEER)), + }, Err(NetError::NotConnected) => Ok(None), Err(e) => Err(e), } @@ -579,6 +601,16 @@ pub(super) fn adopted(connection: Connection, payload: &[u8], handles: ipc::Batc mod tests { use super::*; + /// [`PEER`]'s answer as a holder decodes it: the peer, its port, then the address the + /// socket sends from; an answer without that address is refused. + #[test] + fn a_peers_answer_ends_in_the_address_it_is_sent_from() { + let wire = [192, 0, 2, 7, 53, 0, 0, 0, 198, 51, 100, 9]; + let peer: Peer = ipc::decode_payload(&wire).unwrap(); + assert_eq!(peer, Peer { addr: [192, 0, 2, 7], port: 53, _pad: 0, local_addr: [198, 51, 100, 9] }); + assert!(ipc::decode_payload::(&wire[..8]).is_err()); + } + /// A datagram is one frame's payload: its far end, then exactly its bytes, read back as it /// was written, whatever bytes it carries, an empty one too. #[test] diff --git a/sdk/toyos/src/net/stream.rs b/sdk/toyos/src/net/stream.rs index ca6e068f75f..7a8880aa05d 100644 --- a/sdk/toyos/src/net/stream.rs +++ b/sdk/toyos/src/net/stream.rs @@ -145,6 +145,7 @@ crate::ipc_payload! { pub holder: u16, pub peer_port: u16, pub peer_addr: [u8; 4], + pub local_addr: [u8; 4], pub local_port: u16, pub _pad: u16, } @@ -160,6 +161,9 @@ const HELD_REGION: Rights = Rights::DUP.union(Rights::MAP); /// table has. An estimate: no measurement sets it. pub const MAX_STREAMS: usize = 128; +/// An IPv4 address and a port: one end of a stream. +pub type Endpoint = ([u8; 4], u16); + type Rx = Receiving; type Tx = Sending; @@ -195,8 +199,8 @@ struct End { /// waits through and only [`free`] closes: a descriptor closed under a call /// hangs nothing up. connection: BorrowedHandle<'static>, - peer: ([u8; 4], u16), - local_port: u16, + peer: Endpoint, + local: Endpoint, } /// An end's region, its handle and mapping, the bells netstack rings for it @@ -569,10 +573,17 @@ pub fn failure(handle: BorrowedHandle<'_>) -> Result, Refused> { }) } -/// The peer's address and port, and this end's port. -pub fn endpoints(handle: BorrowedHandle<'_>) -> Result<(([u8; 4], u16), u16), Refused> { +/// The peer's address and port, and this end's: the address netstack sent the +/// stream from. +pub fn endpoints(handle: BorrowedHandle<'_>) -> Result<(Endpoint, Endpoint), Refused> { let end = TABLE.lock().get(handle.raw()).ok_or(Refused::NotThisProcess(handle.raw()))?; - Ok((end.peer, end.local_port)) + Ok((end.peer, end.local)) +} + +/// A stream's own end as netstack names it, where it can be one: a unicast +/// address ([`super::unicast`]) and a port. +fn local_end(addr: [u8; 4], port: u16) -> Option { + (super::unicast(addr) && port != 0).then_some((addr, port)) } /// The handle a watch of `way` is made on, for reading: the holder's receive @@ -723,7 +734,6 @@ pub fn let_go(handle: OwnedHandle) { pub struct Connected { /// Its connection, which reaches its end. pub handle: OwnedHandle, - pub local_port: u16, } /// An active open to `addr`:`port`, answered once the handshake ends or @@ -735,7 +745,7 @@ pub fn connect(addr: [u8; 4], port: u16, timeout_ms: u32, nodelay: bool) -> Resu let opened = open(nodelay, (addr, port), |region| { NetstackConn::connect()?.request_with_handles([region], MsgType::TcpConnect, &request)?.answer_with_handles::() })?; - Ok(Connected { handle: opened.handle, local_port: opened.answer.local_port }) + Ok(Connected { handle: opened.handle }) } /// A stream a listener's accept was handed. @@ -744,7 +754,6 @@ pub struct Accepted { pub handle: OwnedHandle, pub remote_addr: [u8; 4], pub remote_port: u16, - pub local_port: u16, /// Whether Nagle's algorithm is off for it, as its listener had it. pub nodelay: bool, } @@ -768,7 +777,6 @@ pub(super) fn accept(listener: BorrowedHandle<'_>, drain: super::socket::Drain<' handle: opened.handle, remote_addr: answer.remote_addr, remote_port: answer.remote_port, - local_port: answer.local_port, nodelay: answer.nodelay != 0, }) } @@ -781,25 +789,30 @@ struct Opened { /// What an answer says of the stream's endpoints. trait Answered { - fn local_port(&self) -> u16; + /// The message the answer is to, which a malformed one is refused as. + const ASKED: u32; + /// This end, as the answer spells it. + fn local(&self) -> Endpoint; /// The peer, where the answer names it. - fn peer(&self) -> Option<([u8; 4], u16)>; + fn peer(&self) -> Option; } impl Answered for TcpConnectResponse { - fn local_port(&self) -> u16 { - self.local_port + const ASKED: u32 = MsgType::TcpConnect as u32; + fn local(&self) -> Endpoint { + (self.local_addr, self.local_port) } - fn peer(&self) -> Option<([u8; 4], u16)> { + fn peer(&self) -> Option { None } } impl Answered for TcpAcceptResponse { - fn local_port(&self) -> u16 { - self.local_port + const ASKED: u32 = super::socket::ACCEPT; + fn local(&self) -> Endpoint { + (self.local_addr, self.local_port) } - fn peer(&self) -> Option<([u8; 4], u16)> { + fn peer(&self) -> Option { Some((self.remote_addr, self.remote_port)) } } @@ -809,7 +822,7 @@ impl Answered for TcpAcceptResponse { /// stream's first holder. Nothing made outlives a refusal. fn open( nodelay: bool, - peer: ([u8; 4], u16), + peer: Endpoint, ask: impl FnOnce(OwnedHandle) -> Result<(A, [OwnedHandle; BELLS], Connection), NetError>, ) -> Result, NetError> { let slot = TABLE.lock().reserve().ok_or(NetError::ResourceExhausted)?; @@ -825,13 +838,15 @@ fn open( let (rx, tx) = session::client(words, LAYOUT); let lent = region.share().map_err(|_| NetError::Io)?; let (answer, bells, connection) = ask(lent)?; + let (addr, port) = answer.local(); + let local = local_end(addr, port).ok_or(NetError::Protocol(A::ASKED))?; let (region, base) = region.into_raw(); // SAFETY: `into_raw` gave this the region's handle, which nothing else answers for. let held = take_up(connection.into_owned(), bells, unsafe { OwnedHandle::from_raw(region) }, base)?; *HALVES[slot].rx.lock() = Kept::Held(rx); *HALVES[slot].tx.lock() = Kept::Held(tx); let peer = answer.peer().unwrap_or(peer); - let end = held.end(base as usize, peer, answer.local_port()); + let end = held.end(base as usize, peer, local); Ok((held.descriptor, end, answer)) })(); let mut table = TABLE.lock(); @@ -860,12 +875,12 @@ struct Held { impl Held { /// The end, its region mapped at `base`. - fn end(&self, base: usize, peer: ([u8; 4], u16), local_port: u16) -> End { + fn end(&self, base: usize, peer: Endpoint, local: Endpoint) -> End { // SAFETY: the handles `take_up` narrowed for this end, which keeps them until its `free`. let (receive_bell, send_bell, rung, connection) = unsafe { (kept(self.receive_bell), kept(self.send_bell), kept(self.rung), kept(self.connection)) }; let region = Mapped { handle: self.region, base, receive_bell, send_bell, rung }; - End { region: Some(region), connection, peer, local_port } + End { region: Some(region), connection, peer, local } } } @@ -974,24 +989,24 @@ pub(super) fn ask_export(connection: BorrowedHandle<'_>, rung: Option) -> Option { let slot = TABLE.lock().reserve()?; - let taken = first.and_then(adoption).and_then(|(holder, adopt, [region, receive, send, rung])| { + let taken = first.and_then(adoption).and_then(|(holder, adopt, local, [region, receive, send, rung])| { let region = SharedMemory::lent(region, REGION_BYTES).ok()?; let (region, base) = region.into_raw(); // SAFETY: `into_raw` gave this the region's handle, which nothing else answers for. let region = unsafe { OwnedHandle::from_raw(region) }; - Some((holder, adopt, base, [receive, send, rung], region)) + Some((holder, adopt, local, base, [receive, send, rung], region)) }); let made = match taken { - Some((holder, adopt, base, bells, region)) => take_up(connection.into_owned(), bells, region, base).map(|held| { + Some((holder, adopt, local, base, bells, region)) => take_up(connection.into_owned(), bells, region, base).map(|held| { let (rx, tx) = session::holder(LAYOUT, holder); *HALVES[slot].rx.lock() = Kept::Held(rx); *HALVES[slot].tx.lock() = Kept::Held(tx); - let end = held.end(base as usize, (adopt.peer_addr, adopt.peer_port), adopt.local_port); + let end = held.end(base as usize, (adopt.peer_addr, adopt.peer_port), local); (held.descriptor, end) }), None => hold_connection(connection.into_owned()).map(|(descriptor, connection)| { // SAFETY: the end's own connection, which it keeps until its `free`. - (descriptor, End { region: None, connection: unsafe { kept(connection) }, peer: ([0; 4], 0), local_port: 0 }) + (descriptor, End { region: None, connection: unsafe { kept(connection) }, peer: ([0; 4], 0), local: ([0; 4], 0) }) }), }; let mut table = TABLE.lock(); @@ -1007,17 +1022,31 @@ pub(super) fn adopted(connection: Connection, first: Option<(&[u8], ipc::Batch)> } } -/// An [`ADOPT`] frame's holder, payload and handles. -fn adoption((payload, handles): (&[u8], ipc::Batch)) -> Option<(Holder, Adopt, [OwnedHandle; BELLS + 1])> { +/// An [`ADOPT`] frame's holder, payload, this end and handles. +fn adoption((payload, handles): (&[u8], ipc::Batch)) -> Option<(Holder, Adopt, Endpoint, [OwnedHandle; BELLS + 1])> { let adopt: Adopt = ipc::decode_payload(payload).ok()?; let handles = handles.exact::<{ BELLS + 1 }>()?; - Some((Holder::new(adopt.holder)?, adopt, handles)) + let local = local_end(adopt.local_addr, adopt.local_port)?; + Some((Holder::new(adopt.holder)?, adopt, local, handles)) } #[cfg(test)] mod tests { use super::*; + /// A stream's own end is a unicast address and a port: an answer naming + /// the unspecified address, a multicast or the broadcast one, or no port, + /// is no end a stream can have. + #[test] + fn a_streams_own_end_is_a_unicast_address_and_a_port() { + assert_eq!(local_end([192, 0, 2, 1], 49152), Some(([192, 0, 2, 1], 49152))); + assert_eq!(local_end([223, 255, 255, 254], 1), Some(([223, 255, 255, 254], 1))); + for addr in [[0; 4], [224, 0, 0, 1], [239, 255, 255, 250], [255; 4]] { + assert_eq!(local_end(addr, 49152), None, "{addr:?}"); + } + assert_eq!(local_end([192, 0, 2, 1], 0), None); + } + /// A wait for a gate that lasts is reported each stall interval and goes /// on until rung; one with a deadline ends at the deadline, reported only /// where it reached a stall interval first. diff --git a/system/netstack/node/src/datagram.rs b/system/netstack/node/src/datagram.rs index e6987a531d2..5758030d4a1 100644 --- a/system/netstack/node/src/datagram.rs +++ b/system/netstack/node/src/datagram.rs @@ -43,7 +43,7 @@ use toyos_net_wire::{Instant, Port}; use crate::holders::{Bell, HolderId, Holders}; use crate::streams::ExportRefused; -use crate::Node; +use crate::{Node, UdpPeer}; /// One client socket. An id outlives its socket only as a refusal: [udp] never hands a closed /// socket's id to another. @@ -179,8 +179,8 @@ impl Node { connected } - /// The peer [`Self::udp_connect`] named, if any. - pub fn udp_peer(&mut self, id: DatagramId) -> Result, Refused> { + /// The peer [`Self::udp_connect`] named, if any, and the address the socket sends there from. + pub fn udp_peer(&mut self, id: DatagramId) -> Result, Refused> { self.stack.peer(id.0).map_err(refused) } diff --git a/system/netstack/node/src/lease.rs b/system/netstack/node/src/lease.rs index 64d3420028b..7636db9bb48 100644 --- a/system/netstack/node/src/lease.rs +++ b/system/netstack/node/src/lease.rs @@ -26,7 +26,7 @@ use core::net::Ipv4Addr; use toyos_dhcp::{Lease, Transmission}; use toyos_dns::Cache; -use toyos_net_shard::{Config, Credit, Event, IfIndex, Origin, Rank, Refusal, Shard}; +use toyos_net_shard::{Config, Credit, Event, IfIndex, Origin, Rank, Refusal, Shard, UdpPeer}; use toyos_net_udp::{Received, SocketId}; use toyos_net_wire::ethernet::{IndividualMac, MacAddr}; use toyos_net_wire::ipv4::{MulticastAddr, Ttl}; @@ -264,7 +264,7 @@ impl Stack { self.shard.udp_readable(id) } - pub(crate) fn peer(&mut self, id: SocketId) -> Result, toyos_net_udp::Error> { + pub(crate) fn peer(&mut self, id: SocketId) -> Result, toyos_net_udp::Error> { self.shard.udp_peer(id) } diff --git a/system/netstack/node/src/lease/tcp.rs b/system/netstack/node/src/lease/tcp.rs index 099c6c513de..db193d5e819 100644 --- a/system/netstack/node/src/lease/tcp.rs +++ b/system/netstack/node/src/lease/tcp.rs @@ -29,14 +29,14 @@ impl Stack { held(self.shard.status(id)) } - pub(crate) fn tcp_local_port(&mut self, id: ConnId) -> Port { - held(self.shard.tuple(id)).local.port + pub(crate) fn tcp_local(&mut self, id: ConnId) -> Endpoint { + held(self.shard.tuple(id)).local } - /// The connection's peer and its local port. - pub(crate) fn tcp_endpoints(&mut self, id: ConnId) -> (Endpoint, Port) { + /// The connection's peer and its own end. + pub(crate) fn tcp_endpoints(&mut self, id: ConnId) -> (Endpoint, Endpoint) { let tuple = held(self.shard.tuple(id)); - (tuple.remote, tuple.local.port) + (tuple.remote, tuple.local) } /// [`toyos_net_tcp::Tcp::lend`]. diff --git a/system/netstack/node/src/lib.rs b/system/netstack/node/src/lib.rs index 7ca9ccc85fa..95a73008b9e 100644 --- a/system/netstack/node/src/lib.rs +++ b/system/netstack/node/src/lib.rs @@ -55,7 +55,7 @@ use alloc::vec; use alloc::vec::Vec; use toyos_dhcp::{AddressRequest, Client, HostName, Lease, Output}; -pub use toyos_net_shard::{Credit, IfIndex, Rank}; +pub use toyos_net_shard::{Credit, IfIndex, Rank, UdpPeer}; use toyos_net_shard::{Config, Shard}; use toyos_net_wire::ethernet::IndividualMac; use toyos_net_wire::Instant; diff --git a/system/netstack/node/src/listeners.rs b/system/netstack/node/src/listeners.rs index 1a4be4c7f78..e2026c7b369 100644 --- a/system/netstack/node/src/listeners.rs +++ b/system/netstack/node/src/listeners.rs @@ -62,7 +62,7 @@ pub enum ListenRefused { pub struct Accepted { pub id: StreamId, pub remote: Endpoint, - pub local: Port, + pub local: Endpoint, /// Whether Nagle's algorithm is off for the stream as it is handed over: what /// [`Node::nodelay`] answers for as long as the node holds the stream, which may be no /// longer than this accept's own pass. @@ -202,7 +202,7 @@ impl Node { let session = session().ok_or(AcceptRefused::NoRegion)?; let Some((conn, tuple, options)) = self.stack.tcp_accept(bound) else { unreachable!("[tcp] counted a connection it would not hand over") }; let id = self.streams.accepted(&mut self.stack, conn, options, session); - Ok(Accepted { id, remote: tuple.remote, local: tuple.local.port, nodelay: options.nodelay }) + Ok(Accepted { id, remote: tuple.remote, local: tuple.local, nodelay: options.nodelay }) } fn end_listener(&mut self, now: Instant, id: ListenerId) { diff --git a/system/netstack/node/src/streams.rs b/system/netstack/node/src/streams.rs index f7e3e88d4d2..57c316ef634 100644 --- a/system/netstack/node/src/streams.rs +++ b/system/netstack/node/src/streams.rs @@ -32,7 +32,7 @@ use core::time::Duration; use toyos::net::stream::{Server, Words, ADDRESS_GONE, LAYOUT, NODELAY, RECEIVE_BYTES, RESET, TIMED_OUT, UNREACHABLE}; use toyos_net_shard::ConnectError; use toyos_net_tcp::{ConnId, Endpoint, Error, Failure, Lent, Options, State, Status}; -use toyos_net_wire::{Instant, Port}; +use toyos_net_wire::Instant; use toyos::transport::session::{Holder, Rings}; use toyos::transport::Violation; @@ -74,8 +74,8 @@ pub struct StreamId(u64); #[derive(Clone, Copy, Debug, PartialEq, Eq)] pub enum StreamEvent { - /// The answer to a connect: established, from this port. - Connected { id: StreamId, local: Port }, + /// The answer to a connect: established, from this address and port. + Connected { id: StreamId, local: Endpoint }, /// The answer to a connect: the peer refused it or [tcp] gave it up. Failed { id: StreamId, failure: Failure }, /// The answer to a connect: its deadline passed first. @@ -150,7 +150,7 @@ impl Stream { (_, Some(failure)) => StreamEvent::Failed { id, failure }, (State::SynSent | State::SynReceived, None) if self.deadline.is_some_and(|at| at <= now) => StreamEvent::TimedOut { id }, (State::SynSent | State::SynReceived, None) => return true, - (_, None) => StreamEvent::Connected { id, local: stack.tcp_local_port(conn) }, + (_, None) => StreamEvent::Connected { id, local: stack.tcp_local(conn) }, }; events.push_back(event); if !matches!(event, StreamEvent::Connected { .. }) { @@ -325,8 +325,8 @@ impl Node { } /// A holder more for established stream `id`, which takes a place: the stream's peer and - /// its local port, for the holder to be told. - pub fn export(&mut self, id: StreamId) -> Result<(Endpoint, Port), ExportRefused> { + /// its own end, for the holder to be told. + pub fn export(&mut self, id: StreamId) -> Result<(Endpoint, Endpoint), ExportRefused> { if self.room() == 0 { return Err(ExportRefused::Full); } diff --git a/system/netstack/node/tests/datagram.rs b/system/netstack/node/tests/datagram.rs index ae37485013b..8cc10c342ff 100644 --- a/system/netstack/node/tests/datagram.rs +++ b/system/netstack/node/tests/datagram.rs @@ -13,7 +13,7 @@ use std::time::Duration; use common::{from_server, message_of, terms, xid, A, ACK, BROADCAST, DNS, MAC, MAC_B, MAC_R, R}; use lan::{udp, unreachable, Lan, Seen, Udp, B, OFF_LINK}; -use toyos_net_node::{Datagram, Event, ExportRefused, HolderId, Refused}; +use toyos_net_node::{Datagram, Event, ExportRefused, HolderId, Refused, UdpPeer}; use toyos_net_shard::Refusal; use toyos_net_udp::Counter; use toyos_net_wire::Port; @@ -276,7 +276,8 @@ fn a_connected_sockets_peer_is_every_holders() { assert_eq!((lan.node.udp_peer(id), lan.node.udp_send(lan.now, id, b"x")), (Ok(None), Err(Refused::NotConnected))); assert_eq!(lan.node.udp_connect(lan.now, id, B, 0), Err(Refused::InvalidInput), "port 0"); lan.node.udp_connect(lan.now, id, B, 7).expect("one host of the link"); - assert_eq!(lan.node.udp_peer(id), Ok(Some((B, Port::new(7).unwrap())))); + // A is the lease, which the route to B leaves from. + assert_eq!(lan.node.udp_peer(id), Ok(Some(UdpPeer { addr: B, port: Port::new(7).unwrap(), local: A }))); lan.node.udp_send(lan.now, id, b"to the peer").expect("a connected socket's send"); lan.pump(); let last = lan.datagrams().last().map(|(_, udp)| (udp.destination, udp.port, udp.payload.clone())); diff --git a/system/netstack/node/tests/listeners.rs b/system/netstack/node/tests/listeners.rs index 672fd2d69e2..6d7a749606d 100644 --- a/system/netstack/node/tests/listeners.rs +++ b/system/netstack/node/tests/listeners.rs @@ -354,7 +354,7 @@ impl Net { /// `holder`'s accept that takes `peer`'s connection. fn accepts_as(&mut self, id: ListenerId, holder: HolderId, peer: Peer) -> (StreamId, Rc) { let (accepted, client) = self.accept_as(id, holder).expect("a connection and a place"); - assert_eq!((accepted.remote, accepted.local), (peer.endpoint(), Port::new(SSH).unwrap())); + assert_eq!((accepted.remote, accepted.local), (peer.endpoint(), Endpoint { addr: A, port: Port::new(SSH).unwrap() })); (accepted.id, client) } @@ -418,7 +418,7 @@ fn a_listener_answers_a_syn_and_wakes_its_owner_when_the_handshake_ends() { let (client, session) = Client::new(); let accepted = net.node.accept(net.now, id, HolderId::FIRST, || Some(session)).unwrap(); - assert_eq!((accepted.remote, accepted.local), (P1.endpoint(), Port::new(SSH).unwrap())); + assert_eq!((accepted.remote, accepted.local), (P1.endpoint(), Endpoint { addr: A, port: Port::new(SSH).unwrap() })); assert_eq!(inbox(&client), b"hello", "what arrived before the accept moves in it"); assert_eq!((net.node.streams(), net.node.listeners(), net.node.held()), (1, 1, 2)); assert_eq!(client.write(b"welcome"), Ok(7)); diff --git a/system/netstack/node/tests/streams.rs b/system/netstack/node/tests/streams.rs index cad6c41ade1..ad11de887b8 100644 --- a/system/netstack/node/tests/streams.rs +++ b/system/netstack/node/tests/streams.rs @@ -417,7 +417,7 @@ fn established() -> (Net, StreamId, Rc) { fn established_within(timeout: Option) -> (Net, StreamId, Rc) { let mut net = Net::new(); let (id, client) = net.connect(timeout); - let local = Port::new(net.far.port).expect("the SYN's source port"); + let local = Endpoint { addr: A, port: Port::new(net.far.port).expect("the SYN's source port") }; assert_eq!(net.events(), [StreamEvent::Connected { id, local }]); (net, id, client) } @@ -427,7 +427,7 @@ fn established_within(timeout: Option) -> (Net, StreamId, Rc) // RFC 9293 §3.5, figure 6: a SYN that acknowledges nothing, and after the peer's SYN-ACK an ACK // of its sequence number plus one from ours plus one. #[test] -fn a_connect_is_a_handshake_and_its_answer_names_the_port() { +fn a_connect_is_a_handshake_and_its_answer_names_the_address_and_port() { let (net, _, client) = established(); let [syn, ack] = &net.far.segments[..] else { panic!("a SYN and an ACK, not {:?}", net.far.segments) }; assert!(syn.syn && syn.ack.is_none() && syn.text.is_empty() && !syn.fin && !syn.rst, "{syn:?}"); @@ -1073,8 +1073,9 @@ fn a_holder_past_the_first_takes_a_place_and_none_is_made_past_the_places() { let (mut net, id, _client) = established(); net.node.set_places(2); assert_eq!(net.node.held(), 1); - let (remote, _) = net.node.export(id).expect("a place for the second holder"); + let (remote, local) = net.node.export(id).expect("a place for the second holder"); assert_eq!(remote, peer(), "the holder is told the stream's peer"); + assert_eq!(local.addr, A, "and the address the stream left from"); assert_eq!(net.node.held(), 2); assert_eq!(net.node.export(id), Err(ExportRefused::Full)); assert_eq!(net.node.held(), 2, "a refused export took nothing"); diff --git a/system/netstack/shard/src/lib.rs b/system/netstack/shard/src/lib.rs index 4b4bf9be3fa..61e608ad21f 100644 --- a/system/netstack/shard/src/lib.rs +++ b/system/netstack/shard/src/lib.rs @@ -63,6 +63,14 @@ use toyos_net_wire::{Instant, Port}; pub use toyos_net_ip::{Credit, IfIndex, Rank}; pub use toyos_net_udp::Origin; +/// A connected datagram socket's peer, and the address it sends there from. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct UdpPeer { + pub addr: Ipv4Addr, + pub port: Port, + pub local: Ipv4Addr, +} + /// Independent 128-bit keys from the system's randomness: one leaked value predicts no other. #[derive(Clone, Debug)] pub struct Secrets { @@ -674,10 +682,11 @@ impl Shard { self.udp.readable(id) } - /// The peer a connected socket's datagrams go to and come from. - pub fn udp_peer(&mut self, id: SocketId) -> Result, toyos_net_udp::Error> { + /// The peer a connected socket's datagrams go to and come from, and the address it sends + /// from, which its connect fixed. + pub fn udp_peer(&mut self, id: SocketId) -> Result, toyos_net_udp::Error> { self.udp.binding(id).map(|(binding, _)| match binding { - toyos_net_udp::Binding::Connected { peer, peer_port, .. } => Some((peer, peer_port)), + toyos_net_udp::Binding::Connected { local, peer, peer_port } => Some(UdpPeer { addr: peer, port: peer_port, local }), toyos_net_udp::Binding::Any | toyos_net_udp::Binding::Specific(_) => None, }) } diff --git a/system/netstack/src/serve.rs b/system/netstack/src/serve.rs index d237a5119a6..46a395b4c01 100644 --- a/system/netstack/src/serve.rs +++ b/system/netstack/src/serve.rs @@ -533,7 +533,7 @@ impl Sockets { let answer_to = self.by_stream.get(&id).and_then(|key| self.streams.get_mut(key)?.answer.take()); let Some((bells, holders)) = answer_to else { unreachable!("the node answered a connect nobody waits for") }; match answer { - Ok(local) => hand_over(&holders, bells, &TcpConnectResponse { local_port: local.get(), _pad: 0 }), + Ok(local) => hand_over(&holders, bells, &TcpConnectResponse { local_addr: local.addr.octets(), local_port: local.port.get(), _pad: 0 }), Err(code) => answer_refused(&holders, code), } } @@ -716,7 +716,8 @@ impl Sockets { holder: number.number(), peer_port: peer.port.get(), peer_addr: peer.addr.octets(), - local_port: local.get(), + local_addr: local.addr.octets(), + local_port: local.port.get(), _pad: 0, }; let handles = [region, receive_bell, send_bell, rung_waker]; @@ -833,7 +834,8 @@ impl Sockets { let answer = TcpAcceptResponse { remote_addr: accepted.remote.addr.octets(), remote_port: accepted.remote.port.get(), - local_port: accepted.local.get(), + local_port: accepted.local.port.get(), + local_addr: accepted.local.addr.octets(), nodelay: u32::from(accepted.nodelay), }; self.hold_stream(accepted.id, &holders, None); @@ -1017,7 +1019,7 @@ fn asked(kind: Kind, msg_type: u32, payload: &[u8], handles: usize) -> Option), } @@ -1040,7 +1042,12 @@ fn answer_socket(node: &mut Node, now: Instant, socket: Held, holder: HolderId, node.udp_connect(now, id, Ipv4Addr::from(to), port).map(|()| Reply::Bare).map_err(refused) } (Held::Datagram(id), Asked::Peer) => match node.udp_peer(id).map_err(refused)? { - Some((addr, port)) => Ok(Reply::Peer(socket::Datagram { addr: addr.octets(), port: port.get(), _pad: 0 })), + Some(peer) => Ok(Reply::Peer(socket::Peer { + addr: peer.addr.octets(), + port: peer.port.get(), + _pad: 0, + local_addr: peer.local.octets(), + })), None => Err(ERR_NOT_CONNECTED), }, (Held::Datagram(id), Asked::SendPeer(data)) => node.udp_send(now, id, &data).map(|()| Reply::Bare).map_err(refused), diff --git a/tests/guest/rust-tests/src/bin/libc_recv_beside_close.rs b/tests/guest/rust-tests/src/bin/libc_recv_beside_close.rs index a1f72929169..075d974a3a0 100644 --- a/tests/guest/rust-tests/src/bin/libc_recv_beside_close.rs +++ b/tests/guest/rust-tests/src/bin/libc_recv_beside_close.rs @@ -140,7 +140,7 @@ fn test(role: &str) { let (_send, send_bell) = wake::pair().expect("the send bell"); let (rung_waker, rung) = wake::pair().expect("the rung bell"); let bells = [OwnedHandle::from(receive_bell), send_bell.into(), rung_waker.into()]; - conn.send_with(RespType::Result as u32, &TcpConnectResponse { local_port: 49_152, _pad: 0 }, bells) + conn.send_with(RespType::Result as u32, &TcpConnectResponse { local_addr: [192, 0, 2, 1], local_port: 49_152, _pad: 0 }, bells) .expect("answer the connect with the bells"); assert_eq!(line(&mut out), "connected"); diff --git a/tests/guest/rust-tests/src/bin/link_failover.rs b/tests/guest/rust-tests/src/bin/link_failover.rs index da1b47d7165..288660e28b9 100644 --- a/tests/guest/rust-tests/src/bin/link_failover.rs +++ b/tests/guest/rust-tests/src/bin/link_failover.rs @@ -8,6 +8,10 @@ //! names. It asks the harness, on a stream over the other link, to set the //! first link's NIC down; the next datagrams leave by the other link and none //! by the first. Asked to set it up again, the datagrams return to it. +//! +//! A stream and a connected datagram socket to each network's host first +//! report that link's lease as their local address: netstack's, since the two +//! leases differ. use std::collections::BTreeMap; use std::io::{BufRead, BufReader, Write}; @@ -85,22 +89,43 @@ fn main() { let snap = until("both links' leases", |snap| { snap.get("net.links") == Some(&Value::U64(2)) && (0..2).all(|n| snap.get(&format!("net.link.{n}.lease.held")) == Some(&Value::Bool(true))) }); - // Each link's network, by its lease's address: QEMU's host is `.2` of it. - let network = |n: usize| { + // Each link's lease, and its network: QEMU's host is `.2` of it. + let lease = |n: usize| -> Ipv4Addr { let address = text(&snap, &format!("net.link.{n}.lease.address")).unwrap_or_else(|| panic!("link_failover: link {n} holds no lease")); - let address: Ipv4Addr = address.split('/').next().and_then(|a| a.parse().ok()).expect("link_failover: an address"); - let [a, b, c, _] = address.octets(); + address.split('/').next().and_then(|a| a.parse().ok()).expect("link_failover: an address") + }; + let leases = [lease(0), lease(1)]; + let network = |n: usize| { + let [a, b, c, _] = leases[n].octets(); Ipv4Addr::new(a, b, c, 0) }; let networks = [network(0), network(1)]; + assert_ne!(networks[0], networks[1], "link_failover: both links lease on one network"); let socket = UdpSocket::bind("0.0.0.0:0").expect("link_failover: a datagram socket"); let (first, second) = carried(round(&socket)); println!(" PASS: a round left by link {first}, on {}", networks[first]); - // The harness, on a stream over the second link: its network's own host. - let [a, b, c, _] = networks[second].octets(); - let mut control = TcpStream::connect(SocketAddr::from((Ipv4Addr::new(a, b, c, 2), port))).expect("link_failover: a stream to the harness over the second link"); + // The harness on each network's own host: a stream over each link, the + // first's let go and the second's the control. + let harness = |n: usize| { + let [a, b, c, _] = networks[n].octets(); + let stream = TcpStream::connect(SocketAddr::from((Ipv4Addr::new(a, b, c, 2), port))).expect("link_failover: a stream to the harness"); + let local = stream.local_addr().expect("link_failover: the stream's local address"); + assert_eq!(local.ip(), leases[n], "link_failover: the stream over link {n} says it left from {local}"); + println!(" PASS: the stream over link {n} left from its lease, {local}"); + stream + }; + for (n, network) in networks.iter().enumerate() { + let [a, b, c, _] = network.octets(); + let connected = UdpSocket::bind("0.0.0.0:0").expect("link_failover: a datagram socket to connect"); + connected.connect(SocketAddr::from((Ipv4Addr::new(a, b, c, 2), port))).expect("link_failover: a datagram socket's connect"); + let local = connected.local_addr().expect("link_failover: the datagram socket's local address"); + assert_eq!(local.ip(), leases[n], "link_failover: the datagram socket connected over link {n} says it sends from {local}"); + println!(" PASS: the datagram socket connected over link {n} sends from its lease, {local}"); + } + drop(harness(first)); + let mut control = harness(second); control.set_read_timeout(Some(BOUND)).expect("link_failover: a read timeout"); let mut answers = BufReader::new(control.try_clone().expect("link_failover: the stream's reader")); let mut ask = |word: &str| { diff --git a/tests/guest/rust-tests/src/bin/stream_handles.rs b/tests/guest/rust-tests/src/bin/stream_handles.rs index fb12c593014..a1b2e46754b 100644 --- a/tests/guest/rust-tests/src/bin/stream_handles.rs +++ b/tests/guest/rust-tests/src/bin/stream_handles.rs @@ -25,6 +25,9 @@ //! two read its stdout and stderr, and another waits for its exit; a child //! that holds its pipes open is killed from a thread other than the one //! reading them, and that read ends. `sshserver` runs every program so. +//! - **A stream's local address is netstack's answer**: std's `local_addr` +//! reads the address and port the connect's answer named, and a connect +//! whose answer names the unspecified address is refused. //! //! **This test is netstack for its children**: a port of its own served to //! each as `netstack`, which answers a connect with the region judged and its @@ -93,6 +96,8 @@ fn main() { Some("two-threads") => two_threads(), Some("echo") => echo(), Some("hold") => hold(), + Some("local") => local(), + Some("unaddressed") => unaddressed(), Some("child") => std::process::exit(0), Some(other) => panic!("unknown role {other:?}"), None => test(), @@ -108,6 +113,7 @@ fn test() { refuses_a_stream_as_a_file(); reads_and_writes_on_two_threads(); pipes_across_threads(); + takes_its_local_address_from_the_answer(); println!("a stream's holders: one dead inside the gate leaves the other working; netstack's exit ends both"); } @@ -243,6 +249,18 @@ fn refuses_a_stream_as_a_file() { println!(" a stream as a file: refused by name"); } +/// std's `local_addr` is what the connect's answer named, and an answer naming +/// the unspecified address refuses the connect. +fn takes_its_local_address_from_the_answer() { + let (child, _netstack) = run("local", &[]); + let said = child.wait_with_output().expect("wait the child"); + assert_eq!(String::from_utf8_lossy(&said.stdout).trim(), "192.0.2.1:49152", "the stream's local address"); + let (child, _netstack) = run_answering("unaddressed", &[], [0; 4]); + let said = child.wait_with_output().expect("wait the child"); + assert_eq!(String::from_utf8_lossy(&said.stdout).trim(), "refused", "a connect answered with no local address"); + println!(" a stream's local address: the answer's, and none refused"); +} + // ---- this test as netstack ---- /// What the client writes while its other thread reads. @@ -451,7 +469,7 @@ impl Netstack { let theirs = self.exports.1.open("export").expect("a connection of the private port"); let ours = self.exports.0.accept().expect("the private port's connection"); let ([receive_bell, send_bell, rung_waker], (receive, send, rung)) = bells(); - let adopt = Adopt { holder: number.number(), peer_port: 80, peer_addr: [192, 0, 2, 2], local_port: 49_152, _pad: 0 }; + let adopt = Adopt { holder: number.number(), peer_port: 80, peer_addr: [192, 0, 2, 2], local_addr: [192, 0, 2, 1], local_port: 49_152, _pad: 0 }; let region = self.region.share().expect("a handle of the region"); let handles = [region, receive_bell, send_bell, rung_waker]; ours.send_with(ADOPT, &adopt, handles).expect("queue the holder's adoption"); @@ -490,6 +508,11 @@ impl Netstack { /// Spawns `role` with this test as its netstack, and serves its connect, the /// stream's receive ring holding `peer`. fn run(role: &str, peer: &[u8]) -> (Child, Netstack) { + run_answering(role, peer, [192, 0, 2, 1]) +} + +/// [`run`], the connect answered as left from `local`, port 49152. +fn run_answering(role: &str, peer: &[u8], local: [u8; 4]) -> (Child, Netstack) { let (acceptor, connector) = port::create().expect("a port"); // This test's own names, so a child reaches the file server its own // spawns read images through, and this test as its `netstack`. @@ -514,7 +537,7 @@ fn run(role: &str, peer: &[u8]) -> (Child, Netstack) { region.bytes().write(RECEIVE_RING, peer); let _ = server.publish(words_of(®ion), peer.len() as u32).expect("the client keeps the protocol"); let (handed, (receive, send, rung)) = bells(); - conn.send_with(RespType::Result as u32, &TcpConnectResponse { local_port: 49_152, _pad: 0 }, handed) + conn.send_with(RespType::Result as u32, &TcpConnectResponse { local_addr: local, local_port: 49_152, _pad: 0 }, handed) .expect("answer the connect with the bells"); let (exports, export_connector) = port::create().expect("the private port"); let names = namespace::build().add("export", &export_connector).finish().expect("the private port's namespace"); @@ -703,6 +726,22 @@ fn foreign() -> ! { std::process::exit(0) } +/// A client that says its stream's local address. +fn local() -> ! { + let stream = connect(); + say(&stream.local_addr().expect("the stream's local address").to_string()); + std::process::exit(0) +} + +/// A client whose connect's answer names no local address. +fn unaddressed() -> ! { + match TcpStream::connect(ADDR.parse::().expect("an address")) { + Ok(stream) => say(&format!("connected from {:?}", stream.local_addr())), + Err(_) => say("refused"), + } + std::process::exit(0) +} + /// A client that takes its stream's handle as a file. fn as_file() -> ! { let stream = connect(); diff --git a/tests/toyos.rs b/tests/toyos.rs index 5ab2ecebfe8..01dfbf7bef3 100644 --- a/tests/toyos.rs +++ b/tests/toyos.rs @@ -699,6 +699,10 @@ const MACHINE_TESTS: &[&str] = &[ // when it is up. Two drivers' links, a NIC's carrier and netstack's // routes cross three processes and two devices, which the node's and the // shard's host tests of rank, failover and return hold below the drivers. + // std's `local_addr` of a stream and of a connected datagram socket over + // each link is that link's lease, and the two differ, so one is not QEMU's + // 10.0.2.15: what netstack answered through the SDK, which the node's host + // tests hold only as far as the node. // QEMU x86-64 (`qemu::SUITE_ARCH`), on two user networks; the datagrams go // to RFC 5737's TEST-NET-1, which answers nothing, and the harness is the // peer on the host's loopback, so it depends on no public network. The networks are not @@ -5940,6 +5944,8 @@ fn traffic_leaves_by_the_second_link_while_the_first_is_down() -> Result<(), Str let port = peer.local_addr().map_err(|e| format!("the listener's port: {e}"))?.port(); let socket = qemu.qmp_socket().to_path_buf(); let heard = std::thread::spawn(move || -> Result<(), String> { + // The job's stream over its first link only reports its local address. + drop(peer.accept().map_err(|e| format!("the guest's first connect: {e}"))?); let (conn, _) = peer.accept().map_err(|e| format!("the guest's connect: {e}"))?; conn.set_read_timeout(Some(CEILING)).map_err(|e| e.to_string())?; let mut reader = BufReader::new(conn.try_clone().map_err(|e| e.to_string())?); @@ -6090,6 +6096,7 @@ fn stream_holders(test_config: &Path) -> Result<(), String> { "given out and taken back: one end; a duplicate of its handle endowed or sent: refused", "one end, two threads: the write completes beside an unanswered read, which then takes the peer's next bytes", "piped children across threads: fed, read and waited for apart, and one killed beside its reader", + "a stream's local address: the answer's, and none refused", ] { if !result.stdout.contains(claim) { return Err(format!("the job never said {claim:?}:\n{}", result.stdout));