diff --git a/quest/m0/README.md b/quest/m0/README.md index bfa06daea9..d4139613da 100644 --- a/quest/m0/README.md +++ b/quest/m0/README.md @@ -43,6 +43,7 @@ a published `@moq/watch` break. ## Required +- [Expiry wakes](/quest/m0/expiry-wakes.md) - an appended group wakes only the parked reads it expires, so a 2.5 ms-frame publisher with 2 s of parked serves stays near idle and the Python and Go interop rows pass - [Request caps](/quest/m0/request-caps.md) - lite message sizes, IETF request IDs, and per-session announces and subscriptions are bounded - [Prefix route fronts](/quest/m0/prefix-route-fronts.md) - a prefix route cannot be made to mint one front per requested path - [qmux credit](/quest/m0/qmux-credit.md) - qmux returns connection credit for dropped and stopped streams and delivers its close frame, on both lines diff --git a/quest/m0/expiry-wakes.md b/quest/m0/expiry-wakes.md new file mode 100644 index 0000000000..616a6fa1eb --- /dev/null +++ b/quest/m0/expiry-wakes.md @@ -0,0 +1,43 @@ +# [M] An appended group wakes only the parked reads it expires + +## Goal + +A group appended to a track wakes only the parked group serves whose expiry +it crosses, not every parked serve. A publisher sending one 2.5 ms Opus frame +per group to a subscriber with a 2 s max age (about 800 parked serves) keeps +its runtime thread near idle, and `just test interop --all` passes its +Python and Go publisher rows again. + +## Plan + +Root cause, found 2026-10-06 while chasing the red Interop nightly: since +#4162 raised the browser's audio max age to 2 s, an FFI publisher's +subscription parks up to ~800 group serves. Every parked serve registers on +track-state changes and on the next unstamped group past the edge +(`GroupExpiry::is_expired` in `rs/moq-net/src/model/track.rs`, reached from +`group::Consumer::poll_expired_while_pending` in the lite and IETF +publishers' `GroupServe`). Each append wakes all N, so the single moq-ffi +runtime thread sits at ~110% CPU, QUIC idles out, and subscribers see +"internal error" or RESET_STREAM. #4892 made each check O(log N) but kept the +N-wide wake per append. It depends on load and build profile, so it fails on +unrelated branches. + +Decided 2026-10-06: deadline-indexed wakes. A parked read goes stale only +once the live edge passes its reach plus its budget, so keep parked reads in +a track-level map ordered by that point and wake only those the advancing +edge crosses. Keep the register-before-judge order that prevents lost wakes. +Rejected: skipping expiry checks while a serve waits for stream credit +(stale serves linger, lost-wakeup risk, fan-out stays elsewhere), capping +backlog serves per subscription (changes delivery scheduling), and lowering +the browser audio max age or using 20 ms frames in the interop clients +(hides the bug). + +Tests: + +- A mocked-time regression test that counts wakes: N parked serves, one + append, and only the crossed serves wake. It fails today. +- Sweep `track_parked_read` in `rs/moq-net/benches/track.rs` over parked + readers, so a per-append cost that grows with N shows up as a slope. +- `just test interop --all` passes the Python and Go publisher rows on CI. + +Public API: none. Wire: none. diff --git a/quest/m1/README.md b/quest/m1/README.md index 0a44964597..d562b32a74 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -89,6 +89,12 @@ blocks. The quests that gated m0 lines moved under them. - [Datagrams are live-only](/quest/m1/datagram-unfetchable.md) - no FETCH, replay to a new subscriber, or cache fill ever returns a datagram group - [JavaScript FETCH](/quest/m1/js-fetch.md) - browser publishers answer IETF FETCH through the JS ranges request surface - [Bounded HLS playlists](/quest/m1/hls-bounded.md) - `moq-hls` renders a capped sliding window for a fresh viewer, with a stable media sequence, gap slots, and no video before a sync point +- [Request update routing](/quest/m1/ietf-subscribe-update-route.md) - drafts 14 and 15 SUBSCRIBE_UPDATE and draft 16 REQUEST_UPDATE reach the request named by their second field +- [JS request window](/quest/m1/js-request-window.md) - @moq/net grants MAX_REQUEST_ID as requests close, so a peer never stalls after ~21k requests +- [Relay session limits](/quest/m1/relay-session-limits.md) - moq-relay sets per-session request limits, tighter for clients than peers, and the bindings name a refused request +- [Churn with held subscriptions](/quest/m1/session-churn-held.md) - opening and closing a request costs the same with 1 or 1,024 held subscriptions +- [Wide varint tests](/quest/m1/varint-test-gaps.md) - wide sequences and both varint forms are tested where they differ +- [Export linger](/quest/m1/export-ts-linger.md) - `moq export ts --linger` fails on an export error while the broadcast is live, and lingers once it closes or is replaced - [Archive](/quest/m1/archive/README.md) - record selected tracks to any object_store and replay them over FETCH or derived HLS; the catalog entry and format may break in place, since no archives exist - [In-band auth](/quest/m1/auth/README.md) - a session tells its peer what it may publish and subscribe to, unions tokens presented in band, and fails loud on an out-of-scope publish - [Dropped sources](/quest/m1/dropped-sources.md) - track consumers see the producer's real error on every end path, never `Dropped` diff --git a/quest/m1/export-ts-linger.md b/quest/m1/export-ts-linger.md new file mode 100644 index 0000000000..d0fd7a703c --- /dev/null +++ b/quest/m1/export-ts-linger.md @@ -0,0 +1,29 @@ +# [S] An export failure is not the broadcast ending + +## Goal + +`moq export ts --linger` reports an export failure while the broadcast is +still live as a failure, instead of treating it as the broadcast ending and +waiting out the linger (`rs/moq-cli/src/subscribe.rs`). + +## Plan + +Reported by t0ms while grading #4645, who offered to do the CLI side. + +The error alone can't tell the cases apart: a publisher drop or SIGKILL +usually surfaces as a track `Err` too, and that must keep lingering (see the +comment above the linger in `run_ts`). Classify by the broadcast's state +instead. If it is still announced and live, the export failed: report it and +fail the run as before. If it closed or was replaced, it ended, clean or not, +and the linger starts. The track error can arrive before the broadcast's close +does, so the check has to settle that race rather than read the state once. + +A test drives each case: an export failure on a live broadcast fails the run, +a clean finish lingers, and a killed publisher still lingers and resumes when +the broadcast returns. + +Public API: none. Wire: none. + +## Related + +- [#4645 report](https://github.com/moq-dev/moq/pull/4645#issuecomment-6013366214) - the motivating `missed a decode deadline` failure exists only on #4645's branch diff --git a/quest/m1/ietf-subscribe-update-route.md b/quest/m1/ietf-subscribe-update-route.md new file mode 100644 index 0000000000..5f28579c21 --- /dev/null +++ b/quest/m1/ietf-subscribe-update-route.md @@ -0,0 +1,22 @@ +# [S] Request updates reach their target on drafts 14 to 16 + +## Goal + +On moq-transport drafts 14 and 15, a SUBSCRIBE_UPDATE reaches its +Subscription Request ID. On draft 16, a REQUEST_UPDATE reaches its Existing +Request ID. Today `rs/moq-net/src/ietf/adapter.rs` routes both by the update's +own new Request ID (the first field), so it never reaches its target. + +## Plan + +Found while working on request caps (#4820); the bug predates it and can be +fixed independently. Route updates by their second field on drafts 14 to +16, and keep the newer drafts' routing as it is. The codec currently stores +that field as `subscription_request_id` on all three affected drafts. +Land a regression test for each affected draft with distinct update and +target IDs, proving the update reaches the named request and failing without +the fix. Rewrite the existing `test_classify_subscribe_update_followup` the +same way rather than keep it beside them: its body carries a single ID, so it +locks in the current routing. + +Public API: none. Wire: behaviour within the drafts; no format change. diff --git a/quest/m1/js-request-window.md b/quest/m1/js-request-window.md new file mode 100644 index 0000000000..e873acdf24 --- /dev/null +++ b/quest/m1/js-request-window.md @@ -0,0 +1,20 @@ +# [S] @moq/net grants request IDs as requests close + +## Goal + +A Rust peer can make any number of requests over one JS session on +moq-transport drafts 14 to 16. Today `js/net` advertises +`MaxRequestId = 42069` in SETUP, as a server (`js/net/src/connection/accept.ts`) +and as a client (`js/net/src/connection/connect.ts`), and never grants more, +so the peer stalls after about 21k requests. + +## Plan + +Advertise a finite window and send MAX_REQUEST_ID to grant more as requests +close. Refuse an incoming ID past the window with the draft's error. This +can start before request caps (#4820), since the JS bug exists on main; use +the drafts as the contract and align with Rust's landed window when +available. A mocked-time test drives more than 21k requests through one +session. + +Public API: none. Wire: behaviour within the drafts; no format change. diff --git a/quest/m1/lite07-finalize.md b/quest/m1/lite07-finalize.md index b012e1e930..c783cabfb8 100644 --- a/quest/m1/lite07-finalize.md +++ b/quest/m1/lite07-finalize.md @@ -44,6 +44,13 @@ The cut: rather than misparse; no compatibility shim. - The draft's lite-07 changelog matches the wire and `just drafts check` passes. +- Preserve SETUP's rule: "a receiver MUST treat a longer SETUP as a protocol + violation and MAY reject it based on the length prefix alone" (65,536 + bytes). The general cap proposed by request caps (#4820) only permits + rejection (MAY), so it does not replace SETUP's stronger requirement. + Remove the SETUP sentence only if the general rule requires the same + rejection. If SETUP keeps its own rule, the lite-07 changelog still names + that rejection, not only the cap. Rust's lite-07 varints already carry the full 64 bits the draft specifies, so no codec work waits on the cut. diff --git a/quest/m1/relay-session-limits.md b/quest/m1/relay-session-limits.md new file mode 100644 index 0000000000..5406d9874a --- /dev/null +++ b/quest/m1/relay-session-limits.md @@ -0,0 +1,27 @@ +# [S] The relay configures per-session request limits + +## Goal + +`moq-relay` sets moq-net's `session::Limits` from its config, with a tighter +default for client sessions than for cluster peers, and documents it in +`doc/bin/relay/`. The bindings report a refused request as its own error +kind instead of mapping `TooManyRequests` to Unknown. + +## Plan + +Blocked until request caps (#4820) lands. That PR proposes generous +defaults in moq-net (100,000 announces and 10,000 subscriptions per session), +sized for relay meshes, so viewer sessions would get loose limits. Confirm +the landed API and defaults before implementing this quest. The relay knows +which sessions are peers, so it picks the limit per session. Propose the flag +and TOML names in the PR. + +Add the error kind to `rs/moq-ffi` and every wrapper per the Cross-Package +Sync table. + +Public API: a relay config field and flag, and a new error kind in moq-ffi +and the bindings. Wire: none. + +## Required + +- [Request caps](/quest/m0/request-caps.md) - the session limits and TooManyRequests error exist on main diff --git a/quest/m1/session-churn-held.md b/quest/m1/session-churn-held.md new file mode 100644 index 0000000000..1c891c0864 --- /dev/null +++ b/quest/m1/session-churn-held.md @@ -0,0 +1,27 @@ +# [M] Request churn costs the same with many held subscriptions + +## Goal + +Opening and closing a request on a session costs the same whether the +session holds 1 or 1,024 subscriptions. The `session_churn_held` benchmark +proposed by request caps (#4820) is flat in held subscriptions. + +## Plan + +Blocked until request caps (#4820) lands with the benchmark. Measured on +that PR: about 70 µs per churn at held=1 and 10 to 20 ms at held=1024 across +4 sessions, equally on lite-06 and draft-22. Neither has a +request window, so the slope predates request caps. Profile first, find what +scans the held subscriptions on each churn, and make that path proportional +to what it touches. Keep the bench swept over both sessions and held +subscriptions. + +Public API: none. Wire: none. + +## Required + +- [Request caps](/quest/m0/request-caps.md) - the held-subscription churn benchmark exists on main + +## Related + +- [Reproducible relay CPU and allocation profiles](/quest/m1/performance-profiles.md) - the profiling setup this can use diff --git a/quest/m1/test-flakes-2/README.md b/quest/m1/test-flakes-2/README.md index d46678aae5..42a83e4cab 100644 --- a/quest/m1/test-flakes-2/README.md +++ b/quest/m1/test-flakes-2/README.md @@ -32,4 +32,5 @@ Public API: none. Wire: none. - [Subscription cut by disconnect](/quest/m1/test-flakes-2/subscription-cut.md) - a publisher disconnect never ends a subscription clean - [Media late join](/quest/m1/test-flakes-2/media-late-join.md) - `just test media` late join stays within one GOP, or the regression is fixed +- [Import catalog finish](/quest/m1/test-flakes-2/import-catalog-finish.md) - `moq-cli`'s subprocess EOF catalog-finish test holds up under load with event-based fixture coordination - [Relay restart rebind](/quest/m1/test-flakes-2/relay-restart-rebind.md) - the crash drill restarts on its original UDP address under concurrent load diff --git a/quest/m1/test-flakes-2/import-catalog-finish.md b/quest/m1/test-flakes-2/import-catalog-finish.md new file mode 100644 index 0000000000..c7494c4d1e --- /dev/null +++ b/quest/m1/test-flakes-2/import-catalog-finish.md @@ -0,0 +1,22 @@ +# [XS] Import catalog finish + +## Goal + +`moq-cli`'s `import_delivers_the_catalog_finish_at_eof` +(`rs/moq-cli/tests/import.rs`) passes under a loaded `just check`. It runs +8.8 s alone against a 10 s timeout and fails with "announce timed out" under +load. + +## Plan + +Preserve the real `moq` subprocess coverage: closing stdin must finish the +catalog and let the process exit successfully while the subscriber still +receives a clean finish. Pausing the parent's Tokio clock cannot control the +child process, and the real QUIC sockets can race an auto-advancing clock. +Reproduce the slowdown, fix its cause, and coordinate the fixture through +observable readiness and completion events rather than elapsed-time +assumptions. Use an in-memory fixture for separable protocol assertions if +helpful, while retaining the subprocess EOF regression. Never raise the +timeout or add a retry. + +Public API: none. Wire: none. diff --git a/quest/m1/varint-test-gaps.md b/quest/m1/varint-test-gaps.md new file mode 100644 index 0000000000..30f4b11087 --- /dev/null +++ b/quest/m1/varint-test-gaps.md @@ -0,0 +1,20 @@ +# [S] Wide varints are tested where the two forms differ + +## Goal + +The plain-`u64` varint codec (#4437) has tests for the cases its first round +left open: + +- The wide-sequence relay test asserts the error a lite-06 reader sees for a + group of `1 << 62` from a lite-07 publisher, not any error. Today the reader + only sees `Stream(Internal)` over the wire, so decide what it should see. +- A subscription whose groups turn wide partway through fails only that + subscription, and the session and other tracks keep going. +- Announce compression is tested where the QUIC and leading-ones forms size + the same value differently. + +## Plan + +Extend `rs/moq-net/tests/wide_sequence.rs` and the lite announce encode tests. + +Public API: none. Wire: none. diff --git a/quest/m2/README.md b/quest/m2/README.md index 188c7095d9..1d3d3740a3 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -59,6 +59,8 @@ waiting on an upstream release waits in [m4](/quest/m4/README.md). - [Refusal reasons](/quest/m2/refusal-reasons.md) - refused-session metrics tell an expired token from an invalid one, and count gateway admissions - [Bench coverage](/quest/m2/bench-coverage.md) - Criterion targets for moq-pattern matching first, then the stats producer, moq-mux containers, the hang catalog, and moq-auth - [Signed priority](/quest/m2/signed-priority.md) - every API priority is an `i8` with 0 as the unset midpoint, and hang's built-ins sit above it +- [mTLS on tls://](/quest/m2/tls-listener-mtls.md) - a `tls://` listener can identify a cluster peer by its client certificate +- [One transport adapter](/quest/m2/transport-adapter-dedup.md) - the poll transport adapter exists once, and the 64 KiB cap in moq-net's default `poll_read_buf` (not in the adapter) has a test - [Catalog track identity](/quest/m2/catalog-tracks.md) - a changed track configuration becomes a new track name or epoch, never a mutated definition - [Catalog colour model](/quest/m2/color-catalog.md) - the catalog describes a rendition's colour and HDR properties once a renderer consumes them - [Archive S3 wire proof](/quest/m2/archive-s3.md) - the archive proof also runs through the S3 client against an in-process S3-compatible server diff --git a/quest/m2/tls-listener-mtls.md b/quest/m2/tls-listener-mtls.md new file mode 100644 index 0000000000..74c7a843c8 --- /dev/null +++ b/quest/m2/tls-listener-mtls.md @@ -0,0 +1,20 @@ +# [M] tls:// peers can authenticate by certificate + +## Goal + +A relay's `tls://` (qmux over TLS on TCP) listener can identify a cluster +peer by its client certificate, as the QUIC listener does, so an upstream +link doesn't need a token. + +## Plan + +#4816 added `tls://` with no client certificate request, so peers present a +token. `moq_tokio::tcp::Listener::with_tls` (`rs/moq-tokio/src/tcp.rs`) +already takes an `Arc`, so requesting a certificate is +config. The real gap is getting the peer certificate out afterwards: +`qmux::tls::Server::accept` finishes the handshake and returns a `Session` +without it. Either expose it upstream in qmux or own the TLS accept here. +Keep the token path working on the same listener, and keep pinned peers and +CA roots meaning the same as on QUIC. + +Public API: relay config may gain a listener option. Wire: none. diff --git a/quest/m2/transport-adapter-dedup.md b/quest/m2/transport-adapter-dedup.md new file mode 100644 index 0000000000..9d36c1d515 --- /dev/null +++ b/quest/m2/transport-adapter-dedup.md @@ -0,0 +1,22 @@ +# [S] One copy of the poll transport adapter + +## Goal + +The poll transport adapter exists once. #4709 proposes copying the same +~310-line adapter into `rs/moq-ffi`, `rs/moq-uring` and `rs/moq-wasm` +(`src/transport/adapter.rs`), and moq-tokio repeats its error and stats +wrappers. The 64 KiB cap on moq-net's default `poll_read_buf` has a unit test. + +## Plan + +Blocked until transport seam (#4709) lands; confirm the landed adapter +shape first. The transport-seam quest ruled out a new shared adapter crate. +Revisit that here: share one copy from an existing crate (for example moq-net behind a +feature, or one backend re-exporting it to the others), or record why the +copies stay. Add the `poll_read_buf` cap test either way. + +Public API: possibly a feature-gated module. Wire: none. + +## Required + +- [Transport seam](/quest/m1/transport-seam.md) - the owned transport traits and backend adapters exist on main