From 4a645130d04bfcac3b68a4ba292f2fb4cb0bd256 Mon Sep 17 00:00:00 2001 From: Luke Curley <432854+kixelated@users.noreply.github.com> Date: Sat, 7 Nov 2026 07:25:05 -0800 Subject: [PATCH 1/4] quest: plan fixes from the protocol, relay, and auth invariant audit Co-Authored-By: Claude Opus 5.5 --- quest/m0/README.md | 2 ++ quest/m0/peer-arithmetic.md | 16 ++++++++++++++++ quest/m0/qmux-js-credit.md | 19 +++++++++++++++++++ quest/m1/README.md | 9 +++++++++ quest/m1/auth-stream-bounds.md | 14 ++++++++++++++ quest/m1/cluster-peer-identity.md | 15 +++++++++++++++ quest/m1/copy-identity.md | 13 +++++++++++++ quest/m1/drop-runs-prune.md | 13 +++++++++++++ quest/m1/js-net-input-checks.md | 16 ++++++++++++++++ quest/m1/lite-update-refused.md | 9 +++++++++ quest/m1/relay-config-checks.md | 16 ++++++++++++++++ quest/m1/stats-prefix.md | 9 +++++++++ quest/m1/token-parse-strict.md | 15 +++++++++++++++ quest/m2/README.md | 3 +++ quest/m2/anchor-oldest.md | 13 +++++++++++++ quest/m2/auth-serve-policy.md | 14 ++++++++++++++ quest/m2/sequence-reuse.md | 22 ++++++++++++++++++++++ quest/m3/mesh/README.md | 1 + quest/m3/mesh/device-bounds.md | 12 ++++++++++++ quest/m4/auth-session-index.md | 1 + 20 files changed, 232 insertions(+) create mode 100644 quest/m0/peer-arithmetic.md create mode 100644 quest/m0/qmux-js-credit.md create mode 100644 quest/m1/auth-stream-bounds.md create mode 100644 quest/m1/cluster-peer-identity.md create mode 100644 quest/m1/copy-identity.md create mode 100644 quest/m1/drop-runs-prune.md create mode 100644 quest/m1/js-net-input-checks.md create mode 100644 quest/m1/lite-update-refused.md create mode 100644 quest/m1/relay-config-checks.md create mode 100644 quest/m1/stats-prefix.md create mode 100644 quest/m1/token-parse-strict.md create mode 100644 quest/m2/anchor-oldest.md create mode 100644 quest/m2/auth-serve-policy.md create mode 100644 quest/m2/sequence-reuse.md create mode 100644 quest/m3/mesh/device-bounds.md diff --git a/quest/m0/README.md b/quest/m0/README.md index 9141265dc82..43dfbd415f0 100644 --- a/quest/m0/README.md +++ b/quest/m0/README.md @@ -30,6 +30,8 @@ edge, which embeds moq-relay; it moved there on 2026-09-28. - [Nightly passes on main](/quest/m0/nightly/README.md) - the scheduled Nightly passes end to end, so red means a new break - [Interop and Video pass on main](/quest/m0/interop-green/README.md) - the scheduled Interop and Video workflows pass, so red means a new break - [A draft-18 session to Cloudflare outlives its first request](/quest/m0/cloudflare-draft18-session.md) - `moq fetch` against Cloudflare's draft-18 relay waits for its timeout instead of a `protocol violation` +- [JS qmux keeps credit and final sizes consistent with Rust](/quest/m0/qmux-js-credit.md) - a JS WebSocket session no longer stalls on cancelled streams or gets closed by a Rust peer with FINAL_SIZE_ERROR, and its control queue stays bounded +- [Peer and config values never panic or wrap moq-net arithmetic](/quest/m0/peer-arithmetic.md) - a peer's retry interval, object id, drop end or sequence, and huge configured durations no longer panic a task or wrap - [Publish the release line's drain() wrappers](/quest/m0/ffi-shutdown-backport.md) - Dart `moq` 0.1.1 reaches pub.dev from `release` - [Release's audio encoder rolls groups before the cap](/quest/m0/release-audio-group-cap.md) - a long `groupDuration` with short frames never throws `GroupTooLarge` on `release` - [Local checks pass on unchanged main](/quest/m0/local-checks-main.md) - `sh/go/publish-wrapper.test.sh` passes under a global `tag.gpgsign = true`, so `just check` passes on a clean `main` diff --git a/quest/m0/peer-arithmetic.md b/quest/m0/peer-arithmetic.md new file mode 100644 index 00000000000..0bcf0f34f4f --- /dev/null +++ b/quest/m0/peer-arithmetic.md @@ -0,0 +1,16 @@ +# [S] Peer and config values never panic or wrap moq-net arithmetic + +## Goal + +No value a peer sends or an application configures panics a moq-net task or silently wraps: + +- `rs/moq-net/src/ietf/publisher.rs` `refusal`: a REQUEST_ERROR Retry Interval near `u64::MAX` panics through kio's `Instant + Duration`. Any peer we send PUBLISH_NAMESPACE to can trigger it. +- `rs/moq-net/src/ietf/publisher.rs` `save_join`: `largest.object + 1` on an upstream's LARGEST_OBJECT of `u64::MAX`. +- `rs/moq-net/src/reconnect.rs`: `now + duration` and `mul_f64` on `Goaway.handover`, `Config::timeout` and `Backoff` panic for huge configured values once a GOAWAY arrives; `Server::deadline` already treats overflow as no deadline. +- `rs/moq-net/src/lite/subscriber.rs` and `subscriber/fills.rs`: SUBSCRIBE_DROP's inclusive end `u64::MAX` becomes `..u64::MAX` through `saturating_add(1)`, so the drop misses group `u64::MAX`. +- `rs/moq-net/src/lite/publisher.rs` `DatagramSend::send`: a sequence of 2^62 or more relayed from lite-07 to a lite-05/06 subscriber fails to encode and is dropped silently, and the subscription ends in a reset instead of a finish. +- `rs/moq-net/src/route/mod.rs` `requests`: `fd[&id].0 + 1` on an advertised seqno `u64::MAX` (test-only today; becomes reachable when lite-07 drives the router). + +## Plan + +Found by the 2026-11-07 invariant audit (ietf-1, ietf-2, lite-3, lite-4, lite-9, model-3). Each site gets `checked_add` with an explicit meaning for overflow (no deadline, unsupported, refused) and a test at the extreme value. Prefer refusing the impossible value at decode or at `Reconnect::new` where that is simpler than handling it downstream. diff --git a/quest/m0/qmux-js-credit.md b/quest/m0/qmux-js-credit.md new file mode 100644 index 00000000000..be987bcc1e1 --- /dev/null +++ b/quest/m0/qmux-js-credit.md @@ -0,0 +1,19 @@ +# [S] JS qmux keeps credit and final sizes consistent with Rust + +## Goal + +A JS qmux session (the WebSocket fallback) never stalls or gets closed by a Rust peer because of its own flow-control bookkeeping, and a peer cannot grow its memory without bound: + +- Bytes that arrive on a stream we already stopped (late STREAM frames, and the final-size gap of a later RESET_STREAM) count toward connection receive credit. Today `js/qmux/src/session.ts` (`#onStream`, `#onResetStream`) returns early once the receive half is deleted on cancel, so each cancelled stream permanently shrinks the window until MAX_DATA is never sent again. +- A RESET_STREAM's final size is at least every byte already handed to the socket. Today `sendOffset` advances only after `scheduler.enqueueStream` resolves, so a STOP_SENDING handled while a frame is mid-`sink.write` replies with a short final size, which Rust's `rs/moq-quic/src/connection/streams/recv.rs` refuses with FINAL_SIZE_ERROR. +- MAX_DATA, MAX_STREAMS_{BIDI,UNI} and MAX_STREAM_DATA (per stream) replace any queued copy, as Rust does with its pending flags, instead of each taking a new entry on the uncapped control lane. +- An empty binary WebSocket message is a no-op on every qmux version, matching `rs/qmux/src/transport/websocket.rs`; qmux-00 currently throws. +- WebSocket messages larger than the record size are refused before they are buffered where the platform allows it: Rust `rs/qmux/src/ws.rs` `Upgraded` validates or sets the tungstenite `max_message_size` itself instead of trusting the caller, and JS documents the `ws` `maxPayload` it needs. + +## Plan + +Found by the 2026-11-07 invariant audit (qmux-1 to qmux-5). qmux-1 and qmux-2 are the urgent ones: an honest Rust relay triggers both whenever a JS app cancels group streams over WebSocket. + +- Keep each stopped peer stream's receive offset (and final size once known) until FIN or RESET so late bytes are accounted, and port Rust's `credit_returns_for_unread_and_stopped_streams` (`rs/moq-quic/src/connection/mux/tests.rs`). +- Count bytes as sent when the scheduler takes the frame from its queue, or track the in-flight frame per stream and include it in the RESET final size. Test with a sink whose `write` is held open while STOP_SENDING arrives. +- Send the MAX_* frames through `replaceControl` keyed by type (and stream id for MAX_STREAM_DATA); fix the "bounded" claim in `scheduler.ts`. diff --git a/quest/m1/README.md b/quest/m1/README.md index 8167de6e377..15190fbd61d 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -32,6 +32,11 @@ moves what remains of them once their claims merge. ## Required - [Drop set](/quest/m1/drop-set.md) - each cursor judges missing groups against its own budget, so a narrow reader drops promptly and a widening re-offers groups already reported dropped on lite-07 +- [Only the relay's own stats publisher uses the stats prefix](/quest/m1/stats-prefix.md) - a broad grant can no longer publish unmetered media or spoof node stats under `.stats` +- [js/net enforces the input checks Rust does](/quest/m1/js-net-input-checks.md) - GOAWAY redirects to loopback spellings, credential logging, ANNOUNCE_INIT size, sequence range, and version pinning match Rust +- [Rust and JS accept exactly the same tokens and key sets](/quest/m1/token-parse-strict.md) - a null `exp` no longer means never-expiring in Rust, duplicate kids and bad scope roots are refused, and every relay lease is validated +- [A refused lite SUBSCRIBE_UPDATE ends the subscription](/quest/m1/lite-update-refused.md) - the publisher no longer serves a removed range or reports the wrong start after the model refuses an update +- [Cluster peers have one identity each and fresh inputs](/quest/m1/cluster-peer-identity.md) - a duplicate cluster id, a self-dial, or a rotated cluster token fails loud instead of partitioning or looping the mesh - [Media fuzz findings](/quest/m1/fuzz-findings/README.md) - the nightly media fuzz targets run clean: mp4-atom, MKV block, TS step, SPS overflow, and catalog encoder bugs they found are fixed - [A smaller, safer API than release](/quest/m1/api-audit/README.md) - every surface added since release has a consumer or is gone, overlapping operations are collapsed, names mirror across Rust, JS, and the bindings, and every break says how to migrate - [Routing guide](/quest/m1/routing-guide.md) - concepts and practical outbound federation setup @@ -54,6 +59,10 @@ moves what remains of them once their claims merge. - [Tests run on a mocked clock](/quest/m1/mock-clock-tests/README.md) - about 300 tests stop proving nothing happened by waiting, sleeping to settle, or asserting on elapsed time - [moq-mux reads H.265 SPS without scuffle](/quest/m1/h265-sps-reader.md) - moq-mux's own SPS reader replaces scuffle-h265 and scuffle-expgolomb, and the fuzz regressions still pass - [QMux reads the app-limited flag on big-endian Linux](/quest/m1/qmux-app-limited-endian.md) - the send-rate filter's app-limited mask follows the target's byte order +- [Copies of one group identity must agree beyond their bytes](/quest/m1/copy-identity.md) - copies that differ in timestamp, object fields, or frame count fail as a protocol violation +- [Session auth grants and replies stay consistent and bounded](/quest/m1/auth-stream-bounds.md) - an AUTH_ERROR revokes its grant, queued grants and retiring tokens are bounded, and dropping queued requests cannot deadlock +- [Relay settings and admin input that cannot work are refused](/quest/m1/relay-config-checks.md) - clustered stats without a node name, pin without workers, repeated filter keys, and unknown transports fail loud +- [Track drop runs are forgotten below the floor](/quest/m1/drop-runs-prune.md) - a track's dropped-group record stays proportional to its window in Rust and JS ## Related diff --git a/quest/m1/auth-stream-bounds.md b/quest/m1/auth-stream-bounds.md new file mode 100644 index 00000000000..82188051bd1 --- /dev/null +++ b/quest/m1/auth-stream-bounds.md @@ -0,0 +1,14 @@ +# [S] Session auth grants and replies stay consistent and bounded + +## Goal + +moq-net's session-side auth (`rs/moq-net/src/auth.rs`, `auth/tokens.rs`, and the AUTH stream loops in `lite/publisher.rs` and `ietf/auth.rs`) keeps four promises it currently leaves to callers: + +- A token answered with AUTH_ERROR leaves the enforced scope at once. Today an AUTH_OK too large to encode is replaced by AUTH_ERROR and `state.done = true` without `Issue::finish`, so the grant `reply()` already recorded keeps admitting requests for the session. +- Replies queued for a peer that withholds stream credit are bounded: a newer `Reply::Grant` replaces a queued one instead of appending to the outbox. +- Tokens replaced by `Tokens::set` are withdrawn even when they were never answered; a backend failing transiently while the app rotates JWTs does not pile up retrying AUTH streams. +- `Handle::requests` never drops queued `Request`s under the state lock (dropping one re-enters it); take them out and drop them after the guard. + +## Plan + +Found by the 2026-11-07 invariant audit (lite-2, lite-5, lite-6, lite-8). Each gets a test: `allows()` is false after an oversize accept, repeated `Issued::update` on a blocked stream keeps one queued grant, two `set` calls with no answer leave one retiring generation, and swapping the acceptor with queued requests does not deadlock. diff --git a/quest/m1/cluster-peer-identity.md b/quest/m1/cluster-peer-identity.md new file mode 100644 index 00000000000..e8c96fb58fc --- /dev/null +++ b/quest/m1/cluster-peer-identity.md @@ -0,0 +1,15 @@ +# [S] Cluster peers have one identity each and fresh inputs + +## Goal + +A relay mesh fails loud on identity mistakes instead of silently partitioning or looping, and its peer inputs stay current and bounded (`rs/moq-relay/src/cluster.rs`, `relay.rs`): + +- A peer session whose declared hop (`Request::peer_hop`) equals our `cluster.id` is refused with an error naming the duplicate, instead of every route from it being dropped as reflected. +- One relay has one dial key: `DialTarget` keys on `canonicalize_peer_key` (which also strips URL userinfo, so credentials stop reaching logs and the `remote` span), and the self filter applies to static `cluster.connect` entries too. A test covers the self filter. +- `--cluster-lan` on a relay without a QUIC listener is refused at startup instead of advertising SRV port 0. +- The `cluster.token` file is re-read on redial (or watched like the API file), so a rotated or expiring token does not break every redial until restart. +- The `cluster.api.url` body and file are read with a size cap; an oversized answer keeps the last good peer list. + +## Plan + +Found by the 2026-11-07 invariant audit (relay-2, relay-4, relay-5, relay-7, relay-9, relay-10). diff --git a/quest/m1/copy-identity.md b/quest/m1/copy-identity.md new file mode 100644 index 00000000000..0cbbba322fd --- /dev/null +++ b/quest/m1/copy-identity.md @@ -0,0 +1,13 @@ +# [M] Copies of one group identity must agree beyond their bytes + +## Goal + +Two copies of the same broadcast+track+group (another route, a FETCH answer, a re-offer, a mid-frame recovery) that disagree on anything a reader sees fail the later copy with a protocol violation, instead of readers of each copy seeing different content under one identity. Today `rs/moq-net/src/model/group/content.rs` `row()`, `write_chunk` and the recovery in `frame.rs` compare only size, `inflated` and overlapping bytes; nothing compares a frame's timestamp, its IETF object fields, or the group's final frame count, so a copy with one more frame can raise `largest` after `finish_largest` declared it settled. + +## Plan + +Found by the 2026-11-07 invariant audit (model-1). Put the timestamp and a digest of the object fields in the shared row, and the group's fin in `Content`; a copy that disagrees, commits past the fin, or finishes short of it gets `ProtocolViolation`. Tests: two copies with differing timestamps, differing fields, and an extra frame. Check the JS counterpart (`js/net/src/group.ts`) and match it in the same PR. + +## Related + +- [A reader on a failed copy continues on the held re-offer](/quest/m1/copy-recover.md) - also stitches frames across copies diff --git a/quest/m1/drop-runs-prune.md b/quest/m1/drop-runs-prune.md new file mode 100644 index 00000000000..7151a038a7f --- /dev/null +++ b/quest/m1/drop-runs-prune.md @@ -0,0 +1,13 @@ +# [S] Track drop runs are forgotten below the floor + +## Goal + +A track's record of dropped groups stays proportional to the window readers can still ask for, in Rust and JS. Today `drops` only grows: `RangeMap` in `rs/moq-net/src/model/track.rs` is never pruned, and JS `insertDrop` (`js/net/src/internal.ts`) copies and re-merges the whole array for the state and every sink on each SUBSCRIBE_DROP, while `#drop` in `js/net/src/track.ts` scans the whole cache. A peer dropping alternating groups, or a long live track with sporadic gaps, grows it forever at quadratic total cost in JS. + +## Plan + +Found by the 2026-11-07 invariant audit (js-4). Runs below the cache/arrivals floor are forgotten, as `Delivered.forgetBelow` does, and JS inserts in place. This waits on [Drop set](/quest/m1/drop-set.md), which redesigns these runs with their reason and budget; prune that representation rather than today's. + +## Required + +- [Drop set](/quest/m1/drop-set.md) - drop runs carry their reason and budget, the structure this quest prunes diff --git a/quest/m1/js-net-input-checks.md b/quest/m1/js-net-input-checks.md new file mode 100644 index 00000000000..92d4b051709 --- /dev/null +++ b/quest/m1/js-net-input-checks.md @@ -0,0 +1,16 @@ +# [S] js/net enforces the input checks Rust does + +## Goal + +`@moq/net` rejects the inputs `rs/moq-net` rejects, so JS is not the weaker side of a session: + +- A GOAWAY redirect is classified on the host actually dialed. `isLocal` in `js/net/src/connection/goaway.ts` parses the opaque `moql:`/`moqt:` host, so `127.1`, `0x7f000001` and `2130706433` pass as non-local while `dialable()` rewrites them to `https:`, which resolves to 127.0.0.1. With `redirect: "follow"` a relay can steer the client and its tokens to loopback. +- `redact` in `js/net/src/util/log.ts` logs scheme, host and port only. For `moql:` URLs `url.origin` is `"null"` and the path, which can carry a token, is kept. +- The lite-01/02 ANNOUNCE_INIT is capped like Rust's (`frame::SEGMENT - 16`) and its count is checked against the announce budget before decoding (`js/net/src/lite/announce.ts`). +- `checkSequence` (`js/net/src/util/sequence.ts`) refuses a group sequence at or above 2^64, as datagrams already do, so `appendGroup` after group 2^64-1 fails instead of minting a group past `UNBOUNDED`. +- `accept.ts` requires a pinned `AcceptProps.version` to be one the client offered, as `rs/moq-net/src/server.rs` does. +- `open.ts` `patience` treats an RTT of 0 as unknown, so a transport that reports 0 before its first sample does not make every stalled open reset a lower-priority stream. + +## Plan + +Found by the 2026-11-07 invariant audit (js-2, js-5 to js-9). Tests for each, including the three loopback spellings. js-1 (announce queue growth) is already [JS announced coalescing](/quest/m3/js-announced-coalesce.md). diff --git a/quest/m1/lite-update-refused.md b/quest/m1/lite-update-refused.md new file mode 100644 index 00000000000..9610a589ba1 --- /dev/null +++ b/quest/m1/lite-update-refused.md @@ -0,0 +1,9 @@ +# [S] A refused lite SUBSCRIBE_UPDATE ends the subscription + +## Goal + +When the model refuses a lite SUBSCRIBE_UPDATE, the publisher's per-range state is left unchanged and the subscription ends with the refusal, as the initial SUBSCRIBE and the ietf publisher already do. Today `TrackRun::update` in `rs/moq-net/src/lite/publisher.rs` only uses `self.track.update(..).is_ok()` to decide `self.named`, then rewrites `ids`, `unstarted`, `frames`, `floor` and `cap` anyway, so the publisher's view and the model's diverge and SUBSCRIBE_STARTED can name a range with another range's start. + +## Plan + +Found by the 2026-11-07 invariant audit (lite-1). On `Err`, set `self.refused` and return before mutating, as the `Bounds::new` arm does; carve out `Closed` only if it is benign. Add a test: an update that swaps ranges for one the cache cannot settle ends the subscription instead of serving the removed range. diff --git a/quest/m1/relay-config-checks.md b/quest/m1/relay-config-checks.md new file mode 100644 index 00000000000..605cb5b2e87 --- /dev/null +++ b/quest/m1/relay-config-checks.md @@ -0,0 +1,16 @@ +# [S] Relay settings and admin input that cannot work are refused + +## Goal + +moq-relay refuses at load or request time what it currently accepts and then ignores or mishandles: + +- Stats enabled on a clustered relay without `stats.node` is refused, or `stats.node` defaults to `cluster.node`, so nodes stop overwriting each other's `/node` broadcast. +- `runtime.pin` without `runtime.workers` is refused, matching the file's "refused rather than ignored" rule. +- A repeated key on `/sessions` or `/sessions/revalidate` (`session.rs` `Filter::from_query`) is an error instead of the last value winning. +- Refusal counters (`refusals.rs`) stay bounded with stats disabled, where nothing calls the pruning `Registry::report`. +- `auth.rs` maps every `moq_tokio::Transport` without `unreachable!` (the enum is `#[non_exhaustive]`), and the qmux-01-only ALPN choice in `websocket.rs` is derived from the version, not a hand-kept string list. +- `session::Registry` registration drops only their own entry, so an embedder registering a reused `Request` id cannot unlist another live session. + +## Plan + +Found by the 2026-11-07 invariant audit (relay-3, relay-6, relay-8, relay-11 to relay-14). diff --git a/quest/m1/stats-prefix.md b/quest/m1/stats-prefix.md new file mode 100644 index 00000000000..bb667a218a5 --- /dev/null +++ b/quest/m1/stats-prefix.md @@ -0,0 +1,9 @@ +# [XS] Only the relay's own stats publisher uses the stats prefix + +## Goal + +No client session can publish or announce under the relay's stats prefix (`.stats` by default), whatever its grant. Today `Cluster::publisher` scopes purely from the grant's patterns, while the stats registry excludes the whole `/**` subtree from metering, so a root `**` grant can publish unmetered media there and replace a node's `.stats/node/` broadcast that aggregators read. + +## Plan + +Found by the 2026-11-07 invariant audit (relay-1). Cut the prefix out of non-peer publish scopes in `Cluster::mounted`/`scope` (`rs/moq-relay/src/cluster.rs`); peers still relay each other's stats. Test that a `**` publisher cannot announce under the prefix and that subscribers still can read it. diff --git a/quest/m1/token-parse-strict.md b/quest/m1/token-parse-strict.md new file mode 100644 index 00000000000..aa825ef3828 --- /dev/null +++ b/quest/m1/token-parse-strict.md @@ -0,0 +1,15 @@ +# [S] Rust and JS accept exactly the same tokens and key sets + +## Goal + +Token and key parsing is strict and identical in `rs/moq-auth` and `js/auth`, and every grant the relay holds has been validated: + +- `exp`, `iat` and `nbf` refuse `null` and values outside the range both languages represent. Today Rust reads `"exp": null` as no expiry (a token that never expires) while JS refuses it; the Grant wire `expires` has the same gap. +- A KeySet with a duplicate `kid` is refused at parse in both languages instead of the first match winning. +- A JWK's scope root passes `path::validate` on import in both languages, as the `with_scope` doc promises. +- The relay detects a token the way the server parses one (`presents_token` vs form-decoded `jwt`), so `?%6Awt=` cannot fall through to the public grant; key sources are cached instead of re-read and re-parsed per token. +- `Lease::new` (`rs/moq-relay/src/auth.rs`) runs `Grant::validate`, so an embedder's grant cannot skip the tier charset and upstream-implies-peer checks. + +## Plan + +Found by the 2026-11-07 invariant audit (auth-2, auth-4, auth-9, auth-11, auth-12). Each refusal gets a vector in the Rust/JS interop test (`js/auth/src/interop.test.ts`), including a null-`exp` token. diff --git a/quest/m2/README.md b/quest/m2/README.md index ad168b11ccb..2d75ca4be1f 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -184,6 +184,9 @@ quests that require them. - [Unbounded groups](/quest/m2/unbounded-groups/README.md) - groups and frames lose their size caps; a group keeps a 32 MB tail, and readers behind it replay through paced flights or refresh - [Session policy lives in moq-net](/quest/m1/session-policy/README.md) - accept and connect deadlines, admission, window widening and reconnect live once in moq-net; moq-tokio and moq-relay's io_uring path call them (held in m1 by an open claim) - [MoQ on rind](/quest/m1/rind/README.md) - every MoQ crate takes its crypto from rind, a pure-Rust graviola fork, so no build compiles C crypto (held in m1 by an open claim) +- [A recreated or restarted track cannot reuse group sequences](/quest/m2/sequence-reuse.md) - recreating a track or restarting an instance cannot give new content an old group identity unless the app seeds past it, and datagrams are documented as exempt +- [Subscriptions name the oldest group without a u64::MAX sentinel](/quest/m2/anchor-oldest.md) - `Anchor` gets an explicit oldest variant, decoded the same way on every wire +- [moq auth serve bounds what a leaked token or public caller can do](/quest/m2/auth-serve-policy.md) - tokens need an `exp` unless the operator opts out, and a public listener authenticates relays - [Cut readiness](/quest/m2/cut-gate/README.md) - the checks the cut needs that break nothing: main no slower than release, vendored fixes ported, and moq-c at parity - [Probe](/quest/m2/probe/README.md) - a session probes up to the bitrate it wants to send or receive, so a stopped simulcast rendition resumes and a viewer up-switches; browser padding is parked in m3 (held in m2 by open claims; moves to m3) - [Finish the milestone re-cut](/quest/m2/recut-held.md) - claim-held epics move to the milestone their kind and priority give them once their claims merge diff --git a/quest/m2/anchor-oldest.md b/quest/m2/anchor-oldest.md new file mode 100644 index 00000000000..f479a911d38 --- /dev/null +++ b/quest/m2/anchor-oldest.md @@ -0,0 +1,13 @@ +# [S] Subscriptions name the oldest group without a u64::MAX sentinel + +## Goal + +`Anchor` has an explicit variant for "the oldest group", and `Relative(n)` always means n groups before the latest. Today `Range::oldest` is `Anchor::Relative(u64::MAX)` (`rs/moq-net/src/model/subscription.rs`), against the repository's rule never to use a huge value as a marker. A peer on lite-07 or IETF can send `u64::MAX` or `u64::MAX - 1`, which clamp to the same group but differ in window start, demand shape and `is_wire`, and the lite encoder treats any `back > MAX_QUIC` as oldest, which the model does not. + +## Plan + +Found by the 2026-11-07 invariant audit (model-4). Add the variant, decode and encode it consistently on every wire (lite-07, lite-06 and earlier, IETF), and mirror it in JS and the bindings. m2 because `Anchor` is public API. + +## Related + +- [Subscribe ranges](/quest/m2/subscribe-ranges/README.md) - owns the range model this anchor lives in diff --git a/quest/m2/auth-serve-policy.md b/quest/m2/auth-serve-policy.md new file mode 100644 index 00000000000..f459f13278d --- /dev/null +++ b/quest/m2/auth-serve-policy.md @@ -0,0 +1,14 @@ +# [S] moq auth serve bounds what a leaked token or public caller can do + +## Goal + +`moq auth serve` stops relying on operator discipline for two invariants: + +- Admission eventually ends. Today `exp` is never required (`required_spec_claims` is emptied, `moq auth sign` adds none by default) and `Policy.expires` and revalidation are off by default, so a leaked token without `exp` admits forever and admitted sessions never re-check. The server refuses a token without `exp` unless the operator passes an explicit opt-out flag, and `moq auth sign` writes an `exp` by default. +- Only relays drive the session table. Today `POST /` is unauthenticated and `--listen-public` only lifts the loopback bind, so anyone can record sessions with a victim's `remote` (exhausting its `RemoteLimit`) or grow `slots` with random ids. A public listener requires a shared secret or mTLS from relays. + +## Plan + +Found by the 2026-11-07 invariant audit (auth-3, auth-5). + +Decided 2026-11-07 (maintainer, invariant-audit planning): m2, since both change CLI behaviour a deployed operator must adapt to. The opt-out keeps unbounded tokens possible for deployments that revoke by key rotation. diff --git a/quest/m2/sequence-reuse.md b/quest/m2/sequence-reuse.md new file mode 100644 index 00000000000..adf56b73720 --- /dev/null +++ b/quest/m2/sequence-reuse.md @@ -0,0 +1,22 @@ +# [S] A recreated or restarted track cannot reuse group sequences + +## Goal + +moq-net refuses to give new content an identity it already used, in Rust and JS, instead of relying on the documented contract: + +- Recreating a track name in a live broadcast (`create_track` after the previous producer dropped, `rs/moq-net/src/model/broadcast.rs`) no longer restarts `append_group` at 0 under a fresh `TrackState`. The broadcast remembers each name's highest sequence, and a group at or below it on the replacement is an error. +- A publisher restarting under the same instance at a higher epoch, which `js/net/src/origin.ts` splices onto the existing logical front (and Rust's origin likewise continues), is refused loudly when its sequences go backwards, instead of resume silently dropping its groups as already delivered or a recovering reader stitching frames from two streams into one group. +- An application that really wants to reuse a name seeds it explicitly with a sequence past the predecessor's highest; the library never does it silently. +- Datagrams are documented as outside the identity rule: on the datagram API in Rust and JS and in the moq-lite draft, a datagram sequence names no cached content and repeats are delivered. + +## Plan + +Found by the 2026-11-07 invariant audit (model-2, js-3, model-7, js-10). + +Decided 2026-11-07 (maintainer, invariant-audit planning): reuse is misuse that produces unintended behaviour, so the library prevents it and an application that really wants it seeds from the high-water mark itself; the library does not seed automatically. Datagrams get no duplicate detection: spending resources to catch application bugs there is not worth it, so they are documented as exempt instead. + +m2 because `append_group`/`create_group` and the origin splice start refusing calls that succeed today, which callers recreating tracks must adapt to (the questline's breaking-change test). The API shape for the explicit seed is the implementer's call; prefer reusing `create_group` with an explicit sequence over a new method. [moq-dev/moq#8063](https://github.com/moq-dev/moq/pull/8063) changes how a job's route continues its claim's instance; build on whichever lands first. + +## Related + +- [A job's exact route continues its claim's readers](/quest/m2/announce-takeover.md) - instance continuity across a claim's jobs diff --git a/quest/m3/mesh/README.md b/quest/m3/mesh/README.md index 46b56ce1831..2e1b18d2c53 100644 --- a/quest/m3/mesh/README.md +++ b/quest/m3/mesh/README.md @@ -95,6 +95,7 @@ the third, and survives a relay restart. - [Byte streams](/quest/m3/stream/README.md) - lossless TCP-level streams over MoQ - [Endpoint API](/quest/m3/mesh/endpoint.md) - an iroh-shaped `Endpoint` that connects to an endpoint ID and service and accepts byte streams - [Mesh relay node](/quest/m3/mesh/node.md) - a relay node enrolls into a mesh, dials its relays, and admits only enrolled LAN peers +- [Mesh enrollment is bounded and revocation takes effect](/quest/m3/mesh/device-bounds.md) - a reusable key enrolls a bounded number of devices and a revoked device loses access within a bounded time - [Address hints](/quest/m3/mesh/addr.md) - nodes publish their addresses on mesh tracks and dial each other directly where mDNS cannot find them - [Sparse overlay](/quest/m3/mesh/overlay.md) - each node keeps about log N deterministic direct links, forwards for the rest, and falls back to relays across a partition - [JS mesh enrollment client](/quest/m3/mesh/js-enroll.md) - `@moq/auth` enrolls and refreshes from JS like `moq_auth::mesh::Client` diff --git a/quest/m3/mesh/device-bounds.md b/quest/m3/mesh/device-bounds.md new file mode 100644 index 00000000000..6dffe8c0ccc --- /dev/null +++ b/quest/m3/mesh/device-bounds.md @@ -0,0 +1,12 @@ +# [S] Mesh enrollment is bounded and revocation takes effect + +## Goal + +The mesh coordinator (`rs/moq-auth/src/mesh/coordinator.rs`) stops assuming its device store stays small and its revocations wait for token expiry: + +- A reusable enrollment key enrolls a bounded number of devices, and `/enroll` is rate limited; today each enroll clones the whole store, rewrites the state file and returns every member under one mutex, so a leaked key can mint thousands of endpoints. +- A revoked device loses access within a bounded time, not only at its token's `exp` (1h by default): either `decide` checks revocation for mesh tokens, or mesh tokens are short-lived with revalidation on. + +## Plan + +Found by the 2026-11-07 invariant audit (auth-7, auth-8). diff --git a/quest/m4/auth-session-index.md b/quest/m4/auth-session-index.md index 62b739a70fe..be3b54eb334 100644 --- a/quest/m4/auth-session-index.md +++ b/quest/m4/auth-session-index.md @@ -13,6 +13,7 @@ Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-a - Keep `HashMap` and `HashMap` beside the slots, updated on connect, end, a new token attaching to a known session, and expiry. `Sessions::revalidate` updates them too, since it re-inserts an aged-out or restart-lost slot and appends a token the slot hadn't seen, and it keeps today's rule of enforcing no cap. - Age slots from a queue ordered by `seen`, refreshed on each sighting (a re-push with a generation, or an ordered map keyed by `(seen, id)`), so a sweep pops only expired entries. Keep today's rule: a session that missed two cadences is dropped. - Keep the semantics the doc comment on the limits describes, including a known id attaching another credential counting only against the new JWT's cap. +- Key the per-token count on the token's signing input (header and payload) or its decoded claims, not the whole compact JWS: an ECDSA signature can be malleated to a second valid string, which today doubles `limits.token` for an ES256 or ES384 holder (found by the 2026-11-07 invariant audit, auth-6). - Measure with #7512's `verify/sessions` bench before and after; the slope should be flat. Public API: none. Wire: none. From db5c734dd1380c057ebc24e1be28ffed1db8b911 Mon Sep 17 00:00:00 2001 From: Luke Curley <432854+kixelated@users.noreply.github.com> Date: Sat, 7 Nov 2026 07:48:24 -0800 Subject: [PATCH 2/4] quest: plan root-cause redesigns for the invariant audit findings Co-Authored-By: Claude Opus 5.5 --- REVIEW.md | 1 + quest/m0/README.md | 4 ++-- quest/m0/peer-arithmetic.md | 17 ++++++++--------- quest/m0/qmux-js-credit.md | 20 +++++++++----------- quest/m1/README.md | 8 ++++---- quest/m1/cluster-peer-identity.md | 15 ++++++++------- quest/m1/js-net-input-checks.md | 17 ++++++++--------- quest/m1/relay-config-checks.md | 25 ++++++++++++++++--------- quest/m1/token-parse-strict.md | 19 ++++++++++++------- quest/m3/mesh/README.md | 2 +- quest/m3/mesh/device-bounds.md | 4 ++-- 11 files changed, 71 insertions(+), 61 deletions(-) diff --git a/REVIEW.md b/REVIEW.md index a8347f002a6..3f57446390c 100644 --- a/REVIEW.md +++ b/REVIEW.md @@ -22,6 +22,7 @@ Assume a better approach exists until you have tried to find one: - **Layer**: does the logic live where it belongs (library vs runtime adapter, application vs relay, moq-lite model vs IETF layer, our crate vs upstream)? Does a package boundary leak? - **Public API and wire**: would we want to support this shape for years, in every language? Look for foot-guns, names tied to today's implementation, a new surface that could merge into an existing one, and exports nothing needs yet. Is every API or wire change listed under the body's Impact, with breaking ones marked `!`? - **Completeness**: does it fix every instance of the problem and the gaps it exposes, or land a partial version plus follow-ups? +- **Bounds**: can a peer grow a queue or map here? It should hold the latest state per key, or be bounded by credit the peer already spent, never one entry per event. - **Evidence**: is a performance claim measured, a bug reproduced, and is each new test worth running forever? An approach finding names the alternative and why it holds up better long-term. diff --git a/quest/m0/README.md b/quest/m0/README.md index 43dfbd415f0..1063f4f8e6e 100644 --- a/quest/m0/README.md +++ b/quest/m0/README.md @@ -30,8 +30,8 @@ edge, which embeds moq-relay; it moved there on 2026-09-28. - [Nightly passes on main](/quest/m0/nightly/README.md) - the scheduled Nightly passes end to end, so red means a new break - [Interop and Video pass on main](/quest/m0/interop-green/README.md) - the scheduled Interop and Video workflows pass, so red means a new break - [A draft-18 session to Cloudflare outlives its first request](/quest/m0/cloudflare-draft18-session.md) - `moq fetch` against Cloudflare's draft-18 relay waits for its timeout instead of a `protocol violation` -- [JS qmux keeps credit and final sizes consistent with Rust](/quest/m0/qmux-js-credit.md) - a JS WebSocket session no longer stalls on cancelled streams or gets closed by a Rust peer with FINAL_SIZE_ERROR, and its control queue stays bounded -- [Peer and config values never panic or wrap moq-net arithmetic](/quest/m0/peer-arithmetic.md) - a peer's retry interval, object id, drop end or sequence, and huge configured durations no longer panic a task or wrap +- [JS qmux streams follow QUIC's per-stream state machine](/quest/m0/qmux-js-credit.md) - credit changes only in stream state transitions, so a JS WebSocket session cannot stall on cancelled streams or send a short final size that a Rust peer refuses +- [kio's Instant has no panicking arithmetic, and peer values never wrap](/quest/m0/peer-arithmetic.md) - every deadline states what overflow means, so a peer's retry interval or a huge configured duration cannot panic a task - [Publish the release line's drain() wrappers](/quest/m0/ffi-shutdown-backport.md) - Dart `moq` 0.1.1 reaches pub.dev from `release` - [Release's audio encoder rolls groups before the cap](/quest/m0/release-audio-group-cap.md) - a long `groupDuration` with short frames never throws `GroupTooLarge` on `release` - [Local checks pass on unchanged main](/quest/m0/local-checks-main.md) - `sh/go/publish-wrapper.test.sh` passes under a global `tag.gpgsign = true`, so `just check` passes on a clean `main` diff --git a/quest/m0/peer-arithmetic.md b/quest/m0/peer-arithmetic.md index 0bcf0f34f4f..71fa115dbd9 100644 --- a/quest/m0/peer-arithmetic.md +++ b/quest/m0/peer-arithmetic.md @@ -1,16 +1,15 @@ -# [S] Peer and config values never panic or wrap moq-net arithmetic +# [M] kio's Instant has no panicking arithmetic, and peer values never wrap ## Goal -No value a peer sends or an application configures panics a moq-net task or silently wraps: +No value a peer sends or an application configures can panic a moq task or silently wrap: -- `rs/moq-net/src/ietf/publisher.rs` `refusal`: a REQUEST_ERROR Retry Interval near `u64::MAX` panics through kio's `Instant + Duration`. Any peer we send PUBLISH_NAMESPACE to can trigger it. -- `rs/moq-net/src/ietf/publisher.rs` `save_join`: `largest.object + 1` on an upstream's LARGEST_OBJECT of `u64::MAX`. -- `rs/moq-net/src/reconnect.rs`: `now + duration` and `mul_f64` on `Goaway.handover`, `Config::timeout` and `Backoff` panic for huge configured values once a GOAWAY arrives; `Server::deadline` already treats overflow as no deadline. -- `rs/moq-net/src/lite/subscriber.rs` and `subscriber/fills.rs`: SUBSCRIBE_DROP's inclusive end `u64::MAX` becomes `..u64::MAX` through `saturating_add(1)`, so the drop misses group `u64::MAX`. -- `rs/moq-net/src/lite/publisher.rs` `DatagramSend::send`: a sequence of 2^62 or more relayed from lite-07 to a lite-05/06 subscriber fails to encode and is dropped silently, and the subscription ends in a reset instead of a finish. -- `rs/moq-net/src/route/mod.rs` `requests`: `fd[&id].0 + 1` on an advertised seqno `u64::MAX` (test-only today; becomes reachable when lite-07 drives the router). +- `kio::clock::Instant` drops `Add`, `AddAssign`, `Sub` and `SubAssign` with `Duration` (`rs/kio/src/clock/instant.rs`), which panic past the clock's range. Every call site picks `checked_add`/`checked_sub` with an explicit meaning for overflow (no deadline, refused) or a saturating form. This removes the class behind, among others, the IETF publisher's REQUEST_ERROR retry interval (`rs/moq-net/src/ietf/publisher.rs` `refusal`, which any peer can panic) and `rs/moq-net/src/reconnect.rs` deadlines from huge configured durations. +- Peer-supplied counters never wrap with `+ 1`: the ietf `save_join` `largest.object + 1`, the lite SUBSCRIBE_DROP end `u64::MAX` (`saturating_add(1)` that misses that group in `lite/subscriber.rs` and `subscriber/fills.rs`), and the router's advertised seqno in `route/mod.rs`. +- A datagram sequence of 2^62 or more relayed to a lite-05/06 subscriber is handled explicitly in `lite/publisher.rs` `DatagramSend::send` instead of dropped silently. ## Plan -Found by the 2026-11-07 invariant audit (ietf-1, ietf-2, lite-3, lite-4, lite-9, model-3). Each site gets `checked_add` with an explicit meaning for overflow (no deadline, unsupported, refused) and a test at the extreme value. Prefer refusing the impossible value at decode or at `Reconnect::new` where that is simpler than handling it downstream. +Found by the 2026-11-07 invariant audit (ietf-1, ietf-2, lite-3, lite-4, lite-9, model-3). + +Decided 2026-11-07 (maintainer, invariant-audit root causes): remove the operators rather than make them saturate, so each of the roughly 600 sites states what overflow means; most edits are mechanical. Breaking for the published `kio` crate; it stays in m0 because a remote peer can panic the publisher today. Removing `Sub` alongside `Add` is the same class (underflow before the clock's start). Each peer-reachable site gets a test at the extreme value. diff --git a/quest/m0/qmux-js-credit.md b/quest/m0/qmux-js-credit.md index be987bcc1e1..ec120fb2f6d 100644 --- a/quest/m0/qmux-js-credit.md +++ b/quest/m0/qmux-js-credit.md @@ -1,19 +1,17 @@ -# [S] JS qmux keeps credit and final sizes consistent with Rust +# [M] JS qmux streams follow QUIC's per-stream state machine ## Goal -A JS qmux session (the WebSocket fallback) never stalls or gets closed by a Rust peer because of its own flow-control bookkeeping, and a peer cannot grow its memory without bound: +`js/qmux` tracks each stream's send and receive halves through the states of [RFC 9000 ยง3](https://www.rfc-editor.org/rfc/rfc9000#section-3), as `rs/moq-quic/src/connection/streams/state.rs` does, instead of credit bookkeeping spread across `session.ts` handlers. A JS qmux session (the WebSocket fallback) then cannot leak or double-count credit by construction: -- Bytes that arrive on a stream we already stopped (late STREAM frames, and the final-size gap of a later RESET_STREAM) count toward connection receive credit. Today `js/qmux/src/session.ts` (`#onStream`, `#onResetStream`) returns early once the receive half is deleted on cancel, so each cancelled stream permanently shrinks the window until MAX_DATA is never sent again. -- A RESET_STREAM's final size is at least every byte already handed to the socket. Today `sendOffset` advances only after `scheduler.enqueueStream` resolves, so a STOP_SENDING handled while a frame is mid-`sink.write` replies with a short final size, which Rust's `rs/moq-quic/src/connection/streams/recv.rs` refuses with FINAL_SIZE_ERROR. -- MAX_DATA, MAX_STREAMS_{BIDI,UNI} and MAX_STREAM_DATA (per stream) replace any queued copy, as Rust does with its pending flags, instead of each taking a new entry on the uncapped control lane. -- An empty binary WebSocket message is a no-op on every qmux version, matching `rs/qmux/src/transport/websocket.rs`; qmux-00 currently throws. -- WebSocket messages larger than the record size are refused before they are buffered where the platform allows it: Rust `rs/qmux/src/ws.rs` `Upgraded` validates or sets the tungstenite `max_message_size` itself instead of trusting the caller, and JS documents the `ws` `maxPayload` it needs. +- A stream's flow record lives until both halves are terminal on the wire (all data or a reset received and consumed; FIN or RESET sent), not until the app releases its handle. Today `#maybeDeleteStreamFlow` drops it on cancel, so late STREAM bytes and a later RESET's final-size gap are never counted and the connection window shrinks until MAX_DATA stops (an honest Rust peer stalls the session). +- Connection and stream credit change only inside state transitions, so no handler can return early past the accounting (today twelve call sites do it by hand). +- The send half's offset counts bytes when the scheduler takes them, so a RESET_STREAM's final size covers a frame mid-`sink.write`. Today it does not, and Rust closes the connection with FINAL_SIZE_ERROR. +- Window updates (MAX_DATA, MAX_STREAMS_{BIDI,UNI}, MAX_STREAM_DATA) are pending flags or per-stream latest values drained by the scheduler, as Rust's `pending` flags are, not entries on the uncapped control lane. +- An empty binary WebSocket message is a no-op on every version, and messages over the record size are refused before buffering where the platform allows it (Rust `rs/qmux/src/ws.rs` `Upgraded` validates its own `max_message_size`; JS documents `ws` `maxPayload`). ## Plan -Found by the 2026-11-07 invariant audit (qmux-1 to qmux-5). qmux-1 and qmux-2 are the urgent ones: an honest Rust relay triggers both whenever a JS app cancels group streams over WebSocket. +Found by the 2026-11-07 invariant audit (qmux-1 to qmux-5); qmux-1 and qmux-2 are reproduced as tests before the redesign. -- Keep each stopped peer stream's receive offset (and final size once known) until FIN or RESET so late bytes are accounted, and port Rust's `credit_returns_for_unread_and_stopped_streams` (`rs/moq-quic/src/connection/mux/tests.rs`). -- Count bytes as sent when the scheduler takes the frame from its queue, or track the in-flight frame per stream and include it in the RESET final size. Test with a sink whose `write` is held open while STOP_SENDING arrives. -- Send the MAX_* frames through `replaceControl` keyed by type (and stream id for MAX_STREAM_DATA); fix the "bounded" claim in `scheduler.ts`. +Decided 2026-11-07 (maintainer, invariant-audit root causes): fix the cause, the hand-rolled flow accounting, rather than patching each early return. Mirror Rust's state shapes and port its credit tests (`credit_returns_for_unread_and_stopped_streams` in `rs/moq-quic/src/connection/mux/tests.rs` among them) so both implementations pin the same invariants. A single implementation for both languages is the longer-term route, planned in [Evaluate WASM-to-JS for @moq/net](/quest/m5/rs2ts-wasm2js.md); this does not wait for it. diff --git a/quest/m1/README.md b/quest/m1/README.md index 15190fbd61d..5754d016ffc 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -33,10 +33,10 @@ moves what remains of them once their claims merge. - [Drop set](/quest/m1/drop-set.md) - each cursor judges missing groups against its own budget, so a narrow reader drops promptly and a widening re-offers groups already reported dropped on lite-07 - [Only the relay's own stats publisher uses the stats prefix](/quest/m1/stats-prefix.md) - a broad grant can no longer publish unmetered media or spoof node stats under `.stats` -- [js/net enforces the input checks Rust does](/quest/m1/js-net-input-checks.md) - GOAWAY redirects to loopback spellings, credential logging, ANNOUNCE_INIT size, sequence range, and version pinning match Rust -- [Rust and JS accept exactly the same tokens and key sets](/quest/m1/token-parse-strict.md) - a null `exp` no longer means never-expiring in Rust, duplicate kids and bad scope roots are refused, and every relay lease is validated +- [js/net and moq-net decode the same inputs the same way](/quest/m1/js-net-input-checks.md) - a shared input corpus pins both decoders, and a typed endpoint stops loopback spellings from slipping past GOAWAY checks +- [Rust and JS run one token and key-set corpus](/quest/m1/token-parse-strict.md) - a shared corpus pins both parsers, so a null `exp` no longer means never-expiring in Rust - [A refused lite SUBSCRIBE_UPDATE ends the subscription](/quest/m1/lite-update-refused.md) - the publisher no longer serves a removed range or reports the wrong start after the model refuses an update -- [Cluster peers have one identity each and fresh inputs](/quest/m1/cluster-peer-identity.md) - a duplicate cluster id, a self-dial, or a rotated cluster token fails loud instead of partitioning or looping the mesh +- [A cluster peer's identity is a type, not a string](/quest/m1/cluster-peer-identity.md) - a canonical `PeerKey` and a refused duplicate cluster id stop self-dials, double links, and silent mesh partitions - [Media fuzz findings](/quest/m1/fuzz-findings/README.md) - the nightly media fuzz targets run clean: mp4-atom, MKV block, TS step, SPS overflow, and catalog encoder bugs they found are fixed - [A smaller, safer API than release](/quest/m1/api-audit/README.md) - every surface added since release has a consumer or is gone, overlapping operations are collapsed, names mirror across Rust, JS, and the bindings, and every break says how to migrate - [Routing guide](/quest/m1/routing-guide.md) - concepts and practical outbound federation setup @@ -61,7 +61,7 @@ moves what remains of them once their claims merge. - [QMux reads the app-limited flag on big-endian Linux](/quest/m1/qmux-app-limited-endian.md) - the send-rate filter's app-limited mask follows the target's byte order - [Copies of one group identity must agree beyond their bytes](/quest/m1/copy-identity.md) - copies that differ in timestamp, object fields, or frame count fail as a protocol violation - [Session auth grants and replies stay consistent and bounded](/quest/m1/auth-stream-bounds.md) - an AUTH_ERROR revokes its grant, queued grants and retiring tokens are bounded, and dropping queued requests cannot deadlock -- [Relay settings and admin input that cannot work are refused](/quest/m1/relay-config-checks.md) - clustered stats without a node name, pin without workers, repeated filter keys, and unknown transports fail loud +- [Relay settings parse once into a validated configuration](/quest/m1/relay-config-checks.md) - settings that cannot work together are refused at load instead of accepted and ignored - [Track drop runs are forgotten below the floor](/quest/m1/drop-runs-prune.md) - a track's dropped-group record stays proportional to its window in Rust and JS ## Related diff --git a/quest/m1/cluster-peer-identity.md b/quest/m1/cluster-peer-identity.md index e8c96fb58fc..2d01f8bddc5 100644 --- a/quest/m1/cluster-peer-identity.md +++ b/quest/m1/cluster-peer-identity.md @@ -1,15 +1,16 @@ -# [S] Cluster peers have one identity each and fresh inputs +# [S] A cluster peer's identity is a type, not a string ## Goal -A relay mesh fails loud on identity mistakes instead of silently partitioning or looping, and its peer inputs stay current and bounded (`rs/moq-relay/src/cluster.rs`, `relay.rs`): +Relay mesh identity mistakes are unrepresentable or fail loud (`rs/moq-relay/src/cluster.rs`, `session.rs`): -- A peer session whose declared hop (`Request::peer_hop`) equals our `cluster.id` is refused with an error naming the duplicate, instead of every route from it being dropped as reflected. -- One relay has one dial key: `DialTarget` keys on `canonicalize_peer_key` (which also strips URL userinfo, so credentials stop reaching logs and the `remote` span), and the self filter applies to static `cluster.connect` entries too. A test covers the self filter. -- `--cluster-lan` on a relay without a QUIC listener is refused at startup instead of advertising SRV port 0. +- A `PeerKey` type, constructible only through `canonicalize_peer_key`, keys the dial map, the self filter, LAN discovery and `/nodes`. Today `DialTarget` keys on the raw URL with only the query cleared, so one relay can get two links, a node can dial itself, and URL userinfo reaches logs and the `remote` span. The self filter applies to static `cluster.connect` entries too, and a test covers it. +- A peer session whose declared hop (`Request::peer_hop`) equals our `cluster.id` is refused with an error naming the duplicate, instead of every route from it being dropped as reflected and the mesh silently partitioning. - The `cluster.token` file is re-read on redial (or watched like the API file), so a rotated or expiring token does not break every redial until restart. -- The `cluster.api.url` body and file are read with a size cap; an oversized answer keeps the last good peer list. +- A `session::Registry` registration removes only its own entry, so a reused `Request` id cannot unlist another live session. ## Plan -Found by the 2026-11-07 invariant audit (relay-2, relay-4, relay-5, relay-7, relay-9, relay-10). +Found by the 2026-11-07 invariant audit (relay-2, relay-4, relay-5, relay-9, relay-13). + +Decided 2026-11-07 (maintainer, invariant-audit root causes): make the canonical key a type so no code path can key on a raw URL again. The audit's suggestion of a byte cap on the `cluster.api.url` body is dropped: that endpoint is operator-trusted, and a constant cap is not a design bound. diff --git a/quest/m1/js-net-input-checks.md b/quest/m1/js-net-input-checks.md index 92d4b051709..b2ef492063b 100644 --- a/quest/m1/js-net-input-checks.md +++ b/quest/m1/js-net-input-checks.md @@ -1,16 +1,15 @@ -# [S] js/net enforces the input checks Rust does +# [M] js/net and moq-net decode the same inputs the same way ## Goal -`@moq/net` rejects the inputs `rs/moq-net` rejects, so JS is not the weaker side of a session: +Rust and JS cannot drift apart on what they accept, because both run one shared corpus, and URLs are parsed once into a typed endpoint: -- A GOAWAY redirect is classified on the host actually dialed. `isLocal` in `js/net/src/connection/goaway.ts` parses the opaque `moql:`/`moqt:` host, so `127.1`, `0x7f000001` and `2130706433` pass as non-local while `dialable()` rewrites them to `https:`, which resolves to 127.0.0.1. With `redirect: "follow"` a relay can steer the client and its tokens to loopback. -- `redact` in `js/net/src/util/log.ts` logs scheme, host and port only. For `moql:` URLs `url.origin` is `"null"` and the path, which can carry a token, is kept. -- The lite-01/02 ANNOUNCE_INIT is capped like Rust's (`frame::SEGMENT - 16`) and its count is checked against the announce budget before decoding (`js/net/src/lite/announce.ts`). -- `checkSequence` (`js/net/src/util/sequence.ts`) refuses a group sequence at or above 2^64, as datagrams already do, so `appendGroup` after group 2^64-1 fails instead of minting a group past `UNBOUNDED`. -- `accept.ts` requires a pinned `AcceptProps.version` to be one the client offered, as `rs/moq-net/src/server.rs` does. -- `open.ts` `patience` treats an RTT of 0 as unknown, so a transport that reports 0 before its first sample does not make every stalled open reset a lower-priority stream. +- A shared corpus of moq-lite inputs, each with its expected outcome (decoded value, or refused), is replayed by `rs/moq-net` and `js/net` tests. It starts from the fuzz regressions in `rs/moq-net/fuzz/regressions` plus hand cases for every difference this audit found: the lite-01/02 ANNOUNCE_INIT size and count budget (JS allows 64 MiB and checks the announce budget only after building the array), group sequences at 2^64 (`checkSequence` accepts them), and a pinned server version the client did not offer (`accept.ts`). +- JS parses a dial target once into a typed endpoint (scheme, host as an IP or a name, port), the way `rs/moq-net/src/reconnect/redirect.rs` does, and GOAWAY classification, redirect checks, dialing and log redaction all use it. Today `isLocal` (`js/net/src/connection/goaway.ts`) reads the opaque `moql:` host, so `127.1`, `0x7f000001` and `2130706433` pass as non-local while `dialable()` turns them into 127.0.0.1, which lets a relay steer a `redirect: "follow"` client and its tokens to loopback. `redact` logs `null/` for `moql:` URLs, path tokens included. +- `open.ts` `patience` treats an RTT of 0 as unknown. ## Plan -Found by the 2026-11-07 invariant audit (js-2, js-5 to js-9). Tests for each, including the three loopback spellings. js-1 (announce queue growth) is already [JS announced coalescing](/quest/m3/js-announced-coalesce.md). +Found by the 2026-11-07 invariant audit (js-2, js-5 to js-9). + +Decided 2026-11-07 (maintainer, invariant-audit root causes): the cause is two hand-written decoders with no shared inputs; the corpus makes any divergence a failing test on both sides. Pick the corpus location and format to suit both test runners (a directory of JSON or hex cases is the obvious start). The single-implementation route stays [Evaluate WASM-to-JS for @moq/net](/quest/m5/rs2ts-wasm2js.md). js-1 (announce queue growth) is [JS announced coalescing](/quest/m3/js-announced-coalesce.md). diff --git a/quest/m1/relay-config-checks.md b/quest/m1/relay-config-checks.md index 605cb5b2e87..52801d5d04d 100644 --- a/quest/m1/relay-config-checks.md +++ b/quest/m1/relay-config-checks.md @@ -1,16 +1,23 @@ -# [S] Relay settings and admin input that cannot work are refused +# [S] Relay settings parse once into a validated configuration ## Goal -moq-relay refuses at load or request time what it currently accepts and then ignores or mishandles: +moq-relay turns its CLI, environment and TOML settings into one validated configuration at load, and the rest of the relay takes that type, so a setting cannot be accepted and then ignored. Cross-field rules live in that one step instead of across `Config::build`, `Cluster::new`, `Cluster::start` and `Relay::load`. The cases this audit found are refused there: -- Stats enabled on a clustered relay without `stats.node` is refused, or `stats.node` defaults to `cluster.node`, so nodes stop overwriting each other's `/node` broadcast. -- `runtime.pin` without `runtime.workers` is refused, matching the file's "refused rather than ignored" rule. -- A repeated key on `/sessions` or `/sessions/revalidate` (`session.rs` `Filter::from_query`) is an error instead of the last value winning. -- Refusal counters (`refusals.rs`) stay bounded with stats disabled, where nothing calls the pruning `Registry::report`. -- `auth.rs` maps every `moq_tokio::Transport` without `unreachable!` (the enum is `#[non_exhaustive]`), and the qmux-01-only ALPN choice in `websocket.rs` is derived from the version, not a hand-kept string list. -- `session::Registry` registration drops only their own entry, so an embedder registering a reused `Request` id cannot unlist another live session. +- Stats enabled on a clustered relay without `stats.node`; or `stats.node` defaults to `cluster.node`. Without either, nodes overwrite each other's `/node` broadcast. +- `--cluster-lan` without a QUIC listener; today it advertises SRV port 0. +- `runtime.pin` without `runtime.workers`. + +The same parse-once rule applies to the admin endpoints: a repeated key on `/sessions` or `/sessions/revalidate` (`Filter::from_query`) is an error, not last-wins. + +Smaller gaps fixed in the same PR: + +- Refusal counters (`refusals.rs`) stay bounded with stats disabled, where nothing prunes them. +- `auth.rs` maps every `moq_tokio::Transport` without `unreachable!`. +- `websocket.rs` derives its qmux-01-only ALPN choice from the version instead of a hand-kept list. ## Plan -Found by the 2026-11-07 invariant audit (relay-3, relay-6, relay-8, relay-11 to relay-14). +Found by the 2026-11-07 invariant audit (relay-3, relay-6, relay-7, relay-8, relay-11, relay-12, relay-14). + +Decided 2026-11-07 (maintainer, invariant-audit root causes): parse, don't validate. The existing settings drift test stays and covers the new type. diff --git a/quest/m1/token-parse-strict.md b/quest/m1/token-parse-strict.md index aa825ef3828..8aeea378630 100644 --- a/quest/m1/token-parse-strict.md +++ b/quest/m1/token-parse-strict.md @@ -1,15 +1,20 @@ -# [S] Rust and JS accept exactly the same tokens and key sets +# [S] Rust and JS run one token and key-set corpus ## Goal -Token and key parsing is strict and identical in `rs/moq-auth` and `js/auth`, and every grant the relay holds has been validated: +`rs/moq-auth` and `js/auth` load one shared corpus of tokens, key sets and JWKs, each marked accepted (with the resulting grant) or refused, so a parser difference fails a test on both sides. The cases this audit found pass: -- `exp`, `iat` and `nbf` refuse `null` and values outside the range both languages represent. Today Rust reads `"exp": null` as no expiry (a token that never expires) while JS refuses it; the Grant wire `expires` has the same gap. -- A KeySet with a duplicate `kid` is refused at parse in both languages instead of the first match winning. +- `exp`, `iat` and `nbf` refuse `null` and values outside the range both languages represent. Rust reads `"exp": null` as no expiry, a token that never expires, while JS refuses it; the Grant wire `expires` has the same gap. +- A key set with a duplicate `kid` is refused in both languages instead of the first match winning. - A JWK's scope root passes `path::validate` on import in both languages, as the `with_scope` doc promises. -- The relay detects a token the way the server parses one (`presents_token` vs form-decoded `jwt`), so `?%6Awt=` cannot fall through to the public grant; key sources are cached instead of re-read and re-parsed per token. -- `Lease::new` (`rs/moq-relay/src/auth.rs`) runs `Grant::validate`, so an embedder's grant cannot skip the tier charset and upstream-implies-peer checks. + +Two relay-side gaps found beside them are fixed in the same PR: + +- The relay detects a token the way the server parses one (`presents_token` vs the form-decoded `jwt`), so `?%6Awt=` cannot fall through to the public grant, and key sources are cached rather than re-read per token. +- `Lease::new` (`rs/moq-relay/src/auth.rs`) runs `Grant::validate`. ## Plan -Found by the 2026-11-07 invariant audit (auth-2, auth-4, auth-9, auth-11, auth-12). Each refusal gets a vector in the Rust/JS interop test (`js/auth/src/interop.test.ts`), including a null-`exp` token. +Found by the 2026-11-07 invariant audit (auth-2, auth-4, auth-9, auth-11, auth-12). + +Decided 2026-11-07 (maintainer, invariant-audit root causes): the cause is two hand-written parsers checked only by JS loading a few Rust-signed fixtures (`js/auth/src/interop.test.ts`). The corpus replaces those fixtures and runs in both directions. diff --git a/quest/m3/mesh/README.md b/quest/m3/mesh/README.md index 2e1b18d2c53..1a4ffc22529 100644 --- a/quest/m3/mesh/README.md +++ b/quest/m3/mesh/README.md @@ -95,7 +95,7 @@ the third, and survives a relay restart. - [Byte streams](/quest/m3/stream/README.md) - lossless TCP-level streams over MoQ - [Endpoint API](/quest/m3/mesh/endpoint.md) - an iroh-shaped `Endpoint` that connects to an endpoint ID and service and accepts byte streams - [Mesh relay node](/quest/m3/mesh/node.md) - a relay node enrolls into a mesh, dials its relays, and admits only enrolled LAN peers -- [Mesh enrollment is bounded and revocation takes effect](/quest/m3/mesh/device-bounds.md) - a reusable key enrolls a bounded number of devices and a revoked device loses access within a bounded time +- [Mesh enrollment is bounded and revocation takes effect](/quest/m3/mesh/device-bounds.md) - a reusable key enrolls at most its own use budget and a revoked device loses access within a bounded time - [Address hints](/quest/m3/mesh/addr.md) - nodes publish their addresses on mesh tracks and dial each other directly where mDNS cannot find them - [Sparse overlay](/quest/m3/mesh/overlay.md) - each node keeps about log N deterministic direct links, forwards for the rest, and falls back to relays across a partition - [JS mesh enrollment client](/quest/m3/mesh/js-enroll.md) - `@moq/auth` enrolls and refreshes from JS like `moq_auth::mesh::Client` diff --git a/quest/m3/mesh/device-bounds.md b/quest/m3/mesh/device-bounds.md index 6dffe8c0ccc..2a9976d684f 100644 --- a/quest/m3/mesh/device-bounds.md +++ b/quest/m3/mesh/device-bounds.md @@ -4,9 +4,9 @@ The mesh coordinator (`rs/moq-auth/src/mesh/coordinator.rs`) stops assuming its device store stays small and its revocations wait for token expiry: -- A reusable enrollment key enrolls a bounded number of devices, and `/enroll` is rate limited; today each enroll clones the whole store, rewrites the state file and returns every member under one mutex, so a leaked key can mint thousands of endpoints. +- A reusable enrollment key carries a use budget set when it is created, so the coordinator holds at most what its keys allow. Today a reusable key enrolls any number of endpoints, and each enroll clones the whole store, rewrites the state file and returns every member under one mutex, so a leaked key can mint thousands. - A revoked device loses access within a bounded time, not only at its token's `exp` (1h by default): either `decide` checks revocation for mesh tokens, or mesh tokens are short-lived with revalidation on. ## Plan -Found by the 2026-11-07 invariant audit (auth-7, auth-8). +Found by the 2026-11-07 invariant audit (auth-7, auth-8). The budget belongs to the key's owner, not a server-wide constant or rate limit, per REVIEW.md's rule against constant caps. From 915a613f804fce14b55ffbd994b6f8153ac99d36 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 7 Nov 2026 10:34:21 -0800 Subject: [PATCH 3/4] quest: name Config::resolve in relay-config-checks Co-Authored-By: Claude Opus 5.5 --- quest/m1/relay-config-checks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/quest/m1/relay-config-checks.md b/quest/m1/relay-config-checks.md index 52801d5d04d..18200179b47 100644 --- a/quest/m1/relay-config-checks.md +++ b/quest/m1/relay-config-checks.md @@ -2,7 +2,7 @@ ## Goal -moq-relay turns its CLI, environment and TOML settings into one validated configuration at load, and the rest of the relay takes that type, so a setting cannot be accepted and then ignored. Cross-field rules live in that one step instead of across `Config::build`, `Cluster::new`, `Cluster::start` and `Relay::load`. The cases this audit found are refused there: +moq-relay turns its CLI, environment and TOML settings into one validated configuration at load, and the rest of the relay takes that type, so a setting cannot be accepted and then ignored. Cross-field rules live in that one step instead of across `Config::resolve`, `Cluster::new`, `Cluster::start` and `Relay::load`. The cases this audit found are refused there: - Stats enabled on a clustered relay without `stats.node`; or `stats.node` defaults to `cluster.node`. Without either, nodes overwrite each other's `/node` broadcast. - `--cluster-lan` without a QUIC listener; today it advertises SRV port 0. From 857c00dcbcdb69f587517413ee51b81784d03732 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 7 Nov 2026 14:22:22 -0800 Subject: [PATCH 4/4] quest: cross-link qmux stream states with the explicit-state close quest Co-Authored-By: Claude Opus 5.5 --- quest/m0/qmux-js-credit.md | 4 ++++ quest/m1/explicit-state/qmux-close.md | 4 ++++ 2 files changed, 8 insertions(+) diff --git a/quest/m0/qmux-js-credit.md b/quest/m0/qmux-js-credit.md index ec120fb2f6d..499c10e159d 100644 --- a/quest/m0/qmux-js-credit.md +++ b/quest/m0/qmux-js-credit.md @@ -15,3 +15,7 @@ Found by the 2026-11-07 invariant audit (qmux-1 to qmux-5); qmux-1 and qmux-2 are reproduced as tests before the redesign. Decided 2026-11-07 (maintainer, invariant-audit root causes): fix the cause, the hand-rolled flow accounting, rather than patching each early return. Mirror Rust's state shapes and port its credit tests (`credit_returns_for_unread_and_stopped_streams` in `rs/moq-quic/src/connection/mux/tests.rs` among them) so both implementations pin the same invariants. A single implementation for both languages is the longer-term route, planned in [Evaluate WASM-to-JS for @moq/net](/quest/m5/rs2ts-wasm2js.md); this does not wait for it. + +## Related + +- [js/qmux close](/quest/m1/explicit-state/qmux-close.md) - the session close phase and `recv.ts` stream-end sweep; land the per-stream state machine so both share one stream state diff --git a/quest/m1/explicit-state/qmux-close.md b/quest/m1/explicit-state/qmux-close.md index ff7158236cc..344adc8c121 100644 --- a/quest/m1/explicit-state/qmux-close.md +++ b/quest/m1/explicit-state/qmux-close.md @@ -39,3 +39,7 @@ Also sweep `js/qmux/src/recv.ts` and `js/qmux/src/credit.ts`: [moq-dev/moq#8136](https://github.com/moq-dev/moq/pull/8136) and [moq-dev/moq#8088](https://github.com/moq-dev/moq/pull/8088) also edit `session.ts`. Expect merge conflicts. + +## Related + +- [JS qmux streams follow QUIC's per-stream state machine](/quest/m0/qmux-js-credit.md) - its stream states subsume the `recv.ts` end flags; sweep `recv.ts` on top of it