From e841f4771e6504e6b1aa529a8c645434a6de38dc Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 22 Oct 2026 20:56:11 -0700 Subject: [PATCH 01/10] chore(quest): claim quest/m1/compress/ietf-range-widen.md Quest-Claim: 793e6971637808ef From fdd55e513710fc4994e4494568c870714b24d003 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 22 Oct 2026 22:50:39 -0700 Subject: [PATCH 02/10] fix(net): widen moq-transport mid-group starts on compressed tracks A moq-transport subscriber now widens a mid-group SUBSCRIBE start to the group's first frame when it lacks the compressed prefix, as lite-07 does. A cold request learns compression only from SUBSCRIBE_OK, so on a session that negotiated it every mid-group start widens. Groups whose ingest pins Object IDs to frame ordinals (a resumed start or its fill) now get the reader's frame window, so an application on an IETF hop reads only the frames it asked for, on plain tracks too. Co-Authored-By: Claude Opus 5.5 --- quest/m1/compress/README.md | 1 - quest/m1/compress/ietf-range-widen.md | 58 -------- rs/moq-net/src/ietf/subscriber.rs | 42 +++++- rs/moq-net/src/lite/ietf.rs | 4 + rs/moq-net/src/model/group.rs | 13 +- rs/moq-net/src/model/resume.rs | 3 + rs/moq-net/src/model/track.rs | 70 +++++---- rs/moq-net/tests/integration/compress.rs | 182 +++++++++++++++++------ 8 files changed, 231 insertions(+), 142 deletions(-) delete mode 100644 quest/m1/compress/ietf-range-widen.md diff --git a/quest/m1/compress/README.md b/quest/m1/compress/README.md index 4b94a3b0d9a..b2d4c867931 100644 --- a/quest/m1/compress/README.md +++ b/quest/m1/compress/README.md @@ -81,7 +81,6 @@ This README owns the end-to-end proof and the guide once every child lands: ## Required -- [moq-transport mid-group subscriptions widen](/quest/m1/compress/ietf-range-widen.md) - an application range starting mid-group on a compressed track widens to frame 0 at a moq-transport subscriber instead of failing with `Lagged` - [Migrate](/quest/m1/compress/migrate.md) - catalogs, stats, and timelines publish one compressed track; moq-flate, @moq/flate, and every `.z` name are gone ## Related diff --git a/quest/m1/compress/ietf-range-widen.md b/quest/m1/compress/ietf-range-widen.md deleted file mode 100644 index 62b818c0fc0..00000000000 --- a/quest/m1/compress/ietf-range-widen.md +++ /dev/null @@ -1,58 +0,0 @@ -# [M] A moq-transport mid-group subscription to a compressed track widens to frame 0 - -## Goal - -An application range starting mid-group on a compressed track, requested over -moq-transport draft-17+ with the Flate extension negotiated, reads the frames -it asked for instead of failing with `Lagged` against a publisher that serves -exactly the requested Object range. The requesting IETF subscriber widens the -SUBSCRIBE start to Object 0 when it lacks the group's prefix, as lite-07 does -since [moq-dev/moq#6242](https://github.com/moq-dev/moq/pull/6242). The server -never widens and keeps serving exactly what is asked. - -## Plan - -The rule is the [epic](/quest/m1/compress/README.md)'s Ranges rule, decided -with the maintainer on 2026-10-20: the subscriber widens, never the server; -one that holds frames 0..N-1 still asks from N; the reader still starts at the -frame it asked for. Lite-07 applies it through `track::Producer::widen_compressed` -(`rs/moq-net/src/model/track.rs`), which a warm IETF copy can reuse. - -Split from #6242 because a two-hop matrix (lite-07 and moq-transport-17, 18 -and 22 on either hop) found three gaps under the IETF requester that are not -about compression: - -- A cold SUBSCRIBE (`Target::Request` in `rs/moq-net/src/ietf/subscriber.rs`) - learns the track is compressed only from SUBSCRIBE_OK. Widening every cold - mid-group start on a Flate session would make plain tracks pay for the - prefix too. Decide between learning the properties first (TRACK_STATUS where - `TrackStatusOk::describes_track` holds) and widening cold starts only on - Flate sessions. -- Readers get IETF-received groups whole: `track::Subscriber::deliver` skips - the frame window when `group.relayed()`, since Object IDs may not be frame - ordinals. An application on an IETF hop therefore already reads frames below - its start on plain tracks, and widening would hand it more. Filter by - ordinal when the copy's Object IDs are dense, or by Object ID. -- REQUEST_UPDATE carries no range, so a live range change never reaches - upstream; [moq-transport ranges](/quest/m1/subscribe-ranges/ietf.md) owns - that. - -In the matrix, our own IETF publisher served the whole group for a mid-group -Absolute start, so the `Lagged` failure needs a publisher that honors the -exact start; test against the publisher model directly. - -`js/net` needs no mirror: its subscriptions carry group ranges only -(`Subscription.groups`), so a JS SUBSCRIBE always starts at frame 0, and a -compressed group that does not is refused with `TooFarBehind`. - -Regression: extend -`application_ranges_keep_their_filter_after_requesting_the_compressed_prefix` -(`rs/moq-net/tests/integration/compress.rs`) over moq-transport-17, 18 and 22 -hops. - -Public API: none. Wire: none. - -## Related - -- [moq-transport ranges](/quest/m1/subscribe-ranges/ietf.md) - carries a live range change upstream on REQUEST_UPDATE -- [Bounded tails](/quest/m1/subscribe-ranges/bounded-tail.md) - a copy cut at a request's end must accept its tail before a warm prefix can be extended diff --git a/rs/moq-net/src/ietf/subscriber.rs b/rs/moq-net/src/ietf/subscriber.rs index 8eb9a27b13f..3decb27ce95 100644 --- a/rs/moq-net/src/ietf/subscriber.rs +++ b/rs/moq-net/src/ietf/subscriber.rs @@ -486,6 +486,18 @@ impl Target { } } + /// Widen a mid-group start to the group's first frame unless the copy holds the frames + /// below it, so a compressed group can inflate. A request learns whether the track is + /// compressed only from SUBSCRIBE_OK, so with compression `negotiated` it widens every + /// mid-group start, and a plain track pays for that group's prefix. + fn widen(&self, subscription: &mut track::Subscription, negotiated: bool) { + match self { + Self::Request(request) if negotiated => request.widen_compressed(subscription), + Self::Request(_) => {} + Self::Resume(idle) => idle.track.widen_compressed(subscription), + } + } + /// The subscription could not be made: the track fails with `err`. fn fail(self, err: Error) { match self { @@ -576,9 +588,11 @@ struct TrackState { // The joining FETCH's own request id, when one was sent. fetch_id: Option, - // Where the subscription asked to start, when that is partway through a group: a - // resumed subscription picking up after the frames a previous route delivered. The - // stream for that group legitimately starts there, without a head. + // Where the subscription asked to start, when the request starts partway through a + // group: a resumed subscription picking up after the frames a previous route + // delivered, or an application range. The stream for that group legitimately starts + // there, without a head, and its Object IDs stay pinned to the ordinals the readers' + // windows count in. A compressed group's start is widened to its first frame. resume: Option, // The data streams read so far, which PUBLISH_DONE's Stream Count is checked against. @@ -2813,7 +2827,12 @@ where gate: &mut crate::auth::Gate, ) -> Option { let validity = self.validity.start(); - let subscription = target.subscription(); + let requested = target.subscription(); + let negotiated = self.peer_setup.get().await.flate; + let subscription = requested.clone().map(|mut subscription| { + target.widen(&mut subscription, negotiated); + subscription + }); let start = subscription.as_ref().and_then(|s| s.start()); let join = match subscribe_join(start, subscription.as_ref().and_then(|s| s.end()), self.version) { Ok(join) => join, @@ -2936,10 +2955,11 @@ where state.subscribes.insert( request_id, TrackState { - resume: subscription + resume: requested .as_ref() .and_then(|s| s.start()) - .filter(|start| start.frame != 0), + .filter(|start| start.frame != 0) + .and(start), // A resumed copy is the session's to abort from here on. producer: target.producer().cloned(), epoch: self.epoch.clone(), @@ -4338,10 +4358,14 @@ where } }; + // A resumed group's ingest refuses gaps and drops what the copy already holds, so a + // stream starting at or below the copy's next frame keeps each Object ID its ordinal. + let stitch = resume.map(|_| start); if passthrough || producer.wire().is_none() { let first_id = start.max(producer.frame_count() as u64); producer.set_wire(group::Wire { sparse: false, + pinned: stitch.is_some_and(|start| start <= producer.frame_count() as u64), first_object: group.flags.first_object && first_id == start, first_id, extensions: group.flags.has_extensions, @@ -4356,7 +4380,7 @@ where let mut producer = crate::recv::Group::new(producer); let observing = producer.handle().clone(); - let mut ingest = GroupIngest::new(self, &group, timescale, resume.map(|_| start)); + let mut ingest = GroupIngest::new(self, &group, timescale, stitch); let res = { kio::wait(|waiter| { if let Poll::Ready(err) = track.poll_closed(waiter) { @@ -5058,6 +5082,8 @@ where && let Some(mut wire) = producer.wire() { wire.first_id = id; + // Resume refuses gaps here and in the tail stream that continues this group. + wire.pinned = resume.is_some_and(|resume| resume.group == *sequence); producer.set_wire(wire)?; } if *next == 0 @@ -5530,6 +5556,7 @@ where let (producer, track, timescale, start, end) = taken?; producer.set_wire(group::Wire { sparse: false, + pinned: false, first_object: start == 0, first_id: start, extensions: true, @@ -5801,6 +5828,7 @@ fn open_fill_group( let producer = track.create_group(group::Info { sequence })?; producer.set_wire(group::Wire { sparse: false, + pinned: false, first_object: true, first_id: 0, extensions: true, diff --git a/rs/moq-net/src/lite/ietf.rs b/rs/moq-net/src/lite/ietf.rs index 83b88b2843a..d673771bf00 100644 --- a/rs/moq-net/src/lite/ietf.rs +++ b/rs/moq-net/src/lite/ietf.rs @@ -69,6 +69,7 @@ pub(crate) fn decode_state(state: u64, frame_start: u64) -> Result bool { - self.cursor.state.read().wire().is_some() + /// Whether frame ordinals address this group's objects: it was not relayed from + /// moq-transport, or its ingest pinned every Object ID to its ordinal. + pub(super) fn ordinal(&self) -> bool { + self.cursor.state.read().wire().is_none_or(|wire| wire.pinned) } /// Read a compressed group's stored DEFLATE slices instead of plain frames, with each @@ -3420,6 +3423,7 @@ mod test { group .set_wire(Wire { sparse: true, + pinned: false, first_object: false, first_id: 6, extensions: true, @@ -3871,6 +3875,7 @@ mod test { }; let wire = Wire { sparse: false, + pinned: false, first_object: false, first_id: 2, extensions: true, diff --git a/rs/moq-net/src/model/resume.rs b/rs/moq-net/src/model/resume.rs index 9279f581176..907cbfcee84 100644 --- a/rs/moq-net/src/model/resume.rs +++ b/rs/moq-net/src/model/resume.rs @@ -4617,6 +4617,7 @@ mod test { group .set_wire(group::Wire { sparse: false, + pinned: false, first_object: true, first_id: 6, extensions: false, @@ -4685,6 +4686,7 @@ mod test { group .set_wire(group::Wire { sparse: false, + pinned: false, first_object: true, first_id: 6, extensions: false, @@ -4731,6 +4733,7 @@ mod test { group .set_wire(group::Wire { sparse: false, + pinned: false, first_object: false, first_id: 6, extensions: false, diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index aa2ebf373a3..21f039b4612 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -812,6 +812,34 @@ impl Holdings for TrackState { } impl TrackState { + /// Start each mid-group range at its group's first frame unless a copy holds the + /// frames below it; see [`Producer::widen_compressed`]. + #[cfg_attr( + not(feature = "async"), + allow(dead_code, reason = "Only session adapters request upstream.") + )] + fn widen_compressed(&self, subscription: &mut Subscription) { + for range in &mut subscription.ranges { + if range.is_empty() { + continue; + } + let Anchor::Absolute(start) = &mut range.start else { + continue; + }; + if start.frame == 0 { + continue; + } + let prefix = self + .lookup + .get(&start.group) + .and_then(|slot| slot.copies().find(|group| group.holds_prefix(start.frame))); + match prefix { + Some(group) => group.cache_refresh(), + None => start.frame = 0, + } + } + } + /// A consumer joined at the live edge: ask for a group past every sequence written when /// the newest group's bytes after its first frame exceed [`Info::refresh`] times that /// frame. @@ -3369,28 +3397,8 @@ impl Producer { )] // private: capability gap pub(crate) fn widen_compressed(&self, subscription: &mut Subscription) { - if !self.info.compress { - return; - } - let state = self.state.read(); - for range in &mut subscription.ranges { - if range.is_empty() { - continue; - } - let Anchor::Absolute(start) = &mut range.start else { - continue; - }; - if start.frame == 0 { - continue; - } - let prefix = state - .lookup - .get(&start.group) - .and_then(|slot| slot.copies().find(|group| group.holds_prefix(start.frame))); - match prefix { - Some(group) => group.cache_refresh(), - None => start.frame = 0, - } + if self.info.compress { + self.state.read().widen_compressed(subscription); } } @@ -6037,10 +6045,11 @@ impl Cursor { /// /// Positioned without clamping, so a group missing the head asked for reads it as /// [`Error::Lagged`], which a route copy recovers from another. A group relayed from - /// moq-transport is handed out whole, since its ranges may name Object IDs rather than - /// frame ordinals, so its reader filters by ID. + /// moq-transport is handed out whole unless its ingest pinned Object IDs to ordinals: + /// its ranges may name Object IDs, so an ordinal window could drop objects they cover, + /// and its reader filters by ID. fn deliver(&self, mut group: group::Consumer) -> group::Consumer { - if !group.relayed() { + if group.ordinal() { let (start, end) = self.filter.window.frames(group.sequence); group.skip_to(start); if let Some(end) = end { @@ -7561,6 +7570,17 @@ impl Request { snapshot_subscription(&subs, bound) } + /// As [`Producer::widen_compressed`], for a track whose info is not known yet: the + /// requester calls it when the answer may declare the track compressed. + #[cfg_attr( + not(feature = "async"), + allow(dead_code, reason = "Only session adapters request upstream.") + )] + // private: capability gap + pub(crate) fn widen_compressed(&self, subscription: &mut Subscription) { + self.state.read().widen_compressed(subscription); + } + /// Block until the aggregate [`subscription`](Self::subscription) changes, /// yielding `None` once nobody is waiting. #[cfg(feature = "async")] diff --git a/rs/moq-net/tests/integration/compress.rs b/rs/moq-net/tests/integration/compress.rs index a846b1efac3..ee9d419b17f 100644 --- a/rs/moq-net/tests/integration/compress.rs +++ b/rs/moq-net/tests/integration/compress.rs @@ -52,17 +52,45 @@ struct Fixture { _origins: Vec, } -/// A compressed track crosses a relay between the requested protocols. -async fn connect(upstream: &str, downstream: &str) -> Fixture { - connect_with(upstream, downstream, moq_net::track::Subscription::default()).await +/// A track announced by the original publisher, before any relay carries it. +struct Published { + publisher: moq_net::origin::Producer, + broadcast: moq_net::broadcast::Producer, + track: moq_net::track::Producer, } -/// As [`connect`], with the downstream application asking for `subscription`. -async fn connect_with(upstream: &str, downstream: &str, subscription: moq_net::track::Subscription) -> Fixture { +/// Announce one track described by `info`. +fn publish(info: moq_net::track::Info) -> Published { let publisher = produce_origin(1); let broadcast = publisher.create_broadcast("bcast").unwrap(); - let track = broadcast.create_track("data", compressed()).unwrap(); + let track = broadcast.create_track("data", info).unwrap(); broadcast.announce(Default::default()).unwrap(); + Published { + publisher, + broadcast, + track, + } +} + +/// A compressed track crosses a relay between the requested protocols. +async fn connect(upstream: &str, downstream: &str) -> Fixture { + relay( + publish(compressed()), + [upstream, downstream], + moq_net::track::Subscription::default(), + ) + .await +} + +/// `published` crosses a relay over the `[upstream, downstream]` protocols to an application +/// asking for `subscription`. +async fn relay(published: Published, hops: [&str; 2], subscription: moq_net::track::Subscription) -> Fixture { + let Published { + publisher, + broadcast, + track, + } = published; + let [upstream, downstream] = hops; let relay = produce_origin(2); let mut options = MockConnectOptions::new(upstream.parse::().unwrap()); @@ -91,52 +119,112 @@ async fn connect_with(upstream: &str, downstream: &str, subscription: moq_net::t } } -/// A cold subscriber needs the compressed prefix, but its application sees only its -/// requested half-open frame range, including when it changes that range while live. +/// Hops whose requester carries an application's mid-group start: lite-07, and +/// moq-transport with compression negotiated. +const RANGED: &[&str] = &[ + "moq-lite-07-wip", + "moq-transport-17", + "moq-transport-18", + "moq-transport-22", +]; + +/// A cold subscriber needs a compressed group's prefix, but its application sees only its +/// requested half-open frame range across every pair of hops, whether the group began before +/// it subscribed or after, and on a plain track too. Lite-07 hops also carry a live change. #[moq_clock::test] async fn application_ranges_keep_their_filter_after_requesting_the_compressed_prefix() { use moq_net::track::{Anchor, Position, Range, Subscription}; - moq_clock::timeout(TIMEOUT, async { - let range = |group, start, end| { - Subscription::default().with_ranges([Range { - start: Anchor::Absolute(Position { group, frame: start }), - end: Some(Anchor::Absolute(Position { group, frame: end })), - }]) - }; - let mut fixture = connect_with("moq-lite-07-wip", "moq-lite-07-wip", range(5, 2, 4)).await; - assert_eq!( - fixture.track.subscription_changed().await.unwrap().unwrap().ranges[0].start, - Anchor::Absolute(Position::group(5)), - "upstream asks for the prefix" - ); - for sequence in [5, 6] { - if sequence == 6 { - fixture.subscriber.update(range(6, 1, 3)).unwrap(); - let updated = fixture.track.subscription_changed().await.unwrap().unwrap(); - // Old route copies can keep their earlier floor until a newer group arrives. - // The updated end must already permit the newly requested group. - assert_eq!( - updated.ranges[0].end, - Some(Anchor::Absolute(Position { group: 6, frame: 3 })) - ); - } - let mut group = fixture.track.create_group(moq_net::group::Info { sequence }).unwrap(); - for index in 0..5 { - group - .write_frame(Timestamp::from_millis(index).unwrap(), payload(index)) - .unwrap(); - } - group.finish().unwrap(); - let mut reading = fixture.subscriber.recv_group().await.unwrap().unwrap(); - let (start, end) = if sequence == 5 { (2, 4) } else { (1, 3) }; - for index in start..end { - assert_eq!(reading.read_frame().await.unwrap().unwrap().payload, payload(index)); + let range = |group, start, end: Option| { + Subscription::default().with_ranges([Range { + start: Anchor::Absolute(Position { group, frame: start }), + end: end.map(|end| Anchor::Absolute(Position { group, frame: end })), + }]) + }; + let write = |group: &mut moq_net::group::Producer, frames: std::ops::Range| { + for index in frames { + group + .write_frame(Timestamp::from_millis(index).unwrap(), payload(index)) + .unwrap(); + } + }; + for &upstream in RANGED { + for &downstream in RANGED { + let hops = [upstream, downstream]; + let lite = hops.iter().all(|hop| *hop == "moq-lite-07-wip"); + // A pre-draft-20 join cannot carry an end, so its range stays open. + let end = hops + .iter() + .all(|hop| matches!(*hop, "moq-lite-07-wip" | "moq-transport-22")) + .then_some(4); + for compress in [true, false] { + // Frames of the group written before the subscription, which a joining FETCH + // or fill delivers. + for head in [0, 3] { + let case = format!("{upstream} -> {downstream}, compress {compress}, head {head}"); + moq_clock::timeout(TIMEOUT, async { + let info = match compress { + true => compressed(), + false => moq_net::track::Info::default().with_timescale(moq_net::Timescale::MILLI), + }; + let published = publish(info); + let mut group = published + .track + .create_group(moq_net::group::Info { sequence: 5 }) + .unwrap(); + write(&mut group, 0..head); + let mut fixture = relay(published, hops, range(5, 2, end)).await; + if lite { + assert_eq!( + fixture.track.subscription_changed().await.unwrap().unwrap().ranges[0].start, + Anchor::Absolute(Position { + group: 5, + frame: if compress { 0 } else { 2 } + }), + "{case}: upstream asks for the prefix a compressed group needs" + ); + } + write(&mut group, head..5); + group.finish().unwrap(); + let mut reading = fixture.subscriber.recv_group().await.unwrap().unwrap(); + for index in 2..end.unwrap_or(5) { + assert_eq!( + reading.read_frame().await.unwrap().unwrap().payload, + payload(index), + "{case}" + ); + } + assert!(reading.read_frame().await.unwrap().is_none(), "{case}"); + + // REQUEST_UPDATE carries no range: /quest/m1/subscribe-ranges/ietf.md. + if !lite { + return; + } + fixture.subscriber.update(range(6, 1, Some(3))).unwrap(); + let updated = fixture.track.subscription_changed().await.unwrap().unwrap(); + // Old route copies can keep their earlier floor until a newer group arrives. + // The updated end must already permit the newly requested group. + assert_eq!( + updated.ranges[0].end, + Some(Anchor::Absolute(Position { group: 6, frame: 3 })) + ); + let mut group = fixture + .track + .create_group(moq_net::group::Info { sequence: 6 }) + .unwrap(); + write(&mut group, 0..5); + group.finish().unwrap(); + let mut reading = fixture.subscriber.recv_group().await.unwrap().unwrap(); + for index in 1..3 { + assert_eq!(reading.read_frame().await.unwrap().unwrap().payload, payload(index)); + } + assert!(reading.read_frame().await.unwrap().is_none()); + }) + .await + .unwrap_or_else(|_| panic!("{case}: the requested frames never arrived")); + } } - assert!(reading.read_frame().await.unwrap().is_none()); } - }) - .await - .expect("the requested frames arrive"); + } } /// Whole and chunked frames on a compressed track read back as written on every version, From cce4513ec5235d252d81ff858f8763e54a082253 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 23 Oct 2026 11:44:29 -0700 Subject: [PATCH 03/10] fix(net): start every upstream subscription at a group's first frame Compressed tracks are no longer special: a mid-group start widens to frame 0 on every track and protocol unless a cached copy holds the frames below it, as when resuming from another route. The lite draft's SUBSCRIBE Frame Start says so. Co-Authored-By: Claude Opus 5.5 --- drafts/draft-lcurley-moq-lite.md | 3 +- quest/m1/compress/README.md | 15 +++++---- quest/m2/net-wrap/e2ee.md | 6 ++-- rs/moq-net/benches/codec.rs | 12 +++----- rs/moq-net/src/fuzz.rs | 6 ++-- rs/moq-net/src/ietf/subscriber.rs | 19 +++++------- rs/moq-net/src/lite/subscriber.rs | 26 ++++++++-------- rs/moq-net/src/model/track.rs | 39 +++++++++++------------- rs/moq-net/tests/integration/compress.rs | 20 ++++++------ 9 files changed, 70 insertions(+), 76 deletions(-) diff --git a/drafts/draft-lcurley-moq-lite.md b/drafts/draft-lcurley-moq-lite.md index 08890fd1b7e..304193a409b 100644 --- a/drafts/draft-lcurley-moq-lite.md +++ b/drafts/draft-lcurley-moq-lite.md @@ -1256,7 +1256,7 @@ A non-zero value is the absolute group sequence + 1. The index of the first frame to deliver within the `Group Start` group (see [Positions](#positions)). A value of 0 means from the start of that group (default), so the group is delivered whole. Frames before this index are not delivered, even when delivery begins at that exact group; a start resolved at a later group is always delivered from frame 0. -A subscriber that has received no group MUST send 0, since it cannot number the frames of a group it has not seen. +A subscriber MUST send 0 unless it already holds every frame below this index, from this publisher or another, since a Group is the unit of decodability (see [Positions](#positions)). **Frame End**: The last frame to deliver (inclusive) within the end group (see [Positions](#positions)). @@ -1775,6 +1775,7 @@ The `Message Length` describes the payload size on the wire. - Added TRACK_INFO `Refresh` and SUBSCRIBE_UPDATE `Wanted Group`, carrying ratio-based refresh demand through relays. - Added `Compression` to TRACK_INFO: none (0x0) or DEFLATE (0x1), whose Groups are DEFLATE streams per draft-lcurley-moq-flate. FRAME and datagrams carry an `Inflated Size` when it is not none. Earlier versions receive the Track inflated. +- A SUBSCRIBE carries a non-zero `Frame Start` only when the subscriber holds every frame below it; otherwise it asks for the whole Group. - Replaced ANNOUNCE_OK Active Count with an ordered, empty SYNCED event (0x4), marking the initial set without retaining routes that end before delivery. diff --git a/quest/m1/compress/README.md b/quest/m1/compress/README.md index b2d4c867931..aa84a144698 100644 --- a/quest/m1/compress/README.md +++ b/quest/m1/compress/README.md @@ -52,12 +52,15 @@ Every interview prompt and the selected answer (✅) are in the planning PR. parameter get the inflated copy, with no flag. A sealed compression declaration is preserved, so delivery is refused when plain fallback would remove it. -- **Ranges.** A range starting at frame N of a compressed group is served - from N. The requester only asks from N because it holds 0..N-1, so it - inflates from its own cached prefix. Reading frame N without 0..N-1 is a - non-goal: groups are delta encoded anyway. The requesting model enforces - it: a start on a compressed group it lacks the prefix for widens to frame - 0 before the request leaves, and a server serves exactly what is asked. +- **Ranges.** Compressed tracks are not special: every subscription, new or + resumed, starts at the beginning of a group, and one resuming from another + route may take the group's start from that route's cached prefix + (maintainer, on [moq-dev/moq#6465](https://github.com/moq-dev/moq/pull/6465)). + The requesting model enforces it on every protocol: a mid-group start it + lacks the prefix for widens to frame 0 before the request leaves, the + reader still starts at the frame it asked for, and a server serves exactly + what is asked. A compressed group therefore always inflates from a prefix + its requester holds. - **Specs:** draft-lcurley-moq-flate keeps the codec and becomes the IETF extension; draft-lcurley-moq-lite specifies the lite-07 field and size and references it for the codec. diff --git a/quest/m2/net-wrap/e2ee.md b/quest/m2/net-wrap/e2ee.md index 24fc09fb1ed..10a61612e54 100644 --- a/quest/m2/net-wrap/e2ee.md +++ b/quest/m2/net-wrap/e2ee.md @@ -58,9 +58,9 @@ the reverse (decided 2026-10-20, maintainer): without changing moq-net's `track(name)`; propose the encoding in the PR. Record it in `draft-lcurley-moq-e2ee.md`, replacing the Catalogs section's `.z` sibling rule. -- Ranges: moq-net widens a range that starts mid-group on a compressed group - to frame 0 when the requester lacks the prefix, but it sees only the - uncompressed physical track. The wrapper's subscriber applies that rule +- Ranges: moq-net widens a range that starts mid-group to frame 0 when the + requester lacks the prefix, but it sees only the uncompressed physical + track. The wrapper's subscriber applies that rule itself: it widens such a physical request to frame 0, inflates from there, and yields only the requested logical range. Test a resumed mid-group read, not only full round trips. diff --git a/rs/moq-net/benches/codec.rs b/rs/moq-net/benches/codec.rs index 651e4c8b636..f9fda728403 100644 --- a/rs/moq-net/benches/codec.rs +++ b/rs/moq-net/benches/codec.rs @@ -70,17 +70,15 @@ fn bench(c: &mut Criterion) { /// An outgoing request reads the counted aggregate and checks only its named group, /// independent of the audience beside it and the cache's unrelated groups. -fn compressed_request(c: &mut Criterion) { +fn request_start(c: &mut Criterion) { use moq_net::{broadcast, track}; use std::hint::black_box; - let mut bench = c.benchmark_group("codec_compressed_request"); + let mut bench = c.benchmark_group("codec_request_start"); for groups in [1, 64, 1024] { for readers in [1, 64, 1024] { for held in [false, true] { let broadcast = broadcast::Info::default().produce(); - let track = broadcast - .create_track("compressed", track::Info::default().with_compress(true)) - .unwrap(); + let track = broadcast.create_track("track", track::Info::default()).unwrap(); for _ in 0..groups { let mut group = track.append_group().unwrap(); group.write_frame(None, bytes::Bytes::from_static(b"prefix")).unwrap(); @@ -98,7 +96,7 @@ fn compressed_request(c: &mut Criterion) { bench.bench_function(BenchmarkId::new(format!("held_{held}_{readers}r"), groups), |b| { b.iter(|| { let mut subscription = track.subscription().unwrap(); - moq_net::fuzz::widen_compressed(&track, &mut subscription); + moq_net::fuzz::widen_start(&track, &mut subscription); black_box(subscription); }); }); @@ -108,5 +106,5 @@ fn compressed_request(c: &mut Criterion) { bench.finish(); } -criterion_group!(benches, bench, compressed_request); +criterion_group!(benches, bench, request_start); criterion_main!(benches); diff --git a/rs/moq-net/src/fuzz.rs b/rs/moq-net/src/fuzz.rs index 2c752971a7f..9de866ff38f 100644 --- a/rs/moq-net/src/fuzz.rs +++ b/rs/moq-net/src/fuzz.rs @@ -73,9 +73,9 @@ pub fn object_group(frames: usize, properties: bool) -> (broadcast::Producer, tr (broadcast, track, group) } -/// Normalize a compressed wire request for the private requesting-model benchmark. -pub fn widen_compressed(track: &track::Producer, subscription: &mut track::Subscription) { - track.widen_compressed(subscription); +/// Widen an outgoing request's mid-group starts for the private requesting-model benchmark. +pub fn widen_start(track: &track::Producer, subscription: &mut track::Subscription) { + track.widen_start(subscription); } /// Every target, keyed by the name of its `fuzz_targets/.rs` shim. diff --git a/rs/moq-net/src/ietf/subscriber.rs b/rs/moq-net/src/ietf/subscriber.rs index 3decb27ce95..da8de5fa586 100644 --- a/rs/moq-net/src/ietf/subscriber.rs +++ b/rs/moq-net/src/ietf/subscriber.rs @@ -487,14 +487,11 @@ impl Target { } /// Widen a mid-group start to the group's first frame unless the copy holds the frames - /// below it, so a compressed group can inflate. A request learns whether the track is - /// compressed only from SUBSCRIBE_OK, so with compression `negotiated` it widens every - /// mid-group start, and a plain track pays for that group's prefix. - fn widen(&self, subscription: &mut track::Subscription, negotiated: bool) { + /// below it. + fn widen_start(&self, subscription: &mut track::Subscription) { match self { - Self::Request(request) if negotiated => request.widen_compressed(subscription), - Self::Request(_) => {} - Self::Resume(idle) => idle.track.widen_compressed(subscription), + Self::Request(request) => request.widen_start(subscription), + Self::Resume(idle) => idle.track.widen_start(subscription), } } @@ -590,9 +587,10 @@ struct TrackState { // Where the subscription asked to start, when the request starts partway through a // group: a resumed subscription picking up after the frames a previous route - // delivered, or an application range. The stream for that group legitimately starts + // delivered, or an application range, either widened to its group's first frame unless + // the copy holds the frames below it. The stream for that group legitimately starts // there, without a head, and its Object IDs stay pinned to the ordinals the readers' - // windows count in. A compressed group's start is widened to its first frame. + // windows count in. resume: Option, // The data streams read so far, which PUBLISH_DONE's Stream Count is checked against. @@ -2828,9 +2826,8 @@ where ) -> Option { let validity = self.validity.start(); let requested = target.subscription(); - let negotiated = self.peer_setup.get().await.flate; let subscription = requested.clone().map(|mut subscription| { - target.widen(&mut subscription, negotiated); + target.widen_start(&mut subscription); subscription }); let start = subscription.as_ref().and_then(|s| s.start()); diff --git a/rs/moq-net/src/lite/subscriber.rs b/rs/moq-net/src/lite/subscriber.rs index e33b87164ba..8b2e29d719f 100644 --- a/rs/moq-net/src/lite/subscriber.rs +++ b/rs/moq-net/src/lite/subscriber.rs @@ -4308,14 +4308,20 @@ mod tests { assert_eq!((msg.start_frame, msg.end_frame), (0, None)); } - /// A lite-07 SUBSCRIBE keeps a mid-group start: its answer carries the largest position, - /// which says whether what the subscriber holds is current. Lite-06's answer does not, - /// so it asks from the head of the group instead, and the first frame says where the - /// live feed is; its end keeps the frame bound. + /// A lite-07 SUBSCRIBE keeps a mid-group start whose prefix the copy holds: its answer + /// carries the largest position, which says whether what the subscriber holds is + /// current. Lite-06's answer does not, so it asks from the head of the group instead, + /// and the first frame says where the live feed is; its end keeps the frame bound. #[moq_clock::test] async fn a_mid_group_start_survives_only_where_the_answer_has_the_largest() { for (version, start_frame) in [(Version::Lite07, 3), (Version::Lite06, 0)] { let mut h = Harness::new(version); + let mut group = h.producer.create_group(crate::group::Info { sequence: 5 }).unwrap(); + for frame in 0..3 { + group + .write_frame(crate::Timestamp::from_millis(frame).unwrap(), b"prefix".as_ref()) + .unwrap(); + } let mut sub = Sub::None; h.serve @@ -4334,10 +4340,10 @@ mod tests { } } - /// A partial prefix, a tail, or a released group cannot seed the inflater. Only - /// a complete live prefix permits requesting the tail without widening its start. + /// A partial prefix, a tail, or a released group lacks frames below a mid-group start. + /// Only a complete live prefix permits requesting the tail without widening its start. #[moq_clock::test] - async fn compressed_frame_bounds_require_the_entire_cached_prefix() { + async fn frame_bounds_require_the_entire_cached_prefix() { for (first, count, aborted, expected) in [ (0, 0, false, 0), (0, 2, false, 0), @@ -4346,10 +4352,6 @@ mod tests { (0, 3, true, 0), ] { let mut h = Harness::new(Version::Lite07); - h.producer = h - ._broadcast - .create_track("compressed", track::Info::timed().with_compress(true)) - .unwrap(); let mut group = h.producer.create_group(crate::group::Info { sequence: 5 }).unwrap(); group.start_at(first).unwrap(); for frame in first..first + count { @@ -6291,7 +6293,7 @@ impl TrackServe { // The wire carries one range: the span of every range asked for, which the // subscribers' own reads narrow again. subscription.set_bounds(subscription.start(), subscription.end()); - producer.widen_compressed(&mut subscription); + producer.widen_start(&mut subscription); self.widen_frame_bounds(&mut subscription); match sub { Sub::None => { diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 21f039b4612..6c835ffb135 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -813,12 +813,12 @@ impl Holdings for TrackState { impl TrackState { /// Start each mid-group range at its group's first frame unless a copy holds the - /// frames below it; see [`Producer::widen_compressed`]. + /// frames below it; see [`Producer::widen_start`]. #[cfg_attr( not(feature = "async"), allow(dead_code, reason = "Only session adapters request upstream.") )] - fn widen_compressed(&self, subscription: &mut Subscription) { + fn widen_start(&self, subscription: &mut Subscription) { for range in &mut subscription.ranges { if range.is_empty() { continue; @@ -3383,9 +3383,12 @@ impl Producer { poll_combined_changed(&subs, bound, &mut self.prev_subscription, waiter).map(Ok) } - /// Widen outgoing compressed ranges that cannot inflate from a cached prefix. - /// Only requesters call this: the original preferences keep filtering local reads, - /// and a server serves the frame bounds it was actually asked for. + /// Start outgoing ranges at their group's first frame, unless a cached copy holds the + /// frames below a mid-group start, such as ones a previous route delivered. A group is + /// often useless without its head (a keyframe, or a compressed group's window), so a + /// request skips only frames a copy already holds. Only requesters call this: the + /// original preferences keep filtering local reads, and a server serves the frame + /// bounds it was actually asked for. /// /// A range kept mid-group relies on the cached prefix, so it counts as a read of that /// copy and normal eviction takes it last. Nothing pins it: a prefix still lost @@ -3396,10 +3399,8 @@ impl Producer { allow(dead_code, reason = "Only session adapters request upstream.") )] // private: capability gap - pub(crate) fn widen_compressed(&self, subscription: &mut Subscription) { - if self.info.compress { - self.state.read().widen_compressed(subscription); - } + pub(crate) fn widen_start(&self, subscription: &mut Subscription) { + self.state.read().widen_start(subscription); } /// Create a [`Dynamic`] handle that serves on-demand fetches of uncached @@ -7570,15 +7571,14 @@ impl Request { snapshot_subscription(&subs, bound) } - /// As [`Producer::widen_compressed`], for a track whose info is not known yet: the - /// requester calls it when the answer may declare the track compressed. + /// As [`Producer::widen_start`], for a track not accepted yet. #[cfg_attr( not(feature = "async"), allow(dead_code, reason = "Only session adapters request upstream.") )] // private: capability gap - pub(crate) fn widen_compressed(&self, subscription: &mut Subscription) { - self.state.read().widen_compressed(subscription); + pub(crate) fn widen_start(&self, subscription: &mut Subscription) { + self.state.read().widen_start(subscription); } /// Block until the aggregate [`subscription`](Self::subscription) changes, @@ -14922,17 +14922,12 @@ mod test { assert!(consumer.peek_group(1).is_none(), "unread group is evicted instead"); } - /// Requesting a compressed tail from frame N reads the cached prefix it relies on, as a - /// FETCH hit does, so pressure evicts a never-read group first and the prefix stays to - /// seed the tail's inflater. + /// Requesting a tail from frame N reads the cached prefix it relies on, as a FETCH hit + /// does, so pressure evicts a never-read group first and the prefix stays to complete + /// the group. #[test] fn a_requested_prefix_survives_eviction() { let (producer, pool) = pooled_producer(u64::MAX); - let producer = Producer::new( - producer.broadcast.clone(), - "compressed", - Info::timed().with_compress(true), - ); let consumer = producer.consume(); // Incompressible, so each group caches about its payload. let mut seed = 1u32; @@ -14960,7 +14955,7 @@ mod test { let start = Anchor::Absolute(Position { group: 0, frame: 1 }); let mut request = Subscription::default().with_ranges([Range { start, end: None }]); - producer.widen_compressed(&mut request); + producer.widen_start(&mut request); assert_eq!(request.ranges[0].start, start, "a held prefix keeps the request at N"); elapse(&producer, Duration::from_millis(500)); diff --git a/rs/moq-net/tests/integration/compress.rs b/rs/moq-net/tests/integration/compress.rs index ee9d419b17f..074d401babc 100644 --- a/rs/moq-net/tests/integration/compress.rs +++ b/rs/moq-net/tests/integration/compress.rs @@ -119,8 +119,8 @@ async fn relay(published: Published, hops: [&str; 2], subscription: moq_net::tra } } -/// Hops whose requester carries an application's mid-group start: lite-07, and -/// moq-transport with compression negotiated. +/// Hops that carry a mid-group start and negotiate compression: lite-07, and moq-transport +/// draft-17 and later. const RANGED: &[&str] = &[ "moq-lite-07-wip", "moq-transport-17", @@ -128,11 +128,12 @@ const RANGED: &[&str] = &[ "moq-transport-22", ]; -/// A cold subscriber needs a compressed group's prefix, but its application sees only its -/// requested half-open frame range across every pair of hops, whether the group began before -/// it subscribed or after, and on a plain track too. Lite-07 hops also carry a live change. +/// A cold subscriber asks from the group's first frame, compressed or not, but its +/// application sees only its requested half-open frame range across every pair of hops, +/// whether the group began before it subscribed or after. Lite-07 hops also carry a live +/// change. #[moq_clock::test] -async fn application_ranges_keep_their_filter_after_requesting_the_compressed_prefix() { +async fn application_ranges_keep_their_filter_after_requesting_the_group_head() { use moq_net::track::{Anchor, Position, Range, Subscription}; let range = |group, start, end: Option| { Subscription::default().with_ranges([Range { @@ -176,11 +177,8 @@ async fn application_ranges_keep_their_filter_after_requesting_the_compressed_pr if lite { assert_eq!( fixture.track.subscription_changed().await.unwrap().unwrap().ranges[0].start, - Anchor::Absolute(Position { - group: 5, - frame: if compress { 0 } else { 2 } - }), - "{case}: upstream asks for the prefix a compressed group needs" + Anchor::Absolute(Position::group(5)), + "{case}: upstream asks from the group's first frame" ); } write(&mut group, head..5); From a0b760d55d3e05ef399d4ab5f40216ca32a7bd77 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 23 Oct 2026 14:23:19 -0700 Subject: [PATCH 04/10] fix(net)!: every range and fetch starts at a group's first frame Drop frame-level application ranges: track::Anchor::Absolute names a group, group::Fetch::frame_start is crate-private (only route-change hole fills set it), and the bindings' absolute anchors lose their frame. Publishers still honor a peer's wire frame bounds, cutting frames per group. Co-Authored-By: Claude Opus 5.5 --- cpp/moq/MIGRATE.md | 4 + dart/moq/MIGRATE.md | 4 + dart/moq_ffi/lib/src/moq.dart | 20 +- doc/concept/moq-lite.md | 2 +- drafts/draft-lcurley-moq-lite.md | 8 +- go/wrapper/MIGRATE.md | 4 + go/wrapper/types.go | 2 +- kt/moq/MIGRATE.md | 4 + .../kotlin/dev/moq/Aliases.kt | 2 +- py/moq-rs/MIGRATE.md | 4 + py/moq-rs/tests/test_local.py | 2 +- py/moq-rs/tests/test_testing_clock.py | 2 +- quest/m0/broadcast-epoch/README.md | 9 +- quest/m1/compress/README.md | 9 +- quest/m1/release-audit/README.md | 2 +- quest/m1/release-audit/js-partial-group.md | 3 - quest/m1/resolved-demand.md | 3 +- quest/m1/subscribe-ranges/README.md | 8 +- quest/m1/subscribe-ranges/bounded-tail.md | 42 -- quest/m1/subscribe-ranges/dynamic.md | 6 +- quest/m1/subscribe-ranges/js.md | 9 +- quest/m2/net-wrap/e2ee.md | 9 +- rs/moq-archive/src/producer/track.rs | 2 +- rs/moq-archive/src/reader/tests.rs | 145 ++----- rs/moq-c/MIGRATE.md | 4 + rs/moq-c/src/api.rs | 13 +- rs/moq-c/src/test.rs | 53 +-- rs/moq-ffi/MIGRATE.md | 4 + rs/moq-ffi/src/consumer.rs | 6 +- rs/moq-ffi/src/test.rs | 4 +- rs/moq-hls/src/export/rendition.rs | 11 +- rs/moq-net/MIGRATE.md | 6 + rs/moq-net/benches/codec.rs | 40 +- rs/moq-net/benches/track.rs | 2 +- rs/moq-net/src/coding/reader.rs | 35 -- rs/moq-net/src/fuzz.rs | 5 - rs/moq-net/src/ietf/publisher.rs | 13 +- rs/moq-net/src/ietf/subscriber.rs | 359 ++++-------------- rs/moq-net/src/lite/ietf.rs | 71 +--- rs/moq-net/src/lite/publisher.rs | 132 ++++--- rs/moq-net/src/lite/subscriber.rs | 336 ++-------------- rs/moq-net/src/model/group.rs | 39 +- rs/moq-net/src/model/origin.rs | 4 +- rs/moq-net/src/model/resume.rs | 53 ++- rs/moq-net/src/model/subscription.rs | 134 +++---- rs/moq-net/src/model/subscription/window.rs | 194 ++-------- rs/moq-net/src/model/track.rs | 310 ++++----------- rs/moq-net/src/model/track/demand.rs | 7 +- rs/moq-net/src/model/track/demand/ranges.rs | 41 +- rs/moq-net/tests/integration/compress.rs | 58 +-- rs/moq-net/tests/integration/head_fetch.rs | 171 --------- .../tests/integration/late_lower_group.rs | 12 +- rs/moq-net/tests/integration/main.rs | 1 - rs/moq-transcode/src/lib.rs | 64 ---- swift/MIGRATE.md | 4 + swift/Sources/Moq/Aliases.swift | 2 +- 56 files changed, 562 insertions(+), 1931 deletions(-) delete mode 100644 quest/m1/subscribe-ranges/bounded-tail.md delete mode 100644 rs/moq-net/tests/integration/head_fetch.rs diff --git a/cpp/moq/MIGRATE.md b/cpp/moq/MIGRATE.md index ed65c7f0dd6..6838c3b9da8 100644 --- a/cpp/moq/MIGRATE.md +++ b/cpp/moq/MIGRATE.md @@ -4,6 +4,10 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Absolute anchors name a group ([#6465](https://github.com/moq-dev/moq/pull/6465)) + +`MoqAnchor::Absolute` drops `frame`: a range always starts at a group's first frame. Set only `group`. + ### `announce` takes an `AnnounceRoute` `BroadcastProducer::announce` takes `moq::AnnounceRoute{cost, epoch}` instead of a `moq::Route`: the origin builds the hop chain, so `hops` is no longer an input. diff --git a/dart/moq/MIGRATE.md b/dart/moq/MIGRATE.md index 219980e3eb7..eaa0cfa5a71 100644 --- a/dart/moq/MIGRATE.md +++ b/dart/moq/MIGRATE.md @@ -4,6 +4,10 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Absolute anchors name a group ([#6465](https://github.com/moq-dev/moq/pull/6465)) + +`AbsoluteMoqAnchor` drops `frame`: a range always starts at a group's first frame. Write `AbsoluteMoqAnchor(n)`. + ### `announce` takes an `AnnounceRoute` `BroadcastProducer.announce(route:)` takes `AnnounceRoute(cost:, epoch:)` instead of a `MoqRoute`: the origin builds the hop chain, so `hops` is no longer an input. diff --git a/dart/moq_ffi/lib/src/moq.dart b/dart/moq_ffi/lib/src/moq.dart index 7e60959cf9d..9c2edf413b7 100644 --- a/dart/moq_ffi/lib/src/moq.dart +++ b/dart/moq_ffi/lib/src/moq.dart @@ -4469,9 +4469,8 @@ class FfiConverterMoqAnchor { class AbsoluteMoqAnchor extends MoqAnchor { final int group; - final int frame; - AbsoluteMoqAnchor({required int this.group, required int this.frame}); - AbsoluteMoqAnchor._(int this.group, int this.frame); + AbsoluteMoqAnchor(int this.group); + AbsoluteMoqAnchor._(int this.group); static LiftRetVal read(Uint8List buf) { int new_offset = buf.offsetInBytes; final group_lifted = FfiConverterUInt64.read( @@ -4479,12 +4478,7 @@ class AbsoluteMoqAnchor extends MoqAnchor { ); final group = group_lifted.value; new_offset += group_lifted.bytesRead; - final frame_lifted = FfiConverterUInt64.read( - Uint8List.view(buf.buffer, new_offset), - ); - final frame = frame_lifted.value; - new_offset += frame_lifted.bytesRead; - return LiftRetVal(AbsoluteMoqAnchor._(group, frame), new_offset); + return LiftRetVal(AbsoluteMoqAnchor._(group), new_offset); } @override @@ -4496,9 +4490,7 @@ class AbsoluteMoqAnchor extends MoqAnchor { @override int allocationSize() { - return FfiConverterUInt64.allocationSize(group) + - FfiConverterUInt64.allocationSize(frame) + - 4; + return FfiConverterUInt64.allocationSize(group) + 4; } @override @@ -4509,10 +4501,6 @@ class AbsoluteMoqAnchor extends MoqAnchor { group, Uint8List.view(buf.buffer, new_offset), ); - new_offset += FfiConverterUInt64.write( - frame, - Uint8List.view(buf.buffer, new_offset), - ); return new_offset; } } diff --git a/doc/concept/moq-lite.md b/doc/concept/moq-lite.md index f278dcc22b7..60e3e8181e8 100644 --- a/doc/concept/moq-lite.md +++ b/doc/concept/moq-lite.md @@ -243,7 +243,7 @@ Each subscription carries the knobs that decide behavior under congestion: | **Priority** (-128..127, default 0) | Higher-priority tracks get bandwidth first. Audio above video, base layer above enhancement. The publisher's track priority breaks ties between subscriptions at one priority. From a native Rust publisher over QUIC or WebTransport (iroh excepted), subscriptions at the same priority and publisher priority split bandwidth evenly by bytes, streams and datagrams alike, however many groups each has queued. A browser publisher and other transports without send groups are not covered. | | **Order** | Which group to send first when several are pending. Newest first for live (the default), oldest first for catch-up. Needs moq-lite 07 or moq-transport; older moq-lite always sends newest first. | | **Max delay** | How long a non-latest group stays worth delivering before it is skipped. Zero means "live edge only", and raising it is also what asks for history. | -| **Ranges** | Which groups to ask for: from a group, a number of groups back from the latest, or the group containing a presentation time, optionally up to an end. The default asks for everything max delay allows. Max delay still caps every range. | +| **Ranges** | Which groups to ask for: from a group, a number of groups back from the latest, or the group containing a presentation time, optionally up to an end. Each group arrives whole, from its first frame. The default asks for everything max delay allows. Max delay still caps every range. | On moq-lite 05 and later, a publisher explicitly reports groups that will not arrive. This lets a finished subscription settle without waiting for missing groups. diff --git a/drafts/draft-lcurley-moq-lite.md b/drafts/draft-lcurley-moq-lite.md index 937e08e709b..e5ccf59c2ae 100644 --- a/drafts/draft-lcurley-moq-lite.md +++ b/drafts/draft-lcurley-moq-lite.md @@ -211,15 +211,15 @@ The resolved start is therefore always either exactly the requested Position or The asymmetry with Groups is deliberate. A Group is the unit of decodability: dropping leading *Groups* leaves a stream the application can still decode, whereas dropping leading *Frames* often does not, whether because the Group opens with a keyframe the rest depends on or because its compression state lives in the frames that went missing. -Only the subscriber knows whether a partial Group is any use to it, so only the subscriber may ask for one. +A subscription therefore starts at a Group's first frame, and a subscriber asks for a partial Group only to finish one whose earlier frames it already holds, from this publisher or another. Frame indices are only meaningful relative to a Group the subscriber has already begun receiving, so: - A `Frame Start` qualifies the group `Group Start` names, group 0 included. - A subscriber that has seen no such group MUST send `Frame Start` = 0, since it cannot number the frames of a group it has not received. + A subscriber MUST send `Frame Start` = 0 unless it already holds every frame of that group below it. - A `Frame End` is meaningless without a `Group End`; an unbounded subscription (`Group End` = 0) MUST send `Frame End` = 0. -A publisher MUST treat a violation of either rule as a protocol violation and reset the stream. +A publisher MUST treat a non-zero `Frame End` without a `Group End` as a protocol violation and reset the stream. # Flow This section outlines the flow of messages within a moq-lite session. @@ -1797,7 +1797,7 @@ The `Message Length` describes the payload size on the wire. - Added TRACK_INFO `Refresh` and SUBSCRIBE_UPDATE `Wanted Group`, carrying ratio-based refresh demand through relays. - Added `Compression` to TRACK_INFO: none (0x0) or DEFLATE (0x1), whose Groups are DEFLATE streams per draft-lcurley-moq-flate. FRAME and datagrams carry an `Inflated Size` when it is not none. Earlier versions receive the Track inflated. -- A SUBSCRIBE carries a non-zero `Frame Start` only when the subscriber holds every frame below it; otherwise it asks for the whole Group. +- A subscription starts at a Group's first frame: SUBSCRIBE carries a non-zero `Frame Start` only when the subscriber holds every frame below it, compressed Track or not. - Replaced ANNOUNCE_OK Active Count with an ordered, empty SYNCED event (0x4), marking the initial set without retaining routes that end before delivery. diff --git a/go/wrapper/MIGRATE.md b/go/wrapper/MIGRATE.md index 9db258a3ce0..4db162e904a 100644 --- a/go/wrapper/MIGRATE.md +++ b/go/wrapper/MIGRATE.md @@ -4,6 +4,10 @@ How to update your code for each breaking change, newest first. ## Unreleased +### `AnchorAbsolute` names a group ([#6465](https://github.com/moq-dev/moq/pull/6465)) + +`AnchorAbsolute` drops `Frame`: a range always starts at a group's first frame. Write `AnchorAbsolute{Group: n}`. + ### `Announce` takes an `AnnounceRoute` `BroadcastProducer.Announce` takes `moq.AnnounceRoute{Cost, Epoch}` instead of a `moq.Route`: the origin builds the hop chain, so `Hops` is no longer an input, and `ErrInvalidRoute` is gone. diff --git a/go/wrapper/types.go b/go/wrapper/types.go index 1e1f32e68af..45d5c1aff7c 100644 --- a/go/wrapper/types.go +++ b/go/wrapper/types.go @@ -62,7 +62,7 @@ type ( Range = ffi.MoqRange // Anchor is where one end of a Range falls: AnchorAbsolute, AnchorRelative, or AnchorTime. Anchor = ffi.MoqAnchor - // AnchorAbsolute is an exact group and frame. + // AnchorAbsolute is a group, delivered from its first frame. AnchorAbsolute = ffi.MoqAnchorAbsolute // AnchorRelative is the group Groups before the latest one, clamped to group 0: 0 is the latest. AnchorRelative = ffi.MoqAnchorRelative diff --git a/kt/moq/MIGRATE.md b/kt/moq/MIGRATE.md index 717cf625e20..412803f3863 100644 --- a/kt/moq/MIGRATE.md +++ b/kt/moq/MIGRATE.md @@ -4,6 +4,10 @@ How to update your code for each breaking change, newest first. ## Unreleased +### `AnchorAbsolute` names a group ([#6465](https://github.com/moq-dev/moq/pull/6465)) + +`AnchorAbsolute` drops `frame`: a range always starts at a group's first frame. Write `AnchorAbsolute(group)`. + ### `announce` takes an `AnnounceRoute` `BroadcastProducer.announce` takes `AnnounceRoute(cost, epoch)` instead of a `Route`: the origin builds the hop chain, so `hops` is no longer an input. diff --git a/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt b/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt index 84da35126c6..eb0d96c06c1 100644 --- a/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt +++ b/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt @@ -109,7 +109,7 @@ typealias Subscription = uniffi.moq.MoqSubscription typealias Range = uniffi.moq.MoqRange /** Where one end of a [Range] falls: [AnchorAbsolute], [AnchorRelative], or [AnchorTime]. */ typealias Anchor = uniffi.moq.MoqAnchor -/** An exact group and frame. */ +/** A group, delivered from its first frame. */ typealias AnchorAbsolute = uniffi.moq.MoqAnchor.Absolute /** The group `groups` before the latest one, clamped to group 0: 0 is the latest. */ typealias AnchorRelative = uniffi.moq.MoqAnchor.Relative diff --git a/py/moq-rs/MIGRATE.md b/py/moq-rs/MIGRATE.md index 3c22a3b0cc9..d1d698b976e 100644 --- a/py/moq-rs/MIGRATE.md +++ b/py/moq-rs/MIGRATE.md @@ -4,6 +4,10 @@ How to update your code for each breaking change, newest first. ## Unreleased +### `Anchor.ABSOLUTE` names a group ([#6465](https://github.com/moq-dev/moq/pull/6465)) + +`Anchor.ABSOLUTE` drops `frame`: a range always starts at a group's first frame. Write `Anchor.ABSOLUTE(group=n)`. + ### `announce` takes an `AnnounceRoute` `BroadcastProducer.announce` takes `moq.AnnounceRoute(cost=..., epoch=...)` instead of a `Route`: the origin builds the hop chain, so `hops` is no longer an input. diff --git a/py/moq-rs/tests/test_local.py b/py/moq-rs/tests/test_local.py index 022e163df76..49c5633bc8f 100644 --- a/py/moq-rs/tests/test_local.py +++ b/py/moq-rs/tests/test_local.py @@ -11,7 +11,7 @@ import pytest # Every group from the first, rather than only what the live edge still reaches. -FROM_GROUP_0 = [moq.Range(start=cast(moq.Anchor, moq.Anchor.ABSOLUTE(group=0, frame=0)))] +FROM_GROUP_0 = [moq.Range(start=cast(moq.Anchor, moq.Anchor.ABSOLUTE(group=0)))] async def until_wanted(demand: moq.TrackDemand, wanted: bool = True) -> None: diff --git a/py/moq-rs/tests/test_testing_clock.py b/py/moq-rs/tests/test_testing_clock.py index 8dd4a345221..e5898428c9e 100644 --- a/py/moq-rs/tests/test_testing_clock.py +++ b/py/moq-rs/tests/test_testing_clock.py @@ -26,7 +26,7 @@ async def main(): # The origin's driver runs the timers that judge a group stale. origin = moq.OriginProducer() track = origin.create_broadcast("clock").publish_track("track") - from_group_0 = [moq.Range(start=moq.Anchor.ABSOLUTE(group=0, frame=0))] + from_group_0 = [moq.Range(start=moq.Anchor.ABSOLUTE(group=0))] consumer = track.consume(moq.Subscription(max_delay=timedelta(seconds=1), ranges=from_group_0)) # Group 0 stalls with no frame, so the in-order read waits on it until diff --git a/quest/m0/broadcast-epoch/README.md b/quest/m0/broadcast-epoch/README.md index b185c3ed52d..b990aed5278 100644 --- a/quest/m0/broadcast-epoch/README.md +++ b/quest/m0/broadcast-epoch/README.md @@ -86,11 +86,10 @@ Decided: lite-06). A release that needs seamless failover promotes lite-07 first. Decided 2026-10-06: older versions and moq-transport lose cross-route resume, -since their routes have no epoch, but their mid-group start handling stays. -The IETF joining FETCH is the normal live join for every IETF subscription, and -the IETF resume point and lite-05/06 `widen_frame_bounds` still serve any -mid-group start: a public `Subscription::with_start` or a downstream Frame -Start a relay forwards upstream. +since their routes have no epoch. +The IETF joining FETCH is the normal live join for every IETF subscription. +Every subscription starts at a group's first frame ([moq-dev/moq#6465](https://github.com/moq-dev/moq/pull/6465)), +so no version needs a mid-group start. This README owns an end-to-end relay test: republish a name under a new epoch while the old publisher's session stays open. A lite-07 viewer and a lite-06 or IETF viewer diff --git a/quest/m1/compress/README.md b/quest/m1/compress/README.md index 03721457584..b3b660e94ed 100644 --- a/quest/m1/compress/README.md +++ b/quest/m1/compress/README.md @@ -55,12 +55,9 @@ Every interview prompt and the selected answer (✅) are in the planning PR. - **Ranges.** Compressed tracks are not special: every subscription, new or resumed, starts at the beginning of a group, and one resuming from another route may take the group's start from that route's cached prefix - (maintainer, on [moq-dev/moq#6465](https://github.com/moq-dev/moq/pull/6465)). - The requesting model enforces it on every protocol: a mid-group start it - lacks the prefix for widens to frame 0 before the request leaves, the - reader still starts at the frame it asked for, and a server serves exactly - what is asked. A compressed group therefore always inflates from a prefix - its requester holds. + (maintainer, on [moq-dev/moq#6465](https://github.com/moq-dev/moq/pull/6465)). Application + ranges and fetches name whole groups, so a compressed group always + inflates from a prefix its requester holds. - **Specs:** draft-lcurley-moq-flate keeps the codec and becomes the IETF extension; draft-lcurley-moq-lite specifies the lite-07 field and size and references it for the codec. diff --git a/quest/m1/release-audit/README.md b/quest/m1/release-audit/README.md index 42a06c9fc67..ac476b63206 100644 --- a/quest/m1/release-audit/README.md +++ b/quest/m1/release-audit/README.md @@ -109,7 +109,7 @@ child records whether its behavior is new on main or inherited from release. - [JS subscription ranges](/quest/m1/subscribe-ranges/js.md) - range lists and frame offsets build on the bounds fixed here - [OBS video source](/quest/m2/obs-moq-video/source.md) - reproduced inherited timing and EOF defects remain with the decoder replacement owner - [QPACK dependency fix](/quest/m0/web-transport-proto-0.6.3.md) - decoded header expansion is already tracked -- [Compressed tracks](/quest/m1/compress/README.md) - mid-group widening and application migration have existing owners +- [Compressed tracks](/quest/m1/compress/README.md) - application migration has an existing owner - [Archive consumer](/quest/m1/archive/consumer.md) - current producer and CLI reader still need the new-format roundtrip - [Measured release performance](/quest/m1/release-perf/README.md) - existing paired benchmark regressions and their confirmation runs - [Transport performance audit](/quest/m2/audit-perf.md) - congestion-control coverage remains outstanding diff --git a/quest/m1/release-audit/js-partial-group.md b/quest/m1/release-audit/js-partial-group.md index e666ce3b56c..278ff70c284 100644 --- a/quest/m1/release-audit/js-partial-group.md +++ b/quest/m1/release-audit/js-partial-group.md @@ -29,7 +29,4 @@ way and is swallowed too; fix both, letting the violation escape `runGroup` while ordinary group errors stay stream-scoped. Test that each closes the session. -[JS ranges](/quest/m1/subscribe-ranges/js.md) carries the offset through the -group model when it adds frame bounds. - Public API: none. Wire: none. diff --git a/quest/m1/resolved-demand.md b/quest/m1/resolved-demand.md index 0ab1b24a644..a98b2006b89 100644 --- a/quest/m1/resolved-demand.md +++ b/quest/m1/resolved-demand.md @@ -71,8 +71,7 @@ with no wire change. because watching for it needs more than this plan carries: a verdict depends on two groups (`drifted` uses G's successor for `reach` and a possibly later presentation edge in `live_above`, and losing either can - free G), a range may end mid-group so an edge needs a frame position, and - a run skipped at the max age bound would no longer be final. Likewise, + free G), and a run skipped at the max age bound would no longer be final. Likewise, a stale drop in flight across an aggregate range change (lite-07, see [Drop set](/quest/m1/drop-set.md)) may settle a reader whose own range would not convict, since SUBSCRIBE_UPDATE has no acknowledgement and the diff --git a/quest/m1/subscribe-ranges/README.md b/quest/m1/subscribe-ranges/README.md index 77d4b797449..90dcd1f3210 100644 --- a/quest/m1/subscribe-ranges/README.md +++ b/quest/m1/subscribe-ranges/README.md @@ -83,9 +83,10 @@ relay bounds a sparse FETCH only if the model can request ranges: its group starts, so a time end settles across a sequence that arrived only as a datagram, and a stream copy of that group arriving later is read even if it starts past the end. -- Ranges are frame-precise (`Position`), not whole groups. The IETF joining - FETCH for a mid-group SUBSCRIBE's uncached prefix stays, because today's - bridge relies on it (maintainer, 09-29). +- Ranges are whole groups: a range always starts at a group's first frame + (maintainer, on [moq-dev/moq#6465](https://github.com/moq-dev/moq/pull/6465)). The IETF joining FETCH for the + live group's uncached prefix stays, because today's bridge relies on it + (maintainer, 09-29). - The relay half of fetch-span moves here; #4558 landed only the no-handler skip. @@ -100,7 +101,6 @@ every sequence delivered or dropped. - [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - the lite-07 wire, Rust publisher and subscriber, and the draft - [moq-transport ranges](/quest/m1/subscribe-ranges/ietf.md) - range FETCH upstream per missing run, and non-blocking FETCH served downstream - [JS ranges](/quest/m1/subscribe-ranges/js.md) - `@moq/net` model and lite-07 parity -- [Bounded tails](/quest/m1/subscribe-ranges/bounded-tail.md) - a copy cut at a request's mid-group end accepts the rest of its group instead of rejecting it as `Duplicate` ## Related diff --git a/quest/m1/subscribe-ranges/bounded-tail.md b/quest/m1/subscribe-ranges/bounded-tail.md deleted file mode 100644 index edd6a9065f1..00000000000 --- a/quest/m1/subscribe-ranges/bounded-tail.md +++ /dev/null @@ -1,42 +0,0 @@ -# [M] A copy that stopped at a request's end bound accepts its tail - -## Goal - -A group received for a request whose end falls mid-group stays open to the -rest of that group. When later demand asks the same sequence past that end, -the tail is accepted instead of rejected as `Duplicate`, so a second -subscriber on the same relay reads the frames it asked for. This holds for -plain and compressed tracks on every version that carries frame bounds. - -## Plan - -Found on [moq-dev/moq#6242](https://github.com/moq-dev/moq/pull/6242) while -answering Codex's -[P1](https://github.com/moq-dev/moq/pull/6242#discussion_r4345457940): through -one relay, a subscriber reads group 5 frames 0..3 and a second then asks for -3..5. The tail never arrives, on a plain track and on a compressed one alike. - -- `GroupRecv` (`rs/moq-net/src/lite/subscriber.rs`) calls `group.finish()` - when a group stream ends cleanly, including one that ended only because the - request's end bound was reached. The truncated copy then looks complete. -- `claim_sequence` and `covering_group` (`rs/moq-net/src/model/track.rs`) - judge a copy only by its first frame, so every later stream for that - sequence is `Duplicate`, whatever frame it was asked from. -- Fix it in the receive path: a stream that stopped at the request's end - bound is not the group's end. Weigh leaving the copy unfinished so a tail - appends from its end (one copy; a compressed group keeps inflating against - its own prefix, which `widen_compressed` relies on when it keeps a request at - N) against letting a bounded copy stop covering a request past its end so - the new stream replaces it (compressed requests would then widen to frame 0 - again). -- A peer must not be able to pass off an early FIN as reaching the bound. - -Test on mock time through a relay: a bounded first subscriber, then a second -asking past its end, on plain and compressed lite-07 tracks, and on -moq-transport draft-20+ if its Absolute end reaches the same path. - -Public API: none expected. Wire: none. - -## Related - -- [Compressed tracks](/quest/m1/compress/README.md) - its Ranges rule keeps a warm request at N, which this makes reachable diff --git a/quest/m1/subscribe-ranges/dynamic.md b/quest/m1/subscribe-ranges/dynamic.md index dd7dc86dab6..508707e261b 100644 --- a/quest/m1/subscribe-ranges/dynamic.md +++ b/quest/m1/subscribe-ranges/dynamic.md @@ -38,11 +38,9 @@ Decided by the maintainer on 2026-10-20, when the quest was first started - **Split.** This quest replaces `requested_group`; removing `fetch_group` is [fetch as a range](/quest/m1/subscribe-ranges/fetch-range.md), so each - PR breaks one API once. That half needs head fills for a range starting - mid-group, which build on frame bounds - ([moq-dev/moq#6129](https://github.com/moq-dev/moq/pull/6129)). + PR breaks one API once. - **Handler shape.** `Dynamic::requested_fill` yields a `track::Fill` naming - one bounded run: `start: Position`, exclusive end group, priority, the most + one bounded run: start group, exclusive end group, priority, the most tolerant budget among its readers, and order. Each reader's cursor still skips what is stale under its own `max_delay`, so sharing never relaxes it. The handler calls `accept(sequence, info)` for each group it has and diff --git a/quest/m1/subscribe-ranges/js.md b/quest/m1/subscribe-ranges/js.md index cc6f369af33..041f5c7dbe2 100644 --- a/quest/m1/subscribe-ranges/js.md +++ b/quest/m1/subscribe-ranges/js.md @@ -28,10 +28,9 @@ IETF FETCH dispatch. Resized from [M] to [L] for it. was never cached, covering exact replay, empty and missing groups, concurrent requests, and cancellation while awaiting a reply. -When ranges add frame bounds, the group model carries a lite group's -`frameStart` instead of refusing a nonzero one, as -[partial lite groups](/quest/m1/release-audit/js-partial-group.md) does until -then. Preserve the per-reader group-bound rule for reliable reads, live-edge +Ranges stay whole groups ([moq-dev/moq#6465](https://github.com/moq-dev/moq/pull/6465)), so the group model keeps +refusing a nonzero lite `frameStart`, as +[partial lite groups](/quest/m1/release-audit/js-partial-group.md) does. Preserve the per-reader group-bound rule for reliable reads, live-edge selection, subscription updates, and publisher datagrams. ## Required @@ -40,4 +39,4 @@ selection, subscription updates, and publisher datagrams. ## Related -- [Partial lite groups](/quest/m1/release-audit/js-partial-group.md) - refuses a nonzero `frameStart` until this models it +- [Partial lite groups](/quest/m1/release-audit/js-partial-group.md) - refuses a nonzero `frameStart` diff --git a/quest/m2/net-wrap/e2ee.md b/quest/m2/net-wrap/e2ee.md index 10a61612e54..65e77bf7af0 100644 --- a/quest/m2/net-wrap/e2ee.md +++ b/quest/m2/net-wrap/e2ee.md @@ -58,12 +58,9 @@ the reverse (decided 2026-10-20, maintainer): without changing moq-net's `track(name)`; propose the encoding in the PR. Record it in `draft-lcurley-moq-e2ee.md`, replacing the Catalogs section's `.z` sibling rule. -- Ranges: moq-net widens a range that starts mid-group to frame 0 when the - requester lacks the prefix, but it sees only the uncompressed physical - track. The wrapper's subscriber applies that rule - itself: it widens such a physical request to frame 0, inflates from - there, and yields only the requested logical range. Test a resumed - mid-group read, not only full round trips. +- Ranges: every moq-net range starts at a group's first frame + ([moq-dev/moq#6465](https://github.com/moq-dev/moq/pull/6465)), so the wrapper reads each physical group whole + and inflates from frame 0. - Frames: AEAD authenticates a whole frame, so the wrapper's frame producer buffers chunks from `write` and seals at `finish`, and its frame consumer yields chunks only after the frame authenticates. diff --git a/rs/moq-archive/src/producer/track.rs b/rs/moq-archive/src/producer/track.rs index 7059a8619c7..7fafaca200d 100644 --- a/rs/moq-archive/src/producer/track.rs +++ b/rs/moq-archive/src/producer/track.rs @@ -163,7 +163,7 @@ impl Track { let config = &shared.config; let range = match self.next { Some(next) => Range { - start: Anchor::Absolute(next), + start: Anchor::Absolute(next.group), end: None, }, None => Range::oldest(), diff --git a/rs/moq-archive/src/reader/tests.rs b/rs/moq-archive/src/reader/tests.rs index d21f28732fe..349bb982afa 100644 --- a/rs/moq-archive/src/reader/tests.rs +++ b/rs/moq-archive/src/reader/tests.rs @@ -194,16 +194,14 @@ async fn open_with(archive: &Archive, config: Config) -> (broadcast::Producer, R (broadcast, reader) } -/// FETCH one group starting at `frame_start`, returning each frame's timestamp and payload. +/// FETCH one group, returning each frame's timestamp and payload. async fn fetch( broadcast: &broadcast::Producer, track: &str, sequence: u64, - frame_start: u64, ) -> std::result::Result, moq_net::Error> { let track = broadcast.consume().track(track)?; - let options = group::Fetch::default().with_frame_start(frame_start); - let mut group = track.fetch_group(sequence, options).await?; + let mut group = track.fetch_group(sequence, None).await?; let mut frames = Vec::new(); while let Some(frame) = group.read_frame().await? { frames.push((frame.timestamp.unwrap().value(), frame.payload)); @@ -234,22 +232,9 @@ async fn fetch_replays_original_groups() { let (broadcast, _reader) = open(&archive).await; - assert_eq!( - fetch(&broadcast, "video", 0, 0).await.unwrap(), - expected("video", 0, 0..3) - ); - assert_eq!( - fetch(&broadcast, "video", 2, 0).await.unwrap(), - expected("video", 2, 0..2) - ); - assert_eq!( - fetch(&broadcast, "audio", 5, 0).await.unwrap(), - expected("audio", 5, 0..2) - ); - - // A requested frame_start keeps the original frame indices and timestamps. - let tail = fetch(&broadcast, "video", 1, 1).await.unwrap(); - assert_eq!(tail, expected("video", 1, 1..3)); + assert_eq!(fetch(&broadcast, "video", 0).await.unwrap(), expected("video", 0, 0..3)); + assert_eq!(fetch(&broadcast, "video", 2).await.unwrap(), expected("video", 2, 0..2)); + assert_eq!(fetch(&broadcast, "audio", 5).await.unwrap(), expected("audio", 5, 0..2)); let track = broadcast.consume().track("video").unwrap(); let info = track.query().await.unwrap(); @@ -270,18 +255,8 @@ async fn a_group_split_across_records_is_served_whole() { archive.commit("video", &tail, 0).await; let (broadcast, _reader) = open(&archive).await; - assert_eq!( - fetch(&broadcast, "video", 7, 0).await.unwrap(), - expected("video", 7, 0..5) - ); - assert_eq!( - fetch(&broadcast, "video", 7, 4).await.unwrap(), - expected("video", 7, 4..5) - ); - assert_eq!( - fetch(&broadcast, "video", 8, 0).await.unwrap(), - expected("video", 8, 0..5) - ); + assert_eq!(fetch(&broadcast, "video", 7).await.unwrap(), expected("video", 7, 0..5)); + assert_eq!(fetch(&broadcast, "video", 8).await.unwrap(), expected("video", 8, 0..5)); } #[tokio::test] @@ -366,11 +341,7 @@ async fn an_expired_head_is_not_found() { archive.commit("video", &tail, 1).await; let (broadcast, _reader) = open(&archive).await; - not_found(fetch(&broadcast, "video", 7, 0).await); - assert_eq!( - fetch(&broadcast, "video", 7, 3).await.unwrap(), - expected("video", 7, 3..5) - ); + not_found(fetch(&broadcast, "video", 7).await); } #[tokio::test] @@ -384,7 +355,7 @@ async fn requests_download_only_their_object() { for sequence in 0..3 { assert_eq!( - fetch(&broadcast, "audio", sequence, 0).await.unwrap(), + fetch(&broadcast, "audio", sequence).await.unwrap(), expected("audio", sequence, 0..1) ); } @@ -424,18 +395,15 @@ async fn unadvertised_or_unreadable_groups_are_not_found() { let (broadcast, _reader) = open(&archive).await; - not_found(fetch(&broadcast, "audio", 2, 0).await); // between records - not_found(fetch(&broadcast, "audio", 5, 0).await); // past the last record - not_found(fetch(&broadcast, "video", 0, 0).await); // table disagrees with the record - not_found(fetch(&broadcast, "bad", 0, 0).await); // malformed envelope - not_found(fetch(&broadcast, "missing", 0, 0).await); // no object - not_found(fetch(&broadcast, "chat", 0, 0).await); // a track no timeline indexes + not_found(fetch(&broadcast, "audio", 2).await); // between records + not_found(fetch(&broadcast, "audio", 5).await); // past the last record + not_found(fetch(&broadcast, "video", 0).await); // table disagrees with the record + not_found(fetch(&broadcast, "bad", 0).await); // malformed envelope + not_found(fetch(&broadcast, "missing", 0).await); // no object + not_found(fetch(&broadcast, "chat", 0).await); // a track no timeline indexes // Siblings remain usable. - assert_eq!( - fetch(&broadcast, "audio", 4, 0).await.unwrap(), - expected("audio", 4, 0..1) - ); + assert_eq!(fetch(&broadcast, "audio", 4).await.unwrap(), expected("audio", 4, 0..1)); } #[tokio::test] @@ -444,23 +412,20 @@ async fn refresh_follows_new_records_and_pops() { archive.add("video", 0..2, 1).await; let (broadcast, mut reader) = open(&archive).await; - assert!(fetch(&broadcast, "video", 0, 0).await.is_ok()); + assert!(fetch(&broadcast, "video", 0).await.is_ok()); let next = archive.record("video", Position::group(2), Position::group(3), 1).await; // Refreshing before the commit finds nothing new; Not Found is not finality. reader.refresh().await.unwrap(); - not_found(fetch(&broadcast, "video", 2, 0).await); + not_found(fetch(&broadcast, "video", 2).await); archive.commit("video", &next, 1).await; reader.refresh().await.unwrap(); - assert_eq!( - fetch(&broadcast, "video", 2, 0).await.unwrap(), - expected("video", 2, 0..1) - ); + assert_eq!(fetch(&broadcast, "video", 2).await.unwrap(), expected("video", 2, 0..1)); // The popped record's groups are gone, although its object is still stored and was cached. archive.store.inner().take(); - not_found(fetch(&broadcast, "video", 1, 0).await); + not_found(fetch(&broadcast, "video", 1).await); assert_eq!(archive.store.inner().gets(), Vec::::new()); } @@ -472,7 +437,7 @@ async fn a_track_enrolled_after_opening_is_followed() { archive.add("audio", 0..2, 1).await; let (broadcast, mut reader) = open(&archive).await; - not_found(fetch(&broadcast, "video", 0, 0).await); + not_found(fetch(&broadcast, "video", 0).await); archive.add("video", 0..2, 2).await; let timelines = archive.store.timelines().await.unwrap(); @@ -484,10 +449,7 @@ async fn a_track_enrolled_after_opening_is_followed() { ); reader.refresh().await.unwrap(); - assert_eq!( - fetch(&broadcast, "video", 1, 0).await.unwrap(), - expected("video", 1, 0..2) - ); + assert_eq!(fetch(&broadcast, "video", 1).await.unwrap(), expected("video", 1, 0..2)); let timeline = broadcast.consume().track(&timeline("video")).unwrap(); assert!(timeline.subscribe(None).await.is_ok(), "the late timeline replays"); } @@ -523,7 +485,7 @@ async fn a_missing_timeline_segment_recovers_from_the_next_checkpoint() { // Record 2's checkpoint restates record 1, so its group is still served. for sequence in 0..3 { assert_eq!( - fetch(&broadcast, "video", sequence, 0).await.unwrap(), + fetch(&broadcast, "video", sequence).await.unwrap(), expected("video", sequence, 0..1) ); } @@ -539,8 +501,8 @@ async fn a_missing_tail_is_retried_on_the_next_refresh() { archive.store.inner().hide_gets("segments/0000000000000000002"); let (broadcast, mut reader) = open(&archive).await; - assert!(fetch(&broadcast, "video", 1, 0).await.is_ok()); - not_found(fetch(&broadcast, "video", 2, 0).await); + assert!(fetch(&broadcast, "video", 1).await.is_ok()); + not_found(fetch(&broadcast, "video", 2).await); archive.store.inner().heal(); archive.store.inner().take(); @@ -556,10 +518,7 @@ async fn a_missing_tail_is_retried_on_the_next_refresh() { ], "the cursor stays before the missing tail" ); - assert_eq!( - fetch(&broadcast, "video", 2, 0).await.unwrap(), - expected("video", 2, 0..1) - ); + assert_eq!(fetch(&broadcast, "video", 2).await.unwrap(), expected("video", 2, 0..1)); } #[tokio::test] @@ -574,7 +533,7 @@ async fn following_lists_only_new_timeline_keys() { .record("video", Position::group(group), Position::group(group + 1), 1) .await; reader.refresh().await.unwrap(); - not_found(fetch(&broadcast, "video", group, 0).await); + not_found(fetch(&broadcast, "video", group).await); archive.commit("video", &record, 0).await; archive.store.inner().take(); @@ -592,7 +551,7 @@ async fn following_lists_only_new_timeline_keys() { "following record {group} touches no media listing" ); assert_eq!( - fetch(&broadcast, "video", group, 0).await.unwrap(), + fetch(&broadcast, "video", group).await.unwrap(), expected("video", group, 0..1) ); } @@ -622,11 +581,11 @@ async fn an_unordered_listing_replays_in_segment_order() { ); for group in 0..3 { - not_found(fetch(&broadcast, "video", group, 0).await); + not_found(fetch(&broadcast, "video", group).await); } for group in 3..5 { assert_eq!( - fetch(&broadcast, "video", group, 0).await.unwrap(), + fetch(&broadcast, "video", group).await.unwrap(), expected("video", group, 0..1) ); } @@ -651,12 +610,9 @@ async fn a_track_without_usable_info_is_not_found() { let (broadcast, _reader) = open(&archive).await; for track in ["bare", "future"] { - not_found(fetch(&broadcast, track, 0, 0).await); + not_found(fetch(&broadcast, track, 0).await); } - assert_eq!( - fetch(&broadcast, "audio", 0, 0).await.unwrap(), - expected("audio", 0, 0..1) - ); + assert_eq!(fetch(&broadcast, "audio", 0).await.unwrap(), expected("audio", 0, 0..1)); } #[tokio::test] @@ -674,7 +630,7 @@ async fn multi_frame_timeline_groups_decode() { archive.timeline("video", 0, groups).await; let (broadcast, _reader) = open(&archive).await; - assert!(fetch(&broadcast, "video", 1, 0).await.is_ok()); + assert!(fetch(&broadcast, "video", 1).await.is_ok()); } #[tokio::test] @@ -716,11 +672,8 @@ async fn a_record_contradicting_its_timeline_is_not_indexed() { archive.timeline("video", untimed.sequence, groups).await; let (broadcast, _reader) = open(&archive).await; - assert_eq!( - fetch(&broadcast, "video", 0, 0).await.unwrap(), - expected("video", 0, 0..1) - ); - not_found(fetch(&broadcast, "video", 1, 0).await); + assert_eq!(fetch(&broadcast, "video", 0).await.unwrap(), expected("video", 0, 0..1)); + not_found(fetch(&broadcast, "video", 1).await); } #[tokio::test] @@ -745,20 +698,11 @@ async fn evicted_timeline_groups_remain_fetchable() { .await .unwrap(); let serve = tokio::spawn(reader.serve()); - let expected = fetch(&broadcast, &timeline("video"), 0, 0).await.unwrap(); + let expected = fetch(&broadcast, &timeline("video"), 0).await.unwrap(); let now = moq_net::time::Instant::now(); pool.gc(now, &kio::Waiter::noop()); pool.gc(now + Duration::from_secs(1), &kio::Waiter::noop()); - assert_eq!(fetch(&broadcast, &timeline("video"), 0, 0).await.unwrap(), expected); - // Past the end of an evicted complete group is EOF, as it is for a cached one. - pool.gc(now + Duration::from_secs(2), &kio::Waiter::noop()); - pool.gc(now + Duration::from_secs(3), &kio::Waiter::noop()); - assert!( - fetch(&broadcast, &timeline("video"), 0, expected.len() as u64 + 1) - .await - .unwrap() - .is_empty() - ); + assert_eq!(fetch(&broadcast, &timeline("video"), 0).await.unwrap(), expected); serve.abort(); let _ = serve.await; } @@ -778,7 +722,7 @@ async fn enrolled_timeline_fetch_survives_finish() { archive.add("audio", 1..2, 1).await; reader.track("audio", &timeline("audio")).await.unwrap(); reader.refresh().await.unwrap(); - let expected = fetch(&broadcast, &timeline("audio"), 0, 0).await.unwrap(); + let expected = fetch(&broadcast, &timeline("audio"), 0).await.unwrap(); reader.finish().unwrap(); let now = moq_net::time::Instant::now(); broadcast.info().pool.gc(now, &kio::Waiter::noop()); @@ -786,19 +730,12 @@ async fn enrolled_timeline_fetch_survives_finish() { .info() .pool .gc(now + Duration::from_secs(31), &kio::Waiter::noop()); - assert_eq!(fetch(&broadcast, &timeline("audio"), 0, 0).await.unwrap(), expected); - let missing = fetch(&broadcast, &timeline("audio"), 2, 0).await; + assert_eq!(fetch(&broadcast, &timeline("audio"), 0).await.unwrap(), expected); + let missing = fetch(&broadcast, &timeline("audio"), 2).await; assert!( matches!(missing, Err(moq_net::Error::NotFound)), "unknown timeline group: {missing:?}" ); - // Complete groups retain their known end, so a read past it is EOF. - assert!( - fetch(&broadcast, &timeline("audio"), 0, expected.len() as u64 + 1) - .await - .unwrap() - .is_empty() - ); serving.abort(); let _ = serving.await; } @@ -849,7 +786,7 @@ async fn benchmark_timeline_fetch_fanout() { "benchmark must fetch an evicted group" ); let started = std::time::Instant::now(); - let results = futures::future::join_all((0..readers).map(|_| fetch(&broadcast, &name, 0, 0))).await; + let results = futures::future::join_all((0..readers).map(|_| fetch(&broadcast, &name, 0))).await; elapsed += started.elapsed(); assert!( results diff --git a/rs/moq-c/MIGRATE.md b/rs/moq-c/MIGRATE.md index 73a236f8faa..f03f8167757 100644 --- a/rs/moq-c/MIGRATE.md +++ b/rs/moq-c/MIGRATE.md @@ -4,6 +4,10 @@ How to update your code for each breaking change, newest first. ## Unreleased +### `moq_anchor` names a group ([#6465](https://github.com/moq-dev/moq/pull/6465)) + +`moq_anchor` drops `frame`, which changes its ABI: `MOQ_ANCHOR_ABSOLUTE` always starts at the group's first frame. Remove any `frame` you set. + ### A broadcast reference waits for its sibling's announcement (#6254) `moq_consume_video` and `moq_consume_audio` on a rendition whose catalog `broadcast` field names a sibling now wait for a sibling nothing announces yet instead of failing as unroutable, so a late sibling plays once it is announced. diff --git a/rs/moq-c/src/api.rs b/rs/moq-c/src/api.rs index 3f7d56937a1..0cd367a7cc0 100644 --- a/rs/moq-c/src/api.rs +++ b/rs/moq-c/src/api.rs @@ -618,7 +618,7 @@ pub struct moq_range { pub end_present: bool, } -/// Where one end of a [moq_range] falls. Zeroing it means group 0, frame 0. +/// Where one end of a [moq_range] falls. Zeroing it means group 0. #[repr(C)] #[derive(Clone, Copy)] pub struct moq_anchor { @@ -627,8 +627,6 @@ pub struct moq_anchor { /// The group, or with `MOQ_ANCHOR_RELATIVE` how many groups before the latest one /// (0 is the latest, and an offset past the first group clamps to it). pub group: u64, - /// The frame within `group`, read only with `MOQ_ANCHOR_ABSOLUTE`. - pub frame: u64, /// The presentation time in microseconds, read only with `MOQ_ANCHOR_TIME`. pub timestamp_us: u64, } @@ -638,7 +636,7 @@ pub struct moq_anchor { #[expect(non_camel_case_types, reason = "C ABI type names match moq.h")] #[derive(Clone, Copy, Debug)] pub enum moq_anchor_kind { - /// A fixed group and frame. + /// A fixed group, delivered from its first frame. MOQ_ANCHOR_ABSOLUTE = 0, /// A count of groups back from the latest one. MOQ_ANCHOR_RELATIVE = 1, @@ -659,12 +657,7 @@ pub enum moq_order { fn parse_anchor(anchor: &moq_anchor) -> Result { Ok(match anchor.kind { - v if v == moq_anchor_kind::MOQ_ANCHOR_ABSOLUTE as u32 => { - moq_net::track::Anchor::Absolute(moq_net::track::Position { - group: anchor.group, - frame: anchor.frame, - }) - } + v if v == moq_anchor_kind::MOQ_ANCHOR_ABSOLUTE as u32 => moq_net::track::Anchor::Absolute(anchor.group), v if v == moq_anchor_kind::MOQ_ANCHOR_RELATIVE as u32 => moq_net::track::Anchor::Relative(anchor.group), v if v == moq_anchor_kind::MOQ_ANCHOR_TIME as u32 => { moq_net::track::Anchor::Time(moq_net::Timestamp::from_micros(anchor.timestamp_us)?) diff --git a/rs/moq-c/src/test.rs b/rs/moq-c/src/test.rs index aabec7da78b..c7900caf79d 100644 --- a/rs/moq-c/src/test.rs +++ b/rs/moq-c/src/test.rs @@ -1710,13 +1710,11 @@ fn raw_track_subscription_options_and_update() { start: moq_anchor { kind: moq_anchor_kind::MOQ_ANCHOR_ABSOLUTE as u32, group: 1, - frame: 0, timestamp_us: 0, }, end: moq_anchor { kind: moq_anchor_kind::MOQ_ANCHOR_ABSOLUTE as u32, group: 2, - frame: 0, timestamp_us: 0, }, end_present: true, @@ -1824,13 +1822,11 @@ fn raw_track_time_anchor() { start: moq_anchor { kind: moq_anchor_kind::MOQ_ANCHOR_TIME as u32, group: 0, - frame: 0, timestamp_us: 30_000, }, end: moq_anchor { kind: moq_anchor_kind::MOQ_ANCHOR_ABSOLUTE as u32, group: 0, - frame: 0, timestamp_us: 0, }, end_present: false, @@ -1924,13 +1920,11 @@ fn raw_track_update_refuses_time_on_untimed() { start: moq_anchor { kind: moq_anchor_kind::MOQ_ANCHOR_TIME as u32, group: 0, - frame: 0, timestamp_us: 0, }, end: moq_anchor { kind: moq_anchor_kind::MOQ_ANCHOR_ABSOLUTE as u32, group: 0, - frame: 0, timestamp_us: 0, }, end_present: false, @@ -1990,7 +1984,6 @@ fn raw_track_relative_update_counts_from_the_track_edge() { let anchor = |kind: moq_anchor_kind, group: u64| moq_anchor { kind: kind as u32, group, - frame: 0, timestamp_us: 0, }; let subscribe = |ranges: &[moq_range]| moq_subscription { @@ -3172,7 +3165,6 @@ fn fetch_group( name: &str, sequence: u64, priority: i8, - frame_start: u64, ) -> mpsc::Receiver), moq_net::Error>> { let consumer = State::lock() .publish @@ -3184,9 +3176,7 @@ fn fetch_group( crate::ffi::RUNTIME.spawn(async move { let res = async { let track = consumer.track(&name)?; - let options = moq_net::group::Fetch::default() - .with_priority(priority) - .with_frame_start(frame_start); + let options = moq_net::group::Fetch::default().with_priority(priority); let mut group = track.fetch_group(sequence, options).await?; let index = group.index(); let frame = group.read_frame().await?.expect("expected a fetched frame"); @@ -3209,7 +3199,7 @@ fn track_dynamic_serves_a_fetch_miss() { let group_cb = Callback::new(); let dynamic = id(unsafe { moq_publish_track_dynamic(track, Some(channel_callback), group_cb.ptr) }); - let fetch = fetch_group(broadcast, "data", 5, 11, 0); + let fetch = fetch_group(broadcast, "data", 5, 11); let request = id(group_cb.recv()); let mut sequence = 0u64; let mut priority = 0i8; @@ -3232,7 +3222,7 @@ fn track_dynamic_serves_a_fetch_miss() { assert_eq!(fetch.recv_timeout(TIMEOUT).unwrap().unwrap(), (0, payload.to_vec())); // A rejected fetch fails with the application code. - let fetch = fetch_group(broadcast, "data", 6, 0, 0); + let fetch = fetch_group(broadcast, "data", 6, 0); let request = id(group_cb.recv()); assert_eq!(moq_group_request_abort(request, 9), 0); let err = fetch @@ -3248,41 +3238,6 @@ fn track_dynamic_serves_a_fetch_miss() { assert_eq!(moq_origin_close(origin), 0); } -#[test] -fn track_dynamic_serves_a_fetch_from_frame_start() { - let origin = id(moq_origin_create()); - let broadcast = publish_broadcast(origin, b"track-dynamic-frame-start"); - let name = b"data"; - let track = - id(unsafe { moq_publish_track(broadcast, name.as_ptr() as *const c_char, name.len(), std::ptr::null()) }); - - let group_cb = Callback::new(); - let dynamic = id(unsafe { moq_publish_track_dynamic(track, Some(channel_callback), group_cb.ptr) }); - - let fetch = fetch_group(broadcast, "data", 5, 0, 3); - let request = id(group_cb.recv()); - let mut frame_start = 0u64; - assert_eq!(unsafe { moq_group_request_frame_start(request, &mut frame_start) }, 0); - assert_eq!(frame_start, 3); - - let group = id(moq_group_request_accept(request)); - let payload = b"tail"; - assert_eq!( - unsafe { moq_publish_group_frame(group, payload.as_ptr(), payload.len(), 100_000) }, - 0 - ); - assert_eq!(moq_publish_group_finish(group), 0); - let (index, got) = fetch.recv_timeout(TIMEOUT).unwrap().unwrap(); - assert_eq!(index, 3, "the consumer resumes at the requested frame"); - assert_eq!(got, payload); - - assert_eq!(moq_publish_dynamic_cancel(dynamic), 0); - assert_eq!(group_cb.recv_terminal(), 0); - assert_eq!(moq_publish_track_finish(track), 0); - assert_eq!(moq_publish_close(broadcast), 0); - assert_eq!(moq_origin_close(origin), 0); -} - #[test] fn track_request_dynamic_survives_accept() { let origin = id(moq_origin_create()); @@ -3291,7 +3246,7 @@ fn track_request_dynamic_survives_accept() { let _dynamic = id(unsafe { moq_publish_dynamic(broadcast, Some(channel_callback), request_cb.ptr) }); // A fetch of a track the broadcast has not declared requests the track first. - let fetch = fetch_group(broadcast, "archive", 9, 0, 0); + let fetch = fetch_group(broadcast, "archive", 9, 0); let request = id(request_cb.recv()); let group_cb = Callback::new(); let track_dynamic = id(unsafe { moq_track_request_dynamic(request, Some(channel_callback), group_cb.ptr) }); diff --git a/rs/moq-ffi/MIGRATE.md b/rs/moq-ffi/MIGRATE.md index a28d1fbdda2..05a258f1c47 100644 --- a/rs/moq-ffi/MIGRATE.md +++ b/rs/moq-ffi/MIGRATE.md @@ -5,6 +5,10 @@ These are the UniFFI names; each language wrapper has its own `MIGRATE.md` in it ## Unreleased +### `MoqAnchor::Absolute` names a group ([#6465](https://github.com/moq-dev/moq/pull/6465)) + +`MoqAnchor::Absolute` drops `frame`: a range always starts at a group's first frame. Pass `MoqAnchor::Absolute { group }`. + ### `announce` takes a `MoqAnnounceRoute` `MoqBroadcastProducer::announce` takes `MoqAnnounceRoute { cost, epoch }` instead of a `MoqRoute`: the origin builds the hop chain, so a publisher no longer passes `hops` or `anonymous`. diff --git a/rs/moq-ffi/src/consumer.rs b/rs/moq-ffi/src/consumer.rs index 4b34bdae9d6..88a72845b17 100644 --- a/rs/moq-ffi/src/consumer.rs +++ b/rs/moq-ffi/src/consumer.rs @@ -66,12 +66,10 @@ pub struct MoqRange { /// Where one end of a [`MoqRange`] falls, mirroring [`moq_net::track::Anchor`]. #[derive(Clone, Copy, uniffi::Enum)] pub enum MoqAnchor { - /// This exact group and frame. + /// This group, delivered from its first frame. Absolute { /// The absolute group sequence number. group: u64, - /// The zero-based frame index within the group. - frame: u64, }, /// The group this many before the latest, clamped to group 0: `0` is the latest group. Relative { @@ -100,7 +98,7 @@ impl TryFrom for moq_net::track::Anchor { fn try_from(anchor: MoqAnchor) -> Result { Ok(match anchor { - MoqAnchor::Absolute { group, frame } => Self::Absolute(moq_net::track::Position { group, frame }), + MoqAnchor::Absolute { group } => Self::Absolute(group), MoqAnchor::Relative { groups } => Self::Relative(groups), MoqAnchor::Time { timestamp_us } => { Self::Time(moq_net::Timestamp::from_micros(timestamp_us).map_err(|_| MoqError::TimeOverflow)?) diff --git a/rs/moq-ffi/src/test.rs b/rs/moq-ffi/src/test.rs index d8e0d73e959..5a23adade3a 100644 --- a/rs/moq-ffi/src/test.rs +++ b/rs/moq-ffi/src/test.rs @@ -1055,7 +1055,7 @@ async fn raw_track_update_does_not_wait_for_pending_read() { priority: 10, max_delay_us: 25_000, ranges: Some(vec![MoqRange { - start: MoqAnchor::Absolute { group: 0, frame: 0 }, + start: MoqAnchor::Absolute { group: 0 }, end: None, }]), order: None, @@ -4349,7 +4349,7 @@ async fn raw_track_update_during_pending_group_still_reads_datagram() { priority: 10, max_delay_us: 25_000, ranges: Some(vec![MoqRange { - start: MoqAnchor::Absolute { group: 0, frame: 0 }, + start: MoqAnchor::Absolute { group: 0 }, end: None, }]), order: None, diff --git a/rs/moq-hls/src/export/rendition.rs b/rs/moq-hls/src/export/rendition.rs index d4d1a797fbc..3ca5aaf9396 100644 --- a/rs/moq-hls/src/export/rendition.rs +++ b/rs/moq-hls/src/export/rendition.rs @@ -1301,18 +1301,19 @@ impl Drop for Rendition { /// Fetch one group, mapping "no longer (or not yet) servable" to the error that says so /// rather than a bare `None`: a caller abandoning a segment can only explain itself with it. /// -/// Reads from `start` and, when `end` is set, stops before that frame of the group. +/// Fetches the whole group, since a group is the unit of decodability, and reads from `start` +/// and, when `end` is set, stops before that frame of the group. async fn fetch( track: &moq_net::track::Consumer, start: hang::timeline::Position, end: Option, broadcast: &moq_net::broadcast::Consumer, ) -> Result> { - let options = moq_net::group::Fetch::default().with_frame_start(start.frame); - match track.fetch_group(start.group, options).await { + match track.fetch_group(start.group, None).await { Ok(mut group) => { - if let Some(end) = end { - group.set_frames(..end); + match end { + Some(end) => group.set_frames(start.frame..end), + None => group.set_frames(start.frame..), } Ok(Ok(group)) } diff --git a/rs/moq-net/MIGRATE.md b/rs/moq-net/MIGRATE.md index fb52968088d..36f429f5ca4 100644 --- a/rs/moq-net/MIGRATE.md +++ b/rs/moq-net/MIGRATE.md @@ -4,6 +4,12 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Subscriptions and fetches start at a group's first frame ([#6465](https://github.com/moq-dev/moq/pull/6465)) + +A group is the unit of decodability, so every range names whole groups. +`track::Anchor::Absolute` takes a group sequence instead of a `Position`: replace `Anchor::Absolute(Position::group(n))` with `Anchor::Absolute(n)`, and as an exclusive end, `Position::after_group(n)` with `n + 1`. +`group::Fetch::frame_start` and `with_frame_start` are gone: fetch the whole group and skip frames with `group::Consumer::set_frames`. + ### A listing says whether a removed track finished (#6399) `broadcast::TracksEvent` gains `Finished { name }`: the track left the listing because its producer declared its end. `Removed { name }` now means it left unfinished (dropped or aborted without an end, or its route withdrawn). Handle both where you matched `Removed`. diff --git a/rs/moq-net/benches/codec.rs b/rs/moq-net/benches/codec.rs index f9fda728403..39f6dad9347 100644 --- a/rs/moq-net/benches/codec.rs +++ b/rs/moq-net/benches/codec.rs @@ -68,43 +68,5 @@ fn bench(c: &mut Criterion) { group.finish(); } -/// An outgoing request reads the counted aggregate and checks only its named group, -/// independent of the audience beside it and the cache's unrelated groups. -fn request_start(c: &mut Criterion) { - use moq_net::{broadcast, track}; - use std::hint::black_box; - let mut bench = c.benchmark_group("codec_request_start"); - for groups in [1, 64, 1024] { - for readers in [1, 64, 1024] { - for held in [false, true] { - let broadcast = broadcast::Info::default().produce(); - let track = broadcast.create_track("track", track::Info::default()).unwrap(); - for _ in 0..groups { - let mut group = track.append_group().unwrap(); - group.write_frame(None, bytes::Bytes::from_static(b"prefix")).unwrap(); - group.finish().unwrap(); - } - let sequence = if held { groups - 1 } else { groups }; - let subscription = track::Subscription::default().with_ranges([track::Range { - start: track::Anchor::Absolute(track::Position { - group: sequence, - frame: 1, - }), - end: None, - }]); - let _readers: Vec<_> = (0..readers).map(|_| track.subscribe(subscription.clone())).collect(); - bench.bench_function(BenchmarkId::new(format!("held_{held}_{readers}r"), groups), |b| { - b.iter(|| { - let mut subscription = track.subscription().unwrap(); - moq_net::fuzz::widen_start(&track, &mut subscription); - black_box(subscription); - }); - }); - } - } - } - bench.finish(); -} - -criterion_group!(benches, bench, request_start); +criterion_group!(benches, bench); criterion_main!(benches); diff --git a/rs/moq-net/benches/track.rs b/rs/moq-net/benches/track.rs index 32d2159cd93..ee990632f7d 100644 --- a/rs/moq-net/benches/track.rs +++ b/rs/moq-net/benches/track.rs @@ -1411,7 +1411,7 @@ impl TimeEnd { let subscription = track::Subscription::default() .with_max_delay(AGED_BUDGET) .with_ranges([track::Range { - start: track::Anchor::Absolute(track::Position::group(0)), + start: track::Anchor::Absolute(0), end: Some(track::Anchor::Time(Timestamp::from_secs(u32::MAX.into()).unwrap())), }]); let waiter = kio::Waiter::noop(); diff --git a/rs/moq-net/src/coding/reader.rs b/rs/moq-net/src/coding/reader.rs index 32382d93715..77f60c36862 100644 --- a/rs/moq-net/src/coding/reader.rs +++ b/rs/moq-net/src/coding/reader.rs @@ -413,23 +413,6 @@ impl Reader { Poll::Ready(Ok(bytes)) } - /// Poll to discard `remaining` bytes, counting it down as they arrive. Each byte is - /// released as it is skipped, so a peer cannot make a skipped payload cost credit. - pub(crate) fn poll_skip(&mut self, cx: &mut Context<'_>, remaining: &mut usize) -> Poll> { - self.want = 0; - while *remaining > 0 { - let mut lease = ready!(self.poll_lease(cx, 1))?; - if !lease.has_remaining() { - return Poll::Ready(Err(DecodeError::Short.into())); - } - let skipped = lease.remaining().min(*remaining); - lease.advance(skipped); - lease.release(); - *remaining -= skipped; - } - Poll::Ready(Ok(())) - } - /// Read exactly the given number of bytes from the stream. pub async fn read_exact(&mut self, size: usize) -> Result { std::future::poll_fn(|cx| self.poll_read_exact(cx, size)).await @@ -709,24 +692,6 @@ mod tests { } } - /// A skipped payload is discarded across reads, leaving what follows it, and a skip - /// waiting on more has counted down what it already discarded. - #[test] - fn skip_discards_across_reads() { - let mut reader = queue(&[b"abc", b"defg", b"hi"], crate::lite::Version::Lite05); - let mut cx = Context::from_waker(std::task::Waker::noop()); - - let mut remaining = 6; - assert!(matches!(reader.poll_skip(&mut cx, &mut remaining), Poll::Ready(Ok(())))); - assert_eq!(&reader.read_exact(3).now_or_never().unwrap().unwrap()[..], b"ghi"); - - let mut reader = queue(&[b"ab", b"cd"], crate::lite::Version::Lite05); - let mut remaining = 5; - assert!(reader.poll_skip(&mut cx, &mut remaining).is_pending()); - assert_eq!(remaining, 1); - assert_eq!(leased(&mut reader), 0, "nothing skipped is kept"); - } - /// A message cut across chunks decodes from a copy, waiting for exactly what its /// prefix declared, and consumes only itself. #[test] diff --git a/rs/moq-net/src/fuzz.rs b/rs/moq-net/src/fuzz.rs index 7d666235375..79c431c5b76 100644 --- a/rs/moq-net/src/fuzz.rs +++ b/rs/moq-net/src/fuzz.rs @@ -73,11 +73,6 @@ pub fn object_group(frames: usize, properties: bool) -> (broadcast::Producer, tr (broadcast, track, group) } -/// Widen an outgoing request's mid-group starts for the private requesting-model benchmark. -pub fn widen_start(track: &track::Producer, subscription: &mut track::Subscription) { - track.widen_start(subscription); -} - /// Every target, keyed by the name of its `fuzz_targets/.rs` shim. pub const TARGETS: &[(&str, Target)] = &[ ("lite", lite_wire), diff --git a/rs/moq-net/src/ietf/publisher.rs b/rs/moq-net/src/ietf/publisher.rs index 9b3d2afc764..fe9b7725686 100644 --- a/rs/moq-net/src/ietf/publisher.rs +++ b/rs/moq-net/src/ietf/publisher.rs @@ -16420,17 +16420,12 @@ struct ServeRange { } impl ServeRange { - /// Bound `subscription` to this range. + /// Bound `subscription` to the whole groups this range touches: a model subscription + /// starts and ends on a group boundary, and [`GroupSlice`] cuts the objects when serving. fn apply(&self, subscription: &mut Subscription) { subscription.set_bounds( - self.start.map(|start| track::Position { - group: start.group, - frame: start.object, - }), - self.end.and_then(|end| match end.object { - Some(object) => track::Position::after(end.group, object), - None => track::Position::after_group(end.group), - }), + self.start.map(|start| start.group), + self.end.and_then(|end| end.group.checked_add(1)), ); } } diff --git a/rs/moq-net/src/ietf/subscriber.rs b/rs/moq-net/src/ietf/subscriber.rs index 6eca3d9faeb..3a804e00028 100644 --- a/rs/moq-net/src/ietf/subscriber.rs +++ b/rs/moq-net/src/ietf/subscriber.rs @@ -486,15 +486,6 @@ impl Target { } } - /// Widen a mid-group start to the group's first frame unless the copy holds the frames - /// below it. - fn widen_start(&self, subscription: &mut track::Subscription) { - match self { - Self::Request(request) => request.widen_start(subscription), - Self::Resume(idle) => idle.track.widen_start(subscription), - } - } - /// The subscription could not be made: the track fails with `err`. fn fail(self, err: Error) { match self { @@ -583,14 +574,6 @@ struct TrackState { // The joining FETCH's own request id, when one was sent. fetch_id: Option, - // Where the subscription asked to start, when the request starts partway through a - // group: a resumed subscription picking up after the frames a previous route - // delivered, or an application range, either widened to its group's first frame unless - // the copy holds the frames below it. The stream for that group legitimately starts - // there, without a head, and its Object IDs stay pinned to the ordinals the readers' - // windows count in. - resume: Option, - // The data streams read so far, which PUBLISH_DONE's Stream Count is checked against. tail: kio::Producer, } @@ -614,7 +597,6 @@ impl TrackState { timescale: None, fill, fetch_id: None, - resume: None, tail: Default::default(), } } @@ -2845,11 +2827,7 @@ where gate: &mut crate::auth::Gate, ) -> Option { let validity = self.validity.start(); - let requested = target.subscription(); - let subscription = requested.clone().map(|mut subscription| { - target.widen_start(&mut subscription); - subscription - }); + let subscription = target.subscription(); let start = subscription.as_ref().and_then(|s| s.start()); let join = match subscribe_join(start, subscription.as_ref().and_then(|s| s.end()), self.version) { Ok(join) => join, @@ -2972,11 +2950,6 @@ where state.subscribes.insert( request_id, TrackState { - resume: requested - .as_ref() - .and_then(|s| s.start()) - .filter(|start| start.frame != 0) - .and(start), // A resumed copy is the session's to abort from here on. producer: target.producer().cloned(), epoch: self.epoch.clone(), @@ -3306,7 +3279,7 @@ where }; // Pending until the answer is written in below, so a relay serving the copy // never reads the requested floor as where the feed starts. - let request = request.request_start(start.map(|start| start.group)); + let request = request.request_start(start); // Serves cache misses with a group FETCH. Registered before accepting, so a // miss queued meanwhile waits for it rather than failing for want of a handler. let dynamic = request.dynamic(); @@ -3330,7 +3303,7 @@ where // may still arrive after a higher one's. This resolves the pending start. let _ = match live { true => track.start_at(largest.map_or(0, |largest| largest.group)), - false => track.start_at(start.map(|start| start.group)), + false => track.start_at(start), }; if let Some(staged) = staged { staged.commit(); @@ -4304,15 +4277,13 @@ where } }; - let (mut track, timescale, fill, resume, mut reading) = { + let (mut track, timescale, fill, mut reading) = { let state = self.state.lock(); let track = state.subscribes.get(&request_id).ok_or(Error::NotFound)?; ( track.producer.clone().ok_or(Error::NotFound)?, track.timescale, track.fill.clone(), - // Only the group the subscription resumes partway through. - track.resume.filter(|resume| resume.group == group.group_id), // Every data stream counts toward PUBLISH_DONE's Stream Count, even one // dropped below, and the subscription's end waits until it is read. Reading::open(&track.tail, Some(group.group_id), self.runtime.now()), @@ -4324,19 +4295,18 @@ where stream.set_priority(crate::priority::recv_group(priority, group.group_id, order)); // Peek before creating a group: a standalone END_OF_TRACK has no payload - // group, and a fill or resume may already own the prefix. + // group, and a fill may already own the prefix. // // The peek inside blocks until the publisher produces the group's first object, so // race it against the subscription going away the same way the group read below is. // Otherwise dropping the local subscriber cannot end this handler. - let passthrough = !fill.read().outstanding() && resume.is_none(); + let passthrough = !fill.read().outstanding(); let opened = { let mut opening = track.clone(); let mut open = std::pin::pin!(crate::util::sendable(self.open_group( stream, &mut opening, &fill, - resume, &group, &mut reading ))); @@ -4374,14 +4344,10 @@ where } }; - // A resumed group's ingest refuses gaps and drops what the copy already holds, so a - // stream starting at or below the copy's next frame keeps each Object ID its ordinal. - let stitch = resume.map(|_| start); if passthrough || producer.wire().is_none() { let first_id = start.max(producer.frame_count() as u64); producer.set_wire(group::Wire { sparse: false, - pinned: stitch.is_some_and(|start| start <= producer.frame_count() as u64), first_object: group.flags.first_object && first_id == start, first_id, extensions: group.flags.has_extensions, @@ -4396,7 +4362,7 @@ where let mut producer = crate::recv::Group::new(producer); let observing = producer.handle().clone(); - let mut ingest = GroupIngest::new(self, &group, timescale, stitch); + let mut ingest = GroupIngest::new(self, &group, timescale); let res = { kio::wait(|waiter| { if let Poll::Ready(err) = track.poll_closed(waiter) { @@ -4637,7 +4603,6 @@ where stream: &mut Reader, track: &mut track::Producer, fill: &kio::Producer, - resume: Option, header: &ietf::GroupHeader, reading: &mut Reading, ) -> Result { @@ -4686,21 +4651,9 @@ where }); } - // Without a fill or resume, a relay preserves the subgroup's first Object ID - // and FIRST_OBJECT claim independently. Plain readers see only frame ordinals. - + // Without a fill, a relay preserves the subgroup's first Object ID and FIRST_OBJECT + // claim independently. Plain readers see only frame ordinals. if !fill.read().outstanding() { - // The group a resumed subscription picks up partway through starts where it - // asked, and a stream with no objects is the end of a group complete there. - // A publisher sending more of it (a pre-draft-20 join asks for all of it) has - // the objects below the start dropped as they arrive. - if let Some(resume) = resume - && first.is_none_or(|first| first.id <= resume.frame) - { - let mut producer = create(track)?; - producer.start_at(resume.frame)?; - return Ok(Opened::Group(producer, first.map_or(resume.frame, |first| first.id))); - } return Ok(Opened::Group(create(track)?, first.map_or(0, |first| first.id))); } @@ -4806,14 +4759,11 @@ struct GroupIngest { timescale: Option, version: Version, prior_object: Option, - start: Option, properties: crate::model::object::Properties, /// The property list `properties` was read from, for a status object, which has no frame. raw: bytes::Bytes, gap: Option, inflated: Option, - /// The object being read is one the group started past, so it is read and dropped. - dropping: bool, phase: IngestPhase, } @@ -4830,8 +4780,6 @@ enum IngestPhase { Status { timestamp: Option }, /// Streaming the object payload. Payload { frame: frame::Producer }, - /// Discarding a dropped object's payload: the bytes of it still to read. - Skip { size: usize }, /// An explicit end-of-group or end-of-track status arrived. Finished(Ended), } @@ -4841,19 +4789,16 @@ impl GroupIngest { subscriber: &Subscriber, group: &ietf::GroupHeader, timescale: Option, - start: Option, ) -> Self { Self { has_extensions: group.flags.has_extensions, timescale, version: subscriber.version, prior_object: None, - start, properties: Default::default(), raw: bytes::Bytes::new(), gap: None, inflated: None, - dropping: false, phase: IngestPhase::Delta, } } @@ -4881,7 +4826,7 @@ where return self.recv_group_fetch(stream, slot).await; } - let (subscribe_id, fill, resume, _counted) = { + let (subscribe_id, fill, _counted) = { let state = self.state.lock(); // A draft-20 fill is named by the SUBSCRIBE's request id. A pre-draft-20 joining // FETCH has its own id, which `fetches` maps back to that subscription. @@ -4893,7 +4838,7 @@ where let counted = joined .is_none() .then(|| Reading::open(&track.tail, None, self.runtime.now())); - (subscribe_id, track.fill.clone(), track.resume, counted) + (subscribe_id, track.fill.clone(), counted) }; // SUBSCRIBE_OK declares the units these object timestamps are in, and this stream can @@ -4941,12 +4886,7 @@ where // track does not close a group producer, since those lifecycles are independent. let res = { let mut serving = track.clone(); - let mut serve = std::pin::pin!(crate::util::sendable(self.run_fill( - stream, - &mut serving, - timescale, - resume - ))); + let mut serve = std::pin::pin!(crate::util::sendable(self.run_fill(stream, &mut serving, timescale))); kio::wait(|waiter| { if let Poll::Ready(err) = track.poll_closed(waiter) { return Poll::Ready(Err(err)); @@ -4996,11 +4936,10 @@ where stream: &mut Reader, track: &mut track::Producer, timescale: Option, - resume: Option, ) -> Result { let mut head: Option<(u64, u64, crate::recv::Group)> = None; - match self.run_fill_objects(stream, track, timescale, resume, &mut head).await { + match self.run_fill_objects(stream, track, timescale, &mut head).await { Ok(()) => Ok(match head { Some((sequence, next, producer)) => Fill::Ready { sequence, @@ -5022,17 +4961,12 @@ where /// absolute IDs. /// /// A joining FETCH and a draft-20 fill are one group, the head the live stream - /// continues. A Prior Object ID Gap must account for every missing ID. Ordinal resume - /// still requires contiguous IDs; refusing its fill leaves the subscription alone. - /// - /// A pre-draft-20 join resuming partway through a group asks for all of it, so the - /// objects below `resume` are dropped: another route already delivered them. + /// continues. A Prior Object ID Gap must account for every missing ID. async fn run_fill_objects( &mut self, stream: &mut Reader, track: &mut track::Producer, timescale: Option, - resume: Option, head: &mut Option<(u64, u64, crate::recv::Group)>, ) -> Result<(), Error> { let mut prior_group = None; @@ -5081,7 +5015,7 @@ where if id < expected { return Err(Error::ProtocolViolation); } - if id > expected && (resume.is_some() || decoded.gap.is_none_or(|gap| gap < id - expected)) { + if id > expected && decoded.gap.is_none_or(|gap| gap < id - expected) { return Err(Error::Unsupported); } match head.as_ref().map(|(sequence, _, _)| *sequence) { @@ -5093,21 +5027,13 @@ where _ => {} } - let (sequence, next, producer) = head.as_mut().expect("the head was created above"); + let (_, next, producer) = head.as_mut().expect("the head was created above"); if *next == 0 && let Some(mut wire) = producer.wire() { wire.first_id = id; - // Resume refuses gaps here and in the tail stream that continues this group. - wire.pinned = resume.is_some_and(|resume| resume.group == *sequence); producer.set_wire(wire)?; } - if *next == 0 - && let Some(resume) = resume.filter(|resume| resume.group == *sequence) - { - producer.start_at(resume.frame)?; - } - let keep = *next >= producer.frame_count() as u64; if self .recv_fetch_payload( stream, @@ -5116,7 +5042,6 @@ where decoded, raw, timescale, - keep, id, }, ) @@ -5130,8 +5055,7 @@ where Ok(()) } - /// Read one fetch object's length and payload into `producer`, after its header, or - /// past it unless `keep`. + /// Read one fetch object's length and payload into `producer`, after its header. async fn recv_fetch_payload( &self, stream: &mut Reader, @@ -5142,12 +5066,9 @@ where decoded, raw, timescale, - keep, id, } = payload; - if keep { - producer.mark_gap(id, decoded.gap)?; - } + producer.mark_gap(id, decoded.gap)?; let size = stream.varint().await?; if size == 0 && matches!(self.version, Version::Draft14 | Version::Draft15) { let status = stream.varint().await?; @@ -5163,9 +5084,7 @@ where match (status, unframed) { (0, _) => {} (END_OF_GROUP, Some(properties)) => { - if keep { - producer.set_end_object(crate::model::object::Fields { id, properties })?; - } + producer.set_end_object(crate::model::object::Fields { id, properties })?; return Ok(Fetched::EndOfGroup); } // END_OF_TRACK is track-level, so there is nowhere to keep its properties. @@ -5174,11 +5093,6 @@ where _ => return Err(Error::Unsupported), } } - if !keep { - let mut remaining = usize::try_from(size).map_err(|_| Error::FrameTooLarge)?; - std::future::poll_fn(|cx| stream.poll_skip(cx, &mut remaining)).await?; - return Ok(Fetched::Frame); - } super::object::check_frame(&decoded.properties, timescale)?; let timestamp = object_time(timescale, decoded.timestamp)?; let plain = id == producer.frame_count() as u64 && decoded.properties.is_empty(); @@ -5573,7 +5487,6 @@ where let (producer, track, timescale, start, end) = taken?; producer.set_wire(group::Wire { sparse: false, - pinned: false, first_object: start == 0, first_id: start, extensions: true, @@ -5687,7 +5600,6 @@ where decoded, raw, timescale, - keep: true, id, }, ) @@ -5739,7 +5651,6 @@ struct FetchPayload { /// The property list `decoded` was read from, for a status object, which has no frame. raw: bytes::Bytes, timescale: Option, - keep: bool, id: u64, } @@ -5845,7 +5756,6 @@ fn open_fill_group( let producer = track.create_group(group::Info { sequence })?; producer.set_wire(group::Wire { sparse: false, - pinned: false, first_object: true, first_id: 0, extensions: true, @@ -5925,13 +5835,12 @@ impl GroupIngest { let Some(id_delta) = ready!(reader.poll_varint_maybe(&mut cx))? else { return Poll::Ready(Ok(Ended::Group)); }; - let id = next_object_id(self.prior_object, id_delta, self.start)?; + let id = next_object_id(self.prior_object, id_delta)?; self.prior_object = Some(id); self.properties = Default::default(); self.raw = bytes::Bytes::new(); self.gap = None; self.inflated = None; - self.dropping = self.start.is_some() && id < group.frame_count() as u64; self.phase = match self.has_extensions { true => IngestPhase::ExtSize, false => IngestPhase::Size { timestamp: None }, @@ -5957,11 +5866,6 @@ impl GroupIngest { self.phase = IngestPhase::Status { timestamp: *timestamp }; continue; } - if self.dropping { - let size = usize::try_from(size).map_err(|_| Error::FrameTooLarge)?; - self.phase = IngestPhase::Skip { size }; - continue; - } // `create_object` is the allocation chokepoint: it rejects an oversized // `size` and allocates up front only within the origin's budget, so no // pre-check is needed. @@ -5973,12 +5877,10 @@ impl GroupIngest { IngestPhase::Status { timestamp } => { let status = ready!(reader.poll_varint(&mut cx))?; if status == 0 { - if !self.dropping { - let timestamp = object_time(self.timescale, *timestamp)?; - let header = self.header(group, frame::Info { size: 0, timestamp })?; - let frame = group.create_object(header)?; - group.restore(frame.finish()?); - } + let timestamp = object_time(self.timescale, *timestamp)?; + let header = self.header(group, frame::Info { size: 0, timestamp })?; + let frame = group.create_object(header)?; + group.restore(frame.finish()?); self.phase = IngestPhase::Delta; continue; } @@ -6021,10 +5923,6 @@ impl GroupIngest { } } } - IngestPhase::Skip { size } => { - ready!(reader.poll_skip(&mut cx, size))?; - self.phase = IngestPhase::Delta; - } IngestPhase::Finished(ended) => { let ended = std::mem::replace(ended, Ended::Group); return Poll::Ready(Ok(ended)); @@ -12319,11 +12217,7 @@ struct Join { /// /// A start group we already know is absolute on draft-20 and needs no fill: the /// subscription's own range covers it, which is what our publisher serves from its cache. -fn subscribe_join( - start: Option, - end: Option, - version: Version, -) -> Result { +fn subscribe_join(start: Option, end: Option, version: Version) -> Result { if !Filter::is_draft20(version) { if end.is_some() { return Err(Error::Unsupported); @@ -12347,7 +12241,7 @@ fn subscribe_join( fetch: false, }, // An absolute {0, 0} with no end is defined as unfiltered, so it spells itself. - Some(start) if start == track::Position::group(0) && end.is_none() => Join { + Some(0) if end.is_none() => Join { filter: Filter::Unfiltered, fill: None, fetch: false, @@ -12355,21 +12249,14 @@ fn subscribe_join( Some(start) => Join { filter: Filter::Absolute { start: ietf::Location { - group: start.group, - object: start.frame, + group: start, + object: 0, }, end: end.and_then(|end| { - if end.frame == 0 { - Some(ietf::EndLocation { - group: end.group.checked_sub(1)?, - object: None, - }) - } else { - Some(ietf::EndLocation { - group: end.group, - object: Some(end.frame - 1), - }) - } + Some(ietf::EndLocation { + group: end.checked_sub(1)?, + object: None, + }) }), }, fill: None, @@ -12378,69 +12265,28 @@ fn subscribe_join( }) } -/// Resolve the first absolute ID or a later ID delta without overflowing. -/// -/// Ordinary subgroup relay ingest preserves gaps. A fill or resume supplies `start` -/// and requires contiguous IDs because its public frame ordinals stitch onto another copy. -fn next_object_id(prior: Option, delta: u64, start: Option) -> Result { - let object = match prior { - None => delta, +/// Resolve the first absolute ID or a later ID delta without overflowing. Subgroup relay +/// ingest preserves gaps. +fn next_object_id(prior: Option, delta: u64) -> Result { + match prior { + None => Ok(delta), Some(prior) => prior .checked_add(delta) .and_then(|id| id.checked_add(1)) - .ok_or(Error::Session(SessionError::ProtocolViolation))?, - }; - - let expected = prior.map_or(start.unwrap_or(object), |prior| prior.saturating_add(1)); - if start.is_some() && object != expected { - tracing::warn!( - object, - expected, - "object IDs must start at the group's start and increment by 1" - ); - return Err(Error::Unsupported); + .ok_or(Error::Session(SessionError::ProtocolViolation)), } - - Ok(object) } #[cfg(test)] mod object_id_tests { use super::*; - /// A zero delta throughout is a group numbered from 0 with no gaps, which is the only - /// shape moq-lite can represent. - #[test] - fn accepts_sequential_ids_from_zero() { - let mut prior = None; - for expected in 0..4 { - let object = next_object_id(prior, 0, Some(0)).expect("sequential"); - assert_eq!(object, expected); - prior = Some(object); - } - } - - /// The first object's delta is its absolute Object ID, so a non-zero one means the - /// group starts partway through and has a hole at the front. - #[test] - fn rejects_a_group_that_does_not_start_at_zero() { - assert!(matches!(next_object_id(None, 6, Some(0)), Err(Error::Unsupported))); - } - - /// The tail of a stitched group starts where the fill's head stopped, and nowhere else. - #[test] - fn accepts_a_tail_that_starts_where_the_fill_stopped() { - assert_eq!(next_object_id(None, 6, Some(6)).expect("the fill's next object"), 6); - assert_eq!(next_object_id(Some(6), 0, Some(6)).expect("then sequential"), 7); - assert!(matches!(next_object_id(None, 5, Some(6)), Err(Error::Unsupported))); - assert!(matches!(next_object_id(None, 7, Some(6)), Err(Error::Unsupported))); - } - - /// A later delta skips objects, which would renumber every frame after it. + /// The first delta is the absolute Object ID and each later one counts past the prior. #[test] - fn rejects_a_gap() { - assert!(matches!(next_object_id(Some(0), 1, Some(0)), Err(Error::Unsupported))); - assert!(matches!(next_object_id(Some(3), 9, Some(0)), Err(Error::Unsupported))); + fn resolves_ids_and_gaps() { + assert_eq!(next_object_id(None, 6).unwrap(), 6); + assert_eq!(next_object_id(Some(6), 0).unwrap(), 7); + assert_eq!(next_object_id(Some(6), 2).unwrap(), 9); } /// The running ID is bounded, and the draft makes an overflow a protocol violation @@ -12448,7 +12294,7 @@ mod object_id_tests { #[test] fn rejects_an_overflow() { assert!(matches!( - next_object_id(Some(u64::MAX), 0, Some(0)), + next_object_id(Some(u64::MAX), 0), Err(Error::Session(SessionError::ProtocolViolation)) )); } @@ -12481,12 +12327,7 @@ mod filter_tests { #[test] fn a_past_start_is_absolute() { assert_eq!( - subscribe_join( - Some(track::Position::group(7)), - track::Position::after_group(9), - Version::Draft20, - ) - .unwrap(), + subscribe_join(Some(7), Some(10), Version::Draft20).unwrap(), Join { filter: Filter::Absolute { start: ietf::Location { group: 7, object: 0 }, @@ -12502,7 +12343,7 @@ mod filter_tests { #[test] fn the_whole_track_is_unfiltered() { assert_eq!( - subscribe_join(Some(track::Position::group(0)), None, Version::Draft20).unwrap(), + subscribe_join(Some(0), None, Version::Draft20).unwrap(), Join { filter: Filter::Unfiltered, fill: None, @@ -12526,11 +12367,7 @@ mod filter_tests { #[test] fn older_drafts_always_join_live() { for version in JOINING_DRAFTS { - for start in [ - None, - Some(track::Position::group(7)), - Some(track::Position { group: 7, frame: 1 }), - ] { + for start in [None, Some(7)] { assert_eq!( subscribe_join(start, None, version).unwrap(), Join { @@ -12549,14 +12386,7 @@ mod filter_tests { fn older_drafts_refuse_a_bounded_end() { for version in JOINING_DRAFTS { assert!( - matches!( - subscribe_join( - Some(track::Position::group(7)), - track::Position::after_group(9), - version - ), - Err(Error::Unsupported) - ), + matches!(subscribe_join(Some(7), Some(10), version), Err(Error::Unsupported)), "{version}" ); } @@ -12840,15 +12670,6 @@ mod stitch_tests { self } - /// Resume the subscription partway through a group another route delivered the - /// head of. - fn with_resume(self, resume: track::Position) -> Self { - if let Some(track) = self.subscriber.state.lock().subscribes.get_mut(&REQUEST) { - track.resume = Some(resume); - } - self - } - /// Run the next scripted stream through `recv_group` and the dispatcher's handling of /// its result, asserting that stream alone was refused with STOP_SENDING. async fn dispatch_refused(&self) { @@ -13654,7 +13475,7 @@ mod stitch_tests { let mut run = GroupFetchRun::new(version, objects).await; let fill = h .subscriber - .run_fill(&mut run.stream, &mut h.track, None, None) + .run_fill(&mut run.stream, &mut h.track, None) .await .unwrap(); *h.fill.write().ok().expect("fill open") = fill; @@ -14721,59 +14542,6 @@ mod stitch_tests { assert_eq!(frames.len(), 1, "the group is whatever one producer wrote, not both"); } - /// A resumed subscription that gets its group whole (a pre-draft-20 join can only ask - /// for all of it) keeps only the objects from where it asked: another route already - /// delivered the rest. - #[moq_clock::test] - async fn a_resumed_whole_group_drops_the_delivered_head() { - let h = Harness::new(Fill::Done, vec![tail_stream(SEQUENCE, 0, &[b"0", b"1", b"2"])]).with_resume( - track::Position { - group: SEQUENCE, - frame: 2, - }, - ); - let mut consumer = h.track.subscribe(None); - let mut whole = h.stream().await; - h.subscriber.clone().recv_group(&mut whole).await.expect("the group"); - - let mut group = consumer.recv_group().await.unwrap().expect("the group arrived"); - group.start_at(0); - assert_eq!(group.index(), 2, "the group starts where the subscription asked"); - assert_eq!(&group.read_frame().await.unwrap().unwrap().payload[..], b"2"); - assert!(group.read_frame().await.unwrap().is_none()); - } - - /// The same for a joining FETCH's head: the objects below the resume point are - /// dropped, and the live tail continues what is left. - #[moq_clock::test] - async fn a_resumed_fill_drops_the_delivered_head() { - let h = Harness::new( - Fill::Serving(Some(Timescale::MICRO)), - vec![ - fill_stream(SEQUENCE, &[b"0", b"1", b"2"]), - tail_stream(SEQUENCE, 3, &[b"3"]), - ], - ) - .with_resume(track::Position { - group: SEQUENCE, - frame: 2, - }); - let mut consumer = h.track.subscribe(None); - let mut fill = h.stream().await; - let mut tail = h.stream().await; - h.subscriber.clone().recv_fill(&mut fill).await.expect("fill"); - h.subscriber.clone().recv_group(&mut tail).await.expect("tail"); - - let mut group = consumer.recv_group().await.unwrap().expect("the group arrived"); - group.start_at(0); - assert_eq!(group.index(), 2, "the group starts where the subscription asked"); - let mut frames = Vec::new(); - while let Some(frame) = group.read_frame().await.unwrap() { - frames.push(frame.payload.to_vec()); - } - assert_eq!(frames, [b"2".to_vec(), b"3".to_vec()]); - } - /// A standalone subgroup tail keeps its identity while plain readers see its payload. #[moq_clock::test] async fn a_tail_without_a_fill_is_preserved() { @@ -15612,13 +15380,13 @@ mod joining_fetch_tests { } impl JoinRun { - async fn start(version: Version, start: Option, ok: Vec, fetch: Vec) -> Self { + async fn start(version: Version, start: Option, ok: Vec, fetch: Vec) -> Self { Self::start_with_control(version, start, [ok, fetch], Control::new(None, false)).await } async fn start_with_control( version: Version, - start: Option, + start: Option, scripts: [Vec; 2], control: Control, ) -> Self { @@ -16010,13 +15778,7 @@ mod joining_fetch_tests { let largest = Some(ietf::Location { group: 9, object: 0 }); for version in JOINING_DRAFTS { for start in [7, 9, 11] { - let run = JoinRun::start( - version, - Some(track::Position::group(start)), - subscribe_ok(version, largest), - fetch_ok(version), - ) - .await; + let run = JoinRun::start(version, Some(start), subscribe_ok(version, largest), fetch_ok(version)).await; let messages = decode_messages(&run.session.log, version); let fetch = messages.iter().find(|(id, _)| *id == ietf::Fetch::ID).expect("FETCH"); @@ -16503,8 +16265,13 @@ mod joining_fetch_tests { ); let dynamic = track.dynamic(); let consumer = track.consume(); - let mut fetch = - std::pin::pin!(consumer.fetch_group(GROUP, group::Fetch::default().with_frame_start(START))); + let mut fetch = std::pin::pin!(consumer.fetch_group( + GROUP, + group::Fetch { + frame_start: START, + ..Default::default() + } + )); assert!(futures::poll!(fetch.as_mut()).is_pending()); let request = dynamic.requested_group().await.expect("no group requested"); @@ -16644,7 +16411,13 @@ mod joining_fetch_tests { ); let dynamic = track.dynamic(); let consumer = track.consume(); - let mut fetch = std::pin::pin!(consumer.fetch_group(GROUP, group::Fetch::default().with_frame_start(START))); + let mut fetch = std::pin::pin!(consumer.fetch_group( + GROUP, + group::Fetch { + frame_start: START, + ..Default::default() + } + )); assert!(futures::poll!(fetch.as_mut()).is_pending()); let request = dynamic.requested_group().await.expect("no group requested"); diff --git a/rs/moq-net/src/lite/ietf.rs b/rs/moq-net/src/lite/ietf.rs index 90b2d9aac69..fae719f149a 100644 --- a/rs/moq-net/src/lite/ietf.rs +++ b/rs/moq-net/src/lite/ietf.rs @@ -69,7 +69,6 @@ pub(crate) fn decode_state(state: u64, frame_start: u64) -> Result) -> Result<(), crate::Error> { @@ -1525,13 +1466,11 @@ mod hop_tests { .unwrap(); assert_eq!(next(&mut fetched).await.unwrap().timestamp, negotiated.then(|| ts(100))); assert!(next(&mut fetched).await.is_none()); - let mut subscriber = moq_clock::timeout( - TIMEOUT, - copy.subscribe(track::Subscription::default().with_start(track::Position::group(1))), - ) - .await - .unwrap() - .unwrap(); + let mut subscriber = + moq_clock::timeout(TIMEOUT, copy.subscribe(track::Subscription::default().with_start(1))) + .await + .unwrap() + .unwrap(); // A peer may have stamped arrival times. The second group's rewind must not // acquire native moq-lite checks merely because it crossed a relay hop. let mut group = track.create_group(1u64.into()).unwrap(); diff --git a/rs/moq-net/src/lite/publisher.rs b/rs/moq-net/src/lite/publisher.rs index 561a8f475e7..7449d3ee0d9 100644 --- a/rs/moq-net/src/lite/publisher.rs +++ b/rs/moq-net/src/lite/publisher.rs @@ -2090,7 +2090,7 @@ impl Request for SubscribeServe { priority: msg.priority, max_delay: serving_max_delay(shared.version, msg.max_delay), order: msg.order, - ..Bounds::from(&msg).positions() + ..Bounds::from(&msg).groups() }; // One subscriber for the whole subscription: the run loop polls its groups and its @@ -2542,7 +2542,7 @@ mod test { let mut producer = track_producer("test"); let mut subscriber = producer.subscribe( track::Subscription::default() - .with_start(track::Position::group(0)) + .with_start(0) .with_max_delay(std::time::Duration::from_secs(60)), ); @@ -3548,25 +3548,17 @@ struct Bounds { } impl Bounds { - /// The requested range as the model's half-open pair of [`track::Position`]s. + /// The whole groups the requested range touches, as the model's half-open range. /// - /// The wire carries the two halves of each bound separately and both ends - /// inclusive; the model carries whole positions with an exclusive end. The - /// [`Subscription`] builders own that conversion, so this goes through them - /// rather than repeating it. - fn positions(&self) -> crate::track::Subscription { + /// A model subscription always starts and ends on a group boundary, so a frame bound + /// widens to its whole group here and [`position_group`] applies it when serving. + fn groups(&self) -> crate::track::Subscription { let mut sub = crate::track::Subscription::default(); if let Some(group) = self.start_group { - sub = sub.with_start(track::Position { - group, - frame: self.start_frame, - }); + sub = sub.with_start(group); } if let Some(group) = self.end_group { - sub = sub.with_end(match self.end_frame { - Some(frame) => track::Position::after(group, frame), - None => track::Position::after_group(group), - }); + sub = sub.with_end(group.checked_add(1)); } sub } @@ -3879,14 +3871,14 @@ impl TrackRun { // Feed the full update into the model subscriber so the producer's // aggregate reflects it (and a relay re-forwards it upstream). // Read first: `update` replaces these preferences. - let requested = self.track.subscription().start().map(|start| start.group); + let requested = self.track.subscription().start(); let floored = requested.is_some_and(|group| group > 0); let bounds = Bounds::from(&upd); let _ = self.track.update(crate::track::Subscription { priority: upd.priority, max_delay: serving_max_delay(self.ctx.version, upd.max_delay), order: upd.order, - ..bounds.positions() + ..bounds.groups() }); // An explicit start moves the read cursor. Lite-06+ encodes an absent // start as no floor, so a subscription that had one above group 0 has to @@ -4007,14 +3999,20 @@ impl TrackRun { && self.ctx.version.has_largest() && let Some(start) = self.track.subscription().start() && let Poll::Ready(Some(largest)) = self.track.poll_live(waiter) - && start > largest + && (track::Position { + group: start, + frame: self + .start_frame + .filter(|(group, _)| *group == start) + .map_or(0, |(_, frame)| frame), + }) > largest { - self.start = Some(start.group); + self.start = Some(start); writer.buffer(&lite::SubscribeResponse::Start(lite::SubscribeStart { - group: start.group, + group: start, largest: Some(largest), }))?; - self.track.raise_start_to(start.group); + self.track.raise_start_to(start); return Poll::Ready(Ok(ControlFlow::Continue(()))); } @@ -4120,6 +4118,14 @@ impl TrackRun { return; } let sequence = datagram.sequence; + // A Frame Start inside the start group excludes its datagram, the group's only frame. + // The model bounds whole groups, so the frame bound is applied here. + if self + .start_frame + .is_some_and(|(group, frame)| group == sequence && frame > 0) + { + return; + } // Metered in the form the wire carries. let (payload, _) = datagram.sent(self.ctx.version.has_compression()); self.track.meter_datagram(payload.len() as u64); @@ -4176,7 +4182,8 @@ impl TrackRun { return Ok(()); }; // A cap below the end bounds what the subscriber is owed, as on its side. - let end = final_sequence(self.track.subscription().end().map_or(fin, |end| end.min(fin))); + let fin = final_sequence(fin); + let end = self.track.subscription().end().map_or(fin, |end| end.min(fin)); self.drop_range(start..end, &StreamError::NotFound, writer) } } @@ -4723,26 +4730,23 @@ mod serve_group_test { use crate::{Timestamp, broadcast}; use futures::FutureExt; - /// The wire's inclusive pair maps to the model's exclusive end. - /// - /// The inverse of the subscriber's `WireBounds`, and the same off-by-one risk: a - /// whole end group becomes the head of the next one, a capped frame becomes the head - /// of the frame above it. + /// The wire's inclusive bounds map to the model's whole groups with an exclusive end: a + /// frame bound widens to its group, which `position_group` cuts when serving. #[test] - fn bounds_convert_to_positions() { + fn bounds_convert_to_whole_groups() { let whole = Bounds { start_group: None, start_frame: 0, end_group: Some(5), end_frame: None, }; - assert_eq!(whole.positions().end(), Some(track::Position::group(6))); + assert_eq!(whole.groups().end(), Some(6)); let capped = Bounds { end_frame: Some(2), ..whole }; - assert_eq!(capped.positions().end(), Some(track::Position { group: 5, frame: 3 })); + assert_eq!(capped.groups().end(), Some(6)); let started = Bounds { start_group: Some(5), @@ -4750,9 +4754,7 @@ mod serve_group_test { end_group: None, end_frame: None, }; - let positions = started.positions(); - assert_eq!(positions.start(), Some(track::Position { group: 5, frame: 3 })); - assert_eq!(positions.end(), None); + assert_eq!((started.groups().start(), started.groups().end()), (Some(5), None)); // A frame bound the peer sent without its group has nothing to count from, so it // cannot reach the model at all. @@ -4762,7 +4764,7 @@ mod serve_group_test { end_group: None, end_frame: Some(7), }; - assert_eq!((orphan.positions().start(), orphan.positions().end()), (None, None)); + assert_eq!((orphan.groups().start(), orphan.groups().end()), (None, None)); } /// A group whose head the publisher no longer holds is skipped, not served short: @@ -5654,6 +5656,50 @@ mod serve_group_test { assert!(order > 1); } + /// A Frame Start inside the start group excludes that group's datagram, its only frame, + /// while a later group's datagram is served. + #[moq_clock::test] + async fn a_frame_start_drops_the_start_groups_datagram() { + let mut track = track::Producer::new( + Arc::new(broadcast::Info::default()), + "test", + crate::track::Info::timed(), + ); + let subscriber = track.subscribe(track::Subscription::default().with_max_delay(Duration::from_secs(3600))); + let mut session = SinkSession::new(Log::default()).with_groups().with_datagrams(1200); + let log = session.log.clone(); + let (mut run, mut writer, _) = lite_run(Version::Lite06, session.clone(), subscriber); + run.ctx.group = crate::transport::poll::Session::send_group(&mut session, 3); + run.update(lite::SubscribeUpdate { + wanted_group: None, + order: Order::Descending, + priority: 0, + max_delay: Duration::from_secs(3600), + start_group: Some(5), + end_group: None, + start_frame: 1, + end_frame: None, + }); + + for sequence in [5, 6] { + track + .insert_datagram( + sequence, + Timestamp::from_millis(sequence).unwrap(), + bytes::Bytes::from_static(b"d"), + ) + .unwrap(); + } + { + let mut poll = std::pin::pin!(kio::wait(|waiter| run.poll(&mut writer, waiter))); + assert!(futures::poll!(poll.as_mut()).is_pending()); + } + assert_eq!( + log.datagram_ranks(), + [(Some(0), crate::priority::in_group(6, 0, Order::Descending))] + ); + } + /// A SUBSCRIBE_UPDATE that flips the group order reranks the next datagram, even with no /// group served in between. #[moq_clock::test] @@ -5959,7 +6005,7 @@ mod serve_group_test { crate::track::Info::timed(), ); let subscription = track::Subscription::default() - .with_start(track::Position::group(5)) + .with_start(5) .with_max_delay(Duration::from_secs(60)); let subscriber = track.subscribe(subscription); let (mut run, mut writer, log) = lite_run(version, SinkSession::new(Log::default()), subscriber); @@ -6333,7 +6379,7 @@ mod serve_group_test { crate::track::Info::timed(), ); let subscription = track::Subscription::default() - .with_start(track::Position::group(5)) + .with_start(5) .with_max_delay(Duration::from_secs(60)); let subscriber = track.subscribe(subscription); let (mut run, mut writer, log) = lite_run(version, SinkSession::new(Log::default()), subscriber); @@ -6379,7 +6425,7 @@ mod serve_group_test { crate::track::Info::timed(), ); let subscription = track::Subscription::default() - .with_start(track::Position::group(5)) + .with_start(5) .with_max_delay(Duration::from_secs(1)); let subscriber = track.subscribe(subscription); let (mut run, mut writer, log) = lite_run(version, SinkSession::new(Log::default()), subscriber); @@ -6426,7 +6472,7 @@ mod serve_group_test { crate::track::Info::timed(), ); let subscription = track::Subscription::default() - .with_start(track::Position::group(0)) + .with_start(0) .with_max_delay(Duration::from_secs(1)); let subscriber = track.subscribe(subscription); let (mut run, mut writer, log) = lite_run(version, SinkSession::new(Log::default()), subscriber); @@ -6473,7 +6519,7 @@ mod serve_group_test { crate::track::Info::timed(), ); let subscription = track::Subscription::default() - .with_start(track::Position::group(0)) + .with_start(0) .with_max_delay(Duration::from_secs(10)); let subscriber = track.subscribe(subscription); let (mut run, mut writer, log) = lite_run(version, SinkSession::new(Log::default()), subscriber); @@ -6517,7 +6563,7 @@ mod serve_group_test { ); let subscriber = track.subscribe( track::Subscription::default() - .with_start(track::Position::group(0)) + .with_start(0) .with_max_delay(Duration::from_secs(60)), ); let (mut run, mut writer, _log) = lite_run(Version::Lite05, SinkSession::new(Log::default()), subscriber); @@ -6558,7 +6604,7 @@ mod serve_group_test { ); let subscriber = track.subscribe( track::Subscription::default() - .with_start(track::Position::group(5)) + .with_start(5) .with_max_delay(Duration::from_secs(60)), ); let (mut run, mut writer, _log) = lite_run(Version::Lite05, SinkSession::new(Log::default()), subscriber); @@ -6626,7 +6672,7 @@ mod serve_group_test { fn new(track: &mut track::Producer, start_group: u64) -> Self { track.request_start(Some(0)).unwrap(); let subscription = track::Subscription::default() - .with_start(track::Position::group(start_group)) + .with_start(start_group) .with_max_delay(Duration::from_secs(30)); let subscriber = track.subscribe(subscription); diff --git a/rs/moq-net/src/lite/subscriber.rs b/rs/moq-net/src/lite/subscriber.rs index 09e9da34168..9de0faf69fd 100644 --- a/rs/moq-net/src/lite/subscriber.rs +++ b/rs/moq-net/src/lite/subscriber.rs @@ -2205,7 +2205,7 @@ mod tests { let lowered = Some(Subscription::default().with_groups(2..)); // The SUBSCRIBE in force, already answered. - let floor = Position::group(if update { 5 } else { 2 }); + let floor = if update { 5 } else { 2 }; let mut sub = Sub::Active(SubStream { order: crate::track::Order::Descending, stream: Stream::open(&mut h.session.clone(), Version::Lite04).await.unwrap(), @@ -2612,9 +2612,9 @@ mod tests { stream, id: 0, max_delay: Duration::ZERO, - start: Some(Position::group(2)), + start: Some(2), priority: 0, - requested: Some(Position::group(2)), + requested: Some(2), tail: Default::default(), served: None, end: Some(lite::SubscribeEnd { group: 10 }), @@ -2632,9 +2632,9 @@ mod tests { Some(2..10), "the groups below START are accounted for on arrival" ); - sub.start = Some(Position::group(6)); + sub.start = Some(6); assert_eq!(sub.owed(None), Some(6..10), "a raised floor owes nothing below it"); - sub.start = Some(Position::group(1)); + sub.start = Some(1); assert_eq!( sub.owed(None), Some(1..10), @@ -4270,164 +4270,22 @@ mod tests { } } - /// Mid-group demand: a subscriber resuming at frame 3 of group 5, capped by a later - /// bound in the same group. - fn mid_group_demand() -> Subscription { - Subscription::default() - .with_start(Position { group: 5, frame: 3 }) - .with_end(Position::after(5, 7)) - } - - /// A mid-group resume boundary handed to a peer that predates lite-06 is widened to - /// the whole group rather than refused. - /// - /// The codec rejects a frame bound such a peer cannot carry, so passing the demand - /// through unchanged fails the SUBSCRIBE and hands the track back. The origin then - /// asks the same route again indefinitely, since asking keeps succeeding and the - /// retry budget never trips. - #[moq_clock::test] - async fn frame_bounds_widen_for_an_older_peer() { - let mut h = Harness::new(Version::Lite05); - let mut sub = Sub::None; - - h.serve - .handle_subscription(&mut h.producer, &mut sub, Some(mid_group_demand()), true) - .await - .expect("an older peer must not fail the subscribe"); - - let wire = h.wire(); - let mut wire = wire.as_slice(); - assert_eq!( - crate::coding::decode_buf(&mut wire, Version::Lite05, lite::ControlType::decode).unwrap(), - lite::ControlType::Subscribe - ); - let msg = crate::coding::decode_buf(&mut wire, Version::Lite05, lite::Subscribe::decode).unwrap(); - // The group bounds survive; only the frame offsets are widened away. - assert_eq!((msg.start_group, msg.end_group), (Some(5), Some(5))); - assert_eq!((msg.start_frame, msg.end_frame), (0, None)); - } - - /// A lite-07 SUBSCRIBE keeps a mid-group start whose prefix the copy holds: its answer - /// carries the largest position, which says whether what the subscriber holds is - /// current. Lite-06's answer does not, so it asks from the head of the group instead, - /// and the first frame says where the live feed is; its end keeps the frame bound. - #[moq_clock::test] - async fn a_mid_group_start_survives_only_where_the_answer_has_the_largest() { - for (version, start_frame) in [(Version::Lite07, 3), (Version::Lite06, 0)] { - let mut h = Harness::new(version); - let mut group = h.producer.create_group(crate::group::Info { sequence: 5 }).unwrap(); - for frame in 0..3 { - group - .write_frame(crate::Timestamp::from_millis(frame).unwrap(), b"prefix".as_ref()) - .unwrap(); - } - let mut sub = Sub::None; - - h.serve - .handle_subscription(&mut h.producer, &mut sub, Some(mid_group_demand()), true) - .await - .unwrap(); - - let wire = h.wire(); - let mut wire = wire.as_slice(); - assert_eq!( - crate::coding::decode_buf(&mut wire, version, lite::ControlType::decode).unwrap(), - lite::ControlType::Subscribe - ); - let msg = crate::coding::decode_buf(&mut wire, version, lite::Subscribe::decode).unwrap(); - assert_eq!((msg.start_frame, msg.end_frame), (start_frame, Some(7)), "{version:?}"); - } - } - - /// A partial prefix, a tail, or a released group lacks frames below a mid-group start. - /// Only a complete live prefix permits requesting the tail without widening its start. - #[moq_clock::test] - async fn frame_bounds_require_the_entire_cached_prefix() { - for (first, count, aborted, expected) in [ - (0, 0, false, 0), - (0, 2, false, 0), - (0, 3, false, 3), - (1, 3, false, 0), - (0, 3, true, 0), - ] { - let mut h = Harness::new(Version::Lite07); - let mut group = h.producer.create_group(crate::group::Info { sequence: 5 }).unwrap(); - group.start_at(first).unwrap(); - for frame in first..first + count { - group - .write_frame(crate::Timestamp::from_millis(frame).unwrap(), b"prefix".as_ref()) - .unwrap(); - } - if aborted { - group.abort(Error::Cancel).unwrap(); - } else { - group.finish().unwrap(); - } - let mut sub = Sub::None; - h.serve - .handle_subscription(&mut h.producer, &mut sub, Some(mid_group_demand()), true) - .await - .unwrap(); - let wire = h.wire(); - let mut wire = wire.as_slice(); - assert_eq!( - crate::coding::decode_buf(&mut wire, Version::Lite07, lite::ControlType::decode).unwrap(), - lite::ControlType::Subscribe - ); - let msg = crate::coding::decode_buf(&mut wire, Version::Lite07, lite::Subscribe::decode).unwrap(); - assert_eq!( - (msg.start_group, msg.start_frame, msg.end_group, msg.end_frame), - (Some(5), expected, Some(5), Some(7)), - "first={first}, count={count}, aborted={aborted}" - ); - } - } - - /// The model's exclusive end maps back to the wire's inclusive pair. - /// - /// The two disagree deliberately (see [`Position::after_group`]), so this pins the seam: - /// an end at the head of a group means the group below it, served whole, while one - /// mid-group caps the frame below it. Off by one here would silently drop or - /// duplicate a frame at every relay hop. + /// The model's exclusive group end maps back to the wire's inclusive one, and every + /// SUBSCRIBE asks from a group's first frame. Off by one here would silently drop or + /// duplicate a group at every relay hop. #[test] fn wire_bounds_convert_the_exclusive_end() { - // The whole of group 5 is the head of group 6. - let bounds = WireBounds::new(None, Some(Position::group(6))); + let bounds = WireBounds::new(Some(5), Some(6)); + assert_eq!((bounds.start_group, bounds.start_frame), (Some(5), 0)); assert_eq!((bounds.end_group, bounds.end_frame), (Some(5), None)); - // Group 5 through frame 2 is the head of frame 3. - let bounds = WireBounds::new(None, Some(Position { group: 5, frame: 3 })); - assert_eq!((bounds.end_group, bounds.end_frame), (Some(5), Some(2))); - - // Unbounded stays unbounded. let bounds = WireBounds::new(None, None); assert_eq!((bounds.end_group, bounds.end_frame), (None, None)); - - // Starts are inclusive on both sides, so they pass straight through. - let bounds = WireBounds::new(Some(Position { group: 5, frame: 3 }), None); - assert_eq!((bounds.start_group, bounds.start_frame), (Some(5), 3)); - } - - /// The builders produce exactly what the wire conversion expects, so an inclusive - /// bound survives the trip out to a peer unchanged. - #[test] - fn wire_bounds_match_the_builders() { - let whole = Subscription::default().with_end(Position::after_group(5)); - let bounds = WireBounds::new(whole.start(), whole.end()); - assert_eq!((bounds.end_group, bounds.end_frame), (Some(5), None)); - - let capped = Subscription::default().with_end(Position::after(5, 2)); - let bounds = WireBounds::new(capped.start(), capped.end()); - assert_eq!((bounds.end_group, bounds.end_frame), (Some(5), Some(2))); - - let started = Subscription::default().with_start(Position { group: 5, frame: 3 }); - let bounds = WireBounds::new(started.start(), started.end()); - assert_eq!((bounds.start_group, bounds.start_frame), (Some(5), 3)); } /// A subscription that asks for nothing opens nothing. /// - /// `Position::group(0)` is the empty range: nothing sorts below it. The wire cannot + /// An end of group 0 is the empty range: nothing sorts below it. The wire cannot /// say that, and the nearest thing it can say is "through group 0", which would /// deliver the single group the caller excluded. #[moq_clock::test] @@ -4435,7 +4293,7 @@ mod tests { let mut h = Harness::new(Version::Lite06); let mut sub = Sub::None; - let empty = Subscription::default().with_end(Position::group(0)); + let empty = Subscription::default().with_end(0); h.serve .handle_subscription(&mut h.producer, &mut sub, Some(empty), true) .await @@ -4459,7 +4317,7 @@ mod tests { assert!(matches!(sub, Sub::Active(_)), "the first subscriber opens one"); let established = h.wire().len(); - let empty = Subscription::default().with_end(Position::group(0)); + let empty = Subscription::default().with_end(0); h.serve .handle_subscription(&mut h.producer, &mut sub, Some(empty), true) .await @@ -4480,9 +4338,7 @@ mod tests { let mut h = Harness::new(Version::Lite06); let mut sub = Sub::None; - let empty = Subscription::default() - .with_start(Position::group(5)) - .with_end(Position::group(5)); + let empty = Subscription::default().with_start(5).with_end(5); h.serve .handle_subscription(&mut h.producer, &mut sub, Some(empty), true) .await @@ -4506,9 +4362,7 @@ mod tests { assert!(matches!(sub, Sub::Active(_)), "the first subscriber opens one"); let established = h.wire().len(); - let empty = Subscription::default() - .with_start(Position::group(5)) - .with_end(Position::group(5)); + let empty = Subscription::default().with_start(5).with_end(5); h.serve .handle_subscription(&mut h.producer, &mut sub, Some(empty), true) .await @@ -4518,57 +4372,6 @@ mod tests { assert_eq!(h.wire().len(), established, "no SUBSCRIBE_UPDATE inverting the range"); } - /// Widening rounds the end outward at the last group too, keeping the range at least - /// as wide as the request. - /// - /// Rounding inward there would empty a range the caller asked for, and the filter in - /// `handle_subscription` runs before the widening, so nothing downstream would catch - /// it. - #[moq_clock::test] - async fn frame_bounds_widen_outward_at_the_last_group() { - let h = Harness::new(Version::Lite05); - - let mut subscription = Subscription::default() - .with_start(Position { - group: u64::MAX, - frame: 1, - }) - .with_end(Position::after(u64::MAX, 5)); - h.serve.widen_frame_bounds(&mut subscription); - - assert_eq!(subscription.start(), Some(Position::group(u64::MAX))); - // Past the last group there is no position to round up to, and unbounded is the - // wider request. - assert_eq!(subscription.end(), None); - } - - /// The widening covers SUBSCRIBE_UPDATE too: a downstream peer asking for a frame - /// offset must not tear down an older upstream that is already serving. - #[moq_clock::test] - async fn frame_bounds_widen_on_update() { - let mut h = Harness::new(Version::Lite05); - let mut sub = Sub::None; - - h.serve - .handle_subscription(&mut h.producer, &mut sub, Some(Subscription::default()), true) - .await - .unwrap(); - let established = h.wire().len(); - - // A lite-06 subscriber downstream now wants to resume mid-group. - h.serve - .handle_subscription(&mut h.producer, &mut sub, Some(mid_group_demand()), true) - .await - .expect("a downstream frame offset must not tear down an older upstream"); - - // SUBSCRIBE_UPDATE rides the subscribe stream with no control type ahead of it. - let wire = h.wire(); - let mut wire = &wire[established..]; - let msg = crate::coding::decode_buf(&mut wire, Version::Lite05, lite::SubscribeUpdate::decode).unwrap(); - assert_eq!((msg.start_group, msg.start_frame), (Some(5), 0)); - assert_eq!((msg.end_group, msg.end_frame), (Some(5), None)); - } - /// Concurrent joins over the refresh ratio send one SUBSCRIBE_UPDATE upstream, a new /// group past the largest sends the next, and every update restates the latest. #[moq_clock::test] @@ -4631,7 +4434,7 @@ mod tests { let mut h = Harness::new(Version::Lite05); let mut sub = Sub::None; - let demand = |group: u64| Some(Subscription::default().with_start(Position::group(group))); + let demand = |group: u64| Some(Subscription::default().with_start(group)); let applies = |sub: &Sub| matches!(sub, Sub::Active(active) if active.start == active.requested); // Establish from group 3; the peer's START is considered in flight. @@ -4646,11 +4449,7 @@ mod tests { .handle_subscription( &mut h.producer, &mut sub, - Some( - Subscription::default() - .with_start(Position::group(3)) - .with_end(Position::group(9)), - ), + Some(Subscription::default().with_start(3).with_end(9)), true, ) .await @@ -4682,7 +4481,7 @@ mod tests { let mut h = Harness::new(Version::Lite05); let mut sub = Sub::None; - let demand = |group: u64| Some(Subscription::default().with_start(Position::group(group))); + let demand = |group: u64| Some(Subscription::default().with_start(group)); // Establish from group 5: the floor tracks the request until START lands. h.serve @@ -5807,10 +5606,10 @@ mod tests { /// The four wire fields a subscription's half-open range encodes to. /// -/// The inverse of the publisher's `Bounds::positions`: the model carries whole -/// positions with an exclusive end, while the wire splits each bound into a group and a -/// frame and states both ends inclusive. The range must be non-empty, since an inclusive -/// end has nothing to say below the first position it excludes. +/// The model carries whole groups with an exclusive end, while the wire states both ends +/// inclusive. A subscription always starts and ends at a group boundary, so both frame +/// fields stay at their whole-group defaults. The range must be non-empty, since an +/// inclusive end has nothing to say below the first group it excludes. struct WireBounds { start_group: Option, start_frame: u64, @@ -5819,7 +5618,7 @@ struct WireBounds { } impl WireBounds { - fn new(start: Option, end: Option) -> Self { + fn new(start: Option, end: Option) -> Self { // An empty range has no wire encoding: flooring its end below asks for the // position it excludes, or inverts the range. `handle_subscription` drops such a // subscription instead. An absent start is the live edge, so it only makes the @@ -5829,19 +5628,11 @@ impl WireBounds { "an empty range cannot be encoded; it should have been dropped as no demand" ); - let (end_group, end_frame) = match end { - // An exclusive end at the head of a group means the group below it is the - // last one, and it is served whole. - Some(end) if end.frame == 0 => (Some(end.group.saturating_sub(1)), None), - Some(end) => (Some(end.group), Some(end.frame - 1)), - None => (None, None), - }; - Self { - start_group: start.map(|start| start.group), - start_frame: start.map_or(0, |start| start.frame), - end_group, - end_frame, + start_group: start, + start_frame: 0, + end_group: end.map(|end| end.saturating_sub(1)), + end_frame: None, } } } @@ -5854,7 +5645,7 @@ struct SubStream { /// Original SUBSCRIBE params, echoed in every SUBSCRIBE_UPDATE; refreshed as the /// downstream aggregate changes. max_delay: Duration, - start: Option, + start: Option, priority: i8, order: crate::track::Order, /// The start the SUBSCRIBE itself carried, fixed for the stream's life. A @@ -5862,7 +5653,7 @@ struct SubStream { /// so a buffered START only applies while `start` still equals it: while the /// start sits elsewhere the request-tracked floor stands instead, and demand /// returning here makes the declaration valid again. - requested: Option, + requested: Option, /// The groups received for this subscription, shared with its [`TrackEntry`]. tail: kio::Producer, /// The first group the publisher serves (SUBSCRIBE_START), once declared. @@ -5885,7 +5676,7 @@ impl SubStream { let end = self.end.as_ref()?.group; let end = requested_end.map_or(end, |requested| requested.min(end)); let start = match self.served { - Some(served) => self.start.map_or(served, |start| start.group), + Some(served) => self.start.unwrap_or(served), None => end, }; Some(start..end) @@ -6236,43 +6027,6 @@ impl TrackServe { ) } - /// The mid-group start a peer without frame bounds can't be asked for, recorded so - /// the frames below it are dropped when its whole group arrives. - fn widen_frame_bounds(&self, subscription: &mut Subscription) { - // An answer that does not carry the largest position cannot vouch for what a - // subscriber holds, so ask from the head of its group instead: the first frame then - // arrives at once and says where the live feed is. - if !self.subscriber.version.has_largest() { - subscription.set_bounds( - subscription.start().map(|start| Position::group(start.group)), - subscription.end(), - ); - } - if self.subscriber.version.has_frame_bounds() { - return; - } - - // Round both bounds outward to the enclosing group, so the peer sends at least - // what was asked for and never less. Rounding the end down instead would be able - // to empty a non-empty range, which has no wire encoding at all. - let start = subscription.start().map(|start| Position::group(start.group)); - let end = subscription.end().and_then(|end| match end.frame { - 0 => Some(end), - // Past the last group there is no position to round up to, and unbounded is - // the wider request. - _ => Position::after_group(end.group), - }); - - if (start, end) != (subscription.start(), subscription.end()) { - tracing::debug!( - track = %self.name, - version = ?self.subscriber.version, - "widening frame bounds to whole groups for an older peer" - ); - } - subscription.set_bounds(start, end); - } - /// Apply a subscription-demand change: hand back an [`Establish`] to open the /// upstream SUBSCRIBE on the first subscriber, buffer a SUBSCRIBE_UPDATE while /// live (the caller flushes), or cancel outright when the last one leaves. @@ -6296,15 +6050,13 @@ impl TrackServe { // The wire carries one range: the span of every range asked for, which the // subscribers' own reads narrow again. subscription.set_bounds(subscription.start(), subscription.end()); - producer.widen_start(&mut subscription); - self.widen_frame_bounds(&mut subscription); match sub { Sub::None => { // An idle copy is asked from the head of the newest group it cached when // the subscriber has no start of its own: a quiet track (a catalog) sends // that group again, which says the cache is current. if subscription.start().is_none() { - subscription.set_bounds(producer.idle_newest().map(Position::group), subscription.end()); + subscription.set_bounds(producer.idle_newest(), subscription.end()); } // Open an upstream SUBSCRIBE for the first subscriber. Ok(Begin::Establish(self.prepare_establish(producer, subscription))) @@ -6320,8 +6072,8 @@ impl TrackServe { // tail as well as in the track's arrivals, which its start declaration // restarts. let lowered = subscription.start().map(|start| { - let floor = active.start.map(|start| start.group).or(active.served); - start.group..floor.unwrap_or(u64::MAX) + let floor = active.start.or(active.served); + start..floor.unwrap_or(u64::MAX) }); if let Ok(mut tail) = active.tail.write() { tail.set_grace(tail::grace(subscription.max_delay)); @@ -6340,7 +6092,7 @@ impl TrackServe { // start returns to the demand that produced it (see // `SubStream::requested`). if start_moved { - let _ = producer.start_at(active.start.map(|start| start.group)); + let _ = producer.start_at(active.start); } buffer_update(active, subscription.end())?; } @@ -6379,14 +6131,12 @@ impl TrackServe { fn prepare_establish(&self, producer: &mut track::Producer, subscription: Subscription) -> Establish { let id = self.subscriber.next_id.fetch_add(1, atomic::Ordering::Relaxed); - // Both halves of each bound come from the same position, so a frame can never - // reach the wire without the group it counts from (which the peer would reject). // The floor tracks the requested start until this subscription's own // SUBSCRIBE_START refines it: the peer never serves below the request, a // previous subscription's declaration must not outlive its demand, and // live-edge demand (None) starts with no floor at all. Lite-06+ only resolves // the start with its SUBSCRIBE_START, so until then the floor is just a request. - let floor = subscription.start().map(|start| start.group); + let floor = subscription.start(); let _ = match self.subscriber.version.resolves_start() { true => producer.request_start(floor), false => producer.start_at(floor), @@ -6485,10 +6235,7 @@ enum Begin { } /// Buffer the current SUBSCRIBE parameters with an updated end. The caller flushes. -fn buffer_update( - active: &mut SubStream, - end: Option, -) -> Result<(), Error> { +fn buffer_update(active: &mut SubStream, end: Option) -> Result<(), Error> { let bounds = WireBounds::new(active.start, end); active.stream.writer.buffer(&lite::SubscribeUpdate { wanted_group: active.wanted, @@ -7417,7 +7164,7 @@ impl ServeLoop { if let Some(requested) = active.requested && let Ok(mut tail) = active.tail.write() { - tail.account(requested.group..start.group, serve.subscriber.runtime.now()); + tail.account(requested..start.group, serve.subscriber.runtime.now()); } } // The publisher will never send these groups, so they @@ -7455,14 +7202,7 @@ impl ServeLoop { // parks, since a cap can be raised). Those streams can still // be in flight, so wait for the tail before finishing. let subscription = self.serving.subscription(); - let requested_end = - subscription - .as_ref() - .and_then(|sub| sub.end()) - .map(|end| match end.frame { - 0 => end.group, - _ => end.group.saturating_add(1), - }); + let requested_end = subscription.as_ref().and_then(|sub| sub.end()); // The effective max delay is the stopgap grace: the wrong clock // (it bounds presentation-time drift), but it is how long the // subscriber was willing to wait for a late group anyway. diff --git a/rs/moq-net/src/model/group.rs b/rs/moq-net/src/model/group.rs index 32b90da758d..1be2246fcec 100644 --- a/rs/moq-net/src/model/group.rs +++ b/rs/moq-net/src/model/group.rs @@ -505,9 +505,6 @@ enum Source { pub(crate) struct Wire { /// Object IDs differ from the public frame ordinals, so ordinal recovery cannot stitch. pub sparse: bool, - /// Ingest refuses an Object ID other than its frame ordinal, so a window in ordinals - /// selects the objects it names. - pub pinned: bool, pub first_object: bool, pub first_id: u64, pub extensions: bool, @@ -1920,13 +1917,6 @@ impl Handle { state.abort.is_none().then_some(state.offset) } - /// Whether this copy still holds every committed frame below `end`, so a compressed - /// tail requested there can inflate against it. Sparse object IDs are not ordinals. - pub(super) fn holds_prefix(&self, end: u64) -> bool { - let state = self.state.read(); - state.abort.is_none() && state.offset == 0 && !state.sparse() && state.committed as u64 >= end - } - /// One past the last frame committed to an unfinished group, when that is past its /// first: where a replacement route resumes. `None` once the group is finished, or /// while it holds nothing a replacement could continue. @@ -2517,12 +2507,6 @@ impl Consumer { self.cursor.state.read().wire().cloned() } - /// Whether frame ordinals address this group's objects: it was not relayed from - /// moq-transport, or its ingest pinned every Object ID to its ordinal. - pub(super) fn ordinal(&self) -> bool { - self.cursor.state.read().wire().is_none_or(|wire| wire.pinned) - } - /// Read a compressed group's stored DEFLATE slices instead of plain frames, with each /// frame's inflated size beside it ([`frame::Object::inflated`], /// [`frame::Consumer::inflated`]), for egress that carries compression. A no-op on a @@ -3234,15 +3218,14 @@ pub struct Fetch { /// Index of the first frame to fetch within the group. Defaults to 0, the whole group. /// - /// Use this to fill a hole left by a route change: the group's head is already - /// cached locally and only the tail is missing. + /// Only a route change sets it, filling the hole after a head another route already + /// delivered. An application always fetches from a group's first frame, since a group is + /// the unit of decodability, and skips frames locally with [`Consumer::set_frames`]. /// - /// There is no matching end: a fetch always runs to the end of the group, and a - /// caller wanting less caps the returned consumer with [`Consumer::set_frames`]. Stopping - /// the *fetch* short would put a group in the cache that is indistinguishable from a - /// complete one, so a later fetch of the whole group would resolve from it and come - /// up short. - pub frame_start: u64, + /// There is no matching end: stopping the *fetch* short would put a group in the cache + /// that is indistinguishable from a complete one, so a later fetch of the whole group + /// would resolve from it and come up short. + pub(crate) frame_start: u64, } impl Fetch { @@ -3257,12 +3240,6 @@ impl Fetch { self.max_delay = Some(max_delay); self } - - /// Set the first frame to fetch, returning `self` for chaining. - pub fn with_frame_start(mut self, frame_start: u64) -> Self { - self.frame_start = frame_start; - self - } } /// A consumer's request for a single past group, handed to a handler via @@ -3397,7 +3374,6 @@ mod test { group .set_wire(Wire { sparse: true, - pinned: false, first_object: false, first_id: 6, extensions: true, @@ -3849,7 +3825,6 @@ mod test { }; let wire = Wire { sparse: false, - pinned: false, first_object: false, first_id: 2, extensions: true, diff --git a/rs/moq-net/src/model/origin.rs b/rs/moq-net/src/model/origin.rs index 2259b66a9ac..e66351c6218 100644 --- a/rs/moq-net/src/model/origin.rs +++ b/rs/moq-net/src/model/origin.rs @@ -11746,13 +11746,13 @@ mod tests { let mut handler = replacement.dynamic(); queued(&standby).await.accept(&replacement); let request = handler.requested_track().await.unwrap(); - assert_eq!(request.subscription().unwrap().start(), Some(track::Position::group(5))); + assert_eq!(request.subscription().unwrap().start(), Some(5)); let track = request.accept(crate::track::Info::timed()); let mut preferences = rig.subscription.subscription(); preferences.priority = 7; rig.subscription.update(preferences).unwrap(); settle(|| track.subscription().is_some_and(|sub| sub.priority == 7)).await; - assert_eq!(track.subscription().unwrap().start(), Some(track::Position::group(5))); + assert_eq!(track.subscription().unwrap().start(), Some(5)); } /// A route updated in place to a new first hop still serves the same broadcast: diff --git a/rs/moq-net/src/model/resume.rs b/rs/moq-net/src/model/resume.rs index 16b7992d564..26b506e8614 100644 --- a/rs/moq-net/src/model/resume.rs +++ b/rs/moq-net/src/model/resume.rs @@ -112,11 +112,7 @@ impl Producer { Some(copy) if copy.from == preferences.ranges => copy.resolved.clone(), _ => Vec::new(), }; - ranges.extend( - preferences - .clamped(reader.newest.map(Position::group), None, &resolved) - .ranges, - ); + ranges.extend(preferences.clamped(reader.newest, None, &resolved).ranges); } demand.ranges = ranges; demand @@ -369,7 +365,7 @@ struct Copy { track: track::Consumer, sub: Sub, /// Where the subscription was asked to start, so demand updates never ask for less. - floor: Option, + floor: Option, /// The reader's ranges [`Self::resolved`] are for. from: Vec, /// Each of those ranges resolved, by index, so clamping never widens one: as the @@ -418,7 +414,7 @@ impl Copy { self.from = reader.ranges.clone(); } } - reader.clamped(self.floor, self.end().map(Position::group), &self.resolved) + reader.clamped(self.floor, self.end(), &self.resolved) } /// The first group past where this copy was replaced, if any: one replaced before @@ -441,7 +437,7 @@ impl Copy { if let Sub::Pending(pending) = &self.sub { let mut sub = ready!(pending.poll_ok(waiter))?; // An anchor the wire kept may resolve below where the reader got to. - sub.raise_start_to(self.floor.map_or(groups.0, |floor| groups.0.max(floor.group))); + sub.raise_start_to(self.floor.map_or(groups.0, |floor| groups.0.max(floor))); sub.end_at(groups.1); self.sub = Sub::Ready(Box::new(sub)); } @@ -487,7 +483,7 @@ impl Copy { { let start = reader.groups.0; // From where it was asked to start, or else from what it first got. - let floor = self.floor.map(|floor| floor.group); + let floor = self.floor; let from = floor.map_or(start, |floor| floor.max(start)); // A group still to land is waited for once something superseded it, as a gap is. if self.since.is_some() @@ -520,7 +516,7 @@ impl Copy { let backs = |sequence: u64| { // A group at or past where it was asked to start may land even before its first // arrival; without a floor, only what follows its first arrival can. - let asked = self.floor.is_some_and(|floor| floor.group <= sequence); + let asked = self.floor.is_some_and(|floor| floor <= sequence); self.track.peek_group(sequence).is_some() || (lands && self @@ -873,15 +869,13 @@ impl Reader { track::Range { start: match range.start { // The farthest offset reaches back to the oldest group, which names no floor. - Anchor::Relative(back) if back != u64::MAX => Anchor::Absolute(Position::group(resolved.start)), - Anchor::Time(_) => Anchor::Absolute(Position::group(resolved.start)), + Anchor::Relative(back) if back != u64::MAX => Anchor::Absolute(resolved.start), + Anchor::Time(_) => Anchor::Absolute(resolved.start), start => start, }, // A time end the copy has not settled stays one: its cap is not the end. end: match (range.end, resolved.end) { - (Some(Anchor::Relative(_) | Anchor::Time(_)), Some(end)) => { - Some(Anchor::Absolute(Position::group(end))) - } + (Some(Anchor::Relative(_) | Anchor::Time(_)), Some(end)) => Some(Anchor::Absolute(end)), (end, _) => end, }, } @@ -987,15 +981,15 @@ impl Reader { copy.datagrams_done = None; // Resume where a fresh copy would: a group it buffered while replaced, below the // newest another route handed out, is not handed out. - copy.floor = copy.floor.max(self.newest.map(Position::group)); + copy.floor = copy.floor.max(self.newest); if let (Sub::Ready(sub), Some(floor)) = (&mut copy.sub, copy.floor) { - sub.raise_start_to(floor.group); + sub.raise_start_to(floor); } copy.update(&self.mirrored); self.copies.push(copy); return; } - let floor = self.newest.map(Position::group); + let floor = self.newest; // The wire request carries the floor from the first message: some sessions read // the demand only once. let wire = self.mirrored.clamped(floor, None, &resolved); @@ -1577,7 +1571,7 @@ impl Reader { self.groups.0 = start; for copy in &mut self.copies { if let Sub::Ready(sub) = &mut copy.sub { - sub.start_at(copy.floor.map_or(start, |floor| start.max(floor.group))); + sub.start_at(copy.floor.map_or(start, |floor| start.max(floor))); } } } @@ -3870,7 +3864,7 @@ mod test { .subscribe(kio::Producer::new(Subscription::default()).into()); let mut reader = sub.reader.lock().unwrap(); let copy = &mut reader.copies[0]; - copy.floor = Some(Position::group(0)); + copy.floor = Some(0); let ranges = vec![track::Range { start: track::Anchor::Time(ts(10_000)), end: None, @@ -3960,7 +3954,7 @@ mod test { assert!(sub.recv_group().now_or_never().is_none()); let start = c.subscription().expect("subscribed").ranges[0].start; assert!( - matches!(start, track::Anchor::Absolute(position) if position.group >= 10), + matches!(start, track::Anchor::Absolute(group) if group >= 10), "the replacement asks from where b resolved the time: {start:?}" ); } @@ -3977,7 +3971,7 @@ mod test { zero.write_frame(ts(3000), b"x".as_ref()).unwrap(); zero.finish().unwrap(); let range = track::Range { - start: track::Anchor::Absolute(Position::group(0)), + start: track::Anchor::Absolute(0), end: Some(track::Anchor::Time(ts(3500))), }; let subscription = Subscription::default() @@ -4009,7 +4003,7 @@ mod test { #[test] fn a_replaced_copy_at_the_last_sequence_has_no_overflow_cap() { - for (last, end) in [(Some(u64::MAX), None), (None, Some(Position::default()))] { + for (last, end) in [(Some(u64::MAX), None), (None, Some(0))] { let routes = Producer::new(); let a = copy(); routes.serve(a.consume()); @@ -4020,8 +4014,8 @@ mod test { old.until = Some(last); assert_eq!(old.wire(&Subscription::default()).end(), end); // A reader's narrower bound survives an unbounded replacement cap. - let requested = Subscription::default().with_end(Position::group(7)); - assert_eq!(old.wire(&requested).end(), Some(end.unwrap_or(Position::group(7)))); + let requested = Subscription::default().with_end(7); + assert_eq!(old.wire(&requested).end(), Some(end.unwrap_or(7))); } } @@ -4226,15 +4220,15 @@ mod test { // One reader has not read anything yet, so it still needs the oldest range. assert_eq!(routes.subscription(demand.clone(), 2).start(), None); assert_eq!(recv(&mut second).sequence, 5); - assert_eq!(routes.subscription(demand.clone(), 2).start(), Some(Position::group(5))); + assert_eq!(routes.subscription(demand.clone(), 2).start(), Some(5)); write(&a, 10); assert_eq!(recv(&mut first).sequence, 10); - assert_eq!(routes.subscription(demand.clone(), 2).start(), Some(Position::group(5))); + assert_eq!(routes.subscription(demand.clone(), 2).start(), Some(5)); let pending = front.consume().subscribe(Subscription::default().with_max_delay(HOUR)); assert_eq!(routes.subscription(front.subscription().unwrap(), 3).start(), None); drop(pending); drop(second); - assert_eq!(routes.subscription(demand, 1).start(), Some(Position::group(10))); + assert_eq!(routes.subscription(demand, 1).start(), Some(10)); } #[moq_clock::test] @@ -4662,7 +4656,6 @@ mod test { group .set_wire(group::Wire { sparse: false, - pinned: false, first_object: true, first_id: 6, extensions: false, @@ -4731,7 +4724,6 @@ mod test { group .set_wire(group::Wire { sparse: false, - pinned: false, first_object: true, first_id: 6, extensions: false, @@ -4778,7 +4770,6 @@ mod test { group .set_wire(group::Wire { sparse: false, - pinned: false, first_object: false, first_id: 6, extensions: false, diff --git a/rs/moq-net/src/model/subscription.rs b/rs/moq-net/src/model/subscription.rs index 26d9cf27aa7..230d1e068cf 100644 --- a/rs/moq-net/src/model/subscription.rs +++ b/rs/moq-net/src/model/subscription.rs @@ -76,8 +76,8 @@ pub struct Subscription { /// for nothing. /// /// A request to the publisher, aggregated across every live subscriber as the union of - /// their ranges, and this subscriber's read filter: it reads only the groups and frames - /// its own ranges cover, whatever another subscriber widens the union to. + /// their ranges, and this subscriber's read filter: it reads only the groups its own + /// ranges cover, whatever another subscriber widens the union to. /// [`crate::track::Subscriber::set_groups`] narrows the reads further. See [Local cursor /// vs wire preference](crate::track::Subscriber#local-cursor-vs-wire-preference). pub ranges: Vec, @@ -104,9 +104,12 @@ pub enum Order { Descending, } -/// Where one end of a [`Range`] falls: a fixed position, a count of groups back from the +/// Where one end of a [`Range`] falls: a fixed group, a count of groups back from the /// latest one, or a presentation time. /// +/// Every anchor names a whole group: a subscription always starts at a group's first frame, +/// since a group is the unit of decodability. +/// /// A relative or time anchor resolves at the hop that serves it, when it starts serving /// and again whenever the subscriber replaces its ranges. Before any group exists a /// relative anchor resolves at the first one, so [`Range::live`] starts there. @@ -117,8 +120,8 @@ pub enum Order { /// starts there. #[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord, Hash)] pub enum Anchor { - /// This exact position. - Absolute(Position), + /// The group with this sequence. + Absolute(u64), /// The group this many before the latest, clamped to group 0: `Relative(0)` is the latest group. Relative(u64), /// The group containing this presentation time: the newest one starting at or before it. @@ -138,10 +141,9 @@ pub enum Anchor { /// A span of a track to deliver: from `start` up to `end`, or onward when `end` is `None`. /// -/// Relative and time anchors cover whole groups: such a start is frame 0 of its group, and -/// such an end includes its group, so `Absolute(0)..Relative(0)` delivers through the latest -/// group. An absolute end is exclusive, like the end of a [`std::ops::Range`]: see -/// [`Position::after`] and [`Position::after_group`]. +/// An absolute end is exclusive, like the end of a [`std::ops::Range`], so `Absolute(2).. +/// Absolute(6)` delivers groups 2 through 5. Relative and time ends include their group, so +/// `Absolute(0)..Relative(0)` delivers through the latest group. #[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord, Hash)] pub struct Range { /// Where delivery starts. @@ -173,16 +175,16 @@ impl Range { /// The whole groups in `groups`, such as `2..=5`, `2..`, or `..6`. /// /// An omitted start is [`Self::oldest`]'s, and an inclusive end past the last group is - /// no end, as [`Position::after_group`] spells it. + /// no end. pub fn groups(groups: impl RangeBounds) -> Self { let unbounded_start = matches!(groups.start_bound(), Bound::Unbounded); let (start, end) = sequence_bounds(groups); Self { start: match unbounded_start { true => Self::oldest().start, - false => Anchor::Absolute(Position::group(start)), + false => Anchor::Absolute(start), }, - end: end.map(|end| Anchor::Absolute(Position::group(end))), + end: end.map(Anchor::Absolute), } } @@ -209,7 +211,7 @@ pub(super) fn union(ranges: impl IntoIterator) -> Vec { return vec![Range::oldest()]; } - let mut spans: Vec<(Position, Option)> = ranges + let mut spans: Vec<(u64, Option)> = ranges .iter() .filter_map(|range| match (range.start, range.end) { (Anchor::Absolute(start), None) => Some((start, None)), @@ -218,7 +220,7 @@ pub(super) fn union(ranges: impl IntoIterator) -> Vec { }) .collect(); spans.sort_unstable(); - let mut runs: Vec<(Position, Option)> = Vec::new(); + let mut runs: Vec<(u64, Option)> = Vec::new(); for (start, end) in spans { match runs.last_mut() { Some(last) if last.1.is_none_or(|last| start <= last) => { @@ -302,13 +304,13 @@ impl Subscription { self } - /// The lowest absolute position any range asks for, or `None` when one counts back - /// from the latest group (or none is asked for). + /// The lowest absolute group any range asks for, or `None` when one counts back from + /// the latest group (or none is asked for). /// /// The form a wire without ranges carries: a floor below which nothing is requested. #[cfg(any(feature = "async", test))] - pub(crate) fn start(&self) -> Option { - let mut floor: Option = None; + pub(crate) fn start(&self) -> Option { + let mut floor: Option = None; for range in &self.ranges { let Anchor::Absolute(start) = range.start else { return None; @@ -318,13 +320,13 @@ impl Subscription { floor } - /// The exclusive end past every range, or `None` when one is open or counts back from - /// the latest group. An empty list is the empty range, `Position::group(0)`. + /// The exclusive end group past every range, or `None` when one is open or counts back + /// from the latest group. An empty list is the empty range, ending at group 0. /// /// The form a wire without ranges carries, alongside [`Self::start`]. #[cfg(any(feature = "async", test))] - pub(crate) fn end(&self) -> Option { - let mut end = Some(Position::default()); + pub(crate) fn end(&self) -> Option { + let mut end = Some(0); for range in &self.ranges { let Some(Anchor::Absolute(next)) = range.end else { return None; @@ -337,7 +339,7 @@ impl Subscription { /// Replace the ranges with one from `start` (or [`Range::oldest`]'s) to `end`: the form /// a wire without ranges carries. #[cfg(any(feature = "async", test))] - pub(crate) fn set_bounds(&mut self, start: Option, end: Option) { + pub(crate) fn set_bounds(&mut self, start: Option, end: Option) { self.ranges = vec![Range { start: start.map_or(Range::oldest().start, Anchor::Absolute), end: end.map(Anchor::Absolute), @@ -346,14 +348,14 @@ impl Subscription { /// Replace the start of [`Self::set_bounds`]'s one range, keeping its end. #[cfg(feature = "async")] - pub(crate) fn with_start(mut self, start: impl Into>) -> Self { + pub(crate) fn with_start(mut self, start: impl Into>) -> Self { self.set_bounds(start.into(), self.end()); self } /// Replace the end of [`Self::set_bounds`]'s one range, keeping its start. #[cfg(any(feature = "async", test))] - pub(crate) fn with_end(mut self, end: impl Into>) -> Self { + pub(crate) fn with_end(mut self, end: impl Into>) -> Self { self.set_bounds(self.start(), end.into()); self } @@ -366,12 +368,7 @@ impl Subscription { /// keeps its anchor, since the floor may lie before it, so the caller drops what it /// resolves to below `floor`; an end not resolved becomes the bound, except a time end, /// which stays so the caller enforces `end` apart from it. - pub(super) fn clamped( - &self, - floor: Option, - end: Option, - resolved: &[Option], - ) -> Self { + pub(super) fn clamped(&self, floor: Option, end: Option, resolved: &[Option]) -> Self { let clamping = floor.is_some() || end.is_some(); let ranges = self .ranges @@ -380,15 +377,11 @@ impl Subscription { .map(|(index, range)| { let resolved = resolved.get(index).copied().flatten().filter(|_| clamping); let start = match (range.start, resolved) { - (Anchor::Relative(_) | Anchor::Time(_), Some(resolved)) => { - Anchor::Absolute(Position::group(resolved.start)) - } + (Anchor::Relative(_) | Anchor::Time(_), Some(resolved)) => Anchor::Absolute(resolved.start), (start, _) => start, }; let requested = match (range.end, resolved.and_then(|resolved| resolved.end)) { - (Some(Anchor::Relative(_) | Anchor::Time(_)), Some(resolved)) => { - Some(Anchor::Absolute(Position::group(resolved))) - } + (Some(Anchor::Relative(_) | Anchor::Time(_)), Some(resolved)) => Some(Anchor::Absolute(resolved)), (requested, _) => requested, }; Range { @@ -599,8 +592,8 @@ mod tests { combined.finish() } - fn absolute(group: u64, frame: u64) -> Anchor { - Anchor::Absolute(Position { group, frame }) + fn absolute(group: u64) -> Anchor { + Anchor::Absolute(group) } fn range(start: Anchor, end: impl Into>) -> Range { @@ -611,16 +604,16 @@ mod tests { combine(subscriptions).unwrap().ranges } - /// A group range spells both positions at once, in whichever form the caller has. + /// A group range spells both ends at once, in whichever form the caller has. #[test] - fn group_ranges_build_whole_group_positions() { - assert_eq!(Range::groups(2..=5), range(absolute(2, 0), absolute(6, 0))); - assert_eq!(Range::groups(2..6), range(absolute(2, 0), absolute(6, 0))); - assert_eq!(Range::groups(..6), range(Range::oldest().start, absolute(6, 0))); - assert_eq!(Range::groups(2..), range(absolute(2, 0), None)); + fn group_ranges_build_absolute_anchors() { + assert_eq!(Range::groups(2..=5), range(absolute(2), absolute(6))); + assert_eq!(Range::groups(2..6), range(absolute(2), absolute(6))); + assert_eq!(Range::groups(..6), range(Range::oldest().start, absolute(6))); + assert_eq!(Range::groups(2..), range(absolute(2), None)); assert_eq!(Range::groups(..), Range::oldest()); - // Through the last group is unbounded, as `after_group` spells it. + // Through the last group is unbounded. assert_eq!(Range::groups(..=u64::MAX).end, None); let sub = Subscription::default().with_groups(2..=5).with_groups(..); @@ -670,7 +663,7 @@ mod tests { assert_eq!(floor(&Subscription::default(), None), Some(0)); // The lowest range wins, and one unresolved range leaves the whole floor unresolved. - let mixed = Subscription::default().with_ranges([range(absolute(4, 0), None), Range::live()]); + let mixed = Subscription::default().with_ranges([range(absolute(4), None), Range::live()]); assert_eq!(floor(&mixed, Some(3)), Some(3)); assert_eq!(floor(&mixed, Some(9)), Some(4)); assert_eq!(floor(&mixed, None), None); @@ -688,32 +681,26 @@ mod tests { let sub = Subscription::default(); assert_eq!((sub.start(), sub.end()), (None, None)); - let sub = Subscription::default().with_ranges([Range::groups(2..4), range(absolute(7, 1), absolute(9, 0))]); - assert_eq!( - (sub.start(), sub.end()), - (Some(Position::group(2)), Some(Position::group(9))) - ); + let sub = Subscription::default().with_ranges([Range::groups(2..4), Range::groups(7..9)]); + assert_eq!((sub.start(), sub.end()), (Some(2), Some(9))); // An explicit group 0 is a floor, unlike the default. let sub = Subscription::default().with_groups(0..); - assert_eq!((sub.start(), sub.end()), (Some(Position::group(0)), None)); + assert_eq!((sub.start(), sub.end()), (Some(0), None)); let sub = Subscription::default().with_ranges([Range::groups(2..4), Range::live()]); assert_eq!((sub.start(), sub.end()), (None, None)); - let sub = Subscription::default().with_ranges([range(absolute(2, 0), Anchor::Relative(0))]); - assert_eq!((sub.start(), sub.end()), (Some(Position::group(2)), None)); + let sub = Subscription::default().with_ranges([range(absolute(2), Anchor::Relative(0))]); + assert_eq!((sub.start(), sub.end()), (Some(2), None)); // Nothing asked for is the empty range. let sub = Subscription::default().with_ranges([]); - assert_eq!(sub.end(), Some(Position::group(0))); + assert_eq!(sub.end(), Some(0)); let mut sub = Subscription::default(); - sub.set_bounds(Some(Position { group: 5, frame: 2 }), Position::after_group(9)); - assert_eq!( - (sub.start(), sub.end()), - (Some(Position { group: 5, frame: 2 }), Some(Position::group(10))) - ); + sub.set_bounds(Some(5), Some(10)); + assert_eq!((sub.start(), sub.end()), (Some(5), Some(10))); sub.set_bounds(None, None); assert_eq!(sub, Subscription::default()); } @@ -775,31 +762,6 @@ mod tests { ); } - /// Ranges fold as whole positions: two ranges in the same group are separated only by - /// their frames, so folding group and frame independently would invent a bound. - #[test] - fn combined_ranges_fold_whole_positions() { - let sub = |range: Range| Subscription::default().with_ranges([range]); - - assert_eq!( - ranges(&[ - sub(range(absolute(5, 9), absolute(5, 10))), - sub(range(absolute(5, 2), absolute(5, 3))) - ]), - vec![ - range(absolute(5, 2), absolute(5, 3)), - range(absolute(5, 9), absolute(5, 10)) - ] - ); - assert_eq!( - ranges(&[ - sub(range(absolute(5, 2), absolute(5, 9))), - sub(range(absolute(5, 4), absolute(6, 0))) - ]), - vec![range(absolute(5, 2), absolute(6, 0))] - ); - } - /// Combining is order-independent and stops changing once it covers everyone. #[test] fn combined_ranges_settle() { diff --git a/rs/moq-net/src/model/subscription/window.rs b/rs/moq-net/src/model/subscription/window.rs index 0867c7e561e..eb86c987b90 100644 --- a/rs/moq-net/src/model/subscription/window.rs +++ b/rs/moq-net/src/model/subscription/window.rs @@ -389,37 +389,28 @@ impl Window { } /// Whether a datagram is in range, judged on its own and never through a time end's - /// settling: its `timestamp` against each range's times, its `sequence` against the - /// range's absolute groups and the relative ones resolved so far, and its `object` ID - /// against the frames that same range covers in its group. - pub fn admits_datagram(&self, sequence: u64, timestamp: Option, object: u64) -> bool { - let framed = |range: &Range| { - let (start, end) = frames(range, sequence); - object >= start && end.is_none_or(|end| object < end) - }; + /// settling: its `timestamp` against each range's times, and its `sequence` against the + /// range's absolute groups and the relative ones resolved so far. + pub fn admits_datagram(&self, sequence: u64, timestamp: Option) -> bool { let Some(timestamp) = timestamp else { - return self.pending - || self - .spans - .iter() - .any(|span| span.covers(sequence) && framed(&self.ranges[span.index])); + return self.pending || self.spans.iter().any(|span| span.covers(sequence)); }; self.ranges .iter() .enumerate() - .filter(|(_, range)| !range.is_empty() && framed(range)) + .filter(|(_, range)| !range.is_empty()) .any(|(index, range)| { // A relative anchor still waiting on a latest group bounds nothing yet. let span = self.spans.iter().find(|span| span.index == index); let started = match range.start { Anchor::Time(pts) => !later(pts, timestamp), - Anchor::Absolute(start) => sequence >= start.group, + Anchor::Absolute(start) => sequence >= start, Anchor::Relative(_) => span.is_none_or(|span| sequence >= span.start), }; let ended = match range.end { None => true, Some(Anchor::Time(pts)) => !later(timestamp, pts), - Some(Anchor::Absolute(end)) => end.final_sequence().is_none_or(|end| sequence < end), + Some(Anchor::Absolute(end)) => sequence < end, Some(Anchor::Relative(_)) => { span.is_none_or(|span| !matches!(span.end, End::Before(end) if sequence >= end)) } @@ -485,28 +476,6 @@ impl Window { all.into_iter().chain(runs.into_iter().flatten()) } - /// The frames of group `sequence` in range: from a start frame up to an exclusive end, - /// `None` through the group's end. - /// - /// Only an absolute anchor names a frame, so a group a range starts or ends inside reads - /// part of itself and every other group reads whole. Ranges that cover a group - /// differently read the hull of their frames, since a group is handed out once, as one - /// run of frames. Everything while a relative anchor waits. - pub fn frames(&self, sequence: u64) -> (u64, Option) { - if self.pending { - return (0, None); - } - let mut hull = None; - for span in self.spans.iter().filter(|span| span.covers(sequence)) { - let (start, end) = frames(&self.ranges[span.index], sequence); - hull = Some(match hull { - None => (start, end), - Some((low, high)) => (start.min(low), end.zip(high).map(|(end, high)| end.max(high))), - }); - } - hull.unwrap_or((0, None)) - } - /// The first run of `range` that is in range. pub fn clip(&self, range: ops::Range) -> Option> { let last = range.end.checked_sub(1).filter(|last| *last >= range.start)?; @@ -573,26 +542,11 @@ impl Window { } } -/// The frames of group `sequence` that `range` names, if it covers that group: from a start -/// frame up to an exclusive end, `None` through the group's end. Only an absolute anchor -/// names a frame; a head-of-group end covers nothing of its group, so it never gets here. -fn frames(range: &Range, sequence: u64) -> (u64, Option) { - let start = match range.start { - Anchor::Absolute(start) if start.group == sequence => start.frame, - _ => 0, - }; - let end = match range.end { - Some(Anchor::Absolute(end)) if end.group == sequence => Some(end.frame), - _ => None, - }; - (start, end) -} - /// The first group a range starts at, and whether it names that group rather than /// reaching back to the oldest one. `None` while it waits on a latest group. fn start(anchor: Anchor, latest: Option, holdings: &impl Holdings) -> Option<(u64, bool)> { match anchor { - Anchor::Absolute(start) => Some((start.group, true)), + Anchor::Absolute(start) => Some((start, true)), // Nothing sits below group 0, so the farthest offset needs no latest group. Anchor::Relative(u64::MAX) => Some((0, false)), Anchor::Relative(back) => Some((latest?.saturating_sub(back), true)), @@ -611,7 +565,7 @@ fn start(anchor: Anchor, latest: Option, holdings: &impl Holdings) -> Optio fn end(anchor: Option, latest: Option) -> Option { Some(match anchor { None => End::Open, - Some(Anchor::Absolute(end)) => end.final_sequence().map_or(End::Open, End::Before), + Some(Anchor::Absolute(end)) => End::Before(end), Some(Anchor::Relative(back)) => latest? .saturating_sub(back) .checked_add(1) @@ -622,7 +576,6 @@ fn end(anchor: Option, latest: Option) -> Option { #[cfg(test)] mod tests { - use super::super::Position; use super::*; use std::cell::Cell; @@ -761,63 +714,12 @@ mod tests { ] ); - // A mid-group end includes its group, a head-of-group end excludes it. - let window = Window::new( - &[range( - Anchor::Absolute(Position { group: 1, frame: 2 }), - Anchor::Absolute(Position { group: 3, frame: 1 }), - )], - None, - &Untimed, - ) - .unwrap(); - assert_eq!(covered(&window, 6), vec![1, 2, 3]); - assert_eq!(window.end(), Some(4)); - // Nothing asked for is nothing in range, floored nowhere. let window = Window::new(&[], None, &Untimed).unwrap(); assert_eq!(covered(&window, 4), Vec::::new()); assert_eq!((window.floor(), window.end()), (Some(0), Some(0))); } - /// Only a group an absolute anchor falls inside reads part of itself, and overlapping - /// ranges read the hull of their frames there. - #[test] - fn frames_bound_only_the_groups_an_anchor_falls_inside() { - let at = |group, frame| Anchor::Absolute(Position { group, frame }); - let window = Window::new(&[range(at(4, 5), at(6, 3))], None, &Untimed).unwrap(); - assert_eq!(window.frames(4), (5, None)); - assert_eq!(window.frames(5), (0, None)); - assert_eq!(window.frames(6), (0, Some(3))); - - // Within one group, both ends bound it. - let window = Window::new(&[range(at(4, 2), at(4, 7))], None, &Untimed).unwrap(); - assert_eq!(window.frames(4), (2, Some(7))); - - // A second range reaching further either way widens the group to cover both. - let window = Window::new(&[range(at(4, 5), at(4, 7)), range(at(4, 2), at(4, 6))], None, &Untimed).unwrap(); - assert_eq!(window.frames(4), (2, Some(7))); - let window = Window::new(&[range(at(4, 5), None), range(at(2, 0), at(4, 3))], None, &Untimed).unwrap(); - assert_eq!(window.frames(4), (0, None)); - - // Relative anchors cover whole groups, and a pending one bounds nothing yet. - let window = Window::new(&[range(Anchor::Relative(1), at(9, 2))], Some(9), &Untimed).unwrap(); - assert_eq!((window.frames(8), window.frames(9)), ((0, None), (0, Some(2)))); - let window = Window::new(&[range(at(4, 5), None), Range::live()], None, &Untimed).unwrap(); - assert_eq!(window.frames(4), (0, None)); - - // A time end covers whole groups, through its cap while it waits to settle. - let groups = Groups::new([Groups::started(4, 4000), Groups::started(5, 5000), (6, Held::Pending)]); - let window = Window::new( - &[range(at(6, 2), None), range(time(4500), time(5000))], - Some(6), - &groups, - ) - .unwrap(); - assert!(!window.is_settled()); - assert_eq!(window.frames(6), (0, None)); - } - /// Relative anchors count back from the latest group, both ends whole groups, and wait /// for a first group when there is none. #[test] @@ -924,12 +826,7 @@ mod tests { #[test] fn an_unsettled_time_end_reads_one_group_on_speculation() { let mut groups = Groups::new([(3, Held::Pending), (4, Held::Pending), (5, Held::Pending)]); - let mut window = Window::new( - &[range(Anchor::Absolute(Position::group(3)), time(3500))], - Some(5), - &groups, - ) - .unwrap(); + let mut window = Window::new(&[range(Anchor::Absolute(3), time(3500))], Some(5), &groups).unwrap(); assert_eq!(covered(&window, 8), vec![3]); groups.held.insert(3, Held::Started(at(3000))); @@ -985,14 +882,9 @@ mod tests { /// A datagram is one frame, so its own timestamp places it against a time end. #[test] fn a_datagram_is_judged_by_its_own_timestamp() { - let window = Window::new( - &[range(Anchor::Absolute(Position::group(0)), time(3500))], - None, - &Groups::default(), - ) - .unwrap(); - assert!(window.admits_datagram(9, Some(at(3500)), 0)); - assert!(!window.admits_datagram(9, Some(at(3600)), 0)); + let window = Window::new(&[range(Anchor::Absolute(0), time(3500))], None, &Groups::default()).unwrap(); + assert!(window.admits_datagram(9, Some(at(3500)))); + assert!(!window.admits_datagram(9, Some(at(3600)))); } /// Groups whose `racing` one starts past the end after the first time it is asked @@ -1040,12 +932,7 @@ mod tests { racing: 1, asked: Cell::new(false), }; - let window = Window::new( - &[range(Anchor::Absolute(Position::group(0)), time(5000))], - None, - &racing, - ) - .unwrap(); + let window = Window::new(&[range(Anchor::Absolute(0), time(5000))], None, &racing).unwrap(); assert_eq!(window.resolved(), vec![Some(Resolved { start: 0, end: Some(1) })]); } @@ -1069,12 +956,7 @@ mod tests { #[test] fn a_time_end_stops_walking_at_the_last_sequence() { let groups = Groups::new([Groups::started(u64::MAX, 3000)]); - let window = Window::new( - &[range(Anchor::Absolute(Position::group(u64::MAX)), time(5000))], - None, - &groups, - ) - .unwrap(); + let window = Window::new(&[range(Anchor::Absolute(u64::MAX), time(5000))], None, &groups).unwrap(); assert_eq!( window.resolved(), vec![Some(Resolved { @@ -1096,53 +978,49 @@ mod tests { }; let window = Window::new(&[range(time(0), time(5000))], None, &groups).unwrap(); assert_eq!(window.resolved(), vec![Some(Resolved { start: 0, end: Some(2) })]); - assert!(window.admits_datagram(1, Some(at(4000)), 0)); - assert!(!window.admits_datagram(1, Some(at(7000)), 0)); + assert!(window.admits_datagram(1, Some(at(4000)))); + assert!(!window.admits_datagram(1, Some(at(7000)))); } #[test] fn a_waiting_window_judges_a_datagram_against_both_times() { let window = Window::new(&[range(time(3000), time(3500))], None, &Groups::default()).unwrap(); assert!(window.is_pending(), "no group has started"); - assert!(!window.admits_datagram(0, Some(at(2900)), 0)); - assert!(window.admits_datagram(1, Some(at(3000)), 0)); - assert!(window.admits_datagram(2, Some(at(3500)), 0)); - assert!(!window.admits_datagram(3, Some(at(3600)), 0)); + assert!(!window.admits_datagram(0, Some(at(2900)))); + assert!(window.admits_datagram(1, Some(at(3000)))); + assert!(window.admits_datagram(2, Some(at(3500)))); + assert!(!window.admits_datagram(3, Some(at(3600)))); } #[test] fn a_waiting_window_judges_a_datagram_against_absolute_groups_too() { let window = Window::new( - &[range(time(3000), Some(Anchor::Absolute(Position::group(10))))], + &[range(time(3000), Some(Anchor::Absolute(10)))], None, &Groups::default(), ) .unwrap(); assert!(window.is_pending(), "no group has started"); - assert!(window.admits_datagram(9, Some(at(3500)), 0)); - assert!(!window.admits_datagram(10, Some(at(3500)), 0)); - assert!(!window.admits_datagram(9, Some(at(2900)), 0)); + assert!(window.admits_datagram(9, Some(at(3500)))); + assert!(!window.admits_datagram(10, Some(at(3500)))); + assert!(!window.admits_datagram(9, Some(at(2900)))); } - /// A datagram is in range only when one range admits its sequence, time, and Object ID - /// together, not when each passes a different range. + /// A datagram is in range only when one range admits its sequence and time together, + /// not when each passes a different range. #[test] fn a_datagram_is_judged_against_one_range_at_a_time() { - let pos = |group, frame| Anchor::Absolute(Position { group, frame }); let groups = Groups::new([Groups::started(5, 5000)]); // The time range covers group 5 whole, but only from 5500 on. - let window = Window::new(&[range(pos(5, 1), None), range(time(5500), None)], Some(5), &groups).unwrap(); - assert!(!window.admits_datagram(5, Some(at(5000)), 0)); - assert!(window.admits_datagram(5, Some(at(5000)), 1)); - assert!(window.admits_datagram(5, Some(at(5500)), 0)); - - // Untimed, two ranges splitting a group admit only the Object IDs each names. - let window = Window::new(&[range(pos(4, 1), pos(4, 2)), range(pos(4, 5), None)], None, &Untimed).unwrap(); - assert_eq!(window.frames(4), (1, None), "a group stream reads the hull"); - let admitted: Vec = (0..7) - .filter(|object| window.admits_datagram(4, None, *object)) - .collect(); - assert_eq!(admitted, vec![1, 5, 6]); + let window = Window::new( + &[range(Anchor::Absolute(6), None), range(time(5500), None)], + Some(5), + &groups, + ) + .unwrap(); + assert!(!window.admits_datagram(5, Some(at(5000)))); + assert!(window.admits_datagram(5, Some(at(5500)))); + assert!(window.admits_datagram(6, Some(at(5000)))); } #[test] diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index e629b069470..6a74d0bf126 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -797,34 +797,6 @@ impl Holdings for TrackState { } impl TrackState { - /// Start each mid-group range at its group's first frame unless a copy holds the - /// frames below it; see [`Producer::widen_start`]. - #[cfg_attr( - not(feature = "async"), - allow(dead_code, reason = "Only session adapters request upstream.") - )] - fn widen_start(&self, subscription: &mut Subscription) { - for range in &mut subscription.ranges { - if range.is_empty() { - continue; - } - let Anchor::Absolute(start) = &mut range.start else { - continue; - }; - if start.frame == 0 { - continue; - } - let prefix = self - .lookup - .get(&start.group) - .and_then(|slot| slot.copies().find(|group| group.holds_prefix(start.frame))); - match prefix { - Some(group) => group.cache_refresh(), - None => start.frame = 0, - } - } - } - /// A consumer joined at the live edge: ask for a group past every sequence written when /// the newest group's bytes after its first frame exceed [`Info::refresh`] times that /// frame. @@ -3338,25 +3310,6 @@ impl Producer { poll_combined_changed(&subs, bound, &mut self.prev_subscription, waiter).map(Ok) } - /// Start outgoing ranges at their group's first frame, unless a cached copy holds the - /// frames below a mid-group start, such as ones a previous route delivered. A group is - /// often useless without its head (a keyframe, or a compressed group's window), so a - /// request skips only frames a copy already holds. Only requesters call this: the - /// original preferences keep filtering local reads, and a server serves the frame - /// bounds it was actually asked for. - /// - /// A range kept mid-group relies on the cached prefix, so it counts as a read of that - /// copy and normal eviction takes it last. Nothing pins it: a prefix still lost - /// before its tail lands fails that group with [`Error::Lagged`], which takes a - /// consumer slower than the cache's turnover. - #[cfg_attr( - not(feature = "async"), - allow(dead_code, reason = "Only session adapters request upstream.") - )] - pub(crate) fn widen_start(&self, subscription: &mut Subscription) { - self.state.read().widen_start(subscription); - } - /// Create a [`Dynamic`] handle that serves on-demand fetches of uncached /// (old) groups. Most producers never need this; a relay creates one to fetch /// past groups from upstream. @@ -5979,22 +5932,11 @@ impl Cursor { } } - /// Hand `group` out: bounded to the frames this cursor's ranges cover in it, and expiring - /// against its budget. + /// Hand `group` out whole from its first frame, expiring against its budget. /// - /// Positioned without clamping, so a group missing the head asked for reads it as - /// [`Error::Lagged`], which a route copy recovers from another. A group relayed from - /// moq-transport is handed out whole unless its ingest pinned Object IDs to ordinals: - /// its ranges may name Object IDs, so an ordinal window could drop objects they cover, - /// and its reader filters by ID. - fn deliver(&self, mut group: group::Consumer) -> group::Consumer { - if group.ordinal() { - let (start, end) = self.filter.window.frames(group.sequence); - group.skip_to(start); - if let Some(end) = end { - group.end_at(Bound::Excluded(end)); - } - } + /// A group missing its head reads it as [`Error::Lagged`], which a route copy recovers + /// from another. + fn deliver(&self, group: group::Consumer) -> group::Consumer { let sequence = group.sequence; group.with_expiry(Arc::new(GroupExpiry { state: self.state.weak(), @@ -6154,9 +6096,8 @@ impl Cursor { let sequence = datagram.sequence; // A datagram is a whole group, so its own timestamp places it against a time end. - let object = datagram.wire.as_ref().and_then(|wire| wire.object_id).unwrap_or(0); let admits = super::subscription::before_end(sequence, self.end_sequence) - && self.filter.window.admits_datagram(sequence, datagram.timestamp, object); + && self.filter.window.admits_datagram(sequence, datagram.timestamp); if sequence >= self.floor() && admits { return Poll::Ready(Ok(Some(datagram))); } @@ -7494,15 +7435,6 @@ impl Request { snapshot_subscription(&subs, bound) } - /// As [`Producer::widen_start`], for a track not accepted yet. - #[cfg_attr( - not(feature = "async"), - allow(dead_code, reason = "Only session adapters request upstream.") - )] - pub(crate) fn widen_start(&self, subscription: &mut Subscription) { - self.state.read().widen_start(subscription); - } - /// Block until the aggregate [`subscription`](Self::subscription) changes, /// yielding `None` once nobody is waiting. #[cfg(feature = "async")] @@ -7926,7 +7858,7 @@ mod test { /// in arrival and sequence order alike, however wide another subscriber made the /// aggregate. #[test] - fn a_range_reads_only_its_frames() { + fn a_range_reads_only_its_groups() { let producer = track_producer("test", Info::default()); let _groups: Vec<_> = (3..8) .map(|sequence| { @@ -7937,23 +7869,22 @@ mod test { .collect(); let _wide = producer.subscribe(replay()); let range = Range { - start: Anchor::Absolute(Position { group: 4, frame: 5 }), - end: Some(Anchor::Absolute(Position { group: 6, frame: 3 })), + start: Anchor::Absolute(4), + end: Some(Anchor::Absolute(6)), }; let bounded = || producer.subscribe(replay().with_ranges([range])); - let want = vec![(4, vec![5, 6, 7]), (5, (0..8).collect()), (6, vec![0, 1, 2])]; + let want = vec![(4, (0..8).collect()), (5, (0..8).collect())]; let mut arrival = bounded(); assert_eq!(read_groups(|waiter| arrival.poll_recv_group(waiter)), want); let mut ordered = bounded().ordered(); assert_eq!(read_groups(|waiter| ordered.poll_next_group(waiter)), want); } - /// A group missing the frames a range asks for is never served short: reading them is - /// [`Error::Lagged`], which a route copy recovers. One holding them from where the range - /// starts is served from there. + /// A group missing its head is never served short: reading it is [`Error::Lagged`], + /// which a route copy recovers. #[test] - fn a_group_missing_its_asked_head_lags() { + fn a_group_missing_its_head_lags() { let producer = track_producer("test", Info::default()); let mut headless = producer.create_group(group::Info { sequence: 4 }).unwrap(); headless.start_at(2).unwrap(); @@ -7963,50 +7894,13 @@ mod test { headless.finish().unwrap(); let noop = kio::Waiter::noop(); - for frame in [0, 1] { - let start = Anchor::Absolute(Position { group: 4, frame }); - let mut sub = producer.subscribe(replay().with_ranges([Range { start, end: None }])); - let Poll::Ready(Ok(Some(mut group))) = sub.poll_recv_group(&noop) else { - panic!("from frame {frame}: no group"); - }; - assert_eq!(group.index(), frame); - assert!( - matches!(group.poll_read_frame(&noop), Poll::Ready(Err(Error::Lagged))), - "from frame {frame}" - ); - } - - let start = Anchor::Absolute(Position { group: 4, frame: 2 }); + let start = Anchor::Absolute(4); let mut sub = producer.subscribe(replay().with_ranges([Range { start, end: None }])); - assert_eq!( - read_groups(|waiter| sub.poll_recv_group(waiter)), - vec![(4, vec![2, 3, 4])] - ); - } - - /// A range replaced between a peek and the take re-bounds the reserved group: it is - /// still the group handed out, positioned by the ranges in force when it is taken. - #[test] - fn a_peeked_group_is_bounded_by_the_ranges_at_handout() { - for ordered in [false, true] { - let producer = track_producer("test", Info::default()); - framed(&producer, 4, 8).finish().unwrap(); - let mut subscriber = producer.subscribe(replay()); - let waiter = kio::Waiter::noop(); - assert_eq!( - subscriber.poll_peek_group(ordered, &waiter).map(Result::unwrap), - Poll::Ready(Some(4)) - ); - let start = Anchor::Absolute(Position { group: 4, frame: 5 }); - subscriber - .control() - .update(replay().with_ranges([Range { start, end: None }])) - .unwrap(); - let Poll::Ready(Ok(Some(group))) = subscriber.poll_group(ordered, &waiter) else { - panic!("ordered={ordered}: the reserved group must be returned"); - }; - assert_eq!((group.sequence, group.index()), (4, 5), "ordered={ordered}"); - } + let Poll::Ready(Ok(Some(mut group))) = sub.poll_recv_group(&noop) else { + panic!("no group"); + }; + assert_eq!(group.index(), 0); + assert!(matches!(group.poll_read_frame(&noop), Poll::Ready(Err(Error::Lagged)))); } /// The largest reports only while it is within the bounds: a cap below it or a floor @@ -8700,7 +8594,7 @@ mod test { #[cfg(feature = "async")] fn the_group_range_bounds_datagrams() { let mut producer = track_producer("test", Info::timed()); - let mut subscriber = producer.subscribe(Subscription::default().with_start(Position::group(5))); + let mut subscriber = producer.subscribe(Subscription::default().with_start(5)); subscriber.set_groups(..7); for sequence in [4, 5, 7, 6] { producer @@ -8713,33 +8607,6 @@ mod test { assert!(subscriber.poll_recv_datagram(&kio::Waiter::noop()).is_pending()); } - /// A start past the start group's first frame excludes that group's datagram, its only - /// frame, even behind an earlier range; a start at frame 0 keeps it. - #[test] - #[cfg(feature = "async")] - fn a_mid_group_start_drops_the_start_groups_datagram() { - let mut producer = track_producer("test", Info::timed()); - let mut mid = producer.subscribe(Subscription::default().with_start(Position { group: 5, frame: 1 })); - let mut split = producer.subscribe(Subscription::default().with_ranges([ - Range::groups(2..4), - Range { - start: Anchor::Absolute(Position { group: 5, frame: 1 }), - end: None, - }, - ])); - let mut whole = producer.subscribe(Subscription::default().with_start(Position::group(5))); - for sequence in [5, 6] { - producer - .insert_datagram(sequence, Timestamp::ZERO, bytes::Bytes::from_static(b"x")) - .unwrap(); - } - - assert_eq!(recv_datagram(&mut mid).sequence, 6); - assert_eq!(recv_datagram(&mut split).sequence, 6); - assert_eq!(recv_datagram(&mut whole).sequence, 5); - assert_eq!(recv_datagram(&mut whole).sequence, 6); - } - /// A range update judges the datagrams still buffered when they are read, and one it /// dropped stays dropped once the cap rises again: nothing holds it like a parked group. #[test] @@ -9895,20 +9762,20 @@ mod test { // The budget is the only thing that asks for data; a named start only bounds how // far back it may reach. Naming group 1 at real time still delivers the live edge // alone, since the zero budget calls everything older stale. - let named = Subscription::default().with_start(Position::group(1)); + let named = Subscription::default().with_start(1); let mut subscriber = producer.subscribe(named); assert_eq!(drain(&mut subscriber), vec![4]); // A budget reaching further back than the floor is cut off at it. let floored = Subscription::default() - .with_start(Position::group(3)) + .with_start(3) .with_max_delay(Duration::from_secs(10)); let mut subscriber = producer.subscribe(floored); assert_eq!(drain(&mut subscriber), vec![3, 4]); // A floor below what the budget admits changes nothing. let slack = Subscription::default() - .with_start(Position::group(1)) + .with_start(1) .with_max_delay(Duration::from_secs(2)); let mut subscriber = producer.subscribe(slack); assert_eq!(drain(&mut subscriber), vec![2, 3, 4]); @@ -9925,7 +9792,7 @@ mod test { // A resumed subscription names where it left off, which may not exist yet. The // cursor sits at the floor rather than sliding back to what is cached. let resumed = Subscription::default() - .with_start(Position::group(7)) + .with_start(7) .with_max_delay(Duration::from_secs(10)); let mut subscriber = producer.subscribe(resumed); assert_eq!(drain(&mut subscriber), Vec::::new()); @@ -10722,13 +10589,13 @@ mod test { // Asking to start at the beginning is a filter, not a request for reliability. // Backfill needs a budget that covers it; without one the live edge still wins. - let mut subscriber = producer.subscribe(Subscription::default().with_start(Position::group(0))); + let mut subscriber = producer.subscribe(Subscription::default().with_start(0)); subscriber.start_at(0); assert_eq!(drain(&mut subscriber), vec![3]); let mut patient = producer.subscribe( Subscription::default() - .with_start(Position::group(0)) + .with_start(0) .with_max_delay(Duration::from_secs(10)), ); patient.start_at(0); @@ -10869,9 +10736,9 @@ mod test { append_untimed(&mut producer); } - let mut subscriber = producer.subscribe(replay().with_start(Position::group(2))); + let mut subscriber = producer.subscribe(replay().with_start(2)); assert_eq!(drain(&mut subscriber), vec![2, 3, 4]); - let mut live = producer.subscribe(Subscription::default().with_start(Position::group(2))); + let mut live = producer.subscribe(Subscription::default().with_start(2)); assert_eq!(drain(&mut live), vec![4]); } @@ -10885,7 +10752,7 @@ mod test { append_untimed(&mut producer); } - let mut subscriber = producer.subscribe(Subscription::default().with_end(Position::group(3))); + let mut subscriber = producer.subscribe(Subscription::default().with_end(3)); subscriber.end_at(Position::group(3).group_end()); assert_eq!(drain(&mut subscriber), vec![2]); } @@ -11338,7 +11205,7 @@ mod test { assert_eq!(next.sequence, 1); // A budget that covers it delivers the range, and nothing above it. - let mut subscriber = producer.subscribe(replay().with_end(Position::after_group(1))); + let mut subscriber = producer.subscribe(replay().with_end(2)); assert_eq!(drain(&mut subscriber), vec![0, 1]); } @@ -11404,7 +11271,7 @@ mod test { let subscription = Subscription::default() .with_max_delay(Duration::ZERO) .with_ranges([Range { - start: Anchor::Absolute(Position::group(0)), + start: Anchor::Absolute(0), end: Some(Anchor::Relative(0)), }]); let mut subscriber = producer.subscribe(subscription); @@ -11654,7 +11521,7 @@ mod test { use std::sync::atomic::{AtomicBool, Ordering}; let mut producer = track_producer("test", Info::timed()); - let _narrow = producer.subscribe(Subscription::default().with_end(Position::after_group(3))); + let _narrow = producer.subscribe(Subscription::default().with_end(4)); let mut wide = producer.subscribe(Subscription::default()); let woken = Arc::new(AtomicBool::new(false)); @@ -11667,14 +11534,13 @@ mod test { assert!(producer.poll_subscription_changed(&waiter).is_pending()); assert!(!woken.load(Ordering::SeqCst), "nothing happened yet"); - wide.update(Subscription::default().with_end(Position::after_group(5))) - .unwrap(); + wide.update(Subscription::default().with_end(6)).unwrap(); assert!( woken.load(Ordering::SeqCst), "the widest subscriber changing must wake the aggregate watcher", ); match producer.poll_subscription_changed(&waiter) { - Poll::Ready(Ok(Some(sub))) => assert_eq!(sub.end(), Position::after_group(5)), + Poll::Ready(Ok(Some(sub))) => assert_eq!(sub.end(), Some(6)), other => panic!("expected the narrowed aggregate, got {other:?}"), } } @@ -14135,7 +14001,13 @@ mod test { let dynamic = producer.dynamic(); let consumer = producer.consume(); - let _tail = consumer.fetch_group(5, group::Fetch::default().with_frame_start(3)); + let _tail = consumer.fetch_group( + 5, + group::Fetch { + frame_start: 3, + ..Default::default() + }, + ); let req = dynamic .requested_group() .now_or_never() @@ -14233,7 +14105,13 @@ mod test { group.finish().unwrap(); // A fetch for the tail is covered and resolves from the cache. - let fetch = consumer.fetch_group(0, group::Fetch::default().with_frame_start(3)); + let fetch = consumer.fetch_group( + 0, + group::Fetch { + frame_start: 3, + ..Default::default() + }, + ); let cached = fetch.now_or_never().expect("covered by the cache").unwrap(); assert_eq!(cached.index(), 3); @@ -14515,12 +14393,24 @@ mod test { let dynamic = producer.dynamic(); let consumer = producer.consume(); - let _narrow = consumer.fetch_group(0, group::Fetch::default().with_frame_start(5)); + let _narrow = consumer.fetch_group( + 0, + group::Fetch { + frame_start: 5, + ..Default::default() + }, + ); let mut narrow = std::pin::pin!(_narrow); assert!(futures::poll!(narrow.as_mut()).is_pending()); // Still queued: widening is honored, because nothing has read the range yet. - let _wider = consumer.fetch_group(0, group::Fetch::default().with_frame_start(2)); + let _wider = consumer.fetch_group( + 0, + group::Fetch { + frame_start: 2, + ..Default::default() + }, + ); let mut wider = std::pin::pin!(_wider); assert!(futures::poll!(wider.as_mut()).is_pending()); @@ -14529,7 +14419,13 @@ mod test { // Handed off now: the handler holds its own copy of the range, so a later wider // caller cannot move what is already being served. - let _widest = consumer.fetch_group(0, group::Fetch::default().with_frame_start(0)); + let _widest = consumer.fetch_group( + 0, + group::Fetch { + frame_start: 0, + ..Default::default() + }, + ); let mut widest = std::pin::pin!(_widest); assert!(futures::poll!(widest.as_mut()).is_pending()); assert_eq!(request.frame_start(), 2, "the in-flight range is already on the wire"); @@ -14851,51 +14747,6 @@ mod test { assert!(consumer.peek_group(1).is_none(), "unread group is evicted instead"); } - /// Requesting a tail from frame N reads the cached prefix it relies on, as a FETCH hit - /// does, so pressure evicts a never-read group first and the prefix stays to complete - /// the group. - #[test] - fn a_requested_prefix_survives_eviction() { - let (producer, pool) = pooled_producer(u64::MAX); - let consumer = producer.consume(); - // Incompressible, so each group caches about its payload. - let mut seed = 1u32; - let mut append = || { - let payload: Vec = (0..3_000) - .map(|_| { - seed = seed.wrapping_mul(1_103_515_245).wrapping_add(12_345); - (seed >> 16) as u8 - }) - .collect(); - let mut group = producer.append_group().unwrap(); - group.write_frame(Timestamp::ZERO, payload).unwrap(); - group.finish().unwrap(); - }; - - append(); // seq 0 - elapse(&producer, Duration::from_secs(1)); - append(); // seq 1 - elapse(&producer, Duration::from_secs(1)); - let used = pool.used(); - append(); // seq 2 - // Room for the three groups and a little more, so each later append evicts one. - pool.resize(pool.used() + (pool.used() - used) / 3); - elapse(&producer, Duration::from_millis(500)); - - let start = Anchor::Absolute(Position { group: 0, frame: 1 }); - let mut request = Subscription::default().with_ranges([Range { start, end: None }]); - producer.widen_start(&mut request); - assert_eq!(request.ranges[0].start, start, "a held prefix keeps the request at N"); - elapse(&producer, Duration::from_millis(500)); - - append(); // seq 3 - elapse(&producer, Duration::from_secs(1)); - append(); // seq 4 - - assert!(consumer.peek_group(0).is_some(), "the requested prefix survives"); - assert!(consumer.peek_group(1).is_none(), "an unread group is evicted instead"); - } - /// A consumer holding an evicted group surfaces the eviction, not a hang or a /// truncated clean end. #[moq_clock::test] @@ -16420,23 +16271,6 @@ mod test { assert_eq!(drain(&mut ahead), vec![8]); } - /// A frame-precise start survives the merge with a live range, for the publisher to - /// trim on the wire. Locally it floors reads at its group. - #[test] - #[cfg(feature = "async")] - fn a_frame_precise_start_merged_with_live_keeps_its_frame() { - let producer = quiet_track(3); - let precise = Range { - start: Anchor::Absolute(Position { group: 1, frame: 2 }), - end: None, - }; - let mut from_frame = producer.subscribe(replay().with_ranges([precise])); - let mut live = producer.subscribe(replay().with_ranges([Range::live()])); - assert_eq!(producer.subscription().unwrap().ranges, vec![precise, Range::live()]); - assert_eq!(drain(&mut from_frame), vec![1, 2, 3]); - assert_eq!(drain(&mut live), vec![3]); - } - /// An untimed track replays from group 0 only what the wall clock hasn't made stale. #[test] #[cfg(feature = "async")] @@ -17060,7 +16894,7 @@ mod test { two.write_frame(at(2000), bytes::Bytes::from_static(b"x")).unwrap(); let mut subscriber = producer.subscribe(replay().with_ranges([Range { - start: Anchor::Absolute(Position::group(0)), + start: Anchor::Absolute(0), end: Some(Anchor::Time(at(500))), }])); hole.write_frame(at(1000), bytes::Bytes::from_static(b"x")).unwrap(); @@ -17075,7 +16909,7 @@ mod test { fn a_time_end_waits_on_an_absent_group_below_it() { let at = |millis| Timestamp::from_millis(millis).unwrap(); let ranges = [Range { - start: Anchor::Absolute(Position::group(0)), + start: Anchor::Absolute(0), end: Some(Anchor::Time(at(500))), }]; for dropped in [false, true] { @@ -17153,7 +16987,7 @@ mod test { .map(|sequence| producer.create_group(group::Info { sequence }).unwrap()) .collect(); let mut subscriber = producer.subscribe(replay().with_ranges([Range { - start: Anchor::Absolute(Position::group(0)), + start: Anchor::Absolute(0), end: Some(Anchor::Time(at(3500))), }])); assert_eq!(drain(&mut subscriber), vec![0]); @@ -17198,7 +17032,7 @@ mod test { let at = |millis| Timestamp::from_millis(millis).unwrap(); let mut producer = track_producer("test", Info::timed()); let mut subscriber = producer.subscribe(replay().with_ranges([Range { - start: Anchor::Absolute(Position::group(0)), + start: Anchor::Absolute(0), end: Some(Anchor::Time(at(3500))), }])); for millis in [3000, 3400, 3600, 3700] { diff --git a/rs/moq-net/src/model/track/demand.rs b/rs/moq-net/src/model/track/demand.rs index 80618c813b2..16e2015318a 100644 --- a/rs/moq-net/src/model/track/demand.rs +++ b/rs/moq-net/src/model/track/demand.rs @@ -288,7 +288,7 @@ impl From> for Preferences { #[cfg(test)] mod tests { use super::super::super::subscription::Combined; - use super::super::{Anchor, Position, Range}; + use super::super::{Anchor, Range}; use super::*; use rand::{RngExt, SeedableRng}; use std::sync::atomic::{AtomicUsize, Ordering}; @@ -305,10 +305,7 @@ mod tests { let anchor = |rng: &mut rand::rngs::StdRng| match rng.random_range(0..4) { 0 => Anchor::Relative(rng.random_range(0..3)), 1 => Anchor::Relative(u64::MAX), - _ => Anchor::Absolute(Position { - group: rng.random_range(0..8), - frame: rng.random_range(0..4), - }), + _ => Anchor::Absolute(rng.random_range(0..8)), }; let ranges: Vec = (0..rng.random_range(0..3)) .map(|_| Range { diff --git a/rs/moq-net/src/model/track/demand/ranges.rs b/rs/moq-net/src/model/track/demand/ranges.rs index 88bae86d296..30f4595b89d 100644 --- a/rs/moq-net/src/model/track/demand/ranges.rs +++ b/rs/moq-net/src/model/track/demand/ranges.rs @@ -1,7 +1,7 @@ //! The union of every subscriber's ranges, kept current per change rather than refolded. //! -//! Absolute ranges become signed boundaries over positions, +1 where one starts and -1 -//! where it ends, so a position is covered while the running sum stays above zero. A treap +//! Absolute ranges become signed boundaries over groups, +1 where one starts and -1 +//! where it ends, so a group is covered while the running sum stays above zero. A treap //! keeps each subtree's sum and lowest running sum, so a change touches two boundaries and //! asks one range-minimum question, whatever else the audience holds. Ranges anchored to //! the latest group or to a time can't be placed on that line and are counted by value @@ -11,7 +11,7 @@ use std::cell::OnceCell; use std::collections::BTreeMap; use std::hash::{BuildHasher, RandomState}; -use super::super::{Anchor, Position, Range}; +use super::super::{Anchor, Range}; use super::{add, remove}; /// Every subscriber's ranges, counted so that one change costs O(log n). @@ -32,7 +32,7 @@ pub(super) struct Ranges { enum Shape { Empty, Oldest, - Absolute(Position, Option), + Absolute(u64, Option), Relative(u64), Mixed, } @@ -146,7 +146,7 @@ impl Ranges { } } -/// Signed boundary counts over positions: the coverage of the absolute ranges. +/// Signed boundary counts over groups: the coverage of the absolute ranges. #[derive(Default)] struct Coverage { root: Link, @@ -156,7 +156,7 @@ struct Coverage { type Link = Option>; struct Node { - key: Position, + key: u64, /// Ranges starting here, minus ranges ending here. Never zero. delta: i64, priority: u64, @@ -170,14 +170,14 @@ struct Node { impl Coverage { /// Count `delta` more ranges over `start..end`, open when `end` is `None`. - fn add(&mut self, start: Position, end: Option, delta: i64) { + fn add(&mut self, start: u64, end: Option, delta: i64) { self.shift(start, delta); if let Some(end) = end { self.shift(end, -delta); } } - fn shift(&mut self, key: Position, delta: i64) { + fn shift(&mut self, key: u64, delta: i64) { let (below, rest) = split(self.root.take(), &key, false); let (at, above) = split(rest, &key, true); let delta = at.map_or(0, |node| node.delta) + delta; @@ -185,8 +185,8 @@ impl Coverage { self.root = merge(merge(below, at), above); } - /// The fewest ranges covering any position in `start..end`. - fn min(&mut self, start: Position, end: Option) -> i64 { + /// The fewest ranges covering any group in `start..end`. + fn min(&mut self, start: u64, end: Option) -> i64 { let (through, rest) = split(self.root.take(), &start, true); let (inside, above) = match end { Some(end) => split(rest, &end, false), @@ -199,14 +199,14 @@ impl Coverage { } /// The covered spans in order, the last one open while an open range remains. - fn runs(&self) -> Vec<(Position, Option)> { + fn runs(&self) -> Vec<(u64, Option)> { let mut runs = Vec::new(); walk(&self.root, 0, &mut runs); runs } /// Where the open span starts, if an open range remains. - fn tail(&self) -> Option { + fn tail(&self) -> Option { if sum(&self.root) <= 0 { return None; } @@ -218,7 +218,7 @@ impl Coverage { } impl Node { - fn new(key: Position, delta: i64, priority: u64) -> Box { + fn new(key: u64, delta: i64, priority: u64) -> Box { Box::new(Self { key, delta, @@ -246,7 +246,7 @@ fn low(link: &Link) -> i64 { } /// Split into the keys below `key` (through it when `inclusive`) and the rest. -fn split(link: Link, key: &Position, inclusive: bool) -> (Link, Link) { +fn split(link: Link, key: &u64, inclusive: bool) -> (Link, Link) { let Some(mut node) = link else { return (None, None); }; @@ -281,7 +281,7 @@ fn merge(left: Link, right: Link) -> Link { } } -fn first(link: &Link) -> Option { +fn first(link: &Link) -> Option { let mut node = link.as_ref()?; while let Some(left) = &node.left { node = left; @@ -290,7 +290,7 @@ fn first(link: &Link) -> Option { } /// The first key above `after`. -fn successor(mut link: &Link, after: &Position) -> Option { +fn successor(mut link: &Link, after: &u64) -> Option { let mut found = None; while let Some(node) = link { if node.key > *after { @@ -307,7 +307,7 @@ fn successor(mut link: &Link, after: &Position) -> Option { /// /// A subtree entered covered whose running sum never reaches zero holds no boundary of the /// union, so it is skipped whole: the walk visits O(log n) nodes per span it returns. -fn walk(link: &Link, offset: i64, runs: &mut Vec<(Position, Option)>) { +fn walk(link: &Link, offset: i64, runs: &mut Vec<(u64, Option)>) { let Some(node) = link else { return; }; @@ -326,7 +326,7 @@ fn walk(link: &Link, offset: i64, runs: &mut Vec<(Position, Option)>) } /// The last key where the running sum, starting at `offset`, is zero. -fn last_zero(link: &Link, offset: i64) -> Option { +fn last_zero(link: &Link, offset: i64) -> Option { let node = link.as_ref()?; if offset.saturating_add(node.low) > 0 { return None; @@ -355,10 +355,7 @@ mod tests { let anchor = |rng: &mut rand::rngs::StdRng| match rng.random_range(0..6) { 0 => Anchor::Relative(rng.random_range(0..4)), 1 => Anchor::Relative(u64::MAX), - _ => Anchor::Absolute(Position { - group: rng.random_range(0..12), - frame: rng.random_range(0..3), - }), + _ => Anchor::Absolute(rng.random_range(0..12)), }; let mut ranges = Ranges::default(); let mut held: Vec = Vec::new(); diff --git a/rs/moq-net/tests/integration/compress.rs b/rs/moq-net/tests/integration/compress.rs index 074d401babc..c173cba6986 100644 --- a/rs/moq-net/tests/integration/compress.rs +++ b/rs/moq-net/tests/integration/compress.rs @@ -119,7 +119,7 @@ async fn relay(published: Published, hops: [&str; 2], subscription: moq_net::tra } } -/// Hops that carry a mid-group start and negotiate compression: lite-07, and moq-transport +/// Hops that carry a start group and negotiate compression: lite-07, and moq-transport /// draft-17 and later. const RANGED: &[&str] = &[ "moq-lite-07-wip", @@ -128,19 +128,11 @@ const RANGED: &[&str] = &[ "moq-transport-22", ]; -/// A cold subscriber asks from the group's first frame, compressed or not, but its -/// application sees only its requested half-open frame range across every pair of hops, -/// whether the group began before it subscribed or after. Lite-07 hops also carry a live -/// change. +/// A subscription starting at a group delivers it whole from its first frame across every +/// pair of hops, compressed or not, whether the group began before it subscribed or after. #[moq_clock::test] -async fn application_ranges_keep_their_filter_after_requesting_the_group_head() { - use moq_net::track::{Anchor, Position, Range, Subscription}; - let range = |group, start, end: Option| { - Subscription::default().with_ranges([Range { - start: Anchor::Absolute(Position { group, frame: start }), - end: end.map(|end| Anchor::Absolute(Position { group, frame: end })), - }]) - }; +async fn a_start_group_arrives_whole() { + use moq_net::track::{Anchor, Subscription}; let write = |group: &mut moq_net::group::Producer, frames: std::ops::Range| { for index in frames { group @@ -152,11 +144,6 @@ async fn application_ranges_keep_their_filter_after_requesting_the_group_head() for &downstream in RANGED { let hops = [upstream, downstream]; let lite = hops.iter().all(|hop| *hop == "moq-lite-07-wip"); - // A pre-draft-20 join cannot carry an end, so its range stays open. - let end = hops - .iter() - .all(|hop| matches!(*hop, "moq-lite-07-wip" | "moq-transport-22")) - .then_some(4); for compress in [true, false] { // Frames of the group written before the subscription, which a joining FETCH // or fill delivers. @@ -173,18 +160,19 @@ async fn application_ranges_keep_their_filter_after_requesting_the_group_head() .create_group(moq_net::group::Info { sequence: 5 }) .unwrap(); write(&mut group, 0..head); - let mut fixture = relay(published, hops, range(5, 2, end)).await; + let mut fixture = relay(published, hops, Subscription::default().with_groups(5..)).await; if lite { assert_eq!( fixture.track.subscription_changed().await.unwrap().unwrap().ranges[0].start, - Anchor::Absolute(Position::group(5)), - "{case}: upstream asks from the group's first frame" + Anchor::Absolute(5), + "{case}: upstream asks for the start group" ); } write(&mut group, head..5); group.finish().unwrap(); let mut reading = fixture.subscriber.recv_group().await.unwrap().unwrap(); - for index in 2..end.unwrap_or(5) { + assert_eq!(reading.sequence, 5, "{case}"); + for index in 0..5 { assert_eq!( reading.read_frame().await.unwrap().unwrap().payload, payload(index), @@ -192,33 +180,9 @@ async fn application_ranges_keep_their_filter_after_requesting_the_group_head() ); } assert!(reading.read_frame().await.unwrap().is_none(), "{case}"); - - // REQUEST_UPDATE carries no range: /quest/m1/subscribe-ranges/ietf.md. - if !lite { - return; - } - fixture.subscriber.update(range(6, 1, Some(3))).unwrap(); - let updated = fixture.track.subscription_changed().await.unwrap().unwrap(); - // Old route copies can keep their earlier floor until a newer group arrives. - // The updated end must already permit the newly requested group. - assert_eq!( - updated.ranges[0].end, - Some(Anchor::Absolute(Position { group: 6, frame: 3 })) - ); - let mut group = fixture - .track - .create_group(moq_net::group::Info { sequence: 6 }) - .unwrap(); - write(&mut group, 0..5); - group.finish().unwrap(); - let mut reading = fixture.subscriber.recv_group().await.unwrap().unwrap(); - for index in 1..3 { - assert_eq!(reading.read_frame().await.unwrap().unwrap().payload, payload(index)); - } - assert!(reading.read_frame().await.unwrap().is_none()); }) .await - .unwrap_or_else(|_| panic!("{case}: the requested frames never arrived")); + .unwrap_or_else(|_| panic!("{case}: the group never arrived whole")); } } } diff --git a/rs/moq-net/tests/integration/head_fetch.rs b/rs/moq-net/tests/integration/head_fetch.rs deleted file mode 100644 index 7dcb5459d49..00000000000 --- a/rs/moq-net/tests/integration/head_fetch.rs +++ /dev/null @@ -1,171 +0,0 @@ -#![cfg(feature = "async")] - -//! A relay that fetches the head of a group it is receiving mid-group keeps delivering -//! that group to arrival-order subscribers, along with the frames written into it after. -//! -//! A mesh peer subscribing from partway into an open group (a catalog: snapshot, then -//! deltas) has its relay pull the group headless. Reading the peer's copy from frame 0 -//! then fetches the head through the relay, whose cache must not trade the live group -//! for the fetched copy. - -use crate::support; - -use std::time::Duration; - -use moq_net::track::{Anchor, Position, Range, Subscription}; -use moq_net::{Hop, Timestamp, Version}; -use support::harness::{MockConnectOptions, connect_mock}; - -/// A millisecond track: the frames written here carry timestamps. -fn timed() -> moq_net::track::Info { - moq_net::track::Info::default().with_timescale(moq_net::Timescale::MILLI) -} - -const TIMEOUT: Duration = Duration::from_secs(5); - -const VERSIONS: &[&str] = &[ - "moq-lite-03", - "moq-lite-05", - "moq-lite-06", - "moq-lite-07-wip", - "moq-transport-14", - "moq-transport-17", - "moq-transport-22", -]; - -fn produce_origin(hop: u64) -> moq_net::origin::Producer { - let (producer, driver) = moq_net::origin::Producer::new(moq_net::origin::Config::new(Hop::new(hop).unwrap())); - support::harness::spawn(driver); - producer -} - -async fn resolve(origin: &moq_net::origin::Producer) -> moq_net::broadcast::Consumer { - let consumer = origin.consume(); - moq_clock::timeout(TIMEOUT, consumer.routed("bcast")) - .await - .expect("announce timeout") - .expect("routed"); - moq_clock::timeout(TIMEOUT, consumer.request_broadcast("bcast", None)) - .await - .expect("resolve timeout") - .expect("broadcast resolves") -} - -async fn read(group: &mut moq_net::group::Consumer) -> Vec { - moq_clock::timeout(TIMEOUT, group.read_frame()) - .await - .expect("no frame") - .expect("read_frame") - .expect("group ended") - .payload - .to_vec() -} - -/// Publisher P holds an open group, relay R pulls P, peer M pulls R from the group's -/// second frame and then reads its copy from the first. A fresh subscriber on R must -/// still get the whole group, and a frame P writes afterward. -async fn round(version: &str) { - let version: Version = version.parse().unwrap(); - let publisher = produce_origin(1); - let broadcast = publisher.create_broadcast("bcast").unwrap(); - let track = broadcast.create_track("catalog.json", timed()).unwrap(); - broadcast.announce(Default::default()).unwrap(); - - let mut group = track.append_group().unwrap(); - let sequence = group.sequence; - for payload in [&b"snapshot"[..], b"delta1", b"delta2"] { - group.write_frame(Timestamp::now(), payload).unwrap(); - } - - let relay = produce_origin(2); - let mut options = MockConnectOptions::new(version); - options.server_publish = Some(publisher.consume()); - options.client_subscribe = Some(relay.clone()); - let upstream = connect_mock(options).await; - - let peer = produce_origin(3); - let mut options = MockConnectOptions::new(version); - options.server_publish = Some(relay.consume()); - options.client_subscribe = Some(peer.clone()); - let downstream = connect_mock(options).await; - - let remote = resolve(&relay).await; - let peer_remote = resolve(&peer).await; - - let start = Position { - group: sequence, - frame: 1, - }; - let mut peer_sub = peer_remote - .track("catalog.json") - .unwrap() - .subscribe(Subscription::default().with_ranges([Range { - start: Anchor::Absolute(start), - end: None, - }])) - .await - .expect("peer subscribe"); - let mut peer_group = moq_clock::timeout(TIMEOUT, peer_sub.recv_group()) - .await - .expect("peer got no group") - .unwrap() - .unwrap(); - // The peer reads its headless copy from frame 0: a fetch of the head through R. - let _ = moq_clock::timeout(Duration::from_millis(500), peer_group.read_frame()).await; - - // A fresh reader on R wants the latest group from its snapshot. - let mut sub = remote.track("catalog.json").unwrap().subscribe(None).await.unwrap(); - let mut got = moq_clock::timeout(TIMEOUT, sub.recv_group()) - .await - .unwrap_or_else(|_| panic!("{version}: no group")) - .unwrap() - .unwrap(); - assert_eq!(got.sequence, sequence, "{version}"); - assert_eq!(read(&mut got).await, b"snapshot", "{version}"); - assert_eq!(read(&mut got).await, b"delta1", "{version}"); - assert_eq!(read(&mut got).await, b"delta2", "{version}"); - - // The publisher keeps writing into the group: the reader on R hears it. - moq_clock::sleep(Duration::from_secs(1)).await; - group.write_frame(Timestamp::now(), &b"delta3"[..]).unwrap(); - assert_eq!( - read(&mut got).await, - b"delta3", - "{version}: frame written after the fetch" - ); - - // Every reader of the fetched copy leaves, so its fetch is abandoned, while both - // subscriptions keep the live feed running. The live group still holds R's slot: a - // later reader gets the whole group again. - drop((got, peer_group)); - moq_clock::sleep(Duration::from_secs(1)).await; - let mut late_sub = remote.track("catalog.json").unwrap().subscribe(None).await.unwrap(); - let mut late = moq_clock::timeout(TIMEOUT, late_sub.recv_group()) - .await - .unwrap_or_else(|_| panic!("{version}: no group after the fetch was abandoned")) - .unwrap() - .unwrap(); - assert_eq!(late.sequence, sequence, "{version}"); - for payload in [&b"snapshot"[..], b"delta1", b"delta2", b"delta3"] { - assert_eq!( - read(&mut late).await, - payload, - "{version}: after the fetch was abandoned" - ); - } - - drop(( - late, late_sub, sub, peer_sub, group, track, broadcast, downstream, upstream, publisher, relay, peer, - )); -} - -#[moq_clock::test] -async fn a_fetched_head_keeps_the_live_group_visible() { - let mut failures = Vec::new(); - for version in VERSIONS { - if moq_clock::sim::spawn(round(version)).await.is_err() { - failures.push(*version); - } - } - assert!(failures.is_empty(), "failed: {failures:?}"); -} diff --git a/rs/moq-net/tests/integration/late_lower_group.rs b/rs/moq-net/tests/integration/late_lower_group.rs index 2cf29eee790..a7aa035cdbd 100644 --- a/rs/moq-net/tests/integration/late_lower_group.rs +++ b/rs/moq-net/tests/integration/late_lower_group.rs @@ -11,7 +11,7 @@ use crate::support; use std::time::Duration; -use moq_net::track::{Anchor, Info, Position, Range, Subscription}; +use moq_net::track::{Anchor, Info, Range, Subscription}; use moq_net::{Timestamp, Version}; use support::harness::{MockConnectOptions, connect_mock}; @@ -28,7 +28,7 @@ const LITE: &[&str] = &["moq-lite-05", "moq-lite-06", "moq-lite-07-wip"]; const TRANSPORT: &[&str] = &["moq-transport-14", "moq-transport-17", "moq-transport-22"]; /// From `start`, or the oldest group still around when there is none. -fn floored(start: Option) -> Range { +fn floored(start: Option) -> Range { Range { start: start.map_or(Range::oldest().start, Anchor::Absolute), end: None, @@ -79,7 +79,7 @@ async fn read_all(sub: &mut moq_net::track::Subscriber) -> Vec<(u64, usize)> { } /// Group 1, then a fresh group 0, over `version`. `relay` inserts one hop. -async fn session(version: &str, relay: bool, start: Option) -> Vec<(u64, usize)> { +async fn session(version: &str, relay: bool, start: Option) -> Vec<(u64, usize)> { let version: Version = version.parse().unwrap(); let publisher = produce_origin(1); let broadcast = publisher.create_broadcast("bcast").unwrap(); @@ -156,7 +156,7 @@ async fn session(version: &str, relay: bool, start: Option) -> Vec<(u6 got } -async fn in_process(start: Option) -> Vec<(u64, usize)> { +async fn in_process(start: Option) -> Vec<(u64, usize)> { let publisher = produce_origin(1); let broadcast = publisher.create_broadcast("bcast").unwrap(); let track = broadcast @@ -190,14 +190,14 @@ async fn an_explicit_floor_receives_a_group_created_below_the_first_served_group let want = vec![(1, 4), (0, 4)]; let mut failures = Vec::new(); - let got = in_process(Some(Position::group(0))).await; + let got = in_process(Some(0)).await; if got != want { failures.push(format!("in-process: {got:?}")); } for version in LITE.iter().chain(TRANSPORT) { for relay in [false, true] { - let got = session(version, relay, Some(Position::group(0))).await; + let got = session(version, relay, Some(0)).await; if got != want { failures.push(format!("{version} relay={relay}: {got:?}")); } diff --git a/rs/moq-net/tests/integration/main.rs b/rs/moq-net/tests/integration/main.rs index 7669be7f7b2..2b63be5e8cd 100644 --- a/rs/moq-net/tests/integration/main.rs +++ b/rs/moq-net/tests/integration/main.rs @@ -30,7 +30,6 @@ mod finished_broadcast_mock; mod finite_relay; mod goaway; mod group_order; -mod head_fetch; mod history_groups; mod idle_cancel_refetch; mod idle_session; diff --git a/rs/moq-transcode/src/lib.rs b/rs/moq-transcode/src/lib.rs index 7ab0b48c9ea..d94543a3b3c 100644 --- a/rs/moq-transcode/src/lib.rs +++ b/rs/moq-transcode/src/lib.rs @@ -1402,70 +1402,6 @@ mod tests { transcoder.abort(); } - /// A fetch that starts partway through a group is refused rather than served. - /// A fresh encode of that group need not match the bytes of the head a reader - /// holds, so its tail cannot continue it. The reader moves on to the next group. - #[tokio::test] - async fn a_mid_group_fetch_is_refused() { - let source = source_catalog(320, 240); - // Serve any source group on demand, so a fetch that is not refused has - // something to transcode and resolves. - let source_fetches = source._track.dynamic(); - let server = tokio::spawn(async move { - while let Ok(request) = source_fetches.requested_group().await { - let mut group = request.accept(None).unwrap(); - write_keyframe(&mut group); - group.finish().unwrap(); - } - }); - - let config = Config { - ladder: Ladder::new([Rung::new(120, moq_net::bandwidth::Rate::from_bps(100_000))]).unwrap(), - encoder: moq_video::encode::Kind::Software, - decoder: moq_video::decode::Kind::Software, - source: None, - ..Default::default() - }; - let output = moq_net::broadcast::Info::default().produce(); - let consumer = output.consume(); - let transcoder = tokio::spawn(run(source.broadcast.consume(), output, config)); - - let catalog = loop { - match consumer.track(hang::Catalog::DEFAULT_NAME) { - Ok(track) => break track, - Err(moq_net::Error::NotFound) => tokio::task::yield_now().await, - Err(err) => panic!("catalog track: {err}"), - } - }; - let mut catalogs = moq_mux::catalog::hang::Consumer::<()>::new(catalog.subscribe(None).await.unwrap()); - await_catalog(&mut catalogs, |snapshot| { - snapshot.video.renditions.contains_key("video/120p") - }) - .await; - - // The source publishes no live groups, so group 7 is a cache miss that - // reaches the rung's fetch handler. - let rung = consumer.track("video/120p").unwrap(); - rung.query().await.unwrap(); - - let partial = rung - .fetch_group(7, moq_net::group::Fetch::default().with_frame_start(2)) - .await; - match partial { - Err(moq_net::Error::NotFound) => {} - Err(err) => panic!("expected a NotFound refusal, got {err}"), - Ok(_) => panic!("served a fetch that starts mid-group"), - } - - // The whole group still serves. - let mut whole = rung.fetch_group(7, None).await.unwrap(); - while whole.read_frame().await.unwrap().is_some() {} - assert!(whole.finished().await.unwrap() > 0, "the whole group had no frames"); - - server.abort(); - transcoder.abort(); - } - /// A fetch dropped before its group is encoded never opens that encode, and /// the request goes with it. A later fetch of the same group is a new /// request: the abandoned one must not still be encoding into the cache, diff --git a/swift/MIGRATE.md b/swift/MIGRATE.md index 4e6bf56fcd1..e181b57caa7 100644 --- a/swift/MIGRATE.md +++ b/swift/MIGRATE.md @@ -4,6 +4,10 @@ How to update your code for each breaking change, newest first. ## Unreleased +### `Subscription.Anchor.absolute` names a group ([#6465](https://github.com/moq-dev/moq/pull/6465)) + +`.absolute(group:frame:)` is `.absolute(group:)`: a range always starts at a group's first frame. + ### `announce` takes an `AnnounceRoute` `BroadcastProducer.announce(route:)` takes `AnnounceRoute(cost:epoch:)` instead of a `Route`: the origin builds the hop chain, so `hops` is no longer an input. diff --git a/swift/Sources/Moq/Aliases.swift b/swift/Sources/Moq/Aliases.swift index d01eccb6feb..bba8892e77c 100644 --- a/swift/Sources/Moq/Aliases.swift +++ b/swift/Sources/Moq/Aliases.swift @@ -118,7 +118,7 @@ public typealias ProtocolError = MoqFFI.MoqProtocolError extension Subscription { /// A span of a track to deliver: from `start` up to `end`, or onward when `end` is `nil`. public typealias Range = MoqFFI.MoqRange - /// Where one end of a `Range` falls: `.absolute(group:frame:)`, `.relative(groups:)` + /// Where one end of a `Range` falls: `.absolute(group:)`, `.relative(groups:)` /// back from the latest group, or `.time(timestampUs:)` for the group containing it. public typealias Anchor = MoqFFI.MoqAnchor /// Which pending group the publisher sends first: `.ascending` or `.descending`. From cec4011bbff337f6ad370fc5fd6dcbfb75eec6ff Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 23 Oct 2026 14:34:34 -0700 Subject: [PATCH 05/10] fix(net): drop the doc link to the private frame_start and a stale quest link Co-Authored-By: Claude Opus 5.5 --- quest/m1/ietf-extension/README.md | 6 +++--- rs/moq-net/src/model/track.rs | 11 ++++++----- 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/quest/m1/ietf-extension/README.md b/quest/m1/ietf-extension/README.md index cee1283e8db..251d8fc8ff2 100644 --- a/quest/m1/ietf-extension/README.md +++ b/quest/m1/ietf-extension/README.md @@ -112,9 +112,9 @@ them yet. Remaining, in order: edge's TRACK_INFO query becomes TRACK_STATUS upstream, and the pinned MoQTestPublisher inherits NOT_SUPPORTED. Preserve that refusal: widening query-only demand into a SUBSCRIBE is not an authorized fallback. - [Range widening](/quest/m1/compress/ietf-range-widen.md), claimed by - [moq-dev/moq#6465](https://github.com/moq-dev/moq/pull/6465), owns that - separate cold-metadata boundary. Needs a Moxygen revision whose test + No cold request needs that metadata to pick its start, since + [moq-dev/moq#6465](https://github.com/moq-dev/moq/pull/6465) starts every + subscription at a group's first frame. Needs a Moxygen revision whose test publisher answers TRACK_STATUS; then bump the pins, rebase the profile, and add the two-relay cell back to `test/ietf-conformance/run.sh` (it ran in [d96126c65a](https://github.com/moq-dev/moq/commit/d96126c65a52027820884cee7cb9741e79fb674b)). diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 6a74d0bf126..f8dcc642e7e 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -4777,13 +4777,14 @@ impl group::Request { /// The first frame of the group the consumer wants; 0 is the whole group. /// - /// The group [`accept`](Self::accept) returns already starts here, so the frames a - /// handler writes carry the indices they have in the group rather than restarting at - /// 0. A handler serving from elsewhere moves it with - /// [`start_at`](group::Producer::start_at) before the first frame. + /// Only a peer's partial FETCH or a route change filling the rest of a group sets it; + /// an application always fetches whole groups. The group [`accept`](Self::accept) + /// returns already starts here, so the frames a handler writes carry the indices they + /// have in the group rather than restarting at 0. A handler serving from elsewhere + /// moves it with [`start_at`](group::Producer::start_at) before the first frame. /// /// There is no end: the handler fetches through the end of the group so the result - /// is cacheable for anyone (see [`group::Fetch::frame_start`]). + /// is cacheable for anyone. pub fn frame_start(&self) -> u64 { self.frame_start } From 06ad3f25bad1b8e5983b18b8e1039014969d4dac Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 23 Oct 2026 18:23:26 -0700 Subject: [PATCH 06/10] style(net): rustfmt after merging main Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/ietf/subscriber.rs | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/rs/moq-net/src/ietf/subscriber.rs b/rs/moq-net/src/ietf/subscriber.rs index 09037fad417..b0c87e02b62 100644 --- a/rs/moq-net/src/ietf/subscriber.rs +++ b/rs/moq-net/src/ietf/subscriber.rs @@ -4787,11 +4787,7 @@ enum IngestPhase { } impl GroupIngest { - fn new( - subscriber: &Subscriber, - group: &ietf::GroupHeader, - timescale: Option, - ) -> Self { + fn new(subscriber: &Subscriber, group: &ietf::GroupHeader, timescale: Option) -> Self { Self { has_extensions: group.flags.has_extensions, timescale, From f2261135372d49ed11397510461434ad6820d781 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 23 Oct 2026 19:02:33 -0700 Subject: [PATCH 07/10] fix(net): cut lite datagrams by Object ID against both frame bounds A relayed moq-transport datagram keeps its Object ID, so the start bound compares against it rather than dropping the whole start group, and the end bound applies too, as the model window did before. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/lite/publisher.rs | 104 +++++++++++++++++++------------ 1 file changed, 64 insertions(+), 40 deletions(-) diff --git a/rs/moq-net/src/lite/publisher.rs b/rs/moq-net/src/lite/publisher.rs index f6c35791304..1166f4b89ed 100644 --- a/rs/moq-net/src/lite/publisher.rs +++ b/rs/moq-net/src/lite/publisher.rs @@ -4172,11 +4172,15 @@ impl TrackRun { return; } let sequence = datagram.sequence; - // A Frame Start inside the start group excludes its datagram, the group's only frame. - // The model bounds whole groups, so the frame bound is applied here. + // The model bounds whole groups, so the peer's frame bounds are applied here, against + // the Object ID a relayed moq-transport datagram keeps (a public one is frame 0). + let object = datagram.wire.as_ref().and_then(|wire| wire.object_id).unwrap_or(0); if self .start_frame - .is_some_and(|(group, frame)| group == sequence && frame > 0) + .is_some_and(|(group, frame)| group == sequence && object < frame) + || self + .end_frame + .is_some_and(|(group, frame)| group == sequence && object > frame) { return; } @@ -5721,48 +5725,68 @@ mod serve_group_test { assert!(order > 1); } - /// A Frame Start inside the start group excludes that group's datagram, its only frame, - /// while a later group's datagram is served. + /// The peer's frame bounds cut datagrams by Object ID: a public datagram is frame 0, a + /// relayed moq-transport one keeps its own, and both ends are inclusive. #[moq_clock::test] - async fn a_frame_start_drops_the_start_groups_datagram() { - let mut track = track::Producer::new( - Arc::new(broadcast::Info::default()), - "test", - crate::track::Info::timed(), - ); - let subscriber = track.subscribe(track::Subscription::default().with_max_delay(Duration::from_secs(3600))); - let session = SinkSession::new(Log::default()).with_groups().with_datagrams(1200); - let log = session.log.clone(); - let (mut run, mut writer, _) = lite_run(Version::Lite06, session.clone(), subscriber); - run.ctx.group = crate::transport::poll::Session::send_group(&mut run.ctx.session, 3); - run.update(lite::SubscribeUpdate { - wanted_group: None, - order: Order::Descending, - priority: 0, - max_delay: Duration::from_secs(3600), - start_group: Some(5), - end_group: None, - start_frame: 1, - end_frame: None, - }); + async fn frame_bounds_cut_datagrams_by_object_id() { + for (sequence, object_id, served) in [ + (5, None, false), + (5, Some(1), false), + (5, Some(2), true), + (6, None, true), + (7, Some(3), true), + (7, Some(4), false), + ] { + let mut track = track::Producer::new( + Arc::new(broadcast::Info::default()), + "test", + crate::track::Info::timed(), + ); + let subscriber = track.subscribe(track::Subscription::default().with_max_delay(Duration::from_secs(3600))); + let session = SinkSession::new(Log::default()).with_groups().with_datagrams(1200); + let log = session.log.clone(); + let (mut run, mut writer, _) = lite_run(Version::Lite06, session.clone(), subscriber); + run.ctx.group = crate::transport::poll::Session::send_group(&mut run.ctx.session, 3); + run.update(lite::SubscribeUpdate { + wanted_group: None, + order: Order::Descending, + priority: 0, + max_delay: Duration::from_secs(3600), + start_group: Some(5), + end_group: Some(7), + start_frame: 2, + end_frame: Some(3), + }); - for sequence in [5, 6] { track - .insert_datagram( - sequence, - Timestamp::from_millis(sequence).unwrap(), - bytes::Bytes::from_static(b"d"), - ) + .insert_wire_datagram(crate::model::datagram::Stored { + datagram: crate::model::datagram::Datagram { + sequence, + timestamp: Some(Timestamp::from_millis(sequence).unwrap()), + payload: bytes::Bytes::from_static(b"d"), + }, + wire: object_id.map(|object_id| { + Arc::new(crate::model::datagram::Wire { + object_id: Some(object_id), + priority: None, + end_of_group: false, + status: None, + properties: Default::default(), + }) + }), + deflated: None, + }) .unwrap(); + { + let mut poll = std::pin::pin!(kio::wait(|waiter| run.poll(&mut writer, waiter))); + assert!(futures::poll!(poll.as_mut()).is_pending()); + } + assert_eq!( + log.datagram_ranks().len(), + usize::from(served), + "group {sequence} object {object_id:?}" + ); } - { - let mut poll = std::pin::pin!(kio::wait(|waiter| run.poll(&mut writer, waiter))); - assert!(futures::poll!(poll.as_mut()).is_pending()); - } - assert_eq!( - log.datagram_ranks(), - [(Some(0), crate::priority::in_group(6, 0, Order::Descending))] - ); } /// A SUBSCRIBE_UPDATE that flips the group order reranks the next datagram, even with no From 73291be02761922791376b05999c240cd80a13ac Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 24 Oct 2026 06:14:51 -0700 Subject: [PATCH 08/10] test(net): anchor main's range test on a group sequence Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/tests/integration/range_anchors.rs | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/rs/moq-net/tests/integration/range_anchors.rs b/rs/moq-net/tests/integration/range_anchors.rs index 7a4dcb2fea5..21f5bc31e90 100644 --- a/rs/moq-net/tests/integration/range_anchors.rs +++ b/rs/moq-net/tests/integration/range_anchors.rs @@ -9,7 +9,7 @@ use crate::support; use std::time::Duration; -use moq_net::track::{Anchor, Info, Order, Position, Range, Subscription}; +use moq_net::track::{Anchor, Info, Order, Range, Subscription}; use moq_net::{Error, Hop, Timescale, Timestamp, Version}; use support::harness::peer; @@ -107,7 +107,7 @@ fn range(start: Anchor, end: Option) -> Range { /// Relative and time anchors at each end, with what a reader of the held track reads. fn anchored() -> [(Range, Vec); 6] { - let zero = Anchor::Absolute(Position::group(0)); + let zero = Anchor::Absolute(0); [ (range(Anchor::Relative(2), None), vec![2, 3, 4]), (range(Anchor::Time(millis(2500)), None), vec![2, 3, 4]), From ef5d764b798f0e7893750bac188e559e2f0fbca8 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 24 Oct 2026 16:17:40 -0700 Subject: [PATCH 09/10] test(net): format the crate-internal partial fetch in the refused-group test Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/track.rs | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index bd575325ade..33f1178095a 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -16740,15 +16740,13 @@ mod test { } evict(&producer, 3); let waiter = kio::Waiter::noop(); - let tail = producer - .consume() - .fetch_group( - 3, - group::Fetch { - frame_start: 2, - ..Default::default() - }, - ); + let tail = producer.consume().fetch_group( + 3, + group::Fetch { + frame_start: 2, + ..Default::default() + }, + ); let Poll::Ready(Ok(first)) = dynamic.poll_requested_fill(&waiter) else { panic!("an evicted group is a miss"); }; From 666c2f6c7522fd178ac7c7b093cf663a8b6129db Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 24 Oct 2026 17:01:12 -0700 Subject: [PATCH 10/10] quest: subscribe-ranges names group-only absolute anchors Co-Authored-By: Claude Opus 5.5 --- quest/m1/subscribe-ranges/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/quest/m1/subscribe-ranges/README.md b/quest/m1/subscribe-ranges/README.md index 712537bb9bb..fcd21b4bbc1 100644 --- a/quest/m1/subscribe-ranges/README.md +++ b/quest/m1/subscribe-ranges/README.md @@ -45,7 +45,7 @@ relay bounds a sparse FETCH only if the model can request ranges: an IETF FETCH is served from the model's ranges, capped at the Largest Object. - **Relative ends** (maintainer, 2026-10-11, superseding the 10-07 `Live` - flag and closing #5085). Each range end is `Absolute(group, frame)` or + flag and closing #5085). Each range end is `Absolute(group)` or `Relative(n)`, naming the group n before the latest; either end may be relative, and an omitted end is live. An offset past the track's start clamps to group 0. Before any group exists, relative ends resolve when the