From c7e59e1dbb520d4dfc78b5753ac7bff8dc779d6c Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 1 Oct 2026 15:46:56 -0700 Subject: [PATCH 1/3] quest: plan T-STD follow-ups (FETCH_OK properties, 2022-7 legs, E-AC-3, harness controls) Co-Authored-By: Claude Opus 5.5 --- quest/m1/README.md | 1 + quest/m1/fetch-ok-properties.md | 35 ++++++++++++++++++++++++ quest/m1/ietf-fetch-only.md | 14 ++++++++++ quest/m1/open-gop-leading-pictures.md | 5 ++++ quest/m2/README.md | 3 +++ quest/m2/ts-eac3.md | 24 +++++++++++++++++ quest/m2/ts-hitless.md | 38 +++++++++++++++++++++++++++ quest/m2/tstd-controls.md | 24 +++++++++++++++++ 8 files changed, 144 insertions(+) create mode 100644 quest/m1/fetch-ok-properties.md create mode 100644 quest/m2/ts-eac3.md create mode 100644 quest/m2/ts-hitless.md create mode 100644 quest/m2/tstd-controls.md diff --git a/quest/m1/README.md b/quest/m1/README.md index 805509a56df..e38a9a224d8 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -80,6 +80,7 @@ QUIC studies there on that rule. - [Omit empty catalog sections](/quest/m1/catalog-omit-empty.md) - a Rust catalog with no video or audio leaves those keys out, as JS does - [Delete SourceMap](/quest/m1/source-map-removal.md) - on dev, `Anchor` with its `Lane`s is the one way onto the broadcast clock - [Draft-20 FETCH](/quest/m1/ietf-fetch-location.md) - a draft-20+ FETCH within one group is served from its LOCATION_FILTER, as older drafts are +- [FETCH_OK properties](/quest/m1/fetch-ok-properties.md) - our FETCH_OK carries the track properties SUBSCRIBE_OK does, as draft 16+ requires - [Fetch without SUBSCRIBE](/quest/m1/ietf-fetch-only.md) - a relay fetches from an IETF upstream without subscribing, finished tracks included, with End of Track always reported - [IETF timestamp units](/quest/m1/ietf-timestamp-units.md) - drafts 14-16 stop sending Timestamp properties they can't give units for - [JS IETF datagrams](/quest/m1/js-ietf-datagram.md) - `@moq/net` sends and receives datagram groups over moq-transport, like Rust diff --git a/quest/m1/fetch-ok-properties.md b/quest/m1/fetch-ok-properties.md new file mode 100644 index 00000000000..9991a81662a --- /dev/null +++ b/quest/m1/fetch-ok-properties.md @@ -0,0 +1,35 @@ +# [S] FETCH_OK carries the track's properties + +## Goal + +Our IETF publisher's FETCH_OK carries the same track properties its +SUBSCRIBE_OK does: max cache duration, timescale, priority, and group order. +A fetch-only reader then learns the track as a subscriber would, and a relay +meets draft 16+'s rule to include all of a track's properties. + +## Plan + +Found (2026-10-01) while merging main into dev (#4647): a group FETCH with no +prior SUBSCRIBE_OK now reads its max age from FETCH_OK, but our publisher +sends an empty block. + +- `rs/moq-net/src/ietf/publisher.rs` sends FETCH_OK with + `properties: Default::default()`. SUBSCRIBE_OK fills them in the block near + `publisher.rs:655`. Share that code so the two can't diverge. +- Drafts 14-15 allow the omission: MAX_CACHE_DURATION is a MAY there, and + FETCH_OK has no properties block. From draft 16, a relay MUST include all + Extension Headers / Properties associated with a track in FETCH_OK (d16 + §8.6, d22 §7.7). An empty block also misleads the reader: it infers + Ascending order and priority 128 where SUBSCRIBE_OK says Descending. +- Honour INCLUDE_PROPERTIES (0x35) on FETCH from draft 20. It defaults to + sending the properties; at 0 the block is present but empty. `fetch.rs` + doesn't parse it today. +- Test per draft range: FETCH_OK round-trips the properties SUBSCRIBE_OK would + carry, and INCLUDE_PROPERTIES = 0 empties the block. + +Public API: none. Wire: FETCH_OK gains its properties on drafts that define +the block. Interop: run `just test interop --all`. + +## Related + +- [Fetch without SUBSCRIBE](/quest/m1/ietf-fetch-only.md) - asks TRACK_STATUS for the same properties when a publisher omits them diff --git a/quest/m1/ietf-fetch-only.md b/quest/m1/ietf-fetch-only.md index a7bdc908e66..d7a05be8d3f 100644 --- a/quest/m1/ietf-fetch-only.md +++ b/quest/m1/ietf-fetch-only.md @@ -24,6 +24,20 @@ TRACK_STATUS is the likely source of the info. The publisher refuses it today, so both sides are in scope. Keep the SUBSCRIBE path for real subscription demand. +Decided (2026-10-01): + +- Our subscriber learns a track's properties from TRACK_STATUS, sent in + parallel with every FETCH or SUBSCRIBE. From draft 20 it sets + INCLUDE_PROPERTIES to 0 on those requests, so the properties aren't sent + twice. Before draft 20 there's no opt-out, so it ignores the duplicate. +- The publisher serves TRACK_STATUS. Its reply carries what it would set in a + SUBSCRIBE_OK (d22 §9.13), from the same code that fills SUBSCRIBE_OK and + FETCH_OK. +- The publisher still follows the spec for subscribers that don't opt out: + [FETCH_OK properties](/quest/m1/fetch-ok-properties.md). + ## Related +- [FETCH_OK properties](/quest/m1/fetch-ok-properties.md) - the publisher side of the same properties + - [JavaScript FETCH](/quest/m1/js-fetch.md) - the browser publisher answers these fetches diff --git a/quest/m1/open-gop-leading-pictures.md b/quest/m1/open-gop-leading-pictures.md index 0dc7c0af266..fe83d04f39b 100644 --- a/quest/m1/open-gop-leading-pictures.md +++ b/quest/m1/open-gop-leading-pictures.md @@ -50,6 +50,11 @@ decode, and the JS test should drive the software decoder. Every continuous group is passed through untouched. - The same rule in the Rust decode path (`moq-video` decode consumers), so native playback and the transcoder tune in the same way. +- The same rule in `moq export ts`. Decided (2026-10-01): the fixed-delay + export (#4645) still sends a join's orphaned leading pictures, so 3 of 500 + frames on the open-GOP fixture decode after they present + (`dts-before-pts`). Trim them at tune-in from the same signal, and make + `just test ts --open-gop` pass under `--strict`. - This quest owns the Rust non-continuous signal, which audio warmup and consumer warmup reuse rather than each adding one. Today `moq_mux::container::Consumer::poll_read` returns a bare frame, and diff --git a/quest/m2/README.md b/quest/m2/README.md index a2b5090f7d5..2e3854aafcc 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -20,6 +20,9 @@ waiting on an upstream release waits in [m4](/quest/m4/README.md). - [Ladder](/quest/m2/ladder/README.md) - a transcode ladder adapts to the uplink it publishes over, instead of encoding every live rung at its ceiling - [Processor](/quest/m2/processor/README.md) - a customer-run worker publishes an on-demand contribution under its own service prefix with scoped access - [Stream sessions](/quest/m2/uring-tcp/README.md) - serve WebSocket and HTTP from the io_uring workers, where io_uring pays off most +- [Hitless TS legs](/quest/m2/ts-hitless.md) - two `--sync` export legs emit packet-identical TS for ST 2022-7 +- [DVB E-AC-3](/quest/m2/ts-eac3.md) - E-AC-3 private data is split per sync frame and buffer-modelled in TS export +- [T-STD controls](/quest/m2/tstd-controls.md) - the harness gains an MB-overflow control and an AAC broadcast reference - [fMP4 emsg](/quest/m2/emsg.md) - settles the shared framing and missing-data semantics before this section adopts them - [ID3 catalog section](/quest/m2/id3.md) - timed ID3 as a first-class container-neutral catalog section - [FLV script tags](/quest/m2/flv-script.md) - onMetaData and AMF data messages survive RTMP and FLV import diff --git a/quest/m2/ts-eac3.md b/quest/m2/ts-eac3.md new file mode 100644 index 00000000000..57103cb5eae --- /dev/null +++ b/quest/m2/ts-eac3.md @@ -0,0 +1,24 @@ +# [S] DVB E-AC-3 gets a buffer model in TS export + +## Goal + +`moq export ts` splits DVB E-AC-3 private-data PES one sync frame per PES, +schedules each frame against the 13818-1 E-AC-3 buffer (12,896 B), and +`test/ts/compliance.py` grades it. That's the same treatment DVB AC-3 got in +the fixed-delay export (#4645). + +## Plan + +Found in #4645: E-AC-3 carried as DVB private data still goes out unsplit, +with no buffer model, so a multi-frame PES can overflow B. + +- Split it the way #4645 splits DVB AC-3. Read frame sizes from each E-AC-3 + sync frame's `frmsiz`, and drop a frame shorter than its header says. +- Add E-AC-3 to the schedule's per-PID admission (TB at 2 Mb/s, B at + 12,896 B) and to the `tstd` model in `compliance.py`. +- Test: a synthetic multi-frame E-AC-3 PES exports one frame per PES, and a + control in `tstd-controls.py` overflows B without the split. + +## Required + +- [T-STD TS export](/quest/m1/tstd/README.md) - the per-PID schedule this joins diff --git a/quest/m2/ts-hitless.md b/quest/m2/ts-hitless.md new file mode 100644 index 00000000000..3b2ef3010f1 --- /dev/null +++ b/quest/m2/ts-hitless.md @@ -0,0 +1,38 @@ +# [L] Two TS export legs emit identical packets for ST 2022-7 + +## Goal + +Two `moq export ts --sync` legs of one broadcast, joined at different +moments, emit packet-identical TS from their first common packet, across +skips and source drift, continuity counters included. A ST 2022-7 selector +can then switch between them hitlessly. + +## Plan + +Decided (2026-10-01), from t0ms's review of the fixed-delay export (#4645). +They call 2022-7 a differentiator for primary distribution, not a +requirement, so this waits until the T-STD line settles. + +- Measured acquisition stays the default. `--sync` opts in to anchoring on + the catalog clock: release at `clock.wall + pts + delay`, so every leg + anchors the same way however it joined. This needs NTP-synced hosts. If the + measured age is implausible (negative, or beyond the delay), the export + fails loud. +- Continuity counters restart at each group. Each media PID's first packet in + a group sets `discontinuity_indicator`, and its counter is numbered from + there, so it comes from the media and not from what a leg has sent. The PCR + moves to its own PID, because a discontinuity flag on the PCR PID would + declare a time-base break. Loss is still detected within a group. +- Once anchored identically, both legs must make the same skip decision and + lay out the same slots. t0ms's `two_legs_render_a_skip_the_same_way` stays + ignored in #4645 until this lands. With the audio late, it also showed a + PES header byte that differs at the skip; find and fix that. +- Test with mocked time: + - two legs, joined a second apart, give byte-identical output including + the counters; + - the same across a mid-run skip, and under ±30 ppm drift; + - `--sync` with an implausible age fails loud. + +## Required + +- [T-STD TS export](/quest/m1/tstd/README.md) - the export, clock recovery, and schedule these legs share diff --git a/quest/m2/tstd-controls.md b/quest/m2/tstd-controls.md new file mode 100644 index 00000000000..5d8d20ffe16 --- /dev/null +++ b/quest/m2/tstd-controls.md @@ -0,0 +1,24 @@ +# [S] The T-STD harness covers MB overflow and an AAC broadcast reference + +## Goal + +`just test ts-tstd` has a negative control that overflows the video multiplex +buffer (MB), and a positive reference that carries AAC the way a broadcast +encoder does. Every buffer and codec the export emits then has a control on +each side. + +## Plan + +Found in #4643. Grading against the declared HRD made MB about 3.6 MB on the +Kyrion capture, more than its 4 s holds, so no control reaches MB any more. +The only positive reference carries MPEG audio, but the export emits AAC. + +- MB: a longer clip with no declared HRD (level defaults), restamped to a + burst, which overflows MB and passes as captured. +- AAC: a shareable broadcast-style reference whose AAC passes B. + - FFmpeg's default 0.7 s audio lead overflows the 3,584 B AAC buffer, so try + its mux delay options first. + - If no generated clip passes, ask t0ms whether a short AAC capture can be + shared. +- Each control fails or passes for the stated reason, and `interop.yml` + runs them. From 715ec129fd7173e72fe338358d2229b4ff4ec20b Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 1 Oct 2026 16:13:05 -0700 Subject: [PATCH 2/3] quest: address review on T-STD follow-ups ts-eac3 takes DVB's buffer from TS 101 154 and keeps dependent substreams on their independent frame's interval; ietf-fetch-only decouples object timestamps from INCLUDE_PROPERTIES first; fetch-ok-properties applies the properties on the fetch accept path; tstd-controls requires the T-STD export. Co-Authored-By: Claude Opus 5.5 --- quest/m1/fetch-ok-properties.md | 5 ++++- quest/m1/ietf-fetch-only.md | 6 ++++++ quest/m2/ts-eac3.md | 27 ++++++++++++++++++--------- quest/m2/tstd-controls.md | 4 ++++ 4 files changed, 32 insertions(+), 10 deletions(-) diff --git a/quest/m1/fetch-ok-properties.md b/quest/m1/fetch-ok-properties.md index 9991a81662a..69271af05bf 100644 --- a/quest/m1/fetch-ok-properties.md +++ b/quest/m1/fetch-ok-properties.md @@ -16,6 +16,9 @@ sends an empty block. - `rs/moq-net/src/ietf/publisher.rs` sends FETCH_OK with `properties: Default::default()`. SUBSCRIBE_OK fills them in the block near `publisher.rs:655`. Share that code so the two can't diverge. +- Our group-fetch accept path reads only the max age from FETCH_OK and + hardcodes a microsecond timescale. Apply the same properties SUBSCRIBE_OK + does (timescale, priority, group order) there too. - Drafts 14-15 allow the omission: MAX_CACHE_DURATION is a MAY there, and FETCH_OK has no properties block. From draft 16, a relay MUST include all Extension Headers / Properties associated with a track in FETCH_OK (d16 @@ -23,7 +26,7 @@ sends an empty block. Ascending order and priority 128 where SUBSCRIBE_OK says Descending. - Honour INCLUDE_PROPERTIES (0x35) on FETCH from draft 20. It defaults to sending the properties; at 0 the block is present but empty. `fetch.rs` - doesn't parse it today. + decodes it on draft 20+ and drops it, so it is not honoured today. - Test per draft range: FETCH_OK round-trips the properties SUBSCRIBE_OK would carry, and INCLUDE_PROPERTIES = 0 empties the block. diff --git a/quest/m1/ietf-fetch-only.md b/quest/m1/ietf-fetch-only.md index d7a05be8d3f..c904d30e600 100644 --- a/quest/m1/ietf-fetch-only.md +++ b/quest/m1/ietf-fetch-only.md @@ -30,6 +30,12 @@ Decided (2026-10-01): parallel with every FETCH or SUBSCRIBE. From draft 20 it sets INCLUDE_PROPERTIES to 0 on those requests, so the properties aren't sent twice. Before draft 20 there's no opt-out, so it ignores the duplicate. +- First decouple object timestamps from INCLUDE_PROPERTIES: the publisher + derives its serving timescale from `properties_wanted` + (`publisher.rs:628`), so opting out today strips the Timestamp property + from every object and the subscriber falls back to arrival time. Test + TRACK_STATUS plus the opt-out with source timestamps that differ from + arrival. - The publisher serves TRACK_STATUS. Its reply carries what it would set in a SUBSCRIBE_OK (d22 §9.13), from the same code that fills SUBSCRIBE_OK and FETCH_OK. diff --git a/quest/m2/ts-eac3.md b/quest/m2/ts-eac3.md index 57103cb5eae..424fba20598 100644 --- a/quest/m2/ts-eac3.md +++ b/quest/m2/ts-eac3.md @@ -2,9 +2,10 @@ ## Goal -`moq export ts` splits DVB E-AC-3 private-data PES one sync frame per PES, -schedules each frame against the 13818-1 E-AC-3 buffer (12,896 B), and -`test/ts/compliance.py` grades it. That's the same treatment DVB AC-3 got in +`moq export ts` splits DVB E-AC-3 private-data PES (`stream_type` 0x06 with +an `enhanced_AC3_descriptor`) one access unit per PES, schedules each against +the E-AC-3 buffer ETSI TS 101 154 gives DVB, and `test/ts/compliance.py` +grades it. That's the same treatment DVB AC-3 got in the fixed-delay export (#4645). ## Plan @@ -12,12 +13,20 @@ the fixed-delay export (#4645). Found in #4645: E-AC-3 carried as DVB private data still goes out unsplit, with no buffer model, so a multi-frame PES can overflow B. -- Split it the way #4645 splits DVB AC-3. Read frame sizes from each E-AC-3 - sync frame's `frmsiz`, and drop a frame shorter than its header says. -- Add E-AC-3 to the schedule's per-PID admission (TB at 2 Mb/s, B at - 12,896 B) and to the `tstd` model in `compliance.py`. -- Test: a synthetic multi-frame E-AC-3 PES exports one frame per PES, and a - control in `tstd-controls.py` overflows B without the split. +- Split it the way #4645 splits DVB AC-3, but per access unit: an + independent sync frame plus the dependent substreams that follow it share + one presentation interval (ETSI TS 102 366 Annex E), so they go out in one + PES on one decode time. Read sizes from each sync frame's `frmsiz`, and + drop a frame shorter than its header says. +- The buffer is the DVB one, not ATSC's: #4645's 12,896 B is A/52 Annex G's + for `stream_type` 0x87, and DVB AC-3 uses 5,696 B. Take the E-AC-3 value + from ETSI TS 101 154, cite the clause, and add it to the schedule's + per-PID admission and to the `tstd` model in `compliance.py`, which + refuses DVB E-AC-3 today. +- Test: a synthetic multi-frame E-AC-3 PES exports one access unit per PES; + an independent-plus-dependent fixture keeps both substreams on one + interval (32 ms for six 48 kHz blocks, not 64); and a control in + `tstd-controls.py` overflows B without the split. ## Required diff --git a/quest/m2/tstd-controls.md b/quest/m2/tstd-controls.md index 5d8d20ffe16..07ef58ed98d 100644 --- a/quest/m2/tstd-controls.md +++ b/quest/m2/tstd-controls.md @@ -22,3 +22,7 @@ The only positive reference carries MPEG audio, but the export emits AAC. shared. - Each control fails or passes for the stated reason, and `interop.yml` runs them. + +## Required + +- [T-STD TS export](/quest/m1/tstd/README.md) - `just test ts-tstd` and `tstd-controls.py` land with it (#4645) From 4bd40d1ad3e9d9ce4142a59a01d450546a6942bf Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 1 Oct 2026 16:59:56 -0700 Subject: [PATCH 3/3] quest: link TS passthrough and ts-hitless Co-Authored-By: Claude Opus 5.5 --- quest/m1/ts-passthrough.md | 3 ++- quest/m2/ts-hitless.md | 4 ++++ 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/quest/m1/ts-passthrough.md b/quest/m1/ts-passthrough.md index 03fb1c0057e..6a3aa3b5605 100644 --- a/quest/m1/ts-passthrough.md +++ b/quest/m1/ts-passthrough.md @@ -78,7 +78,7 @@ Decided (2026-10-01), from a discussion with t0ms: continuity counters included, since passthrough never rewrites them; an object one leg drops late is a gap in that leg only. Aligning the legs in time is left to the `--sync` anchor planned for the demultiplexed export's - 2022-7 legs (`quest/m2/ts-hitless.md`, #4680), which passthrough can adopt. + 2022-7 legs, which passthrough can adopt. That is TS identity, not ST 2022-7 recovery, which also needs matching RTP headers from a coordinated RTP egress. - Rust only. `js/hang` does not parse `m2ts`, and a player sees no @@ -113,5 +113,6 @@ hang catalog gains an `m2ts` root section; additive. ## Related +- [TS hitless](/quest/m2/ts-hitless.md) - the demultiplexed lane's 2022-7 legs and the `--sync` anchor - [MSFTS convergence](/quest/m2/msfts-convergence.md) - the ES-level side of the same mapping - [TS byte schedule](/quest/m1/tstd/byte-schedule.md) - the remux's equivalent of pacing on the source PCR diff --git a/quest/m2/ts-hitless.md b/quest/m2/ts-hitless.md index 3b2ef3010f1..a055a89fc30 100644 --- a/quest/m2/ts-hitless.md +++ b/quest/m2/ts-hitless.md @@ -36,3 +36,7 @@ requirement, so this waits until the T-STD line settles. ## Required - [T-STD TS export](/quest/m1/tstd/README.md) - the export, clock recovery, and schedule these legs share + +## Related + +- [TS passthrough](/quest/m1/ts-passthrough.md) - byte-identical legs for free in its own lane; adopts `--sync` for alignment