From 4d057a11f141ec6c3aa90a46a064bebf2adf79b6 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 13 Oct 2026 10:16:55 -0700 Subject: [PATCH 1/2] quest(m1): plan PSI pacing within the system buffers Drop adjacent-copy suppression: #5306's padded planner already paces TBsys and passes #5304's regression. Rewrite psi-runs as the remaining work: Bsys beside TBsys in the planner, PID 1-3 passthrough in the same buffers, and a hard system-buffer check in compliance.py. Co-Authored-By: Claude Opus 5.5 --- quest/m1/tstd/README.md | 2 +- quest/m1/tstd/psi-runs.md | 89 +++++++++++++++++---------------------- 2 files changed, 39 insertions(+), 52 deletions(-) diff --git a/quest/m1/tstd/README.md b/quest/m1/tstd/README.md index 155cfbd6f8f..c165a36aea8 100644 --- a/quest/m1/tstd/README.md +++ b/quest/m1/tstd/README.md @@ -50,7 +50,7 @@ MB/AAC controls. Do not relabel the current subset a full standards check. ## Required - [Send-ahead within the delay](/quest/m1/tstd/send-ahead.md) - total lag is `--delay`, send-ahead included, with a 1 s default -- [PSI runs](/quest/m1/tstd/psi-runs.md) - the PAT and PMT are deduplicated and paced so the system buffers never overflow +- [PSI pacing](/quest/m1/tstd/psi-runs.md) - padded output paces the PAT and PMT so TBsys and Bsys never overflow - [Video at its full leak](/quest/m1/tstd/video-leak.md) - a leak-bound video PID is admitted at its full Rbx, not about 3 % below it - [Mux overhead](/quest/m1/tstd/mux-overhead.md) - the PCR rides in the video's adaptation field and audio PES carry several frames, so a full CBR source re-muxes at its own rate diff --git a/quest/m1/tstd/psi-runs.md b/quest/m1/tstd/psi-runs.md index 9877a574014..74b0011fdb6 100644 --- a/quest/m1/tstd/psi-runs.md +++ b/quest/m1/tstd/psi-runs.md @@ -1,60 +1,47 @@ -# [L] TS export paces PSI within the system buffers +# [M] TS export keeps PSI within the system buffers ## Goal -The export's program tables never overflow the program's system buffers at any -mux rate: TBsys (512 bytes drained at Rxsys = 1 Mb/s) and Bsys (1,536 bytes -drained at Rsys = max(80 kb/s, mux rate / 500)), per ISO 13818-1 2.4.2.3. -Identical runs that the same flush would lay back to back go out once. Distinct -tune-in runs (each send-ahead unit leads with its own tables) and changed tables -(a new PMT version) are paced, never dropped, and changed tables go out in -order. `compliance.py` grades both buffers too, so every TS Interop arm catches -a regression. +Padded `moq export ts` output keeps the program's system buffers within +ISO 13818-1 2.4.2.3 at every mux rate: TBsys (512 B at Rxsys = 1 Mb/s, shared +by PIDs 0-3 and the PMT PID) and Bsys (1,536 B drained at +Rsys = max(80 kb/s, mux rate / 500)). Distinct tune-in runs and changed table +versions are paced, never dropped, and changed versions go out in order. +`compliance.py` grades both buffers, so every TS Interop arm catches a +regression. Unpadded output has no mux rate and makes no system-buffer claim; +say so in the profile report. ## Plan -Found by Codex on #5207 (`schedule.rs`, repeated PSI groups). It exists on -`main` before #5207 too. Keeping each table once per slot was tried and broke -`aac_program_config_follows_each_table`, because send-ahead units need their own -tables. +Decided (2026-10-13), from #5304: adjacent-copy suppression is dropped. It +would only reach unpadded output, which makes no buffer claim, and #5306's +padded planner already passes #5304's 3/4/8-PID regression with every tune-in +copy kept. -The PAT and the PMT share one TBsys: it takes PIDs 0 to 3 and the selected -program's PMT PID (ISO 13818-1 2.4.2.3). One PAT and PMT pair (376 B) fits. At -25 Mb/s, two pairs back to back deliver 752 B in about 180 µs, of which about -23 B drains, peaking near 729 B; three peak near 1090 B. At high rates two -distinct pairs fit only about 1.9 ms apart; pacing derives the spacing from -simulated occupancy at the actual mux rate. - -- Start from the author's narrower diff on #5207 - (https://github.com/moq-dev/moq/pull/5207#discussion_r4239051134): drop a run - only when the same flush would lay it right after an identical run, and pad - the dropped packets back with nulls in a padded slot. `Slot::nulls` becomes - `Option` so an unpadded slot gains none. A slot still opens with its - clock packet and keeps its length, so no PCR moves. #5304 implements this step - and stays a draft until pacing lands. -- Then pace what suppression leaves: a distinct tune-in run or a changed version - goes out only once TBsys and Bsys have drained enough to take it, within the - slot budget and the units' decode deadlines. Bsys caps sustained PSI at Rsys, - 80 kb/s below 40 Mb/s. #5304 found equal-weight aligned units on four and - eight video PIDs still overflow after suppression. -- #5220 (multi-packet PMT) adds PMT pacing at the system-buffer rate, since a - PMT section of up to 1,024 B spans several packets and can exceed 512 B alone. - Build on that pacing rather than adding a second one, or own it here if #5220 - lands without it. Pace every PID that enters TBsys as one lane: the PAT, the - PMT, and any `mpegts.si` passthrough on PIDs 1 to 3, which the scheduler gives - its own `Buffer::SYSTEM` lane today. -- Test against buffer fill with the T-STD model, not a packet count: a unit test - runs laid slots' PIDs 0 to 3 and PMT packets through one shared TBsys and - Bsys, with keyframes aligned at 25 Mb/s on two and three video PIDs (both fail - without the fix) and on one (passes either way). Add a changed-version - regression, #5304's distinct-run cases, a PID 1 to 3 passthrough packet beside - a pair (fails while it keeps its own lane), and a sustained burst that fits - TBsys but overflows Bsys. The AAC tune-in test still passes. -- `test/ts/compliance.py` gains a hard-severity system-buffer check: for each - program, one TBsys fed by PIDs 0 to 3 and that program's PMT PID, draining - into one Bsys, both checked for overflow, and TBsys for emptying at least - once a second (2.4.2.6), with a negative control for each. Its `tstd` check - models only elementary streams today. Other SI (NIT, SDT, EIT) and SCTE-35 - never enter TBsys. Every existing TS arm still passes. +- Build on #5306's padded planner, which paces TBsys at absolute positions + shared by tables, nulls and media. t0ms found on #5304 (a real DVB recording) + that table and padding positions laid ahead of a capped video lane, outside + its interleave count, compress the video and overrun its TB. Any further + pacing must stay inside that planner's positions, never a pre-pass that moves + or pads tables outside it. Confirm with t0ms's replay against the planner. +- Add Bsys beside TBsys in the planner. Settle first what enters Bsys (section + bytes, or whole payloads with stuffing): for single-packet tables it decides + whether Bsys can overflow at all. A probe on #5306 (`a5640abee`) counting + full payloads peaked near 1.35 KB at 4 aligned video PIDs and 2.3-2.7 KB + at 8. +- Open: when Bsys pacing would push a table past its unit's decode deadline, + fail loud like any missed deadline, or allow dropping a copy identical to one + still queued. Recommend failing loud unless a real source hits it. +- Pace `mpegts.si` passthrough on PIDs 1-3 in the same TBsys and Bsys; the + planner's `transport_buffer` maps only PID 0 and the PMT to TBsys. +- Delete the round-robin `SystemBuffer` stall in `Slot::layout` (#5220), which + only unit tests reach once the planner handles padded output, or move its + cases to the planner. +- Tests: planner unit tests with 8 aligned PIDs, a multi-packet PMT, a PID 1-3 + passthrough packet beside a PAT/PMT pair, and a sustained burst that fits + TBsys but overflows Bsys. `test/ts/compliance.py` gains a hard-severity check + per program: one TBsys fed by PIDs 0-3 and the PMT PID, draining into one + Bsys, both checked for overflow, and TBsys emptying at least once a second + (2.4.2.6), each with a negative control. Every existing TS arm still passes. Public API: none. Wire: none. From 610f9a73c4ee2f0f79a829d49528b4b9b487ac78 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 13 Oct 2026 10:23:30 -0700 Subject: [PATCH 2/2] quest(m1): address review on PSI pacing List whole packets as a Bsys accounting candidate, and keep the changed-version, multi-slot Bsys burst and AAC tune-in tests. Co-Authored-By: Claude Opus 5.5 --- quest/m1/tstd/psi-runs.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/quest/m1/tstd/psi-runs.md b/quest/m1/tstd/psi-runs.md index 74b0011fdb6..12d54fb9105 100644 --- a/quest/m1/tstd/psi-runs.md +++ b/quest/m1/tstd/psi-runs.md @@ -24,9 +24,9 @@ copy kept. its interleave count, compress the video and overrun its TB. Any further pacing must stay inside that planner's positions, never a pre-pass that moves or pads tables outside it. Confirm with t0ms's replay against the planner. -- Add Bsys beside TBsys in the planner. Settle first what enters Bsys (section - bytes, or whole payloads with stuffing): for single-packet tables it decides - whether Bsys can overflow at all. A probe on #5306 (`a5640abee`) counting +- Add Bsys beside TBsys in the planner. Settle first, from 2.4.2.3's text, what + enters Bsys (whole 188 B packets, payloads with stuffing, or section bytes): + for single-packet tables it decides whether Bsys can overflow at all. A probe on #5306 (`a5640abee`) counting full payloads peaked near 1.35 KB at 4 aligned video PIDs and 2.3-2.7 KB at 8. - Open: when Bsys pacing would push a table past its unit's decode deadline, @@ -38,8 +38,9 @@ copy kept. only unit tests reach once the planner handles padded output, or move its cases to the planner. - Tests: planner unit tests with 8 aligned PIDs, a multi-packet PMT, a PID 1-3 - passthrough packet beside a PAT/PMT pair, and a sustained burst that fits - TBsys but overflows Bsys. `test/ts/compliance.py` gains a hard-severity check + passthrough packet beside a PAT/PMT pair, a changed PMT version going out in + order, and a burst spanning several slots that fits TBsys but overflows Bsys. + The AAC tune-in test still passes. `test/ts/compliance.py` gains a hard-severity check per program: one TBsys fed by PIDs 0-3 and the PMT PID, draining into one Bsys, both checked for overflow, and TBsys emptying at least once a second (2.4.2.6), each with a negative control. Every existing TS arm still passes.