Repository navigation
perf(mux): parse an MPTS once for every program - #6371
Conversation
Quest-Claim: 84557ef1d36a7199
`ts::Programs` fed the whole multiplex to an independent `Import` per program, so framing, continuity, the PAT, SI reassembly and health grading ran once per program: ingest cost was program count times input rate. A shared `Demux` now frames the input and reads what every program shares (sync, per-PID continuity, the latest PCR per PID, the PAT, SI sections). `Programs` routes each packet only to the programs whose PAT entry or PMT claims its PID, through a PID-indexed reader table, and flushes only programs a call touched. A single `Import` runs the same parser. `health::Health` splits into the multiplex checks and the per-program timers; the mux-rate meter takes the parser's packet count. New `ts_programs` bench: at 16 programs, 4 Mb/s each, throughput goes from 87 to ~770-850 MiB/s, flat with program count. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EeyiEy5FWz3z44DYZVo7vU
kixelated
changed the title
|
Outcome: the quest is complete and the quest file is deleted in this PR. One shared parser now serves every program under Suggested follow-ups: none new. Stream routes and PAT changes unblock once this merges. (Written by Claude Opus 5.5) |
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 31 minutes. View limit details
No actionable comments were generated in the recent review.
 |
| Check name | Status | Explanation |
|---|---|---|
| Title check |  |
The title clearly summarizes the main change: parsing an MPTS once for all programs. |
| Description check | |
The description explains the problem, implementation, performance measurements, impact, and validation. It is directly related to the changeset. |
| Docstring Coverage | |
Docstring coverage is 81.18% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 85 functions across 11 files. (4 skipped: 4… |
| Linked Issues check | |
Check skipped because no linked issues were found for this pull request. |
| Out of Scope Changes check | |
Check skipped because no linked issues were found for this pull request. |
Full details: Docstring Coverage
-
Explanation
Docstring coverage is 81.18% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 85 functions across 11 files. (4 skipped: 4 unsupported.)
Finishing Touches
-
✨ Simplify code
-
- Commit to this branch
- Create a new PR
-
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Comment @coderabbitai help to get the list of available commands.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Merging. Iterated to settle the maintainer's decisions, then chosen for /quest-merge. CI is green, the current head has a non-Grok review, and (Written by Claude Opus 5.5) |
…ty/shared-parser # Conflicts: # quest/m1/ts-reliability/README.md # quest/m1/ts-reliability/shared-parser.md
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0d85e9036b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…laims it The shared demux tracks one continuity chain per PID. A program registering a PID another program already reads reset that chain, so a duplicate packet right after the PMT read as contiguous and a counter gap went uncounted for the existing reader. Only the PID's sole reader restarts it now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Merge summary:
(Written by Claude Opus 5.5) |
…ty/shared-parser # Conflicts: # quest/m1/ts-reliability/README.md # quest/m1/ts-reliability/pat-lifecycle.md
|
Addendum: merged (Written by Claude Opus 5.5) |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # rs/moq-mux/src/container/ts/import.rs # sh/gh/nightly.sh
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#6371 replaced the per-program reparse with one Demux. On it, Programs no longer hands scan state to a late importer: it watches the PMT PIDs of waiting programs, and when a PMT for one completes it creates the importer (seeded with the PAT and that PID's reassembler as it stood before the packet) and routes the packet through the importer's own PMT path. The shared parser already owns the PAT reader, grading and PCR samples, so rounds 1 to 4 of review findings fall away. The `Import::resume` handoff, `Resume` and `reads_pmt` are gone; only `Import::with_pmt` remains. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem
ts::Programs(moq publish ... import ts --program all) fed the whole multiplex to an independentImportper program. Framing, continuity, the PAT, SI reassembly and TR 101 290 grading all ran once per program, so an MPTS ingest cost program count times input rate. Completes Shared parser, which the stream routes and PAT changes quests were waiting on.Approach
ts::demux::Demuxframes the input and reads what every program shares: sync, per-PID continuity, the latest PCR on each PID, the PAT, and the standalone SI sections. It hands out onePacketat a time with those results.Importkeeps its per-program logic behindImport::packet;Import::decoderuns its ownDemux, so the single-program path andProgramsshare one code path.Programsruns oneDemuxand routes each packet only to the programs whose PAT entry or PMT claims its PID (a PID-indexed reader table, updated when a PAT or PMT is handled). The PAT and completed SI sections go to every program. It flushes SI and announces only the programs adecodecall touched, so a 1316-byte SRT/UDP datagram costs what it carried, not a pass over every program.health::Healthsplits intoMultiplex(sync, continuity, transport errors, counted once) and the per-programHealth(PAT/PMT/PCR/PTS timers on that program's clock).mux_rate::Metertakes the parser's cumulative packet count instead of counting packets itself;Passthroughpasses its own.Measurements
New
rs/moq-mux/benches/ts_programs.rs: one second of a generated H.264 MPTS, sweeping program count and per-program bitrate, asserting every program delivers every frame. Throughput over the whole input (MiB/s):The single-program rows are within run-to-run noise: interleaved repeats measured 948/887 on
mainagainst 953/957 here. The remaining dip at 16 programs and 1 Mb/s is the PAT, which every program still reads.Impact
Import,Programsand their methods keep their signatures. Internal:si::Capture::sectiontakesBytes, so one reassembled SI section is shared by every program without copying.--program all, sync, continuity and transport errors before the first PAT are now graded too, since one parser reads the whole input. A duplicate packet no longer refreshes the PCR a later PMT starts its clock from. When two programs read the same elementary PID, a PMT that re-routes it in one program restarts the shared continuity chain for both.Notes
Programs::readersand extends this bench, and PAT changes notes thatProgramssees each whole PAT first.Validation
just check: passed (Rust lint and tests, quest, markdown, flake).cargo test -p moq-mux --lib container::ts: 606 passed.cargo bench --bench ts_programsonmain(with the bench copied in) and on this branch, numbers above.🤖 Generated with Claude Code
https://claude.ai/code/session_01EeyiEy5FWz3z44DYZVo7vU
(Written by Claude Opus 5.5)