Skip to content

refactor(mux): drive TS exports from catalog streams - #5561

Merged
kixelated merged 4 commits into
mainfrom
quest/m1/ts-export-catalog
Oct 15, 2026
Merged

kixelated merged 4 commits into
mainfrom
quest/m1/ts-export-catalog

Conversation

@kixelated

Copy link
Copy Markdown
Collaborator

Problem

MPEG-TS export chose and subscribed its own catalog, so callers could not supply rendition selection or a custom catalog stream as they can with other exporters. Its follower also needed encoding metadata after catalog EOF, while the ended catalog reader retained its subscription.

Approach

Accept the caller's catalog stream in Export::new(source, catalog) and follow(broadcast, catalog). Keep subscription and format policy in the concrete Follower::new(export) path. On EOF, the unified catalog consumer drops its reader and decoder and retains only its format in a hidden closed state. Migrate the CLI, SRT, benchmarks and tests, and document the constructor migration.

Impact

  • Breaking Rust API: ts::Export is generic over catalog::Stream; new is synchronous and takes two arguments. Remove with_catalog_format and with_ts; choose encoding and extension on the catalog stream. follow is synchronous and takes the replacement stream.
  • Follower::new(export) keeps its one-argument shape and accepts exports over the concrete catalog consumer. The existing non-exhaustive catalog::Consumer gains a doc-hidden Closed implementation state; existing MSF fields are preserved.
  • An arbitrary custom stream remains owned until the export is dropped or follows another broadcast. It is never polled again after EOF. Concrete consumers release their subscription and decoder at EOF.
  • No MoQ wire, catalog encoding, MPEG-TS format or CLI flag changes.

Validation

Before cleanup, regressions failed for retained EOF subscriptions and HangZ follower cleanup. After cleanup, all three focused regressions pass: all-format subscription release, custom selected stream export/follow, and HangZ resubscription after EOF. The full scoped native suite passes 3,096 tests with nine skips.

The full just check exits successfully, including native tests, lint/rustdoc, feature/WASM and JavaScript/documentation checks. just test interop --all passes all 45 publisher/subscriber pairings, denial assertions, browser refusal close code, CLI FETCH and finite tails. The final quest graph (470 documents), whitespace and current-main merge checks pass.

Alternatives

The maintainer chose caller-owned catalog selection, follow(broadcast, catalog), and preserving Follower::new(export) through concrete EOF cleanup. No compatibility shim, new public type, policy hook or format constructor argument is added.

Follow-ups

The concurrent same-instance draft #5545 also edits TS/CLI tests and needs constructor adaptation when combining the branches.

(Written by GPT-6.1)

kixelated and others added 3 commits October 14, 2026 19:21
Co-Authored-By: Codex <noreply@openai.com>
Continue the approved parent claim in an isolated worktree. Maintainer decisions: Export::new(source, catalog), follow(broadcast, catalog), preserve Follower::new(export), and retain private concrete format metadata while releasing catalog subscriber and decoder resources at EOF. No compatibility shim or new public policy type.

Co-Authored-By: GPT-6.1 <noreply@openai.com>
Co-Authored-By: GPT-6.1 <noreply@openai.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Completed the maintainer-approved catalog API decisions: synchronous Export::new(source, catalog) and follow(broadcast, catalog), preserving one-argument Follower::new(export). Encoding, extension and rendition selection belong to the caller's stream. No shim or new public type is added.

The approved EOF lifecycle refactor adds a doc-hidden closed state to the existing non-exhaustive unified consumer. It preserves format metadata and existing MSF fields while dropping the concrete subscriber and decoder. Arbitrary custom streams remain owned by the export after EOF, are not polled again, and are released on drop or replacement. Before-fix cleanup and HangZ regressions fail; all three focused regressions pass after the fix.

The full just check exits successfully, including native tests, lint/rustdoc, feature/WASM and JavaScript/documentation checks. just test interop --all passes all 45 publisher/subscriber pairings, denial assertions, browser refusal close code, CLI FETCH and finite tails. The final quest graph (470 documents), whitespace and current-main merge checks pass.

The constructor migration overlaps the TS/CLI tests in #5545; adapt those calls to subscribe the catalog before building the export. This PR stays draft.

(Written by GPT-6.1)

@kixelated
kixelated marked this pull request as ready for review October 15, 2026 15:32
@coderabbitai

coderabbitai Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 12 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 0c6ae148-ce62-4c03-8f06-bca09720dea7
📥 Commits

Reviewing files that changed from the base of the PR and between 5beead6 and 2dfbe2e.

📒 Files selected for processing: 16
  • doc/lib/rs/moq-mux.md
  • doc/setup/upgrade.md
  • quest/m1/README.md
  • quest/m1/ts-export-catalog.md
  • rs/moq-cli/src/publish.rs
  • rs/moq-cli/src/subscribe.rs
  • rs/moq-mux/benches/export.rs
  • rs/moq-mux/src/catalog/consumer.rs
  • rs/moq-mux/src/catalog/hang/consumer.rs
  • rs/moq-mux/src/container/ts/export.rs
  • rs/moq-mux/src/container/ts/export_test.rs
  • rs/moq-mux/src/container/ts/export_timing_test.rs
  • rs/moq-mux/src/container/ts/follower.rs
  • rs/moq-mux/src/container/ts/import.rs
  • rs/moq-mux/src/container/ts/import_test.rs
  • rs/moq-srt/src/ts.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-15T15:49:56.587558Z 2dfbe2e Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 4a3c200.

No actionable bugs found. Moving catalog selection to the caller aligns TS with the other exporters without adding a policy hook. The EOF flag avoids re-polling arbitrary non-fused streams, while the concrete consumer releases its subscription/decoder and retains the format needed by Follower. Same-epoch follow resets catalog state; replacement follow rebuilds the export against the pinned broadcast.

The CLI/SRT migrations retain ts::Ext and delay settings, and the breaking constructor/follow changes are documented. Focused regressions cover selected custom streams, all-format EOF release, and HangZ following after EOF. The existing fixed PAT/PMT restriction remains unchanged.

Verification: static GitHub review of the changes and surrounding source/follower paths. No tests, builds, or interop runs executed.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4a3c200c73

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

#[doc(hidden)]
Msf(super::msf::Consumer<E>),
#[doc(hidden)]
Closed(CatalogFormat),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep the EOF sentinel private

When downstream crates use Consumer, this variant remains constructible and matchable despite #[doc(hidden)], so internal EOF bookkeeping becomes a supported public API and lets callers fabricate a consumer that was never sourced from a broadcast. Removing or reshaping that state later would require another breaking change; retain the format and release the decoder within private state instead of adding a public variant.

AGENTS.md reference: AGENTS.md:L57-L61

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Disagree, keeping it. catalog::Consumer is already a #[non_exhaustive] enum whose Hang/Msf variants are #[doc(hidden)] and constructed directly by the benches, so Closed adds no new kind of exposure: hidden variants are not supported API, and the type is documented as driven through Stream only. Moving EOF into private state would need either making msf::Consumer::track non-public or turning the unified consumer into an opaque struct (plus a constructor for the benches), both wider API changes than this PR's approved scope. An opaque catalog::Consumer is a reasonable follow-up on its own.

(Written by Claude Opus 5.5)

@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. What shall we delve into next?

Reviewed commit: 2dfbe2ecdc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@kixelated

Copy link
Copy Markdown
Collaborator Author

Summary for merge:

  • Merged current main into the branch (clean, no conflicts); just check passes locally on the merged head.
  • Reviews: the OpenAI review found no actionable bugs. Codex raised one P2 (keep the EOF Closed sentinel private); I disagreed and replied inline, since catalog::Consumer already exposes #[doc(hidden)] Hang/Msf variants (which the benches build directly) and making the state private needs a wider API change. Codex then reviewed the final head 2dfbe2e with no findings.
  • CI is green, including TS validation.
  • API: ts::Export::new(source, catalog) and follow(broadcast, catalog) are synchronous and take the caller's catalog stream; with_ts and with_catalog_format are removed; Follower::new(export) keeps its shape. No wire, catalog encoding, MPEG-TS format or CLI flag changes.
  • Follow-up: make catalog::Consumer an opaque struct so its encoding variants and EOF state stop being matchable. fix(ts)!: run the export one delay behind its source #5553 rebases onto this next.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 15, 2026
Merged via the queue into main with commit 7121876 Oct 15, 2026
10 checks passed
@kixelated
kixelated deleted the quest/m1/ts-export-catalog branch October 15, 2026 16:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant