Skip to content

refactor(cli): give independent importers distinct epochs - #5967

Merged
kixelated merged 10 commits into
mainfrom
quest/m0/broadcast-epoch/distinct-importer-epochs
Oct 20, 2026
Merged

kixelated merged 10 commits into
mainfrom
quest/m0/broadcast-epoch/distinct-importer-epochs

Conversation

@kixelated

@kixelated kixelated commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Independent importers cannot publish identical tracks. Each one has its own loss, start point, and group history. A shared --epoch let a relay resume a subscription mid-group on another importer's media, and moq export ts exited on the rewind (#4354).

Approach

Implements the distinct importer epochs quest, as replanned in moq-dev/moq#5963 and moq-dev/moq#5964. This PR deletes the quest and its references.

  • moq --epoch and MOQ_EPOCH are removed. Each run mints its own epoch. Route and binding epochs stay for replicas of one publisher's bytes.
  • The shared-epoch alignment is reverted (moq-dev/moq#5625, moq-dev/moq#5894):
    • TS PTS and PCR wraps follow the source's own history instead of the era nearest the catalog clock. The 13.25 h bound and the half-period failure cases go with it.
    • The passthrough fallback cut is one second after a group's start again, not a catalog-clock grid. Extrapolated object times are clamped monotonic at a forward jump.
    • The m2ts section's randomAccess covers every published group, and muxRate is the measured source rate, instead of two-second windows.
    • HLS imports anchor their own timebase. Timebase::is_on is removed.
  • Whole private PES (moq-dev/moq#5921) and TS sections (moq-dev/moq#6005), both merged meanwhile, are placed by their PCR in the wrap nearest the program's PTS history, the same rule as the media.
  • The PTS history survives a flagged clock break, so a forward jump across the wrap carries on and a step back still reads as a rewind, like the passthrough's PCR count.
  • A stream still behind a wrap the import's first PTS already crossed steps back into the wrap before, rather than a whole wrap ahead. Its PES before the broadcast clock's zero are dropped as unplaced.

Kept after the audit, since a single importer is better with each:

  • Timebase::place, for sources whose timestamps are real wall clocks (moqsink uses it).
  • A placed clock waits for the catalog's first track, so an importer never publishes an empty catalog.
  • An importer announces once its catalog names a servable track.
  • One PTS history per program, so a stream found after a wrap lands on the program's timeline.
  • The sliding mux-rate meter, for a single importer's rate.

Decisions

No maintainer input was available for this round, so each recommended option was taken.

How should the --passthrough pair be tested, given that export ts --passthrough refuses --stitch to keep source bytes?

  • ✅ Pair the passthrough importers with export ts --passthrough. It stays on its importer while a backup starts, and ends when that importer goes. This is the quest Goal's "any other consumer ends, as on any restart".
  • Let passthrough exports stitch, which rewrites source packets.

How deep should the in-tree bufferbloat run throttle?

  • ✅ 160 kbit/s behind a 2 s queue: below the clip's media rate, so the shaper throttles every run, but each switch still lands. Overflow is logged, not asserted. BBR keeps the queue short, as the TS harness documents.
  • 64 kbit/s, which starved the export: CI's first switch never arrived within 30 s.

Should MIGRATE notes cover the removed Timebase::is_on, import::Config::timebase and Error::ForeignTimebase?

  • No, since none of them shipped (release has no Timebase).
  • ✅ Yes (maintainer, after review): Unreleased entries in rs/moq-cli, rs/moq-hls and rs/moq-mux MIGRATE.md cover --epoch/MOQ_EPOCH, Config::timebase/Error::ForeignTimebase, and Timebase::is_on, each with how to migrate.

How should an import started just past a wrap place a stream still before it?

  • ✅ Keep timestamps verbatim and read the step back as negative. PES before the broadcast clock's zero are dropped and counted in stats::Stream::unplaced, as a whole PES there already is. That costs at most the few hundred ms the stream lags behind the wrap.
  • Start the PTS history one wrap in, which keeps those frames but shifts every TS-imported timestamp and the catalog clock by 26.5 h (it broke 32 tests, including the export round trips).

Validation

  • just check: lint, and all tests passed.
  • export ts exits with "frame timestamp is below the live edge" when the relay switches to a same-hop standby quickly #4354's 1+1 setup on lite-07: one relay, two moq import ts fed one live stream, two moq export ts --stitch. Starting the backup, killing the newest, and restarting it each produce one marked program switch (a new PAT version, every PID flagging the break), and no exit. The bufferbloat variant passed 5 of 5 local runs.
  • a_flagged_jump_across_a_wrap_carries_on fails without the kept history (the import rejects the jump as a rewind) and passes with it.
  • a_stream_behind_the_first_wrap_steps_back fails without the signed history (the peer lands a whole wrap ahead) and passes with it.
  • The passthrough pair: the export stays up while the backup announces its own epoch, and ends once its importer is killed, with the backup still live.

Viewer output gap at each switch (export --delay 3s):

Switch Clean Bufferbloat (160 kbit/s, 2 s queue)
Backup starts (newer epoch wins) 2.9 s 5.4 s
Newest killed (older one wins back) 11.3 s 11.8 s
Backup restarts 2.9 s 5.7 s

When the newest importer is killed, the 11 s gap is the relay noticing the dead session (QUIC idle timeout) before the older route wins back. Each other gap is about the export's delay, plus the queue under bufferbloat.

Impact

  • CLI: the unpublished --epoch flag and MOQ_EPOCH are gone.
  • Rust: moq_hls::import::Config::timebase, moq_hls::Error::ForeignTimebase and moq_mux::catalog::Timebase::is_on are removed, with MIGRATE entries.
  • Wire: the m2ts catalog section (in no published draft) goes back to track-lifetime randomAccess, a measured muxRate, history-based PCR unwrap, and a one-second fallback cut. drafts/draft-lcurley-moq-mpegts.md is updated. The e2ee draft drops "explicit or shared epoch".

Follow-ups

  • The relay drill bursts_cross_a_transport_peer::impaired failed once on the io_uring job of the previous head ("the subscription aborted: internal error"). It passed on this head, and this PR does not touch the relay.
  • TS validation's "full multiplex at its own rate" fails with PID 258 B underflow x1 on this head. The same failure hit unrelated branches in the same hour (fix/quic-no-default-clippy, quest/m1/auth/request-race), and no open PR fixes it. The job is not required.

Closes #4354

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 3 commits October 19, 2026 11:02
Remove the CLI shared epoch, HLS clock override, and epoch-only TS grid/window machinery. Restore passthrough history-based cuts and clamping; unwrap PES against program history instead of wall time.

Checkpoint: cargo fmt and git diff --check passed. CLI, mux, and HLS suites are compiling. Full just check, draft validation, interop, demux 1+1 and bufferbloat acceptance are not yet complete. The original before-fix regression build was interrupted by source changes and is not evidence of reproduction.

Unresolved: quest requests passthrough --stitch acceptance, but passthrough intentionally refuses --stitch to preserve source bytes. Await maintainer choice before altering or asserting that behavior.

Co-Authored-By: GPT-6 <noreply@openai.com>
Preserve in-progress demux 1+1 and finite bufferbloat acceptance tests. The plain 1+1 test and fast-file regression passed before arrival-gap instrumentation and the shaped variant were added. The shaped build and draft validation were interrupted on maintainer instruction to skip this quest while passthrough acceptance is unresolved. No scope decision was made; passthrough identity and refusal remain unchanged.

Checks: original targeted CLI/mux/HLS suite 1907 passed, 5 skipped; initial new demux and fast-file regressions 2 passed. Formatting and diff checks pass. Current shaped test build, full just check, drafts, interop, and benchmark validation remain incomplete.

Co-Authored-By: GPT-6 <noreply@openai.com>
@kixelated

kixelated commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator Author

Stopped at the maintainer's request to skip uncertain quests. Checkpoints 49f8ca9 and 0715e18 preserve all source; passthrough byte identity and its refusal of --stitch remain unchanged.

Initial CLI/mux/HLS checks passed 1907 tests (5 skipped), and the initial demux 1+1 plus fast-file regressions passed 2 tests. Later bufferbloat instrumentation/build and draft validation were interrupted; full checks, interop, benchmarks, and gap measurements remain incomplete. The draft body records the unresolved passthrough acceptance choice.

Recommendation when resumed: retain byte identity, test passthrough termination on restart, and reserve marked program switching for demultiplexed exports, subject to a maintainer decision.

(written by GPT-6.1-Sol)

kixelated and others added 2 commits October 19, 2026 16:47
Place whole PES on the program's PTS history instead of the removed clock-nearest era.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Window each stitched program from the PCR that opens its slot, throttle the bufferbloat run below the media rate without starving it, and cover a passthrough pair: its export stays on its importer and ends with it. Delete the finished quest, drop MIGRATE notes for APIs that never shipped, and rewrap the docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review October 20, 2026 00:47
@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 1 minute.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 402ede26-a712-4371-95a3-14021a2e9fc9
📥 Commits

Reviewing files that changed from the base of the PR and between 3b77d23 and bfb8593.

⛔ Files ignored due to path filters: 1
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing: 38
  • doc/bin/cli.md
  • doc/bin/relay/cluster.md
  • doc/concept/hang.md
  • doc/concept/moq-lite.md
  • drafts/draft-lcurley-moq-e2ee.md
  • drafts/draft-lcurley-moq-mpegts.md
  • quest/m0/broadcast-epoch/README.md
  • quest/m0/broadcast-epoch/distinct-importer-epochs.md
  • quest/m1/README.md
  • quest/m1/redundant-ingest-guide.md
  • quest/m2/e2ee/cli.md
  • quest/m2/release/feature-review.md
  • quest/m2/ts-hitless.md
  • rs/hang/MIGRATE.md
  • rs/hang/src/catalog/m2ts.rs
  • rs/moq-cli/Cargo.toml
  • rs/moq-cli/MIGRATE.md
  • rs/moq-cli/src/announced.rs
  • rs/moq-cli/src/args.rs
  • rs/moq-cli/src/complete.rs
  • rs/moq-cli/src/fetch.rs
  • rs/moq-cli/src/hls.rs
  • rs/moq-cli/src/main.rs
  • rs/moq-cli/src/publish.rs
  • rs/moq-cli/tests/integration/export.rs
  • rs/moq-cli/tests/integration/import.rs
  • rs/moq-hls/MIGRATE.md
  • rs/moq-hls/src/error.rs
  • rs/moq-hls/src/import.rs
  • rs/moq-mux/MIGRATE.md
  • rs/moq-mux/src/catalog/mod.rs
  • rs/moq-mux/src/catalog/producer.rs
  • rs/moq-mux/src/catalog/timebase.rs
  • rs/moq-mux/src/container/ts/import.rs
  • rs/moq-mux/src/container/ts/mux_rate.rs
  • rs/moq-mux/src/container/ts/passthrough.rs
  • rs/moq-mux/src/container/ts/passthrough_test.rs
  • rs/moq-mux/src/container/ts/stats.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-20T04:06:25.804831Z bfb8593 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ac8d3dc5fb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rs/moq-mux/src/container/ts/import.rs Outdated
kixelated and others added 2 commits October 19, 2026 18:44
Place TS sections (#6005) on the program's PTS history like whole PES, and keep that history across a flagged clock break, so a forward jump across the 33-bit wrap carries on instead of rewinding.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4a1705fff2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rs/moq-cli/MIGRATE.md
Comment thread rs/moq-hls/src/import.rs
Comment thread rs/moq-mux/src/catalog/timebase.rs
Add Unreleased MIGRATE entries for moq --epoch / MOQ_EPOCH, moq_hls import::Config::timebase and Error::ForeignTimebase, and moq_mux catalog::Timebase::is_on, and stop clearing MOQ_EPOCH in CLI tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d8d1b2a0c3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rs/hang/src/catalog/m2ts.rs
Add Unreleased entries to hang and moq-mux MIGRATE.md: randomAccess stays false once a group starts off a random access point, muxRate follows the measured source rate, and passthrough fallback cuts follow each group's start.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 85c88fa007

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rs/moq-mux/src/container/ts/import.rs Outdated
An import started just past a wrap read a lagging stream's pre-wrap PTS
as a whole wrap forward. The PTS history is now signed, so it reads as a
small step back; a PES landing before the broadcast clock's zero is
dropped and counted as unplaced, as a whole PES there already is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated added this pull request to the merge queue Oct 20, 2026
Merged via the queue into main with commit 9b90cdd Oct 20, 2026
16 checks passed
@kixelated
kixelated deleted the quest/m0/broadcast-epoch/distinct-importer-epochs branch October 20, 2026 04:42
t0ms added a commit to t0ms/moq-dev that referenced this pull request Oct 20, 2026
--epoch is gone (moq-dev#5967), so its refusal of --stats goes with it; the stats
report asks Demand::wanted (moq-dev#5930) whether anyone reads it.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

export ts exits with "frame timestamp is below the live edge" when the relay switches to a same-hop standby quickly

1 participant