Skip to content

feat(mux): report TS health and the export's release clock - #5849

Merged
kixelated merged 23 commits into
moq-dev:mainfrom
t0ms:quest/m2/ts-health-stats
Oct 21, 2026
Merged

kixelated merged 23 commits into
moq-dev:mainfrom
t0ms:quest/m2/ts-health-stats

Conversation

@t0ms

@t0ms t0ms commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Replaces #5784, which GitHub closed when #5643 merged and its branch was deleted. Stacked on the Rust reporters PR, #5848; until that merges, this diff shows both commits. Merged #5848's head, which carries main at f13283b30, bounds the report sinks (a cap on a report's reconstructed state, a 5 s drain for an ended echo reader, 1,024 rendition IDs per stats registry, ended ones making way past it), and lets export stats read the connection root and export echo a hidden prefix. From main: the decoder's stats handle is report there, main's unplaced and refused counters are omitted at zero like the rest and the stats doc names them, and a declared break is not a gap (#5815). Rebased onto main, which now carries #5643: #5713's trimmed and gaps sit under release beside dropped, and the passthrough counts its gaps too. That is the re-timing section the decision recorded on #5643 describes.

Problem

ts::stats counts the TR 101 290 checks and each elementary stream's delivery, but only logs them.
The TS health stats quest
asks for them in the media reports. On #5643, the maintainer asked for the export's re-timing stats
(frames dropped at the release deadline, the release margin, and the output clock's offset from the
source) to sit in their own section beside mpegts.

Approach

  • serde on ts::stats. Snapshot, Stream, and Class are camelCase, with every field
    defaulted. Counters are omitted at zero, as hang's are. quiet is whole milliseconds. An unknown
    class reads as data. PID keys go through DisplayFromStr: flattened into a snapshot, a section
    loses serde_json's integer-key parsing, and hang::stats::Snapshot<Report> would not read back
    ("invalid type: string "256", expected u16").

  • ts::stats::Report { mpegts, release } is the E of both hang::stats::Snapshot and
    hang::echo::Snapshot. An import fills mpegts from Import::stats(). A TS export fills it with
    its rows and adds release. A passthrough export reports release alone.

  • ts::stats::Release: dropped, margin, offset, drift (ppm), and outOfTolerance.
    stats::Export becomes { streams, release }, and ts::Release::stats() gives the passthrough's.
    The jitter buffer measures the two new gauges:

    • margin: the least time any frame arrived before its deadline over each whole second of our
      clock, negative when one missed.
    • offset: the gap between the slack of the track sent latest and the delay, which the clock pulls
      shut, as last measured. Like drift, it is absent until the rate is measured.

    In Rust both are a SignedDuration, signed nanoseconds. A report carries whole milliseconds,
    rounded away from zero, so a miss under a millisecond stays negative.

  • CLI: the import's decode loop sets mpegts at each one-second sample and after the final
    drain. The TS export and the passthrough set theirs as they sample. The release-clock log line
    gains margin_ms and offset_ms. Both sinks stop reading a report past 8 MiB rather than 1 MiB:
    twice the largest a TS import can publish, a row for each of the 8,192 PIDs.

  • Docs: a doc/concept/stats.md § MPEG-TS sections covers:

    • each counter's TR 101 290 check;
    • the monitoring point (an import grades what arrived; an export runs no checks; neither grades the
      downstream wire);
    • the PCR checks graded on values;
    • PID_error covered by units and quiet;
    • the release fields.

    Also a paragraph in doc/bin/cli.md. The quest file is removed, with its links.

Decisions a reviewer may want to revisit:

  • The section name release, and trimmed and gaps in it: they are the
    export's own counts, not a row's, though gaps is about the source rather than the clock.
  • margin over one second (a report's interval) rather than since the last sample, so two readers do
    not interfere.
  • Whole milliseconds on the wire for margin and offset, a typed span in Rust.

Impact

  • Breaking for readers of ts::stats::Export: dropped, trimmed, gaps, drift, and
    out_of_tolerance move under release. rs/moq-mux/MIGRATE.md gives the new path.
  • Class::Data moves last, which #[serde(other)] requires; explicit discriminants keep each
    class's as value.
  • Reports gain sections only for TS. The .z deltas stay small, since counters only climb.

Results

just ci passes: 3,557 and 480 tests, bar two of main's own checks that fail on macOS only
(moq-carla-bridge's rustdoc and bench/relay.test.py), the steps after each run by hand. On macOS,
play's link step trips build.warnings on the __eh_frame linker note; with warnings allowed it
passes 224 (one skipped), that note being its only warning.

Test What it shows Result
a_report_flattens_beside_the_media_counters the wire shape (camelCase, PID keys, quiet in ms, zeros omitted) and a full read-back through hang::stats::Snapshot<Report>; an unknown class reads as data new, pass; fails on the read-back without DisplayFromStr
class_keeps_its_discriminants Class's as values stay Data = 0, Audio = 1, Video = 2 with Data last; an unknown name reads as data new, pass; fails with implicit discriminants
the_margin_is_the_least_spare_over_a_second a frame 20 ms late gives a −20 ms margin once the second closes, held until the next new, pass
a_release_span_under_a_millisecond_keeps_its_sign a 0.3 ms miss is negative and reports -1, 0.3 ms to spare 1, −12 ms -12 new, pass; the old rounding gave 0
the_largest_ts_report_reads_whole (CLI) a report with a row for each of the 8,192 PIDs, every counter and field at its limit and 1,024 renditions, reads back whole through moq_json::snapshot: 3.48 MB, under half the cap new, pass; TooLarge at 1 MiB
the_clock_follows_a_drifting_source_for_a_day now also: offset within the run's existing bound at ±25 and ±30 ppm, and margin equals the delay plus offset to 1 ms pass
a_slow_source_clock_keeps_the_delay (export) now also: a positive margin and a measured offset pass
import_stats_count_what_each_rendition_wrote (CLI) now also: mpegts present, every TR 101 290 counter zero on the clean clip, every PID's units advancing and ending equal to an in-process import's, and no release at import pass
import_stats_report_a_silent_video_pid (CLI) the fault arm offered on #5643: the video PES suppressed 2.5 to 4.5 s with its PCR and continuity legal. Video quiet peaks (the import logged 1.24 s) while audio stays under 500 ms; video counts again after; no priority 1 check fires new, pass
export_echo_reports_what_the_export_reads (CLI) now also: the export's rows advance and stay clean, and release carries a margin (2,737 ms on a 4 s delay, nothing dropped) pass

(Written by Claude Opus 5.5)

@t0ms
t0ms force-pushed the quest/m2/ts-health-stats branch from 01560c5 to 3f2cb92 Compare October 17, 2026 05:12
@t0ms
t0ms marked this pull request as ready for review October 17, 2026 06:49
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-21T09:35:59.190456Z bda2cf4 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 73f03874-8040-4086-8104-cd42e411dac1
📥 Commits

Reviewing files that changed from the base of the PR and between 0f0d474 and 9c56ab2.

📒 Files selected for processing: 6
  • doc/bin/cli.md
  • rs/moq-cli/src/main.rs
  • rs/moq-cli/src/report.rs
  • rs/moq-mux/src/catalog/producer.rs
  • rs/moq-mux/src/source.rs
  • rs/moq-mux/src/stats.rs
🚧 Files skipped from review as they are similar to previous changes: 3
  • doc/bin/cli.md
  • rs/moq-mux/src/catalog/producer.rs
  • rs/moq-mux/src/source.rs

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 2 remain after this review.

Walkthrough

The change adds cumulative rendition counters and MPEG-TS and release metrics to the mux. The CLI adds import --stats, export --echo, and JSON-line stats and echo sinks. Validation covers supported sources, sinks, and catalog formats. Import and export reports include transport and MPEG-TS data when available. Tests cover counters, serialization, CLI validation, and end-to-end reporting. Documentation and planning references are updated.

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to 9c56a

No actionable merge-blocking risk remains from the assessed changes; normal checks can proceed.

Pre-merge checks: Passed 5
Check name Status Explanation
Title check Passed The title clearly summarizes the main changes: MPEG-TS health reporting and export release-clock reporting.
Docstring Coverage Passed Docstring coverage is 84.66% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 176 functions across 22 files. (1 skipped: …
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description check Passed The pull request description directly explains the MPEG-TS health statistics, export release statistics, CLI behavior, serialization changes, documentation updates, tests, and migration impact covered…
Full details: Docstring Coverage

Explanation

Docstring coverage is 84.66% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 176 functions across 22 files. (1 skipped: 1 unsupported.)

Full details: Description check

Explanation

The pull request description directly explains the MPEG-TS health statistics, export release statistics, CLI behavior, serialization changes, documentation updates, tests, and migration impact covered by the changeset.

✨ Finishing Touches: 💡 1
⚔️ Resolve merge conflicts: 💡
  • Resolve merge conflict in branch quest/m2/ts-health-stats
✨ Simplify code
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@t0ms
t0ms marked this pull request as draft October 17, 2026 10:51
t0ms and others added 3 commits October 17, 2026 11:51
`moq import --stats` names a stats track in the catalog the import builds and
fills it with what each rendition has written: frames, payload bytes, and video
keyframes, with the connection's transport. `moq export --echo <name>`
publishes the feedback a catalog solicits, counting what the export reads and
its newest arrival. `moq export stats` and `moq export echo` print either as
JSON lines.

moq-mux counts at the container producer and the source, so every container
import and export shares the counters. Capture encoders, gateways, `archive`,
and `ts --program all` refuse `--stats`; MSF catalogs refuse `--echo`.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
- Count a buffered frame once the container writes it, not on the way in: an abort discards the buffer.
- Refuse `--echo` for an MSF catalog detected from a `.msf` broadcast, not only one named.
- Refuse an echo name starting with `.`, which discovery hides.
- `export echo` replaces a reader on a restart, lets one drain after its broadcast ends, and
  bounds the lines it queues for stdout.
- A report track that refuses a write is closed with the cause and reported once, instead of
  logged at debug every second; a failed import aborts its stats track with the import's error.
- The stats integration test joins `tests/integration`; the arrival test fakes the wall clock.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
A TS import's stats and a TS export's feedback now carry an `mpegts` section:
each elementary stream's access units and quiet time, and the TR 101 290
counters. An export adds `release`, how its output clock keeps up with the
source: frames dropped at the deadline, the least margin frames arrived with
over the last second, the clock's offset and drift from the source, and
measurements past the 30 ppm a system clock may run off.

`ts::stats` gains serde and `Report`, the extension both snapshots flatten.
`stats::Export` moves its clock fields under `release`, and `ts::Release`
reports the same for a passthrough export.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@t0ms
t0ms force-pushed the quest/m2/ts-health-stats branch from 3f2cb92 to 9974c23 Compare October 17, 2026 11:21
@t0ms
t0ms marked this pull request as ready for review October 17, 2026 11:23
@t0ms

t0ms commented Oct 17, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Something went wrong. Try again later by commenting “@codex review”.

Failed to sample tokens
ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

1 similar comment
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Something went wrong. Try again later by commenting “@codex review”.

Failed to sample tokens
ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

kixelated and others added 5 commits October 17, 2026 20:42
`--epoch` makes instances one publisher of identical tracks, and a stats
track carries this instance's own counts and transport, so the pair is
refused. Each `.echo` feedback broadcast now announces a fresh epoch like
every other CLI publish, so a replaced viewer reads as a restart. The echo
target is a struct instead of a tuple, and the CLI page drops the edge
cases and unshipped fields the API docs and quest already carry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	rs/moq-cli/src/main.rs
#	rs/moq-cli/src/publish.rs
#	rs/moq-cli/tests/integration/main.rs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	rs/moq-cli/src/main.rs
spawn_import gained a shared-epoch argument on main, which pushed it past
clippy's argument limit; the stats report now attaches to the returned
Publish instead, still before any track is declared.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
t0ms and others added 2 commits October 20, 2026 12:48
--epoch is gone (moq-dev#5967), so its refusal of --stats goes with it; the stats
report asks Demand::wanted (moq-dev#5930) whether anyone reads it.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@t0ms
t0ms marked this pull request as draft October 20, 2026 11:52
Takes moq-dev#5848's merge of main. The decoder's stats handle is `report` there, so the
MPEG-TS snapshot closure becomes `report_mpegts`; main's `unplaced` and `refused`
counters are omitted at zero like the rest, and the stats doc names them; a
declared break is not a gap (moq-dev#5815).

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@t0ms
t0ms marked this pull request as ready for review October 20, 2026 13:16

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8a0af79cdc

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread rs/moq-cli/src/report.rs Outdated
track = match &stats {
Some(name) => {
let subscriber = broadcast.track(name)?.subscribe(None).await?;
Some(moq_json::snapshot::Consumer::new(subscriber, Default::default()))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Cap reconstructed report snapshots

When either report sink reads a track from an untrusted publisher, this constructs moq_json::snapshot::Consumer with the default config, whose max_size is unset and explicitly leaves reconstructed state size unlimited; read_echo does the same at line 505. A publisher can send merge patches that continually add keys and make moq export stats or moq export echo retain unbounded JSON, so both consumers need a finite state budget.

AGENTS.md reference: AGENTS.md:L18-L18

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 7bc5705 (#5848, merged here): export stats and export echo both read a report through one reader capped at 1 MiB of reconstructed state (MAX_REPORT). A report is a few KiB; patches that grow one past the cap end the read with TooLarge. a_report_past_the_cap_ends_the_read fails without the cap.

(Written by Claude Opus 5.5)

Comment thread rs/moq-cli/src/report.rs Outdated
} else {
// Detached, not aborted: the final snapshot can still be in flight, and the reader
// ends with its track.
readers.remove(&path);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep retracted feedback readers cancellable

When a peer retracts a feedback route without closing its already-subscribed track, removing and dropping this JoinHandle detaches the read_echo task rather than cancelling it; the origin contract permits tracks already in flight to continue after route retraction. Repeated announce/retract cycles can therefore leave an unbounded number of hidden tasks and subscriptions, and a later start at the same path can interleave old and new snapshots, so ended readers must remain tracked under a bound or be cancelled.

AGENTS.md reference: AGENTS.md:L18-L18

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 7bc5705 (#5848, merged here): an ended reader now stays in the map, so a start at the same path aborts it before the new reader prints, and it is aborted 5 s after the End (DRAIN) even if its track stays open; finished readers are pruned. The drain keeps the final snapshot that the earlier review asked to let through. an_ended_feedback_reader_stops_after_draining retracts a route whose track stays open and fails with the reader detached.

(Written by Claude Opus 5.5)

Comment thread rs/moq-mux/src/stats.rs
/// The counters for `name`, continuing those of an earlier track under that name.
pub(crate) fn track(&self, name: &str) -> Arc<T> {
let mut entries = self.0.lock().unwrap_or_else(PoisonError::into_inner);
entries.entry(name.to_string()).or_default().clone()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Bound lifetime rendition registries

When a remote catalog repeatedly rotates through fresh rendition or stream names, every ExportSource::for_* call inserts another entry here, while the registry intentionally never evicts entries and is shared for the source's lifetime even when --echo is not enabled. An fMP4 or FLV exporter waiting for its initial header can process arbitrarily many valid catalog updates, letting a hostile publisher grow both memory and later stats() sampling cost without bound; cap lifetime distinct IDs or reject further churn.

AGENTS.md reference: AGENTS.md:L18-L18

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 7bc5705 (#5848, merged here): a registry keeps the first 1,024 rendition IDs (MAX_ENTRIES). Past that a new name counts into counters no report reads, so neither the map nor stats() sampling grows; a known name still continues its counts. renditions_past_the_cap_are_not_kept fails without the cap.

(Written by Claude Opus 5.5)

Comment thread rs/moq-cli/src/report.rs
continue;
}
if active {
let reader = tokio::spawn(read_echo(origin.clone(), path.clone(), lines.clone()));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Benchmark route and echo fanout

When export echo starts on a large origin, announcement replay visits all routes and this path creates one task and subscription for every matching feedback broadcast, but the change adds no benchmark sweeping total announced routes against matching echo publishers. Add that two-axis benchmark so growth with the whole route table rather than the touched feedback set is visible.

AGENTS.md reference: AGENTS.md:L40-L40

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not adding one here. The replay over the whole route table is the origin's announced(), shared by every announcement consumer, so a sweep of it belongs with moq-net's benchmarks rather than this sink's. What export echo adds is a prefix comparison per announcement and one reader per matching .echo broadcast, which is the touched set by design, and readers are now bounded in time after their broadcast ends (7bc5705).

(Written by Claude Opus 5.5)

Comment thread rs/moq-mux/src/container/ts/stats.rs Outdated
Comment on lines +222 to +224
#[default]
#[serde(other)]
Data,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve Class's existing discriminants

For downstream code that casts or persists this public fieldless enum, moving Data after Audio and Video silently changes the implicit values from Data=0, Audio=1, Video=2 to Audio=0, Video=1, Data=2. Keep the old discriminants explicitly while satisfying serde(other) so existing Class as ... values remain stable.

AGENTS.md reference: rs/AGENTS.md:L48-L48

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 0f0d474. #[serde(other)] has to sit on the last variant, so Data stays last, and explicit discriminants (Audio = 1, Video = 2, Data = 0) keep the values main gave each class. class_keeps_its_discriminants checks the casts and that an unknown name reads as data; it fails with the implicit values.

(Written by Claude Opus 5.5)

/// One row per elementary stream PID the exporter writes, ordered by PID.
pub streams: BTreeMap<u16, Stream>,
/// How the release clock is keeping up with the source.
pub release: Release,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Add a migration for the Export field move

This replaces the public ts::stats::Export fields dropped, trimmed, gaps, drift, and out_of_tolerance with the nested release field, so existing callers must rewrite field access, yet rs/moq-mux/MIGRATE.md's ## Unreleased section is untouched. Add an entry explaining the move and the new access path.

AGENTS.md reference: AGENTS.md:L63-L63

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 0f0d474: rs/moq-mux/MIGRATE.md now has an Unreleased entry saying that the five counters are fields of Export::release, a ts::stats::Release, which ts::Release::stats also returns, and that callers read export.release.dropped where they read export.dropped.

(Written by Claude Opus 5.5)

t0ms and others added 3 commits October 20, 2026 15:04
- `export stats` and `export echo` read a report with a 1 MiB cap on its reconstructed
  state, so patches that keep adding keys end the read instead of growing the sink.
- `export echo` keeps a reader whose broadcast ended, so a start at the path replaces it,
  and aborts it after 5 s: the final snapshot still drains, and a track that outlives its
  route no longer leaves a reader behind.
- A stats registry keeps the first 1,024 rendition IDs; a catalog that keeps renaming its
  renditions no longer grows it, or each report, without bound.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Takes moq-dev#5848's bounds on the report sinks: a 1 MiB cap on a report's reconstructed state, a
5 s drain for an ended echo reader, and 1,024 rendition IDs per stats registry.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
`Data` stays last for `#[serde(other)]`, and explicit discriminants keep
the `as` values `main` gave each class. MIGRATE.md tells callers where
`Export`'s release counters went.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@t0ms
t0ms marked this pull request as draft October 20, 2026 14:46
@t0ms
t0ms marked this pull request as draft October 20, 2026 17:13
@t0ms
t0ms marked this pull request as ready for review October 20, 2026 17:13
kixelated and others added 2 commits October 20, 2026 13:31
…readers

At the registry cap, a new rendition ID now evicts a single ended one, so the
others keep their final counts. `export echo` forgets a reader once its drain
passes instead of on the next `End`. The sinks read `latest`, unwrapping
main's `Timed` snapshots.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated added a commit that referenced this pull request Oct 20, 2026
Drop recovery's stale #5849 gate (gaps is on main), move the encrypted
moq-json round trip to the json adopter, scope the e2ee consumer to Rust
and JS, aim binding docs at the generated C package, keep Unix listeners
on one acceptor, and tighten several goals.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator

Summary before merge: merged #5848's current head (4b1952a7, the cap eviction and drained echo reader reaping) and then main into this branch, as merge commits on top of t0ms's work. The only conflict was rs/moq-mux/MIGRATE.md, where this PR's ts::stats::Export entry and main's Timed<T> entry (#5677) both landed under Unreleased; both are kept.

Every review finding on this PR is fixed or answered on its thread: Class keeps its discriminants, the Export move has a MIGRATE entry, and the #5848 findings raised here (report caps, reader reaping, registry eviction, connection root, hidden prefixes) are fixed there and merged in. Nothing in this PR's own code changed since the last Codex and CodeRabbit reviews on 0f0d4746.

Local just check passes on 070338f5 (3577 + 516 Rust tests, clippy, JS, docs), and quest check passes.

This PR contains all of #5848, so it should land after #5848.

(Written by Claude Opus 5.5)

@kixelated

kixelated commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Merged main (now carrying #5848 as a3bd488) in 1bc982c, a plain merge commit.

The conflicts all came from #5848 landing as a squash: both sides added its code, and this branch adds the TS sections on top. Each hunk keeps this branch's side; main's later changes in the same files carry over untouched (--listen-timeout bounding admission from #5765, the redundant-ingest link in doc/bin/cli.md). In quest/m2/README.md, the socket-close entry keeps main's rename to quic-socket-close.md, and the ts-health-stats.md links main still had are dropped, since this PR removes that quest.

Against the pre-merge head, the merge adds only main's own changes. The resolution is mechanical, so the earlier review still covers the PR's content.

Checks: just check passes in the Nix dev shell (3,590, 516, and 223 tests), and quest check passes.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 21, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1bc982cff0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rs/moq-mux/src/container/ts/stats.rs Outdated

impl Release {
pub(crate) fn of<K: Ord + Clone, T>(jitter: &crate::jitter::Buffer<K, T>) -> Self {
let millis = |seconds: f64| (seconds * 1e3).round() as i64;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep release timing in a typed signed duration

Release::of rounds both signed second values into bare i64 milliseconds; besides making the public API unit-unsafe, a deadline missed by less than 0.5 ms becomes margin == 0, even though the field promises that every miss is negative, so sign-based consumers cannot detect that deficit. Keep a typed signed duration through the Rust API and perform a sign-preserving millisecond conversion only at the serde edge.

AGENTS.md reference: rs/AGENTS.md:L37-L37

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in bda2cf4. Release::margin and offset are now Option<SignedDuration>, a signed nanosecond span (std::time::Duration cannot go below zero), so the Rust API keeps the unit and the precision. Whole milliseconds appear only at the serde edge, rounded away from zero, so a deadline missed by 0.3 ms reports -1, where the old rounding gave 0. a_release_span_under_a_millisecond_keeps_its_sign pins that, and the doc/concept/stats.md rows say so.

(Written by Claude Opus 5.5)

Comment thread rs/moq-cli/src/publish.rs
let sections = report.as_ref().map(crate::report::Stats::sections);
let report_mpegts = |snapshot: &ts::stats::Snapshot| {
if let Some(sections) = &sections {
sections.set(|report| report.mpegts = Some(snapshot.clone()));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep generated TS reports below the reader cap

When a TS feed rotates its PMT through enough elementary PIDs, Import::stats retains one row per PID across those routes, and populated health fields can make the mpegts object installed here exceed report.rs's 1 MiB MAX_REPORT. The project's own export stats consumer then returns TooLarge and permanently stops monitoring that otherwise valid producer report; either bound/evict the reported PID rows below that limit or choose a bounded reader limit that covers every report this producer can generate.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in bda2cf4 by sizing the cap to what this producer can publish rather than evicting rows. An import keeps one row per PID, and PIDs are 13 bits, so a report holds at most 8,192 rows. the_largest_ts_report_reads_whole builds that worst case (every PID, every counter and field at its limit, 1,024 renditions) and reads it back through moq_json::snapshot: 3.48 MB, which the old 1 MiB cap refused as TooLarge. MAX_REPORT is now 8 MiB, over twice that, and doc/bin/cli.md states the limit.

(Written by Claude Opus 5.5)

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Oct 21, 2026
# Conflicts:
#	rs/moq-mux/MIGRATE.md
@kixelated

Copy link
Copy Markdown
Collaborator

Merged main again in 602fb36, a plain merge commit, after the queue dropped 1bc982c for a conflict.

The only conflict was rs/moq-mux/MIGRATE.md: this PR's ts::stats::Export entry and main's catalog read errors entry from #6114 both landed at the top of ## Unreleased. Both stay, this one first since it lands later. The resolution is mechanical, so the earlier review still covers the PR's content.

Checks: quest check passes, and just check passes in the Nix dev shell (3,598, 516, and 223 tests). Two earlier just check runs each hit a flake main already tracks, neither in code this PR touches: bench/relay.test.py's foreign-listener test (quest/m1/test-flakes-2/relay-bench-holder.md) and export::redundant_importers_switch_through_bufferbloat_on_lite_07 with "stitching export exited" (quest/m1/test-flakes-2/redundant-failover-drift.md). This PR's jitter.rs change only records margin and offset and leaves the clock's decisions alone.

(Written by Claude Opus 5.5)

@kixelated
kixelated enabled auto-merge October 21, 2026 04:35
@kixelated
kixelated added this pull request to the merge queue Oct 21, 2026
@t0ms
t0ms marked this pull request as draft October 21, 2026 08:35
…the report cap for the largest TS report

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@t0ms
t0ms marked this pull request as ready for review October 21, 2026 09:22

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bda2cf44bc

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

sampled = tokio::time::Instant::now();
let stats = ts.export().stats();
let counts = (stats.dropped, stats.trimmed, stats.gaps, stats.out_of_tolerance);
report(&stats);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Sample TS feedback while output is pending

When --delay is still buffering or every incoming frame is late, ts.next().await remains pending while it consumes, queues, or drops input, so this output-only block is never reached even though Echo::run keeps emitting snapshots once per second. The new mpegts and release fields therefore stay absent or stale and can hide an indefinitely rising dropped count or negative margin until a frame is emitted or the source ends; drive section sampling from an independent interval, including in the passthrough loop that repeats this pattern.

Useful? React with 👍 / 👎.

Comment thread doc/concept/stats.md
Comment on lines +48 to +50
`mpegts` holds `streams`, one row per elementary stream keyed by its PID in
decimal, and the stream-wide counters. A row has `track` (the track suffix),
`class` (`audio`, `video`, or `data`; an unknown class reads as `data`),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Move the TS field inventory to reference docs

This new section enumerates the mpegts and release wire shape, including PID-key encoding and individual fields, even though scoped doc/AGENTS.md requires feature docs to leave field lists and wire encodings to API docs or drafts. Keeping a second schema copy here will drift from the serde types, so retain the user-facing monitoring behavior and link to a colocated reference instead.

AGENTS.md reference: doc/AGENTS.md:L4-L6

Useful? React with 👍 / 👎.

Comment thread doc/concept/stats.md
Comment on lines +95 to +97
| `margin` | Gauge, milliseconds rounded away from zero: the least time any frame arrived before its deadline over the last second. Negative when one missed, however narrowly. |
| `offset` | Gauge, milliseconds rounded away from zero: how far the output clock sits from the source's plus the delay. Positive holds frames longer than the delay. The clock pulls it shut over ten minutes. |
| `drift` | Gauge, parts per million: the source clock's rate against ours, positive running fast. |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge State the delay required for clock gauges

With export ts --delay 0, including passthrough, Buffer::queue deliberately skips margin collection and steering never runs, so margin, offset, and drift remain absent forever rather than merely pending a measurement. The new table describes these gauges without that condition, which can make consumers treat a healthy zero-delay export as missing data; state the nonzero-delay requirement alongside the claim.

AGENTS.md reference: doc/AGENTS.md:L9-L9

Useful? React with 👍 / 👎.


impl<'de> serde_with::DeserializeAs<'de, SignedDuration> for SignedMillis {
fn deserialize_as<D: serde::Deserializer<'de>>(deserializer: D) -> Result<SignedDuration, D::Error> {
i64::deserialize(deserializer).map(SignedDuration::from_millis)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject signed report durations that do not fit

When a peer report contains margin or offset outside the roughly 292-year nanosecond range but still inside i64 milliseconds, this deserializer calls from_millis, whose saturating multiplication silently changes the value. The typed report then reserializes to a different duration instead of refusing malformed input; use checked conversion here and return a serde error.

AGENTS.md reference: AGENTS.md:L17-L18

Useful? React with 👍 / 👎.

Merged via the queue into moq-dev:main with commit b54030c Oct 21, 2026
1 check passed
t0ms added a commit to t0ms/moq-dev that referenced this pull request Oct 21, 2026
main now carries moq-dev#5849, whose stats::Release holds the release clock's
counters. --sync's overruns and late join it: set beside gaps after
Release::of, logged on the release-clock line and watched for change
there, serialized only when nonzero like the rest, and listed in the
stats concept page.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
kixelated added a commit that referenced this pull request Oct 21, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants