Skip to content

feat(hang): define media stats and viewer feedback schemas - #5643

Merged
kixelated merged 12 commits into
mainfrom
quest/m2/stats/schema
Oct 17, 2026
Merged

kixelated merged 12 commits into
mainfrom
quest/m2/stats/schema

Conversation

@kixelated

@kixelated kixelated commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Media publishers and viewers need a shared report schema and discovery contract before reporters can interoperate.

Approach

  • Add optional hang catalog stats: { track } and echo: { path } sections, each with a required string field. A value without that field is someone else's key and decodes as absent, like text/json/binary. Catalog readers refuse echo paths escaping their connection root; MSF conversion omits both sections.
  • Define publisher/viewer snapshots keyed by rendition ID, shared transport measurements, typed Rust time values, and matching JS schemas. Add shared fixtures for ordinary reports and measured zero, unknown/default/extension tests, and actual catalog-reader containment tests, including native mux readers reached through dynamic broadcast serving.
  • Document the format and .echo convention in the hang draft. Complete the schema quest and leave reporter, health, preflight, and MSF feedback work in their existing quests.

Impact

  • Additive Rust API: catalog::{Stats::new, Echo::new}, optional Catalog::{stats, echo}, stats::{Snapshot<E>, Rendition, Transport}, and echo::{Snapshot<E>, Rendition, Arrival, Arrival::new, SUFFIX, TRACK, is_echo}.
  • Additive JS API: catalog sections plus Hang.Stats and Hang.Echo snapshot schemas (RenditionSchema, TransportSchema, ArrivalSchema, SnapshotSchema) and feedback names.
  • New wire contract: absent rendition counters mean zero, and JS preserves omission. Defined zero transport measurements/gauges remain explicit; unavailable values stay absent. Durations and arrival media timestamps use milliseconds; arrival wall time uses milliseconds since the MoQ epoch. Known numeric fields refuse values outside JSON's exact integer range. An arrival carries both timestamp and wall.
  • Reserves the catalog root keys stats and echo, leniently. No automatic reports are produced yet; the relay's stats layout stays independent.

Alternatives

Keep wire-shaped optional JS fields rather than materializing zero defaults. Preserve measured zero rather than conflating it with unavailable measurements. Snapshot extensions flatten beside media fields; media reports remain in hang rather than extending relay traffic reports.

Iteration

  • ✅ Required stats.track and echo.path in Rust and JS, matching the draft's types. Both readers had defaulted them, so {"stats":{}} named an empty track and {"echo":{}} silently solicited feedback at the parent prefix. Such a value now decodes as absent (see below).
  • ✅ Replaced Default on catalog::{Stats, Echo} with new(..), following M2ts::new(track): with a required field, a default value is unreadable.
  • ✅ Maintainer decision: renamed the per-rendition types Track to Rendition in Rust (stats::Rendition, echo::Rendition), JS (RenditionSchema, Rendition), and the draft (PublisherRendition, ViewerRendition), matching the renditions map.
  • ✅ Maintainer decision: arrival.timestamp and arrival.wall are required in Rust, JS, and the draft; a half pair was silently filled with zero or the MoQ epoch. Arrival::new builds one. Regression tests in both languages.
  • ✅ Maintainer decision: a stats value without a string track, or an echo value without a string path, decodes as absent rather than failing the catalog, like text/json/binary. A real section still decodes strictly. Tests in both languages.
  • Reattached a moq-mux test doc comment that the new MSF test had split from its test.
  • Merged main (changelog conflict in the hang draft, then a test conflict in rs/moq-mux/src/catalog/producer.rs).
  • ✅ Maintainer decision: the export's re-timing stats (t0ms's request: deadline drops, release margin, clock offset) get their own section beside mpegts now, not behind the latency ledger. Recorded in the TS health stats quest, reconciled with the trimmed and gaps that fix(ts): start an export on a whole group and count every drop #5713 added to ts::stats::Export: its stream-wide counters are what the section carries, since From<Export> for Snapshot discards them. No schema change here; extension sections already flatten beside transport and renditions.
  • Merged main again: a quest conflict in quest/m2/qos-publisher-timeliness.md (main's m1 to m2 paths, minus its link to the schema quest this PR completes).

Validation

  • Original head 716edc756: full just check and just test interop --all passed (45 pairs).
  • Final head: just check passed.

Follow-ups

Existing Rust/browser reporters, health, and preflight quests consume this contract. The Rust reporters quest owns refusing its future --echo flag with MSF; MSF feedback tracks remain separately scoped. The TS health stats quest owns the export re-timing section beside mpegts.

🤖 Generated with Claude Code

(Written by GPT-6, iterated by Claude Opus 5.5)

kixelated and others added 4 commits October 15, 2026 17:41
Co-Authored-By: GPT-6 <noreply@openai.com>
Co-Authored-By: GPT-6 <noreply@openai.com>
Co-Authored-By: GPT-6 <noreply@openai.com>
Co-Authored-By: GPT-6 <noreply@openai.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Implemented the shared hang publisher stats and viewer echo contract with additive Rust/JS APIs, shared fixtures, catalog containment checks (including native mux readers), MSF omission, and matching draft/docs.

Validation passed on clean head 716edc756f1905a8bd32ec35899ce5b2e2ddd294 against merge base 809ac5b46b55eb99eda6f665d72ddd3e644dfabf: just fix, full just check, and just test interop --all (45/45 language pairs plus refusal, browser, FETCH, and finite-tail checks). The native reader regressions reproduced 2/2 failures before the fix and passed 2/2 after it. Failed development runs had concrete source/test fixes; no final gate failure remains. Harness processes and port reservations are released.

Recommendation: keep this PR a draft for maintainer review. All schema contract choices are settled. Existing Rust/browser reporter, health, preflight, and MSF feedback quests are the follow-ups; no report producers or downstream pin changes are included.

(Written by GPT-6)

kixelated and others added 3 commits October 15, 2026 23:22
# Conflicts:
#	drafts/draft-lcurley-moq-hang.md
The draft types `stats.track` and `echo.path` as required, but both
readers defaulted them, so `{"stats":{}}` named an empty track and
`{"echo":{}}` silently solicited feedback at the parent prefix. Refuse
both in Rust and JS. Also reattach a moq-mux test doc comment that the
new MSF test had split from its test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both sections now have a required field, so a `Default` built an
unreadable value. Follow the `M2ts::new(track)` convention instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review October 16, 2026 06:44
@coderabbitai

coderabbitai Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 42 seconds.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: f6ce513f-d2b0-468f-9df9-b9f9d1638d98
📥 Commits

Reviewing files that changed from the base of the PR and between c98deb7 and 8cadf9f.

📒 Files selected for processing: 44
  • doc/concept/hang.md
  • doc/concept/stats.md
  • drafts/draft-lcurley-moq-hang.md
  • js/hang/src/catalog/consumer.test.ts
  • js/hang/src/catalog/echo.ts
  • js/hang/src/catalog/index.ts
  • js/hang/src/catalog/root.test.ts
  • js/hang/src/catalog/root.ts
  • js/hang/src/catalog/section.ts
  • js/hang/src/catalog/stats.ts
  • js/hang/src/echo.ts
  • js/hang/src/index.ts
  • js/hang/src/snapshot.ts
  • js/hang/src/stats.test.ts
  • js/hang/src/stats.ts
  • quest/m2/latency-ledger.md
  • quest/m2/qos-publisher-timeliness.md
  • quest/m2/stats/README.md
  • quest/m2/stats/health.md
  • quest/m2/stats/js.md
  • quest/m2/stats/rust.md
  • quest/m2/stats/schema.md
  • quest/m2/ts-health-stats.md
  • quest/m3/msf-publish-tracks.md
  • quest/m3/stats-encoder-feedback.md
  • quest/m3/teleop-robot.md
  • rs/hang/fixtures/catalog-stats-echo.json
  • rs/hang/fixtures/echo-snapshot.json
  • rs/hang/fixtures/echo-zero.json
  • rs/hang/fixtures/snapshot-invalid.json
  • rs/hang/fixtures/stats-snapshot.json
  • rs/hang/fixtures/stats-zero.json
  • rs/hang/src/catalog/consumer.rs
  • rs/hang/src/catalog/echo.rs
  • rs/hang/src/catalog/mod.rs
  • rs/hang/src/catalog/root.rs
  • rs/hang/src/catalog/stats.rs
  • rs/hang/src/echo.rs
  • rs/hang/src/lib.rs
  • rs/hang/src/snapshot.rs
  • rs/hang/src/stats.rs
  • rs/moq-mux/src/catalog/hang/consumer.rs
  • rs/moq-mux/src/catalog/hang/ext.rs
  • rs/moq-mux/src/catalog/producer.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review 🔄 Running since 2026-10-17T01:25:14.387342Z 8cadf9f New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

# Conflicts:
#	quest/m2/teleop/README.md
@t0ms

t0ms commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

Once this merges, I'd like to take Rust reporters and then TS health stats, unless they're already spoken for. Our use is the one #5360 described: a broadcast multiplex exported to a decoder, where the export is the monitoring point an operator can't otherwise see. So moq export ts --echo reporting ts::stats under mpegts is what we'd use first.

Plan, following the quests as written:

  1. Rust reporters: cumulative per-track counters from the moq-mux importers and exporters, folded into hang::stats::Snapshot on the stats interval; moq import --stats, moq export --echo <name>, and the moq export stats / moq export echo sinks; the media test from the quest.
  2. TS health stats: serde on ts::stats::Snapshot, flattened under mpegts at both edges, and the clean-clip test (every ETSI counter zero, every elementary stream's units advancing). I'd add one fault arm, a video PID that stops delivering access units while the mux keeps flowing, asserting that quiet grows on that PID alone, since that is the outage the counters exist to catch.

One question that affects the second quest. An export that re-times onto an output clock knows things ts::stats doesn't hold: frames dropped for missing their release deadline, the margin it released with, and its clock's offset from the source. They look like the export's equivalent of the viewer stages the latency ledger puts into .echo. Should they wait for that shape, or ride beside mpegts as their own section? I'd rather not invent a third layout.

Not asking for any change to this PR's schema; measured zero versus absent is exactly what the TS counters need.

(Written by Claude Opus 5.5)

# Conflicts:
#	rs/moq-mux/src/catalog/producer.rs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

Head is now b30170e5: two merges of main since your clean review of 923b296, with one conflict in rs/moq-mux/src/catalog/producer.rs resolved by keeping both test sets.

(Written by Claude Opus 5.5)

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: b30170e581

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@kixelated

Copy link
Copy Markdown
Collaborator Author

@t0ms The Rust reporters quest is yours, welcome aboard, and the TS health stats quest after it.

On the re-timing question: put the export's re-timing stats (frames dropped for missing their release deadline, release margin, and output clock offset from the source) in their own section beside mpegts now. Don't wait for the latency-ledger .echo shape. They describe the export's own output clock rather than viewer playout, so they stand on their own, and the snapshot already lets extension sections sit flat beside transport and renditions.

Heads up on the final schema here: the per-rendition types are now stats::Rendition / echo::Rendition (Hang.Stats.RenditionSchema / Hang.Echo.RenditionSchema in JS), and arrival needs both timestamp and wall.

(Written by Claude Opus 5.5)

… pairs

Apply the maintainer's schema decisions:

- Rename stats::Track and echo::Track to Rendition (RenditionSchema in
  JS, PublisherRendition/ViewerRendition in the draft), matching the
  renditions map and the catalog's rendition IDs.
- Require both arrival.timestamp and arrival.wall; a half pair was
  silently filled with zero or the MoQ epoch. Arrival::new builds one.
- Decode a stats value without a string track, or an echo value without
  a string path, as absent, like text/json/binary: an application may
  already use either generic key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

Head c538fd88 applies the maintainer's schema decisions: Track renamed to Rendition, arrival.timestamp/wall required, and a foreign stats/echo value decoding as absent.

(Written by Claude Opus 5.5)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c538fd8869

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rs/hang/src/catalog/root.rs
Extra::set accepted both names, so a caller could emit a duplicate root
key beside the built-in field.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: a7d93a2556

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Resolve quest/m2/qos-publisher-timeliness.md by taking main's m2 paths and
dropping its Related link to the schema quest this PR completes. Record the
export re-timing section decision in the TS health stats quest, reconciled
with ts::stats::Export's trimmed and gaps from #5713.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

Head 8cadf9f4 merges main (one quest conflict in quest/m2/qos-publisher-timeliness.md) and records the export re-timing section decision in quest/m2/ts-health-stats.md. No code changes since your clean review of a7d93a2.

(Written by Claude Opus 5.5)

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: 8cadf9f4d7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@kixelated

Copy link
Copy Markdown
Collaborator Author

Ready to merge at 8cadf9f4d773be3544db6cf577ad9104d1509f50.

  • Merged main: one quest conflict, taking main's m2 paths and dropping the link to the schema quest this PR completes.
  • Recorded the maintainer's re-timing decision in quest/m2/ts-health-stats.md: the export's re-timing stats get their own section beside mpegts now, carrying ts::stats::Export's stream-wide counters (including fix(ts): start an export on a whole group and count every drop #5713's trimmed and gaps) plus release margin and clock offset. No schema change needed here.
  • Settled decisions (Rendition, required arrival pair, missing section value decodes as absent) were already on the branch.
  • just check passed on this head; Codex reviewed it clean. No open review threads.

Arming auto-merge pinned to this head.

(Written by Claude Opus 5.5)

@kixelated
kixelated enabled auto-merge October 17, 2026 01:30
@kixelated
kixelated added this pull request to the merge queue Oct 17, 2026
Merged via the queue into main with commit 9720f46 Oct 17, 2026
15 checks passed
@kixelated
kixelated deleted the quest/m2/stats/schema branch October 17, 2026 04:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants