Skip to content

feat(mux): measure encoder flush jitter against a shared baseline - #3940

Draft
kixelated wants to merge 4 commits into
mainfrom
quest/main/jitter-flush-clock
Draft

kixelated wants to merge 4 commits into
mainfrom
quest/main/jitter-flush-clock

Conversation

@kixelated

@kixelated kixelated commented Sep 23, 2026

Copy link
Copy Markdown
Collaborator

Problem

Catalog jitter used presentation timestamp spacing as a provisional floor, which could permanently overstate a reordered stream. Browser encoders advertised fixed frame duration hints instead of measuring when encoded frames reached the transport. A slower rendition also needs to be compared with its broadcast peers.

Approach

  • Use a recent minimum encode lateness shared across a broadcast, with a lifetime maximum per rendition. Retain clock-free batch and reorder measurements for imported media.
  • Add explicit flush(timestamp, now) observations to the Rust estimator, container producer, and codec importers. The native audio and video encoders call them after successful writes.
  • Add moq_publish_media_flush to libmoq and MoqMediaProducer.flush to UniFFI. Sample Instant::now() inside Rust for C and language bindings, then expose the call through Python, Swift, Go, and the Kotlin and Dart FFI objects. OBS calls it for encoded audio and video packets; generic file, pipe, and network imports remain clock-free.
  • Measure browser encoder handoffs, normalize text jitter, update the HANG draft and library docs, and add regression tests. Complete the jitter quest and create a separate GStreamer provenance quest.

Impact

  • Rust API: additive Estimator::flush(timestamp: Timestamp, now: Instant), container::Producer::flush(timestamp, now), codec/importer forwarding methods, and moq_mux::Error::JitterDecreased. Existing rendition updates now refuse decreasing jitter, which violates the catalog rule.
  • C and language APIs: additive moq_publish_media_flush(media, timestamp_us) and MediaProducer.flush(timestamp_us) in the affected bindings. The timestamp is a PTS on the broadcast media clock; invalid handles and unrepresentable timestamps return errors.
  • JS API: no signature change. Browser encoder publishers now report measured jitter, and text renditions refuse zero or decreasing jitter.
  • Wire: no framing, field, or schema change. Emitted jitter values can change, and submillisecond text jitter is rounded up to milliseconds.

Alternatives

  • A per-rendition minimum hides a constant delay on a slower encoder, so the baseline is shared across the broadcast.
  • Automatically observing all imports would turn ingest stalls into encoder jitter. The GStreamer path needs an explicit encoder provenance boundary before it can call flush.

Follow-ups

  • quest/next/gst-encoder-jitter-provenance.md scopes opt-in GStreamer encoder observations. Generic GStreamer imports remain clock-free.

Validation: just check, just test (1,992 Rust tests, 68 Python tests, and affected JS suites), just obs test, and just test smoke --all. The local Nix shell has no Swift toolchain, so the Swift check was skipped locally.
The first full smoke run marked GStreamer unavailable after its plugin probe. Direct gst-inspect, isolated rust -> gst, and a second full run with all 32 pairs passed without source changes.
After rebasing onto the new release on origin/main, just check is blocked by an unchanged upstream moq-relay dead-code lint at rs/moq-relay/src/relay.rs:698 (serve is never used). The earlier full check passed before that release merge; cargo check -p moq-mux -p moq-video -p moq-audio -p moq-ffi -p libmoq passes on the rebased branch.

(written by GPT-6 Astra)

opencode agent and others added 4 commits September 22, 2026 21:20
Co-Authored-By: GPT-6 Astra <codex@openai.com>
Co-Authored-By: GPT-6 Astra <codex@openai.com>
Keep catalog jitter monotonic and serialize text jitter with ceiling precision. Browser encoders report measured flush delay instead of fixed frame hints.

Co-Authored-By: GPT-6 Astra <codex@openai.com>
Expose explicit flush observations to native encoders and language bindings, and report OBS encoder handoffs. Keep generic imports clock-free and defer GStreamer provenance to a follow-up quest.

Co-Authored-By: GPT-6 Astra <codex@openai.com>
@kixelated
kixelated force-pushed the quest/main/jitter-flush-clock branch from e5856ce to 3cadf3b Compare September 23, 2026 04:20

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant