Repository navigation
Audio slows down and speeds up around its target in every player - #8507
Conversation
Quest-Claim: 583a6d7cd221feb1
moq play, a live <moq-watch>, and a buffered one decide each 30 ms window by the same NetEq rule on a filtered level, the playhead's lag behind the shared clock, and a skip bound from the subscription's max delay. Park, the write-time band skip, and fit() are gone; underruns resume on a cushion and slow down to rebuild. Shared decision and filter vectors pin both ports. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A join plays its first read onto the clock rather than stretching through the backlog a subscription opens on, in the browser ring and natively. After an underrun, playback resumes on a cushion of one window and one read, capped at the floor. A capacity re-anchor counts as a skip only when it dropped audio. The ring reads the host clock through one expected alias, and the mixer's command channel carries the consumer boxed, as the producer's resampler is. Replay budgets are re-measured with the stretch model, and the quest file and its links are removed. Closes #8031 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e it starts - The browser decoder tells the ring what the consumer holds encoded, so the level the stretch steers on reaches past the decode gate, as the replay's already did. - A target that falls moves the clock ahead by as much; both rings now catch that up by speeding up however small the bound, and skip only lateness past what of the fall is left. - The mixer reads the host clock once per device callback and starts each 1024-frame chunk at its own offset, so a later chunk is not read as ahead of the clock. - The native clock and bound are published as one word, released, so the reader never pairs one anchor's clock with another's bound, and sees the target set before it. - The bus boxes its live consumer, as the mixer's command does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…chunk A target that rises after a fall moves the clock back, so it cancels the allowance the fall added rather than leaving a skip deferred past a lag that no longer exists. Both rings do this, with a test each that fails under the add-only rule. The native driver reads the host clock once per device callback and hands each staging chunk its own start, so a callback larger than the staging buffer no longer reads every chunk as if it started with the callback. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mixer::fill takes the device's sample type and converts each chunk as it copies it out, so the driver hands it the whole callback against one clock reading. The driver's staging buffer and the loop over it, which read every staging pass as if it started the callback, go away. The mixer test now fills 4096-frame i16 callbacks, and fails with 2640 accelerated frames when every chunk reads the callback's start. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…arget allowance The bound alone decides a lag skip in both players, so the retarget allowance and its tests go. When the shared clock's reference moves earlier, audio skips as much, as far as it trails the clock, once the jumps add up to one stretch window; target falls and stalls still stretch. Replay budgets re-measured. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Design change: ✅ 1A drop the retarget allowance (rejected: 1B carry the delay with the clock, 1C merge as is). The bound alone decides a lag skip in both players: (Written by Claude Opus 5.5 with medium effort) |
|
Design change: ✅ 2A audio follows the reference's skip-ahead (rejected: 2B hold the consumer until the clock settles, 2C merge as is). When the shared clock's reference moves earlier, the ring skips as much, as far as the playhead trails the clock; target falls and stalls still stretch. Jumps skip once they add up to more than one 30 ms stretch window, so a publisher clock drifting fast, which moves the reference a little at a time, stays the stretch's to catch up (the 1% fast replay test caught per-jump skips raising its lag from under 23 to 60 ms). Browser: (Written by Claude Opus 5.5 with medium effort) |
…efill for the whole callback A reference jump past what is queued keeps its remainder and skips it once the audio lands. A truncate rebases the received horizon to the cut, so the predecessor's retired audio never reads as depth. Native refills after an underrun until what is left of the device callback is queued, not the 1024-frame slice the mixer reads it in. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both rings skip the larger of the lag past the bound and the pending jump, and the jump keeps whatever that skip did not cover. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the target A player sitting behind its clock on a low buffer let the reference's jitter moves pile up to a window and skip into an underrun (native-aac near-zero and wide skipped 4.3 and 5.4 times a minute live). A jump now counts only as far as the ring holds audio past its target, which the early media that moved the reference leaves it. Native derives the jump from the timestamp the clock placed, so rounding never reads as one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 8 minutes. View limit details
Pre-merge checks:
|
…-target/stretch # Conflicts: # js/watch/src/audio/decoder.test.ts
…ed its arrivals Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
ℹ️ No critical issues — one process suggestion on the new corpora.
Reviewed changes
- One decision rule in both players:
decideandLevelinrs/moq-audio/src/stretch.rsandjs/watch/src/audio/stretch.tsnow implement NetEq'sDecisionLogicandBufferLevelFilteron the same five inputs (target,level,raw,lag,bound), and every seam is capped by the decision's allowance. - Native clock anchor:
Sink::due(at, late, jumped)carries the caller's instant onto the host clock, andring::Producer::duepacks the clock and bound into oneAtomicU64while accumulating reference jumps. - Native ring:
ring::Consumergainsplayed/level/now/demand/jumped/joined,read_at(output, now, demand), a cushion refill of one window plus the device callback, and askip()that jumps past the bound and past reference jumps only. - Mixer and driver:
Mixer::fillconverts to the device's sample format itself and dates each chunk from one host reading, so the driver's staging buffer and conversion loop are gone and sinks move boxed through the command channel. - Browser ring:
AudioRingBufferreplacespark/peak/chunk/bufferedwithreceived/level/clock/jumpedandTerms, skips by the clock's bound or a reference jump, resumes on a cushion, and skips a declared silence by its lag. - Browser decode gate:
Backpressuredrops parking and keeps a started ring's depth below its floor plus one window, counting frames still decoding and ignoring each frame's timestamp so a hole plays as silence. - Browser clock posting:
AudioBufferpairssync.now()withAudioContext.currentTimebehind a min-filtered offset, takes real jumps whole, and forwards the newest received end and reference deltas to the worklet. - CLI:
moq playanchors the sink after each write fromtiming.endwithat - end - delayas the reference, andfit/AUDIO_SLACKplus the write-path re-anchor are removed. - Corpora, tests, docs, budgets: the shared PCM corpus now carries
decision/allowance, a newdecisions.jsonadds decision and filter vectors, the JS unit and replay suites are rewritten, every browser and native ceiling is re-measured, andaudio-jitter.mdstates the one rule.
ℹ️ The two stretch corpora have no generator and no drift test
doc/concept/audio-jitter.md now says these vectors cannot be derived from either player, but nothing enforces it: unlike the estimator corpus in the same directory (corpus.ts regenerates, corpus.test.ts fails when the checked-in files no longer match), stretch/corpus.json and stretch/decisions.json are hand-edited JSON that only the two test suites read. Nothing catches a future rule change that quietly moves the vectors to whatever the implementation under test produces, and the new allowance fields in corpus.json are exactly that kind of hand-derived value.
Technical details
# Add a reference implementation and drift test for the stretch corpora
## Affected sites
- `doc/concept/audio-jitter/stretch/decisions.json` — 23 decision and 9 filter vectors, read by `stretch.rs::tests::shared_decision_vectors` / `shared_filter_vectors` and `stretch.test.ts`; no script produces it.
- `doc/concept/audio-jitter/stretch/corpus.json` — `buffered`/`target`/`cooldown` were replaced by `decision`/`allowance`, which only the documented rule can produce; the expected PCM still legitimately needs the pitch search.
- `doc/concept/audio-jitter/corpus.ts` + `corpus.test.ts` — the pattern to copy: `reference.ts` is a non-normative reference the doc wins over, `corpus.ts` rewrites the JSON, and the test fails on drift.
- `doc/concept/audio-jitter.md:833-838` — the paragraph claiming both players consume exact vectors that were not generated from the implementation under test.
## Required outcome
- The decision and filter vectors, and the `allowance` fields in `corpus.json`, are produced by a reference of the documented rule that neither player imports, and a checked-in test fails when the committed files no longer match it — the same guarantee `corpus.test.ts` gives the estimator cases.
## Suggested approach (optional)
- A `reference.ts` beside the corpora implementing only `decide` and `Level`, a `corpus.ts` that writes `decisions.json` and refreshes `corpus.json`'s `decision`/`allowance`, and a `corpus.test.ts` that recomputes and compares, wired into `just check`.
- Leave the PCM `output` arrays out of the generator: the expected seams come from the shared pitch search, which is the thing being graded.
## Open questions for the human (optional)
- If the vectors were in fact produced by a generator that stayed out of the tree, committing that generator may be all this needs.deepseek-v4.1-flash-1017 (free via Pullfrog for OSS) | 𝕏
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # quest/m1/audio-jitter-target/calibrate.md # test/audio-quality/budgets.json
There was a problem hiding this comment.
ℹ️ No new issues in this delta — the new commits are quest bookkeeping only, so approval is withheld solely for the prior review's still-open suggestion about the stretch corpora (
decisions.json/corpus.jsonhave no generator or drift test); the code it concerns is untouched by these commits.
Reviewed changes
- Recorded the maintainer's 2026-11-11 merge pick in
quest/m1/audio-jitter-target/calibrate.md, including how to read a native-opus near-zero tap-underrun breach. - Turned the four follow-ups into quests —
clock-step.md,hole-wait.md,render-callback.md, andspeech-replay.md— each citing the measurement that motivates it.
deepseek-v4.1-flash-1017 (free via Pullfrog for OSS) | 𝕏
There was a problem hiding this comment.
ℹ️ No new issues in this delta — the merge only brings main's already-merged AAC budget re-measure and quest bookkeeping; no source changed.
Reviewed changes
- Merged
main's AAC re-measure:284c9771f(#8309) brings the six AAC browser rows' budgets re-measured after the TS importer grouped each PES, and deletes the now-completedcapped-budgetsquest, leavingcalibrate.mdwithout itsRelatedlink. - Recorded the merge pick and follow-ups:
96c641edrecords the 2026-11-11 merge decision and turns the four follow-ups intoclock-step.md,hole-wait.md,render-callback.md, andspeech-replay.md. - Merge commit:
52ae44755is a clean merge ofmain; its three-file delta is exactly284c9771f's changes, so no source file was touched.
The prior review's open concern — that doc/concept/audio-jitter/stretch/corpus.json and decisions.json are hand-edited with no generator or drift test — remains open and is not addressed by this merge.
deepseek-v4.1-flash-1017 (free via Pullfrog for OSS) | 𝕏

Closes #8031. Finishes and deletes
quest/m1/audio-jitter-target/stretch.md.Decisions
auto, not for a fixed delay lowered at runtime past its bound.Problem
Buffered playback lost lip sync and cut audio out on a live stream (moq-dev/moq#8031), and the two players disagreed on how to keep audio near its target:
Approach
One decision rule, NetEq's
DecisionLogic, runs in both players on the same inputs, and a shared corpus (doc/concept/audio-jitter/stretch/decisions.json) pins it in both test suites.BufferLevelFilter, once per 30 ms window, less what the stretch changed), the raw level, the lag behind the shared clock, and the bound.low = max(3/4 target, target - 85 ms)andhigh = max(target, low + 20 ms), withbuffer = min(filtered, raw).buffer >= 4 high: fast accelerate, even in cooldown.buffer >= high: accelerate.filtered < lowand not past the bound: expand.moq playanchors the sink with the newSink::due(at, late). Without a clock, the lag is the level less the target.auto, and the lookahead in buffered mode.AudioRingBuffer.parkand the skip on a lowered target.moq play's write-path re-anchor (fit,AUDIO_SLACK).Impact
moq_audio::Sink::due(at, late, jumped)anchors the audio written so far to the caller's clock;jumpedis how far the clock's reference moved earlier since the last call. No JS export changes: the ring, the worklet messages, and the decode gate are internal to@moq/watch. No migration note.auto) on a missing group, as native does.Consumer::read_attakes the frames left in the device callback, so a refill after an underrun waits for the whole callback rather than the mixer's 1024-frame slice.Instantandhost(), each behind anexpect, as the workspace'sclippy.tomlasks.Mixer::fillwrites the device's sample format itself, so the driver's staging buffer and its loop are gone.Measurements
Replays, deterministic, old ceiling to new (
test/audio-quality/budgets.json, note dated 2026-11-11):Live rows, this branch at f55a89d (the head before the last review fix, which only lowers how much a pending jump may skip), local host (x86_64 Linux, seed 7, 60 s a row, load average about 43, report only; the browser and native ceilings stay with the calibrate quest). Underruns are the tap's:
baseLatency42.7 ms), and the clock is paired withcurrentTimejust after each callback, so the ring trails by up to one callback.Benchmarks:
just rs bench-playback: stretch on costs 1 to 4% more per callback thanorigin/main, for example 891.6 against 864.0 µs at 64 sinks of 2048 frames, and 1022.8 against 1009.2 ns at one sink of 128. Stretch off is unchanged within 2%.bun watch/bench/ring.ts: per-callback CPU at 1, 4 and 16 players is 148, 50 and 39 µs onorigin/main. On this branch it is 122 to 172, 63 to 76 and 35 to 53 µs over two runs, within the bench's round-to-round spread (main's own rounds ranged 132 to 161 and 42 to 67 µs). Zero underruns throughout.Refill after an underrun, measured both ways as the quest asked (browser replays): the cushion beats or ties refilling to the whole target on every row. bursty-aac stalled 0.005 against 0.009, silence 0.007 against 0.01, and lag p95 162 against 224 ms. local-aac silence 0.001 against 0.004, and recovery 9.8 against 13.5 s.
Validation
quest gate just check: passes.quest gate just check: passes at a16e156, after mergingorigin/main(itsdecoder.arrivals()removal: the replay counts delivered frames instead).origin/mainat 284c977 (moq-dev/moq#8309's AAC re-measure): the budgets note keeps main's text plus this branch's 2026-11-11 paragraphs;just test audio-quality --replays --enforcewithin budget andquest checkpasses at 52ae447.just test audio-quality --replays --enforce: within budget at a16e156, re-measured for the reference skip and each review fix.i16callback read where it starts (2640 accelerated frames without it).a_reference_jump_skips_as_much, and its JS pair), andAudioBuffersends a jump and takes it whole in its offset filter. The 1% fast publisher replay keeps its lag under 23 ms, which skipping each small jump broke (60 ms).Choices made without asking
delay + lookahead, the replays measured 1651.8 tap underruns a minute on relay-bbb and 2123.6 on relay-mic-stall, because the age skip dropped what the stretch would have caught up on.delay + lookaheaduntil its first frame was tried and reverted. relay-mic-rtt40's first frame arrives alone before its backlog, so it changed nothing.moq playderives the reference from consecutive anchors (at - end - delay) rather than exposing it frommoq_mux::sync, so no new mux API.AtomicU64, so the device thread never pairs one call's clock with another's bound.park.Alternatives
delay + lookahead: measured above, rejected.Follow-ups
delay + lookaheadfor holes, the max for aged media, in JS and Rust.Caused by
No single PR: the browser's trim-and-park ring and native's raw-depth expand predate the stretch model.
Models
(Written by Claude Opus 5.5 with medium effort; iterate round by Claude Opus 5.5 with low effort)
🤖 Generated with Claude Code
Review
Rewritten by
quest reviewafter each round.01a1cc4e-ada3-7591-9202-8be661e62954, round 7Incremental review found no new issues in the follow-up notes or merge resolution, and budget changes match those inherited from main; builds and tests were not run.