Skip to content

Audio slows down and speeds up around its target in every player - #8507

Merged
kixelated merged 17 commits into
mainfrom
quest/m1/audio-jitter-target/stretch
Nov 12, 2026
Merged

kixelated merged 17 commits into
mainfrom
quest/m1/audio-jitter-target/stretch

Conversation

@kixelated

@kixelated kixelated commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #8031. Finishes and deletes quest/m1/audio-jitter-target/stretch.md.

Decisions

  • Design change: ✅ 1A drop the retarget allowance (rejected: 1B carry the delay with the clock, 1C merge as is). The bound alone decides a lag skip in both players. "Nothing skips buffered audio on a fall" holds by the bound in auto, not for a fixed delay lowered at runtime past its bound.
  • Design change: ✅ 2A audio follows the reference's skip-ahead (rejected: 2B hold the consumer until the clock settles, 2C merge as is). When the shared clock's reference moves earlier, the ring skips as much, while target falls and stalls still stretch. Fixes the tune-in lag, not relay-mic-rtt40's raised target, which stays with the hole-wait follow-up.
  • Merge pick 2026-11-11 (maintainer, /quest-merge): ✅ merge now, treating native-opus near-zero's 64 to 77 tap underruns a minute (ceiling 42, host load about 43) as host noise for the calibrate quest's nightly runs to settle (rejected: re-measure on a quiet host first; investigate the review fixes first). The calibrate Plan names that row to watch.
  • Follow-ups 2026-11-11 (maintainer): ✅ keep all four as quests.

Problem

Buffered playback lost lip sync and cut audio out on a live stream (moq-dev/moq#8031), and the two players disagreed on how to keep audio near its target:

  • The browser never slowed down. After an underrun it stayed silent until the whole target refilled, it parked on a target rise, it turned the stretch off in buffered mode, and it trimmed whatever trailed the clock, so a stall cost the audio that arrived late.
  • Native expanded whenever its raw depth was below the target, which is true between every arrival, so audio settled 60 to 88 ms behind. It had neither NetEq's level filter nor its low limit.

Approach

One decision rule, NetEq's DecisionLogic, runs in both players on the same inputs, and a shared corpus (doc/concept/audio-jitter/stretch/decisions.json) pins it in both test suites.

  • Inputs, in frames: the target, the filtered level (NetEq's BufferLevelFilter, once per 30 ms window, less what the stretch changed), the raw level, the lag behind the shared clock, and the bound.
  • Limits: low = max(3/4 target, target - 85 ms) and high = max(target, low + 20 ms), with buffer = min(filtered, raw).
  • Rule, in order:
    1. Behind with buffer >= 4 high: fast accelerate, even in cooldown.
    2. In cooldown: hold.
    3. Behind with buffer >= high: accelerate.
    4. filtered < low and not past the bound: expand.
    5. Ahead of the clock: expand.
    6. Otherwise hold.
  • Clock:
    • The browser posts where the shared clock stands against the context's time, and the decoder reports what arrived, encoded or not.
    • Natively, moq play anchors the sink with the new Sink::due(at, late). Without a clock, the lag is the level less the target.
  • Skip:
    • Only on a read whose lag is past the bound plus one window, and only back to the bound.
    • The bound is the subscription's max delay less the delay: 2 s less the delay in auto, and the lookahead in buffered mode.
    • Until the player has read once on the clock, the bound is zero, so a join lands on the clock instead of playing out the backlog at the stretch's pace.
    • When the shared clock's reference moves earlier, as through a tune-in backlog, audio skips as much, as far as it trails the clock and holds audio past its target (which the early media that moved the reference leaves it), once those jumps add up to more than one window. Video skips the same frames. A target fall and a stall still stretch, and a jump past what has arrived skips the rest once it lands.
  • Underrun: playback resumes on a cushion of one window and one read, capped at the floor, then slows down until the buffer is back at the target.
  • Declared silence: a playhead behind the clock skips a declared silence by its lag rather than stretching across it.
  • Browser decode gate: once playing, the gate releases a frame while what it was given, landed or still decoding, reaches less than the floor plus one window past the playhead.
  • Delivery: the browser's container consumer waits and skips at the subscription's max delay, as native already did, so a stall's late audio is delivered and caught up on.
  • Removed:
    • AudioRingBuffer.park and the skip on a lowered target.
    • The browser's 20 ms fast accelerate.
    • moq play's write-path re-anchor (fit, AUDIO_SLACK).

Impact

  • Public API: additive. moq_audio::Sink::due(at, late, jumped) anchors the audio written so far to the caller's clock; jumped is how far the clock's reference moved earlier since the last call. No JS export changes: the ring, the worklet messages, and the decode gate are internal to @moq/watch. No migration note.
  • Behavior:
    • Audio trails the clock through a stall and catches up by playing faster, instead of trimming.
    • A target change is followed by stretching, not parking or skipping, except a fixed delay lowered at runtime past its bound, which skips to the bound.
    • A reference that moves earlier skips audio by as much, as video does.
    • The browser waits up to the subscription's max delay (2 s in auto) on a missing group, as native does.
  • Wire: none.
  • Native internals:
    • Consumer::read_at takes the frames left in the device callback, so a refill after an underrun waits for the whole callback rather than the mixer's 1024-frame slice.
    • The ring reads the host clock through one aliased Instant and host(), each behind an expect, as the workspace's clippy.toml asks.
    • The mixer, the sink's pending hand-off, and the bus's live queue hold the consumer boxed, as the producer's resampler already is.
    • Mixer::fill writes the device's sample format itself, so the driver's staging buffer and its loop are gone.

Measurements

Replays, deterministic, old ceiling to new (test/audio-quality/budgets.json, note dated 2026-11-11):

row player underruns/min trimmed ms/min stalled silence A/V lag p50 A/V lag p95 target p95
bursty-aac 4.4 → 4.4 0 → 0 0.009 → 0.005 0.012 → 0.007 -22.4 → 10.4 -22.4 → 162.1 343.2 → 343.2
local-aac 12.1 → 4 0 → 0 0.017 → 0 0.021 → 0.002 -22.2 → 1.8 -4.2 → 39.2 203.2 → 203.2
relay-bbb 16.1 → 0 0 → 0 0.025 → 0 0.02 → 0 -22.6 → -1.3 -4.2 → 2.1 223.2 → 223.2
relay-mic 6.1 → 0 0 → 0 0 → 0 0.002 → 0 -19 → 0.3 -18.5 → 0.3 80 → 80
relay-mic-rtt40 10.6 → 2.7 0 → 0 0.003 → 0 0.004 → 0 -20.1 → 0.4 -13.4 → 1.9 320 → 780
relay-mic-stall 6.1 → 4 2152.5 → 0 0.067 → 0.05 0.076 → 0.049 -19.5 → 25.3 -9.5 → 501.8 1540 → 1540
  • Lag p50: audio now rests on the clock, not one frame ahead of it.
  • Lag p95: the tune-in backlog's reference jumps are now skipped before the graded window, so the p95 sits within a frame or two of the clock on relay-bbb and relay-mic. Where it still rises, audio fell behind through a stall or a target fall and is caught up by speeding up rather than trimmed. On the replay's constant tone that pace is one 2.5 ms period a window, slower than on speech.
  • relay-mic-stall: its p50 of 25.3 ms is the catch-up after its target falls from 1540 to 80 ms.
  • bursty-aac: rests 5 ms behind its clock, against about 1 ms on the Opus rows.
  • Tap underruns: the tap files 2 to 3.3 short quanta a minute on local-aac and bursty-aac, under 0.2 ms of missing audio.
  • relay-mic-rtt40's target: its consumer waits up to the subscription's max on the reordered join backlog's holes, which the estimator counts as jitter. Left to the hole-wait follow-up.

Live rows, this branch at f55a89d (the head before the last review fix, which only lowers how much a pending jump may skip), local host (x86_64 Linux, seed 7, 60 s a row, load average about 43, report only; the browser and native ceilings stay with the calibrate quest). Underruns are the tap's:

row underruns/min trimmed ms/min A/V lag p50 A/V lag p95 target p95
chromium-aac near-zero / mild / wide / bursty 0 / 0 / 0 / 0 0 29.1 / 28.4 / 39.5 / 58.6 53.2 / 50.1 / 65.6 / 84.3 203 / 186 / 283 / 363
chromium-opus near-zero / mild / wide / bursty 2.2 / 152.8 / 0 / 0 0 34.2 / 44.7 / 43.2 / 34.7 56.5 / 853.3 / 71.8 / 55.3 80 / 320 / 380 / 180
chromium-aac / opus step 0 / 6.5 0 35.2 / 41.8 57.6 / 65.1 223 / 100
chromium-aac / opus congestion 0 / 0 611.4 / 461.7 44.3 / 52.5 1098.3 / 1057.2 663 / 720
native-aac near-zero / mild / wide / bursty 0 / void / 2.2 / 0 0 / void / 0 / 0 -12.8 / void / 23.5 / 42.6 -7 / void / 32.6 / 51.4 186 / void / 283 / 323
native-opus near-zero / mild / wide / bursty 63.8 / 4.3 / 0 / 32.6 81.2 / 78.8 / 0 / 0 -5.1 / -6.2 / 10.3 / -0.7 6.1 / 0.5 / 22.7 / 23.7 100 / 100 / 180 / 160
native-aac / opus step 29.3 / 17.4 0 5.7 / 17.3 37.5 / 42 223 / 100
native-aac / opus congestion 911.1 / 918.7 461 / 364.2 -6 / -5 972.6 / 966.5 663 / 860
  • Tune-in: no browser row carries the tune-in backlog's lag any more. chromium-opus wide's p95 was 341.7 ms before this round (opus mild's 1133 ms the run before that), 71.8 now.
  • Host noise: native-aac mild and fixed-250 voided on a device clock that ran at 0.94x and 0.979x, and chromium-opus mild met a 405 ms/min arrival stall (skips and trimmed both 0). A rerun of the native near-zero, mild and fixed-250 rows on the same head skipped nothing and trimmed nothing on any of the six; native-opus near-zero still filed 77.1 tap underruns a minute (ceiling 42) with no skip, where the two runs before this round's review fixes filed 0. Whether that is the host's load or this branch is unresolved.
  • Native skips: the run before the surplus rule skipped 4.3 and 5.4 times a minute on native-aac near-zero and wide, where audio sat about 30 ms behind its clock on a low buffer and the reference's jitter moves piled up to a window. A jump now counts only as far as audio is held past the target, and the rerun above has no native skip.
  • Step rows, for the deferred clock-step decision: lag p50 35.2 and 41.8 ms in the browser and 5.7 and 17.3 ms natively, p95 37.5 to 65.1 ms. The underruns of moq-dev/moq#8361's native AAC step (614 a minute) stay down, at 29.3.
  • Underruns: native mild, wide, bursty and step, and chromium-opus bursty, file 13 to 29.4 tap underruns a minute with no skip in the graded window; the same native rows read 0 to 32.6 in the run before this change. All are under their ceilings.
  • Congestion rows gate audio dropped, not audio late (the bufferbloat quest). Trimmed audio is under the old ceilings (2300, 2400, 3000, 2500 ms/min). Browser underruns fell from the ceilings' 270 and 280 a minute to 0.
  • Report-only breaches besides the noise above, unchanged from the runs before: the browser's clock-pairing trail below (every non-congestion browser row's lag over its provisional ceiling), the browser congestion lags, and native-aac congestion's p95 (972.6 against 570).
  • Browser rest:
    • Where it rests: the browser's near-zero, mild and fixed-250 rows rest 28 to 35 ms behind the clock, where native's near-zero and mild rows rest 4 to 13 ms ahead of it. Headless Chromium renders 2048-frame callbacks (baseLatency 42.7 ms), and the clock is paired with currentTime just after each callback, so the ring trails by up to one callback.
    • Pairing on the wall clock instead: a max-filtered pairing put audio on the clock (p50 -1.3 to 4.5 ms on near-zero and fixed-250). It also underran near-zero Opus 45 times a minute, since the target does not count the 42.7 ms a callback pulls at once, so this PR keeps the pairing and documents it.

Benchmarks:

  • just rs bench-playback: stretch on costs 1 to 4% more per callback than origin/main, for example 891.6 against 864.0 µs at 64 sinks of 2048 frames, and 1022.8 against 1009.2 ns at one sink of 128. Stretch off is unchanged within 2%.
  • bun watch/bench/ring.ts: per-callback CPU at 1, 4 and 16 players is 148, 50 and 39 µs on origin/main. On this branch it is 122 to 172, 63 to 76 and 35 to 53 µs over two runs, within the bench's round-to-round spread (main's own rounds ranged 132 to 161 and 42 to 67 µs). Zero underruns throughout.

Refill after an underrun, measured both ways as the quest asked (browser replays): the cushion beats or ties refilling to the whole target on every row. bursty-aac stalled 0.005 against 0.009, silence 0.007 against 0.01, and lag p95 162 against 224 ms. local-aac silence 0.001 against 0.004, and recovery 9.8 against 13.5 s.

Validation

  • quest gate just check: passes.
  • quest gate just check: passes at a16e156, after merging origin/main (its decoder.arrivals() removal: the replay counts delivered frames instead).
  • Merged origin/main at 284c977 (moq-dev/moq#8309's AAC re-measure): the budgets note keeps main's text plus this branch's 2026-11-11 paragraphs; just test audio-quality --replays --enforce within budget and quest check passes at 52ae447.
  • just test audio-quality --replays --enforce: within budget at a16e156, re-measured for the reference skip and each review fix.
  • Both live runtimes at 60 s a row at 3ca238c, 08ddd85 and f55a89d, and the native near-zero, mild and fixed-250 rows again at f55a89d: report only, the last numbers above.
  • Each review fix has a unit test that fails without it: the decoder's held-audio level and each chunk of a 4096-frame i16 callback read where it starts (2640 accelerated frames without it).
  • Review round fixes, each with a test that fails without it, in both languages where the code is in both: a jump past what has arrived skips the rest once it lands, also when the bound skip goes first; a truncate rebases the received horizon to the cut (JS); a native refill waits for the whole device callback (Rust); a pending jump never skips below the target, nor counts while the buffer holds nothing past it.
  • The reference skip: both rings skip a reference jump to the clock and catch up on the same lag from a fallen target (a_reference_jump_skips_as_much, and its JS pair), and AudioBuffer sends a jump and takes it whole in its offset filter. The 1% fast publisher replay keeps its lag under 23 ms, which skipping each small jump broke (60 ms).

Choices made without asking

  • One gate rule: the browser's decode gate keeps one fill rule once playing, which also releases the frame past a hole, rather than a separate hole clause.
  • The join rule: bound zero until the first read on the clock, in both players.
  • The cushion cap: one window and one read, capped at the floor in the browser and at the target natively, so a small floor never refills past itself.
  • Skip count: a capacity re-anchor counts as a skip only when it dropped audio.
  • Consumer delay: the browser consumer's local max delay is the subscription's. At delay + lookahead, the replays measured 1651.8 tap underruns a minute on relay-bbb and 2123.6 on relay-mic-stall, because the age skip dropped what the stretch would have caught up on.
  • A join-time max: holding the consumer at delay + lookahead until its first frame was tried and reverted. relay-mic-rtt40's first frame arrives alone before its backlog, so it changed nothing.
  • Clock pairing: kept as is, as measured above.
  • Jumps skip past one window: the reference skip waits until jumps add up to more than one 30 ms window, capped at the lag, so a fast publisher clock's small steps stay the stretch's, the same window the bound skip allows.
  • Jumps skip only the surplus: pending jumps are capped at the audio held past the target, so a reference moving on jitter while audio trails on a low buffer never skips into an underrun. The early media that moves the reference is what makes the surplus in the tune-in case.
  • Native reference: moq play derives the reference from consecutive anchors (at - end - delay) rather than exposing it from moq_mux::sync, so no new mux API.
  • One anchor word: native packs the clock and the bound into one AtomicU64, so the device thread never pairs one call's clock with another's bound.
  • Per-chunk clock: the mixer reads the host clock once per device callback and dates each 1024-frame chunk by the frames before it, so a large callback does not read on-time audio as ahead of the clock.
  • Level counts encoded audio: the browser decoder reports the newest end its consumer holds as received, so the stretch level counts audio not yet decoded, as the replay already did.
  • Tests: the unit tests pinning the old rule were rewritten rather than worked around, and the park tests went with park.

Alternatives

  • Refill to the whole target: measured above, rejected.
  • A few percent both ways, or a capped slow-down: rejected in the quest's plan for NetEq's untuned rates.
  • Consumer at delay + lookahead: measured above, rejected.

Follow-ups

Caused by

No single PR: the browser's trim-and-park ring and native's raw-depth expand predate the stretch model.

Models

  • Size: XL
  • Implementer: Claude Code, Claude Opus 5.5, xhigh effort
  • Implementer: Claude Code, claude-opus-5-5, medium effort (iterate, M)
  • Implementer: Claude Code, claude-opus-5-5, low effort (iterate, S)

(Written by Claude Opus 5.5 with medium effort; iterate round by Claude Opus 5.5 with low effort)

🤖 Generated with Claude Code

Review

Rewritten by quest review after each round.

  • Reviewer: Codex, gpt-6.1-sol, xhigh effort
  • Session: 01a1cc4e-ada3-7591-9202-8be661e62954, round 7
  • Head: 52ae447
  • Base: origin/main at 284c977
  • Score: 10/10, Incremental review found no new issues in the follow-up notes or merge resolution, and budget changes match those inherited from main; builds and tests were not run.
  • Findings: none

kixelated and others added 7 commits November 11, 2026 01:23
moq play, a live <moq-watch>, and a buffered one decide each 30 ms window by
the same NetEq rule on a filtered level, the playhead's lag behind the shared
clock, and a skip bound from the subscription's max delay. Park, the
write-time band skip, and fit() are gone; underruns resume on a cushion and
slow down to rebuild. Shared decision and filter vectors pin both ports.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A join plays its first read onto the clock rather than stretching through the
backlog a subscription opens on, in the browser ring and natively. After an
underrun, playback resumes on a cushion of one window and one read, capped at
the floor. A capacity re-anchor counts as a skip only when it dropped audio.

The ring reads the host clock through one expected alias, and the mixer's
command channel carries the consumer boxed, as the producer's resampler is.

Replay budgets are re-measured with the stretch model, and the quest file and
its links are removed.

Closes #8031

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e it starts

- The browser decoder tells the ring what the consumer holds encoded, so the
  level the stretch steers on reaches past the decode gate, as the replay's
  already did.
- A target that falls moves the clock ahead by as much; both rings now catch
  that up by speeding up however small the bound, and skip only lateness past
  what of the fall is left.
- The mixer reads the host clock once per device callback and starts each
  1024-frame chunk at its own offset, so a later chunk is not read as ahead
  of the clock.
- The native clock and bound are published as one word, released, so the
  reader never pairs one anchor's clock with another's bound, and sees the
  target set before it.
- The bus boxes its live consumer, as the mixer's command does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…chunk

A target that rises after a fall moves the clock back, so it cancels the
allowance the fall added rather than leaving a skip deferred past a lag
that no longer exists. Both rings do this, with a test each that fails
under the add-only rule.

The native driver reads the host clock once per device callback and hands
each staging chunk its own start, so a callback larger than the staging
buffer no longer reads every chunk as if it started with the callback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mixer::fill takes the device's sample type and converts each chunk as it
copies it out, so the driver hands it the whole callback against one
clock reading. The driver's staging buffer and the loop over it, which
read every staging pass as if it started the callback, go away.

The mixer test now fills 4096-frame i16 callbacks, and fails with 2640
accelerated frames when every chunk reads the callback's start.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated and others added 2 commits November 11, 2026 11:54
…arget allowance

The bound alone decides a lag skip in both players, so the retarget
allowance and its tests go. When the shared clock's reference moves
earlier, audio skips as much, as far as it trails the clock, once the
jumps add up to one stretch window; target falls and stalls still
stretch. Replay budgets re-measured.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Design change: ✅ 1A drop the retarget allowance (rejected: 1B carry the delay with the clock, 1C merge as is).

The bound alone decides a lag skip in both players: retarget and its four tests are gone from js/watch/src/audio/ring-buffer.ts and rs/moq-audio/src/ring.rs, and with them the race quest review round 3 found. "Nothing skips buffered audio on a fall" now holds by the bound in auto (the max delay less the target always covers a fall of the target), not for a fixed delay lowered at runtime by more than its bound and a window, which skips to the bound as any lag past it does. doc/concept/audio-jitter.md says so.

(Written by Claude Opus 5.5 with medium effort)

@kixelated

Copy link
Copy Markdown
Collaborator Author

Design change: ✅ 2A audio follows the reference's skip-ahead (rejected: 2B hold the consumer until the clock settles, 2C merge as is).

When the shared clock's reference moves earlier, the ring skips as much, as far as the playhead trails the clock; target falls and stalls still stretch. Jumps skip once they add up to more than one 30 ms stretch window, so a publisher clock drifting fast, which moves the reference a little at a time, stays the stretch's to catch up (the 1% fast replay test caught per-jump skips raising its lag from under 23 to 60 ms). Browser: AudioBuffer reads sync.out.reference with each clock reading and sends the drop as jumped on the clock message, and its offset filter takes a jump whole. Native: moq play derives the reference from each anchor (at - end - delay) and passes the drop as a new third argument, Sink::due(at, late, jumped). This fixes the tune-in lag, not relay-mic-rtt40's raised target, which stays with the hole-wait follow-up.

(Written by Claude Opus 5.5 with medium effort)

kixelated and others added 4 commits November 11, 2026 12:51
…efill for the whole callback

A reference jump past what is queued keeps its remainder and skips it
once the audio lands. A truncate rebases the received horizon to the
cut, so the predecessor's retired audio never reads as depth. Native
refills after an underrun until what is left of the device callback is
queued, not the 1024-frame slice the mixer reads it in.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both rings skip the larger of the lag past the bound and the pending
jump, and the jump keeps whatever that skip did not cover.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the target

A player sitting behind its clock on a low buffer let the reference's
jitter moves pile up to a window and skip into an underrun (native-aac
near-zero and wide skipped 4.3 and 5.4 times a minute live). A jump now
counts only as far as the ring holds audio past its target, which the
early media that moved the reference leaves it. Native derives the jump
from the timestamp the clock placed, so rounding never reads as one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 11, 2026 23:17
@coderabbitai

coderabbitai Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 8 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 68d9f5b9-f970-4dd8-94e0-dc633f7b61fb
Commits

Reviewing files that changed from the base of the PR and between 284c977 and 52ae447.

Files selected for processing: 41

  • doc/concept/audio-jitter.md
  • doc/concept/audio-jitter/stretch/corpus.json
  • doc/concept/audio-jitter/stretch/decisions.json
  • doc/lib/js/watch.md
  • doc/lib/rs/moq-audio.md
  • js/watch/src/audio/buffer.test.ts
  • js/watch/src/audio/buffer.ts
  • js/watch/src/audio/decoder.test.ts
  • js/watch/src/audio/decoder.ts
  • js/watch/src/audio/render-worklet.test.ts
  • js/watch/src/audio/render-worklet.ts
  • js/watch/src/audio/render.ts
  • js/watch/src/audio/replay.test.ts
  • js/watch/src/audio/replay.ts
  • js/watch/src/audio/ring-buffer.test.ts
  • js/watch/src/audio/ring-buffer.ts
  • js/watch/src/audio/stretch.test.ts
  • js/watch/src/audio/stretch.ts
  • js/watch/src/element.ts
  • js/watch/src/sync.ts
  • quest/m1/audio-jitter-target/README.md
  • quest/m1/audio-jitter-target/calibrate.md
  • quest/m1/audio-jitter-target/clock-step.md
  • quest/m1/audio-jitter-target/hole-wait.md
  • quest/m1/audio-jitter-target/render-callback.md
  • quest/m1/audio-jitter-target/speech-replay.md
  • quest/m1/audio-jitter-target/stretch-setting.md
  • quest/m1/audio-jitter-target/stretch.md
  • quest/m4/audio-bufferbloat-av.md
  • rs/moq-audio/src/bus.rs
  • rs/moq-audio/src/playback/driver.rs
  • rs/moq-audio/src/playback/mixer.rs
  • rs/moq-audio/src/playback/sink.rs
  • rs/moq-audio/src/ring.rs
  • rs/moq-audio/src/stretch.rs
  • rs/moq-cli/src/play/fake.rs
  • rs/moq-cli/src/play/media.rs
  • rs/moq-cli/src/play/output.rs
  • rs/moq-cli/src/play/timeline.rs
  • test/audio-quality/README.md
  • test/audio-quality/budgets.json
Pre-merge checks: Inconclusive 2

Failed Failed checks: 2 inconclusive

Check name Status Explanation Resolution
Title check Pending This assessment has not completed. The check will be retried on the next pre-merge check run.
Description check Pending This assessment has not completed. The check will be retried on the next pre-merge check run.
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

kixelated and others added 2 commits November 11, 2026 15:18
…-target/stretch

# Conflicts:
#	js/watch/src/audio/decoder.test.ts
…ed its arrivals

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No critical issues — one process suggestion on the new corpora.

Reviewed changes

  • One decision rule in both players: decide and Level in rs/moq-audio/src/stretch.rs and js/watch/src/audio/stretch.ts now implement NetEq's DecisionLogic and BufferLevelFilter on the same five inputs (target, level, raw, lag, bound), and every seam is capped by the decision's allowance.
  • Native clock anchor: Sink::due(at, late, jumped) carries the caller's instant onto the host clock, and ring::Producer::due packs the clock and bound into one AtomicU64 while accumulating reference jumps.
  • Native ring: ring::Consumer gains played/level/now/demand/jumped/joined, read_at(output, now, demand), a cushion refill of one window plus the device callback, and a skip() that jumps past the bound and past reference jumps only.
  • Mixer and driver: Mixer::fill converts to the device's sample format itself and dates each chunk from one host reading, so the driver's staging buffer and conversion loop are gone and sinks move boxed through the command channel.
  • Browser ring: AudioRingBuffer replaces park/peak/chunk/buffered with received/level/clock/jumped and Terms, skips by the clock's bound or a reference jump, resumes on a cushion, and skips a declared silence by its lag.
  • Browser decode gate: Backpressure drops parking and keeps a started ring's depth below its floor plus one window, counting frames still decoding and ignoring each frame's timestamp so a hole plays as silence.
  • Browser clock posting: AudioBuffer pairs sync.now() with AudioContext.currentTime behind a min-filtered offset, takes real jumps whole, and forwards the newest received end and reference deltas to the worklet.
  • CLI: moq play anchors the sink after each write from timing.end with at - end - delay as the reference, and fit/AUDIO_SLACK plus the write-path re-anchor are removed.
  • Corpora, tests, docs, budgets: the shared PCM corpus now carries decision/allowance, a new decisions.json adds decision and filter vectors, the JS unit and replay suites are rewritten, every browser and native ceiling is re-measured, and audio-jitter.md states the one rule.

ℹ️ The two stretch corpora have no generator and no drift test

doc/concept/audio-jitter.md now says these vectors cannot be derived from either player, but nothing enforces it: unlike the estimator corpus in the same directory (corpus.ts regenerates, corpus.test.ts fails when the checked-in files no longer match), stretch/corpus.json and stretch/decisions.json are hand-edited JSON that only the two test suites read. Nothing catches a future rule change that quietly moves the vectors to whatever the implementation under test produces, and the new allowance fields in corpus.json are exactly that kind of hand-derived value.

Technical details
# Add a reference implementation and drift test for the stretch corpora

## Affected sites
- `doc/concept/audio-jitter/stretch/decisions.json` — 23 decision and 9 filter vectors, read by `stretch.rs::tests::shared_decision_vectors` / `shared_filter_vectors` and `stretch.test.ts`; no script produces it.
- `doc/concept/audio-jitter/stretch/corpus.json` — `buffered`/`target`/`cooldown` were replaced by `decision`/`allowance`, which only the documented rule can produce; the expected PCM still legitimately needs the pitch search.
- `doc/concept/audio-jitter/corpus.ts` + `corpus.test.ts` — the pattern to copy: `reference.ts` is a non-normative reference the doc wins over, `corpus.ts` rewrites the JSON, and the test fails on drift.
- `doc/concept/audio-jitter.md:833-838` — the paragraph claiming both players consume exact vectors that were not generated from the implementation under test.

## Required outcome
- The decision and filter vectors, and the `allowance` fields in `corpus.json`, are produced by a reference of the documented rule that neither player imports, and a checked-in test fails when the committed files no longer match it — the same guarantee `corpus.test.ts` gives the estimator cases.

## Suggested approach (optional)
- A `reference.ts` beside the corpora implementing only `decide` and `Level`, a `corpus.ts` that writes `decisions.json` and refreshes `corpus.json`'s `decision`/`allowance`, and a `corpus.test.ts` that recomputes and compares, wired into `just check`.
- Leave the PCM `output` arrays out of the generator: the expected seams come from the shared pitch search, which is the thing being graded.

## Open questions for the human (optional)
- If the vectors were in fact produced by a generator that stayed out of the tree, committing that generator may be all this needs.

Pullfrog  | Fix it ➔ | View workflow run | Using deepseek-v4.1-flash-1017 (free via Pullfrog for OSS) | 𝕏

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	quest/m1/audio-jitter-target/calibrate.md
#	test/audio-quality/budgets.json

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No new issues in this delta — the new commits are quest bookkeeping only, so approval is withheld solely for the prior review's still-open suggestion about the stretch corpora (decisions.json/corpus.json have no generator or drift test); the code it concerns is untouched by these commits.

Reviewed changes

  • Recorded the maintainer's 2026-11-11 merge pick in quest/m1/audio-jitter-target/calibrate.md, including how to read a native-opus near-zero tap-underrun breach.
  • Turned the four follow-ups into quests — clock-step.md, hole-wait.md, render-callback.md, and speech-replay.md — each citing the measurement that motivates it.

Pullfrog  | Fix it ➔ | View workflow run | Using deepseek-v4.1-flash-1017 (free via Pullfrog for OSS) | 𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No new issues in this delta — the merge only brings main's already-merged AAC budget re-measure and quest bookkeeping; no source changed.

Reviewed changes

  • Merged main's AAC re-measure: 284c9771f (#8309) brings the six AAC browser rows' budgets re-measured after the TS importer grouped each PES, and deletes the now-completed capped-budgets quest, leaving calibrate.md without its Related link.
  • Recorded the merge pick and follow-ups: 96c641ed records the 2026-11-11 merge decision and turns the four follow-ups into clock-step.md, hole-wait.md, render-callback.md, and speech-replay.md.
  • Merge commit: 52ae44755 is a clean merge of main; its three-file delta is exactly 284c9771f's changes, so no source file was touched.

The prior review's open concern — that doc/concept/audio-jitter/stretch/corpus.json and decisions.json are hand-edited with no generator or drift test — remains open and is not addressed by this merge.

Pullfrog  | Fix it ➔ | View workflow run | Using deepseek-v4.1-flash-1017 (free via Pullfrog for OSS) | 𝕏

@kixelated
kixelated added this pull request to the merge queue Nov 12, 2026
Merged via the queue into main with commit 4262a06 Nov 12, 2026
7 checks passed
@kixelated
kixelated deleted the quest/m1/audio-jitter-target/stretch branch November 12, 2026 00:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

JS Watch: Configuring a buffer of more than 0 causes audio sync and play issue

1 participant