From 7bc1b5e244717849c76de29be43ad8cd672a4485 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 9 Nov 2026 02:57:00 -0800 Subject: [PATCH 1/5] chore(quest): claim quest/m1/push-to-talk-latency.md Quest-Claim: 6d36a39ba87e671f From 34b94075b41416d74304bed356afbeb4ea273de2 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 9 Nov 2026 03:13:32 -0800 Subject: [PATCH 2/5] test(audio-quality): measure push-to-talk on the browser defaults Drop the fixed 300 ms delay and 60 ms / 128 kbit/s Opus from the page, so just test push-to-talk measures what a listener gets: 20 ms Opus and delay "auto". Report latency.onset, the press to the render onset, which the press-to-audible budget will gate once it is measured on kixtop. Co-Authored-By: Claude Opus 5.5 --- quest/m1/push-to-talk-latency.md | 35 ++++++++++++++++++-- test/audio-quality/README.md | 11 +++--- test/audio-quality/clients/js/ptt-analyze.ts | 4 ++- test/audio-quality/clients/js/src/ptt.ts | 2 -- test/justfile | 2 +- 5 files changed, 42 insertions(+), 12 deletions(-) diff --git a/quest/m1/push-to-talk-latency.md b/quest/m1/push-to-talk-latency.md index 0e08084622d..aef0c430d57 100644 --- a/quest/m1/push-to-talk-latency.md +++ b/quest/m1/push-to-talk-latency.md @@ -47,9 +47,21 @@ head was not decoded"): 2 of 22 on the default 20 ms Opus and 1 on 60 ms. The group after a head could overtake it, and the read took the mute's timestamp jump as proof the head was too late to wait for; [moq-dev/moq#8123](https://github.com/moq-dev/moq/pull/8123) fixed it, and two -20 ms runs after the fix kept all 44 heads. What remains is to switch the -page to the defaults, measure on `kixtop` again, and set and enforce the -budget. +20 ms runs after the fix kept all 44 heads. +[moq-dev/moq#8364](https://github.com/moq-dev/moq/pull/8364) switches the page +to the defaults and reports `latency.onset`, the press to the render onset, +which the budget will gate. What remains is to measure on `kixtop` and set and +enforce the budget. + +Validated 2026-11-09 on the defaults with Watch's stretch, one run on the Linux +agent host (load average 3 to 5): every completeness check passed and all 44 +heads arrived, but the browser counted 596 underruns and 136 stalls, against +64 / 33 on 2026-11-05. The stretch only accelerates and never expands, so it +cannot cover a target set below the arrival spread; the premise of the +2026-11-06 decision does not hold. Press to render onset was 56 / 83 ms +(median / max over 15 separated presses), the `"auto"` target 40 to 60 ms, +`device` 114 / 120 ms. This run checks the harness only; the budget comes from +`kixtop`. Measured 2026-11-05 on the defaults, before Watch, one run of 22 presses each, Chromium 156 headless through a local relay. `kixtop` is a MacBook Air @@ -91,6 +103,23 @@ measured term (`js/watch/src/audio/latency.ts`); device and render follow the watch AudioContext's `latencyHint` (`"interactive"` in `js/watch/src/audio/decoder.ts`), otherwise the platform. +## Waiting + +- Maintainer: free about 30 GB on `kixtop`, which had 193 MiB free on + 2026-11-09, too little for the relay build and Chromium. An agent does not + delete files on a personal laptop unasked. Check with `ssh kixtop df -h /`. + The harness's native listener needs PulseAudio, so the agent then runs the + browser half ad hoc there, as on 2026-11-05. +- Maintainer: decide how `just test push-to-talk` treats browser underruns on + the defaults, since the stretch does not bring them to zero: (a) gate on a + ceiling measured on `kixtop` and the nightly runner, as `budgets.json` does + for the browser rows, leaving zero to + [Audio jitter target](/quest/m1/audio-jitter-target/README.md) + (recommended: it keeps the latency budget enforceable now and records + today's player honestly); (b) keep the zero check, so the nightly step fails + until the jitter target covers push-to-talk's spread; (c) hold the fixed + 300 ms delay and budget latency only on the defaults' onset in a second row. + ## Related - [Browser ceilings](/quest/m1/audio-jitter-target/browser-ceilings.md) - `"auto"` underruns less by time-stretching the ring, so the run can hold the default delay; the stretch landed in [moq-dev/moq#8058](https://github.com/moq-dev/moq/pull/8058), and its nightly ceilings are re-measured there diff --git a/test/audio-quality/README.md b/test/audio-quality/README.md index 57a8072ad54..2f7d612acda 100644 --- a/test/audio-quality/README.md +++ b/test/audio-quality/README.md @@ -129,10 +129,11 @@ just test push-to-talk ``` This separate, roughly nine-minute run drives the shipped publish and watch elements in one Chromium -page, and the native `moq play` as a second listener. A constant tone loops through the fake microphone, encoded as 128 kbit/s Opus with 60 ms -frames. A release mutes the publisher, which sends nothing until the next press while its rendition -stays enabled, so the listener keeps its subscription and plays comfort noise in between. The -listener holds a fixed 300 ms delay. After warmup and a mute, it measures 22 presses, +page, and the native `moq play` as a second listener. Both elements keep their defaults: Opus with +20 ms frames, and `delay: "auto"`. A constant tone loops through the fake microphone. A release +mutes the publisher, which sends nothing until the next press while its rendition stays enabled, +so the listener keeps its subscription and plays comfort noise in between. After warmup and a +mute, it measures 22 presses, alternating two and three seconds: the first after the warmup's five-second mute, seven with no intervening wait, seven after five seconds, and seven after sixty seconds. Both elements reach a local relay directly over WebTransport, with no shaping, loss, or queueing, so every millisecond @@ -172,7 +173,7 @@ The latency goal is at most 300 ms of playout added on top of one-way delivery. press, `delivery` is the slowest of the utterance's frames that arrived before its onset, measured from that frame's first sample (its media offset after the press). `playout` is what remains of the render onset after the press and that delivery, and `output` is the projected device output after -that onset. `path` follows the head frame from the press to projected output in parts that sum to +that onset. `onset` is the press to the render onset, everything but the device's own output. `path` follows the head frame from the press to projected output in parts that sum to it: `capture` (worklet delivery and framing, after the media the frame still needed), `packetization` (that media, placed on the media clock by the newest chunk captured before the press), `encode` (codec and container write), `network` (write to arrival, through the relay), diff --git a/test/audio-quality/clients/js/ptt-analyze.ts b/test/audio-quality/clients/js/ptt-analyze.ts index ba00ffe7474..92e69fe9a98 100644 --- a/test/audio-quality/clients/js/ptt-analyze.ts +++ b/test/audio-quality/clients/js/ptt-analyze.ts @@ -118,6 +118,7 @@ const presses = result.presses firstDecoded: input[0] ? decoded.has(input[0].timestamp) : false, arrival: earliest ? earliest[0] - press.at : null, audible: audible ? audible.at - press.at : null, + onset: audible ? audible.rendered - press.at : null, delivery, spread, delay: filled ? (delayAt(filled.at) ?? null) : null, @@ -168,8 +169,9 @@ const nativePlayback = { }; const parts = ["capture", "packetization", "encode", "network", "decode", "jitter_buffer", "render", "device"] as const; const summary = { - // The goal: at most 300 ms of playout added on top of one-way delivery. latency: { + // Press to render onset: what the budget covers, everything but the device's own output. + onset: summarize(presses.map((press) => press.onset)), delivery: summarize(presses.map((press) => press.delivery)), playout: summarize(presses.map((press) => press.playout)), output: summarize(presses.map((press) => press.output)), diff --git a/test/audio-quality/clients/js/src/ptt.ts b/test/audio-quality/clients/js/src/ptt.ts index ba0ab94c536..bd20176c1bb 100644 --- a/test/audio-quality/clients/js/src/ptt.ts +++ b/test/audio-quality/clients/js/src/ptt.ts @@ -116,7 +116,6 @@ const publish = document.createElement("moq-publish") as MoqPublish; publish.url = required("relay"); publish.name = "ptt.hang"; publish.invisible = true; -publish.audio.codec.set({ mime: "opus", frameDuration: Milli(60), bitrate: 128_000 }); publish.source = "camera"; document.body.append(publish); @@ -128,7 +127,6 @@ const listener = new URL(required("relay")); listener.searchParams.set("role", "listener"); watch.url = listener.toString(); watch.name = publish.name; -watch.setAttribute("delay", "300ms"); watch.setAttribute("visible", "always"); watch.volume = 1; watch.muted = false; diff --git a/test/justfile b/test/justfile index d098610b5ff..e4a498103a6 100644 --- a/test/justfile +++ b/test/justfile @@ -165,7 +165,7 @@ native-play: audio-quality *args: ./audio-quality/run.sh "$@" -# Measure 22 push-to-talk presses at 300 ms on an ideal local path. +# Measure 22 push-to-talk presses on the browser defaults over an ideal local path. push-to-talk: ./audio-quality/ptt.sh From 1ab8bcb287549f63fdc07ba7772cb3e388678a12 Mon Sep 17 00:00:00 2001 From: Luke Curley <432854+kixelated@users.noreply.github.com> Date: Mon, 9 Nov 2026 17:19:32 -0800 Subject: [PATCH 3/5] test(audio-quality): gate push-to-talk's press to render onset at 140 ms Set from a kixtop run on the browser defaults (106 ms max x 1.25, rounded up), and move the analyzer's fixture to 20 ms Opus and a 60 ms delay. Co-Authored-By: Claude Opus 5.5 --- quest/m1/push-to-talk-latency.md | 49 +++++++++++------ test/audio-quality/README.md | 10 ++-- .../clients/js/ptt-analyze.test.ts | 52 ++++++++++++------- test/audio-quality/clients/js/ptt-analyze.ts | 8 +++ 4 files changed, 78 insertions(+), 41 deletions(-) diff --git a/quest/m1/push-to-talk-latency.md b/quest/m1/push-to-talk-latency.md index 1aa1052705c..26502c34b5d 100644 --- a/quest/m1/push-to-talk-latency.md +++ b/quest/m1/push-to-talk-latency.md @@ -57,8 +57,32 @@ timestamp jump as proof the head was too late to wait for; 20 ms runs after the fix kept all 44 heads. [moq-dev/moq#8364](https://github.com/moq-dev/moq/pull/8364) switches the page to the defaults and reports `latency.onset`, the press to the render onset, -which the budget will gate. What remains is to measure on `kixtop` and set and -enforce the budget. +and gates its max at 140 ms, set from the `kixtop` run below. What remains is +the underrun and stall ceiling that replaces `ptt-analyze.ts`'s zero gate. + +Measured 2026-11-09 on `kixtop` on the defaults with Watch's stretch, the +browser half only (macOS has no PulseAudio for the native listener, so the +driver ran without it), headless Chromium 156, load average 10 to 12 from +other agents' work. Every browser completeness check passed. Medians and maxes +in ms over the separated presses: + +| Part | `kixtop` | +| --- | --- | +| capture | 2.3 / 11 | +| packetization | 15 / 20 | +| encode | 0.7 / 0.9 | +| network | 2.9 / 4.1 | +| decode | 0.6 / 2.1 | +| jitter buffer | 40 / 60 | +| render | 15 / 30 | +| press to render onset | 78 / 106 | +| device | 261 / 262 | +| `"auto"` target | 60 (40 to 80) | +| underruns / stalls | 17 / 8 | + +The budget is that 106 ms max times 1.25, rounded up to two significant +figures, as `budgets.json` sizes its ceilings. `device` is Bluetooth +headphones, the laptop's default output that day, and the budget excludes it. Validated 2026-11-09 on the defaults with Watch's stretch, one run on the Linux agent host (load average 3 to 5): every completeness check passed and all 44 @@ -112,20 +136,13 @@ watch AudioContext's `latencyHint` (`"interactive"` in ## Waiting -- Maintainer: free about 30 GB on `kixtop`, which had 193 MiB free on - 2026-11-09, too little for the relay build and Chromium. An agent does not - delete files on a personal laptop unasked. Check with `ssh kixtop df -h /`. - The harness's native listener needs PulseAudio, so the agent then runs the - browser half ad hoc there, as on 2026-11-05. -- Maintainer: decide how `just test push-to-talk` treats browser underruns on - the defaults, since the stretch does not bring them to zero: (a) gate on a - ceiling measured on `kixtop` and the nightly runner, as `budgets.json` does - for the browser rows, leaving zero to - [Audio jitter target](/quest/m1/audio-jitter-target/README.md) - (recommended: it keeps the latency budget enforceable now and records - today's player honestly); (b) keep the zero check, so the nightly step fails - until the jitter target covers push-to-talk's spread; (c) hold the fixed - 300 ms delay and budget latency only on the defaults' onset in a second row. +- moq-dev/moq: the browser half of the + [stretch model](/quest/m1/audio-jitter-target/stretch.md) lands (decided + 2026-11-09, maintainer, /quest-complete on + [moq-dev/moq#8364](https://github.com/moq-dev/moq/pull/8364)). Then measure + browser underruns and stalls on `kixtop` and the nightly runner, and replace + `ptt-analyze.ts`'s zero gate with that ceiling. Check with + `quest unblocked quest/m1/audio-jitter-target/stretch.md` and its PR. ## Related diff --git a/test/audio-quality/README.md b/test/audio-quality/README.md index 2f7d612acda..34b2c5ada03 100644 --- a/test/audio-quality/README.md +++ b/test/audio-quality/README.md @@ -162,15 +162,15 @@ later onset can belong to the preceding utterance. Their audible latency is alwa separated presses have an identifiable onset. `summary.json` matches inputs, outputs, writes, and arrivals by media timestamp, so a late arrival -is still attributed to its utterance. The command checks pipeline completeness, not the epic's -latency verdict. It refuses missing encoder inputs, a head that never reaches the listener or never -decodes, submitted frames that never encode or reach the container, written frames that never reach +is still attributed to its utterance. The command checks pipeline completeness and one latency +budget: every separated press's `onset` stays within 140 ms, the worst measured on a MacBook Air +with macOS 26 times 1.25, rounded up to two significant figures. It refuses missing encoder inputs, +a head that never reaches the listener or never decodes, submitted frames that never encode or reach the container, written frames that never reach the listener, reopened capture, a new encoder (a probe or a rebuilt codec) or subscription, a mute that disables or drops the rendition, any underrun or stall across the presses, and separated utterances with no audible output. It also fails on a broken driver or invalid capture. -The latency goal is at most 300 ms of playout added on top of one-way delivery. For each separated -press, `delivery` is the slowest of the utterance's frames that arrived before its onset, measured +For each separated press, `delivery` is the slowest of the utterance's frames that arrived before its onset, measured from that frame's first sample (its media offset after the press). `playout` is what remains of the render onset after the press and that delivery, and `output` is the projected device output after that onset. `onset` is the press to the render onset, everything but the device's own output. `path` follows the head frame from the press to projected output in parts that sum to diff --git a/test/audio-quality/clients/js/ptt-analyze.test.ts b/test/audio-quality/clients/js/ptt-analyze.test.ts index 1917b0be4d0..cd89a92cb54 100644 --- a/test/audio-quality/clients/js/ptt-analyze.test.ts +++ b/test/audio-quality/clients/js/ptt-analyze.test.ts @@ -5,13 +5,15 @@ import { join } from "node:path"; import { fileURLToPath } from "node:url"; import type { Sample } from "./src/schema"; -const FRAME = 60; +const FRAME = 20; +/** The playout delay `"auto"` holds through the run. */ +const DELAY = 60; /** How far the native clock runs ahead of the page's, delivery included. */ const NATIVE = 5000; -// 22 complete presses on an ideal path: the media clock is the page clock, every 60 ms frame is +// 22 complete presses on an ideal path: the media clock is the page clock, every 20 ms frame is // encoded, written, received and decoded a millisecond apart, and each separated press renders -// once the 300 ms delay is decoded, with 100 ms of projected device output after it. +// once the 60 ms delay is decoded, with 100 ms of projected device output after it. function trace() { const result = { submitted: [] as { at: number; timestamp: number; duration: number }[], @@ -28,14 +30,14 @@ function trace() { playback: { underruns: 0, stalls: 0 }, errors: [] as string[], arrivals: [] as [number, number, number, number][], - delays: [{ at: 0, delay: 300 }], + delays: [{ at: 0, delay: DELAY }], }; for (let i = 0; i < 22; i++) { const press = 10_000 + i * 10_000; const release = press + 2000; - // The press lands 20 ms into a frame, so the head frame still needs 40 ms of media. + // The press lands 10 ms into a frame, so the head frame still needs 10 ms of media. result.presses.push({ at: press, on: true, capture: { at: press, end: press * 1000 } }); - const start = press - 20; + const start = press - 10; for (let media = start; media + FRAME <= release; media += FRAME) { const at = media + FRAME + 25; if (at >= release) break; @@ -48,8 +50,8 @@ function trace() { result.decoded.push({ at: at + 3, timestamp, duration: FRAME * 1000 }); } if (i === 0 || i >= 8) { - // The fifth frame completes the delay; the render onset follows its PCM by 10 ms. - const rendered = start + 5 * FRAME + 25 + 3 + 10; + // The third frame completes the delay; the render onset follows its PCM by 10 ms. + const rendered = start + DELAY + 25 + 3 + 10; result.audible.push({ at: rendered + 100, rendered, received: rendered + 5 }); } result.presses.push({ at: release, on: false }); @@ -106,29 +108,29 @@ test("a complete run passes, reports its latency, and splits each press exactly" const press = summary.presses[0]; expect(press.path).toEqual({ capture: 25, - packetization: 40, + packetization: 10, encode: 1, network: 1, decode: 1, - jitter_buffer: 4 * FRAME, + jitter_buffer: DELAY - FRAME, render: 10, device: 100, }); const parts = Object.values(press.path as Record).reduce((sum, part) => sum + part, 0); expect(parts).toBe(press.audible); - expect(summary.delay).toEqual({ min: 300, median: 300, max: 300 }); + expect(summary.delay).toEqual({ min: DELAY, median: DELAY, max: DELAY }); }); test("a target that rises before the delay fills is filled by the frame that reaches it", async () => { - // "auto" raises the target a frame as the press starts, so the sixth frame fills it. + // "auto" raises the target a frame as the press starts, so the fourth frame fills it. const result = trace(); - result.delays.push({ at: result.presses[0].at, delay: 300 + FRAME }); + result.delays.push({ at: result.presses[0].at, delay: DELAY + FRAME }); result.audible[0].rendered += FRAME; result.audible[0].at += FRAME; const { exitCode, summary } = await analyze(result); expect(exitCode).toBe(0); - expect(summary.presses[0].delay).toBe(300 + FRAME); - expect(summary.presses[0].path.jitter_buffer).toBe(5 * FRAME); + expect(summary.presses[0].delay).toBe(DELAY + FRAME); + expect(summary.presses[0].path.jitter_buffer).toBe(DELAY); expect(summary.presses[0].path.render).toBe(10); }); @@ -145,15 +147,15 @@ test("a frame held back by a capture burst widens its press's spread", async () }); test("the delay is filled by the first frame to reach it, not the newest before the onset", async () => { - // The sixth frame decodes just before the onset, adding to a ring already leaving its stall. + // The fourth frame decodes just before the onset, adding to a ring already leaving its stall. const result = trace(); - const sixth = (result.presses[0].at - 20 + 5 * FRAME) * 1000; - const late = result.decoded.find((frame) => frame.timestamp === sixth); - if (!late) throw new Error("no sixth frame"); + const fourth = (result.presses[0].at - 10 + DELAY) * 1000; + const late = result.decoded.find((frame) => frame.timestamp === fourth); + if (!late) throw new Error("no fourth frame"); late.at = result.audible[0].rendered - 1; const { exitCode, summary } = await analyze(result); expect(exitCode).toBe(0); - expect(summary.presses[0].path.jitter_buffer).toBe(4 * FRAME); + expect(summary.presses[0].path.jitter_buffer).toBe(DELAY - FRAME); expect(summary.presses[0].path.render).toBe(10); }); @@ -193,6 +195,16 @@ test("a mute that disables the rendition fails the run", async () => { expect(stderr).toContain("a mute disabled or dropped the rendition"); }); +test("a press past the onset budget fails the run", async () => { + const result = trace(); + // The first press renders at 88 ms; this one renders a millisecond past the budget. + result.audible[0].rendered += 53; + result.audible[0].at += 53; + const { exitCode, stderr } = await analyze(result); + expect(exitCode).toBe(1); + expect(stderr).toContain("press to render onset exceeded its 140 ms budget"); +}); + // The silence between presses is declared, so playback must cross it without counting it. test("an underrun or stall across the presses fails the run", async () => { for (const playback of [ diff --git a/test/audio-quality/clients/js/ptt-analyze.ts b/test/audio-quality/clients/js/ptt-analyze.ts index 92e69fe9a98..14527ed8ce0 100644 --- a/test/audio-quality/clients/js/ptt-analyze.ts +++ b/test/audio-quality/clients/js/ptt-analyze.ts @@ -206,7 +206,15 @@ console.log( JSON.stringify({ latency, path, delay, unencoded, unsent, unreceived, playback, native: summary.native }, null, 2), ); +// Press to render onset, the slowest separated press: kixtop's worst (106 ms on 2026-11-09, macOS +// 26, headless Chromium 156) times 1.25, rounded up to two significant figures, as budgets.json sizes +// its ceilings. +const ONSET_BUDGET_MS = 140; + const failures: string[] = []; +if (!latency.onset || latency.onset.max > ONSET_BUDGET_MS) { + failures.push(`press to render onset exceeded its ${ONSET_BUDGET_MS} ms budget`); +} if (presses.some((press, i) => (i === 0 || i >= 8) && press.audible === null)) { failures.push("a separated utterance never reached the output"); } From 4f9d0a77796985c9c600991bd0813f8da235ecf0 Mon Sep 17 00:00:00 2001 From: Luke Curley <432854+kixelated@users.noreply.github.com> Date: Mon, 9 Nov 2026 17:36:13 -0800 Subject: [PATCH 4/5] quest: offer landing push-to-talk with a provisional underrun ceiling Co-Authored-By: Claude Opus 5.5 --- quest/m1/push-to-talk-latency.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/quest/m1/push-to-talk-latency.md b/quest/m1/push-to-talk-latency.md index 26502c34b5d..1d0585b1112 100644 --- a/quest/m1/push-to-talk-latency.md +++ b/quest/m1/push-to-talk-latency.md @@ -143,6 +143,14 @@ watch AudioContext's `latencyHint` (`"interactive"` in browser underruns and stalls on `kixtop` and the nightly runner, and replace `ptt-analyze.ts`'s zero gate with that ceiling. Check with `quest unblocked quest/m1/audio-jitter-target/stretch.md` and its PR. +- Maintainer: whether to wait that long, since the model needs grading, + native A/V lag, and native backpressure first: (a) keep waiting, as + decided; (b) land with a provisional ceiling from today's player, measured + on `kixtop` (17 / 8) and one nightly run on the branch, and let + [calibrate](/quest/m1/audio-jitter-target/calibrate.md) re-measure it after + the model (recommended: it enforces the latency budget now and drops a zero + gate that [grading](/quest/m1/audio-jitter-target/grading.md) says pushes + the wrong way). ## Related From 5fec26ff8ae939826138856846b36aa53356b3dd Mon Sep 17 00:00:00 2001 From: Luke Curley <432854+kixelated@users.noreply.github.com> Date: Mon, 9 Nov 2026 18:13:29 -0800 Subject: [PATCH 5/5] test(audio-quality): report push-to-talk onset and underruns without gating Per the 2026-11-09 maintainer decisions: drop the 140 ms onset budget and the browser and native underrun/stall failures; keep completeness checks. This finishes the push-to-talk latency quest. Co-Authored-By: Claude Opus 5.5 --- quest/m1/README.md | 1 - quest/m1/audio-jitter-target/grading.md | 12 +- quest/m1/push-to-talk-latency.md | 158 ------------------ test/audio-quality/README.md | 14 +- .../clients/js/ptt-analyze.test.ts | 52 +++--- test/audio-quality/clients/js/ptt-analyze.ts | 20 +-- 6 files changed, 37 insertions(+), 220 deletions(-) delete mode 100644 quest/m1/push-to-talk-latency.md diff --git a/quest/m1/README.md b/quest/m1/README.md index 9ed7f402118..140271b3f66 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -40,7 +40,6 @@ moves what remains of them once their claims merge. - [Stream credit by priority](/quest/m1/stream-credit/README.md) - every stream opens through one priority entry point, and a higher-priority group takes over or resets a lower one when credit runs out - [Audio jitter target](/quest/m1/audio-jitter-target/README.md) - the playout target is default in both languages; audio that slows down and catches up around it in every player, graded in that direction, and a native trace replay remain - [Buffered playback subscribes in ascending order](/quest/m1/buffered-ascending.md) - a buffered `` asks for the oldest group first, so a voice reply's opening arrives first, and audio's head hold shrinks to the floor -- [Push-to-talk's press-to-audible latency fits a budget on the defaults](/quest/m1/push-to-talk-latency.md) - press to audible on the browser defaults, excluding device output, stays under a kixtop-measured budget that `just test push-to-talk` enforces - [Lifecycles are explicit state](/quest/m1/explicit-state/README.md) - every lifecycle the 2026-11-07 audit found implicit is one enum or union, fixing the bugs in the 11 lifecycles it redesigns, with review and agent rules against new ones - [Track drop runs are forgotten below the floor](/quest/m1/drop-runs-prune.md) - a track's dropped-group record stays proportional to its window in Rust and JS - [A macOS camera delivers frames the renderer imports](/quest/m1/camera-nv12-macos.md) - AVFoundation capture asks for NV12, so a camera whose native format is `2vuy` reaches the renderer diff --git a/quest/m1/audio-jitter-target/grading.md b/quest/m1/audio-jitter-target/grading.md index 07c7f60a16c..6b664f0a5d3 100644 --- a/quest/m1/audio-jitter-target/grading.md +++ b/quest/m1/audio-jitter-target/grading.md @@ -13,9 +13,7 @@ before the model lands, `test/audio-quality` grades: regression. - Dropped audio is the player's own count (`trimmed_ms`, `player_skips`). Speeding up is not a skip. -- No row's budget is fixed at zero underruns or zero stalls. Push-to-talk's - own zero gate, in `ptt-analyze.ts`, goes with - [Push-to-talk latency](/quest/m1/push-to-talk-latency.md), not here. +- No row's budget is fixed at zero underruns or zero stalls. - A congestion row, where the path stalls for longer than the target and then delivers the backlog, reports the audio dropped and how long audio takes to return to the clock. @@ -45,10 +43,10 @@ The gates the audit of `main` found pushing the wrong way (2026-11-09, `discarded_ms`). - The 24 browser and native rows cap underruns with no ceiling on lag. Native settled 60 to 88 ms behind its clock without failing one. -- `test/audio-quality/clients/js/ptt-analyze.ts` fails on any underrun or - stall. [Push-to-talk latency](/quest/m1/push-to-talk-latency.md) owns that - change in - [moq-dev/moq#8364](https://github.com/moq-dev/moq/pull/8364); leave it there. +- `test/audio-quality/clients/js/ptt-analyze.ts` failed on any underrun or + stall. [moq-dev/moq#8364](https://github.com/moq-dev/moq/pull/8364) made it + report the counts without gating them (maintainer, /quest-unblock, + 2026-11-09), leaving their judgment to this quest. - The unit tests that pin the old rule belong to the model quest, which rewrites them with the behavior. diff --git a/quest/m1/push-to-talk-latency.md b/quest/m1/push-to-talk-latency.md deleted file mode 100644 index 1d0585b1112..00000000000 --- a/quest/m1/push-to-talk-latency.md +++ /dev/null @@ -1,158 +0,0 @@ -# [M] Push-to-talk's press-to-audible latency fits a budget on the defaults - -## Goal - -On an ideal path with the browser defaults (`` `delay: "auto"`, -20 ms Opus), push-to-talk's press-to-audible latency, excluding device output, -stays under a budget set from a real-device measurement on `kixtop`, and -`just test push-to-talk` enforces it with the per-part breakdown. Native -playback is out of scope. - -## Plan - -Decided 2026-11-04 (maintainer, planning in -[moq-dev/moq#7816](https://github.com/moq-dev/moq/pull/7816)): - -- ✅ Budget the default path with realistic settings, not the fixed 300 ms / - 60 ms setup [moq-dev/moq#7760](https://github.com/moq-dev/moq/pull/7760) - measured; drop the fixed-delay row from `just test push-to-talk` (once Watch - lands, per the 2026-11-06 decision below). The maintainer is fine assuming the constants are the problem. -- ✅ Measure capture and output latency on real macOS hardware (`kixtop`) - first; set the budget from those numbers, and keep output-side knobs until - then. -- ✅ Browser only. -- The catalog toggle forced at least a round trip of jitter buffer, so the - budget is measured on silence suppression, which - [moq-dev/moq#7942](https://github.com/moq-dev/moq/pull/7942) landed. - -Decided 2026-11-06 (maintainer, on -[moq-dev/moq#8009](https://github.com/moq-dev/moq/pull/8009)): how should -`just test push-to-talk` run until Watch lands, since its no-underrun check -fails on the defaults by design? - -- ✅ (a) Require Watch's time stretch ([moq-dev/moq#8058](https://github.com/moq-dev/moq/pull/8058)) for this - quest. The run holds a fixed 300 ms delay until then, and the defaults are - budgeted after Watch lands, on the jitter buffer users will get. -- (b) Run the defaults now and stop failing on underruns while `"auto"` is - on. -- (c) Run both rows, roughly doubling the nine-minute nightly step. - -Decided 2026-11-09 (maintainer, /quest-plan of the -[audio jitter target](/quest/m1/audio-jitter-target/README.md)'s model): zero -underruns is not a goal of that epic either, so the run gates underruns and -stalls on a measured ceiling for good, not until the epic reaches zero. -[moq-dev/moq#8364](https://github.com/moq-dev/moq/pull/8364)'s wording -"leaving zero to the epic" goes. - -[moq-dev/moq#8009](https://github.com/moq-dev/moq/pull/8009) landed the -analyzer's delay timeline: the page records the resolved delay over time, so -each press's `jitter_buffer` ends at the first PCM that reaches the target the -ring held when that PCM landed. The run also keeps 60 ms / 128 kbit/s Opus, -the setup the nightly passes on. On the loaded agent host, 2026-11-06 runs at -the fixed 300 ms delay had no underrun or stall but lost heads ("utterance -head was not decoded"): 2 of 22 on the default 20 ms Opus and 1 on 60 ms. -The group after a head could overtake it, and the read took the mute's -timestamp jump as proof the head was too late to wait for; -[moq-dev/moq#8123](https://github.com/moq-dev/moq/pull/8123) fixed it, and two -20 ms runs after the fix kept all 44 heads. -[moq-dev/moq#8364](https://github.com/moq-dev/moq/pull/8364) switches the page -to the defaults and reports `latency.onset`, the press to the render onset, -and gates its max at 140 ms, set from the `kixtop` run below. What remains is -the underrun and stall ceiling that replaces `ptt-analyze.ts`'s zero gate. - -Measured 2026-11-09 on `kixtop` on the defaults with Watch's stretch, the -browser half only (macOS has no PulseAudio for the native listener, so the -driver ran without it), headless Chromium 156, load average 10 to 12 from -other agents' work. Every browser completeness check passed. Medians and maxes -in ms over the separated presses: - -| Part | `kixtop` | -| --- | --- | -| capture | 2.3 / 11 | -| packetization | 15 / 20 | -| encode | 0.7 / 0.9 | -| network | 2.9 / 4.1 | -| decode | 0.6 / 2.1 | -| jitter buffer | 40 / 60 | -| render | 15 / 30 | -| press to render onset | 78 / 106 | -| device | 261 / 262 | -| `"auto"` target | 60 (40 to 80) | -| underruns / stalls | 17 / 8 | - -The budget is that 106 ms max times 1.25, rounded up to two significant -figures, as `budgets.json` sizes its ceilings. `device` is Bluetooth -headphones, the laptop's default output that day, and the budget excludes it. - -Validated 2026-11-09 on the defaults with Watch's stretch, one run on the Linux -agent host (load average 3 to 5): every completeness check passed and all 44 -heads arrived, but the browser counted 596 underruns and 136 stalls, against -64 / 33 on 2026-11-05. The stretch only accelerates and never expands, so it -cannot cover a target set below the arrival spread; the premise of the -2026-11-06 decision does not hold. Press to render onset was 56 / 83 ms -(median / max over 15 separated presses), the `"auto"` target 40 to 60 ms, -`device` 114 / 120 ms. This run checks the harness only; the budget comes from -`kixtop`. - -Measured 2026-11-05 on the defaults, before Watch, one run of 22 presses -each, Chromium 156 headless through a local relay. `kixtop` is a MacBook Air -on macOS 26 with its built-in speakers as the default output. Linux is the -agent host under heavy load, with PipeWire output. The table gives the median -and max in ms over the separated presses with a breakdown: 15 on `kixtop`, 13 -on Linux, which lost two heads. - -| Part | `kixtop` | Linux | -| --- | --- | --- | -| capture | 9 / 12 | 10 / 11 | -| packetization | 16 / 20 | 16 / 20 | -| encode | 0.5 / 0.6 | 0.5 / 8 | -| network | 2 / 12 | 3 / 5 | -| decode | 0.5 / 3 | 0.6 / 17 | -| jitter buffer | 21 / 59 | 40 / 43 | -| render | 23 / 34 | -1 / 24 | -| press to render onset | 73 / 110 | 71 / 95 | -| device | 54 / 57 | 105 / 113 | -| `"auto"` target | 40 (max 80) | 60 | -| underruns / stalls | 10 / 9 | 64 / 33 | - -Both runs failed the no-underrun check. `"auto"` targets the 0.95 quantile of -arrival lateness and accepts the re-stalls past it by design until Watch -time-stretches the ring. Push-to-talk's capture spread on this ideal path -reached 60 ms on `kixtop` and 78 ms on Linux, past a 40 or 60 ms target. -Linux's much higher count is not separated from its host load (load average -above 20) or its 43 ms render callbacks draining a 60 ms target. - -Measurement limits: the harness's headless Chromium mutes its output and reads -a fake microphone, so `device` is Chromium's projected output for the default -device and `capture` excludes the microphone's own input latency. Linux's -render callbacks are 43 ms, so its `render` can be negative by up to one -callback. The lost heads were the ordering defect -[moq-dev/moq#8123](https://github.com/moq-dev/moq/pull/8123) fixed. - -Knobs: the target's frame term (`frameDuration`, 20 ms by default) and its -measured term (`js/watch/src/audio/latency.ts`); device and render follow the -watch AudioContext's `latencyHint` (`"interactive"` in -`js/watch/src/audio/decoder.ts`), otherwise the platform. - -## Waiting - -- moq-dev/moq: the browser half of the - [stretch model](/quest/m1/audio-jitter-target/stretch.md) lands (decided - 2026-11-09, maintainer, /quest-complete on - [moq-dev/moq#8364](https://github.com/moq-dev/moq/pull/8364)). Then measure - browser underruns and stalls on `kixtop` and the nightly runner, and replace - `ptt-analyze.ts`'s zero gate with that ceiling. Check with - `quest unblocked quest/m1/audio-jitter-target/stretch.md` and its PR. -- Maintainer: whether to wait that long, since the model needs grading, - native A/V lag, and native backpressure first: (a) keep waiting, as - decided; (b) land with a provisional ceiling from today's player, measured - on `kixtop` (17 / 8) and one nightly run on the branch, and let - [calibrate](/quest/m1/audio-jitter-target/calibrate.md) re-measure it after - the model (recommended: it enforces the latency budget now and drops a zero - gate that [grading](/quest/m1/audio-jitter-target/grading.md) says pushes - the wrong way). - -## Related - -- [Stretch model](/quest/m1/audio-jitter-target/stretch.md) - `"auto"` underruns less once the ring slows down near empty and resumes on a cushion, which moves this run's measured ceiling -- [Audio jitter target](/quest/m1/audio-jitter-target/README.md) - the `"auto"` target this latency feeds diff --git a/test/audio-quality/README.md b/test/audio-quality/README.md index 34b2c5ada03..e06edf3202d 100644 --- a/test/audio-quality/README.md +++ b/test/audio-quality/README.md @@ -142,13 +142,13 @@ measured belongs to the media pipeline rather than the path. The native listener, `moq play --video off --delay 300ms --metrics native.ndjson`, joins on its own WebTransport session to the same relay while the publisher talks, plays into a private null sink ([`../lib/pulse.py`](../lib/pulse.py)) as the [native rows](#native-rows) do, and must exit cleanly -on Ctrl-C. It is held to the epic's correctness checks, not its latency budget. Its metrics add what +on Ctrl-C. It is held to the same correctness checks as the browser. Its metrics add what a mute needs: `utterances`, the media timestamp each resume after a declared silence starts from, and `comfort`, the device time filled with comfort noise. Each press must resume on its head frame's timestamp, so a lost head (which would resume one frame later) fails. The native clock is placed on the page's by those resumes, each less its head's container write. Each separated press's mute, -from its release to the next press, must play comfort noise, and from the first press on the -speaker must report no gap or stall. Each press in `summary.json` carries its `native` result +from its release to the next press, must play comfort noise. Its gaps and stalls from the first +press on are counted and reported. Each press in `summary.json` carries its `native` result beside the browser's, and `native` sums them. `presses.json` records captured chunks at each press, WebCodecs encoder and decoder inputs and @@ -162,12 +162,12 @@ later onset can belong to the preceding utterance. Their audible latency is alwa separated presses have an identifiable onset. `summary.json` matches inputs, outputs, writes, and arrivals by media timestamp, so a late arrival -is still attributed to its utterance. The command checks pipeline completeness and one latency -budget: every separated press's `onset` stays within 140 ms, the worst measured on a MacBook Air -with macOS 26 times 1.25, rounded up to two significant figures. It refuses missing encoder inputs, +is still attributed to its utterance. The command checks pipeline completeness and reports latency, +underruns, and stalls without gating them; the [audio grading](../../quest/m1/audio-jitter-target/grading.md) +judges those. It refuses missing encoder inputs, a head that never reaches the listener or never decodes, submitted frames that never encode or reach the container, written frames that never reach the listener, reopened capture, a new encoder (a probe or a rebuilt codec) or subscription, a mute -that disables or drops the rendition, any underrun or stall across the presses, and separated +that disables or drops the rendition, and separated utterances with no audible output. It also fails on a broken driver or invalid capture. For each separated press, `delivery` is the slowest of the utterance's frames that arrived before its onset, measured diff --git a/test/audio-quality/clients/js/ptt-analyze.test.ts b/test/audio-quality/clients/js/ptt-analyze.test.ts index cd89a92cb54..d30e700b77f 100644 --- a/test/audio-quality/clients/js/ptt-analyze.test.ts +++ b/test/audio-quality/clients/js/ptt-analyze.test.ts @@ -195,28 +195,14 @@ test("a mute that disables the rendition fails the run", async () => { expect(stderr).toContain("a mute disabled or dropped the rendition"); }); -test("a press past the onset budget fails the run", async () => { +// The silence between presses is declared, so playback crosses it without counting it. Underruns +// and stalls the player does count are reported, not failed: the audio grading judges them. +test("an underrun or stall across the presses is reported and passes", async () => { const result = trace(); - // The first press renders at 88 ms; this one renders a millisecond past the budget. - result.audible[0].rendered += 53; - result.audible[0].at += 53; - const { exitCode, stderr } = await analyze(result); - expect(exitCode).toBe(1); - expect(stderr).toContain("press to render onset exceeded its 140 ms budget"); -}); - -// The silence between presses is declared, so playback must cross it without counting it. -test("an underrun or stall across the presses fails the run", async () => { - for (const playback of [ - { underruns: 1, stalls: 0 }, - { underruns: 0, stalls: 1 }, - ]) { - const result = trace(); - result.playback = playback; - const { exitCode, stderr } = await analyze(result); - expect(exitCode).toBe(1); - expect(stderr).toContain("playback underran or stalled"); - } + result.playback = { underruns: 3, stalls: 2 }; + const { exitCode, summary } = await analyze(result); + expect(exitCode).toBe(0); + expect(summary.playback).toEqual({ underruns: 3, stalls: 2 }); }); test("the native listener's results sit beside each press", async () => { @@ -257,10 +243,10 @@ test("a native mute without comfort noise fails the run", async () => { expect(stderr).toContain("native: a mute played no comfort noise"); }); -test("a native underrun or stall after the first press fails the run, and one before it does not", async () => { - for (const [page, fails] of [ - [trace().presses[0].at - 1000, false], - [trace().presses[6].at, true], +test("a native underrun or stall counts from the first press on, and passes", async () => { + for (const [page, counted] of [ + [trace().presses[0].at - 1000, 0], + [trace().presses[6].at, 1], ] as const) { for (const event of ["gap", "stall"] as const) { const run = trace(); @@ -271,14 +257,16 @@ test("a native underrun or stall after the first press fails the run, and one be const at = sample.render - 250; if (event === "gap") sample.gaps = [{ at, ms: 20, quanta: 8, short: 1 }]; else sample.stalls = [{ at, stalled: true }]; - const { exitCode, stderr } = await analyze(run, native); - expect(exitCode).toBe(fails ? 1 : 0); - if (fails) expect(stderr).toContain("native: playback underran or stalled"); + const { exitCode, summary } = await analyze(run, native); + expect(exitCode).toBe(0); + expect(summary.native.playback).toEqual( + event === "gap" ? { underruns: counted, stalls: 0 } : { underruns: 0, stalls: counted }, + ); } } }); -test("a native underrun open as the first press lands fails the run", async () => { +test("a native underrun open as the first press lands counts", async () => { const result = trace(); const native = nativeTrace(result); const sample = native.find((sample) => sample.at >= result.presses[0].at + NATIVE + 250); @@ -286,7 +274,7 @@ test("a native underrun open as the first press lands fails the run", async () = // It starts 100 ms before the window, which opens at the first press's head, and runs 200 ms into it. const start = result.presses[0].at + NATIVE + 2; sample.gaps = [{ at: sample.render - (sample.at - (start - 100)), ms: 300, quanta: 113, short: 0 }]; - const { exitCode, stderr } = await analyze(result, native); - expect(exitCode).toBe(1); - expect(stderr).toContain("native: playback underran or stalled"); + const { exitCode, summary } = await analyze(result, native); + expect(exitCode).toBe(0); + expect(summary.native.playback.underruns).toBe(1); }); diff --git a/test/audio-quality/clients/js/ptt-analyze.ts b/test/audio-quality/clients/js/ptt-analyze.ts index 14527ed8ce0..910f604656e 100644 --- a/test/audio-quality/clients/js/ptt-analyze.ts +++ b/test/audio-quality/clients/js/ptt-analyze.ts @@ -170,7 +170,7 @@ const nativePlayback = { const parts = ["capture", "packetization", "encode", "network", "decode", "jitter_buffer", "render", "device"] as const; const summary = { latency: { - // Press to render onset: what the budget covers, everything but the device's own output. + // Press to render onset: everything but the device's own output. onset: summarize(presses.map((press) => press.onset)), delivery: summarize(presses.map((press) => press.delivery)), playout: summarize(presses.map((press) => press.playout)), @@ -206,15 +206,8 @@ console.log( JSON.stringify({ latency, path, delay, unencoded, unsent, unreceived, playback, native: summary.native }, null, 2), ); -// Press to render onset, the slowest separated press: kixtop's worst (106 ms on 2026-11-09, macOS -// 26, headless Chromium 156) times 1.25, rounded up to two significant figures, as budgets.json sizes -// its ceilings. -const ONSET_BUDGET_MS = 140; - +// Latency, underruns, and stalls are reported, never gated: the audio grading quest judges them. const failures: string[] = []; -if (!latency.onset || latency.onset.max > ONSET_BUDGET_MS) { - failures.push(`press to render onset exceeded its ${ONSET_BUDGET_MS} ms budget`); -} if (presses.some((press, i) => (i === 0 || i >= 8) && press.audible === null)) { failures.push("a separated utterance never reached the output"); } @@ -222,21 +215,18 @@ if (result.captures.length) failures.push("capture reopened"); if (result.encoders.length) failures.push("an encoder was created again"); if (result.subscriptions.length) failures.push("subscription reopened"); if (result.catalog.length) failures.push("a mute disabled or dropped the rendition"); -// The silence between presses is declared, so it is neither an underrun nor a stall. -if (result.playback.underruns || result.playback.stalls) failures.push("playback underran or stalled"); if (presses.some((press) => !press.submitted || !press.firstEncoded)) failures.push("utterance head was not encoded"); if (summary.unencoded || summary.unsent) failures.push("submitted audio was not encoded and sent completely"); if (presses.some((press) => !press.firstReceived)) failures.push("utterance head was not received"); if (presses.some((press) => press.firstReceived && !press.firstDecoded)) failures.push("utterance head was not decoded"); if (summary.unreceived) failures.push("written audio did not all reach the listener"); -// The native listener: each utterance resumes on its head, every separated mute plays comfort -// noise, and the declared silences cost no underrun or stall. +// The native listener: each utterance resumes on its head, and every separated mute plays comfort +// noise. if (presses.some((press) => !press.native.head)) failures.push("native: an utterance did not resume on its head"); if (presses.some((press, i) => i >= 8 && !press.native.comfort)) failures.push("native: a mute played no comfort noise"); -if (start === undefined || !windowed.length || nativePlayback.underruns || nativePlayback.stalls) - failures.push("native: playback underran or stalled"); +if (start === undefined || !windowed.length) failures.push("native: no playback from the first press"); if (failures.length) { console.error(failures.join("; ")); process.exitCode = 1;