test(publish): isolate capture format overrides, keep the media fixture unpinned

quest/m1/README.md

@@ -142,7 +142,7 @@ above them.
 - [Watch worker](/quest/m1/watch-worker.md) - watch playback runs in a worker onto an OffscreenCanvas, so main-thread jank never stalls video or audio
 - [Rust papercuts](/quest/m1/papercuts-rs.md) - HTTP refusals are counted, and a `u64::MAX` resume is unbounded
 - [Duration parser defaults](/quest/m1/cli-duration-defaults.md) - a moq-tokio duration set in code beats the parser default on a parsed or merged config, zero included, after #5350
-- [Publish channel count](/quest/m1/publish-audio-channel-count.md) - forcing a channel count on an Audio.Capture stops costing the subscriber gaps of silence
+- [Publish channel count](/quest/m1/publish-audio-channel-count.md) - forcing a channel count on an Audio.Capture stops costing the subscriber gaps of silence, and the media fixture's cold-start audio-tone flake is fixed at its cause
 - [JS request deadline](/quest/m1/js-request-deadline.md) - each JS request, PUBLISH_NAMESPACE included, has one fatal 10 s timer from create to answer and fails fast without stream credit, so nothing queues or retries
 - [Handshake deadline covers admission](/quest/m1/handshake-admission-deadline.md) - `listen.timeout` bounds a connection through auth admission on both listeners, replacing the hardcoded admission timeout
 - [Container reads drop backfill](/quest/m1/read-backfill.md) - the JS container consumer drops groups below its read cursor like Rust

quest/m1/publish-audio-channel-count.md

@@ -2,63 +2,53 @@
 
 ## Goal
 
-`new Audio.Capture({ source, channelCount: 1 })` publishes continuous audio.
-A subscriber hears no gaps that the same capture without the override does not
-also have, so the documented way to pin a publisher's channel count stops being
-a way to degrade it.
+`new Audio.Capture({ source, channelCount: 1 })` publishes audio a subscriber
+plays as smoothly as the same capture without the override, so pinning a
+publisher's channel count stops being a way to degrade it. The browser media
+fixture can then pin mono again.
 
-Boundaries: `channelCount` keeps its meaning, "force the captured channel
-count", and the published shape does not move. Whether the fix is to stop
-forcing `channelCountMode: "explicit"`, to do the downmix off the realtime
-graph, or to keep the mode and remove whatever it starves, is the quest's to
-decide.
+This is also the home for the media fixture's cold-start audio-tone flake
+without the pin, which shows the same symptom. Fix it at its cause, not by
+lowering the tone floor.
 
-## Plan
-
-`js/publish/src/audio/capture.ts` sets
-`channelCountMode: requestedChannels !== undefined ? "explicit" : "max"`, so
-supplying `channelCount` is what puts the AudioWorklet behind a forced Web
-Audio downmix. Omitting it does not always take the `"max"` path:
-`requestedChannels` falls back to `requestedChannelCount(source.track)`,
-so an applied `getUserMedia` channelCount constraint forces `"explicit"` too.
-Account for that path when isolating the cause.
-
-What was measured, on macOS headless Chromium through `just test media`,
-publishing a `MediaStreamAudioDestinationNode` track into a graph already
-running at 48 kHz:
-
-- With `sampleRate: 48000, channelCount: 1`: 11 probe samples across a run read
-  a flat spectrum (peak equal to median, below `AUDIBLE_RMS`), which is
-  silence, not a weak tone. The `audio tone` check fell to 82-88% against its
-  90% floor and took two negative controls down with it.
-- With both overrides removed: zero such samples, and every media check passed.
-- `main`, whose fixture pinned the same format on the encoder rather than the
-  capture, reported `tone 150dB above the floor` and no silent samples at all.
+`channelCount` keeps its meaning, "force the captured channel count".
 
-What is not established, and should be first:
-
-- The two overrides were removed together, so `channelCount` is implicated by
-  reading the `channelCountMode` choice, not by an isolated run. Re-run with
-  `sampleRate` alone and with `channelCount` alone before believing the
-  attribution.
-- The mechanism. A forced downmix starving or stalling the worklet is the
-  obvious guess and is not evidence.
-- Whether a `getUserMedia` microphone track shows it, or only a
-  destination-node track, whose channel count Web Audio reports as 2 by
-  default. `requestedChannelCount` exists for the macOS
-  mono-mic misreport, so that path has a real caller and cannot simply lose the
-  override.
-
-The regression belongs where the evidence came from: the browser fixture can
-pin a channel count again once this is fixed, and
-`test/interop/clients/js/src/fixture.ts` carries a comment saying why it does
-not. A unit test that counts worklet callbacks under an explicit downmix would
-be cheaper than a full media run, if one can be made to fail reliably.
+## Plan
 
-The `just test media` audio-tone misses under load were not this: the fixture
-takes the `"max"` path, and every miss was the player's own buffering silence
-with audio still arriving.
+What is known:
+
+- The real-worklet matrix in
+  `test/interop/clients/js/capture-channel-count.browser.ts` (in
+  `just test harness`) finds nothing override-specific: no timestamp gaps,
+  the requested format, and audible output in every channel, on Linux and
+  macOS. It only asserts each channel is audible at least once. An uncommitted
+  per-quantum trace (#5270) found one short silent run during startup in
+  every case, the no-override baseline included, and none after. Under heavy
+  load the baseline also showed a later run, so a strict per-quantum check
+  flakes on the source. Tightening the matrix needs a baseline-relative
+  measure.
+- The player stalls. In CI (#5270), the pinned fixture failed the `just test media`
+  audio-tone check in 2 of 3 runs, against ~5% of unpinned runs (5 of 108,
+  all cold start). Failures look the same either way: player samples go
+  `stalled` while received audio bytes keep growing, and the jitter delay jumps
+  from ~77 ms to 500-700 ms.
+
+The working hypothesis is that the explicit downmix (`channelCountMode:
+"explicit"` in `js/publish/src/audio/capture.ts`) changes something the player
+reacts to, such as frame cadence or burstiness, rather than dropping PCM.
+Neither half is proven. Measure the
+publisher's frame arrival timing pinned vs unpinned before picking a fix, and
+check whether the player's re-stall behavior
+([watch jitter target](/quest/m1/audio-jitter-target/watch.md)) is the
+amplifier. A loaded local host fails the media fixture either way, so compare
+on CI or a quiet machine.
+
+Not covered: an applied `getUserMedia` channelCount constraint also takes the
+explicit path, but a destination-node track refuses that constraint, so the
+harness cannot exercise it.
 
 ## Related
 
+- [Watch jitter target](/quest/m1/audio-jitter-target/watch.md) - a ring that runs dry re-stalls and refills to the target
+- [Test flakes, round 2](/quest/m1/test-flakes-2/README.md) - load flakes fixed at their cause
 - [Browser benchmarks](/quest/m2/browser-benchmarks.md) - the other place browser-side capture and encode costs get measured

quest/m1/test-flakes-2/README.md

@@ -45,3 +45,4 @@ Public API: none. Wire: none.
 ## Related
 
 - [Controlled time](/quest/m1/time/README.md) - the systemic fix these rules ask for: every crate on a controlled clock
+- [Publish channel count](/quest/m1/publish-audio-channel-count.md) - tracks the media fixture's cold-start audio-tone flake, a player stall it shares with the mono pin

test/interop/clients/js/capture-channel-count.browser.ts

@@ -0,0 +1,25 @@
+/** Capture diagnostic uses the real browser and worklet, with each format override isolated. */
+import { expect, test } from "bun:test";
+import { createServer } from "vite";
+import { launch } from "./harness";
+
+test("capture format override matrix", async () => {
+	const server = await createServer({ server: { host: "127.0.0.1", port: 0 } });
+	await server.listen();
+	const browser = await launch(["--autoplay-policy=no-user-gesture-required"]);
+	try {
+		const page = await browser.newPage();
+		await page.goto(`${server.resolvedUrls?.local[0]}capture-channel-count.html`);
+		await page.waitForFunction(() => typeof window.captureMatrix === "function");
+		const result = await page.evaluate(() => window.captureMatrix());
+		expect(result).toEqual([
+			{ mode: "none", channelsAudible: [true, true], gaps: 0, format: { sampleRate: 48_000, channelCount: 2 } },
+			{ mode: "rate", channelsAudible: [true, true], gaps: 0, format: { sampleRate: 48_000, channelCount: 2 } },
+			{ mode: "channels", channelsAudible: [true], gaps: 0, format: { sampleRate: 48_000, channelCount: 1 } },
+			{ mode: "both", channelsAudible: [true], gaps: 0, format: { sampleRate: 48_000, channelCount: 1 } },
+		]);
+	} finally {
+		await browser.close();
+		await server.close();
+	}
+}, 30_000);

test/interop/clients/js/capture-channel-count.html

@@ -0,0 +1,2 @@
+<!doctype html>
+<script type="module" src="/src/capture-channel-count.ts"></script>

test/interop/clients/js/harness.browser.ts

@@ -22,7 +22,7 @@ test("pause activates its button when a canvas intercepts pointer input", async
 	} finally {
 		await browser.close();
 	}
-});
+}, 30_000);
 
 for (const stall of ["audio", "video"]) {
 	test(`the paused play button stays clickable during ${stall} stalling`, async () => {

test/interop/clients/js/src/capture-channel-count.ts

@@ -0,0 +1,61 @@
+/** Real capture measurements isolating format overrides before encoding or playout. */
+import * as Publish from "@moq/publish";
+import { Effect } from "@moq/signals";
+
+declare global {
+	interface Window {
+		captureMatrix: () => Promise<unknown[]>;
+	}
+}
+
+window.captureMatrix = async () => {
+	const results = [];
+	for (const mode of ["none", "rate", "channels", "both"]) {
+		const source = new AudioContext({ sampleRate: 48_000 });
+		const oscillator = source.createOscillator();
+		const destination = source.createMediaStreamDestination();
+		oscillator.connect(destination);
+		oscillator.start();
+		await source.resume();
+		const track = destination.stream.getAudioTracks()[0];
+		const capture = new Publish.Audio.Capture({
+			source: { track: track as Publish.Audio.StreamTrack, kind: "music" },
+			sampleRate: mode === "rate" || mode === "both" ? 48_000 : undefined,
+			channelCount: mode === "channels" || mode === "both" ? 1 : undefined,
+		});
+		const effect = new Effect();
+		try {
+			let fanout = capture.out.frames.peek();
+			while (!fanout) {
+				await capture.out.frames.changed();
+				fanout = capture.out.frames.peek();
+			}
+			const reader = fanout.subscribe(effect).getReader();
+			const channelsAudible: boolean[] = [];
+			let gaps = 0;
+			let expected: number | undefined;
+			for (let i = 0; i < 512; i++) {
+				const { value: frame } = await reader.read();
+				if (!frame) throw new Error("capture ended");
+				for (const [channel, samples] of frame.channels.entries()) {
+					if (!samples.length || samples.some((sample) => !Number.isFinite(sample))) {
+						throw new Error(`capture channel ${channel} has invalid PCM`);
+					}
+					const rms = Math.sqrt(samples.reduce((sum, value) => sum + value * value, 0) / samples.length);
+					channelsAudible[channel] = (channelsAudible[channel] ?? false) || rms > 0.01;
+				}
+				if (expected !== undefined && Math.abs(frame.timestamp - expected) > 2) gaps++;
+				expected = frame.timestamp + (frame.channels[0].length / 48_000) * 1_000_000;
+			}
+			await reader.cancel();
+			results.push({ mode, channelsAudible, gaps, format: capture.out.format.peek() });
+		} finally {
+			effect.close();
+			capture.close();
+			track.stop();
+			oscillator.stop();
+			await source.close();
+		}
+	}
+	return results;
+};

test/interop/clients/js/src/fixture.ts

@@ -109,10 +109,10 @@ export class Fixture {
 		// suspended graph.
 		//
 		// No sampleRate or channelCount override: the graph already runs at SAMPLE_RATE, and asking
-		// the capture for one channel puts the AudioWorklet behind an explicit Web Audio downmix
-		// whose output drops PCM under load, which reaches the player as gaps of silence. Publishing
-		// the destination node's own format costs nothing here, since the tone is the same in both
-		// channels.
+		// the capture for one channel puts the AudioWorklet behind an explicit Web Audio downmix that
+		// made the player stall far more often in CI, with no capture loss found in isolation; see
+		// /quest/m1/publish-audio-channel-count.md. Publishing the destination node's own format
+		// costs nothing here, since the tone is the same in both channels.
 		const audioCapture = new Publish.Audio.Capture({
 			source: { track: audioTrack, kind: "music" },
 		});

test/interop/clients/js/tsconfig.json

@@ -8,6 +8,7 @@
 		"driver.ts",
 		"harness.ts",
 		"harness.browser.ts",
+		"capture-channel-count.browser.ts",
 		"buffering",
 		"media.ts",
 		"src",

test/justfile

@@ -26,7 +26,7 @@ harness:
     cd .. && bun test test/interop/compat/resolve.test.ts
     cd .. && bun install --frozen-lockfile
     cd interop/clients/js && bunx playwright install chromium
-    cd interop/clients/js && bun test ./harness.browser.ts ./src/contract.test.ts ./src/resume.test.ts
+    cd interop/clients/js && bun test ./harness.browser.ts ./capture-channel-count.browser.ts ./src/contract.test.ts ./src/resume.test.ts
 
 # Cross-language media interop test, built from this checkout. Stands up a
 # relay and runs the publisher x subscriber matrix. Default: rust only (fast