Skip to content

test(publish): isolate capture format overrides, keep the media fixture unpinned - #5270

Merged
kixelated merged 10 commits into
mainfrom
quest/m1/publish-audio-channel-count
Oct 14, 2026
Merged

kixelated merged 10 commits into
mainfrom
quest/m1/publish-audio-channel-count

Conversation

@kixelated

@kixelated kixelated commented Oct 10, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

A macOS just test media run lost audio with sampleRate: 48000, channelCount: 1 on the fixture's capture, and recovered when both overrides were removed. That blamed the explicit Web Audio downmix, which channelCount selects, without isolating it.

Approach

Isolate each override against the production Audio.Capture AudioWorklet in a real browser, then rerun the full media fixture with the pin restored.

  • Capture matrix (capture-channel-count.browser.ts, in just test harness): none / rate / channels / both, 512 quanta each. Asserts the requested format, finite audible PCM in every output channel, and zero timestamp gaps. Passes on Linux and macOS Chromium 153. It finds nothing override-specific. It only asserts each channel is audible at least once; an uncommitted per-quantum trace found one short silent run during startup in every case, the no-override baseline included.
  • Full media fixture with the pin: passed 3 of 3 on macOS. In CI, though, the audio-tone check failed 2 of 3 runs, against ~5% of unpinned runs (5 of the last 108, all cold start). Failures look the same either way: player samples go stalled while received audio bytes keep growing, and the jitter delay jumps from ~77 ms to 500-700 ms.

So the pin makes the player stall more often in CI; whether through capture cadence or something else is the open question. The fixture stays unpinned, and the quest stays open, rewritten around capture cadence and player stalls. It is also now the home for the unpinned cold-start audio-tone flake.

Decisions

CI fails the media audio-tone check in 2 of 3 runs with the mono pin, against ~5% unpinned. What should this PR do?

  • ✅ Drop the pin, land the capture matrix only, keep the quest open with this evidence (Recommended)
  • Rerun Interop on the pinned head a few more times before deciding
  • Keep the pin and accept a higher media flake rate

Impact

  • No production code, public API, wire format, dependency, or version changes.
  • New browser test wired into just test harness. The first harness.browser.ts test gets the same 30 s timeout as its siblings, since a cold browser launch on a loaded host can exceed bun's 5 s default.
  • test/interop/clients/js/src/fixture.ts: comment only, now stating what the evidence shows.
  • quest/m1/publish-audio-channel-count.md rewritten; linked from the test-flakes-2 README.

Not covered

A real getUserMedia microphone with an applied channelCount constraint: a destination-node track rejects the constraint with OverconstrainedError, so this harness cannot exercise that path.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 6 commits October 10, 2026 14:16
Co-Authored-By: GPT-6 <noreply@openai.com>
Co-Authored-By: GPT-6 <noreply@openai.com>
Co-Authored-By: GPT-6 Astra <noreply@openai.com>
Reject invalid PCM and prove a missing stereo channel cannot pass the override matrix.

Co-Authored-By: GPT-6 Astra <noreply@openai.com>
…o-channel-count

# Conflicts:
#	test/interop/clients/js/tsconfig.json
#	test/justfile
The capture override matrix and the full media fixture pass on macOS with
channelCount 1, so the reported loss was not the explicit downmix. Restore
the pin so every media run covers that path, assert zero timestamp gaps in
the matrix, and delete the investigation quest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title test(publish): isolate capture audio format overrides test(publish): pin the media fixture to mono capture again Oct 13, 2026
@kixelated
kixelated marked this pull request as ready for review October 14, 2026 16:37
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-14T18:23:51.184944Z 87f85b9 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 34 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: ecb83fd1-6695-42be-99e3-981ebed23515
📥 Commits

Reviewing files that changed from the base of the PR and between 68cc5fc and 60c76cf.

📒 Files selected for processing: 10
  • quest/m1/README.md
  • quest/m1/publish-audio-channel-count.md
  • quest/m1/test-flakes-2/README.md
  • test/interop/clients/js/capture-channel-count.browser.ts
  • test/interop/clients/js/capture-channel-count.html
  • test/interop/clients/js/harness.browser.ts
  • test/interop/clients/js/src/capture-channel-count.ts
  • test/interop/clients/js/src/fixture.ts
  • test/interop/clients/js/tsconfig.json
  • test/justfile
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 69ff3c3.

No actionable bugs found. Restoring the mono fixture exercises explicit downmix through the existing encode/playback checks, while the isolated override matrix is a useful diagnostic. Keeping this test-only rather than adding an unproven capture workaround is sensible.

Coverage caveat: capture-channel-count.ts:45 ORs audibility across all 512 quanta, so it proves each channel is audible at least once, not continuously audible. Contiguous timestamps also do not rule out silent PCM. A post-warmup silent-quanta fraction or longest-run measurement with baseline-aware tolerance would strengthen the diagnostic; the restored full-media tone checks provide broader coverage now. Applied getUserMedia channel constraints remain explicitly outside this fixture.

Verification: GitHub-only static inspection of capture/worklet, diagnostic, fixture, media assertions and harness wiring. No browser tests executed, and the reported macOS/Linux results were not independently reproduced.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 69ff3c328d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

throw new Error(`capture channel ${channel} has invalid PCM`);
}
const rms = Math.sqrt(samples.reduce((sum, value) => sum + value * value, 0) / samples.length);
channelsAudible[channel] = (channelsAudible[channel] ?? false) || rms > 0.01;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject silent quanta after capture becomes audible

When the explicit downmix emits one audible quantum and then intermittent or sustained silence, this assignment permanently latches the channel to true, so even 511 silent frames can produce the expected result. Those silent frames also retain contiguous timestamps, leaving gaps at zero. Count or bound inaudible quanta after startup so this test can detect the PCM-silence regression it is intended to cover.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed the latch can't see intermittent silence, but I measured the stricter check and it doesn't hold, so I'm keeping the latch.

I traced per-quantum audibility over 5 runs (20 captures). Every mode, the no-override baseline included, drops exactly one 3-quantum silent run, somewhere in quanta 12 to 108. Then I required zero silent quanta after quantum 256 and ran it 8 times on a loaded host (load average ~44). One run failed, and it failed in mode none (silent: [3, 3]), the baseline with no override at all.

So the silence comes from the MediaStream source under load, not from the channelCount or sampleRate path. A strict check would flake on the baseline. A bound relative to the baseline would flake too, because each mode captures independently and the runs land at random. End-to-end coverage for sustained silence comes from the full media fixture's tone checks, which now run pinned to mono.

(Written by Claude Opus 5.5)

@kixelated

Copy link
Copy Markdown
Collaborator Author

Merged main (quest README conflict only) and ran the checks. Not enqueuing: CI suggests the mono pin raises the rate of the media fixture's audio-tone failure, so whether to keep the pin needs a maintainer decision.

Media fixture (just test media in CI)

  • Pinned (this PR): the audio-tone check failed 2 of 3 runs. Both failures were on 69ff3c3: run 38599664938 failed the first attempt at after enable: 56/65 and the rerun at cold start: 62/70. The earlier pinned head b05e881 passed once.
  • Unpinned (every other branch, last 108 completed runs back to 2026-10-10): 5 audio-tone failures (~5%), all in cold start.
  • The failures look the same both ways: stalled player samples, with received audio bytes still growing and the jitter delay jumping from ~77 ms to ~500 to 700 ms. Delivery continues and playback stalls. So the PR's "player-side, not a PCM defect" reading still fits, but the downmix path seems to make the stall more frequent, which matches the original report.
  • I couldn't A/B locally: at load average 40 to 65, just test media failed all 10 runs, pinned or not.

Other results

  • just test interop --all: all cross-language pairs pass locally. The full CI matrix also passed on the rerun. The earlier failures in run 38446079864 were -> c "timed out waiting for data" and the unroutable WebSocket-upgrade bug that fix(tokio)!: keep the transport that won the connect race #5484 fixed. The JS publisher passed every pair.
  • just check passes. In just test harness, the new capture matrix passes. The untouched pause activates its button... test times out at bun's default 5 s on a loaded host, and passes with --timeout 30000.
  • Review finding (silence latch): replied inline. A strict check after onset flakes on the no-override baseline itself.

Options

  1. Drop the pin, land the capture matrix only, and keep the quest open with this CI evidence. The PCM is continuous, so the investigation moves to capture cadence and player stalls. (Recommended)
  2. Rerun Interop on this head a few more times to firm up the rate before deciding.
  3. Keep the pin and accept a higher media flake rate.

(Written by Claude Opus 5.5)

kixelated and others added 2 commits October 14, 2026 11:17
CI failed the media audio-tone check in 2 of 3 runs with the mono pin,
against ~5% unpinned, with the same player stall either way while the
captured PCM stays continuous. Drop the pin, keep the capture matrix, and
keep the quest open as the home for that stall and the unpinned cold-start
flake. Give the first harness browser test the same 30 s timeout as its
siblings, since a cold browser launch can exceed bun's default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title test(publish): pin the media fixture to mono capture again test(publish): isolate capture format overrides, keep the media fixture unpinned Oct 14, 2026
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

Scope changed per the maintainer: the mono pin is dropped, and the PR now lands the capture matrix plus the rewritten quest.

(Written by Claude Opus 5.5)

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 👍

Reviewed commit: 87f85b9c89

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 87f85b9, separating the fixture/quest update from the merge of main.

No new actionable code bugs found. Leaving the fixture unpinned and reopening the investigation is reasonable given the pinned media failures. The real-worklet override matrix and wiring are unchanged from the previous review; the browser timeout change is scoped to the harness.

Coverage correction: full-media checks no longer exercise explicitly pinned mono downmix. The existing silence-latch caveat still applies: #5270 (comment). The diagnostic proves each channel becomes audible at least once and checks timestamps, not that every captured quantum remains audible. Consequently the quest's categorical PCM-versus-player conclusion is stronger than this committed test establishes; cadence and player re-stalling are investigation hypotheses. No duplicate inline finding added.

Verification: static GitHub comparison plus inspection of the prior pinned Interop rerun logs, which show the matrix passing while cold-start audio-tone fails and player delay rises during continued audio receipt. I did not reconstruct the aggregate failure-rate comparison or execute tests. Current-head CI was still pending at inspection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Summary for the merge, at the maintainer's choice of option 1:

  • Scope: the mono pin is dropped and the fixture stays unpinned; its comment now states what the evidence shows. The PR lands the real-worklet capture matrix in just test harness, keeps quest/m1/publish-audio-channel-count.md rewritten around capture cadence and player stalls, and makes that quest the home for the unpinned cold-start audio-tone flake (linked from test-flakes-2).
  • Harness: the first harness.browser.ts test now has the same 30 s timeout as its siblings.
  • Evidence: the pinned media fixture failed the audio-tone check in 2 of 3 CI runs, against ~5% unpinned (5 of 108), with the same player-stall signature either way.
  • Reviews: Codex 👍 on 87f85b9. The OpenAI review found no bugs. It noted the quest overstated what the matrix proves, since the matrix only asserts each channel is audible at least once. Fixed in 60c76cf by rewording the quest and fixture comment (wording only). The earlier silence-latch finding is answered inline with measurements.
  • Checks: quest check and just check pass; just test harness passes; just test interop --all passed locally on the earlier head.

(Written by Claude Opus 5.5)

@kixelated
kixelated enabled auto-merge October 14, 2026 18:27
@kixelated
kixelated added this pull request to the merge queue Oct 14, 2026
Merged via the queue into main with commit 83c9619 Oct 14, 2026
9 checks passed
@kixelated
kixelated deleted the quest/m1/publish-audio-channel-count branch October 14, 2026 19:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant