Skip to content

quest: close the cpp -> js interop budget flake, fixed by #6231 - #6442

Merged
kixelated merged 4 commits into
mainfrom
quest/m1/test-flakes-2/interop-cpp-js
Oct 23, 2026
Merged

kixelated merged 4 commits into
mainfrom
quest/m1/test-flakes-2/interop-cpp-js

Conversation

@kixelated

@kixelated kixelated commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Closes quest/m1/test-flakes-2/interop-cpp-js.md with no code change: the cause was already fixed by moq-dev/moq#6231. This PR deletes the quest and its link.

What overran

None of the three phases the quest guessed. In run 39734065658 the browser started fast, painted 3 video frames, then got no more video and no audio for the whole 30 s audio wait (videoFrames: 3, audioBytes: 0). That was a stalled player, not a slow one.

Cause

The run's artifacts show the C++ publisher fell back to WebSocket under load (using WebSocket fallback). Its log then shows "subscribed error ... internal error" each time the relay cancelled an upstream subscription as a fast subscriber left. The relay's WebSocket sessions ran on crates.io qmux 0.6, which returned connection credit (MAX_DATA) only for bytes the application read (RecvStream::report_consumed). A stream that was stopped or dropped with unread bytes never gave its credit back. So every cancelled group stream used up part of the publisher's connection window. By the time the browser subscribed, the window was empty: the catalog got through, plus a few frames, then nothing.

#6231 replaced qmux 0.6 with the in-tree qmux on the QUIC stream state machine, which credits unread data on stop. credit_returns_for_unread_and_stopped_streams in rs/moq-quic/src/connection/mux/tests.rs already covers this.

Reproduction

The harness rebuilds every client on each run, so I used a scratch loop instead. It ran a relay, a C++ publisher forced onto ws://, eight short-lived cpp-interop subscribers, and one browser driver.ts subscribe, all pinned to 2 CPUs:

Build Browser cells stalled
CI commit c59c3ad51 3 / 12
#6231's parent 72259ab3c~1 3 / 12 (one of them is the CI failure exactly: "audible audio" timeout after 30 s)
#6231 72259ab3c 0 / 20
main 0 / 28

On the two older builds the publisher logged about 7 "subscribed error" lines per run. On #6231 and main it logged none. Without the WebSocket publisher, 500+ browser cells on main at 2 to 4 CPUs showed no stall (7 fast tone-check failures, see follow-ups).

Public API / wire impact

None.

Checks

  • just check: passed.
  • quest check --base origin/main: passed.
  • Interop: reproduction runs as above. just test interop --all was not rerun, since this PR changes no code.

Follow-ups (not filed)

  • Under extreme CPU starvation (8 browsers on 2 CPUs), main's browser cell can fail fast (in under 12 s) in the tone check. The auto playout delay rises to 1 to 5 s, the ring re-stalls to refill, and the 3 s TONE_GRACE_MS sees silence. In one case it saw stuttered audio and reported a false "wrong tone". This was never seen in CI. A fix would charge the grace only for audible playback.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

https://claude.ai/code/session_01EeyiEy5FWz3z44DYZVo7vU

kixelated and others added 2 commits October 22, 2026 20:27
The 38 s cell in run 39734065658 was a stalled player, not a slow one: the
browser decoded 3 video frames and no audio for 30 s. Under load the C++
publisher fell back to WebSocket, and crates.io qmux 0.6 never returned
connection credit for bytes a stopped or dropped stream left unread. Every
subscription the relay cancelled as the fast subscribers left leaked its
window, until the publisher could not send. The in-tree qmux on the QUIC
stream state machine (#6231) returns that credit, and
`credit_returns_for_unread_and_stopped_streams` covers it.

Reproduced with a WebSocket publisher, eight short-lived subscribers, and one
browser on two CPUs: 3/12 browser cells stall at #6231's parent and at the
failing commit, 0/20 at #6231, 0/28 on main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EeyiEy5FWz3z44DYZVo7vU
@kixelated
kixelated force-pushed the quest/m1/test-flakes-2/interop-cpp-js branch from 9a37edb to 8b3211f Compare October 23, 2026 03:28
@kixelated kixelated changed the title The interop matrix's cpp to js cell stays inside its budget quest: close the cpp -> js interop budget flake, fixed by #6231 Oct 23, 2026
@kixelated
kixelated marked this pull request as ready for review October 23, 2026 03:28
@kixelated

Copy link
Copy Markdown
Collaborator Author

Outcome: the cause was a qmux 0.6 connection-credit leak behind a WebSocket publisher, already fixed by #6231. I reproduced it with a forced-WebSocket publisher on 2 CPUs: 3/12 cells stalled at #6231's parent, 0/20 at #6231, and 0/28 on main. This PR only deletes the quest. One follow-up is suggested in the body (the tone-grace check under extreme CPU starvation), not filed.

(Written by Claude Opus 5.5)

@coderabbitai

coderabbitai Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 5 seconds.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: ea996772-37ec-4188-9c7b-30568f059804
📥 Commits

Reviewing files that changed from the base of the PR and between c3b2c1b and c61cff5.

📒 Files selected for processing: 3
  • quest/m1/test-flakes-2/README.md
  • quest/m1/test-flakes-2/interop-cpp-js.md
  • quest/m1/test-flakes-2/tone-grace.md
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-23T18:33:00.823345Z
We couldn’t complete this review. Comment @codex review to try again.
c61cff5 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: 8b3211f660

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

# Conflicts:
#	quest/m1/test-flakes-2/README.md

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: eda6b04691

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread quest/m1/test-flakes-2/README.md
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Merging. Closes the cpp to js interop budget flake quest, fixed by #6231, and records the tone-check grace follow-up as a quest. CI is green, the current head has a non-Grok review, and quest check passes against main.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 23, 2026
Merged via the queue into main with commit 2f146e8 Oct 23, 2026
10 checks passed
@kixelated
kixelated deleted the quest/m1/test-flakes-2/interop-cpp-js branch October 23, 2026 20:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant