Repository navigation
fix(net): never leave a torn request message on the stream - #6433
Conversation
Quest-Claim: 9cf04398847f47fb
The union renewal flake was the publisher writing REQUEST_OK's type and body in two writes: the shared write budget can yield to the host after any write, and under load the subscriber's FIN landed in that gap, so the publisher FINed after a bare type and the subscriber closed the session as malformed. #6255 already made every JS message one write; this pins that with a test that yields after every write, and closes the rest: - JS: the IETF subscriber no longer sends an update of defaults when its last subscriber leaves (the logged priority=255 was that), matching lite and Rust, and an update that fails to encode ends the track loudly instead of being swallowed. - Rust: every IETF control message buffers its type with its body, all or nothing, via Writer::buffer_message / encode_message, so a failed encode or a cancelled write can no longer strand a bare type for the next write or a FIN to complete. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EeyiEy5FWz3z44DYZVo7vU
e6230ef to
9894b11
Compare
|
Outcome: I reproduced the flake on the parent of #6255: the publisher's REQUEST_OK was written as a type and then a body, and the write budget's yield let the subscriber's FIN land between the two. On main, #6255 already writes each message whole. This PR pins that with a test that yields after every write, on every IETF draft. It also stops the JS subscriber's update of defaults when its last subscriber leaves (the logged priority=255), makes an update that fails to encode end the track instead of being swallowed, and makes every Rust IETF control message buffer its type and body together. (written by Claude Opus 5.5) |
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 40 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9894b11179
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
buffer_message now buffers the type and body separately so control stats count the body without its type selector, as main's stats require. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drafts 14 and 15 require the next request ID, so an update that spent one and never sent it would stall every later request. As in Rust's legacy header path, that now closes the session with an internal error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Finishes quest/m1/test-flakes-2/auth-union-renewal.md.
Root cause
auth.test.ts's "a request waits for its token and survives union renewal" failed with "malformed request stream message". I reproduced it on the parent of moq-dev/moq#6255: 4 failures in 3,600 loaded runs, all on moqt-17/22. With extra logging I found the torn message. The publisher's REQUEST_OK answering the subscriber's update was cut short. The subscriber's update itself went out whole. The publisher wrote the type and the body in two writes.Writer.writecan hand control back to the host after any write: the shared write budget yields once 4 ms of CPU time pass, which happens more often under load. The subscriber's FIN landed in that gap, so the publisher FINed after a bare type and the subscriber closed the session.moq-dev/moq#6255 (merged today, after this quest was written) already makes every JS IETF/lite message a single write. I found no failures in 10,800 loaded runs on main. This PR adds a test that pins that fix and covers the related problems below.
The logged
priority=255was real but harmless. Before moq-dev/moq#6142, 255 was the wire value for priority 0. The subscriber sent it as a defaults-only update when its last subscriber left:maxAgefell back toMAX_SAFE_INTEGER, which differed from what had been sent. Rust and the JS lite subscriber never send that update. The JS IETF subscriber now matches them.Changes
JS (
js/net):ietf/subscriber.ts: with no subscriber left, the update loop sends nothing (as on lite and in Rust).ietf/subscriber.ts: an update that fails to encode now closes the track with the error and logs a warning, instead of being silently dropped. Encoding happens before the write (new internalMessage.frame), so only a failed write still means "the publisher stopped reading".ietf/subscriber.ts: on drafts 14 and 15 that failure also closes the session with an internal error. The update already spent a request ID, and those drafts require the next ID, so every later request would stall behind it. Rust's legacy header path does the same (a_failed_legacy_header_closes_the_session). Drafts 16+ accept a skipped ID, so they keep the session.subscribe_priority.test.ts: three new cases, each run on every IETF ALPN (drafts 14 to 22, which also covers the draft 14-16SubscribeUpdateencoding):Writer.writeyielding a host task (the worst case of the write budget) and a mocked clock, the last subscribers leave while an update is being answered. The session must stay up. On the parent of fix: m0 interop audit #6255 this fails 6/6 on drafts 17+ every run; it passes on this branch.Rust (
rs/moq-net): the same split write existed in about 40 IETF control-message call sites:varint(T::ID).await?thenencode(&msg), orbuffer_varint(T::ID)?thenbuffer(&msg). It never tore a message in practice, because every error path resets the stream. But a failed encode, or a write future dropped between the two calls, left a bare type ID for the next write or FIN to complete. Some places could reach that:TrackStatusOk's write is dropped mid-way on a grant revocation.PublishDoneignores its errors, then closes the stream.Writer::buffer_message(id, &msg)now buffers the type and the body together, all or nothing, andencode_messageis built on it. Every production site uses one of the two.writer.rs's partial-encode test covers the type too. Control-byte stats still count the body without its type selector, as main's stats require; the adapter's stats test now writes throughencode_message, so it pins that.a_failed_legacy_header_closes_the_sessionused to force its header to fail through the split: with a 0-byte stream buffer, the body waited behind the type while the peer stopped the stream. A whole message can no longer fail that way on a fresh in-memory stream. The header now fails because its token is too big for the u16 length, after the stream opened and the request ID was spent. The assertions are unchanged: drafts 14/15 close the session, and draft 19 offers the namespace again.The union renewal test still runs only on drafts 17 and 22. Drafts 14-16 have no grant, so the scenario does not exist there. The new tests cover the legacy encoding instead.
Public API / wire
Message.frameis internal tojs/net/src/ietf, andWriteris crate-private in Rust.Checks
just checkpasses after merging main.just test interop --all. The wire format did not change. The only behavior change on the wire is one redundant update the JS subscriber no longer sends.Decisions
Settled while iterating, on the recommended option:
Follow-ups (not in this PR)
Writer.write's write budget can still yield between the multiple writes of a data-stream object header and its payload. Those streams are reset rather than FINed on teardown, so this was left alone. Worth an audit if a truncated object ever shows up.🤖 Generated with Claude Code
https://claude.ai/code/session_01EeyiEy5FWz3z44DYZVo7vU