Skip to content

perf(moq-net): store large frames as sealed segments - #6628

Merged
kixelated merged 15 commits into
mainfrom
quest/m1/unbounded-groups/segmented-frames
Oct 26, 2026
Merged

kixelated merged 15 commits into
mainfrom
quest/m1/unbounded-groups/segmented-frames

Conversation

@kixelated

@kixelated kixelated commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Finishes quest/m1/unbounded-groups/segmented-frames.md.

Problem

A completed frame was one contiguous Bytes, and a declared-size frame preallocated its whole declared size (up to the session Budget). Its QUIC receive credit came back only once the whole frame landed, so the connection window had to fit the largest frame. Front eviction and byte holes need frame storage that the cache can account for and evict in pieces.

Approach

  • frame::SEGMENT (1 MiB, crate-private) is the storage unit. A frame larger than one segment that arrives in pieces is stored as Payload::Chain: sealed segments of exactly SEGMENT bytes (the last shorter), so segment i always holds bytes from i * SEGMENT. Smaller frames, and any frame written whole, stay one Bytes (Payload::Whole, same size as Bytes).
  • Receiving allocates only the segment being written, whatever size was declared. Budget reserves min(size, SEGMENT) per frame.
  • coding::Reader stops each lease chunk at the open segment's seal and releases the lease's credit there, so an in-flight frame holds at most one segment of the receive window. The frame header is still read as one unit.
  • Readers stream chains segment by segment with no copy (poll_read_chunk, wire egress through Payload: Buf). A whole-frame read (read_frame) copies a chain into one buffer, out of line. The group prefetch keeps chains in a separate side table, so reading small frames does the same work as before.
  • Merged onto main's lock-free group log (moq-dev/moq#6782): a log Entry stores the Cached frame, and both prefetch fill paths (the lock-free log and the locked state) set chains aside.
  • Benches: frame_recv_large sweeps 4 to 32 MiB frames over 1 and 4 streams. memory.rs adds a 4 MiB size and skips cases that would write more than 512 MiB per round.

Measured on a loaded shared host (load ~20 to 100), min of 2 runs:

  • frame_recv_large (before the log merge), budget mode: 4 MiB x 1 stream 2.26 ms -> 0.48 ms, 16 MiB x 1 10.4 -> 5.8 ms, 32 MiB x 4 111 -> 61 ms. frame_recv at 16 to 256 KiB stayed within noise.
  • group_read_frames (after the log merge) against main: 512 to 8192 frames per group sit at 0.81 to 1.12x, within noise on this host.
  • memory allocations per frame: unchanged count, +8 bytes.

Impact

  • Public API: none. frame::SEGMENT, Payload, Chain, and Cached are crate-private, and transport::MIN_RECEIVE_WINDOW keeps its value: the 64 MiB announcement term already exceeded the old 32 MiB frame term, and now outweighs the 1 MiB segment term.
  • Wire: none.
  • Behavior: a frame larger than the receive window now completes rather than stalling the window (lite_frame_of_many_windows_completes), provided the window fits one segment.

Choices made without asking

  • Segment size 1 MiB, from the frame.rs sweep. Smaller segments mean more allocations per byte, larger ones a bigger allocation and more credit held ahead of the data. Most media frames fit in one segment, so a whole read of them never copies.
  • A frame written whole (one write_frame, or one write or transport chunk carrying all of it) stays one Bytes at any size instead of being split. Byte holes will have to slice or copy those frames; that is recorded in byte-holes.md.
  • A whole read of a chained frame copies it once, since frame::Frame::payload is a public Bytes. A public chunked payload type would be an API change, so it is out of scope here.

Alternatives

  • Store every frame as a chain, even small ones: this costs an indirection and an Arc<[Bytes]> per frame on the common path.
  • Expose chains through frame::Frame: avoids the copy on a whole read of a huge frame, but breaks the public API in every binding.

Checks

just ci passes locally. It was run with the schedule-bench fix from moq-dev/moq#6824 applied but not committed, because benches/schedule.rs is broken on main after moq-dev/moq#6809.

Follow-ups

  • moq-dev/moq#6824 has to land for just check on moq-net to pass on main.
  • Byte holes and front eviction build on this. Front eviction's Required section is now empty.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 14 commits October 24, 2026 00:16
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A frame larger than one 1 MiB segment that arrives in pieces is kept as a
chain of fixed-size sealed segments instead of one buffer of its declared
size. Receiving it allocates at most the segment being written, and the
wire ingest returns receive credit as each segment seals, so a frame never
needs a matching connection window. Frames within a segment, and whole
writes of any size, keep their single buffer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ports segmented payloads onto the lock-free group log: a log Entry stores a Cached frame, and the prefetch sets chains aside on both fill paths.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Segmented frames perf(moq-net): store large frames as sealed segments Oct 26, 2026
@kixelated
kixelated marked this pull request as ready for review October 26, 2026 00:30
@coderabbitai

coderabbitai Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 3 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 6dd733fd-964e-471f-9c0a-f3ee8ab9b57a
Commits

Reviewing files that changed from the base of the PR and between 09da543 and c5a88d0.

Files selected for processing: 17
  • quest/m1/unbounded-groups/README.md
  • quest/m1/unbounded-groups/byte-holes.md
  • quest/m1/unbounded-groups/front-eviction.md
  • quest/m1/unbounded-groups/segmented-frames.md
  • quest/m2/qmux-receive-lease.md
  • rs/moq-net/benches/frame.rs
  • rs/moq-net/benches/memory.rs
  • rs/moq-net/src/coding/reader.rs
  • rs/moq-net/src/ietf/publisher.rs
  • rs/moq-net/src/lite/ietf.rs
  • rs/moq-net/src/lite/publisher.rs
  • rs/moq-net/src/model/frame.rs
  • rs/moq-net/src/model/group.rs
  • rs/moq-net/src/transport.rs
  • rs/moq-net/tests/integration/main.rs
  • rs/moq-net/tests/integration/segmented_frame.rs
  • rs/moq-tokio/tests/integration/receive_credit.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-26T02:28:24.726037Z c5a88d0 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 7ed616d

No actionable bugs found in this review. I checked segment allocation/sealing and reader offsets, receive-credit release boundaries, whole/streamed reads through prefetch, compression adaptation, and the lite/IETF egress changes against the current base. There are no prior published findings to carry forward.

Direction: keeping small and whole-written frames as Bytes, with chains only for large incremental writes, is a reasonable way to bound receive-side allocation without changing the public API or wire. The prefetch side table adds complexity, but preserves the common small-frame path; making every frame a chain would impose that cost universally. Coalescing only at whole-frame/compression boundaries is a justified compatibility tradeoff.

Verification limits: static GitHub code and test inspection only; I did not run builds, tests, benchmarks, or concurrency checks. The added segment-boundary, mixed-prefetch, protocol round-trip, and multi-window tests cover the principal changed paths, but their execution and reported performance remain unverified here.

@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: c5a88d07df

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@kixelated

Copy link
Copy Markdown
Collaborator Author

Landing summary: merged main onto the lock-free group log (log entries store the segmented Cached frame, both prefetch fill paths set chains aside), gated three async-only items for the no-async build, and confirmed just ci passes locally on c5a88d07df with the schedule bench fix from main. The OpenAI review (on 7ed616d) and the Codex review (on c5a88d07df) found no issues; CodeRabbit was rate-limited. Choices made without asking are listed in the PR body: 1 MiB segments, frames written whole stay one Bytes, and a whole read of a chain copies it once.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 26, 2026
Merged via the queue into main with commit f6d0ace Oct 26, 2026
10 checks passed
@kixelated
kixelated deleted the quest/m1/unbounded-groups/segmented-frames branch October 26, 2026 05:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant