Repository navigation
feat(quic): hold connection credit until the receiver releases it - #5675
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…batim Unmodified copies, not yet compiled, so a reviewer can diff the adaptation that follows against upstream: - quinn-rs/quinn 7616e6b2782722f3ce4a1b181ef08817d7f545e4 quinn/src -> rs/moq-tokio/src/quic/driver (lib.rs -> mod.rs), the commit moq-quic forked from. - moq-dev/noq e27c0ce00741d81c8620b98e345d4f2c88e0768c web-transport-moq/src -> rs/moq-tokio/src/quic/web (lib.rs -> mod.rs). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop every moq-noq dependency. moq-tokio's QUIC backend now drives moq-quic through quinn's async layer (`quic::driver`) with web-transport-moq folded in as `quic::web`; moq-uring's sans-IO driver moves from moq-noq-proto to moq-quic. Upstream noq stays only behind moq-tokio's `iroh` feature. - The `noq` cargo feature is `quic` everywhere it was forwarded. - `moq_tokio::noq::Error` is `moq_tokio::quic::Error`; `Error::Noq` is `Error::Quic`. - moq-quic takes `moq_clock::Instant` and raises a once-per-connection Draining endpoint event, which backs the driver's `wait_all_draining`. - qlog writes one trace per endpoint (per dial on moq-uring's connect side), since moq-quic holds one sink per transport config. - Session stats report the congestion controller's bandwidth estimate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The capsule reader maps a lost stream to UnexpectedEnd, so every ordinary session close warned "failed to read capsule". A read error while the connection is closed is now debug; one on a live connection still warns. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Relay peak RSS on the fanout workloads matches moq-noq on origin/main, which was the last open part of the Goal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A moq-quic read can now keep its connection flow control credit: `Chunks::hold` takes it as a `Credit`, and `Connection::release` returns it. Stream credit still follows reads, so a finite stream window never forces a whole message to fit. Every terminal path already returned only unread bytes, so held bytes account once whether they are released before or after the stream is reclaimed. Credit too small to clear the usual MAX_DATA batching threshold is announced once the peer is within that threshold of its limit, on release or on the next arrival. Held credit may never add up to the threshold, and the receiver ignores DATA_BLOCKED, so without this a small release could leave the peer blocked indefinitely. The tokio and io_uring drivers opt in per receive stream with `hold` and return bytes with `release`; dropping the stream releases the rest. io_uring's read-ahead is not releasable by the application and returns on stop. `ConnectionStats::credit` reports held and still-available credit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Quest outcome: the receive-credit seam is in, the tests pass, and the PR stays a draft stacked on #5664. Open decisions for the maintainer, each with a recommendation:
Follow-ups:
(Written by Claude Opus 5.5) |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…iter/receive-credit
Carry #5638's steering changes onto the renamed moq-quic drivers: the shard CID comes from moq_sock::shard::cid, and moq-uring stops greasing the QUIC bit on sharded endpoints. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace RecvStream::hold/release(bytes) in both drivers with a copying poll_read_held that yields a driver Credit for exactly the bytes read. Credit splits and merges, and dropping it returns the connection credit, so over-release and cross-connection release cannot be written. io_uring's poll_closed read-ahead now keeps its credit until the application reads it: a plain read returns it, a held read takes it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Endpoint::close now refuses attempts still queued for accept, and wait_all_draining waits for accepts partway through Incoming::accept, so a server shutdown cannot release the socket before those peers hear back. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…iter/receive-credit
The server socket retried every failed recv, so a persistent error spun the endpoint task while tokio kept readiness asserted. Only WouldBlock retries now, matching the driver's tokio runtime. Also cap moq_quic at info like noq was, fix the driver doc example, and point the moq-uring README at the just recipe. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…iter/receive-credit
Match the tokio driver and quinn: credit is read as stats().credit in both drivers. The inherent stats() shadows the poll::Session trait's on concrete calls, so the web test reads UDP bytes from it directly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # quest/m1/quic/receive-credit.md # rs/moq-tokio/src/quic/driver/connection.rs # rs/moq-tokio/src/quic/driver/mod.rs # rs/moq-tokio/src/quic/driver/recv_stream.rs # rs/moq-tokio/src/quic/driver/tests.rs # rs/moq-uring/src/quic/driver/stream.rs # rs/moq-uring/src/quic/mod.rs
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 888523e47b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 26 minutes. View limit details
✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
# Conflicts: # quest/m1/quic/README.md # quest/m1/quic/receive-credit.md
A Credit from one connection released into another succeeded whenever the other held at least as many bytes, silently moving receive window between them. Each StreamsState now carries a process-unique ConnectionTag that Chunks::hold stamps on its Credit; release and merge panic on a foreign one. An empty default Credit adopts the tag of the first credit merged in. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@codex review (Written by Claude Opus 5.5) |
|
Codex Review: Didn't find any major issues. You're on a roll. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
Merging at
(Written by Claude Opus 5.5) |
Checkpoint now carries the connection tag #5675 added for Credit, next to the streams' generation. A checkpoint from another connection panics, as a foreign Credit does, even where that connection's stream of the same id holds as much data, which the old offset assert let through. A stale generation on the right connection, from rejected 0-RTT, still fails with ClosedStream since that is a legitimate race. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in receive credit (#5675). Its ConnectionTag now also tags the send group and domain tokens, so moq-quic refuses a token from another connection as its docs promised, instead of indexing whichever of its own groups shares the slot. The drivers' pointer checks became redundant and are gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Completes
quest/m1/quic/receive-credit.md(deleted here) and unblocksquest/m1/quic/receive-memory.md, which gets a short note on the seam it should finalize.What
Native receivers can read stream bytes without returning connection credit until they drop an owned
Credithandle. Stream credit (MAX_STREAM_DATA) still advances on reads, so a stream window smaller than a message keeps it flowing while the connection window (MAX_DATA) stays spent.moq-quic
Chunks::hold() -> Credittakes the connection credit for the bytes read so far;finalizethen returns only stream credit.Connection::release(Credit) -> ShouldTransmitreturns it.Creditis a move-only token (len,is_empty,split,merge), so double release cannot be written. Each connection stamps its credit with a process-unique, crate-privateConnectionTag, andreleaseandmergepanic on a credit from another connection (an emptyCredit::default()adopts the tag of the first credit merged in). Releasing more than the connection holds also panics.ConnectionStats::credit: CreditStats { held, available }: bytes held by the application, and bytes the peer may still send (including credit not yet advertised). Together they separate held memory from bytes that can still arrive.diff > 0and the peer is within the batching threshold (window/8) of the last limit it was told. This is checked on release/read and on each arrival. Without it, a release below the threshold never goes out: held credit may never add up to window/8, and the receiver ignores DATA_BLOCKED. This applies to the plain read path too. The extra frames only appear when the receiver is the bottleneck.moq-tokio driver / moq-uring driver (same shape in both)
RecvStream::poll_read_held(cx, dst) -> Poll<Result<Option<Credit>, _>>is the copying read moq-net uses, except it yields the driver'sCreditfor exactly the bytes copied intodst(credit.len()is the count), orNoneat FIN. tokio also hasasync read_held. Plain reads are unchanged and return credit as before.Creditwrapsmoq_quic::Creditwith its connection:len,is_empty,split(n),merge(other). Dropping it releases it, even after the stream or connection is gone.mergepanics across connections. So over-release, release to the wrong connection, and leaks are all unwritable (short ofmem::forget), and the moq-quic panic is unreachable from the drivers.poll_closed's read-ahead keeps its credit until the application reads those bytes, as if they were still in moq-quic's reassembly buffer. A plain read returns it as it drains the backlog; a held read moves it into the returnedCredit;stopand drop return the rest.Connection::stats() -> moq_quic::ConnectionStats, matching the tokio driver and quinn; both drivers read credit asconn.stats().credit, which counts outstanding driverCredits and io_uring read-ahead as held. On io_uring this inherentstats()shadows thepoll::Session::statstrait method on concrete calls (call the trait by path for theweb_transport_trait::Statsshape); tokio avoids that because its driver and WebTransport session are separate types.No callbacks, no sender-side shedding, no victim selection. Those belong to receive-memory.
The seam receive-memory builds on
A reader buffer keeps one merged
Creditnext to its bytes: eachpoll_read_heldappends to the buffer andmerges the credit. When a complete unit ofnbytes is parsed and handed off or discarded,drop(credit.split(n)); read-ahead stays in the remainder and is attributed to the next unit. Cancellation, errors, and teardown release by dropping the buffer. What is left to decide there is how moq-net'stransport::poll::RecvStreamcarries it (likely an associatedCredittype with a no-op for browser/iroh/WebTransport).Settled decisions
hold()plusrelease(bytes)by countCredithandles returned by each held read (poll_read_held -> Option<Credit>), withsplit/mergecovering partial consumption. Over-release cannot be written.Credit:#[must_use]token that leaks on dropCredit:moq_quic::Creditand release throughConnectionpoll_closedread-ahead (new, recommended and taken by the agent):hold()mode and itsunheldbookkeeping, and matches how unread bytes in moq-quic already behave. Plain streams now advertise read-ahead bytes'MAX_DATAwhen the application reads them rather than whenpoll_closedreads ahead (at most 64 KiB per watched stream).hold()modeMAX_DATArule on ordinary reads.moq_quic::Checkpoint(feat(quic)!: reliable reset from a send-stream checkpoint #5690) can reuse it.poll_read_heldseparate for now; receive-memory folds it intopoll_read.Credithandle:Connection::credit()forConnection::stats() -> moq_quic::ConnectionStats, matching the tokio driver and quinn; read credit asstats().creditin both driverscredit_stats()credit()Public API / wire impact
moq_quic::{Credit, CreditStats},Chunks::hold,Connection::release,ConnectionStats::credit.moq_tokio::quic::driver::Credit(len,is_empty,split,merge, release on drop),moq_tokio::quic::driver::RecvStream::{read_held, poll_read_held}.moq_uring::quic::Credit(same methods),moq_uring::quic::RecvStream::poll_read_held,moq_uring::quic::Connection::stats.hold/releaseon the driverRecvStreams.transport::poll::RecvStreamtrait and the WebTransport wrappers (moq_tokio::quic::web,moq_uring::quic::web). The consumer decides that shape.MAX_DATAframes when a peer is close to its limit, and on io_uring, read-ahead bytes'MAX_DATAwaits until the application reads them. No MoQ message or negotiation changes. Browser and iroh are untouched.Tests
moq-quicstate tests:held_credit_returns_once(FIN / stop / reset / stop-then-reset with a gap and late data, release after reclamation),held_credit_pays_shrink_debt,release_from_another_connection(panics even when the other connection holds enough to absorb it; fails without the tag check),merge_from_another_connection(panics),default_credit_adopts_its_connection.moq-quicsimulator:held_credit_spends_connection_window_only. A 1000 B stream window delivers a 4000 B connection window while it is held. A 100 B release (threshold 500) unblocks the peer, and so does one queued while the peer still had room, with no read in between. Both halves fail without the newMAX_DATArule.moq-tokiodriver:held_reads_spend_the_connection_window(real sockets): held reads spend the window, dropping a 100 Bsplitunblocks, and credit outlives its stream until dropped.moq-uringtests/credit.rs: the same flow, plus read-ahead keeping its credit, a plain read returning it, a held read taking it,stopreturning only the read-ahead, and credit outliving the stream. Passed underMOQ_STRICT=1.benchmark_receive_credit(ignored; in the Rust nightly group) sweeps 1/64/4096 streams × 1200 B/64 KiB reads. Held and plain reads cost the same within noise (43-73 ns per read with the connection tag, on a loaded shared box), flat across stream counts. It measures moq-quic; the driverCreditadds oneArc/Rcclone per held read and one connection lock per non-empty drop, whichrelease(bytes)also took.just check: all 4695 tests pass after merging the latestmain, lint clean.just test interop --all: there is no wire, ffi, or gateway change.Not covered
RESET_STREAM_AT) waits until that quest lands. A reliable prefix would be held like any other read; its final-size gap goes through the existing reset accounting.🤖 Generated with Claude Code
(Written by Claude Opus 5.5)