Skip to content

feat(net): expose native stream occupancy and remaining peer opening credit - #5982

Closed
kixelated wants to merge 10 commits into
mainfrom
feat/transport-stream-stats
Closed

kixelated wants to merge 10 commits into
mainfrom
feat/transport-stream-stats

Conversation

@kixelated

@kixelated kixelated commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Transport users cannot tell locally initiated stream occupancy apart from exhausted peer stream-ID credit. Cumulative MAX_STREAMS is not a fixed concurrent capacity, so neither number can be derived from the other. A metrics endpoint that holds one long-lived session (a stats collector subscribed to many tracks, for example) needs both to graph its stream headroom; MoQ handle counts under-report it, since setup and track-info requests hold streams too and a stream stays open until the transport retires it.

Approach

Forward an optional structured snapshot through moq-net's existing session sampler.

  • moq-quic keeps a constant-time count of locally initiated streams until full protocol retirement: a bidi stream counts until both halves retire, and a FIN or reset counts until acknowledged. Rejected 0-RTT resets the counts; a closed connection reports zero.
  • available is the peer's cumulative limit minus the next stream ID, so closing a stream never restores it on its own.
  • The native QUIC sessions in moq-tokio and moq-uring (raw QUIC and WebTransport) hand moq-quic's snapshot to moq-net's transport::Stats::streams() through each crate's sealed Grouped hook, next to wire(). Nothing goes through web-transport-trait, and no moq-dev/web-transport change or patch is needed.
  • Every other backend reports None, the trait default: qmux (WebSocket, TCP, WebRTC) and iroh in moq-tokio, and the browser backend behind the moq-ffi and moq-wasm adapters.
  • rs/moq-quic/README.md lists the stream count among the fork's carried changes, and the JS Stats doc names streams as native-only.

Impact

  • moq-net (additive): transport::Streams { bidi, uni }, transport::StreamState { open, available }, transport::Stats::streams() -> Option<Streams> with a default None, and a session::Stats::streams field (the struct is #[non_exhaustive]). The moq-net structs stay exhaustive because backend adapters in other crates construct them.
  • moq-quic (additive): StreamStats, StreamState (both #[non_exhaustive], like their sibling stats), and a ConnectionStats::streams field.
  • moq-tokio, moq-uring: none. The new Grouped::streams hook is on a sealed, unnameable trait.
  • moq-ffi, moq-c, language bindings: none. MoqConnectionStats does not gain the snapshot until a binding consumer needs it.
  • Wire: none. doc/concept/stats.md is unchanged because the stats tracks do not carry the snapshot.

Alternatives

Settled during /quest-iterate, picking the recommended option:

  1. How moq-quic's snapshot reaches moq-net. Maintainer, 2026-10-20: moq takes no new work in, or dependency on, moq-dev/web-transport; moq-quic powers qmux and WebTransport (moq-dev/moq#6083).
    • ✅ Through the sealed Grouped hook moq-tokio and moq-uring already use for wire() and send groups. Native QUIC keeps the whole point of the PR; only the out-of-tree qmux crate loses it, and it gains it when qmux moves onto moq-quic (quest/m1/quic-qmux.md). When quest/m2/drop-web-transport-trait.md deletes the adapters, the hook body moves into each native session's own transport::poll impl.
    • Implement moq-net's transport::poll directly on the native sessions first. Rejected: that is the [L] drop-web-transport-trait quest, far past this PR.
    • Forward web_transport_trait::Stats::streams() (moq-dev/web-transport#440, now published as web-transport-trait 0.5.1 and qmux 0.6.4) so qmux reports too. Rejected by the maintainer.
  2. Exposing the snapshot through moq-ffi's MoqConnectionStats:
    • ✅ Not yet: no binding consumer, and every binding would need the same addition.
    • Add it now across rs/moq-c and the bindings.
  3. #[non_exhaustive] on the new moq-quic stats structs:
    • ✅ Yes, matching CreditStats and the other moq-quic stats, which only moq-quic constructs.
    • No, leaving them constructible downstream.

quest/m1/rs-request-credit.md stays consistent: its credit check is a non-waiting open, not this stats snapshot, and its native path already asks moq-quic directly.

Follow-ups

Validation

  • Regression tests: moq-quic stream-state unit tests for occupancy and non-recycled credit; the moq-tokio and moq-uring adapter tests assert the Grouped::streams hook reaches Stats::streams(); moq-tokio's native QUIC and WebTransport integration tests assert the server session reports Some streams.
  • just ci on 286c8fb8 (merged with main at 2e7473d1): exit 0, 8,477 Rust tests passed plus every other selected check. On 3e0353e6 (a JS doc comment on top): fail-fast stopped at 7,095 of 8,477 on one failure, export::redundant_importers_switch_through_bufferbloat_on_lite_07, is the known flake in quest/m1/test-flakes-2/redundant-failover-drift.md (fix open in moq-dev/moq#6113) and passed rerun once alone.
  • just test interop --all was not run: no wire, moq-ffi or gateway change.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 2 commits October 19, 2026 11:54
Co-Authored-By: GPT-6 <noreply@openai.com>
Co-Authored-By: GPT-6 <noreply@openai.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Checkpoint af762c71c73fc38b7ab98a7ba444371e93d4dcba is clean and pushed. Full fix and focused backend/adapter/session regressions pass. Full check passed8,852 Rust tests (35skipped), native/WASM/docs/features, language bindings, OBS/Nix and selected JS, then failed one local Python timer fixture (100others pass).

One characterization catches Protocol("old") at simulated t=1, then reads b"later" successfully. This matches the unchanged native Ordered contract for an expired handed-out group. The test and local data/clock code are unchanged from base2e1; no baseline rebuild or full rerun is claimed. Update the stale expectation in a separate test-fix PR. No transport/clock behavior was weakened here.

Required all-language interop is NOT RUN. Matching shared/QMux implementation is web-transport440 at13bd407, with full fix/check/test and real Rust/TS WebSocket interop passing. Compatible publication/adoption remains pending. No package release, version bump, deployment or merge occurred.

(written by GPT-6)

@kixelated kixelated changed the title Expose native stream occupancy and remaining peer opening credit feat(net): expose native stream occupancy and remaining peer opening credit Oct 20, 2026
kixelated and others added 4 commits October 19, 2026 17:29
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hook

Drop the web-transport-trait Stats::streams plumbing and its git patch on
moq-dev/web-transport#440. Native QUIC sessions in moq-tokio and moq-uring
now hand moq-quic's snapshot to moq-net through the crate-private Grouped
hook, next to wire(). The FFI and WASM adapters only wrap the browser
backend, which has no counts, so they are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review October 20, 2026 20:33
@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 8d004427-248a-4953-bcdc-00b8671b7548
📥 Commits

Reviewing files that changed from the base of the PR and between 2e7473d and 286c8fb.

📒 Files selected for processing: 15
  • rs/moq-net/src/client.rs
  • rs/moq-net/src/session.rs
  • rs/moq-net/src/transport.rs
  • rs/moq-quic/README.md
  • rs/moq-quic/src/connection/mod.rs
  • rs/moq-quic/src/connection/stats.rs
  • rs/moq-quic/src/connection/streams/state.rs
  • rs/moq-quic/src/lib.rs
  • rs/moq-tokio/src/transport.rs
  • rs/moq-tokio/src/transport/grouped.rs
  • rs/moq-tokio/src/transport/owned.rs
  • rs/moq-tokio/tests/integration/backend.rs
  • rs/moq-uring/src/quic/web.rs
  • rs/moq-uring/src/transport.rs
  • rs/moq-uring/src/transport/adapter.rs

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 0 remain after this review.

Walkthrough

The change adds bidirectional and unidirectional stream occupancy and available-credit statistics to QUIC and transport APIs. QUIC tracks locally initiated streams until full retirement and reports reset values after rejected 0-RTT or connection closure. Tokio and uring adapters pass stream statistics into session snapshots. Tests cover QUIC accounting, adapter propagation, and snapshots.

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to 286c8

No confirmed issue blocks merging based on the available evidence.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 32.86% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 70 functions across 14 files. (1 skipped:… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Title check Passed The title clearly and concisely describes the main change: exposing native stream occupancy and remaining peer opening credit.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description check Passed The pull request description directly explains the added native transport stream occupancy and peer stream-credit statistics, the affected crates, implementation approach, scope limits, tests, and val…
Full details: Description check

Explanation

The pull request description directly explains the added native transport stream occupancy and peer stream-credit statistics, the affected crates, implementation approach, scope limits, tests, and validation results.

Full details: Docstring Coverage

Explanation

Docstring coverage is 32.86% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 70 functions across 14 files. (1 skipped: 1 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

✨ Finishing Touches
✨ Simplify code
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-21T00:51:21.127002Z 286c8fb Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4bff2c394d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

pub(super) recv: FxHashMap<StreamId, Option<StreamRecv>>,
pub(super) free_recv: Vec<StreamRecv>,
pub(super) next: [u64; 2],
local_open: [u64; 2],

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Add this fork change to the carried-change inventory

This adds custom stream-accounting behavior directly to the hard-forked moq-quic, but rs/moq-quic/README.md has no corresponding entry under “Carried changes.” CONTRIBUTING.md:78-79 requires every carried fork change to be recorded with its upstream PR or the reason it has none; without that entry, future Quinn cherry-pick audits can overlook this custom state-management behavior. Add the inventory entry in this change.

AGENTS.md reference: AGENTS.md:L8-L8

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 286c8fb: rs/moq-quic/README.md now lists the stream count under "Carried changes" (not offered upstream, since quinn exposes no stream counts), naming insert_local, stream_freed, and zero_rtt_rejected as the spots a cherry-pick must keep.

(Written by Claude Opus 5.5)

@kixelated

Copy link
Copy Markdown
Collaborator Author

Closing: nothing consumes the stream snapshot. It isn't published on the stats tracks or exposed through moq-ffi or the bindings, and request-credit checks ask moq-quic directly. Per AGENTS.md we keep API private until a consumer needs it. If a stats or debugging need shows up, this branch is a good starting point. (moq-dev/web-transport#440 already shipped and stays as is; it's an additive default method.)

(Written by Claude Opus 5.5)

@kixelated kixelated closed this Oct 20, 2026
@kixelated kixelated reopened this Oct 21, 2026
@kixelated
kixelated marked this pull request as draft October 21, 2026 00:37
@kixelated

Copy link
Copy Markdown
Collaborator Author

Reopening: there is a consumer now. A downstream relay operator's metrics endpoint needs to graph a long-lived session's transport stream headroom (moq_net::session::Stats::streams: bidi/uni open and remaining available credit), so stream-limit exhaustion shows up as a gauge instead of log forensics. MoQ handle counts under-report it: a subscription's setup and track-info requests hold streams of their own, and a stream stays open until the transport retires it.

The scope is unchanged: moq-net and moq-quic only, no moq-ffi or binding surface, and no stats-track or wire change. Back to draft while I merge main, add the carried-change entry Codex asked for, and rerun just ci.

(Written by Claude Opus 5.5)

@kixelated
kixelated marked this pull request as ready for review October 21, 2026 00:48

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 286c8fb8f1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rs/moq-net/src/session.rs
Comment on lines +47 to +49
/// Locally initiated stream occupancy and remaining peer stream-ID credit.
pub streams: Option<crate::transport::Streams>,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Synchronize the new stats API across packages

This adds public moq-net stream statistics only on the Rust side, while a repo-wide search shows js/net/src/connection/stats.ts and doc/concept remain unchanged even though the JavaScript Stats interface says its fields match Rust's session::Stats. Update the JS surface and concept documentation, or explicitly document the native-only unsupported field so the cross-language APIs do not silently diverge.

AGENTS.md reference: AGENTS.md:L107-L110

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partly fixed in 3e0353e: the JS Stats doc no longer claims a full match and names streams as Rust's native-only field, since W3C getStats() has no stream counts to report. Declining the rest: an always-absent JS field or a concept-doc section would publish surface no browser consumer needs, and doc/concept and the binding docs describe only what the stats tracks and moq-ffi carry, which this PR leaves unchanged.

(Written by Claude Opus 5.5)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Closing: moq-net only forwards the snapshot and nothing in moq reads it. The request-credit work needs an atomic non-waiting open, not a racy snapshot, so the one in-repo use of stream credit doesn't need this either. A single out-of-repo metrics consumer doesn't justify the new exported API across moq-quic, moq-net, moq-tokio, and moq-uring. The branch stays as a starting point if an in-repo consumer appears.

(Written by Claude Opus 5.5)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant