Skip to content

perf(net): serve whole cached groups from a parked track's copy - #6286

Merged
kixelated merged 6 commits into
mainfrom
quest/m1/release-perf/fetch-fanout
Oct 23, 2026
Merged

kixelated merged 6 commits into
mainfrom
quest/m1/release-perf/fetch-fanout

Conversation

@kixelated

@kixelated kixelated commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

When many viewers FETCH the same group through a relay over moq-lite, the relay already sends one upstream FETCH per group. Each viewer's FETCH still revived the relay's parked logical track, though: the front saw demand, queried the source copy, spliced it, and parked it again once the group was read. The copy's cache already held the whole group, so all of that work was accidental.

Approach

  • When the front parks a track, the route keeps a lazy handle to the copy it let go of. It is not demand, so the copy still releases its route's demand once idle.
  • track::Consumer::fetch_group serves a group straight from that parked copy when the copy holds it whole: finished, starting at frame 0, on a copy that is neither closed nor aborted. The logical track is never revived, so the front never wakes. Nothing more feeds a whole group, so no route has to stay up while it is read.
  • Anything less takes the normal route path, unchanged. That covers open, headless, aborted, and expired (evicted) groups, and groups on a failed or closed copy. The caller's frame start, age cap (drift check), fetch count, and egress meter apply as before, because the hit resolves through the same Fetching as a direct cache hit.
  • The fetch joins the copy's demand while it resolves, so a copy kept busy only by fetches stays cached at its session.
  • The shortcut is dropped when the front serves another copy, when any source detaches (replacement or close), and when the track ends, concludes, or is forgotten. A route leaving can't leave a stale copy serving.
  • Tests:
    • A parked track serves a whole cached group without waiting on the front, while an open group still waits for the route.
    • A new source takes over a parked track: a group only the old copy holds is asked of the new route, not served from the old copy.
    • Both tests fail without the change.
    • The one-FETCH-per-group regression test (lite-05/06/07, 64 sequential viewers) and the bench fixture assertion are kept.
  • The held-query control and the session_delivery_fetch_demand group are dropped. The normal /fetch sweep now amortizes. The control's remaining advantage is per-viewer client work that no ownership-preserving change can remove (see below). With the control gone, the group's normal half duplicates session_delivery_group_cost's /fetch cases.
  • The quest is retired. The accepted residual is recorded in the release-performance epic, as the quest's plan asked.

Measurements

These are callgrind instruction counts, because the shared host was too loaded for stable Criterion timings (load average around 30 on 24 cores). Each cell is the instructions per viewer FETCH for moq-lite-06, one relay, and the default 1 MiB window. I ran each case for 1 s and for 4 s under --profile-time and divided the instruction difference by the difference in fetches, so setup cancels out. "Before" is this branch's merge of main without the change. The held-query control is the dropped diagnostic, measured on the same base.

Publishers Viewers Before After Change Held-query control
1 1 333k 335k +0.4% 332k
1 16 271k 225k -17% 164k
1 64 258k 211k -19% 155k
16 1 344k 345k +0.4% 343k
16 16 490k 491k 0% 326k
16 64 314k 277k -12% 195k

From 1 to 64 viewers, per-viewer cost now falls 37% with one publisher, against 23% before. At 16 publishers and 16 viewers each viewer watches its own broadcast, so no group is shared and nothing changes. Single-viewer rows are within noise.

The rest of the gap to the control is the viewers' own work. A viewer's fetch of a group it lacks is a miss on its own copy. That miss revives the viewer's logical track so the open group keeps its route and can fail over mid-read, and the viewer's copy releases its route's demand once idle. That cost is per viewer, so it cannot shrink as viewers grow. A viewer that holds a query avoids it, which is all the control did.

Impact

  • Public API: none. The new functions are private.
  • Wire: none.
  • Consumer: every relay serving FETCH from a remote route (moq-relay, through the lite and IETF publishers' fetch_group).

Decisions

Settled while iterating. The recommended option was taken for each one, since this ran unattended:

  1. What a parked track serves without reviving it
    • ✅ Only whole cached groups. An open group needs its route and logical track for mid-read failover.
    • Every fetch through the parked copy, misses included. Rejected: an open group would lose failover and keep the route up without the front knowing.
  2. Invalidating the shortcut
    • ✅ Drop every parked shortcut when any source detaches. That event is rare, and the check is cheap.
    • Track which source each parked copy came from. Rejected: more state for the same outcome.
  3. Held-query control
    • ✅ Drop it now that the /fetch sweep amortizes. Record what remains in the epic.
    • Keep it as a permanent control.
  4. Remaining gap to the control
    • ✅ Accept it. It is the viewer-side ownership and failover cost, recorded in the release-performance epic.
    • Keep optimizing. No ownership-preserving design removes a miss's revival.

Validation

  • cargo nextest run -p moq-net after merging the latest main: 3034 passed, 11 skipped.
  • just check: the Rust and quest modules pass. The flake module fails only in the runner-image build, because this host cannot create a user namespace (unshare: write failed /proc/self/uid_map: Operation not permitted). That failure predates this branch.
  • quest check --base origin/main --branch quest/m1/release-perf/fetch-fanout passes, with 598 documents.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

@kixelated kixelated changed the title FETCH fan-out test(net): isolate FETCH fan-out demand cost Oct 22, 2026
Co-Authored-By: GPT-6 <noreply@openai.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Checkpoint pushed: the 64-viewer lite-05/06/07 regression proves one upstream FETCH per group and one TRACK total. Three paired CPU rounds isolate demand lifecycle overhead: at one publisher/64 viewers, normal FETCH costs 34.5 us/viewer versus 19.9 for the held-query diagnostic control. No production optimization or API/wire change is retained; the quest remains open and this PR remains draft with the ownership decision under Decisions needed.

Verification: all 36 benchmark smoke cases, 471 quest documents, lints, and capture/playback (511) plus CLI playback (220) tests pass. The default suite passes 7,956/7,957; its sole failure is the base stats readiness issue fixed separately in #6290. just ci also hits the host's user-namespace restriction while building the Nix runner image; remaining selected checks were continued directly.

(Written by GPT-6)

kixelated and others added 3 commits October 22, 2026 15:26
…4d23c593d92

# Conflicts:
#	rs/moq-net/benches/session.rs
A relay revived its parked logical track for every viewer's FETCH, even
when the parked copy already cached the whole group. A finished group
needs no route kept up, so serve it straight from that copy without
waking the front. Open, headless, aborted, and failed-copy groups still
take the route. A source detaching drops the shortcut.

Drop the held-query bench control and retire the FETCH fan-out quest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title test(net): isolate FETCH fan-out demand cost perf(net): serve whole cached groups from a parked track's copy Oct 22, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review October 22, 2026 23:32
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-22T23:35:47.261263Z 9d83886 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 49 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: de391474-b7ba-4ade-965f-93571c1e17a6
📥 Commits

Reviewing files that changed from the base of the PR and between bc03da5 and 7763e8a.

📒 Files selected for processing: 8
  • quest/m1/release-perf/README.md
  • quest/m1/release-perf/fetch-fanout.md
  • rs/moq-net/benches/session.rs
  • rs/moq-net/src/model/origin.rs
  • rs/moq-net/src/model/resume.rs
  • rs/moq-net/src/model/track.rs
  • rs/moq-net/tests/integration/fetch_fanout.rs
  • rs/moq-net/tests/integration/main.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Hooray!

Reviewed commit: 9d83886d79

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@kixelated
kixelated added this pull request to the merge queue Oct 23, 2026
Merged via the queue into main with commit facb474 Oct 23, 2026
9 checks passed
@kixelated
kixelated deleted the quest/m1/release-perf/fetch-fanout branch October 23, 2026 03:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant