Skip to content

feat(lite): carry refresh demand through relays - #6030

Merged
kixelated merged 17 commits into
mainfrom
quest/m1/refresh/lite
Oct 20, 2026
Merged

kixelated merged 17 commits into
mainfrom
quest/m1/refresh/lite

Conversation

@kixelated

@kixelated kixelated commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Refresh ratios and wanted groups stop at a lite session, so a late join behind a relay cluster cannot request fresh content from the origin.

Approach

Carry Refresh in TRACK_INFO and Wanted Group in SUBSCRIBE_UPDATE on lite-07. Each hop judges joins against its own cache.

Wanted Group is plain subscription state, like the other SUBSCRIBE_UPDATE fields:

  • A relay sends an update when Demand::wanted asks for a new group past its largest. It restates that value in every later update, indefinitely, including after the group arrives. Joins asking for the same group coalesce into one update.
  • A receiver keeps the latest update. A value past its largest counts as a join under its ratio, and repeating it just repeats the pending ask. A value at or below its largest is ignored.
  • A new SUBSCRIBE is itself a join, and a new upstream stream starts with no wanted group.

Rust and JS gate both fields on a default-forward has_refresh / hasRefresh version helper, matching the other lite-07 features. A nightly benchmark sweeps retained groups x unrelated tracks x joined readers.

This completes the lite wire quest, so the PR deletes it and its references in the refresh epic, the lite-07 finalize quest, and the IETF extension quest.

Impact

  • Public API: none beyond the existing track::Info::refresh and Demand::wanted. The new Version::has_refresh is pub(crate); JS hasRefresh is internal to lite/version.ts.
  • Wire: two lite-07-wip fields, TRACK_INFO Refresh (0 = none, else ratio + 1) and SUBSCRIBE_UPDATE Wanted Group (0 = none, else sequence + 1), appended after the existing fields. lite-03 to lite-06 layouts are unchanged; those peers receive no ratio and forward no wanted requests. No enum or version allocation; moq-dev/moq#6029 has since landed and only collided textually (changelog, version.rs, a JS import).
  • Draft: draft-lcurley-moq-lite.md documents both fields, the restated-state rule, and the lite-07 changelog entry. doc/concept/standard.md no longer says moq-lite hops carry no refresh requests.

Decisions

  1. Should Wanted Group be an event sent once or state restated in every update? (Maintainer's call: "Yes, make it state".)
    • ✅ State: restated in every SUBSCRIBE_UPDATE indefinitely, latest wins, stale ignored. No receiver needs to fold updates, so the earlier Request::coalesce hook and the draft's keep-the-highest MUST are gone.
    • Event: sent once until answered, receivers keep the highest. Rejected: a buffer that keeps only the latest update silently drops a request.
    • A separate refresh message. Rejected: more wire surface for no behavioral gain.
  2. Does "an older peer sees a track that never refreshes" forbid origin-local refresh on a legacy SUBSCRIBE? (Recommended option taken.)
    • ✅ No: the legacy peer sees no ratio and forwards no wanted groups, but its SUBSCRIBE is still a join at the hop that holds the ratio, per the Plan's SUBSCRIBE-is-a-join rule.
    • Yes: suppress join judgment for pre-lite-07 sessions. Rejected: per-version state in the model, and refresh would depend on who joined rather than on the cache.
  3. How to gate the fields? (Recommended option taken.)
    • ✅ A default-forward version helper, like has_epoch, so lite-08 keeps the behavior.
    • Inline matches! per site.

Validation

  • Codec tests failed before the fix, demonstrating the lost Refresh and Wanted Group.
  • refresh_joins_coalesce_upstream (Rust relay): eight concurrent joins write one SUBSCRIBE_UPDATE, the next wanted group sends the next, and a later parameter update restates the latest instead of clearing it. JS refresh.test.ts checks the same coalescing and restatement through one hop. Both fail if the restatement is removed.
  • a_subscribe_update_is_not_a_close (Rust publisher): two updates restating the same wanted group before the subscription starts still join under the ratio once it runs; after the asked-for group exists, a restated value is ignored (fails with Some(3) if the stale check is removed).
  • Rust two-relay integration test covers lite-06 and lite-07 on simulated time.
  • On 3fd4249a86: just ci exits 0 (7,734 Rust tests plus lints), just drafts check passes, just test interop --all exits 0 (74 cells, 0 fail, browser included), and the JS @moq/net check plus its dependents' tests pass (net: 2,403 tests).
  • One just ci run hit moq-cli redundant_importers_switch_through_bufferbloat_on_lite_07 ("stitching export exited"); the rerun passed it. Main already tracks that flake as quest/m1/test-flakes-2/redundant-failover-drift.md (3 of 10 failures on main), and the refresh paths are inert there since TS imports set no ratio.

Follow-ups

  • The refresh epic README still owns the end-to-end moq-json test through a two-relay cluster and the doc/concept Refresh section, after its json and video children land.
  • Cross-language refresh with a JS publisher behind two Rust relays is not covered; the epic's end-to-end test is the natural home.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 8 commits October 19, 2026 18:35
Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1-Sol <noreply@openai.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Draft checkpoint: 59618c8ecc9e13e5624c0461c61cbf6a97ec1dfd.

Implementation/default-native/interop tested head: 22b3e9d41e52b112183fe6fe803d24dd30ee19c4. The exact later delta is nonbehavioral: an async feature guard on the new integration test, then a corrected buffer_update helper doc comment. The raw production-source diff contains that comment; executable production, JS, draft and interop source are unchanged.

Validation outcomes:

  • Before fix: both new codec regressions failed, demonstrating lost Refresh and Wanted Group.
  • First broad native attempt exited100: 4,314 passed, one new stale-request fixture used the wrong simulated-clock wrapper. Wrapper corrected, then all34 new Rust cases passed explicitly.
  • Next scoped gate exited101 after 7,624 native passes/31 skips because the new integration module lacked its async guard. Guard-only correction: all34 new cases and full net-features passed, including1,035 no-default tests and minimal WASM.
  • Remaining gate phases completed individually with exit0: repository/flake checks and selected JS runtime suites (net2,232 passed/2 skipped), WASM, capture515/12 skips, play217/1 skip, uring and media-feature checks (123 base video tests plus10 vpx). No final whole-gate-zero claim.
  • Required all-interop attempt exited1: five wire preflights and32 native cells passed;13 browser matrix cells plus the refused-browser case failed with pinned Chromium's missing libglib-2.0.so.0. Separate available native subset exited0, including all32 cells, denial, FETCH and finite/drop-tail checks.
  • Drafts parsed;27 benchmark shapes passed across retained groups{1,64,1024} × unrelated tracks{1,8,32} × joined readers{1,8,32}. Timings include existing retained-cache replay, so no whole-path constant-cost or speedup claim. Benchmark is wired into nightly.
  • Final comment-only correction passed Nix formatting and diff checks. Earlier superseded checks retain their formal130 outcomes separately from the fixture failures.

No new public model API; only unpublished lite-07-wip wire fields. Legacy receives no ratio and forwards no wanted requests. Whether the Goal also forbids origin-local refresh on a new legacy SUBSCRIBE remains an unresolved maintainer interpretation; existing core behavior is preserved. JS endpoint/manual-clock coverage and Rust actual two-relay coverage are distinct. Cross-language JS-publisher/two-Rust-relay refresh acceptance and working browser coverage remain gaps.

The original quest is retained. All owned commands formally ended; the exact-owned child audit was empty. Keeping this PR draft with no ready, auto-merge, merge, or CI/review polling, as requested.

(written by GPT-6.1-Sol)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add has_refresh/hasRefresh in the existing default-forward form and use it for TRACK_INFO Refresh, SUBSCRIBE_UPDATE Wanted Group and the relay's upstream driver, which previously only matched lite-07 exactly. Add a unit test that concurrent joins send one SUBSCRIBE_UPDATE and the arrival of the requested group re-arms the next. Fix a doc comment the new codec test split from its test, simplify the JS refresh test setup, and retire the finished lite wire quest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 11 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 2303a4e2-07bb-428b-9ab2-dfa3164c2290
📥 Commits

Reviewing files that changed from the base of the PR and between 5284d96 and 3fd4249.

📒 Files selected for processing: 28
  • doc/concept/standard.md
  • drafts/draft-lcurley-moq-lite.md
  • js/net/bench/refresh.ts
  • js/net/src/internal.ts
  • js/net/src/lite/publisher.ts
  • js/net/src/lite/refresh.test.ts
  • js/net/src/lite/subscribe.test.ts
  • js/net/src/lite/subscribe.ts
  • js/net/src/lite/subscriber.ts
  • js/net/src/lite/track.test.ts
  • js/net/src/lite/track.ts
  • js/net/src/lite/version.ts
  • js/net/src/resume.ts
  • js/net/src/track.ts
  • quest/m1/ietf-extension/lite.md
  • quest/m1/lite07-finalize.md
  • quest/m1/refresh/README.md
  • quest/m1/refresh/lite.md
  • rs/moq-net/src/fuzz.rs
  • rs/moq-net/src/lite/publisher.rs
  • rs/moq-net/src/lite/subscribe.rs
  • rs/moq-net/src/lite/subscriber.rs
  • rs/moq-net/src/lite/track.rs
  • rs/moq-net/src/lite/version.rs
  • rs/moq-net/src/test_interop.rs
  • rs/moq-net/tests/integration/main.rs
  • rs/moq-net/tests/integration/refresh.rs
  • sh/gh/nightly.sh
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-20T16:43:23.767059Z 3fd4249 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 220e48cebf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rs/moq-net/src/lite/subscribe.rs
kixelated and others added 2 commits October 19, 2026 23:47
A SUBSCRIBE_UPDATE arriving before the subscription starts replaced any earlier one, so a later parameter update dropped a Wanted Group that the relay sends only once. Fold pending updates through Request::coalesce, keeping the highest Wanted Group, and say so in the draft. The pre-start update test now subscribes from a cached group, as a relay does, so the SUBSCRIBE is not itself a live-edge join and the lost request shows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7c0577c7ae

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rs/moq-net/src/lite/version.rs
kixelated and others added 2 commits October 20, 2026 00:42
The dynamic-groups note still said moq-lite hops carry no requests. lite-07 now carries the ratio in TRACK_INFO and the wanted group in SUBSCRIBE_UPDATE; earlier versions carry neither.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated and others added 3 commits October 20, 2026 09:00
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Per the maintainer: a relay restates its current wanted group in every SUBSCRIBE_UPDATE, indefinitely, and sends a new update only when a new group past its largest is wanted. Receivers keep the latest update and ignore a value at or below their largest, so restating it is harmless and no update needs folding. Drop Request::coalesce and the draft's keep-the-highest MUST, store the wanted group on the upstream SubStream in Rust, and mirror the send rule in js/net. Tests now prove a parameter update restates the wanted group, a restated value still joins before its group exists, and a stale one is ignored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Holding the wanted group on SubStream tipped Sub past clippy's variant-size threshold. There is one per track copy and it holds a live stream while anyone reads, so boxing would add an allocation per upstream subscription without saving memory; same reasoning as TrackRunState.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated added this pull request to the merge queue Oct 20, 2026
Merged via the queue into main with commit 9b52143 Oct 20, 2026
6 checks passed
@kixelated
kixelated deleted the quest/m1/refresh/lite branch October 20, 2026 18:22
kixelated added a commit that referenced this pull request Oct 20, 2026
Lite TRACK_INFO has no refresh yet, so the hop compared a sealed
DYNAMIC_GROUPS with false and refused real moq-transport tracks crossing
a cluster. The hop now forwards it as it arrived; direct moq-transport
ingress still compares it. #6030 restores the check.

The concept doc no longer claims default settings never carry the
extension's state: moq-relay offers it on moq-lite-07.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated added a commit that referenced this pull request Oct 20, 2026
TRACK_INFO now ends Refresh (#6030) then Compression on lite-07, in Rust,
JS, and the draft. Both lite-07 quests drop out of lite07-finalize.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated added a commit that referenced this pull request Oct 20, 2026
Lite TRACK_INFO now carries Refresh (#6030), so the hop restates
DYNAMIC_GROUPS like the other understood properties and drops its exception.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated added a commit that referenced this pull request Oct 20, 2026
Resolve against refresh demand (#6030), announce SYNCED (#6029) and the
ietf extension (#6058): TRACKS_ADD now carries TRACK_INFO's Refresh, the
draft shares one SYNCED section between the Announce and Tracks streams,
and the ietf draft keeps the Track Stream for listed tracks since
TRACKS_ADD has no TRACK_PROPERTIES.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant