Skip to content

feat(net): ask a client's peer only about watched prefixes on moq-lite - #6165

Merged
kixelated merged 24 commits into
mainfrom
quest/m1/lazy-discovery/rust
Oct 22, 2026
Merged

kixelated merged 24 commits into
mainfrom
quest/m1/lazy-discovery/rust

Conversation

@kixelated

@kixelated kixelated commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Quest: Rust lazy discovery, following the newer plan in moq-dev/moq#6092 (Rust has no blind requests or answers).

Summary

A moq-lite client session now asks its peer for announcements only under the prefixes its subscribe origin's readers watch, mirroring JS connection/forward.ts:

  • The origin counts watched literal heads in its route trie (interests, interests_below) and hands each client session only the covering-set changes a watch causes, with a snapshot taken under the lock so a new reader never reads synced before the request goes out.
  • discovery.rs (crate-private) turns those changes into announce requests: a narrower watch rides a broader request, a narrower request already open is held until the broader one goes, and a request lingers 2s after nothing at or beneath it is watched.
  • Watches are announced(), follow, routed, routed_broadcast, and announced_broadcast. A broadcast resolved through a received route keeps its path watched while the front lives.
  • Reads that serve the peer (the lite and IETF publishers, IETF grant enforcement) use a crate-private serving() consumer and never count as watches, so a client that publishes and consumes through one origin stays lazy.
  • Server sessions, and clients fed into a peer() or upstream() handle, still ask for everything they may, so relays and clusters discover as before.
  • moq-lite-03/04 clients stay eager too: their announce streams mark no initial set, so a watch would read as synced before the peer answered it.
  • The WHEP handler (moq-rtc whep::accept) resolves with routed_broadcast inside its existing 5s bound, so export rtc --listen keeps working through a lazy client; no route in time is a 404.
  • moq-mux Source sibling bindings now watch the sibling path and request once that watch syncs, so cross-broadcast renditions keep resolving through a lazy client. An origin that already knows its routes syncs at once, so the request still goes out on bind.

Public API and wire impact

  • Wire: none. Same ANNOUNCE requests, just fewer of them on a client.
  • Behavior (breaking, documented in rs/moq-net/MIGRATE.md): on a lite client, request_broadcast for a path nothing watches fails Unroutable even when the peer announces it. Use routed_broadcast / announced_broadcast, or hold announced() over the prefix. A hand-rolled relay that forwards over a plain origin must feed its client session upstream() or peer().
  • Rust API: no new public items. New crate-private Interests, InterestChange, Consumer::serving, Producer::interests, Producer::relays (marked pending: /quest/m1/public-model/answer.md). The hidden fuzz module gains Interests for the bench.
  • Bindings / C: no signature change. moq_origin_request and moq-ffi request_broadcast docs now say a lite client only reaches watched paths (regenerated the Dart checksum). The Dart decode example now waits through announcedBroadcast(...).available().

Tests

  • rs/moq-net/tests/integration/lazy_discovery.rs: 11 tests over lite-05/06/07 (nothing asked until watched, only the watched prefix, a watch made before connecting, linger and rewatch, narrower rides broader, broader holds narrower, a resolved broadcast keeps its route, routed/announced waits ask, peer/upstream links and servers ask for everything, serving the peer asks nothing).
  • moq-hls a_sibling_resolves_through_a_client_session: fails Unroutable without the moq-mux fix.
  • The integration harness keeps a root announced() on the client by default (client_discovers_all), so existing tests keep their eager discovery.
  • Bench rs/moq-net/benches/interests.rs sweeps (watches, sessions) over (10, 1000)². Covered changes stay flat (~0.6-0.9 µs) on both axes; an uncovered change grows with sessions only (~6 µs at 10 sessions, ~0.7-1 ms at 1000, one snapshot per session) and is flat over held watches.

Checks

  • just check origin/main: every lint and binding stage passes (including the Dart, Go, Kotlin, Python, C++ binding checks). It stops at bench/relay.test.py test_a_foreign_listener_never_proves_relay_readiness, an unrelated busy-port hang under load fixed by moq-dev/moq#6121; it passes when run alone.
  • cargo nextest run --workspace --tests: 9299 passed, 0 failed.
  • just test interop --all: all checks passed.
  • cargo bench -p moq-net --features fuzz --bench interests: numbers above.
  • No drafts/ change: there is no wire change.

Decisions

Settled by the maintainer (2026-10-21): every recommendation below is accepted.

  1. moq-transport stays eager. IETF sessions still ask for their whole scope. Lazy SUBSCRIBE_NAMESPACE needs an UNSUBSCRIBE_NAMESPACE path on drafts 14/15, and peers still push unsolicited PUBLISH_NAMESPACE. Recommendation: keep it eager and split it into its own quest, given moq-lite takes priority.
  2. Serving the peer is not a watch. Recommendation: keep it (matches JS, where the publisher reads the table directly). The cost is that a hand-rolled relay on a plain origin must use upstream()/peer(), now in MIGRATE. Alternative: per-session self-exclusion so a client forwarding over local links propagates its peers' watches.
  3. follow folded into the request path was read as: every waiting read (follow, routed, routed_broadcast, announced_broadcast) registers interest, with no API removed. Recommendation: keep; removing follow is a separate API change.
  4. moq-mux sibling lookup waits for the watch to sync, then asks once, rather than switching siblings to routed_broadcast. That keeps Binding's contract (asks on bind, never retries, a missing sibling fails). Recommendation: keep. Alternative: move "watch, then answer" into moq-net request_broadcast itself for lazy clients, so every plain request keeps working after one round trip; that is close to the blind-answer behavior the plan rejected, so it needs your call.

Quests

Deletes the finished Rust lazy discovery quest and its now-empty lazy-discovery epic (JS landed earlier; the IETF half lives in its own m2 quest), and drops the links to them from quest/m1/README.md, quest/m1/public-model/{README,answer}.md, and the two m2 follow-ups below.

Follow-ups

Already planned on main by moq-dev/moq#6186:

🤖 Generated with Claude Code

kixelated and others added 4 commits October 20, 2026 22:01
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bench the interest fan-out over watches and sessions, watch a sibling
path before moq-mux requests it, and point docs and examples at
announced_broadcast where a client resolves a path nobody lists.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Quest outcome: started quest/m1/lazy-discovery/rust.md, following the newer plan in moq-dev/moq#6092. Lazy discovery works for moq-lite clients. Relays, servers, and peer/upstream links still ask for everything. This is a draft because four decisions are open (listed in the body):

  1. moq-transport stays eager. I recommend a follow-up quest.
  2. Serving the peer does not count as a watch. I recommend keeping it.
  3. The follow fold means every waiting read registers interest, with no API removed. I recommend keeping it.
  4. A moq-mux sibling waits for its watch to sync, then asks once. I recommend keeping it, rather than putting "watch, then answer" into request_broadcast.

The quest file stays in place until these are settled.

(Written by Claude Opus 5.5)

@kixelated
kixelated marked this pull request as ready for review October 21, 2026 16:19
@coderabbitai

coderabbitai Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 30 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: b4b775e7-666a-4439-b0ec-2f735bf75e36
📥 Commits

Reviewing files that changed from the base of the PR and between a2ebd8e and d6cbfc4.

📒 Files selected for processing: 50
  • cpp/moq/MIGRATE.md
  • dart/moq/MIGRATE.md
  • dart/moq_ffi/MIGRATE.md
  • dart/moq_ffi/lib/src/moq.dart
  • doc/lib/dart/index.md
  • doc/lib/rs/moq-net.md
  • go/ffi/MIGRATE.md
  • go/wrapper/MIGRATE.md
  • kt/moq-ffi/MIGRATE.md
  • kt/moq/MIGRATE.md
  • py/moq-ffi/MIGRATE.md
  • py/moq-rs/MIGRATE.md
  • quest/m1/README.md
  • quest/m1/lazy-discovery/README.md
  • quest/m1/lazy-discovery/rust.md
  • quest/m1/public-model/README.md
  • quest/m1/public-model/answer.md
  • quest/m2/ietf-lazy-discovery.md
  • quest/m2/keyed-prefix-polling.md
  • rs/moq-c/MIGRATE.md
  • rs/moq-c/src/api.rs
  • rs/moq-c/src/test.rs
  • rs/moq-ffi/MIGRATE.md
  • rs/moq-ffi/src/origin.rs
  • rs/moq-hls/src/server/mod.rs
  • rs/moq-hls/src/server/routes.rs
  • rs/moq-mux/src/container/test_util.rs
  • rs/moq-mux/src/source.rs
  • rs/moq-net/Cargo.toml
  • rs/moq-net/MIGRATE.md
  • rs/moq-net/benches/interests.rs
  • rs/moq-net/src/discovery.rs
  • rs/moq-net/src/fuzz.rs
  • rs/moq-net/src/ietf/publisher.rs
  • rs/moq-net/src/ietf/session.rs
  • rs/moq-net/src/lib.rs
  • rs/moq-net/src/lite/publisher.rs
  • rs/moq-net/src/lite/session.rs
  • rs/moq-net/src/lite/subscriber.rs
  • rs/moq-net/src/model/mod.rs
  • rs/moq-net/src/model/origin.rs
  • rs/moq-net/tests/integration/idle_session.rs
  • rs/moq-net/tests/integration/lazy_discovery.rs
  • rs/moq-net/tests/integration/main.rs
  • rs/moq-net/tests/integration/support/harness.rs
  • rs/moq-relay/tests/integration/goaway_cluster.rs
  • rs/moq-rtc/Cargo.toml
  • rs/moq-rtc/src/server/whep.rs
  • swift/MIGRATE.md
  • swift/ffi/MIGRATE.md

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 8d7d600a-2060-47fa-a5c1-29ae153b2d59
📥 Commits

Reviewing files that changed from the base of the PR and between 90670cb and a2ebd8e.

📒 Files selected for processing: 7
  • dart/moq_ffi/lib/src/moq.dart
  • rs/moq-hls/src/server/routes.rs
  • rs/moq-mux/src/source.rs
  • rs/moq-net/src/ietf/session.rs
  • rs/moq-net/src/lite/subscriber.rs
  • rs/moq-rtc/Cargo.toml
  • rs/moq-rtc/src/server/whep.rs

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 2 remain after this review.

Walkthrough

MoQ-lite client sessions now request announcements for watched paths, with a two-second linger after a prefix loses its watchers. Origin interest tracking drives those requests, while server and relay paths retain broader discovery behavior. Broadcast lookup and HLS and WHEP integrations now use discovered routes. The change also adds tests and a benchmark, updates a Dart example and migration notes across language bindings, and removes the completed lazy-discovery quest documents.

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to a2ebd

No identified issue prevents merging after normal checks.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 76.52% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 115 functions across 23 files. (2 skipped… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Title check Passed The title clearly and concisely identifies the main change: lazy announcement requests for watched prefixes in moq-lite client sessions.
Description check Passed The description directly explains the lazy discovery implementation, behavior changes, affected components, tests, compatibility details, and follow-up work.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 76.52% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 115 functions across 23 files. (2 skipped: 2 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

✨ Finishing Touches
✨ Simplify code
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review 🔄 Running since 2026-10-22T12:43:40.978613Z d6cbfc4 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Keep both MIGRATE entries, and delete the finished lazy discovery quests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Merged origin/main at 3f24f22. The only conflict was rs/moq-net/MIGRATE.md: kept both entries (this PR's lazy-discovery entry, which covers feeding a relaying client upstream()/peer(), and #5678's claim entry). The merge also deletes the finished quest/m1/lazy-discovery/ quest and epic and drops the links to them; the IETF and keyed-polling follow-ups are already m2 quests.

Decisions: all four recommendations in the body were accepted by the maintainer. Codex gave a 👍 on e4ff1c8; this head only adds the mechanical merge on top of that.

Checks on this head: just check passed, just test interop --all passed, quest check passed. No drafts/ change, so just drafts check was not run.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 21, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3f24f22f3b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rs/moq-mux/src/source.rs
request: None,
};
// Ask now when the origin already knows, so a dynamic handler hears it on binding.
let _ = lookup.poll(&kio::Waiter::noop());

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep bind eager after discovery completes

When a lazy remote route is not already synced, this no-op poll returns pending and nothing drives Lookup after the announcement arrives. Consequently Source::bind no longer fulfills its documented guarantee to issue the request without awaiting or polling: the upstream broadcast request and any dynamic handler invocation are deferred until somebody later calls poll_broadcast or broadcast, which breaks callers that bind early specifically to start acquisition before consuming the result.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Disagree that this breaks early binding. What bind exists to start eagerly is the network acquisition, and that still happens at bind: Lookup::new opens the hidden announced() watch on the literal path, which is what makes the client session ask its peer. The only step deferred to the first poll is the local request_broadcast once that watch syncs, which resolves against the route the announcement just installed and sends nothing on its own (tracks subscribe when read). Any origin that already knows its routes, including one with a local dynamic handler, syncs inside the constructor's noop poll, so the request and handler invocation still happen on binding there. Driving the request without a poll would need a runtime in Source, which it deliberately does not hold. The bind doc already states the deferral for a lazily discovering client session.

(Written by Claude Opus 5.5)

Comment on lines +655 to +657
let mut ended = Vec::new();
for (prefix, machine) in &mut self.prefixes {
match machine.poll(waiter) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Key polling to the announce prefix that woke

When a client holds many disjoint watches, every subscriber-driver wake scans and polls every open AnnouncePrefix, including wakes caused by unrelated media, datagrams, or bandwidth feedback. The new watch-driven design makes this set application-controlled and potentially large, so the loop adds O(number of watched prefixes) CPU to each transport wake and can hurt live-media latency; use keyed readiness/tasks rather than polling the whole map, and benchmark the watch-count and wake-rate axes as required.

AGENTS.md reference: AGENTS.md:L39-L40

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already scoped as a measurement-gated follow-up with the maintainer: quest/m2/keyed-prefix-polling.md, added in this PR. It first extends rs/moq-net/benches/interests.rs to sweep open prefixes against wakes, and keys polling by prefix only if the per-wake cost actually slopes. Leaving it out of this PR.

(Written by Claude Opus 5.5)

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Oct 22, 2026
Resolve rs/moq-mux/src/source.rs: the epoch-checked Binding from main now
binds through the watch-then-request Lookup. Keep both MIGRATE entries.
Pass the new epoch argument in the moq-hls sibling test, and drop the
stale Timescale argument from main's SUBSCRIBE_DROP fixture (same fix as
#6223) so moq-net tests compile.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 376dfcfd93

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rs/moq-ffi/src/origin.rs
Comment on lines 499 to +502
/// Calling this straight after connecting therefore races the session's announcements
/// and can report a live broadcast as unroutable. Await `announced_broadcast` first.
/// and can report a live broadcast as unroutable. Await `announced_broadcast` first. A
/// moq-lite client only asks the relay about paths something watches, through `announced`
/// or `announced_broadcast`, so a path nothing watches stays unroutable.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Add migration notes to the binding packages

For users of the published moq-ffi, C, or generated language APIs on a moq-lite client, this changes request_broadcast from discovering any permitted path to returning Unroutable unless another handle watches it, requiring migration to announced_broadcast; the documentation edit also changes the UniFFI checksum. Only rs/moq-net/MIGRATE.md records this break, so add the corresponding guidance to rs/moq-ffi/MIGRATE.md, rs/moq-c/MIGRATE.md, and the affected wrapper-package migration files.

AGENTS.md reference: AGENTS.md:L62-L63

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Fixed in 116a535: every binding package now has an Unreleased MIGRATE entry. That covers moq-ffi, moq-c, the five generated *-ffi packages (they also note the changed request_broadcast checksum), and the py, swift, kt, go, dart, and cpp wrappers, each in its own casing.

(Written by Claude Opus 5.5)

…h a watch

Lite-03/04 announce streams mark no initial set, so a watch reads as synced
before the peer answers it and moq-mux's watch-then-request sibling lookup
pinned Unroutable. Those clients keep asking for everything at connect.

The WHEP handler requested its broadcast blind, which a lazy client origin
(moq --connect ... export rtc --listen) can never route. It now resolves with
routed_broadcast inside the existing 5s bound, answering 404 when nothing
announces it in time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Merged origin/main again (it conflicted once moq-dev/moq#6090 landed) and fixed what a fresh Codex review of the merge found. Head: 57a0d52.

Merge (376dfcf)

  • rs/moq-mux/src/source.rs: main's epoch-checked Binding now resolves through this PR's watch-then-request Lookup. The epoch check runs on whatever the lookup pins, unchanged.
  • rs/moq-net/MIGRATE.md: kept both Unreleased entries.
  • The moq-hls sibling test passes the new epoch argument.
  • main's SUBSCRIBE_DROP fixture in lite/subscriber.rs still passed a removed Timescale argument, so moq-net tests did not compile on main. Applied the same one-line fix as moq-dev/moq#6223, so the two merge cleanly.

Codex review fixes (57a0d52)

  • lite-03/04 clients stay eager. Their announce streams mark no initial set, so a watch reads as synced as soon as the request flushes. The sibling Lookup then requested before the peer answered and pinned Unroutable. a_sibling_resolves_through_a_client_session now loops over lite-01 to lite-07. Without the fix it fails on lite-03. MIGRATE and doc/lib/rs/moq-net.md mention the exception.
  • WHEP resolves through a watch. whep::accept called request_broadcast on the subscribe origin with nothing watching it, so moq --connect ... export rtc --listen could never route a play through a lazy client. It now uses routed_broadcast, with the route and the first catalog sharing the existing 5s bound. A route timeout maps to Unroutable (404, which WHEP clients retry), and a missing catalog still maps to CatalogTimeout. New test a_broadcast_resolves_through_a_lazy_client_session fails with Unroutable without the fix.

Codex findings I disagree with

  • On lite-03/04, a catalog announcement can still arrive before its sibling's, so the sibling lookup can come back Unroutable. This is the behavior main already has: bind there requests at once and a client asks for everything at connect, which is exactly what this PR now restores for those versions. Closing it needs an initial-set boundary those wire versions do not have, so it is out of scope for a legacy-version gap.
  • I replied inline to the two earlier findings (bind eagerness, keyed prefix polling, already a measurement-gated m2 quest).

Checks: just check passes every step except nix flake check. That step fails only on this shared runner's user-namespace limit (unshare: write failed /proc/self/uid_map) while building the github-runner VM image. I ran the steps after it by hand: ts-paths, markdown, shell, taplo, gh lint, just rs test --workspace (9431 passed), play, capture, JS doc check and tests, Python. All pass. While the machine was at load ~22, redundant_importers_switch_through_bufferbloat_on_lite_07 failed twice inside the full suite. At normal load it passes on both this head and the pre-fix head. Some moq-uring tests failed only with the shared RLIMIT_MEMLOCK exhausted.

Public API: none new. Wire: none.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 22, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 57a0d524c2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +183 to +186
if want && !covered {
let snapshot = snapshot.expect("a wanted prefix comes with its snapshot");
self.requests.insert(prefix.clone(), Request::default());
self.changes.push_back(Change::Open(prefix, snapshot));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Hold lingering child requests under a new broader prefix

If a narrow watch is dropped and its request enters the two-second linger, then a broader watch opens before that timer expires, this branch creates the broader request but never adopts the already-lingering descendant. If the timer fires before the broader request's initial announcements arrive, expire closes the narrow request and temporarily retracts its routes; an announcement consumer can observe an End, and a broadcast resolved during this interval can lose its only route even though the peer continuously serves it. Cancel or hold lingering descendant requests when opening the broader prefix.

AGENTS.md reference: rs/moq-net/AGENTS.md:L7-L7

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, and it breaks Discovery's own documented invariant: "A narrower request already open when a broader one opens stays open until the broader one goes." Only a narrower request released after the broader one opens gets held. One that is already lingering keeps its own timer. The fix is small. When interest opens a new request, it moves each lingering request beneath that prefix into the new request's held set and cancels the linger timer. A test like a_broader_request_holds_the_narrower_open would cover it, with the narrower watch dropped before the root watch opens.

Not pushed yet. This PR has reached its design checkpoint, so the maintainer is deciding the next step together with the WHEP finding below.

(Written by Claude Opus 5.5)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 634107b. A new request now holds every narrower request still lingering beneath its prefix and cancels their timers, the same as for one released after it opened. New test: a_broader_request_holds_a_lingering_narrower_open, which fails without the fix.

(Written by Claude Opus 5.5)

Comment thread rs/moq-rtc/src/server/whep.rs Outdated
Comment on lines +197 to +200
let deadline = tokio::time::Instant::now() + CATALOG_TIMEOUT;
let broadcast = tokio::time::timeout_at(deadline, subscriber.routed_broadcast(path))
.await
.map_err(|_| Error::Moq(moq_net::Error::Unroutable))??;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Bound concurrent WHEP discovery watches

On the bundled unauthenticated WHEP router, a caller can submit valid offers for many distinct nonexistent paths; each call now holds a routed_broadcast interest for five seconds, causing every lazy client session on the origin to open a distinct announce-request stream, which then lingers for another two seconds. Because neither the router nor resolve limits concurrent unique lookups, a remote request burst can grow per-session state and exhaust bidirectional stream credit, preventing legitimate playback lookups; add admission control or otherwise bound these untrusted watches.

AGENTS.md reference: AGENTS.md:L18-L18

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that it's real, and it reaches beyond WHEP. Since moq-dev/moq#6190, moq_hls::Server::broadcaster calls origin.follow(name) for each HTTP GET. Under this PR that registers an interest too, so a plain GET flood of distinct paths does the same thing. The moq-cli gateways scope to subtree(name), but every distinct name/<anything> beneath it still opens its own announce request upstream. The only bound is the upstream connection's stream credit, which legitimate lookups share.

An admission cap would be an arbitrary limit. The bounded-by-design options are:

  1. An HTTP gateway holds one watch over its whole served scope for its lifetime (for moq-cli, the name subtree), so every per-request lookup rides that one request. A gateway that serves other clients is acting as a relay, so being eager within its scope matches what a relay link already does.
  2. Merge as is and track the bound as an m2 follow-up quest.

Maintainer decision pending.

(Written by Claude Opus 5.5)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5d9bf72 with option 1, as the maintainer chose. moq_hls::Server and the bundled WHEP router each hold one announced() watch over their whole scope for as long as they live, so every per-request lookup rides that single covering request instead of opening its own. The cost is now one announce request per gateway, however many paths callers name. The whep::accept doc tells embedders that own the route to do the same. New tests, lookups_ride_the_servers_scope_watch and offers_ride_the_routers_scope_watch, each fail without the watch. JS has no HTTP gateway, so there is nothing to mirror.

(Written by Claude Opus 5.5)

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Oct 22, 2026
kixelated and others added 2 commits October 21, 2026 22:45
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
request_broadcast on a moq-lite client now fails for a path nothing watches, and its doc change moves the UniFFI checksum, so moq-ffi, moq-c, the generated packages, and each wrapper say how to migrate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Merged origin/main again after the queue dropped this PR for a conflict. Head: 116a535.

Merge (a3f1f4b)

  • quest/m1/lazy-discovery/: main's audit (moq-dev/moq#6092) edited the quest, but this PR finishes it, so the deletion stands. quest/m1/README.md drops its entry.
  • quest/m1/public-model/answer.md: took main's narrowed text (JS-only fallback, no blind answers in Rust, which matches this PR) and removed the links to the deleted quest.
  • rs/moq-hls/src/server/routes.rs test: keeps this PR's routed_broadcast wait and main's close-only-the-sibling teardown (moq-dev/moq#6190).

Codex finding fixed (116a535): MIGRATE entries for every binding package.

Not re-queued. This is the fourth review round since the design last changed, and two findings on 57a0d52 are still open: lingering narrower requests (a small fix, agreed) and bounding gateway-driven watches, which needs a maintainer decision.

Checks: quest check passes, and so do all just check steps except nix flake check, which fails only on this runner's user-namespace limit (unshare: write failed /proc/self/uid_map). I ran the steps after it by hand, and they pass: ts-paths, markdown, shell, taplo, gh, just rs test --workspace (9301 passed), just js test, and just py test. just test interop --all was not rerun on this head.

(Written by Claude Opus 5.5)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 116a535706

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rs/moq-mux/src/source.rs Outdated
if let Some(watch) = &mut self.watch {
// Synced only counts once every event queued before it was read.
while let Poll::Ready(Some(_)) = watch.poll_next(waiter) {}
ready!(watch.poll_synced(waiter));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Request once any discovery session supplies the route

When an origin is fed by multiple lazy client sessions and one peer never completes its initial ANNOUNCE response, poll_synced keeps waiting on that peer's snapshot even after another peer has delivered a usable route, whose event is discarded by the loop above. The binding therefore never issues its request through the healthy route, allowing one stalled or hostile session to block sibling-broadcast playback across the shared origin; request as soon as the watch observes a route, and use synchronization only to conclude that none exists.

AGENTS.md reference: AGENTS.md:L18-L18

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, and reproduced. Fixed in 3db5d52: Lookup now requests as soon as its watch reports a route from any session, and uses sync only to conclude that none serves the path. New test: a_stalled_session_does_not_hold_up_a_sibling feeds one origin from two lazy lite-07 sessions, one of whose peers never accepts a request stream. Before the fix it timed out on the sibling.

(Written by Claude Opus 5.5)

kixelated and others added 4 commits October 21, 2026 23:32
A narrower request already lingering when a broader one opened kept its own timer, so it could close and retract its routes before the broader request synced. The new request now holds it, as it would one released after it opened.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each HTTP lookup watched only its own path, so a lazily discovering client session opened one announce request upstream per distinct path a caller named, bounded only by stream credit. The HLS Server and the WHEP router now hold one watch over their scope, so lookups ride a single request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Applied the maintainer's decisions on the two open findings. Head: 80ee299.

  • 634107b: a new broader announce request holds the narrower requests still lingering beneath it, so none of them closes before the broader one syncs.
  • 5d9bf72: the HLS Server and the WHEP router each hold one watch over their whole scope while they serve, so the paths HTTP callers name no longer open one upstream announce request each. doc/lib/rs/moq-net.md mentions it.
  • 0ecfec1: merged origin/main with no conflicts.

Public API: unchanged. Wire: unchanged.

Checks: every just check step passes except nix flake check, which fails only on this runner's user-namespace limit. I ran the steps after it by hand, and they pass: ts-paths, markdown, shell, taplo, gh, just rs test --workspace (9309 passed), just js test, just py test. just test interop --all also passes.

(Written by Claude Opus 5.5)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 90670cb514

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rs/moq-mux/src/source.rs Outdated

/// A sibling resolves through the session that announces it, even while another lazily
/// discovering session on the origin never answers its announce request.
#[tokio::test]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Pause time in the in-memory async test

This test runs entirely over the in-memory Link::lazy transport but uses an unpaused Tokio clock with real five-second timeouts, so a regression occupies a CI worker for wall-clock time instead of failing promptly under simulated time. Make it #[tokio::test(start_paused = true)], as required for non-socket time-dependent Rust tests.

AGENTS.md reference: rs/AGENTS.md:L53-L54

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Fixed in 99f927c: the test now runs with start_paused = true. With the lookup forced back to waiting for sync, it fails in 8 ms instead of 5 s.

(Written by Claude Opus 5.5)

kixelated and others added 3 commits October 22, 2026 02:26
It runs over the in-memory transport, so a regression now fails at once under simulated time instead of holding a worker for real timeouts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ietf/session.rs: main's grant check judges the publisher's serving origin (#6259); keep this branch's serving() view on it, so the check registers no discovery interest. moq-rtc whep.rs: keep the watch-then-request resolve, moved onto moq-clock like the rest of the crate (#6135).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main disallows tokio::time in moq-rtc since #6135.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 38dbdab7f7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rs/moq-mux/src/source.rs Outdated
Comment on lines +416 to +417
self.watch = None;
self.request = Some(self.origin.request_broadcast(&self.path, None).into_inner());

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Retry if the route vanishes before the sibling request

On a multithreaded runtime, a route can retract after the loop drains its Start and poll_next returns Pending, but before request_broadcast reads the route table. The queued End is then discarded with self.watch = None, the request returns an immediate Unroutable, and this binding permanently fails even if another unsynced session later supplies the sibling. Fresh evidence beyond the earlier fix is that following the selected route still does not make the event read and table lookup atomic; retain the follow until the request actually joins a route, or resume following after an unqueued Unroutable.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Fixed in a2ebd8e: the lookup keeps its follower until the request settles. If it gets Unroutable before every session has synced, it drops the request and follows again, the same way routed_broadcast does. No spin is possible, because the next pass has no new events and parks on poll_synced. There's no dedicated test, since the race needs the route to retract between two adjacent reads.

(Written by Claude Opus 5.5)

A route could retract between the follower reporting it and the request reading the table, failing the binding for good although another unsynced session might still serve it. An Unroutable answer before every session has synced now resumes following, as routed_broadcast does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

The automatic review of a2ebd8e has shown "running" for about an hour; re-requesting it.

(Written by Claude Opus 5.5)

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: a2ebd8e00b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@kixelated
kixelated added this pull request to the merge queue Oct 22, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Oct 22, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Merged origin/main (53a9dea) to clear the merge-queue conflict. The only conflict was rs/moq-net/MIGRATE.md ## Unreleased, resolved by keeping both entries (this PR's lazy-discovery entry and main's "Frames carry their arrival time"). quest check (both modes), cargo check --workspace --all-targets, clippy -D warnings, tests for moq-net/moq-mux/moq-hls/moq-rtc/moq-c/moq-ffi, and moq-relay nextest all pass.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 22, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Oct 22, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Merged origin/main (d6cbfc4) after the merge-queue drop. Mechanical resolution only: each MIGRATE.md keeps both ## Unreleased entries (this PR's first, then #6250's), and quest/m1/lazy-discovery/README.md stays deleted since this PR finishes that quest. quest check, workspace check and clippy, and tests for moq-net, moq-mux, moq-hls, moq-rtc, moq-c, moq-ffi and moq-relay pass locally.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 22, 2026
Merged via the queue into main with commit e54010b Oct 22, 2026
10 checks passed
@kixelated
kixelated deleted the quest/m1/lazy-discovery/rust branch October 22, 2026 13:01
kixelated added a commit that referenced this pull request Oct 22, 2026
Take main's deletion of the finished lazy-discovery epic (#6165).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated added a commit that referenced this pull request Oct 22, 2026
…/ingest

Resolve conflicts with main's fMP4 import refactor (#6250), frame arrival
stamps, the clock rename, and the #6165 MIGRATE.md entries. The owned
frame writer now builds frames with `Frame::new`, and main's arrival-stamp
test takes the group writer back from `finish`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated added a commit that referenced this pull request Oct 22, 2026
moq-lite clients already ask only for watched prefixes (#6165), and
moq-mux Source watches the sibling broadcasts a catalog references, so the
subtree scope only lost those siblings. The moxygen draft-14 cell stays
red until IETF lazy discovery asks for watched prefixes too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant