Skip to content

fix(net): retire epochless handles only once their Restart is deliverable - #5443

Merged
kixelated merged 6 commits into
mainfrom
quest/m0/upgrade-unroutable
Oct 13, 2026
Merged

kixelated merged 6 commits into
mainfrom
quest/m0/upgrade-unroutable

Conversation

@kixelated

@kixelated kixelated commented Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator

Implements step 1 and the regression of quest/m0/upgrade-unroutable.md. The consumer half is split out to quest/m1/restart-followers.md, moved to m1 because the WebSocket-to-QUIC upgrade is being removed (maintainer, 2026-10-13).

Problem

When WebSocket wins the connect race, #4180's upgrade re-announces the relay's routes over QUIC, and every epochless route comes back as a new instance. Since #5294, RequestValidity retired the handles resolved before the upgrade at once. Their Restart was still held by the origin's update_hold (300 ms), so a request in that window failed with Unroutable before any Restart could reach the holder. That is Interop's catalog consumer: unroutable / container consumer: unroutable.

Fix

Retirement and its Restart now share one Release in rs/moq-net/src/model/origin.rs:

  • A table change that restarts a prefix onto an epochless route makes one Release, anchored like the existing hold (on the first clock reading after the change, then update_hold later).
  • Announce cursors hold that pending Restart on the Release instead of their own per-consumer anchor.
  • A Selection whose peer replaced its instance at the same prefix with an epochless route keeps its period open until the same Release. RequestValidity::start now captures the period with its selection (Period), so check refuses once either the period closes or its release fires.
  • Everything else retires at once, as before: a newer epoch, a more specific route, the route gone, origin teardown. A selection whose instance wins again before the release continues its period. After the release, it starts a new period, so nothing refused revives.

The old session is still draining during the hold, so it serves those requests. Subscriptions already open stay sticky on it, as before. This also covers a relay hop when a publisher upgrades, because the lite and IETF subscribers both build their validity through Dynamic::request_validity.

Tests

rs/moq-net/tests/upgrade.rs simulates an upgrade with two sessions from the same peer into one origin, under simulated time, on lite-06 and lite-07:

  • Epochless: a request through the pre-upgrade handle right after the upgrade is served. It failed with Unroutable before this change. Past the hold, the old handle refuses with Unroutable while the Restart is already deliverable (try_next, with no wait). Following it, a fresh request_broadcast serves a new track.
  • Epoch (lite-07): no Restart, and the old handle keeps serving past the hold.

The race can't be forced in websocket_upgrades_to_quic (QUIC wins locally, on the wall clock), so this sibling carries the proof.

Public API / wire

  • Public API: none. Behavior: an epochless replacement's handles retire when its held Restart is released, not when the replacement is announced. Config::update_hold documents this.
  • Wire: none.

Not in this PR

First-party consumers following Restart (moq export/subscribe, moq-ffi and moq-c consumers, interop clients) need API and behavior decisions, such as what an fmp4 export does on an epochless restart. Those moved to Restart followers. Without them, a consumer that requests within the hold, as every interop subscriber does right after connecting, now works. A long-lived consumer's sticky subscriptions still end when the old session closes.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 2 commits October 13, 2026 11:46
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…able

A WebSocket-to-QUIC upgrade re-announces every epochless route as a new
instance. RequestValidity retired the handles resolved before it at once,
while their Restart waited out the origin's update hold, so a request in
that window failed Unroutable before any Restart could be followed.

One Release per table change now ends both: announce cursors hold the
pending Restart on it, and a selection replaced at the same prefix by an
epochless route keeps serving until it fires. Everything else still
retires at once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Quest outcome

  • Done: step 1 (handle retirement waits for the held Restart's release, in the origin, so a relay hop behaves the same) and step 3 (the regression in rs/moq-net/tests/upgrade.rs, using simulated time, epochless on lite-06/07 and epoch on lite-07; it failed with Unroutable before the fix).
  • Split out: step 2 (first-party consumers follow Restart) moved to quest/m0/restart-followers.md. It needs maintainer calls on API and behavior, for example what an fmp4/mkv export does on an epochless restart, given the TS Follower's existing Replaced-unless-stitching design.
  • Checks: just check passed. On a rerun, moq-relay::drills bursts_cross_a_transport_peer::impaired flaked on a handshake timeout and passed when run alone. That is the known lost-ACK flake (quest(m1): a lost handshake ACK stalls the impaired cluster drill's connect #5415). just test interop --all passed locally, but QUIC wins the race locally, so the unit test carries the proof.

Open decisions:

  1. Keep restart-followers in m0, or move it to m1 once Interop on main stays green with this PR. Recommended: m1 if Interop goes green, because what remains only bites long-lived consumers after the old session closes.
  2. What an epochless Restart means for non-TS moq export formats: restart the output, stitch like TS, or end the export. Recommended: match the TS Follower (sticky, then Replaced unless stitching), and follow inside moq-ffi/moq-c for the bindings.

(Written by Claude Opus 5.5)

kixelated and others added 3 commits October 13, 2026 13:00
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review October 13, 2026 20:11
@coderabbitai

coderabbitai Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 81d39010-7b2f-42fc-9afe-3164e997f41a
📥 Commits

Reviewing files that changed from the base of the PR and between 4e3e23b and eea0f99.

📒 Files selected for processing: 10
  • doc/concept/moq-lite.md
  • quest/m0/README.md
  • quest/m0/upgrade-unroutable.md
  • quest/m1/README.md
  • quest/m1/restart-followers.md
  • quest/m1/transport-upgrade/README.md
  • rs/moq-net/src/fuzz.rs
  • rs/moq-net/src/model/broadcast.rs
  • rs/moq-net/src/model/origin.rs
  • rs/moq-net/tests/upgrade.rs
💤 Files with no reviewable changes: 1
  • quest/m0/upgrade-unroutable.md

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 0 remain after this review.

Walkthrough

Origin route synchronization now shares a clock-based hold between epochless restart delivery and selection retirement. Request validity tracks the selected route period and returns Unroutable when the route closes or the selection retires. Simulated-session tests cover epochless and epoch-bearing upgrades. Protocol and quest documentation now describe held restart delivery and first-party consumer behavior after Restart.

Priority: ➖ Normal

Severity of issue fixed: Medium

Merge Risk: ⚪ Minimal · up to eea0f

The change delays retirement of old epochless handles until the matching Restart can be delivered, and it adds regression tests. No merge-blocking risk is established from the supplied evidence.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 71.11% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 45 functions across 4 files. (5 skipped: … Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description check Passed The description clearly explains the epochless handle-retirement fix, regression coverage, scope, and excluded consumer work.
Title check Passed The title clearly and concisely identifies the main change: delaying epochless handle retirement until the corresponding Restart is deliverable.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 71.11% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 45 functions across 4 files. (5 skipped: 5 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches
✨ Simplify code
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Some tools did not complete. Review the errors below.

🔧 Clippy (1.98.1)

Clippy execution timed out

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-13T20:15:43.618527Z 75b6fa9 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@kixelated

Copy link
Copy Markdown
Collaborator Author

Merge summary

Changes since the last outcome comment:

  • Merged main twice, resolving quest/m0/README.md conflicts. wire-compat-green and ts-duration-fidelity completed on main, so the Interop paragraph now covers only the unroutable fix.
  • Moved the consumer half to quest/m1/restart-followers.md (maintainer, 2026-10-13). The WebSocket-to-QUIC upgrade is being removed in a separate m0 quest, so an epochless Restart no longer follows every connect; it remains for GOAWAY drains and publisher replacement. Its Goal is now framed on those cases, and every reference (m0/m1 READMEs, transport-upgrade README) is updated.

Decisions:

  • The split of the consumer half is accepted.
  • The export-on-epochless-Restart decision (fmp4/mkv/h264/h265/flv: restart, stitch, or end) stays open in the quest. No follow-up to force the WebSocket-first path in CI Interop.

Checks: CI green on 75b6fa9 (the newest merge only touches quest text); Codex reviewed 75b6fa9 with no findings. Local just check passed.

(Written by Claude Opus 5.5)

@kixelated
kixelated enabled auto-merge October 13, 2026 20:37

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: eea0f99 against 4e3e23b, including the earlier full review at 75b6fa9 and subsequent base merges.

Two P2 races remain: an anchoring wake can be mistaken for retirement, and a synthesized broad-prefix Restart can use a different release from the retiring selection. Both can still expose Unroutable before Restart is available.

Sharing a release is a sound direction and avoids another independent timeout policy. The ordinary new-period/no-revival logic is sensible; these two paths need the same actual release boundary. The explicitly deferred consumer Restart-following work is not a new finding here.

Verification: GitHub-only full diff and origin/broadcast/time/subscriber/test inspection, plus current-head comparison. No builds or tests run; the concurrent interleavings have not been executed.

Comment on lines +716 to +718
let until = state.until(now);
state.deadline.set(Some(until));
state.deadline.poll(waiter)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Recheck the release after the anchoring wake

With Held::Anchoring(S), until(S) returns S + 1ns without starting update_hold. The independently running origin driver can advance Clock after try_now() but before Deadline::poll(); the latter rereads the clock and can return Ready for that temporary anchoring deadline. Period::check then reports Unroutable before the real hold starts. An announce scan instead starts the hold at the fresh reading and remains Pending, and the same Period can subsequently appear valid again. Re-read/recompute after deadline readiness and return Ready only when the actual hold has elapsed. Add an interleaving test advancing the clock between these reads.

Comment on lines +3470 to +3473
let held = best.is_some_and(|entry| entry.epoch.is_none() && entry.prefix == self.prefix);
match (held, &self.retiring) {
(true, Some(_)) => None,
(true, None) => Some(hold.release().map_or(Change::End, Change::Retire)),

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Reuse retirement's release for a synthesized prefix Restart

A broad prefix's announcement winner can stay unchanged while a path beneath it switches between two instances from the same peer. The first sync_route creates Release1 for this selection but queues no Restart. run_front later detects the moved path and calls renew (2762–2770), which calls sync_route again (4042–4052) and creates Release2 for the cursor's Restart. This branch retains Release1. Their separate lazy anchors can expire at different times, so the old request becomes Unroutable while the Restart is still held. Propagate/reuse the selection's release during the synthesized renewal, or coordinate both synchronously. Extend a_moved_path_restarts_a_scoped_cursor with nonzero hold and a same-via replacement; its current zero-hold helper misses this case.

@kixelated
kixelated added this pull request to the merge queue Oct 13, 2026
Merged via the queue into main with commit 35d935f Oct 13, 2026
8 checks passed
@kixelated
kixelated deleted the quest/m0/upgrade-unroutable branch October 13, 2026 21:49
kixelated added a commit that referenced this pull request Oct 13, 2026
Drop the repro note for the now-completed upgrade-unroutable quest (#5443).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant