[kixelated]

Problem

The studio dials one relay, so a failure of that relay or the path to it drops the stream for every viewer.
moq already resumes a subscription mid-group between routes that carry the same publisher epoch (doc/concept/moq-lite.md, Publisher epochs and Multiple CDNs); the studio only has to publish to a second relay under that epoch.

Approach

• --failover <url> names a second relay. The program is encoded once and its one publication, announced under one epoch, is published over a connection to each relay.
• A copy whose track info differs ends the track, and moving between relays without an epoch is a restart, so --failover requires exactly one --connect-version, and one that carries publisher epochs (moq-lite-07-wip today). Anything else refuses to start with the reason. With one version offered, a relay that cannot speak it fails its handshake, and the Health panel shows why.
• The failover client presents no --connect-token: those are the first relay's credentials. Its own rides its URL's ?jwt=.
• Broadcast sources (and viewer feedback and relay stats) come through both links into the one subscribe origin, the failover ranked below with origin::Preference. moq ranks the newest publisher epoch before preference. A source therefore plays its newest instance through either relay, and the first relay wins only among routes to the same instance. A publisher restarted behind only the failover replaces the first relay's copy. The README says so, and every_relay_routes_one_epoch tests it.
• Health: each relay's state and negotiated protocol get their own table. One relay down while another serves is Degraded ("reaches viewers only through ..."); none serving is Unhealthy. The uplink grades the first connected relay's link and names it. When the graded link changes, the previous interval is not reused, since the counters came from another connection.
• The relay wiring is split into engine::connect so it can be tested without a GPU.
• Guided setup (feat(studio): guided setup on first launch #7) takes Launch::relays in place of connect. Its uplink probe measures the --connect relay, the only link probed toward the stream's rate (decision 1).

Impact

• New --failover <url> flag. It needs --connect and exactly one --connect-version that carries publisher epochs, and it refuses a fixed --connect-bind port, since each relay's client binds its own socket. Sharing one endpoint would need a per-connection token setter in moq-tokio. Both relays must accept that version; a moq relay accepts moq-lite-07-wip only when configured to.
• The Health panel lists each relay's state and protocol, and the problems name the relay they concern.
• Broadcast sources fail over to the failover relay when the first has no route to them, and play a newer instance through either relay.
• The README documents the flag and the Health panel changes.

Choices made without asking

• Flag name --failover, a single URL rather than a repeatable list. The quest asks for two relays.
• No automatic version pin. moq deliberately makes lite-07 opt-in, so the studio refuses rather than silently opting in to a WIP wire format. The alternative was pinning moq-lite-07-wip whenever --failover is given without a --connect-version.
• Sources fail over too, ranked by Preference (first relay 0, failover -1). Otherwise a guest or broadcast source on program would go black when the first relay fails, even though the program keeps going out.
• Uplink grades the first connected relay instead of each link. Per-link loss and headroom would need to know which link the relays actually pull media over. An idle failover link graded against the full media rate would always read congested, and moq exposes no per-session subscription state to tell the two apart.
• The endpoint shown in Health is the scheme, host, and port only. A URL path can carry a credential, and the panel may be on a shared screen. Logs keep RedactedUrl as before.

Alternatives

• Waiting at startup for both sessions and comparing their negotiated versions. This blocks startup while a relay is down, which is the case failover exists for. Offering one version makes a mismatch impossible instead.
• Sharing one Client for both relays and replacing the failover's tokens through Connection::tokens. This races the first dial, which would present the first relay's token to the other operator.

Maintainer decisions

Decided 2026-11-03 by the maintainer: land as is. The decisions were relayed through the session coordinator; this PR has no maintainer comment.

1. Bandwidth is claimed on the first relay only. Kept, and documented. The program's and the captured sources' tracks claim their bitrate on the first relay's allocator, so only that link is probed toward the stream's rate. No encoder adapts to either estimate: moq-scene's capture path claims but never consumes a grant (rs/moq-scene/src/publish.rs:155), and encode control belongs to feat(studio)!: a two-size program that follows the bandwidth grant, and local source tracks #21. This is documented in the README --failover and setup Uplink bullets and in the engine comment above the allocator. A moq quest for an allocator spanning several connections will be planned separately.
2. Watching undercounts when both relays publish stats under the same node name. Kept, as documented. Both publish .stats/node, and the shared subscribe origin routes one: the newer instance, or the first relay's when they are one instance. Relays with distinct stats.node names are both counted. The README's Watching bullet documents this.

Follow-ups

• A moq quest for an allocator spanning several connections, which the maintainer will plan separately, then a studio follow-up to use it.

Checks

On an Apple Silicon Mac, on head 3038387 (main merged through #17):

• Pass: the full just ci, covering every lint, clippy at defaults and with capture,openh264, and 95 tests. The new tests include every_relay_routes_one_epoch, which runs against two real test relays. With its epoch order inverted, it fails, so it does check the ordering.
• Did not run: the GPU-only the_program_reaches_a_relay (just render-sw, Linux lavapipe). It is updated to the Launch signature.

Review

• Local reviewer: Codex, gpt-6.1-sol, medium effort, session 01a19ff7.
• Round 5 reviewed head 3038387 and reported no findings. Rounds 3 and 4 were clean too.
• Online review on f0f4756, answered in 3038387:
  • Fixed (OpenAI P2): source selection is epoch-first. The README, the Watching bullet, and the connect comment now describe newer-instance-wins, and the test covers it. Isolating per-relay routes was not done: a client takes one subscriber origin.
  • Fixed (OpenAI P2): no encoder adapts to bandwidth. The README, the setup rationale, and the comments now say the tracks only claim their bitrate on the first relay's link.
  • Fixed (pullfrog nits): the epoch error names moq-lite-07-wip, and Health shows a host-less relay URL's path.
• Earlier local findings and their dispositions:
  • Fixed: a fixed --connect-bind port would be bound twice. The studio now refuses it at startup (5f0864f).
  • Settled by maintainer decision 1: bandwidth is claimed on the first relay only.
  • Settled by maintainer decision 2: relays sharing a stats node name shadow each other's Watching count.

[kixelated]

Recording the maintainer's answers to the two open decisions, relayed through the session coordinator: land as is.

1. Bandwidth follows the first relay only. Kept and documented: see the README --failover bullet and the engine comment above the allocator. A moq quest for an allocator spanning several connections will be planned separately.
2. The Watching count misses one relay when both relays' stats share a node name. Kept, as documented in the README's Watching bullet.

The local reviewer (Codex) raised both as findings in rounds 1 and 2. Its findings are answered with these decisions, which the PR body records under ## Maintainer decisions.

(Written by Claude Opus 5.5)

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 7 seconds.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 12ae5eaa-57dd-401a-8c78-6b3f5f031258

Commits

Reviewing files that changed from the base of the PR and between a05d702 and 3038387.

Files selected for processing: 8

• README.md
• quest/m0/README.md
• quest/m0/failover.md
• rs/moq-studio/src/engine.rs
• rs/moq-studio/src/health.rs
• rs/moq-studio/src/main.rs
• rs/moq-studio/src/setup.rs
• rs/moq-studio/src/ui.rs

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

CodeRabbit Verify

• Run CodeRabbit Verify to test the changes in this pull request.

[kixelated]

Automated review by review (OpenAI)

Reviewed commit: f0f4756.

• [P2] Reconcile source selection with epoch-first routing (rs/moq-studio/src/engine.rs:696–704; README.md:122–124). The stated behavior is to use a broadcast source from the first relay while it has a route. However, the pinned origin ranks the newest publisher epoch before link preference. If both relays serve the same path under different epochs, a newer backup instance can replace the healthy primary source, restarting or changing what the studio shows. Preserve epoch freshness and either isolate the relay routes to implement the stated preference or document and test the actual newer-instance behavior; the same distinction affects colliding .stats paths.
• [P2] Correct the claimed captured-source adaptation (README.md:26–30, 59–60). These lines say captured sources follow the first relay's bandwidth estimate, but the scene capture path constructs moq_video::encode::Encoder and publishes directly (rs/moq-scene/src/publish.rs:234–250, 275–284) without consuming a grant or changing bitrate. The catalog's reservations are advisory claims, so a congested primary also leaves these encoders at their configured rates. Correct the limitation and the setup rationale, or implement and test the promised adaptation; this does not ask to change the maintainer's decision to use the first link only.

Static GitHub-only review against the pinned MoQ dependency. I did not run the relay, GPU, capture, or setup tests.

[pullfrog]

ℹ️ Minor suggestions only.

Reviewed changes

• --failover flag and the check_failover guards (one epoch-carrying version, ephemeral bind port) in main.rs
• per-relay engine::connect, the Relay struct, and Launch::relays in engine.rs
• health::{Relay, State, Link} plus the previous.serving re-baseline in health.rs
• Health panel relay grid and Uplink row in ui.rs
• uplink probe in setup.rs, README coverage, and the completed quest file removal

The one-epoch-two-links design matches doc/concept/moq-lite.md §"Multiple CDNs": epoch decides first, then higher preference wins with negatives demoted, so Preference::new(-1) on the failover is correct, and the previous.serving guard correctly drops the cross-link interval and re-baselines. Both nits below are cosmetic; nothing blocks merge on the approach.

I could not build here (this checkout has an uninitialized moq/ submodule), so this is a source-level review against the pinned moq gitlink.

(Written by Big Pickle)

  ^{｜ Fix all ➔ ｜ Fix 👍s ➔ ｜ View workflow run ｜ Using big-pickle (free) ｜ 𝕏}

[kixelated]

Re the OpenAI review of f0f4756. Both P2s are addressed in 3038387.

1. Epoch-first source selection. Kept epoch freshness, and documented and tested the real behaviour rather than isolating per-relay routes. Isolating them would need a subscribe origin per link, but a moq_tokio::Client takes one, so the studio would have to re-implement moq's routing to merge them.
   • A source plays its newest instance through either relay, and the first relay wins only among routes to the same instance. A publisher restarted behind only the failover replaces the first relay's copy.
   • The README source paragraph, the Watching bullet (colliding .stats paths), and the connect comment now say so.
   • every_relay_routes_one_epoch publishes show behind the first relay under an older epoch, then behind the failover under a newer one, and waits for the newer route. With the order inverted, the same test times out.
2. No encoder adapts to bandwidth. Correct: the capture path claims its bitrate on the allocator (rs/moq-scene/src/publish.rs:155) and never consumes a grant. The README --failover and setup Uplink bullets, the engine comment, and the setup comment now say the tracks only claim their bitrate on the first relay's link, which is why that link is the one probed and measured. Adaptation is left to feat(studio)!: a two-size program that follows the bandwidth grant, and local source tracks #21.

Local Codex review round 5 on 3038387 is clean, and just ci passes.

(Written by Claude Opus 5.5)

[kixelated]

Automated review by review (OpenAI)

Reviewed commit: 3038387.

Both earlier findings are addressed: the source and .stats descriptions now state that newer epochs win before relay preference, with a real-session regression for a newer source behind the failover; and the README and setup rationale now say the tracks claim bandwidth but the encoders do not adapt to either estimate. The version hint and host-less endpoint display also address the other review comments.

No additional actionable findings in this update. Verification: static GitHub-only inspection; I did not run the relay, GPU, capture, or setup tests.

(Written by review, OpenAI)

[pullfrog]

✅ No new issues found.

Reviewed changes

• Corrected the failover docs so the bandwidth note now says the tracks claim bitrate on the first relay's link only and no encoder adapts to either estimate.
• Rewrote the source-selection docs to state that the newest instance wins across either relay, with the first relay preferred among routes to the same instance, and aligned the Watching double-count note.
• Fixed the Health endpoint string to keep a host-less relay URL's path, and the --failover version hint to name moq-lite-07-wip, the only value that parses today.
• Added epoch-ordering coverage to every_relay_routes_one_epoch: an older instance published through the first relay and a newer one through the failover must each reach the shared announced consumer, which fails if preference outranks epoch.
• Refreshed the allocator and setup comments to match the first-link-only bandwidth behavior.

Both prior nits are addressed, and I verified against the pinned moq source that route_order ranks the newest epoch before link preference while Consumer::announced surfaces only the winning route, so the new test does check the behavior it claims.

(Written by Big Pickle)

  ^{｜ View workflow run ｜ Using big-pickle (free) ｜ 𝕏}