Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/.vitepress/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ export default defineConfig({
{ text: "Install", link: "/setup/install" },
{ text: "Development", link: "/setup/dev" },
{ text: "Production", link: "/setup/prod" },
{ text: "Redundant ingest", link: "/setup/redundancy" },
{ text: "Upgrade", link: "/setup/upgrade" },
{ text: "Coding agents", link: "/setup/agent" },
],
Expand Down
10 changes: 3 additions & 7 deletions doc/bin/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -347,13 +347,9 @@ Other versions carry no epoch, so a restart from another session is the newest
route at an equal cost, or a better one.

Each independent importer publishes its own epoch, even when two importers read
one stream and publish one path. On a version that carries epochs, the newest
importer wins new requests. If it leaves, a still-live older importer wins again
as a restart. Players following announcements and `moq export ts --stitch`
follow those restarts; other consumers end. `--stitch` also switches as soon as a newer
importer starts, even while the current importer remains healthy. Each switch
marks a program discontinuity, so redundant ingest does not promise hitless
output.
one stream and publish one path. See [Redundant ingest](/setup/redundancy) for
same-path switching, subscriber-controlled selection on separate paths, and
several routes to one publisher.

An importer announces once its catalog names a servable track. Input that ends
before then, such as an empty input or a header that configures no track,
Expand Down
2 changes: 2 additions & 0 deletions doc/bin/relay/cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ worker.

Failover routes must carry copies of one publisher's broadcast. Independent
importers use distinct epochs and switch as a restart, even when fed one stream.
See [Redundant ingest](/setup/redundancy) to choose between publisher switching,
subscriber selection, and several routes to one publisher.
A source whose track differs in timescale, retention, publisher priority, or
group order is refused for that track.

Expand Down
128 changes: 128 additions & 0 deletions doc/setup/redundancy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
---
title: Redundant Ingest
description: Protect publishers and delivery routes without confusing broadcast identity
---

# Redundant ingest

Choose what must survive: a publisher failing, a delivery route failing, or
both. Two independent publishers can announce one path for automatic switching,
or use separate paths so the subscriber chooses. Several routes carrying one
publisher's broadcast can resume the same stream.

| Setup | What a failure does |
| --- | --- |
| Independent publishers, one path | Switches to another broadcast at that path. |
| Independent publishers, separate paths | Leaves selection to the subscriber. |
| One publisher, several routes | Resumes the same broadcast on another route. |

A [path and publisher epoch](/concept/moq-lite#publisher-epochs) identify one
broadcast. Every independent publisher mints its own epoch, even if both read
the same source. Sharing an epoch would claim that their group and frame
numbers name the same content, which independently running importers cannot
promise.

## Independent publishers on one path

1. Run the publishers on separate hosts and ingress routes. Connect their
relays into a [cluster](/bin/relay/cluster) so the viewer's relay can reach
both. Give both publishers permission to announce the same full path.
2. Publish the same broadcast name under the same auth path. Each publisher
keeps its own epoch; `moq` creates one automatically.
3. Use epoch-capable connections on every hop: moq-lite 07, which is opt-in,
or moq-transport draft 17 or later with the [epoch extension](/draft/moq-epoch)
enabled. Keep the publisher hosts' clocks synchronized, since epochs order
by their UUIDv7 creation time.

For example, two HLS importers can publish `event.hang` through different
relays. Run each command on its publisher host, with its own source URL:

```bash
# Publisher A
moq --connect https://ingest-a.example.com/anon \
--connect-version moq-transport-17 --broadcast event.hang \
import hls https://source-a.example.com/live/index.m3u8

# Publisher B
moq --connect https://ingest-b.example.com/anon \
--connect-version moq-transport-17 --broadcast event.hang \
import hls https://source-b.example.com/live/index.m3u8
```

The relays must forward these announcements under the same path; publishing to
two disconnected relays does not make one a fallback for the other. The same
setup applies to other publishers, including browser capture and ingest
gateways. See [Production](/setup/prod) for TLS and authentication.

At the same path, the newest epoch wins new requests, ahead of route cost.
Starting B after A therefore switches followers to B even while A is healthy.
If B's route goes away and A remains live and reachable, A wins again as a new
broadcast. This is not a fixed primary/standby priority: restarting either
publisher can make it the newest.

The switch is announced as a `Restart`. Players that follow announcements
request the path again and begin the replacement broadcast. Existing
subscriptions stay on the old instance while it remains available, unless the
consumer chooses to follow the restart.

For MPEG-TS output, opt into following replacements:

```bash
moq --connect https://edge.example.com/anon \
--connect-version moq-transport-17 --broadcast event.hang \
export ts --stitch --linger 10s > event.ts
```

`--stitch` follows a replacement as a program switch: its catalog supplies a
new PMT and every PID flags the discontinuity. Starting a backup can cause
this switch too. `--linger 10s` bounds waiting after an end and waiting for a
replacement's first catalog; choose that window for your application. It does
not make the switch hitless. Without `--stitch`, the TS export stays on the
old instance while it is up, then fails on a replacement. Other stdout export
formats end with their broadcast; they do not follow a replacement. TS
passthrough also cannot stitch. See [CLI retention and latency](/bin/cli#retention-and-latency).

A connection without epochs cannot prove continuity or apply the newest-epoch
rule: routing chooses among its epochless routes instead, and another route
winning is a restart. Do not rely on same-path continuity through such a hop.

## Independent publishers on separate paths

Publish each stream under its own name, such as `event/a.hang` and
`event/b.hang`. The end subscriber can subscribe to both and decide whether to
switch between them, combine them, or keep them separate. This is useful when
selection should depend on the media or operator policy rather than which
publisher started last.

Each publisher produces its own stream with its own epoch. The relay routes
the two paths independently; it does not align their frames or stitch their
output. The subscriber owns any transition between them.

## One publisher through several routes

Keep a single publisher instance and distribute copies of its broadcast
through multiple relays or CDNs. All copies retain its path, epoch, group
numbers, and frame content.

Connect the publisher's relay to multiple peers, then make those routes
reachable from the subscriber's relay. An application can also hold sessions
to [multiple CDNs](/concept/moq-lite#multiple-cdns) feeding one shared origin
and rank its links. Do not start a second importer to create the second copy.

When the serving route fails or another route with the same epoch wins, the
subscription resumes from the first frame it lacks. This needs an explicit
matching epoch across every hop and a surviving route that can still serve
those frames within the broadcast's retention. It preserves broadcast
continuity, but a late recovery can still miss playback deadlines. Losing the
only publisher cannot be repaired by adding routes to it.

## Hitless protection at the boundaries

For ST 2022-7 protection downstream, form both RTP legs from one broadcast and
supply matching RTP packetization in the downstream system. Independent
publishers switching at one path are different broadcasts, so their output is
not a pair of identical protection legs.

To protect the input to an importer hitlessly, merge duplicate legs before a
single importer, using upstream protection equipment. That importer then
publishes one broadcast, which can travel over several MoQ routes.
1 change: 0 additions & 1 deletion quest/m1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,6 @@ lands before the cut.
- [Announcement takeover](/quest/m1/announce-takeover.md) - a new independent route explicitly continues instances served through its predecessor, with shared Rust/JS semantics and lite-07 wire
- [IETF takeover](/quest/m1/ietf-takeover.md) - carry the shared takeover contract through the negotiated IETF epoch extension
- [Identity and recovery guide](/quest/m1/identity-guide.md) - explain pinned catalogs, lazy jobs, takeover, restart and version compatibility
- [Redundant ingest guide](/quest/m1/redundant-ingest-guide.md) - a setup page on redundant ingest over multiple routes, for any publisher
- [Generated C bindings](/quest/m1/c/README.md) - C generated from moq-ffi ships as `moq-c` 0.8.0 and replaces the hand-written libmoq
- [moq-ffi reports why a first dial fails](/quest/m1/ffi-first-dial-failure.md) - OBS names the reason while the first dial retries, as moq-c already can
- [Scope track priority](/quest/m1/track-priority-scope.md) - priority orders one owner's streams, and a shared cluster session is fair across tenants
Expand Down
37 changes: 0 additions & 37 deletions quest/m1/redundant-ingest-guide.md

This file was deleted.

Loading