Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,8 @@ cargo run --release -p moq-studio --features capture,openh264 -- \
Security.
- The `--connect-*`, `--quic-*`, and `--log-level` flags are
[moq-cli's](https://doc.moq.dev/bin/cli); see `moq-studio --help`.
- `moq-studio --watch <url>` opens no scene file and plays a studio broadcast
instead, as a viewer sees it; see [Watching](#watching).

## First launch

Expand Down Expand Up @@ -993,6 +995,29 @@ A take changes the layout on the frame the program starts showing the new
scene, so the published sources of the scene leaving go off and those of the
scene coming in go on at once, even through a transition.

#### Watching

`moq-studio --watch https://relay.example/anon/my-show` plays a studio
broadcast in a window as a viewer sees it: the program, with each picture of
its layer layout over it and each sound mixed with it. The URL's path names
the broadcast, as `moq play`'s does, and `--connect-token` and the `--quic-*`
flags apply to it; `--connect`, `--failover`, and `--room` are refused
beside it. It is a prototype for settling the layout's format, which goes
once `moq play` plays layers.

- Each layout takes effect when the program plays the frame it is stamped
with, and its pictures' keyframes play from its `start`.
- Each source plays its own track on the program's clock: a picture shows
when the program's picture stamped alike does, and a sound plays with the
program's sound stamped alike, at its `gain`, `muted`, `pan`, and `delay`.
A frame too late for its moment is dropped.
- A broadcast without a layout, or with a layout of another `version`, plays
its program alone; the window says why for the latter.
- Guests' broadcasts are not in the layout yet, so only sources local to the
studio play as layers. `priority` and `toggle` are not applied.
- Sound needs a build with `capture`; without it the window says so in the
log and plays the pictures alone.

### Recording

**Start recording** in the Output panel saves the program at the canvas size,
Expand Down
8 changes: 2 additions & 6 deletions quest/m2/compose/controls.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ the studio.
## Plan

Decided 2026-11-08 (maintainer, /quest-plan): the controls land in the native
player, the [watch prototype](/quest/m2/compose/watch.md) (rejected: browser
only). moq's players take them from hang's layer spec.
player, the `moq-studio --watch` prototype (`rs/moq-studio/src/watch.rs`;
rejected: browser only). moq's players take them from hang's layer spec.

Decided 2026-11-09 (maintainer, /quest-audit): ✅ viewer toggles also land in
moq's native player,
Expand All @@ -28,7 +28,3 @@ layer is unsubscribed, so it costs the viewer and the relay nothing. The
studio keeps encoding it, so toggling it back on can reach its recent past.
Subscriptions are per track, not per layer: a source shown by several layers
stays subscribed while any of them is on, at the highest priority among them.

## Required

- [moq-studio watch prototype](/quest/m2/compose/watch.md) - the viewer that exposes the toggles
5 changes: 2 additions & 3 deletions quest/m2/compose/hang-layers.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,15 @@ Decided 2026-11-08 (maintainer, /quest-plan): the layer layout becomes
official moq, so `moq play` and `@moq/watch` play a studio broadcast. Once
the wait clears, advance the gitlink with `just upgrade-moq` in this PR, move
the studio to hang's types, and delete the
[watch prototype](/quest/m2/compose/watch.md) and its flag.
`moq-studio --watch` prototype (`rs/moq-studio/src/watch.rs`) and its flag.

Decided 2026-11-09 (maintainer, /quest-audit): ✅ the wait names moq's three
quests (rejected: "both quests", unnamed, which left out `@moq/watch`).

## Waiting

- moq-dev/moq: its three layer quests land, planned in moq's m3 from the
[watch prototype](/quest/m2/compose/watch.md)'s format:
`moq-studio --watch` prototype's format, the README's "Layer layout":
[hang specifies a layer layout track](https://github.com/moq-dev/moq/blob/main/quest/m3/layers/spec.md),
[moq play composites layers](https://github.com/moq-dev/moq/blob/main/quest/m3/layers/play.md),
and
Expand All @@ -34,6 +34,5 @@ quests (rejected: "both quests", unnamed, which left out `@moq/watch`).

## Required

- [moq-studio watch prototype](/quest/m2/compose/watch.md) - the format moq specifies
- [Viewer controls](/quest/m2/compose/controls.md) - toggles and priorities the spec carries
- [The studio is caught up to moq's main](/quest/m0/pin-main.md) - moq's breaking changes since moq-dev/moq#8272, so this pin takes only its own
9 changes: 5 additions & 4 deletions quest/m2/compose/layout.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,11 @@ studio's broadcast.

Bump the layout's `version` if the change is not additive.

The `moq-studio --watch` prototype (`rs/moq-studio/src/watch.rs`) plays only
local sources' layers today: its layers play tracks of the program's own
broadcast on the program's clock, so a guest's layer needs the prototype to
follow the named broadcast, on the offset the layout carries.

## Waiting

- Maintainer: decide how the layout names a guest's broadcast. The epic
Expand All @@ -43,7 +48,3 @@ Bump the layout's `version` if the change is not additive.

Check: the maintainer answers in /quest-unblock, and the choice is recorded
in this Plan and the epic's.

## Related

- [moq-studio watch prototype](/quest/m2/compose/watch.md) - the first player of the layout
30 changes: 0 additions & 30 deletions quest/m2/compose/watch.md

This file was deleted.

21 changes: 21 additions & 0 deletions rs/moq-scene/src/collection.rs
Original file line number Diff line number Diff line change
Expand Up @@ -469,6 +469,13 @@ impl Collection {
"`{name}` draws scene `{scene}`, which is not a scene"
)));
}
if let Source::Layer { broadcast, .. } = source
&& !matches!(self.sources.get(broadcast), Some(Source::Broadcast { .. }))
{
return Err(Error::Invalid(format!(
"`{name}` plays on `{broadcast}`, which is not a broadcast source"
)));
}
let Source::Clone { source } = source else { continue };
match self.sources.get(source) {
Some(Source::Clone { .. }) => {
Expand Down Expand Up @@ -1848,6 +1855,19 @@ pub enum Source {
/// same frame. A scene drawing it cannot go on program, and the top layer
/// cannot draw it, since either would draw the program into itself.
Program,
/// A source a studio broadcast publishes independently, played from the
/// broadcast the [`Source::Broadcast`] called `broadcast` plays, on that
/// source's playout clock, so it lands with the program as the
/// [layer layout](crate::layout) places it. Never in a scene file: only
/// a player of the layout makes one.
#[serde(skip)]
Layer {
/// The broadcast source whose broadcast and clock it plays on.
broadcast: String,
/// Its entry under the broadcast catalog's
/// [`SECTION`](crate::publish::SECTION) `sources`.
source: String,
},
}

impl Source {
Expand Down Expand Up @@ -1904,6 +1924,7 @@ impl Source {
| Self::Broadcast { .. }
| Self::Browser { .. }
| Self::Replay
| Self::Layer { .. }
)
}

Expand Down
6 changes: 6 additions & 0 deletions rs/moq-scene/src/compositor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -655,6 +655,12 @@ impl Compositor {
Ok(())
}

/// Play the program scene's keyframes as if it went on program at
/// `start`, such as one a viewer joined after it went on.
pub fn set_program_start(&mut self, start: Instant) {
self.program_start = Some(start);
}

/// Refuse to take a preview scene to program that cannot go there.
fn can_take(&self, collection: &Collection) -> Result<(), Error> {
match self.preview_scene.as_deref() {
Expand Down
69 changes: 67 additions & 2 deletions rs/moq-scene/src/playout.rs
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ pub(crate) const VIDEO_DELAY: Duration = Duration::from_millis(100);
/// represent, and so how long a stalled group is waited on before it is
/// skipped, as under `moq play --delay auto`. A shorter wait would cap the
/// estimate at it, since a skipped group is never measured.
pub(crate) const MAX_DELAY: Duration = Duration::from_secs(2);
pub const MAX_DELAY: Duration = Duration::from_secs(2);

/// How far ahead of its moment a sound is handed to the bus.
///
Expand Down Expand Up @@ -179,16 +179,56 @@ impl Drop for Speaker<'_> {
}
}

/// How soon a picture held by [`heard`] looks at the playhead again: another
/// broadcast's playhead moves, or re-anchors across a gap, without telling
/// anyone reading its clock, so a held picture polls it.
pub const RECHECK: Duration = Duration::from_millis(10);

/// Whether a picture stamped `timestamp` may show at `now` on `schedule`:
/// while audio plays, only once the sound stamped with it has played, so a
/// stall in the audio alone holds the picture too.
pub(crate) fn heard(schedule: &timeline::Consumer, timestamp: moq_net::Timestamp, now: Instant) -> bool {
pub fn heard(schedule: &timeline::Consumer, timestamp: moq_net::Timestamp, now: Instant) -> bool {
schedule.target().is_none()
|| schedule
.playhead(now)
.is_some_and(|head| Duration::from(timestamp) <= head + CONTIGUOUS)
}

/// What to do at `now` with a sound stamped `timestamp` that plays on
/// `schedule`, another track's clock.
#[derive(Debug, PartialEq, Eq)]
pub(crate) enum Follow {
/// Write it to sound at this moment.
Write(Instant),
/// Look again at this moment.
Wait(Instant),
/// Its moment has passed, or the clock has none.
Drop,
}

/// Decide what to do with a followed sound from the clock as it is now, not
/// as it was when the sound arrived, so a re-anchor moves it. The clock's own
/// sound stamped alike must have been written first: when that stalls, this
/// one waits with the pictures, until its moment has passed.
pub(crate) fn follow(schedule: &timeline::Consumer, timestamp: moq_net::Timestamp, now: Instant) -> Follow {
let Some(sounds) = schedule.due(timestamp) else {
return Follow::Drop;
};
if sounds < now {
return Follow::Drop;
}
// The clock re-anchors without telling its readers, so a wait for a
// moment it may yet move earlier is rechecked too.
let write = sounds - WRITE_LEAD;
if write > now {
return Follow::Wait(write.min(now + RECHECK));
}
if !heard(schedule, timestamp, sounds) {
return Follow::Wait(now + RECHECK);
}
Follow::Write(sounds)
}

/// Writes placed frames into a positioned bus channel, each at the bus
/// position its moment maps to, so it sounds exactly then.
///
Expand Down Expand Up @@ -330,6 +370,31 @@ mod tests {
assert!(!heard(&schedule, stamp(1_100), at(500)), "shown past its sound");
}

/// A followed sound, rechecked until then, is written the bus's latency
/// before its moment once the clock's own sound stamped alike has been,
/// waits while that has not, and is dropped once its moment has passed.
#[test]
fn a_followed_sound_waits_for_the_clocks_own() {
let clock = timeline::Producer::new(VIDEO_DELAY);
let schedule = clock.consume();
let changed = Notify::new();
let mut speaker = Speaker::new(clock.clone(), &changed);
speaker.place(ms(1_000), FRAME, TARGET, at(0));
assert_eq!(
follow(&schedule, stamp(1_000), at(0)),
Follow::Wait(at(10)),
"rechecked"
);
assert_eq!(follow(&schedule, stamp(1_000), at(95)), Follow::Wait(at(100)));
assert_eq!(follow(&schedule, stamp(1_000), at(100)), Follow::Write(at(150)));
// Past what the clock's own sound has written: it stalled.
assert_eq!(follow(&schedule, stamp(1_100), at(200)), Follow::Wait(at(210)));
assert_eq!(follow(&schedule, stamp(1_100), at(250)), Follow::Wait(at(260)));
assert_eq!(follow(&schedule, stamp(1_100), at(251)), Follow::Drop);
let fresh = clock.clone().reset(VIDEO_DELAY).consume();
assert_eq!(follow(&fresh, stamp(1_000), at(0)), Follow::Drop, "no moment yet");
}

/// A stall longer than the target re-buffers: the next frame sounds the
/// target after it arrived, and the pictures stamped with it wait as long.
#[test]
Expand Down
Loading