From 2632dc99a4a47bcad7feb11fa2c42af437d4d981 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 4 Nov 2026 10:27:12 -0800 Subject: [PATCH 1/3] quest: settle keyframes; plan a media file source and stinger transitions Co-Authored-By: Claude Opus 5.5 --- quest/m0/README.md | 4 +++- quest/m0/animation.md | 46 +++++++++++++++++++----------------------- quest/m0/media-file.md | 26 ++++++++++++++++++++++++ quest/m0/stinger.md | 19 +++++++++++++++++ 4 files changed, 69 insertions(+), 26 deletions(-) create mode 100644 quest/m0/media-file.md create mode 100644 quest/m0/stinger.md diff --git a/quest/m0/README.md b/quest/m0/README.md index 4eea230..8c2a94f 100644 --- a/quest/m0/README.md +++ b/quest/m0/README.md @@ -94,7 +94,9 @@ repository. - [Broadcast sources play in sync](/quest/m0/broadcast-sync.md) - subscribed broadcasts and guests play audio and video on the publisher's timestamps, with `moq play`'s delay - [Wayland display and window sources](/quest/m0/wayland-sources.md) - portal-picked sources that reopen without the picker - [Live encode control](/quest/m0/encode-control.md) - simulcast rungs that follow the bandwidth grant, live bitrate and resolution, keyframes on request -- [Keyframes and stingers](/quest/m0/animation.md) - keyframed item properties with easing, and stinger transitions +- [Keyframes](/quest/m0/animation.md) - keyframed item properties with easing, starting when their scene goes on program +- [Media file source](/quest/m0/media-file.md) - a local video or audio file plays as a source, looping or from the start on demand +- [Stinger transitions](/quest/m0/stinger.md) - a clip played over the program that switches scenes at its cut point - [ML video effects](/quest/m0/effects.md) - background removal and other ML effects, baked into their source's track - [Destinations](/quest/m0/destinations.md) - RTMP, RTMPS, and WHIP push with per-destination codecs and shared encoders, on a UI-free egress library a hosted service can reuse - [Multiple canvases](/quest/m0/multi-canvas.md) - landscape and portrait canvases composite at once from shared sources diff --git a/quest/m0/animation.md b/quest/m0/animation.md index 64a8c92..a2c69ed 100644 --- a/quest/m0/animation.md +++ b/quest/m0/animation.md @@ -1,10 +1,10 @@ -# [M] Keyframes and stingers +# [M] Keyframes ## Goal Any item property in `moq-studio` can be keyframed with easing, so an item -can fly in or a camera can push in, and a scene change can run a stinger: a -video clip played over the program that switches scenes at its cut point. +can fly in or a camera can push in, and the same animation plays in the +program, a recording, and a replay. ## Plan @@ -12,28 +12,24 @@ Fade, slide, wipe, and move transitions are drawn by moq-scene's compositor on the program clock ([moq-dev/studio#33](https://github.com/moq-dev/studio/pull/33)). Animate in moq-scene on the media clock, so the program, a recording, and a replay match. -Carrying transitions and keyframes in the layout track, for viewers to -compose, waits for the m1 [composition epic](/quest/m1/compose/README.md) -(changed 2026-11-04, maintainer: the program comes first). Overlays animate -through their own CSS. No Lottie runtime for now (decided 2026-10-26): the -open ones either lack text (velato) or add a C++ build (ThorVG). Keyframes and -stinger settings change through moq-scene's `Edit`, so they undo like any edit. - -A stinger clip is a media source like any other, but no source plays a media -file yet, and a stinger must start its clip as the transition starts or its -cut point lands at random. It waits on a media-file source that plays from -the start on demand, which no quest plans yet. - -Open questions for the maintainer: - -- What starts an item's keyframes: its scene going on program (and into - preview, for the preview), a loop on the program clock, or a trigger such - as a hotkey. Recommended: the time since its scene went on program, playing - once and holding the last keyframe. -- How a keyframed item edits: a drag in the preview sets the keyframe at the - preview's time, or the item's resting place, from which keyframes are - offsets. +Carrying keyframes in the layout track, for viewers to compose, waits for the +m1 [composition epic](/quest/m1/compose/README.md) (changed 2026-11-04, +maintainer: the program comes first). Overlays animate through their own CSS. +No Lottie runtime for now (decided 2026-10-26): the open ones either lack text +(velato) or add a C++ build (ThorVG). Keyframes change through moq-scene's +`Edit`, so they undo like any edit. + +Decided 2026-11-04 (maintainer): + +- **Start:** an item's keyframes run on the time since its scene went on + program (and into preview, for the preview), play once, and hold the last + keyframe. Rejected: a loop on the program clock, and a trigger or hotkey. +- **Editing:** a drag in the preview moves the item's resting place, and + keyframes are offsets from it, so editing needs no timeline scrubber. + Rejected: setting the keyframe at the preview's time. + +Stingers are their own quest, [Stinger transitions](/quest/m0/stinger.md). ## Related -- [Layout track](/quest/m1/compose/layout.md) - where transitions and keyframes travel to viewers in m1 +- [Layout track](/quest/m1/compose/layout.md) - where keyframes travel to viewers in m1 diff --git a/quest/m0/media-file.md b/quest/m0/media-file.md new file mode 100644 index 0000000..9b59089 --- /dev/null +++ b/quest/m0/media-file.md @@ -0,0 +1,26 @@ +# [M] Media file source + +## Goal + +A source in `moq-studio` plays a local video or audio file, such as an intro +clip or background music: looping, or from the start each time something asks, +with its sound joining the mixes like any other source. + +## Plan + +Decided 2026-11-04 (maintainer): planned so stinger transitions have a clip +that starts on demand. Demux and decode through the moq crates the studio +already uses (moq-mux import, moq-video and moq-audio decode) rather than a +new media stack; anything they lack lands in moq first, with a moq-side test, +then a pin. Start with the container and codecs those crates already read, and +refuse anything else at setup with the reason. The file path is a local path +in the scene file, like an image source's. + +A file is played on the program clock. Settings: loop, or play once and hold +the last frame; and whether going on program restarts it. A restart on demand +(the stinger's need) is an API on the source, not only a setting. A file that +ends or fails reads as ended or failed in the Sources panel and Health. + +## Related + +- [Stinger transitions](/quest/m0/stinger.md) - the first consumer of a clip that starts on demand diff --git a/quest/m0/stinger.md b/quest/m0/stinger.md new file mode 100644 index 0000000..9c6847e --- /dev/null +++ b/quest/m0/stinger.md @@ -0,0 +1,19 @@ +# [S] Stinger transitions + +## Goal + +A scene change in `moq-studio` can run a stinger: a video clip played over the +program that switches scenes at its cut point, as broadcast graphics do. + +## Plan + +Split from [Keyframes](/quest/m0/animation.md) on 2026-11-04 (maintainer). A +stinger is a transition kind beside fade, slide, wipe, and move +([moq-dev/studio#33](https://github.com/moq-dev/studio/pull/33)), drawn on the +program clock, whose clip is a media file source. It must start its clip as +the transition starts, or its cut point lands at random. Its settings (clip +and cut point) change through `Edit`, so they undo. + +## Required + +- [Media file source](/quest/m0/media-file.md) - a clip that starts on demand From 8991592a47762fa2bbeb42265579f6e90a0b4938 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 4 Nov 2026 11:54:44 -0800 Subject: [PATCH 2/3] quest: say how the web studio gets a media file Co-Authored-By: Claude Opus 5.5 --- quest/m0/media-file.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/quest/m0/media-file.md b/quest/m0/media-file.md index 9b59089..c96c9a2 100644 --- a/quest/m0/media-file.md +++ b/quest/m0/media-file.md @@ -14,7 +14,10 @@ already uses (moq-mux import, moq-video and moq-audio decode) rather than a new media stack; anything they lack lands in moq first, with a moq-side test, then a pin. Start with the container and codecs those crates already read, and refuse anything else at setup with the reason. The file path is a local path -in the scene file, like an image source's. +in the scene file, like an image source's. The web studio has no local paths: +there a file comes from a browser file picker for the session, and resolving +scene-file paths in the browser belongs to the [web studio](/quest/m0/web.md), +as it does for image sources. A file is played on the program clock. Settings: loop, or play once and hold the last frame; and whether going on program restarts it. A restart on demand From 41a6bab897b0dece71e5e2cd4c50e6551dbe0b63 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 4 Nov 2026 12:14:29 -0800 Subject: [PATCH 3/3] quest: an audio-only media file goes silent after playing once Co-Authored-By: Claude Opus 5.5 --- quest/m0/media-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/quest/m0/media-file.md b/quest/m0/media-file.md index c96c9a2..8f5008d 100644 --- a/quest/m0/media-file.md +++ b/quest/m0/media-file.md @@ -20,7 +20,7 @@ scene-file paths in the browser belongs to the [web studio](/quest/m0/web.md), as it does for image sources. A file is played on the program clock. Settings: loop, or play once and hold -the last frame; and whether going on program restarts it. A restart on demand +the last frame (an audio-only file goes silent after its last sample); and whether going on program restarts it. A restart on demand (the stinger's need) is an API on the source, not only a setting. A file that ends or fails reads as ended or failed in the Sources panel and Health.