diff --git a/quest/README.md b/quest/README.md index 8d7f9648..ccf43bd7 100644 --- a/quest/README.md +++ b/quest/README.md @@ -18,7 +18,7 @@ there first. A quest here that needs it waits on a pin quest, which ## Required -- [m0: the studio](/quest/m0/README.md) - the native and wasm studio for creators who rely on OBS today -- [m1: after the studio ships](/quest/m1/README.md) - viewers and a server compose a broadcast from its sources and a layout track, plus studio features held past m0 +- [m0: the studio](/quest/m0/README.md) - the native studio for creators who rely on OBS today +- [m1: after the studio ships](/quest/m1/README.md) - studio features held past m0; empty now - [m2: later](/quest/m2/README.md) - work with no consumer yet - [m3: someday](/quest/m3/README.md) - work no customer asks for, kept so its plans are not lost diff --git a/quest/m0/README.md b/quest/m0/README.md index de75730d..f2b570a7 100644 --- a/quest/m0/README.md +++ b/quest/m0/README.md @@ -3,7 +3,7 @@ ## Goal A native, cross-platform streaming studio (`rs/moq-studio`) for creators and -teams who rely on OBS today, plus the same app built to wasm. It composites +teams who rely on OBS today. It composites cameras, screens, media, browser sources, and remote MoQ guests into scenes on the GPU with moq-video and moq-audio, publishing several scenes at once (landscape and portrait), and goes live to a MoQ relay and directly to RTMP, @@ -29,9 +29,8 @@ Decided 2026-10-22 with the maintainer (/quest-plan): feature, gpui has no wgpu on macOS or Windows, and Tauri is a webview. The app owns the wgpu device. - **Compositor:** a UI-free crate, `rs/moq-scene`: scene graph, sources, - transforms, transitions, and canvases on wgpu. The program audio bus goes - in moq-audio, whose only mixer today is output-only - (`rs/moq-audio/src/playback/mixer.rs`). Both grow slice by slice with the + transforms, transitions, and canvases on wgpu. The program audio bus is + moq-audio's `bus` module. Both grow slice by slice with the studio UI that consumes them, never ahead of it. - **Outputs:** the studio publishes to MoQ and pushes every RTMP(S) and WHIP destination itself. It remuxes its own broadcast through a UI-free egress @@ -47,6 +46,12 @@ Decided 2026-10-22 with the maintainer (/quest-plan): control. Its media runs on new browser backends in moq-video and moq-audio (getUserMedia, getDisplayMedia, WebCodecs) rather than a bridge to js/publish and js/watch, so one Rust media path drives both builds. + Changed 2026-11-09 (maintainer, /quest-audit): ✅ the + [web studio](/quest/m2/web.md) and its two pins moved to m2 (rejected: + keeping them in m0), so m0 ships the native studio alone. moq removed all + WASM support on 2026-10-31, its browser-backend quests are deferred with + no other consumer, and nothing plans a browser WebTransport session for + moq-net. - **Guests:** guests and phone cameras join a moq-room prefix the studio owns, through a web join link or QR code. - **Naming:** quests name no streaming platforms; ingest requirements are @@ -76,9 +81,10 @@ broadcast reaches viewers, not the fallback. Sources are not published to the relay by default (decided 2026-11-03 on [moq-dev/studio#21](https://github.com/moq-dev/studio/pull/21)), because each encoding costs an encoder session and consumer GPUs have few, so the -[viewer-side composition](/quest/m2/compose/README.md) epic moved to m1. -Overlays and animation render into the program here, and travel to viewers -in the layout track only once that epic lands. +[viewer-side composition](/quest/m2/compose/README.md) epic left m0 (now m2, +replanned 2026-11-08 around independently published sources layered over the +program). Overlays and animation render into the program here; that epic's +layout track carries only the layers of independently published sources. Dropped from the first scoping round: relay fan-out ("encode once, the relay restreams") and its upload savings. Direct push with shared encoders matches @@ -89,24 +95,25 @@ repository. - [Studio on macOS with capture granted](/quest/m0/app-macos.md) - camera, screen, and audio capture run on a Mac, not only Linux - [nixos2 boots Windows for agents](/quest/m0/windows-host.md) - a Windows machine agents reach over ssh, with a camera and microphone +- [The studio builds for Windows in CI](/quest/m0/windows-build.md) - a nightly `windows-latest` build, which needs no Windows host - [Studio on Windows](/quest/m0/app-windows.md) - the studio runs on Windows, not only Linux - [Wayland display and window sources](/quest/m0/wayland-sources.md) - portal-picked sources that reopen without the picker - [Destinations](/quest/m0/destinations.md) - RTMP, RTMPS, and WHIP push with per-destination codecs and shared encoders, on a UI-free egress library a hosted service can reuse +- [AAC for destinations](/quest/m0/destinations-aac.md) - AAC for the ingests that require it, from the platform encoders - [Clap-to-measure sync delay](/quest/m0/sync-clap.md) - a clap sets a microphone's delay against its camera -- [Audio filters](/quest/m0/audio-filters.md) - per-source noise suppression, gate, compressor, limiter, and plugins, configured from the studio -- [Browser sources on macOS and Windows](/quest/m0/browser-source.md) - the out-of-process CEF pages Linux has, with shared GPU textures on the other desktops -- [Web studio](/quest/m0/web.md) - the full studio in wasm on WebGPU and WebCodecs, served from this repository +- [Audio filters](/quest/m0/audio-filters.md) - noise suppression, gate, compressor, limiter, and plugins on each microphone and system audio source, configured from the studio +- [Browser sources on macOS and Windows](/quest/m0/browser-source/README.md) - the out-of-process CEF pages Linux has, with shared GPU textures on the other desktops - [Virtual camera](/quest/m0/vcam/README.md) - the program or a published scene exposed as a system camera on Linux, Windows, and macOS -- [The Apple signing identity is available](/quest/m0/signing.md) - the Apple credentials packaging and the macOS camera need; Windows ships unsigned -- [Packaging](/quest/m0/packaging.md) - a signed macOS app, a Windows installer, and a Flatpak for Linux +- [The Apple signing identity is available](/quest/m0/signing.md) - the Apple credentials the macOS app and its camera need; Windows ships unsigned +- [Packaging](/quest/m0/packaging/README.md) - a signed macOS app, a Windows installer, and a Flatpak for Linux, each its own quest - [The persistable portal restore token is pinned](/quest/m0/pin-portal-token.md) - moq-video's restore token, for Wayland sources -- [Platform AAC encoders are pinned](/quest/m0/pin-aac-encode.md) - moq-audio's AAC encoders, for destinations +- [Platform AAC encoders are pinned](/quest/m0/pin-aac-encode.md) - moq-audio's AAC encoders, for AAC destinations - [moq-audio's filter chain is pinned](/quest/m0/pin-audio-filters.md) - the chain audio filters configure -- [moq-video's browser backends are pinned](/quest/m0/pin-browser-video.md) - the web studio's video slice and browser composition -- [moq-audio's browser backends are pinned](/quest/m0/pin-browser-audio.md) - the web studio's audio slice +- [moq's egress dial changes are pinned](/quest/m0/pin-egress-dial.md) - RTMPS that keeps `tcUrl` and a caller-supplied dial policy, for destinations - [moq-audio's source tap is pinned](/quest/m0/pin-audio-tap.md) - the samples clap-to-measure sync times a clap from - [Playout writes land on positioned bus channels](/quest/m0/playout-positioned.md) - broadcast, media, and replay sound at the bus position its pictures are timed for - [moq-audio's positioned bus channel is pinned](/quest/m0/pin-bus-positioned.md) - `Bus::positioned`, which playout writes through +- [moq-mux's decoupled playout clock is pinned](/quest/m0/pin-sync.md) - `moq_mux::sync` in place of `moq_mux::timeline`, which every later pin crosses ## Related diff --git a/quest/m0/app-windows.md b/quest/m0/app-windows.md index 52e76066..9ed96baa 100644 --- a/quest/m0/app-windows.md +++ b/quest/m0/app-windows.md @@ -12,16 +12,25 @@ the studio landed. - System audio is `Unsupported` there until [Windows capture parity](https://github.com/moq-dev/moq/blob/main/quest/m3/capture-windows.md) adds WASAPI loopback, so that quest runs it. -- Build with `--features capture`, since a default build has image, color, - and broadcast sources only. +- Build with `--features capture`, since a default build has no camera, + display, window, microphone, or system-audio sources. - Hotkeys, from [moq-dev/studio#84](https://github.com/moq-dev/studio/pull/84), only compiled for Windows so far: run the keypad keys, the refusal of keypad Enter (Windows' hotkey API cannot tell it from Enter), and a keyboard layout switch while a game has focus, which re-registers every binding on its physical key. +- The effects setting's read of Windows' animation setting + (`rs/moq-studio/src/fx/motion.rs`, from + [moq-dev/studio#85](https://github.com/moq-dev/studio/pull/85)) has not run + there either. + +Decided 2026-11-09 (maintainer, /quest-audit): ✅ compiling for Windows is +[its own quest](/quest/m0/windows-build.md) (rejected: folding it in here), +since a hosted runner does it without the host this quest waits on. ## Required +- [The studio builds for Windows in CI](/quest/m0/windows-build.md) - the crate compiling for Windows before it is run there - [nixos2 boots Windows for agents](/quest/m0/windows-host.md) - the Windows machine this runs on ## Related diff --git a/quest/m0/audio-filters.md b/quest/m0/audio-filters.md index a5569833..3ce623df 100644 --- a/quest/m0/audio-filters.md +++ b/quest/m0/audio-filters.md @@ -2,30 +2,38 @@ ## Goal -Each audio source's filter chain (gain, noise suppression, noise gate, -compressor, limiter, ducking, and VST3 and CLAP plugins) is configured from -the studio and saved with the scene collection. Ducking lowers one source -(music) when another (the microphone) is active. A plugin that fails or keeps -missing its deadline shows as failed on its source. +Each microphone's and system audio source's filter chain (noise suppression, +noise gate, compressor, limiter, ducking, and VST3 and CLAP plugins) is +configured from the studio and saved with the scene collection. Ducking +lowers one source (music) when another (the microphone) is active. A plugin +that fails or keeps missing its deadline shows as failed on its source. + +Guest, broadcast, media, browser, and replay sound stays unfiltered. ## Plan Changed 2026-11-02 (maintainer), when the studio moved out of moq: the chain itself, its filters, and the out-of-process plugin host are moq-audio work in -[moq-audio filter chain](https://github.com/moq-dev/moq/blob/main/quest/m5/audio-filters.md). +[moq-audio filter chain](https://github.com/moq-dev/moq/blob/main/quest/m3/audio-filters.md), +which moves to moq's m3 in [moq-dev/moq#8421](https://github.com/moq-dev/moq/pull/8421). This quest is the studio's side: the panel, the scene-file shape, and wiring each source's chain into the program bus. Each microphone's and system audio's own track encodes its capture, before the bus applies gain and mute (`Driver::listen` in `rs/moq-scene/src/source.rs`). The chain goes before that tap, so the track -carries filtered sound, as video bakes in its ML effects (decided 2026-11-02, -maintainer). +carries filtered sound (decided 2026-11-02, maintainer), unlike a picture's +track, which is its capture before any filter. The scene file names filters and their settings, never a plugin's absolute path, so a collection stays portable; a missing plugin loads as failed rather than refusing the file. +Decided 2026-11-09 (maintainer, /quest-audit): ✅ the Goal covers +microphones and system audio only (rejected: widening moq's chain to bus +channels). moq's chain runs on the capture producer, so only captured sources +have one; sound that reaches the bus another way never passes through it. + ## Required - [moq-audio's filter chain is pinned](/quest/m0/pin-audio-filters.md) - the chain this configures diff --git a/quest/m0/browser-source.md b/quest/m0/browser-source.md deleted file mode 100644 index 7c4c5247..00000000 --- a/quest/m0/browser-source.md +++ /dev/null @@ -1,51 +0,0 @@ -# [L] Browser sources on macOS and Windows - -## Goal - -Browser sources run on macOS and Windows as they do on Linux: a URL (an -overlay, alert box, or chat widget) renders into the compositor at the canvas -frame rate, the creator can interact with it, its audio joins the bus, and a -crash drops browser sources, never the stream: a page's renderer only its own -source, the shared browser process every browser source. - -## Plan - -Decided 2026-10-22: use CEF through `cef-rs`, desktop only. -- **Rejected engines:** Servo's site compatibility is not yet enough for - common overlay sites. Blitz runs no JavaScript. -- **wasm build:** omits browser sources until a web API can draw HTML into a - canvas. -- **Rendering:** CEF offscreen rendering with a shared texture - (`OnAcceleratedPaint`), imported into wgpu as a D3D11 shared handle on - Windows, an IOSurface on macOS, and a dmabuf on Linux. The CPU `OnPaint` - path is the fallback and shows in stats. -- **Process:** CEF runs out of process. - -Linux landed first (`moq_scene::browser`): one helper process every page -shares, on one persistent profile, which is the studio's executable started -again, talking over a Unix socket on its stdin that carries the DMA-BUF -descriptors, and `cef-rs`'s `osr_texture_import` copies each texture on the -studio's device. The profile lives in the XDG data directory -(`engine::browser_profile`); each platform keeps it in its own per-user data -directory instead. What the other platforms need on top: -- **Windows:** a channel without Unix sockets, such as the helper's stdio - pipes. The studio duplicates CEF's NT handle out of the helper with - `DuplicateHandle` before `cef-rs` imports it. -- **macOS:** the helper is CEF's browser process, so it needs an app bundle - with CEF's helper apps (`cef-rs`'s `bundle-cef-app`) and its own - NSApplication. An IOSurface crosses processes only as a Mach port - (`IOSurfaceCreateMachPort`), so the channel needs a Mach port rendezvous. -- **Exiting with the studio:** the studio holds the channel open until the - helper exits and stops it with `Command::Stop`, so the helper reads the - channel's end, from before CEF starts, as the studio dying, and kills the - process group it leads with CEF's processes. macOS keeps that as is. - Windows has no process groups: start the helper in a job object with - `JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE`, which the studio's handle keeps alive. - -## Required - -- [Studio on Windows](/quest/m0/app-windows.md) - the studio runs there at all; it already runs on macOS - -## Related - -- [Studio on macOS with capture granted](/quest/m0/app-macos.md) - the rest of the macOS app, which pages do not need diff --git a/quest/m0/browser-source/README.md b/quest/m0/browser-source/README.md new file mode 100644 index 00000000..2f7074ae --- /dev/null +++ b/quest/m0/browser-source/README.md @@ -0,0 +1,41 @@ +# Browser sources on macOS and Windows + +## Goal + +Browser sources run on macOS and Windows as they do on Linux: a URL (an +overlay, alert box, or chat widget) renders into the compositor at the canvas +frame rate, the creator can interact with it, its audio joins the bus, and a +crash drops browser sources, never the stream: a page's renderer only its own +source, the shared browser process every browser source. Each platform lands +on its own. + +## Plan + +Decided 2026-10-22: use CEF through `cef-rs`, desktop only. +- **Rejected engines:** Servo's site compatibility is not yet enough for + common overlay sites. Blitz runs no JavaScript. +- **wasm build:** omits browser sources until a web API can draw HTML into a + canvas. +- **Rendering:** CEF offscreen rendering with a shared texture + (`OnAcceleratedPaint`), imported into wgpu as a D3D11 shared handle on + Windows, an IOSurface on macOS, and a dmabuf on Linux. The CPU `OnPaint` + path is the fallback and shows in stats. +- **Process:** CEF runs out of process. + +Linux landed first (`moq_scene::browser`): one helper process every page +shares, on one persistent profile, which is the studio's executable started +again, talking over a Unix socket on its stdin that carries the DMA-BUF +descriptors, and `cef-rs`'s `osr_texture_import` copies each texture on the +studio's device. The profile lives in the XDG data directory +(`engine::browser_profile`); each platform keeps it in its own per-user data +directory instead. + +The studio holds the channel open until the helper exits and stops it with +`Command::Stop`, so the helper reads the channel's end, from before CEF +starts, as the studio dying, and kills the process group it leads with CEF's +processes. Each platform's quest says what it needs on top. + +Decided 2026-11-09 (maintainer, /quest-audit): ✅ one quest per platform +(rejected: Related links only; leaving one quest). One quest held macOS +behind the Windows host, and shaped the macOS app bundle with no link to the +[quest that owns it](/quest/m0/packaging/macos.md). diff --git a/quest/m0/browser-source/macos.md b/quest/m0/browser-source/macos.md new file mode 100644 index 00000000..7db84120 --- /dev/null +++ b/quest/m0/browser-source/macos.md @@ -0,0 +1,28 @@ +# [M] Browser sources on macOS + +## Goal + +Browser sources run on macOS as the +[epic](/quest/m0/browser-source/README.md) describes, each page's shared +texture imported into wgpu as an IOSurface. + +## Plan + +On top of what Linux landed: + +- The helper is CEF's browser process, so it needs an app bundle with CEF's + helper apps (`cef-rs`'s `bundle-cef-app`) and its own NSApplication. They + go into the bundle [the macOS app](/quest/m0/packaging/macos.md) owns, and + through its signing, rather than a second bundle. +- An IOSurface crosses processes only as a Mach port + (`IOSurfaceCreateMachPort`), so the channel needs a Mach port rendezvous. +- Exiting with the studio stays as on Linux: the helper kills the process + group it leads. + +## Required + +- [macOS app](/quest/m0/packaging/macos.md) - the app bundle CEF's helper apps join + +## Related + +- [Studio on macOS with capture granted](/quest/m0/app-macos.md) - the rest of the macOS app, which pages do not need diff --git a/quest/m0/browser-source/windows.md b/quest/m0/browser-source/windows.md new file mode 100644 index 00000000..7b78d2b5 --- /dev/null +++ b/quest/m0/browser-source/windows.md @@ -0,0 +1,22 @@ +# [M] Browser sources on Windows + +## Goal + +Browser sources run on Windows as the +[epic](/quest/m0/browser-source/README.md) describes, each page's shared +texture imported into wgpu as a D3D11 shared handle. + +## Plan + +On top of what Linux landed: + +- A channel without Unix sockets, such as the helper's stdio pipes. The + studio duplicates CEF's NT handle out of the helper with `DuplicateHandle` + before `cef-rs` imports it. +- Windows has no process groups: start the helper in a job object with + `JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE`, which the studio's handle keeps + alive. + +## Required + +- [Studio on Windows](/quest/m0/app-windows.md) - the studio runs there at all diff --git a/quest/m0/destinations-aac.md b/quest/m0/destinations-aac.md new file mode 100644 index 00000000..971e7314 --- /dev/null +++ b/quest/m0/destinations-aac.md @@ -0,0 +1,29 @@ +# [S] AAC for destinations + +## Goal + +A destination can take AAC: the creator, or a preset, picks it for a +destination whose ingest requires it, and the studio encodes it with the +platform's encoder on macOS and Windows. Destinations that share a picture +scene, an audio scene, and encode settings still share one encoder. On Linux, +choosing AAC fails loudly when no encoder is available. + +## Plan + +Decided 2026-11-09 (maintainer, /quest-audit): ✅ split from +[Destinations](/quest/m0/destinations.md) (rejected: leaving that quest +blocked on AAC), which lands with Opus first. + +Decided 2026-10-22 with the maintainer, carried from that quest: + +- **AAC for RTMP ingests,** which commonly require H.264 and AAC. Opus stays + the default. +- **AAC encoders:** platform encoders only, from the AudioToolbox and Media + Foundation quests. moq-audio has no software AAC tier, and none is + planned. On Linux, choosing AAC for a destination fails loudly when no + encoder is available. It never silently falls back to another codec. + +## Required + +- [Destinations](/quest/m0/destinations.md) - the egress library and the per-destination codec choice AAC joins +- [Platform AAC encoders are pinned](/quest/m0/pin-aac-encode.md) - AAC on macOS and Windows diff --git a/quest/m0/destinations.md b/quest/m0/destinations.md index 1e8b45c9..8fefbb2b 100644 --- a/quest/m0/destinations.md +++ b/quest/m0/destinations.md @@ -7,7 +7,8 @@ MoQ, each with its own codec and bitrate, and goes live to all of them at once. Destinations that take the same picture scene and audio scene with identical encode settings share one encoder. Each destination's status and errors show in the UI. Going live and stopping are actions the Hotkeys panel can bind, -as recording is. +as recording is. Audio is Opus here; AAC is +[AAC for destinations](/quest/m0/destinations-aac.md). The fan-out lands here as a UI-free egress library, with the studio as its first consumer. The library subscribes once to a MoQ broadcast and remuxes it @@ -33,22 +34,20 @@ Decided 2026-10-22 with the maintainer: - **Redials:** a dropped destination redials with backoff and can be cancelled. A refusal such as a rejected stream key is terminal and is reported instead of retried. - - **RTMPS:** moq-rtmp dials `rtmps://` natively and keeps the URL-derived - `tcUrl` that plaintext `Client::connect` advertises. Today the TLS route, - `Client::with_stream`, sends no `tcUrl`, and several ingest servers - reject that. + - **RTMPS:** dials `rtmps://` through moq-rtmp, whose TLS route sends no + `tcUrl` today, which several ingest servers reject. moq's + [RTMPS keeps `tcUrl`](https://github.com/moq-dev/moq/blob/main/quest/m3/rtmp-rtmps-tcurl.md) + fixes that, planned in + [moq-dev/moq#8421](https://github.com/moq-dev/moq/pull/8421). - **Untrusted destination URLs:** a server-side host must be able to - refuse private and loopback addresses, so dialing goes through a - caller-supplied policy rather than resolving any URL it is handed. + refuse private and loopback addresses, so dialing goes through the + policy moq's + [caller-supplied dial policy](https://github.com/moq-dev/moq/blob/main/quest/m3/webrtc-dial-policy.md) + adds (also planned in #8421), rather than resolving any URL it is handed. - **The studio chooses codecs, not a server:** - Opus by default. - - AAC for RTMP ingests, which commonly require H.264 and AAC. - E-RTMP HEVC or AV1 only when the creator picks it for a destination that accepts it. -- **AAC encoders:** platform encoders only, from the AudioToolbox and Media - Foundation quests. moq-audio has no software AAC tier, and none is - planned. On Linux, choosing AAC for a destination fails loudly when no - encoder is available. It never silently falls back to another codec. - **Stream keys** go in the OS credential store, never in the scene file, which is meant to be shared and versioned. Prefer a maintained crate such as `keyring`. @@ -63,6 +62,20 @@ Decided 2026-10-22 with the maintainer: CBR) for common ingest constraints without naming platforms. The creator can override every field. +Decided 2026-11-09 (maintainer, /quest-audit): + +- ✅ **Unblock it from AAC** (rejected: leave it blocked on AAC). RTMP with + Opus and WHIP work at today's pin, so AAC, its codec decisions, and the + wait on moq's platform encoders moved to + [AAC for destinations](/quest/m0/destinations-aac.md). +- ✅ **The two moq changes are moq's quests** (rejected: planning them inline + here, with no moq quest or pin). This quest uses them through + [the pin](/quest/m0/pin-egress-dial.md). + ## Required -- [Platform AAC encoders are pinned](/quest/m0/pin-aac-encode.md) - AAC on macOS and Windows +- [moq's egress dial changes are pinned](/quest/m0/pin-egress-dial.md) - RTMPS that keeps `tcUrl`, and a dial policy the caller supplies + +## Related + +- [Server composition](/quest/m2/compose/server.md) - a source published independently leaves the program, so a destination shows it only once the server flattens it diff --git a/quest/m0/packaging.md b/quest/m0/packaging.md deleted file mode 100644 index f8d6703f..00000000 --- a/quest/m0/packaging.md +++ /dev/null @@ -1,34 +0,0 @@ -# [M] Studio packaging - -## Goal - -Every release ships a signed and notarized macOS app, an unsigned Windows -installer, and a Linux Flatpak of `moq-studio`, all built in CI. Each artifact -is verified to launch and list devices on its platform. - -## Plan - -Decided 2026-10-26 (maintainer): as for the `moq` binary in -[Ship capture and playback](https://github.com/moq-dev/moq/blob/main/quest/m3/cli-packaging.md), moq-studio's defaults -skip any feature that is expensive to build or relies on a system library, -and a `release` feature turns on every backend its platform can load without -one (`capture`, `openh264`, `vaapi`, `nvidia`, and the platform frameworks). -Every artifact builds with `release`. `pipewire` stays out of `release` -because it links libpipewire at load time, but the Flatpak bundles -libpipewire, so it adds `pipewire` itself: inside the sandbox the camera -portal is the only way to a camera, and its PipeWire DMA-BUFs are what the -compositor imports without a copy. - -Decided 2026-11-06 (maintainer, /quest-unblock): Windows ships unsigned -until a release needs SmartScreen reputation; see -[signing](/quest/m0/signing.md). - -## Required - -- [The Apple signing identity is available](/quest/m0/signing.md) - the credentials macOS signing needs -- [Studio on macOS with capture granted](/quest/m0/app-macos.md) - camera, screen, and audio capture run on a Mac -- [Studio on Windows](/quest/m0/app-windows.md) - the studio running on Windows - -## Related - -- [Ship capture and playback](https://github.com/moq-dev/moq/blob/main/quest/m3/cli-packaging.md) - the same distribution checks for the `moq` binary diff --git a/quest/m0/packaging/README.md b/quest/m0/packaging/README.md new file mode 100644 index 00000000..40fd4d04 --- /dev/null +++ b/quest/m0/packaging/README.md @@ -0,0 +1,29 @@ +# Studio packaging + +## Goal + +Every release ships a signed and notarized macOS app, an unsigned Windows +installer, and a Linux Flatpak of `moq-studio`, all built in CI. Each artifact +is verified to launch and list devices on its platform, and lands on its own. + +## Plan + +Decided 2026-10-26 (maintainer): as for the `moq` binary in +[Ship capture and playback](https://github.com/moq-dev/moq/blob/main/quest/m3/cli-packaging.md), moq-studio's defaults +skip any feature that is expensive to build or relies on a system library, +and a `release` feature turns on every backend its platform can load without +one (`capture`, `openh264`, `vaapi`, `nvidia`, and the platform frameworks). +Every artifact builds with `release`, and the first one to land adds the +feature. `pipewire` stays out of `release` because it links libpipewire at +load time. + +Decided 2026-11-09 (maintainer, /quest-audit): ✅ one quest per platform +(rejected: Related links between the quests that shape the macOS bundle; +leaving one packaging quest). One quest held all three artifacts behind the +Apple identity and the Windows host, though the Flatpak needs neither, and +three quests shaped the macOS app bundle with no link between them. The +[macOS app](/quest/m0/packaging/macos.md) owns that bundle. + +## Related + +- [Ship capture and playback](https://github.com/moq-dev/moq/blob/main/quest/m3/cli-packaging.md) - the same distribution checks for the `moq` binary diff --git a/quest/m0/packaging/flatpak.md b/quest/m0/packaging/flatpak.md new file mode 100644 index 00000000..b1b60945 --- /dev/null +++ b/quest/m0/packaging/flatpak.md @@ -0,0 +1,15 @@ +# [M] Linux Flatpak + +## Goal + +Every release ships a Flatpak of `moq-studio`, built in CI with the `release` +and `pipewire` features and verified to launch and list devices on Linux. + +## Plan + +The feature policy is the [epic](/quest/m0/packaging/README.md)'s. + +Decided 2026-10-26 (maintainer): the Flatpak bundles libpipewire, so it adds +`pipewire`, which `release` leaves out, itself: inside the sandbox the camera +portal is the only way to a camera, and its PipeWire DMA-BUFs are what the +compositor imports without a copy. diff --git a/quest/m0/packaging/macos.md b/quest/m0/packaging/macos.md new file mode 100644 index 00000000..2e1f09a6 --- /dev/null +++ b/quest/m0/packaging/macos.md @@ -0,0 +1,30 @@ +# [M] macOS app + +## Goal + +Every release ships a signed and notarized macOS app of `moq-studio`, built +in CI with the `release` feature and verified to launch and list devices on a +Mac. + +This quest owns the app bundle: its layout, its `Info.plist` and +entitlements, and the step that signs and notarizes it. The +[camera extension](/quest/m0/vcam/macos.md) and +[browser sources](/quest/m0/browser-source/macos.md) add to that bundle +rather than build one of their own. + +## Plan + +The feature policy is the [epic](/quest/m0/packaging/README.md)'s. + +Today the studio is not an app bundle, so macOS's capture prompts and grants +belong to whatever launched it. A bundle is asked itself, so it carries the +camera, microphone, and screen recording usage strings. + +Lay the bundle out and order the signing so nested code can be added: a +system extension and CEF's helper apps are each signed before the app that +holds them. + +## Required + +- [The Apple signing identity is available](/quest/m0/signing.md) - the credentials macOS signing needs +- [Studio on macOS with capture granted](/quest/m0/app-macos.md) - camera, screen, and audio capture run on a Mac diff --git a/quest/m0/packaging/windows.md b/quest/m0/packaging/windows.md new file mode 100644 index 00000000..57064fde --- /dev/null +++ b/quest/m0/packaging/windows.md @@ -0,0 +1,19 @@ +# [S] Windows installer + +## Goal + +Every release ships an unsigned Windows installer of `moq-studio`, built in +CI with the `release` feature and verified to launch and list devices on +Windows. + +## Plan + +The feature policy is the [epic](/quest/m0/packaging/README.md)'s. + +Decided 2026-11-06 (maintainer, /quest-unblock): Windows ships unsigned +until a release needs SmartScreen reputation; see +[signing](/quest/m0/signing.md). + +## Required + +- [Studio on Windows](/quest/m0/app-windows.md) - the studio running on Windows diff --git a/quest/m0/pin-aac-encode.md b/quest/m0/pin-aac-encode.md index 849dee77..15606eec 100644 --- a/quest/m0/pin-aac-encode.md +++ b/quest/m0/pin-aac-encode.md @@ -3,8 +3,8 @@ ## Goal The `moq` submodule includes moq-audio's platform AAC encoders, from -[AudioToolbox AAC encode](https://github.com/moq-dev/moq/blob/main/quest/m5/audio-encode-audiotoolbox.md) on macOS and -[Media Foundation AAC encode](https://github.com/moq-dev/moq/blob/main/quest/m5/audio-encode-mediafoundation.md) on +[AudioToolbox AAC encode](https://github.com/moq-dev/moq/blob/main/quest/m3/audio-encode-audiotoolbox.md) on macOS and +[Media Foundation AAC encode](https://github.com/moq-dev/moq/blob/main/quest/m3/audio-encode-mediafoundation.md) on Windows, which destinations whose ingest requires AAC use. ## Plan @@ -14,7 +14,13 @@ fixing whatever the new API breaks here. ## Waiting -- moq-dev/moq: both upstream quests above land. Check: both quests' PRs have +- moq-dev/moq: both upstream quests above land. They move to moq's m3 in + [moq-dev/moq#8421](https://github.com/moq-dev/moq/pull/8421), so each link + resolves once that merges. Check: both quests' PRs have merged on moq's `main`, deleting their files, and moq-audio's AAC encoders are there. A quest file that moved, or went away without its work landing, does not count. + +## Required + +- [moq-mux's decoupled playout clock is pinned](/quest/m0/pin-sync.md) - moq-dev/moq#8272, which no later pin can skip diff --git a/quest/m0/pin-audio-filters.md b/quest/m0/pin-audio-filters.md index 0787f955..432b8f08 100644 --- a/quest/m0/pin-audio-filters.md +++ b/quest/m0/pin-audio-filters.md @@ -3,7 +3,7 @@ ## Goal The `moq` submodule includes moq-audio's per-source filter chain and plugin -host, from [moq-audio filter chain](https://github.com/moq-dev/moq/blob/main/quest/m5/audio-filters.md). +host, from [moq-audio filter chain](https://github.com/moq-dev/moq/blob/main/quest/m3/audio-filters.md). ## Plan @@ -12,7 +12,13 @@ fixing whatever the new API breaks here. ## Waiting -- moq-dev/moq: the upstream quest above lands. Check: that quest's PR has +- moq-dev/moq: the upstream quest above lands. It moves to moq's m3 in + [moq-dev/moq#8421](https://github.com/moq-dev/moq/pull/8421), so the link + resolves once that merges. Check: that quest's PR has merged on moq's `main`, deleting its file, and moq-audio's filter chain is there. A quest file that moved, or went away without its work landing, does not count. + +## Required + +- [moq-mux's decoupled playout clock is pinned](/quest/m0/pin-sync.md) - moq-dev/moq#8272, which no later pin can skip diff --git a/quest/m0/pin-audio-tap.md b/quest/m0/pin-audio-tap.md index 9bd475cf..c83e231b 100644 --- a/quest/m0/pin-audio-tap.md +++ b/quest/m0/pin-audio-tap.md @@ -17,3 +17,7 @@ fixing whatever the new API breaks here. merged on moq's `main`, deleting its file, and `moq_audio::bus` there offers a tap of a source's samples with bus positions. A quest file that moved, or went away without its work landing, does not count. + +## Required + +- [moq-mux's decoupled playout clock is pinned](/quest/m0/pin-sync.md) - moq-dev/moq#8272, which no later pin can skip diff --git a/quest/m0/pin-bus-positioned.md b/quest/m0/pin-bus-positioned.md index fe9dbeff..5088f837 100644 --- a/quest/m0/pin-bus-positioned.md +++ b/quest/m0/pin-bus-positioned.md @@ -12,9 +12,15 @@ PR also renamed `Bus::channel`/`bus::Channel` to `Bus::live`/`bus::Live`. Planned 2026-11-08 (maintainer, moq's /quest-complete), a follow-up of moq-dev/moq#8273, which has merged on moq's `main`. -Advance it with `just upgrade-moq` in its own PR, fixing whatever the new API -breaks here: at least the `Bus::live` rename at every `bus.channel(..)` call. -If [moq-dev/moq#8272](https://github.com/moq-dev/moq/pull/8272) has landed by -then, `moq_mux::timeline` is `moq_mux::sync`, and `playout.rs` follows that -PR's migration notes too. Move no audio onto positioned writes here; that is +Decided 2026-11-09 (maintainer, /quest-audit): ✅ pin exactly #8273, at +`79960a9dd` with `just upgrade-moq 79960a9dd` (rejected: bump to moq `main` +here, which crosses the playout clock's replacement below and resizes this +quest). + +Land it in its own PR, fixing whatever the new API breaks here: at least the +`Bus::live` rename at every `bus.channel(..)` call. +[moq-dev/moq#8272](https://github.com/moq-dev/moq/pull/8272) merged after +#8273, as `644d48996`. It replaces `moq_mux::timeline` with `moq_mux::sync` +and is not crossed here; [pin-sync](/quest/m0/pin-sync.md) crosses it. Move no +audio onto positioned writes here; that is [Playout writes land on positioned bus channels](/quest/m0/playout-positioned.md). diff --git a/quest/m0/pin-egress-dial.md b/quest/m0/pin-egress-dial.md new file mode 100644 index 00000000..40e389a1 --- /dev/null +++ b/quest/m0/pin-egress-dial.md @@ -0,0 +1,32 @@ +# [XS] moq's egress dial changes are pinned + +## Goal + +The `moq` submodule includes the two moq changes destinations dial through: +moq-rtmp's `rtmps://` dial keeps the URL-derived `tcUrl`, from +[RTMPS keeps `tcUrl`](https://github.com/moq-dev/moq/blob/main/quest/m3/rtmp-rtmps-tcurl.md), +and a dial takes a policy its caller supplies, so a host can refuse private +and loopback addresses, from +[a caller-supplied dial policy](https://github.com/moq-dev/moq/blob/main/quest/m3/webrtc-dial-policy.md). + +## Plan + +Decided 2026-11-09 (maintainer, /quest-audit): ✅ both changes are planned in +moq's tree with this pin here (rejected: planning them inline in +[Destinations](/quest/m0/destinations.md), with no moq quest or pin). + +Once the wait clears, advance it with `just upgrade-moq` in its own PR, +fixing whatever the new API breaks here. + +## Waiting + +- moq-dev/moq: both upstream quests above land. They were being planned in + moq's tree on 2026-11-09, so each link resolves once its plan merges. + Check: both quests' PRs have merged on moq's `main`, deleting their files, + moq-rtmp's TLS dial sends the URL-derived `tcUrl`, and a dial takes the + caller's policy. A quest file that moved, or went away without its work + landing, does not count. + +## Required + +- [moq-mux's decoupled playout clock is pinned](/quest/m0/pin-sync.md) - moq-dev/moq#8272, which no later pin can skip diff --git a/quest/m0/pin-portal-token.md b/quest/m0/pin-portal-token.md index cfa9827c..350144ae 100644 --- a/quest/m0/pin-portal-token.md +++ b/quest/m0/pin-portal-token.md @@ -15,3 +15,7 @@ fixing whatever the new API breaks here. - moq-dev/moq: Linux capture parity lands the persistable token. Check: moq-video on moq's `main` exposes the persistable restore token (the quest no longer lists it because it landed, not because it moved). + +## Required + +- [moq-mux's decoupled playout clock is pinned](/quest/m0/pin-sync.md) - moq-dev/moq#8272, which no later pin can skip diff --git a/quest/m0/pin-sync.md b/quest/m0/pin-sync.md new file mode 100644 index 00000000..c0c3fd4d --- /dev/null +++ b/quest/m0/pin-sync.md @@ -0,0 +1,87 @@ +# [M] moq-mux's decoupled playout clock is pinned + +## Goal + +The `moq` submodule is at `644d48996`, +[moq-dev/moq#8272](https://github.com/moq-dev/moq/pull/8272), which replaces +`moq_mux::timeline` with `moq_mux::sync`, and a broadcast source's playout +(`rs/moq-scene/src/playout.rs`, `rs/moq-scene/src/source.rs`) runs on the new +clock. `README.md`'s broadcast-source paragraph under "Keyframes" says what a +stall does on it. + +## Plan + +Decided 2026-11-09 (maintainer, /quest-audit): ✅ crossing #8272 is this +quest, after +[Playout writes land on positioned bus channels](/quest/m0/playout-positioned.md) +(rejected: crossing it in +[pin-bus-positioned](/quest/m0/pin-bus-positioned.md)). Both rewrite +`playout.rs`, so the positioned work finishes on the API it was planned +against, then this crosses. + +Advance with `just upgrade-moq 644d48996` in its own PR. #8272's Impact holds +the migration: + +- `timeline::Kind` is `moq_mux::container::Kind`, which `Producer::track` + takes. +- `Producer::video(ts, now)` is `track.received(ts, now)` on the video + `Track`. `Producer::audio(Audio { .. }, now)` is `track.received(ts, + arrival)` on the audio `Track` plus `track.set_target(jitter_target)`, and + `timeline::Audio` is gone. +- `video_restarted` and `audio_restarted` are `track.reset()`; + `audio_stopped` is dropping or retiring the track. +- `Consumer::target()` is `Consumer::delay()`, always resolved. + `Consumer::playhead()` is gone, which `playout::heard` reads: the speaker + tracks what it sounds. +- Audio no longer owns the clock. Each frame sounds at `producer.due(..)`, + which the `Speaker` steers toward itself. + +The other breaking changes in `79960a9dd..644d48996` that reach studio code, +read against the tree on 2026-11-09; `just ci` finds any other: + +- [moq-dev/moq#8274](https://github.com/moq-dev/moq/pull/8274): + `mix.minus(strip)` takes every strip to remove, `mix.minus([&strip])` + (`rs/moq-studio/src/feed.rs`). +- [moq-dev/moq#8269](https://github.com/moq-dev/moq/pull/8269): + `moq_video::encode::Producer::new` takes a track, not a broadcast, and + `with_track` is gone; `render::Exporter::new` takes `&encode::Config` and + returns `(Exporter, encode::Config, VideoConfig)`. + +Not crossed here: [moq-dev/moq#8270](https://github.com/moq-dev/moq/pull/8270) +(`241d2ce92`), [moq-dev/moq#8372](https://github.com/moq-dev/moq/pull/8372) +(`147327e14`), and +[moq-dev/moq#8220](https://github.com/moq-dev/moq/pull/8220) (`16a673036`, +kio's `Instant` loses `+` and `-` with a `Duration`, which `playout.rs` and +`source.rs` use throughout) all merged after #8272. The first pin past them +fixes them. + +## Waiting + +- Maintainer: decide what a sound stall does to a broadcast source's picture. + `README.md` says "A sound stall longer than that delay pauses the picture + too and starts both again together", which rested on audio owning the + clock's anchor and on `Consumer::playhead()`. #8272 removed both, after the + 2026-11-05 ruling on + [moq-dev/moq#7941](https://github.com/moq-dev/moq/pull/7941): lip sync in + the steady state, desync under congestion, and one slow track never stalls + or moves another. An agent cannot pick: it changes what a creator sees when + a guest's audio stalls. Options: + 1. Adopt moq's decoupled rule (recommended). The picture keeps playing + through a sound stall, and the sound comes back on the clock: each frame + is written at the bus position `due` gives it, and the late part of a + burst is trimmed, as the positioned writer already does. The studio + plays a broadcast as `moq play` and the browser do, which `playout.rs` + sets out to, and `README.md` drops "Each picture waits for the sound + stamped with it" and the pause sentence. + 2. Keep the studio's rule. The `Speaker` keeps a playhead of its own, + `heard` holds pictures on it, and the studio holds the picture back + itself after a stall, since `sync` no longer lets audio hold video. Sound + and picture never part, at the cost of studio-only coupling over an API + built to prevent it. + + Check: the maintainer answers in `/quest-unblock`, and the choice is + recorded here. + +## Required + +- [Playout writes land on positioned bus channels](/quest/m0/playout-positioned.md) - the `playout.rs` rewrite this crosses on top of diff --git a/quest/m0/playout-positioned.md b/quest/m0/playout-positioned.md index d925b542..f680879e 100644 --- a/quest/m0/playout-positioned.md +++ b/quest/m0/playout-positioned.md @@ -4,8 +4,11 @@ A broadcast, media, or replay source's sound plays at exactly the bus position its pictures are timed for. `rs/moq-scene`'s playout writes it -through moq-audio's `Bus::positioned`, and its private copy of moq-audio's bus -latency (`BUS_LATENCY`) and of the live ring's pad rule are deleted. +through moq-audio's `Bus::positioned`, and its copy of the live ring's pad +rule is deleted. How far ahead of its moment a sound is handed to the bus +stays a constant the studio owns: today's `BUS_LATENCY`, renamed for what it +is (such as `WRITE_LEAD`) and no longer a copy of moq-audio's private +`bus::LATENCY`. Non-goals: the `Speaker`'s jitter targeting stays as it is, and live inputs (microphones, capture, guests) stay on `Bus::live`. @@ -37,6 +40,20 @@ Decided with the maintainer: the frames whose position has passed and writes the rest at its position, as today's writer does with a late part. +Decided 2026-11-09 (maintainer, /quest-audit): + +- ✅ **Keep a studio-owned write lead** (rejected: deleting `BUS_LATENCY` + outright). The Goal deleted it while the `Speaker`'s targeting, a non-goal, + is built on it: `Speaker::place` sounds a frame `target + BUS_LATENCY` + after it arrives and re-buffers one that would follow less than half of it + away, and `source.rs` hands each sound over that far ahead of its moment + (`ahead`, the broadcast write loop, and where a media or replay play + begins). A positioned write needs that lead too: the bus mixes a block at a + time, and a write with any part already mixed fails. So the constant stays, + within `bus::REACH`. What goes is the `Writer`'s pad rule and the claim + that the constant is moq-audio's `bus::LATENCY`, which + `a_write_sounds_when_placed` pins today. + Recommendations, not settled: - Trim against the later of the already-mixed position and the previous diff --git a/quest/m0/vcam/README.md b/quest/m0/vcam/README.md index a7b19cc1..e708872b 100644 --- a/quest/m0/vcam/README.md +++ b/quest/m0/vcam/README.md @@ -4,7 +4,7 @@ The studio exposes the program or a published scene as a system camera, so video-call and conferencing apps, which offer no ingest API, can use it. The studio has one -toggle per published scene, and each OS backend lands on its own. +toggle for the program and one per published scene, and each OS backend lands on its own. ## Plan diff --git a/quest/m0/vcam/macos.md b/quest/m0/vcam/macos.md index 4cd89fa1..769b3255 100644 --- a/quest/m0/vcam/macos.md +++ b/quest/m0/vcam/macos.md @@ -6,7 +6,14 @@ The macOS app bundle carries a CoreMediaIO Camera Extension. Once the user approves it, apps list a camera that carries the program or a published scene. An app update installs the new extension without a reboot. +## Plan + +Decided 2026-11-09 (maintainer, /quest-audit): ✅ the extension joins the +bundle the [macOS app](/quest/m0/packaging/macos.md) owns and goes through +its signing (rejected: a Related link only; leaving the quests unlinked). + ## Required +- [macOS app](/quest/m0/packaging/macos.md) - the app bundle that carries and signs the extension - [The Apple signing identity is available](/quest/m0/signing.md) - system extensions only load when signed - [Studio on macOS with capture granted](/quest/m0/app-macos.md) - camera, screen, and audio capture run on a Mac diff --git a/quest/m0/wayland-sources.md b/quest/m0/wayland-sources.md index 7c68d11a..75699ce1 100644 --- a/quest/m0/wayland-sources.md +++ b/quest/m0/wayland-sources.md @@ -18,11 +18,12 @@ Decided 2026-10-26 (maintainer): keyed by scene collection and source), never in the scene file, so the scene file stays portable and diffable. On another machine, or when the portal rejects the token, the picker shows once and the new token is stored. -- **Scene file:** the scene item records that the source is portal-picked, not +- **Scene file:** the source records that it is portal-picked, not which monitor or window was picked, since only the portal knows that. -Add the source kinds next to the X11 ones in moq-scene and the Sources panel, -and update `README.md`, which says display and window sources are +Add portal-picked display and window sources beside the id-selected +`Source::Display` and `Source::Window` (`rs/moq-scene/src/collection.rs`), in +the Sources and Devices panels and guided setup, and update `README.md`, which says display and window sources are X11-only. ## Required diff --git a/quest/m0/windows-build.md b/quest/m0/windows-build.md new file mode 100644 index 00000000..8ae1f36c --- /dev/null +++ b/quest/m0/windows-build.md @@ -0,0 +1,25 @@ +# [S] The studio builds for Windows in CI + +## Goal + +`moq-studio` builds for `x86_64-pc-windows-msvc` with `--features capture` on +a `windows-latest` job in the nightly workflow +(`.github/workflows/nightly.yml`), and whatever fails to compile there is +fixed. + +## Plan + +Decided 2026-11-09 (maintainer, /quest-audit): ✅ its own quest (rejected: +folding it into [Studio on Windows](/quest/m0/app-windows.md)). A hosted +runner compiles it, so it needs no Windows host and is not blocked behind +one. + +Nothing has compiled the whole crate for Windows. +[moq-dev/studio#84](https://github.com/moq-dev/studio/pull/84) reported that +`cargo check` for the target fails off a Windows host, in C build scripts +(`tract-linalg`) that need an MSVC toolchain, so it checked only +`hotkeys/native.rs` and `hotkeys/layout.rs`, in a scratch crate. + +The job is nightly, beside the macOS one, not on the merge gate. Windows has +no Nix dev shell, so the job installs the toolchain `rust-toolchain.toml` +names itself. diff --git a/quest/m0/windows-host.md b/quest/m0/windows-host.md index ada3389e..27e50b07 100644 --- a/quest/m0/windows-host.md +++ b/quest/m0/windows-host.md @@ -32,9 +32,8 @@ Manager); NixOS boots through systemd-boot on `nvme0n1p4`. The switch: ## Waiting -- Maintainer: say nixos2 is free to reboot, once the Mesa trial is done. On - 2026-11-08 it ran an interactive `codex` session (open since 2026-11-06), a - `codex app-server`, and a rind `astro dev` server, and an agent cannot tell +- Maintainer: say nixos2 is free to reboot. On 2026-11-09 it ran a Claude + Code session and a rind worktree's node process, and an agent cannot tell whether such work is finished. Check: the maintainer says so, or `w` and `ps` on nixos2 show no other session's work. - Maintainer: at nixos2's console once it boots Windows, turn on OpenSSH diff --git a/quest/m1/README.md b/quest/m1/README.md index 970c2e3e..642d9f48 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -2,12 +2,12 @@ ## Goal -The work held until m0 ships. The studio gains features m0 left out: -overlay assets and restarts, and isolated guest recordings. +The work held until m0 ships: studio features planned as follow-ups of m0 +work but not needed to ship it. Nothing is planned here now. ## Plan -Moved out of m0 on 2026-11-04 (maintainer): the program is how a studio +Viewer-side composition moved out of m0 on 2026-11-04 (maintainer): the program is how a studio broadcast reaches viewers, and sources are not published to the relay by default (decided 2026-11-03 on [moq-dev/studio#21](https://github.com/moq-dev/studio/pull/21)), since @@ -17,7 +17,8 @@ GPU has. Viewer-side composition waits until m0 ships. Viewer-side composition moved to m2 on 2026-11-07 (maintainer, on [moq-dev/studio#97](https://github.com/moq-dev/studio/pull/97)): no viewer can fetch a source's track until a source is published independently, which -is m2. +was m2 then and shipped in +[moq-dev/studio#117](https://github.com/moq-dev/studio/pull/117). Broadened 2026-11-04 (maintainer, /quest-plan): m1 also holds studio features planned as follow-ups of m0 work but not needed to ship it, rather diff --git a/quest/m2/README.md b/quest/m2/README.md index 8b41de99..73a53b61 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -12,10 +12,19 @@ when a consumer appears. Publishing a source independently and viewer-side composition moved on to [m3](/quest/m3/README.md) on 2026-11-07 (maintainer): no customer asks for them. They came back to m2 on 2026-11-08 (maintainer, /quest-plan), replanned around players that composite independent sources -over the program. +over the program. Publishing a source independently shipped the same day +([moq-dev/studio#117](https://github.com/moq-dev/studio/pull/117)). + +The web studio and its two pins moved here from m0 on 2026-11-09 (maintainer, +/quest-audit; rejected: keeping them in m0): moq removed all WASM support on +2026-10-31, its browser-backend quests are deferred with no other consumer, +and nothing plans a browser WebTransport session for moq-net. ## Required - [Overlays drop their Blitz panic guard](/quest/m2/overlay-blitz-release.md) - the next Blitz release lets overlay checks parse without `catch_unwind` - [Viewer-side composition](/quest/m2/compose/README.md) - players composite independently published sources over the program from a layer layout track -- [Browser sources paint on the GPU under Mesa AMD Wayland](/quest/m2/mesa-amd.md) - the Intel two-plane fix, checked on AMD once a host exists +- [Browser sources paint on the GPU under Mesa AMD Wayland](/quest/m2/mesa-amd.md) - the Intel two-plane fix, checked on the `desktop` host's AMD GPU +- [Web studio](/quest/m2/web.md) - the full studio in wasm on WebGPU and WebCodecs, served from this repository +- [moq-video's browser backends are pinned](/quest/m2/pin-browser-video.md) - the web studio's video slice +- [moq-audio's browser backends are pinned](/quest/m2/pin-browser-audio.md) - the web studio's audio slice diff --git a/quest/m2/compose/README.md b/quest/m2/compose/README.md index 26e755e3..f0452892 100644 --- a/quest/m2/compose/README.md +++ b/quest/m2/compose/README.md @@ -40,8 +40,11 @@ land before replanning: (rejected: straight to `moq play`). moq's spec quest starts from it. - **Audio:** ✅ players mix independently published audio sources with the program's, at the gain and mute the layout carries (rejected: video only). + Since 2026-11-09 the layout carries pan and delay too; see + [layout](/quest/m2/compose/layout.md). - **Controls:** ✅ viewer toggles and priorities land in the native player - (rejected: browser only). + (rejected: browser only). Since 2026-11-09 that includes `moq play`; see + [controls](/quest/m2/compose/controls.md). - **Browser composition** (moq-scene's browser build layered on `@moq/watch`): ✅ deleted, since `@moq/watch` layering supersedes it. Server composition stays, replanned to flatten program and layers for @@ -50,8 +53,10 @@ land before replanning: moq's m3. Still settled from 2026-10-26: local sources are tracks in the studio's one -broadcast, and guests stay their own broadcasts, named through the hang -catalog's `contributions` references. +broadcast, and guests stay their own broadcasts. How the layout names a +guest is open in the [layer layout](/quest/m2/compose/layout.md)'s Waiting: +hang's `include` requires an included broadcast to share the listing +catalog's timeline, clock, and join interval, which a guest's does not. moq's side, in its own quest tree (m3): hang specifies a layer layout track, and `moq play` and `@moq/watch` composite it. diff --git a/quest/m2/compose/controls.md b/quest/m2/compose/controls.md index fccf44db..a79b3c2e 100644 --- a/quest/m2/compose/controls.md +++ b/quest/m2/compose/controls.md @@ -3,7 +3,7 @@ ## Goal A viewer watching a studio broadcast can switch off any layer the layout -marks toggleable (a facecam, a chat overlay), and stops fetching its track. +marks toggleable (a facecam, a screen share), and stops fetching its track. The layout's per-layer priorities steer delivery, so under congestion the least important sources degrade first. The creator sets both per source in the studio. @@ -14,8 +14,16 @@ Decided 2026-11-08 (maintainer, /quest-plan): the controls land in the native player, the [watch prototype](/quest/m2/compose/watch.md) (rejected: browser only). moq's players take them from hang's layer spec. +Decided 2026-11-09 (maintainer, /quest-audit): ✅ viewer toggles also land in +moq's native player, +[moq play composites layers](https://github.com/moq-dev/moq/blob/main/quest/m3/layers/play.md) +(rejected: leaving them to the prototype alone, which +[hang-layers](/quest/m2/compose/hang-layers.md) deletes, so the native +player's toggles would have no home). moq adds the toggles to that quest in +its own tree. + Map layer priority onto the subscription priorities moq-lite already has -(`doc/concept/moq-lite.md`), rather than a second mechanism. A toggled-off +(`moq/doc/concept/moq-lite.md`), rather than a second mechanism. A toggled-off layer is unsubscribed, so it costs the viewer and the relay nothing. The studio keeps encoding it, so toggling it back on can reach its recent past. Subscriptions are per track, not per layer: a source shown by several layers diff --git a/quest/m2/compose/hang-layers.md b/quest/m2/compose/hang-layers.md index 2b452554..4ec5803b 100644 --- a/quest/m2/compose/hang-layers.md +++ b/quest/m2/compose/hang-layers.md @@ -15,16 +15,25 @@ the wait clears, advance the gitlink with `just upgrade-moq` in this PR, move the studio to hang's types, and delete the [watch prototype](/quest/m2/compose/watch.md) and its flag. +Decided 2026-11-09 (maintainer, /quest-audit): ✅ the wait names moq's three +quests (rejected: "both quests", unnamed, which left out `@moq/watch`). + ## Waiting -- moq-dev/moq: hang's layer layout spec and `moq play`'s layer compositing - land, planned in moq's m3 from the - [watch prototype](/quest/m2/compose/watch.md)'s format. Check: both quests' - PRs have merged on moq's `main`, deleting their files, and hang exposes the - layer layout. A quest file that moved, or went away without its work - landing, does not count. +- moq-dev/moq: its three layer quests land, planned in moq's m3 from the + [watch prototype](/quest/m2/compose/watch.md)'s format: + [hang specifies a layer layout track](https://github.com/moq-dev/moq/blob/main/quest/m3/layers/spec.md), + [moq play composites layers](https://github.com/moq-dev/moq/blob/main/quest/m3/layers/play.md), + and + [The watch player composites layers](https://github.com/moq-dev/moq/blob/main/quest/m3/layers/watch.md), + which the [epic](/quest/m2/compose/README.md)'s test plays in. Check: all + three quests' PRs have merged on moq's `main`, deleting their files, hang + exposes a layer layout whose audio layers carry gain, mute, pan, and delay, + and both players apply all four. A quest file that moved, or went away + without its work landing, does not count. ## Required - [moq-studio watch prototype](/quest/m2/compose/watch.md) - the format moq specifies - [Viewer controls](/quest/m2/compose/controls.md) - toggles and priorities the spec carries +- [moq-mux's decoupled playout clock is pinned](/quest/m0/pin-sync.md) - moq-dev/moq#8272, which no later pin can skip diff --git a/quest/m2/compose/layout.md b/quest/m2/compose/layout.md index 835991c9..77b97e3c 100644 --- a/quest/m2/compose/layout.md +++ b/quest/m2/compose/layout.md @@ -6,7 +6,7 @@ A studio broadcast that publishes a source independently also carries a layer layout track. For each such source, it says where and when that source shows over the program: its transform, crop, z-order, opacity, and visibility, its item's keyframes and when its scene went on program, all on the media clock, -and for audio its gain and mute. Until hang specifies it, the track is the +and for audio its gain, mute, pan, and delay. Until hang specifies it, the track is the provisional `dev.moq.studio.layers` in the studio catalog. ## Plan @@ -18,7 +18,7 @@ independent layers over it, rather than the full scene state; see the A JSON track in snapshot mode, so a late joiner gets the current state. Version it, and have a player refuse an unknown version loudly and play the program alone. Keep it free of local paths and credentials: layers name -tracks and contribution references, never files. +tracks and broadcasts, never files. Decided 2026-11-08 (maintainer, on [moq-dev/studio#117](https://github.com/moq-dev/studio/pull/117)): a @@ -37,8 +37,17 @@ needs that effect baked into its track, or the option refused for it. Settle which here. Publishing a source independently refuses one with its background removed for now, since H.264 carries no transparency, so lifting that refusal is this track's call too. An audio source's track already carries its sound -after its filters (decided 2026-11-02), so its layer carries only gain and -mute. +after its filters (decided 2026-11-02), so its layer carries what its strip +applies after them: gain, mute, pan, and delay. + +Decided 2026-11-09 (maintainer, /quest-audit): ✅ an audio layer carries pan +and delay as well as gain and mute (rejected: stating that a player drops +them). Without the delay, a delayed microphone published independently plays +out of sync with its camera. moq's +[layer spec](https://github.com/moq-dev/moq/blob/main/quest/m3/layers/spec.md) +starts from the format the prototype settles; +[moq-dev/moq#8421](https://github.com/moq-dev/moq/pull/8421) adds pan and +delay to it and has both players apply them. Items carry their keyframes (`moq_scene::Keyframe`), which play from when their scene went on program. The layout says when that was, so a late joiner @@ -49,3 +58,28 @@ Each layer also carries its priority and whether a viewer may toggle it, for The [watch prototype](/quest/m2/compose/watch.md) is this track's first reader. What the two settle becomes moq's hang spec draft. + +## Waiting + +- Maintainer: decide how the layout names a guest's broadcast. The epic + settled on 2026-10-26 that guests stay their own broadcasts, named through + hang's catalog references, then `contributions`, now `include` + ([moq-dev/moq#7943](https://github.com/moq-dev/moq/pull/7943)). The hang + draft at the pin requires an include to share the listing catalog's PTS + timeline, `media.clock`, and `media.join.interval`, and a guest's broadcast + keeps its own, so a player that takes `include` at its word places a + guest's frames at the wrong time. Options: + - The layout names each guest's broadcast itself and carries the offset + from the guest's timeline onto the program's media clock, the one the + studio applies when it plays the guest (recommended): `include` keeps + its meaning, and the timing stays in the track that already says when + each layer shows. + - moq relaxes `include`, or adds a sibling map, for broadcasts on their own + timeline, with the offset in the catalog: a moq quest and a pin, and every + hang consumer learns a second kind of include. + - The studio republishes each guest's tracks in its own broadcast, restamped + onto the program's clock: no spec change, but it reverses the settled + "guests stay their own broadcasts" and relays every guest twice. + + Check: the maintainer answers in /quest-unblock, and the choice is recorded + in this Plan and the epic's. diff --git a/quest/m2/compose/server.md b/quest/m2/compose/server.md index 324cdb1f..fec44bf2 100644 --- a/quest/m2/compose/server.md +++ b/quest/m2/compose/server.md @@ -29,3 +29,4 @@ and document it in `README.md`. ## Required - [Layer layout track](/quest/m2/compose/layout.md) - what the server flattens +- [Destinations](/quest/m0/destinations.md) - the egress library the server pushes RTMP and WHIP through diff --git a/quest/m2/mesa-amd.md b/quest/m2/mesa-amd.md index 9c85b396..fec0faed 100644 --- a/quest/m2/mesa-amd.md +++ b/quest/m2/mesa-amd.md @@ -15,12 +15,12 @@ buffers, so the browser helper now starts with `INTEL_DEBUG=noccs` the same way. Run `binary(browser)` on an AMD Mesa host, and only if it fails the same way, add `AMD_DEBUG=nodcc` beside `noccs`. -Planned 2026-11-08 (maintainer, /quest-plan): ✅ m2, waiting on a host -(rejected: m0; dropping it), since no user has reported AMD failing. +Planned 2026-11-08 (maintainer, /quest-plan): ✅ m2 (rejected: m0; dropping +it), since no user has reported AMD failing. It waited on a host then. -## Waiting - -- Maintainer: provide an AMD GPU host running Mesa with GNOME on Wayland, - reachable over ssh. No known host has one, and an agent cannot acquire - hardware. Check: `ssh glxinfo -B` (or `vulkaninfo --summary`) names an - AMD device on the radeonsi or RADV driver. +Decided 2026-11-09 (maintainer, /quest-audit): ✅ agents may run the check on +the host named `desktop`, in its live GNOME Wayland session (rejected: wait +for a dedicated host). It has an RX 9070 on radeonsi. It also has an Intel +integrated GPU (`lspci`), so the run confirms the helper paints on the AMD +one. The check opens browser windows in the maintainer's session for a few +minutes. diff --git a/quest/m0/pin-browser-audio.md b/quest/m2/pin-browser-audio.md similarity index 74% rename from quest/m0/pin-browser-audio.md rename to quest/m2/pin-browser-audio.md index 64fa5b1c..1a927e21 100644 --- a/quest/m0/pin-browser-audio.md +++ b/quest/m2/pin-browser-audio.md @@ -11,9 +11,16 @@ capture and playback, WebCodecs Opus and AAC), from Once the wait clears, advance it with `just upgrade-moq` in its own PR, fixing whatever the new API breaks here. +Moved to m2 with the [web studio](/quest/m2/web.md) on 2026-11-09 +(maintainer, /quest-audit). + ## Waiting - moq-dev/moq: the upstream quest above lands. Check: that quest's PR has merged on moq's `main`, deleting its file, and moq-audio's browser backends are there. A quest file that moved, or went away without its work landing, does not count. + +## Required + +- [moq-mux's decoupled playout clock is pinned](/quest/m0/pin-sync.md) - moq-dev/moq#8272, which no later pin can skip diff --git a/quest/m0/pin-browser-video.md b/quest/m2/pin-browser-video.md similarity index 76% rename from quest/m0/pin-browser-video.md rename to quest/m2/pin-browser-video.md index bb4bf2e5..bf08776f 100644 --- a/quest/m0/pin-browser-video.md +++ b/quest/m2/pin-browser-video.md @@ -12,9 +12,16 @@ wasm32 builds of hang and moq-mux, from Once the wait clears, advance it with `just upgrade-moq` in its own PR, fixing whatever the new API breaks here. +Moved to m2 with the [web studio](/quest/m2/web.md) on 2026-11-09 +(maintainer, /quest-audit). + ## Waiting - moq-dev/moq: the upstream quest above lands. Check: that quest's PR has merged on moq's `main`, deleting its file, and moq-video's browser backends are there. A quest file that moved, or went away without its work landing, does not count. + +## Required + +- [moq-mux's decoupled playout clock is pinned](/quest/m0/pin-sync.md) - moq-dev/moq#8272, which no later pin can skip diff --git a/quest/m0/web.md b/quest/m2/web.md similarity index 62% rename from quest/m0/web.md rename to quest/m2/web.md index d4901162..2b97922e 100644 --- a/quest/m0/web.md +++ b/quest/m2/web.md @@ -10,7 +10,7 @@ serves: - publishing over WebTransport through moq-net Browser sources are absent. So are RTMP and WHIP destinations. A page cannot -open raw TCP, and moq-rtc's WHIP client binds native UDP sockets, so only MoQ +open raw TCP, and moq-webrtc's WHIP client binds native UDP sockets, so only MoQ output remains. Each missing feature is hidden, not shown broken. The page lands in vertical slices: video first (camera and screen sources on @@ -35,7 +35,23 @@ Decided 2026-11-04 (maintainer): a page has no local paths, so an image or media source's file comes from a browser file picker for the session, and resolving the scene file's paths in the browser is settled here. +Decided 2026-11-09 (maintainer, /quest-audit): ✅ m2, with both pins +(rejected: keep in m0). moq removed all WASM support on 2026-10-31, and its +browser-backend quests +([video](https://github.com/moq-dev/moq/blob/main/quest/m5/video-web.md), +[audio](https://github.com/moq-dev/moq/blob/main/quest/m5/audio-web.md)) are +deferred in its m5 with no other consumer. Nothing in moq's tree plans a +browser WebTransport session for moq-net either, which the page publishes +through, so this waits on one (see Waiting). + +## Waiting + +- moq-dev/moq: plans a browser WebTransport session for moq-net, the + transport the page publishes through. Check: a quest in moq's tree names + it. Then this quest gains a pin quest for it under `Required`, and this + item goes. + ## Required -- [moq-video's browser backends are pinned](/quest/m0/pin-browser-video.md) - the video slice's capture, codecs, and GPU import -- [moq-audio's browser backends are pinned](/quest/m0/pin-browser-audio.md) - the audio slice's capture, playback, and codecs +- [moq-video's browser backends are pinned](/quest/m2/pin-browser-video.md) - the video slice's capture, codecs, and GPU import +- [moq-audio's browser backends are pinned](/quest/m2/pin-browser-audio.md) - the audio slice's capture, playback, and codecs