Repository navigation
quest: apply the studio audit's decisions #8421
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| # [M] Media Foundation decode on Windows | ||
|
|
||
| ## Goal | ||
|
|
||
| On Windows, `moq-audio` decodes AAC-LC, HE-AAC v1 and v2, and multichannel | ||
| AAC through the Media Foundation AAC decoder MFT, and whichever of MP3, FLAC, | ||
| AC-3, and E-AC-3 the installed MFTs open. | ||
|
|
||
| ## Plan | ||
|
|
||
| The audio counterpart of `rs/moq-video/src/decode/backend/mediafoundation.rs`, | ||
| using the `windows` crate features moq-video already enables plus the audio | ||
| ones. Behind the decode seam as the first candidate on `target_os = | ||
| "windows"`. | ||
|
|
||
| - The AAC decoder MFT takes an `MF_MT_USER_DATA` blob built from the catalog | ||
| AudioSpecificConfig and reports the output `WAVEFORMATEXTENSIBLE`, whose | ||
| channel mask maps to `Layout`. | ||
| - Optional MFTs (Dolby, FLAC) are probed at open: absent means the codec is | ||
| not advertised on that host, not an error at decode time. | ||
| - Fixtures and layout-order tests as in the AudioToolbox quest. | ||
| - Verification runs on a Windows host; the per-PR CI only compiles the | ||
| platform code (`just rs windows`). | ||
|
|
||
| Changed 2026-11-07 (maintainer, /quest-plan from the API sweep of open PRs and remaining quests): moq-audio's codecs return zero or more outputs with `flush` before the cut, so this backend adds no break. `decode::Consumer` still places each packet by the one output it returns and refuses any other count, so if this backend holds or splits output, it also teaches the consumer to place that output. | ||
|
|
||
| Changed 2026-11-09 (moq-dev/studio /quest-audit): this moved from m5 to m3 | ||
| with [Media Foundation encode](/quest/m3/audio-encode-mediafoundation.md), | ||
| which the maintainer moved for moq-dev/studio's destinations and which | ||
| requires this for its round-trip regression. A quest an m3 quest requires | ||
| joins m3. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| # [M] AudioToolbox AAC encode on macOS and iOS | ||
|
|
||
| ## Goal | ||
|
|
||
| On macOS and iOS, `Codec::Aac` encodes through AudioToolbox at the input's | ||
| layout, mono through 5.1 and 7.1, and the result plays in the browser, | ||
| `moq play`, and OBS. 6.1 is refused: since #4973 a synthesized ASC names no | ||
| program config element. Carrying the converter's magic cookie verbatim would | ||
| lift that once a caller needs it. | ||
|
|
||
| ## Plan | ||
|
|
||
| An `AudioConverter` from interleaved `f32` to `kAudioFormatMPEG4AAC`, behind | ||
| the encode seam as the platform candidate on macOS and iOS. | ||
|
|
||
| - The catalog ASC is synthesized at construction per the encode seam; read | ||
| `kAudioConverterCompressionMagicCookie` at open and assert it matches. | ||
| `kAudioConverterPrimeInfo` gives the delay the timestamps fold in. | ||
| - Bitrate through `kAudioConverterEncodeBitRate`, updated live where the | ||
| converter allows. | ||
| - Gate the seam's "AAC refused without a platform encoder" test to hosts | ||
| without one. | ||
| - Regression: a stereo and a 5.1 encode round-trip through the AudioToolbox | ||
| decoder and through symphonia (stereo only), with timestamps continuous | ||
| across the priming. | ||
| - That continuity fails today for an epoch below the delay: `Producer::timestamp` | ||
| (`rs/moq-audio/src/encode/producer.rs`) clamps each priming packet to zero, | ||
| so a zero epoch stamps `[0, 0, 0, 20_000]`, and `decode::Consumer` reads the | ||
| duplicates as holes and resets AAC between them. Shift the whole timeline by | ||
| the deficit instead, including the terminal end (found in #4081 review). | ||
|
|
||
| This backend implements the zero-or-more packet return and `flush` that | ||
| moq-audio's encoder took in m2 before the cut | ||
| (decided 2026-11-07, maintainer, /quest-plan from the API sweep of open PRs | ||
| and remaining quests), so it adds no break and waits on no other encoder backend; its round-trip test still needs its platform decoder. | ||
|
|
||
| Decided 2026-11-09 (maintainer, moq-dev/studio /quest-audit): this moved from | ||
| m5 to m3. The studio's m0 | ||
| [destinations](https://github.com/moq-dev/studio/blob/main/quest/m0/destinations.md) | ||
| wait on it through the | ||
| [AAC pin quest](https://github.com/moq-dev/studio/blob/main/quest/m0/pin-aac-encode.md) | ||
| for RTMP ingests that require AAC, and m3 is where work a named consumer | ||
| waits on belongs. | ||
|
|
||
| ## Required | ||
|
|
||
| - [AudioToolbox decode](/quest/m3/audio-decode-audiotoolbox.md) - the round-trip regression decodes through it | ||
|
|
||
| ## Related | ||
|
|
||
| - [Media Foundation AAC encode](/quest/m3/audio-encode-mediafoundation.md) - the same encode seam on Windows | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| # [M] Media Foundation AAC encode on Windows | ||
|
|
||
| ## Goal | ||
|
|
||
| On Windows, `Codec::Aac` encodes through the Media Foundation AAC encoder | ||
| MFT, mono, stereo, and up to 5.1. | ||
|
|
||
| ## Plan | ||
|
|
||
| The audio counterpart of `rs/moq-video/src/encode/backend/mediafoundation.rs`, | ||
| behind the encode seam on Windows. | ||
|
|
||
| - The encoder MFT fixes its output rates and bitrates per channel count; | ||
| refuse configurations outside that table at construction rather than | ||
| letting the MFT pick silently. | ||
| - The catalog ASC is synthesized at construction per the encode seam; the | ||
| output type's `MF_MT_USER_DATA` must match it, asserted in tests. | ||
| - Round-trip regression through the Media Foundation decoder; verification on | ||
| a Windows host, since the per-PR CI compiles only. | ||
|
|
||
| This backend implements the zero-or-more packet return and `flush` that | ||
| moq-audio's encoder took in m2 before the cut | ||
| (decided 2026-11-07, maintainer, /quest-plan from the API sweep of open PRs | ||
| and remaining quests), so it adds no break and waits on no other encoder backend; its round-trip test still needs its platform decoder. | ||
|
|
||
| Decided 2026-11-09 (maintainer, moq-dev/studio /quest-audit): this moved from | ||
| m5 to m3. The studio's m0 | ||
| [destinations](https://github.com/moq-dev/studio/blob/main/quest/m0/destinations.md) | ||
| wait on it through the | ||
| [AAC pin quest](https://github.com/moq-dev/studio/blob/main/quest/m0/pin-aac-encode.md) | ||
| for RTMP ingests that require AAC, and m3 is where work a named consumer | ||
| waits on belongs. | ||
|
|
||
| ## Required | ||
|
|
||
| - [Media Foundation decode](/quest/m3/audio-decode-mediafoundation.md) - the round-trip regression decodes through it | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| # [L] moq-audio filter chain | ||
|
|
||
| ## Goal | ||
|
|
||
| Each audio source can run an ordered processing chain in moq-audio before the | ||
| program bus: noise suppression, noise gate, compressor, and limiter, plus | ||
| ducking, which lowers one source (music) when another (the microphone) is | ||
| active. Third-party VST3 and CLAP plugins load into the chain. The | ||
| [studio's audio filters](https://github.com/moq-dev/studio/blob/main/quest/m0/audio-filters.md) in moq-dev/studio | ||
| configure it per source. | ||
|
|
||
| ## Plan | ||
|
|
||
| Changed 2026-11-02 (maintainer): the studio moved to moq-dev/studio. The | ||
| chain lands here with its own tests and benchmark; the studio's panel and | ||
| scene-file shape follow a pin bump there. | ||
|
|
||
| Decided 2026-11-09 (maintainer, moq-dev/studio /quest-audit): this moved from | ||
| m5 to m3. The studio's m0 audio filters wait on it through the | ||
| [filter chain pin quest](https://github.com/moq-dev/studio/blob/main/quest/m0/pin-audio-filters.md), | ||
|
Comment on lines
+18
to
+20
Contributor
There was a problem hiding this comment. Update the studio filter-chain links.
 |
||
| and m3 is where work a named consumer waits on belongs. | ||
|
|
||
| Decided 2026-11-02 (maintainer): the chain runs on the capture producer | ||
| before `capture::Producer` fans out to its consumers, so | ||
| the bus and a source's own track both carry filtered audio; gain and mute | ||
| stay after the split, in the bus `Strip`, so the chain has no gain stage of | ||
| its own; a compressor's makeup gain covers level fixes. | ||
|
|
||
| Prefer maintained crates, such as `nnnoiseless` for RNNoise suppression, over | ||
| hand-rolled DSP. Benchmark the chain per source so its cost per added source | ||
| shows up as a slope. Host plugins out of process, so a crashing or hung plugin | ||
| drops that filter rather than the audio bus: the audio thread never blocks on | ||
| the plugin process, a block the plugin misses its deadline for passes through | ||
| unfiltered, and a plugin that keeps missing is bypassed and reported as failed. | ||
| Use a maintained host crate (such as `clack` for CLAP) rather than | ||
| hand-rolling the plugin ABIs. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| # [S] moq-rtmp dials rtmps:// and always advertises a tcUrl | ||
|
|
||
| ## Goal | ||
|
|
||
| moq-rtmp's client dials an `rtmp://` or `rtmps://` URL itself: it resolves | ||
| the host, connects, and for `rtmps://` completes the TLS handshake against | ||
| the host's name. Its `connect` command advertises a `tcUrl` derived from that | ||
| URL on both schemes: its scheme, host, port if it names one, and app, never | ||
| the stream key. | ||
| `moq import rtmp` and `moq export rtmp` take `rtmps://` in `--connect`. | ||
|
|
||
| A test dials the crate's own TLS listener by URL and asserts the server saw | ||
| that `tcUrl`, and a server certificate the client does not trust fails the | ||
| dial. | ||
|
|
||
| ## Plan | ||
|
|
||
| Decided 2026-11-09 (maintainer, moq-dev/studio /quest-audit): this is its own | ||
| quest in m3, since moq-dev/studio's | ||
| [destinations](https://github.com/moq-dev/studio/blob/main/quest/m0/destinations.md) | ||
| wait on it to push to RTMPS ingests. | ||
|
|
||
| Today (`rs/moq-rtmp/src/dial.rs`): | ||
|
|
||
| - `Client::connect(addr, app)` takes a resolved `SocketAddr`, dials plaintext | ||
| TCP, and advertises `rtmp://<addr>/<app>`. That `tcUrl` carries the | ||
| resolved IP, not the name the URL gave, so a server that picks a virtual | ||
| host from it sees the wrong one. | ||
| - `Client::with_stream(stream, app)` is the only route to `rtmps://`: the | ||
| caller makes the TLS stream. It passes `tc_url: None`, and several ingest | ||
| servers reject a `connect` without one. | ||
| - moq-cli resolves the host itself and refuses every scheme but `rtmp` | ||
| (`parse_url` in `rs/moq-cli/src/rtmp.rs`), so `doc/bin/rtmp.md` says the | ||
| CLI is plaintext only. | ||
|
|
||
| Guidance: | ||
|
|
||
| - No route leaves the `tcUrl` out. `with_stream` either takes what it needs | ||
| to advertise one or goes away; propose the shape, changing `connect` | ||
| rather than adding a second entry point beside it. | ||
| - The caller supplies the TLS client configuration, as `Server::with_tls` | ||
| takes the server's, behind the same `tls` feature. | ||
| - `Request::tc_url()` on the server already reports what a client sent, | ||
| and `tls_fixture` in `rs/moq-rtmp/src/server.rs` builds a self-signed | ||
| listener, so the test needs no new harness. | ||
| - Update `doc/bin/rtmp.md`, and put the migration note in the PR's Impact | ||
| section if a published entry point changes. | ||
|
|
||
| ## Related | ||
|
|
||
| - [moq-webrtc and moq-rtmp dial through a caller's policy](/quest/m3/webrtc-dial-policy.md) - applies a host's address policy to the URL dial this adds |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Update the studio AAC quest links.
moq-dev/studio’squest/m0/pin-aac-encode.md, Lines 5–6, still links to both encoders underm5. Once these moves land on moq’s main branch, those links will break. Coordinate an update to them3paths or retain redirects. (raw.githubusercontent.com)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Declining here: those links live in moq-dev/studio, which this PR cannot edit, and quest files keep no redirects. The studio's pin quests need their m5 paths switched to m3 in a studio PR once this merges.
(Written by Opus 5.5 with low effort)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@kixelated Understood. The studio PR can update the links after this PR merges.
Would you like a follow-up issue in
moq-dev/moqto track that update?Learnings used
You are interacting with an AI system.