diff --git a/quest/m0/README.md b/quest/m0/README.md index 49ae345d17b..55bddeb5c1f 100644 --- a/quest/m0/README.md +++ b/quest/m0/README.md @@ -35,7 +35,8 @@ since cutting segments from first-frame pts waits on nothing in m1. - [Backport the joined-head FETCH race fix](/quest/m0/backport-fill-head-race.md) - a `release` relay stops FETCHing a group head its own SUBSCRIBE is bringing, which left live readers hanging - [CI cost and latency](/quest/m0/ci/README.md) - cloud CI runs only in a batched merge gate as one `just ci` pass per batch, expensive checks run nightly and gate `release`, caches move to R2, and runners are chosen last - [More tests under load](/quest/m0/test-flakes-2/README.md) - load-only and queue failures, one quest per flake, fixed at the cause, since each one fails a whole batch -- [Backport the dot-segment and auth-redirect refusals](/quest/m0/backport-release-hardening.md) - `release`'s relay refuses dot-segment paths and its auth client refuses redirects +- [Backport the relay and auth hardening to release](/quest/m0/backport-release-hardening.md) - `release`'s relay refuses dot-segment paths, serves its HTTP routes with main's response hardening, and withdraws advertisements a peer starves of send credit; its auth client refuses redirects +- [Release's MKV importer bounds what it buffers](/quest/m0/backport-mkv-buffer.md) - on `release`, an MKV input that declares a huge element and trickles its bytes no longer grows the importer's memory without limit - [HLS export on a declared grid](/quest/m0/hls-media-grid.md) - the catalog's `hls` field and record tracks go away, and the export numbers segments on `media.join.interval` from media tracks - [Agents use quest's worktree hooks](/quest/m0/quest-worktree-hooks.md) - adopt `quest worktree hook` once it's shown to coexist with T3 Code's worktrees - [HLS lists contributions](/quest/m0/hls-contributions.md) - an HLS master lists each catalog contribution's renditions, and a contribution's media is subscribed only while players fetch it diff --git a/quest/m0/backport-mkv-buffer.md b/quest/m0/backport-mkv-buffer.md new file mode 100644 index 00000000000..0b76d9c765e --- /dev/null +++ b/quest/m0/backport-mkv-buffer.md @@ -0,0 +1,58 @@ +# [S] Release's MKV importer bounds what it buffers + +## Goal + +On `release`, an MKV input that declares a huge element and trickles its +bytes no longer grows the importer's memory without limit: once the bytes +held for one incomplete element pass a size bound, the import fails with an +error. Ordinary files of any total size still import, and the published API +and supported inputs are unchanged. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 /quest-complete run), from [moq-dev/moq#7234](https://github.com/moq-dev/moq/pull/7234). + +Backport #7234's MKV unbounded-buffer fix to `release`? ✅ Trimmed +backport, m0: confirm with the 1 GiB repro on `release`, size bound only. + +Why (the recommendation's reasoning, from #7234's Impact): #7234 is a real memory-exhaustion fix, but the full change is a +rewrite that also refuses inputs `release` accepts today (subtitle tracks, +CMAF output, video `BlockGroup`s with fields after their `Block`). A size +bound fixes the exhaustion on `release` without changing what it imports. + +- **Confirm first**: port #7234's reproduction, + `ignored_payload_does_not_accumulate_under_a_large_metadata_master` (a + 1 GiB EBML header holding a nearly 1 GiB Void, fed in 1 KiB chunks), onto + `release` and show it grows memory there. If it doesn't, report that and + stop. +- **Where it buffers**: `release`'s + `rs/moq-mux/src/container/mkv/import.rs:139` builds ebml-iterable's + `TagDecoder` to collect the `Ebml`, `Info`, `Tracks`, `TrackEntry`, + `Audio`, `Video`, and `BlockGroup` masters whole, and `decode` (`:170`) + keeps every byte of an incomplete tag in `self.buffer` until it completes; + `SimpleBlock` and ignored binary leaves such as `Void` are held the same + way. +- **The bound**: refuse once the bytes retained for the element in progress + exceed a cap, checked in `decode` before appending more. `Segment` and + `Cluster` arrive as start and end events and are never held whole, so a + large file is unaffected. Do not set ebml-iterable's + `max_allowable_tag_size`: it applies to every sized tag including + `Segment`, which [moq-dev/moq#7234](https://github.com/moq-dev/moq/pull/7234) + (merged) ruled out. Pick the cap from the largest legitimate block `release` can + publish (its transport frame limit, if it has one), and name the choice in + the PR. The cap applies to every held element, so it must also clear the + largest legitimate `Tracks`/`TrackEntry` (CodecPrivate) and `Void`; if one + cap can't cover both blocks and metadata, use a cap per element kind, or + skip an ignored leaf such as `Void` without holding its bytes. +- Tests on `release`: the reproduction now fails with the error and + bounded memory, a single element just over its cap fails while one just + under imports, and an ordinary file larger than the cap still imports. +- Version bumps stay with a human, per `CONTRIBUTING.md`: this PR bumps + nothing. + +Public API: none; if refusing needs a new `mkv::Error` variant on +`release`, report it, since the enum is published. Wire: none. + +## Related + +- [Backport the relay and auth hardening to release](/quest/m0/backport-release-hardening.md) - other security backports for the same patch release diff --git a/quest/m0/backport-release-hardening.md b/quest/m0/backport-release-hardening.md index 1da9042ee4b..4e4bf984035 100644 --- a/quest/m0/backport-release-hardening.md +++ b/quest/m0/backport-release-hardening.md @@ -1,8 +1,8 @@ -# [S] Backport the dot-segment and auth-redirect refusals to release +# [M] Backport the relay and auth hardening to release ## Goal -The `release` branch's relay refuses connection paths with `.` or `..` segments before asking the auth server, and `release`'s auth client refuses redirects from the auth server, matching main. The fixes ship in the next patch release a human cuts. +The `release` branch's relay refuses connection paths with `.` or `..` segments before asking the auth server, and `release`'s auth client refuses redirects from the auth server, matching main. `release`'s relay also serves its HTTP routes with main's response hardening and ops-listener checks, and withdraws a moq-transport advertisement a peer starves of send credit, with the published API unchanged. The fixes ship in the next patch release a human cuts. ## Plan @@ -13,7 +13,13 @@ Decided 2026-10-30 (maintainer, follow-up planning after the /quest-afk run). - Hand-port rather than cherry-pick where `release` lacks main's structure, as [moq-dev/moq#7389](https://github.com/moq-dev/moq/pull/7389) did, with a regression test for each on `release`. - Version bumps stay with a human: several backports combine into one patch release, so this PR bumps nothing. -Public API: none. Wire: none. +Added 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 /quest-complete run): + +- **Relay HTTP hardening:** port [moq-dev/moq#7210](https://github.com/moq-dev/moq/pull/7210) (merged): `/fetch` serves bytes as `application/octet-stream`, `/fetch` and `/announced` send `nosniff` and `private, no-store` (refusals included), and the ops listener refuses any request with `Origin` and any `Host` or URI authority that isn't the accepted socket's destination. `release` has its own `rs/moq-relay/src/fetch.rs` and `internal.rs`; port the behavior and #7210's tests, not its layout. +- **Withdraw without credit:** port [moq-dev/moq#7414](https://github.com/moq-dev/moq/pull/7414) (merged), keeping `release`'s published API: a withdrawn PUBLISH_NAMESPACE or REQUEST_UPDATE that a peer starves of send credit is reset in both directions instead of pinning the announce loop. Port its zero- and partial-credit withdrawal tests for the drafts `release` speaks. +- **Landed:** the dot-segment and auth-redirect items above merged into `release` on 2026-11-02 in [moq-dev/moq#7599](https://github.com/moq-dev/moq/pull/7599), which also settles the redirect follow-up that [moq-dev/moq#7298](https://github.com/moq-dev/moq/pull/7298) and [moq-dev/moq#7278](https://github.com/moq-dev/moq/pull/7278) suggested. What remains is the two ports above, then the patch release. [moq-dev/moq#7596](https://github.com/moq-dev/moq/pull/7596) (open) holds this quest's main-side checkpoint. + +Public API: none. Wire: none; as on main, a withdrawal resets a partially written request without waiting for peer credit. ## Related diff --git a/quest/m0/ci/README.md b/quest/m0/ci/README.md index 8d65d78b557..2b375cdecd8 100644 --- a/quest/m0/ci/README.md +++ b/quest/m0/ci/README.md @@ -163,5 +163,11 @@ The historical samples locate candidates; they are not controlled benchmarks. - [Drop main from the publish Environment](/quest/m0/ci/publish-release-only.md) - the maintainer narrows `publish` to `release` and release tags once no job naming it runs as `main` - [Binary release publishers run on their tag](/quest/m0/ci/release-tag-publishers.md) - brew, Go and Swift releases stop running through `workflow_run` on main - [The queue host keeps the queue's disk](/quest/m0/ci/queue-host-disk.md) - agent sessions can't starve the queue runner of disk, and a crashed runner restarts +- [The bench comparison builds moq-gst's catalog bench on its base](/quest/m0/ci/bench-base-build.md) - A paired bench comparison that selects `rust/gst/moq-gst/catalog` against `origin/main` builds the base side and reports a comparison, instead of marking it unavailable with "base build failed" +- [A jq error in the Rust selection fails the run](/quest/m0/ci/check-selection.md) - `just check`'s Rust selection fails the run when one of its jq filters errors, instead of selecting less +- [Code scanning has no untriaged alert on main](/quest/m0/ci/code-scanning-triage.md) - every open code-scanning alert on `main`'s default branch is either fixed or dismissed with a reason and a comment that names the evidence +- [The workspace-hack refuses trybuild edges and spares non-media builds](/quest/m0/ci/hakari-followups.md) - `hakari.py` refuses trybuild dev-dependency edges, and non-media Linux test builds stop needing libpipewire and libclang +- [Pullfrog reviews finish inside their time limit](/quest/m0/ci/pullfrog-timeout.md) - A Pullfrog review of a large PR finishes instead of failing on the action's run limit +- [Long test runs keep their built binaries](/quest/m0/ci/shared-target.md) - A long `just test interop` or wire-compat run never fails because the binaries it built were removed from the shared Cargo target mid-run, such as by mbx pruning its managed build storage - [One nextest pin](/quest/m0/ci/nextest-pin.md) - cargo-nextest's version lives in one place - [The flake lane skips hosts that can't map user namespaces](/quest/m0/ci/userns-uid-map.md) - `check flake` skips instead of failing where `/proc/self/uid_map` is unwritable diff --git a/quest/m0/ci/bench-base-build.md b/quest/m0/ci/bench-base-build.md new file mode 100644 index 00000000000..97fe607b218 --- /dev/null +++ b/quest/m0/ci/bench-base-build.md @@ -0,0 +1,44 @@ +# [S] The bench comparison builds moq-gst's catalog bench on its base + +## Goal + +A paired bench comparison that selects `rust/gst/moq-gst/catalog` against +`origin/main` builds the base side and reports a comparison, instead of +marking it unavailable with "base build failed". Local reviews and gates +that run `quest` use the build `flake.nix` pins, or say which they ran. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 /quest-complete run), from [moq-dev/moq#7347](https://github.com/moq-dev/moq/pull/7347). Milestone: ✅ m0 (user, "all recommended" for quests 1-8). + +- **Base build**: #7347's `just ci` bench comparison reported "base build + failed" for `rust/gst/moq-gst/catalog` against `origin/main`, from a + change that doesn't touch moq-gst. `bench/run.sh:396-405` (at + be39188000) builds the base in a detached `git worktree` through + `criterion_build_base` (`:330`), and any failure there marks every + selected target unavailable without showing why. The bench target + (`rs/moq-gst/Cargo.toml:48`, `benches/catalog.rs`) has existed since + 2026-10-14, so a missing target is not the cause. Check first whether + it was the nested-checkout Cargo alias merge Nightly hit: #7347's run + predates both [moq-dev/moq#7245](https://github.com/moq-dev/moq/pull/7245), + which made those aliases strings, and + [moq-dev/moq#7445](https://github.com/moq-dev/moq/pull/7445) (merged), + which moved the base worktree out of the checkout to + `${TMPDIR:-/tmp}/moq-bench-base.*` (`checkout_base`, `:226`). Nightly's + baseline has built since + ([moq-dev/moq#7592](https://github.com/moq-dev/moq/pull/7592)). If a + comparison on current main builds the base, only the logging below is + left. Otherwise look for what the fresh worktree lacks that the checkout + has (GStreamer's build environment, a generated or untracked input, + submodules), reproducing with the bench command #7347's `just ci` ran, + and fix the cause. Either way, have the base build log its error into + the run's output. +- **quest version**: in the same run, the `quest` on the host's PATH was + not the build `flake.nix` pinned (then `github:moq-dev/quest/1ef2ca7`, + `5e66075` at be39188000, served by the dev shell); the review was rerun + with the pinned one. Check which scripts or recipes call `quest` (the + gate and `quest review` paths) and make them resolve the pinned build, + or fail with a message naming both versions when they differ. Settle on + whichever is simpler; the quest binary's own behavior is out of scope. + +Public API: none. Wire: none. diff --git a/quest/m0/ci/check-selection.md b/quest/m0/ci/check-selection.md new file mode 100644 index 00000000000..d7cf6f47d19 --- /dev/null +++ b/quest/m0/ci/check-selection.md @@ -0,0 +1,36 @@ +# [XS] A jq error in the Rust selection fails the run + +## Goal + +`just check`'s Rust selection fails the run when one of its jq filters +errors, instead of logging the error and selecting less. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 /quest-complete run), from [moq-dev/moq#7303](https://github.com/moq-dev/moq/pull/7303), [moq-dev/moq#7280](https://github.com/moq-dev/moq/pull/7280), [moq-dev/moq#7304](https://github.com/moq-dev/moq/pull/7304), and [moq-dev/moq#6946](https://github.com/moq-dev/moq/pull/6946). Milestone: ✅ m0 (user, "all recommended" for quests 1-8). + +The other two parts of the original quest are gone from main: +[moq-dev/moq#7473](https://github.com/moq-dev/moq/pull/7473) (merged) moved +`just rs send-depth` out of the gate into Nightly's Tests matrix +(`.github/workflows/nightly.yml:355`), which runs it whatever changed, and +[moq-dev/moq#7475](https://github.com/moq-dev/moq/pull/7475) (merged) +deleted `test/wasm/uniffi/` and `wasm-baseline` with the rest of WASM. + +- **The noise**: #7303's `just check` logged `jq: error ... null cannot be + matched` without failing. That message matches what + `sh/rs/uring.test.py` (run by `.github/justfile:72`) printed four times + from a cargo-metadata fixture without `workspace_root` or `targets`; + [moq-dev/moq#7621](https://github.com/moq-dev/moq/pull/7621) (merged) + completed the fixture and asserts empty stderr. Confirm a `just check` on + current main logs no jq error; if one remains, find and fix its filter. +- **Failing the run**: `sh/rs/select.sh:9` sets `set -euo pipefail`, but + `select_packages` runs as `packages=$(select_packages)` (`:112`, at + be39188000), and bash clears `-e` inside a command substitution, so a jq + failure in its filters (`:38`, `:64`, `:77`, `:104`) lets the function + carry on and print a smaller selection. #7621's "before" run shows it: + io_uring selection passed with four jq errors. Make the failure propagate + (`shopt -s inherit_errexit`, or checking each `jq` status), check + `sh/rs/c-tests.sh:36` for the same pattern, and add a + `sh/rs/select.test.py` case where malformed metadata fails selection. + +Public API: none. Wire: none. diff --git a/quest/m0/ci/code-scanning-triage.md b/quest/m0/ci/code-scanning-triage.md new file mode 100644 index 00000000000..3260a16a054 --- /dev/null +++ b/quest/m0/ci/code-scanning-triage.md @@ -0,0 +1,51 @@ +# [XS] Code scanning has no untriaged alert on main + +## Goal + +Every open code-scanning alert on `main`'s default branch is either fixed +or dismissed with a reason and a comment that names the evidence: +the `rust/cleartext-logging` alerts +[58](https://github.com/moq-dev/moq/security/code-scanning/58), +[59](https://github.com/moq-dev/moq/security/code-scanning/59), and +[61](https://github.com/moq-dev/moq/security/code-scanning/61), and the +`rust/hard-coded-cryptographic-value` alerts +[35](https://github.com/moq-dev/moq/security/code-scanning/35), +[36](https://github.com/moq-dev/moq/security/code-scanning/36), and +[64](https://github.com/moq-dev/moq/security/code-scanning/64). + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 /quest-complete run), from [moq-dev/moq#7362](https://github.com/moq-dev/moq/pull/7362) and [moq-dev/moq#7027](https://github.com/moq-dev/moq/pull/7027). CodeQL: ✅ Dismiss + triage quest. Milestone: ✅ m0 (user: "use recommended for any CI/flakes"). + +Alerts [67](https://github.com/moq-dev/moq/security/code-scanning/67) and +[68](https://github.com/moq-dev/moq/security/code-scanning/68) were +dismissed on 2026-10-31 as "used in tests" (test assertion messages in +`rs/moq-relay/src/web.rs`'s `mod tests`); follow that precedent, and the +earlier dismissals 25 to 34. Open alerts, as of 2026-10-31 on `38566f5a84`: + +- **58** `rs/moq-auth/src/client.rs:1150` and **61** + `rs/moq-rtc/src/lib.rs:434`: both are inside `mod tests` (from + `client.rs:497` and `lib.rs:74`). The "secret" is the literal strings + `jwt`, `secret`, and `user:pass` in an assertion that proves those never + appear in errors or logs. Dismiss as "used in tests". +- **59** `rs/moq-tokio/src/server.rs:1416`: production code. CodeQL traces + `read_certs(...)` / `certs.load_certs(...)` (`rs/moq-tokio/src/tls.rs:211`, + `:1364`, `:1546`) into a log. Certificates are public, but confirm the + flow cannot carry the private key or its path's contents (for example + through a TLS load error's message); fix the log if it can, otherwise + dismiss as a false positive with that reasoning. #7027 already judged + the `server.rs` cert alert a false positive. +- **35**, **36** `rs/moq-tokio/src/mdns.rs:778-779`: the test keys + `KEY_A`/`KEY_B` inside `mod tests` (from `:775`). Dismiss as "used in + tests". +- **64** `rs/moq-quic/src/varint.rs:142`: the zeroed scratch buffer a + varint decodes into, not a key. #7027 judged it a false positive; dismiss + as one. + +Dismiss through `gh api -X PATCH repos/moq-dev/moq/code-scanning/alerts/` +with `state=dismissed`, `dismissed_reason` (`false positive` or +`used in tests`), and `dismissed_comment`, the comment ending in the model +line. If a fix is needed, it lands in this quest's PR; a dismissal-only finish +needs no code change, so the PR only deletes this quest. + +Public API: none. Wire: none. diff --git a/quest/m0/ci/hakari-followups.md b/quest/m0/ci/hakari-followups.md new file mode 100644 index 00000000000..64c6135830d --- /dev/null +++ b/quest/m0/ci/hakari-followups.md @@ -0,0 +1,40 @@ +# [XS] The workspace-hack refuses trybuild edges and spares non-media builds + +## Goal + +`sh/rs/hakari.py` fails when a crate that dev-depends on trybuild also +dev-depends on `workspace-hack`, and a Linux test build of a crate with no +media dependency needs neither libpipewire nor libclang. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 /quest-complete run), from [moq-dev/moq#6985](https://github.com/moq-dev/moq/pull/6985). Milestone: ✅ m0 (user: "use recommended for any CI/flakes"). + +- **trybuild edge**: trybuild copies a crate's dev-dependencies into the + project it compiles each UI case in, so `kio-macros`' hack edge made + `kio-macros::ui` compile the whole hack (88-120 s), time out, and starve + the tests beside it; #6985 (merged) was ejected from the land queue for it + ([run 41074871604](https://github.com/moq-dev/moq/actions/runs/41074871604)) + and fixed it by excluding `kio-macros` by hand (`.config/hakari.toml`'s + `[final-excludes]`). `sh/rs/hakari.py:37-53` (at be39188000) only checks + that excluded members lack the edge and the rest carry it as a path + dev-dependency. Make it derive the rule: a crate with `trybuild` in its + dev-dependencies must not carry the `workspace-hack` edge, with a test + that plants one. +- **libpipewire**: CodeRabbit flagged on #6985 that the hack's unified + features pull PipeWire (through cpal) and libclang into ordinary Linux + test builds of crates that use no media; `rs/workspace-hack/Cargo.toml` + still lists `pipewire` (`:353`) and `clang-sys` (`:372`). The dev shell + provides both, so only builds outside it notice. Recommended: exclude the + pipewire/cpal feature from the hack with hakari's final-excludes, then + measure that the cache hit rate across jobs barely moves. If it does + move, bring the numbers to the maintainer instead. + +`cargo-hakari` is already in the dev shell (`flake.nix:162`, in `rustDeps`), +so that part of the original quest is done. + +Public API: none. Wire: none. + +## Related + +- [Unify dependency features with cargo-hakari](/quest/m0/ci/hakari.md) - records the hosted jobs' cache hit rate around the workspace-hack, the measurement the pipewire exclusion needs diff --git a/quest/m0/ci/pullfrog-timeout.md b/quest/m0/ci/pullfrog-timeout.md new file mode 100644 index 00000000000..19a1d0537c7 --- /dev/null +++ b/quest/m0/ci/pullfrog-timeout.md @@ -0,0 +1,53 @@ +# [XS] Pullfrog reviews finish inside their time limit + +## Goal + +A Pullfrog review of a large PR finishes instead of failing on the +action's run limit: `.github/workflows/pullfrog.yml` sets a `timeout` +and `effort` for the `pullfrog/pullfrog` step that fit the reviews it is +dispatched for, and the choice is recorded in the PR with the run +durations that support it. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 /quest-complete run). Milestone: ✅ m0 (user: "use recommended for any CI/flakes"). The fix is to raise the timeout or lower the effort. + +Facts from the last 100 runs of the workflow (58 runs, all +`workflow_dispatch` reviews; 55 passed): + +- [41178266611](https://github.com/moq-dev/moq/actions/runs/41178266611) + ("Review #7202"): `agent run timed out after 1h`. The log shows + `effort: high` and `timeout: 1h (default)`. +- [41188250827](https://github.com/moq-dev/moq/actions/runs/41188250827) + ("Review #7234", 56 min): aborted by the action's activity watchdog after + the model sent nothing for 903 s. Not the run limit, but a lower effort + likely shortens those silent turns too. +- [41197083466](https://github.com/moq-dev/moq/actions/runs/41197083466) + ("Review #7211", 23 min): failed installing dependencies + (`@moq/libav@0.1.0 is not published to npm yet, and building it needs + Nix`). Unrelated to the limit; note it in the PR as a follow-up rather + than fixing it here. + +It still recurs after this plan: runs +[41362378749](https://github.com/moq-dev/moq/actions/runs/41362378749) +("Review #7586") and +[41404718134](https://github.com/moq-dev/moq/actions/runs/41404718134) +("Review #7599") both failed with `agent run timed out after 1h` at +`effort: high`. + +The action takes `timeout` ("e.g. 10m, 1h30m. Default: 1h") and `effort` +(`low` to `max`, or a position in [0, 1]) inputs; today neither is set in +`with:` (`pullfrog.yml:34-35`, at be39188000), so the `high` comes from +the repo's Pullfrog settings. The file's header allows +edits only where indicated; `with:` is the place. Recommended: set +`effort: medium` first, since it bounds cost as well as time, and raise +`timeout` only if medium-effort reviews of the largest recent +PRs still approach an hour. Keep the job's own `timeout-minutes` (default +360) above whatever `timeout` is set. + +[moq-dev/moq#7317](https://github.com/moq-dev/moq/pull/7317) (merged, pins +the action) and [moq-dev/moq#7424](https://github.com/moq-dev/moq/pull/7424) +(merged, gates its secrets to main through the `pullfrog` environment) +already reshaped this file; edit on top of them. + +Public API: none. Wire: none. diff --git a/quest/m0/ci/shared-target.md b/quest/m0/ci/shared-target.md new file mode 100644 index 00000000000..ef1aeb814ba --- /dev/null +++ b/quest/m0/ci/shared-target.md @@ -0,0 +1,33 @@ +# [S] Long test runs keep their built binaries + +## Goal + +A long `just test interop` or wire-compat run never fails because the +binaries it built were removed from the shared Cargo target mid-run, such +as by mbx pruning its managed build storage. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 /quest-complete run), from [moq-dev/moq#7211](https://github.com/moq-dev/moq/pull/7211). Milestone: ✅ m0 (user, "all recommended" for quests 1-8). + +- **Target removed mid-run**: the 2026-10-31 run reported the shared + `target/` deleted by mbx under long interop and wire-compat runs. mbx + links each checkout's target to managed storage that prunes itself + (`mbx gc` collects managed targets to fit its budget), and + `test/interop/interop.sh:578` (at be39188000) resolves `TARGET_BASE` + once, then runs `moq-relay`, `moq-cli`, its examples, and moq-c's library + from there for the whole matrix; `test/interop/compat.sh:72-73` does the + same. Confirm from mbx's logs or `mbx gc --dry-run` which collection + removes them. Then either copy everything the run loads from the target + into the run's own directory (`$HARNESS_RUN`) right after building, + which needs no mbx change: binaries, examples, moq-c's headers and + library, and any GStreamer plugin the matrix loads, with every consumer + pointed at the copies. Or use an mbx lease or pin if mbx offers one. + Prefer whichever keeps one rule for every long-running script. + +The one-off `Argument list too long` Cargo launch #7211 saw is fixed: +[moq-dev/moq#7594](https://github.com/moq-dev/moq/pull/7594) (merged) +traced it to mbx copying a PATH that grew with every nested dev shell and +dedupes PATH in `flake.nix`'s `shellHook`. + +Public API: none. Wire: none. diff --git a/quest/m0/interop-green/README.md b/quest/m0/interop-green/README.md index fdedf1c7ae7..b0c9617b4c4 100644 --- a/quest/m0/interop-green/README.md +++ b/quest/m0/interop-green/README.md @@ -1,4 +1,4 @@ -# [XS] Interop and Video pass on main +# Interop and Video pass on main ## Goal @@ -24,7 +24,11 @@ Already owned: the intermittent media-matrix timestamp-gap cells [test flakes](/quest/m0/test-flakes-2/README.md), with [moq-dev/moq#6990](https://github.com/moq-dev/moq/pull/6990) open. -Every child has landed. What remains: confirm a scheduled Interop +This README's own work, after the children: confirm a scheduled Interop and Video run each pass. Public API: none. Wire: none. + +## Required + +- [The browser js -> js moq-transport cell stays fixed](/quest/m0/interop-green/js-pause-resume.md) - interop's browser `js -> js` cell passes on moq-transport drafts 20 and 22 in the scheduled run on `main`, with a regression test for the fix diff --git a/quest/m0/interop-green/js-pause-resume.md b/quest/m0/interop-green/js-pause-resume.md new file mode 100644 index 00000000000..c25da4bf2fe --- /dev/null +++ b/quest/m0/interop-green/js-pause-resume.md @@ -0,0 +1,69 @@ +# [S] The browser js -> js moq-transport cell stays fixed + +## Goal + +Interop's browser `js -> js` cell passes on moq-transport drafts 20 and 22 +in the scheduled run on `main`: the browser subscriber gets the catalog +and media through the relay, and playback resumes after a pause, without +the relay forwarding a `PUBLISH_DONE` with an internal error. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 /quest-complete run), from [moq-dev/moq#7236](https://github.com/moq-dev/moq/pull/7236). Milestone: ✅ m0 (user, "all recommended" for quests 1-8). + +The cell fails on `main` in scheduled Interop +[41075505055](https://github.com/moq-dev/moq/actions/runs/41075505055) +(2026-10-30) and +[40896908858](https://github.com/moq-dev/moq/actions/runs/40896908858) +(2026-10-29). Two symptoms so far: + +- #7236's local run (`just test interop --versions ietf --publishers rust,js + --subscribers rust,js,js-native-node,js-native-bun`): every cell passes + on drafts 18 and 22 except browser `js -> js` on draft 22, which times out + waiting for playback to resume after a pause. After the resume, the + browser publisher ends the resumed subscriptions and the relay forwards + `PUBLISH_DONE` INTERNAL_ERROR. It fails the same way on main at + `1d8bf24998`. That report names no draft-20 result; record it in the + reproduction. The Goal's drafts are 20 and 22; draft 18's cells must stay + green but are not this quest's target. +- In 41075505055 the cell fails earlier: the browser subscriber's + `catalog.json` SUBSCRIBE gets SUBSCRIBE_OK and then + `publish done: status=0x0 reason=internal error`, so playback never starts. + +It has since gone green: the next scheduled run, +[41317901057](https://github.com/moq-dev/moq/actions/runs/41317901057) +(2026-11-01, `c95a9b1201`), passes browser `js -> js` on drafts 18, 20, and +22, where +[41209722015](https://github.com/moq-dev/moq/actions/runs/41209722015) +(2026-10-31, `38566f5a84`) failed it on 20 (`35s of 30s`) and 22 (`5s of +30s`). That run's own failure is the browser media tests in +[browser media flakes](/quest/m0/test-flakes-2/browser-media-flakes.md), +not this cell. Both PRs this quest suspected landed in between: +[moq-dev/moq#7383](https://github.com/moq-dev/moq/pull/7383) (merged, +draft-22 resume keeps DYNAMIC_GROUPS) and +[moq-dev/moq#7236](https://github.com/moq-dev/moq/pull/7236) (merged, +refuses inbound IETF track listings), along with `js/net`'s +[moq-dev/moq#7355](https://github.com/moq-dev/moq/pull/7355) (merged, ends +IETF requests once through `RequestStream`). + +What is left: + +- Confirm the cell stays green in the next scheduled Interop on `main`, or + locally with `just test interop --versions ietf --publishers js + --subscribers js` repeated. If it fails again, the original plan applies: + start at the `js/net` IETF publisher's side of a relay-forwarded + subscription and find which path ends it with an internal error, from the + relay log and the browser console in the cell's output. +- Name the fix: bisect `38566f5a84..c95a9b1201` over `js/net`, + `rs/moq-net/src/ietf`, and `rs/moq-relay` with the command above. +- Add a scripted-peer or unit test for that cause in `js/net` (or + `rs/moq-net` if the fix was there), so the regression shows without the + browser matrix, unless the fixing PR already added one that covers the + pause and resume. + +Public API: none expected. Wire: none expected; a fix that changes what +the publisher sends is reported in the PR. + +## Related + +- [Browser buffering and Firefox audio-tone tests pass under load](/quest/m0/test-flakes-2/browser-media-flakes.md) - other browser failures in the same Interop runs diff --git a/quest/m0/test-flakes-2/README.md b/quest/m0/test-flakes-2/README.md index 206ac2612f0..70edce71fe2 100644 --- a/quest/m0/test-flakes-2/README.md +++ b/quest/m0/test-flakes-2/README.md @@ -69,6 +69,8 @@ Public API: none. Wire: none. ## Required - [IETF22 fallback](/quest/m0/test-flakes-2/ietf22-fallback.md) - an IETF22 browser viewer follows a fallback within the media lane's bound, keeping its session and its audio +- [Browser buffering and Firefox audio-tone tests pass under load](/quest/m0/test-flakes-2/browser-media-flakes.md) - the two `harness.browser.ts` buffering tests and the Firefox player's audio-tone checks pass reliably in Interop and Nightly, each fixed at its cause under the epic's rules +- [Timing-bound tests pass beside a busy build](/quest/m0/test-flakes-2/load-timing-bounds.md) - four tests that pass alone but have failed under a loaded `just check` pass reliably there, each fixed at its cause under the epic's rules - [moq-tokio's close-drain test survives load](/quest/m0/test-flakes-2/tokio-close-drain.md) - `quic_client_close_drains_finished_track` stops failing land batches with "Unroutable" - [@moq/net's close-drain tests survive load](/quest/m0/test-flakes-2/js-close-drain.md) - the lite close-drain cases pass on every full suite run - [@moq/net's boundary-sync test survives load](/quest/m0/test-flakes-2/js-boundary-sync.md) - "a version without a boundary syncs" passes under load diff --git a/quest/m0/test-flakes-2/browser-media-flakes.md b/quest/m0/test-flakes-2/browser-media-flakes.md new file mode 100644 index 00000000000..df8a323f2b5 --- /dev/null +++ b/quest/m0/test-flakes-2/browser-media-flakes.md @@ -0,0 +1,47 @@ +# [S] Browser buffering and Firefox audio-tone tests pass under load + +## Goal + +The two `harness.browser.ts` buffering tests and the Firefox player's +audio-tone checks pass reliably in Interop and Nightly, each fixed at its +cause under the epic's rules. The Nix dev shell runs local Firefox media +tests without the host's libraries on `LD_LIBRARY_PATH`. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 /quest-complete run), from [moq-dev/moq#7402](https://github.com/moq-dev/moq/pull/7402) and [moq-dev/moq#7255](https://github.com/moq-dev/moq/pull/7255). Milestone: ✅ m0 (user, "all recommended" for quests 1-8). + +- **Buffering tests**: `the paused play button stays clickable during + {audio,video} stalling` (`test/interop/clients/js/harness.browser.ts:30`, + at be39188000) hit their 30 s timeout on main. They passed in scheduled + Interop + [40896908858](https://github.com/moq-dev/moq/actions/runs/40896908858) + (2026-10-29, under 1 s each) and failed in + [41075505055](https://github.com/moq-dev/moq/actions/runs/41075505055) + (2026-10-30), and #7402 saw them time out locally on `origin/main`. They + still time out in + [41317901057](https://github.com/moq-dev/moq/actions/runs/41317901057) + (2026-11-01, "waiting for locator('.buffering') to be hidden"). Find + which wait never resolves (the `.buffering` spinner, the Pause click, or + `.center-play`) and whether a change between those runs caused it; the + fixture is `test/interop/clients/js/buffering/`. If it is a regression in + `@moq/watch`, fix the element, not the test. +- **Firefox audio tone**: `audio tone: after resume` and + `audio tone: coded-size growth` (the `audio tone` check in `measure`, + `test/interop/clients/js/media.ts:604`, reached from the `after resume` + and `coded-size growth` steps at `:1171` and `:1100`) flake under host + load with either media fixture: #7255 measured main's fixture at 2/3 for + `rename,pause`. Run the Firefox lane + (`just test media --media-engines firefox`) under load to find what the + tone check samples too early. +- **Dev shell**: in the Nix dev shell, Firefox needs libavcodec and + libpulse for media; CI installs them, and #7255 had to expose the host's + through `LD_LIBRARY_PATH`. Provide them to the Playwright Firefox from the + flake, scoped to the test launch rather than the whole shell if that is + simpler. + +Public API: none. Wire: none. + +## Related + +- [Interop and Video pass on main](/quest/m0/interop-green/README.md) - the scheduled Interop run these tests fail in diff --git a/quest/m0/test-flakes-2/load-timing-bounds.md b/quest/m0/test-flakes-2/load-timing-bounds.md new file mode 100644 index 00000000000..d73050a0397 --- /dev/null +++ b/quest/m0/test-flakes-2/load-timing-bounds.md @@ -0,0 +1,53 @@ +# [S] Timing-bound tests pass beside a busy build + +## Goal + +Four tests that pass alone but have failed under a loaded `just check` pass +reliably there, each fixed at its cause under the epic's rules: +moq-c's `video_raw_*` tests, moq-cli's `instances::*` route-change bounds, +kio's `log::test` eviction tests, and +`sh/dispatch-query.test.py`'s +`test_terminating_the_run_stops_the_running_phase`. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 /quest-complete run), from [moq-dev/moq#6985](https://github.com/moq-dev/moq/pull/6985), [moq-dev/moq#7036](https://github.com/moq-dev/moq/pull/7036), and [moq-dev/moq#7360](https://github.com/moq-dev/moq/pull/7360). Milestone: ✅ m0 (user, "all recommended" for quests 1-8). + +They share no fixture, but each fails only under load, so they land +together: three are wall-clock bounds a starved host misses, and kio's +eviction tests depend on global state other tests in the same binary hold. +Reproduce each under load first (for example +beside a `cargo build` of the workspace, or with `stress-ng`), and fix the +cause rather than raising a bound. + +- **moq-c `video_raw_*`** (`rs/moq-c/src/test.rs:5265` onward, at + be39188000): timed out while `kio-macros::ui` compiled the whole + workspace-hack beside them in #6985's land runs. Callbacks wait `TIMEOUT` + (10 s, `test.rs:8`) on wall time, and some assert silence with a 50 ms + `recv_timeout`. Find what the starved path waits on (decode thread, + encoder, or the runtime) before changing the wait. +- **moq-cli `instances::*`** (`rs/moq-cli/tests/integration/instances.rs:18`, + `epochs.rs` until [moq-dev/moq#7489](https://github.com/moq-dev/moq/pull/7489) + renamed it): `routing()` is twice `origin::Config::default().update_hold` + plus a 1 ms `ROUND_TRIP`, 601 ms of wall time across a real relay and + real sockets. Drive the holds on a manual or simulated clock, or assert + the route change by event, so the bound measures the hold rather than the + scheduler. +- **kio `log::test`** (`rs/kio/src/log.rs:960`, `:804`, `:786`): + `reader_outlives_writer`, `a_read_outlasts_eviction`, and + `eviction_frees_whatever_readers_do` failed in the parallel run of #7036. + They take `COLLECTOR` and then `collect()` the global `crossbeam_epoch` + collector, which other tests in the same binary may still pin. Give these + tests a local `crossbeam_epoch::Collector`, or otherwise make the reclaim + they assert independent of other threads. +- **`sh/dispatch-query.test.py:539`**: failed under host load about 27 in + #7360. With `CHECK_JOBS=3` it reads stderr non-blocking (`:559-561`) + right after the stub opens the FIFO and asserts `ci: check rs started` is + already there, which races the writer. Wait for the line instead of + sampling it once. + +Public API: none. Wire: none. + +## Related + +- [Unify dependency features with cargo-hakari](/quest/m0/ci/hakari.md) - its `kio-macros` edge was the load that exposed the first two diff --git a/quest/m1/README.md b/quest/m1/README.md index d8a307f3e3e..360826fa74b 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -173,6 +173,7 @@ per-frame CPU growth since #5926. - [Container reads drop backfill](/quest/m1/read-backfill.md) - the JS container consumer drops groups below its read cursor like Rust - [Buffered playback subscribes in ascending order](/quest/m1/buffered-ascending.md) - a buffered `` asks for the oldest group first, so a voice reply's opening arrives first, and audio's head hold shrinks to the floor - [Push-to-talk loses no speech](/quest/m1/push-to-talk/README.md) - consumers hold a disabled rendition, and an unmute encodes at once with the mic still open +- [Buffered audio survives a capture stop and a disable](/quest/m1/audio-across-stop.md) - audio retained by buffered capture is still served, or expires on its own `max_age`, after capture stops or the input is disabled - [One audio capture feeds the bus and an encoder](/quest/m1/audio-capture-fanout.md) - a capture Producer/Consumer replaces `bus::Capture` and `publish_capture`, so the studio's audio sources feed the mix and their own tracks from one device open - [moq-audio routes sources into several mixes](/quest/m1/audio-mix-router.md) - several mixes with routing, mix-minus, and per-source sync offsets, for the studio's audio mixes - [moq-mux presents a broadcast on the publisher's timestamps](/quest/m1/presentation-timeline.md) - `moq play`'s audio-anchored playout timing as a moq-mux API the studio's broadcast sources share @@ -189,6 +190,17 @@ per-frame CPU growth since #5926. - [Poll-first async](/quest/m1/poll-first/README.md) - library crates compile no coroutines by default: async wrappers are generated from `poll_*` and gated behind an `async` feature - [Authored tracks default to a 30 s max-age](/quest/m1/max-age-defaults.md) - raw and media tracks advertise 30 s unless their author chooses otherwise, with explicit unlimited in every binding - [HLS export lists a same-codec reconfigure's first group under the new init](/quest/m1/hls-reinit-same-codec.md) - the hand-over pick, whichever of the catalog and the records arrives first, after the catalog `hls` removal is planned +- [A cancelled MoqTrackDemand wait never loses the next change](/quest/m1/demand-cancel-race.md) - cancelling a `MoqTrackDemand::wanted_changed` call, from any binding, never consumes a demand change +- [moq-ffi reports an unsupported media configuration as a typed error](/quest/m1/ffi-unsupported-config.md) - `MoqAudioConsumer::subscribe` and the matching video subscribe fail with a typed "unsupported configuration" error +- [A live keyframe interval change never outruns the declared HLS duration](/quest/m1/js-hls-keyframe-duration.md) - raising a live keyframe interval, even to infinite, never pushes a segment past the declared HLS target duration, and the browser interop cell asserts the catalog `hls` field +- [A request stream that opens after its deadline is aborted once and never sends](/quest/m1/js-request-deadline.md) - A `@moq/net` IETF request whose stream finishes opening only after its deadline fired is aborted exactly once, through its `RequestStream`, and never writes its request +- [One Ed25519 public key type with one z-base-32 codec](/quest/m1/mesh-key-codec.md) - `moq_auth::mesh::EndpointId` is merged into `moq_sign::PublicKey` +- [moqsink reports track demand only once the track is in the catalog](/quest/m1/moqsink-demand-catalog.md) - A `moqsink` request pad's `track-demand` stays null until its track is listed in the published catalog, and only then reports `wanted` +- [A discontinuity before the first frame keeps the init's parameter sets](/quest/m1/mux-discontinuity-headers.md) - an avc3, hev1, or av01 track that marks a discontinuity before its first frame still publishes that first keyframe with its init's parameter sets +- [Ordered reads can skip owed holes, and snapshot latest() does](/quest/m1/ordered-owed-holes.md) - moq-net's `Ordered` gains a read that skips past an owed hole to the newest buffered group, while keeping the reader's bounds +- [A Rust relay never replays from group 0 to a lite-03/04 subscriber](/quest/m1/relay-inflight-replay.md) - A Rust relay or publisher serving a lite-03 or lite-04 SUBSCRIBE from a track copy that is still resolving starts at the newest group once the copy resolves +- [The Python wrapper publish can be retried by hand](/quest/m1/release-py-dispatch.md) - `.github/workflows/release-py.yml` has a `workflow_dispatch` trigger, so a maintainer can retry a failed `moq-rs` publish from `release` without a new `moq-ffi-v*` tag or a path-matching push +- [A returning route serves past the cap its old copy was given](/quest/m1/returning-route-cap.md) - when a resumed reader's route returns, the reused serving copy asks with the reader's current resolution, not the capped resolution it was left with when it was replaced - [The resume model passes its long run again](/quest/m1/resume-model-7542.md) - fix #7542's regression that drops groups a long-budget reader can still get - [A resume more than one group behind never hangs](/quest/m1/ietf-resume-behind.md) - a resumed read gets every group or a drop on every wire version - [moq-quic's idle connections keep no burst-sized state](/quest/m1/quic-readable-dedupe.md) - readable events are deduplicated and a quiet connection keeps no per-peak-stream bytes diff --git a/quest/m1/audio-across-stop.md b/quest/m1/audio-across-stop.md new file mode 100644 index 00000000000..811c7e64155 --- /dev/null +++ b/quest/m1/audio-across-stop.md @@ -0,0 +1,38 @@ +# [S] Buffered audio survives a capture stop and a disable + +## Goal + +Audio retained by default (buffered) capture is still served, or expires on +its own `max_age`, after capture stops: in Rust, across `Control::stop` and +parking after a terminal input failure; in JS, across disabling the input. A +subscriber who arrives while capture is stopped gets the retained history it +would have got a moment earlier. Stopping never encodes the retained history +eagerly. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7238](https://github.com/moq-dev/moq/pull/7238). It was planned +as the last child of the always-on encoding epic, which +[moq-dev/moq#7554](https://github.com/moq-dev/moq/pull/7554) finished and +deleted on `main`, so it now stands alone in m1. + +Milestone: ✅ m1. Not chosen: m2. + +- [moq-dev/moq#7238](https://github.com/moq-dev/moq/pull/7238)'s overnight decision (2026-10-31, on the maintainer's standing + authorization): "retained audio when capture stops: accept for now, follow + up in a new quest" (option 1). Rejected there: flushing retained history on + stop (encode CPU on every mute), and keeping demand and expiry running + while stopped inside [moq-dev/moq#7238](https://github.com/moq-dev/moq/pull/7238) (the largest change, especially in JS). +- Today, in Rust, `Control::stop` or parking stops demand and expiry, so the + retained PCM is unreachable; in JS, disabling discards the buffer. Keep the + demand watch and expiry alive while stopped, and encode on demand from the + retained buffer as when capturing. +- JS overlaps [push-to-talk](/quest/m1/push-to-talk/README.md)'s plan to keep + the codec setup alive across a disable; share that mechanism rather than + adding a second one. +- The CLI's `buffered` audio default belongs to + [moq-dev/moq#7261](https://github.com/moq-dev/moq/pull/7261) (merged), not this quest. + +Public API: none expected. Wire: none. diff --git a/quest/m1/c/README.md b/quest/m1/c/README.md index 7e1845fc16c..d3c9cc83661 100644 --- a/quest/m1/c/README.md +++ b/quest/m1/c/README.md @@ -30,3 +30,4 @@ Public API: moq-c gains each m1 binding feature. Wire: none. ## Required - [API gaps](/quest/m1/c/api-gaps.md) - disabling pre-encoded simulcast rungs after the shared importer support lands +- [moq-c's media consume applies the catalog warmup](/quest/m1/c/raw-consume-warmup.md) - `moq_consume_audio` and `moq_consume_video` start where the moq-ffi and Rust media consumers do diff --git a/quest/m1/c/raw-consume-warmup.md b/quest/m1/c/raw-consume-warmup.md new file mode 100644 index 00000000000..97504319416 --- /dev/null +++ b/quest/m1/c/raw-consume-warmup.md @@ -0,0 +1,36 @@ +# [XS] moq-c's media consume applies the catalog warmup + +## Goal + +`moq_consume_audio` and `moq_consume_video` start where the moq-ffi and Rust +media consumers do: the catalog's `warmup` widens the start, and the +subscription budget grows by the same warmup, so a warmup rendition's decoder +gets the history it needs. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7311](https://github.com/moq-dev/moq/pull/7311) and +[moq-dev/moq#7029](https://github.com/moq-dev/moq/pull/7029). + +Milestone: ✅ m1. Not chosen: m2. + +- Warmup: `rs/moq-c/src/consume.rs:493` (`media`, at be39188000) subscribes + with `with_delay(delay)` (`:531`) and builds the container consumer with + only `with_delay(delay)` (`:534`), so the catalog's `warmup` never + reaches it. Add `moq_mux::container::Consumer::with_warmup` from the + rendition's catalog `warmup`, and add the same warmup to the + subscription's max delay, as `moq_audio::decode::Consumer` does + (`rs/moq-audio/src/decode/consumer.rs:193-206` and `:243-246`) and + `moq_video::decode::Consumer` likewise. + [moq-dev/moq#7311](https://github.com/moq-dev/moq/pull/7311) (merged) + replaced `with_max_delay` with a `Delay { min, max }`, which moq-c + already mirrors as `moq_delay` (`rs/moq-c/src/api.rs:717`). +- Update `moq.h`, `doc/lib/c`, and the C tests. + +The pre-encoded rung disable this quest also carried +([moq-dev/moq#7029](https://github.com/moq-dev/moq/pull/7029)'s decision 4) +is planned in [moq-c disables pre-encoded simulcast rungs](/quest/m1/c/api-gaps.md). + +Public API: moq-c gains the warmup on media consume (behavior). Wire: none. diff --git a/quest/m1/cut-audit/README.md b/quest/m1/cut-audit/README.md index 38c0fd89001..7441dd98a4e 100644 --- a/quest/m1/cut-audit/README.md +++ b/quest/m1/cut-audit/README.md @@ -53,6 +53,7 @@ leaving JS-only redesigns of `js/net` to [rs2ts](/quest/m2/rs2ts-js.md). ## Required +- [Each export-clock rule kept by the jitter audit has its own test](/quest/m1/cut-audit/jitter-rule-tests.md) - `stale`, `unheard_lag`, and the horizon shift in `rs/moq-mux/src/jitter.rs` each have a test that fails when that rule alone is removed - [Text and data sections keep a prototype-named ID](/quest/m1/cut-audit/catalog-proto-ids.md) - text and data entries keep the ID JS's `z.record` drops ## Related diff --git a/quest/m1/cut-audit/jitter-rule-tests.md b/quest/m1/cut-audit/jitter-rule-tests.md new file mode 100644 index 00000000000..51c1f12b026 --- /dev/null +++ b/quest/m1/cut-audit/jitter-rule-tests.md @@ -0,0 +1,38 @@ +# [S] Each export-clock rule kept by the jitter audit has its own test + +## Goal + +`stale`, `unheard_lag`, and the horizon shift in `rs/moq-mux/src/jitter.rs` +each have a test that fails when that rule alone is removed. Today the +`moq-mux` suite passes with any one of them knocked out. No rule's behavior +changes. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7412](https://github.com/moq-dev/moq/pull/7412). + +Milestone: ✅ m1. Not chosen: m2. + +[moq-dev/moq#7412](https://github.com/moq-dev/moq/pull/7412)'s knockout measurement kept these rules because each guards behavior +admission does not provide, but none has a test of its own: + +- `stale` (`judge`, `queue` at `jitter.rs:992`, `:1025`, at be39188000): + drops a frame that arrived before its deadline but was read after it. + Test that such a frame never goes out past its deadline. +- `unheard_lag` (`jitter.rs:955`, `:1291`) and the horizon shift + (`jitter.rs:970`, `:1041`): both keep a new generation behind a track sent + later than the anchor, and in the tests each covers for the other + (removing both fails `a_track_crossing_after_the_new_clock_keeps_its_lead` + and one more). Find an input where only one applies for each, such as a + track never heard during acquisition versus one heard but queued past the + horizon. + +Rerun the knockouts the way [moq-dev/moq#7412](https://github.com/moq-dev/moq/pull/7412) did (one lib run per rule removed) and +record in the PR which new test fails for each. If a rule turns out to have +no case only it covers, stop and report rather than delete it. + +[moq-dev/moq#7412](https://github.com/moq-dev/moq/pull/7412) (merged) removed the `leapt` arm from the same file. + +Public API: none (`jitter` is crate-private). Wire: none. diff --git a/quest/m1/demand-cancel-race.md b/quest/m1/demand-cancel-race.md new file mode 100644 index 00000000000..663a4fe958d --- /dev/null +++ b/quest/m1/demand-cancel-race.md @@ -0,0 +1,31 @@ +# [S] A cancelled MoqTrackDemand wait never loses the next change + +## Goal + +Cancelling a `MoqTrackDemand::wanted_changed` call, from any binding, never +consumes a demand change: the next call returns that change instead of +waiting for the one after it. `MoqFill::wanted_changed` already behaves this +way. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7036](https://github.com/moq-dev/moq/pull/7036). + +Milestone: ✅ m1 (moved from m2). Not chosen: m2. + +- `MoqTrackDemand::wanted_changed` (`rs/moq-ffi/src/demand.rs:45`) runs the + watch through `crate::ffi::detached`, a spawned task. A call cancelled + before the binding frees it leaves that task running; it advances the + shared watcher and its result is dropped, so the change is lost. +- `MoqFill::wanted_changed` (`rs/moq-ffi/src/producer.rs:511`, at + be39188000) fixed the same race by driving the watch in place through + `crate::ffi::Task::run` ("a cancelled call left unfreed must not advance + the watcher"). Use the same + shape for `MoqTrackDemand`, and check the other `detached` watchers in + moq-ffi and moq-c's demand callbacks for the same pattern. +- Test: start a wait, cancel it, change demand, and assert the next call + returns the change at once. + +Public API: none. Wire: none. diff --git a/quest/m1/drop-set.md b/quest/m1/drop-set.md index df29ff5092a..a04d02993c1 100644 --- a/quest/m1/drop-set.md +++ b/quest/m1/drop-set.md @@ -303,6 +303,15 @@ a cancelled fetch below a track's end still holds the end open until the cache expiry scan removes it, so how fast a subscription ends depends on expiry timing. Settle it here, where cancelled and expired runs get a reason. +### Untimed fills below a resolved start + +Noted 2026-10-31 from [moq-dev/moq#7235](https://github.com/moq-dev/moq/pull/7235), +which made everything below a resolved SUBSCRIBE_START fillable from upstream: +a lite-07 copy with no media time asks a fill below the start even for a reader +whose budget the publisher already exceeded, and that fill ends empty. If the +extra FETCH shows up in relay traffic, bound it here, where a run below the +start can carry the budget it was resolved under. + ## Related - [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - fills the cache miss an evicted run leaves diff --git a/quest/m1/ffi-unsupported-config.md b/quest/m1/ffi-unsupported-config.md new file mode 100644 index 00000000000..a84c9998a07 --- /dev/null +++ b/quest/m1/ffi-unsupported-config.md @@ -0,0 +1,33 @@ +# [S] moq-ffi reports an unsupported media configuration as a typed error + +## Goal + +`MoqAudioConsumer::subscribe` and the matching video subscribe fail with a +typed "unsupported configuration" error (a distinct `MoqError` variant, not a +reason string) when moq-audio or moq-video refuses a rendition's codec, +layout, or description. The OBS source falls through to the next ranked +rendition on that error and drops its copy of moq-audio's codec checks. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7331](https://github.com/moq-dev/moq/pull/7331). + +Milestone: ✅ m1 (moved from m2). Not chosen: m2. + +- [moq-dev/moq#7331](https://github.com/moq-dev/moq/pull/7331) (merged) gates audio in OBS with `moq_source_audio_decodes` (opus, pcm, mono + or stereo AAC-LC), mirroring moq-audio's rules, because moq-ffi reports a + setup refusal only as an async `MoqError::Audio { reason }` + (`rs/moq-ffi/src/error.rs:235`; video at `:243`). A configuration moq-audio + refuses under a supported codec string, such as an AAC description that + declares SBR, still discards the candidate without trying the next. +- Add the typed variant, raised from `subscribe` (`rs/moq-ffi/src/audio.rs:645`, + `video.rs:1293` at be39188000) wherever moq-audio or moq-video refuses the + config before decoding starts. Mirror it in moq-c and the wrappers. +- In OBS, fall through on the typed error and delete the mirrored gate and + its tests. +- Not in scope: falling through after a later async failure (subscribe + rejected by the peer, first decode fails), which [moq-dev/moq#7331](https://github.com/moq-dev/moq/pull/7331) also listed. + +Public API: adds a `MoqError` variant (bindings and moq-c). Wire: none. diff --git a/quest/m1/js-hls-keyframe-duration.md b/quest/m1/js-hls-keyframe-duration.md new file mode 100644 index 00000000000..6eda922f040 --- /dev/null +++ b/quest/m1/js-hls-keyframe-duration.md @@ -0,0 +1,37 @@ +# [S] A live keyframe interval change never outruns the declared HLS duration + +## Goal + +A browser publisher (`@moq/publish`) or a moq-video encoder whose keyframe +interval is raised on a live track, including to infinite, never produces +segments longer than the HLS target duration its catalog declared when the +track was created. The browser publisher's interop cell asserts the +catalog's `hls` field. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7367](https://github.com/moq-dev/moq/pull/7367). + +Milestone: ✅ m1 (moved from m2). Not chosen: m2. + +- The declaration is fixed at track creation: JS in the `hlsHint` that + [moq-dev/moq#7367](https://github.com/moq-dev/moq/pull/7367) (merged) added, read only when a track is created + (`js/publish/src/video/encoder.ts:356`), Rust in `publish_capture` + (`rs/moq-video/src/encode/producer.rs:430`, `with_hls_hint(gop)` from the + probe's GOP; lines at be39188000), while the live GOP follows later config + (`encoder.ts:534`). A raised interval therefore pushes segments past the + declared duration. +- Recommended: keep a keyframe at least every declared duration for the + track's life, by capping the live interval at it. This matches + [time-addressing consumers](/quest/m1/time-addressing/consumers.md), whose + `media.join.interval` is fixed for the broadcast instance. Re-declaring + the duration mid-track is the alternative; ask the maintainer before taking + it. +- If consumers.md lands first, the `hls` field becomes `media.join.interval`; + apply the same cap to it and assert that field in the interop cell instead. +- Interop: the browser publisher's cell checks only `@moq/hang` under Node + and Bun today. Assert the catalog's `hls` field from `@moq/publish` too. + +Public API: none (behavior: a raised keyframe interval is capped). Wire: none. diff --git a/quest/m1/js-request-deadline.md b/quest/m1/js-request-deadline.md new file mode 100644 index 00000000000..8c52aff30e0 --- /dev/null +++ b/quest/m1/js-request-deadline.md @@ -0,0 +1,29 @@ +# [XS] A request stream that opens after its deadline is aborted once and never sends + +## Goal + +A `@moq/net` IETF request whose stream finishes opening only after its +deadline fired is aborted exactly once, through its `RequestStream`, and +never writes its request. A test proves it. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7355](https://github.com/moq-dev/moq/pull/7355). + +Milestone: ✅ m1 (moved from m2). Not chosen: m2. + +- [moq-dev/moq#7355](https://github.com/moq-dev/moq/pull/7355) (merged) routes a deadline failure through the owning `RequestStream` + (`js/net/src/ietf/request_stream.ts`, `js/net/src/util/deadline.ts`), so a + timed-out request is ended once. Its online review suggested one more test, + left as this follow-up: resolve the opening stream only after the deadline + expires, then assert the owner aborts it exactly once and the request bytes + are never written. +- Add it beside `request_stream.test.ts`'s "a request's deadline ends it + through the request" case. Fix the code if it fails. +- The deleted quest of the same name (one 10 s deadline from create to + answer) planned this reset; this quest only proves it through the new + owner. + +Public API: none. Wire: none. diff --git a/quest/m1/mesh-key-codec.md b/quest/m1/mesh-key-codec.md new file mode 100644 index 00000000000..f533e46b06e --- /dev/null +++ b/quest/m1/mesh-key-codec.md @@ -0,0 +1,48 @@ +# [S] One Ed25519 public key type with one z-base-32 codec + +## Goal + +`moq_auth::mesh::EndpointId` is merged into `moq_sign::PublicKey`: one Rust +type for a mesh endpoint's Ed25519 public key, with one z-base-32 text codec +(52 lowercase characters, zero trailing bits, no other text form) that +moq-auth and moq-sign share. JS has one codec too, used by `@moq/sign` and +by `@moq/auth` if it grows one. The text form, vectors, and refusals stay as +both PRs specified. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7301](https://github.com/moq-dev/moq/pull/7301) and +[moq-dev/moq#7092](https://github.com/moq-dev/moq/pull/7092). + +Milestone: ✅ m1 (moved from m2). Not chosen: m2. There is no m1 mesh epic, +so this sits at the m1 top level; the [mesh epic](/quest/m2/mesh/README.md) +is related. + +- [moq-dev/moq#7301](https://github.com/moq-dev/moq/pull/7301) (merged) added `moq_auth::mesh::EndpointId` (an Ed25519 key with z-base-32 + text, via `data-encoding` with a custom `Specification`). [moq-dev/moq#7092](https://github.com/moq-dev/moq/pull/7092) (merged) added + `rs/moq-sign` with `SecretKey`/`PublicKey` using the same specification, + and a separate JS codec in `js/sign/src/zbase32.ts`. Both chose z-base-32 + so they would share one format; whichever landed second was to dedupe. +- Merge `EndpointId` into `moq_sign::PublicKey` (moq-auth depends on + moq-sign for it, or the type moves to a small shared crate if the + dependency direction is wrong; say which in the PR). Keep the shared + vectors in `drafts/moq-sign-00.json` as the single source of test vectors. +- Both copies are still on `main` at be39188000: + `rs/moq-auth/src/mesh/mod.rs:52` (the specification) and `:65` + (`EndpointId`), beside `rs/moq-sign/src/key.rs:18`. +- [moq-crypto replaces moq-sign](/quest/m1/moq-crypto.md) renames + `moq-sign`/`@moq/sign` to `moq-crypto`/`@moq/crypto`. If it lands first, + merge `EndpointId` into its public key type and use its JS codec instead. +- Note, not in scope: persist a browser JS `SecretKey` (a nonextractable + `CryptoKey`) in IndexedDB only once a browser mesh node needs a stable + endpoint ID. + +Public API: breaking for moq-auth (`mesh::EndpointId` becomes +`moq_sign::PublicKey` or a re-export). Wire: none. + +## Related + +- [Mesh](/quest/m2/mesh/README.md) - the epic both types serve +- [moq-crypto replaces moq-sign](/quest/m1/moq-crypto.md) - renames the crate this quest merges into diff --git a/quest/m1/moqsink-demand-catalog.md b/quest/m1/moqsink-demand-catalog.md new file mode 100644 index 00000000000..f023fc4cb2b --- /dev/null +++ b/quest/m1/moqsink-demand-catalog.md @@ -0,0 +1,36 @@ +# [M] moqsink reports track demand only once the track is in the catalog + +## Goal + +A `moqsink` request pad's `track-demand` stays null until its track is listed +in the published catalog, and only then reports `wanted`. An application that +gates its encoder on `track-demand` can no longer stop on the initial +`wanted=false` before the first catalog and hide its own rendition, so the +startup trap documented today is gone. The same holds for a `/` +replacement. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7252](https://github.com/moq-dev/moq/pull/7252) (merged as +[38566f5a84](https://github.com/moq-dev/moq/commit/38566f5a84)). + +Milestone: ✅ m1 (moved from m2). Not chosen: m2. + +- [moq-dev/moq#7252](https://github.com/moq-dev/moq/pull/7252) reports demand once the pad negotiates + (`rs/moq-gst/src/sink/request_pad.rs:474` returns the `Bandwidth` + snapshot whenever one exists). But the first catalog needs a timed buffer + and any in-band codec config, so an encoder gated on the initial + `wanted=false` keeps the rendition out of the catalog and demand never + arrives. `doc/bin/gstreamer.md:66` warns about it instead. +- Report null until the catalog that lists the track is published, then + notify `track-demand`. Clear it again on the existing terminal events + (individual EOS, pad failure, session failure), as today. +- Rewrite the doc paragraph to the new contract and drop the warning. Update + the element tests (`rs/moq-gst/src/sink/imp.rs` around `:1681`, + `request_pad.rs` around `:616`) to assert null before the catalog and a + value after. + +Public API: breaking for GStreamer users: `track-demand` stays null longer +(until the track is in the catalog). Wire: none. diff --git a/quest/m1/mux-discontinuity-headers.md b/quest/m1/mux-discontinuity-headers.md new file mode 100644 index 00000000000..b5e22fc4653 --- /dev/null +++ b/quest/m1/mux-discontinuity-headers.md @@ -0,0 +1,45 @@ +# [S] A discontinuity before the first frame keeps the init's parameter sets + +## Goal + +An avc3, hev1, or av01 track that marks a discontinuity (or seeks) before its +first frame still publishes that first keyframe with the parameter sets its +init's extra data supplied. Any caller that drives `moq_mux` this way, through +moq-ffi, moq-c, OBS, or Rust, gets a decodable first group without repeating +the headers in band. A discontinuity after the first frame keeps today's +behavior: it drops the partial access unit and its timestamps. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7029](https://github.com/moq-dev/moq/pull/7029). + +Milestone: ✅ m1. Not chosen: m2. + +- `import::Track::discontinuity` and `seek` + (`rs/moq-mux/src/import/track.rs:458`, `:465`) call `reset_split`, which + calls each splitter's `reset`: `rs/moq-mux/src/codec/h264/split.rs:245`, + `codec/h265/split.rs:250`, `codec/av1/split.rs:162`. `reset` clears the + in-flight access unit (`current`, `tail`). Before the first frame that unit + holds only the SPS/PPS (or AV1 sequence header) fed from extra data, which + a keyframe has not adopted into the retained parameter-set cache yet. An + encoder that sends headers only as extra data, such as OBS's x264, never + repeats them, so the first group is undecodable. +- Fix it in the splitters, not in callers: on `reset`, keep header NALs + that no slice has followed, or adopt them into the retained cache, so the + next keyframe re-injects them. Drop only slice data and pending frames. +- AV1 differs: `av1::Split` keeps no parameter-set cache and re-inserts + nothing (`rs/moq-mux/src/codec/av1/split.rs:7-9`). First write the av01 + test below; if a reset before the first frame does lose the extra-data + sequence header, keep that pending sequence-header OBU across `reset` + without adding a cache. If the test passes on main, drop av01 from the + Goal in the PR. +- [moq-dev/moq#7029](https://github.com/moq-dev/moq/pull/7029) (merged) worked around it in OBS by gating the discontinuity on `Track::started` + (`cpp/obs/src/moq-output.cpp:734` at + [61cb70456d](https://github.com/moq-dev/moq/commit/61cb70456d)). Once the splitters keep the headers, that + gate can stay as a no-op guard or go; keep its regression test either way. +- Test each codec in `moq_mux`: extra-data headers, a discontinuity, then a + bare keyframe, asserting the published frame carries the parameter sets. + +Public API: none. Wire: none. diff --git a/quest/m1/ordered-owed-holes.md b/quest/m1/ordered-owed-holes.md new file mode 100644 index 00000000000..994c6a7139d --- /dev/null +++ b/quest/m1/ordered-owed-holes.md @@ -0,0 +1,42 @@ +# [M] Ordered reads can skip owed holes, and snapshot latest() does + +## Goal + +moq-net's `Ordered` gains a read that skips past an owed hole to the newest +buffered group, while keeping the reader's bounds (range, budget, delivered +set). `moq_json::snapshot::Consumer::latest()` moves onto it, so on a +received track with a nonzero or unlimited `max_delay` it returns the newest +buffered snapshot at once instead of waiting for an owed lower group. The +`poll_latest` caveat goes. `@moq/net`'s `Ordered` and `@moq/json`'s +`latest()` do the same. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7403](https://github.com/moq-dev/moq/pull/7403). + +Milestone: ✅ m1 (moved from m2). Not chosen: m2. + +- [moq-dev/moq#7403](https://github.com/moq-dev/moq/pull/7403) decision 7, taken overnight on the maintainer's standing + authorization (2026-10-31): ✅ land with the documented caveat (the + `poll_latest` rustdoc, commit + [1aad05c565](https://github.com/moq-dev/moq/commit/1aad05c565)) and plan + this follow-up. Not chosen: fix it in [moq-dev/moq#7403](https://github.com/moq-dev/moq/pull/7403), threading new public API + through `Cursor::poll_next_with`, `Subscriber::poll_group`, and the resume + layer's multi-copy peek (`rs/moq-net/src/model/resume.rs`). +- Today `latest()` drains `Ordered::poll_next_group`, which since [moq-dev/moq#7403](https://github.com/moq-dev/moq/pull/7403) parks + above an owed hole until it lands, the track says it won't, or the budget + gives up. Every in-repo snapshot reader uses the zero default and is + unaffected; a reader with a budget is not. +- The new read is that threading: skip owed holes but not the reader's range + or budget, and never hand out a group twice. The caveat sits at + `rs/moq-json/src/snapshot/consumer.rs:84` at be39188000. +- [moq-dev/moq#7609](https://github.com/moq-dev/moq/pull/7609) (merged) made + JS ordered reads wait on owed groups too, so `@moq/json`'s `latest()` + (`js/json/src/snapshot/consumer.ts:59`, draining `Ordered.tryNextGroup`) + now parks the same way. Mirror the new read in `@moq/net`'s `Ordered` + (`js/net/src/track.ts:3111`) and move `latest()` onto it. + +Public API: adds an `Ordered` read (Rust and JS); `snapshot::Consumer::latest()` +and `@moq/json`'s `latest()` stop waiting at owed holes. Wire: none. diff --git a/quest/m1/poll-first/README.md b/quest/m1/poll-first/README.md index 882587d683b..61c24bb8314 100644 --- a/quest/m1/poll-first/README.md +++ b/quest/m1/poll-first/README.md @@ -75,4 +75,5 @@ and their alternatives: ## Required - [Generated async wrappers behind a feature](/quest/m1/poll-first/async-feature.md) - each library's `async fn` wrappers are generated from its `poll_*` signatures and compile only with its `async` feature +- [Server::accept and Handshake::ok are kio machines](/quest/m1/poll-first/server-machines.md) - moq-net's `Server::accept` and `server::Handshake::ok` are `#[kio::machine]` `poll_*` methods with one-line async wrappers, like the other straight-line bodies converted in [moq-dev/moq#7284](https://github.com/moq-dev/moq/pull/7284) - [Libraries spawn nothing](/quest/m1/poll-first/no-spawn/README.md) - moq-ffi, moq-stats, and moq-relay's cluster and auth run as polled `kio::Tasks` children instead of spawned tasks diff --git a/quest/m1/poll-first/server-machines.md b/quest/m1/poll-first/server-machines.md new file mode 100644 index 00000000000..dd2ee6bc699 --- /dev/null +++ b/quest/m1/poll-first/server-machines.md @@ -0,0 +1,34 @@ +# [S] Server::accept and Handshake::ok are kio machines + +## Goal + +moq-net's `Server::accept` and `server::Handshake::ok` are `#[kio::machine]` +`poll_*` methods with one-line async wrappers, like the other straight-line +bodies converted in [moq-dev/moq#7284](https://github.com/moq-dev/moq/pull/7284). Behavior, errors, and their public async +signatures stay the same. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7284](https://github.com/moq-dev/moq/pull/7284). + +Milestone: ✅ m1. Not chosen: m2. + +- Both are straight-line in spirit (`rs/moq-net/src/server.rs:283` and + `:769` at be39188000; [moq-dev/moq#7284](https://github.com/moq-dev/moq/pull/7284) called the second + `Request::ok`), but they wait on the session handshake, which has no poll form, so + [moq-dev/moq#7284](https://github.com/moq-dev/moq/pull/7284) left them as `async fn`. + [Generated async wrappers](/quest/m1/poll-first/async-feature.md) converts + the handshake by hand; once it has, convert these two through + `#[kio::machine]`, following the conventions in `kio::machine`'s docs and + the bodies [moq-dev/moq#7284](https://github.com/moq-dev/moq/pull/7284) (merged) converted. +- If the handshake's poll form makes either body a loop rather than + straight-line, hand-write that `poll_*` instead and say so in the PR. + +Public API: adds `poll_accept`/`poll_ok` beside the existing async methods. +Wire: none. + +## Required + +- [Generated async wrappers](/quest/m1/poll-first/async-feature.md) - gives the handshake a poll form these bodies wait on diff --git a/quest/m1/relay-inflight-replay.md b/quest/m1/relay-inflight-replay.md new file mode 100644 index 00000000000..a46d7a5a1d5 --- /dev/null +++ b/quest/m1/relay-inflight-replay.md @@ -0,0 +1,33 @@ +# [S] A Rust relay never replays from group 0 to a lite-03/04 subscriber + +## Goal + +A Rust relay or publisher serving a lite-03 or lite-04 SUBSCRIBE from a track +copy that is still resolving (an in-flight upstream request) starts at the +newest group once the copy resolves, as it does for lite-05+ and as JS does +for every version. It never replays the cache from group 0. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7328](https://github.com/moq-dev/moq/pull/7328). + +Milestone: ✅ m1 (moved from m2). Not chosen: m2. There is no m1 +pipeline-requests epic, so this sits at the m1 top level. + +- `cursor_floor` (`rs/moq-net/src/lite/publisher.rs:127` at be39188000) + floors a pre-06 SUBSCRIBE at `latest`, the newest group when it arrived, + and at 0 when the copy has none yet. [moq-dev/moq#7328](https://github.com/moq-dev/moq/pull/7328) (merged) re-reads the newest group of an unresolved track + once it resolves, but only for lite-05+ (`SubscribeServe`, the + `has_track_stream()` match at `:2259`). JS no longer replays for any + version. +- Extend the re-read to lite-03/04, so a resolved copy floors them at its + newest group, never at a lower first group it reports; match what JS + does. +- Test with a lite-03 and a lite-04 subscriber on a relay whose upstream + answers after the SUBSCRIBE, with cached groups below the live edge. + [moq-dev/moq#7328](https://github.com/moq-dev/moq/pull/7328)'s interop was not rerun after its last changes; run the lite-03/04 + cells here. + +Public API: none. Wire: none. diff --git a/quest/m1/release-py-dispatch.md b/quest/m1/release-py-dispatch.md new file mode 100644 index 00000000000..c6ce45766b6 --- /dev/null +++ b/quest/m1/release-py-dispatch.md @@ -0,0 +1,45 @@ +# [XS] The Python wrapper publish can be retried by hand + +## Goal + +`.github/workflows/release-py.yml` has a `workflow_dispatch` trigger, so a +maintainer can retry a failed `moq-rs` publish from `release` without a new +`moq-ffi-v*` tag or a path-matching push. A dispatch from any ref other than +`release` never publishes. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#6634](https://github.com/moq-dev/moq/pull/6634). + +Milestone: ✅ m1 (moved from m2). Not chosen: m2. There is no m1 release +epic, so this sits at the m1 top level. + +- Today the workflow runs only on a `release` push touching `py/moq-rs/**`, + the workflow, or `sh/gh/release.sh`, or on `workflow_run` after a + `moq-ffi-v*` tag (`release-py.yml:15`). When the `moq-ffi-v0.4.20` chain + failed, the only retries were a new tag or a matching push. +- Follow `release-kt-lib.yml`, which allows `workflow_dispatch` and publishes + only when `github.ref == 'refs/heads/release'` (`release-kt-lib.yml:148` + at be39188000, as [moq-dev/moq#7431](https://github.com/moq-dev/moq/pull/7431) left it). + Keep `release-py.yml`'s existing gates (already on PyPI, moq-ffi published). + Those FFI steps (`ffi-gate`, `ffi-pypi`, `release-py.yml:98-112`) run only + on `push`, and `build.outputs.publish` (`:52`) needs them for every event + but `workflow_run`, so run them on `workflow_dispatch` too, or a dispatch + never publishes. The `build` job's own `if` (`:43`) admits only `push` and + `workflow_run`, so it needs the dispatch too. Check both states: `moq-ffi` already on PyPI publishes, + and an unpublished `moq-ffi` does not. +- A dispatch with `--ref release` runs `release`'s copy of the workflow, so + the trigger helps only once it is on `release` too. Backporting it is a + `release` push touching the workflow, which itself publishes; coordinate + with the maintainer's cut instead of pushing it. +- Keep it consistent with + [Release credentials off main](/quest/m0/ci/publish-release-only.md): a + dispatch adds no new path to the `pypi` environment from `main`. + +Public API: none. Wire: none. + +## Related + +- [Publish and adopt the breaking release](/quest/m2/release/README.md) - release publishing diff --git a/quest/m1/returning-route-cap.md b/quest/m1/returning-route-cap.md new file mode 100644 index 00000000000..d52cf30a157 --- /dev/null +++ b/quest/m1/returning-route-cap.md @@ -0,0 +1,44 @@ +# [S] A returning route serves past the cap its old copy was given + +## Goal + +When a resumed reader's route returns, the reused serving copy asks with the +reader's current resolution, not the capped resolution it was left with when +it was replaced. A range ending at a relative anchor, or at a time that has +already resolved, keeps delivering past the old cap on the returning route, +as it would on a fresh route. Rust first; JS only if the same reuse exists +there. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7364](https://github.com/moq-dev/moq/pull/7364). + +- Prompt: "The capped serving copy on a returning route (pre-existing on + main): fix now, or leave it to the resume redesign?" ✅ Own S quest, m1. + Not chosen: leave it to [resume as producer replacement](/quest/m2/resume-redesign.md), + which is blocked until after the cut. +- Fix via the reader's resolution, consistent with [moq-dev/moq#7364](https://github.com/moq-dev/moq/pull/7364)'s widened-backfill + fix: there a widened backfill asks with the reader's resolution, because + the copy's own resolution ends at its earlier cap and would pin a range + ending relative to the latest group. +- The bug: returning to a route reuses its newest copy + (`rs/moq-net/src/model/resume.rs:1192` at be39188000), clears `until`, and + calls `copy.update(&self.mirrored)`. `Copy::wire` (`resume.rs:511`) then + takes the `Sub::Ready(sub) if reader.ranges == self.from` arm and re-reads + `sub.resolved_ranges()`, which is the capped wire it was last sent. So the + reused copy stays at its old cap and delivers nothing past it. Re-resolve + the reused copy against the reader's window instead + (`sub.resolve_ranges(&reader.ranges)`), as a new backfill does. +- Test: a range ending at a relative anchor (and one at a resolved time end) + flaps away and back; the returning route delivers groups past the old cap. + Add the shrunk case to the resume model's regressions if the nightly sweep + finds one. +- [moq-dev/moq#7364](https://github.com/moq-dev/moq/pull/7364) (merged) edited the same code first. + +Public API: none. Wire: none. + +## Related + +- [Resume as producer replacement](/quest/m2/resume-redesign.md) - would remove copy reuse altogether diff --git a/quest/m1/ts-reliability/README.md b/quest/m1/ts-reliability/README.md index 28aa3e660ea..8d7c3dd64d0 100644 --- a/quest/m1/ts-reliability/README.md +++ b/quest/m1/ts-reliability/README.md @@ -100,6 +100,7 @@ catalog linger (#5305). ## Required - [A small step back keeps the TS export's video](/quest/m1/ts-reliability/step-back.md) - the export re-acquires on a sub-group step back from any source instead of dropping its video +- [TS timing checks survive a stalled PCR](/quest/m1/ts-reliability/pcr-stall.md) - the TS importer's TR 101 290 timing checks keep running while a program's PCR stalls or goes missing ## Related diff --git a/quest/m1/ts-reliability/pcr-stall.md b/quest/m1/ts-reliability/pcr-stall.md new file mode 100644 index 00000000000..eba91b509da --- /dev/null +++ b/quest/m1/ts-reliability/pcr-stall.md @@ -0,0 +1,40 @@ +# [S] TS timing checks survive a stalled PCR + +## Goal + +The TR 101 290 timing checks the TS importer reports (PAT and PMT +repetition, PTS intervals) keep running while a program's PCR stalls or goes +missing, for a single program and for programs that share a PID. An outage on +a stream with a stalled PCR is counted, not hidden, and a stalled PCR is +itself reported as an input fault. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7294](https://github.com/moq-dev/moq/pull/7294). + +Milestone: ✅ m1. Not chosen: m2. + +- [moq-dev/moq#7294](https://github.com/moq-dev/moq/pull/7294) (merged)'s overnight decision (2026-10-31, on the maintainer's standing + authorization) merged shared-PID stats as is and filed this root-cause fix: + a stalled owner's clock hides shared-PID timing errors + ([review](https://github.com/moq-dev/moq/pull/7294#pullrequestreview-5709848438)), + and a lone program with a stalled PCR already loses the same outage on main. +- Each program's `Health` (`rs/moq-mux/src/container/ts/health.rs:214` at be39188000) times + its tables, PCR, and PTS on its own PCR clock, passed in as `now` from the + importer's `Liveness`. When the PCR stops, `now` stops, and no interval + check fires. Fall back to a clock that keeps running, such as the multiplex + clock (`mux_rate`) or arrival time, while the PCR is stalled, and report the + stall through the PCR checks. Today `health.tick` runs only on the + PCR-bearing packet path and `Health::pcr` checks only when a PCR arrives, + so advance the checks from every packet on the fallback clock. A fully + stopped input has no packet to advance them: the next packet or EOF + reports that outage, with no timer. Report a missing PCR once per outage; + a returning PCR clears it so a later outage reports again. +- Test single-program and shared-PID streams with a PCR that stops while + tables and PES continue, and with all three stopping: the outage shows in + the counters either way. Add the case to the TS guide's coverage table. + +Public API: none expected (stats fields only if a new counter is needed). +Wire: none. diff --git a/quest/m1/unbounded-groups/README.md b/quest/m1/unbounded-groups/README.md index e0cc227b249..003c394c37e 100644 --- a/quest/m1/unbounded-groups/README.md +++ b/quest/m1/unbounded-groups/README.md @@ -154,3 +154,5 @@ covered only at the model layer (maintainer, 2026-10-30). - [Byte holes](/quest/m1/unbounded-groups/byte-holes.md) - segments inside a frame shed like front frames and flights start at `(frame, byte)` on the current lite wip; the frame cap goes - [JS byte holes](/quest/m1/unbounded-groups/js-byte-holes.md) - `@moq/net` mirrors the byte-offset wire and continues a frame cut off mid-way - [Log lock profile](/quest/m1/unbounded-groups/log-lock-profile.md) - a lock profile on a perf host confirms the epoch-based `kio::log` cut `GroupState` contention +- [Compressed frames have no size cap and inflate as a bounded stream](/quest/m1/unbounded-groups/compressed-stream.md) - A compressed frame of any size publishes and is received in Rust and `@moq/net`, like a plain one +- [FETCH egress and ingest stream large frames without buffering](/quest/m1/unbounded-groups/fetch-stream.md) - `@moq/net` serves a moq-transport FETCH object by object as it reads them, never collecting a requested group slice whole, so a FETCH of a group with large frames holds no more than the group tail diff --git a/quest/m1/unbounded-groups/compressed-stream.md b/quest/m1/unbounded-groups/compressed-stream.md new file mode 100644 index 00000000000..a2cde9929e1 --- /dev/null +++ b/quest/m1/unbounded-groups/compressed-stream.md @@ -0,0 +1,58 @@ +# [M] Compressed frames have no size cap and inflate as a bounded stream + +## Goal + +A compressed frame of any size publishes and is received in Rust and +`@moq/net`, like a plain one. The 32 MiB guard on a compressed frame goes. +The wire still declares each compressed slice's size (and its `Inflated +Size`) up front, so a publisher still compresses a frame whole before it +sends it. A receiver never inflates a frame whole in memory: it inflates as +a stream into the frame's segments, which shed under the group's 32 MB tail +like any frame. Inflating past the declared `Inflated Size` fails the frame +as malformed the moment the output passes it, so a zip bomb costs at most one +segment beyond what was declared. A compressed frame near any size no longer +encodes into a slice that receivers refuse. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7411](https://github.com/moq-dev/moq/pull/7411). + +- [moq-dev/moq#7411](https://github.com/moq-dev/moq/pull/7411) kept a 32 MiB whole-frame guard on compressed frames in JS, reasoning + that lifting it needs a wire change. The user: "remove the 32MB guard, make + sure we stream any decompression to avoid zip bombs." And: "we still need + compressed size on the wire, so we don't need to uncompress to learn it." + So no wire change: keep the declared compressed size, drop only the guard, + and bound decompression while streaming. +- Milestone: ✅ m1. Not chosen: m2. +- JS ([moq-dev/moq#7411](https://github.com/moq-dev/moq/pull/7411) merged): drop the guard and the `Encoder.bound` preflight in + `js/net/src/group.ts` (`writeFrame`, `createFrame`, binding compression) + and JSON's mirrored `MAX_FRAME_BYTES`. Inflate a received slice in chunks + into the chunked frame storage [moq-dev/moq#7411](https://github.com/moq-dev/moq/pull/7411) added, checking output against the + declared `Inflated Size`. +- Rust: the guard is `rs/moq-net/src/model/group/content.rs:117` (`size` and + `inflated` against `MAX_CACHE_BYTES`). The inflated copy inflates each + frame whole (`Inflate::inflate`, `rs/moq-net/src/model/group.rs:632` at be39188000). + Inflate into `frame::SEGMENT` segments instead, bounded the same way. +- This also resolves [moq-dev/moq#7411](https://github.com/moq-dev/moq/pull/7411)'s Rust follow-up: `GroupProducer::write_frame` and + `write_frames` (`group.rs:1719`, `:1798`) check only the plain payload + against 32 MB, so an incompressible frame near the cap encodes to a slice + every receiver refuses. With no compressed cap, there is nothing to + mismatch; [Byte holes](/quest/m1/unbounded-groups/byte-holes.md) removes + the plain cap. +- The lite draft's FRAME text ("a receiver MUST check it against its limits + before inflating") stays; add that a receiver may inflate incrementally and + must stop at the declared size. Update `doc/concept` and the compression + docs that state the 32 MiB cap. +- Tests, both languages: a compressed frame several times 32 MB round-trips + over lite-07 and a relay chain with memory held near the tail; a slice + whose payload inflates past its declared size fails as malformed without + allocating the excess. + +Public API: removes the compressed-frame `FrameTooLarge` refusal (JS, Rust, +JSON preflight). Wire: none. + +## Required + +- [Byte holes](/quest/m1/unbounded-groups/byte-holes.md) - Rust frames past 32 MB at all diff --git a/quest/m1/unbounded-groups/fetch-stream.md b/quest/m1/unbounded-groups/fetch-stream.md new file mode 100644 index 00000000000..3a0bfe47c8c --- /dev/null +++ b/quest/m1/unbounded-groups/fetch-stream.md @@ -0,0 +1,35 @@ +# [M] FETCH egress and ingest stream large frames without buffering + +## Goal + +`@moq/net` serves a moq-transport FETCH object by object as it reads them, +never collecting a requested group slice whole, so a FETCH of a group with +large frames holds no more than the group tail. A lite FETCH whose readers +all leave in the middle of a frame stops reading that frame's payload at +once. The Chromium group-tail memory measurement and the frame benches are +rerun on a quiet host after [moq-dev/moq#7411](https://github.com/moq-dev/moq/pull/7411) and after this quest. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7411](https://github.com/moq-dev/moq/pull/7411). + +- moq-transport FETCH egress: the user: "absolutely don't buffer." + `readFetched` (`js/net/src/ietf/publisher.ts:2760` at be39188000, called at + `:2408` and `:2521`) collects each requested group slice before writing it. Stream it + the way [moq-dev/moq#7411](https://github.com/moq-dev/moq/pull/7411)'s subgroup egress (the `serveGroup` closure, `:811`) does, with `nextFrame`-style + reads and chunked writes for a payload past one segment. +- lite FETCH whose readers all leave mid-frame: the user: "fix that I + think." Today it keeps reading the frame's payload before it notices. Watch + the reader count while streaming a frame and stop the stream (reset or + `STOP_SENDING`) as soon as none is left. +- Benches: [moq-dev/moq#7411](https://github.com/moq-dev/moq/pull/7411)'s small-frame hot-path results were inconclusive (paired + head/base medians 0.91 to 1.11 at load 24 to 29), and the Chromium + group-tail memory numbers predate chunked storage. Rerun both on a quiet + host (`js/net/bench/frames.ts` and the Chromium group-tail measurement) + against the pre-[moq-dev/moq#7411](https://github.com/moq-dev/moq/pull/7411) base and record them in the PR. A small-frame + regression found here is fixed here. +- Milestone: ✅ m1. Not chosen: m2. + +Public API: none. Wire: none. diff --git a/quest/m2/README.md b/quest/m2/README.md index a8438125b8c..4c04ff593e6 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -220,7 +220,6 @@ lite-07; bitrate enforcement is not part of this release. - [Audio track changes are gapless](/quest/m2/gapless-audio-switch.md) - no silence beyond a frame on audio format or rendition changes, enforced by media QA - [PCM encodes in one pass](/quest/m2/pcm-bulk-encode.md) - the PCM encoder converts a frame in bulk instead of one sample at a time - [We own our WebRTC stack](/quest/m2/webrtc/README.md) - moq-rtc becomes moq-webrtc with moq-ice, the data-channel transport, and a vendored, trimmed str0m/dimpl/is/sctp-proto -- [Certificate reload and qmux's tasks are supervised](/quest/m2/tls-reload-supervision.md) - a panic in either fails loudly instead of silently stopping - [PUBLISH_DONE says why](/quest/m2/publish-done-codes.md) - GOING_AWAY, TOO_FAR_BEHIND, and EXPIRED round-trip as stream codes - [moq repair refuses a live recording](/quest/m2/mp4-repair-lock.md) - the recorder's advisory lock keeps `moq repair mp4` off a file still being written - [The error-queue reader decodes truncated control data](/quest/m2/errqueue-truncation.md) - `MSG_ERRQUEUE` keeps what fit on `MSG_CTRUNC`, like the receive paths @@ -230,9 +229,41 @@ lite-07; bitrate enforcement is not part of this release. - [An ended HLS playlist has no trailing gap](/quest/m2/hls-trailing-gap.md) - no zero-duration `EXT-X-GAP` before `EXT-X-ENDLIST` - [moq-video's Apple Sink encodes on its own thread](/quest/m2/apple-sink-thread.md) - macOS and iOS ladders never encode on a tokio worker - [/fetch answers from the announce sync](/quest/m2/fetch-synced.md) - a missing broadcast gets a 404 once announcements are synced, not after 30 s +- [An advertisement stuck on a zero-credit peer gives up](/quest/m2/advertise-credit-timeout.md) - A Rust moq-transport publisher whose PUBLISH_NAMESPACE or namespace update cannot leave because the peer grants no send credit gives up on it within a bounded time, as it already does on a silent answer: it resets the request and frees its state +- [aec::Processor's edge cases are tested](/quest/m2/aec-tests.md) - moq-audio tests cover `aec::Processor` at the extremes of its accepted input +- [A failed archive time lookup fails the subscription](/quest/m2/archive-time-followups.md) - when an `archive::Consumer` cannot answer a `Time(pts)` lookup because a store GET fails or an object does not decode, the subscription fails with a typed error naming the lookup failure instead of silently falling back to the cache +- [moq-sock's control buffer is sized for the send options too](/quest/m2/cmsg-len-send.md) - `cmsg::LEN` in moq-sock is derived from, or checked at compile time against, both the receive options and the send options +- [A panic inside moq-tokio's connection internals fails callers instead of panicking them](/quest/m2/connection-panic-isolation.md) - after a moq-tokio QUIC connection driver panics, every client-side caller gets the connection's terminal error instead of panicking in its own task, and the certificate reload watcher and qmux driver tasks are supervised +- [No large future in the FFI layers can overflow a runtime or foreign thread's stack](/quest/m2/ffi-future-size.md) - every future moq-c and moq-ffi spawn or block on is below a size bound that a lint enforces, so unoptimized builds stay inside the runtime thread stack +- [A browser test covers GPU encoder death and a broadcast rename](/quest/m2/gpu-encoder-death-test.md) - A real-browser test makes `@moq/publish`'s video encoder fail mid-stream +- [An empty tail stream for a headless moq-transport group creates no group](/quest/m2/headless-empty-tail.md) - when a moq-transport subscription's fill for the Largest Group fails, an empty subgroup stream for that headless group is dropped instead of creating an empty group that reads as complete +- [A recording stalled on an open group reports what it waits for](/quest/m2/hls-stalled-segment.md) - when a moq-hls recording waits on a listed segment whose group a still connected publisher never closes, the wait is visible +- [A kept draft-18 join pins its head group](/quest/m2/ietf-join-pin-head.md) - on draft 18, a joining FETCH that our publisher keeps open after a quick finish is answered even if the publisher drops its `track::Producer` right after `finish()` +- [The JS player keeps a track's start when decoding from its first packet](/quest/m2/js-warmup-trim.md) - `@moq/watch` trims an audio track's catalog `warmup` only on a mid-stream join, as moq-audio does +- [kio::Tasks tolerates a removed task's waker firing into its reused slot](/quest/m2/kio-tasks-waker-test.md) - A unit test in `rs/kio/src/task.rs` shows that when a task taken out with `Tasks::remove` wakes after another task was `insert`ed into its slot, the new occupant is polled at most once spuriously, still finishes on its own wake (handed back through `poll_each`), and its `TaskKey` still removes it +- [Nightly's Miri job stays inside its time budget](/quest/m2/miri-nightly-budget.md) - Nightly's Miri job stays inside its 60-minute job and 15-minute-per-test budgets +- [moqsrc reconnects correctly whatever order events arrive in](/quest/m2/moqsrc-reconnect-order.md) - A moq-gst test reconnects `moqsrc` while a rendition pad is blocked downstream, delivering the reconnect's events +- [A library recorder gets the takeover cut](/quest/m2/mux-source-cut.md) - A recorder built directly with `moq_hls::export::Broadcaster::new` ends a replaced instance's recording at what it listed when another publisher takes the broadcast over, exactly as one built through `Server::broadcaster` does +- [The relay NixOS module's peer list is tested](/quest/m2/nix-relay-peers-test.md) - `nix flake check` evaluates the moq-relay NixOS module with no, one, and several `cluster.rootUrl` peers and refuses an empty list, asserting the `MOQ_CLUSTER_CONNECT` value each produces +- [The OBS source joins audio at the live edge](/quest/m2/obs-decode-latest.md) - the OBS MoQ source's audio decoder starts at the newest cached group instead of replaying the track's backlog, so a reconnect or rendition change plays live audio at once +- [OBS simulcast layers reach moq-watch from a live OBS, including the dock's Go Live](/quest/m2/obs-multitrack-check.md) - A live OBS 32.2.2 run, recorded in the PR, shows the `WHIPSimulcastTotalLayers` profile setting attaching two or more scaled video encoders to `moq_output`, each published as its own rendition and watched in ``, both through the obs-websocket recipe in `doc/bin/obs.md` and through the MoQ dock's Go Live, which honors the profile's layer count instead of creating a single encoder +- [Opus export never writes a channel mapping it can't stand behind](/quest/m2/opus-surround-export.md) - exporting a catalog Opus rendition with 3 to 8 channels and no description writes a `dOps` with the Vorbis family 1 mapping table, as the TS importer assumes, instead of an invalid family 0 `dOps` +- [Open-GOP MPEG-2 export trims leading B-pictures in zero-stuffed CBR streams](/quest/m2/pes-hold-open-gop.md) - A TS export starting or resuming inside an open GOP of MPEG-2 video drops the orphaned leading B-pictures even when their PES ends on zero stuffing, as CBR encoders emit, without ever trimming part of a reference picture whose start code straddles two PES packets +- [The lite publisher tells transport faults from cancels](/quest/m2/publish-cancel-classify.md) - `rs/moq-net`'s lite publisher no longer counts every `Error::Transport` as a clean cancel +- [moq-publish hides previews that show nothing](/quest/m2/publish-empty-previews.md) - `` hides a preview that renders nothing +- [moq-quic's sim pair fails a test that sends an oversized packet](/quest/m2/quic-sim-mtu.md) - A moq-quic test whose endpoint sends a datagram larger than the simulated path MTU fails, instead of the `Pair` silently dropping it +- [A stalled reader within budget gets no Old resets through a relay](/quest/m2/relay-capped-credit-test.md) - A test proves that a subscriber which stalls before its first read, behind a relay over a transport with capped stream credit, receives no `Old` stream resets for groups still within its max-delay budget, and reads them in order once it resumes +- [In-memory transports refuse a request without stream credit](/quest/m2/sim-credit-open.md) - moq-net's in-memory test transports model the peer's bidirectional stream credit, and `poll_open_request` returns `Ok(None)` +- [The tenant matrix's HTTP checks assert exact refusals](/quest/m2/tenant-leak-checks.md) - the relay tenant matrix's HTTP check asserts a definite outcome for each cross-tenant request, so it cannot pass by timing out or by an unexpected status +- [The test inventory follows every branch selection](/quest/m2/test-inventory-followups.md) - `sh/gh/test-inventory.py` judges every check by the dispatcher branch its caller actually selects, including the cargo-fuzz caller check, the recipe-runs-tests check +- [tls:// sessions report their negotiated ALPN](/quest/m2/tls-link-alpn.md) - A session accepted on a `tls://` listener reports the negotiated moq ALPN in `Link::alpn`, as raw QUIC, WebTransport, and WebSocket sessions already do, so admission and logs see the protocol on every transport +- [PUBLISH token renewal is consistent across pause and forward, and the multi-token cost is profiled](/quest/m2/token-auth-followups.md) - on a moq-transport session, a REQUEST_UPDATE carrying an AUTHORIZATION TOKEN on a PUBLISH gets the same answer whether the PUBLISH is paused or forwarding +- [moq-uring's handle lifecycle tests cover the newly spawned queue and a woken close watcher](/quest/m2/uring-lifecycle-tests.md) - two regressions cover the shutdown paths moq-dev/moq#7309 added +- [Watch tests wait for the change they expect](/quest/m2/watch-test-waits.md) - `js/watch` tests wait for the signal or state they assert on instead of a fixed number of event-loop yields, so they neither flake under load nor pass before the change happens +- [Nightly runs zizmor's online workflow audits](/quest/m2/workflow-audit-followups.md) - nightly runs zizmor's online audits - [moq auth sign mints capped tokens](/quest/m2/auth-sign-caps.md) - CLI flags for the bitrate caps - [QUIC-LB forwarding reads the cluster's peers and reports its counts](/quest/m2/quic-lb-forward-extras.md) - a cluster-API peer map and forward counters -- [An audio-only broadcast that can't play hides its play button](/quest/m2/audio-only-play.md) - center-play follows the audio-only rule +- [An audio-only broadcast that can't play hides its play button](/quest/m2/audio-only-play.md) - center-play follows the audio-only rule, proven by a browser test - [The browser bench keeps its page consoles](/quest/m2/browser-bench-consoles.md) - publisher and consumer page logs in the run directory - [moq-quic's long header type is one variant](/quest/m2/quic-long-type.md) - collapse LongType after the 0-RTT removal - [One predicate for the lite close wait](/quest/m2/subscriber-fin-fold.md) - fold waits_for_subscriber_fin into has_track_stream diff --git a/quest/m2/advertise-credit-timeout.md b/quest/m2/advertise-credit-timeout.md new file mode 100644 index 00000000000..03455ee9f22 --- /dev/null +++ b/quest/m2/advertise-credit-timeout.md @@ -0,0 +1,41 @@ +# [S] An advertisement stuck on a zero-credit peer gives up + +## Goal + +A Rust moq-transport publisher whose PUBLISH_NAMESPACE or namespace update +cannot leave because the peer grants no send credit gives up on it within a +bounded time, as it already does on a silent answer: it resets the request +and frees its state. Withdrawal still frees it at once, and a peer that +grants credit late but in time is unaffected. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7414](https://github.com/moq-dev/moq/pull/7414): + +- Bound a stuck advertisement under a zero-credit peer: + - ✅ Quest in m2. Withdrawal already frees it after [moq-dev/moq#7414](https://github.com/moq-dev/moq/pull/7414), so the + remaining cost is one buffered request per namespace, not a leak. + - Drop it. + +[moq-dev/moq#7414](https://github.com/moq-dev/moq/pull/7414) (merged) +buffers each message and starts `ADVERTISE_TIMEOUT` +(`rs/moq-net/src/ietf/publisher.rs:563` at be39188000, 5 s) only once the bytes +reach the transport (`:3749`); its `NamespacePending::Writing` state has no deadline, and the PR +notes "send backpressure alone does not start answer timeouts or retries". +Build on it: + +- Give `Writing` a deadline too, reusing `ADVERTISE_TIMEOUT` from when the + message was buffered rather than adding a second constant, and on expiry + take the same path as an unanswered request (reset both directions, end + the request). +- Test on drafts 18 and 22 with the zero-credit test peer [moq-dev/moq#7414](https://github.com/moq-dev/moq/pull/7414) adds: a + never-granting peer times out the initial advertisement and an update; a + peer granting credit before the deadline is answered normally. + +Public API: none. Wire: none. + +## Related + +- [One request lease](/quest/m2/ietf-request-lease.md) - other request lifetimes on the same streams diff --git a/quest/m2/aec-tests.md b/quest/m2/aec-tests.md new file mode 100644 index 00000000000..08cce01b113 --- /dev/null +++ b/quest/m2/aec-tests.md @@ -0,0 +1,24 @@ +# [S] aec::Processor's edge cases are tested + +## Goal + +moq-audio tests cover `aec::Processor` at the extremes of its accepted +input: 8 kHz and 384 kHz sample rates, render and capture streams with +mismatched formats (refused, or converted as documented), and a `reset` +while a partial frame is buffered, which neither panics nor leaks the +partial frame into the next output. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7369](https://github.com/moq-dev/moq/pull/7369): m2, as +recommended. [moq-dev/moq#7369](https://github.com/moq-dev/moq/pull/7369) (merged) exposes `aec::Processor`, +`aec::Stream`, and `aec::Stats` for caller-supplied PCM. + +Add the tests beside [moq-dev/moq#7369](https://github.com/moq-dev/moq/pull/7369)'s unit tests (`mod tests`, +`rs/moq-audio/src/aec/processor.rs:358` at be39188000). Fix any +bug they find in the same PR; if a rate is outside what Sonora accepts, +assert the refusal rather than widening support. + +Public API: none. Wire: none. diff --git a/quest/m2/archive-lookup-withdrawal.md b/quest/m2/archive-lookup-withdrawal.md index 60acbebd62e..7eb0a04dbf4 100644 --- a/quest/m2/archive-lookup-withdrawal.md +++ b/quest/m2/archive-lookup-withdrawal.md @@ -6,7 +6,7 @@ A `track::Fill` exposes a public signal for "no reader waits on this fill any mo ## Plan -Follow-up of [moq-dev/moq#7310](https://github.com/moq-dev/moq/pull/7310), which resolves archive time ranges through a fill lookup: a fill over `0..reach` carrying the unresolved time in `Fill::time()` and owing no group. moq-archive used `fill.unused()` to stop a slow store read once nobody waited, but [moq-dev/moq#7036](https://github.com/moq-dev/moq/pull/7036) removed it, and a lookup owes no group, so `unwanted(sequence)` can't stand in. #7310 dropped lookup cancellation by maintainer decision (2026-11-01, "no new moq-net API" in that PR), so an abandoned lookup now finishes its read and answers nobody. Group fills still stop on `unwanted(sequence)`. This waits on #7310 landing. +Follow-up of [moq-dev/moq#7310](https://github.com/moq-dev/moq/pull/7310), which resolves archive time ranges through a fill lookup: a fill over `0..reach` carrying the unresolved time in `Fill::time()` and owing no group. moq-archive used `fill.unused()` to stop a slow store read once nobody waited, but [moq-dev/moq#7036](https://github.com/moq-dev/moq/pull/7036) removed it, and a lookup owes no group, so `unwanted(sequence)` can't stand in. #7310 dropped lookup cancellation by maintainer decision (2026-11-01, "no new moq-net API" in that PR), so an abandoned lookup now finishes its read and answers nobody. Group fills still stop on `unwanted(sequence)`. Decided 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-afk run): m2, adding the public withdrawal signal #7310 deferred, since wasted store reads under churn are a real cost for slow stores. @@ -19,4 +19,4 @@ Public API: a public withdrawal signal on `moq_net::track::Fill`. Wire: none. ## Related -- [A failed archive time lookup fails the subscription](https://github.com/moq-dev/moq/pull/7440) - planned in that open PR as `quest/m2/archive-time-followups.md`: a lookup refusal on `Fill`, the other half of a lookup's lifecycle; whichever lands second shares its `Fill` surface +- [A failed archive time lookup fails the subscription](/quest/m2/archive-time-followups.md) - a lookup refusal on `Fill`, the other half of a lookup's lifecycle; whichever lands second shares its `Fill` surface diff --git a/quest/m2/archive-time-followups.md b/quest/m2/archive-time-followups.md new file mode 100644 index 00000000000..54297f65a1b --- /dev/null +++ b/quest/m2/archive-time-followups.md @@ -0,0 +1,52 @@ +# [M] A failed archive time lookup fails the subscription + +## Goal + +When an `archive::Consumer` cannot answer a `Time(pts)` lookup because a +store GET fails or an object does not decode, the subscription fails with a +typed error naming the lookup failure instead of silently falling back to +the cache. A time range on a history-served track also counts an unverified +group's start even when its handler registered after the group's first +frame, and moq-ffi plus the bindings expose `Fill::time()`. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7310](https://github.com/moq-dev/moq/pull/7310) (merged), +which added `track::Fill::time()` and the lookup and finished the archive +time ranges quest. + +- What a lookup's storage or decode failure does: + - Silent cache fallback, as [moq-dev/moq#7310](https://github.com/moq-dev/moq/pull/7310) documents with a warning. + - ✅ Refuse + report. Add a lookup refusal to `Fill` (for example + `Fill::refuse(err)` valid on a lookup fill); the reader's span fails + and the subscription ends with a typed error. No retry here: the store + client already retries. Reason: a cache fallback silently serves the + wrong range, which a DVR seek cannot tell from success. + - Retry the lookup in moq-net. + +Also, from [moq-dev/moq#7310](https://github.com/moq-dev/moq/pull/7310)'s follow-ups: + +- Unverified starts are indexed when a group's first frame is written, + using whether a history-serving handler exists then. A handler registered + later misses those groups, so index them (or re-check) when the handler + registers. +- Mirror `Fill::time()` and the new refusal in `rs/moq-ffi` + (`MoqFill`, `rs/moq-ffi/src/producer.rs:458` at be39188000) and the binding wrappers, + so a binding-side history handler can answer lookups. +- A time end walks the history one fill round trip per group. Measure a + long range against the in-memory store; if DVR seeks need it, look the + end up as well. Otherwise record the cost in the `Fill::time()` docs. + +Tests: a failing GET and a malformed object each fail the subscription with +the typed error; a handler registered after a group started still resolves +a time range over it; an FFI handler answers a lookup. + +Public API: `track::Fill` gains a lookup refusal; moq-ffi `MoqFill` gains +`time()` and the refusal. Wire: none. + +## Related + +- [Archive lookup withdrawal](/quest/m2/archive-lookup-withdrawal.md) - the other [moq-dev/moq#7310](https://github.com/moq-dev/moq/pull/7310) follow-up, a public withdrawal signal on `Fill` +- [DVR rewind](/quest/m2/archive-dvr.md) - the consumer of time seeks diff --git a/quest/m2/audio-only-play.md b/quest/m2/audio-only-play.md index 8cc5e7f67ee..edab5866b45 100644 --- a/quest/m2/audio-only-play.md +++ b/quest/m2/audio-only-play.md @@ -1,4 +1,4 @@ -# [XS] An audio-only broadcast that can't play hides its play button +# [S] An audio-only broadcast that can't play hides its play button ## Goal @@ -9,3 +9,16 @@ as it does when video has an error. The unsupported notice from ## Plan Planned 2026-11-01 (maintainer, follow-up planning after the 2026-11-01 /quest-complete run). Use #7359's rule: a broadcast with no video renditions counts as audio. + +- `js/watch/src/ui/components/center-play.ts:16` reads only + `watch.video.source.out.error`; with no video rendition, hide on the audio + source's error. +- [moq-dev/moq#7359](https://github.com/moq-dev/moq/pull/7359) ran nothing in + a browser. Add a browser test (in `test/interop/clients/js/`, beside + `harness.browser.ts`) that a codec the browser refuses reaches the indicator + through `AudioDecoder.isConfigSupported`, and that the play button hides for + audio-only. Folded in 2026-11-02 from the duplicate quest + [moq-dev/moq#7440](https://github.com/moq-dev/moq/pull/7440) planned from the + same PR (user, 2026-10-31: m2). + +Public API: none. Wire: none. diff --git a/quest/m2/audit-interop.md b/quest/m2/audit-interop.md index 7e1b5b9fb98..3fd93f7f433 100644 --- a/quest/m2/audit-interop.md +++ b/quest/m2/audit-interop.md @@ -150,6 +150,33 @@ moq-transport drafts 14 to 17 (Cloudflare's draft-14 and draft-16 relays), quiche-moq (connect idle timeout), xquic's docker relay (binds loopback), and moqtopus and xquic-draft-18 over WebTransport (raw QUIC only). +Added 2026-10-31, from the runner's 2026-10-31 nightly +[41149054952](https://github.com/englishm/moq-interop-runner/actions/runs/41149054952) +as read by [moq-dev/moq#7050](https://github.com/moq-dev/moq/pull/7050) and +[moq-dev/moq#7104](https://github.com/moq-dev/moq/pull/7104) (moq-dev-js on +`@moq/net` 0.4.7): + +- moq-dev-js fails `setup-only` in about 505 ms with an empty message in + three cells: remote QUIC against Cloudflare draft-18, remote QUIC against + aiomoqt, and aiomoqt docker, on 0.4.4 and 0.4.7 alike. WebTransport to the + same hosts passes. The two remote QUIC cells are probably the raw-QUIC + limitation under Check first; the aiomoqt docker cell is unexplained. +- The same nightly shows moq-dev-js `setup-only` failures against + libquicr, moqlivemock, moq-dev-rs, moqtail, moqx, quiche-moq, red5, + stitcher-moq, and xquic-draft-18. Classify each cell from its own log, + and group cells only when their logs show a shared cause. +- imquic: moq-dev-js `subscribe-before-announce` fails in both remote cells + (`lminiero.it`), and `publish-namespace-done` in the remote-quic cell; all + three pass in the docker cell. +- Runner image staleness: a merged moq-dev-* bump reaches the nightly only + after someone dispatches an image rebuild, because the schedule in the + runner's `rebuild-all-images.yml` is commented out. After a bump merges, + check its GHCR tag. Asking upstream to restore the schedule is outside + the standing approval, so get the maintainer's first. +- The runner's `moq-rs-draft-18-relay` and `moq-rs-draft-18-client` image + builds (Cloudflare draft-18) failed in the 2026-10-22 rebuild and both + 2026-10-28 rebuilds, so those cells run stale images. + ## Required - [Release cut](/quest/m2/release/cut.md) - waits for the release in flight to ship diff --git a/quest/m2/audit-lints.md b/quest/m2/audit-lints.md index 881e58c306a..d53ddf45833 100644 --- a/quest/m2/audit-lints.md +++ b/quest/m2/audit-lints.md @@ -3,8 +3,9 @@ ## Goal Every lint suppression in first-party code, in every language, is still -needed and says why, or is removed. Nothing calls a deprecated upstream API, -and our own deprecated items are removed or planned for removal once their +needed and says why, or is removed. Nothing calls an upstream API +deprecated at the supported toolchain (the workspace MSRV), and our own +deprecated items are removed or planned for removal once their migration window has passed. ## Plan @@ -74,6 +75,19 @@ Python wrapper. This run only reviews what those gates can't see. enable `reportUnnecessaryTypeIgnoreComment` without falsely flagging its three needed ignores. Audit it with the bindings installed. +- Atomic `fetch_update` (added 2026-10-31, from + [moq-dev/moq#7280](https://github.com/moq-dev/moq/pull/7280) and + [moq-dev/moq#7315](https://github.com/moq-dev/moq/pull/7315)): the pinned + nightly warns that it is deprecated in favor of `try_update`. Nine files + call it on be39188000 (`git grep fetch_update -- rs`), six in moq-net, + plus kio, moq-c, and moq-mux. `try_update` is stable since Rust 1.95 and + the workspace `rust-version` is 1.91, so switching needs that MSRV bump. + `fetch_update` is not deprecated at the 1.91 MSRV, so keeping it there is + what the Goal allows: if the MSRV stays, keep the calls, list them under + this run's Results with the MSRV that unblocks `try_update`, and allow the + nightly's deprecation warning where it fires. Fixing it keeps the nightly + pin easy to bump. + - `set_var`/`remove_var` (decided 2026-10-31, maintainer, the /quest-complete requeue session): replace `rs lint`'s `git grep` text check (`rs/justfile`, the `lint` recipe) with a `disallowed-methods` entry diff --git a/quest/m2/cmsg-len-send.md b/quest/m2/cmsg-len-send.md new file mode 100644 index 00000000000..46ee5801335 --- /dev/null +++ b/quest/m2/cmsg-len-send.md @@ -0,0 +1,34 @@ +# [XS] moq-sock's control buffer is sized for the send options too + +## Goal + +`cmsg::LEN` in moq-sock is derived from, or checked at compile time +against, both the receive options and the send options (ECN, the GSO +segment size, and the source-address pktinfo) on every Unix target, so a +send cmsg that outgrows the buffer fails the build instead of truncating. +Apple's IPv4 arm is sized for the `in_pktinfo` it now sends. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 /quest-complete run), from [moq-dev/moq#7394](https://github.com/moq-dev/moq/pull/7394) +and [moq-dev/moq#7418](https://github.com/moq-dev/moq/pull/7418). + +[moq-dev/moq#7394](https://github.com/moq-dev/moq/pull/7394) (merged) made `cmsg::LEN` (`rs/moq-sock/src/udp/cmsg/unix.rs:72`) +a compile-time `cmsg_space` sum over each target's receive set and dropped +the runtime subset assert in `UdpSocketState::new`. Its doc comment says the +sum covers the send set, but nothing checks it. [moq-dev/moq#7418](https://github.com/moq-dev/moq/pull/7418) (landed as +[48aaca7d50](https://github.com/moq-dev/moq/commit/48aaca7d50023ce80308487e69b84d419bbc17f3)) +switched Apple's IPv4 source to `IP_PKTINFO` with `in_pktinfo` +(`rs/moq-sock/src/udp/unix.rs:686`), but `LEN` still sizes Apple's IPv4 arm +for `in_addr` (`cmsg/unix.rs:78`). Move `apple` to the `in_pktinfo` arm here. + +Take the larger of the receive and send sums per family, or add a +`const` assert that the send sum fits; prefer the max, which needs no +reasoning about which set is bigger. Drop `encode_src_ip` on Apple if it +is still unused. Verify with cross-target `cargo check` +(macOS, FreeBSD, Linux) or the `platform.yml` workflow, and watch the next +macOS Nightly's `ipv4_source_address_override` (and `fast-apple-datapath`). +The Windows receive path's truncation was fixed separately in +[moq-dev/moq#7543](https://github.com/moq-dev/moq/pull/7543). + +Public API: none. Wire: none. diff --git a/quest/m2/connection-panic-isolation.md b/quest/m2/connection-panic-isolation.md new file mode 100644 index 00000000000..3161b7e4ba7 --- /dev/null +++ b/quest/m2/connection-panic-isolation.md @@ -0,0 +1,57 @@ +# [M] A panic inside moq-tokio's connection internals fails callers instead of panicking them + +## Goal + +After a moq-tokio QUIC connection driver panics, every client-side caller +(`Connecting`, the connection, its streams) gets the connection's terminal +error instead of panicking in its own task, and the connection still closes +to the peer when its last handle drops. Two background tasks +[moq-dev/moq#7360](https://github.com/moq-dev/moq/pull/7360) left out are +supervised by its rule: a panic in a listener's certificate reload watcher +(`tls::Reload`) fails `Listener::accept` like a dead endpoint, and a panic +in a qmux session's driver ends that session with an error and is counted. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 /quest-complete run), from [moq-dev/moq#7360](https://github.com/moq-dev/moq/pull/7360) (merged), +which finished the task-supervision quest. + +[moq-dev/moq#7360](https://github.com/moq-dev/moq/pull/7360) wakes a dead driver's waiters and isolates each handshake in +`Server`'s shared accept queue, but keeps the connection mutex fail-fast +(`rs/moq-tokio/src/quic/driver/mutex.rs`, `lock()` unwraps). So on the +server every accessor runs isolated, while a client's `Connecting` or +stream call that touches the poisoned lock panics in the application's task; +each caller would have to isolate it. And `ConnectionRef::drop` +(`rs/moq-tokio/src/quic/driver/connection.rs:808` at be39188000) skips the implicit close +when the lock is poisoned, so the peer waits for its idle timeout. + +Record the panic as the connection's terminal error once, where the +driver's isolation catches it, and have every accessor (`Connecting`, the +connection, its streams) check that error before locking instead of reading +the poisoned state. [moq-dev/moq#7360](https://github.com/moq-dev/moq/pull/7360) tried per-accessor poison recovery +([34c7666147](https://github.com/moq-dev/moq/commit/34c7666147)) and reverted it because later handshake steps stayed +exposed; a single terminal error checked before locking avoids that. Not +chosen: keeping the mutex fail-fast and documenting that client callers must +isolate, since that leaves those callers panicking, which the Goal rules out. + +For the drop: send the close from the terminal-error path (the driver's +isolation) rather than from the last handle, so a poisoned lock doesn't +matter. + +`tls::Reload` is in `rs/moq-tokio/src/tls.rs:3323` (at be39188000), and +`Reload::spawn` runs its watcher with a plain `runtime::spawn`. +These two spawns were planned on 2026-10-30 (maintainer) in a separate +certificate-reload quest, folded in here on 2026-11-02 since both quests +named them: the reload watcher's panic surfaces as an error from +`Listener::accept`, as [moq-dev/moq#7360](https://github.com/moq-dev/moq/pull/7360) +does for a dead QUIC endpoint. +[moq-dev/moq#7494](https://github.com/moq-dev/moq/pull/7494) (merged) folded qmux's reader, writer, and +WebSocket pump tasks into one caller-spawned `qmux::Driver` per session, so +qmux spawns nothing and the only task left is the one moq-tokio's +`spawn_qmux` (`rs/moq-tokio/src/lib.rs:114`) starts with a plain +`runtime::spawn`. Wrap both spawns in `crate::panic::isolate`; qmux needs +no isolate of its own. Add a regression per class that injects a panic, as +[moq-dev/moq#7360](https://github.com/moq-dev/moq/pull/7360)'s `driver_panic_fails_connecting` does. + +Public API: none expected (behavior only); a new `Error` variant for a +panicked driver if one is needed. Wire: none. diff --git a/quest/m2/e2ee/README.md b/quest/m2/e2ee/README.md index 08ff5de6f01..53f1580d475 100644 --- a/quest/m2/e2ee/README.md +++ b/quest/m2/e2ee/README.md @@ -57,6 +57,18 @@ Rejected: The broadcast wrappers merged in [#5876](https://github.com/moq-dev/moq/pull/5876) with `reserve_track`, which still serves declared names; [rust-publish](/quest/m2/e2ee/rust-publish.md) needs nothing more (maintainer, 2026-10-19). +Noted 2026-10-31 from [moq-dev/moq#7307](https://github.com/moq-dev/moq/pull/7307): + +- The draft's epoch rule (`drafts/draft-lcurley-moq-e2ee.md:136`, clones of a + producing generation share permanent track claims so reopening a physical + track cannot reset its nonce counters) leaves its resume rule implicit. + State explicitly in that section how a reopened or resumed track continues + from those claims. +- moq-net's `Lister` has only `poll_wanted`/`poll_unwanted` + (`rs/moq-net/src/model/listing.rs:601` on be39188000), no async forms, so + neither the e2ee wrapper nor the trait mirror has them. Add them to the + wrapper once moq-net does. + Before `@moq/e2ee` is first published, add its entrypoints to the API Extractor declaration checks, which analyze only published packages since [moq-dev/moq#6794](https://github.com/moq-dev/moq/pull/6794). diff --git a/quest/m2/ffi-exporter.md b/quest/m2/ffi-exporter.md index 989589a00db..f135664d885 100644 --- a/quest/m2/ffi-exporter.md +++ b/quest/m2/ffi-exporter.md @@ -94,6 +94,16 @@ which publishes `rust2-uniffi`, `rust2-uniffi-core`, `rust2-uniffi-macros`, and - Update the quests that name a `kixelated/uniffi-*` fork ([Generated C bindings](/quest/m2/c/README.md)) to point at rust2, then archive the three forks. +- Cancelled Go results (2026-10-31, from + [moq-dev/moq#7397](https://github.com/moq-dev/moq/pull/7397), open): the + Go generator's fix that destroys a cancelled async call's result lives on + `kixelated/uniffi-bindgen-go`'s `cancelled-result` branch and tag + `v0.10.0-kixelated.2+v0.32.0`, not on the fork's `main`, like the earlier + fork tags. The exporter's Go generator must destroy that result too: port + the fork's `destroy` fixture tests to it before archiving the fork. It + should also destroy the error value of a cancelled call that completed + with an error, which the fork still leaks (moq-ffi's errors hold no + handles today). Public API: moq-ffi's dependency names change, generated C type names become CamelCase, and the generated record `MoqAudioEncoderFinished` becomes diff --git a/quest/m2/ffi-future-size.md b/quest/m2/ffi-future-size.md new file mode 100644 index 00000000000..b13a498a869 --- /dev/null +++ b/quest/m2/ffi-future-size.md @@ -0,0 +1,35 @@ +# [S] No large future in the FFI layers can overflow a runtime or foreign thread's stack + +## Goal + +Every future moq-c and moq-ffi spawn or block on is below a size bound +that a lint enforces, so unoptimized builds stay inside the runtime thread +stack (2 MiB) and the 512 KiB foreign threads Apple callers use, with +margin. A future over the bound is boxed or split, not allowed by default. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 /quest-complete run), from [moq-dev/moq#7090](https://github.com/moq-dev/moq/pull/7090). +Merging it pushed moq-c's `a_failing_dial_reports_its_failure` past the +2 MiB runtime stack in a debug build: unoptimized code copied the session +task's `select!` future into each wrapper's frame (`Background::watch` +twice, `isolate`, `run`, the task), about 2 MB in all. Boxing it, as +moq-tokio's dial futures already are, cut the need under 768 KiB. Other +futures in the FFI layers may sit just under the edge the same way. + +- Turn on clippy's `large_futures` for moq-c and moq-ffi (in their + `[lints]`, or crate attributes, since both inherit the workspace lints) + with a `future-size-threshold` chosen from what the audit finds, and box + or split what it flags at the spawn or `block_on` boundary rather than + deep inside moq-net. The lint measures futures where they are awaited, so + it can miss one handed straight to a spawn or `block_on`; check those + calls by hand (or with a `size_of_val` assertion in a test) and record the + sizes. +- Measure the worst remaining case: run each crate's tests in a debug build + on a smaller stack (the existing + `accept_on_a_foreign_thread` in `rs/moq-ffi/src/server.rs` uses + 512 KiB) and record the margin. +- If the lint proves useful, consider it workspace-wide in + [audit lints](/quest/m2/audit-lints.md) instead. + +Public API: none. Wire: none. diff --git a/quest/m2/gpu-encoder-death-test.md b/quest/m2/gpu-encoder-death-test.md new file mode 100644 index 00000000000..852fe5db9c8 --- /dev/null +++ b/quest/m2/gpu-encoder-death-test.md @@ -0,0 +1,24 @@ +# [S] A browser test covers GPU encoder death and a broadcast rename + +## Goal + +A real-browser test makes `@moq/publish`'s video encoder fail mid-stream (as +when the GPU process dies), then renames the network broadcast, and asserts +the failed rendition stays out of the catalog while failed, then comes back +under a fresh track name on the new broadcast with no carried-over failure. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7371](https://github.com/moq-dev/moq/pull/7371): m2, as +recommended. [moq-dev/moq#7371](https://github.com/moq-dev/moq/pull/7371) (merged) moved encoder failure into +each publication's slot, so a new network broadcast starts clean, and covered +that only with unit tests and fake encoders. + +- Put it in `test/interop/clients/js/` beside `publish-ui.browser.ts`. + Inject the failure through the encoder (closing the `VideoEncoder` or a + test hook on its error path) rather than killing a real GPU process. +- Check the catalog and track names from a subscriber, not internal state. + +Public API: none. Wire: none. diff --git a/quest/m2/headless-empty-tail.md b/quest/m2/headless-empty-tail.md new file mode 100644 index 00000000000..1822beb345b --- /dev/null +++ b/quest/m2/headless-empty-tail.md @@ -0,0 +1,26 @@ +# [S] An empty tail stream for a headless moq-transport group creates no group + +## Goal + +When a moq-transport subscription's fill for the Largest Group fails, an +empty subgroup stream (no objects) for that headless group is dropped like a +mid-group stream with objects, in either arrival order, instead of creating +an empty group that reads as complete. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 /quest-complete run), from [moq-dev/moq#7322](https://github.com/moq-dev/moq/pull/7322) (merged), +which finished the headless-groups quest. + +[moq-dev/moq#7322](https://github.com/moq-dev/moq/pull/7322) drops a headless group's mid-group stream that carries objects, +whichever of the stream and the failed fill arrives first, and keeps the +failure as a watermark across fill replacements. It left an empty stream +alone: it still creates an empty group in both orders, and a moq-lite +subscriber downstream of the relay sees a finished, empty group where the +head was lost. Route the empty stream through the same headless check (the +watermark and the parked-tail path in `rs/moq-net/src/ietf/subscriber.rs`), +and add both orders beside `a_headless_tail_after_the_fill_settles_is_dropped` +and `a_headless_tail_before_the_fill_settles_is_dropped`. Streams for +groups the fill didn't owe are unchanged. + +Public API: none. Wire: none; behavior only. diff --git a/quest/m2/hls-stalled-segment.md b/quest/m2/hls-stalled-segment.md new file mode 100644 index 00000000000..9e5c3154a47 --- /dev/null +++ b/quest/m2/hls-stalled-segment.md @@ -0,0 +1,40 @@ +# [S] A recording stalled on an open group reports what it waits for + +## Goal + +When a moq-hls recording waits on a listed segment whose group a still +connected publisher never closes, the wait is visible: the export's status +and a log line name the rendition, segment, and group it waits on. The wait +itself stays unbounded; nothing is cut or discarded on a timer. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7399](https://github.com/moq-dev/moq/pull/7399) (merged; it finished +the restart drain cutoff quest): + +- A recording waiting on a listed segment whose group never closes: + - Bound it with a timer. + - ✅ Report, no timer. Reason: consistent with the 2026-10-24 decision in + the drain cutoff quest, which rejected wall-clock bounds and discarding + listed rows; a disconnecting publisher already errors the read. + - Leave it as is. + +This is a general stalled-publisher case, not only takeover: a replaced +instance's cut and a live recording hang the same way. Find where a cursor waits on a listed row's media +(`rs/moq-hls/src/export/rendition.rs`, the read [moq-dev/moq#7399](https://github.com/moq-dev/moq/pull/7399)'s Codex P2 pointed +at) and surface the wait through the existing status or stats the export +exposes, plus one `warn` per stalled segment, logged when the wait starts +on a group that is still open, so no wakeup or timer is needed. + +Test: a publisher that lists a segment then holds its group open shows the +segment and group in status and logs, and the recording completes normally +once the group closes. + +Public API: possibly a status field; none if existing stats carry it. +Wire: none. + +## Related + +- [A library recorder gets the takeover cut](/quest/m2/mux-source-cut.md) - the other [moq-dev/moq#7399](https://github.com/moq-dev/moq/pull/7399) follow-up diff --git a/quest/m2/ietf-fill-refill.md b/quest/m2/ietf-fill-refill.md index c8af7d782fd..dc21281f2f0 100644 --- a/quest/m2/ietf-fill-refill.md +++ b/quest/m2/ietf-fill-refill.md @@ -22,8 +22,11 @@ Group stitching across two streams drew repeated fixes: [#6407](https://github.com/moq-dev/moq/pull/6407), plus [#7322](https://github.com/moq-dev/moq/pull/7322) (headless groups) and join after finish ([moq-dev/moq#7306](https://github.com/moq-dev/moq/pull/7306)). The `Fill` enum -and its handoff (`rs/moq-net/src/ietf/subscriber.rs` about lines 273-396 -and 4712-4862) exist only because the model has one producer per group. +and its handoff exist only because the model has one producer per group. In +`rs/moq-net/src/ietf/subscriber.rs` on be39188000 (refreshed 2026-11-02): +`Fill`, `FillRequest`, and `Filling` at lines 261-441, `claim_fill` at +5070-5110, and `recv_fill`, `run_fill`, and `run_fill_objects` from 5177. +Search by those names if the lines have moved again. Decided 2026-10-28 (maintainer): nothing is written into a group copy below its live log, so the head no longer fills the tail's group in place. Once diff --git a/quest/m2/ietf-join-pin-head.md b/quest/m2/ietf-join-pin-head.md new file mode 100644 index 00000000000..8bb40c2f405 --- /dev/null +++ b/quest/m2/ietf-join-pin-head.md @@ -0,0 +1,36 @@ +# [S] A kept draft-18 join pins its head group + +## Goal + +On draft 18, a joining FETCH that our publisher keeps open after a quick +finish is answered even if the publisher drops its `track::Producer` right +after `finish()`: the kept join pins the head group in the cache until it is +answered, instead of the late FETCH being refused as not found. The +draft-18 interop `tail` and `finish` cells pass with a Rust publisher. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7306](https://github.com/moq-dev/moq/pull/7306): m2, as +recommended. [moq-dev/moq#7306](https://github.com/moq-dev/moq/pull/7306) (merged) kept the join after finish. + +[moq-dev/moq#7306](https://github.com/moq-dev/moq/pull/7306) keeps the subscription's request open after PUBLISH_DONE but does not +hold the head: the retained `Joined::Group` holds only a `track::Consumer` +(`rs/moq-net/src/ietf/publisher.rs:885`, `Retain` at `:927` as of +be39188000), so when the producer drops, the cache demotes the latest group, +and the late FETCH gets a loud refusal (accepted by its quest's Plan). + +- Hold the head group from the retained subscription (a cache pin or a + held `group::Consumer`) until the join is answered or the request closes, + bounded by the request's lifetime, so no new constant is needed. +- Test: finish then drop the producer, then a late joining FETCH on draft + 18 still gets the head. +- Rerun the draft-18 interop `tail` and `finish` cells with the Rust + publisher, as [IETF finish](/quest/m2/ietf-finish.md) asks. + +Public API: none. Wire: none. + +## Related + +- [IETF finish](/quest/m2/ietf-finish.md) - owns the interop cells diff --git a/quest/m2/js-warmup-trim.md b/quest/m2/js-warmup-trim.md new file mode 100644 index 00000000000..9765aacb1e5 --- /dev/null +++ b/quest/m2/js-warmup-trim.md @@ -0,0 +1,29 @@ +# [XS] The JS player keeps a track's start when decoding from its first packet + +## Goal + +`@moq/watch` trims an audio track's catalog `warmup` only on a mid-stream +join, as moq-audio does: decoding from a track's first packet (group 0) +drops only the codec's pre-skip, so a recording's first 80 ms of Opus plays +and a short finished track is not empty. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7384](https://github.com/moq-dev/moq/pull/7384): m2, as +recommended. + +[moq-dev/moq#7384](https://github.com/moq-dev/moq/pull/7384) (merged) stopped `moq_audio::decode::Consumer` from arming the warmup when +the first packet read is frame 0 of group 0. JS arms it unconditionally: +`js/watch/src/audio/terminal.ts:76` sets the trim boundary to the first +frame's timestamp plus warmup, fed from `audio/decoder.ts:557,645` (as of +be39188000). + +- First confirm the JS behavior with a test (decode a short Opus track from + group 0 and count frames). If it trims, apply [moq-dev/moq#7384](https://github.com/moq-dev/moq/pull/7384)'s rule: no warmup when + the first packet is group 0's first frame; a loss or a refused first + packet still arms it. +- Mirror [moq-dev/moq#7384](https://github.com/moq-dev/moq/pull/7384)'s tests, including a finished track shorter than the warmup. + +Public API: none (behavior change in `@moq/watch`). Wire: none. diff --git a/quest/m2/kio-tasks-waker-test.md b/quest/m2/kio-tasks-waker-test.md new file mode 100644 index 00000000000..a7c8bc84f0b --- /dev/null +++ b/quest/m2/kio-tasks-waker-test.md @@ -0,0 +1,30 @@ +# [XS] kio::Tasks tolerates a removed task's waker firing into its reused slot + +## Goal + +A unit test in `rs/kio/src/task.rs` shows that when a task taken out with +`Tasks::remove` wakes after another task was `insert`ed into its slot, the +new occupant is polled at most once spuriously, still finishes on its own +wake (handed back through `poll_each`), and its `TaskKey` still removes +it. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 /quest-complete run), from [moq-dev/moq#7340](https://github.com/moq-dev/moq/pull/7340), whose +online review suggested it. + +[moq-dev/moq#7340](https://github.com/moq-dev/moq/pull/7340) added `insert`, `remove`, `TaskKey`'s generation stamp, and +`poll_each`. The existing +`a_stale_wake_costs_at_most_one_spurious_poll` (`task.rs:944` as of +be39188000) covers a *finished* occupant's waker through `push`, and +`removed_tasks_free_their_slot_and_key` (`:662`) opens the removed task's +gate only while its slot is empty. Neither fires a removed task's waker +after the slot is reused, which is the path the lite subscriber's +`Change::Close` takes. Write that case with the `gated` helper and the +removed task's leftover slot waker, as the stale-wake test does. + +Not a quest: [moq-dev/moq#7340](https://github.com/moq-dev/moq/pull/7340) also noted that no bench sweeps lazy discovery with many +prefixes (its viewers use `client_discovers_all`). Add a lazy axis to the +session bench only if a later change to announce discovery needs it. + +Public API: none. Wire: none. diff --git a/quest/m2/miri-nightly-budget.md b/quest/m2/miri-nightly-budget.md new file mode 100644 index 00000000000..2a8937c0338 --- /dev/null +++ b/quest/m2/miri-nightly-budget.md @@ -0,0 +1,37 @@ +# [S] Nightly's Miri job stays inside its time budget + +## Goal + +Nightly's `Tests (rs miri)` entry finishes well inside the job's 60-minute +timeout on the 4-core ARM runner, and no single Miri test comes near +nextest's 15-minute `miri` profile cap, with headroom recorded from a real +Nightly run. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 /quest-complete run), from [moq-dev/moq#7315](https://github.com/moq-dev/moq/pull/7315) +(merged), which finished the Miri for in-memory unsafe quest. +Milestone m2 (user: "use recommended for any CI/flakes"). + +Act on the first Nightly runs' timings, not ahead of them. Locally (24 +cores) `just rs miri` took 11.5 min wall: strict pass 118 tests in 109 s, +epoch pass 94 in 307 s. + +- No Nightly timing exists yet: the first run with the job + ([run 41483654426](https://github.com/moq-dev/moq/actions/runs/41483654426), + 2026-11-02) failed in 2 minutes before any test, as cargo-miri found no + `rust-src` ("given Rust source directory ... does not exist"). Install it + with the toolchain (`components: miri` in `.github/workflows/nightly.yml` + becomes `miri, rust-src`), then read the timings from the next run. +- `a_group_has_no_frame_cap` (`rs/moq-net/src/model/group.rs:5228` as of + be39188000) took 307 s alone, writing 10,000 one-byte frames. Under `cfg(miri)` + write fewer, still more than any former per-group frame cap, so the test + keeps its meaning. +- `rs/moq-net/src/coding/reader.rs:1232` builds its payload byte by byte + (`(0..SIZE).map(|i| (i % 251) as u8)`); switch to a cheaper fill if + Miri starts running that module. +- If the job nears 60 minutes, split the strict and epoch passes into + separate matrix entries (`.github/workflows/nightly.yml` Tests matrix, + `timeout-minutes: 60` at line 328) rather than raising the timeout. + +Public API: none. Wire: none. diff --git a/quest/m2/moqsrc-reconnect-order.md b/quest/m2/moqsrc-reconnect-order.md new file mode 100644 index 00000000000..30fe96c6d9f --- /dev/null +++ b/quest/m2/moqsrc-reconnect-order.md @@ -0,0 +1,23 @@ +# [XS] moqsrc reconnects correctly whatever order events arrive in + +## Goal + +A moq-gst test reconnects `moqsrc` while a rendition pad is blocked +downstream, delivering the reconnect's events (session close, new catalog, +restart, track end) in several orders, and the source always ends with one +live pad per rendition, no duplicate or leaked pads, and no deadlock. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7368](https://github.com/moq-dev/moq/pull/7368): m2, as +recommended. [moq-dev/moq#7368](https://github.com/moq-dev/moq/pull/7368) (merged) moved each rendition pad +into one actor (`rs/moq-gst/src/source/`). + +Reuse [moq-dev/moq#7368](https://github.com/moq-dev/moq/pull/7368)'s restart/relist/retire/finish tests and their blocked-pad +setup. Enumerate the orders explicitly (a small table of permutations) +rather than relying on stress runs, and check pad count and EOS state after +each. + +Public API: none. Wire: none. diff --git a/quest/m2/mux-source-cut.md b/quest/m2/mux-source-cut.md new file mode 100644 index 00000000000..1ad448d6acd --- /dev/null +++ b/quest/m2/mux-source-cut.md @@ -0,0 +1,43 @@ +# [S] A library recorder gets the takeover cut + +## Goal + +A recorder built directly with `moq_hls::export::Broadcaster::new` ends a +replaced instance's recording at what it listed when another publisher takes +the broadcast over, exactly as one built through `Server::broadcaster` does. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7399](https://github.com/moq-dev/moq/pull/7399): a separate S +quest in m2 for the `moq_mux::Source` API, alongside reporting stalled +segments. + +[moq-dev/moq#7399](https://github.com/moq-dev/moq/pull/7399) (merged) detects takeover in the server's follower (`Followed::start` +in `rs/moq-hls/src/server/mod.rs`) and cuts through the crate-private +`Broadcaster::replaced`. `Broadcaster::new(source, config)` +(`rs/moq-hls/src/export/mod.rs:141`) pins one instance and +never sees the takeover, because `moq_mux::Source` exposes no way to follow +its own path: `Source::follow(None, ..)` returns +`follow::Instances::fixed` for the source's own path +(`rs/moq-mux/src/source.rs:390` as of be39188000). + +- Give `Source` a public way to follow its own path's announcements (a + `follow_self`, or `follow` returning live instances for the own path when + the source was built from an origin), reusing `follow::Instances`; do + not add a second follower. +- Have `Broadcaster` watch it and call its existing cut on the first + replacement, so `Server` and library callers share one path; drop the + server-side special case if it becomes redundant. + +Test: a `Broadcaster::new` recording replaced by a new epoch drains exactly +its listed segments and returns `Ok(None)`, mirroring [moq-dev/moq#7399](https://github.com/moq-dev/moq/pull/7399)'s +`a_replaced_recording_ends_at_what_it_listed`. + +Public API: `moq_mux::Source` gains a way to follow its own path. Wire: +none. + +## Related + +- [A recording stalled on an open group reports what it waits for](/quest/m2/hls-stalled-segment.md) - the other [moq-dev/moq#7399](https://github.com/moq-dev/moq/pull/7399) follow-up diff --git a/quest/m2/mux-vr-ambisonic-fmp4.md b/quest/m2/mux-vr-ambisonic-fmp4.md index 4ea1c891536..bc6d4f2182f 100644 --- a/quest/m2/mux-vr-ambisonic-fmp4.md +++ b/quest/m2/mux-vr-ambisonic-fmp4.md @@ -18,4 +18,16 @@ already import as `ambisonic` through that family 2 validation. What remains is proving it: test with an fMP4 first-order ambisonic fixture and a refused non-diegetic stereo one. +Noted 2026-10-31 from [moq-dev/moq#7308](https://github.com/moq-dev/moq/pull/7308), +which fills `projection`, `stereo`, and `layout` on import only +(`rs/moq-mux/src/codec/spatial.rs`). The fMP4 exporter already writes a +family 2 `dOps` from the OpusHead (`dops_mapping` in +`rs/moq-mux/src/container/fmp4/mod.rs:1069` on be39188000 refuses only family +3), so also round-trip the ambisonic fixture through `synthesize_audio_trak`, +as `opus_dops_surround_round_trips` does for family 1. No exporter writes +`projection` or `stereo` back yet: the fMP4 exporter should write +`sv3d`/`st3d`, and the Matroska exporter `Projection` and `StereoMode`, so +an imported VR track round-trips. Do the fMP4 side here if it stays small; +otherwise split the exporters into their own quest. + Public API: none. Wire: none. diff --git a/quest/m2/nix-relay-peers-test.md b/quest/m2/nix-relay-peers-test.md new file mode 100644 index 00000000000..38c9b3af44d --- /dev/null +++ b/quest/m2/nix-relay-peers-test.md @@ -0,0 +1,32 @@ +# [S] The relay NixOS module's peer list is tested + +## Goal + +`nix flake check` evaluates the moq-relay NixOS module with no, one, and +several `cluster.rootUrl` peers and refuses an empty list, asserting the +`MOQ_CLUSTER_CONNECT` value each produces. Optionally a NixOS VM test runs a +root and two leaves from the module and sees them connect. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7343](https://github.com/moq-dev/moq/pull/7343): m2, as +recommended. + +[moq-dev/moq#7343](https://github.com/moq-dev/moq/pull/7343) made `services.moq-relay.cluster.rootUrl` a +`nullOr (nonEmptyListOf str)` (`nix/modules/moq-relay.nix:153`), joined into +`MOQ_CLUSTER_CONNECT` at `:380`, and verified it only by a manual `nix eval`. + +- Add an eval check beside `nix/runner/config-test.nix`, wired into + `flake.nix`'s `checks` (`:729` as of be39188000): `null` sets no `MOQ_CLUSTER_CONNECT`, one + and two URLs join with commas, `[ ]` fails the type check, and a leaf + without peers trips the assertion at `:219`. +- A VM test (root plus two leaves) only if it stays cheap in CI; otherwise + leave it out and say so. + +The string-to-list break needs a `MIGRATE.md` line; the +[migration audit](/quest/m2/release/migrate-audit.md) handles it, not this +quest. + +Public API: none. Wire: none. diff --git a/quest/m2/obs-decode-latest.md b/quest/m2/obs-decode-latest.md new file mode 100644 index 00000000000..f91eebdea07 --- /dev/null +++ b/quest/m2/obs-decode-latest.md @@ -0,0 +1,35 @@ +# [XS] The OBS source joins audio at the live edge + +## Goal + +The OBS MoQ source's audio decoder starts at the newest cached group instead +of replaying the track's backlog, so a reconnect or rendition change plays +live audio at once. The Go wrapper no longer re-exports the raw ffi decoder +output records. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7288](https://github.com/moq-dev/moq/pull/7288), which added +`DecodeStart` to the UniFFI decoders and left the plugin on the default: + +- OBS plugin decode start: + - Keep the default (oldest). + - ✅ Switch to `DecodeStart::Latest`. Reason: the source is a live player, + so replaying the cache only adds latency. + +- `cpp/obs/src/moq-source.cpp:1068` (as of be39188000): set `output.start` to + `moq::DecodeStart::kLatest` on the `moq::AudioDecoderOutput`. Video reads + raw frames through `MediaTrackConsumer` and its own decoder, so it is + unaffected. +- Delete the `AudioDecoderOutput` and `VideoDecoderOutput` aliases in + `go/wrapper/types.go:15,81`: no wrapper API accepts them + (`audio.DecoderOutput`/`video.DecoderOutput` are the real inputs). Their + migration note is gathered by the + [migration audit](/quest/m2/release/migrate-audit.md). +- If cheap, an e2e check that an audio decoder with `Latest` starts at the + newest group, beside [moq-dev/moq#7288](https://github.com/moq-dev/moq/pull/7288)'s `video_decode_start`. + +Public API: Go `wrapper.AudioDecoderOutput`/`VideoDecoderOutput` removed +(breaking). Wire: none. diff --git a/quest/m2/obs-multitrack-check.md b/quest/m2/obs-multitrack-check.md new file mode 100644 index 00000000000..8edade70472 --- /dev/null +++ b/quest/m2/obs-multitrack-check.md @@ -0,0 +1,40 @@ +# [S] OBS simulcast layers reach moq-watch from a live OBS, including the dock's Go Live + +## Goal + +A live OBS 32.2.2 run, recorded in the PR, shows the +`WHIPSimulcastTotalLayers` profile setting attaching two or more scaled +video encoders to `moq_output`, each published as its own rendition and +watched in ``, both through the obs-websocket recipe in +`doc/bin/obs.md` and through the MoQ dock's Go Live, which honors the +profile's layer count instead of creating a single encoder. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 /quest-complete run), from [moq-dev/moq#7333](https://github.com/moq-dev/moq/pull/7333), +which finished the OBS multitrack quest as docs only. +[moq-dev/moq#7333](https://github.com/moq-dev/moq/pull/7333) derived the recipe from OBS and obs-websocket sources without running +OBS, so the manual run never happened. + +- Manual check: follow `doc/bin/obs.md`'s Simulcast renditions section on + OBS 32.2.2 (the version `cpp/obs/buildspec.json` pins). Fix the doc if a + step is wrong. +- Dock: `MoQDock::CreateVideoEncoder` (`cpp/obs/src/moq-dock.cpp:1050`) + creates one encoder and attaches it with `obs_output_set_video_encoder` + (:1112), ignoring `Stream1/WHIPSimulcastTotalLayers`. Read the count from + the profile and create the scaled layers with OBS's ladder math (layer + `i` at `i/N` of the output size and bitrate), attached with + `obs_output_set_video_encoder2`. This gives a GUI path to simulcast, + since stock Settings > Stream can't select `moq_service`. +- Cover the dock's layers with `just obs test`; verify with + `just obs compile` and the manual run. +- [moq-dev/moq#7029](https://github.com/moq-dev/moq/pull/7029) (merged) + added a multi-encoder ladder to the OBS output tests + (`g_ladder` in `cpp/obs/test/moq-output-test.cpp`). Extend it to assert + each rendition advertises its own bitrate in the catalog. + +Public API: none. Wire: none. + +## Related + +- [Simulcast rung disable](/quest/m1/simulcast-rung-disable.md) - the shaped-uplink proof of the rung disable [moq-dev/moq#7029](https://github.com/moq-dev/moq/pull/7029) added, which turns these layers off as the grant falls diff --git a/quest/m2/one-port/rtc-feed.md b/quest/m2/one-port/rtc-feed.md index 667799c2c00..cc1f30e0aec 100644 --- a/quest/m2/one-port/rtc-feed.md +++ b/quest/m2/one-port/rtc-feed.md @@ -75,6 +75,11 @@ the fed socket may belong in the server's constructor rather than the config. The fed mux has nothing to bind lazily, and a dropped feed should fail the server loudly rather than idle. +Noted 2026-10-31 from [moq-dev/moq#7095](https://github.com/moq-dev/moq/pull/7095): +its demux `Datagram` carries no ECN marks or kernel receive timestamp. If +the fed mux's RTP congestion control needs either, add them to the demux +here rather than reading them around it. + Tests: a fed mux completes ICE and DTLS with a client through an in-memory `Transmit`, reports the selected tuple on `pinned`, and releases it when the session closes. A fed dual-stack socket serves an IPv4 peer. The existing diff --git a/quest/m2/one-port/srt-demux.md b/quest/m2/one-port/srt-demux.md index 06086824fd8..3b80ed3611c 100644 --- a/quest/m2/one-port/srt-demux.md +++ b/quest/m2/one-port/srt-demux.md @@ -43,6 +43,11 @@ that sees every inbound packet can tell. Test both with a mocked clock, which the driver allows by taking `now` as input; srt-tokio reads std's clock, so `tokio::time::pause` cannot drive its idle timeout. +Noted 2026-10-31 from [moq-dev/moq#7095](https://github.com/moq-dev/moq/pull/7095): +a pinned lookup in the demux's flow table hashes with SipHash, about 20 ns. +Once SRT pins send every SRT datagram through that path, measure a faster +keyed hasher with the `demux_route` bench and switch if it pays. + Tests: `moq-srt`'s existing integration test runs against the shared socket with a QUIC client active on the same port; a QUIC short header from a new 4-tuple during an SRT session still reaches QUIC. diff --git a/quest/m2/opus-surround-export.md b/quest/m2/opus-surround-export.md new file mode 100644 index 00000000000..3050a79e4c0 --- /dev/null +++ b/quest/m2/opus-surround-export.md @@ -0,0 +1,34 @@ +# [S] Opus export never writes a channel mapping it can't stand behind + +## Goal + +Exporting a catalog Opus rendition with 3 to 8 channels and no description +writes a `dOps` with the Vorbis family 1 mapping table, as the TS importer +assumes, instead of an invalid family 0 `dOps`. A family 3 OpusHead +(RFC 8486 projection with a demixing matrix) is refused by +`opus::Mapping::parse` with a typed error, or carried as `ambisonic`, +instead of being read as a channel table. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 /quest-complete run), from [moq-dev/moq#7336](https://github.com/moq-dev/moq/pull/7336) (merged), +which finished the surround-Opus-through-CMAF quest, +and [moq-dev/moq#7308](https://github.com/moq-dev/moq/pull/7308). + +- No description: on main the `None` arm in + `rs/moq-mux/src/container/fmp4/mod.rs` (line 1105 as of be39188000) builds + `opus::Config::new` and writes `channel_mapping: None` (family 0) for any + channel count, which RFC 7845 allows only for 1 or 2 channels. Use + `opus::Mapping::vorbis` (`rs/moq-mux/src/codec/opus/mapping.rs`) for 3 + to 8 channels and refuse more. +- Family 3: `Mapping::parse` (`mapping.rs:109`) reads a family 3 head as + a plain table, so it imports as plain channels or fails as an invalid + table. [moq-dev/moq#7336](https://github.com/moq-dev/moq/pull/7336) refuses family 3 in fMP4 only. Refuse it in `parse` itself + (recommended, since no consumer can render a demixing matrix today), or + carry it as `ambisonic` if [VR and ambisonic fMP4](/quest/m2/mux-vr-ambisonic-fmp4.md) + wants it. + +Cover both with unit tests beside [moq-dev/moq#7336](https://github.com/moq-dev/moq/pull/7336)'s +`opus_dops_mapping_the_head_cannot_describe_is_refused`. + +Public API: none, or a new error variant. Wire: none. diff --git a/quest/m2/perf/stream/codec.md b/quest/m2/perf/stream/codec.md index 03bee6655d8..5455f8c631f 100644 --- a/quest/m2/perf/stream/codec.md +++ b/quest/m2/perf/stream/codec.md @@ -14,6 +14,22 @@ Gate: re-run the bench against the driver's numbers in the epic README. If the p shows tungstenite's read-buffer zero-fill, its `out_buffer` copy, or its memmove after a partial write as a real share, abandon the quest. +Stopgap first (2026-10-31, from +[moq-dev/moq#7086](https://github.com/moq-dev/moq/pull/7086)'s benchmark; +moved here when [moq-dev/moq#7494](https://github.com/moq-dev/moq/pull/7494) +landed the driver and kept tungstenite): tungstenite's default read buffer +dominates its share of the profile. Shrink +`WebSocketConfig::read_buffer_size` to near qmux's maximum record size +(about 16 KiB) in `qmux::ws` (`capped` in `rs/qmux/src/ws.rs`) and on the +relay's axum `WebSocketUpgrade` (`rs/moq-relay/src/websocket.rs`) until this +codec replaces tungstenite. That setting is the buffer tungstenite allocates +up front, not a memory cap: a larger frame still grows past it, so the +existing message and frame size limits stay the bound. The stopgap lowers +the baseline allocation per connection and the zero-fill per read attempt. +Land it as its own commit with a bench run that also reports retained +memory, then run the gate above before deciding whether replacing +tungstenite still pays. + The subset to implement, from the 2026-10-22 audit: - **Frames.** 7-, 16-, and 64-bit lengths. Reject unmasked client frames and diff --git a/quest/m2/pes-hold-open-gop.md b/quest/m2/pes-hold-open-gop.md new file mode 100644 index 00000000000..518a510b0a1 --- /dev/null +++ b/quest/m2/pes-hold-open-gop.md @@ -0,0 +1,36 @@ +# [S] Open-GOP MPEG-2 export trims leading B-pictures in zero-stuffed CBR streams + +## Goal + +A TS export starting or resuming inside an open GOP of MPEG-2 video drops +the orphaned leading B-pictures even when their PES ends on zero stuffing, +as CBR encoders emit, without ever trimming part of a reference picture +whose start code straddles two PES packets. The H.264 and HEVC open-GOP +recovery tests also cover a join at an in-flight group. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 /quest-complete run), from [moq-dev/moq#7365](https://github.com/moq-dev/moq/pull/7365), +which finished the open-GOP MPEG-2 export quest. + +[moq-dev/moq#7365](https://github.com/moq-dev/moq/pull/7365) never trims a PES that ends on a zero byte or inside a picture +header, since the next PES may hold the rest of a reference picture's start +code. Zero-stuffed CBR streams therefore keep some leading B-pictures that +decoders skip. Hold each whole-PES MPEG-2 frame until the next one arrives +(or the track ends), then decide with both in hand: if the next PES does +not continue a start code, the trailing zeros were stuffing and the orphan +can go. At the track's end nothing can prove the start code was complete, +so a held PES is kept untrimmed, as today. The cost is one PES of latency on MPEG-2 video only, which the +export's existing `PesPacketStream::hold` path +(`rs/moq-mux/src/container/ts/import.rs`) and [moq-dev/moq#7365](https://github.com/moq-dev/moq/pull/7365)'s per-track `orphan` +state in `container/ts/export.rs` should absorb; check what the hold does +to the live edge before settling it. + +Extend `mpeg2_orphans_are_whole_pes_of_leading_b_pictures` with a +zero-stuffed orphan, a P-picture start code split across PES packets, and +a held final PES ending in an incomplete start code, which stays. +For H.264 (`Open`) and HEVC (`Rasl`) in +`rs/moq-mux/src/container/ts/recovery_test.rs`, add a join with a nonzero +`join` so the export starts at an in-flight group, as the MPEG-2 arm does. + +Public API: none. Wire: none. diff --git a/quest/m2/probe/README.md b/quest/m2/probe/README.md index 540d2a9bca5..898e2316410 100644 --- a/quest/m2/probe/README.md +++ b/quest/m2/probe/README.md @@ -90,6 +90,7 @@ This README's own work, once the children land: - [Viewer targets](/quest/m2/probe/viewer-target.md) - a session sends its viewer handles as its PROBE target, probes to the peer's, splits the receive estimate into viewer grants, and advertises `Increase` where a sender exists - [Players set viewer targets](/quest/m2/probe/players.md) - players hold a viewer handle per watched broadcast at its highest enabled rendition's bitrate - [Viewer up-switch](/quest/m2/probe/viewer-upswitch.md) - a viewer capped by its small rendition finds headroom through PROBE and moves up +- [Viewer targets are tested end to end and documented in Python](/quest/m2/probe/viewer-followups.md) - three gaps moq-dev/moq#7090 left are closed ## Related diff --git a/quest/m2/probe/viewer-followups.md b/quest/m2/probe/viewer-followups.md new file mode 100644 index 00000000000..fd94679b999 --- /dev/null +++ b/quest/m2/probe/viewer-followups.md @@ -0,0 +1,43 @@ +# [M] Viewer targets are tested end to end and documented in Python + +## Goal + +Three gaps [moq-dev/moq#7090](https://github.com/moq-dev/moq/pull/7090) +left are closed: + +- A `js/net` test shows a viewer's target reaching a fake peer that + advertises PROBE `Increase` over a pooled connection, and reaching it + again after the connection reconnects. +- A test in each language stalls the PROBE writer and shows one target in + flight, with the latest target sent once the writer drains. +- `py/moq-rs/docs/index.md` has a Bandwidth section listing `Allocator`, + `Reservation`, `BandwidthConsumer`, `Viewers`, and `Viewer`. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 /quest-complete run), from [moq-dev/moq#7090](https://github.com/moq-dev/moq/pull/7090), +which finishes [viewer targets](/quest/m2/probe/viewer-target.md). + +- Pooled reconnect: the `Reload`/pool `viewers` wiring + (`js/net/src/connection/reload.ts`) is covered only by type checks and + the subscriber's wire tests. Drive a reconnect through the pool against a + fake peer and assert the live handles' sum arrives on the new session. +- Stalled PROBE: [moq-dev/moq#7090](https://github.com/moq-dev/moq/pull/7090) fixed targets piling up behind a blocked PROBE + writer but couldn't test it, since the test setups can't stall a stream. + Use a transport double whose PROBE stream stops granting credit (Rust: + the in-memory session; JS: a fake writer whose pending write the test + resolves by hand). Assert one target is in flight, change the target + several times, release the write, and assert exactly the latest target is + sent. +- Python docs: list the five types with one-line descriptions, following + the file's existing sections. + +Native players (`moq play`, `moqsrc`, and the OBS source) holding a +viewer once they adapt renditions is [players](/quest/m2/probe/players.md)' +work, not this quest's. + +Public API: none (tests and docs). Wire: none. + +## Required + +- [Viewer probe targets and grants](/quest/m2/probe/viewer-target.md) - its PR, [moq-dev/moq#7090](https://github.com/moq-dev/moq/pull/7090), adds the handles and PROBE wiring these test diff --git a/quest/m2/publish-cancel-classify.md b/quest/m2/publish-cancel-classify.md new file mode 100644 index 00000000000..1a2e4e7d98d --- /dev/null +++ b/quest/m2/publish-cancel-classify.md @@ -0,0 +1,30 @@ +# [XS] The lite publisher tells transport faults from cancels + +## Goal + +`rs/moq-net`'s lite publisher no longer counts every `Error::Transport` as a +clean cancel: a peer walking away logs at debug, a real transport fault at +warn. The probe stream and publisher share one classifier, and a test covers +a clean probe cancel at the `Control` boundary. + +## Plan + +Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 +/quest-complete run), from +[moq-dev/moq#7350](https://github.com/moq-dev/moq/pull/7350): m2, as +recommended. + +`is_cancel` (`rs/moq-net/src/lite/publisher.rs:1559` as of be39188000, +"TODO better classify WebTransport errors") matches all `Transport(_)` and +`IdleTimeout`. +[moq-dev/moq#7350](https://github.com/moq-dev/moq/pull/7350) left it alone and wrote its own three-variant match for probe ends in +`lite/publisher.rs` and `lite/subscriber.rs`. + +- Classify `Transport` errors by what the transport says (a stream reset or + stop with the cancel code, or a closed session, is a cancel; anything else + is a fault), then use one helper at every log site, including [moq-dev/moq#7350](https://github.com/moq-dev/moq/pull/7350)'s. +- Test: a subscriber cancelling a probe at the `Control` boundary logs no + warning (or the classifier returns cancel for it), and an injected + transport error is not a cancel. + +Public API: none. Wire: none. diff --git a/quest/m2/publish-empty-previews.md b/quest/m2/publish-empty-previews.md new file mode 100644 index 00000000000..8fe89ab55f8 --- /dev/null +++ b/quest/m2/publish-empty-previews.md @@ -0,0 +1,30 @@ +# [XS] moq-publish hides previews that show nothing + +## Goal + +`` hides a preview that renders nothing (an idle `