diff --git a/quest/m0/nightly/README.md b/quest/m0/nightly/README.md index fac16fe2dc5..c80c7fb2227 100644 --- a/quest/m0/nightly/README.md +++ b/quest/m0/nightly/README.md @@ -16,6 +16,10 @@ Decided 2026-10-24 (maintainer), from a triage of run child per distinct cause, each landing on its own. The PR gate keeps skipping benches and examples; a green Nightly catches them within a day. +Decided 2026-10-27 (maintainer), from the coverage audit's triage of run +[40506938673](https://github.com/moq-dev/moq/actions/runs/40506938673): more +children for the causes nothing owned. + This README's own work, after the children: - Nightly's multi-step jobs (Features first) keep going past a failed step @@ -35,5 +39,8 @@ Public API: none. Wire: none. - [The browser backend baseline completes on IETF22](/quest/m0/nightly/browser-baseline.md) - the WASM and UniFFI backend baseline completes every supported transport case without an internal error - [Resolve the paired catalog fanout regressions](/quest/m0/nightly/paired-shared.md) - the reported catalog fanout regressions are attributed and genuine extra costs are fixed with paired measurements - [Probing beside Cubic](/quest/m0/nightly/competing-cubic.md) - the `competing_cubic` probe test stays within its deadline budget +- [OBS TSan passes without a wall-clock bound](/quest/m0/nightly/obs-tsan.md) - `destroySource` asserts ordering instead of `elapsed < 500ms` +- [Paired bench regressions bisected](/quest/m0/nightly/bench-regressions.md) - retirement/selection's +21-24% and two 10% rows fixed or shown to be noise +- [Firefox's player closes every VideoFrame](/quest/m0/nightly/firefox-videoframe.md) - the Media job's Firefox player stops leaking an unclosed VideoFrame - [Gate stops hanging](/quest/m0/nightly/gate-hang.md) - `just gate` finishes instead of hanging to the timeout, and a hang names its module - [moq-quic rustdoc](/quest/m0/nightly/quic-rustdoc.md) - `--all-features` rustdoc for moq-quic builds and a lane checks it diff --git a/quest/m0/nightly/bench-regressions.md b/quest/m0/nightly/bench-regressions.md new file mode 100644 index 00000000000..fa4b6ec4a42 --- /dev/null +++ b/quest/m0/nightly/bench-regressions.md @@ -0,0 +1,24 @@ +# [M] Paired bench regressions bisected + +## Goal + +Each regression the paired Rust comparison confirmed on 2026-10-27 is fixed at +its cause or shown to be noise by an A/A rerun, and Nightly's Rust bench +targets job passes. + +## Plan + +Nightly [40506938673](https://github.com/moq-dev/moq/actions/runs/40506938673) (2026-10-27, at 22808bbb2f) confirmed five regressions: +retirement/selection at 8x32, 32x32, and 128x32 (+21 to 24%), +server_admission/workers=4 (+10%), and track_range_demand/1000r (+10%). +The retirement/selection rows agree across three sizes, so treat them as +real: bisect with the paired base across the previous night's commits. The two +10% rows sit at the threshold that [bench CI](/quest/m1/bench-ci.md) has yet +to calibrate; rerun them before bisecting. bench-ci owns the threshold, this +quest owns the regressions. + +Public API: none. Wire: none. + +## Related + +- [Calibrate the paired nightly benchmarks](/quest/m1/bench-ci.md) - the threshold these crossed diff --git a/quest/m0/nightly/firefox-videoframe.md b/quest/m0/nightly/firefox-videoframe.md new file mode 100644 index 00000000000..809eec03c31 --- /dev/null +++ b/quest/m0/nightly/firefox-videoframe.md @@ -0,0 +1,16 @@ +# [S] Firefox's player closes every VideoFrame + +## Goal + +Nightly's Media job no longer fails Firefox's player on the console error "A +VideoFrame was garbage collected without being closed". + +## Plan + +Unowned in Nightly [40506938673](https://github.com/moq-dev/moq/actions/runs/40506938673) (2026-10-27, at 22808bbb2f). +[Firefox publishing](/quest/m2/publish-firefox.md) covers only the publisher +side. Find the player path that drops a decoded frame without `close()`, such +as one replaced, skipped, or discarded on teardown. The unclosed `VideoFrame` +is a leak in the player, not a console-noise filter to add. + +Public API: none. Wire: none. diff --git a/quest/m0/nightly/obs-tsan.md b/quest/m0/nightly/obs-tsan.md new file mode 100644 index 00000000000..f49be61a63c --- /dev/null +++ b/quest/m0/nightly/obs-tsan.md @@ -0,0 +1,18 @@ +# [S] OBS TSan passes without a wall-clock bound + +## Goal + +Nightly's OBS TSan job passes, and `destroySource` in +`cpp/obs/test/moq-source-test.cpp` no longer asserts elapsed wall time. + +## Plan + +Nightly [40506938673](https://github.com/moq-dev/moq/actions/runs/40506938673) (2026-10-27, at 22808bbb2f) reported zero ThreadSanitizer +diagnostics. The only failure is `elapsed < 500ms` at +`moq-source-test.cpp:553` in the connect-failure case, and TSan's slowdown +is enough to break it. Per AGENTS.md, tests do not depend on wall-clock +timing: assert the ordering the bound stood in for (destroy returns after the +connect attempt is cancelled, not after it times out) through an event or a +mocked clock. + +Public API: none. Wire: none. diff --git a/quest/m2/README.md b/quest/m2/README.md index e058b9a8723..afbbbe73b49 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -21,6 +21,15 @@ lite-07; bitrate enforcement is not part of this release. ## Required - [Release execution](/quest/m2/release/README.md) - publication, installed-artifact proof, cleanup, and downstream adoption after m1 +- [Untrusted media parsers have fuzz targets](/quest/m2/media-fuzz.md) - fMP4, MKV, FLV, TS, Annex-B, catalog, and AMF0 parsing fuzzed with a bounded-work invariant +- [Wire fuzz coverage](/quest/m2/wire-fuzz-coverage.md) - the audited moq-net decoders have targets that assert an invariant, and the unsafe peer input reaches runs under Miri +- [In-memory unsafe runs under Miri](/quest/m2/miri-memory.md) - kio's and moq-net's memory-only unsafe runs under a nightly Miri job +- [VA-API renderer tests](/quest/m2/vaapi-render-tests.md) - the VA-API renderer, decode, and transcode tests run in `just rs gpu` on AMD and Intel and fail instead of skipping, and hosted CI reports them ignored +- [Every opt-in test target has a CI caller](/quest/m2/test-inventory.md) - the merge gate flags ignored tests, fuzz and bench targets, recipes, and scripts no workflow runs +- [Workflow and dependency security linting](/quest/m2/workflow-security.md) - zizmor gates workflow changes, and Nightly audits Go and Python dependencies +- [Worklet bundler matrix and React hooks run in CI](/quest/m2/browser-test-gaps.md) - the worklet matrix runs nightly on pinned bundlers, and the React signal hooks have tests +- [Every handwritten language has a format gate](/quest/m2/format-gates.md) - gofmt, ktfmt, swift-format, and clang-format join `just check`, and Ruff covers all Python +- [Weekly mutation testing on critical modules](/quest/m2/mutants.md) - cargo-mutants runs weekly on moq-auth and moq-net's permission, resume, and expiry code - [Video quality](/quest/m2/video-quality/README.md) - a nightly VMAF harness grades every encoder backend and on-demand GoP pops against floors, and moq-boy's encode is fixed under it - [noq adopts moq-congestion](/quest/m2/noq-congestion.md) - noq's BBR3 and Cubic delegate to the published crate behind noq's unchanged API - [Byte streams](/quest/m2/stream/README.md) - TCP connections ride MoQ losslessly through relays, proxied per hop rather than as IP packets @@ -67,7 +76,6 @@ lite-07; bitrate enforcement is not part of this release. - [CMAF surround Opus](/quest/m2/cmaf-opus-surround.md) - a released mp4-atom carries the `dOps` channel mapping table, and fMP4 import and export carry it through the OpusHead - [moqsrc surround Opus](/quest/m2/moqsrc-opus-surround.md) - moqsrc sets the Opus mapping caps, so opusdec plays 3 to 8 channels - [GPU CI](/quest/m2/gpu-ci.md) - NVIDIA tests gate the merge queue on a self-hosted GPU runner VM, and `just rs nvidia` runs them locally instead of skipping -- [VA-API renderer tests](/quest/m2/vaapi-render-tests.md) - the ignored VA-API renderer import tests run in `just rs gpu` on AMD and Intel and fail instead of skipping - [Vendor-free CARLA bridge](/quest/m2/carla-bridge.md) - the CARLA demo's GPU bridge lets moq-video pick the encoder, and its stand-in test runs on AMD as well as NVIDIA - [V4L2 virtual camera](/quest/m2/v4l2-virtual-camera.md) - V4L2 capture runs nightly against the `vivid` virtual driver instead of only on a developer's camera - [Probe](/quest/m2/probe/README.md) - a session probes up to the bitrate it wants to send or receive, so a stopped simulcast rendition resumes and a viewer up-switches; the browser pads where native sessions retransmit @@ -117,8 +125,6 @@ lite-07; bitrate enforcement is not part of this release. - [Bindings in the interop matrix](/quest/m2/interop-bindings.md) - Kotlin, Swift, and Dart publish and subscribe against every other interop client - [WebGPU HDR](/quest/m2/webgpu-hdr.md) - HDR renditions play as HDR where the browser and display can show it, and tone-map to SDR elsewhere - [HLS restart drain cutoff](/quest/m2/hls-restart-drain-cutoff.md) - a moq-hls recording of a replaced run finishes at a finite boundary while the successor starts at once -- [Wire fuzz coverage](/quest/m2/wire-fuzz-coverage.md) - the audited moq-net decoders have targets that assert an invariant, and the unsafe peer input reaches runs under Miri -- [Untrusted media parsers have fuzz targets](/quest/m2/media-fuzz.md) - fMP4, MKV, FLV, TS, Annex-B, catalog, and AMF0 parsing fuzzed with a bounded-work invariant - [Resume as producer replacement](/quest/m2/resume-redesign.md) - route changes replace the writer of one track state, deleting the per-reader resume path in Rust and JS - [moq-transport requests end through one lease](/quest/m2/ietf-request-lease.md) - every IETF request ends through one polled lease, and the two 18k-line files split by mechanism - [A moq-transport group head fills its hole in place](/quest/m2/ietf-fill-refill.md) - fills and joining FETCHes write into the tail's group, deleting the `Fill` handoff diff --git a/quest/m2/browser-test-gaps.md b/quest/m2/browser-test-gaps.md new file mode 100644 index 00000000000..6af0adda1b4 --- /dev/null +++ b/quest/m2/browser-test-gaps.md @@ -0,0 +1,26 @@ +# [S] Worklet bundler matrix and React hooks run in CI + +## Goal + +`just js worklet-matrix` runs nightly against pinned bundler versions, and +`@moq/signals/react`'s `useValue` and `useSignal` have tests. + +## Plan + +From the 2026-10-27 coverage audit. + +- `js/common/worklet-matrix.ts` checks that `?worklet` output survives Vite, + webpack, esbuild, and Bun consumers, including under a strict CSP, but says + "Not in CI". It also runs `bun add webpack webpack-cli` unpinned at run + time. Pin the bundlers through the repo's lockfile so a night's failure + means a change on our side or a deliberate bump, then add it to Nightly. +- `js/signals/src/react.ts` exports two hooks that no test exercises. Cover + mount and unmount, switching the signal a component reads, functional + updates through `useSignal`'s setter, and that unmount drops the + subscription. + +Public API: none. Wire: none. + +## Related + +- [Every opt-in test target has a CI caller](/quest/m2/test-inventory.md) - would flag the worklet matrix diff --git a/quest/m2/format-gates.md b/quest/m2/format-gates.md new file mode 100644 index 00000000000..ec692c13c34 --- /dev/null +++ b/quest/m2/format-gates.md @@ -0,0 +1,29 @@ +# [M] Every handwritten language has a format gate + +## Goal + +`just check` and `just fix` format and check Go, Kotlin, Swift, and the C/C++ +wrappers the way they already do Rust, TypeScript, shell, and Python, and Ruff +covers every Python file in the repo. Generated bindings stay excluded. + +## Plan + +From the 2026-10-27 coverage audit. On main at 8d2fa6653a, Go runs +`go vet` but no formatter (`gofmt -l go/wrapper` lists `client.go`, +`errors_test.go`, `audio/audio.go`, and `video/video.go`), Kotlin and Swift +only compile, clang-format runs only for `cpp/obs` (`sh/obs/check.sh`), and +the root Ruff config does not reach `sh/`, `test/`, or `bench/` Python. + +Decided 2026-10-27 (maintainer): + +- Pure formatters, like rustfmt and gofmt: `gofmt`, `ktfmt` (kotlinlang + style), Apple's `swift-format` (bundled with the Swift 6 toolchain as + `swift format`), and `clang-format` reusing OBS's config for `c/` and + `cpp/` wrappers. No style linters (ktlint, SwiftFormat rules) in this quest. +- Take each tool from the Nix dev shell so CI and local runs match. +- One commit per language: add the check and fix recipes, then the + reformatting churn. +- Extend Ruff's lint to the uncovered Python first, then its formatter, + each as its own commit. + +Public API: none. Wire: none. diff --git a/quest/m2/gpu-ci.md b/quest/m2/gpu-ci.md index 1f78d0ad6d1..47db9a5718f 100644 --- a/quest/m2/gpu-ci.md +++ b/quest/m2/gpu-ci.md @@ -19,7 +19,9 @@ skipping inside the Nix shell. and `safe::session::tests::start_session_refuses_held_frames` in `rs/moq-nvenc` need hardware and match none of them. Find them by their driver probes (`hw_available`, `driver_libs_present`, `Api::get` and - friends in `moq-nvenc` and `moq-video`). Recommendation: follow the + friends in `moq-nvenc` and `moq-video`, and `hardware_available` in + `moq-transcode`, whose `live_multi_rung_hardware` and + `end_to_end_hardware` pass on Linux without a GPU). Recommendation: follow the existing `#[ignore = "requires ..."]` convention (as `frame/vulkan_test.rs` does) and put them in `nvidia` test modules, so hosted CI reports them ignored instead of passed and one filter, `--run-ignored only -E diff --git a/quest/m2/miri-memory.md b/quest/m2/miri-memory.md new file mode 100644 index 00000000000..dff1868cc15 --- /dev/null +++ b/quest/m2/miri-memory.md @@ -0,0 +1,36 @@ +# [M] In-memory unsafe runs under Miri + +## Goal + +The hand-written unsafe in `kio` and `moq-net` that touches only memory (no +sockets, GPU, or FFI) runs under Miri nightly, so a use-after-free, data race, +or aliasing violation fails a test instead of waiting for Loom or production. + +## Plan + +From the 2026-10-27 coverage audit. No Miri job exists anywhere in the repo. +[Wire fuzz coverage](/quest/m2/wire-fuzz-coverage.md) covers the +peer-reachable unsafe in `moq-c`, `moq-uring`, and `moq-sock`; this quest +covers the in-memory primitives it leaves out. `kio` has 33 `unsafe` sites +and `moq-net` 21 (2026-10-27), and Loom flagged concurrent `UnsafeCell` +access in kio's `a_log_reader_racing_eviction_never_touches_freed_memory` +that night, so start with kio's log and channels. + +Decided 2026-10-27 (maintainer): + +- Nightly only, never the merge gate: Miri is orders of magnitude slower than + native tests. +- Run kio's unit tests whole, and the `moq-net` tests that reach its unsafe, + selected by filter. Skip a test Miri cannot run (foreign calls, real + sockets, wall-clock time) with `#[cfg_attr(miri, ignore = "...")]` naming + why, not by narrowing the filter silently. +- Share one `just rs miri` recipe and nightly job with wire fuzz coverage: + whichever lands first creates it, the other adds crates. +- Fix what it finds at the cause. + +Public API: none. Wire: none. + +## Related + +- [Wire fuzz coverage](/quest/m2/wire-fuzz-coverage.md) - Miri for the peer-reachable unsafe +- [Nightly passes on main](/quest/m0/nightly/README.md) - the nightly this job joins diff --git a/quest/m2/mutants.md b/quest/m2/mutants.md new file mode 100644 index 00000000000..6eb62d156e6 --- /dev/null +++ b/quest/m2/mutants.md @@ -0,0 +1,27 @@ +# [M] Weekly mutation testing on critical modules + +## Goal + +A weekly scheduled job runs cargo-mutants on `moq-auth` and on `moq-net`'s +path permission, resume, and expiry code, and reports surviving mutants, +so assertions that never notice a behavior change get found. + +## Plan + +From the 2026-10-27 coverage audit. The failure drills already use +mutation-style negative controls; nothing does it systematically. + +Decided 2026-10-27 (maintainer): + +- Weekly, not nightly: the scoped code changes slowly and each mutant + rebuilds and reruns its crate's tests. +- Scope to the modules above with cargo-mutants' file filters, and bound each + mutant with its timeout so one hung mutant cannot eat the job. +- Survivors go to the job summary, and a run with survivors alerts through + `alert.yml`. +- No mutation-score gate. The PR that adds the job fixes or annotates the + first run's survivors: add a test where one is worth killing, or exclude the + mutant in cargo-mutants' config with the reason (an equivalent mutant, or + logging only). + +Public API: none. Wire: none. diff --git a/quest/m2/test-inventory.md b/quest/m2/test-inventory.md new file mode 100644 index 00000000000..82154370ee9 --- /dev/null +++ b/quest/m2/test-inventory.md @@ -0,0 +1,32 @@ +# [M] Every opt-in test target has a CI caller + +## Goal + +The merge gate fails when a test that CI does not pick up by default has no +CI caller, so AGENTS.md's "every test runs in CI, at least nightly" is +enforced instead of rediscovered by each coverage audit. + +## Plan + +From the 2026-10-27 coverage audit. nextest and `bun test` find ordinary tests +by themselves; the gaps are the opt-in ones. The m1 +[coverage audit](/quest/m2/audit-test-coverage.md) found uncalled VA-API +tests by hand, and `js/common/worklet-matrix.ts` still says "Not in CI". + +Decided 2026-10-27 (maintainer): check the opt-in targets, not everything. + +- Every `#[ignore]` test is matched by some recipe's `--run-ignored` filter. +- Every fuzz target and every bench target runs in some workflow. +- Every `just` recipe that runs tests is reachable from a workflow. +- Every standalone test script (like `worklet-matrix.ts`) has a caller. + +It runs in the merge gate beside the CI-selection tests, and reports each +orphan by name with where it was found. An intentionally local-only target +needs an explicit allowlist entry that names its owning quest. After this +lands, the coverage audit's `#[ignore]` walk can rely on the check. + +Public API: none. Wire: none. + +## Related + +- [Test coverage audit](/quest/m2/audit-test-coverage.md) - the recurring audit this automates part of diff --git a/quest/m2/vaapi-render-tests.md b/quest/m2/vaapi-render-tests.md index b8bbd5de68b..4cfd598d698 100644 --- a/quest/m2/vaapi-render-tests.md +++ b/quest/m2/vaapi-render-tests.md @@ -1,4 +1,4 @@ -# [S] VA-API renderer tests +# [M] VA-API renderer tests ## Goal @@ -8,7 +8,11 @@ The four ignored VA-API renderer import tests in `decoded_frames_reach_the_gpu_without_a_download`, and `recycled_decoder_surfaces_draw_the_latest_picture`) and `packed_dmabuf_renders_through_vulkan` run in `just rs gpu` on AMD and Intel -hosts and fail there instead of skipping. +hosts and fail there instead of skipping. So do the VA-API decode tests that +return early without a device today (`vaapi_h264_round_trip` and +`native_frames_still_answer_into_i420` in +`rs/moq-video/src/decode/backend/vaapi.rs`) and moq-transcode's +`vaapi_decoder_available` tests, which hosted CI reports as passed. ## Plan @@ -30,6 +34,11 @@ which covers only NVIDIA. if the recipe cannot provide one, leave it to [Video hardware validation](/quest/m3/video-hardware.md) and say so in its reason. +- Added 2026-10-27 (maintainer) from the coverage audit: convert the + early-return VA-API decode and transcode tests to + `#[ignore = "requires ..."]`, as [GPU CI](/quest/m2/gpu-ci.md) does for + NVIDIA's `hw_available`, so hosted CI reports them ignored and this + recipe's filter selects them. - No hosted runner has these GPUs, so this stays a local recipe until a GPU runner exists ([GPU CI](/quest/m2/gpu-ci.md) owns that decision). diff --git a/quest/m2/workflow-security.md b/quest/m2/workflow-security.md new file mode 100644 index 00000000000..067c152463f --- /dev/null +++ b/quest/m2/workflow-security.md @@ -0,0 +1,27 @@ +# [S] Workflow and dependency security linting + +## Goal + +The merge gate fails on a new zizmor finding in `.github/workflows/`, and +Nightly audits the Go and Python dependencies the way it already audits Rust +(`cargo deny`) and Bun. + +## Plan + +From the 2026-10-27 coverage audit. actionlint already runs; nothing checks +workflow security. An offline zizmor run on 82f7159bb2 gave 230 diagnostics, +182 of them about local-action references, so most are noise to configure +away, not vulnerabilities. Worth fixing on review: explicit permission scopes +for the release App token and narrower workflow-level OIDC permissions. + +Decided 2026-10-27 (maintainer): + +- zizmor is offline and fast, so it runs in the merge gate. Review every + finding once: fix it, or record it in zizmor's config with the reason (the + privileged triggers and caches several workflows already justify in + comments). Only new findings fail. +- `govulncheck` on the staged Go modules and `pip-audit` on the resolved + Python environment need advisory databases, so they run nightly beside + `cargo deny` and alert through `alert.yml`. + +Public API: none. Wire: none.