From bb004fd418a406b5cb43c35f64a2f3cbc21520c7 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 27 Oct 2026 22:13:57 -0700 Subject: [PATCH 1/2] docs(quest): plan the coverage audit's findings Six Nightly children own the 2026-10-27 failures nothing owned (Loom on kio's log, render-sw, OBS TSan, paired bench regressions, browser legs, and three mechanical breaks). Six m2 quests add Miri for in-memory unsafe, a test-inventory gate, workflow and dependency security linting, the worklet matrix and React hooks tests, format gates for every language, and weekly mutation testing. VA-API and GPU CI absorb the remaining silent hardware passes, and the fuzz quests move up in m2. Co-Authored-By: Claude Opus 5.5 --- quest/m0/nightly/README.md | 11 ++++++++ quest/m0/nightly/bench-regressions.md | 24 ++++++++++++++++++ quest/m0/nightly/browser-legs.md | 27 ++++++++++++++++++++ quest/m0/nightly/loom-log.md | 25 +++++++++++++++++++ quest/m0/nightly/obs-tsan.md | 18 ++++++++++++++ quest/m0/nightly/render-sw.md | 20 +++++++++++++++ quest/m0/nightly/small-breaks.md | 26 +++++++++++++++++++ quest/m2/README.md | 12 ++++++--- quest/m2/browser-test-gaps.md | 26 +++++++++++++++++++ quest/m2/format-gates.md | 29 +++++++++++++++++++++ quest/m2/gpu-ci.md | 4 ++- quest/m2/miri-memory.md | 36 +++++++++++++++++++++++++++ quest/m2/mutants.md | 27 ++++++++++++++++++++ quest/m2/test-inventory.md | 32 ++++++++++++++++++++++++ quest/m2/vaapi-render-tests.md | 13 ++++++++-- quest/m2/workflow-security.md | 31 +++++++++++++++++++++++ 16 files changed, 355 insertions(+), 6 deletions(-) create mode 100644 quest/m0/nightly/bench-regressions.md create mode 100644 quest/m0/nightly/browser-legs.md create mode 100644 quest/m0/nightly/loom-log.md create mode 100644 quest/m0/nightly/obs-tsan.md create mode 100644 quest/m0/nightly/render-sw.md create mode 100644 quest/m0/nightly/small-breaks.md create mode 100644 quest/m2/browser-test-gaps.md create mode 100644 quest/m2/format-gates.md create mode 100644 quest/m2/miri-memory.md create mode 100644 quest/m2/mutants.md create mode 100644 quest/m2/test-inventory.md create mode 100644 quest/m2/workflow-security.md diff --git a/quest/m0/nightly/README.md b/quest/m0/nightly/README.md index a153222320e..cf09a2d56c2 100644 --- a/quest/m0/nightly/README.md +++ b/quest/m0/nightly/README.md @@ -16,6 +16,11 @@ Decided 2026-10-24 (maintainer), from a triage of run child per distinct cause, each landing on its own. The PR gate keeps skipping benches and examples; a green Nightly catches them within a day. +Decided 2026-10-27 (maintainer), from the coverage audit's triage of run +[40506938673](https://github.com/moq-dev/moq/actions/runs/40506938673): six +more children for the causes nothing owned. The three one-line breaks share +one child rather than one each, and TS's stale env var rides along with them. + This README's own work, after the children: - Nightly's multi-step jobs (Features first) keep going past a failed step @@ -31,6 +36,12 @@ Public API: none. Wire: none. ## Required +- [kio's log survives Loom's eviction model](/quest/m0/nightly/loom-log.md) - the Loom causality violation in kio's log reader is fixed at its cause, possibly a use-after-free - [moq-net compiles in Fuzz and send-depth](/quest/m0/nightly/send-overflow.md) - the ietf publisher future stops overflowing the Send check - [Probing beside Cubic](/quest/m0/nightly/competing-cubic.md) - the `competing_cubic` probe test stays within its deadline budget - [cargo deny passes](/quest/m0/nightly/cargo-deny.md) - Audit stops failing on the yanked crypto-bigint and moq-studio's egui licenses +- [Three mechanical scheduled-run breaks](/quest/m0/nightly/small-breaks.md) - Features' dead-code `refuse`, the stale drill mutation, and TS's stale `MOQ_TEST_RUNS` +- [Software rendering runs every group](/quest/m0/nightly/render-sw.md) - the DMA-BUF assertion stops failing render-sw's first group under software Vulkan +- [OBS TSan passes without a wall-clock bound](/quest/m0/nightly/obs-tsan.md) - `destroySource` asserts ordering instead of `elapsed < 500ms` +- [Paired bench regressions bisected](/quest/m0/nightly/bench-regressions.md) - retirement/selection's +21-24% and two 10% rows fixed or shown to be noise +- [Nightly browser legs pass](/quest/m0/nightly/browser-legs.md) - serve-bench's 0/1024 streams, wasm-baseline's missing frame, and Firefox's unclosed VideoFrame diff --git a/quest/m0/nightly/bench-regressions.md b/quest/m0/nightly/bench-regressions.md new file mode 100644 index 00000000000..fa4b6ec4a42 --- /dev/null +++ b/quest/m0/nightly/bench-regressions.md @@ -0,0 +1,24 @@ +# [M] Paired bench regressions bisected + +## Goal + +Each regression the paired Rust comparison confirmed on 2026-10-27 is fixed at +its cause or shown to be noise by an A/A rerun, and Nightly's Rust bench +targets job passes. + +## Plan + +Nightly [40506938673](https://github.com/moq-dev/moq/actions/runs/40506938673) (2026-10-27, at 22808bbb2f) confirmed five regressions: +retirement/selection at 8x32, 32x32, and 128x32 (+21 to 24%), +server_admission/workers=4 (+10%), and track_range_demand/1000r (+10%). +The retirement/selection rows agree across three sizes, so treat them as +real: bisect with the paired base across the previous night's commits. The two +10% rows sit at the threshold that [bench CI](/quest/m1/bench-ci.md) has yet +to calibrate; rerun them before bisecting. bench-ci owns the threshold, this +quest owns the regressions. + +Public API: none. Wire: none. + +## Related + +- [Calibrate the paired nightly benchmarks](/quest/m1/bench-ci.md) - the threshold these crossed diff --git a/quest/m0/nightly/browser-legs.md b/quest/m0/nightly/browser-legs.md new file mode 100644 index 00000000000..64cd098bf63 --- /dev/null +++ b/quest/m0/nightly/browser-legs.md @@ -0,0 +1,27 @@ +# [M] Nightly browser legs pass + +## Goal + +Nightly's browser publisher fairness benchmark, browser backend baseline, and +Firefox player test pass. + +## Plan + +Three unowned failures in Nightly [40506938673](https://github.com/moq-dev/moq/actions/runs/40506938673) (2026-10-27, at 22808bbb2f), possibly +one cause: + +- The Audio quality job's "Browser publisher fairness benchmark" step + (`js/net/bench/serve-browser-entry.ts`) fails with "only 0/1024 streams". +- The browser backend baseline (`wasm-baseline`) fails with "missing WASM + frame 0". +- The Media job's Firefox player fails on the console error "A VideoFrame + was garbage collected without being closed". + [Firefox publishing](/quest/m2/publish-firefox.md) covers only the + publisher side. + +Triage together first: if the first two share a cause, fix it once. If the +causes differ and one fix is large, split it into its own child of this epic. +The unclosed `VideoFrame` is a leak in the player, not a console-noise +filter to add. + +Public API: none. Wire: none. diff --git a/quest/m0/nightly/loom-log.md b/quest/m0/nightly/loom-log.md new file mode 100644 index 00000000000..9e73acfd5a8 --- /dev/null +++ b/quest/m0/nightly/loom-log.md @@ -0,0 +1,25 @@ +# [S] kio's log survives Loom's eviction model + +## Goal + +`just rs loom` passes `a_log_reader_racing_eviction_never_touches_freed_memory` +in `rs/kio/src/loom.rs`, with the bug it found fixed at its cause, whether +that is the log or the model's instrumentation. + +## Plan + +Nightly [40506938673](https://github.com/moq-dev/moq/actions/runs/40506938673) (2026-10-27, at 22808bbb2f) panics with "Causality violation: +Concurrent read and write accesses to `UnsafeCell`", then aborts on a panic +in a destructor. The model came with the lock-free log in +[moq-dev/moq#6782](https://github.com/moq-dev/moq/pull/6782). Treat it as a +possible use-after-free in `rs/kio/src/log.rs` until shown otherwise: a +reader's hint into a segment the collector frees is the case the model was +written to catch. Reproduce from the failing interleaving, then decide from +the trace whether the log orders the read before the free. Never loosen the +model or its preemption bound to pass. + +Public API: none. Wire: none. + +## Related + +- [In-memory unsafe runs under Miri](/quest/m2/miri-memory.md) - Miri over the same log diff --git a/quest/m0/nightly/obs-tsan.md b/quest/m0/nightly/obs-tsan.md new file mode 100644 index 00000000000..f49be61a63c --- /dev/null +++ b/quest/m0/nightly/obs-tsan.md @@ -0,0 +1,18 @@ +# [S] OBS TSan passes without a wall-clock bound + +## Goal + +Nightly's OBS TSan job passes, and `destroySource` in +`cpp/obs/test/moq-source-test.cpp` no longer asserts elapsed wall time. + +## Plan + +Nightly [40506938673](https://github.com/moq-dev/moq/actions/runs/40506938673) (2026-10-27, at 22808bbb2f) reported zero ThreadSanitizer +diagnostics. The only failure is `elapsed < 500ms` at +`moq-source-test.cpp:553` in the connect-failure case, and TSan's slowdown +is enough to break it. Per AGENTS.md, tests do not depend on wall-clock +timing: assert the ordering the bound stood in for (destroy returns after the +connect attempt is cancelled, not after it times out) through an event or a +mocked clock. + +Public API: none. Wire: none. diff --git a/quest/m0/nightly/render-sw.md b/quest/m0/nightly/render-sw.md new file mode 100644 index 00000000000..f0bdcc44f6f --- /dev/null +++ b/quest/m0/nightly/render-sw.md @@ -0,0 +1,20 @@ +# [S] Software rendering runs every group + +## Goal + +Nightly's Tests (rs render-sw) passes all three of its groups under software +Vulkan, so the export, scene, and studio checks after the first one run again. + +## Plan + +Nightly [40506938673](https://github.com/moq-dev/moq/actions/runs/40506938673) (2026-10-27, at 22808bbb2f) fails the first group on +`the_output_follows_the_device` (`rs/moq-video/src/render/renderer.rs:528`, +from [moq-dev/moq#6806](https://github.com/moq-dev/moq/pull/6806)), which +asserts "the adapter imports DMA-BUFs". Software Vulkan cannot, so the +assertion belongs only where the device can: make the test check what the +device reports it supports, or mark the DMA-BUF half with +`#[ignore = "requires ..."]` and let `just rs gpu` select it. The recipe's +later groups must not depend on the first passing (see this epic's +continue-past-failure work). + +Public API: none. Wire: none. diff --git a/quest/m0/nightly/small-breaks.md b/quest/m0/nightly/small-breaks.md new file mode 100644 index 00000000000..2a301ceaa16 --- /dev/null +++ b/quest/m0/nightly/small-breaks.md @@ -0,0 +1,26 @@ +# [S] Three mechanical scheduled-run breaks + +## Goal + +Nightly's Features and drill-sensitivity steps and TS's MPEG-2 video step get +past their mechanical failures and run their real checks. + +## Plan + +All found in the 2026-10-27 scheduled runs (Nightly [40506938673](https://github.com/moq-dev/moq/actions/runs/40506938673) at 22808bbb2f, TS +[40515802751](https://github.com/moq-dev/moq/actions/runs/40515802751)): + +- Features: `-D dead-code` rejects `Incoming::refuse` + (`rs/moq-tokio/src/quic/driver/incoming.rs:30`), which only the server and + tests call, so the driver-only feature build sees it unused. Gate it on the + feature that uses it, rather than allowing dead code. +- drill-sensitivity: `test/drill/mutations/fetch-never-sent.patch` no + longer compiles (E0425 `request`), so `test/drill/sensitivity.sh` reports + that it "broke the build". Re-anchor the patch to the current code so it + still breaks the behavior the drill expects to catch. +- TS: `.github/workflows/ts.yml`'s MPEG-2 step sets `MOQ_TEST_RUNS`, which + the relay refuses as an undeclared `MOQ_*` variable since + `check_env`. Rename it to the variable the harness reads + (`HARNESS_RUNS`), and grep the workflows for any other stale `MOQ_*`. + +Public API: none. Wire: none. diff --git a/quest/m2/README.md b/quest/m2/README.md index 67de43cc562..0d0ecf922e1 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -21,6 +21,15 @@ lite-07; bitrate enforcement is not part of this release. ## Required - [Release execution](/quest/m2/release/README.md) - publication, installed-artifact proof, cleanup, and downstream adoption after m1 +- [Untrusted media parsers have fuzz targets](/quest/m2/media-fuzz.md) - fMP4, MKV, FLV, TS, Annex-B, catalog, and AMF0 parsing fuzzed with a bounded-work invariant +- [Wire fuzz coverage](/quest/m2/wire-fuzz-coverage.md) - the audited moq-net decoders have targets that assert an invariant, and the unsafe peer input reaches runs under Miri +- [In-memory unsafe runs under Miri](/quest/m2/miri-memory.md) - kio's and moq-net's memory-only unsafe runs under a nightly Miri job +- [VA-API renderer tests](/quest/m2/vaapi-render-tests.md) - the VA-API renderer, decode, and transcode tests run in `just rs gpu` on AMD and Intel and fail instead of skipping, and hosted CI reports them ignored +- [Every opt-in test target has a CI caller](/quest/m2/test-inventory.md) - the merge gate flags ignored tests, fuzz and bench targets, recipes, and scripts no workflow runs +- [Workflow and dependency security linting](/quest/m2/workflow-security.md) - zizmor gates workflow changes, and Nightly audits Go and Python dependencies +- [Worklet bundler matrix and React hooks run in CI](/quest/m2/browser-test-gaps.md) - the worklet matrix runs nightly on pinned bundlers, and the React signal hooks have tests +- [Every handwritten language has a format gate](/quest/m2/format-gates.md) - gofmt, ktfmt, swift-format, and clang-format join `just check`, and Ruff covers all Python +- [Weekly mutation testing on critical modules](/quest/m2/mutants.md) - cargo-mutants runs weekly on moq-auth and moq-net's permission, resume, and expiry code - [noq adopts moq-congestion](/quest/m2/noq-congestion.md) - noq's BBR3 and Cubic delegate to the published crate behind noq's unchanged API - [Byte streams](/quest/m2/stream/README.md) - TCP connections ride MoQ losslessly through relays, proxied per hop rather than as IP packets - [Mesh](/quest/m2/mesh/README.md) - `moq mesh` enrolls a host or robot and reaches any node by endpoint ID or name for SSH, forwards, and agents @@ -63,7 +72,6 @@ lite-07; bitrate enforcement is not part of this release. - [CMAF surround Opus](/quest/m2/cmaf-opus-surround.md) - a released mp4-atom carries the `dOps` channel mapping table, and fMP4 import and export carry it through the OpusHead - [moqsrc surround Opus](/quest/m2/moqsrc-opus-surround.md) - moqsrc sets the Opus mapping caps, so opusdec plays 3 to 8 channels - [GPU CI](/quest/m2/gpu-ci.md) - NVIDIA tests gate the merge queue on a self-hosted GPU runner VM, and `just rs nvidia` runs them locally instead of skipping -- [VA-API renderer tests](/quest/m2/vaapi-render-tests.md) - the ignored VA-API renderer import tests run in `just rs gpu` on AMD and Intel and fail instead of skipping - [Vendor-free CARLA bridge](/quest/m2/carla-bridge.md) - the CARLA demo's GPU bridge lets moq-video pick the encoder, and its stand-in test runs on AMD as well as NVIDIA - [V4L2 virtual camera](/quest/m2/v4l2-virtual-camera.md) - V4L2 capture runs nightly against the `vivid` virtual driver instead of only on a developer's camera - [Probe](/quest/m2/probe/README.md) - a session probes up to the bitrate it wants to send or receive, so a stopped simulcast rendition resumes and a viewer up-switches; the browser pads where native sessions retransmit @@ -113,8 +121,6 @@ lite-07; bitrate enforcement is not part of this release. - [Bindings in the interop matrix](/quest/m2/interop-bindings.md) - Kotlin, Swift, and Dart publish and subscribe against every other interop client - [WebGPU HDR](/quest/m2/webgpu-hdr.md) - HDR renditions play as HDR where the browser and display can show it, and tone-map to SDR elsewhere - [HLS restart drain cutoff](/quest/m2/hls-restart-drain-cutoff.md) - a moq-hls recording of a replaced run finishes at a finite boundary while the successor starts at once -- [Wire fuzz coverage](/quest/m2/wire-fuzz-coverage.md) - the audited moq-net decoders have targets that assert an invariant, and the unsafe peer input reaches runs under Miri -- [Untrusted media parsers have fuzz targets](/quest/m2/media-fuzz.md) - fMP4, MKV, FLV, TS, Annex-B, catalog, and AMF0 parsing fuzzed with a bounded-work invariant - [Resume as producer replacement](/quest/m2/resume-redesign.md) - route changes replace the writer of one track state, deleting the per-reader resume path in Rust and JS - [moq-transport requests end through one lease](/quest/m2/ietf-request-lease.md) - every IETF request ends through one polled lease, and the two 18k-line files split by mechanism - [A moq-transport group head fills its hole in place](/quest/m2/ietf-fill-refill.md) - fills and joining FETCHes write into the tail's group, deleting the `Fill` handoff diff --git a/quest/m2/browser-test-gaps.md b/quest/m2/browser-test-gaps.md new file mode 100644 index 00000000000..6af0adda1b4 --- /dev/null +++ b/quest/m2/browser-test-gaps.md @@ -0,0 +1,26 @@ +# [S] Worklet bundler matrix and React hooks run in CI + +## Goal + +`just js worklet-matrix` runs nightly against pinned bundler versions, and +`@moq/signals/react`'s `useValue` and `useSignal` have tests. + +## Plan + +From the 2026-10-27 coverage audit. + +- `js/common/worklet-matrix.ts` checks that `?worklet` output survives Vite, + webpack, esbuild, and Bun consumers, including under a strict CSP, but says + "Not in CI". It also runs `bun add webpack webpack-cli` unpinned at run + time. Pin the bundlers through the repo's lockfile so a night's failure + means a change on our side or a deliberate bump, then add it to Nightly. +- `js/signals/src/react.ts` exports two hooks that no test exercises. Cover + mount and unmount, switching the signal a component reads, functional + updates through `useSignal`'s setter, and that unmount drops the + subscription. + +Public API: none. Wire: none. + +## Related + +- [Every opt-in test target has a CI caller](/quest/m2/test-inventory.md) - would flag the worklet matrix diff --git a/quest/m2/format-gates.md b/quest/m2/format-gates.md new file mode 100644 index 00000000000..ec692c13c34 --- /dev/null +++ b/quest/m2/format-gates.md @@ -0,0 +1,29 @@ +# [M] Every handwritten language has a format gate + +## Goal + +`just check` and `just fix` format and check Go, Kotlin, Swift, and the C/C++ +wrappers the way they already do Rust, TypeScript, shell, and Python, and Ruff +covers every Python file in the repo. Generated bindings stay excluded. + +## Plan + +From the 2026-10-27 coverage audit. On main at 8d2fa6653a, Go runs +`go vet` but no formatter (`gofmt -l go/wrapper` lists `client.go`, +`errors_test.go`, `audio/audio.go`, and `video/video.go`), Kotlin and Swift +only compile, clang-format runs only for `cpp/obs` (`sh/obs/check.sh`), and +the root Ruff config does not reach `sh/`, `test/`, or `bench/` Python. + +Decided 2026-10-27 (maintainer): + +- Pure formatters, like rustfmt and gofmt: `gofmt`, `ktfmt` (kotlinlang + style), Apple's `swift-format` (bundled with the Swift 6 toolchain as + `swift format`), and `clang-format` reusing OBS's config for `c/` and + `cpp/` wrappers. No style linters (ktlint, SwiftFormat rules) in this quest. +- Take each tool from the Nix dev shell so CI and local runs match. +- One commit per language: add the check and fix recipes, then the + reformatting churn. +- Extend Ruff's lint to the uncovered Python first, then its formatter, + each as its own commit. + +Public API: none. Wire: none. diff --git a/quest/m2/gpu-ci.md b/quest/m2/gpu-ci.md index 339782be695..14c38a11cb2 100644 --- a/quest/m2/gpu-ci.md +++ b/quest/m2/gpu-ci.md @@ -19,7 +19,9 @@ skipping inside the Nix shell. and `safe::session::tests::start_session_refuses_held_frames` in `rs/moq-nvenc` need hardware and match none of them. Find them by their driver probes (`hw_available`, `driver_libs_present`, `Api::get` and - friends in `moq-nvenc` and `moq-video`). Recommendation: follow the + friends in `moq-nvenc` and `moq-video`, and `hardware_available` in + `moq-transcode`, whose `live_multi_rung_hardware` and + `end_to_end_hardware` pass on Linux without a GPU). Recommendation: follow the existing `#[ignore = "requires ..."]` convention (as `frame/vulkan_test.rs` does) and put them in `nvidia` test modules, so hosted CI reports them ignored instead of passed and one filter, `--run-ignored only -E diff --git a/quest/m2/miri-memory.md b/quest/m2/miri-memory.md new file mode 100644 index 00000000000..dff1868cc15 --- /dev/null +++ b/quest/m2/miri-memory.md @@ -0,0 +1,36 @@ +# [M] In-memory unsafe runs under Miri + +## Goal + +The hand-written unsafe in `kio` and `moq-net` that touches only memory (no +sockets, GPU, or FFI) runs under Miri nightly, so a use-after-free, data race, +or aliasing violation fails a test instead of waiting for Loom or production. + +## Plan + +From the 2026-10-27 coverage audit. No Miri job exists anywhere in the repo. +[Wire fuzz coverage](/quest/m2/wire-fuzz-coverage.md) covers the +peer-reachable unsafe in `moq-c`, `moq-uring`, and `moq-sock`; this quest +covers the in-memory primitives it leaves out. `kio` has 33 `unsafe` sites +and `moq-net` 21 (2026-10-27), and Loom flagged concurrent `UnsafeCell` +access in kio's `a_log_reader_racing_eviction_never_touches_freed_memory` +that night, so start with kio's log and channels. + +Decided 2026-10-27 (maintainer): + +- Nightly only, never the merge gate: Miri is orders of magnitude slower than + native tests. +- Run kio's unit tests whole, and the `moq-net` tests that reach its unsafe, + selected by filter. Skip a test Miri cannot run (foreign calls, real + sockets, wall-clock time) with `#[cfg_attr(miri, ignore = "...")]` naming + why, not by narrowing the filter silently. +- Share one `just rs miri` recipe and nightly job with wire fuzz coverage: + whichever lands first creates it, the other adds crates. +- Fix what it finds at the cause. + +Public API: none. Wire: none. + +## Related + +- [Wire fuzz coverage](/quest/m2/wire-fuzz-coverage.md) - Miri for the peer-reachable unsafe +- [Nightly passes on main](/quest/m0/nightly/README.md) - the nightly this job joins diff --git a/quest/m2/mutants.md b/quest/m2/mutants.md new file mode 100644 index 00000000000..6eb62d156e6 --- /dev/null +++ b/quest/m2/mutants.md @@ -0,0 +1,27 @@ +# [M] Weekly mutation testing on critical modules + +## Goal + +A weekly scheduled job runs cargo-mutants on `moq-auth` and on `moq-net`'s +path permission, resume, and expiry code, and reports surviving mutants, +so assertions that never notice a behavior change get found. + +## Plan + +From the 2026-10-27 coverage audit. The failure drills already use +mutation-style negative controls; nothing does it systematically. + +Decided 2026-10-27 (maintainer): + +- Weekly, not nightly: the scoped code changes slowly and each mutant + rebuilds and reruns its crate's tests. +- Scope to the modules above with cargo-mutants' file filters, and bound each + mutant with its timeout so one hung mutant cannot eat the job. +- Survivors go to the job summary, and a run with survivors alerts through + `alert.yml`. +- No mutation-score gate. The PR that adds the job fixes or annotates the + first run's survivors: add a test where one is worth killing, or exclude the + mutant in cargo-mutants' config with the reason (an equivalent mutant, or + logging only). + +Public API: none. Wire: none. diff --git a/quest/m2/test-inventory.md b/quest/m2/test-inventory.md new file mode 100644 index 00000000000..82154370ee9 --- /dev/null +++ b/quest/m2/test-inventory.md @@ -0,0 +1,32 @@ +# [M] Every opt-in test target has a CI caller + +## Goal + +The merge gate fails when a test that CI does not pick up by default has no +CI caller, so AGENTS.md's "every test runs in CI, at least nightly" is +enforced instead of rediscovered by each coverage audit. + +## Plan + +From the 2026-10-27 coverage audit. nextest and `bun test` find ordinary tests +by themselves; the gaps are the opt-in ones. The m1 +[coverage audit](/quest/m2/audit-test-coverage.md) found uncalled VA-API +tests by hand, and `js/common/worklet-matrix.ts` still says "Not in CI". + +Decided 2026-10-27 (maintainer): check the opt-in targets, not everything. + +- Every `#[ignore]` test is matched by some recipe's `--run-ignored` filter. +- Every fuzz target and every bench target runs in some workflow. +- Every `just` recipe that runs tests is reachable from a workflow. +- Every standalone test script (like `worklet-matrix.ts`) has a caller. + +It runs in the merge gate beside the CI-selection tests, and reports each +orphan by name with where it was found. An intentionally local-only target +needs an explicit allowlist entry that names its owning quest. After this +lands, the coverage audit's `#[ignore]` walk can rely on the check. + +Public API: none. Wire: none. + +## Related + +- [Test coverage audit](/quest/m2/audit-test-coverage.md) - the recurring audit this automates part of diff --git a/quest/m2/vaapi-render-tests.md b/quest/m2/vaapi-render-tests.md index b8bbd5de68b..4cfd598d698 100644 --- a/quest/m2/vaapi-render-tests.md +++ b/quest/m2/vaapi-render-tests.md @@ -1,4 +1,4 @@ -# [S] VA-API renderer tests +# [M] VA-API renderer tests ## Goal @@ -8,7 +8,11 @@ The four ignored VA-API renderer import tests in `decoded_frames_reach_the_gpu_without_a_download`, and `recycled_decoder_surfaces_draw_the_latest_picture`) and `packed_dmabuf_renders_through_vulkan` run in `just rs gpu` on AMD and Intel -hosts and fail there instead of skipping. +hosts and fail there instead of skipping. So do the VA-API decode tests that +return early without a device today (`vaapi_h264_round_trip` and +`native_frames_still_answer_into_i420` in +`rs/moq-video/src/decode/backend/vaapi.rs`) and moq-transcode's +`vaapi_decoder_available` tests, which hosted CI reports as passed. ## Plan @@ -30,6 +34,11 @@ which covers only NVIDIA. if the recipe cannot provide one, leave it to [Video hardware validation](/quest/m3/video-hardware.md) and say so in its reason. +- Added 2026-10-27 (maintainer) from the coverage audit: convert the + early-return VA-API decode and transcode tests to + `#[ignore = "requires ..."]`, as [GPU CI](/quest/m2/gpu-ci.md) does for + NVIDIA's `hw_available`, so hosted CI reports them ignored and this + recipe's filter selects them. - No hosted runner has these GPUs, so this stays a local recipe until a GPU runner exists ([GPU CI](/quest/m2/gpu-ci.md) owns that decision). diff --git a/quest/m2/workflow-security.md b/quest/m2/workflow-security.md new file mode 100644 index 00000000000..b78ae47a3cc --- /dev/null +++ b/quest/m2/workflow-security.md @@ -0,0 +1,31 @@ +# [S] Workflow and dependency security linting + +## Goal + +The merge gate fails on a new zizmor finding in `.github/workflows/`, and +Nightly audits the Go and Python dependencies the way it already audits Rust +(`cargo deny`) and Bun. + +## Plan + +From the 2026-10-27 coverage audit. actionlint already runs; nothing checks +workflow security. An offline zizmor run on 82f7159bb2 gave 230 diagnostics, +182 of them about local-action references, so most are noise to configure +away, not vulnerabilities. Worth fixing on review: explicit permission scopes +for the release App token and narrower workflow-level OIDC permissions. + +Decided 2026-10-27 (maintainer): + +- zizmor is offline and fast, so it runs in the merge gate. Review every + finding once: fix it, or record it in zizmor's config with the reason (the + privileged triggers and caches several workflows already justify in + comments). Only new findings fail. +- `govulncheck` on the staged Go modules and `pip-audit` on the resolved + Python environment need advisory databases, so they run nightly beside + `cargo deny` and alert through `alert.yml`. + +Public API: none. Wire: none. + +## Related + +- [cargo deny passes](/quest/m0/nightly/cargo-deny.md) - the Rust audit these join From 4052518b66a4031280aa53c433bc1934ac5ec4a9 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 29 Oct 2026 08:25:02 -0700 Subject: [PATCH 2/2] Merge main and drop nightly children main already owns Main fixed the Loom log race (#7134) and the three mechanical breaks (#7047, #7120, #7121), and #7119 planned render-capabilities and browser-baseline, so remove loom-log, render-sw, and small-breaks, and narrow browser-legs to the Firefox VideoFrame leak. Drop the Related link to the finished cargo-deny quest. Co-Authored-By: Claude Opus 5.5 --- quest/m0/nightly/README.md | 10 +++------- quest/m0/nightly/browser-legs.md | 27 -------------------------- quest/m0/nightly/firefox-videoframe.md | 16 +++++++++++++++ quest/m0/nightly/loom-log.md | 25 ------------------------ quest/m0/nightly/render-sw.md | 20 ------------------- quest/m0/nightly/small-breaks.md | 26 ------------------------- quest/m2/workflow-security.md | 4 ---- 7 files changed, 19 insertions(+), 109 deletions(-) delete mode 100644 quest/m0/nightly/browser-legs.md create mode 100644 quest/m0/nightly/firefox-videoframe.md delete mode 100644 quest/m0/nightly/loom-log.md delete mode 100644 quest/m0/nightly/render-sw.md delete mode 100644 quest/m0/nightly/small-breaks.md diff --git a/quest/m0/nightly/README.md b/quest/m0/nightly/README.md index 6aedc979311..c80c7fb2227 100644 --- a/quest/m0/nightly/README.md +++ b/quest/m0/nightly/README.md @@ -17,9 +17,8 @@ child per distinct cause, each landing on its own. The PR gate keeps skipping benches and examples; a green Nightly catches them within a day. Decided 2026-10-27 (maintainer), from the coverage audit's triage of run -[40506938673](https://github.com/moq-dev/moq/actions/runs/40506938673): six -more children for the causes nothing owned. The three one-line breaks share -one child rather than one each, and TS's stale env var rides along with them. +[40506938673](https://github.com/moq-dev/moq/actions/runs/40506938673): more +children for the causes nothing owned. This README's own work, after the children: @@ -36,15 +35,12 @@ Public API: none. Wire: none. ## Required -- [kio's log survives Loom's eviction model](/quest/m0/nightly/loom-log.md) - the Loom causality violation in kio's log reader is fixed at its cause, possibly a use-after-free - [The software renderer tests both capability paths](/quest/m0/nightly/render-capabilities.md) - the software-renderer lane verifies output selection with and without DMA-BUF import support without requiring hardware the runner lacks - [The browser backend baseline completes on IETF22](/quest/m0/nightly/browser-baseline.md) - the WASM and UniFFI backend baseline completes every supported transport case without an internal error - [Resolve the paired catalog fanout regressions](/quest/m0/nightly/paired-shared.md) - the reported catalog fanout regressions are attributed and genuine extra costs are fixed with paired measurements - [Probing beside Cubic](/quest/m0/nightly/competing-cubic.md) - the `competing_cubic` probe test stays within its deadline budget -- [Three mechanical scheduled-run breaks](/quest/m0/nightly/small-breaks.md) - Features' dead-code `refuse`, the stale drill mutation, and TS's stale `MOQ_TEST_RUNS` -- [Software rendering runs every group](/quest/m0/nightly/render-sw.md) - the DMA-BUF assertion stops failing render-sw's first group under software Vulkan - [OBS TSan passes without a wall-clock bound](/quest/m0/nightly/obs-tsan.md) - `destroySource` asserts ordering instead of `elapsed < 500ms` - [Paired bench regressions bisected](/quest/m0/nightly/bench-regressions.md) - retirement/selection's +21-24% and two 10% rows fixed or shown to be noise -- [Nightly browser legs pass](/quest/m0/nightly/browser-legs.md) - serve-bench's 0/1024 streams, wasm-baseline's missing frame, and Firefox's unclosed VideoFrame +- [Firefox's player closes every VideoFrame](/quest/m0/nightly/firefox-videoframe.md) - the Media job's Firefox player stops leaking an unclosed VideoFrame - [Gate stops hanging](/quest/m0/nightly/gate-hang.md) - `just gate` finishes instead of hanging to the timeout, and a hang names its module - [moq-quic rustdoc](/quest/m0/nightly/quic-rustdoc.md) - `--all-features` rustdoc for moq-quic builds and a lane checks it diff --git a/quest/m0/nightly/browser-legs.md b/quest/m0/nightly/browser-legs.md deleted file mode 100644 index 64cd098bf63..00000000000 --- a/quest/m0/nightly/browser-legs.md +++ /dev/null @@ -1,27 +0,0 @@ -# [M] Nightly browser legs pass - -## Goal - -Nightly's browser publisher fairness benchmark, browser backend baseline, and -Firefox player test pass. - -## Plan - -Three unowned failures in Nightly [40506938673](https://github.com/moq-dev/moq/actions/runs/40506938673) (2026-10-27, at 22808bbb2f), possibly -one cause: - -- The Audio quality job's "Browser publisher fairness benchmark" step - (`js/net/bench/serve-browser-entry.ts`) fails with "only 0/1024 streams". -- The browser backend baseline (`wasm-baseline`) fails with "missing WASM - frame 0". -- The Media job's Firefox player fails on the console error "A VideoFrame - was garbage collected without being closed". - [Firefox publishing](/quest/m2/publish-firefox.md) covers only the - publisher side. - -Triage together first: if the first two share a cause, fix it once. If the -causes differ and one fix is large, split it into its own child of this epic. -The unclosed `VideoFrame` is a leak in the player, not a console-noise -filter to add. - -Public API: none. Wire: none. diff --git a/quest/m0/nightly/firefox-videoframe.md b/quest/m0/nightly/firefox-videoframe.md new file mode 100644 index 00000000000..809eec03c31 --- /dev/null +++ b/quest/m0/nightly/firefox-videoframe.md @@ -0,0 +1,16 @@ +# [S] Firefox's player closes every VideoFrame + +## Goal + +Nightly's Media job no longer fails Firefox's player on the console error "A +VideoFrame was garbage collected without being closed". + +## Plan + +Unowned in Nightly [40506938673](https://github.com/moq-dev/moq/actions/runs/40506938673) (2026-10-27, at 22808bbb2f). +[Firefox publishing](/quest/m2/publish-firefox.md) covers only the publisher +side. Find the player path that drops a decoded frame without `close()`, such +as one replaced, skipped, or discarded on teardown. The unclosed `VideoFrame` +is a leak in the player, not a console-noise filter to add. + +Public API: none. Wire: none. diff --git a/quest/m0/nightly/loom-log.md b/quest/m0/nightly/loom-log.md deleted file mode 100644 index 9e73acfd5a8..00000000000 --- a/quest/m0/nightly/loom-log.md +++ /dev/null @@ -1,25 +0,0 @@ -# [S] kio's log survives Loom's eviction model - -## Goal - -`just rs loom` passes `a_log_reader_racing_eviction_never_touches_freed_memory` -in `rs/kio/src/loom.rs`, with the bug it found fixed at its cause, whether -that is the log or the model's instrumentation. - -## Plan - -Nightly [40506938673](https://github.com/moq-dev/moq/actions/runs/40506938673) (2026-10-27, at 22808bbb2f) panics with "Causality violation: -Concurrent read and write accesses to `UnsafeCell`", then aborts on a panic -in a destructor. The model came with the lock-free log in -[moq-dev/moq#6782](https://github.com/moq-dev/moq/pull/6782). Treat it as a -possible use-after-free in `rs/kio/src/log.rs` until shown otherwise: a -reader's hint into a segment the collector frees is the case the model was -written to catch. Reproduce from the failing interleaving, then decide from -the trace whether the log orders the read before the free. Never loosen the -model or its preemption bound to pass. - -Public API: none. Wire: none. - -## Related - -- [In-memory unsafe runs under Miri](/quest/m2/miri-memory.md) - Miri over the same log diff --git a/quest/m0/nightly/render-sw.md b/quest/m0/nightly/render-sw.md deleted file mode 100644 index f0bdcc44f6f..00000000000 --- a/quest/m0/nightly/render-sw.md +++ /dev/null @@ -1,20 +0,0 @@ -# [S] Software rendering runs every group - -## Goal - -Nightly's Tests (rs render-sw) passes all three of its groups under software -Vulkan, so the export, scene, and studio checks after the first one run again. - -## Plan - -Nightly [40506938673](https://github.com/moq-dev/moq/actions/runs/40506938673) (2026-10-27, at 22808bbb2f) fails the first group on -`the_output_follows_the_device` (`rs/moq-video/src/render/renderer.rs:528`, -from [moq-dev/moq#6806](https://github.com/moq-dev/moq/pull/6806)), which -asserts "the adapter imports DMA-BUFs". Software Vulkan cannot, so the -assertion belongs only where the device can: make the test check what the -device reports it supports, or mark the DMA-BUF half with -`#[ignore = "requires ..."]` and let `just rs gpu` select it. The recipe's -later groups must not depend on the first passing (see this epic's -continue-past-failure work). - -Public API: none. Wire: none. diff --git a/quest/m0/nightly/small-breaks.md b/quest/m0/nightly/small-breaks.md deleted file mode 100644 index 2a301ceaa16..00000000000 --- a/quest/m0/nightly/small-breaks.md +++ /dev/null @@ -1,26 +0,0 @@ -# [S] Three mechanical scheduled-run breaks - -## Goal - -Nightly's Features and drill-sensitivity steps and TS's MPEG-2 video step get -past their mechanical failures and run their real checks. - -## Plan - -All found in the 2026-10-27 scheduled runs (Nightly [40506938673](https://github.com/moq-dev/moq/actions/runs/40506938673) at 22808bbb2f, TS -[40515802751](https://github.com/moq-dev/moq/actions/runs/40515802751)): - -- Features: `-D dead-code` rejects `Incoming::refuse` - (`rs/moq-tokio/src/quic/driver/incoming.rs:30`), which only the server and - tests call, so the driver-only feature build sees it unused. Gate it on the - feature that uses it, rather than allowing dead code. -- drill-sensitivity: `test/drill/mutations/fetch-never-sent.patch` no - longer compiles (E0425 `request`), so `test/drill/sensitivity.sh` reports - that it "broke the build". Re-anchor the patch to the current code so it - still breaks the behavior the drill expects to catch. -- TS: `.github/workflows/ts.yml`'s MPEG-2 step sets `MOQ_TEST_RUNS`, which - the relay refuses as an undeclared `MOQ_*` variable since - `check_env`. Rename it to the variable the harness reads - (`HARNESS_RUNS`), and grep the workflows for any other stale `MOQ_*`. - -Public API: none. Wire: none. diff --git a/quest/m2/workflow-security.md b/quest/m2/workflow-security.md index b78ae47a3cc..067c152463f 100644 --- a/quest/m2/workflow-security.md +++ b/quest/m2/workflow-security.md @@ -25,7 +25,3 @@ Decided 2026-10-27 (maintainer): `cargo deny` and alert through `alert.yml`. Public API: none. Wire: none. - -## Related - -- [cargo deny passes](/quest/m0/nightly/cargo-deny.md) - the Rust audit these join