From 4229a1e975220b6f6c4152109c0ea8b38c4a5293 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 01:48:51 -0800 Subject: [PATCH 1/5] chore(quest): claim quest/m0/ci/hakari-followups.md Quest-Claim: 65b310d43ab379af From e8793a2de3c87674ad665515b97c3bd1d581dbce Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 01:55:02 -0800 Subject: [PATCH 2/5] fix(ci): keep native media dependencies out of the workspace hack Derive the trybuild exclusion from Cargo metadata and verify renamed and target-specific UI dependencies. Preserve the scheduled Linux cache measurement as the remaining quest work. Co-Authored-By: GPT-6.1 Sol --- .config/hakari.toml | 3 ++ .github/justfile | 1 + Cargo.lock | 3 -- quest/m0/ci/hakari-followups.md | 77 ++++++++++++++++++----------- rs/workspace-hack/Cargo.toml | 10 ---- sh/rs/hakari.py | 8 ++- sh/rs/hakari.test.py | 88 +++++++++++++++++++++++++++++++++ 7 files changed, 147 insertions(+), 43 deletions(-) create mode 100644 sh/rs/hakari.test.py diff --git a/.config/hakari.toml b/.config/hakari.toml index 71289bde5b2..0ce14b4ccc0 100644 --- a/.config/hakari.toml +++ b/.config/hakari.toml @@ -26,3 +26,6 @@ workspace-members = ["moq-mux-fuzz", "moq-net-fuzz", "moq-quic-fuzz", "moq-rtmp- [final-excludes] workspace-members = ["kio-macros"] +# Keep native media dependencies out of unrelated test builds. Final excludes +# preserve feature unification for their other dependencies. +third-party = [{ name = "pipewire" }, { name = "libspa" }, { name = "clang-sys" }] diff --git a/.github/justfile b/.github/justfile index 73d61a8e986..bf643244a6e 100644 --- a/.github/justfile +++ b/.github/justfile @@ -61,6 +61,7 @@ test: python3 sh/dart/build.test.py python3 sh/rs/select.test.py python3 sh/rs/config.test.py + python3 sh/rs/hakari.test.py python3 sh/js/workspace.test.py python3 sh/js/audit.test.py python3 sh/rs/test-lints.test.py diff --git a/Cargo.lock b/Cargo.lock index 302d669a3a3..a2084f5f4d5 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -11201,7 +11201,6 @@ dependencies = [ "camino", "cc", "chrono", - "clang-sys", "clap", "clap_builder", "const-oid 0.9.6", @@ -11252,7 +11251,6 @@ dependencies = [ "jsonwebtoken", "lazy_static", "libc", - "libspa", "linux-raw-sys 0.12.1", "log", "md-5 0.10.6", @@ -11279,7 +11277,6 @@ dependencies = [ "object_store", "once_cell", "percent-encoding", - "pipewire", "pkcs1", "pkcs8 0.10.2", "pkcs8 0.11.0", diff --git a/quest/m0/ci/hakari-followups.md b/quest/m0/ci/hakari-followups.md index 64c6135830d..9e8fcf750be 100644 --- a/quest/m0/ci/hakari-followups.md +++ b/quest/m0/ci/hakari-followups.md @@ -1,40 +1,59 @@ -# [XS] The workspace-hack refuses trybuild edges and spares non-media builds +# [S] Measure the Linux cache effect of native media exclusions ## Goal -`sh/rs/hakari.py` fails when a crate that dev-depends on trybuild also -dev-depends on `workspace-hack`, and a Linux test build of a crate with no -media dependency needs neither libpipewire nor libclang. +Scheduled Linux jobs record the cache hit rate and build-time effect of +removing native media requirements from unrelated test builds. Bring a +material cache regression to the maintainer before declaring the tradeoff +settled. ## Plan -Decided 2026-10-31 (user, /quest-plan of the follow-ups from the 2026-10-31 /quest-complete run), from [moq-dev/moq#6985](https://github.com/moq-dev/moq/pull/6985). Milestone: ✅ m0 (user: "use recommended for any CI/flakes"). - -- **trybuild edge**: trybuild copies a crate's dev-dependencies into the - project it compiles each UI case in, so `kio-macros`' hack edge made - `kio-macros::ui` compile the whole hack (88-120 s), time out, and starve - the tests beside it; #6985 (merged) was ejected from the land queue for it - ([run 41074871604](https://github.com/moq-dev/moq/actions/runs/41074871604)) - and fixed it by excluding `kio-macros` by hand (`.config/hakari.toml`'s - `[final-excludes]`). `sh/rs/hakari.py:37-53` (at be39188000) only checks - that excluded members lack the edge and the rest carry it as a path - dev-dependency. Make it derive the rule: a crate with `trybuild` in its - dev-dependencies must not carry the `workspace-hack` edge, with a test - that plants one. -- **libpipewire**: CodeRabbit flagged on #6985 that the hack's unified - features pull PipeWire (through cpal) and libclang into ordinary Linux - test builds of crates that use no media; `rs/workspace-hack/Cargo.toml` - still lists `pipewire` (`:353`) and `clang-sys` (`:372`). The dev shell - provides both, so only builds outside it notice. Recommended: exclude the - pipewire/cpal feature from the hack with hakari's final-excludes, then - measure that the cache hit rate across jobs barely moves. If it does - move, bring the numbers to the maintainer instead. - -`cargo-hakari` is already in the dev shell (`flake.nix:162`, in `rustDeps`), -so that part of the original quest is done. +The implementation in [moq-dev/moq#7713](https://github.com/moq-dev/moq/pull/7713) +adds the derived trybuild-edge guard and excludes `pipewire`, `libspa`, and +`clang-sys` from the workspace-hack's final output. Other transitive features +remain unified. Linux x86_64 and ARM test graphs of `moq-json` no longer reach +PipeWire or libclang; an uncached x86_64 nextest run passes with native media +pkg-config probes refused and an invalid libclang path. The original manifests +fail under the same restriction. + +Local Cargo 1.98.1 unit graphs for `moq-json`, `moq-video --features pipewire`, +and `moq-audio --features pipewire`, each built with `--tests`, have 531 distinct +third-party recursive compilation identities both before and after. The JSON +build loses 30 units; video retains 501 total units. Four media identities +change. These are graph identities, not mbx hit rates or CPU measurements. + +Use the same scheduled Linux job before and after this PR's merge, beginning +with `Nightly (Rust benchmark tests)`, and retain its mbx cache summaries and +available compiler-time fields. Compare the last scheduled `main` run before +the merge with the first two after it. Failed or cancelled builds and missing summaries +are incomplete evidence, not zero cache hits. Record source/build differences +that prevent attributing a change to these exclusions. + +The 2026-11-02 scheduled run is a pre-merge candidate: +[run 41483654426](https://github.com/moq-dev/moq/actions/runs/41483654426), +[Rust benchmark tests](https://github.com/moq-dev/moq/actions/runs/41483654426/job/125408205554). +Its 12 mbx object-cache summaries total 986 hits, 721 misses, 323 not +looked up, 8 bypassed, and 11 remote failures. The conditional hit rate is +57.76% (`986 / (986 + 721)`); this excludes not-looked-up and bypassed units. +The summaries report estimated compiler time avoided, not measured CPU totals. +The remote failures and mixed build phases limit attribution. Replace this +candidate if a later scheduled run precedes this PR's merge. Public API: none. Wire: none. +## Waiting + +- moq-dev/moq: after the native-media exclusions merge, collect the last + scheduled Linux baseline and first two post-merge Nightlies' mbx hit rates + and available compiler-time fields. No post-merge evidence for these + exclusions exists yet. If the cache cost is material, present the measured tradeoff to the + maintainer; recommend keeping the narrow final exclusions unless the + measurements justify another design. + ## Related -- [Unify dependency features with cargo-hakari](/quest/m0/ci/hakari.md) - records the hosted jobs' cache hit rate around the workspace-hack, the measurement the pipewire exclusion needs +- [Unify dependency features with cargo-hakari](/quest/m0/ci/hakari.md) - owns + the separate macOS/Windows evidence and FFI invocation alignment in + [moq-dev/moq#7666](https://github.com/moq-dev/moq/pull/7666); do not duplicate + those measurements or depend on its unmerged implementation diff --git a/rs/workspace-hack/Cargo.toml b/rs/workspace-hack/Cargo.toml index 767776371e0..0b615435683 100644 --- a/rs/workspace-hack/Cargo.toml +++ b/rs/workspace-hack/Cargo.toml @@ -329,12 +329,10 @@ hyper-rustls = { version = "0.27", default-features = false, features = ["aws-lc hyper-util = { version = "0.1", default-features = false, features = ["client-legacy", "client-proxy"] } itertools = { version = "0.13" } libc = { version = "0.2", features = ["extra_traits"] } -libspa = { version = "0.10", default-features = false, features = ["v0_3_33"] } linux-raw-sys = { version = "0.12", features = ["auxvec", "elf", "if_ether", "ioctl", "net", "netlink", "no_std", "prctl", "system", "xdp"] } log = { version = "0.4", default-features = false, features = ["release_max_level_debug"] } nix = { version = "0.31", features = ["net", "uio"] } nom-15128469a54ed75a = { package = "nom", version = "7" } -pipewire = { version = "0.10", default-features = false, features = ["v0_3_53"] } proc-macro2 = { version = "1" } quick-xml = { version = "0.41", features = ["overlapped-lists", "serialize"] } quote = { version = "1" } @@ -351,7 +349,6 @@ zerocopy = { version = "0.8", default-features = false, features = ["derive", "s [target.aarch64-unknown-linux-gnu.build-dependencies] bytemuck = { version = "1", default-features = false, features = ["aarch64_simd"] } -clang-sys = { version = "1", default-features = false, features = ["clang_11_0", "runtime"] } crossbeam-epoch = { version = "0.9" } cudarc = { version = "0.19", default-features = false, features = ["cuda-12020", "driver", "fallback-dynamic-loading", "nvrtc"] } event-listener = { version = "5" } @@ -364,12 +361,10 @@ hyper-rustls = { version = "0.27", default-features = false, features = ["aws-lc hyper-util = { version = "0.1", default-features = false, features = ["client-legacy", "client-proxy"] } itertools = { version = "0.13" } libc = { version = "0.2", features = ["extra_traits"] } -libspa = { version = "0.10", default-features = false, features = ["v0_3_33"] } linux-raw-sys = { version = "0.12", features = ["auxvec", "elf", "if_ether", "ioctl", "net", "netlink", "no_std", "prctl", "system", "xdp"] } log = { version = "0.4", default-features = false, features = ["release_max_level_debug"] } nix = { version = "0.31", features = ["net", "uio"] } nom-15128469a54ed75a = { package = "nom", version = "7" } -pipewire = { version = "0.10", default-features = false, features = ["v0_3_53"] } quick-xml = { version = "0.41", features = ["overlapped-lists", "serialize"] } regex = { version = "1" } rustc-hash = { version = "1" } @@ -431,12 +426,10 @@ hyper-rustls = { version = "0.27", default-features = false, features = ["aws-lc hyper-util = { version = "0.1", default-features = false, features = ["client-legacy", "client-proxy"] } itertools = { version = "0.13" } libc = { version = "0.2", features = ["extra_traits"] } -libspa = { version = "0.10", default-features = false, features = ["v0_3_33"] } linux-raw-sys = { version = "0.12", features = ["auxvec", "elf", "if_ether", "ioctl", "net", "netlink", "no_std", "prctl", "system", "xdp"] } log = { version = "0.4", default-features = false, features = ["release_max_level_debug"] } nix = { version = "0.31", features = ["net", "uio"] } nom-15128469a54ed75a = { package = "nom", version = "7" } -pipewire = { version = "0.10", default-features = false, features = ["v0_3_53"] } proc-macro2 = { version = "1" } quick-xml = { version = "0.41", features = ["overlapped-lists", "serialize"] } quote = { version = "1" } @@ -453,7 +446,6 @@ zerocopy = { version = "0.8", default-features = false, features = ["derive", "s [target.x86_64-unknown-linux-gnu.build-dependencies] bytemuck = { version = "1", default-features = false, features = ["aarch64_simd"] } -clang-sys = { version = "1", default-features = false, features = ["clang_11_0", "runtime"] } crossbeam-epoch = { version = "0.9" } cudarc = { version = "0.19", default-features = false, features = ["cuda-12020", "driver", "fallback-dynamic-loading", "nvrtc"] } event-listener = { version = "5" } @@ -466,12 +458,10 @@ hyper-rustls = { version = "0.27", default-features = false, features = ["aws-lc hyper-util = { version = "0.1", default-features = false, features = ["client-legacy", "client-proxy"] } itertools = { version = "0.13" } libc = { version = "0.2", features = ["extra_traits"] } -libspa = { version = "0.10", default-features = false, features = ["v0_3_33"] } linux-raw-sys = { version = "0.12", features = ["auxvec", "elf", "if_ether", "ioctl", "net", "netlink", "no_std", "prctl", "system", "xdp"] } log = { version = "0.4", default-features = false, features = ["release_max_level_debug"] } nix = { version = "0.31", features = ["net", "uio"] } nom-15128469a54ed75a = { package = "nom", version = "7" } -pipewire = { version = "0.10", default-features = false, features = ["v0_3_53"] } quick-xml = { version = "0.41", features = ["overlapped-lists", "serialize"] } regex = { version = "1" } rustc-hash = { version = "1" } diff --git a/sh/rs/hakari.py b/sh/rs/hakari.py index 4fb46d84822..feae8258c8d 100755 --- a/sh/rs/hakari.py +++ b/sh/rs/hakari.py @@ -5,7 +5,9 @@ every third-party crate while `cargo publish` strips it. `cargo hakari manage-deps` only writes normal dependencies, so this checks the edges instead: every member dev-depends on the hack, except those `.config/hakari.toml` -excludes, which must not depend on it at all. Run from `just rs lint`. +excludes, and crates whose UI tests use trybuild. Neither may depend on it +at all: trybuild copies dev-dependencies into every UI test project. Run +from `just rs lint`. """ import json @@ -38,6 +40,10 @@ name = package["name"] edges = [dep for dep in package["dependencies"] if dep["name"] == hack] manifest = Path(package["manifest_path"]).relative_to(ROOT) + if any(dep["name"] == "trybuild" and dep["kind"] == "dev" for dep in package["dependencies"]): + if edges: + errors.append(f"{manifest}: trybuild copies dev-dependencies into UI tests; remove the {hack} edge") + continue if name in excluded: if edges: errors.append(f"{manifest}: {name} is excluded in .config/hakari.toml but depends on {hack}") diff --git a/sh/rs/hakari.test.py b/sh/rs/hakari.test.py new file mode 100644 index 00000000000..4feb059ac68 --- /dev/null +++ b/sh/rs/hakari.test.py @@ -0,0 +1,88 @@ +"""Keep UI test projects and non-media test builds free of native media dependencies.""" + +import shutil +import subprocess +import tempfile +import unittest +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[2] + + +class EdgesTest(unittest.TestCase): + def test_trybuild_cannot_inherit_the_hack(self): + scratch = ROOT / ".tmp" + scratch.mkdir(exist_ok=True) + for excluded in (False, True): + for target in (False, True): + with ( + self.subTest(excluded=excluded, target=target), + tempfile.TemporaryDirectory(dir=scratch) as directory, + ): + root = Path(directory) + (root / "sh/rs").mkdir(parents=True) + shutil.copyfile(ROOT / "sh/rs/hakari.py", root / "sh/rs/hakari.py") + (root / ".config").mkdir() + (root / ".config/hakari.toml").write_text( + 'hakari-package = "workspace-hack"\n[final-excludes]\nworkspace-members = ' + + ('["ui", "trybuild"]\n' if excluded else '["trybuild"]\n') + ) + (root / "Cargo.toml").write_text( + '[workspace]\nmembers = ["ui", "trybuild", "workspace-hack"]\nresolver = "2"\n' + ) + for name in ("ui", "trybuild", "workspace-hack"): + crate = root / name + (crate / "src").mkdir(parents=True) + (crate / "src/lib.rs").touch() + (crate / "Cargo.toml").write_text(f'[package]\nname = "{name}"\nversion = "0.0.0"\n') + manifest = root / "ui/Cargo.toml" + with manifest.open("a") as output: + output.write( + ("[target.'cfg(unix)'.dev-dependencies]\n" if target else "[dev-dependencies]\n") + + 'ui-runner = { package = "trybuild", path = "../trybuild" }\n' + ) + safe = manifest.read_text() + with manifest.open("a") as output: + output.write('workspace-hack = { path = "../workspace-hack" }\n') + result = subprocess.run(["python3", "sh/rs/hakari.py"], cwd=root, capture_output=True, text=True) + self.assertNotEqual(result.returncode, 0, result.stdout + result.stderr) + self.assertIn("trybuild", result.stderr) + self.assertIn("workspace-hack", result.stderr) + # A new UI crate is safe without having to copy kio-macros' exclusion. + manifest.write_text(safe) + result = subprocess.run(["python3", "sh/rs/hakari.py"], cwd=root, capture_output=True, text=True) + self.assertEqual(result.returncode, 0, result.stdout + result.stderr) + + +class NativeDependenciesTest(unittest.TestCase): + def test_non_media_linux_test_graph(self): + for target in ("x86_64-unknown-linux-gnu", "aarch64-unknown-linux-gnu"): + with self.subTest(target=target): + result = subprocess.run( + [ + "cargo", + "tree", + "--locked", + "-p", + "moq-json", + "--target", + target, + "--edges", + "normal,build,dev", + "--prefix", + "none", + "--format", + "{p}", + ], + cwd=ROOT, + capture_output=True, + text=True, + ) + self.assertEqual(result.returncode, 0, result.stdout + result.stderr) + packages = {line.split()[0] for line in result.stdout.splitlines()} + self.assertIn("workspace-hack", packages) + self.assertTrue(packages.isdisjoint({"pipewire-sys", "libspa-sys", "clang-sys"}), packages) + + +if __name__ == "__main__": + unittest.main() From e86e0ea1afbfb858d1b6cbca7b8f249c2aa47cf6 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 01:57:24 -0800 Subject: [PATCH 3/5] docs(quest): track the remaining native-media cache measurement Co-Authored-By: GPT-6.1 Sol --- quest/m0/ci/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/quest/m0/ci/README.md b/quest/m0/ci/README.md index 2b375cdecd8..019561b5864 100644 --- a/quest/m0/ci/README.md +++ b/quest/m0/ci/README.md @@ -166,7 +166,7 @@ The historical samples locate candidates; they are not controlled benchmarks. - [The bench comparison builds moq-gst's catalog bench on its base](/quest/m0/ci/bench-base-build.md) - A paired bench comparison that selects `rust/gst/moq-gst/catalog` against `origin/main` builds the base side and reports a comparison, instead of marking it unavailable with "base build failed" - [A jq error in the Rust selection fails the run](/quest/m0/ci/check-selection.md) - `just check`'s Rust selection fails the run when one of its jq filters errors, instead of selecting less - [Code scanning has no untriaged alert on main](/quest/m0/ci/code-scanning-triage.md) - every open code-scanning alert on `main`'s default branch is either fixed or dismissed with a reason and a comment that names the evidence -- [The workspace-hack refuses trybuild edges and spares non-media builds](/quest/m0/ci/hakari-followups.md) - `hakari.py` refuses trybuild dev-dependency edges, and non-media Linux test builds stop needing libpipewire and libclang +- [Measure the Linux cache effect of native media exclusions](/quest/m0/ci/hakari-followups.md) - scheduled Linux jobs measure cache reuse after unrelated tests stop compiling PipeWire and libclang - [Pullfrog reviews finish inside their time limit](/quest/m0/ci/pullfrog-timeout.md) - A Pullfrog review of a large PR finishes instead of failing on the action's run limit - [Long test runs keep their built binaries](/quest/m0/ci/shared-target.md) - A long `just test interop` or wire-compat run never fails because the binaries it built were removed from the shared Cargo target mid-run, such as by mbx pruning its managed build storage - [One nextest pin](/quest/m0/ci/nextest-pin.md) - cargo-nextest's version lives in one place From f65786c916f4303d345c8f7461245afc793eed59 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 3 Nov 2026 02:01:44 -0800 Subject: [PATCH 4/5] docs(quest): qualify the native-build and cache evidence Co-Authored-By: GPT-6.1 Sol --- quest/m0/ci/hakari-followups.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/quest/m0/ci/hakari-followups.md b/quest/m0/ci/hakari-followups.md index 9e8fcf750be..9873ae6da7b 100644 --- a/quest/m0/ci/hakari-followups.md +++ b/quest/m0/ci/hakari-followups.md @@ -13,9 +13,9 @@ The implementation in [moq-dev/moq#7713](https://github.com/moq-dev/moq/pull/771 adds the derived trybuild-edge guard and excludes `pipewire`, `libspa`, and `clang-sys` from the workspace-hack's final output. Other transitive features remain unified. Linux x86_64 and ARM test graphs of `moq-json` no longer reach -PipeWire or libclang; an uncached x86_64 nextest run passes with native media -pkg-config probes refused and an invalid libclang path. The original manifests -fail under the same restriction. +PipeWire or libclang. An x86_64 nextest run with shared compiler caching +bypassed passes with native media pkg-config probes refused and an invalid +libclang path. The original manifests fail under the same restriction. Local Cargo 1.98.1 unit graphs for `moq-json`, `moq-video --features pipewire`, and `moq-audio --features pipewire`, each built with `--tests`, have 531 distinct @@ -37,7 +37,8 @@ Its 12 mbx object-cache summaries total 986 hits, 721 misses, 323 not looked up, 8 bypassed, and 11 remote failures. The conditional hit rate is 57.76% (`986 / (986 + 721)`); this excludes not-looked-up and bypassed units. The summaries report estimated compiler time avoided, not measured CPU totals. -The remote failures and mixed build phases limit attribution. Replace this +The 5 GiB download budget was exhausted during the first build; remote +failures and mixed build phases limit attribution. Replace this candidate if a later scheduled run precedes this PR's merge. Public API: none. Wire: none. From 50eb0a060c5764313a5e4e86c1e5168e94064d30 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 4 Nov 2026 14:48:04 -0800 Subject: [PATCH 5/5] docs(ci): record the VAAPI cache tradeoff --- quest/m0/ci/hakari-followups.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/quest/m0/ci/hakari-followups.md b/quest/m0/ci/hakari-followups.md index 9873ae6da7b..8bfecc2801e 100644 --- a/quest/m0/ci/hakari-followups.md +++ b/quest/m0/ci/hakari-followups.md @@ -23,6 +23,19 @@ third-party recursive compilation identities both before and after. The JSON build loses 30 units; video retains 501 total units. Four media identities change. These are graph identities, not mbx hit rates or CPU measurements. +Additional Linux x86_64 unit graphs at the merged main baseline +`ac0f04e5dcca9a9a07d15135dc2e904215c494f3` compare `moq-video --features vaapi` +and `moq-video --features pipewire`, each with `--tests`. VAAPI-only drops from +478 to 456 total units; PipeWire stays at 491. Recursive third-party identities +shared by those builds drop from 453 to 428, and their union grows from 483 to +486. Excluding `clang-sys` lets VAAPI's bindgen 0.70 request `clang_6_0`, while +PipeWire's bindgen 0.72 requests `clang_11_0`; the library and its two build-script +units split into three extra compilation identities. The earlier three-build +sample does not cover this cost. Record this VAAPI/PipeWire split when interpreting +the scheduled cache results. These graphs measure compilation identities, not +CPU cost or cache hit rates; the post-merge measurements still decide whether +the tradeoff is material. + Use the same scheduled Linux job before and after this PR's merge, beginning with `Nightly (Rust benchmark tests)`, and retain its mbx cache summaries and available compiler-time fields. Compare the last scheduled `main` run before