From d11f5aa8841082aecfadd6d1421edc798f903d80 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 6 Nov 2026 20:30:06 -0800 Subject: [PATCH 1/3] chore(quest): claim quest/m3/video-quality/corpus-bucket.md Quest-Claim: 29ff3442f20d26d5 From 268ce92ba6cd3da1ef418337e41850ea42262dcf Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 6 Nov 2026 20:36:43 -0800 Subject: [PATCH 2/3] feat(infra): R2 buckets for the video-quality corpus `just infra corpus create` creates corpus-moq-dev, public at corpus.moq.dev, and corpus-private-moq-dev, with no public access. infra/README.md documents the private bucket's read-only token, the main-only `corpus` Environment, and the CORPUS_R2_* secret names. Co-Authored-By: Claude Opus 5.5 --- infra/README.md | 53 ++++++- infra/corpus/bun.lock | 200 ++++++++++++++++++++++++ infra/corpus/justfile | 12 ++ infra/corpus/package.json | 11 ++ infra/justfile | 1 + quest/m3/video-quality/corpus-bucket.md | 24 ++- sh/js/workers.sh | 4 +- 7 files changed, 293 insertions(+), 12 deletions(-) create mode 100644 infra/corpus/bun.lock create mode 100644 infra/corpus/justfile create mode 100644 infra/corpus/package.json diff --git a/infra/README.md b/infra/README.md index b5c9e6ea4eb..c9dc018dc40 100644 --- a/infra/README.md +++ b/infra/README.md @@ -10,6 +10,7 @@ infrastructure. | `infra/moq-sh` | `moq.sh` | | The `moq` installer script | | `infra/mbx` | | `mbx-moq-dev` | Private Rust build cache for CI | | `infra/nix` | `nix.moq.dev` | `nix-moq-dev`, `nix-ci-moq-dev` | Nix binary cache | +| `infra/corpus` | `corpus.moq.dev` | `corpus-moq-dev`, `corpus-private-moq-dev` | Video-quality corpus | Each worker is a standalone bun package with a `wrangler.jsonc` and a small TypeScript handler. The apt and rpm workers proxy GETs out of the R2 @@ -18,7 +19,8 @@ file kind, and the nix worker does the same across its two buckets. The moq.sh worker serves `install.sh` from its bundle at every path, with the newest `moq-cli` release filled in as the default version. `infra/mbx` has no Worker: CI reaches its bucket directly through mbx's S3 -client. +client. `infra/corpus` has none either: R2 serves the open bucket on its custom +domain, and CI reads the private one over S3. ## Deploying a worker @@ -57,6 +59,7 @@ names the Environment, and a branch that is not on the list fails to start it. | `publish` | `release`, `moq-ffi-v*`, `moq-relay-v*`, `moq-cli-v*` | Registry, signing, and deploy credentials, and the moq-bot key for jobs that also need one of those | | `alert` | `main` | `DISCORD_WEBHOOK`, for `alert.yml`'s failure posts | | `cache-write` | `main`, its merge queue, binary release tags | The mbx and Nix cache write keys ([below](#bootstrapping-the-rust-build-cache)) | +| `corpus` | `main` | The private corpus's read key ([below](#bootstrapping-the-video-corpus)) | | `npm` | `release` | `JSR_TOKEN`; npm's trusted publisher names it | | `pypi` | `main`, `release`, `moq-ffi-v*` | Nothing; PyPI's trusted publisher names it | | `pub.dev` | `moq-dart-v*`, `moq-ffi-v*` | Nothing; pub.dev's automated publishing names it | @@ -72,7 +75,7 @@ A job that needs only the moq-bot key, such as Dependabot's, names `bot`; the me build holds no secrets, and the queue watcher runs on the maintainer's host, outside Actions. A job that also runs dry on other refs names the Environment only when it publishes, as in `environment: ${{ github.ref_type == 'tag' && 'publish' || '' }}`. Read-only secrets, such as the mbx read key, stay at the -repository level. +repository level, unless what they read is private. A tag policy only helps if the tag itself is protected, so the `release tags` ruleset ([below](#bootstrapping-the-nix-cache)) also covers `moq-ffi-v*` @@ -371,3 +374,49 @@ secrets. To rotate the key, run `keygen` again and add the new public key next to the old one in `flake.nix`. Drop the old key once nothing signed with it matters: 30 days for CI paths, but never for released ones unless they are re-signed. + +## Bootstrapping the video corpus + +The nightly video-quality job reads its clips from two buckets. +`corpus-moq-dev` holds open clips and is public at `corpus.moq.dev`. +`corpus-private-moq-dev` holds clips that cannot be redistributed. It has no +custom domain and its r2.dev URL is off, since R2's public access exposes a +whole bucket. The harness's manifest records each object's sha256, so a +replaced clip fails the run. Uploading clips is a maintainer action. + +The private bucket's read key lives in the `corpus` Environment, deployable +from `main` only. A dispatched run uses the workflow file from the branch it +names, so a job-level `if` cannot keep the key from other branches. + +All of this runs once, by a maintainer. Step 1 is done. + +1. **Create the buckets and the custom domain** under your `wrangler login`: + + ```bash + just infra corpus create + ``` + +2. **Create one R2 API token** in the dashboard (R2 -> Manage API tokens -> + Create Account API token) with **Object Read only** on + `corpus-private-moq-dev` only. Wrangler cannot create R2 tokens. It must + not share write credentials with the caches. + +3. **Create the `corpus` Environment**, deployable from `main` only: + + ```bash + gh api -X PUT repos/moq-dev/moq/environments/corpus --input - <<'EOF' + {"deployment_branch_policy": {"protected_branches": false, "custom_branch_policies": true}} + EOF + gh api -X POST repos/moq-dev/moq/environments/corpus/deployment-branch-policies \ + -f name=main -f type=branch + ``` + +4. **Store the key** in it: + + ```bash + gh secret set CORPUS_R2_ACCESS_KEY_ID --env corpus --repo moq-dev/moq + gh secret set CORPUS_R2_SECRET_ACCESS_KEY --env corpus --repo moq-dev/moq + ``` + + The endpoint is `https://dd618f5dbd5da77b8296f1613c301f5c.r2.cloudflarestorage.com`, + as for mbx, so no account id secret is needed. diff --git a/infra/corpus/bun.lock b/infra/corpus/bun.lock new file mode 100644 index 00000000000..2d65e057d87 --- /dev/null +++ b/infra/corpus/bun.lock @@ -0,0 +1,200 @@ +{ + "lockfileVersion": 3, + "configVersion": 1, + "workspaces": { + "": { + "name": "corpus-moq-dev", + "devDependencies": { + "wrangler": "^4", + }, + }, + }, + "overrides": { + "undici@7": { + ".": "^7.30.1", + }, + }, + "packages": { + "@cloudflare/kv-asset-handler": ["@cloudflare/kv-asset-handler@0.5.0", "", {}, "sha512-jxQYkj8dSIzc0cD6cMMNdOc1UVjqSqu8BZdor5s8cGjW2I8BjODt/kWPVdY+u9zj3ms75Q5qaZgnxUad83+eAg=="], + + "@cloudflare/unenv-preset": ["@cloudflare/unenv-preset@2.16.2", "", { "peerDependencies": { "unenv": "2.0.0-rc.24", "workerd": ">1.20260305.0 <2.0.0-0" }, "optionalPeers": ["workerd"] }, "sha512-JBP1+Z7ZSNG/d4mRP+y8VC5dka3tZVMLEZRvS+rzQ4DGV1EoxRFQckcJTTkXbHSQiTj0DtNI01Zwb/V2fX0mvQ=="], + + "@cloudflare/workerd-darwin-64": ["@cloudflare/workerd-darwin-64@1.20261023.1", "", { "os": "darwin", "cpu": "x64" }, "sha512-lsEx0i7nR0PNG/erUFFHmQmiidEhLfDE5uQmAL9421/y4/ycuxDDOSaxu3VGGYFFPK5sduyUeMmjIlX4CwOBFg=="], + + "@cloudflare/workerd-darwin-arm64": ["@cloudflare/workerd-darwin-arm64@1.20261023.1", "", { "os": "darwin", "cpu": "arm64" }, "sha512-5e79fHgabdDD8EgmjOhJ1S+xMsf6DbC7fL7IgTBizMnzOp0P2isDn7pYmUxI9AJZE9FVQ4RUilH/I36YNg+GQQ=="], + + "@cloudflare/workerd-linux-64": ["@cloudflare/workerd-linux-64@1.20261023.1", "", { "os": "linux", "cpu": "x64" }, "sha512-VHQ6jKFSihjR2+x1buby8XTdBLLJztVnYiB7OLOTYO8aDXB+c1Rd2410PdgFuFYc4AgFEk6PCJLyCLL5L1KBnw=="], + + "@cloudflare/workerd-linux-arm64": ["@cloudflare/workerd-linux-arm64@1.20261023.1", "", { "os": "linux", "cpu": "arm64" }, "sha512-ilpRHzfDN8YFRM5CLRKuFw40x/LJBS89yVdhdQmxehlVDuBqjKpJB++zPFbTgKRRFaNupgnXc8RHktXAIhgCmQ=="], + + "@cloudflare/workerd-windows-64": ["@cloudflare/workerd-windows-64@1.20261023.1", "", { "os": "win32", "cpu": "x64" }, "sha512-279WLVjNsMycAwr0EZNiNEf0l2SY6NUtsN80NwZzbTHzakmiN+WSx4w38muxPm+PAU0oSPPAPXKVD+H6Hya8xw=="], + + "@cspotcode/source-map-support": ["@cspotcode/source-map-support@0.8.1", "", { "dependencies": { "@jridgewell/trace-mapping": "0.3.9" } }, "sha512-IchNf6dN4tHoMFIn/7OE8LWZ19Y6q/67Bmf6vnGREv8RSbBVb9LPJxEcnwrcwX6ixSvaiGoomAUvu4YSxXrVgw=="], + + "@emnapi/runtime": ["@emnapi/runtime@1.11.3", "", { "dependencies": { "tslib": "^2.4.0" } }, "sha512-Xz4Tpyki7XyrpbUK1jR1AhdAdaXyhhY4lZ3neLodmhpuWfy2PAQN5B46sAiU4liOXGLkHypn/qU+jvfWSCYYLA=="], + + "@esbuild/aix-ppc64": ["@esbuild/aix-ppc64@0.28.2", "", { "os": "aix", "cpu": "ppc64" }, "sha512-XExcO+dvLKvVtNTibSTBej1NCAbaGhWn9Ww1ZPx80qsahhPFe/8jgWP0IchNe0F3HwkU7n8ejhH8bjonqht8mQ=="], + + "@esbuild/android-arm": ["@esbuild/android-arm@0.28.2", "", { "os": "android", "cpu": "arm" }, "sha512-kXXoiPVVGQcnIYGOeaovwOURpniDBpSq4A03qkQ+BMQqtGG6HYap3xne9C1O1yo4TR3qxlCX5IqqmX6fFo2Lqg=="], + + "@esbuild/android-arm64": ["@esbuild/android-arm64@0.28.2", "", { "os": "android", "cpu": "arm64" }, "sha512-5YfKeeI8qWfBZIX+u2xZC3Zlb3Os/gLS2sbEKM+I4ZOcsWmHS2WLysCcQZDAFRslDUU5Oiq44gf6PYN1vGwG5A=="], + + "@esbuild/android-x64": ["@esbuild/android-x64@0.28.2", "", { "os": "android", "cpu": "x64" }, "sha512-O387ite7SzUyCcy3JQX4P4bLtEA7bLLkx+esve5JHnyYfNTxcVpXZo9jhdB0lTKN44gztELTdU7nS8Nr16Fs1Q=="], + + "@esbuild/darwin-arm64": ["@esbuild/darwin-arm64@0.28.2", "", { "os": "darwin", "cpu": "arm64" }, "sha512-n4KqkOQrraxHJcgjM1RvwbigfQKIKJVpM7xp+KsxiyUSrRdIXnt73VhrPAx0fV44hgfmIVKjxMN9J1t5jySVkw=="], + + "@esbuild/darwin-x64": ["@esbuild/darwin-x64@0.28.2", "", { "os": "darwin", "cpu": "x64" }, "sha512-uq6suIWYP37qzGddBKPw5QEQPi6HiLGsO7UmkpfyaYNQ3D+rN6w6WfwH+nuqcGXWvawGwxOEroO4YGnFh95azw=="], + + "@esbuild/freebsd-arm64": ["@esbuild/freebsd-arm64@0.28.2", "", { "os": "freebsd", "cpu": "arm64" }, "sha512-n+I0BTSRIoy+d6RPKnEVwql5UwBJolytvY4mAOIEJorKlqgPII8ix6slVVrfZ5Tnj7glIZvloylbB/EJPMWEXw=="], + + "@esbuild/freebsd-x64": ["@esbuild/freebsd-x64@0.28.2", "", { "os": "freebsd", "cpu": "x64" }, "sha512-78XJTJkvPs0kz2w61301PJjXl4g7q3JqiYMZ/M/yVI73EHBrCRTgkhu9oqG7vPqq+a/yadEW8aD+agKlk5xrmg=="], + + "@esbuild/linux-arm": ["@esbuild/linux-arm@0.28.2", "", { "os": "linux", "cpu": "arm" }, "sha512-XlDnu2q5yoqems+xay6wSAcg9DDD7K9RLKZEBOMZm3ckNpJBvOX20tSfby8KfrrhINDyv9V2YVZKY/SpoGJI8w=="], + + "@esbuild/linux-arm64": ["@esbuild/linux-arm64@0.28.2", "", { "os": "linux", "cpu": "arm64" }, "sha512-pW4AC0P3it8c7do9MVM4p51FzHzdM/TZrerurgRcHJ2WTa1VQ1CIq18xncfpBJw4ojkiZZrKW2yIBWBP92j6Ug=="], + + "@esbuild/linux-ia32": ["@esbuild/linux-ia32@0.28.2", "", { "os": "linux", "cpu": "ia32" }, "sha512-CYbnj78HsIeA+DhgUKgFCfvNsTHFhMMrinUrMZpDXJXKN8T3XViTZ/+wtHeVxEWY8ewSzTFN+nRmSwO2tZaLUQ=="], + + "@esbuild/linux-loong64": ["@esbuild/linux-loong64@0.28.2", "", { "os": "linux", "cpu": "none" }, "sha512-buwkd8nsph4R+ajRvw0qM5Hja/TXQow3ptzWO2EbG/cqcIkHloRrdlBtQlshyYGTNFvfkfJ5tpPLVkY4DtsPfQ=="], + + "@esbuild/linux-mips64el": ["@esbuild/linux-mips64el@0.28.2", "", { "os": "linux", "cpu": "none" }, "sha512-ZVykbDyk7519VwiNb9Lcj9m8XM6v5V9uKPvrEMkkEedVewf+0itkhahp4HDpgERXhwLRpWFypsGbG/J8s0QjJA=="], + + "@esbuild/linux-ppc64": ["@esbuild/linux-ppc64@0.28.2", "", { "os": "linux", "cpu": "ppc64" }, "sha512-CAXl+Dtd9UUuJd8pKKdwh6MLm3MUMiqMPmhZ3tTSXPqfyQ3vDl6R5hZdZ/kYojK4ofXtdfSv1tFq8XzWx3heNQ=="], + + "@esbuild/linux-riscv64": ["@esbuild/linux-riscv64@0.28.2", "", { "os": "linux", "cpu": "none" }, "sha512-GeXCej4IQtU1B+QlDV8W/RRvbzI3O/Stss+/bCXv4lZls5WGRtu2a+3JkA3i4qIUlMXpcHebWpF8AkJhATowuA=="], + + "@esbuild/linux-s390x": ["@esbuild/linux-s390x@0.28.2", "", { "os": "linux", "cpu": "s390x" }, "sha512-3H1weTYZPxt/WOhByszQZybS9w5lKzUn1FDMsgEChbHWQwHYQQRfBxgCcZvPhjHfKyJjIievvMmEUawJrdY9Dg=="], + + "@esbuild/linux-x64": ["@esbuild/linux-x64@0.28.2", "", { "os": "linux", "cpu": "x64" }, "sha512-4xTZr1FUmSoQW4XIWmit3tzQrUTZM+N3P0XV8xROKYF50XfI7xeO90+1bZvNwxIufQ9hDQVRJH5YhgPVF8A/HQ=="], + + "@esbuild/netbsd-arm64": ["@esbuild/netbsd-arm64@0.28.2", "", { "os": "none", "cpu": "arm64" }, "sha512-sSATRjPeDBg3pdgHoQfoYBob11Kk1FGa9lui5RIHZCoCkJa9QKlvl3/vKz2usCmYYjs7ymJR/2Nnsqe+Hjt5nw=="], + + "@esbuild/netbsd-x64": ["@esbuild/netbsd-x64@0.28.2", "", { "os": "none", "cpu": "x64" }, "sha512-lqnzCV+mM0gIADaKihiCg6ifgfU2L3h5E33rNQBN1Y4MaVGnzryzmvvf7UHxprpQdE8hpqLolJ9Rl+SkIRDpyw=="], + + "@esbuild/openbsd-arm64": ["@esbuild/openbsd-arm64@0.28.2", "", { "os": "openbsd", "cpu": "arm64" }, "sha512-AL2qJILH7lNjrDmCQDvdxMfAUIv8KMNZOvrwAQ8i8//ntL9FflhOyMJ8OZSMBb8/AWXe3/5v5S20y3zCoZWKoQ=="], + + "@esbuild/openbsd-x64": ["@esbuild/openbsd-x64@0.28.2", "", { "os": "openbsd", "cpu": "x64" }, "sha512-QtiuPytchRyC4rwUKhexJdQKvDuZ6hWloi3igqPQNUJCS1/v9EiO3UTOXR6A3FoMo4fnAKbWJdqaIwhOzh8qEw=="], + + "@esbuild/openharmony-arm64": ["@esbuild/openharmony-arm64@0.28.2", "", { "os": "none", "cpu": "arm64" }, "sha512-WkhYDmpTjLvGlScA1rwjRUmhl4k8oXR3cIbtqWmELgU/dFeHHlEllxDvdWcNJV9rbzCexB5vz8gtNewWLgCT7Q=="], + + "@esbuild/sunos-x64": ["@esbuild/sunos-x64@0.28.2", "", { "os": "sunos", "cpu": "x64" }, "sha512-GPMSkTOtMnv2U2F8gxe4Io6qmVs+YKyp832Etqqxr0hFngmXQ3rzwytelm3GIn7T4VviRUlf3sOgBOiTdvaf7g=="], + + "@esbuild/win32-arm64": ["@esbuild/win32-arm64@0.28.2", "", { "os": "win32", "cpu": "arm64" }, "sha512-PIhhEkE9uPBleRBrQEJpUn7MBnibZzbGzYWPmY3x+YoVg/95zbjB4CxPPOQ8l5tYYM4mMaCthF8/1DIfBQQyWQ=="], + + "@esbuild/win32-ia32": ["@esbuild/win32-ia32@0.28.2", "", { "os": "win32", "cpu": "ia32" }, "sha512-YmJbfTlvU7Sdn9BB+4PRES4oB6pxgS37MAONj+hBr/cpXS1aBPKXxNnDbu+QCWPj0o9dgyxeq79g6c5P8KeuYA=="], + + "@esbuild/win32-x64": ["@esbuild/win32-x64@0.28.2", "", { "os": "win32", "cpu": "x64" }, "sha512-5ebpxr3nWMzrL/rnUI755Jkuee0bHL/Gq0WTF9lvcpv73wAp5eu8MfBUgWK9bhWvZjj7yX8etf/8tI8Ney695g=="], + + "@img/colour": ["@img/colour@1.1.0", "", {}, "sha512-Td76q7j57o/tLVdgS746cYARfSyxk8iEfRxewL9h4OMzYhbW4TAcppl0mT4eyqXddh6L/jwoM75mo7ixa/pCeQ=="], + + "@img/sharp-darwin-arm64": ["@img/sharp-darwin-arm64@0.35.5", "", { "optionalDependencies": { "@img/sharp-libvips-darwin-arm64": "1.3.4" }, "os": "darwin", "cpu": "arm64" }, "sha512-QRUlFQ0WxvdWyqqG/WtI3iupfD5rBzmCHXSdPsY91sAtVtTo7Q4cb6zOccZ3gqEqkr0f1As1ehLqmEpDsRf+lg=="], + + "@img/sharp-darwin-x64": ["@img/sharp-darwin-x64@0.35.5", "", { "optionalDependencies": { "@img/sharp-libvips-darwin-x64": "1.3.4" }, "os": "darwin", "cpu": "x64" }, "sha512-+BR255RhDlpygUpOc/Jdt1nT6DQ3XG/ERo5wbcdOf5Q320dKtPCKPLR1LJs9VGXRaMa8l1uUa0tkCNOXiAxZUw=="], + + "@img/sharp-freebsd-wasm32": ["@img/sharp-freebsd-wasm32@0.35.5", "", { "dependencies": { "@img/sharp-wasm32": "0.35.5" }, "os": "freebsd" }, "sha512-Y/z91nEZ4uIBX5X3nfTovjU9lHNKFYbL2lpHCLVNmXQK03VIZvXBBt0KxbPGp2SdGSF+2mQU4e+hQaWOt86iAw=="], + + "@img/sharp-libvips-darwin-arm64": ["@img/sharp-libvips-darwin-arm64@1.3.4", "", { "os": "darwin", "cpu": "arm64" }, "sha512-5R89nBYiRdUlSWJxPhO+GVtaXzXSxKnRu/xqMn3KTA3L9EB9Oy/P+Nn2f2vlhPuUdy/Zusb2DarbyTpGCfEDuw=="], + + "@img/sharp-libvips-darwin-x64": ["@img/sharp-libvips-darwin-x64@1.3.4", "", { "os": "darwin", "cpu": "x64" }, "sha512-iR2OKH80yi0U+dUplyh3/xdpFvps6YkCwsXenIJxqxR1v9o+xtKTGbS9H7cps+2Vxjc8B1j96p75NmTGjIhtpQ=="], + + "@img/sharp-libvips-linux-arm": ["@img/sharp-libvips-linux-arm@1.3.4", "", { "os": "linux", "cpu": "arm" }, "sha512-LmRtTsOHuvM2+wlO2Db37dx5MiZhB0FvSunciw48YjdOkZz9KAiRbm8ujeMOA1INqmei5NapFxYEK1D1ZSidmw=="], + + "@img/sharp-libvips-linux-arm64": ["@img/sharp-libvips-linux-arm64@1.3.4", "", { "os": "linux", "cpu": "arm64" }, "sha512-Y3dgX/6lE2QhQb+Gxy0WZxfg9MEm/JBjamZpS2IklP7xIQoKN4hzAm7KcMVGtaVDt3neE9OKBC7vAfonA/Lr1A=="], + + "@img/sharp-libvips-linux-ppc64": ["@img/sharp-libvips-linux-ppc64@1.3.4", "", { "os": "linux", "cpu": "ppc64" }, "sha512-Le6boB8Tai0Nis+gIxIpKx68UDVVIqdR8Tin5Yf1z2LJJQLDJvCDRqRu+jC2qCoD+eIomonmOwB4smBRxfVpYQ=="], + + "@img/sharp-libvips-linux-riscv64": ["@img/sharp-libvips-linux-riscv64@1.3.4", "", { "os": "linux", "cpu": "none" }, "sha512-aHkkIEHPRdQEegJN20MLmGtxYD9R2wQr3Cwpddnu5+YKMt6Uzax7S9h5gpZTo8wyrGuZSlfQ63OevL5mTyOC7Q=="], + + "@img/sharp-libvips-linux-s390x": ["@img/sharp-libvips-linux-s390x@1.3.4", "", { "os": "linux", "cpu": "s390x" }, "sha512-ra/mB6MikESDUO7Yg+Mi95bFBb9GsObURuhnOv3OqknjGe9sZrG8tCe9q0xSIGrtLgvgw0gKnFWcK4blSgQOuQ=="], + + "@img/sharp-libvips-linux-x64": ["@img/sharp-libvips-linux-x64@1.3.4", "", { "os": "linux", "cpu": "x64" }, "sha512-GJ//SSXbnwSDes02umB3nDJLFcQzw8a18V8fyhqr6tV515tOEMdImjjxj1AoafMRz56F3PHgftnj1QEKSU1zkw=="], + + "@img/sharp-libvips-linuxmusl-arm64": ["@img/sharp-libvips-linuxmusl-arm64@1.3.4", "", { "os": "linux", "cpu": "arm64" }, "sha512-hvulFwtjUcagsis6BBxHwGFwWoNZjgYmULGVrZcyfNbjA8hKILbRxGg15/7w5HDyXHXUos/j6baAWqnCyQ2DWA=="], + + "@img/sharp-libvips-linuxmusl-x64": ["@img/sharp-libvips-linuxmusl-x64@1.3.4", "", { "os": "linux", "cpu": "x64" }, "sha512-6zXKeE/p39I1AmA3cJG35eyBGNqNddLnUXjhwBnsGjFPWqf5VKkDBEqaEkPDoTEtkxwi2vv8Tcr2mDyP4So7Fg=="], + + "@img/sharp-linux-arm": ["@img/sharp-linux-arm@0.35.5", "", { "optionalDependencies": { "@img/sharp-libvips-linux-arm": "1.3.4" }, "os": "linux", "cpu": "arm" }, "sha512-LEaXK2WdXVK5ykcw0buWyPMsmLLL2vpHLD6yrNSW+JGEL3BZPA4tpKN6iaMc4AxTTAoaX/sU1rOL51lcIz48ZQ=="], + + "@img/sharp-linux-arm64": ["@img/sharp-linux-arm64@0.35.5", "", { "optionalDependencies": { "@img/sharp-libvips-linux-arm64": "1.3.4" }, "os": "linux", "cpu": "arm64" }, "sha512-LYVx5JTsOM2CBzmxreh+nl64/3H6Xb09iSLknqH47z2T2DFFxDeFLP5y4dJwe6H7uGQlHPyEEtIqyo3DYsRwdQ=="], + + "@img/sharp-linux-ppc64": ["@img/sharp-linux-ppc64@0.35.5", "", { "optionalDependencies": { "@img/sharp-libvips-linux-ppc64": "1.3.4" }, "os": "linux", "cpu": "ppc64" }, "sha512-QVxAAq8evVRI9ia2vqgwrmWucn5Dfv+JdWzj75pD8omHLPSP7f8p20O8jxzjCcuCEQEOtYOZUmX1hkiZ0kdevA=="], + + "@img/sharp-linux-riscv64": ["@img/sharp-linux-riscv64@0.35.5", "", { "optionalDependencies": { "@img/sharp-libvips-linux-riscv64": "1.3.4" }, "os": "linux", "cpu": "none" }, "sha512-LtdreXguaavKODPIfzJ4kffx7UNt1omwtK0rch4EBbbSTXPnxWmYSayXdLJw0fJzQ97kHt1gL/yh4tvU+nCyRQ=="], + + "@img/sharp-linux-s390x": ["@img/sharp-linux-s390x@0.35.5", "", { "optionalDependencies": { "@img/sharp-libvips-linux-s390x": "1.3.4" }, "os": "linux", "cpu": "s390x" }, "sha512-UZasTOFiYzotTsGOCu42BfUzP6Tu6Do/947iRm1RsLKvlllxwGcn4RN27LibGWceix4Y+Pmw3jsnTcCQIgWjqA=="], + + "@img/sharp-linux-x64": ["@img/sharp-linux-x64@0.35.5", "", { "optionalDependencies": { "@img/sharp-libvips-linux-x64": "1.3.4" }, "os": "linux", "cpu": "x64" }, "sha512-SxFtLTeJInhAA9Q836kux2vZNeOBQEx658qvbboZScr0wIARym3IcGmW7KpVD5sbVg0Ojy+udFQdayYIZyoNog=="], + + "@img/sharp-linuxmusl-arm64": ["@img/sharp-linuxmusl-arm64@0.35.5", "", { "optionalDependencies": { "@img/sharp-libvips-linuxmusl-arm64": "1.3.4" }, "os": "linux", "cpu": "arm64" }, "sha512-9HbMclmI1zlNkFRs3z9/eBtDjfD0sGlrX1z6b1qwmiFY5ElDLh4BC0LPBdVp7z1DXFiKlIcznf+ZlsuZzLxQqg=="], + + "@img/sharp-linuxmusl-x64": ["@img/sharp-linuxmusl-x64@0.35.5", "", { "optionalDependencies": { "@img/sharp-libvips-linuxmusl-x64": "1.3.4" }, "os": "linux", "cpu": "x64" }, "sha512-4KOphqB035HrVdqLZfCgMzzERrQkkzOwRhl4OAkRO1YCldbaFjySXMaK534Mo0V+LndnlJk+sbUyLeU0ULyD1A=="], + + "@img/sharp-wasm32": ["@img/sharp-wasm32@0.35.5", "", { "dependencies": { "@emnapi/runtime": "^1.11.3" } }, "sha512-Ptsga1su4tQx+LLF1ECS9U6nz5kmrXKo6XVbtR48Ke3ZRxxgaWBu7IDtEe1quo8hiupwm6WFqxVlXaSf7IINGQ=="], + + "@img/sharp-webcontainers-wasm32": ["@img/sharp-webcontainers-wasm32@0.35.5", "", { "dependencies": { "@img/sharp-wasm32": "0.35.5" }, "cpu": "none" }, "sha512-hfhF/FmoQyTUkA0bIKFOtw536BQSeBMe6BF6QyWlrPxT754+TFLaZ7sKKTfvvM0yJgKgaYTwnFCIZ/GuDw5SUA=="], + + "@img/sharp-win32-arm64": ["@img/sharp-win32-arm64@0.35.5", "", { "os": "win32", "cpu": "arm64" }, "sha512-X4t7g+7ZA5DKblCBEXGjUqqemj4vczING/5viFwAL8h4N3qYeyjwdCvRLHi4EdOUI+2Z7UFlp1VM+p/AuEtm6Q=="], + + "@img/sharp-win32-ia32": ["@img/sharp-win32-ia32@0.35.5", "", { "os": "win32", "cpu": "ia32" }, "sha512-5Zm82LoBc43nhwNybZlG7Y1KO//Zhsn306fQl29ZOuStHLGTo3BWL83q3cznX0poxSAMuYL1On/BHBxkBeKr6A=="], + + "@img/sharp-win32-x64": ["@img/sharp-win32-x64@0.35.5", "", { "os": "win32", "cpu": "x64" }, "sha512-x76eH0vEiHlcMQu8Y8IenntaACtddpT6W0wmXtWrnKcnKI7ME5DdgqhAD6SEWOEl1v2zDvkZDhFA9KnURwpfqg=="], + + "@jridgewell/resolve-uri": ["@jridgewell/resolve-uri@3.1.2", "", {}, "sha512-bRISgCIjP20/tbWSPWMEi54QVPRZExkuD9lJL+UIxUKtwVJA8wW1Trb1jMs1RFXo1CBTNZ/5hpC9QvmKWdopKw=="], + + "@jridgewell/sourcemap-codec": ["@jridgewell/sourcemap-codec@1.6.0", "", {}, "sha512-T7jf+5zgsZHwNJ4lvQ7/aezbyk0nNX+zJVWpmHA7VYsEx7a7qr5Rg5IbtJFqkgze5Y2sruq1RUY8Q837Od7iFw=="], + + "@jridgewell/trace-mapping": ["@jridgewell/trace-mapping@0.3.9", "", { "dependencies": { "@jridgewell/resolve-uri": "^3.0.3", "@jridgewell/sourcemap-codec": "^1.4.10" } }, "sha512-3Belt6tdc8bPgAtbcmdtNJlirVoTmEb5e2gC94PnkwEW9jI6CAHUeoG85tjWP5WquqfavoMtMwiG4P926ZKKuQ=="], + + "@poppinss/colors": ["@poppinss/colors@4.1.6", "", { "dependencies": { "kleur": "^4.1.5" } }, "sha512-H9xkIdFswbS8n1d6vmRd8+c10t2Qe+rZITbbDHHkQixH5+2x1FDGmi/0K+WgWiqQFKPSlIYB7jlH6Kpfn6Fleg=="], + + "@poppinss/dumper": ["@poppinss/dumper@0.6.5", "", { "dependencies": { "@poppinss/colors": "^4.1.5", "@sindresorhus/is": "^7.0.2", "supports-color": "^10.0.0" } }, "sha512-NBdYIb90J7LfOI32dOewKI1r7wnkiH6m920puQ3qHUeZkxNkQiFnXVWoE6YtFSv6QOiPPf7ys6i+HWWecDz7sw=="], + + "@poppinss/exception": ["@poppinss/exception@1.2.3", "", {}, "sha512-dCED+QRChTVatE9ibtoaxc+WkdzOSjYTKi/+uacHWIsfodVfpsueo3+DKpgU5Px8qXjgmXkSvhXvSCz3fnP9lw=="], + + "@sindresorhus/is": ["@sindresorhus/is@7.2.0", "", {}, "sha512-P1Cz1dWaFfR4IR+U13mqqiGsLFf1KbayybWwdd2vfctdV6hDpUkgCY0nKOLLTMSoRd/jJNjtbqzf13K8DCCXQw=="], + + "@speed-highlight/core": ["@speed-highlight/core@1.2.24", "", {}, "sha512-qeW2e1l78afw8VhRPfPQ1Gjj+KU5XFQ/OFV5ti6eTa9bruO7mJyZtA4vw0ofqmA3tKCkROE9xLk3VZoeRc98nw=="], + + "blake3-wasm": ["blake3-wasm@2.1.5", "", {}, "sha512-F1+K8EbfOZE49dtoPtmxUQrpXaBIl3ICvasLh+nJta0xkz+9kF/7uet9fLnwKqhDrmj6g+6K3Tw9yQPUg2ka5g=="], + + "cookie": ["cookie@1.1.1", "", {}, "sha512-ei8Aos7ja0weRpFzJnEA9UHJ/7XQmqglbRwnf2ATjcB9Wq874VKH9kfjjirM6UhU2/E5fFYadylyhFldcqSidQ=="], + + "detect-libc": ["detect-libc@2.1.2", "", {}, "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ=="], + + "error-stack-parser-es": ["error-stack-parser-es@1.0.5", "", {}, "sha512-5qucVt2XcuGMcEGgWI7i+yZpmpByQ8J1lHhcL7PwqCwu9FPP3VUXzT4ltHe5i2z9dePwEHcDVOAfSnHsOlCXRA=="], + + "esbuild": ["esbuild@0.28.2", "", { "optionalDependencies": { "@esbuild/aix-ppc64": "0.28.2", "@esbuild/android-arm": "0.28.2", "@esbuild/android-arm64": "0.28.2", "@esbuild/android-x64": "0.28.2", "@esbuild/darwin-arm64": "0.28.2", "@esbuild/darwin-x64": "0.28.2", "@esbuild/freebsd-arm64": "0.28.2", "@esbuild/freebsd-x64": "0.28.2", "@esbuild/linux-arm": "0.28.2", "@esbuild/linux-arm64": "0.28.2", "@esbuild/linux-ia32": "0.28.2", "@esbuild/linux-loong64": "0.28.2", "@esbuild/linux-mips64el": "0.28.2", "@esbuild/linux-ppc64": "0.28.2", "@esbuild/linux-riscv64": "0.28.2", "@esbuild/linux-s390x": "0.28.2", "@esbuild/linux-x64": "0.28.2", "@esbuild/netbsd-arm64": "0.28.2", "@esbuild/netbsd-x64": "0.28.2", "@esbuild/openbsd-arm64": "0.28.2", "@esbuild/openbsd-x64": "0.28.2", "@esbuild/openharmony-arm64": "0.28.2", "@esbuild/sunos-x64": "0.28.2", "@esbuild/win32-arm64": "0.28.2", "@esbuild/win32-ia32": "0.28.2", "@esbuild/win32-x64": "0.28.2" }, "bin": { "esbuild": "bin/esbuild" } }, "sha512-HKVLS8dvII+xoKW9kmqxbRKrnWEXfJJr/FZhhJmiqIB0e053QNYFqOBouTMO/k5sID4MvCiUCvv8b9M4h32wIA=="], + + "fsevents": ["fsevents@2.3.3", "", { "os": "darwin" }, "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw=="], + + "kleur": ["kleur@4.1.5", "", {}, "sha512-o+NO+8WrRiQEE4/7nwRJhN1HWpVmJm511pBHUxPLtp0BUISzlBplORYSmTclCnJvQq2tKu/sgl3xVpkc7ZWuQQ=="], + + "miniflare": ["miniflare@5.20261023.2-beta", "", { "dependencies": { "@cspotcode/source-map-support": "0.8.1", "sharp": "0.35.5", "undici": "7.30.1", "workerd": "1.20261023.1", "ws": "8.21.0", "youch": "4.1.0-beta.10" } }, "sha512-sXpZloQnIUe4raXXdwwqd2ioRHox9a/1B4nB2/p4JWKDwGuVgEK4DugCNgIeQf0iVuSgBItdYpodJe4iwY6SrQ=="], + + "path-to-regexp": ["path-to-regexp@6.3.0", "", {}, "sha512-Yhpw4T9C6hPpgPeA28us07OJeqZ5EzQTkbfwuhsUg0c237RomFoETJgmp2sa3F/41gfLE6G5cqcYwznmeEeOlQ=="], + + "pathe": ["pathe@2.0.3", "", {}, "sha512-WUjGcAqP1gQacoQe+OBJsFA7Ld4DyXuUIjZ5cc75cLHvJ7dtNsTugphxIADwspS+AraAUePCKrSVtPLFj/F88w=="], + + "semver": ["semver@7.8.5", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-Y7/KDsb8LjooZpwaqGyulO6DQlksgCncchHGk+sZIY4SBvUocMBEFH5Ur1fI4dV+Jvl0w6cjvucaIi40puRioA=="], + + "sharp": ["sharp@0.35.5", "", { "dependencies": { "@img/colour": "^1.1.0", "detect-libc": "^2.1.2", "semver": "^7.8.5" }, "optionalDependencies": { "@img/sharp-darwin-arm64": "0.35.5", "@img/sharp-darwin-x64": "0.35.5", "@img/sharp-freebsd-wasm32": "0.35.5", "@img/sharp-libvips-darwin-arm64": "1.3.4", "@img/sharp-libvips-darwin-x64": "1.3.4", "@img/sharp-libvips-linux-arm": "1.3.4", "@img/sharp-libvips-linux-arm64": "1.3.4", "@img/sharp-libvips-linux-ppc64": "1.3.4", "@img/sharp-libvips-linux-riscv64": "1.3.4", "@img/sharp-libvips-linux-s390x": "1.3.4", "@img/sharp-libvips-linux-x64": "1.3.4", "@img/sharp-libvips-linuxmusl-arm64": "1.3.4", "@img/sharp-libvips-linuxmusl-x64": "1.3.4", "@img/sharp-linux-arm": "0.35.5", "@img/sharp-linux-arm64": "0.35.5", "@img/sharp-linux-ppc64": "0.35.5", "@img/sharp-linux-riscv64": "0.35.5", "@img/sharp-linux-s390x": "0.35.5", "@img/sharp-linux-x64": "0.35.5", "@img/sharp-linuxmusl-arm64": "0.35.5", "@img/sharp-linuxmusl-x64": "0.35.5", "@img/sharp-webcontainers-wasm32": "0.35.5", "@img/sharp-win32-arm64": "0.35.5", "@img/sharp-win32-ia32": "0.35.5", "@img/sharp-win32-x64": "0.35.5" } }, "sha512-Ywn4OnzGukp7CDMrp08RQ50YKmuwG47brZgIVPTvBaaAfQlRlygrRqSrxdCiL9M+LlzLBiJ68IR1QqvzHyjC7g=="], + + "supports-color": ["supports-color@10.2.2", "", {}, "sha512-SS+jx45GF1QjgEXQx4NJZV9ImqmO2NPz5FNsIHrsDjh2YsHnawpan7SNQ1o8NuhrbHZy9AZhIoCUiCeaW/C80g=="], + + "tslib": ["tslib@2.8.1", "", {}, "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w=="], + + "undici": ["undici@7.30.1", "", {}, "sha512-03KrMbjIFDWQR5RNPrSw3bd1r12JpyFQl9/yxjUbJtqkjmllqC5lFjZh6ujeuJCKr8EOQvRW872Oy/az+nLYxg=="], + + "unenv": ["unenv@2.0.0-rc.24", "", { "dependencies": { "pathe": "^2.0.3" } }, "sha512-i7qRCmY42zmCwnYlh9H2SvLEypEFGye5iRmEMKjcGi7zk9UquigRjFtTLz0TYqr0ZGLZhaMHl/foy1bZR+Cwlw=="], + + "workerd": ["workerd@1.20261023.1", "", { "optionalDependencies": { "@cloudflare/workerd-darwin-64": "1.20261023.1", "@cloudflare/workerd-darwin-arm64": "1.20261023.1", "@cloudflare/workerd-linux-64": "1.20261023.1", "@cloudflare/workerd-linux-arm64": "1.20261023.1", "@cloudflare/workerd-windows-64": "1.20261023.1" }, "bin": { "workerd": "bin/workerd" } }, "sha512-NeofJFyaYNpqKpFRJW3ApuonaerPd6/JWlfTtLDYf2A6ygqrDEFnYgEXmc0hhOxqnJsSM+kXJN6VebPojzRrsg=="], + + "wrangler": ["wrangler@4.155.0", "", { "dependencies": { "@cloudflare/kv-asset-handler": "0.5.0", "@cloudflare/unenv-preset": "2.16.2", "blake3-wasm": "2.1.5", "esbuild": "0.28.2", "miniflare": "5.20261023.2-beta", "path-to-regexp": "6.3.0", "unenv": "2.0.0-rc.24", "workerd": "1.20261023.1" }, "optionalDependencies": { "fsevents": "2.3.3" }, "peerDependencies": { "@cloudflare/workers-types": "^5.20261023.1" }, "optionalPeers": ["@cloudflare/workers-types"], "bin": { "wrangler": "bin/wrangler.js", "wrangler2": "bin/wrangler.js", "cf-wrangler": "bin/cf-wrangler.js" } }, "sha512-gBKtAg2e0abQDvXPRQu611YMTd/cfAaJTOLAs/89tTxFxEeUyC260MRJRK+m3gLmQjDYWFwJv+ErwuJ/GuALpA=="], + + "ws": ["ws@8.21.0", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-Vsp28b7DRcimFQvrqu2Wek3z1iYxDCWqHYB8Qsnk/S4RfaCQzPGPyBNuVjJV3cd6UiKtUtp6sNM77gWvzcCH+g=="], + + "youch": ["youch@4.1.0-beta.10", "", { "dependencies": { "@poppinss/colors": "^4.1.5", "@poppinss/dumper": "^0.6.4", "@speed-highlight/core": "^1.2.7", "cookie": "^1.0.2", "youch-core": "^0.3.3" } }, "sha512-rLfVLB4FgQneDr0dv1oddCVZmKjcJ6yX6mS4pU82Mq/Dt9a3cLZQ62pDBL4AUO+uVrCvtWz3ZFUL2HFAFJ/BXQ=="], + + "youch-core": ["youch-core@0.3.3", "", { "dependencies": { "@poppinss/exception": "^1.2.2", "error-stack-parser-es": "^1.0.5" } }, "sha512-ho7XuGjLaJ2hWHoK8yFnsUGy2Y5uDpqSTq1FkHLK4/oqKtyUU1AFbOOxY4IpC9f0fTLjwYbslUz0Po5BpD1wrA=="], + } +} diff --git a/infra/corpus/justfile b/infra/corpus/justfile new file mode 100644 index 00000000000..4eb6f5e8ee0 --- /dev/null +++ b/infra/corpus/justfile @@ -0,0 +1,12 @@ +set fallback + +# No Worker config names the account here, so wrangler would ask. +export CLOUDFLARE_ACCOUNT_ID := "dd618f5dbd5da77b8296f1613c301f5c" + +# Create the open corpus bucket on corpus.moq.dev and the private one, which has no public access. Run once. +create: + bun install --frozen-lockfile + bun wrangler r2 bucket create corpus-moq-dev + bun wrangler r2 bucket create corpus-private-moq-dev + # The zone id is moq.dev's. + bun wrangler r2 bucket domain add corpus-moq-dev --domain corpus.moq.dev --zone-id 6e4171f84c4faa5bb2c9549723eda5d7 --min-tls 1.2 --force diff --git a/infra/corpus/package.json b/infra/corpus/package.json new file mode 100644 index 00000000000..7cfe43705c2 --- /dev/null +++ b/infra/corpus/package.json @@ -0,0 +1,11 @@ +{ + "name": "corpus-moq-dev", + "private": true, + "devDependencies": { + "wrangler": "^4" + }, + "//overrides": "Drop this when upstream stops pinning a vulnerable undici.", + "overrides": { + "undici@7": "^7.30.1" + } +} diff --git a/infra/justfile b/infra/justfile index 3244cfaa900..8e29d57561a 100644 --- a/infra/justfile +++ b/infra/justfile @@ -5,6 +5,7 @@ mod rpm mod moq-sh mod mbx mod nix +mod corpus # Deploy all infra workers. deploy: diff --git a/quest/m3/video-quality/corpus-bucket.md b/quest/m3/video-quality/corpus-bucket.md index 5c283e6e2de..87dadb2c816 100644 --- a/quest/m3/video-quality/corpus-bucket.md +++ b/quest/m3/video-quality/corpus-bucket.md @@ -9,16 +9,24 @@ public access exposes a whole bucket and a prefix cannot be private behind it. A read-only token for the private bucket is a secret the nightly video-quality jobs can read. -This waits on a Cloudflare account action. Check it by fetching one public -object anonymously, one private object with the secret from a nightly -`workflow_dispatch` run, and the same private object anonymously, which must -fail. Delete this quest once all three hold. - ## Plan -- Follow the existing R2 secret naming (`MBX_R2_*`, `R2_*`), with a separate - token that is read-only and scoped to the private bucket. The corpus must - never share write credentials with the caches. +Done 2026-11-07: `just infra corpus create` created `corpus-moq-dev`, public +at `corpus.moq.dev`, and `corpus-private-moq-dev`, with no custom domain and +its r2.dev URL off. Each holds a `probe.txt`; `infra/README.md` ("Bootstrapping +the video corpus") documents the token, the `corpus` Environment, and the +`CORPUS_R2_*` secret names. + - Upload is a maintainer action. The manifest the harness owns records each object's sha256, so a replaced object fails the run instead of silently changing the baseline. + +## Waiting + +- Maintainer: mint the private bucket's read-only token in the dashboard and + store it in the `corpus` Environment, steps 2 to 4 of `infra/README.md`'s + "Bootstrapping the video corpus". Wrangler cannot create R2 API tokens, and + an agent never handles the secret. Check by fetching one public object + anonymously, one private object with the secret from a nightly + `workflow_dispatch` run, and the same private object anonymously, which must + fail. Delete this quest once all three hold. diff --git a/sh/js/workers.sh b/sh/js/workers.sh index dd47a3126e2..6fa6a3c1046 100755 --- a/sh/js/workers.sh +++ b/sh/js/workers.sh @@ -4,8 +4,8 @@ # # These Workers have lockfiles outside the Bun workspace, so `just js check` # never installs or builds them. Every tracked lockfile but the workspace's is -# one, so a new Worker can't fall out of the check; infra/mbx only drives -# wrangler's CLI and has no bundle to build. +# one, so a new Worker can't fall out of the check; infra/mbx and infra/corpus +# only drive wrangler's CLI and have no bundle to build. set -euo pipefail cd "$(git rev-parse --show-toplevel)" From 4b1fd5eabafa64e7e38b486b3ca4d38b3b637000 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 6 Nov 2026 20:55:19 -0800 Subject: [PATCH 3/3] quest: finish the corpus bucket quest The token is minted and stored in the `corpus` Environment, so every bootstrap step is done. The dispatched-run check moves to the harness quest, which adds the first workflow that uses the Environment. Co-Authored-By: Claude Opus 5.5 --- infra/README.md | 6 +++-- quest/m3/video-quality/README.md | 1 - quest/m3/video-quality/corpus-bucket.md | 32 ------------------------- quest/m3/video-quality/harness.md | 13 +++++----- 4 files changed, 11 insertions(+), 41 deletions(-) delete mode 100644 quest/m3/video-quality/corpus-bucket.md diff --git a/infra/README.md b/infra/README.md index c9dc018dc40..7aa345d580e 100644 --- a/infra/README.md +++ b/infra/README.md @@ -377,7 +377,9 @@ old one in `flake.nix`. Drop the old key once nothing signed with it matters: ## Bootstrapping the video corpus -The nightly video-quality job reads its clips from two buckets. +The nightly video-quality job, which [the harness +quest](../quest/m3/video-quality/harness.md) adds, reads its clips from two +buckets. `corpus-moq-dev` holds open clips and is public at `corpus.moq.dev`. `corpus-private-moq-dev` holds clips that cannot be redistributed. It has no custom domain and its r2.dev URL is off, since R2's public access exposes a @@ -388,7 +390,7 @@ The private bucket's read key lives in the `corpus` Environment, deployable from `main` only. A dispatched run uses the workflow file from the branch it names, so a job-level `if` cannot keep the key from other branches. -All of this runs once, by a maintainer. Step 1 is done. +All of this runs once, by a maintainer. All four steps are done. 1. **Create the buckets and the custom domain** under your `wrangler login`: diff --git a/quest/m3/video-quality/README.md b/quest/m3/video-quality/README.md index 99cd9ab06c5..98498eedc9a 100644 --- a/quest/m3/video-quality/README.md +++ b/quest/m3/video-quality/README.md @@ -74,6 +74,5 @@ Decided in planning (2026-10-28): - [Harness](/quest/m3/video-quality/harness.md) - `just test video-quality` scores OpenH264 rows with per-frame VMAF and boundary pops against floors, nightly - [On-demand pops](/quest/m3/video-quality/on-demand.md) - cold starts, resubscribes, and fetch-encoded groups spliced with live ones stay under the pop ceiling - [moq-boy quality](/quest/m3/video-quality/moq-boy.md) - moq-boy's encode is measured on generated clips and fixed -- [Corpus buckets](/quest/m3/video-quality/corpus-bucket.md) - R2 serves open clips publicly and private clips, from a separate bucket, to a read-only CI secret - [GPU runners](/quest/m3/video-quality/gpu-runners.md) - `nixos2` and the desktop host self-hosted runners for the nightly job - [Hardware rows](/quest/m3/video-quality/hardware.md) - NVENC and VA-API rows from three GPU hosts gate the nightly and compare side by side diff --git a/quest/m3/video-quality/corpus-bucket.md b/quest/m3/video-quality/corpus-bucket.md deleted file mode 100644 index 87dadb2c816..00000000000 --- a/quest/m3/video-quality/corpus-bucket.md +++ /dev/null @@ -1,32 +0,0 @@ -# [XS] The video corpus has R2 buckets - -## Goal - -Two R2 buckets hold the video-quality corpus. Open clips are in a public-read -bucket at a moq.dev hostname, the way ROMs are served from `rom.moq.dev`. -Private clips are in a separate bucket with public access off, since R2's -public access exposes a whole bucket and a prefix cannot be private behind -it. A read-only token for the private bucket is a secret the nightly -video-quality jobs can read. - -## Plan - -Done 2026-11-07: `just infra corpus create` created `corpus-moq-dev`, public -at `corpus.moq.dev`, and `corpus-private-moq-dev`, with no custom domain and -its r2.dev URL off. Each holds a `probe.txt`; `infra/README.md` ("Bootstrapping -the video corpus") documents the token, the `corpus` Environment, and the -`CORPUS_R2_*` secret names. - -- Upload is a maintainer action. The manifest the harness owns records each - object's sha256, so a replaced object fails the run instead of silently - changing the baseline. - -## Waiting - -- Maintainer: mint the private bucket's read-only token in the dashboard and - store it in the `corpus` Environment, steps 2 to 4 of `infra/README.md`'s - "Bootstrapping the video corpus". Wrangler cannot create R2 API tokens, and - an agent never handles the secret. Check by fetching one public object - anonymously, one private object with the secret from a nightly - `workflow_dispatch` run, and the same private object anonymously, which must - fail. Delete this quest once all three hold. diff --git a/quest/m3/video-quality/harness.md b/quest/m3/video-quality/harness.md index 0ecf9fdacc4..e9398b418e0 100644 --- a/quest/m3/video-quality/harness.md +++ b/quest/m3/video-quality/harness.md @@ -42,10 +42,15 @@ Decisions are recorded in the [epic](/quest/m3/video-quality/README.md). `just gh benchmark-report`, as the other nightly sources do. - Corpus manifest: synthetic clips generated at run time (cheap, always available), open clips such as Blender open movies (CC-BY) with each license - checked before listing, and private clips from the - [bucket](/quest/m3/video-quality/corpus-bucket.md). Cover screen content, + checked before listing, and private clips from `corpus-private-moq-dev` + (see `infra/README.md`, "Bootstrapping the video corpus"). Open clips go in + `corpus-moq-dev`, served at `corpus.moq.dev`. Cover screen content, talking head, high motion, and fades. Keep clips short: nightly CPU time is a budget. +- The job reads private clips with `CORPUS_R2_*` from the `corpus` + Environment, which admits only `main`. Its first dispatched run must read + `probe.txt` from `corpus-private-moq-dev` through that Environment, and a + dispatch on an unprotected branch must not get the secret. - A run selects clips by name. A selected clip that cannot be fetched, or whose sha256 differs from the manifest, fails the run. - OpenH264 is deterministic, so its floors can sit close to the measured @@ -53,10 +58,6 @@ Decisions are recorded in the [epic](/quest/m3/video-quality/README.md). Public API: none. Wire: none. -## Required - -- [Corpus bucket](/quest/m3/video-quality/corpus-bucket.md) - where open and private clips live - ## Related - [Audio ViSQOL](/quest/m4/audio-visqol.md) - the same floor rule for audio