Repository navigation
ci(bench): publish measured nightly benchmark trends - #5759
Conversation
Co-Authored-By: GPT-6 <noreply@openai.com>
Co-Authored-By: GPT-6 <noreply@openai.com>
|
Implementation and local checks are complete. This stays a draft for the maintainer's Pages choice: enable Source: GitHub Actions (recommended), then verify the first main-branch Nightly deployment; or keep the site disabled. Repository settings have not changed. (written by GPT-6) |
Chart the split Rust bench targets job and the new moq-quic and e2ee benchmarks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bench-browser and bench-ffi-gen also match the bench-* download pattern; the browser report has its own schema. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The origin now holds route updates and restarts for 300 ms against its clock, so announce_duplicate and reprice_duplicate saw nothing pending and panicked. Both now run with a zero hold, as the auth renewal bench does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The moq-quic targets time thread CPU through their own Measurement, which Criterion does not record, so the report labelled them elapsed time. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 10 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Merging at 1bfcd7e. The maintainer has enabled repository Pages (Source: GitHub Actions), the prerequisite in the description. Codex thumbs-up on this head; CI green. Check the first main deployment at https://moq-dev.github.io/moq/bench/ after the next Nightly. (Written by Claude Opus 5.5) |
Resolves sh/gh/nightly.sh against #5759 (resume joins the measured JS benches) and ports #5725's streamed largest to resumed JS readers: the serving copy's largest, never moving back, ending only with the route, as Rust's Consumer::poll_largest does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Important
Maintainer prerequisite before merge: enable repository Pages with Settings > Pages > Source: GitHub Actions. Merging does not change repository settings.
Without it, the main-branch Nightly's Benchmark site job still validates the reports and pushes the history to
gh-pages, then fails atactions/deploy-pageswith "Ensure GitHub Pages has been enabled", so the Nightly run goes red every night. It never skips silently.Problem
Nightly retained binding results, but most Rust Criterion targets only smoke-tested with
--test, and the JS measurements had no durable charts.Approach
just gh criterion,bench/run.sh --criterion). The JS group tees each benchmark's CSV. Both write a validatedreport.json; the binding benchmark already emits one.bench-*artifact that records an outcome, then stores cost and throughput charts on one serializedgh-pagesbranch with github-action-benchmark and deploys them explicitly to Pages. Alerts and comments stay off.Manualclock, protocol diagnostics stay off CSV stdout, and moq-net'sannounce_duplicateandreprice_duplicatebenches run with a zeroupdate_hold. Both currently fail Nightly on main.Impact
Validation
just checkand the tooling regressions (sh/gh/nightly.test.py,sh/gh/bench-report.test.py) pass. A measured localjust gh nightly jsproduced 567 valid entries across all 16 suites, including the new e2ee bench (forward stopped early, see Follow-ups). A measured moq-quicwire+receiverun throughjust gh criterionproduced 32 Criterion estimates. The moq-netoriginbench passes--testagain. The pinned upstream action generated both chart families twice in a temporary repository and kept both history points. The full measured Rust nightly suite was not run locally.Alternatives
Smoke mode supplies no Criterion estimates. Pushing a Pages branch with
GITHUB_TOKENalone does not deploy a site, so the publisher deploys its generated artifact explicitly.Follow-ups
js/net/bench/forward.tshangs at 100% CPU at 2048 held watches since feat(net): announce consumers report when they have synced #5724 (bisected:866e72783~1finishes in about 2 s,866e72783never finishes). Nightly's JS group will stall on it until fixed, and a failed group publishes no JS point.(Written by Claude Opus 5.5, building on GPT-6's draft)
🤖 Generated with Claude Code