Repository navigation
bench: backport main's benchmarks to release for before/after runs - #6103
Conversation
Port main's Criterion targets onto release with main's case IDs, so `just bench origin/release` on main pairs them. Cases whose subject doesn't exist on release are dropped rather than emulated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8f0f1b798f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const effect = new Effect((effect) => { | ||
| checksum += effect.get(signal) + 1; | ||
| }); | ||
| effect.close(); |
There was a problem hiding this comment.
Effect schedules its callback in a microtask, but this closes the instance synchronously, so every scheduled run observes the closed state and returns without executing effect.get(signal). Consequently, the create-get-close row measures scheduling and immediate closure rather than creating and closing an effect with a tracked signal, while the queued callbacks drain outside the timed section and the checksum from the other rows masks the omission.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Valid: the callback never ran, so the row timed scheduling plus an immediate close. Fixed in 465fb4a: the row yields once so the effect tracks the signal before close(), the timing loop awaits only rows that return a promise (the sync rows pay no extra hop), and a separate tracked counter fails the run if the effect never executes. main's js/signals/bench/effect.ts has the same bug and needs the same fix for the before/after rows to pair.
(Written by Claude Opus 5.5)
release has no producer_tick bench, so moq-stats' benches never use criterion and cargo shear fails. create-get-close closed the effect before its microtask ran, so it never tracked the signal; yield once before closing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ness Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@codex review (Written by Claude Opus 5.5) |
|
Codex Review: Didn't find any major issues. Already looking forward to the next diff. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Ports main's benchmarks onto
releasesojust bench origin/release, run from main, has a baseline to compare against.bench/run.sh <base>pairs Criterion cases by manifest, target, and case ID, so every case kept here has the same ID as on main.When the code a case measures doesn't exist on release, the case is dropped rather than emulated. A missing baseline is better than a misleading one.
Criterion targets (cases listed on release)
tracktrack::Range/with_ranges, untimed sparse reads, and retired empty runs.groupgroup_relay_frames(needsfuzz::object_group).statsrequest,write, andretag(APIs not on release).originnarrowandnarrow_delivery(no session auth on release).sessiontests/support. Dropped thewindow=64axis (no constrained mock),session_churn_*/moq-lite-06(see below), and theSESSION_CPUmode.export(new)containers(new)with_max_delaybecomeswith_max_age(the rename in #4917).contended(new)matching(new)chunk_interleaving(new)catalog(new)worker_turns(new)TxBufhas noextend_from_sliceon release, so the bench writes the slice and passeslen.Also:
moq-statsgains theaggregatetable bench. It keeps only thegrace=never, known-epoch rows, because release has no grace and no epochs.net/bench/frames.tsstdout fix, plusjson/bench/codec.tsandsignals/bench/effect.tswith ajust js signals-benchrecipe. Thecreate-get-closerow now yields once before closing, so the effect actually runs and tracks the signal (it closed before its microtask on main too, so main needs the same fix for the rows to pair).criterionis added as a dev-dependency where the new targets need it.Cargo.lockonly gains those edges. moq-stats keeps none, since its criterion bench (producer_tick) isn't on release.Unchanged, because main's version is equivalent to release's or needs main-only code: announce, priority, kio tasks, e2ee protect, uring udp/session_lite, and the other JS benches.
Not ported, because the code they measure isn't on release: moq-net admission, auth, codec, memory, refresh, request-ids, retirement, schedule, synced, tail, viewers; moq-mux catalog, jitter, ts_schedule; moq-relay auth, http_idle; moq-stats producer_tick; moq-archive writer; moq-uring echo_quic, timers, unreachable_dials.
Notes
session_churn_*/moq-lite-06hangs on release. After one open and close, reopening the subscription never reaches the relay. This may be a real release bug and is worth checking separately.rs/**, so release-plz will propose patch bumps for the crates whose bench files changed. Only benchmark code changed.Verification
just check origin/releasepasses in the Nix dev shell (includescargo shearandcargo sort).cargo clippy --benches -D warningsandcargo fmt --checkare clean on the touched crates. Run on macOS with rustup 1.98.1, outside Nix.Every target above lists its cases. Each one ran briefly without panics; moq-uring ran in a Linux container.
🤖 Generated with Claude Code
(Written by Claude Opus 5.5)