Repository navigation
Conversation
|
I'm a bit hesitant to bake our own here as it's somewhat nontrivial to flesh out custom benchmarking, for example you kind of want things like statistics, saving records to files to compare against previous iterations, etc, and that can get pretty gnarly pretty quickly. Since wasi-sdk is typically used to compile wasi-libc and it's already got libc++, would it be possible to rely on that when compiling benchmarks? They don't have to compile with every toolchain under the sun for example. |
|
i should have mentioned this in the pr description: bench.h prints go's benchmark format so benchstat could potentially handle the complexity you're describing (stats and cross-run comparisons, etc): i did prototype google benchmark on top of wasi-sdk, which works, though it comes with a few restrictions:
would documenting this benchstat workflow address your concern, or would you still prefer something like the google benchmark setup? or is there another framework that could work? |
|
Ok that makes sense to me, so yeah seems like the Google C++ benchmark framework isn't worth using here. Documenting something like |
Add a small header-only C benchmark runner under test/bench that calibrates iteration batches between CLOCK_MONOTONIC reads so host clock overhead does not dominate nanosecond-scale libc calls. Wire benchmark binaries into test/CMakeLists.txt via a bench target using the configured Wasm engine, with a 1ms CTest smoke pass on CLI runners to keep benchmarks building cleanly.
f53afb5 to
281eb67
Compare
|
So ubench.h did come up when i was investigating but:
It also weighs ~1k LoC, much of it repeating the logic benchstat provides us. Admittedly isn't not ideal to roll our own here but it's <100 LoC of pretty straightforward code. i also updated the logic to fix some edge-cases i found. ptal |
Add a small header-only C benchmark runner under test/bench that
calibrates iteration batches between CLOCK_MONOTONIC reads so host clock
overhead does not dominate nanosecond-scale libc calls.
Wire benchmark binaries into test/CMakeLists.txt via a bench target
using the configured Wasm engine, with a 1ms CTest smoke pass on CLI
runners to keep benchmarks building cleanly.