Skip to content
This repository was archived by the owner on Sep 11, 2026. It is now read-only.
This repository was archived by the owner on Sep 11, 2026. It is now read-only.

Auto sampling for statistical significance #215

Description

@neonstalwart

https://github.com/google/tachometer?tab=readme-ov-file#sampling describes flags that tachometer uses to automatically determine sample size so that performance comparisons are statistically significant. A similar approach could be used to automatically set the sample size for web-codegen-scorer. Use the first n samples (e.g. 50) to estimate current rate of pass/fail and then find the sample size needed for 95% confidence and 5% margin of error for the results. Initial sample size, confidence level, margin of error could all be configurable via flags.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions