fix(results): exact fractional confidence-level labels family-wide - #815
Merged
Conversation
Every text surface that names a confidence level computed the percent with int((1 - alpha) * 100) (truncation), int(round(...)) (rounding), or :.0f, so alpha=0.025 printed "97%" or "98%" over a 97.5% interval. All 34 sites now route through one shared formatter in results_base (_coverage_level / _coverage_pct, plus _alpha_pct for the dCDH "Significant at" line, which printed 2% for alpha=0.025). Default-alpha output is byte-identical. BusinessReport headline.ci_level carries the exact level: int when integral (95 unchanged), float otherwise (97.5). No schema-version bump (REPORTING.md Note). tests/test_coverage_label.py pins the formatter, the representative surfaces at alpha=0.025, and adds a source guard against reintroducing an inline percent computation. Retires the TODO.md row.
Overall Assessment✅ Looks good — no unmitigated P0 or P1 findings. Executive Summary
Methodology
Code QualityNo findings. The shared formatter removes inconsistent inline calculations. PerformanceNo findings. Formatting overhead is negligible. MaintainabilityNo findings. Tech DebtNo findings. Removing the completed SecurityNo findings or exposed secrets. Documentation / TestsNo findings. The reporting note, release-note fragment, formatter tests, representative surface tests, and source guard are appropriate. Static parsing and changelog validation passed; runtime tests could not be executed because this environment lacks |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
int((1 - alpha) * 100)(truncation),int(round((1 - alpha) * 100))(rounding), or an:.0fformat, soalpha=0.025printed "97% Confidence Interval" or "98%" over a 97.5% interval. All 34 sites now route through one shared formatter inresults_base(_coverage_level/_coverage_pct, plus_alpha_pctfor the dCDH HonestDiD "Significant at" line, which printed2%foralpha=0.025).summary()headers (DiD/TWFE/MultiPeriod/SyntheticDiD, CallawaySantAnna, staggered and 2x2x2 TripleDifference, StackedDiD, ImputationDiD, TwoStageDiD, EfficientDiD, ContinuousDiD, dCDH, SunAbraham, TROP), theEventStudyResults/ HAD / RDD / ETWFE / LWDiD / LPDiD / ChangesInChanges table headers, CS and dCDH sup-t band labels,WildBootstrapResultsand LWDiD wild-cluster-bootstrap summaries,HonestDiDResults/PlaceboTestResultssummaries, BusinessReport / DiagnosticReport prose, and theplot_event_study/plot_dose_responsecoverage labels.headline.ci_levelcarries the exact level:intwhen integral (95is byte-unchanged) andfloatotherwise (97.5). No schema-version bump; the decision is recorded as a REPORTING.md Note.Methodology references (required if estimator / math changes)
summary()contract (stored inference is never recomputed or relabeled).Validation
tests/test_coverage_label.py(formatter pins incl.np.float32alpha; representativealpha=0.025surfaces: DiD, CS with sup-t, WooldridgeDiD, TWFE event-study table header, dCDH HonestDiD block, BusinessReportci_leveltype + prose, DiagnosticReport prose; a source guard that fails on any reintroduced inline percent computation, includingfloat(...)-wrapped forms).test_trop.py -k summary4 passed).Security / privacy
Changelog
changelog.d/20260903-fractional-confidence-label.md)