Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,6 @@ Related tracking surfaces:
| `WooldridgeDiD` does not apply the W2025 Sec 5.4 `D_{G_max} x X` covariate normalization, and three sibling covariate rank deficiencies are pre-existing. Measured with the period range pinned and only the never-treated units toggled: (1) time-invariant `exovar` is absorbed by the unit FE, 4 of 26 columns, IDENTICALLY with and without never-treated units; (2) `xgvar`'s cell x covariate block, 19 of 41, identical on both panels; (3) `xtvar` under `demean_covariates=False` does exhibit the `sum_g D_g x = x` dependency that the default demeaning removes; (4) the newly-reachable case -- time-VARYING data passed through `exovar`, which its own docstring reserves for time-invariant covariates -- where the paper's `dT_i` rule would give a deterministic `D_{G_max} x X` drop instead of QR's arbitrary pick (coefficients unaffected, `1.35e-14`; `rank_deficient_action="error"` raises). REGISTRY's narrowed Sec 5.4 note cross-references this row. **Trap for whoever takes it:** `xtvar` under the DEFAULT `demean_covariates=True` is FULL RANK -- the raw block carries demeaned values while `D_g x X` carries raw ones -- and forcing the drop there moves `overall_att` 1.11903 -> 1.46269. Pinned as-is by `TestComparisonSupportFiltering::test_cells_derived_groups_did_not_leak_into_the_design`. | `diff_diff/wooldridge.py` | #729-followup | Heavy | Medium |
| `WooldridgeDiD.n_control_units` counts never-treated UNITS on `control_group="never_treated"` regardless of method, but on the nonlinear paths (`logit`/`poisson`) treated units' pre-treatment rows ARE the identifying comparison -- only the OLS path absorbs them into their own cells. So the reported count under-states the comparison pool exactly where the REGISTRY control-pool asymmetry note applies. Widen to `not_yet_treated or (never_treated and method != "ols")`, or document the count as never-treated-units-by-definition. Behavior is PRE-EXISTING; documented for now in the REGISTRY control-pool Note rather than changed, because widening moves a public results field and wants its own ledger row and test matrix. | `diff_diff/wooldridge.py` | #729-followup | Mid | Low |
| `WooldridgeDiD` has no opt-out for comparison-support period filtering: a user who would rather see the refusal than a reduced sample cannot ask for it. Adding one means a constructor parameter (`get_params`/`set_params` propagation, transactional validation), a ledger row, and a test matrix across both predicate branches and all three `rank_deficient_action` modes -- deliberately out of scope for the change that introduced the filter. The always-on warning is the interim answer. | `diff_diff/wooldridge.py` | #729-followup | Mid | Low |
| Fractional confidence-level display unification: every `summary()` header computes `int((1 - alpha) * 100)` (~19 truncating sites incl. the shipped M-146 staggered family), so a fit `alpha=0.025` prints "97% Confidence Interval" over a 97.5% interval; `plot_dose_response`'s band label now renders exact fractional coverage via its `_coverage_label` helper - unify the summary surfaces on the same formatter (cross-family display change: moves every summary golden/doctest that pins a header, so it wants one sweep with its own test recapture, not a per-estimator drip) | `diff_diff/results_base.py`, `diff_diff/results.py` | alpha-guard review | Quick | Low |

### Performance

Expand Down
19 changes: 19 additions & 0 deletions changelog.d/20260903-fractional-confidence-label.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
### Fixed
- **Exact fractional confidence-level labels, family-wide**: every text surface
that names a confidence level now prints the exact coverage (`97.5%` for
`alpha=0.025`; previously truncated to `97%` by `int((1 - alpha) * 100)` or
rounded to `98%` by `int(round(...))` / `:.0f`) via one shared
`results_base._coverage_pct` formatter: the 14 `summary()` headers
(DiD/TWFE/MultiPeriod/SyntheticDiD, CallawaySantAnna, staggered and 2x2x2
TripleDifference, StackedDiD, ImputationDiD, TwoStageDiD, EfficientDiD,
ContinuousDiD, dCDH, SunAbraham, TROP), the `EventStudyResults` / HAD / RDD /
ETWFE / LWDiD / LPDiD / ChangesInChanges table headers, the CS and dCDH sup-t
band labels, the dCDH HonestDiD block (whose "Significant at" line printed
`2%` for `alpha=0.025`; now `2.5%`), `WildBootstrapResults` and the LWDiD
wild-cluster-bootstrap summaries, `HonestDiDResults` / `PlaceboTestResults`
summaries, and BusinessReport / DiagnosticReport prose. The BusinessReport
headline `ci_level` field carries the exact level as an `int` when integral
(`95` is byte-unchanged) and a `float` otherwise (`97.5`); no schema-version
bump (REPORTING.md Note). Default-alpha output is byte-identical. A source
guard (`tests/test_coverage_label.py`) rejects any reintroduced inline
percent computation.
10 changes: 5 additions & 5 deletions diff_diff/business_report.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@

from diff_diff._reporting_helpers import describe_target_parameter
from diff_diff.diagnostic_report import DiagnosticReport, DiagnosticReportResults
from diff_diff.results_base import Diagnostic
from diff_diff.results_base import Diagnostic, _coverage_level, _coverage_pct

BUSINESS_REPORT_SCHEMA_VERSION = "2.0"

Expand Down Expand Up @@ -534,7 +534,7 @@ def _build_schema(self) -> Dict[str, Any]:
"ci_upper": None,
"alpha_was_honored": True,
"alpha_override_caveat": None,
"ci_level": int(round((1.0 - self._context.alpha) * 100)),
"ci_level": _coverage_level(self._context.alpha),
"p_value": None,
"is_significant": False,
"near_significance_threshold": False,
Expand Down Expand Up @@ -683,7 +683,7 @@ def _extract_headline(self, dr_schema: Optional[Dict[str, Any]]) -> Dict[str, An
f"for the confidence interval because this fit uses "
f"{inference_label} inference; the displayed CI remains "
f"at the fit's native level "
f"({int(round((1.0 - result_alpha) * 100))}%). The "
f"({_coverage_pct(result_alpha)}%). The "
f"significance phrasing still uses the requested alpha."
)

Expand All @@ -700,7 +700,7 @@ def _extract_headline(self, dr_schema: Optional[Dict[str, Any]]) -> Dict[str, An
)
if att is None or not np.isfinite(att):
sign = "undefined"
ci_level = int(round((1.0 - display_alpha) * 100))
ci_level = _coverage_level(display_alpha)
# bool(...) coerces away numpy bool_ — when ``p`` is a numpy NaN (e.g.
# SyntheticControl, whose analytical p_value is always NaN), ``np.isfinite``
# yields a numpy bool that is NOT JSON-serializable in the schema.
Expand Down Expand Up @@ -2073,7 +2073,7 @@ def _significance_phrase(p: Optional[float], alpha: float) -> str:
"""
if p is None or not np.isfinite(p):
return "statistical significance cannot be assessed (p-value unavailable)"
ci_level = int(round((1.0 - alpha) * 100))
ci_level = _coverage_pct(alpha)
if p < 0.001:
return "the direction of the effect is strongly supported by the data"
if p < 0.01:
Expand Down
16 changes: 11 additions & 5 deletions diff_diff/chaisemartin_dhaultfoeuille_results.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,13 @@
from diff_diff._deprecation import deprecated_field_property
from diff_diff.aggregation import AggregationMixin, AggregationResult
from diff_diff.results import _get_significance_stars
from diff_diff.results_base import BaseResults, _require_fit_alpha, build_event_study_surface
from diff_diff.results_base import (
BaseResults,
_alpha_pct,
_coverage_pct,
_require_fit_alpha,
build_event_study_surface,
)

__all__ = [
"ChaisemartinDHaultfoeuilleResults",
Expand Down Expand Up @@ -935,7 +941,7 @@ def summary(self, alpha: Optional[float] = None) -> str:
decomposition diagnostic, and a footer of significance codes.
"""
alpha = _require_fit_alpha(alpha, self.alpha)
conf_level = int((1 - alpha) * 100)
conf_level = _coverage_pct(alpha)
width = 85
sep = "=" * width
thin = "-" * width
Expand Down Expand Up @@ -1590,7 +1596,7 @@ def _render_path_effects_section(
if self.path_sup_t_bands is not None and path in self.path_sup_t_bands:
crit_p = self.path_sup_t_bands[path].get("crit_value", np.nan)
if np.isfinite(crit_p):
conf_level = int((1 - self.alpha) * 100)
conf_level = _coverage_pct(self.alpha)
lines.append(
f" Sup-t critical value: {crit_p:.4f} "
f"(simultaneous {conf_level}% bands)"
Expand All @@ -1605,7 +1611,7 @@ def _render_honest_did_section(self, lines: List[str], width: int, thin: str) ->
method_label = hd.method.replace("_", " ").title()
m_val = hd.M
sig_label = "Yes" if hd.is_significant else "No"
conf_pct = int((1 - hd.alpha) * 100)
conf_pct = _coverage_pct(hd.alpha)
lines.extend(
[
thin,
Expand All @@ -1625,7 +1631,7 @@ def _render_honest_did_section(self, lines: List[str], width: int, thin: str) ->
f"{'Identified set:':<35} " f"[{_fmt_float(hd.lb)}, {_fmt_float(hd.ub)}]",
f"{'Robust ' + str(conf_pct) + '% CI:':<35} "
f"[{_fmt_float(hd.ci_lb)}, {_fmt_float(hd.ci_ub)}]",
f"{'Significant at ' + str(int(hd.alpha * 100)) + '%:':<35} " f"{sig_label:>10}",
f"{'Significant at ' + _alpha_pct(hd.alpha) + '%:':<35} " f"{sig_label:>10}",
thin,
"",
]
Expand Down
4 changes: 2 additions & 2 deletions diff_diff/changes_in_changes_results.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
import pandas as pd

from diff_diff._deprecation import deprecated_field_property
from diff_diff.results_base import BaseResults
from diff_diff.results_base import BaseResults, _coverage_pct

_ESTIMATOR_TITLES = {
"cic": "Changes-in-Changes (Athey & Imbens 2006) Results",
Expand Down Expand Up @@ -159,7 +159,7 @@ def summary(self) -> str:
"""Fixed-width text summary: headline ATT block plus the quantile-effects table."""
from diff_diff.results import _get_significance_stars

ci_pct = int(round((1 - self.alpha) * 100))
ci_pct = _coverage_pct(self.alpha)
width = 88
bar = "=" * width
dash = "-" * width
Expand Down
3 changes: 2 additions & 1 deletion diff_diff/continuous_did_results.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
from diff_diff.results_base import (
_SUMMARY_ALPHA_MESSAGE,
BaseResults,
_coverage_pct,
_require_fit_alpha,
build_event_study_surface,
)
Expand Down Expand Up @@ -274,7 +275,7 @@ def summary(self, alpha: Optional[float] = None) -> str:
or relabeling. Re-fit at the desired alpha instead.
"""
alpha = _require_fit_alpha(alpha, self.alpha, message=_SUMMARY_ALPHA_MESSAGE)
conf_level = int((1 - alpha) * 100)
conf_level = _coverage_pct(alpha)
w = 85

lines = [
Expand Down
6 changes: 3 additions & 3 deletions diff_diff/diagnostic_report.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@
import pandas as pd

from diff_diff._reporting_helpers import describe_target_parameter # noqa: E402 (top-level import)
from diff_diff.results_base import Diagnostic
from diff_diff.results_base import Diagnostic, _coverage_pct

DIAGNOSTIC_REPORT_SCHEMA_VERSION = "2.0"

Expand Down Expand Up @@ -4461,9 +4461,9 @@ def _render_overall_interpretation(schema: Dict[str, Any], labels: Dict[str, str
# stays consistent with the rendered interval when alpha != 0.05.
headline_alpha = headline.get("alpha") if isinstance(headline, dict) else None
if isinstance(headline_alpha, (int, float)) and 0 < headline_alpha < 1:
ci_level = int(round((1.0 - headline_alpha) * 100))
ci_level = _coverage_pct(headline_alpha)
else:
ci_level = 95
ci_level = "95"
ci_finite = (
isinstance(ci, (list, tuple))
and len(ci) == 2
Expand Down
4 changes: 2 additions & 2 deletions diff_diff/diagnostics.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
from diff_diff._deprecation import NOT_SUPPLIED, require_arg, resolve_renamed_kwarg
from diff_diff.estimators import DifferenceInDifferences
from diff_diff.results import _get_significance_stars
from diff_diff.results_base import Diagnostic
from diff_diff.results_base import Diagnostic, _coverage_pct
from diff_diff.utils import safe_inference, validate_binary


Expand Down Expand Up @@ -87,7 +87,7 @@ def significance_stars(self) -> str:

def summary(self) -> str:
"""Generate formatted summary of placebo test results."""
conf_level = int((1 - self.alpha) * 100)
conf_level = _coverage_pct(self.alpha)

lines = [
"=" * 65,
Expand Down
9 changes: 7 additions & 2 deletions diff_diff/efficient_did_results.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,12 @@
from diff_diff.efficient_did_aggregation import _EfficientAggregationMixin
from diff_diff.efficient_did_bootstrap import EfficientDiDBootstrapMixin
from diff_diff.results import _format_survey_block, _get_significance_stars
from diff_diff.results_base import BaseResults, _require_fit_alpha, build_event_study_surface
from diff_diff.results_base import (
BaseResults,
_coverage_pct,
_require_fit_alpha,
build_event_study_surface,
)

if TYPE_CHECKING:
from diff_diff.efficient_did_bootstrap import EDiDBootstrapResults
Expand Down Expand Up @@ -647,7 +652,7 @@ def summary(self, alpha: Optional[float] = None) -> str:
never recomputed or relabeled - re-fit at the desired alpha).
"""
alpha = _require_fit_alpha(alpha, self.alpha)
conf_level = int((1 - alpha) * 100)
conf_level = _coverage_pct(alpha)

lines = [
"=" * 85,
Expand Down
6 changes: 3 additions & 3 deletions diff_diff/had.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@
BiasCorrectedFit,
bias_corrected_local_linear,
)
from diff_diff.results_base import BaseResults, build_event_study_surface
from diff_diff.results_base import BaseResults, _coverage_pct, build_event_study_surface
from diff_diff.survey import (
SurveyMetadata,
compute_survey_metadata,
Expand Down Expand Up @@ -444,7 +444,7 @@ def __repr__(self) -> str:
def summary(self) -> str:
"""Formatted summary table."""
width = 72
conf_level = int((1 - self.alpha) * 100)
conf_level = _coverage_pct(self.alpha)
lines = [
"=" * width,
"HeterogeneousAdoptionDiD Estimation Results".center(width),
Expand Down Expand Up @@ -884,7 +884,7 @@ def __repr__(self) -> str:
def summary(self) -> str:
"""Formatted per-horizon summary table."""
width = 80
conf_level = int((1 - self.alpha) * 100)
conf_level = _coverage_pct(self.alpha)
lines = [
"=" * width,
"HeterogeneousAdoptionDiD Event-Study Results".center(width),
Expand Down
4 changes: 2 additions & 2 deletions diff_diff/honest_did.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
from diff_diff.results import (
MultiPeriodDiDResults,
)
from diff_diff.results_base import Diagnostic, _validate_vcov_subblock
from diff_diff.results_base import Diagnostic, _coverage_pct, _validate_vcov_subblock
from diff_diff.utils import _get_critical_value

# =============================================================================
Expand Down Expand Up @@ -271,7 +271,7 @@ def summary(self) -> str:
str
Formatted summary.
"""
conf_level = int((1 - self.alpha) * 100)
conf_level = _coverage_pct(self.alpha)

method_names = {
"smoothness": "Smoothness (Delta^SD)",
Expand Down
9 changes: 7 additions & 2 deletions diff_diff/imputation_results.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,12 @@
from diff_diff.aggregation import AggregationMixin, AggregationResult, build_total_relay_row
from diff_diff.imputation_aggregation import _ImputationAggregationMixin
from diff_diff.results import _format_survey_block, _get_significance_stars
from diff_diff.results_base import BaseResults, _require_fit_alpha, build_event_study_surface
from diff_diff.results_base import (
BaseResults,
_coverage_pct,
_require_fit_alpha,
build_event_study_surface,
)


class _ImputationKitAggregator(_ImputationAggregationMixin):
Expand Down Expand Up @@ -563,7 +568,7 @@ def summary(self, alpha: Optional[float] = None) -> str:
Formatted summary.
"""
alpha = _require_fit_alpha(alpha, self.alpha)
conf_level = int((1 - alpha) * 100)
conf_level = _coverage_pct(alpha)

lines = [
"=" * 85,
Expand Down
4 changes: 2 additions & 2 deletions diff_diff/lpdid_results.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
import pandas as pd

from diff_diff._deprecation import warn_deprecated_kwarg
from diff_diff.results_base import BaseResults
from diff_diff.results_base import BaseResults, _coverage_pct


@dataclass
Expand Down Expand Up @@ -227,7 +227,7 @@ def summary(self) -> str:
# fit time using ``self.alpha``; the displayed level must match them, so
# summary() does not accept an alpha override (it would relabel without
# recomputing the intervals).
ci_pct = int(round((1 - self.alpha) * 100))
ci_pct = _coverage_pct(self.alpha)
width = 88
bar = "=" * width
dash = "-" * width
Expand Down
4 changes: 2 additions & 2 deletions diff_diff/lwdid_results.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
import pandas as pd

from diff_diff.aggregation import AggregationMixin, AggregationResult
from diff_diff.results_base import BaseResults, EventStudyResults
from diff_diff.results_base import BaseResults, EventStudyResults, _coverage_pct


# How the overall staggered standard error was obtained. Cohort effects that
Expand Down Expand Up @@ -549,7 +549,7 @@ def summary(self) -> str:
"""
from diff_diff.results import _format_vcov_label, _get_significance_stars

ci_pct = int(round((1 - self.alpha) * 100))
ci_pct = _coverage_pct(self.alpha)
width = 88
bar = "=" * width
dash = "-" * width
Expand Down
3 changes: 2 additions & 1 deletion diff_diff/lwdid_wild_bootstrap.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@
import numpy as np

from diff_diff.linalg import solve_ols
from diff_diff.results_base import _coverage_pct
from diff_diff.utils import wild_bootstrap_se

_VALID_WEIGHT_TYPES = ("rademacher", "mammen", "webb")
Expand Down Expand Up @@ -106,7 +107,7 @@ def summary(self) -> str:
if self.p_value < 0.01
else "**" if self.p_value < 0.05 else "*" if self.p_value < 0.1 else ""
)
level = int(round((1 - self.alpha) * 100))
level = _coverage_pct(self.alpha)
return (
f"Wild Cluster Bootstrap Results\n"
f"{'=' * 50}\n"
Expand Down
6 changes: 3 additions & 3 deletions diff_diff/rdd.py
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@
rdbwselect,
rdrobust_fit,
)
from diff_diff.results_base import BaseResults
from diff_diff.results_base import BaseResults, _coverage_pct
from diff_diff.utils import safe_inference, validate_covariate_names

__all__ = [
Expand Down Expand Up @@ -308,7 +308,7 @@ def __setstate__(self, state: Dict[str, Any]) -> None:
def summary(self) -> str:
"""Human-readable summary with the three-row rdrobust table."""
width = 72
conf_level = 100 * (1 - self.alpha)
conf_level = _coverage_pct(self.alpha)
lines = []
lines.append("=" * width)
design = "Fuzzy" if self.first_stage is not None else "Sharp"
Expand Down Expand Up @@ -343,7 +343,7 @@ def summary(self) -> str:
lines.append("-" * width)
header = (
f"{'Method':<16}{'Coef.':>11}{'Std. Err.':>11}{'z':>9}"
f"{'P>|z|':>9}{'[' + f'{conf_level:g}% Conf. Int.]':>16}"
f"{'P>|z|':>9}{'[' + f'{conf_level}% Conf. Int.]':>16}"
)
if self.first_stage is not None:
# Fuzzy: R prints a first-stage block above the treatment
Expand Down
Loading
Loading