fix(dml): fail closed when cross-fit learners cannot be isolated (mirror of #816)

[igerber]

Mirror of third-party PR #816 (fixes #814) so the label-gated CI and the CI AI reviewer can run on the contributor's change. The contributor's two commits are carried verbatim; one maintainer commit on top narrows a documentation claim. Do not merge this PR on its own - findings feed back to #816.

Summary

• Reject custom DML learner templates whose deepcopy fails or returns the original object at configuration time, before any group-time cell is fitted (contributor commit 9cd5649).
• Keep lazy per-fold cloning as a fail-closed backstop for direct cross_fit_predict callers; a per-fold clone failure propagates as a sanitized TypeError, never a NaN-cell skip; a failed re-fit clears prior fitted state; fold clones are re-validated for the learner protocol and, on declared survey fits, sample_weight support (contributor commit bb187a3).
• Narrow the "before any cell" wording on the changelog fragment, API guide, REGISTRY, and tutorial 32 so only the one-copy preflight carries that guarantee (maintainer commit 50109fe).

Methodology references (required if estimator / math changes)

• Method name(s): DMLDiD cross-fitting learner isolation (_crossfit.py, _learners.py, dml_did.py)
• Paper / source link(s): Chang (2020), Double/debiased machine learning for difference-in-differences models - the out-of-fold nuisance contract this hardens
• Any intentional deviations from the source (and why): None. No score, weighting, aggregation, or variance change; built-in learner outputs are unchanged.

Validation

• Tests added/updated: tests/test_crossfit.py, tests/test_dml_did.py (copy failure, self-returning __deepcopy__, clone protocol loss, weighted clone loss, panel/RCS hard-error propagation, re-fit state clearing, survey preflight, sanitized traceback, sklearn preflight via importorskip)
• Backtest / simulation / notebook evidence (if applicable): tutorial 32 markdown-only wording update, no re-execution needed

Security / privacy

• Confirm no secrets/PII in this PR: Yes (the token=SECRET-... strings in the new tests are fake sentinels asserting that foreign exception text is withheld)

Changelog

• changelog.d/ fragment added (or N/A - no user-visible change): Yes (changelog.d/20260903-dml-crossfit-learner-isolation.md)

[github-actions]

Overall assessment

✅ Looks good — no unmitigated P0 or P1 findings.

Executive summary

• Affects DMLDiD learner isolation only; scores, weighting, aggregation, and variance remain unchanged.
• Hard failure preserves Chang’s requirement that nuisance learners train only on fold complements.
• Per-fold protocol and survey-weight checks correctly propagate TypeError.
• Failed refits now consistently clear prior fitted state.
• One minor documentation ambiguity remains.

Methodology

• Severity: P3 — informational, mitigated
  • Impact: Only top-level clone identity is verified; a custom __deepcopy__ may still share nested mutable state. This is explicitly documented in docs/methodology/REGISTRY.md:L838-L846, so it is not a defect under the review policy.
  • Concrete fix: None required.
  • Chang Algorithm 1’s auxiliary-sample contract remains intact; no score or SE formulas changed (docs/methodology/papers/chang-2020-review.md:L171-L181, diff_diff/_crossfit.py:L579-L607).

Code Quality

No findings. Clone, protocol, and sample-weight validation are centralized and errors remain targeted.

Performance

No findings. The additional configuration-time copies are bounded and appropriate for validating learner isolation.

Maintainability

No findings. The same helpers cover DMLDiD and direct cross_fit_predict callers.

Tech Debt

No new untracked correctness debt identified.

Security

No findings. Foreign copy-exception messages and traceback context are suppressed at diff_diff/_crossfit.py:L70-L86; tests cover secret-bearing sentinel messages.

Documentation/Tests

• Severity: P3
  • Impact: The fallback-era phrase “deep-copies the template where copyable” remains in diff_diff/dml_did.py:L294-L298 and docs/methodology/REGISTRY.md:L2930-L2937. It is ambiguous now that uncopyable templates always raise.
  • Concrete fix: Replace it with “requires a distinct deep copy and raises TypeError otherwise”; explicitly say callers must supply an initially unfitted template.

Test coverage is strong across panel/RCS, survey weighting, protocol loss, sanitized errors, and failed-refit cleanup. Runtime tests could not be executed because pytest/NumPy are unavailable; changed Python files passed AST parsing, the notebook passed JSON parsing, and the changelog-fragment checker passed.