OLS Bug + misc

[henrydingliu]

Summary of Changes

fixed a bug in OLS path of weighted regression where mismatching NaN in X and y result in incorrect parameters

deprecated python 3.10 in order to raise minimum sparse version to 0.18 for sp.repeat

added a test to show that current implementation of full triangle weight already works on irregular triangles

moved TriangleWeight and WeightedRegressions tests to their own files

Related GitHub Issue(s)

closes #1326, #1343, #1348

Additional Context for Reviewers

we start the simple case
x = [1, 2, 3, 4, 5]
y = [1, 2, 3, 4, 5]
regression will come back with a slope of 1. ezpz

if one of these pairs of observations are incomplete, such as
x = [1, 2, 3, NaN, 5]
y = [1, 2, 3, 4, 5]
convention dictates that the fourth pair of observations is not considered in the regression, i.e. the weight bacomes [1, 1, 1, 0, 1].

there is also the extreme case of having incomplete, mismatching observations in both datasets
x = [1, 2, 3, NaN, 5]
y = [1, 2, NaN, 4, 5]
we want weight to become [1, 1, NaN, NaN, 1] to ensure we get the correct regression coefficient.

Checklist

• I passed tests locally for both code (uv run pytest) and documentation changes (uv run --directory docs jb build . --builder=custom --custom-builder=doctest)

---

Note

Medium Risk
Changes OLS coefficient logic used by reserving workflows; mitigated by new numpy/sparse tests but still affects numerical results where NaNs were mishandled.

Overview
Fixes weighted OLS in WeightedRegression._fit_ols so observations with NaN in X or y are excluded (weights zeroed) instead of producing wrong slopes when missingness does not line up between X and y. The sparse path applies the same rule by masking weights with valid x/y coordinates.

Adds test_wtd_reg.py covering dense NumPy and sparse COO cases, and moves full-triangle TriangleWeight coverage into test_tri_w.py (including an irregular-grain case).

Support matrix: minimum Python is now 3.11 (CI matrices and requires-python), sparse>=0.18, and Python 3.10 is dropped from classifiers.

^{Reviewed by Cursor Bugbot for commit ebc24f3. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

Pyright Type Completeness

View the full pyright --verifytypes output for this commit

Project (full chainladder package, at this PR's head): 15.2% of exported symbols fully typed (211 / 1384)

	Known	Ambiguous	Unknown	Total
Project (head)	211	111	1062	1384

Other symbols referenced but not exported by chainladder: 13

	Known	Ambiguous	Unknown	Total
Other (head)	3	1	9	13

Symbols without documentation:

• Functions without docstring: 327
• Functions without default param: 0
• Classes without docstring: 10

Patch (exported symbols added or changed by this PR): 60.0% fully typed (3 / 5); 1 no longer exported

	Known	Ambiguous	Unknown	Total
Patch	3	0	2	5

Patch symbol details

Symbol	Status	Change
chainladder.utils.tests.test_tri_w.TestFullTri	❌ unknown	new
chainladder.utils.tests.test_tri_w.TestFullTri.test_triangleweight_full_irregular_triangle	✅ known	new
chainladder.utils.tests.test_tri_w.TestFullTri.test_triangleweight_full_triangle	❌ unknown	new
chainladder.utils.tests.test_wtd_reg.TestOLS	✅ known	new
chainladder.utils.tests.test_wtd_reg.TestOLS.test_missing_data	✅ known	new
chainladder.utils.tests.test_utilities.test_triangleweight_full_triangle	—	no longer exported (was ❌ unknown)

[cursor]

Stale Bugbot comment from a previous run.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.88%. Comparing base (ba25a42) to head (ebc24f3).
⚠️ Report is 40 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1335      +/-   ##
==========================================
+ Coverage   91.76%   91.88%   +0.12%     
==========================================
  Files          96       98       +2     
  Lines        5475     5507      +32     
  Branches      706      709       +3     
==========================================
+ Hits         5024     5060      +36     
+ Misses        327      324       -3     
+ Partials      124      123       -1     

Flag	Coverage Δ
unittests	91.88% <100.00%> (+0.12%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 53ce194. Configure here.}

[kennethshsu]

@genedan do you want to take this one? Regression is not my strong suit at all.

[genedan]

I can try, I think one thing we need to add is review practices on PRs that have a higher level of mathematics than the reviewer does (for me, the bar is not high either).

One day a professor will show up and make a PR with crazy math. If a paper comes from a reputable journal, has a good amount of numerical examples, and the author puts them in the tests, I think we can trust the mathematics.

[kennethshsu]

I agree. I think I mentioned something along the lines of us considering the submitter's actuarial credentials, it helps gauge their background so the reviewer can adjust the review depth accordingly. Maybe credentials aren't a good "bar", but considering their background will definitely help. For example, if a very reputable/credible actuary submits a PR because they implemented their own method, I would consider carefully with any pushbacks, unless it's very obvious.

In this case, I know @henrydingliu and his background, and I also know I am not the most knowledgable person especially when it comes to regressions, I feel I can look at the tests and say ya ok good enough lol

[henrydingliu]

dudes, there's no math in this PR. the changes are entirely around handling nan. i will be more descriptive in the summary