Repository navigation
Conversation
This is the first step in making base tree "regressor" be used by all the tree classifiers. Next step is to make the implementation generic over the used Criterion with callers deciding it. Add a Criterion trait with MSE and Gini implementations in src/tree/criterion.rs. Use MeanStats in BaseTreeRegressor in place of explicit sum/mass bookkeeping. - Store node stats in NodeVisitor; remove true/false_child_output - Keep scratch stats in SplitWorkspace to avoid new allocations - Rank splits by proxy improvement; store the real impurity improvement in split_score - Pass the sample count to find_best_cutoff and check min_samples_split at the caller
Make sure no leaf has a split_core. Add tests to check this. Also check that no "proxy" values are leftover by verifying that the sum of all split_scores adds up to the total impurity decrease.
- Compute the MSE improvement as (S_T·M_F − S_F·M_T)² / (M_T·M_F·M_P). This prevents cancellation for targets with a large offset. - Remove rows with mass 0 in grow(). Zero-weight rows no longer count toward min_samples_leaf/min_samples_split and do not move thresholds. - Remove the MASS_EPS clamp from difference(). - In split(), add the false side directly instead of parent − true. - Add tests for target-offset invariance and zero-weight rows.
Add compute_feature_importances (mean decrease in impurity, as in scikit-learn) to DecisionTreeRegressor, RandomForestRegressor and ExtraTreesRegressor. The forests normalize the importances of each tree, take the mean and normalize again. The fitted models store new fields (root_mass, num_features), so models serialized with an earlier version do not deserialize.
BaseTree only finds the leaf. DecisionTreeRegressor and BaseForestRegressor convert the leaf output to TY.
The forest passes a Criterion to its trees. The predict methods are restricted to Output = f64. The RF and ET regressors use MSECriterion.
BaseForest only gives access to its trees and samples. The mean of the tree predictions is in predict_mean and predict_mean_oob.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #502 +/- ##
===========================================
+ Coverage 43.97% 85.70% +41.72%
===========================================
Files 85 97 +12
Lines 7281 9490 +2209
===========================================
+ Hits 3202 8133 +4931
+ Misses 4079 1357 -2722 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
BaseTree and BaseForest do not store the criterion. Only the type of the leaf value is kept in the tree. Make the types generic over this output type and move the Criterion type parameter to the fit functions. Move output_approximate_eq from Criterion to a new LeafOutput trait on the output type.
…into make-generic
Mec-iS
approved these changes
Oct 24, 2026
Merged
Collaborator
|
Superseded by #504 (v0.7 Release): these changes are merged into the v0.7 release candidate on |
Collaborator
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #501
Follows up on PR#497
Checklist
Current behaviour
BaseTreeRegressorandBaseForestRegressorare hard-coded for regression: MSE splits,f64leaf values and mean predictions. The classification trees and forests cannot reuse them.New expected behaviour
The base tree and base forest are generic over the output type of the leaves. A split
Criterionis passed to the fit functions. The two structs have no regression-specific code:BaseTree<.., Out: LeafOut>: training uses only the criterion passed to fit (target,add,leaf_value, ...), and prediction returns the leaf output (leaf_output_for_row).BaseForest<.., Out: LeafOutput>: uses the criterion in fit and gives access to the trees and the bootstrap samples.TY, mean over the trees, OOB mean) is now inDecisionTreeRegressor,predict_meanandpredict_mean_oob.RandomForestRegressorandExtraTreesRegressoruseMSECriterion.No public API or behaviour change: the modules are internal, and the predictions are identical. The created trees are identical and the performance is the same.
Change logs
Changed
BaseTreeRegressor→BaseTreeandBaseForestRegressor→BaseForest(with their parameter structs and files).LeafOutput, and move the regression prediction code to the regressors.This prepares
DecisionTreeClassifierandRandomForestClassifierto use the same base types.