Repository navigation
Add final report option, fix previous bugs - #3
Merged
Merged
Conversation
`SeedLike = int | str | Iterable[int | str]` is evaluated at import time, so `from __future__ import annotations` does not help and the package failed to import on Python 3.9 (requires-python >= 3.9). The alias is now a typing.Union. tests/test_compat.py checks every source file: no `A | B` between type expressions outside annotations, and the 3.9 grammar parses it.
… stacking Columns were cast to float32 before the selector sort, which ties values that differ beyond 24 bits (16777216 vs 16777217). They are now read at the file dtype (bool -> int8, unsigned -> int64 so that the negation of the sort key is exact) and cast to float32 only in H5Dataset._stack. The example files store float32, so their selection is unchanged (test_lexicographic_order_matches_old_key, test_hh4b_small, test_bbww_small). Test: test_selector_sorts_at_file_precision.
n_folds = len(np.unique(folds)) miscounted non-contiguous ids, so `--exclude-folds next` = (k + 1) % n_folds picked the wrong fold. The loader now raises a ValueError naming the ids; corenet-validate reports FAIL (WARN with --n-folds K, which ignores the column). Tests: test_fold_ids_must_be_0_to_k, test_broken_file[EVENT/fold: fold ids ...], test_fold_ids_not_contiguous_with_n_folds.
A file with neither EVENT/fold nor --n-folds used to fail in H5Dataset, which broke `corenet-evaluate --fold -1` and corenet-export on such files (Dataset.md: the fold is needed for training). The file now loads without folds (has_folds False, every fold -1, n_folds 0, fold_rule "none"); fold_indices, corenet-train and corenet-evaluate with a fold >= 0 stop with a message naming --n-folds K / --fold -1. Tests: test_missing_fold_column, test_file_without_folds (train fails, evaluate --fold -1 and export succeed), test_n_folds_rule.
GhostBatchNorm1d splits a batch with view(nGB, bs // nGB, ...), so --finetune-batch-size 300 with the schedule capped at 16 ghost batches failed in the reshape. finetune_batch now takes the largest number <= the schedule cap that divides the batch size with >= MIN_GHOST_BATCH_SIZE events per ghost batch (ghost_batches); powers of two are unchanged, 0 stays 0. Test: test_finetune_batch_size_divisor (300 -> 15 / 30 ghost batches, a training run).
Prefixes matching nothing froze every parameter at the freeze epoch and the next backward failed. check_finetune_layers raises a ConfigError before the first epoch (and in freeze_heads) naming the prefixes and the prefixes that exist; a single unmatched prefix among matching ones stays allowed, as the default head_reduce. / aux_heads. match nothing in the HH -> 4b network. Test: test_finetune_layers_must_match.
With --early-stop-patience, an empty validation set (--fold 0 --exclude-folds 0) silently disabled early stopping, and a validation fold with a single class crashed in the log line (best_auc None). corenet-train now stops with a message at startup when the validation set is empty and at the first epoch without a finite validation AUC, naming the class counts of the validation folds. Without early stopping both cases run as before. Test: test_early_stopping_needs_validation.
Non-kinematic features of a role used only as `context:` (e.g. btag on `other` in hh4b) were reported by _extras but never fed to the network: _Context.embed had 4 input channels and _prepare_context built the objects from kinematics and the flag only. The context objects now carry 4 + E channels, E = the extra features of the context role; an object of another leaf gets an extra it shares from its own tensor and -1 (the padding value) for one it lacks; the pair features stay kinematic. Without extras the network is unchanged: 1,508 / 3,832 parameters, embed input 4, reference parity. Docs: EventInfo.md, README.md. Test: test_context_role_extras_reach_the_network.
…des the batch size Any --batch-size-init works now, not only powers of two; the schedule is unchanged for powers of two.
corenet-report compares the candidate assignments with TARGETS in file-slot space.
Run summary, training curves, ROC/AUC/scores (test, validation and training folds when corenet-evaluate --fold -1 output is present), confusion matrix and auxiliary heads, view probabilities and pairing accuracy against TARGETS, the inputs as the network embeds them, the raw inputs, the event weights, weight profiles, and the list of what could not be produced. matplotlib becomes a dependency (pixi: matplotlib-base).
… pairing accuracy
…abels in the pairing figure
…ME lists report.html under the evaluation outputs
…ure without inputs is left out - SECTIONS has eight entries; the gaps stay internal and are logged at DEBUG only - the wall-time column of the Folds table appears only when the history records a wall time - notes announcing an absent figure removed (no TARGETS, no training-fold predictions, no history, no prediction file) - tests: the page holds exactly the eight sections in order and no 'not produced' / 'wall time' text; without TARGETS and training-fold predictions there is no pairing-accuracy table and no train / validation AUC or overtraining overlay - docs/Report.md and README: section list and the 'left out' rule
…hors in CITATION.cff
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR fixes the concerns raised in the previously merged PR: #2
Additionally, it adds a
corenet-reportfunction that generates all the relevant final ML results (AUC, loss vs epoch curve, etc) along with the inputs and how they transform at different steps of the network.