Skip to content

Add final report option, fix previous bugs - #3

Merged
aniketkhanal merged 23 commits into
mainfrom
copilot-review
Oct 22, 2026
Merged

aniketkhanal merged 23 commits into
mainfrom
copilot-review

Conversation

@aniketkhanal

Copy link
Copy Markdown
Collaborator

This PR fixes the concerns raised in the previously merged PR: #2

Additionally, it adds a corenet-report function that generates all the relevant final ML results (AUC, loss vs epoch curve, etc) along with the inputs and how they transform at different steps of the network.

`SeedLike = int | str | Iterable[int | str]` is evaluated at import time, so
`from __future__ import annotations` does not help and the package failed to
import on Python 3.9 (requires-python >= 3.9). The alias is now a typing.Union.
tests/test_compat.py checks every source file: no `A | B` between type
expressions outside annotations, and the 3.9 grammar parses it.
… stacking

Columns were cast to float32 before the selector sort, which ties values that
differ beyond 24 bits (16777216 vs 16777217). They are now read at the file
dtype (bool -> int8, unsigned -> int64 so that the negation of the sort key is
exact) and cast to float32 only in H5Dataset._stack. The example files store
float32, so their selection is unchanged (test_lexicographic_order_matches_old_key,
test_hh4b_small, test_bbww_small). Test: test_selector_sorts_at_file_precision.
n_folds = len(np.unique(folds)) miscounted non-contiguous ids, so
`--exclude-folds next` = (k + 1) % n_folds picked the wrong fold. The loader
now raises a ValueError naming the ids; corenet-validate reports FAIL (WARN with
--n-folds K, which ignores the column). Tests: test_fold_ids_must_be_0_to_k,
test_broken_file[EVENT/fold: fold ids ...], test_fold_ids_not_contiguous_with_n_folds.
A file with neither EVENT/fold nor --n-folds used to fail in H5Dataset, which
broke `corenet-evaluate --fold -1` and corenet-export on such files (Dataset.md:
the fold is needed for training). The file now loads without folds (has_folds
False, every fold -1, n_folds 0, fold_rule "none"); fold_indices, corenet-train
and corenet-evaluate with a fold >= 0 stop with a message naming --n-folds K /
--fold -1. Tests: test_missing_fold_column, test_file_without_folds (train
fails, evaluate --fold -1 and export succeed), test_n_folds_rule.
GhostBatchNorm1d splits a batch with view(nGB, bs // nGB, ...), so
--finetune-batch-size 300 with the schedule capped at 16 ghost batches failed in
the reshape. finetune_batch now takes the largest number <= the schedule cap
that divides the batch size with >= MIN_GHOST_BATCH_SIZE events per ghost batch
(ghost_batches); powers of two are unchanged, 0 stays 0. Test:
test_finetune_batch_size_divisor (300 -> 15 / 30 ghost batches, a training run).
Prefixes matching nothing froze every parameter at the freeze epoch and the next
backward failed. check_finetune_layers raises a ConfigError before the first
epoch (and in freeze_heads) naming the prefixes and the prefixes that exist; a
single unmatched prefix among matching ones stays allowed, as the default
head_reduce. / aux_heads. match nothing in the HH -> 4b network. Test:
test_finetune_layers_must_match.
With --early-stop-patience, an empty validation set (--fold 0 --exclude-folds 0)
silently disabled early stopping, and a validation fold with a single class
crashed in the log line (best_auc None). corenet-train now stops with a message
at startup when the validation set is empty and at the first epoch without a
finite validation AUC, naming the class counts of the validation folds. Without
early stopping both cases run as before. Test: test_early_stopping_needs_validation.
Non-kinematic features of a role used only as `context:` (e.g. btag on `other`
in hh4b) were reported by _extras but never fed to the network: _Context.embed
had 4 input channels and _prepare_context built the objects from kinematics and
the flag only. The context objects now carry 4 + E channels, E = the extra
features of the context role; an object of another leaf gets an extra it shares
from its own tensor and -1 (the padding value) for one it lacks; the pair
features stay kinematic. Without extras the network is unchanged: 1,508 / 3,832
parameters, embed input 4, reference parity. Docs: EventInfo.md, README.md.
Test: test_context_role_extras_reach_the_network.
…des the batch size

Any --batch-size-init works now, not only powers of two; the schedule is unchanged for powers of two.
corenet-report compares the candidate assignments with TARGETS in file-slot space.
Run summary, training curves, ROC/AUC/scores (test, validation and training folds when
corenet-evaluate --fold -1 output is present), confusion matrix and auxiliary heads, view
probabilities and pairing accuracy against TARGETS, the inputs as the network embeds them,
the raw inputs, the event weights, weight profiles, and the list of what could not be produced.
matplotlib becomes a dependency (pixi: matplotlib-base).
…ME lists report.html under the evaluation outputs
…ure without inputs is left out

- SECTIONS has eight entries; the gaps stay internal and are logged at DEBUG only
- the wall-time column of the Folds table appears only when the history records a wall time
- notes announcing an absent figure removed (no TARGETS, no training-fold predictions, no history,
  no prediction file)
- tests: the page holds exactly the eight sections in order and no 'not produced' / 'wall time' text;
  without TARGETS and training-fold predictions there is no pairing-accuracy table and no train /
  validation AUC or overtraining overlay
- docs/Report.md and README: section list and the 'left out' rule
@aniketkhanal aniketkhanal self-assigned this Oct 22, 2026
@aniketkhanal
aniketkhanal merged commit fd30b19 into main Oct 22, 2026
5 checks passed
@aniketkhanal
aniketkhanal deleted the copilot-review branch October 22, 2026 20:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant