diff --git a/.zenodo.json b/.zenodo.json index 9e38da7..8e25c69 100644 --- a/.zenodo.json +++ b/.zenodo.json @@ -1,7 +1,7 @@ { "title": "CoReNet: Combinatorial Residual Networks for multijet topologies", "upload_type": "software", - "description": "Reference implementation of CoReNet, a configuration-driven combinatorial residual network for jet assignment and event classification in multijet final states, with the HH to bbbb and HH to bbWW examples of the accompanying paper.", + "description": "Reference implementation of CoReNet, a configuration-driven combinatorial residual network for jet assignment and event classification in multijet final states, with the HH to bbbb, HH to bbWW and ttH examples.", "creators": [ {"name": "Khanal, Aniket", "affiliation": "Carnegie Mellon University", "orcid": "0009-0007-5557-9821"}, {"name": "Alison, John", "affiliation": "Carnegie Mellon University", "orcid": "0000-0003-0843-1641"}, diff --git a/CITATION.cff b/CITATION.cff index 74ac8ba..f059209 100644 --- a/CITATION.cff +++ b/CITATION.cff @@ -3,10 +3,10 @@ cff-version: 1.2.0 message: "If you use CoReNet, please cite it as below." title: "CoReNet: Combinatorial Residual Network" type: software -version: 0.1.0 -date-released: "YYYY-MM-DD" # to be confirmed +version: 1.0.0 +date-released: "2026-10-22" license: MIT -repository-code: "https://github.com/[owner]/CoReNet" # to be confirmed +repository-code: "https://github.com/cms-cmu/CoReNet" authors: - family-names: "Khanal" given-names: "Aniket" diff --git a/README.md b/README.md index daedef5..55ea878 100644 --- a/README.md +++ b/README.md @@ -45,8 +45,9 @@ requires by construction but have not all been tested. ## Example -The repository ships two small example files of 20,000 events each, so that everything below runs in -under a minute. The HH → bb̄bb̄ example, from the repository root: +The repository ships three small example files of 20,000 events each (`hh4b_small.h5`, `bbww_small.h5`, +`tth_small.h5` in `examples/data/`), so that everything below runs in under a minute. The HH → bb̄bb̄ +example, from the repository root: ```bash # 1. check the configuration and the data file against each other @@ -68,9 +69,10 @@ corenet-export --checkpoint runs/hh4b/model_fold0.pkl --input examples/data/hh4b corenet-report --run-dir runs/hh4b --input examples/data/hh4b_small.h5 --output runs/hh4b/report.html ``` -The same five commands with `bbww` in place of `hh4b` run the HH → bb̄WW example. +The same five commands with `bbww` or `tth` in place of `hh4b` run the HH → bb̄WW and ttH examples. [docs/HH4b.md](docs/HH4b.md) walks through the HH → bb̄bb̄ example step by step; -[docs/BBWW.md](docs/BBWW.md) does the same for HH → bb̄WW against tt̄. +[docs/BBWW.md](docs/BBWW.md) does the same for HH → bb̄WW against tt̄, and +[docs/TTH.md](docs/TTH.md) for ttH (H → bb̄) against semileptonic tt̄. The decay chain of the HH → bb̄bb̄ example is @@ -101,9 +103,9 @@ two jets with the lowest value instead. The selector reads its column from the f need not be a feature, and a feature named `btag` would be one more network input. The input file holds all jets of the event, and the b jets are chosen when it is loaded. For more details on specifying the event decay, check [docs/EventInfo.md](docs/EventInfo.md), and for the input file itself, check [docs/Dataset.md](docs/Dataset.md). -The full-statistics files (454,261 HH → bb̄bb̄ events, 292,285 HH → bb̄WW events) are distributed -separately on Zenodo ([doi:10.5281/zenodo.23567532](https://doi.org/10.5281/zenodo.23567532)); download them into `examples/data/` as `hh4b_all.h5` and -`bbww_all.h5`. +The full-statistics files (454,261 HH → bb̄bb̄ events, 292,285 HH → bb̄WW events, 423,753 ttH and tt̄ +events) are distributed separately on Zenodo ([doi:10.5281/zenodo.23567532](https://doi.org/10.5281/zenodo.23567532)); download them into `examples/data/` as `hh4b_all.h5`, +`bbww_all.h5` and `tth_all.h5`. ## Usage diff --git a/docs/Dataset.md b/docs/Dataset.md index 02dfad9..ffdae51 100644 --- a/docs/Dataset.md +++ b/docs/Dataset.md @@ -85,17 +85,19 @@ A collection named `x` is read from `INPUTS/x`, matched case-insensitively, or f ## Example -The example files `examples/data/hh4b_small.h5` and `examples/data/bbww_small.h5` (20,000 events each) -follow this layout: +The example files `examples/data/hh4b_small.h5`, `examples/data/bbww_small.h5` and +`examples/data/tth_small.h5` (20,000 events each) follow this layout: * `hh4b`: jets in `INPUTS/Source` (12 slots: `pt eta phi mass btag MASK`, `btag` is 0 or 1), labels in `CLASSIFICATIONS/EVENT/signal` (0 multijet, 1 HH), `EVENT/weight`, `EVENT/fold` (5 folds), `EVENT/event`; * `bbww`: jets in `INPUTS/Jets` (12 slots), `INPUTS/Lepton` (`pt eta phi mass isE isM`), `INPUTS/Met` (`pt phi`), labels in `EVENT/label` (1 HH, 2 tt̄; no class 0), `EVENT/weight`, `EVENT/fold`, - `EVENT/event`. + `EVENT/event`; +* `tth`: the layout of `bbww` without `TARGETS` groups, at least 6 jets per event, labels in + `EVENT/label` (1 ttH, 0 tt̄). -They are random subsamples of the full files `hh4b_all.h5` and `bbww_all.h5`, written with +They are random subsamples of the full files `hh4b_all.h5`, `bbww_all.h5` and `tth_all.h5`, written with `python scripts/make_small_h5.py --events 20000`. Their `EVENT/fold` column equals the fold rule with 5 folds and seed 0 (`--n-folds 5 --fold-seed 0`), so the column and `--n-folds 5` give the same folds on these files; `corenet-validate --n-folds 5` reports that agreement. Check any diff --git a/docs/EventInfo.md b/docs/EventInfo.md index c7c7ec2..2a12d77 100644 --- a/docs/EventInfo.md +++ b/docs/EventInfo.md @@ -79,7 +79,8 @@ min_jets: 4 Optional, default 4: the minimum number of jets per event in your selection. The jet multiplicity enters the network as `log(nSelJets - (min_jets - 1))`, so that the minimum multiplicity maps to zero; an event -with fewer jets than `min_jets` would give a non-finite input. +with fewer jets than `min_jets` would give a non-finite input. The ttH example (`examples/tth/tth.yaml`) +selects at least six jets and sets `min_jets: 6`. ## `hypotheses` diff --git a/docs/TTH.md b/docs/TTH.md new file mode 100644 index 0000000..a7b6cfa --- /dev/null +++ b/docs/TTH.md @@ -0,0 +1,227 @@ +# ttH → bb̄ Example Guide + +tt̄H with H → bb̄, tt̄ → b qq̄′ b ℓν, against semileptonic tt̄ → b qq̄′ b ℓν. The Higgs boson decays to +two b jets, one top quark decays hadronically and the other leptonically. The background, semileptonic +tt̄, has the same objects except the two b jets of the Higgs boson. This guide builds on the +[HH → bb̄WW guide](./BBWW.md) and shows what is new: four b jets shared by three particles, a +`min_jets` above the default, and W candidates that are declared under a hypothesis of their own +because two top quarks use them. + +## Data + +`examples/data/tth_small.h5` (20,000 events: 10,009 ttH, 9,991 tt̄; 7.9 MB) is a random subsample of +`examples/data/tth_all.h5` (423,753 events: 212,067 ttH, 211,686 tt̄; 166 MB), which is distributed +separately on Zenodo ([doi:10.5281/zenodo.23567532](https://doi.org/10.5281/zenodo.23567532)); download it into `examples/data/`. + +The events are generated at 13 TeV and passed through the Delphes fast simulation of the CMS detector: + +* ttH: `p p → t t̄ H` from a MadGraph5_aMC@NLO leading-order gridpack, showered and decayed with + Pythia 8, H → bb̄ forced; +* tt̄: inclusive tt̄ production from Pythia 8. + +Both samples keep the events with exactly one W boson decaying to an electron or a muon and a neutrino +and one W boson decaying to quarks (W → τν rejected), with a generator lepton of pT > 20 GeV. The +selection asks for one tight lepton, a Z-boson veto and a τ veto; jets with pT > 25 GeV, |η| < 2.4 and +ΔR > 0.4 from the lepton; at least six jets and at least two b-tagged jets. + +The layout ([Dataset.md](./Dataset.md)) is that of the HH → bb̄WW file, without `TARGETS` groups: + +| path | shape | meaning | +|---|---|---| +| `INPUTS/Jets/pt`, `eta`, `phi`, `mass`, `btag`, `MASK` | (N, 12) | all jets of the event, 12 slots | +| `INPUTS/Lepton/pt`, `eta`, `phi`, `mass`, `isE`, `isM` | (N,) | the lepton; `isE`, `isM` flag electron or muon | +| `INPUTS/Met/pt`, `phi` | (N,) | the missing transverse momentum | +| `EVENT/label` | (N,) | 1 = ttH, 0 = tt̄ | +| `EVENT/weight`, `EVENT/fold`, `EVENT/event` | (N,) | weight, fold (5 folds), event identifier | + +## Event `.yaml` File + +The decay-chain part of `examples/tth/tth.yaml`: + +```yaml +objects: + jets: {features: [pt, eta, phi, mass], max: 12} # INPUTS/Jets; features = network inputs + b: jets btag 4 # the 4 highest-btag jets: two from the tops, two from the Higgs boson + q: jets rest 2 # the 2 leading remaining jets: the hadronic W + lepton: {features: [pt, eta, phi, mass, isE, isM]} # INPUTS/Lepton, a single object (max 1) + met: {features: [pt, phi]} # INPUTS/Met, a single object (max 1) +event: [nSelJets, HT] +min_jets: 6 + +hypotheses: + ttH: + H: [b, b] # the Higgs boson: one of the 6 pairs of b jets + t_had: [b, W_had] # the hadronic top: a b jet and the hadronic W + t_lep: [b, W_lep] # the leptonic top: a b jet and the leptonic W; 12 assignments of the 4 b jets in all + WW: # declares the two W candidates that both tops use (a shared node needs a parent; + W_had: [q, q] # in bbww.yaml that parent is H_WW); WW has a single candidate + W_lep: [lepton, met] # the leptonic W, from the lepton and the missing momentum + +attention: + - W_lep: W_had + - t_had: t_lep # the two tops never use the same b jet + +relations: # pairs the classifier looks at directly + - [lepton, met] + - [lepton, W_had] + +outputs: + signal: ttH # label 1: ttH; label 0: semileptonic ttbar +combine: [ttH, WW] # WW is listed so that its W candidates reach the output + +training: + features: 8 + attention_heads: 2 + attention_gate: true + kv_projection: true + phase_symmetric: true + reflect_eta: false +``` + +### `objects` + +* `jets`: the jet collection, up to 12 jets per event, read from `INPUTS/Jets`; the four kinematic + columns are its network inputs. +* `b: jets btag 4`: the four jets most likely to be b jets: two from the top quarks, two from the Higgs + boson. The rule reads `INPUTS/Jets/btag` for this choice only. +* `q: jets rest 2`: the two leading remaining jets, the light-quark jets of the hadronic W. +* `lepton`: a single object per event; `isE` and `isM` tell the network whether it is an electron or a + muon. +* `met`: the missing transverse momentum, a single object given by `pt` and `phi`. +* `min_jets: 6`: the selection keeps events with at least six jets, so the jet multiplicity enters the + network as `log(nSelJets - 5)`, zero for six jets ([EventInfo.md](./EventInfo.md)). + +### `hypotheses` + +* `ttH → H t_had t_lep`, `H → bb̄`, `t_had → b W_had`, `t_lep → b W_lep`. The Higgs boson takes one of + the six pairs of the four b jets, and the two remaining b jets go to the hadronic and the leptonic top + in one of two ways: 12 assignments in all. Their probabilities are written to `views/ttH` in the + prediction files. +* `WW → W_had W_lep`, `W_had → qq̄′`, `W_lep: [lepton, met]`: the hadronic W from the two light jets and + the leptonic W from the lepton and the missing momentum. Both tops use these W candidates. A particle + used by two parents is declared once, under a hypothesis of its own, and that hypothesis is listed in + `combine`; `t_had` and `t_lep` then refer to `W_had` and `W_lep` by name. With two light jets and one + lepton, `WW` has a single candidate. + +### `attention` and `relations` + +The leptonic W candidate is compared with the hadronic W candidate, and each top candidate with the +other one; a top never pairs with the top that uses the same b jet. The classifier also looks directly +at the pairs lepton–MET and lepton–W_had. + +### `outputs` and `combine` + +One output, `signal: ttH`, hence two classes: 0 = rest (semileptonic tt̄), 1 = signal (ttH), read from +`EVENT/label`. The classifier is built from both hypotheses in `combine`: `ttH` and `WW`. + +### `training` + +All network settings at their defaults except `reflect_eta: false`. [Options.md](./Options.md) +describes every setting. + +## Training, evaluation and export + +Every command is run from the repository root in the pixi environment of the [README](../README.md). + +1. Validate the configuration against the input file: + + ```bash + corenet-validate --config examples/tth/tth.yaml --input examples/data/tth_small.h5 + ``` + + * `--config`: the event description. + * `--input`: the file to check against it. + + | output | content | + |---|---| + | report on the terminal | every check, ending with `valid` or `NOT VALID` | + +2. Train a quick test on the small file (2 epochs): + + ```bash + corenet-train --config examples/tth/tth.yaml --input examples/data/tth_small.h5 \ + --fold 0 --exclude-folds 1 --epochs 2 --device cuda --output-dir runs/tth + ``` + + * `--fold 0`: hold out fold 0 for testing. + * `--exclude-folds 1`: keep fold 1 out of the training and use it for validation. + * `--epochs 2`: train for 2 epochs instead of 20. + * `--device cuda`: train on the GPU. + + | file in `runs/tth` | content | + |---|---| + | `model_fold0.pkl` | checkpoint; evaluation and export need no configuration file | + | `history_fold0.json` | training settings and per-epoch history | + | `predictions_fold0.h5` | predictions on the held-out fold 0 | + +3. Evaluate the model: + + ```bash + corenet-evaluate --checkpoint runs/tth/model_fold0.pkl --input examples/data/tth_small.h5 \ + --output runs/tth/eval_fold0.h5 + ``` + + * `--fold -1`: evaluate every event of the file instead of the held-out fold 0. + * `--device cuda`: evaluate on the GPU (default: CPU). + + | file | content | + |---|---| + | `eval_fold0.h5` | `class_probabilities` (rest, signal), `views/ttH` (the 12 assignments of the four b jets), `roc/`, `EVENT/` | + | `eval_fold0.h5.json` | the metrics | + +4. Export the model: + + ```bash + corenet-export --checkpoint runs/tth/model_fold0.pkl --input examples/data/tth_small.h5 \ + --output-dir runs/tth/export + ``` + + * `--input`: events on which the exported models are checked against PyTorch. + * `--formats torchscript` or `--formats onnx`: export only one of the two formats. + + | file in `runs/tth/export` | content | + |---|---| + | `model_fold0.pt` | TorchScript model | + | `model_fold0.onnx` | ONNX model | + | `model_fold0.json` | description of the inputs and outputs | + + The exported model takes `b` (N, 5, 4), `q` (N, 5, 2), `lepton` (N, 7, 1), `met` (N, 3, 1) and + `event` (N, 2). + +5. Write the results page of the run ([Report.md](./Report.md)): + + ```bash + corenet-report --run-dir runs/tth --input examples/data/tth_small.h5 --output runs/tth/report.html + ``` + + * `--run-dir`: the directory written by `corenet-train` and `corenet-evaluate`. + * `--max-events 0`: use every event of the input file for the input and weight distributions. + + | file | content | + |---|---| + | `report.html` | the page, figures embedded | + +6. Full training (20 epochs) on the full file, on a GPU, then evaluation of every event and the report: + + ```bash + corenet-validate --config examples/tth/tth.yaml --input examples/data/tth_all.h5 + corenet-train --config examples/tth/tth.yaml --input examples/data/tth_all.h5 \ + --fold 0 --exclude-folds 1 --device cuda --output-dir runs/tth_full + corenet-evaluate --checkpoint runs/tth_full/model_fold0.pkl --input examples/data/tth_all.h5 \ + --output runs/tth_full/eval_fold0.h5 --fold -1 + corenet-report --run-dir runs/tth_full --input examples/data/tth_all.h5 \ + --output runs/tth_full/report.html + ``` + + * `--fold 0 --exclude-folds 1`: fold 0 held out for testing, fold 1 for validation. + * `--fold -1` (evaluation): predictions for every event, so the report shows the training and + validation folds next to the test fold. + + | file in `runs/tth_full` | content | + |---|---| + | `model_fold0.pkl`, `history_fold0.json`, `predictions_fold0.h5` | as in step 2 | + | `eval_fold0.h5`, `eval_fold0.h5.json` | predictions and metrics for every event | + | `report.html` | the results page | + +The commands, their options and their output files are the same as in the +[HH → bb̄bb̄ guide](./HH4b.md). diff --git a/environment.yml b/environment.yml deleted file mode 100644 index 19dcd48..0000000 --- a/environment.yml +++ /dev/null @@ -1,16 +0,0 @@ -# conda / mamba environment: conda env create -f environment.yml && conda activate corenet -# then install the package itself: pip install -e . -name: corenet -channels: - - conda-forge -dependencies: - - python>=3.9 - - pytorch>=2.0 - - numpy>=1.21 - - h5py>=3.0 - - pyyaml>=5.4 - - scikit-learn>=1.0 - - onnx>=1.14 - - onnxruntime>=1.15 - - pytest>=7 - - pip diff --git a/examples/data/tth_small.h5 b/examples/data/tth_small.h5 new file mode 100644 index 0000000..ccfb404 Binary files /dev/null and b/examples/data/tth_small.h5 differ diff --git a/examples/tth/tth.yaml b/examples/tth/tth.yaml new file mode 100644 index 0000000..cad3da1 --- /dev/null +++ b/examples/tth/tth.yaml @@ -0,0 +1,40 @@ +# ttH (H -> bb) with one leptonic top against semileptonic ttbar, on the objects of the bbWW example. +# Input file: tth_all.h5 (Zenodo), layout as the bbWW file without TARGETS groups (CoReNet needs none). +# The selection asks for at least six jets and two b tags; min_jets sets the multiplicity offset accordingly. +objects: + jets: {features: [pt, eta, phi, mass], max: 12} # INPUTS/Jets; features = network inputs + b: jets btag 4 # the 4 highest-btag jets: two from the tops, two from the Higgs boson + q: jets rest 2 # the 2 leading remaining jets: the hadronic W + lepton: {features: [pt, eta, phi, mass, isE, isM]} # INPUTS/Lepton, a single object (max 1) + met: {features: [pt, phi]} # INPUTS/Met, a single object (max 1) +event: [nSelJets, HT] +min_jets: 6 + +hypotheses: + ttH: + H: [b, b] # the Higgs boson: one of the 6 pairs of b jets + t_had: [b, W_had] # the hadronic top: a b jet and the hadronic W + t_lep: [b, W_lep] # the leptonic top: a b jet and the leptonic W; 12 assignments of the 4 b jets in all + WW: # declares the two W candidates that both tops use (a shared node needs a parent; + W_had: [q, q] # in bbww.yaml that parent is H_WW); WW has a single candidate + W_lep: [lepton, met] # the leptonic W, from the lepton and the missing momentum + +attention: + - W_lep: W_had + - t_had: t_lep # the two tops never use the same b jet + +relations: # pairs the classifier looks at directly + - [lepton, met] + - [lepton, W_had] + +outputs: + signal: ttH # label 1: ttH; label 0: semileptonic ttbar +combine: [ttH, WW] # WW is listed so that its W candidates reach the output + +training: + features: 8 + attention_heads: 2 + attention_gate: true + kv_projection: true + phase_symmetric: true + reflect_eta: false diff --git a/pixi.toml b/pixi.toml index d3429e2..47c5f8d 100644 --- a/pixi.toml +++ b/pixi.toml @@ -5,7 +5,7 @@ # pixi.lock records the exact versions; commit it with the code. [workspace] name = "corenet" -version = "0.1.0" +version = "1.0.0" description = "CoReNet (Combinatorial Residual Network): a decay-chain-configurable neural network for event classification in collider physics" channels = ["conda-forge"] platforms = ["linux-64"] diff --git a/pyproject.toml b/pyproject.toml index 0bb615a..51d19af 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" [project] name = "CoReNet" -version = "0.1.0" +version = "1.0.0" description = "CoReNet (Combinatorial Residual Network): a decay-chain-configurable neural network for event classification in collider physics" readme = "README.md" license = "MIT" diff --git a/requirements.txt b/requirements.txt deleted file mode 100644 index 53a1541..0000000 --- a/requirements.txt +++ /dev/null @@ -1,11 +0,0 @@ -# core dependencies (same minimum versions as pyproject.toml) -torch>=2.0 -numpy>=1.21 -h5py>=3.0 -pyyaml>=5.4 -scikit-learn>=1.0 -# model export (corenet-export) -onnx>=1.14 -onnxruntime>=1.15 -# tests -pytest>=7 diff --git a/src/corenet/__init__.py b/src/corenet/__init__.py index d0f8e5d..1e49c99 100644 --- a/src/corenet/__init__.py +++ b/src/corenet/__init__.py @@ -4,7 +4,7 @@ from .models.config import ConfigError, NetworkSpec, load_config, parse_config from .models.network import CoReNet -__version__ = "0.1.0" +__version__ = "1.0.0" __all__ = [ "CoReNet", diff --git a/tests/test_config.py b/tests/test_config.py index 5c0e20b..7c507d4 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -3,10 +3,14 @@ from __future__ import annotations import copy +from pathlib import Path import pytest -from corenet.models.config import ConfigError, parse_config +from corenet.models.config import ConfigError, load_config, parse_config +from corenet.models.network import CoReNet + +TTH = Path(__file__).resolve().parent.parent / "examples/tth/tth.yaml" HH4B = { "objects": { @@ -260,3 +264,14 @@ def test_min_jets(): c["min_jets"] = bad with pytest.raises(ConfigError): parse_config(c) + + +def test_tth_example_parses(): + """examples/tth/tth.yaml: min_jets 6, hypotheses ttH and WW, 12 ttH views, 3,543 parameters.""" + s = load_config(TTH) + assert s.min_jets == 6 + assert set(s.hypotheses) == {"ttH", "WW"} and s.combine == ["ttH", "WW"] + model = CoReNet(s, device="cpu") + assert model.tables["ttH"].nodes["ttH"].n == 12 + n_par = sum(p.numel() for p in model.parameters()) + assert n_par == 3543, n_par diff --git a/tests/test_validate.py b/tests/test_validate.py index 0530a2d..ed3d983 100644 --- a/tests/test_validate.py +++ b/tests/test_validate.py @@ -68,6 +68,7 @@ def test_valid_synthetic(tmp_path, topology): [ (HH4B, "hh4b_small.h5"), (BBWW, "bbww_small.h5"), + (repo / "examples/tth/tth.yaml", "tth_small.h5"), (repo / "examples/hh4b/hh4b_hyperparameters.yaml", "hh4b_small.h5"), (repo / "examples/bbww/bbww_hyperparameters.yaml", "bbww_small.h5"), ],