OpenRec deployable artifacts

This repository is the deployable output cache for OpenRec. Raw inputs belong in example/data/<dataset>/{user,item,event}.csv; recall tables, fitted feature spaces, entity feature snapshots and rank checkpoints belong here.

Treat deployable model files as generated, versioned artifacts. Do not hand-edit checkpoints, feature-space sidecars, manifests, feature-value snapshots, or large recall tables; regenerate and validate the complete bundle instead. The canonical catalog is reviewed source metadata and is updated through its compatibility rules and validator.

default.manifest.json                  # raw-input and output SHA-256 contract
feature/catalog/
├── feature.catalog.json               # implementation-independent canonical feature registry
├── catalog.schema.json                # machine-readable catalog format
├── validate_catalog.py                # catalog and fitted-sidecar compatibility validation
└── README.md                           # ownership and compatibility rules
rank/item/
├── lr.pth
├── lr.manifest.json
├── fm.pth
├── fm.manifest.json
├── lr.features.json                   # fitted LR encoding contract
├── fm.features.json                   # fitted FM encoding contract
├── user_feature.csv                   # point-in-time user values for this rank release
└── item_feature.csv                   # point-in-time candidate values for this rank release
rank/user/
├── lr.pth
├── lr.manifest.json
├── fm.pth
├── fm.manifest.json
├── lr.features.json
├── fm.features.json
└── user_feature.csv
recall/
├── content_i2i.csv
├── item_cf_i2i.csv
├── item_seq_emb.csv
├── hot.csv
├── new.csv
└── user_cf_u2i.csv

Each rank/{target_type} directory is a self-contained default rank release. Its *.features.json defines model-specific column order, vocabularies, scaling and input dimension and is loaded as a file by rank-engine. Its feature CSVs hold actual entity values; InitStandalone converts their event_* columns into the same JSON snapshot shape used by the streaming data-processor.

feature/catalog/feature.catalog.json is the implementation-independent source of truth for the logical features available to rank models. It defines stable meaning, time, identity, invalid-input, aggregation, and materialization semantics. A rank model selects an ordered subset and keeps its model-specific encoding in *.features.json; fitted vocabularies and normalization statistics do not belong in the global catalog. Validate changes with:

python feature/catalog/validate_catalog.py

Build and reuse

Both example modes run example/scripts/ensure-model-artifacts.sh. It hashes the three raw CSVs and reuses this bundle only when every required output exists and its hash matches default.manifest.json. Otherwise it invokes:

python -m tool.build_default_artifacts \
  --data /path/to/example/data/test \
  --model-root /path/to/model

The build uses the first 80% of event time as frozen feature history and the final 20% as rank labels. A clicked impression's preceding expose remains in feature history but is not treated as a negative training label. User ranking derives positives from users sharing positive item behaviour and balances them with non-cooccurring user pairs. Both sides use the persisted UserFeature space; the sidecar's target_type tells serving whether candidate vectors are Item or User vectors. Training restores the best temporal-validation checkpoint and Item LR/FM must pass AUC 0.70 before the bundle is atomically promoted. Recall generation produces item_cf_i2i, content_i2i, user_cf_u2i, semantic-hash item_seq_emb, hot, and new tables from the same inputs.

Cluster-produced defaults may overwrite standalone defaults. The manifest makes this safe: a bundle computed from another raw dataset is stale rather than silently reused.