Skip to content

Reduce redundant framework checks during trace metadata conversion - #156

Merged
Essoz merged 5 commits into
mainfrom
codex/metadata-classification
Oct 27, 2026
Merged

Essoz merged 5 commits into
mainfrom
codex/metadata-classification

Conversation

@Essoz

@Essoz Essoz commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Metadata conversion repeatedly tests ordinary functions and bound methods against PyTorch tensor/parameter/module types, then discards them. In the final baseline profiles, built-in isinstance consumed 12.39% of profiled MNIST time and 15.27% of profiled Transformer time.

Recognize exact built-in callable types before those framework checks. Preserve _TRAINCHECK values and retain the existing path for custom objects, custom metaclasses and dynamic __class__ behavior. Attribute getters, warning scopes, ordering, identities, tensor contents, hashes, errors and tracing coverage stay unchanged. Changed helpers have explicit type contracts.

Final matched study: five alternating fresh-process pairs on labmach CPU, three cold steps, both API and subclass model tracing active:

Workload Baseline median Candidate median Reduction in traced training-body time
MNIST 667.776 ms 601.247 ms 9.96%
Small Transformer 959.440 ms 782.596 ms 18.43%

Every one of the ten pairs improves. Baseline 7a40649d and candidate 035f7f19 include the same final #154/#155/#157 changes; only dumper.py differs in their 78 executed production files. These numbers supersede the initial study for this final composition. Normal API exclusions remain enabled (use_full_instr=False); this does not measure the broader flag, GPU/MPS/distributed training or selective checking.

All 30 final cases (six correctness, 20 ordinary, four profile) match explicit plain-training references for initial/final model state, every loss and Python/NumPy/PyTorch RNG progression. Raw trace counts/bytes and complete API pairs agree between variants; all tracked parameters show changing data hashes. Hash algorithm/backend and call counts are unchanged. Transformer manual SGD remains intact, but existing observations do not establish gradient snapshots or pre/post observations for every manual update.

The denominator is the traced training body. Whole-process medians do not improve in these short jobs: imports, setup, writer shutdown and shared-host variation dominate. Median sampled peak RSS is 556.8→561.5 MiB for MNIST and 515.7→515.7 MiB for Transformer; full ranges and individual runs are in the report.

Validation:

  • Seven metadata-contract regressions pass on both baseline and candidate under PyTorch 2.2.2 and 2.16 development builds, including exact framework lookup side effects, identities, mutation, warning restoration and propagated errors.
  • Final composed suite: 246/247 pass; the sole test_inject_eval_loop failure also occurs on unchanged main. Pinned hooks and configured mypy across 78 production files pass; strict new-code/test checks with installed Torch types pass. Stronger legacy dumper checking has 22 baseline versus 19 candidate errors, with none introduced.
  • The measurement report and adjacent JSON contain exact commits/trees/data hashes, all pair values, profile denominators, reproducible commands, resource accounting and limits. Raw evidence and checked harnesses remain in the documented experiment directories; the initial study remains in signed history and outputs.

File writing is already buffered/background on this path; metadata conversion and JSON encoding remain synchronous. Final encoding is only 0.31–0.50% of these profiles. Warning setup remains a measured next target; asynchronous live-object conversion needs a separate snapshot/error contract. This PR adds no asynchronous behavior or hashing/algorithm changes.

All commits are signed. Targets main directly. Production change is independent of the other PRs; the final measurements validate the composed result.

@Essoz
Essoz merged commit 892ad8d into main Oct 27, 2026
0 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant