Skip to content

Index function-call time windows while preserving pandas query semantics - #147

Merged
Essoz merged 8 commits into
codex/inference-identity-cachefrom
codex/inference-time-range-index
Oct 26, 2026
Merged

Essoz merged 8 commits into
codex/inference-identity-cachefrom
codex/inference-time-range-index

Conversation

@Essoz

@Essoz Essoz commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

Repeated function-call time-window queries scan all event rows before grouping, including full-column type membership tests. Add a lazy process/thread timestamp index for conservative native numeric cases, then restore original row order and retain the existing pandas grouping, assertions, and fresh record construction. Unsupported dtypes, mixed-precision/scalar comparisons, and malformed queries fall back to the original mask.

This PR is stacked on #144. The full-trace experiment applies the separate #145 pruning-state correction to both variants; that correctness change is not bundled in this PR.

Measured result

Three sequential alternating pairs on the complete saved tiny-Llama trace (573,494 events, six enabled relations, one CPU worker/native thread):

Pair/order Baseline Indexed
1: baseline, candidate 102.655 s 76.748 s
2: candidate, baseline 101.796 s 76.788 s
3: baseline, candidate 101.610 s 76.658 s

Median 101.796 → 76.748 s: 1.326×, 24.61% less wall time. All six complete outputs match the corrected reference: 836 invariants and 104 failed-hypothesis records, with arrays and multiplicity preserved. Raw output and full/stage context files are also byte-identical across variants. Source/input integrity checks passed; independent audit verified the retained source/output/context evidence and recomputed the statistics.

Median sampled child-session PSS increased 2.129 → 2.168 GiB (+40.15 MiB). The first baseline peak was higher than later baseline peaks; all per-run peaks are retained in the report. Fresh-process timings include lazy index construction. A separate 200k-row/1k-query synthetic benchmark measures 2.662 → 0.340 s, with first-query build cost shown separately.

Validation and limits

  • 18 new differential tests; 61 combined local tests and 67 in the matching Linux runtime, including six shared Isolate pruning state between independent precondition solves #145 tests.
  • Independent parent-method oracle: 144 dtype/offset cases and 6,480 queries in each of two pandas/NumPy runtimes, zero differences.
  • Pinned Ruff, Mypy, Black, and isort checks pass. Production and benchmark checkpoints are signed.
  • Conservative fallbacks cover dtype/null/category behavior, numeric precision, strict/reversed bounds, incomplete/duplicate/exception records, and unusual scalar comparisons.
  • Cache contract requires stable time/type/PID/TID values, dtypes, and row order. Frame replacement or length changes invalidate it; same-size edits require a new TracePandas. Storage grows O(N).
  • Shared host, three pairs on one saved trace, six enabled relations, no CPU affinity; this does not establish training overhead or distributed scaling. CPU Torch matches the original CUDA wheel's source revision, but Python/build differ. Do not multiply this incremental result by the earlier single-pair cache result.

Measurement report and compact provenance include reproduction instructions, all run timings, memory tradeoffs, source/trace/output digests, and retained archive identity. Raw trace/output records remain private study artifacts.

Engineering hardening

Typed numeric time-index storage and query boundaries, retaining conservative dtype/scalar validation and exact fallback semantics (8dc1fd1), plus signed parent hardening updates. All 97 affected inference tests, all three benchmark smoke checks, pinned hooks, and six stronger Mypy 1.9 module/tool/test scopes pass. A new bounded replay of source ce25c12 plus the common #145 production patch emits all 836 invariants and 104 failed hypotheses byte-for-byte identical to the corrected reference; both context files are also identical. The verification run took 76.794 seconds, without a new paired speedup claim. Exact source/runtime/equality evidence is in docs/measurements/inference-hardening-validation-20261025.json.

Separately composed updated production sources pass 140 affected Linux regression tests, whole-source Mypy (77 files), Ruff, and isort. Whole-source Black still reports three unchanged legacy files, down from six in the original base; changed-file hooks pass. Strict checks are scoped and do not establish repository-wide strict typing. Historical performance numbers above retain their original measured revisions; this hardening pass does not re-estimate them.

Integration with current main

Based on #144 (codex/inference-identity-cache), with signed merge checkpoint 9b43ff096c26. The shared base is #150, which reconciles Daniel's parallel-inference tip 122e769 with main ed54c7d. History is preserved; no force push or GitHub PR merge was used. All nine PR heads merge cleanly with their declared bases and current main, and the full stack composes without conflicts.

The resolved composition passes 140 production + 28 harness tests locally (Python 3.10.16 / Torch 2.2.2), pinned Mypy over 77 production files, and stricter Mypy over 9 harness/test files. Conflict-file commit hooks pass. Full-stack Ruff/Mypy/isort pass; Black retains the same three pre-existing formatting failures (config.py, proxy_wrapper/hash.py, proxy_wrapper/subclass.py). Remote CI is ignored because the self-hosted runner is unavailable. These checks validate integration; existing performance measurements retain their original revisions and are not remeasured here.

Land the parent in main first, then retarget this PR to main before merging it. Repeat for its children. Use merge commits to preserve stack ancestry, or refresh descendants after a squash/rebase merge. Exact heads and local logs are retained in outputs/conflict-resolution-20261026/.

@Essoz
Essoz marked this pull request as ready for review October 25, 2026 11:03
@Essoz
Essoz merged commit 4d60091 into codex/inference-identity-cache Oct 26, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant