[MaxGhenis]

Finite anchored period recursions are evaluated in full regardless of previously calculated periods, tracing, or persistent cache policy. Unanchored recursions still use the existing depth limit, and values reached by a cut are discarded when the calculation ends.

Repair and invariants

• Deeper periods are scheduled iteratively. Once a deferred key starts evaluation, recursion cannot defer it again; its pending calculation is retried after deeper dependencies finish. For a fixed registered branch graph with finitely many reachable calculation keys, scheduling makes progress without a cache-dependent 10,000-deferral budget.
• Only actual readable inputs of the variable's own period unit, or formula boundaries in the direction of recursion, authorize deeper evaluation. Deleted inputs and derived replacements cannot remain anchors through stale registry entries.
• Outstanding scheduling requests are recorded before they are raised and checked outside formula exception handlers before publishing a result.
• Deferred periods retain the caller scope required by requires_computation_after.
• A temporary cache holds completed, cut-free deferred results independently of tracing, _do_not_store, and cache blacklisting.
• Spiral cleanup removes exact derived entries from their owner; independent descendant inputs and contained periods survive. Clones drop copied cut values before becoming usable. Existing explicit invalidation behavior remains separate.
• Flat traces prefer accepted completed calculations over abandoned or provisional attempts, retaining their dependencies and parameter accesses ahead of later cache reads.

Round-2 findings

All eight review findings are addressed: descendant input deletion; swallowed scheduling exceptions; lost prerequisite caller; invalid clone snapshots; carry-over from calculated values (settled by master's input-only provenance); None retry traces; tracing/cache-policy differences; and cache-dependent deferral exhaustion. The inherited memory sharing, input/derived metadata, disk directory lifetime, clone aliases, system entities and reform baselines from master are preserved.

The original 200-example recurrence order property is retained. Added differential properties compare finite forward/backward recurrences with an independent iterative NumPy evaluator across tracing, storage policies, guarded formulas and caller scope; additional properties cover cut copies, descendant inputs and deleted input provenance. Exact deletion is compared across copied memory, shared memory and disk snapshots.

Previous validation

Targeted, serial single-file checks: 245 passed in total.

File under tests/core/	Result
test_spiral_order.py	109 passed; includes a cold/warm 10,002-month anchored chain
test_spiral_order_property.py	1 passed, original 200 generated examples retained
test_spiral_scheduler_property.py	4 passed, 60/40/24/24 generated examples plus explicit examples
test_spiral_storage.py	12 passed
test_spiral_trace_retries.py	6 passed
test_cycles.py	5 passed
test_fast_cache_guards.py	5 passed
test_clone_invalidated_caches.py	14 passed
test_carry_over_order.py	4 focused provenance cases passed, 60 deselected
test_docs_deployment.py	5 passed
test_simulation_copy_pickle.py	78 passed after the final master merge
test_simulation_copy_pickle_property.py	2 passed after the final master merge, 60/40 generated examples

The final merge leaves the repaired scheduler/storage/tracer source and spiral tests unchanged. The eight deleted-input cases also passed again through the offline uv run launcher before the final clean master merge. Ruff format/check passed for all 19 repaired or incoming Python files, and git diff --check passed. Final copy checks emitted only the existing NumPy generic-timedelta deprecation warning.

Previous repaired head: f34b54c569c6d700ca18046e8ff106c0379ce5e3.

Baseline checks on the history-preserving merge 2ed6c34db0f5e6eae1b9dee6397c5c034938bb93 reproduced 32 failures, with 42 passing and 27 deselected cases. Carry-over cases passed there because master's input-only provenance already repairs them. The additional deleted-input regression failed in all eight cases before the provenance fix.

The fresh dependency install failed on PyPI DNS. Successful checks use the existing read-only Python 3.13 environment, with this checkout first on PYTHONPATH; the final focused provenance and copy checks launch it through offline, frozen, no-sync uv run. The first run of the added recurrence differential file hit Hypothesis's input-generation timing health check (one failed, three passed); only too_slow was suppressed for its two bounded generators, with every example count and assertion retained, and all four tests then passed. No full suite, country tests, partner baselines, documentation build or new microsimulation was run. Earlier full-suite and impact results are historical evidence only.

Composition and status

This remains draft and held for Max's core-semantics merge decision. Master 7399c4ed8b9401febd546512cc55ba1c3e896ab0 is merged without rewriting history; the current merge and validation are recorded below. #576 is included, and this PR's exact Holder.delete_array now calls its _evict_fast_cache while preserving input protection. Current-head green CI and downstream impact remain required before landing. No decision ID is associated with this bounded repair.

Conditional formula-only base cases and defined_for masks still use the existing cut behavior. Mixed-direction finite recurrence graphs have less differential coverage than same-direction anchored graphs; their existing regressions and the original order property are preserved.

Historical downstream evidence

The following measurements belong to the original PR head and its original scratch merge. They have not been repeated on f34b54c569c6d700ca18046e8ff106c0379ce5e3; current-head impact and partner gates remain pending.

Real runs on master b78b0ba and on all three of #571/#572/#573 merged together (scratch merge 37aadada), each output compared array by array (compare.py). The PE-US 2026 run was also repeated on each branch alone, and the 2035 run on #572 alone; all identical too.

Run	Outputs compared	Result
policyengine-us 6c5170fd, eCPS 2024, 3,000-household subsample, 2026 (income tax, itemizing and SALT branches, state taxes, CTC, EITC, SNAP, household and SPM net income, weights, marginal tax rates)	25 arrays	bitwise identical
same, 2035	25 arrays	bitwise identical
policyengine-uk 7b9fc379, enhanced FRS 2024-25, full sample, 2026 (income tax, NI, UC and legacy benefits, Pension Credit, HB, council tax, HBAI income, poverty, marginal tax rates)	36 arrays	bitwise identical

A probe of the PE-US 2026 and 2035 and PE-UK 2026 and 2042 runs on master found one recursion cut by the limit: PE-US weeks_worked in 2035 (below).

PE-US weeks_worked (formula_2025: last year's value) is the one recursion the probe saw reach the limit on real data: a new 2035 simulation cuts it at 2025. The eCPS file carries no weeks_worked, so every year is the default 0 and the cut changes nothing above. Where the input exists it does. A real policyengine-us run on a household with one worker (40 weeks entered for 2024):

2035	New simulation	After calculating 2030
master: weeks_worked, spm_unit_work_expenses	0, $0	40, $1,800.14
this branch	40, $1,800.14	40, $1,800.14

So household calculations from 2035 on (ten years after the 2025 formula start) change, to the value calculating year by year gives.

Merge with master

Merged master 7399c4ed8b9401febd546512cc55ba1c3e896ab0 into the confirmed PR head f34b54c569c6d700ca18046e8ff106c0379ce5e3 with merge commit e5455b8a306ca71e8795f1d443a17ed73c207923. Current head: 7bc9a284efd96915ca1242e5e84d2dcd7e573722. The branch history is preserved and this PR remains draft for Max's core-semantics ruling.

Conflict resolutions (two source files, six conflict blocks):

• simulation.py imports: retain the scheduler's dataclass field, and master's Lock and Tuple imports.
• simulation.py calculation cleanup: retain master's store-history propagation and calculation-frame/counter teardown, then end the trace, release the scheduler's cut frame, and purge invalid results. Deferred exceptions unwind both sets of state.
• simulation.py formula execution: retain master's formula-simulation context token/reset and the PR's outstanding-deferral check after the formula returns. A formula cannot swallow a scheduling request and publish its fallback value.
• simulation.py fast cache: keep master's input-unchanged guard and the PR's cut-result marking inside that guard, so obsolete input-dependent results are not cached and cut-derived cached results are purged.
• simulation.py clone setup: retain master's independent store history/requested-variable state and reset its active frames/input contexts before dropping the PR's copied cut values. Reform baselines, aliases and system entities retain master's clone behavior.
• holder.py cache writes: keep master's _set(..., is_input=False, derived=derived) and store-history recording, together with the PR's _note_cached_value. Calculated writes retain provenance and are marked for exact cleanup when reached by a cut.

Non-textual interactions and fixes:

• Exact holder deletion now evicts master's fast cache, forgets actual supplied-input provenance only when deleting the supplied value, and drops both storage tiers' new sequence metadata. Derived-only cleanup still protects inputs and contained periods. Four memory/disk regressions cover this integration.
• The reported CI run 37847869045 was read with gh run view --log-failed. On the merged code, the unchanged test_every_clone_has_a_baseline_of_its_own reproduced the exact set() == {('income_tax', '2022-01')} failure. This was a PR ownership bug, independent of the depth heuristic: a parent purge consumed a child's explicit invalidation. Commit eac7ec566431f7b5ea9bdffefda10b2048e7ac58 restricts descendant cleanup to cut-derived values. The original 60-example property now passes unchanged. Ten deterministic cases cover nested branches, tracing, mixed invalidations, an explicit/cut overlap, and cloning before cleanup. Overlapping cut values are deleted exactly while the explicit marker remains pending.
• Commits 1d8e8e255a21395c93ce8539524a8269e51ecfbb and 7bc9a284efd96915ca1242e5e84d2dcd7e573722 integrate master's input-change retries and ADD/DIVIDE caches. Completed deferred values retain foreign dependencies and mutation versions, are invalidated only for their own/dependent simulations, and reopen only completed keys; pending keys still guard cycles. Mutating inputs use master's existing ten-retry bound; stable anchored scheduling retains no deferral-count limit. Stale deferred frames cannot publish finished values. Unrelated scratch-simulation writes do not consume the retry budget. Direct option results that encountered a cut are returned without storing them, preserving master's routing, dtype, provenance and input-set-meanwhile rules.
• Master's test_value_a_spiral_defaults_is_tracked initially failed because it searched for retained cut-derived storage. That storage is deliberately purged by this PR. It now derives the cut period from the recorded dependency history, asserts the cut cache is empty and the dependency is recorded, and retains the original fresh-simulation differential assertion. Its entire file passes (98 tests). No clone-baseline invariant or generated example count was weakened.
• Anchoring rules and unanchored depth limits are preserved. Master's subsampling rebuild, restore provenance, disk-directory lifetime, input helpers and branch fixed-point behavior are covered by the focused master files below.

Validation at the current head: 1737 passed, 2 expected failures, 0 skipped across 91 files, each invoked independently as uv run pytest -q <file> in the foreground. Every test used OMP_NUM_THREADS=4 OPENBLAS_NUM_THREADS=4 MKL_NUM_THREADS=4 VECLIB_MAXIMUM_THREADS=4 NUMEXPR_NUM_THREADS=4 OMP_WAIT_POLICY=PASSIVE. The workspace-local environment was installed with uv sync --extra dev; test invocations used frozen/no-sync uv with CPython 3.14.7's free-threading build (h5py enables the GIL). No folder run, full suite, country run or microsimulation was performed.

The first 14 integration cases reproduced 12 failures before the cache-integration repair; all 24 final integration cases pass, including changing own/foreign inputs, six successive settling input changes, an unsettled deferred frame, continuously changing inputs, cut ADD/DIVIDE input precedence, and unrelated scratch writes. The 10,002-month cold/warm anchored-chain regression and the original 200-example order property pass.

File under tests/core/	Result
test_apply_reform_invalidates_cache.py	1 passed
test_apply_reform_preserves_user_inputs.py	9 passed
test_branch_helper_invalidation.py	8 passed
test_branch_input_fixed_point.py	23 passed
test_branch_input_fixed_point_property.py	2 passed
test_branch_input_invalidation.py	98 passed
test_branch_input_invalidation_property.py	1 passed
test_branch_scoped_delete_arrays.py	6 passed
test_branch_shared_arrays.py	47 passed
test_branch_shared_arrays_differential.py	1 passed
test_carry_over_order.py	64 passed
test_carry_over_order_property.py	4 passed
test_clone_invalidated_caches.py	24 passed
test_clone_rebinding.py	24 passed
test_clone_rebinding_property.py	1 passed
test_clone_reform_baseline.py	22 passed
test_clone_reform_baseline_property.py	1 passed
test_cycles.py	5 passed
test_data_storage_dir.py	40 passed, 2 xfailed
test_data_storage_dir_property.py	1 passed
test_dataset_default_role.py	41 passed
test_dataset_default_role_property.py	3 passed
test_disk_storage_differential.py	1 passed
test_disk_storage_ownership.py	33 passed
test_dump_restore.py	1 passed
test_fast_cache.py	7 passed
test_fast_cache_contained_periods.py	7 passed
test_fast_cache_contained_periods_property.py	2 passed
test_fast_cache_guards.py	5 passed
test_holder_branch_fallback.py	5 passed
test_holder_write_fast_cache.py	16 passed
test_holder_write_fast_cache_property.py	2 passed
test_holders.py	22 passed
test_input_tier_invalidation.py	5 passed
test_join_with_persons_identity.py	23 passed
test_join_with_persons_identity_property.py	6 passed
test_macro_cache_not_singleton.py	2 passed
test_on_disk_storage_clones.py	33 passed
test_opt_out_cache.py	3 passed
test_option_result_cache.py	24 passed
test_option_result_cache_property.py	2 passed
test_reform_baseline_system.py	17 passed
test_registered_input_deletion.py	54 passed
test_restore_input_provenance.py	2 passed
test_restore_input_registry.py	14 passed
test_restore_input_registry_property.py	1 passed
test_set_input_helper_order.py	60 passed
test_set_input_helper_order_property.py	3 passed
test_simulation_builder.py	35 passed
test_simulations.py	9 passed
test_spiral_merge_integration.py	24 passed
test_spiral_order.py	109 passed
test_spiral_order_property.py	1 passed
test_spiral_scheduler_property.py	4 passed
test_spiral_storage.py	16 passed
test_spiral_trace_retries.py	6 passed
test_stale_frame_cache.py	60 passed
test_storage_anchored_periods.py	5 passed
test_storage_delete_branch.py	3 passed
test_storage_input_index.py	18 passed
test_storage_shared_index.py	22 passed
test_storage_shared_index_differential.py	1 passed
test_subsample_baseline_branch.py	2 passed
test_subsample_computed_values.py	7 passed
test_subsample_computed_values_property.py	1 passed
test_subsample_deleted_inputs.py	21 passed
test_subsample_entity_weight_totals.py	28 passed
test_subsample_inputs_only.py	14 passed
test_subsample_inputs_only_property.py	1 passed
test_subsample_invalidates_cache.py	2 passed
test_subsample_membership_ids.py	8 passed
test_subsample_restore_inputs.py	8 passed
test_subsample_sampling_period.py	22 passed
test_subsample_sampling_period_property.py	1 passed
test_subsample_structural_ids.py	27 passed
test_subsample_weight_fallback.py	15 passed
test_tax_benefit_system_cache_identity.py	2 passed
test_tracers.py	42 passed
test_tracing_parameter_isolation.py	49 passed
test_tracing_parameter_isolation_properties.py	1 passed
test_uprating_order.py	141 passed
test_uprating_order_property.py	5 passed
test_user_input_keys.py	29 passed
test_user_input_keys_property.py	3 passed
test_user_input_keys_soundness.py	67 passed
variables/test_non_numeric_uprating_and_defined_for.py	36 passed
test_simulation_copy_pickle.py	78 passed
test_simulation_copy_pickle_property.py	2 passed
test_option_result_input_provenance.py	10 passed
test_disk_period_filenames.py	25 passed
simulations/test_formula_randomness_concurrency.py	1 passed

Ruff on all 91 Python files changed by the PR or incoming master: ruff check → All checks passed!; ruff format --check → 91 files already formatted. git diff --check passed. Documentation review updated the storage/cleanup contract and retained the tracer retry documentation; no documentation build was run.

The old body is saved as untracked evidence in hub-evidence/core572-body-before.md; before/after failure logs, the fix diff and individual-file results are retained there too. The PE-US A/B gate and impact will be rerun by the hub at this merged head. Current-head CI and Max's core-semantics decision remain pending; no review was requested and no PR comment was posted.

axiom: n/a: core engine repair, no policy encoding.

Generated with Claude Code.