[MaxGhenis]

Cuts the CI time of policyengine_us/tests/core/test_behavioral_response_measurement_copies.py (#9913) and keeps every case it ran. Only that test file and a changelog fragment change.

Before this PR, the file took 1,512 s of the Rest core group's 43:46 in main's push run 40031737740; the group took 11:51 on 2026-10-08 (run 38141803955), before the file existed. Since #10439 the file runs in a process of its own (core-2): 1,399 s of test time and 24:13 elapsed in #10439's PR run 40038526568, and 1,523 s and 26:22 in main's release run 40050303812.

Where the time went

The file builds no Microsimulation: every test uses one-person California households. The time goes to copying policy.

• Parameter tree copies. clone_spm_system deep-copies the whole parameter tree: once per reform simulation built, twice per measurement (the reform branch with clone_system=True, and _baseline_measurement_branch) and once per clone().
  • Under cProfile, building a reform simulation and measuring it once took 44.4 s, of which the three tree copies took 25.5 s.
  • Within that, Parameter.clone (323,037 calls) took 23.1 s cumulative, including 16.4 s in ParameterAtInstant.clone (3.77 million calls).
  • In policyengine-core 3.32.27 both deep-copy their metadata (copy.deepcopy(self.metadata)), and Parameter.clone also clones each of its values. copy.deepcopy took 21.4 s over 15 million calls.
• Cycle collection. One tree copy adds 6.4 million objects the collector tracks, and a full collection right after the copy frees none of them. Every full collection traverses every live tree. At the default first threshold (2,000), three rounds of building, measuring, cloning and branching a reform simulation ran 29 full collections.
• Hypothesis. test_a_copy_reports_the_response_for_its_inputs_and_measurement_state took 597 of the 1,512 s (run 40031737740). Under max_examples=3, derandomize=True it ran four explicit pins plus three generated examples. With this test's old strategies, Hypothesis 6.168.3 and CPython 3.14, those three were the same on every run. Each example builds a reform simulation, makes up to two measurements and makes one copy.

The [None]/[reform1] and [entity]/[period] parametrizations each exercise a distinct construction path, so they stay:

• [None]/[reform1]: a simulation with and without a baseline;
• [entity]/[period]: a holder dropped for its entity or for its period.

Changes

1. A module-scoped fixture raises the collector's first threshold from 2,000 to 100,000 while these tests run, and restores the previous thresholds at teardown.
   • The threshold is process-wide, so while this module runs it applies to every collection in its pytest process.
   • Over the three rounds above, full collections fell from 29 to 8.
   • Higher thresholds cost memory: at 500,000, the same rounds peaked at 10.2 GB, against 4.8 GB at 100,000.
   • Freezing the heap at module start (gc.freeze(), with the same threshold) was 6.8% faster again in two paired local runs (415 s against 445 s). It is left out for that gain: it takes every object alive when the module starts, from whichever modules ran before it in the process, out of collection until teardown.
2. The Hypothesis test runs only its explicit examples (phases=[Phase.explicit]). Four of the five pins build their source as one simulation of three households, at 20,000, 50,000 and 100,000.
   • A copy given other wages gets (50,000, 100,000, 20,000), so each household moves to another's level.
   • A fifth pin keeps the old third generated example as it was: a lone 20,000 household (zero response under this reform), measured, cloned and moved to 50,000. It is the only source whose measured response is zero for every household; review found that the three-household sources do not cover that state.
   • The checks are unchanged. A clone, or a branch made before the source measured, is compared household by household with a new one-household reform simulation at the same wages (fresh_response concatenates them). A branch made after measuring is compared, as d1021 requires, with the source's measured responses scaled by its wages, and with a snapshot of the source's measurements.
   • Dropping phases makes Hypothesis generate examples from the new strategies, not the old ones: three households in one source, with the copy's wages a permutation of theirs.

The test now builds 8 reform simulations, makes 10 measurements and makes 3 clones, against 10, 12 and 4 before, counting the one-household references it builds and measures.

Coverage of each removed or merged case

Each pin keeps its kind and after_source_measured. The old pins at 50,000 are household 2 of the new ones:

Old case	kind, measured first, source → copy wages	Now covered by
Pin 1	branch, no, 50,000 → 50,000	pin 1, household 2
Pin 2	clone, no, 50,000 → 50,000	pin 2, household 2
Pin 3	clone, yes, 50,000 → 100,000	pin 3, household 2
Pin 4	branch, yes, 50,000 → 100,000	pin 4, household 2
Generated 1	clone, no, 20,000 → 20,000	pin 2, household 1
Generated 2	branch, no, 100,000 → 100,000	pin 1, household 3
Generated 3	clone, yes, 20,000 → 50,000, lone household	pin 5 (same case); also pin 3, household 1

The pins also run five household cases the old test never ran:

• clone before measuring at 100,000;
• branch before measuring at 20,000;
• clone after measuring, 100,000 → 20,000;
• branch after measuring, 20,000 → 50,000;
• branch after measuring, 100,000 → 20,000.

Mutation check

A pytest plugin patches the code before the tests import it. It re-introduced the three defects the module docstring says #9913 fixed, the first in a representative form (see below), and one break of invariant 3. Every old case and every new pin ran one by one under each (local runs of the four-pin version):

Mutant	Old cases that fail	New pins that fail (households)
none	none	none
baseline measured under the reform (_baseline_measurement_branch returns get_branch("baseline"))	pin 1, pin 3, gen 2, gen 3	pin 1 (households 2, 3), pin 3 (households 1, 2, 3)
clone shares its source's measurements	pin 3, gen 3	pin 3 (households 1, 2, 3)
branch made after measuring starts with none	pin 4	pin 4 (households 1, 2, 3)
branch shares its parent's dictionary	none	none

• Every old failure reappears, with the same actual and expected values, as a failing household of the pin that now covers it. For example, under the first mutant old generated 3 got 3,248.13 against 315.82, and so does pin 3's household 1.
• The last mutant survives both versions of this test. test_copies_own_their_measurements and test_a_branch_measures_a_new_period_without_touching_its_parent, both unchanged, fail under it.
• The first mutant is a representative wrong baseline, not the pre-Measure a branch's behavioural responses against baseline policy, with its own cache #9913 construction: that built a clone_system=True branch under get_branch("baseline").

Pending, held while the host is reserved for another run (results to follow here):

• the pre-Measure a branch's behavioural responses against baseline policy, with its own cache #9913 construction as its own mutant;
• review's counterexample, a clone that keeps its source's measurements when they gave a zero response, which only old generated 3 and pin 5 can catch;
• pin 5 under every mutant.

Timings

CI, this file in its own process (core-2), per-test JUnit times: two before runs, then this PR at the four-pin head 3037f63 (run 40052943537) and at the five-pin head 2ce982c (run 40089604888).

Test	Before, run 40038526568 (s)	Before, run 40050303812 (s)	Four pins, run 40052943537 (s)	Five pins, run 40089604888 (s)
test_a_copy_reports_the_response_for_its_inputs_and_measurement_state	556.5	621.5	307.3	382.8
The other 15 tests	842.4	901.4	666.4	663.4
File total	1,398.9	1,522.8	973.7	1,046.2
Process elapsed (GNU time)	24:13	26:22	17:08	18:20
Process max RSS	5.68 GB	5.68 GB	5.93 GB	5.80 GB

• Against the two before runs, the file's test time fell 25% and 31% at the five-pin head (30% and 36% at four pins), and the 15 tests this PR does not otherwise change ran 21% and 26% faster. The fifth pin costs about 75 s.
• In run 40052943537, shard 1, which ran only this file, took 17m40s, the shortest of five shards; the longest took 28m25s. In run 40089604888 (six shards) it took 18m51s and the longest 23m55s.

Local, four-pin version: full pytest totals and process wall times. Each process ran its tests serially, all four concurrently on a shared 18-CPU host under load 25 to 35:

Version	pytest total (s)	Process wall (s)	Peak RSS (GB)
Before	635.2	649.1	5.99
Pins only	563.0	577.0	6.15
Pins + threshold 100,000	430.7	447.7	6.27

• From before to this PR's version, peak RSS rose 4.7%; the threshold alone added 2.0% (6.15 to 6.27 GB).
• In a later pair of runs, this version took 445.3 and 445.4 s, and the gc.freeze() variant 415.2 and 415.3 s. Compare rows only within one table.

Invariants

The file's four invariants (module docstring) and the tests that check them are unchanged. Only the Hypothesis test changes: which households one pinned run covers, plus the restored lone-household pin.

Shards

#10439 plans the Rest shards from the JUnit reports of recent successful release runs on main. The first release run after this merges records the file's new times, and plans made after that run use them. No Makefile change.

Review: GPT-6.1 Sol (Subfleet, hard tier) requested changes at 3037f63: the lone zero-response source, unsaved figures, and wording. This revision answers them; the re-review follows once the pending runs are in.

axiom: n/a: test-only CI cost change.

🤖 Generated with Claude Code