Repository navigation
Move the UK spine to geography first (#931) - #1538
Draft
juaristi22 wants to merge 61 commits into
Draft
juaristi22 wants to merge 61 commits into
juaristi22 wants to merge 61 commits into
Conversation
…ine-block machinery (#931) Per-clone engine blocks in the dense measure step, and the per-clone spine batches that follow, run policyengine-uk over part of the pool, so the formulas that read beyond their own households see the part. This adds the pieces the next commits wire in. - microcosm.frame.adapters.engine_population is country-agnostic: numpy and duck-typed simulations, no engine import. It holds the pool-share correction c_b = f_b * S_b / S with its zero rules, checks (a) and (b), and injection into a simulation, its baseline and every sibling, refused once a branch exists or a recorder is armed. A per-instance _run_formula recorder, which core's clone rebinds into later branches, and a known-periods walk feed a judge over declarations, parameter predicates and pass policies. A held value is judged unless the simulation holding it ran it after arming, and a run reaches the position window by the last year its own period covers. The verdict carries names, periods and booleans only. Household batches, the nesting check, the scatter back to one engine's order, a float32 share bound that follows numpy's pairwise-sum leaves and counts the adds forming each key, and a per-row bound for values that add share terms to others complete it. - microcosm.frame.adapters.policyengine_uk_population classifies pe-uk 2.174.1's 19 WHOLE_POPULATION_VARIABLES: three corrected shares, two stored inputs, eight zero at policy, attendance inert through private_school_vat_change, six refused ranks. It adds the window from REBALANCING_YEARS (2026 on), the constructor's structural reform factories, sha256 pins of the sources the classes rest on, the spine batch column, and uk_engine_population(), which imports pe-uk and validates only when called, failing closed. Nothing imports either module yet, so no kernel, stage or target hash moves and no output changes. Engine-free tests drive the machinery on fake core-style simulations, covering each guard condition a small change to the module could drop. Engine tests hold the declaration to the installed engine: list, adds, pins and what they cover, predicates at 2024 and 2025 and moved off current law, reform factories, window, the readers of attendance and private-school VAT, and drift lists of axis-free reductions and of whole-array branches in formulas and the engine functions they reach. They correct two synthetic clones that differ to one engine over the pool within the float32 bounds, household_tax per row, with household-local outputs bitwise equal. A build test holds the batch column and the share inputs to the build's own constants. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The annual-income simulation carries each input's other spells from the dataset's year by the factor the engine projects that input by. The factor was the median of the engine's own year-on-year ratios over the rows the simulation holds. That is a statistic of whichever block ran, so per-clone engine blocks that differ would carry the same spell by different factors, and no block would match one engine over the pool. Following María's ruling of 6 Nov, the factor is now the product of one plus the engine's own rate, for each year after the dataset's, in the growth series policyengine-uk's dataset projection uses for the input: average earnings for pay, mixed income per head for profit. The rates are read from the simulation's parameters. The engine's rows must still agree with the factor within 1e-5, or the resolver refuses, as before. This moves data on today's national and dense lines. The median sat within float32 rounding of the parameter, so an annual input with other spells can move by one float32 step. The *_annual matrix entries, calibrated weights and H5 bytes can move, and other_spells_engine_factors changes in every resolver receipt that carries it. A K=1 parity check should expect *_annual inputs within one float32 step, not an identical H5. The full target and problem kernels and the national target kernels re-key with measure_simulation; no spine stage does. An engine-free test holds the factor to the parameters and to the rows' tolerance. An engine test pins the declared series to policyengine-uk's own uprating map and checks the factor on a synthetic frame. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he whole-population formulas it runs (#931) After the move the dense measure step's per-clone engine blocks are no longer identical copies, so policyengine-uk's share formulas (shareholding, corporate_land_value, consumption_shareholding) would read a block's denominator instead of the pool's. - The shared engine_population module gains EngineBlock, the life of one block on its own engine. It validates the declaration, refuses the position window and periods before the data, and runs checks (a) and (b). It sets each share formula to its own value times c_b = f_b * S_b / S in the simulation and its baseline, and arms the formula-run recorder. Every simulation built later gets the same arrays and recorder. At the end it checks that each still holds the block's keys and weights, then judges them all. A record pass records refusals and a raise pass raises; work after the verdict is refused. The country passes its declaration and a pass. validate_pool_shares and PopulationDeclaration.correction_reads come with it. - UKMeasureResolver takes engine_pool_shares (scratch mode only) and opens an EngineBlock with the UK declaration and the measure pass, which records and never raises. A sibling that would set a key, its inputs or the weights is refused. close_engine_block judges the block. - full_measure builds each block's pool shares from the totals the representation already records and passes them to the block's resolver. After the blocks run, exact holds only if every block validated the declaration, passed both checks, applied the correction in every simulation it built, kept its keys and weights to the end, agreed with the others on its growth and annual-income factors, and got a clean verdict. exact_basis names this basis. exact_scope says float32-level, not bitwise, and names the Universal Credit rebalancing draw the window refuses. The identical-copies checks stay as diagnostics. block_sensitivity matches the land rows' engine variables, so present_in_this_run is no longer always empty. - The two frame modules join UK_TARGET_COMPILE_MODULE_NAMES. The fake engine the shared tests use moves to test_support so the build tests can use it too. No data changes on today's lines. Identical copies take a correction of 1 and give the same matrix and local metrics, bitwise, as without it, and a single-block resolver records none of the new keys, so national receipts keep their bytes. Today's K=15 dense receipts change by design: - the representation gains exact_basis, share_correction, population_reach and annual_income_projection_factors; - exact_scope is rewritten, and the release manifest's engine_measurement limitation copies it; - block_sensitivity is rewritten; - each block's resolver receipt gains engine_pool_shares and engine_population_reach; - exact now also needs a clean guard verdict, a stricter release guard on that line. The full and national target kernels re-key; no spine stage does. On a synthetic pool of three clones that differ, three blocks match one block over the pool: household-local and annual rows and every local metric bitwise, and the land rows and each block's share formulas within the float32 bound. Uncorrected land rows miss by more than 1%, and a decile read, a 2026 period and a counterfactual on corporate wealth are each refused. Engine-free tests run EngineBlock on fake engines. They drive checks (a) and (b), a sibling that moves its keys or weights, the window and a raise pass, and hold its receipts to the representation's finalisation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…931) One simulation over the move's K=15 spine pool does not fit in memory, so the UK adapter runs a call over more than one clone clone by clone. It decides from the frame, never from a constructor flag: a household table carrying household_clone_index with more than one clone batches, and a frame without the column, or with one clone, takes the single simulation it always has. National never batches. - The shared engine_population module gains materialize_in_batches and frame_key_sums. Before any engine is built, the driver judges the call's requests: a period in the rebalancing window, a refused rank, a stored formula the frame does not supply, and attendance or its levels and carriers while it is unstored. It forms each batch's share from the frame, c_b = S_b / S at unscaled weights. Each batch, in ascending clone order, then runs an EngineBlock under the spine's raising pass: checks (a) and (b), the pool's shares set in the simulation and its baseline, the recorder armed, the requested values, the verdict. The engine is deleted and collected before the next is built. Every batch must grow the keys and the weights by one factor. People come back in the frame's order and groups by id, as one engine returns them. The driver refuses a recording pass, since its arrays carry no verdict. - PolicyEngineUKEngine passes the UK declaration and the spine pass. It gains simulation_batches(frame), parameters(period) for the stage-52 restatement, and batch_record(). Its optional clones, the spine's K once G1 adds it, refuses a frame carrying any other number of clones, so no call runs per clone or over a sub-pool. microcosm.frame.rules and the RulesEngine.materialize signature are unchanged. - The spine sidecar's rules_engine gains engine_batches only once a call batched. The engine identity is still hashed from the provenance alone, and no batch cap is added. - graph_kernels binds engine_population and policyengine_uk_population beside the adapter in all 16 engine-calling stages. A test derives those stages from their engine calls. No data changes on today's lines: no spine frame carries household_clone_index before G1, so every call takes the single simulation, unchanged. Identities move. Every spine stage kernel hashes graph_kernels.py, so every spine node re-keys, and the 16 engine stages also bind the two modules. engine_population.py is a target compile module, so the full and national target kernels re-key again. The spine graph declaration (the H2 fixture uk_spine.json), source_stages.json and the coverage manifest are unchanged. An engine test runs a synthetic three-clone frame whose clones differ, interleave by id and list people out of order, at the dataset year and projected a year on. Against one simulation over the same frame, the share formulas match within the float32 bound, household_tax within its per-row bound, and household-local values bitwise, with people and groups in one engine's order. A one-clone frame is bitwise the single simulation. A batched attendance request, a decile, a 2026 period, unstored months and a frame with fewer clones than K are each refused before any simulation is built. Engine-free tests run the driver on fake engines, one engine alive at a time. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ine values (#931) policyengine-uk's attends_private_school ranks household net income over the people a simulation holds, so once a spine's engine calls run clone by clone, one clone's simulation would rank that clone alone. Stage 52 now restates the formula's last step itself, over every person in the frame: the members' income tax and National Insurance summed per household as policyengine-core sums them, net income from the household's market income and benefits, the household's weight once per member, microdf's MicroSeries.percentile_rank (the rank the formula calls) over the people of positive weight, the attendance rates and population factor from the engine's own parameters (parameters(period), at the two paths PRIVATE_SCHOOL_ATTENDANCE_PARAMETERS names), the one-million floor, and the stored draw for under-18s. Every input is the engine's own value at the calibration year, decided by a household's own records. - Where the engine computes the frame in one simulation (national, and every spine today), the stage asks for the engine's attendance beside those inputs, stores the engine's flags, and raises if the restatement differs from them bit for bit, so each such build re-checks it on its own data. Where the adapter computes the frame clone by clone (simulation_batches above one), it asks for the inputs only and stores the restatement. An engine without simulation_batches runs one. - The evidence gains rank_route (engine_with_parity or pooled_restatement) and engine_parity_checked, a boolean: whether the engine's flags were checked. A failed check raises, so neither records a mismatch. weighted_pupils stays. The parity error names the formula, never a count. - The stage's notes in sources.yaml describe both routes. The notes in source_stages.json are re-derived from it, and the release input coverage manifest is regenerated (its source_manifest_sha256 fields). - Left for G1, which makes the clone-by-clone route live and regenerates the spine fixtures: the stage's source line ("evaluated by the pinned engine"), the notes' sentence that the stage materializes the engine's attendance, and the H2 fixture's sources/fixture.json, which keeps the stage's earlier notes until tools/graph_uk_spine_fixture.py runs. All three hold while every call takes one simulation. The store_rules_engine_input operation describes both routes: each stores attendance as an engine input from the stored draw, at the calibration year and the spine's design weights. No data changes on today's lines: no spine frame carries household_clone_index before G1, so every call takes one simulation and stores the engine's flags, as before. Identities move. Stage 52's kernel hashes private_school_attendance.py, so stage 52 and every node after it re-key. The packaged resources enter the full and national target kernels' country-spec fingerprint, so those re-key again. National's stage-52 evidence gains the two fields. The spine graph declaration (uk_spine.json) is unchanged, since stage contracts do not cover notes. An engine test runs a synthetic pool of three clones of twelve households: some alike in every clone and some not, most with several children, weights spread about one to ten, ties, a household of no weight, and each clone under the floor while the pool is over it. Household ids interleave the clones and the people are listed out of order. Each child's draw sits 3% under or over the rate the formula gives it over the pool, from the formula's own percentiles and the engine's parameters, so a rank, rate, interpolation or factor a few percent off would store other flags. A spy on microdf's percentile_rank shows the restatement hands it the formula's net incomes and weights and gets its percentiles back, bit for bit, in one simulation and clone by clone. The stage run clone by clone stores what one simulation stores, where a clone's own simulation imputes nobody; a population factor of zero, or 6% lower, breaks the parity check; below the floor nobody attends on either route, while design weights just under it impute on both, since the formula reads the engine's weights at the calibration year; and a frame of one clone stores the engine's attendance as the stage asked for it before. Engine-free tests run both routes, the parity refusal and the floor on a fake engine with a stand-in rank. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…raph doc (#931) The dense runbook and the local pre-flight still said a per-clone engine run was exact when its blocks were identical copies scaled to the pool. Exact now means every block corrected to the pool's share denominator with a clean guard (exact_basis pool_share_injection_with_population_guard): equal to one engine over the pool to float32 rounding, not bitwise, with nothing computed in policyengine-uk's Universal Credit rebalancing years (2026 on). - docs/uk-dense-release-assembly-runbook-762.md says so where it describes --engine-blocks, the pre-flight and the engine_measurement limitation. It adds that a calibration year or dated measure's year of 2026 or later leaves every per-clone run inexact and refuses every batched spine call, and that the pre-flight does not require exact_basis, so a receipt from before the correction passes too; that is open for Max. - docs/uk-full-build-graph.md gains a section on the corrected measure blocks and their guard, on how a spine engine call over more than one clone runs clone by clone and what refuses it, and on stage 52's rank over the whole frame. - tools/preflight_uk_local_release_candidate.py: the docstring and the comment over the engine check only. The gate code is unchanged. test_uk_local_release_preflight gains two cases: a per-clone resolution whose representation has another mode, or another block count, fails however exact it says it is. - Two changelog fragments cover group (0). One describes the engine blocks, the spine batches, stage 52 and the shared modules. The other describes the annual-income factor for other spells, the one data change, which can move an annual input by one float32 step on both lines. Only comments, docs, fragments and tests change. No kernel hashes these files, so no node key, H2 fixture, source_stages.json or coverage manifest changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…estatement (#931) Review follow-ups to batching spine engine calls by clone. - The spine sidecar's engine_batches is now read from the final frame, never from the calls this process ran. A resumed spine whose engine stages hit their checkpoints runs no kernel, so it under-reported batches; now it records what a fresh run does. PolicyEngineUKEngine.batch_record(frame) takes the same decision and refusals as simulation_batches. - PopulationDeclaration gains restated: stored formulas a caller may compute over the pool from household-local engine values, with the parameters each reads. Each pe-uk simulation builds and edits its own parameter tree, and parameters(period) is the adapter's fresh copy. So a new check_restated_parameters holds every batch's tree to the adapter's wherever a restatement reads it. The UK declares attends_private_school with its attendance rates and population factor, the parameters stage 52 reads. - PolicyEngineUKEngine(clones=...) accepts any whole number. A frame that drops the clone column still takes one simulation; G1's integration check rules that out after G1. - The test that finds the engine-calling stages matches a materialize call on any receiver, and any transform taking an engine. It keeps isa_subscriptions, which only checks the country, on an asserted allow-list. No data changes on today's lines: no spine frame carries household_clone_index before G1, so every call still takes one simulation and no sidecar gains engine_batches. Identities move again: every spine node re-keys through graph_kernels.py and the engine modules, and the full and national target kernels re-key through the two frame modules. The spine graph declaration (uk_spine.json), source_stages.json and the coverage manifest are unchanged. An engine test adds a frame carrying none of the keys' inputs, which gives the pool's values bitwise. A batch whose own tree moved a restated parameter is refused before it computes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The WAS donor's income, DVTotinc_bhcR8, is net regular household income before housing costs less council tax: HBAI's concept. Both WAS-trained stages matched it to the engine's household_net_income, which also takes off stamp duty, the TV licence and the business rates and corporate stamp duty that shareholding spreads over the households the engine simulates, and adds benefits in kind. At was_lisa (stage 24) those shares already read the corporate wealth was_wealth had just drawn, at the spine's weights before calibration. Both stages now read the engine's HBAI net income before the reset to zero, the concept LCFS, ETB services and NTS read (#1164), named hbai_household_net_income on donor and recipient (María's ruling, 6 Nov). - was_wealth materializes UK_HBAI_NET_INCOME_VARIABLE, and UK_WAS_ENGINE_PREDICTOR_NAMES gives it the predictor name UK_HBAI_NET_INCOME_PREDICTOR; the donor cleaning maps DVTotinc_bhcR8 to the same name. The chain's segments, their order and seeds are unchanged. - Its derive step declares the donor column in direct_maps, and the drift check refuses a declared direct map the cleaning does not read. - was_lisa takes the WAS household predictor, so the ownership logit, the balance forest and the realised profile's income tertiles read it. The profile key becomes by_hbai_household_net_income_tertile and the logit's feature log1p_hbai_household_net_income. Its declared engine reads are now what it materializes (UK_WAS_LISA_ENGINE_PREDICTORS): num_adults and num_children, from the FRS family roles since #1095, leave the list. - sources.yaml carries the declarations and both stages' notes. source_stages.json is re-derived from it, the release input coverage manifest is regenerated (its source_manifest_sha256 fields), and so is the H2 fixture: sources/fixture.json carries the two stages' declarations and notes, and stage 52's notes from 019f5d4, which waited for the tool to run; uk_spine.json carries the two stages' new contract hashes. No synthetic fact in the fixture moves. - docs/uk-lisa-1003.md and a changelog fragment describe the concept. What moves: every WAS draw can move on both lines, and with them the Lifetime ISA draws and the later stages that read wealth (means-tested capital, CGT selection, property components, high-value support). The size is unmeasured; a sampled f010 arm measures it. was_wealth, was_lisa and every later spine node re-key. Tests: the WAS recipient reads the engine's HBAI income, not a household_net_income decoy, and the donor maps DVTotinc_bhcR8 to the same name; the packaged declarations name both sides; drift to the old engine read or a wrong direct map is refused; the LISA recipient and donor carry the WAS predictor and its declared engine reads are what it materializes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of 640b2cf. Its text said the WAS and Lifetime ISA stages read one income concept on both sides. The code matches the ruling, but the two sides are not the same income: policyengine-uk's HBAI net income also counts HBAI's in-kind items (free school meals, school milk and fruit, the free TV licence's value, Healthy Start vouchers, the Tax-Free Childcare top-up) and the Warm Home Discount, and the WAS user guide defines the income behind DVTotinc_bhcR8 only as total net regular household income less the council tax paid, so how it treats pension contributions, maintenance paid, student loan repayments and negative incomes is not verified. - The was_wealth and was_lisa notes, docs/uk-lisa-1003.md, the changelog fragment and the code comments call the recipient's income the engine's closest concept to the donor's and name those residuals. The was_wealth notes keep the income among the predictors that are not fully like-for-like. - household_net_income is described as adding the imputed value of NHS, education and transport spending, not "benefits in kind", which HBAI's income also counts in part. - Only the LCFS and ETB services stages name the income as a predictor; NTS reads the same engine variable to build gross income. The comments and test docstrings say so, and the two tests are renamed to match. - The changelog's What moves adds the calibrated weights on both lines. - source_stages.json is re-derived from sources.yaml (the two notes), the release input coverage manifest is regenerated (its source_manifest_sha256 fields), and the H2 fixture's sources/fixture.json carries the two notes. uk_spine.json does not move: notes are outside the stage contract hash. What moves: nothing in the data or any contract hash. Text only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#931) María's ruling of 6 Nov on the move, in line with workstream 2 of microcosm#1521 and the first must item of Vahid's review there. Two of the three ONS land rows cannot usefully bind weights: - ons.land.corporate_land_value (E43B, £2,063,534m): policyengine-uk's corporate_land_value hands out its aggregate by each household's share of the weighted corporate-sector wealth, so a released file meets that aggregate at any weights. Bound to the column measured at the starting weights, the row pinned only the weighted mass of the WAS corporate and private pension wealth, which no other row targets, not land. - ons.land.land_value (E44R, £6,872,455m) counts the land of financial corporations, government and NPISH, which the engine's land_value does not hold. With the household row (E44N, £4,485,171m) and the corporate row it asked for £323,750m that no weights could supply. Both get entries in uk/calibration_measure_exclusions.json in the obr.council_tax_england shape (tracking microcosm#1521, approved 2026-11-06, expiring 2027-02-06) and are measured on every evaluation. ons.land.household_land_value stays bound. Its binding note in uk/uk_population_targets.json said households_and_npish; E44N is the households sector alone, as the Chronicle package labels it, and the note now says so. The contract is hand-written. tools/generate_uk_target_references.py reads no binding note, so target_references.json and the membership report do not move (their regeneration test needs the pinned Chronicle feed, which is not on this machine, and skips here). What moves: the solve drops two rows on both lines, so calibrated weights move slightly; a calibrate-only rerun on the move's spine attributes it. Released corporate land value still equals the engine's aggregate. Tests: the packaged register's count and census take the two rows, and a new test pins their fields, the published values the reasons quote against target_reference_membership.json, the household row staying bound and its corrected note. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…931) Follow-up to 410e822. Its test pinned the register's two entries, their fields and the published values they quote, but not what the entries do to a run. The new engine-free test compiles the three committed ONS land references with stand-in values and applies the packaged register's land entries, as both target loaders do after reconciliation. The solve surface keeps ons.land.household_land_value alone, the exclusion receipt carries the two entries' review fields, and the compiled register, which every run keeps beside the solve surface as its band-edge register (a national run writes it as national_contract_registry.json), still holds all three. On a synthetic four-household frame the national problem encoder binds the household row alone, with the receipt and the compiled register's digest in its bindings, while the compiled register materializes and scores all three rows at their weighted values. What moves: nothing. Test only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of 410e822 and 161c777. The two entries hold the right rows, but some of their text contradicted itself, the record of what moves was short, and one claim in 410e822's message was wrong. - The total-economy reason, the changelog, the test docstring and the census comment now tie the £323,750m gap to the rows: E44R also counts the land of financial corporations, government and NPISH, which neither E44N nor E43B covers, and policyengine-uk's land_value is the sum of the two columns those rows read. The old text said the engine's land_value does not represent or hold financial corporations' land, while the corporate entry says policyengine-uk describes its corporate aggregate as the land of non-financial and financial corporations. That entry now says "describes as", not "labels". - The census comment no longer says the corporate row is met by construction: the column meets the engine's own aggregate at any weights, not E43B. The adjudications give Vahid's suggestion in his order (met by construction once the engine's parameter comes from the target's ONS series), and say the calibrate-only rerun is to attribute the change, since it has not run. - The two held rows' binding notes in uk/uk_population_targets.json record the hold, as the other held rows' notes do. The parity register's land concern names the two exclusions; uk_data_target_parity.json is regenerated with tools/census_uk_data_target_parity.py, and --check passes. The UK_BLOCK_SENSITIVE_MEASURE_COLUMNS comment and its test's comment no longer say the national targets bind the two variables. - 410e822's message said the target-reference regeneration test skipped because the pinned Chronicle feed is not on this machine. The feed is here. tools/generate_uk_target_references.py, run with the test's arguments on that feed (facts and manifest hashes and the row count match uk/chronicle_feed.json) and on this commit's contract, reproduces target_references.json and target_reference_membership.json byte for byte. It ran directly, through the engine slot, rather than inside pytest, whose own load of the 10 GB feed would double the memory. The local generator keeps only local-level contract rows, so the land rows' notes cannot reach the local surface. What moves, all from 410e822 (this commit is text, a regenerated parity register and tests): - Both lines split each family's equal share of the objective evenly over its bound rows (the dense line within each grain, where the land rows are amounts). ons.land.household_land_value, now the ons_land family's only bound row, carries the family's whole share: three times its loss weight before, on both lines. No other row's weight changes. The calibrate-only rerun measures this together with the two dropped rows. - The national frozen scoring register (national_target_registry.json, the approved register) loses the two rows, so a national run given a frozen register derived before 410e822 refuses, as with obr.ni_employer. The dense loader checks the reconciled register, which keeps them. - The incumbent and release-cut scores, uk_target_fit and calibration_diagnostics.json read only the rows the solve binds, so the two rows leave them. - Neither line resolves land_value or corporate_land_value any more: the dense measure receipt's national_inputs falls by two, and a per-block dense run's block_sensitivity.present_in_this_run is empty. Evaluations that score the un-excluded register (tools/evaluate_uk_incumbent_surface.py) still measure the two rows, as they do every held row; 161c777's test shows the compiled register can score them, doing the scoring itself. - The dense release lists the two exclusions among its limitations, each with the first 300 characters of its reason, which cite only published ONS figures and code facts. - The edited files are UK country-spec resources, so the national and dense target, measure and solve nodes re-key; full_measure.py feeds only those kernels. Spine nodes, source_stages.json, the release input coverage manifest and the H2 fixture do not move. The regional_home_values stage note still says the home value reaches three national rows, ons.land.land_value among them. It changes with group (c)'s rewrite of that stage, which regenerates source_stages.json and the H2 fixture anyway. Tests: test_the_corporate_and_total_land_rows_are_measured_diagnostics also pins the "4.7 percent" the total reason quotes and the two held notes. test_the_two_land_diagnostics_are_measured_but_not_fitted adds a bound row of another family (obr.tv_licence_fee) and pins the household row's family_equal weight at three times its share beside the other two land rows, with the other row's weight unchanged; its docstring says it applies the entries as the loaders do, without calling them. test_uk_measure_simulation, test_uk_full_measure, test_uk_data_target_parity, test_uk_population_targets, test_uk_incumbent_surface_evaluation, test_spec_source_citations, test_uk_target_references (its feed cases skip without CHRONICLE_UK_FACTS; the generator run above stands in for the national one) and test_uk_solve_scope pass, and ruff is clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every UK copy layer took its id offset from ten to the digit count of the largest id it saw. Once (b) clones households at the spine's start, every offset would grow with K, and the K=1 national spine would stop being the clone-0 slice of the K=15 spine. Copy ids now come from fixed decimal fields declared in uk_runtime/copy_id_table.py, so every copy id is a function of its source id and its copy path, at today's stage order. The table is UK-local (María's rulings of 7 Nov); the shared table, a declaration digest and the int32 check come with the follow-up stage. - The table, least significant first: vintage (10 ordinals at 10**8, kept for pooled FRS vintages; nothing mints it), geographic_support (100 at 10**9), spi_support_channel (3 at 10**11: the SPI copy at 1, the band donor at 2), high_value_support (6 at 10**12), cgt_support_split (10,000 at 10**13), cgt_incidence_clone (2 at 10**17) and cgt_residential_split (8 at 10**18). The largest mintable id, about 7.2e18, is checked below int64 when the module loads. Helpers: radix, require_below, mint (refuses an ordinal outside its field, a row that already carries one and an int64 overflow), ordinal and parent. - Mint sites: frs_spine refuses raw ids at or above 10**8. Stages 16 (with its _remap_ids, which had no overflow check), 17 (sources read from the source-id columns), 26, 42, 43 and 46 mint through the table. Stage 42 refuses a household needing more than 10,000 copies with a fixed message. The anchor (45) pairs each twin by the twin field's radix. The executor takes each id-offset stage's declared radix, refuses an incumbent id at or above it and recovers parents on exact int64 ids; the row walk it replaces upcast all-numeric rows to float64, inexact above 2**53. Refusals print counts, never ids. The tests-only replace path mints through the table and refuses a base whose ids reach the SPI field. - Dense: the dense expand keeps its multiplier above the spine's largest id. A table-minted spine's twins and arms leave no int64 room above them, so every dense entry point (uk.full.expand, the ladder and crosswalk clone routes, the row-wise tool's plans) refuses n_clones > 1 in one message naming #931 and the clone-first spine. Dense K>1 builds wait for (c). K=1, and a spine without copy rows, expand as before; the expand kernel no longer multiplies an empty column by a multiplier above int64. - Declarations: the id_remapping texts in sources.yaml, source_stages.json and the code dicts name the table field. stack_zero_weight_donors and stack_income_band_donor_households gain the key, required by sources.schema.json and checked against the code. copy_id_table is bound into frs_spine and every minting stage, spi_support into stages 16 and 17, rowwise_geography into 43 and 45 and cgt_support into 26. What moves: base FRS rows keep their ids. Every copy row's five id columns change: SPI copies +10**11, band donors +2*10**11, high-value copies k*10**12, CGT copies k*10**13, twins +10**17, arms j*10**18. Relative id order is unchanged at K=1, so sorts, ties, ranks and folds are unchanged. Values move on copy rows through every draw keyed on the copy's own id: stage 18's stage-2 fill, stages 19-24, 27 and 28-41, every twin's Advani-Summers prior (so the CGT gains layer re-realises), stages 48-54 and policyengine-uk's warm_home_discount_random_draw on copy households. Base rows move second-order through pooled steps: stages 26-28's selections and walks, the residential solve, the anchor, the CGT walks and calibration. Draws keyed on person_source_id do not move. National ids pass 2**53 from stage 42 on. Dense K>1 builds are refused until (c). Every node re-keys. The coverage manifest's source digests and the H2 fixture are regenerated: the fixture's declarations, and its synthetic CGT asset-type, property purchase and rental facts, which are sized to the synthetic spine's copy-row values. Left for 1b: the identity tool's decoders still read the retired rule, so 11 tests in test_uk_identity_stability_receipts fail until 1b; also size_evaluation's decoders, the remaining id-printing messages and the stochastic_assignment binding sweep. Tests: the table's fields, bounds, order and ceiling, and the helpers' round trips and refusals; each copy stage's ids against its roots, found by lineage; a K=1 chain through all six layers sorts as the retired rule did and keeps base ids; clone-first K=1 and K=3 frames nest exactly and the retired rule does not (test_support/microcosm_build/ uk_clone_first_frames.py); the anchor pairs at a one-digit CGT ordinal; the executor's refusal and parents, exact above 2**53; the dense refusal and the K=1 path; the fixed CGT message; an AST guard against the retired rule; the binding tests. H2 parity passes on the regenerated fixture. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three reviews of 22bcdbe found texts that overstate or misstate what it does, and refusals and dense paths that no test drives. This commit corrects the texts and adds the tests. The identity tool's decoders and the other items ruled into 1b stay in the next commit. - Stage 46's notes said the geography identity kernel and the identity tool read the arm rule. The kernel reads lineage columns, and the tool decodes the retired rule until 1b, so the notes now name the executor only, as the module docstring does. source_stages.json is re-derived, and the coverage manifest's source digests and the H2 fixture's notes are regenerated. - The row walk the executor replaced read a row of only integer and float columns as float64; an all-integer row stays int64. The _source_lineage docstring and the changelog now say so, and the exactness test carries a float column, on which the replaced walk fails. - The dense refusal fires wherever the dense offset would overflow int64, as a spine with residential arms at multiples of 10**18 does at K >= 2. Its docstring, its message and the changelog now say that. require_below's message no longer says that only a later stage may fill the field. - The changelog says each copy row now sits at its parent's ids plus its layer's offset, where it used to sit a power of ten above the largest id its layer saw. It also says that national ids pass 2**53 from the twin on (and at stage 42 from ordinal 901), that only the new refusals print counts, that create_uk_spi_support_tables no longer takes id_multiplier, and that copy rows join across versions by lineage, not by build ids. - The CGT split's bound check moves into a helper so its boundary can be tested. The lineage test helper reports counts, never ids or keys. What moves: no value or id. Every node re-keys again, through the text in graph_kernels.py, copy_id_table.py and rowwise_dataset.py and the helper in cgt_support.py. Tests: - Stage 16 on clone-first K=1 and K=3 frames nests exactly. - The executor takes the declared radix where the frame's maximum would give another, names each stage's own field, and refuses parentless rows by count. - The id_remapping texts quote the table's radices. - Stages 26 and 42 mint from the immediate parent on frames with SPI copies in FRS id spaces. - frs_spine (end to end), stages 42, 43 and 46 and the replace path refuse incoming ids by count. - The CGT bound holds 10,000 rows and refuses 10,001. - A drifted SPI id_remapping is refused. - The copy-id binding set is derived from imports. - The dense refusal and the K=1 path run through the pool expand, the crosswalk route, both dry-run plans and the graph's uk.full.expand, on ids above 2**53. - The twin's negative-prior count is pinned. - H2 parity passes on the regenerated fixture. The 11 tests in test_uk_identity_stability_receipts still fail until 1b, as at 22bcdbe. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
22bcdbe and its follow-up 2f0f8ce moved every mint site, the executor and the anchor onto the copy-id table's fixed radices. Two decoders outside the stages still used the retired rule. The identity tool read the CGT support copies and the residential arms at ten to the digit count of the largest pre-split id. The dataset-size evaluation multiplied by a dense multiplier of 10**19, which a one-clone expansion of a table-minted spine records, and overflowed int64. This commit moves the identity tool onto the table and both decoders onto exact integers. It also makes the copy path's refusals count-only, adds static guards against the retired rule, and binds the shared keyed-draw helpers into every stage that draws through them. - Identity tool: _support_copy_lineage and _residential_arm_lineage take each copy's ordinal and parent at its field's declared radix (copy_id_table.ordinal and parent). They read the household table only, as exact int64; a non-integer id column is refused. Each receipt's id_multiplier now records the declared radix (10**13 for the support split, 10**18 for the arms). Before any receipt, the tool refuses a spine whose copy ids disagree with their lineage columns under the table, as every spine built before the table does. The refusal is one count-only message saying to run the tool at the commit that built the spine. The refusal and the decode are two small functions, so a retired-rule path could replace them and nothing else. - Size evaluation: dense ids decode back to spine ids exactly, at the multiplier the expansion recorded (the new id_multiplier argument of weight_tables), else the spine's, else the clone-0 ids'. 10**19 is included. A clone offset that int64 cannot hold, a non-integer id column and a multiplier that does not fit the dataset are refused by count. A dense file that keeps its true sources computes its spine stretch again; it used to fail on int(None). - Count-only refusals: - rowwise_geography: _remap_ids' overflow (it printed the multiplier and the largest id), clone_entity_frame's and assign_household_geography's duplicate checks (up to five ids), and the crosswalk route's check for duplicate household ids before cloning; - rowwise_dataset: its uniqueness and person-link checks; - spi_support: its uniqueness and person-link checks, the donor's duplicate and missing-source checks, and the selected-household check; - spi_spine: the stage-14 retained-leaf check; - cgt_structure: the CGT carrier check; - cgt_imputation: the redraw's missing-weight check. - Guards: only the dense line's own modules name id_multiplier_for_values until (c), each with its reason: rowwise_geography, rowwise_dataset, the package re-export, size_evaluation and the row-wise tool. Every module that mints, bounds or decodes copy ids reads the table, the identity tool among them, and names no retired rule. - Bindings: every spine stage whose module or bound helpers import microcosm.build.stochastic_assignment now binds it. A derived test holds the set, and a helper-only edit to it re-keys all 36 such stages and no control. No dense node calls it. - Texts: the stage-46 notes and the residential split's docstring say the identity tool reads the arm rule too. The changelog's binding sentence names only the helpers that mint or weigh copies (spi_support, cgt_support, rowwise_geography). What moves: no data value or id. Stage keys re-key on 31 stages that newly bind stochastic_assignment: frs_spine, frs_take_up, frs_person_draws, frs_household_draws, frs_brma, frs_annual_income_spells, spi_income_band_donors, hmrc_spi_income_spine, spi_housing_shell, was_wealth, was_lisa, property_components, property_purchases, nts_bus_travel, lcfs_consumption, etb_vat, etb_services, uc_reporter_redraw, spi_benefit_coherence, uc_capital_coherence, uc_take_up, uc_legacy_closure_take_up, pension_credit_take_up, child_benefit_take_up, uc_deduction_attributes, cgt_incidence_clone, cgt_incidence_anchor, salary_sacrifice, student_loans, isa_subscriptions and childcare_copy_take_up. Every spine node re-keys anyway through graph_kernels.py, as do the dense nodes that hash rowwise_dataset.py. Regenerated: source_stages.json (the stage-46 notes line), the coverage manifest's 25 source digests and the H2 fixture's notes line in fixture.json; uk_spine.json is unchanged. Tests: the 11 identity-tool tests that failed at 22bcdbe and 2f0f8ce pass again. The hand-built split frame is minted through copy_id_table; the other fixtures come from the real stages. New tests cover: - the old-spine refusal, through main and directly; - the arm decoder above 2**53; - a support copy at ordinal 9,999, which a float read misplaces; - the size evaluation's recorded, spine and clone-0 paths at 10**19, an older dense run at its own multiplier, the int64 refusal and the source-column stretch; - the count-only messages, with no id in them; - the executor's household_source_id branch; - the two static guards and the binding test. Synthetic mutations make each guard fail. The parent's weight_tables fails the four new size tests (OverflowError, TypeError). The 114 touched engine-free test files pass. The H2 fixture regenerates byte-identically, and H2 parity and the engine graph and ladder-clone tests pass in the engine slot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three reviews of 50b7ea1 found no must. This commit takes their should findings and the cheap nits, and corrects four claims 50b7ea1 made. - Bindings: stage 14 (frs_hmrc_spine_leaves) draws its employer pension contributions on keyed uniforms through employer_pension_contributions, but did not bind stochastic_assignment, because the derived test read direct imports only. It now binds it, and the test follows one more step: a stage draws when its module or a bound helper imports the helpers, or imports another first-party module's function that draws through them (a fixed point over the import graph, re-exports and module attributes included). spi_support_channel, which shares spi_spine with stage 14's draw and stage 18's SPI imputation but calls neither, is the one exemption, pinned to those two names. employer_pension_contributions itself stays unbound until the wider binding sweep. - Size evaluation: the spine-stretch refusal printed no count and chained a pandas KeyError listing every missing household id; it now counts the dense rows before the lookup and raises unchained. The misfit refusal says an inferred multiplier reads household ids only and names weight_tables(id_multiplier=), the one route for a recorded multiplier. - Count-only refusals: the row-wise tool's dry-run input checks now call the build's own (rowwise_dataset._require_unique and _require_person_links) instead of listing up to five ids, and the spine sampler's three refusals in national_sampling count families. - Guards: an AST guard refuses ten raised to a len(str(...)) power outside rowwise_geography (the dense rule), copy_id_table (field spans) and national_sampling's legacy candidate decoder, so the retired rule cannot come back written out inline. - Texts, correcting 50b7ea1: - a retired-rule decode would not replace two functions and nothing else; it would also thread the person and benefit-unit tables back through the two lineage functions and their four callers, and pair the twins in the tool at the three pair_clone_households call sites (the tool's comment now says so); - only the identity tool used the retired rule; size evaluation kept the dense line's own rule and overflowed at 10**19, and its non-integer id refusal names the dtype, not a count; - no caller passes a recorded dense multiplier yet: tools/evaluate_uk_dataset_size.py infers it from household ids (the docstrings say so); - the arm tests decoded a synthetic copy chain, not stage output; the stage-built tests below cover that now. The changelog names what each SPI and CGT stage binds and why, limits the count-only sentence to the UK modules (the shared graph executor's EXPAND checks are unchanged), describes the inferred multiplier, and says the identity tool refuses a pre-table spine with copy rows. What moves: no data value or id. Stage 14's key moves with its new binding; every spine node re-keys anyway through graph_kernels.py, and the dense population nodes that hash national_sampling.py re-key, as they already do through rowwise_dataset.py in this push. No generated file changes. Tests: new size tests decode an older dense run at K=15 on odd ids above 2**53 at all three bases (a float64 subtraction fails it), and pin the uncovered-spine refusal (no id in the message, its cause or the traceback) and the negative clone_index refusal. New identity-tool tests hold the old-spine gate to each stored ordinal exactly and to an empty vintage field, refuse a support copy whose parent the split did not copy, and run the gate on frames that stages 16, 17, 26, 42, 43, 45 and 46 built (stage 44's redraw done by hand) and the E8 residential, clone-pair and support-split receipts on the last of them. The dry run, the sampler, the CGT carrier and redraw checks, the SPI uniqueness checks and the support-copy refusals now have exact count-only messages pinned. Synthetic mutations make each new guard and test fail. The 30 touched engine-free test files pass (1,331 tests). The coverage manifest's check passes, the H2 fixture regenerates byte-identically, and H2 parity and the engine graph tests pass in the engine slot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The CGT support split divided each selected household into ceil(w / 60)
copies of its own weight. Once (b) clones households at the spine's start,
each geographic clone carries 1/K of its source's weight, so a clone's count
would shrink with K and the K=1 spine would stop nesting in the K=15 one.
As María ruled on 7 Nov, the count now reads the geographic clone count
times the row's own weight, so every clone of a household gets the count its
source gets at K=1. Nothing moves at K=1.
- Count: cgt_support_copy_counts(weights, maximum_copy_weight, *,
clone_count) returns max(1, ceil(K x w / 60)). For K > 1 the quotient is
snapped to the nearest integer when it is within K ulps of it, so an exact
multiple of 60 keeps its count at any K (at K=7, 7 x (60m / 7) lands above
60m for 626 of m < 20,000 in a synthetic sweep). The snap gives up the
converse case: a quotient within K ulps above an integer counts one copy
fewer than at K=1. At K=1 nothing is snapped, and the count is
ceil(w / 60) bit for bit.
- Stage: split_cgt_support_households takes a required clone_count, and the
transform reads it from the frame (geographic_clone_count): one more than
the largest household_clone_index, or 1 without the column. A non-integer
column, a missing or negative index, and indices that are not exactly 0
to K - 1 are refused by count. Copies keep w / n, so each weighs at most
60 / K. The selection walk still accumulates each row's own weight, and
nothing sums geographic siblings.
- Receipt and gate: the receipt's parameters gain clone_count and nothing
else. stage_health compares K x the heaviest copy and K x the heaviest
selected household with 60, so a receipt at K > 1 no longer fails
falsely. At K=1 both checks and their messages are byte-identical, and a
receipt without a positive integer clone_count is refused. The identity
tool's recompute reads K from the pre-split table and passes it on; its
receipt is unchanged.
- Texts: copy_rule in the code dict, sources.yaml and source_stages.json
now reads 'copies = ceil(geographic clone count x household weight /
maximum_copy_weight); every copy carries weight / copies'. The stage notes
give the cap as 60 divided by the geographic clone count before the
incidence clone. The docstrings, the cap's comment and a changelog
fragment are updated to match. The mass-change reason is unchanged.
What moves: no value at K=1. On synthetic frames the pre-commit and new
code give the same results: the stage's ids, copy counts, weights (bit for
bit), mass records and frame content identity, the identity tool's receipts,
and the gate's verdicts and messages. The only difference is that stage
evidence gains clone_count. Implementation hashes move for
cgt_support_split, and for high_value_home_support, which binds cgt_support.
So node keys move from stage 26 on. The spine gates hash stage_health, and
every stage after the assembled gate waits on its verdict, so in a gated
build every node from stage 14 on re-keys. cgt_support_split's stage
contract moves with its texts. Regenerated:
- source_stages.json (copy_rule and notes, re-derived from the typed
projection);
- the coverage manifest's 25 source digests;
- the H2 fixture: the two declaration lines in fixture.json, and the
stage's contract hash in uk_spine.json.
Tests:
- K=1 counts equal ceil(w / 60) bitwise on random and edge weights: exact
multiples of 60, one ulp either side, tiny and large.
- K=3, 7 and 15 replicas count as their sources, including the K=7 exact
multiples the unsnapped rule overcounts. The snap's documented trade is
pinned.
- The clone-count refusals ({0, 2}, negative, missing, float, bool) give
counts.
- On K=3 and K=7 clone-first frames, the transform gives every clone its
source's copies, with weights nested within rtol 1e-12 and every copy at
most 60 / K.
- The packaged gate passes a K=3 stage receipt with w <= 60 < 3w, and fails
it when the count is ignored. Hand-built receipts pin the K=1 messages and
both scaled checks.
- The identity tool's K=1 receipt equals the one ceil(w / 60) gives, with no
new key. At K=3 it receipts green, and it reports a recompute that ignores
K.
Eight source mutations each fail a new test. The 51 touched engine-free
files pass (2,317 tests). The coverage manifest's check passes, the H2
fixture regenerates and passes its check, and H2 parity passes in the
engine slot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three reviews of 914cae4 found no must. This commit takes their should findings and the cheap nits, and corrects three claims 914cae4 made. - Identity tool at K > 1: the E8 recompute counted each family exactly, on a weight rebuilt by re-summing the family's copies. At K > 1 that sum drifts by more ulps than the count's K-ulp snap absorbs. On the commit's own nesting scenario, the tool flagged the stage's correct count of the 7,620 family in every clone at K = 11 and K = 13. At K > 1 the stored count must now lie between the counts at the folded weight times 1 - 1e-9 and 1 + 1e-9, the tolerance the tool already gives family weights. At K = 1 the check stays exact, so K = 1 receipts and problems are unchanged. The tolerance cannot tell an exact multiple's n from n + 1. The count's unit tests pin the snap, and the flags and family weights still report a moved count. copies_recomputed stays the exact recount. - Gate tolerance: test_split_gives_every_geographic_clone_its_sources_copies now runs the packaged stage gate on the transform's receipt. At K = 7 it also asserts that 7 x the heaviest copy lands above 60, so the copy check's 1e-12 tolerance stays exercised. - Texts, correcting 914cae4: - Its changelog fragment and message said every clone of a household gets its source's count. That holds for a selected clone at exactly 1/K of its source's weight. The walk still accumulates each row's own weight, so the household at which a band's walk reaches its support mass, and households tied on wealth, can be selected in some clones and not others, lowest household id first. The fragment and the split's docstrings now say so, and a new test pins the boundary case. - Its Tests line omitted that the clone-first split scenario exhausts every pool, so its selection is the same at every K. - The spine gates re-key because they hash the country spec as well as stage_health; the fragment now names both. The fragment also says the snap adds a copy at K = 7 only for some exact multiples. The module docstring, the clone-split factor's comment and the split's refusal sentence name the incidence clone stage (cgt_incidence_clone). geographic_clone_count's docstring says a refusal gives a count or the column's dtype. Not changed here: the snap rule itself and a mention of it in copy_rule, which wait for a ruling on the snap; the stage notes' "the clone stage", which changes when the stage's declarations are next regenerated; and the tool's cross-check of K against the stage receipt, which lands in (b). What moves: no value, at any K. cgt_support.py changes only in docstrings and a comment, so its implementation hash moves again, and the stages that bind it re-key as in 914cae4. sources.yaml, source_stages.json, the coverage manifest and the H2 fixture are untouched; the fixture holds stage contract hashes, not code hashes. The tool's verdicts move only at K > 1, which no spine runs yet. Tests: the identity tool's nesting receipts are green at K = 7, 11, 13 and 15; a count moved off a non-multiple is reported at K = 1, 3 and 11; at K = 1 a moved count on an exact multiple is still reported by the count check. Six synthetic mutations each fail a new test: the gate without its tolerance, the tool exact at every K, the tolerance at K = 1, the tolerance ignoring K, a tolerance of 1e-1 and a strict walk. The touched and related engine-free files pass (243 and 275 tests). The coverage manifest's check and the source_stages projection check pass, and ruff is clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
) The re-reviews of da5a350 and 9721448 found no must and no code should. This commit takes their four nits. - Bindings: the stochastic_assignment exemption of spi_support_channel now names the stage's transform, and the binding test checks that the transform names no drawing function, directly or through its module, as well as the drawing names the module imports. Before, a support-channel transform calling stage 14's employer pension draw, which spi_spine already imports, would have passed both binding tests. - The row-wise tool's --source-lineage-modulus help still described the spine's retired copy-id scheme. It now says spine inputs carry copy ids in the copy-id table's fixed fields. - Test comments on the CGT nesting scenario: the 9,000 household is the first candidate for the exact-total correction, not where it always lands. At 7 and 13 clones the shared fallback puts it on the last copy of the 7,620 family's last clone. At 13 clones an exact recount moves 12 of the 13 clones, not every one. - The identity tool's E8 docstring says copies_recomputed stays the exact recount, so at K > 1 it can exceed copies by one for each exact-multiple family while the count check passes. What moves: nothing. No runtime module changes, so no node key moves. Ruff is clean, the stochastic_assignment binding tests pass, and the re-review's mutant (the exempt transform naming the employer pension draw) now fails the binding test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
13 tasks
The land changelog said the two held rows "are measured on every evaluation", and 161c777's test was named for that. That phrase is the exclusions register's convention, and the register's entries keep it, but no build step reports a held row's estimate. As María ruled on 8 Nov, the changelog and the test now say what happens: the rows stay on the run's compiled register, and a tool that scores the un-excluded register, such as tools/evaluate_uk_incumbent_surface.py, measures them, as it does every held row. - The changelog fragment's sentence is reworded. - test_the_two_land_diagnostics_are_measured_but_not_fitted is renamed test_the_two_land_diagnostics_leave_the_solve_but_stay_on_the_compiled_register, and its docstring and the scoring comment say the same. What moves: nothing. Ruff is clean and the renamed test passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
As María ruled on 8 Nov, national_sampling's compact sampler, sample_uk_national_frame, is retired with its family decoder, uk_source_family_units. No build or tool has called the sampler since 8adc836 (#762 PR A) moved the row-wise candidate onto the spine sampler. The decoder re-derived retired id arithmetic (a clone multiplier of ten to the digit count of the largest canonical id, and max + 1 SPI and capital-gains offsets), which kept the module on the copy-id tests' inline-rule allowlist. - national_sampling.py loses the two functions, their __all__ entries and the helpers only they used: the post-sample SPI replacement quota check, the required-column list and the clone-index column alias. The module docstring now describes the spine sampler. The spine sampler, the rung tokens, the default seed and every name other modules read from the module are unchanged. - test_uk_national_sampling.py drops the ten tests of the retired functions and their compact-frame builder. The numeric-region refusal, which only those tests covered, is now tested on the spine sampler. - test_uk_copy_id_table.py: national_sampling leaves _INLINE_DIGIT_COUNT_MODULES, so the inline-rule guard now admits only rowwise_geography and copy_id_table. - experiments/901-uk-main-rebase-receipts.md cited one of the removed tests in its row 4; the row gains a dated note. - A removed changelog fragment. What moves: no value. _PopulationKernel hashes national_sampling, so uk.full.sample, uk.full.normalize, uk.full.expand, uk.full.locations, uk.full.geography_mapping, uk.full.pool and uk.full.geography_gate re-key, and with them every node downstream of uk.full.sample. No other UK kernel binds the module, so no spine node re-keys. No generated file moves: the source_stages projection and coverage manifest checks pass, and the H2 fixture holds no code hashes. Tests: test_uk_national_sampling, test_uk_copy_id_table and test_uk_runtime_exports pass (57 tests). An AST scan finds no tracked Python file that names the two functions, and the inline-rule detector flags the module at the parent commit but not here. A permissive region check fails the new spine test. Ruff is clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ll it (#931) A spine stage's key hashes its stage module and the helper modules listed for it, and source_hash does not follow imports, so a stage that called a function in an unlisted module replayed a stale output when that module changed. As María ruled on 8 Nov, this sweep binds those helpers, with a test that derives them. It only changes cache keys. - test_uk_graph gains the derived rule. From the transform graph_kernels builds for each stage, it follows every definition the code names (annotations included, through re-exports and module attributes) into the module defining it, within microcosm.build, to a fixed point, and requires the stage's hash to cover each module it meets. A check that returns and writes nothing, a function returning one column name, a record class of undefaulted fields, a constant of strings and the stage declaration need no binding; a numeric constant does. The other workspace packages stay outside the rule, and the docstring says why. A second test applies the rule to the seven dense kernels that share _PopulationKernel's hash, with the spine gate admission check listed as refusal-only, and a control test pins each form the rule follows and each kind it skips. - graph_kernels: national_frame joins the modules every stage hashes, and the helper table gains 219 bindings of 49 modules in 48 stages, with two shared groups (the vendored-fact reader with the chronicle feed and its schema epochs; frs_spine's pinned tab reader with trace). The six modules the ruling named are bound where stages 26, 42, 43 and 46 compute through them: property_localisation (26), spi_support (26, 42, 43, 46), cgt_imputation (42, 46), cgt_asset_type (46), advani_summers (43) and cgt_structure (42; stages 26 and 46 reach only its closed-world operations check). Stage 14 binds employer_pension_contributions. - The older import-level tests then ask for more: stochastic_assignment in 8 more stages (the support channel's exemption goes), the copy-id table in stages 14 and 18, 15 stages on the copy-id name-only list and four more counting stages. Their re-key tests get new controls and counts. - graph_population: _PopulationKernel hashes rowwise_geography, and the dense test also requires frame_sampling, national_frame, local_authority_input and gates. - Changelog: a new helper-bindings entry; the copy-id entry drops its sentences about the helpers that stayed unbound. What moves: node keys only. Every spine kernel hashes graph_kernels.py, so every spine node re-keys, and with it every node downstream. Every graph_population kernel re-keys (uk.full.sample, normalize, expand, locations, geography_mapping, identity, property.value, property.band, geography.local_authority, geography.brma, geography.rfdr, private_rent, pool and geography_gate), and so do the calibration and size kernels that hash the module (uk.full.dense, selected, calibrated, size_search, size_draw, size_refit and size_checkpoint_import). No value, declaration or generated file moves: uk_spine.json and the H2 fixture hold stage contract hashes, not code hashes, and the fixture check regenerates byte for byte; the source_stages projection and the coverage manifest are current. Tests: test_uk_graph (33 tests) and 15 related engine-free files pass, and ruff is clean. Mutations under scratch: a value helper newly imported into a stage (by name, by module attribute or through a re-export), a numeric constant newly read and a dropped binding (a helper, a group member, national_frame) each fail the stage test, while a helper the transform never names, a check and a constant of strings pass. Dropping rowwise_geography or frame_sampling from the dense hash, or a dense kernel newly naming an unhashed helper, fails the dense test. The H2 fixture check and the H2 parity test pass through the engine slot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…erance (#931) This was to be the commit that keeps stage 42's exact-total correction off the support copies, as María ruled on 8 Nov: try the other unselected households, then the selected households left whole, never a root or copy. On synthetic frames that rule cannot always make the total exact. In the CGT nesting scenario at K = 7 and 13, every household the split leaves whole sits in the first half of NumPy's pairwise sum. That half lies in the total's binade and the second half carries exactly half an ulp, so round-half-to-even skips the target: no change to the whole households can reach it, and only a copy in the second half can. The rule as written refuses those frames, and 1 to 3 of 24 random 150-household frames at K = 1, 3, 7, 11 and 13. So the correction is unchanged here and goes back to María. This commit lands the two rulings that do not depend on it. - copy_rule names the K-ulp snap ruled on 7 Nov, identically in the code dictionary, sources.yaml and source_stages.json: "copies = ceil(geographic clone count x household weight / maximum_copy_weight), the quotient snapped to the nearest integer within clone-count ulps when the clone count exceeds one; every copy carries weight / copies". The stage notes say "the incidence clone stage (cgt_incidence_clone)" where they meant it. - The identity tool's _e8_support_split holds the stored count between the counts at the folded weight times 1 - 1e-9 and 1 + 1e-9 at K = 1 as well. At K = 1 the fold can land ulps from w: for a weight a few ulps above a multiple of 60 (about half of them in a synthetic sweep), or where the correction fell on a copy. On an exact multiple a moved count is then left to the flag and family-weight checks. The docstring, the receipt's tolerance policy and the test class docstring say so. - The split's docstring and _exact_support_weights' no longer claim the correction always keeps every copy at w / n. - test_at_one_clone_the_count_check_stays_exact becomes test_a_count_moved_on_an_exact_multiple_is_left_to_the_flags, at K = 1 and 3. A new test folds the 900 family just above 900 at K = 1: green, with the exact recount one copy higher. - The changelog fragment describes the snap clause, the K = 1 tolerance and the unchanged correction. What moves: no stage value at any K. The identity tool at K = 1: of 25 synthetic frames as built, the 2 it flagged are green now, both false positives; every tampered frame still fails, but a count moved on an exact multiple is reported by the flags and family weights, not the count check. Its K > 1 verdicts are unchanged. Re-keys: the implementation hashes of cgt_support_split and of high_value_home_support, which binds cgt_support; cgt_support_split's stage contract digest, with its texts; and every node keyed on the UK solve fingerprint, which covers sources.yaml, source_stages.json and the coverage manifest, the spine gates among them, so in a gated build every spine node from stage 14 on re-keys. Regenerated: source_stages.json (copy_rule and notes), the coverage manifest (25 source digests) and the H2 fixture (the two declaration lines in fixture.json and the stage's contract hash in uk_spine.json). Tests: 63 touched engine-free files pass (2,745 tests). The new K = 1 test fails against the parent's tool. The source_stages projection, the coverage manifest and the H2 fixture checks pass, and H2 parity passes in the engine slot. ci_test_plan verify and ruff are clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rating left it (#931) The 1b decoders review found two identity-tool gaps on the synthetic H2 oracle frame. As María ruled on 8 Nov, they land as a tool-only commit before the arms. Both reproduce with origin/main's code and with the parent's. - E5's stored-column check re-applies the regional uprating (stage 25) to the stored values as a fixed point. regional_home_values (stage 27) then rewrites main_residence_value and property_wealth, so E5 was comparing stage 27's values. On the oracle, every mismatching row is one that stage 27 rewrote: 284 of 394 scoped rows on main, 294 of 404 at the parent. Stage 27 keeps the value it replaced in main_residence_value_rank_basis and moves the wealth by the value's change. The scoping now reverses that rewrite for every stage before 27. With the reversal, the mismatches drop to zero on both frames, so the stages are right and the tool was reading the wrong stage's output. - _STACKING_STAGE_BY_FLAG had no entry for household_is_high_value_support_copy (stage 26). The was_wealth scope kept the copies (105 of 394 rows on main, 115 of 404 at the parent), and E4's FRS-only frame kept any copy of an FRS household. The flag joins the map. Dropping the layer folds the copies onto their roots by family, after the CGT folds: each copy adds its w/n back, so the root returns to its pre-split weight, to rounding. A new decoder reads the copies at the copy-id table's high_value_support field. It refuses, by count, a copy without a root, a flag or index that disagrees with the id, and a family whose size disagrees with high_value_support_copies. Dropping the copies while a CGT layer stays in scope is refused. On the oracle every high-value root is an SPI row, so the copies never reach the uprating's FRS-base mean there. The new tests use FRS roots, where they do. What moves: tool verdicts only. On the oracle, through main() and the H5 loader, E5 fails with the parent's tool and passes now. E4 reads the same frame as before. The E6, E7 and E9 receipts are identical, and E8 stops at the same pre-existing error on the fixture with both tools. On a spine whose high-value roots include FRS households, E4's scope and weights change too. Nothing hashes the tool, so no stage, value, node key or generated file moves. The source_stages projection and coverage manifest checks pass unchanged. Tests: 12 new tests on frames the real stages build (stages 25, 26 and 27, then the CGT support split and the clone): - the was_wealth and FRS-only scopes drop every copy and give each root its pre-split weight; - E5 is green on that chain, red with either half of the old scope, and red when an uprated value is moved off its fixed point before stage 26; - the tool's entry point passes on that chain and fails on the moved value; - the reversal restores stage 26's values; - the lineage refusals print counts only. 9 of the 12 fail against the parent's tool. The 3 that pass are the mutation cases, which are red with both tools. The 9 touched engine-free files pass (218 tests). Ruff is clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ts (#931) Three reviews of 2afda31, fe15413, e279b69 and bd292eb. Their one must, the stage-42 correction rule, needs a ruling and is left out with the other ruling items listed below. Bindings (fe15413): - The rule took any constant made only of strings as unable to change a value. A tuple, set or mapping of strings selects rows, columns or codes, so it is now a value, like a number. Only a single string, a function returning one column name and a record whose field types are of these kinds still stop the rule. Newly bound: tenure_constants in was_wealth, was_lisa, high_value_home_support and regional_home_values; council_tax_constants in frs_spine; rowwise_geography in lcfs_consumption; etb_vat in etb_services, which joins the counting stages through it; cgt_imputation in cgt_incidence_clone; frs_annual_income and hmrc_source_contract in hmrc_spi_income_spine; household_income in was_wealth and was_lisa. The dense population hash adds spi_support. - Where the rule stops at a name, a new test requires the name's module to be hashed by the stage or by a node upstream of it, since node keys chain on their predecessors', or the name to be listed with a reason. One stage name is listed (the geographic clone index columns, which no spine node writes, checked) and two dense ones (the LAD23 lookup's pin). - The derived check kind goes: a check is skipped only when listed with its reason, now also cgt_structure's closed-world check and spi_support's entity check. _writes_nothing, which tests each entry, also counts an augmented assignment, out=, copyto, ufunc.at, shuffle, reverse and the like as writes, and lets only a one-parameter function return an argument. A record's field types must be harmless too. - The spine root's rung sampler moves from spine_build, which no node hashes, to national_sampling (UKSampledSpineRootTransform and sample_uk_spine_root, behaviour unchanged). It declares national_sampling and frame_sampling through graph_implementation_dependencies, so the production root node hashes them. A test holds every transform spine_build installs to its stage's module, declared dependencies or a listed adapter. - The scope comment and changelog said microcosm.fit's fitters are bound by hand; three are, in two stages. They now name what stays unbound outside microcosm.build and the dense kernels the rule is not yet applied to, and the changelog names stage 18's employer pension binding. Fallback (e279b69), texts only: the tool's docstring, comment and tolerance policy, the receipts test class and the changelog say that within 1e-9 of a multiple of 60 the count check accepts n or n + 1, so a family split consistently into one copy more goes unreported at every K; a new test pins it. The changelog's what-moves names the tolerance policy and the country- and solve-fingerprint kernels that move, and puts the correction row about an ulp of the frame's total off w / n. Stage 26's docstring names the shared fallback as stage 42's does. Tool (bd292eb): the E5 reversal refuses, by count, a roster where a stage after regional_home_values declares main_residence_value or property_wealth. A test holds the stacking map to the structural stages' declared flags, with two listed exceptions. Sampler (2afda31): the seed comment drops the retired draws, and a test pins uk.full.sample's f100 branch, which experiments/901's row 4 now cites. Left for María: the stage-42 correction rule (on the synthetic nesting scenario no whole household can carry it at K = 7 and 13), an exact K = 1 count check, binding the unbound modules outside microcosm.build, binding cgt_structure in stages 26 and 46 instead of listing its check, and copy_rule's wording, which waits for the correction ruling. For the PR body: 2afda31 adds a changelog fragment of type removed; fe15413 binds employer_pension_contributions in stages 14 and 18; e279b69 also re-keys the spine, full, preflight and holdout gates and the target compilation, target selection, problem, measures, national targets, national problem and national gate kernels. What moves: node keys only. graph_kernels.py changes, so every spine node re-keys; the 12 new bindings would re-key their 9 stages by themselves. national_sampling.py and graph_population.py change, so every graph_population kernel and the calibration and size kernels that hash it re-key. The production root node hashes the rung sampler at every rung. No value, declaration or generated file moves: the source_stages projection, coverage manifest and H2 fixture checks pass, and H2 parity passes. The identity tool's receipts change only in the tolerance policy text. Tests: 34 touched and related engine-free files pass (1,115 tests), and ruff is clean on the 11 edited Python files. In-memory mutations under scratch turn the new tests red: a dropped tenure_constants or cgt_imputation binding; a stage newly reading a collection of strings, calling an np.add.at helper in an unbound module or reading an unhashed single string; an emptied or padded names table; spi_support or rowwise_geography dropped from the dense hash; the rung sampler declaring nothing or only its own module; an unlisted source adapter; the stage-26 flag dropped from the stacking map; uk.full.sample sampling at f100; and the tool before the K = 1 tolerance on the n + 1 pin. The source_stages projection and coverage manifest checks pass, the H2 fixture regenerates byte for byte and H2 parity passes in the engine slot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stage 16 copied exactly round(10,000 x the sample fraction) households,
picked by row position with rng.choice, so whether a household was copied
depended on every other row. Once (b) clones households at the spine's
start, the K=1 spine's copies would not be clone 0's copies at K=15. As
María ruled on 7 Nov, the stage now copies a keyed Bernoulli share whose
probability it computes itself.
- Share: the transform passes expected = count x sample fraction, so the
manifest's 10,000 is the share's numerator. create_uk_spi_support_tables
takes expected_spi_households in place of spi_household_count (None
still copies every household). It counts N, the distinct
source_household_id values entering the stage, and copies a household
when pi = min(1, expected / N) is one or the household's uniform falls
below pi. The uniform is keyed on the household's own household_id,
under the stage's seed (42) and the new salt
spi_support_channel:inclusion. The decision is spi_share_inclusion,
which takes pi explicitly; at pi = 1 it reads no uniform, since an
integer over 2**64 can round to one. Copies keep frame order. The
positional draw and the count-versus-rows refusal are gone. The
tests-only replace path keeps its own draw.
- Gate: e7_spi_synthetic_preclone is renamed e7_spi_synthetic_share. It
declares maximum_zero_weight_rows_rule {rule: binomial_share_band,
sigma: 4} in place of the integer. After the draw, the stage resolves the
rule to min(n, ceil(n x (pi + band(pi, n)))) over the n household rows
entering it, then runs the unchanged zero-weight gate.
stage_health.latent_attribute_tolerance moved into stochastic_assignment
as binomial_share_tolerance(*, target, rows, sigma). The stage_health
name stays as an alias at four sigma, with unchanged values.
_support_stage_parameters checks the draw, salt, gate name, selector,
rule and reason, and returns the declaration with its rule. Its
hmrc_source_contract caller needs no change.
- Per-clone strata (ruling 2): where household_clone_index is present,
_allocate_spi_prior_mass adds a derived clone stratum, which is never
written to the frame. The manifest declares it as
geographic_clone_stratum, and spi_spine checks it. The empty-stratum
refusal now counts strata and names none. Without the column, as on
every national spine today, the allocation is unchanged bit for bit.
- Manifest and schema: the stage has a new source sentence, draw text,
salt, gate reason and notes. In sources.schema.json,
stack_zero_weight_donors requires salt, the allocation accepts
geographic_clone_stratum, and a zero-weight declaration takes the
integer maximum or the rule (oneOf).
- Receipt: new keys expected_spi_households, inclusion_probability
(receipt_counts.floored_share), source_households (rounded_count), and
the gate's name, rule and sigma, not its resolved maximum.
spi_households stays exact, because the stage gate reads it.
- Bindings: the receipt's counting sits in spi_spine. That adds one
counting stage (frs_hmrc_spine_leaves), against three for spi_support
(cgt_support_split, cgt_incidence_clone, cgt_residential_split), which
ten stages bind. Node re-keys are the same either way: the root already
binds receipt_counts. high_value_home_support and cgt_residential_split
now bind stochastic_assignment, as they bind spi_support.
What moves: at K=1 the SPI copy set is redrawn. Its count is Binomial(N,
pi) around 10,000 x f instead of exact: about 1,000 at f010, by design
arithmetic. Per-stratum SPI weights move with it, and so do every
SPI-channel value from stage 18 on, the high-value and CGT selections that
rank SPI rows, and calibration. The claim that stage 17's seating does not
move holds on synthetic frames. On 8 frames run through the parent's stage
16 and this one, stage 17 seated the same households, carriers and bands,
with equal seating scales, mass scales and funding factors. Stage 16's
base weights were bitwise equal except for one row in one frame (relative
7e-14, the exact-total correction). The H2 fixture's 135 households at
f = 135/10,000 give pi = 1 (10,000 x 0.0135 is 135.0 in float), so its
copies are unchanged. Every node re-keys, since graph_kernels,
stochastic_assignment and spi_support changed and the root binds
stochastic_assignment; so does every kernel keyed on the spec or the
coverage manifest. Regenerated: source_stages.json, the coverage
manifest's 25 source digests, the SPI declarations in the H2 fixture
(fixture.json), and the stage's contract hash (uk_spine.json).
Tests:
- With pi held fixed, permuting rows or appending other households leaves
every household's draw unchanged. A row permutation leaves the stage's
selection unchanged.
- Over 200 seeds the count stays inside the 4-sigma band.
- pi = 1 copies every household, even with uniforms patched to 1.0.
- The resolved gate passes at its maximum and refuses one more row. The
stage gate refuses an injected copy above the bound, and an unmatched
zero-weight base row still fails.
- Malformed rules, and drift in the draw, salt, rule, sigma, name or clone
stratum, are refused.
- On clone-first frames at K=3 and 7, N is the same as at K=1, and clone
0's copies are exactly K=1's. The other clones draw their own, and K x
clone-0 weights equal K=1's to rtol 1e-12.
- Each clone keeps its own region masses. An empty clone stratum is
refused by count, and so are a non-integer, missing or negative clone
index. A one-clone index allocates bit for bit as no index.
- The band-donor fixture moves to expected_spi_households=10, and a
comment records that seed 42 keeps copies in both regions at 60 and 200
households. Its weight test now takes the median over the FRS base the
seats are drawn from: over every row, the redrawn copies moved the
median and the statistic fell just under its 0.53 bar, while the
seating itself is unchanged.
- Ten in-memory mutations each fail a new test: keying on the source id,
a row-count denominator, no pi = 1 short-circuit, no clone stratum, the
positional draw, a gate without its band, a three-sigma alias, an exact
share or source count in the receipt, and another salt.
The touched engine-free files pass, with ruff. The coverage manifest is
current. The H2 fixture regenerates and passes its check, and H2 parity
passes in the engine slot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stage 17 seated its donors in one walk over every candidate, and sized the seats on the eligible households counted in rows. Once (b) clones households at the spine's start, the walk would pool the clones, and the seats and the 15% household-share cap would grow with K, so clone 0 of a K=15 spine would not seat the K=1 donors. As María ruled on 7 Nov, the stage now runs today's walk inside each geographic clone, with one common mass scale. - Seating: where household_clone_index is present, the candidates are grouped by their household's clone; without it there is one group. The eligible households are counted in distinct source_household_id values, so the seats and the cap (spi_income_band_donor_seats) are the same at every K. Today's walk runs once per clone with those seats and the same person-keyed uniforms, Efraimidis-Spirakis keys and lexsort, so clone 0, whose ids are the one-clone ids, seats exactly the one-clone donors. The per-band eligibility check runs per clone and refuses by count. selected_household and carrier_person stay shared, since ids differ across clones. A non-integer, missing or negative clone index, or a gap in 0 to K - 1, is refused by count. - Weights and funding: each donor weighs (T_b / count_b) / K. Its funding stratum is its region within its clone: _funding_strata appends the clone where the column is present, and the manifest's funding_strata stays ['region']. So every clone's regions conserve their own mass. One mass scale, the minimum over every clone-region stratum, serves every clone (ruling 2), so each band keeps one donor weight, and stage_health's band-mass reconciliation and the identity tool's E8 replay hold unchanged. The exact-total correction still moves one incumbent row and never a donor, so a band's donors stay equal bit for bit at any K. With exact replicas it lands on clone 0, whose weights then nest to rtol 1e-12, not bitwise. The MassChangeRecord reason is unchanged, so uk_frame_content_identity and the coverage manifest's required_mass_change_reason do not move. - Receipt: new key clone_count, and funding_strata lists household_clone_index where the column is present (the stratum labels read region|clone). expected_donor_households, expected_donor_count and the donor counts cover every clone; donor_weight and planned_donor_weight are one clone's. At the receipts' floor of three, as the design review asked, both eligible counts go through rounded_count over distinct true sources in place of the exact ones; seating_scale reads 'full' or 'capped', since an exact scale below one gives back the eligible count; a band's carrier_mean_age reads '<3' below three carrier source households; and carrier_region_counts and the funding rows' donor_households are rounded. Besides the tests, stage_health reads seating_scale and the identity tool copies it; nothing else reads the changed fields. - Gate: the band-donor check refuses a receipt without a positive integer clone_count. It requires donor weight x K within the maximum, donors that split evenly across the clones, and at full seating donors equal to the receipt's plan (K x one clone's) and at least K x 120. The band-mass check is unchanged. At K = 1 the checks and messages are as they were, except that the seating scale is read as 'full' or 'capped' rather than a number in (0, 1]. - Texts: the draw, weight_rule, funding, seating_scale and mass_scale constants and the stage dictionary, sources.yaml (and the stage's notes) and source_stages.json. The design's draw wording named "geographic_expand", which nothing in the code defines, so the draw names the node that writes the clone index, uk.full.expand. - Bindings: the derived tests in test_uk_graph.py pass with no change. The stage already counts by lineage (it is in _COUNTING_STAGES) and binds spi_support and stochastic_assignment, and the clone column's name is a string constant. What moves: no value at K = 1. On 48 synthetic stage-16-then-17 runs (8 frames, 3 band configurations, 2 seeds; two of the frames carry a one-clone index), the seating, carriers, every household weight (hex), donor weights, mass and seating scales, funding factors, mass log and content identity after the stage equal the parent commit's. The receipt gains clone_count and takes the floor-3 forms above. spi_income_band_donors and every node after it re-key, the spine gates included, as does any kernel keyed on the spec or the coverage manifest. Regenerated: source_stages.json, the coverage manifest's 25 source digests, the stage's declaration in the H2 fixture (fixture.json) and its contract hash (uk_spine.json). Tests: - K = 1: the existing seating, carrier, weight and receipt expectations hold, with seating_scale now 'full' or 'capped'. A pinned test checks uk_frame_content_identity after stage 17 on three synthetic frames (full seating, capped seating, scaled mass) against the parent commit's values. A one-clone index seats, weighs and funds bit for bit as no index. - K = 3 and 7, with clone-first frames run through stage 16: clone 0 seats the K = 1 households, bands and carriers; every clone seats the same seats, and the other clones seat their own. K x clone-0 weights, donors included, equal K = 1's at rtol 1e-12, and so do clone 0's funding factors. - A capped seating scale is the same at K = 1 and K = 3. Each (clone, region) conserves its mass. Where one clone's region binds, its factor is 0.5, the one scale serves every clone and band, and the gate passes the scaled receipt. - Refusals: a clone without candidates, and a float, missing, negative or gapped clone index, each by count. A row permutation at K = 3 leaves the seating unchanged. - Receipt: the eligible counts are rounded, checked against an independent recount; the funding and region counts are rounded; the key sets are pinned. The mean age reads '<3' at K = 1 and where three clones' carriers share one source household. - Gate: the K = 3 receipt passes, a dropped clone_count raises, and a misread one fails. Synthetic K = 3 evidence passes and fails on weight x K, uneven splits and the scaled minimum, with the K = 1 messages pinned. - The identity tool's E8 replays a K = 3 seating with no problems and flags a reweighted clone-2 donor. - 17 mutations on scratch copies of the source each fail a test. The touched engine-free files pass, with ruff. source_stages.json and the coverage manifest are current. The H2 fixture regenerates and passes its check, and H2 parity passes in the engine slot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three reviews of 50b6e62 (stage 16's keyed share) and cd43acf (stage 17's per-clone seating) found no must. This takes their should findings and the cheap nits, all inert at K = 1. The ruling items, and the text changes the reviews placed at the next regeneration, are left out and listed below. Band donors (cd43acf): - Each band's rounded eligible count is clone 0's, in distinct source households. Pooled over the clones it was a union, which grew with K in every band below the top. Clone 0 walks what one clone walks, so the count is now the same at every K. The per-clone refusal is unchanged. - A band's carrier_mean_age is also withheld where the band seats fewer than three households per clone. At K = 1 a band's carriers rest on at most its seats in source households, so no K = 1 receipt changes. - The result docstring and the receipt comment name the fields that are rounded, and say that the band rows' donor and carrier counts and donor_count stay exact row counts, because the gate reads them. The module docstring says the seats are sized on eligible households counted in source households, and when clone 0's weights and funding factors nest. - Gate: a seating_scale in neither form is named by its type, or as missing, in the failure, the full-scale message and the details, so a replayed old receipt's exact scale does not reach the verdict. Share (50b6e62): the spine gate kernel hashes stochastic_assignment beside stage_health, since the latent-attribute checks' band moved there. Changelog fragments: stage 16's says what stays bit for bit without the clone column (the strata, given the same copies), that stage 17's funding factors and mass scale move by rounding under the redraw, and that the empty-stratum refusal reads the new draw. Stage 17's drops the clone-0 premise for the exact-total correction and states the nesting condition, the per-band counts, the mean-age rule and the gate message. Corrections to the earlier messages, which stay as they are: - 50b6e62: under the redraw, stage 17 keeps its seating, carriers, bands and seating scale, but its mass scale and funding factors are not equal. The funding strata's masses sum the redrawn copies' weights, so the factors, and the mass scale where it binds, move in their last bits, and the incumbent and donor weights with them (the reviews' synthetic probes: within about 1e-11 relative). The empty-stratum refusal reads the new draw, so a frame that built before can refuse, and one that refused can build; the reviews found both on synthetic frames. - cd43acf: the seating nests exactly at any K, but clone 0's weights and funding factors nest only while the common mass scale is the one K = 1 takes (one at both, or bound by a clone-0 region). The synthetic tests hold it at one. After stage 16 the clones are not exact replicas, and in the reviews' synthetic probes the exact-total correction often landed outside clone 0, at stage 16 too. - Both: rtol 1e-12 is the small synthetic frames' figure. A corrected row misses by a few ulps of the total, so its relative miss grows about as rows x K x 2.2e-16 (design arithmetic). What moves: no value at K = 1. On six synthetic frame and plan configurations at K = 1, the stage-17 evidence and the frame content identity equal cd43acf's; at K = 3 and 4 only carrier_mean_age differs, withheld in more bands. Every UK stage kernel hashes graph_evidence, so every spine node re-keys (54 of the spine graph's 79 kernel hashes change, the root's among them; 50b6e62 had re-keyed every node already), as do the dense population kernels and the spine gates. No declaration changes: source_stages.json and the coverage manifest are current, and the H2 fixture is untouched. Tests: - K = 3: the carriers per region are recounted in source households (SOUTH_EAST reads '<3' over three or more rows). With the rounding patched out, each band's eligible count equals clone 0's recount and is below the pooled one; the rounded counts equal K = 1's. - K = 3 and 4: the one-seat bands withhold the mean age, though their carriers come from three or more source households. - The K = 1 per-band refusal message is pinned. - The share gate resolves a sigma-2 rule below sigma 4, and refuses a NaN or infinite sigma. A sample fraction whose 10,000 x f is not whole keeps the fraction in the expected count and the share. - The gate names a float or a missing seating scale by type. The gate kernel's hash includes stochastic_assignment. - Ten mutations on scratch copies of the source each fail the intended test: the six the reviews found missed, a pooled per-band count, and one each for the mean-age rule, the gate message and the gate hash. Left out: the draw text's "(one before uk.full.expand)", which waits for María; clone_count in the stage-16 receipt, with (b); and the declaration texts the reviews put at the next stage-16 and stage-17 text regeneration, so that H2 is not regenerated for text alone (the weight rule's band donor count, the stage-17 notes' seats counted in source households, the stage-16 notes' draw on the id alone). Disclosure-control items from the reviews went to María separately. The touched engine-free files pass, with ruff. source_stages.json and the coverage manifest are current. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s, at any clone count (#931) María's 9 Nov evening rulings on the identity tool's count hold: keep its two narrowings, hold a family on the members whose clone halves are still equal, extend the hold to K > 1, and hold the family test to the same tolerance as the count. Tool only; no stage code, value, declaration or node key moves. - Equal halves (ruling 3). Where at least four members of a family of more than three show their two clone halves at equal weights and share one pair sum c, the stored count n is held exactly on c, whatever the other members carry (trimmed or liable members, a corrected one). At most three members carry a corrected weight, so one of the four is uncorrected, and equal halves leave its pair sum at exactly w / n. Before, a family was held only where every member carried c. Both accepted narrowings stay: families of three or fewer keep the bracket, and where every member shares c but fewer than four show equal halves, the hold reaches one float either side of c. - K > 1 (ruling 4). The stage counts n on each clone's own weight and gives every member fl(w / n), and the snapped count still rises with w, so the hold applies at any K, with the K-aware count at the ends of the run of weights. The one-float tier reaches two floats where c sits within two floats of a power of two. c lies in (45 / K, 60 / K], which holds 16 at K = 3, 8 at K = 6 and 7, and 4 at K = 12 to 15, and below a power of two the floats are half as wide. The hold's docstring gives the bound: a trimmed pair's sum lies within three quarters of the float spacing at c before its last rounding, a liable pair's within half. - The family test (ruling 3). Whether a household is a family is no longer the exact recount of the re-summed fold. A household the rule selects is possibly a family where its K-aware count at the folded weight times 1 + 1e-9 exceeds one, and possibly whole where its count times 1 - 1e-9 is one. Its stored state is accepted in between, and the hold's counts decide where the hold applies. The keys and their missing and extra counts are unchanged. The hold is now asked about every stored family, so it takes only a positive normal pair sum: a family of zero-weight members falls to the bracket and is reported missing, where the run of weights would step below zero and the tool would raise. - The premises test (the re-review's first nit) counts the exact-total corrections in every function, method and nested function of the three weighing stages, and matches attribute calls as well as bare names. - Texts: the _e8_support_split docstring, the hold's docstrings and constants, the count-check comment, the receipt's tolerance_policy, the receipts test's class docstring and the changelog fragment (Receipt and gate, What moves, and one Exact total sentence this commit makes stale). What moves: tool verdicts and the tolerance_policy text only. The receipt's keys are unchanged, and copies_recomputed is still the exact recount. No node or fingerprint hashes tools/, and nothing is regenerated. - At K > 1 the count check now reports a family the stage split consistently into one copy more on a multiple. With the floor(q) + 1 stage on the nesting scenario it reports all four families in every clone at K = 2, 3, 4, 6, 7, 11, 13 and 15. The parent tool reported none. - At K = 1 it also reports such a family where four members show equal halves and others differ. A family with one corrected member now reports. - The family test no longer reports a family missing, or extra, where a household's fold crosses 60 / K from its stored state. That removes the 9 Nov probe's false missing family (a two-copy family a float above 60 whose fold lands on 60) and one false extra at K = 2. It also no longer reports a household split into two copies at 60 / K (the floor(q) + 1 stage's two-copy households), nor stage 42's one-copy household that the shared fallback lifts across 60. The rule rerun still counts the latter in copies_recomputed. Ruling 2's walk-moving case is still reported, on all seven of its frames. Synthetic evidence, counts only, against a git-archive snapshot of 8ea25b9's tool: - No false support_split_copies_stored or support_split_selection_stored on frames the real stages build, at K = 1, 2, 3, 4, 6, 7, 11, 13 and 15: - the 8 Nov tool_probe classes and the nesting scenario, split then cloned, where tampered counts get the parent's verdicts; - 30 random frames with source weights on and one to three ulps above multiples of 60, and 20 with clone weights on and above multiples of 60 / K; - the nesting scenario and 12 or 24 random frames through a hand redraw, the anchor and the residential split, in two to four gain mixes; - frames whose c sits at or within three floats of 16, 8 or 4 at K = 3, 6, 7, 12, 13 and 15, as built, with every member moved one float off c (two below where c is the power) and through the full chain. - Families held: 83 to 84 percent exactly, split then cloned. Through the full chain, 64 to 74 percent exactly and 7 to 13 percent within the reach; at K = 15, 927 of 33,600 within two floats. - Detection: floor(q) + 1 gives 2,350 to 20,628 count flags split then cloned at K > 1, and 7,934 to 21,217 through the full chain. The parent gives none. At K = 1 the full chain gives 3,258 against the parent's 1,322. Every floor(q) + 1 frame is flagged. - The false missing family's frame is green, where the parent reports {missing: 1}. - The pair arithmetic replayed at 11 clone counts from 1 to 15, 2.4 to 4.8 million members per K and case, 0.7 to 1.3 million of them within two floats of a power of two where (45 / K, 60 / K] holds one: never more than one float off c, and equal halves never moved. Across 267,035 families near a power of two, a two-float reach never accepts a count one float rejects. - The blind spot, by design arithmetic over every n from 4 to 9,998. At K = 1 it is as before. At K = 2 to 15 the snapped count's step sits 3 to 17 floats above the multiple of 60 / K. Held exactly, at most the float either side of the step, for up to 2,975 of the 9,995 counts. In the one-float tier, up to three floats from it. floor(q) + 1 on a clone of a source multiple is reported for every count where it changes the count. - Mutations: the premises test now catches a second correction written as spi_support._importance_weights_with_exact_total(...) in clone_cgt_incidence, a bare-name one in UKCGTIncidenceCloneStageTransform.__call__, and a wrapper call in a nested function. The test before let the first two through. Eight of nine tool mutations fail the receipts tests. The ninth, not letting the held counts decide the family test, changes no verdict: a held family's fold is n times its shared pair sum. Tests (engine-free, synthetic): floor(q) + 1 is reported at K = 3, 7, 13 and 15; a family with a corrected member is held on its equal halves; four equal halves vouch for a family whose others differ, three do not; families of three or fewer stay on the bracket at K = 1 and 3; through the anchor and the residential split the correct stage stays green at K = 1 and 3 and floor(q) + 1 reports 3 and 12; members near a power of two are held to two floats; the two-copy family that folds onto 60 is green; a family dropped or added away from a multiple is reported at K = 1 and 3; a zero-weight family is reported, not refused; the stage-42 one-copy check case is now green, with its stage weight checked instead. The seven engine-free files that load the tool pass (257 tests), and ruff is clean on both Python files. source_stages.json, the coverage manifest and ci_test_plan verify are current. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…#931) María's rulings of 9 Nov (5, 7, 8 and 9) asked for four things: - bind the first-party modules the injected engine computes through into the engine stages, through a second walk root at the engine class; - have the walk see the names microcosm.graph exports lazily, with the create kernel's H2-only use of load_source listed as a stop; - keep SeedSource.NONE for every spine stage kernel; - keep the graph_frames move. This commit does all four and the re-review's changelog nit. It changes keys only. - The engine root. test_uk_graph roots a second walk at PolicyEngineUKEngine, the class that graph_kernels' hermetic H2 builder and the spine build both construct (checked). - The walk follows the engine methods each engine stage's code names, by attribute on any receiver or through getattr. It also follows the constructor and every method they call on self, at the first-party scope with the stage rule's tables, and _stage_walks adds it to the stage's own walk. - The 16 engine stages call materialize and variable_metadata, and private_school_attendance also calls parameters and simulation_batches through getattr. - Beyond the three hand-bound engine modules, the walk reaches one module: microcosm.frame.materialize, through which the adapter builds each simulation's dataset. graph_kernels gains _ENGINE_COMPUTE_MODULES = (frame_materialize,) beside _ENGINE_MODULES, and every engine stage binds both (_ENGINE_STAGE_MODULES). - A new test derives the group exactly. The engine-modules test keeps _ENGINE_MODULES exact. Another stage may bind a module of the group only where its own transform computes through it, as the SPI spine's three stages and the band donors do. - The engine reaches the concept vocabulary (policyengine_uk_concepts, concept_mapping and concepts) and frame.rules only through methods no engine stage calls (concept_mapping and export_contract). So they are not bound, and the engine stages do not re-key on edits to the concept vocabulary, unlike the consequence stated with the ruling. - The fixture resolver stop's reason no longer names the engine as held. SOURCE_HOUSEHOLD_ID_COLUMN's listing is unchanged. - Lazy exports. - _resolve now follows a name that a module exports through its module __getattr__ to the submodule that defines it. It reads the table that __getattr__ reads (_LAZY_EXPORT_TABLES: microcosm.graph's _RUNTIME_EXPORTS). - The walk now reads type aliases as definitions. Three of those names are type aliases. - A test checks that every lazy name resolves to the object the package returns. It also checks that every first-party module with a module __getattr__ is listed; us_runtime.sources, which only raises, is listed as exporting nothing. - Resolving the lazy names changes no stage's walk. In the UK kernels, two things surface: - six definitions of microcosm.graph.kernel (the capabilities record, three of its enums, and the context and result records), now listed as core names with reasons; - the create kernel's load_source, now a listed stop. - The stop's reason: the create kernel calls load_source only where no root transform is bound, the hermetic registry's data-only path over the H2 fixture's captured root tables. The spine build installs the root's transform, and H2 parity binds it too (checked). - One test checks that the call sits only in the "self.transform is None" branch. Another checks that no spine kernel and no national-line kernel hashes microcosm.graph.codecs. - The blind-spot paragraph is gone from the kernel test's docstring. - SeedSource. The comment by the NONE assertion records the ruling and why NONE holds: regional_home_values' stream is a stage parameter, which its contract hash covers, and it reads no generator the executor seeds. - The changelog fragment. - The What moves sentence on local_doctrine and target_weights is reworded so that each clause is true. It was checked against the recorded source_hash calls and the kernels' walks. - The fragment now describes the engine root, the lazy exports, the load_source stop and the SeedSource ruling. What moves: - Node keys only. No value, declaration or generated file moves. - graph_kernels.py's bytes changed. So the implementation hashes of the create, identity and claim kernels and of all 53 stage and expand kernels move: 56 of the 99 UK kernels. The 16 engine stages now also hash frame.materialize. - Every node re-keys, by its own hash or by chaining: - the hermetic spine, 79 of 79; - the spine with its gates, 81 of 81; - the full graph with its gates, 111 of 111. - The other 43 kernels keep their hashes. So the national line, whose root is the bound spine checkpoint, keeps its keys, as do the transport graphs' atomic geography nodes. The country spec's and the solve's fingerprints do not move. - uk_spine.json holds only the 54 stages' stage_contract_sha256 values, and the H2 fixture holds no code hash. The fixture regenerates byte-identically, and source_stages.json and the coverage manifest are current. Tests: all engine-free, synthetic or code facts, except H2. - test_uk_graph (50 tests) and 15 related files pass (445 tests). - Ten mutations on a scratch copy each fail the tests that should catch them: - materialize newly reads the concept mapping; - frame.materialize leaves the group; - a stage calls the engine's concept_mapping through getattr; - the walk ignores lazy exports; - load_source moves out of the unbound branch; - a kernel newly reads a lazy export it does not hash; - a stage newly reads a lazy export it does not hash; - a non-engine stage binds frame.materialize without computing through it; - a spine stage hashes the codecs; - a module __getattr__ is not listed. - Ruff check and format are clean on the two edited Python files. - Through the engine slot, the H2 fixture check regenerates byte-identically and test_h2_uk_spine_parity passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María ruled on 9 Nov (ruling 6) that each phase of a gate battery keys on what its own gates compute through, with what only one phase computes through carried on that phase's params, and that the single-phase national gates bind the gate registry's whole reach. This commit does both. It moves keys only, and the full gates now refuse a stale preflight code as they refused a stale terminal code. - The model. test_uk_graph derives what each battery phase computes through (_battery_phase_walk). It follows the kernel's code, taking the branch of each `if phase == ...` the phase takes, and the binding of each gate the manifest declares for the phase, read from UK_GATE_REGISTRY's literal mapping. In stage_health it follows only the checks the phase declares, by each entry's `check`. In the spine gates' artifact table (uk_spine_gate_artifacts) it follows only the entries the phase's gates read, so the CGT asset type domain is the transferred phase's alone. The kernel rule also leaves out a method only the implementation hash calls, the full gates' _country_resources. Another test checks each part of the model against the code. - The check. A battery's shared hash holds what every phase computes through and nothing only some phases do. Each phase's own code, carried as `<phase>_code`, holds exactly the rest. The kernel rule's test holds each phase's key to cover what it computes through. - The spine gates. The shared hash has 11 modules. It gains gates, stage_evidence, graph_kernels, national_frame, receipt_counts, council_tax_constants and calibrate.geography_constants, and stochastic_assignment moves to the transferred phase. The assembled node carries a new assembled_code (frs_relationships). energy_pricing_code becomes transferred_code, 17 modules. As before, the spine gate kernel does not check them. - The full gates. The shared hash has 26 modules. local_rowwise, weighted_integrity and diagnostics move to terminal_code. high_value_homes and calibrate.geography_constants move into the shared hash, since the preflight's target surface and ledger parity compute through them. The preflight node carries a new preflight_code (microcosm.build.ledger_targets), and terminal_code hashes 37 modules. The kernel refuses a stale or missing code in either phase. - The national gates. Their hash adds the 50 modules battery_bindings.UK_GATE_REGISTRY_MODULES lists, which a test holds to the registry's derived reach. Their held stop goes, and _HELD_KERNEL_STOPS with it. The three batteries leave _HELD_KERNELS, which keeps the six target kernels. - Listed with reasons: rowwise_dataset's clone index column name, which the calibrated and national gates' support-family fold reads, and source_hash, a core name the full gates' code check reaches. - Tests and text. The spine evidence and terminal tests follow the new params. The stale code test covers both full phases and a missing code, and a new test checks each full gate node's code. The control test pins the key-only method. The changelog fragment moves the batteries and the national gates from held to bound. What moves: node keys only. No value, declaration or generated file moves. - 74 of the 99 kernel hashes move. graph_evidence.py changed, and 65 kernels hash it: every spine stage and expand kernel, uk.create, the seven dense population kernels, the spine gates, the preflight and calibrated gates and the certification. graph_terminal.py also moves the export, package, inspection and holdout kernels. graph_national.py moves the national targets, problem, gates and readback. - So every node of the gated spine re-keys (81 of 81), and every node of the country adapter's in-graph build (113 of 113). On a bound spine checkpoint, every node after the checkpoint re-keys, the preflight and the dense solve among them. On the national line 5 of 6 nodes re-key; the bound checkpoint keeps its key. The transport graphs keep theirs. - Later edits: one to a module only the transferred phase computes through re-keys the transferred gate, not the assembled gate or the stages that wait on it. One to a module only the terminal phase computes through re-keys the calibrated gates and what follows, not the preflight or the solve. On a bound spine, edits to stage_evidence, parity_reference, release_identity and release_input_coverage now re-key the preflight, and so re-solve: the preflight's release input coverage gate computes through them, and its key used to miss them. - No fixture or generated file holds a gate node. uk_spine.json, the H2 fixture, source_stages.json and the coverage manifest do not move. Tests: test_uk_graph, test_uk_graph_evidence and test_uk_graph_terminal pass (98 tests). 25 related engine-free files pass (1,276 tests) but for one test that also fails at the parent on this machine (test_uk_cgt_projection asserts the UK engine is not installed). Through the engine slot, the engine terminal and full-graph admission tests pass, the H2 fixture regenerates byte-identically under --check, and test_h2_uk_spine_parity passes. 17 mutations under scratch are caught: a phase newly computing through a module its key misses (a transferred check or the terminal branch reading an unhashed module, an assembled check reading a transferred-only one, the preflight branch reading a terminal-only one), a phase-only module moved into a shared hash, a shared module moved into one phase's code, a terminal-only module put in the preflight's code, the national gates dropping a registry module or the registry's reach, an unmodelled stage-health branch, another evaluator reading a domain artifact, the run calling the key-only method, the run dropping its code check, and a spine node dropping its code. Ruff is clean on the seven edited Python files, and source_stages.json and the coverage manifest are current. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
) María ruled on 9 Nov (ruling 6) that the derived binding rule replaces microcosm#1497's per-kernel import record for the UK target and problem kernels, that 0a9d208's edit to that test's hash inputs stays, and that the false reason in transport/compile_modules.py is corrected with it. This commit does all three. It moves keys only. - The six kernels leave _HELD_KERNELS. The table, its reason and the hold's liveness test go, so no kernel stays held. test_uk_graph's kernel rule now walks the full build's target compilation, selection, measures and problem, and the national targets and problem, as it walks every other kernel. - Bindings. What all four full kernels compute through goes in the hash they share. What only one computes through goes in its own implementation hash, as for the size search and refit. - The shared hash gains the canonical encoder. - The compilation adds chronicle_epoch, whose schema versions and fact-key epochs ledger_artifact checks and records, and frs_relationships, whose ONS household type domain ledger_targets checks the bindings against. - The measures add the 15 modules that resolving the measures and compiling their national problem compute through: graph_population, national_frame, atomic_area_support, frame.materialize, the engine adapter, dfe_spring_term, frs_annual_income, hmrc_income, income_tax_rate_classes, lcfs_consumption, uc_target_measurements, chronicle_epoch, national_calibration, local_rowwise and calibrate.monetary_binding. - The problem adds graph_population, national_frame, local_geography and calibrate.monetary_binding. - The national kernels' shared hash gains chronicle_epoch, graph_targets and the canonical encoder. The national targets add frs_relationships. The national problem adds the measures' modules except chronicle_epoch and national_calibration, which its shared hash binds, plus diagnostics and gate_battery. - The measures and the national problem also bind frs_spine for one name, the annuity and trust column. The resolver reads zeros where a frame lacks it, so on a bound spine a renamed constant would change the problem without re-keying it. - The clone index columns' name stays listed. Its reason now covers the dense measures, whose population is the expand's and which read it only to split into engine blocks, refusing a frame without it. It also covers the national problem, which resolves in one block and never reads it. The kernel rule's test checks these facts. - No kernel loses a module, though some hash modules the walk does not reach: the selection hashes 51 and computes through 6. - test_uk_target_compile_modules. UNCLASSIFIED_IMPORTS, the kernel groups and the record check go, and the docstring says the derived rule now holds these kernels. The closure check of the shared list, the hash tests and HASH_INPUTS stay, with 0a9d208's seven entries. Three of them (calibrate.monetary_binding, the engine adapter and frame.materialize) are now also imported in the national problem's hash, and their reason says so. A new test checks what the derived rule does not: - each hash input is still imported where listed; - a kernel hashes one only where another module it hashes imports it. So no kernel hashes the compile list or the solve fingerprint's module. - target_compile_modules.py's docstring no longer names the retired record. The file is a hash input, so the edit re-keys nothing. - transport/compile_modules.py. _GRAPH_RUNTIME said no kernel in the repository hashes a microcosm.graph module, which was false. It now says: - what the listed modules use those modules for; - that the compile kernels hash none of them; - that other kernels hash the canonical encoder, each UK graph kernel whose code computes through it among them. No kernel hashes compile_modules.py, so the edit re-keys nothing. - The changelog fragment moves the six kernels from held to bound. What moves: node keys only. No value, declaration or generated file moves. - 11 of the 99 UK kernel hashes move: the six, and five that hash graph_targets.py or graph_national.py (the preflight and calibrated gates, the rotated holdout, the national gates and the readback). - Full build with its gates, in the country adapter's in-graph composition: 14 of 113 nodes re-key. They are the target compilation, selection, measures and problem, the pool checkpoint, the preflight, the dense solve, the size search, draw and refit, the selected and calibrated populations, the calibrated gates and the holdout. Every node after them re-keys by chaining. The spine and the dense population and geography nodes keep their keys. - National line: 5 of 6 nodes re-key (the targets, the problem, the dense solve, the calibrated population and the national gates). The bound spine checkpoint keeps its key. - The gated spine (81 nodes) and the transport graphs keep their keys. source_stages.json, the coverage manifest, uk_spine.json and the H2 fixture do not move. Tests: all synthetic or code facts. - test_uk_graph and test_uk_target_compile_modules pass (104 tests), as do 14 related engine-free files (365 tests). Through the engine slot, engine/uk/test_uk_full_target_graph passes (8 tests). - 19 mutations on a scratch copy are caught: - each of the six kernels, or a helper the measures compute through, newly computing through an unhashed module, or reading a lazy export; - a module leaving each kind of hash: the shared, the compilation's, the measures', the problem's, the national shared and the national problem's, frs_spine among them; - a kernel hashing a hash input; - the national problem resolving in blocks, and the build moving the measures' population. A twentieth, naming the module by a computed string, passes, as the rule documents; its static form is caught. - Ruff is clean on the six edited Python files. source_stages.json and the coverage manifest are current. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three reviews of 8ea25b9, e46f54a, aaa371a, 021182d and 08b437a found no must. This commit takes the shoulds and the nits that need no ruling. Five go to María: the engine walk's root (the methods the stages call or the whole class), the reach near a power of two, where the phase-code tables live and a stale-code check in the spine gates, the two bindings the target rule forces, and the transport compile kernels' canonical encoder. The stage-42 notes' narrowing waits for (b)'s next stage-16/17/42 regeneration, so H2 regenerates once. Gate batteries: - The phase-code texts said an edit only one phase computes through re-keys that phase alone. A phase's own key leaves out what only the other phase computes through, but key chaining still re-keys the other phase wherever a node upstream hashes the module. graph_evidence's and graph_terminal's docstrings and comments, the battery table's comment, the battery test's docstring and the changelog now say so and name where: stages before the assembled gate hash stochastic_assignment and take_up_contract, and of the 38 modules only the terminal phase computes through, nine never re-solve, on a bound spine or in the in-graph build, high_value_home_gates among them. - 021182d put calibration_run, the national calibration runner, on the preflight's key, so in the in-graph build an edit to it re-solved, against microcosm#1492's design. The runner's names the preflight computes through, the spine and local gate scopes and the scoped gate manifest, move unchanged into a new gate_scopes module, which calibration_run re-exports. The full gates' shared hash binds gate_scopes in its place, and the bound spine, the national gates and the certification hash it too. The calibrated gates' terminal_code binds calibration_run, whose admin totals and CGT projection artifact their gate context is built with. A new test holds that an edit to the runner leaves both full gate kernels' hashes and the preflight's own code, and moves the terminal code. - The spine gates' enum gates resolve a domain no artifact supplies through the rules engine they are handed (PolicyEngineUKEngine's enum_domain). The battery walk now follows the engine each battery's kernel is built with wherever a phase's gates read it (_BATTERY_ENGINES: the spine build's rules engine, and the driver's coverage engine for the full gates), so the spine gates' shared hash binds the engine adapter. A test checks that each engine is the one its build hands the kernel, and the artifact that hands it on. The engine root: _engine_reach repeats the engine walk over the engine methods its own reach names outside the engine's classes until the set stops growing, so a helper the engine hands itself to is followed into the methods it calls on that argument. It adds no module today. The control test gains that case. Identity tool (tests and text only): - The premises test builds one wrapper set from all three stage modules and counts references to a helper or wrapper, not only calls, so a second correction through another module's wrapper, an alias or a partial fails it. - A test pins that the exact tier takes c from the equal-halves members. On a synthetic family a float above 900, one member's pair sum a float below c leaves the tool green, which the least pair sum over every member would not. - The tolerance policy, the docstrings and the changelog say the exact tier holds where every member that shows equal halves shares c, which is what the code checks; a member the split's own fallback corrected keeps its family on the bracket. The changelog says c can sit a few floats above 60 / K (at most 16 for K up to 15, by design arithmetic), and its Declarations bullet says what the tool reports of the selection recompute's finds and misses. Texts: fixture_stage_plan_inputs' docstring says H2 parity binds the graph's CREATE kernel to the root transform it adds; the gate registry stop's reason says the rule follows the gate kernels past it; the codecs test reads the national line's kernels from its graph, the calibrated kernel among them; test_uk_target_compile_modules' docstring states the narrower hash-input rule; docs/uk-full-build-graph.md names frame.materialize among the engine stages' bindings. Corrections to the earlier messages, for the PR body: in 021182d's later-edits bullet, the chaining above, and in the in-graph build calibration_run (until this commit), parity_reference and release_identity joined the solve's key; in aaa371a, the 16 engine stages call materialize and 6 of them variable_metadata, and on a bound spine 24 of the local line's 33 nodes re-keyed, from uk.full.expand.owned on; in e46f54a, the exact tier's condition above and the snap's range. What moves: node keys only. No value, declaration or generated file moves, and neither the solve nor the country fingerprint does. - 77 of the 99 UK kernel hashes move: every spine stage and expand kernel, the create, identity and claim kernels and the seven dense population kernels (graph_kernels' docstring and graph_evidence); the spine gates (with the engine adapter); the preflight and calibrated gates (gate_scopes for calibration_run); the export, package, inspection, readback and rotated holdout kernels; the bound spine and the certification; and the national targets, problem, gates and readback (graph_national). The calibrated gates' terminal_code moves too. The calibration and size kernels, the full target kernels, the geography, property and private rent kernels and the four atomic geography kernels keep theirs. - So every node of every UK graph re-keys: the hermetic and gated spine (79 and 81), the in-graph build (113), the local line on a bound spine (33) and the national line (6). The transport kernels keep their hashes. - source_stages.json and the coverage manifest are current. uk_spine.json and the H2 fixture hold stage contract hashes and no declaration changed, so neither regenerates. Tests (engine-free, synthetic or code facts): 26 touched and related files pass (1,133 tests). Ruff is clean on the 13 edited Python files, and ci_test_plan verify is ok. The unmutated tree passes, and 19 mutations under scratch are each caught: - a second correction in the incidence clone by attribute, by bare name in its transform, through cgt_support's wrapper, an alias, a nested function, a lambda or a partial; - an adapter helper calling concept_mapping on the engine it is handed, and a one-pass engine reach; - the exact tier taking the least pair sum over every member; - the spine gates dropping the adapter, the model ignoring the engine, and either build handing its gates another engine; - calibration_run back in the full gates' shared hash, the preflight reading a runner scope, and the full gates or the bound spine dropping gate_scopes; - the national line's calibrated kernel hashing the codecs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ffsets, on their labels (#931) María's ruling 6 of 7 Nov: key cgt_imputation's shared stream, and BADR's band offsets on the band bounds. Stage 44 built one Generator from its seed (552) and the build period and handed it to every Table 3 cell walk, pooled walk and plan in run order, so the n-th uniform belonged to whichever walk or plan ran n-th: a cell that gained or lost its gainers, or skipped, moved the uniform of every walk and plan after it. Stage 47 drew one offset per band by position, so a band added to the facts moved every band after it. - Shared scalar: stochastic_assignment.stable_identity_uniform(key, *, seed, salt) is stable_identity_uniforms' draw for one label, and refuses a sequence. The random() adapter, _KeyedOffset, moves from isa_subscriptions into cgt_imputation beside _walk_bands; the ISA subscriptions take it and the scalar from there. - Stage 44: a cell walk's offset is keyed cell:<income band>:<age group>:<region group> and a pooled walk's pool:<income band>, salted hmrc_cgt_gains_spine:offset:<period>; a plan's jitter plan:<gain band>:<income band>, salted hmrc_cgt_gains_spine:jitter:<period>. Bands and age groups are named by their lower bounds, region groups by their labels. The seed stays 552 and the period moves into the salts. Each walk and plan reads one uniform. _walk_bands and _draw_plan_amounts keep their signatures; their rng annotation names the adapter too. - Stage 47: a band's offset is keyed <lower>|<exclusive upper> (open for the top band), salted assign_badr_qualifying_gains:offset:<period>, seed 555 (_badr_band_offset). - Declarations: stage 44's seed_mixing and stage 47's realization texts say this, and the sub-AEA mapping text no longer names a walk and plan draw order, which no longer exists. The code dictionaries, sources.yaml and source_stages.json agree; the docstrings and comments follow. - Bindings: the derived rule now finds stochastic_assignment in the reach of stages 44 and 47, so both bind it. test_uk_graph's invalidation test takes regional_property_uprating and private_school_attendance as controls in their place. - property_components' receipts-walk offset is unchanged (ruling 6). No nesting is promised. The offsets and jitters are the same at every K and whichever other cells walk, but the walks stay pooled: the rounding carry within an income band, pass 2's walk over the band's unassigned gainers, plans pooled across cells, and each BADR band's walk over its pool. What moves (synthetic counts only): - Stage 44: every walk offset and plan jitter, so band membership near each walk boundary and the within-band quantiles. On three synthetic frames, 9 of 1,002, 9 of 3,000 and 126 of 1,512 gainers changed gain band and 338, 2,920 and 1,300 changed amount; no non-gainer moved. - Stage 47: every band's offset, so each band's claimants. On three synthetic frames of equal weights no claimant row of the old walk claims in the new one, each frame's claimant count moved by one or two, and the main asset types moved with the claims. - ISA: unchanged. On three synthetic frames all five output columns are bit for bit the parent's (4,881 person rows). - Node keys: all 79 spine nodes' implementation hashes move, since graph_kernels and stochastic_assignment changed and the root binds stochastic_assignment, so every node re-keys, as does every node keyed on the spec or the coverage manifest. Values move from stage 44 on; isa_subscriptions re-keys through its cgt_imputation binding and its own module with no change in its output. - Regenerated: source_stages.json; the coverage manifest's 25 source digests; in the H2 fixture, the three texts in fixture.json, the two stages' contract hashes in uk_spine.json, and 22 values of the synthetic cgt_asset_type_facts.json, which the tool sizes to the frame after the redraw. No gates.json note goes stale. Tests (engine-free, synthetic or code facts): - New: the scalar is the array helper's draw bit for bit and refuses a sequence. Each walk's and plan's uniform, recorded under a label taken from the call's own members, is the keyed scalar of that label, salted with the period. It is unchanged when an earlier cell gains members or skips, and the one-cell higher income bands keep their members and amounts. The redraw is invariant to a row permutation. A band inserted into Table 4.1 leaves the other bands' offsets and claims as they were. ISA's offsets and jitters are the uniforms they were. - 16 mutations of a scratch copy are each caught, and the unmodified copy passes. The shared Generator as before, and offsets keyed on the walk's position, fail the invariance tests (under the shared stream 54 of the higher bands' 60 plan amounts move); mis-keyed cells, pools or plans, salts without the period and one salt for offsets and jitters fail the label tests; BADR's positional offsets as before, a position key and a lower-bound key fail the insertion test; and the ISA, scalar and binding mutations fail theirs. - The seven touched test files pass (230 tests), as do the coverage manifest, declared-seed and replay-artifact tests (31). Ruff is clean on the 10 edited Python files, and ci_test_plan verify is ok. source_stages.json and the coverage manifest are current, the H2 fixture regenerates and passes its check, and H2 parity passes through the engine slot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Six walks ordered exactly tied rows by id: stage 42's support walk, stage 26's high-value stratum, stage 28's receipts walk, stage 44's rank and stage 47's BADR and main asset type walks. Ties broken by id alone put the lowest id first every time, and the copies and geographic clones of one household differ only in their copy ordinal, so the first copy or clone won every tie (synthetic probe of the design: the lower clone index ranked first in all of 70,035 tied clone pairs under id ties, in 49.9% under keyed ties). Accepted as designed on 7 Nov, with the verifier corrections. - Helper: stochastic_assignment.keyed_tie_order(primary, ids, *, seed, salt) returns np.lexsort((ids, u, primary)), u the records' keyed_tie_uniforms(ids, *, seed, salt), stable_identity_uniforms of integer_record_keys(ids, label=salt). It is a secondary sort key, the form stage 27 uses, so distinct values keep the id order bit for bit; it draws only the tied records' uniforms. It keys each row on its own id, never on spine_household_keys, and is not the rank-inside-a-tie form. - Sites: stage 42 seed 0, salt cgt_support_split:wealth_tie; stage 26 seed 0, salt high_value_home_support:value_tie, its mid-rank unchanged; stage 28 walk_seed and a new tie_salt allocate_property_rental_income:tie, which ReceiptsAllocationSpec reads and refuses when empty or equal to a salt the stage's finance-cost draws key the same person ids on (the op has no other salt, so the stage passes those two); stage 44 one uniform per person, seed 552, salt hmrc_cgt_gains_spine:rank_tie:<period>, drawn once beside rank_key and passed to _ranked, _draw_plan_amounts and _map_remainder_amounts as a keyword-only tie (None keeps the person_id order, so their direct test calls are unchanged); stage 47 seed 555, salts assign_badr_qualifying_gains:tie:<period> and assign_main_asset_type:tie:<period>. - Declarations: the two split operations declare seed and salt and the receipts operation its tie_salt, in the code dictionaries, the strict schema (three operations), sources.yaml and source_stages.json. The selection, stratum_rule, order, ordering, mapping, realization and values texts, the two stage notes, the docstrings and the receipts walk's and asset types' receipt order texts follow; "no draw, no seed, no salt" and "deterministic, no seed" are gone. spine_build._declared_seeds registers the splits under their salts and walk_seed under the tie salt. - Rule B at stage 42: both selections take seed and salt with no default. The split, keeps_the_split's rerun and the identity tool's recompute pass cgt_support_tie_break(), read from the declared operation. The exact-total correction's candidate order (the heaviest whole households first, tied weights in row order) stays: it places a rounding-sized correction, not a draw, and copy_rule states it. - One stage 42 walk on a synthetic vector the size of a K = 15 spine takes 0.5 to 0.7 s, against 0.38 s before (the uniforms of tied households and the id check); the correction cap's comment gives the new cost. - The five stages already bind stochastic_assignment, so the derived binding rule needs nothing new. No receipt key is added; the band receipt's wealth_threshold is untouched. - Stale, left for (c): uk/gates.json's high-value home support gate note says the stage "makes no draw". gates.json is not edited here. What moves (synthetic counts only): nothing where no exact tie touches a cut or a position schedule. - Stage 42: a band's crossing household in a tie group. With three clones per household, 2 to 8 of about 850 households per frame changed copy count over six frames, and clone 0, which carried the most families in all six, now does in one; at one clone none moved (the tied family members there were too light to copy). - Stage 26: the tie group across a region's cut, 0 to 5 of 3,000 owners per frame over six frames of values on 300 atoms. - Stage 28: receipts among tied landlords; about two thirds of the tied landlords' receipts moved on three frames, none outside the tie groups, band counts unchanged. - Stage 44: 43, 99 and 62 of about 1,800 gainers moved amount where wealth varies (the blended key ties), none where wealth is equal; no non-gainer moved. - Stage 47: bit for bit on three frames. - Node keys: every spine node re-keys, since stochastic_assignment changed and the root binds it; 49 of the 79 nodes' own implementation hashes move. Values move from stage 26 on. - Regenerated: source_stages.json; the coverage manifest's 25 source digests; in the H2 fixture, 12 texts and 5 new keys in fixture.json, the stage contract hash in uk_spine.json, and the synthetic facts the tool sizes to the frames stages 28, 29 and 46 read: property_rental_facts.json (8 values, in their last digits), property_purchase_targets.json (3 values, two by 4 to 5 per cent) and cgt_asset_type_facts.json (28 values, by up to 4.4 per cent). Tests (engine-free, synthetic or code facts): - The helper is the id order on input without ties, is a permutation, equals the full lexsort on tie-heavy input (NaNs and signed zeros included), breaks a tie otherwise than ids would, does not depend on the row order, and refuses float, repeated and misaligned ids. - At each site, a tie across the cut, with ids whose keyed order differs from id order, follows the keyed order computed in the test; distinct values give the id order's result bit for bit; permuting the rows changes nothing. Stage 44's plans and remainder give ties the keyed order reversed, and every _ranked call reads the one keyed draw. - Clone-order guard (verifier P10): with three clones of 60 tied families, clone 0's share of the taken (stages 42 and 26), of the top receipts (stage 28) and of the first ranked (stage 44) lies inside the four-sigma binomial band around a third; under id order it is one (stage 26: the last clone, every time). - The CGT support scenario still selects households 2, 3 and 6; the partway walk test now takes the tied clone the keyed order puts first. The split's walk and its rerun pass the declared pair. The identity tool stays green on a tie at the crossing at three clones, passes the declared pair, and reports the family as missing and clone 0's as extra when its recompute walks by id. The seed and salt pins in test_uk_source_stages and test_uk_frs_spine follow. - 12 mutations of a scratch copy are each caught and the copy as is passes: id order in the helper and in _ranked, keying on the geographic-branch-stripped id (spine_household_keys) in either, the rank-inside-a-tie form at stages 26 and 28, the rerun on another seed, a tie salt equal to a finance-cost salt, salts without the period, the asset type tie held at seed 555 whatever the BADR seed, and the declared seeds without the splits. - The 11 touched test files and test_uk_graph pass (694 tests), as do 8 related files (357: the coverage manifest, spec bundle, country spec, copy-id table, rules-engine provenance and fixture-reading tests). Ruff is clean on the 19 edited Python files and ci_test_plan verify is ok. source_stages.json and the coverage manifest are current, the H2 fixture regenerates and passes its check, and H2 parity passes through the engine slot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's ruling 7 of 7 Nov. Half of each subscriber's ISA amount key at stage 53, and of each Lifetime ISA payer's payment key, is the person's balance rank. _weighted_rank sorted balances stably and gave each row its own mid-rank, so tied balances ranked by row position, although its docstring said ties share a rank. Ties are common: the copies and clone halves made after the WAS wealth draw carry their root's ISA totals, so a single-adult one ties its root, and QRF draws repeat a few atoms. A root sits before its copies, as clone 0 will before its clones, so it ranked lowest among them. The stage was not row-order invariant where balances tie: the design's synthetic probe, re-run, moves 372 of 400 amounts under a row permutation on five atoms at the parent and none now (none on distinct balances at either). - _weighted_rank(values, weights, ids) orders by (value, id). Every member of a tie group takes (F(x-) + F(x)) / 2 over the total weight; a singleton keeps (c - w / 2) / total, computed as before. Both callers pass the person ids, so the keyed uniform half of each key alone orders tied holders, exchangeably, and every rank is the same bit for bit under any row order. The module docstring's step 3 says so. - No seed, salt or manifest key. No declared text says anything about ties (within_band stays "uniform"), so nothing regenerates: source_stages.json and the coverage manifest check current, and the H2 fixture holds no stage 53 value or kernel hash. No gates.json note mentions the rank. The band walk and the LISA strata stay pooled. - The rank stays in isa_subscriptions, its only user, so no shared module moves and only stage 53's kernel hash does. What moves (synthetic counts only): - Nothing where no balance ties: on the three synthetic stage frames of distinct balances every output is bit for bit the parent's. - On the same frames with balances on five atoms, every tied holder's rank moved and no singleton's (the rows arrive in id order, as the spine's tables do). Amounts still move beyond the ties, since the band walk and the within-band order are pooled: 248 to 260 of the 363 to 376 tied subscribers and 329 to 344 of the 420 to 430 others moved amount, and 46 to 56 of the 52 to 68 Lifetime ISA payers, all tied, moved payment. No balance moved. - On those frames a row permutation moved 588 to 599 of the 792 to 796 subscribers' amounts and every payer's payment at the parent, and moves none now. - With three geographic clones of 2,000 holders on five atoms, clone 0 gets the least of its three in 33 (ISA amounts) and 32 (LISA payments) per cent of the holders, against 43 and 40 by row position. - Node keys: 1 of the 99 UK kernel hashes moves, stage 53's. The gated spine re-keys 3 of its 81 nodes (isa_subscriptions, trading_allowance_receipts and the transferred gate) and the in-graph build 35 of its 113, from isa_subscriptions on; the national line on a bound spine keeps its 6. On the spine only stage 53's two subscription columns and its LISA payments move; stage 54 reads none of them. Tests (engine-free, synthetic or code facts): - New: a tie group's members share its mid-rank, by hand and over 3,000 tie-heavy rows; on id-sorted rows singletons keep the old rank bit for bit, and without ties every row does; the ranks are unchanged under five row permutations, where the old rank moved nearly every tied row's; with three clones of 2,000 holders, clone 0's share of the least ISA amount and of the least LISA payment lies inside the four-sigma binomial band around a third, and the old rank, patched in, puts both outside. - test_reordering_the_rows_moves_no_draw gains a five-atom case, which checks the ties are there. It fails at the parent (27.5 per cent of cash_isa_subscriptions differ) and passes now; the distinct case passes at both. The clone test fails at the parent too. - 8 mutations of a scratch copy are each caught and the copy as is passes: the old rank; the group mid-rank on row-order sums; ties at F(x) or at F(x-); singletons by the group formula; either call site on the old rank; ties by descending id. - The ISA stage-health entries pass on the five-atom frames at the parent and now. - test_uk_isa_subscriptions (22), test_uk_isa_stage_health (17) and test_uk_graph (53) pass. Ruff is clean on the 2 edited Python files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mits (#931) Three reviews of 65e50b9, b8bd453 and 0c850b1 found no must. This commit takes the five shoulds and the ten nits (two of them the same national-line point); none needs a ruling. The commits are not amended, so the corrections to their messages are listed below for the PR body. 0c850b1 needed no change. Tests: - Stage 44's label test also runs on a frame with a short second cell in the 37,700 income band, so that band's pooled walk runs too. Above band 0 a band's position and lower bound differ, so a pooled walk keyed on its position, or every pooled walk keyed on the lowest band, now fails it. - Stage 53: a new test records the uniform each of select_subscribers' cell walks and draw_band_amounts' walk reads, and each band jitter the amounts draw, and compares them with the pre-move expressions, computed from each call's own seed, salt and key. A call that keys or salts its draw otherwise now fails, where the helper test passed. - Stage 47: the keyed-ties test also hands each walk in turn the id order and asserts that the tied persons' BADR gains, then their asset types, move. A walk that draws the keyed order and walks another now fails it. No stage 47 clone-order guard is added: gains are plan amounts and rarely tie. - Stage 28: a new test on unequal-weight ties (half of 2,400 landlords on 60 round profits, at weights that differ) holds that, against the id order, no untied landlord changes band, while tied landlords cross and the bands' record counts move. Code: - Stage 47's band offset key refuses an edge that is not an integer, with a TypeError naming the type. _validate_table4 compares edges by value, so a roster read with float edges passed it and would have keyed every band's offset anew. A NumPy integer keys as the Python integer does. Both loaders give ints, so no value moves. - cgt_asset_type's module docstring: the residential flag is carried as weight by cgt_residential_split since microcosm#1063 and read here; the weighted systematic walk is described under the BADR claims, which use it; and the main type is assigned by error diffusion, with the keyed tie-break its only draw. - keyed_tie_order's docstring: the lowest copy ordinal always came first, not always won. Changelog fragments: - 931-cgt-keyed-offsets: the rake couples income bands; gainers cross the exempt amount; the national line's keys; ISA takes the adapter from cgt_imputation and the scalar from stochastic_assignment. - 931-keyed-tie-breaks: stage 28 with unequal-weight ties; stage 42 at one clone; which copy ordinal won (the last at stage 26); copy_rule names the cap of 64 candidates, not their row order; the national line's keys; five stage contract hashes. Corrections to the earlier messages, for the PR body (synthetic counts only): - 65e50b9: - "No nesting is promised" also misses the rake. It seeds every cell from the frame's own gainer shares and fits the seed to age and region margins over all income bands, so a change in one cell's gainers moves the raked targets of cells in other income bands. On a synthetic frame, one added cell in the lowest band moved all 20 positive targets of a two-cell higher band, by up to 0.8 per cent. - On the carry frame 70 of the 1,512 gainers crossed the annual exempt amount (none on the other two frames), so the liable set moves, and with it stage 46's arms and row counts. - On a bound spine the national line re-keys 5 of its 6 nodes, all but the bound checkpoint: the national target, problem and gate kernels hash the country spec, which the commit edits. - ISA takes _KeyedOffset from cgt_imputation and the scalar from stochastic_assignment. - b8bd453: - Stage 28: untied landlords keep their band. "Band counts unchanged" holds only where a tie group shares one weight, as on the commit's family frames. Where a tie group of unequal weights straddles a band boundary, tied landlords cross it, band counts and masses change, and the other landlords' receipts in the bands either side move. On three frames of 2,400 landlords, half on 60 round profits at weights that differ: 24 to 49 tied landlords changed band; 4 to 7 of the 11 bands' record counts changed; no untied landlord changed band; and 398 to 941 of about 1,200 untied landlords' receipts moved, by at most about 0.3 per cent. - Stage 42's zero at one clone comes from light synthetic families. At one clone the copies move where a band's crossing falls in a tie group whose members weigh above 60, as a high-value family's can. - The H2 fixture moved five stage contract hashes in uk_spine.json (cgt_support_split, high_value_home_support, property_components, hmrc_cgt_gains_spine, hmrc_cgt_asset_type_spine), not one. - The first copy or clone won every tie at stages 28, 42 and 44; at stage 26, which takes the top of an ascending order, the last did. - copy_rule names the cap of 64 candidates, not their order: tied weights take row order. - On a bound spine the national line re-keys 5 of its 6 nodes, as for 65e50b9. What moves: node keys only. No value, declaration or generated file moves. - 50 of the 99 UK kernel hashes, those that hash stochastic_assignment or cgt_asset_type: 48 stage kernels, uk.create, and the national gates through the gate registry's reach. - Every node of the hermetic and gated spine (79 and 81) and of the in-graph build (113) re-keys. On a bound spine the national line re-keys 1 of its 6 nodes, the calibrated gates. - source_stages.json and the coverage manifest check current. No declaration changed, so the H2 fixture does not regenerate. No gates.json note goes stale. Tests (engine-free, synthetic or code facts): the four touched test files and test_stochastic_assignment pass (189 tests), as do test_uk_graph, test_uk_isa_stage_health, test_uk_cgt_residential_split and test_uk_identity_stability_receipts (183). Ruff is clean on the 6 edited Python files. Under scratch the unmutated copy passes, and 9 mutations are each caught: - the pooled walk keyed on the band's position, or on the lowest band; - the ISA jitter keyed <band>:<key>, the amounts walk salted :walks, and the cell walks salted :cells; - either stage 47 walk drawing the keyed order and walking the id order; - the band key without the integer check; - stage 28 keying near ties. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The derived binding rule's walk lived in test_uk_graph.py. Holding the
transport kernels to it needs the same walk under the transport's own
tables, and country-neutral machinery belongs in one shared module, with
each country's details passed in as a declaration. This commit moves the
walk and its controls as they were. It changes no UK table, walk or key.
- test_support/microcosm_build/derived_binding_rule.py holds the
machinery. A country declares its tables in a RuleTables (core
modules, refusal-only checks, kernel stops and declaration modules)
and walks through a BindingRule built from it. The functions are
test_uk_graph's, word for word, except that each reads its tables
from the rule:
- table-free: module_definitions, writes_nothing, statement_names,
key_only_methods, merged, hashed_by and the rest;
- the rule's methods: resolve, named, helper_kind, walk, helper_walk,
kernel_body, kernel_walk and the engine-root walk;
- four checks both countries need, taken out of UK test bodies:
kernel_classes_defined (the scan for classes that declare a ref),
findings (each walk's modules outside its key, its stops outside
the key, the listed stops met and the core names outside the
Frame), and the rule's refusal_only_violations and
core_modules_off_run_graph_path.
- The lazy-export tables and the in-place calls are facts about the
first-party packages, so they are shared constants, and a rule's
lazy exports default to them.
- The generalisations the design flagged, as settled here:
- declaration modules are optional and empty by default, so a walk
then skips no module;
- each country gives its own reasons for its core modules; what is
shared is the check that each is on run_graph's path;
- a rule caches each definition's kind, and the lazy exports, per
instance, since both depend on its tables;
- findings leaves the Frame's core names out for every country, as
the UK did, since a country's Frame reason covers them.
The graph-fact reasons of the UK's core names stay in its table here;
the transport commit shares them.
- tests/engine_free/shared/test_derived_binding_rule.py holds the five
controls that moved: the refusal check, the helper rule, the kernel
rule, the engine root and the lazy exports. Each builds a rule from
its own scratch tables instead of monkeypatching the UK's.
- test_uk_graph.py keeps its tables and its UK-only parts (the stage
table, the engine root and the gate-battery phase walk), builds _RULE
once from its tables, and binds the rule's methods to the names its
code already used. Its functions and test bodies read as before, but
for the four that now call the shared checks.
What moves: nothing. No key, value, declaration or generated file.
Tests: all code facts.
- Parity: a scratch dump of every UK walk the rule computes is the same,
byte for byte, at 8ddd0a6 and at this commit. It covers the 54 stage
transforms and the engine methods each names, the 16 engine roots, the
54 merged stage walks, the bodies and walks of the 48 kernel classes
behind the 99 UK kernels (with the default halts, and without the gate
registry's stop), the 99 kernel rule walks with the 4 battery phases
and their phase-code modules, the helper walks the UK tests take
directly, the six refusal-only verdicts, the registry bindings and the
stage-health dispatch, the spine-build installs and the 99 UK kernel
hashes.
- test_uk_graph (48 tests), test_derived_binding_rule (5),
test_uk_target_compile_modules (53) and test_first_party_imports (6)
pass. Ruff is clean, and the test plan verifies.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…usal check (#931) The refusal check (writes_nothing) refused every store through a subscript. So a check that records what it has seen in a dict of its own could not be listed as refusal-only: uk_runtime.ledger_targets._assert_household_type_bindings_declared, which stores covered[value] = target_id, is one. This commit teaches the check that such a container is the function's own. It changes the shared machinery and its controls only. - local_containers finds them. A name is local when: - the function's own scope binds it to a new container: a dict, list or set display or comprehension, or a call of dict, list or set; - every other binding of it anywhere in the function, nested scopes included, is one too; - it is no parameter of the function or of anything nested in it. Bindings of every kind count against it: loop, with, walrus, unpacking and comprehension targets, deletions, imports, nested definitions, and exception and match captures. A call counts only where the function binds no dict, list or set of its own. - writes_nothing now lets the function store into or delete one item of a local container, call an in-place method on it, or augment it as a whole, in its own body or in a closure. It still refuses: - a nested store, into an item of an item or an attribute of one; - an attribute store on the container; - a store through any other name, including an alias of the container or of a parameter; - returning the container. - Two departures from the design: - An augmented store into an item (counts[row] += 1) is still a write. It calls the item's own augmented operator, which changes a list or array item in place, and the item may be an object the function was handed: seen = list(rows) holds the rows' own objects. The plain store counts[row] = counts.get(row, 0) + 1 is allowed. - The fresh binding must be in the function's own scope. The design also counted one in a nested scope, but a name that only a nested function binds is a global in the function itself. - A new control holds 10 functions that write only into their own containers and 14 that still write. What moves: nothing. No table, walk, key, value, declaration or generated file moves, since writes_nothing only checks the listed refusal-only entries. This commit neither lists _assert_household_type_bindings_declared nor unbinds frs_relationships; the UK follow-up does both. Tests: all code facts. - _assert_household_type_bindings_declared fails the check at 8ddd0a6 and passes it here (checked in scratch). - The six UK refusal-only entries still pass, and the scratch dump of every UK walk is unchanged. - Of the design prototype's 27 controls, all but its augmented-item case keep their verdict. - test_derived_binding_rule (6 tests) and test_uk_graph (48) pass. Ruff is clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On 9 Nov María asked for the bigger transport change: hold every transport kernel to the derived binding rule, as #931 did for the UK, instead of only adding the canonical encoder to the transport compile list. This commit does so with the transport's own tables, and binds what the rule finds. It moves keys only. - test_transport_binding_rule holds the 45 kernels build_transport_registry registers: the transport package's 37 kernel classes, the shared atomic geography's four, calibrate.ordered_adam@1 and the three rules kernels. test_support/microcosm_build/transport_kernels.py replays the registry's registration calls with a stand-in AxiomEngine, and the test holds the replay to a real build_transport_registry, built engine-free with the toy package's engines. Each table entry has a reason a test checks: - the eight core modules, each on run_graph's path, with transport reasons (the transport graph tests run composed graphs through run_graph); no transport kernel declares SeedSource.EXECUTOR; - 19 core definitions: 15 graph records, enums and constants, and four with transport reasons; - two refusal-only checks, graph_inputs.require_outputs and reference_profiles._validate_interval_resolution, both of which writes_nothing passes; target compilation gives the second no interval document; - one stop, target_kernels._COMPILE_MODULES, met only through the first quantile map's class and read only inside implementation hashes; - no unbound names and no declaration modules; - what the registry hands kernels whose classes their code need not name: the AxiomEngine behind the three rules kernels and the input closure, and TRANSPORT_GATE_REGISTRY behind the battery, both checked against build_transport_registry's code; and the battery's bindings' source closures, which its key binds beyond its source_hash calls (checked: a changed digest re-keys it). - The shared module gains two things: - keyed_walk, which walks a kernel with what its build hands it. Each handed object is walked on its own, with the kernel's halts, and joins the kernel's walk, so it can only add to what the kernel walk finds, as in the design's prototype. One walk with the engine as one more root would stop following the AxiomEngine class whole where the input closure names it for an isinstance check, and so miss frame.materialize. - GRAPH_RECORD_REASONS, the graph-fact reasons. The UK's _KERNEL_CORE_NAMES takes its eight from there and keeps its 12 keys; ArtifactOutput's text now reads for any kernel. - What is bound: 79 (kernel, module) pairs in 36 kernels, in 14 files, each with a short comment on what computes through it: - the canonical encoder in 32 kernels. The seven that hash the compile list gain it through the list. In ten (the four column kernels, the measure, two money kernels, two reporting kernels and the currency) only canonical_document_param's spelling check computes through it; - reference_factors, ledger_artifact and chronicle_epoch, through which decode_target_surface or validate_reference_resolution checks a surface's factor evidence and its Chronicle feed identity, in targets.problem, takeup.assign, diagnostics.calibration and geography.support_from_facts. calibration.prepare, which hashed reference_factors already, gains the other two, and the battery gains reference_factors alone; - artifact_types in support_from_facts, takeup.assign, takeup.cell_diagnostics and the three reporting kernels; calibrate.target in support_from_facts and takeup.assign; graph_atomic_geography in support_from_facts; - country_spec, gate_battery and gates in export.prepare and transport.package; build.trace in export.readback; takeup_kernels in the battery; - the Axiom input surface in concepts.encode and encode_groups, and so scenario_override; frame.rules and frame.materialize in the input closure; calibrate.hierarchy in the weights audit and the diagnostics, which also gain monetary_binding; - frame.concepts in targets.problem, for one name, the source household column the interval diagnostics count by; - frame.exact_money in the counterfactual bridge, through a new implementation_hash that the zero solve now inherits unchanged but for the encoder; - the ordered solve's internals (exact_k, the L0 gates, their initialization record, the hierarchy, monetary_binding and the encoder) in calibrate.ordered_adam@1's adapter half. calibrate.adam@1's hash, which UK kernels and microcosm-graph's parity pins embed, does not move. - compile_modules.py. The encoder joins TRANSPORT_COMPILE_MODULE_NAMES (now 14 modules) and leaves EXCLUDED_DIRECT_IMPORTS. _GRAPH_RUNTIME now covers only the kernel protocol and the declarations, with why: both are core modules the rule lists, and binding either would re-key every node. The docstring loses the #1497 sentence. No kernel hashes the file. - test_transport_compile_modules drops UNCLASSIFIED_IMPORTS, the kernel groups and the record check, as 08b437a did for the UK. It keeps the closure checks, the geography-label re-key test and HASH_INPUTS. Its list test becomes test_no_transport_kernel_hashes_a_hash_input: the list is still imported where listed, exactly the seven listed kernels hash its modules, and none of the 45 hashes the list itself. first_party_imports' recorded_import_mismatches and unhashed_imports, which nothing else used, go with their two controls. - The pinned compositions follow: test_transport_evidence_kernels' module lists, and the by-ref kernel's three in microcosm-frame's test_rules_kernels. - transport_composed gains toy_rules_bindings, the bindings document make_composed_fixture built inline, which the new test uses too. - The changelog fragment 931-transport-helper-bindings says what moves. What moves: node keys only. No value, declaration or generated file moves. - 40 of the 45 transport kernel hashes move. The four atomic geography kernels and geography.person_location@1 keep theirs. - New Zealand's packaged graph, composed as test_nz_transport_declarations composes it: all 357 nodes re-key, 353 directly and four (nz.geo.assign, derive, gate and person) by chaining. Belgium composes no graph. - 0 of the 99 UK kernel hashes move, calibrate.adam@1 keeps f6554bfe..., simulate.rules@1's hashed files are untouched, and nothing of the US's moves. No committed file records a transport kernel hash or node key. Tests: all code facts. - A scratch dump of every UK walk the rule computes, with the 99 UK kernel hashes, at 8ddd0a6 and at this commit, differs only in ArtifactOutput's reason text. - Six mutations of a scratch copy are caught: a dropped binding (the column kernels' encoder), a new unhashed helper in a money kernel's run, a reader of the compile list outside a hash, an AxiomEngine method computing through a new module, the battery's binding sources left out of its key, and a rules kernel left out of the handed-engine table. - On 8ddd0a6's sources the new test fails, on the 77 unhashed (kernel, module) pairs in 36 kernels; 26 stop names lie outside their keys there. - The 54 engine-free test files that import a changed module pass: 4,268 tests, with 105 skipped where a Belgian or NZ Chronicle artifact, the Axiom native extension or a pinned-source checkout is absent. Two failed in a first run, from a source edit made while the run held a battery kernel, which then refuses its changed binding registry; both files pass on a rerun (500 tests). test_graph_parity_pins passes (11 tests). Ruff is clean, and the test plan verifies. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e shared checks (#931) The review of 354296a and 8c10ecc found that the extraction moves nothing for the UK, and four gaps in the refusal check and its controls. This commit closes them. It changes the shared machinery and its controls only: writes_nothing checks only the listed refusal-only entries, so no table, walk, verdict or key moves. - Augmenting a local container as a whole now needs a new container as the value. x += v can rebind x to whatever v's reflected operator returns, v itself included: Python tries that operator before a list's in-place concatenation, and after a set's in-place operator declines a value that is not a set. So seen = []; seen += values; seen[0] = 1 wrote into values and passed. 8c10ecc's message and docstring said a container's augmented operators change it in place or refuse; they need not. - An in-place call on a local container is a write again when it is handed another object's in-place method: seen.sort(key=values.pop) and seen.extend(iter(values.pop, None)) empty values. Both were refused before 8c10ecc. A method handed to any other call, as in sorted(rows, key=values.pop), was not seen before 8c10ecc and still is not; the docstring now says so. - A global or nonlocal declaration disqualifies a name, as 8c10ecc's message said every binding does. writes_nothing already refused such a function, so no verdict moves. - The controls: - the containers control gains a writing case for each branch that had none: an import, a nested function, an exception capture, a match capture, a class body (which the function's own scope leaves out), a global declaration, the two augmentations by a handed value and the two handed methods; and one more refusing case, a set joined by a new set; - test_the_shared_checks_report_what_they_find plants a violation for each check 354296a moved into the shared module (refusal_only_violations, core_modules_off_run_graph_path and findings), and holds the per-rule cache of each definition's kind: two rules whose tables differ classify the same alias each by its own tables, whichever classifies first. Corrections to the earlier messages, which stay as they are: - 354296a says four UK test bodies now call the shared checks; five do. It also says each moved control builds a rule from its own scratch tables instead of monkeypatching the UK's: the refusal-check control builds no rule, and only the helper-rule and kernel-rule controls monkeypatched the UK's tables before. - 8c10ecc says bindings of every kind count against a local container. Global and nonlocal declarations did not until this commit, and its list of the other bindings leaves out the augmented assignments and bare annotations it accepts. Tests: all code facts. - test_derived_binding_rule passes (7 tests). - The six UK and two transport refusal-only entries still pass, and uk_runtime.ledger_targets._assert_household_type_bindings_declared still passes the check. - Eleven mutations of the shared module each fail a new control: removing the import, nested-definition, capture or global branch, letting class bodies into a function's own scope, dropping the augmented values or the callback check, emptying refusal_only_violations, core_modules_off_run_graph_path or findings' stops, and sharing the kind cache across rules. - Ruff is clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ort reasons (#931) The review of 6c76c90 found no binding the code misses today. It found one way a new kernel could get past the transport's tables, two reasons that were wrong, and claims in the changelog fragment and the message that overstate. This commit addresses them. It changes the transport rule test and the changelog fragment only: two reason texts change, and no table entry is added or dropped, no walk moves and no key moves. - The kernels the registry hands its engines to are now held to the registry as built, not only to its code's spelling. test_the_registry_hands_the_kernels_what_the_tables_list matched only a positional argument named engine_mapping, so a new kernel handed the engines by keyword, whose run calls their methods, passed every test while its key hashed only its own module. The test now: - builds build_transport_registry with the toy package's engines (a module-scoped fixture the registration test shares), and requires the kernels that hold one of those engines, at any depth of their attributes, mappings, collections, bound methods, partials or closures, to be exactly the four HANDED_CLASSES lists; - scans keyword arguments as well as positional ones for the mapping; - requires AxiomEngine to be the only class with the rules-engine protocol's methods that the builder constructs, so a second engine class built for some bindings cannot go unwalked. The review's probe, a kernel handed the engines by keyword and registered in both the registry and the replay, now fails the test, as does the same kernel handed them through **{"engines": engine_mapping}, which only the built-registry check sees. - Two reasons are corrected: - the Frame's said no transport key hashes it. The matrix kernels, the ordered solve, the battery, the rules kernels and export hash its modules for their own reasons; it now says the rule asks no kernel to hash them; - require_outputs' said only the kernels that write the problem's and the diagnostics' outputs compute through those outputs' modules. The kernels that read them do too (the battery, the ordered solve and the weights audit), and the rule holds every one of them to hash those modules. - The changelog fragment's first sentence now says the kernels hash what they compute through as far as the rule can see, apart from the core modules and the two refusal-only checks, and its New Zealand count says the test composition's 18 synthetic quantile stand-ins re-key too. Two bindings are wider than the kernels need, and stay, since they cost keys only: simulate.counterfactual@1 now hashes frame.exact_money, which only the zero solve's quantized branches run, so an edit to it re-keys the counterfactual's nodes needlessly; and the input closure hashes frame.materialize because the rule follows AxiomEngine whole. Corrections to the earlier messages, which stay as they are: - 6c76c90 says all 357 nodes of New Zealand's packaged graph re-key. Composed as test_nz_transport_declarations composes it, the graph has 375 nodes, 18 of them synthetic quantile stand-ins, and all 375 re-key. - 6c76c90 says each table entry has a reason a test checks; the two reasons above were neither true nor checked. - 8f6c369 says 8c10ecc's message and docstring claimed a container's augmented operators change it in place or refuse; only its docstring did. Tests: all code facts. - test_transport_binding_rule passes (8 tests). Ruff is clean. - The review's probe, with the engines handed by keyword and positionally, and its **-mapping variant each fail the test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…le spine phase code (#931) María ruled on 9 Nov (late) that the gate phase-code functions and tables move into a small module that only the stale-code checks read, listed refusal-only (3a), and that the spine gates refuse a stale or missing phase code, as the full gates do (3b). This commit does both and takes the first two nits the re-review of a0933a5 left. It moves keys only, plus that one refusal. - The module. uk_runtime/gate_phase_codes.py holds the four phase-code functions, with their @cache and docstrings, and the two tables, which graph_evidence and graph_terminal defined. Their code is unchanged, so each phase code keeps its value. It adds one check both batteries call, require_current_phase_code(params, phase, *, battery), which raises ValueError, naming the battery and phase, unless params carries `<phase>_code` equal to the phase's code in that battery's table (a phase the table leaves out must carry none). Nothing else is in it. The graph construction imports the tables where it sets the params and the two runs import the check, so neither old module carries the names. - The check takes the battery's name, not its table. A run that named a table would compute through the module by the rule, so its kernel would have to hash it, and an edit to a phase's list would again re-key every node whose kernel hashes the gate kernels' modules. Only the listed check names the tables. - The rule. test_uk_graph lists the check in _REFUSAL_ONLY_HELPERS: it reads its params and the phase's code and refuses, writing nothing, and both gate kernels' runs call it first. The battery walk reads each phase's code table from the new module. source_hash leaves _KERNEL_CORE_NAMES rather than taking a new reason: only the full gates' run named it, through UK_FULL_GATE_PHASE_CODE, and the rule does not follow a listed check, so no kernel's code names it now, and test_the_core_definitions_the_kernels_name_are_listed holds the table to exactly what the walks name. Its reason moves into the check's entry. Two new tests: each gate kernel's run calls the check on its own battery, in a statement of its own, before it reads anything of its node but its parameters, and evaluates the phase it checked; and no UK kernel's key covers gate_phase_codes, the gate kernels' implementation hashes among them, while graph_evidence and graph_terminal carry none of the moved names. - The spine gates. UKSpineGateKernel.run calls the check before it reads or evaluates anything, and its docstring says it refuses a stale or missing code. test_uk_graph_evidence mirrors the full gates' stale-code test: each spine phase refuses a stale and a missing code before any artifact read, and passes its node's own declared code. The check that each spine gate node carries its own phase's code moves there from test_uk_graph's battery test, beside the full gates' in test_uk_graph_terminal. - Nit 1. uk_terminal_only_code_hash's docstring and the changelog said nine terminal-only modules never re-solve, on a bound spine or in the in-graph build. uk_runtime.diagnostics re-solves the national line, whose problem kernel hashes it. Both now say eight never re-solve on any line and diagnostics re-solves only the national line. Two comments beside the full gate nodes said an edit to one phase's own code re-keys only that phase, or never the preflight and the solve; they now say key chaining still re-keys them where a node upstream hashes the module. - Nit 2. gate_scopes' docstring, calibration_run's comment above its gate_scopes import, the docstring of test_an_edit_to_calibration_run_leaves_the_preflight_and_the_solve and docs/uk-full-build-graph.md now say what the changelog says: the move keeps the runner out of the full gate kernels' own hashes and the preflight's own code, so in the in-graph build an edit to the runner re-keys only the calibrated gates and what follows them, while on a bound spine the checkpoint, whose kernel hashes the runner, still re-keys the preflight and the solve. The doc also says the spine and local scopes and the scoped manifest are defined in gate_scopes and re-exported by calibration_run. - The changelog's gate battery fragment records the module, the check, the new refusal and source_hash leaving the core names, and counts seven refusal-only checks. Checked by simulated edits (a comment line through the module's __file__, the phase-code caches cleared), at the parent and here alike, counted as in-graph build / local line on a bound spine / national line: each of the eight re-keys only uk.full.gates.calibrated where it re-keys anything (1 of 113, 1 of 33, and 1 or 0 of 6); uk_runtime.diagnostics re-keys 1, 1 and 4 of 6 (the national problem, the dense solve, the calibrated population and the gates); calibration_run 1 of 113, 33 of 33 and 6 of 6. A static reach of the 38 terminal-only modules splits them the same way at the parent and here: eight re-solve on no line, one on the national line only, two on a bound spine only, seven in the in-graph build only, three in the in-graph build and on the local line, and seventeen on every line. What moves: node keys only, plus the spine gates' new refusal. No value, declaration or generated file moves, and neither the solve nor the country fingerprint does. - 72 of the 99 UK kernel hashes move: exactly those that hash graph_evidence.py (65), graph_terminal.py (8), gate_scopes.py (5) or calibration_run.py (2). No kernel's set of hashed modules changes, and none hashes gate_phase_codes. The other 27 keep theirs: the four atomic geography kernels, the claim and both identity kernels, the calibration and size kernels, the full target kernels, the geography, property and private rent kernels, and the national targets, problem and readback. No transport kernel hashes these files, and none moves. - Of the phase codes only terminal_code moves, since it hashes calibration_run.py; assembled_code, transferred_code and preflight_code keep their values. - So every node of every UK graph re-keys: the hermetic and gated spine (79 and 81), the in-graph build (113), the local line on a bound spine (33) and the national line (6). The bound spine kernel hashes gate_scopes.py and calibration_run.py, whose texts changed, so both lines on a bound spine re-key from their root. - After this, an edit to a phase's list re-keys that phase's node and what follows it, and nothing else. One module added to the transferred phase's list, simulated: at the parent, in graph_evidence.py, 65 kernel hashes move and 79 of 79, 81 of 81, 113 of 113, 32 of 33 and 0 of 6 nodes re-key (hermetic spine, gated spine, in-graph build, local line on a bound spine, national line); here, in gate_phase_codes.py, no kernel hash moves and 0, 1 (the transferred gate), 33 (that gate and every node after it), 0 and 0 re-key. A comment edit to gate_phase_codes.py re-keys nothing. - uk_spine.json and the H2 fixture hold stage contract hashes, and no fixture holds a gate node, so neither regenerates. source_stages.json and the coverage manifest are current. Tests (engine-free; code facts and synthetic values): test_uk_graph, test_uk_graph_evidence, test_uk_graph_terminal, test_uk_target_compile_modules and the shared rule and transport rule tests pass (168 tests), and 15 related files pass (655 tests). Through the engine slot, test_uk_full_graph_admission passes. 15 mutations on a scratch copy are caught: either gate kernel reading its table beside the check, or the full gates reading theirs in its place (the rule finds gate_phase_codes unhashed in both phases' walks); the check writing into its params or returning its code; the spine gates dropping the check, running it after their first artifact read or on the full battery's table, each caught both by the static test and by the spine refusal test; the full gates on the spine battery's table; a gate kernel hashing the module; graph_evidence re-exporting a table; and a phase's list gaining a module its gates do not compute through. Ruff is clean on the 10 edited Python files, and ci_test_plan verify is ok. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María ruled on 9 Nov (late) that the annuity and trust column's name moves into a small module of its own (4a) and that ledger_targets' household-type check is listed refusal-only (4b), so that frs_spine and frs_relationships leave the target kernels' hashes. 08b437a bound both only because of the rule's limits (run A findings CM8 and TGH7), and on a bound spine each cost a national re-solve. This commit does both. It moves keys only. - The column. uk_runtime/annuity_trust_column.py holds ANNUITY_AND_TRUST_INCOME_COLUMN, unchanged, and nothing else. frs_spine, measure_simulation, spi_spine and spi_income import it from there, and frs_spine no longer defines it (its __all__ never listed it). The resolver reads zeros where a frame lacks the column, so the name cannot be listed as unbound: the measures and the national problem hash the small module in place of frs_spine. The frs_spine root, which writes the column and has no node upstream, binds it, as the stage rule asks, and so does hmrc_spi_income_spine, which zeros it on the rows it draws and could chain on the root's key. - The check. test_uk_graph lists ledger_targets._assert_household_type_bindings_declared in _REFUSAL_ONLY_HELPERS, the eighth entry. It compares the contract's ons_household_type bindings with the ONS household type domain and refuses a binding outside it or composition rows that do not cover it once each. It writes only into the dict and set it creates, which the shared writes_nothing counts as local since 8c10ecc and 8f6c369. Its one caller, compile_uk_target_registry, calls it before the loop that compiles the references, and the compilation and the national targets reach it there, through full_targets' two loaders. The compilation's own hash keeps chronicle_epoch alone. The national targets compute through nothing their shared hash leaves out, so their own implementation hash goes. The shared rule module is unchanged. - Tests. test_the_household_type_check_refuses_before_the_targets_compile holds the entry's reason: the check's one caller, its statement before the compile loop, both kernels reaching the caller with no walk meeting frs_relationships or key hashing it, and the kernels that hash it for their own reasons keeping it (the frs_relationships stage and the assembled spine gate's own code). test_every_reader_of_the_annuity_and_trust_column_hashes_its_module checks that the small module holds the name alone, that no other uk_runtime module defines it or imports it from elsewhere, that every stage and kernel whose walk reads it hashes the module, and that the measures and the national problem meet nothing in frs_spine and leave it out of their keys. test_uk_target_compile_modules checks that an edit to frs_spine.py or frs_relationships.py leaves each of the six target and problem kernels' hashes alone, and that an edit to the small module moves the measures' and the national problem's. - Texts. The binding comments in graph_targets, graph_national and graph_kernels, and frs_spine's and measure_simulation's comments on the column. The helper-bindings changelog fragment's target bullets, its refusal-only count and its What moves. test_uk_target_compile_modules and docs/uk-full-build-graph.md stated neither binding, so neither needed a correction. What moves: node keys only. No value, declaration or generated file moves, and neither the solve nor the country fingerprint does. - 68 of the 99 UK kernel hashes move, exactly those that hash a file this commit edits: the 57 spine kernels, which all hash graph_kernels.py (20 of them frs_spine.py too); the six target and problem kernels; the full preflight and calibrated gates and the rotated holdout, which hash graph_targets.py; and the national gates and readback, which hash graph_national.py. Six kernels' hashed modules change: the create kernel and hmrc_spi_income_spine gain annuity_trust_column, the measures and the national problem swap frs_spine for it, and the compilation and the national targets lose frs_relationships, which now only the frs_relationships stage's kernel and the assembled gate's own code hash. The other 31 keep theirs, the bound spine, the dense population, calibration and size kernels, the export kernels and the four atomic geography kernels among them. No transport kernel moves. - Every node of the hermetic and gated spines (79 and 81) and of the in-graph build (113) re-keys, since the root's module changed. On a bound spine the local line re-keys 26 of its 33 nodes (the checkpoint, the sample, normalize, expand and the three geography support nodes keep theirs) and the national line 5 of 6 (all but the checkpoint). - After this, by simulated edits (a comment line through each module's __file__), counted as in-graph build / local line on a bound spine / national line: frs_spine.py re-keys 113, 0 and 0 nodes (113, 12 and 4 at the parent) and frs_relationships.py 108, 0 and 0 (108, 14 and 5), so on a bound spine neither line re-solves for either; annuity_trust_column.py re-keys 113, 12 and 4. - uk_spine.json and the H2 fixture hold stage contract hashes only, so neither regenerates. source_stages.json and the coverage manifest are current (both --check pass). Tests (engine-free; code facts and synthetic values): test_uk_graph and test_uk_target_compile_modules pass (119 tests), and with them 20 related files, the shared rule and transport rule tests among them, pass (838 tests). Through the engine slot, engine/uk/test_uk_full_target_graph passes (8 tests). Eleven mutations on a scratch copy are caught, each failing exactly the tests expected: - the column defined in frs_spine again and read from there by the measures (the kernel rule finds frs_spine's name outside both keys); - the measures importing it through frs_spine, which only re-exports it (the rule follows it to the small module; the reader test refuses it); - the root, the SPI income stage, the measures or the national problem no longer hashing the small module; - the household-type check unlisted while frs_relationships stays unbound (the kernel rule finds frs_relationships unhashed in the compilation's and the national targets' walks); - the check writing into what it is handed, or running after the compile loop; - the compilation hashing frs_relationships again; - the national targets reading the domain themselves. Ruff is clean on the 10 edited Python files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…es test (#931) The identity tool's count hold rests on _SUPPORT_SPLIT_CORRECTED_ROWS = 3: the split, the incidence clone and the residential split each correct their total once. test_the_hold_counts_every_weight_change_after_the_split reads that premise from the stages' code. Run A's re-review of a0933a5 found it still passed a second correction in the incidence clone made through an import alias, through a module-level alias, or by a wrapper that corrects twice, since it counted references only inside function bodies and a wrapper call as one correction whatever the wrapper did. This commit takes that nit, the third. It edits the test only. - Aliases. A name one of the three stage modules binds outside every function, at module level or in a class body, to one of spi_support's two helpers, a wrapper or another alias is an alias, and a reference to it counts as a reference to a wrapper does. That covers a from-import of one (its asname is the alias) and an assignment whose value refers to one, a functools.partial among them. Wrappers and aliases are found to one fixed point over the three modules. A function-local import under another name now counts once, where it names the helper, as a local alias assignment already did. At the tip the scan finds no alias but the helpers' own from-imports. - Wrappers. The test pins the wrapper set to what the code has, cgt_support._exact_support_weights and cgt_residential_split._exact_residential_weights, and each to one reference to each of the two helpers: each tries _exact_total_correction on a household its split left whole and falls back to _importance_weights_with_exact_total. So a second correction inside a wrapper fails, and so does a new _exact... wrapper. Wrappers are keyed by module and qualified name. - Text. The comment over the count says which forms it catches, and that it counts references, not calls: a loop that corrects twice, or a local alias called twice, counts once. No docstring or changelog fragment said the test catches aliases in general; only that comment did. Correction to a0933a5's message, for the PR body: its identity tool bullet said a second correction through an alias or a partial fails the premises test. Then only a local alias or partial inside the correcting function did; an import alias, a module-level alias or partial and a wrapper that corrects twice passed. Now all of them fail. What moves: nothing. No kernel hash, node key, value, declaration or generated file moves; source_stages.json and the coverage manifest are current (both --check pass at this commit). Tests (engine-free; code facts only): the touched file passes (101 tests). Run A's harness and mutations, copied to scratch and applied to a git archive snapshot of f274857 with this test laid over it: the unmutated snapshot passes, Pimport, Pmodule and Pwrapper2 now fail (all three pass at f274857), and the control Pbare still fails. Five more mutations fail here and pass at f274857: a top-level functools.partial alias, an alias of a module-level alias, a function-local import under another name, a class-body alias called from the clone transform's __call__, and a new _exact... wrapper that corrects twice. Run A's seven fix-round mutations still fail (an attribute call, a bare name in __call__, another module's wrapper, a local alias, a nested function, a lambda and a local partial). A loop running the clone's one correction twice passes, as the comment says. Ruff is clean on the edited file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…commits (#931) Three reviews of b289e8f, f274857 and c7e0523 found no must, two shoulds and fifteen nits, some of them the same finding twice. This commit takes the shoulds and the nits that need no ruling. The review's corrections to the earlier messages are below, for the PR body. It moves keys only: every edit to a hashed file is to a comment or docstring. Citations (the shoulds): - The texts for the 9 Nov (late) rulings cited them as microcosm#931 ruling 3, 3a, 3b and 4. In the PR body those numbers belong to the 9 Nov (evening) rulings, and the 9 Nov (late) block is unnumbered, so a reader following one would land on an unrelated ruling. The 24 sites now cite "microcosm#931, ruled 9 Nov (late)", as the changelog does: in gate_phase_codes, graph_evidence, graph_terminal, graph_kernels, graph_national, graph_targets, frs_spine, annuity_trust_column and four test files. "Ruling 6" still means the evening ruling. Phase codes: - The cache hit. Texts said the spine gates' new refusal means a declaration replayed from another process cannot key a spine gate on stale phase code. The check runs only when the kernel runs, and the executor serves a record stored under a node's key without calling run. UKSpineGateKernel's docstring, gate_phase_codes' module docstring, require_current_phase_code's docstring, the spine stale-code test's docstring and the changelog now say what holds: a node with a stale or missing <phase>_code is refused whenever its kernel runs, so no gate evaluates its phase under a key that names other phase code, and the check does not guard a cache hit. - test_each_gate_kernel_checks_its_phase_code_first also holds that no statement before the check returns or yields, outside the functions it defines, so every path through the run reaches the check or refuses first. It also holds that the run binds phase exactly once, from its node's parameter, before the check, so the phase evaluated is the phase checked. A return in a parameter branch before the check, and phase rebound after it, passed before and fail now, in either battery. - A new test, test_the_phase_code_check_refuses_what_its_tables_leave_out, calls the check directly. A phase its battery's table leaves out passes with no <phase>_code and is refused with one, whatever its value, and a battery with no table is refused. No gate node reaches either branch today. - test_no_uk_graph_kernel_hashes_the_phase_codes' docstring said an edit to what one phase computes through re-keys no node whose kernel hashes graph_evidence or graph_terminal. It now says what the test holds: a list edit in gate_phase_codes moves that phase's parameter, which re-keys that phase's node and every node after it, and moves no kernel's hash. - The changelog's phase-codes bullet now gives what a list edit in graph_evidence or graph_terminal used to re-key, chaining included. For a spine phase's list that was every node of the spine and of the in-graph build, and all but the checkpoint of the local line on a bound spine. For a full phase's list it was the preflight, the dense solve and every node after them. Simulated edits to the two files at c7e0523, where they have the same hashers as at 4df4065, re-key 113 of 113, 32 of 33 and 0 of 6 nodes for the first and 9, 9 and 0 for the second (in-graph build, local line on a bound spine, national line). The What moves sentences for the phase codes and for the column now add "and by key chaining every node after one". Target keys: - frs_spine's PENTYPE comment said the nodes that read the column's name key on it without keying on frs_spine. annuity_trust_column's docstring said the same of the target and problem kernels. Both now name the two kernels that do: the dense measures and the national problem. The create kernel and the SPI income stage read the name and hash both modules, and the other four target and problem kernels neither read nor hash it. The docstring also says an edit to frs_spine would otherwise re-solve both lines on a bound spine, not only the national one. - test_every_reader_of_the_annuity_and_trust_column_hashes_its_module now pins frs_spine as the household-type test pins frs_relationships: every key that covers frs_spine is the create kernel's or a spine stage's. Neither runs on the local or the national line on a bound spine, so the test now holds the changelog's claim that an edit to frs_spine re-keys no node of either line there. It also refuses a uk_runtime module reading the column's name as an attribute of any module but annuity_trust_column. measure_simulation reading it through frs_spine, which imports it, passed every target test before. - The changelog's column bullet names both checks, and its gate-battery bullet names the new phase-code checks. Premises test: - The alias scan also reads spi_support's bindings outside every function, since spi_support defines the helpers. So an alias spi_support binds, plain or as a functools.partial, which the clone imports and calls for a second correction, now counts. spi_support's functions are not counted as correction sites, and at the tip the scan finds nothing there. - The comment says the limits. Wrappers are looked for only in the three stage modules. A correction made through another module's function that corrects a total itself, such as spi_band_donors' _exact_incumbent_total or the spi_support functions that call the fallback, is not counted. Nor is one made through a helper looked up by a name built at run time. - The review's other fix was a fixed point over every uk_runtime module. It is not taken, and the test states the limit instead, which the review offered as the alternative. By name, that fixed point takes in 18 functions across 7 modules at the tip, the stage transforms' __call__ methods among them. The count would then have to tell correction sites from their callers, and keying it by module would need import resolution in the test. Not taken, for María: the check does not guard a cache hit. Closing that would need a check before keying, for example the drivers comparing each gate node's declared <phase>_code with gate_phase_codes before run_graph. No production path replays a stored declaration today, since full_build_cli and spine_build build their graphs in-process. Corrections to the earlier messages, for the PR body: - b289e8f, the first What moves bullet. Its per-file counts (65, 8, 5 and 2) overlap. Split disjointly: 65 kernels hash graph_evidence.py; 5 more hash graph_terminal.py (the four export kernels and the rotated holdout); 2 more hash gate_scopes.py (the bound spine and the national gates); none more hash calibration_run.py. That makes 72. - c7e0523, the correction to a0933a5. It should read: an import alias, a module-level alias or partial, and a wrapper that refers to a helper a second time all passed then; all three fail now, as does an alias spi_support binds (this commit). A correction repeated through one reference, in a loop, still passes, as the test's comment says. So does one made through another module's own corrector. What moves: node keys only. No value, declaration or generated file moves, and neither the solve nor the country fingerprint does. - 80 of the 99 UK kernel hashes move: exactly those that hash a file this commit edits. Split disjointly: - 57 hash graph_kernels.py: the spine stage, create, claim and identity kernels, and the spine gates; - 10 more hash graph_evidence.py: the preflight and calibrated gates, the sample, normalize, expand, pool and locations kernels, the geography mapping and gate, and the certification readiness; - 2 more hash annuity_trust_column.py: the measures and the national problem; - 6 more hash graph_targets.py: the compilation, the selection, the problem, the rotated holdout, and the national targets and gates; - 1 more hashes graph_national.py: the national readback; - 4 more hash graph_terminal.py: the export kernels. - frs_spine.py's 20 hashers are all spine kernels counted above. No kernel's set of hashed modules changes, and none hashes gate_phase_codes. The other 19 keep theirs: the bound spine, the dense solve, the calibrated, selected and full identity kernels, the size checkpoint import, draw, refit and search, the geography, property and private rent kernels, and the four atomic geography kernels. No transport module imports a uk_runtime module, and none of the 31 transport and atomic geography kernels the key probe registers moves. No phase code moves. - Node keys: every node of the hermetic and gated spines (79 and 81) and of the in-graph build (113) re-keys. On a bound spine the local line re-keys 32 of 33 nodes and the national line 5 of 6; both keep only the checkpoint, whose kernel hashes none of the seven files. b289e8f already re-keys every node of every UK graph against 4df4065, so the push re-keys nothing more. - source_stages.json and the coverage manifest are current; both --check pass at this commit. uk_spine.json and the H2 fixture hold stage contract hashes, so neither regenerates. Tests (engine-free; code facts and synthetic values): the five touched files and the shared rule and transport rule tests pass (286 tests). Ruff is clean on the 13 edited Python files. On a scratch copy of c7e0523 with this commit laid over it, 17 mutations each do what is expected: - the unmutated copy passes; - run A's Pimport, Pmodule and Pwrapper2 fail the premises test, and so does the control Pbare; - an alias, or a top-level functools.partial, that spi_support binds and the clone calls fails it too; both pass at c7e0523; - the stated limits pass, as the comment says: a correction through spi_band_donors' _exact_incumbent_total, and one through a helper looked up by a built name; - a return before the check, or phase rebound after it, fails the static test alone, in the spine gates and in the full gates; - the check accepting any code on a phase outside its table, or an unknown battery getting past it, fails the new test alone; - measure_simulation reading the column through frs_spine or spi_spine, or the bound spine hashing frs_spine, fails the reader test alone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…emises test looks (#931) The re-review of e5d0acc left two nits, both in tests. This commit takes them. It moves nothing: no kernel hash, node key, value, declaration or generated file. - test_every_reader_of_the_annuity_and_trust_column_hashes_its_module looked an import of the name up by the name it binds, since facts.imports is keyed that way. So an aliased import through frs_spine or spi_spine, both of which import the name, passed it, although its docstring and the changelog's column bullet say every module reads the name from annuity_trust_column. The test now checks every from-import of the name, whatever name it binds, resolving a relative import as module_definitions does, which also catches two imports that bind one name. Keys were safe either way: the kernel rule follows frs_spine's import of the name to annuity_trust_column, and at the tip all four readers import it from there. - The premises test's comment said wrappers and aliases are found to a fixed point over four modules, and then that wrappers are looked for only in the three stages' modules, which is what the code does. It now says aliases are found over the four modules and wrappers over the three, together to a fixed point, and so does the comment over the counted names. Tests (engine-free; code facts only): test_uk_graph and test_uk_identity_stability_receipts pass (154 tests), and ruff is clean on both. On a scratch copy of e5d0acc with this commit laid over it, the re-review's three mutations now fail the reader test alone: the name imported under another name through frs_spine, or through spi_spine, in place of measure_simulation's own import, and the same import through frs_spine added beside it. The first two ran with the kernel, stage and name rules and both target-compile edit tests, the third with the whole of test_uk_graph and test_uk_target_compile_modules. The unmutated copy passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…es share a rank (#931) Step 6 of docs/uk-isa-subscriptions-1415.md has said since 960cd0a that the Lifetime ISA payment rank blends the balance and income ranks with a keyed uniform. The code never did: since the stage was added (bf0abdc), draw_lifetime_isa_payments has blended only the weighted rank of the person's Lifetime ISA balance, over every holder aged 18 to 47 rather than within income bands, with a keyed uniform at the same balance_rank_weight as the subscription amounts. Step 6 now says so (María's ruling 4 of 10 Nov). Step 3 also says, in the module docstring's words, that holders tied on a balance share its rank, so the keyed uniform alone orders them: since 0c850b1, _weighted_rank gives a tie group one mid-rank, and both draws blend it with a keyed uniform. Ruling 1 of 10 Nov has stage 53's declaration state ties in a later change. Text only: no code, declaration, generated file or key moves. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rors at thin ages (#931) The UK Child Benefit take-up stage gate (uk_stage_child_benefit_take_up) held the claimed share of eligible children at each measured single year of age within a fixed maximum_age_claim_rate_deviation, 0.05, of HMRC's published rate for it. A measured age is one with at least minimum_age_child_rows child rows whose solved rate was not clipped. Each measured age is now held within the larger of that deviation and z standard errors of a share drawn at the age's published rate p over its n child rows, z * sqrt(p (1 - p) / n) (María's ruling of 10 Nov). p and n are the age's published_rate and eligible_child_rows in the receipt the gate already reads. z is the new declared parameter maximum_age_claim_rate_standard_errors, 3 in gates.json. The stage-health binding now routes it, since the battery refuses a declared parameter its binding does not route. Three standard errors exceed 0.05 only below 3,600 p (1 - p) child rows, so an age with more rows keeps the fixed deviation and only thin ages loosen. The band treats an age's child rows as independent draws at p, an approximation as ruled: in the stage, siblings share their family's draw, reporters always claim and the realised rate is weighted. The overall claim-rate check and the opt-out check do not change. Why: group (a)'s keyed re-draws move each age's realised claim rate by sampling noise. At an age with few child rows that noise can exceed the fixed deviation, and on a full K = 1 spine at (a)'s head it did, at one thin age, while the pre-(a) head and a copy-ids-only head passed. A private read of the stage evidence found draw noise, not a shift. The draws are keyed, so the failure is deterministic and would repeat on every rebuild, including the national build. The gate now allows for each age's own sampling noise. What moves: the gate's verdicts at thin ages. An age whose realised rate is more than the fixed deviation from its published rate, but within its three standard errors, now passes. A failing age's line names the tolerance it was held to, "the larger of 0.05 and 3 standard errors of a share at the published rate over that age's child rows", and the gate's details count the measured ages held above the fixed deviation (ages_tolerance_above_floor). At a published rate of 0 or 1 the binomial term is 0, so the floor holds. No stage output or generated file moves: the gate reads the receipt and draws nothing. - Of the 99 UK kernel hashes 14 move, and no phase code: the spine gates' (stage_health, battery_bindings and the solve fingerprint, which gates.json is part of); uc_deduction_attributes' and uc_take_up's (stage_health); the full build's target compilation, selection, measures, problem and rotated holdout (the solve fingerprint), its preflight and calibrated gates (battery_bindings and the solve or country fingerprint) and its certification readiness (microcosm.data.contract, which the re-pin edits); and the national targets, problem and gates (the country fingerprint; the gates also hash the gate registry's modules). - Node keys move from the assembled spine gate on: 64 of the gated spine's 81 nodes. On a bound spine the full build re-keys from its target compilation on, so it re-solves, and the national line re-keys all but the bound checkpoint. - The contract re-pins the gate battery's policy, gates-manifest and spec-fingerprint digests and the release certification's spine part, and test_contract mirrors the battery pins. The calibration seam, release cut and dense line digests do not move. Tests (engine-free; invented values only): a thin age passes inside its three standard errors and fails outside them with the new line; a large age keeps the fixed deviation; a published rate of 0 or 1 keeps it even at the fewest rows the gate measures; the fixture's ages carry a weighted child mass far from their row counts, so a gate that took that mass for n would fail on value; the gate refuses a missing multiple with a KeyError and a null, zero, negative, NaN or infinite one with a ValueError; and the committed declaration's parameters equal the test's. test_uk_stage_health, the data shard's test_contract, test_gate_battery_contract_pins, test_uk_battery_bindings and test_uk_release_certification pass on the final tree (565 tests). ruff check and ruff format --check are clean on the five touched Python files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
E8's band-donor check (_e8_band_donors in tools/verify_uk_identity_stability.py) now scopes the artifact to stage 17 with _frame_as_stage_saw, the helper E5 uses, before it reads the stored layer or reruns the stage (María's go-ahead of 10 Nov). Every row minted after stage 17 is dropped with its people and benefit units: the high-value home copies of stage 26 and the CGT support copies, incidence twins and residential arms of stages 42, 43 and 46. Each one's mass folds back onto the row it came from: arms onto their sources, twins onto their originals by pair sum, then support copies and high-value copies onto their roots. Every household left carries the weight stage 17 left, to rounding, and the scope also reverses the salary-sacrifice and regional-home-value rewrites. The stored seats, carriers and band weights are read on that frame, so a later copy of a donor is no longer read as a donor of its own, and each funding stratum is scaled back from there to the weights the stage read. If the scope leaves in place a household whose id holds an ordinal in a copy-id field above spi_support_channel, the check refuses, by count; the fields come from the copy-id table's own order. e8_identity_receipt now hands the check the artifact instead of cgt_frame, whose salary-sacrifice conversion is already reversed: a second reversal finds every converted record's salary sacrifice at zero and refuses. Why: the check folded only the CGT layers, so the high-value copies stayed in the rerun as households the stage saw, and a donor's copies, which carry its donor flag, were read as donors too. Under the copy-id table's fixed radices each later layer mints in a field of its own above spi_support_channel, so the later copies' ids sort above those of the donors the rerun mints at that field's ordinal 2. Appended after the copies, the new donors broke the Frame's ascending ids, and the rerun (--check e8 with the SPI tape) raised instead of running. Under the old max-id rule the rerun ran, but on the wrong population. What moves: E8's verdict, and with it the band-donor block of the tool's receipt, which now reads each donor once; no other receipt moves. On a spine with high-value copies the rerun completes where it raised before. No package module references the tool and no kernel binds it, so no stage output, node key, kernel hash or generated file moves. The identity tool's changelog fragment gains a bullet for E8, and its What moves bullet now names E8's receipt. The verdict on a real spine is unverified until the next --check e8 run. Tests (engine-free; invented values only), in test_uk_spi_band_donors.py: stage 17's output, with and without geographic clones, goes through the real high-value split and regional home values, then a CGT support split, incidence twins at uneven anchor shares and residential arms replayed through the copy-id table, and a salary-sacrifice conversion. The scope gives back stage 17's rows and weights, the check scales each funding stratum back by the stage's own factor, and the rerun reproduces the stored seats, carriers and band weights with the artifact's people in either order. Mass moved within a donor's family passes, and reweighting the whole family is reported as unequal band weights. A copy that has lost its lineage columns is refused by count. The E8 entry, its other checks stubbed out, hands the band-donor check the artifact. Against the tool at aed673d every new test fails, most on the sorted-ids refusal. The touched test file passes; ruff check and ruff format --check are clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…by sorting, and add a shared clone expansion core (#931) The shared fold in microcosm.build.source_lineage is now fold_replicas, which returns a ReplicaFold with one value per record in each of five fields: kept, the rows the fit trains on; weights, on each kept row its variant's selected weights summed exactly (math.fsum), every other weight as given; representative, the row a selected record's variant trains on, -1 off the selection; varied, whether a selected record's key holds more than one variant; and key_group, a code of its replica key, -1 off the selection. Its repr prints none of them. Each variant trains on its selected record with the least order, ties going to the earlier row, or without order on its first selected row, as before. The early returns are decided on the key. When no two selected records share a key the fold is the identity. When keys repeat but no two selected records share a variant, the selection and weights still come back value for value, and every record of a key with more than one variant is varied. order must hold one numpy integer per record (a timedelta is refused); a malformed one is refused with counts only, at any clone count. collapse_replicas is now a wrapper that returns (kept, weights) and takes the same keywords, order included. A selected record with a null key is now refused: replica_keys never gives one, but the fold used to add such a record to the last key's group, or, with inputs, to the other null-key records that shared its inputs. The UK entry, household_lineage.fold_geographic_clones, returns that fold ordered by the entity's id column, and collapse_geographic_clones keeps its signature and return through it. Within a key the ids differ only by each clone's offset (the copy-id table's geographic_support digits on a clone-first spine), so each variant trains on its lowest selected clone: clone 0 whenever clone 0 is selected and agrees with it, whatever the row order. integer_record_keys in microcosm.build.stochastic_assignment finds a repeated id by comparing neighbours in a sorted copy of the ids instead of building np.unique's array. On integer ids its verdicts, errors and return are unchanged. It now checks the dtype's kind rather than the integer subtype, so timedelta ids, which numpy types as integers and whose NaT never compare equal, are refused as ids that are not integers. The new microcosm.build.clone_expansion is a country-neutral core that turns a frame into K clones of every record, clone-major. It names no country's columns, tables or constants: expand_clones(frame, clones=K, declaration=...) reads every entity, id column, membership column and weighted entity from the frame. The country passes a CloneDeclaration: a CloneIdRule (bound, mint, require_below), a clone-index column for every entity, the weight kinds it accepts, the mass records' reason, and optional source-id columns. A declaration compares and hashes by identity. Clone 0 is the frame's own rows. Clone k's ids are minted by the rule in the ids' own dtype, each person's memberships point at its units' clone k, and every other column is copied unchanged, so ids and memberships keep their dtype at every K. Every row gets its clone ordinal and, where declared, its clone-0 id. Every weighted entity's weights become numpy.tile(w / K, K), of its own kind. The strata repeat, the metadata is kept, and the mass log gains one record per weighted entity, in the frame's entity order, with factor 1.0 and the declared reason, each entity's totals agreeing within a relative 1e-9. At K = 1 the rows, ids and weights are the frame's own. The core refuses: - a clone count that is not an integer in [1, bound]; - a declaration that names an entity the frame lacks, or gives one of its entities no clone-index column; - a frame that already carries a declared column or a link table; - weights, on any entity, of a kind it does not accept; - ids or memberships that are not numpy integers, and memberships in another dtype than their unit's ids; - ids the rule's require_below refuses; - a mint that does not return one integer per id in the ids' own dtype, keep the ids' order or put each clone above the one before. Its refusals name entities, columns, dtypes, weight kinds and declared constants only, and the rule's own refusals must keep to counts and constants. Nothing calls it yet. The geographic expand transform (G1) comes next and will pass the UK's declarations. Its mint will be a named call to copy_id_table inside the UK module, so that the binding rule binds the table to G1's key. The G1 stand-in, clone_first_frame in test_support, does not use the core, so it stays an independent oracle. For Max's review: - The core returns a plain Frame with no lineage. The expand kernel recovers lineage from the ids at the declared radix. - Its per-clone columns are the clone index and an optional source id. A country's labels, such as a support channel, derive from the index. - Memberships follow their unit's minted ids, in the units' id dtype. - require_below reads each entity's id column, since memberships are unit ids. - It splits every weighted entity, each with its own mass record, as decision 10's analysis proposed. A country that weights households only, as the UK does, can refuse other weighted entities in its own wrapper. - It copies no link table and gives every record the same number of clones, until a caller needs more. Why: O8b. A variant trained on whichever of its replicas came first in row order, so the clone a fit trained on depended on how the stages before it ordered their rows. Ordered by id, each variant trains on its lowest-id record whatever the row order. The representative map, variety and key codes are groundwork for the next commits: UC capital's donor count, the fold-variety receipts and the twin check (O9a, O14(i)). O2: as ruled on 10 Nov, the sorted check gives np.unique's verdict without building the unique array, and is faster on large id arrays. Decision 10: María ruled on 10 Nov that the expansion core is shared now, rather than UK-local in G1 until a later extraction, so that another country's clone step can use it. Max reviews it in this PR, beside the shared fold. What moves: node keys only. No value moves at one clone or on any spine. Every spine is one clone today, so each fit's replica keys are unique, the fold returns the selection and weights it was given, value for value, and integer_record_keys gives np.unique's verdict on every integer id array. Only frames whose geographic clones are out of clone order fold differently, and today only tests build them: a variant now trains on its lowest-id clone instead of its first row. These re-key: - every UK spine node, since the spine's root binds stochastic_assignment, source_lineage and household_lineage and keys chain; with them the spine gate phases (the transferred phase's own code also hashes stochastic_assignment) and, in the in-graph build, every node after the spine; - on a bound spine, whose root hashes none of the three, the full build's uk.full.identity, uk.full.property.value and uk.full.private_rent, which hash source_lineage and household_lineage, and every node keyed on them; - the national line's gates, whose registry modules include stochastic_assignment. The spine checkpoint's gate digests do not move. clone_expansion is imported only by its tests, so it binds no node until the geographic expand stage is bound. The derived binding tables are unchanged, since the new names live in modules that are already bound wherever they are reached. No generated file or gates.json changes, since uk_spine.json and the H2 fixture hold stage contract hashes, not code hashes. No transport kernel and no US, NZ or BE node hashes an edited module, so their node keys stay; only the US PUF-support builder's whole-tree resume fingerprint moves, as it does on any edit to packaged source. The comments at the six fold call sites still say the first row; they change with their calls in later commits of this PR. The fits fragment 931-uk-fits-fold-geographic-clones and docs/uk-atomic-household-lineage.md now name fold_replicas as the fold and collapse_replicas as its rows and weights. Tests (engine-free; invented values only): - test_source_lineage.py: - the least order wins under all 720 row orders of a three-clone frame, with and without agreeing inputs; - each variant trains on its own least-order replica, ties going to the earlier row; - a fold without replicas is the identity, with an identity representative map; - representative, varied and key_group match hand-built maps and, on 240 random frames, a record-by-record statement of the contract; - without order the earlier fixtures keep their rows and weights, as the row positions taken as the order do; - a two-clone frame whose clones all differ folds nothing and varies everywhere; - malformed orders, a timedelta among them, and null keys are refused, and the order refusal prints no order value. - test_uk_geographic_clone_collapse.py: with every table in reverse row order, each record keeps clone 0 for households, people and benefit units, and each representative is its key's lowest id. A differing clone trains on its own row, and only its record varies. The scan that holds every folding stage to a live lineage now finds a stage through either UK entry, leaving out the module that defines them. - test_stochastic_assignment.py: the sorted check gives np.unique's verdict on 403 arrays (random ids in five integer dtypes, the int64 extremes, a repeated maximum and an empty array), and 2-D and timedelta ids are refused. - test_clone_expansion.py, which imports no country module, uses neutral declarations at K = 1, 2, 3 and the bound. It checks: - ids, memberships and copied columns; - weights bit for bit against numpy.tile on every weighted entity, each with its own mass record; - the K-free mass record; - the K = 1 identity; - strata and metadata; - the frame handed in left unchanged; - int32 ids and memberships staying int32 at K = 1 and 3; - US-shaped clones, weighted on households and people and labelled from their clone index, passing outer_stage_runtime.assert_clone_expansion; - a declaration hashing by identity; - every refusal, the dtype ones included, with no id or weight in its message. - test_uk_clone_expansion_core.py uses the UK's declarations: copy_id_table's geographic_support mint and require_below, the ladder's clone-index columns, source_household_id and design weights. It checks: - the core equals clone_first_frame bit for bit at K = 1, 2, 3, 7 and 15; - every id decodes to its clone and its clone-0 row; - validate_uk_national_frame passes; - the mass record is the same at every K; - Python strings stay Python strings; - the UK refusals print no id. Mutation checks: - a wrapper that drops the order fails the reverse-order test; - an early return decided on the key and the inputs together fails the all-differ test; - a w * (1 / K) split or an off-by-one clone ordinal fails the UK oracle; - a mint checked by kind alone, or no membership dtype check, fails the dtype tests. These five files, the six fold callers' test files and test_derived_binding_rule.py pass: 370 tests, and one existing skip that needs the licensed SPI donor staged. The 16 binding-rule tests selected from test_uk_graph.py pass. tools/ci_test_plan.py verify is ok, and ruff check and ruff format --check are clean on the nine Python files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…stages ready for clones (#931) The geographic clone index columns' names, ladder_clone_index_column and ARTIFACT_CLONE_INDEX_COLUMN, and the clone count, geographic_clone_count, move unchanged to a new leaf module, uk_runtime/geographic_clones.py, which imports nothing first-party. rowwise_dataset and cgt_support re-export them, so every importer keeps working; household_lineage, salary_sacrifice, the new modules and the full build's population and export modules import them from the leaf. The leaf joins graph_kernels' counting modules, so every stage that counts by lineage hashes it. The CGT support split, which reads K through the count, binds it as a helper. The full build's kernels that read the names hash it beside rowwise_dataset: uk.full.expand and the other population kernels, uk.full.identity, uk.full.property.value, uk.full.private_rent and both export descriptions. The derived binding tables follow the move. The stage table's entry for the clone-index name retires, since every stage now reaches it through a hashed module. The kernel table keeps it at the leaf only for the readers that do not hash it, and the check of that entry's premise, that no spine node writes the clone index columns, moves beside it. geographic_expand.py adds G1, the step that gives every FRS household K geographic clones: expand_uk_geographic_clones(frame, clones=K) and its stage transform, UKGeographicExpandStageTransform(stage, clones). As María ruled on 10 Nov (decision 10), it calls the shared core, clone_expansion.expand_clones, with the UK's declaration, UK_GEOGRAPHIC_CLONES: - a CloneIdRule bounded by GEOGRAPHIC_CLONE_LIMIT, the geographic_support field's bound of 100, whose mint and require_below are named functions of the module that call copy_id_table on that field (mint_geographic_clone_ids, require_room_for_geographic_clones), so the derived binding rule binds the table to the key of the stage that runs G1; - a clone index on the person, benefit unit and household tables; - design weights only, and a mass record reason that names no K; - source_household_id, each household's clone-0 id. Clone k takes id + k x 10^9 on the five id columns and w / K of its household's design weight. The UK wrapper makes the refusals the core does not: - weights on any entity but the household, since the core splits every weighted entity and the UK weights households only; - a frame already carrying a clone index, a source column or a UK_COPY_LINEAGE copy column; - id or membership columns that are not int64, the dtype the copy-id table mints, at every K, so one clone takes the same frames as more; - a clone count outside 1 to 100; - a time period that is not a year. Before calling the core it writes source_year (the frame's year, int64) and source_household_key ("{source_year}:{source_household_id}", the values stage 16 writes, as Python strings in an object column, the dtype the legacy plan gives them) on the base household table, and it validates the frame as a UK national frame before and after. Its evidence gives K, the copy-id field and radix, the distinct source households rounded through receipt_counts, and the weight split and kind, with no row count, id or weight. Every refusal gives counts, names and declared constants only. geographic_identity.py adds G1b: key_uk_households_by_clone_path(frame) and UKGeographicIdentityStageTransform(stage). It writes geography_household_key as the dense line's uk.full.identity does: household_draw_key over the source frs, the FRS vintage frs_release.json pins for the household's source_year, its source_household_id and its copy path, read through source_lineage.copy_paths on UK_COPY_LINEAGE: () for clone 0 and (("geographic_support", k),) for clone k. The release maps its survey year and its base year, the frame's time period, to its vintage. The key is in the graph's string dtype, StringDtype("python"). An unmapped source year and repeated keys are refused by count, and no message prints a key. Its evidence gives the distinct source households rounded through receipt_counts, that the keys are unique, the source, the vintages and the columns it reads. Neither step is bound to a stage yet: no declaration, op kind, graph node or kernel names them, and no build module imports them. The next commit declares both and wires them at K = 1. Salary sacrifice draws each SPI-channel adult's bonus within its own geographic clone (O8a). draw_spi_bonuses takes clone_indices, which the stage reads as each person's household_clone_index; a frame without the column is one clone. The pay cells and the donor-source floor still pool every clone's donors, the floor counted in distinct source households. Each adult draws its bonus share from its own clone's paid donors in its cell, in id order, on the same person-keyed uniform, and a cell with no donor in the adult's clone is refused without a count. Where the clones agree on the donors' pay, clone 0 draws on the one-clone pool at w / K, up to the 1/K split's rounding, which can move a pooled cell edge where a quantile falls exactly on a cumulative weight share. UC capital counts a cell's donors in survey records (decision 6, María's ruling of 10 Nov). The stage folds its donors with fold_geographic_clones, and its widening test counts the distinct ReplicaFold.key_group values among a cell's donor rows instead of the rows. A unit's geographic clones count once, whether they fold into one row or donate apart as variants; any other copy, such as a high-value support copy, counts as a record of its own. Its comments and docstring now say a variant donates from its lowest-id clone. The fits' changelog fragment and docs/uk-atomic-household-lineage.md say so too. test_uk_clone_expansion_core.py now takes G1's declaration instead of a copy of its own and keeps clone_first_frame as its independent oracle, and the core's changelog fragment names G1 as its first caller. Why: the next commit should only declare and wire G1 and G1b, so both land here, held to the clone-first stand-in and to the dense identity kernel before any build runs them. The expansion core is shared (decision 10), so G1 is the UK's declaration and the refusals around it, not a UK copy of the core. The leaf lets the stages, G1 and G1b read the clone-index names, and stage 42 the count, without binding the dense line's rowwise_dataset into the spine, and an edit to it re-keys every stage and dense kernel that reads them. With clones ahead of stage 48, a pooled draw would hand an adult another clone's donor, so clone 0 would not draw as a one-clone spine does. Counted in rows, clones that differ would grow a UC capital cell's donor count with K and stop its widening sooner. The nesting test (O10) checks, stage by stage and before G1 is wired, what of a clone-first spine nests on the one-clone spine exactly and what only to rounding. What moves: node keys only. No value moves at one clone or on any spine: every spine is one clone today, and both stages give the parent commit's output bit for bit on invented frames, with or without a one-clone index. The bonus draw's declared text and UC capital's notes change with the declarations. These re-key: - every UK spine node. Every spine kernel (the root's CREATE, each stage, the identity and claim kernels and the spine gates) hashes graph_kernels, which this commit edits. The root and every other stage that counts by lineage also hash household_lineage, whose import moved, and now geographic_clones, which joins the counting modules. The CGT support split (stage 42) binds geographic_clones as a helper. Stages 26 and 42 hash cgt_support. Stage 36, and the six stages that bind it as a helper (34, 35 and 37 to 40), hash uc_capital_coherence. Stage 48 hashes salary_sacrifice. Keys chain, so in the in-graph build every node after the spine re-keys too; - on a bound spine, whose checkpoint kernel hashes none of these modules, the whole dense line after the checkpoint. Its population kernels (sample, normalize, expand, locations, geography mapping, pool and geography gate), uk.full.identity, uk.full.property.value, uk.full.private_rent and both export descriptions hash rowwise_dataset and graph_population, whose bytes change, and now geographic_clones; the identity, property value and private rent kernels hash household_lineage too; - on a bound spine, the national line from its problem on. The national problem and the calibration nodes hash graph_population, which this commit edits to bind geographic_clones into the population kernels' keys, and the national gates' registry modules include uc_capital_coherence. The national targets keep their key; - the full gates' terminal phase, whose own code hashes uc_capital_coherence and graph_population (gate_phase_codes.uk_terminal_only_code_hash). The spine gates' phase codes do not move. No node hashes geographic_expand, geographic_identity or the shared clone_expansion core until the next commit binds the two stages. From G1's transform, the derived rule already reaches the core and the copy-id table's mint, require_below, radix and ordinal_bound, through G1's named mint and require_below. In the binding tables: - the stage table drops rowwise_dataset's ladder_clone_index_column, which every spine node now binds through the root; - the kernel table lists the name at geographic_clones only for the readers that do not hash it: the calibrated gates' terminal phase, the dense measures, the national problem and the national gates. uk.full.expand@1 now hashes it; - stage 42's own hash gains geographic_clones; - the copy-id table's readers gain geographic_expand. No generated file, declaration or gates.json changes, since uk_spine.json and the H2 fixture hold stage contract hashes, not code hashes. No transport kernel and no US, NZ or BE node hashes an edited module, so their node keys stay. Only the US PUF-support builder's whole-tree resume fingerprint moves, as it does on any edit to packaged source. Tests (engine-free; invented values only): - test_uk_geographic_expand.py: G1 equals clone_first_frame bit for bit at K = 1, 2, 3, 7 and 15; one clone adds no row and only the declared columns, and keeps the mass log it is handed with one record more; the clone-major ids, indices, sources and weights at K = 3; the source key's dtype under either string inference; the K-free mass record; the evidence; the declaration's named functions; and each refusal, the int64 one at one clone and at two, with no id or weight in its message. - test_uk_geographic_identity.py: every clone's key equals the dense identity kernel's at K = 1 and 3; the clone path; unique keys that nest across K; the dtype; the vintage for the release's survey and base years; the evidence, in distinct source households at every K; and each refusal, with no key or id in its message. - test_uk_clone_first_nesting.py (O10): an invented frame through G1 and stages 16, 17 and 42 at K = 1, 2, 3, 7 and 15. Stage 17's mass scale is reported first. Clone 0 equals the one-clone frame on its ids, G1's columns, stage 16's copies and stage 17's seats and carriers, and every frame and copy keeps the clone-index invariant. K x each clone-0 weight is held to one clone's: rows an exact-total correction moved within rows x K x 2.2e-16 relative; at stage 17, the other incumbents of a funding stratum a stage-16 correction reached within the stage's step budget at every K; other rows bitwise at K = 2, and at K = 3, 7 and 15 within one ulp where the 1/K split is their only rounding, else one ulp per weight-setting step. Stage 42 nests approximately (family masses, the divided band, the crossed band's mass, rule B), and through G1 rule B's correction lands on a copy at 7 and 13 clones. - test_uk_salary_sacrifice.py: a one-clone index draws as none; three clones keep the one-clone cells, clone 0 draws as one clone does and each adult takes a donor of its own clone; the empty cell-clone refusal holds no digit; and the stage at one and three clones. - test_uk_uc_capital_coherence.py: a one-clone index redraws as none; at three clones that differ, the coarsening levels are three times one clone's; and a high-value copy counts as a donor of its own. - test_uk_cgt_support.py: every spelling of the household clone index equals the leaf's, and the re-exports are the leaf's own objects. Its nesting scenario's data move to test_support/microcosm_build/ uk_cgt_support.py, which the nesting test shares. - test_uk_copy_id_table.py lists geographic_expand among the copy-id readers, and test_uk_graph.py re-derives the binding tables and checks that an edit to the leaf re-keys every counting stage. These files, the whole of test_uk_graph.py, test_uk_graph_terminal.py, test_uk_cgt_structure.py, test_uk_receipt_counts.py, test_uk_geographic_clone_collapse.py, test_uk_atomic_identity_graph.py, test_first_party_imports.py and test_clone_expansion.py pass. tools/ci_test_plan.py verify is ok, and ruff check and ruff format --check are clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…and add geographic_expand and geographic_identity (#931) frs_hmrc_spine_leaves and frs_annual_income_spells (stages 14 and 15) move ahead of frs_take_up, and two new stages follow them: geographic_expand (G1) gives every FRS household K geographic clones, and geographic_identity (G1b) keys each clone. frs_take_up, frs_person_draws, frs_household_draws and frs_brma (stages 10 to 13) then run on every clone. The spine build installs both new stages at K = 1, so every spine is still one clone and no value moves. Positional stage numbers in this message, the changelog, comments and tests follow the order before this commit: 14 and 15 are the two moved stages, 10 to 13 the draws and frs_brma, and 16 onward are as before. The order. sources.yaml moves the two stage blocks unchanged and inserts G1 and G1b after them; the graph follows the manifest. The moved stages read only pay, self-employment income, age and gender; stages 10 to 12 read none of their outputs and write none of their inputs, and none of the five sets weights (a test holds this). Both moved stages join the raw FRS tabs on raw person ids, so they run on the survey's own rows, before the clones. The assembled checkpoint still ends at frs_brma, so the assembled phase and its gates now cover 14, 15, G1, G1b and 10 to 13, and the assembled gate admits spi_support_channel. G1 and G1b, declared and wired at K = 1: - The sources schema and ALLOWED_SOURCE_OPERATION_KINDS gain two strict op kinds. expand_geographic_clones declares the entity, the clone-index columns, the ordinal bound of 100, the copy-id geographic_support field at radix 10**9, clone-major order, an equal design-weight split, the source columns, conservation to the split's rounding, declared factor 1.0 and a reason that names no K. key_households_by_clone_path declares the source frs and frs_release.json, which G1b also declares as its one artifact with runtime_sha256_required, so the release's bytes enter its stage contract. Each transform checks its declaration against its module's reviewed parameters, whole; G1 refuses any artifact and G1b any but that one. - No declaration names K. uk_spine_graph(geographic_clones=K) puts K, 1 to GEOGRAPHIC_CLONE_LIMIT, on G1's node params, and refuses K above 1 for a manifest without G1, so K enters that node's key and every later one, never a stage contract. - graph.py: G1 joins UK_SPINE_STRUCTURAL_STAGES as an EXPAND that conserves mass at design weights, with an open surface and six cells. Its household cells are listed in the order the transform writes them (source_year, source_household_key, source_household_id, household_clone_index), since the graph appends an EXPAND's new cells in that order and H2 parity compares column order; an engine-free test now holds that order too. G1b reads the household source id, source year and clone index and writes geography_household_key. uk.full.identity rewrites the key where the spine already carries it, so the dense graph still compiles. - graph_kernels.py: both stages get their modules and the helpers the derived rule finds (G1: clone_expansion and copy_id_table; G1b: atomic_household_identity, frs_release and microcosm.graph.canonical) and join the counting stages, and G1 joins the copy-id readers at geographic_support. The registry's structural set is UK_SPINE_STRUCTURAL_STAGES. UKExpandStageKernel refuses a transform whose clone count differs from its node's, giving the two declared counts. The fixture registry builds G1 and G1b from an integer config.geographic_clones and its engine with clones=K, and its descriptor check no longer names a stage count. - spine_build: _GEOGRAPHIC_CLONES = 1 feeds the graph, the G1 transform and _rules_engine(clones=1). Its clones keyword defaults to None, so the full build's two engine installs are unchanged. G1 and G1b are installed when the roster names them. - The geography key's name, GEOGRAPHY_HOUSEHOLD_KEY_COLUMN, now lives in the geographic_clones leaf, which G1b hashes as a counting stage, and atomic_area_support re-exports it as IDENTITY_COLUMN, so the derived rule holds every reader to the leaf and the stage table needs no entry for it; no kernel that reads it misses the leaf. G1b names its string dtype through pandas, as atomic_household_lineage does, not through the executor's token table. The identity columns. source_household_id, source_year and source_household_key are G1's outputs and cells. spi_support_channel (stage 16) no longer declares or writes them, and its copies inherit them. The stage and the dense expansion default them from a household's own ids only on a table without household_clone_index, and refuse a clone-indexed table that lacks any of them, naming the columns. source_person_id and source_benunit_id are not minted (decision 5). Their names (UK_GEOGRAPHIC_SOURCE_COLUMNS, SOURCE_YEAR_COLUMN, SOURCE_HOUSEHOLD_KEY_COLUMN) move to the geographic_clones leaf, which spells the source id itself, since it imports nothing first-party (a test holds it equal to microcosm.frame.concepts'); geographic_expand re-exports them, and the SPI support channel and the dense crosswalk read the leaf's tuple instead of copies of their own. Stages 10 to 12 count true sources. frs_take_up, frs_person_draws and frs_household_draws are given household source_household_id and household_clone_index, so the shared reader counts their receipts' support in source households on every clone; it refuses the clone index without the source column. A test holds every stage after G1 that counts before the copies to both columns. Folding consumes. household_clone_index joins the inputs of spi_employment_shell, spi_housing_shell and uc_capital_coherence, which fold geographic clones, and of property_components. The typed root cells. frs_spine writes imd_decile (int64: the household's decile on its own nation's FRS index, IMDE, IMDW, IMDS or IMDN, 1 to 10, else 0, since the root refuses NaN) and council_tax_band_observed (bool: False in Northern Ireland, for a ctband code outside 1 to 9, for a band the region does not use and for an unknown region). council_tax_band itself is unchanged. The root refuses a household tab without one of the four IMD variables, naming it, rather than read zeros. Both cells are typed, leave at the release boundary (UK_RELEASE_EXPORT_DROPPED_COLUMNS) and stay out of every receipt and the sidecar's nonzero shares, whichever route built the records; the root's notes describe them. The root binds geography_constants, since both read each household's nation from the region tier. Their first reader is the area assignment of #931 group (c). microdf in node keys. microdf-python's version joins _numerical_dependency_versions ("not-installed" when absent), so it is in the root's and every stage's node params, and the sidecar records it as microdf_python beside engine_batches when a call batches; on that route private_school_attendance stores microdf's weighted percentile rank. pyproject.toml, uv.lock and us_runtime/worker_identity.py are unchanged; the version is declared at the next lock bump. Texts: - CGT_SUPPORT_EXACT_TOTAL_EXCEPTION, and with it the copy_rule of both support splits, high_value_home_support (stage 26) and cgt_support_split (stage 42), names the candidate order: heaviest first, tied weights in row order. - The notes of isa_subscriptions (stage 53) say tied balances share one weighted mid-rank, so the keyed uniform alone orders them, and the Lifetime ISA payments reuse the amounts operation's balance_rank_weight. docs/uk-isa-subscriptions-1415.md steps 3, 5 and 6 say the rank is on the class's balance, the added people walk on their own, holders over 47 pay nothing, and the cap at each holder's room keeps the baseline under the limit. - The weight rule of spi_income_band_donors (stage 17) names one clone's donor count and its draw a spine without household_clone_index; its notes size the seats on eligible households counted in source households. The notes of spi_support_channel (stage 16) say what its uniform depends on, and its source names every geographic clone. cgt_support_split's notes and the CGT changelog say which household the identity tool's recompute can count as a family the stage never made, and that the bracketed family test does not report it. - The source and notes of private_school_attendance (stage 52) say it computes attendance over the whole frame, from the pinned engine's values and parameters. - BONUS_SACRIFICE_SPI_DRAW and the notes of salary_sacrifice (stage 48) say an SPI adult draws within its own geographic clone from cells pooled over every clone; the draw text now also says the lowest cell joins the one above, as the code does. - uc_capital_coherence's notes count a cell's donors in survey records. - G1's notes say it clones before any other copy step, and childcare_copy_take_up's notes and docstring say frs_take_up draws each geographic clone on its own ids before any other copy exists, and that geographic clones keep those draws. - Texts this change made stale: household_lineage, spi_support, spi_band_donors, geographic_clones, geographic_expand, geographic_identity, salary_sacrifice, graph_population, diagnostics, copy_id_table, gate_phase_codes (six of the transferred-only modules are now hashed at or before frs_brma), receipt_counts, frs_take_up, private_school_attendance, frs_spine's and national_frame's band-flag wording (a band the FRS gives that the region uses), the clone-first stand-in, the identity tool's E7 docstring, docs/uk-full-build-graph.md (56 stages, the clone index on every spine frame, microdf_python in the sidecar), and the earlier #931 changelog fragments that said G1 was unwired, that no spine carries a clone index, or that the stage table and the kernel table read as before; and the new fragment 931-uk-spine-geographic-expand.changed.md. The code-mirrored texts are identical in the code, sources.yaml and source_stages.json. The was_wealth and was_lisa notes are unchanged. Bindings. The national gates hash geographic_clones, since their support-family fold reads the clone index columns, which the national frame carries from a clone-first spine; the kernel table's clone-index reason now covers only the calibrated gates, the dense measures and the national problem. The stage table keeps the source household column's name alone, its reason rewritten: geographic_expand writes it through the constant and declares it by name, every stage after it that reads the name is given the column, and frs_annual_income_spells, before it, is given neither the column nor a copy column. The kernel table's premise check now asserts that geographic_expand alone writes the clone-index columns on the spine and that the spine build installs it at one clone. Tools. tools/graph_uk_spine_fixture.py's oracle builds G1 and G1b from their packaged stages at _GEOGRAPHIC_CLONES = 1 and its engine with clones=1; its config records geographic_clones: 1, and uk_spine.json is written at that K. Its synthetic households gain IMDE, IMDW, IMDS and IMDN (invented deciles in turn, one household in seventeen blank). Its descriptor and PRODUCED_BY.txt no longer give a transform count. tools/build_uk_release_input_coverage_manifest.py gains a geographic_expand family, whose required reason is G1's declared reason, matched verbatim. Dense and national. The dense expansion refuses a spine carrying geographic_expand's clone index at any n_clones, naming the clone-first spine and #931 group (c), and its overflow message limits "n_clones=1 still expands" to a spine without the index. The national line is not yet refused early; a later commit of this group adds that refusal. No national or dense build runs on a clone-first spine before (c) (decision 4). Why: María's rulings of 10 Nov. Decision 3 (X) puts G1 and G1b after stages 14 and 15 and before stage 10, so the stages that read raw tabs by raw person id run on the survey's rows and stages 10 to 13 run on every clone from this commit. Decision 1 (Form B) puts stage 53's tie clause in its notes, which enter no stage contract. Decision 2 binds microdf's version into node keys without a re-lock, which would move the US approved-lock digest. Decision 5 mints no person or benefit unit source columns until something reads them. Every declaration edit of group (b) is in this commit, so H2 regenerates once. The clone columns' names sit in the geographic_clones leaf so that G1, G1b, the SPI support channel and the dense line read one definition their keys hash, rather than copies or a listed exemption. What moves. At K = 1 no value moves: on the H2 fixture the legacy oracle's final frame agrees with the parent's on every existing column, bit for bit and in dtype, on the weights and on every earlier mass record (checked through the engine slot). frs_take_up, frs_person_draws and frs_household_draws now count their receipts' support in true source households, which at one clone are the households themselves. The bytes move: - the stage order: frs_hmrc_spine_leaves and frs_annual_income_spells now run ahead of frs_take_up, with geographic_expand and geographic_identity between them; - the column order, since the households' source columns now come from geographic_expand; - each table's clone index, at 0; - geography_household_key, which a household's copies repeat until group (c); - the root's imd_decile and council_tax_band_observed, which the release drops and no receipt or sidecar share reports; - geographic_expand's mass record and both new stages' evidence; - spi_support_channel's strata, grouped by a constant clone index; - spi_income_band_donors' funding strata, which gain the clone index, so each label ends "|0"; - the national support-family fold, whose key columns gain the clone index. Seven stage contracts move: frs_spine (its outputs), spi_support_channel (its source text, outputs and nonnegative outputs), spi_income_band_donors (weight_rule and draw), high_value_home_support and cgt_support_split (copy_rule), salary_sacrifice (spi_bonus_draw) and private_school_attendance (its source text). geographic_expand's and geographic_identity's contracts are new. K is a parameter of geographic_expand's graph node, never of a contract. No contract hashes a stage's reads or notes. The source id and clone index that frs_take_up, frs_person_draws and frs_household_draws now read move only their nodes' inputs, and so does the clone index that spi_employment_shell, spi_housing_shell, uc_capital_coherence and property_components now read. The notes of isa_subscriptions (the tie clause, Form B), uc_capital_coherence, frs_spine, geographic_expand and childcare_copy_take_up move no contract. The dense line refuses every clone-first spine, and the national line's degenerate gate would refuse the constant indices. The coverage gate now requires geographic_expand's mass record, so a release from a spine built before this commit fails it too. No national or dense build runs between (b) and (c) (decision 4). Positional stage numbers in prose (comments, tests and changelogs) still follow the order before this commit. Hash and fixture. Every UK spine node re-keys: - microdf-python's version joins every node's numerical dependencies; - every stage kernel hashes graph_kernels and national_frame, both edited here; - the root's outputs move. sources.yaml, source_stages.json and the coverage manifest move the UK country fingerprint and the solve fingerprint, so every node keyed on one of them re-keys: the spine gates, the national targets and gates, the target selection, compilation, measures and problem, the full preflight and calibrated gates, and the rotated holdout. The rest of the national and dense lines re-key through the modules they hash: national_frame, graph_population, rowwise_dataset, geographic_clones, atomic_area_support, spi_support, household_lineage, receipt_counts and diagnostics, and the national gates through geographic_clones, which their key now hashes. Only the shared atomic-geography kernels and the export package keep their keys. Outside the UK one kernel re-keys: the shared transport gate battery (gates.battery@1). Its key covers the source closure of its gate bindings, which reaches source_manifest, so NZ's terminal gate and the export and package nodes after it re-key (the NZ spec is inactive). These keep their keys: - every other transport kernel; - the US post-transfer graph's nodes and kernels; - the US, NZ, BE and AM specs, seeds and fingerprints. uv.lock, pyproject.toml and the US approved-lock digest are untouched. The schema's two new operation kinds move only the spec engine's grammar receipt, which every country's emitted, uncommitted bundle lock carries. Regenerated: - source_stages.json, with the stage list merged by stage, so every other stage keeps its bytes and key order; - the release input coverage manifest, without --candidate-h5, then --check: a geographic_expand family, and source_stages.json's new digest in every family; - the H2 fixture, through the engine slot, then --check: uk_spine.json and sources/fixture.json (config.geographic_clones set to 1, both new stages and every declaration edit), household.csv and schema.json (the root's two carriers), frs_raw/househol.tab (synthetic IMDE, IMDW, IMDS and IMDN) and PRODUCED_BY.txt (no transform count). person.csv, benunit.csv, weights.json, NORMALIZATION.md and the donor and facts files regenerate byte for byte, and H2 parity passes. gates.json is unchanged. Tests (engine-free unless named; invented values only): - test_uk_graph.py: 56 stages; G1 in the EXPAND set and G1b a NONE node; eleven copy-id readers; the source household column's reason in the stage table; the geography key named in the leaf, read bound by G1b and re-exported by atomic_area_support; the kernel table's clone-index reason, whose premise is now that G1 alone writes the columns on the spine and the spine build installs it at one clone, and the national gates hashing the leaf; a stage after G1 that counts before the copies reads both lineage columns; every folding stage and property_components read the clone index; the moved stages and the draws read nothing the other writes; the root's typed carriers; G1 owns the source cells; microdf in every node's dependencies. - test_uk_geographic_expand.py and test_uk_geographic_identity.py build their transforms from the packaged stages and add: G1 is an EXPAND that conserves mass at design weights; K is on G1's node, never its contract; the graph's and the kernel's K refusals; no new row at K = 1 and tripled tables at K = 3 through the executor; the node lists each table's new cells in the order G1 writes them, at K = 1 and 3; the coverage family; the source columns named once in the leaf; the H2 registry building G1 and its engine at the fixture config's K, and refusing a K that is not an integer; a drifted declaration, a clones field and artifacts refused; G1b's node, and frs_release.json's bytes in its contract and not G1's. - test_uk_clone_first_nesting.py runs the packaged G1, and a K = 3 test runs stages 10 to 12 on the stand-in and holds their support counts to K = 1's. - test_uk_frs_spine.py: the carriers' types and codes, the absent-IMD refusal, codes that move only the carriers, the sidecar's share omission, and the band-flag test renamed for what the flag means; the driver's synthetic spec follows the packaged order with G1 and G1b, its engine stubs take clones, and the driver asks for its engine at one clone. - test_uk_spi_support.py and test_uk_spi_spine.py: the channel keeps G1's source columns and declares none; the clone-indexed refusal. - test_uk_full_population_graph.py: dense meets the reworded refusal at K = 1 and 2; the dense lineage refusal. test_uk_full_target_graph.py: uk.full.identity rewrites the key the spine carries. - test_uk_copy_id_table.py (G1's radix and the executor's trace at K = 1 and 3), test_uk_terminal_gates.py (the export drop), test_uk_rules_engine_provenance.py (microdf only when batching), test_uk_source_stages.py, test_uk_graph_evidence.py (the assembled gate admits spi_support_channel), test_uk_spine_acceptance_receipt.py, test_uk_identity_stability_receipts.py (G1 in the stacking-map exceptions), test_uk_spi_band_donors.py (frames through the real G1), test_uk_uc_capital_coherence.py, test_uk_cgt_support.py, test_uk_high_value_home_support.py and shared/test_country_spec.py: pins and literals. - Engine tier: tests/engine/uk/test_uk_graph.py subtracts the carriers from the nonzero-share pin, and test_uk_release_input_coverage.py's shipped-manifest pin gains geographic_expand. Checks. Before the review fixes, the whole of T/uk, run once in eight chunks, gave 7712 passed, 44 skipped and 2 failed; both failures fail at the parent too (one expects the UK engine absent, one needs a Node package this machine lacks). After them, every engine-free file they touch or that imports an edited module passes: 2652 passed and 14 skipped in 72 files, the graph package's engine-free H parity tests included. Through the engine slot, the fixture tool and its --check (byte-identical), H2 parity, tests/engine/uk/test_uk_graph.py and the shipped-manifest test pass. The source-stage regeneration and the coverage tool's --check report current, ruff check and ruff format --check are clean, and tools/ci_test_plan.py verify is ok. integration-uk and the rest of the engine-tier UK suite were not run here; this PR gets CI once #1519 merges. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
|
Per-commit notes for groups (0), WAS and land, and (a), moved here from the PR body on 11 Nov to make room for (b) and (c). They are as they stood in the body; "below" and "the next section" mean the body's Arm a1 section. Group (0)After the move, the dense build's per-clone engine blocks are no longer identical copies. policyengine-uk's
Data changes in group (0): only the annual-income factor's float32 step, kept by ruling on 7 Nov. Everything else is inert until G1. What moves without being an output:
The spine graph declaration ( For Max:
WAS and land
What moves:
Group (a)Commits 1a to 7 are in with their review follow-ups, as are the 8 and 9 Nov follow-ups and ruling commits and the transport kernels' derived binding rule (354296a to 4df4065). So are the rest of the 9 Nov (late) rulings for the UK and the ISA doc correction (b289e8f to 30fe64f). Arm a1 ran on 10 Nov (below), and aed673d and fce5828 came out of it.
What moves in (a):
|
…nd clone-aware gate thresholds (#931) On a spine of more than one geographic clone, the UK stage receipts now say how each FRS-trained fit's fold met its survey records. The SPI support channel and the income band donors count in source households, and the checks whose limits count rows read the clone count. The spine still builds one clone, so no value and no gate verdict moves. This follows María's rulings of 10 Nov: - decision 7: a threshold that grows with the clone count is multiplied by the receipt's clone_count, or counted in clone 0's rows; - decision 11: the band donors' funding rows take one form at every seating. Every new count is of distinct true source households, rounded through receipt_counts. Stage numbers here keep the order before stages 14 and 15 moved: 16 is spi_support_channel, 17 spi_income_band_donors, 18 hmrc_spi_income_spine and 25 regional_home_values. Fold receipts: - Every FRS-trained fit folds through household_lineage.fold_geographic_clones: the SPI income stage's reference and stage-2 forests, the SPI employment and housing shells, the UC reporter redraw, UC capital and salary sacrifice. - On more than one clone, each of these stages' evidence gains geographic_fold (household_lineage.geographic_fold_receipt). It gives the clone count, the source households the fit selected, those whose clones differ on what the fit reads, and the share that differ. The share is given only where ten or more of the selected source households differ and none or ten or more do not; otherwise the field repeats the count of those that differ. One clone's evidence is unchanged. - cross_fit_transport takes twin_identities, read only to find each recipient's twin among the training rows. The SPI income stage passes each copy's twin through its forests' fold, so the twin check reaches the copies of every clone. Each copy's uniforms stay keyed on its own identity; on one clone the two keys are the same. Stage 16 (spi_support_channel): the receipt gives the frame's clone count. Its two pension-age counts become rounded distinct source households carrying the mass beside them, on rows of positive weight: the pension-age base households', and those of clone 0's pension-age SPI copies. Both are the same at every clone count, and the masses are unchanged. Stage 17 (spi_income_band_donors): - At every seating, each funding row keeps its stratum and rounded donor households and reads "<3" for its masses and factor. The receipt gains two booleans: - funding_within_share: every factor is at least one less the declared reallocation share, to the stage check's relative tolerance; - funding_matches_bands: the bands and the funding strata each carry the reallocated mass. - On a capped seating, each band's donors and carriers, and the donor count, are rounded distinct source households. Each band's weighted taxpayers and carriers' mean age, and the reallocated mass, read "<3", and each band's carrier regions are left out. A full seating keeps the plan's exact band counts. - The result keeps the exact rows, and its frame, out of its repr. - A band short of candidates or seats is refused naming the band and, on a cloned spine, that a clone fell short, without a count. The band donor check: - It reads the funding from the two booleans and requires one less the receipt's reallocation share to reach its minimum funding factor. Its details drop the smallest funding factor. - On a capped seating it recomputes the seats from the transferred phase's frame. A seat is a band donor row whose id holds no ordinal in a copy-id field minted after the band donors. Every declared band must seat a donor, every seat must hold one carrier, and each band's seats must split evenly across the clones. The bands' mass there is the stage's funding_matches_bands, since later copy layers reweight the seats. The details then carry the clone count, the seating, the mass scale, the booleans and the count of bands. - No failure text gives a count, and a boolean that is present but not true is named as such. - The stage-health binding hands the frame to this check only (frame_predicate). copy_id_table joins the gate registry's modules, and the transferred phase's own code hashes it. The clone count: - geographic_clones.geographic_clone_count takes the calling stage and refuses a malformed clone index in fixed sentences naming it, with no count. The six support-clip stages, property components, the CGT support split and the fold receipts read K through it, and the two WAS stages bind the leaf for it. - GEOGRAPHIC_COPY_ID_FIELD, the copy-id field the clones' ids are minted in, is named in the leaf; geographic_expand re-exports it. Clone-aware thresholds: - The six support-clip stages (WAS wealth, WAS Lifetime ISA, NTS bus travel, LCFS consumption, ETB VAT and ETB services) and property components give clone_count when it is more than one. The support-clip check multiplies each declared row allowance by it, and the property components check its two row maxima. - Stage 25 (regional_home_values) counts the source households at the value cap in clone 0's rows, reading each row's clone from its id through the copy-id table, so its check's one maximum holds at any clone count. A frame whose clone index disagrees with its ids is refused. - The take-up signal receipt counts each flag's population support and distinct values, and the apprentice draw its supports, in clone 0's rows. A figure measured over every clone is withheld where either clone 0's count or the count over every clone is one or two. - test_uk_k_sensitive_constants.py classifies each count-like constant of the UK spine by what it counts and how it meets the clone count, and pins it to the code. Every count-like maximum in the packaged gates must be classified, the relationship grid's mismatch maximum among them. Text: changelog.d/931-uk-clone-receipts.changed.md is new. The unreleased band donor, CGT copy and fit fold fragments and docs/uk-atomic-household-lineage.md now describe the capped band counts, the count-free refusals and fold_geographic_clones as the fits' entry. What moves: - At K = 1 no value and no gate verdict moves. On synthetic frames this tree and its parent give the same frames and the same verdicts (pass or fail, and the failure count): the band donor check at full and capped seating, on a scaled mass, on a full sample seated capped and on a capped frame carried through every later copy layer, at one clone and at three. A one-clone receipt carries no clip or property clone_count, so those checks read their declared allowances as before, with the same texts and details. Every packaged clip and property allowance is zero, so the clone-scaled checks change no verdict at any clone count yet. - At K = 1 the bytes move only at stage 16 (clone_count and the two counts) and stage 17 (the funding rows and booleans at every seating, and the capped forms on a capped seating, as on the f010 and f001 samples), with the band donor check's details and failure texts and the clone count's refusal texts. The take-up signal and apprentice receipts count in clone 0's rows, which at one clone are every row. The geographic_fold receipts and the clip and property clone_count appear only from two clones up. - Node keys only: - Every UK spine node re-keys: graph_kernels' helper table re-keys every kernel it defines, the root and every counting stage hash household_lineage, and keys chain. - Both spine gate nodes re-key through stage_health and battery_bindings. The transferred phase's own code moves; the assembled phase's does not. - The full gates, preflight and terminal, re-key through battery_bindings, and the national gates through the registry's modules, copy_id_table now among them. The terminal phase's own code moves through etb_services, etb_vat, frs_take_up, lcfs_consumption and uc_capital_coherence; the preflight phase's does not. - The national problem and the target measures re-key, as do the dense population kernels (sample, normalize, expand, locations, geography mapping, geography gate and pool), the full build's identity, property value and private rent kernels, and the export preparation kernels, which hash the geographic clones' leaf. - 78 of the 105 UK kernel hashes and phase codes move (kernel.implementation_hash() against the parent). The other 27, among them the geography, export readback and package, target selection and compilation, problem, dense and size kernels, keep their own hashes, though their nodes re-key by chaining. - No generated file, declaration, fixture or gates.json changes, and H2 is not regenerated. No module outside the UK runtime imports an edited file, so no NZ, US, BE or transport key moves. Tests (engine-free; no engine test was needed): - The fold receipt is absent at one clone and rounded at three at all six sites, stage 18's through its checkpoint evidence. Frames with one and two varied sources give the same bytes, as three and nine do; ten varied sources give a share; all but one and all but two give the same bytes, as all but three and all but nine do; all varying give a share of 1. Every module whose fit folds builds the receipt. - On two clones, the twin check reads clone 1's copies against their folded twins and refuses a fold-key cell moved on one of them, while their uniforms stay their own. One clone is bitwise unchanged. - Stage 16's counts are the same at three and seven clones, count distinct sources at two, and leave out a row of zero weight. - Two capped stage 17 frames whose seats differ by one give byte-identical receipts, reprs and check results. - The band donor check fails a dropped carrier, an uneven split, a false, missing or non-boolean boolean, a looser share and a missing frame, with no data digit in any message. The transferred gate's inputs hold the cells it decodes. - Each support-clip stage, property components and regional home values run on three clones, with their checks unchanged at one; regional home values refuses a clone index that disagrees with the ids. The clone count's refusals are fixed sentences naming the stage. - The take-up and apprentice receipts count in clone 0 and withhold a figure that the other clones carry on one or two source households. - The constants table, and the binding, phase-code and copy-id tables. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Part of #931. This is the output-moving half of the geography-first UK spine, stacked on #1519, which changes no outputs.
The spine clones every FRS household K times at its start: K=15 on the dense line, K=1 on national. Each clone gets its own small area and its own later draws (wealth, consumption, SPI copies and so on). The K=1 national spine nests in the K=15 dense one wherever draws are keyed: it is the clone-0 slice.
This is a draft, opened early so the commits can be followed as they land. The base is
uk-931-geography-first-prep(#1519). CI runs only on PRs to main, so this PR gets none until #1519 merges.Commit groups
frs_brmafold, the SPI-copy area draw, G7, G8 inside the WAS chain, gates, dense re-basingGroups (0), WAS and land, and (a)
Their per-commit notes, with what moves at each commit, are in this comment.
Group (b)
Six commits, as designed and ruled on 10 Nov. Stage numbers in these notes keep the order before commit 3, which inserts G1 and G1b and moves stages 14 and 15 up.
ReplicaFold.key_group), and checks record ids by sorting. It adds the shared, country-neutral clone expansion core,microcosm.build.clone_expansion(CloneIdRule,CloneDeclaration,expand_clones), which takes the country's details as a declaration (decision 10; for Max). No value moves: every spine is one clone today.geographic_expand, and G1b,geographic_identity, bound to no stage yet.geography_household_keyas the dense line'suk.full.identitydoes.geographic_clones, which every counting stage and the dense kernels that read the names hash.frs_hmrc_spine_leaves,frs_annual_income_spells) ahead of stage 10 and wires G1 and G1b in after them at K = 1 (decision 3). Stages 10 to 13 now run on every clone, and stages 10 to 12 count their receipts' support in source households.expand_geographic_clonesandkey_households_by_clone_path(for Max). K sits on G1's graph node, never in a stage contract.imd_decileandcouncil_tax_band_observed, which the release drops and no receipt reports. microdf-python's version enters every node key (decision 2). The 10 Nov text items, stage 53's tie clause among them (decision 1), ride the one H2 regeneration.Arm a1 (10 Nov)
The statements are the run book's, filled from the compare's public verdict file.
Arm a1 (full f100 spines, K = 1). ctl = main at the move's merge base, 9d080e7. a0 = f50a29f, the head just before (a). a1 = aed673d, (a)'s head. ids = 0c4ebb1, the ids-only sub-arm (P14). Each arm ran from its own worktree and environment. Their outputs stay private.
Notes on the statements:
student_loan_balance, which stage 50 (student_loans) rewrites after the WAS stage E5 checks. The tool does not account for that rewrite yet.brmaandis_apprentice, which no later stage declares a rewrite of; those two are not explained yet.Rulings applied
exact.copy_rulesays so;copy_rulenames that one exception (the 8 Nov rule of never correcting a copy cannot always keep the total bitwise exact: in the synthetic nesting test at K = 7 and 13, no whole household can carry the correction);cgt_structurein stages 26 and 46 stays a listed refusal-only check;uk.full.expand, is reworded so it holds at any clone count, at (b)'s next stage-17 regeneration, so the H2 fixture regenerates once.copy_rulenames it;regional_home_valueskeepsSeedSource.NONE;microcosm.graphexports lazily, and the create kernel's H2-only use ofload_sourceis a listed stop;graph_framesmove stays.frs_spineandfrs_relationshipsleave the target kernels' hashes;microcosm.graph.canonicalin the transport compile kernels;calibration_runstays off the preflight's and the solve's key, throughgate_scopes.copy_ruletexts of stages 42 and 26 say the exact-total correction takes tied weights in row order, and stage 53's declaration says tied balances share a rank (itswithin_bandfield allows onlyuniform, so (b) settles the form);integer_record_keys' uniqueness check becomes sort-based, in (b);engine_batcheswhen a call batches, and declared at the next lock bump;source_person_idandsource_benunit_idare not minted until something reads them;Proof plan
🤖 Generated with Claude Code