Skip to content

Lock policyengine-uk 2.151.0 and write the UK inputs it reads (#1095, #1278) - #1383

Merged
juaristi22 merged 49 commits into
mainfrom
uk-pe-uk-bump-2572
Oct 24, 2026
Merged

juaristi22 merged 49 commits into
mainfrom
uk-pe-uk-bump-2572

Conversation

@juaristi22

@juaristi22 juaristi22 commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

The next policyengine-uk bump (plan of 2026-10-22, microcosm#1095 and microcosm#1278). It locks policyengine-uk 2.151.0, the first release past pe-uk#2573 and pe-uk#2580, and writes the inputs those releases read. One PR of focused commits in three phases, then the rows arm 2 re-measures, then Vahid's two reviews (below).

Closes #1340, closes #1320, closes #1222, closes #1349. Part of #1319: Scotland opens in the Carer's Allowance draw, and the back-test and the follow-ups listed there stay open.

Phase A, on the 2.147.2 lock (13 commits; receipts in experiments/1095-uk-next-bump-phase-a-receipts.md)

Phase B, the lock (1 commit): policyengine-uk 2.147.2 to 2.151.0, policyengine-core unchanged at 3.33.3. The uprating pins' stamp, the concept coverage (314 to 335 inputs), the coverage manifest, the H2 fixture and APPROVED_UV_LOCK_SHA256 re-derive. The ESA support-group test's legacy tenant no longer claims UC, since 2.149.0 ends income-related ESA on a UC claim (pe-uk#2563). The gambling duty inputs of pe-uk#2590 stay at their defaults (#1359), and so does housing_benefit_residential_capital_exception (2.148.4), signed in UK_ENGINE_DEFAULT_INPUTS: the FRS records no assessment of the accommodation the exception covers, so Housing Benefit is understated only for a qualifying working-age resident, who takes tariff income on capital above £6,000 instead of £10,000.

What the lock alone moves, before any data change (from Vahid's engine-only run on a133f37a):

  • income-related ESA falls from £3.85bn to £1.88bn in 2025, as the engine ends income-related ESA on a UC claim (pe-uk#2563). No household of the SPI channel carries that row. Phase C's UC take-up rule keeps a reported award off a UC claim, which offsets it, and DWP's income-related ESA claimant row fits within half a percent;
  • Welsh LTT and Scottish LBTT rise by 2.150.0's 1/0.045 factor, into gov_tax, household_tax and net income; no bound row carries them;
  • council tax falls under 2.148.5's projection; obr.council_tax binds the total;
  • the targeted two-year-old entitlement falls, as 2.148.1 reads UC earnings before the work allowance; dfe.funded_childcare.early_learning_2_year_olds binds the count;
  • September CPI uprating (2.148.2) and the Pension Credit floor (2.151.1) move benefits from April 2027; no 2025 row reads them;
  • statutory paternity pay is now taxed, inside obr.income_tax, which binds;
  • household net income and the poverty rates move a little. They rest in part on the SPI channel's households, so no figure is given here (SN 9422).

Costings and dashboards built on a133f37a, on 2.147.2, need a full rerun on the next release rather than an engine swap.

Phase C, on 2.151.0 (12 commits; receipts in experiments/1095-uk-next-bump-phase-bc-receipts.md)

The rows arm 2 re-measures (3 commits, with the Phase B and C receipts)

Review

Vahid's first review (round 1, at 74a2a37) and second (round 2, at 3ce1fca). Item numbers are his.

  • Item 1, withholding across grains. 3ce1fca withheld every constituency, local-authority and BRMA model figure from what ships. At María's ruling of 2026-10-24 it is reverted (e3c3ecd), with d48652b, its fixture follow-up: the fit evaluation ships. A row that ten or more distinct source households carry publishes its figures at every grain; a thin row, and any published row that would give it back by difference, is withheld. f1cec40 runs the complementary check over overlapping blocks: the national, regional and BRMA rows; each region's rows of every grain; and each local family with the national and regional targets the cross-grain declarations or a nested measurement tie it to. A row withheld in one block is protected in every other, and the blocks run until none withholds another row. BRMA rows publish from ten households under the national block's check (her rulings of 2026-10-17 and 2026-10-24). With the revert, round 2's items 25 to 31 no longer apply.
  • Items 3, 4, 8 to 12: the energy declaration's end-to-end test, the Housing Benefit engine default, the bottled gas and the pinned fuel parts, the levelling on remainders, obr.vat held as a diagnostic again, the drift flags carried as warn-only diagnostic_warnings, and WPA's positive rate rule.
  • Items 5 to 7: bonus sacrifice added beside pay, the SPI channel's bonus drawn from FRS donors, and the anchor's growth basis stated; its check at the final weights stays with the licensed data.
  • Items 13, 15, 17 and the nits: the ALIAMT approximation, the bound rows that absorb the 52-week rule, the benefit cap's scope, the single adult discount count's vintage (as at September 2025, revised on 21 January 2026), schema 3's refusal, the re-signed energy rationale, Scotland's carers' rate, and the support floor's units checked against the contract.
  • Arm 3: its receipt and the engine checks for items 7, 9, 14 and 17 are in the Phase B and C receipts. The bonus sacrifice base holds at the stage's weights only: by María's ruling of 2026-10-24 this PR records it, and UK bonus sacrifice: meet OBR's base at the final weights, not the stage's #1390 re-picks the sacrificers at the final weights.
  • pe-uk: item 8 is liquid_fuel_consumption: COICOP 04.5.3 does not include LPG policyengine-uk#2642; item 12 is noted on bsp_reported is CPI-uprated but BSP rates are frozen, and the variable is mostly War Widow's Pension policyengine-uk#2381, whose step 3 the split unblocks; item 16 is uprate_student_loan_plans relabels existing holders in 2025 against SLC plan rules (Plan 4, NI Plan 1, Postgraduate Loans, pre-2006 write-off) policyengine-uk#2636, which the student-loan engine test links.

Measurement

All runs are local and licensed; their outputs stay with the spines. Details are in the two receipts.

  • Control: b2572-c0main, main at 6bdb800 on policyengine-uk 2.147.2, one spine and one calibration (review date 2026-10-23).
  • Arm 2: one spine and one calibration at the bump head. All seven terminal gates pass; 1,265 rows bound against 1,259. Over the 1,258 shared rows with a nonzero target, the share within 10 percent goes from 99.21 to 99.05 percent, the mean absolute error from 0.56 to 0.55 percent, the effective sample size from about 7,400 to 7,500, and the share of design weight stretched 9.5 times or more from 0.67 to 0.59 percent. Two shared rows cross the 10 percent line outward: a row of the SPI channel and a UC band whose figures are withheld.
  • Arm 2b: calibrate-only on arm 2's spine with the four UC bands bound (the head's calibration; review date 2026-10-24). All seven gates pass; over the 1,264 shared rows, the share within 10 percent goes from 99.05 to 99.13 percent, and no shared row leaves the band.
  • Arm 2c: arm 2b with the ORR row bound on table 7271. The row reaches its target, but uk_weight_ratio fails and the effective sample size falls by about a tenth, so the row stays held on table 7270.
  • Arm 3: one spine at the review's head for its stage changes (bonus sacrifice, the levelling, WPA, the fuels) and a calibrate-only run on it. All seven gates pass; against arm 2b the share within 10 percent goes from 99.3 to 99.2 percent, the effective sample size is about the same, and one UC household count leaves the band. Its engine checks are in the Phase B and C receipts, as classes.
  • Phase A: arm 1 and arm 1b on Lock policyengine-uk 2.145.0, follow its bus fares and bind thin target rows #1333's 2.145.0 spine, in the Phase A receipts.

Disclosure

No figure in this PR, its commits or its receipts rests on fewer than ten source households or names a record. No row's model figure is given where the row is of the SPI channel or rests in large part or in part on the SPI channel's households: SN 9422's Special Conditions send analyses of the tape to HMRC ten working days before they are distributed, and that ruling is open (#1278). The surface-wide fit and weight statistics are given as in #1308. Publisher figures appear as published. The private checks behind C1, C2 and C10 stay with the licensed data, outside the repository.

Signatures

Signed by María on 2026-10-24:

  • obr.vat's entry, held again, 2026-10-24 to 2027-01-24 (review item 10).
  • The restated ons.household_energy_expenditure rationale.
  • housing_benefit_residential_capital_exception in UK_ENGINE_DEFAULT_INPUTS (review item 4).

For María to confirm:

  • The new uprating hold on mhclg.council_tax_stock.single_adult_discount (in-year snapshot, signed 2026-10-23 to 2026-12-11), drafted as the band rows' hold is.
  • The Housing Benefit and JSA renewals, signed 2026-10-24 to 2027-01-24, and the eight UC bands' reasons restated on arm 2.
  • The two ONS energy rows' reasons and the ORR row's reason and adjudication, each with an arm 2 or arm 2c measurement added.
  • The Phase A drafts listed in the Phase A receipts.

Not in this PR

Tests

Counted with --junitxml, on this machine.

  • At the review head, only the touched surfaces ran locally: the support floor, the take-up contract and the carers stage (203 tests); the registers, signed exclusions, frame and spec readers (770); the withholding, the gates, the gate kernel, the Orrery summaries, the calibration graph, the sign-off draft and the release readers after the revert (342 and 425). All pass, after one gate test frame gained a household region; two are skipped. CI runs the full suites.
  • Before the review: tools/ci_test_plan.py verify ok; tools/spec_engine_coverage.py --check 42,184 of 42,184 fields; the UK engine tier and integration-uk passed at the Phase C head.

🤖 Generated with Claude Code

juaristi22 and others added 29 commits October 24, 2026 00:35
The fifteen dwp.benefit_cap rows (the total and the fourteen amount bands
of DWP's Benefit cap tables to November 2025, Table 4) bound
benefit_cap_reduction. That variable adds the Housing Benefit cap and keeps
the gross regulation 81 amount, while DWP counts Universal Credit
households by the amount the award loses. Every row now binds
uc_benefit_cap_reduction_after_floor at benefit-unit grain, and the
measurement names the UC reduction. The total was the incumbent's 0/1
household indicator; it now counts the benefit units the bands partition,
and the complementary withholding covers the closer difference.

Regenerated on abf0921 (the national references from the feed cut to the
national and local-authority rows): the target references move on the
total's entity alone, household to benefit unit; the membership report and
the national compile-parity receipt are unchanged. The diagnostic tool
reads the bound variable, and the reviewed population variables add it.
The OBR Universal Credit rows split by cap status keep
benefit_cap_reduction.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tire

María's ruling of 2026-10-22 extends the zero-support floor to the
national families. Its share ceiling sums published values, so the
register (schema 4) now holds the floor's rows in scopes of one unit, each
with its own ceilings: the UC payment-distribution and benefit-cap bands
(benefit units, as before), the HMRC SPI income-band counts, the HMRC SPI
income-band amounts, and the band-H council-tax rows by region. A test
checks that every scope holds one unit.

Measured on #1333's 2.145.0 spine, every HMRC £1m-and-over cell and every
band-H region cell has carriers, so their thirteen entries retire and the
rows bind, with their figures withheld where fewer than ten source
households carry them. Plan 5's liable stock fits at design weights and
drifts only while unbound, so its entry retires too. The OBR split of UC
spending at the welfare cap and the England postgraduate repayments leave
the targets for registry_parity.excluded: no household concept marks the
welfare-cap boundary, and the stage assigns no postgraduate plan.
obr.universal_credit still binds the sum of the two cap cells.

Regenerated on abf0921: the national references and membership lose the
three retired rows (1,322 to 1,319 active), and the compile-parity receipt
signs them as fixture-only with their reasons. The other held rows keep
their entries until arm 1 re-measures them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's ruling D3 of 2026-10-08 held obr.vat out until a locked
policyengine-uk split the constant survey-coverage factor in vat.
policyengine-uk#2187, released in 2.135.7 and inside the 2.145.0 lock,
does that. On #1333's 2.145.0 arm the row, measured while held out,
already sits inside the 25 percent bound at both the design and the
calibrated weights, so its entry retires and the row binds. The notes
declare the residual the row keeps: OBR's receipts include VAT on
private school fees from 1 January 2025, which policyengine-uk's
current-law vat leaves out until policyengine-uk#2319.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…'s 95% intervals

The #731 scorecard measured period 2024 against HBAI FYE 2024, but period
2024 is FRS FY2024/25, which HBAI calls FYE 2025 (uk-data#487). The new
microcosm.build.uk_runtime.hbai_validation reads the four statistics the
pinned feed carries for FYE 2025, each with DWP's 95% confidence interval
and the release it came from: the median equivalised household income and
the share of individuals in relative low income, before and after housing
costs. It reports each measured statistic inside or outside its interval
and refuses a missing statistic or fact; it never binds a row or fails a
build (María's rulings of 2026-10-16 and 2026-10-20).

scorecard_731.py takes --hbai-facts and writes an hbai_validation block at
engine year 2024; the FYE 2024 reference figures it compared the other
HBAI-shaped rows with leave the script, and those rows stay as diagnostics.
The classifier is tested on fixture facts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…imation

microcosm writes no self_employment_gross_receipts: no source of the
self-employed's turnover is read yet. Since policyengine-uk 2.108.0 the
trading allowance replaces expenses and does not stack on them, so without
receipts trading_allowance_deduction relieves a profit of £1,000 or less in
full and gives a larger profit no allowance, which puts taxable profit at
most £1,000 a person from where receipts would put it (#1095).

UK_ENGINE_DEFAULT_INPUTS, beside the take-up register in the concept
mapping module, signs the approximation with that bound, as the plan for
the next policyengine-uk bump set by default. An engine test checks each
entry is an engine input the mapping never binds, and an engine-free test
that no spine column in the H2 fixture's schema writes it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's ruling of 2026-10-17 left property_purchases, regional_home_values
and uc_legacy_closure_take_up ungated for a follow-up. Each now has a
release-blocking stage-health gate that blocks on an absent receipt:

- uk_stage_property_purchases_fit: each purchase kind is drawn once, draws
  HMRC's Stamp Tax transactions within 5 percent, and fits HMRC's value
  less a recorded shortfall of at most one fifth, the floor the stage
  itself refuses below; the individuals' shares are proper shares.
- uk_stage_regional_home_values_ranking: rows are re-ranked in every
  English region and Wales on the 2024 knots, at the weighted midpoint
  within each FRS region, with no source at the value cap.
- uk_stage_uc_legacy_closure_take_up_contract: the Move to UC receipt
  withholds what a thin combination realizes, so the gate checks what it
  prints: the design-weight Great Britain basis, a probability and a
  declared source for each rate, a printed support band, and a
  well-formed unreachable flag.

All three pass on #1333's 2.145.0 spine receipts. The gates join the spine
scope, the data contract's battery map and certification scope, and the
mirrored battery digests move. The H2 parity fixture does not.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The tape's Special Conditions add two obligations to the End User
Licence: no attempt to attribute a record to a named individual, and
analyses of the tape go to HMRC at least ten working days before they are
distributed or published. Nothing in a build's outputs said which of them
rest on the tape.

microcosm.build.uk_runtime.spi_dependency reads which of a run's stages
declare the tape among their artifacts (the spec's ukds_study_number,
written bare or as "SN 9422"). The spine's build record carries the marker,
spine_provenance_from_sidecar carries it into the calibrated candidate and
its signed diagnostics, and both release assemblers restate it in
release_manifest.json: on the microdata's artifact entry
(spi_public_use_tape_dependent) and, with the conditions, under
licensed_data. The assemblers rederive it from the stages the signed
provenance names, through the packaged spec, and refuse a recorded marker
that disagrees or a stage the spec does not declare; a provenance from a
spine built before the marker records none, and the rederived one stands.
On the packaged roster three stages read the tape.

How SPI-dependent outputs are released stays María's ruling; the marker
records the dependency only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Arm 1 (calibrate-only on #1333's 2.145.0 spine at the Phase A head) bound
the ten HMRC £1m-and-over cells that A2 released. Bound, they moved
neighbouring rows of the SPI channel that had fitted closely out of the
10 percent band, one cell stayed outside the release fit bound, and the
arm's uk_target_fit gate failed; the other six gates passed. Under
María's rule of 2026-10-21 a measured distortion keeps rows out, so the
ten are held again as a family, re-signed to 2027-01-23 with that reason,
pending a split that binds the cells which do not distort. The register
names no figure of theirs or their neighbours': they rest on the SPI
Public Use Tape.

The band-H region cells, Plan 5's liable stock and obr.vat reached their
targets on the arm without stretching their carriers' weights, and stay
bound. The floor's adjudication and the changelog say so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The twelve UC payment bands and Plan 5's repayments lapsed on 2026-11-26,
the two Plan 2 stocks too, and the any-tenure housing-element count on
2026-11-14. Arm 1 (calibrate-only on #1333's 2.145.0 spine) measured each
unbound at its calibrated weights, close to #1333's own arm:

- every one of the twelve bands stays outside 10 percent, eleven short
  and one over; they stay on their own renewals until the award-band
  take-up of #1344 binds them (María's ruling of 2026-10-21);
- the Plan 2 coverage gap stands;
- Plan 5's repayments: fewer than ten source households carry them, so
  the reason gives no figure and the row's figures are withheld;
- the housing element sits at the other/unknown-tenure gap itself.

The rows rest in large part on the households of the SPI support
channel, so the reasons give no new figure of theirs (SN 9422). Each
entry is re-signed on 2026-10-23 to 2027-01-23, and the adjudications
keep their history. The tests follow the new dates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's ruling of 2026-10-23 extends the ten-household floor for
published figures, with its complementary check, to constituency and
local-authority rows; the area-support rule still decides which local
rows bind. uk_withheld_row_sets and uk_thin_rows now take every row, in
blocks: the national, regional and BRMA rows in one span, as before, and
the local rows one grain and area at a time, since the local surface is
too large for one dense span and binds many families per area. The
positive-weight slice of the problem is built once for all blocks.

The report guard matched a bare area code for every withheld key, which
suits a BRMA's one family but would stop a dense build on any published
row in a constituency with one thin row. It now takes the bare codes
explicitly, and uk_withheld_gate_areas gives them for withheld BRMA rows
only; a withheld local row is matched by its family/area/metric key. The
dense terminal phase passes both.

Tests: a thin row is withheld at every level; in one constituency a
family total less its published band gives a thin band back, so the next
band is withheld beside it, while a thin row in another area stands
alone; and the guard names a published row of another family in the same
constituency but refuses the withheld key and a withheld BRMA's code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
experiments/1095-uk-next-bump-phase-a-receipts.md records how Phase A was
measured without a new spine: #1333's 2.145.0 arm as the control, arm 1
(calibrate-only on its spine with every candidate row bound) and arm 1b
(the plan's contingency run, with the HMRC £1m-and-over cells held again).
Arm 1 failed uk_target_fit on one of those cells and moved neighbouring
SPI-channel rows out of the 10 percent band; arm 1b passes all seven
gates, with the shared rows' fit, effective sample size and stretched
share as on the control, and the band-H region rows, Plan 5's liable stock
and obr.vat at their targets. It also records why the rows still held are
renewed.

The receipt names no SPI-channel row and gives no count of source
households. It gives no row's model figure that rests on fewer than ten
source households or in large part on the SPI channel's households.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The integration-uk smoke build blocked at phase 'transferred' on two of
the gates a6280dba1 added. Both check the frame against published facts,
which a synthetic fixture is not: it holds a few regions, so the regional
home-values receipt rewrites three of the ten regions, and it sizes its
own purchase targets to its few households, so the non-residential draw
misses its count by more than 5% and the receipt carries none of HMRC's
individuals' shares.

Both entries now declare population_fact_check, as #1069 did for the
Pension Credit take-up gate: a smoke build records their failures in the
report and continues, and every other posture blocks on them as before.
The legacy closure gate holds only the receipt's own contract, which the
fixture meets, so it stays blocking in every posture. A test pins all
three. The mirrored battery digests move.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's ruling of 2026-10-23: weekly amounts annualise at 52 wherever a
weekly or monthly rule reads them. policyengine-uk reads an annual amount
back to a weekly or monthly one at policyengine-core's 52 weeks before
such a rule tests it, so at 365.25 / 7 every FRS weekly amount read 0.34%
high at Housing Benefit, Council Tax Reduction, Pension Credit, the
non-dependant deductions and the hours tests. #1329 made the State
Pension the first amount at 52 (her ruling of 2026-10-20); it is now the
rule, not an exception.

- frs_spine.WEEKS_IN_YEAR is 52, and STATE_PENSION_WEEKS_IN_YEAR leaves
  the spine; the readers that import the constant follow it (council tax,
  disability, education grants, legacy proxies);
- the HMRC replay leaves mirror it, and their declared formulas say * 52;
- the private-rent localisation keeps its own copy, now 52, so it imports
  nothing of the build;
- the spec's annualize_periodic_amounts operation declares 52 for every
  amount and drops the State Pension exception; its prose and the HMRC
  leaf formulas say 52; source_stages.json, the release input coverage
  manifest and the H2 parity fixture are regenerated.

LCFS spending keeps 365.25 / 7 and NTS trips 52.14, since no weekly rule
reads them back. A new test holds every reader to the one constant; the
State Pension and manifest tests follow. The 0.34% shift is measured on
arm 2, the bump's one new spine.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The UK lock moves from 2.147.2 to 2.151.0 (policyengine-core stays at
3.33.3), the first release past pe-uk#2573 and #2580 and the newest on
PyPI. The uprating pins' stamp, the concept coverage (314 to 335
inputs), the coverage manifest, the H2 fixture, the cited versions and
APPROVED_UV_LOCK_SHA256 re-derive; no uprating value moves.

One engine test follows the new engine: since 2.149.0 a claim for
Universal Credit ends an income-related ESA award (pe-uk#2563) and the
engine's default unit claims it, so the ESA support-group test's tenant,
on legacy ESA, now claims no Universal Credit. The gambling duty inputs
of pe-uk#2590 stay at their defaults (microcosm#1359).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…526)

policyengine-uk 2.148.1 (PolicyEngine/policyengine-uk#2571) projects
stored domestic energy spending from the price period a dataset declares
in its root energy_price_basis attribute, and keeps undeclared spending
at its stored prices in every later year. A Microcosm H5 declared none,
so every later year kept the 2024 prices lcfs_consumption sets
(PolicyEngine/policyengine-uk#2526, its remaining item 2).

The one physical UK writer now declares {"period_type":
"calendar_year", "period_value": 2024} wherever the household table
stores any of the three columns the engine projects from a declaration,
with the keys sorted as the engine writes its own. The period is the one
the stage-time gate uk_stage_lcfs_consumption_basis holds every LCFS
spend level to (microcosm#1113), and a test keeps the two equal. An H5
without energy declares nothing; a reload and rewrite declares it again.
An engine-tier test reads it back through policyengine-uk and checks the
columns are the engine's own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1095)

policyengine-uk 2.146.0 (pe-uk#2549) closes a legacy award through its
owners: the claimant and partner, or a member's own award, which closes
on its own. The uc_legacy_closure_take_up draw still read every member's
reports, so an award a dependant claims in their own right could put the
family in the draw, or keep it out as a Universal Credit unit
(legacy_closure_take_up.py, flagged in #1095's bump item).

The reports that set a unit's combination, and its Universal Credit
status, now come from members marked is_claimant_or_partner, the FRS
adult roles the spine already writes. The graph node declares the read,
the operation names its reporters, and the spec, its schema entry and
source_stages.json follow. A new test runs a dependant's own ESA (the
family stays out of the draw) and a non-claimant's Universal Credit (the
family stays in it); the stage refuses a frame without the column.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From policyengine-uk 2.149.0 a claim for Universal Credit ends a
claimant's or partner's income-related ESA. The UC take-up redraw kept
each unit that holds a legacy benefit at its earlier single-rate flag,
which sets many units reporting ESA, and no UC, to claim it, so the
engine would end their ESA at the calibration year. A unit whose
claimant or partner reports income-related ESA, and that neither reports
Universal Credit nor is moved off its legacy benefits, now claims none:
it was on the award at the interview, and a reported receipt is a fact.

The rule is declared on the operation (kept_legacy_award) and the
closure draw, which reads the same flag, sees it too; the receipt bands
the units it keeps off UC. Housing Benefit, which the engine also ends
on a claim, keeps the stored flag pending the HB and JSA PR (#867, #869).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's ruling of 2026-10-23: Scotland opens in the Carer's Allowance
draw. The stage kept it closed because Carer Support Payment reads the
same flag and policyengine-uk did not apply its earnings limit; since
2.146.1 (PolicyEngine/policyengine-uk#2473) it does, so a drawn Scottish
carer is paid only under Carer Support Payment's own limit.

The eligible set loses its Scotland mask, and the receipt reports
Scotland apart: its eligible support band, weighted persons (withheld
under ten sources) and drawn support band, in place of the closed-region
check. The texts, the spec and source_stages.json follow. The engine
tier now checks that a drawn Scottish carer under the limit is paid
Carer Support Payment and one over it is not, and the stage-through-
engine test draws the Scottish carer over the 2024-25 limit and finds
them paid at 2025.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FRS benefit code 6 is Bereavement Support Payment with its predecessors,
and the spine summed it all into bsp_reported. policyengine-uk 2.149.0
(pe-uk#2572) gives Widowed Parent's Allowance its own input, because it is
taxed and means-tested where the payment is not. María's ruling of
2026-10-22: split code 6 by rate, after a private check of the amounts.

A weekly amount within 1% of a Bereavement Support Payment rate's weekly
equivalent (GBP 350 or GBP 100 a month, frozen since 6 April 2017; at
12/52 for a month, quarter or year and at 1/4 for four weeks) stays
bsp_reported; any other goes to widowed_parents_allowance_reported. The
new column goes wherever War Widow's Pension goes: the spine's outputs,
the SPI support and spine lists, the SPI benefit-coherence restore and
its gate, the graph, the spec and the export allow-list with its
gates.json mirror. The mirrored digests move, and source_stages.json and
the coverage manifest are regenerated.

A parametrised test runs each rate's two weekly equivalents, the edges
of the 1% band either side, and amounts off every rate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
policyengine-uk 2.149.0 (pe-uk#2572) adds spousal_maintenance_income,
which Universal Credit (UC Regulations 2013 reg. 66(1)(d)), Pension
Credit and Housing Benefit count, unlike the child maintenance in
maintenance_income. The spine read no alimony at all, and kept APAMT,
the allowances from a husband or wife who lives away, in
private_transfer_income.

María's rulings: ALIAMT, or ALUAMT where ALIUS 2 says the last amount
was not the usual one (2026-10-22), plus APAMT, which the FRS asks only
of married adults whose spouse is away or separated (2026-10-23, after a
private check of the recipients). APDAMT, the household bills such a
partner pays directly to third parties, stays a private transfer.

The new column goes to the spine's outputs, its non-negative outputs and
the export allow-list with its gates.json mirror; an adult tab without
the three alimony columns refuses, as one without SPPADJ does. Tests read
each ALIUS answer, refuse each missing column, and keep every FRS field in
one income column. The H2 fixture's adults carry the three columns.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
policyengine-uk 2.149.0 adds pension_contributions_via_bonus_sacrifice,
which the GBP 2,000 NICs exemption cap from April 2029 and the annual
allowance tests read summed with salary sacrifice (pe-uk#2572 section
1). Microcosm wrote salary sacrifice only, so a costing of the cap left out
the bonus route the OBR costs (#1340). María's ruling of 2026-10-22:
impute it; the OBR's base starts in 2027-28, after the calibration year,
so it anchors the stage instead of binding the calibration.

- frs_spine writes frs_bonus_amount, each adult's gross occasional
  bonuses over the last 12 months from the JOB tab (BONAMT1-6, a bonus
  given after tax with its imputed weekly tax over the year); a tab
  without the fields refuses, and the carrier leaves at the release
  boundary.
- salary_sacrifice gains a third operation after the salary conversion.
  Candidates are adults with a bonus whose pay without it reaches the
  upper earnings limit, the 2% band the OBR costs bonus sacrifice at.
  Each converts the whole bonus, within half the pay and the room left
  above the other routes, out of pay, in the order of identity-keyed
  draws, until the engine's projection of the converted bonuses to
  2027-28 (each year's growth compounded, the threshold on the two
  routes summed) is nearest the OBR's base. The receipt bands its
  support on source households and withholds the base under ten.
- The OBR's Table 1.1 tax base is vendored from the pinned feed
  (obr_salary_sacrifice_costing_facts.json, a streamed copy of
  tools/vendor_uk_ledger_facts.py's output), and the stage verifies it
  first. tools/pin_uk_bonus_sacrifice_anchor.py writes
  uk/bonus_sacrifice_anchor.json from it and the locked engine's
  earnings and population growth and upper earnings limit; an
  engine-tier test holds the anchor to the installed engine and the
  bonus route's excess to the engine's own cap arithmetic.
- The spec, its schema and operation registry, the declared seeds, the
  graph's output cells, the export allow-list and its mirror (digests
  re-pinned) and the H2 fixture's jobs follow. The HMRC salary-sacrifice
  rows, which ASHE builds, keep binding the salary route alone, and a
  test says so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
policyengine-uk 2.149.0 (pe-uk#2572) adds liquid_fuel_consumption and
solid_fuel_consumption, parts of housing that add nothing to
consumption. The LCFS stage left the diary's heating oil, bottled gas
and solid fuels inside the housing remainder with no column. María's
ruling of 2026-10-22: LPG with liquid, heat in the remainder, and the
ONS 04.5 row measured on electricity, gas, liquid and solid fuel.

- The donor split writes each fuel from its labelled parts (b017, c45312,
  b018, c45222; c45411, c45412); both are annualised, uprated to
  calendar-2024 prices on the CPI fiscal bridge as every diary spend
  column is (the consumption-basis gate reads them), and drawn last in the
  chain so every earlier draw is the chain's own.
- The housing recomposition nets their draws and adds them back, so the
  total always holds them; a household whose drawn parts outgrow its
  drawn total keeps at least its fuels.
- The recomposed-totals receipt's uncarried block becomes published_fuels:
  ONS 04.5.3 and 04.5.4 beside each fuel's donor total at diary prices and
  the frame's draw at stage weights. The stage-time gate recomputes the
  published rows; the spec, its schema, the graph's cells and the stage's
  outputs follow.
- ons.household_energy_expenditure, which the contract keeps for
  registry parity and the reference surface signs out as superseded by
  its fuel rows, reads the four fuels through a value expression; its
  signed rationale no longer says the frame lacks liquid and solid fuel.
- The consumption drift block reads the two fuels where the frame
  carries them, so an older spine keeps its block, and
  uk_consumption_drift_flags gives warn-only flags at #1278's R1
  thresholds (2 percent for a total, half a point for a share), outside
  the diagnostics payload so its v8 schema stays as it is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each COICOP division whose frame total per household stands above its
donor's by more than the twelve divisions together do comes down to that
shared ratio before the hold to the donor maximum; no other division moves.
A recomposed total moves by its remainder alone, so the levelled parts keep
their published levels. Declared as level_division_excess on the recompose
operation; the receipt records each division's ratio and factor, and the
stage-time gate checks them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1095)

The WAS property draw of María's ruling of 2026-10-23. Three draws are
appended after consumer debt, each from the holders of the column it rests
on, so the first eight segments draw as before:

- the directly held fraction of the share-like holdings outside ISAs, from
  which directly_held_shares (WAS UK shares plus employee shares and
  options) and unit_and_investment_trusts (WAS collective investments) are
  written on the final corporate_wealth less the ISA, so the three sum to
  corporate_wealth and the engine's share_like_capital does not move
  (pe-uk#1974);
- the debt secured on other residential property and on non-residential
  property, zero where the asset is zero and scaled down together where
  they would pass the mortgages on other property.

The recorded UC and Pension Credit capitals and the reporter redraw's
screen net each class's debt after the costs of sale, floored at zero per
class (UC Regs 2013 reg 49(1)(b); SPC Regs 2002 reg 19(b)), as the engine
values UC property capital. The donor cleaning refuses debts that pass
the other mortgages; the wealth coherence gate holds the split and the
debts as identities; the four columns are reviewed export extras.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#953)

policyengine-uk applies the single adult household 25% discount from
2.148.0 (pe-uk#1864, council_tax_discount_rate). The new contract row
mhclg.council_tax_stock.single_adult_discount counts England households
at that rate against MHCLG Council Taxbase line 8, all bands, the England
row of the latest October return not after the build base period. The
billing-authority cells bind in the next dense build; no incumbent row
has this definition, so it is an unmapped declaration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
policyengine-uk projects the 2024 dataset to the 2025 calibration year
and relabels every holder by a course started at 18; the stage dates a
course from the age its borrower left full-time education (#868, #1299).
The engine test pins where the two disagree: a course dated after 2012
by its leaving age but before by the age rule becomes Plan 1, and a
1993 course is written off, so the calibration measures the engine's
plans at 2025. An engine that keeps the stage's dating fails it, and the
SLC rows are then re-measured.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Arm 2 of the next bump (one spine and one calibration on
policyengine-uk 2.151.0) re-measured the rows still held, unbound at its
calibrated weights. Since #1344 draws UC take-up by award band, four of
the twelve UC payment-distribution bands come within 10 percent, so their
entries retire and the rows bind (the plan's rule: bind a held row that
fits); the eight that stay keep their 2026-10-23 renewals, their reasons
restated on arm 2.

The four Housing Benefit rows and DWP's JSA claimants stay outside the 25
percent bound at design weights. Their 2026-12-08 entries are renewed,
signed on 2026-10-24 to 2027-01-24, pending the Housing Benefit and JSA
PR (#867, #869).

The two ONS energy rows, held out since #1113, keep their entries; their
reasons add arm 2's consumption drift block, on which electricity and
gas spend and the gas-connected share stay within the warn-only flags
between the design and the final weights.

The UC bands rest in large part, and the Housing Benefit and energy rows
in part, on households of the SPI support channel, so their reasons give
no new figure (SN 9422). The JSA row's claimants at arm 2 are all in FRS
households, and its reason gives its figures. The register tests follow
the counts and the dates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…atio

The plan of 2026-10-22 set a default on the open 7270-versus-7271 ruling
(microcosm#890 G): bind orr.rail_government_support on ORR table 7271's
net operational government support from all sources, unless an arm
measures a distortion. Arm 2c, a calibrate-only run on arm 2's spine
with the row bound on table 7271 beside the four UC bands, did: the row
reached its target, but the weights concentrated past the release bound,
uk_weight_ratio failed, and the effective sample size fell by about a
tenth.

So the row stays held out on table 7270's basis, its entry keeps the
2026-12-11 window, and its reason and adjudication record the
measurement. The basis ruling stays open.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
experiments/1095-uk-next-bump-phase-bc-receipts.md records how the
2.151.0 lock and Phase C were measured: the control on main, arm 2 (one
spine and one calibration at the bump head), arm 2b (calibrate-only on
arm 2's spine with the four UC payment bands bound) and arm 2c (with the
ORR row bound on table 7271 as well). Arm 2 and arm 2b pass all seven
gates; arm 2c fails uk_weight_ratio, so the ORR row stays held. It also
records each Phase C item's check and why the rows still held are
renewed.

The receipt names no SPI-channel row and gives no count of source
households. It gives no row's model figure that rests on fewer than ten
source households or in large part or in part on the SPI channel's
households.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 and others added 2 commits October 24, 2026 14:52
Vahid's review of microcosm#1383, item 7. The anchor grows each
sacrificed amount with the OBR's average earnings and the weights with
the ONS population to 2027-28; the OBR grows its base with employment
and earnings. The base is met at the stage's weights, and calibration
does not bind it. The stage's comment and the changelog say so; the
check of both OBR lines at the final weights is a private receipt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Vahid's review of microcosm#1383. The Phase B and C receipts now give the
single adult discount count's vintage (as at September 2025, revised on
21 January 2026); the remainders-basis levelling, under which six
divisions come down on arm 3's spine and communication keeps its draw;
the donor's liquid fuels without bottled gas beside ONS's 04.5.3; the
bonus sacrifice rulings and the anchor's growth basis; the bound rows
that absorb the 52-week rule (item 15); and the benefit cap's
complementary withholding and its Universal Credit scope (item 17).
Figures that rest on the SPI channel's households stay with the
licensed data; arm 3's calibration receipt follows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

@vahid-ahmadi thanks for both rounds. This replies to both, at 0640d051e. Item numbers are yours. The body now has a Review section and the "what the lock moves" list.

Round 2 (items 25 to 31)

At María's ruling of 2026-10-24, the blanket withholding is reverted: the fit evaluation ships. e3c3ecde0 reverts 3ce1fca6c and its fixture follow-up d48652beb. f1cec40d5 puts back the joint check for item 1 (below).

  • 25 to 27, 30 and 31 no longer apply. The gate report, targets.csv, the local gate report and the size evaluation again read one withheld set: the thin rows and their difference partners. The fit gates count the same set, as before 3ce1fca6c. So the sign-off tool's checks hold again, the manifest keeps the signed local exclusions, and the size evaluation sees the red local targets.
  • 28, BRMA rows: María's ruling. A BRMA row that ten or more distinct source households carry publishes its figures, under the national block's difference check (her rulings of 2026-10-17 and 2026-10-24). Where a run also carries Scotland's local rows, the BRMA rows sit in Scotland's region block beside them too.
  • 29: the fixture patches full_gates.uk_withheld_rows again, which exists again; the file's 15 tests pass.

Round 1

1 (must), withholding across grains. f1cec40d5 makes uk_withheld_row_sets span overlapping blocks:

  • the national, regional and BRMA rows, as before;
  • each region's rows of every grain: the rows whose positive-weight carriers all live in that region. So a region row less its published local cells is caught, and so is a region's constituencies less its local authorities;
  • each local family with its higher-grain parents: the targets that the cross-grain declarations (bridges, fan-out, sum and band bridges, partitions) or a shared measurement with nested filters tie it to. So a national row less a family's published cells is caught too.

A row that one block withholds is protected in every other block that holds it, and the blocks run again until none withholds another row. Local rows need the frame's household region, so the check refuses without it. The dense gates and the Orrery summary pass each row's contract target.

Tests cover a region row less its cells, constituencies less authorities, a national parent less its family's cells, a row withheld by one block and kept hidden in another, and the refusal; each mechanism is mutant-checked. On a synthetic region of 2,218 rows, the check takes under two seconds.

The limit: a higher-grain row that neither a declaration nor a shared measurement ties to a family is not paired with it.

2. The body's Phase B now lists what the lock alone moves. ESA keeps your figure, since no household of the SPI channel carries it. The other items are qualitative, each with the bound row that absorbs it, where there is one. Net income and the poverty rates get no figure (SN 9422).

3. f995397cd: an engine-tier test writes a file, loads it through Microsimulation, and checks each energy column in 2025 and 2026 against the engine's energy price index over its 1 January 2024 value. An undeclared copy keeps its stored spending. The changelog says what the declaration changes, and that releases built before it stay undeclared.

4. ff8dfdd85: housing_benefit_residential_capital_exception joins UK_ENGINE_DEFAULT_INPUTS with the error it can make. The body states it beside the gambling inputs.

5 and 6. 9bc164c6c, on María's rulings of 2026-10-24:

  • The sacrifice is added beside pay, which stays as it is. The UGRSPAY comment now cites SN 9563's derived-variable notes.
  • An SPI-channel adult draws its bonus share of pay from a base-FRS adult with pay in the same pay cell. The cells are weighted pay quantiles, and a cell with fewer than 20 donor source households joins the one below. The draw is keyed on person_id, and FRS rows keep their own bonus.

7. 865dda866: the stage comment and the changelog now say that the anchor grows amounts with OBR average earnings and the weights with the ONS population, while the OBR grows its base with employment and earnings. The check of both OBR lines at the final weights is a private receipt, and I post its outcome as a threshold class with arm 3's receipt.

8 and nit 20. 94cb840b6:

  • The receipt now gives the donor's liquid fuels without bottled gas beside ONS's 04.5.3, with the concept note.
  • The column comments warn that a costing on either fuel would come out near twice ONS's spend.
  • A test pins each fuel's diary parts, with c45511 in the remainder.

The pe-uk docstring fix is drafted for María to file.

9. a94fc7a05: a division's ratio is now the spend the chain drew for it against the donor's mean of the same spend. For housing and transport, that is the remainder against the donor's total less the same parts. The cut is still one pass, by María's ruling: measured again after each cut, the shared ratio falls until every division sits at the lowest one's. On arm 3's spine, six divisions come down; communication now keeps its draw. The change in total consumption and in the VAT base follows with arm 3's receipt, as threshold classes.

10. 94d4102fd: obr.vat is held as a diagnostic again, signed 2026-10-24 to 2027-01-24. Its reason states the circularity and the private-school residual. It rebinds when pe-uk re-derives the factor on a file that does not bind it.

11. 2ee07b894: each drift flag becomes a warn-only diagnostic_warnings entry (uk_consumption_drift). The entry names the total or share and the threshold, but none of its figures. Nothing is gated on it.

12. 3f120c452: a code-6 amount is now WPA only at its 2024-25 rate (£148.40 a week, within 1%) or below it, as a reduced award. Any other off-rate amount stays bsp_reported. The pe-uk issue on bsp_reported's CPI uprating is drafted for María to file.

13. 7174e821d: the FRS 2024-25 question AliMny asks for payments "from a previous partner", so ALIAMT also holds a former cohabitant's. The survey does not say whether the partner was a spouse or civil partner, so the stage comment and the changelog state the approximation.

14. Open. It needs the engine on arm 3's calibrated file, with and without the secured debt. The result follows with arm 3's receipt, as a threshold class, since UC and Pension Credit rest in part on the SPI channel's households.

15. 32b2ee545 fixes the docstring. The receipt (0640d051e) names the bound rows that absorb the fall: the HMRC SPI income-band rows, the OBR income tax and National Insurance receipts, and the OBR and DWP benefit spending and caseload rows.

16. The pe-uk issue is drafted for María to file. The SLC rows are re-measured on the first arm after the engine fix, and at the latest before the Plan 2 and Plan 5 holds renew on 2027-01-23.

17. The receipt states that Housing Benefit-capped households have no row, and how the complementary check covers the total. The run check follows with arm 3's receipt.

Nits:

  • 18. 7174e821d: the feed vendors the revision of 21 January 2026, whose counts are as at September 2025. The row's note, the changelog and the receipt said "October", after the feed's 2025-10 period label.
  • 19 and 21. 32b2ee545: the changelog says a schema 3 register is refused, and the restated energy rationale is signed on 2026-10-24.
  • 22. e59a53d30: the take-up contract had still applied the rate "outside Scotland", so its citation and population now say Scotland's carers take the same rate.
  • 23. The room rule went with item 5. The docstring now says that a person's place in the order is their own, but where the order stops depends on everyone's contributions.
  • 24. 8ed7835c1: each scope's unit is now checked against its targets' contract measurement, so the run refuses a mislabelled scope. Band H says households.

Still to come

Arm 3's calibration receipt. Arm 3 is one spine at the review's stage changes plus a calibrate-only run, and its receipt brings items 7, 9, 14 and 17's run checks.

🤖 Generated with Claude Code

… nor the donor

The column comments, the stage's notes, the changelog and the receipt
said both fuel columns sit at the diary's level, about twice ONS's, so a
costing on them would come out near twice ONS's spend. That is the
donor's level at diary prices. The stage draws both fuels without
levelling them, and the frame's draws reach neither the donor's totals
nor ONS's, so a costing on either needs levelling to ONS first. The
frame's figures rest in part on the SPI channel's households and stay
with the licensed data. source_stages.json, the coverage manifest and
the H2 fixture follow the notes.

The student-loan engine test now links policyengine-uk#2636, which
tracks the engine fix (Vahid's review of microcosm#1383, item 16).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vahid-ahmadi

Copy link
Copy Markdown
Contributor

@juaristi22 thanks, this is my third review, at f3bd0b42b.

Verdict. No must is left open. The revert and the joint check across grains are a better answer to item 1 than the blanket withholding was. The local fit evaluation ships again, and items 25 to 31 go with the revert. From round 2, I would have treated 25 as release-blocking for the local line. Blanket withholding would have been acceptable to us if the body and changelog said so. Neither point matters now. In the new check, I could not fault the overlapping blocks, the protection carried from one block to another, or the fixpoint loop, and I enumerated the contract's local families to check the limit you state. Two new shoulds: the region blocks may be slow at real scale (32), and the family check fails open when a row has no contract target (33). Items 7, 9, 14 and 17 wait for arm 3's receipt. The other round 1 items are resolved. From our side, this can merge once CI is green at the head, with 33 fixed or accepted. Do you plan to land arm 3's receipt in this PR, or in a follow-up? Either works for me if the body says which. CI at 0640d051e had engine-uk red on the H2 parity test. At f3bd0b42b that test passes locally, and CI was still running when I posted.

What I ran

  • CI at 0640d051e: lint, wheels, select-countries and integration-uk passed. Both engine-uk jobs failed on test_acceptance_h_parity.py::test_h2_uk_spine_parity (1 failed, 255 passed): the H2 golden's frs_person_draws stage_contract_sha256 did not match the head's. At f3bd0b42b, lint, wheels, select-countries and integration-uk pass; both engine-free jobs, both engine-uk jobs and both engine-us jobs were still running when I posted. tools/ci_test_plan.py verify: ok. The head merges cleanly with main (git merge-tree; main is 83 commits ahead).
  • Engine-free, with --extra uk, at 0640d051e: the diagnostics, full gates, graph terminal, full calibration graph, Orrery contract, dense sign-off draft, salary sacrifice, LCFS, support floor, FRS spine, measure simulation, size evaluation and both release assembler files. 839 passed. At f3bd0b42b: the H2 parity test, LCFS and diagnostics files, 178 passed.
  • Engine: test_uk_energy_price_basis.py, 3 passed.
  • I ran uk_local_family_parents() on the committed contract: 435 contract targets, 59 local (36 local authority only, 18 at both grains, 4 constituency only, 1 BRMA), forming 34 families. 27 have higher-grain parents. For the 7 without, I looked for national or regional rows of the same measurement (item 33 has the detail).
  • A timing of numpy.linalg.eigh on dense symmetric matrices of the block sizes in item 32, on my laptop.
  • Primary source: DWP, Benefit and pension rates 2024 to 2025, for Widowed Parent's Allowance (£139.10 in 2023-24, £148.40 in 2024-25), which matches WPA_WEEKLY_RATE.
  • Not run: no build of any kind and no data, so the joint check has not been run on a real local surface. No £ figures from the frame appear here, so your SN 9422 rule is not engaged.

Status of round 2

  1. Sign-off tool refused every local build: resolved by the revert. The kernel writes the unredacted gate report again, uk_published_fit_gate_report and the uncertified-local sets are gone, and test_uk_dense_signoff_draft.py passes.
  2. Signed local exclusions left the manifest: resolved by the revert. reviewed_exclusions is back in the details the assembler reads.
  3. Size evaluation's red ledger: resolved by the revert.
  4. BRMA withheld everywhere: resolved by your ruling, now stated in the body. BRMA rows publish from ten households under the national block's check. Where a run carries Scotland's local rows, they sit in Scotland's region block too, since all their carriers live there.
  5. engine-free fixture: resolved. test_uk_full_calibration_graph.py:67-68 patches uk_withheld_rows, which exists again.
    30, 31. No longer apply.

Status of round 1

  1. Must, withholding across grains: resolved in code, checked on synthetic tests only. _row_blocks (diagnostics.py:1127) now adds one block per region, holding every row whose positive-weight carriers all live in that region (_region_blocks). It also adds one block per local family with its higher-grain parents (_family_blocks, uk_local_family_parents). Rows withheld beside a thin row in one block get an infinite recovery floor in every other block that holds them (_withheld_in_block), and blocks re-run until nothing changes. Rows are only ever added, so the loop ends. The checks I made:

    • local clones keep their donor's region (constrain_to_region defaults to true in rowwise_dataset.py), so region blocks do hold the local rows;
    • Scotland's BRMA rows join Scotland's block beside its local-authority rows, which also covers the case where a local-authority private rent cell overlaps a BRMA;
    • a national row of one nation, such as Scotland's band rows, falls in that nation's region block;
    • the five new tests cover the region-less-cells, constituencies-less-authorities and national-less-family cases, plus protection across blocks and the refusal without regions.

    The limit you state (an untied higher-grain row) does not bite on today's contract, as far as I can see (item 33).

  2. What the lock moves: resolved. The body's Phase B list covers each engine change and the bound row that absorbs it, and gives a figure only for ESA.

  3. energy_price_basis: resolved. The changelog says what the declaration changes, and the new engine test loads a written file through Microsimulation and checks 2025 and 2026. It passes locally.

  4. housing_benefit_residential_capital_exception: resolved. It is in UK_ENGINE_DEFAULT_INPUTS with the error it can make, and the body states it.

  5. Bonus sacrifice and pay: resolved by your ruling. The sacrifice is added beside pay, and pay is unchanged (salary_sacrifice.py, BONUS_SACRIFICE_MOVE). The candidate test now reads pay itself against the limit, and the UGRSPAY comment cites SN 9563.

  6. SPI rows' bonus: resolved. draw_spi_bonuses gives each paid SPI-channel adult a bonus share of pay drawn from base-FRS adults in its weighted pay quantile cell. A cell with fewer than 20 donor source households joins the one below, and the draw is keyed on person_id. FRS rows keep their own bonus.

  7. Bonus anchor after calibration: partly. The comment and changelog now say how the anchor and the OBR grow the base. The check of both OBR lines at the final weights comes with arm 3's receipt.

  8. LPG and fuel levels: resolved here, but the pe-uk docstring fix is not yet filed. The receipt reports bottled gas apart from ONS 04.5.3, and f3bd0b42b now says the frame's draws reach neither ONS nor the donor, so a costing needs levelling to ONS first. That wording is better than the "near twice" it replaces. Please link the pe-uk issue or PR once filed.

  9. Division levelling: partly. Each division's ratio is now the spend the chain drew against the donor's same spend (lcfs_donor_drawn_means), so levelled parts no longer count as over-draw. The single pass is your ruling, with its reason in the comment. The change in total consumption and in the VAT base comes with arm 3's receipt.

  10. obr.vat: resolved. It is held as a diagnostic, signed 2026-10-24 to 2027-01-24, with the circularity and the private-school residual in its reason.

  11. Drift flags: resolved. uk_consumption_drift_warnings feeds the diagnostics payload's diagnostic_warnings (diagnostics.py:2793) as warn-only entries without figures.

  12. WPA rule: resolved. A code-6 amount is now WPA only at the 2024-25 rate within 1%, or below it; the rate matches DWP's table. The pe-uk issue on bsp_reported's CPI uprating is still to be filed. See nit 34.

  13. Spousal maintenance: resolved. The approximation (ALIAMT holds a former cohabitant's payments) is stated in the stage comment and changelog.

  14. Secured debt netting: open, waiting for arm 3.

  15. 52-week rule: resolved. The docstring now says 52, and the receipt names the bound rows that absorb the fall.

  16. Student-loan pin: resolved. The test links policyengine-uk#2636, and your reply says when the SLC rows are re-measured.

  17. Benefit-cap total: partly. The receipt states that Housing Benefit-capped households have no row. The run check comes with arm 3.

18 to 24 (nits): resolved. The single adult discount note now says September 2025, revision of 21 January 2026 (18). The changelog notes schema 3's refusal (19). A test pins the fuel parts (20). The energy rationale is re-signed 2026-10-24 (21). The Scotland take-up reason is fixed (22). The room rule went with item 5 and the order docstring is right (23). Each support-floor scope's unit is checked against its targets' contract (24).

New findings in the delta

Should

  1. The region blocks may be slow at real scale (diagnostics.py, _region_blocks and _small_recovery). A region block holds every local row in the region. Counting local_target_references.json against local_area_crosswalk.json, the South East has 3,755 local rows (64 local authorities, 91 constituencies) before its own region rows, against the 2,218 of your timing. _small_recovery takes a dense eigh of the published rows' Gram matrix on every pass, and _withheld_in_block makes one pass per row it withholds, plus one. The fixpoint loop then re-runs any block that shares a newly withheld row. On my laptop, eigh takes about 3 seconds at 3,760 rows, so a region with a hundred complementary withholds takes several minutes, more with re-runs, across 12 region blocks. The set is also computed separately in full_gates.py:248, in the Orrery summary (orrery_contract.py:66), and in write_uk_calibration_diagnostics from full_build_cli.py:3160, which passes no withheld. Please time it on the first real local surface before the next local release candidate. And compute the set once per build, then pass it to the other callers, as graph_terminal.py already does.
  2. The family check fails open when a row has no contract target (_family_blocks). Without regions, the check refuses (_row_blocks). But a row with no metadata.contract_target_id silently joins no family block. test_a_national_row_less_a_familys_published_cells_withholds_one_more asserts exactly that: without targets, nothing is withheld beside the thin cell. The full gates and the Orrery summary pass the id explicitly. full_build_cli.py:3160 relies on the shared payload's rows carrying it from target.metadata, which the reference authoring does set, but nothing enforces it. Please refuse a local row without its contract target. A parent the build does not bind is not published, so it needs no refusal. On today's contract the limit you state looks empty. The 7 parentless families are the two DESNZ kWh families, the three equivalised income and housing cost families, and the two private rent families. I found no national row of the same measurement for any of them: the national energy rows are spend, not kWh. A contract test asserting that would keep it so as rows are added.

Nit

  1. A WPA award with inherited additional pension above the base rate now reads as bsp_reported, untaxed and not means-tested, as the comment says. A private count of code-6 amounts above £148.40 × 1.01 that match no BSP rate would show whether that is rare.

What looks right

  • The revert brings back one withheld set for the gate report, targets.csv, the local gate report and the size evaluation, so the gates count what the writers withhold.
  • Protection across blocks is conservative in the right direction: any combination holding a row another block withheld counts as too small, so a block cannot hand back what another hid.
  • The region refusal fails closed, and region blocks take any concept, so a region row of one family less another family's cells is covered too.
  • The fuel wording in f3bd0b42b is more accurate than before, and the frame's fuel figures stay with the licensed data.
  • The WPA rule now needs a positive match, and the tests pin the edge cases, including the lump-sum case it cannot tell apart.

Merge order

Unchanged: this PR, then #1381 (stacked on this branch, still draft), then my #1373. Both conflict with this head, mostly in generated files (the coverage manifest, source_stages.json, sources.yaml and the H2 fixture); #1373 also conflicts in contract.py and its test. Both need a rebase once this lands.

🤖 Generated with Claude Code

…nd 17

Arm 3 is one spine at the review's stage changes and a calibrate-only
run on it. Against arm 2b it passes all seven terminal gates and keeps
the fit. The engine checks on its calibrated file are given as classes,
since their figures rest in part on the SPI channel's households: the
bonus sacrifice base falls short of OBR's line at the final weights by
more than a quarter (recorded at María's ruling of 2026-10-24; #1390
re-picks the sacrificers after calibration); consumption and VAT move
by under 3 and 5 percent; the secured debt's netting moves UC and
Pension Credit spend by under 0.1 and 1 percent; and the benefit cap's
complementary check withholds no further row.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

@vahid-ahmadi arm 3's results, which my last reply said would follow. Arm 3 is one spine at the review's stage changes (b1b315ae0) and a calibrate-only run on it, against arm 2b. The receipt is in experiments/1095-uk-next-bump-phase-bc-receipts.md (e570d65ff). The engine figures rest in part on the SPI channel's households, so they are given as classes.

  • Fit. All seven terminal gates pass. The run binds 1,268 rows, which is arm 2b's rows less obr.vat. The share of rows within 10 percent is 99.3 percent in arm 2b and 99.2 percent in arm 3. The effective sample size is about the same, no shared row moves by more than 5 points, and one UC household count leaves the 10 percent band.
  • Item 7. At the final weights, the bonus base above £2,000 at 2027-28 falls short of OBR's line by more than a quarter, while the stage meets it at its own weights. So your concern holds: the anchor does not survive calibration. The salary line comes out within 25 percent of OBR's. Calibration to the HMRC income-band rows lowers the weights of the highest earners, where the sacrificers sit, and no row binds the base. At María's ruling of 2026-10-24, this PR records it, and UK bonus sacrifice: meet OBR's base at the final weights, not the stage's #1390 re-picks the sacrificers at the final weights.
  • Item 9. Against arm 2b, total consumption rises by under 3 percent and VAT by under 5 percent. Both changes mix the levelling on remainders, the bonus no longer taken out of pay, and obr.vat leaving the objective.
  • Item 14. Netting the secured debt raises UC spend by under 0.1 percent and Pension Credit spend by under 1 percent at 2025. It takes many benefit units' assessed capital below £6,000 or £16,000, but few of those units are entitled either way.
  • Item 17. On the run, the thin bands are withheld, and the band that no household carries leaves the solve under the zero-support floor. The complementary check finds no published combination that gives a thin band back, so it withholds no further row.
  • A correction to my last reply (item 8). I wrote that the column comments warn a costing on either fuel would come out near twice ONS's spend. That is the donor's level at diary prices, not the frame's. f3bd0b42b corrects the comments, the notes, the changelog and the receipt: the stage draws both fuels without levelling, and the frame's draws reach neither the donor's totals nor ONS's, so a costing on either column needs levelling to ONS first.
  • pe-uk. Item 8 is liquid_fuel_consumption: COICOP 04.5.3 does not include LPG policyengine-uk#2642, and item 12 is noted on bsp_reported is CPI-uprated but BSP rates are frozen, and the variable is mostly War Widow's Pension policyengine-uk#2381. For item 16, your uprate_student_loan_plans relabels existing holders in 2025 against SLC plan rules (Plan 4, NI Plan 1, Postgraduate Loans, pre-2006 write-off) policyengine-uk#2636 is now linked from the student-loan test.

@juaristi22
juaristi22 merged commit fe1e2ad into main Oct 24, 2026
10 checks passed
juaristi22 added a commit that referenced this pull request Oct 24, 2026
#1383 merged with these open, so they land here.

Item 33: the withholding check refuses a local row that names no contract
target. Such a row used to join no family block, so nothing was withheld
beside a thin cell of its family. A contract test keeps each parentless
local family free of a higher-grain row of its measurement. Today those are
the two DESNZ kWh, three equivalised income and housing cost, and two
private rent families. The synthetic local rows in the gate, terminal and
Orrery tests now name a contract target that no family holds.

Item 32: a build computes the withheld rows once.
- The national path's diagnostics take the set that the national gates
  flagged in their calibration evidence.
- The Orrery result summary takes the set recorded by a gate that reads the
  same result, when that gate saw every summary row; otherwise it counts.
- A local surface's check logs its blocks, block checks and seconds to
  stderr, so the first real local surface times it.

Item 34: on the FRS 2024-25, fewer than ten code-6 amounts sit above Widowed
Parent's Allowance's rate at no Bereavement Support Payment rate. The stage
comment says so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Oct 24, 2026
…m#1359)

#1383's last commits level each COICOP division on the spend the chain drew
against the donor's same spend. After the rebase onto main, recreation comes
out at £151.0bn rather than £148.7bn. The gambling stakes are a share of it,
so they rise to £2.31bn.
- Lottery Duty is now about 0.55 times National Lottery stakes (was 0.56),
  with a base 4.6 times them (was 4.7).
- The other gambling duties are about 6.3 times betting and gaming stakes
  (was 6.4).

Incidence, duty per carrying household, and the alcohol and tobacco figures
do not move. The stage notes, changelog and receipts say so, and a stage run
on the rebased head is in the evidence. All eight LCFS gates pass, and 26 of
the 46 columns match the pre-rebase head. The others are the seven levelled
divisions and the 13 gambling inputs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

@vahid-ahmadi #1383 merged with items 32 to 34 of your third review open, so they're in #1381, now rebased onto main (98a09827a).

  • 33. The withholding check now refuses a local row that names no contract target, instead of leaving it out of every family block. A contract test keeps the parentless local families free of a higher-grain row of their measurement. Those families are the two DESNZ kWh, the three equivalised income and housing cost, and the two private rent families.
  • 32. A build now computes the withheld rows once.
    • The national path's diagnostics take the set that the national gates flagged in their calibration evidence.
    • The Orrery result summary takes the set recorded by a gate on the same result. It counts again only if that gate did not see every summary row.
    • A local surface's check logs its blocks, block checks and seconds, so the first real local surface gives the timing you asked for.
  • 34. On the FRS 2024-25, fewer than ten code-6 amounts sit above Widowed Parent's Allowance's rate at no Bereavement Support Payment rate, so the case is rare. The stage comment says so.

Tests on the touched surfaces pass: the withholding, gates, Orrery and build CLI files, 631 of 631.

🤖 Generated with Claude Code

juaristi22 added a commit that referenced this pull request Oct 26, 2026
The first dense build on policyengine-uk 2.151.0 produced no gate report: the differencing check behind the model-figure withholding, which since #1383 runs over each region's rows of every grain and each local family with its parents, ran for hours on the dense surface and then failed to converge in an eigendecomposition. By María's ruling of 2026-10-26, --withhold-thin-rows-only keeps the withholding of the rows resting on fewer than ten source households and skips that check, in the calibrated gates, the diagnostics writer and the run's evidence summaries, so each BRMA row's carrier verdict stays true to its count.

The gate code it would otherwise change is shared with the source preflight, which keys the solve, so the build command arms the switch at run time and records it on the calibrated gates node and in the build request: only that node and what follows it re-key, and a retry with --require-solve-reuse replays the stored solve. It needs --no-publish-orrery, and the national role refuses it; a default request is unchanged byte for byte. Tests cover the refusals and that only the calibrated gates node is marked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Oct 26, 2026
María's basis ruling of 2026-10-25 (microcosm#890 G, microcosm#1095): the row
reads table 7271's net operational support from all sources, not table
7270's broad total, which adds investment no household's rail use carries.
It stays held as a measured diagnostic, since bound on 7271 it failed the
weight-ratio gate on #1383's arm 2c; its expiry does not move. The national
references and membership regenerate with the bump's shared regeneration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 pushed a commit that referenced this pull request Oct 26, 2026
By María's ruling of 2026-10-26 the UK releases go only to the licensed
private repository, whose users need the full diagnostics, so a build no
longer withholds model figures (microcosm#1262, #1383):

- A target row resting on fewer than ten source households, and a row
  that would give one back by difference, keep their estimates and errors
  in the calibration diagnostics, targets.csv, the dense and national gate
  reports, the national calibration evidence and the run's evidence
  summaries.
- The fit gates judge every row; no binding counts withheld rows, and no
  report guard looks for one.
- Totals, losses and the size receipt are the solve's own, and staged
  epochs carry their loss again.
- BRMA rows carry no carrier verdict, and the release contract no longer
  warns about one.

This removes the thin-row count, the differencing check (its block spans,
residual search and eigendecompositions, which ran for hours on the dense
surface and then failed to converge) and --withhold-thin-rows-only. The
withholding code is gone from the gate modules, so the next dense build
re-solves. Release certification refuses a run built while rows were
withheld, rather than judging it without them; the release contract still
checks the withheld rows of earlier releases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants