Repository navigation
SPI donor savings rebased to SPI's 2023-24 bands (savings follow-up to #1418) - #1428
juaristi22 wants to merge 27 commits into
Conversation
The SPI income stage now moves the 2022-23 tape's interest and other income to HMRC's published 2023-24 Table 3.7 amounts band by band, with the tape banded by its own total income, before any draw. The OBR's savings income growth then carries both columns to the frame's year, the path the Table 3.7 band targets take. Before, each column moved by its engine index from 2022 to 2024: other income by GDP per head (x1.09), interest by ONS household interest (x2.27). SPI's own amounts grew x1.65 and x2.69 in the single year to 2023-24. The frame carried about 70 percent of the savings rows at design weights, and the solve met them by stretching the few households that carry them. On the tape, the £1m-and-over band's other income outweighs its interest, as in the published table; the two indices had flipped that split. - uk/hmrc_spi_table_3_7_savings_rows.json vendors the 26 published rows from the feed pin, declared in the vendoring register and the country package. - spi_income.rebase_spi_donor_savings scales the prepared tape. A band the tape carries none of is left as it is and named in the receipt. The forest and the band-donor resample both read the rebased tape. - _spi_income_uprating_factors grows the two columns from 2023-24 on hmrc_uprating.uk_obr_savings_income_level. Both columns now run on the projection's GDP per head from the frame's year on (SPI_HISTORY_INDEX_DIFFERS_FROM_PROJECTION). - Tests cover the rebase, the published rows, the OBR factor and the stage wiring, and the engine lockstep tests are updated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first spine built from 667fe53 stopped at uk_stage_uc_take_up. Single people's caseload ended 0.0568 of DWP's count further from it than before the redraw, against 0.0523 on PR C's spine and a tolerance of 0.055. The rebased SPI savings feed the UC reporter redraw's investment-income predictor. The few SPI benefit units whose report changes move the take-up stage's pool merging, which joins adjacent award bands until each pool rests on enough sources. Single people's £1,000 to £1,100 award band moved from a reachable pool into one whose DWP target exceeds the entitled units. The reachable pools still realize DWP's counts within 0.005. The tolerance goes to 0.06 (María's ruling of 2026-10-30), as PR A took it from 0.05 to 0.055 for the gilt reshuffle. Pool boundaries that move with small upstream changes are a follow-up on #1334. The gate's note, the committed-parameter test and the data contract's mirrored digests are updated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The national built from f3cb3dd fitted every SPI Table 3.7 interest and other-income band but one, the £1m-and-over other-income row included. It missed the £1m-and-over interest amount row outside the uk_target_fit band. By María's ruling of 2026-10-30, the row is held out of the objective as a measured distortion, as the ten HMRC £1m-and-over cells are, and measured on every evaluation. The register entry carries no figures for the band. How the rebase sits in the top bands at design weights is a follow-up. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SPI Table 3.7 counts taxpayers only, but the rebase divided its amounts by the tape's amount over all records in each band. In the two lowest bands non-taxpayers hold much of the tape's savings income, so the rebased taxpayers reached only part of the published amounts there. The prepared tape now carries the tape's TAXPAYER flag. Each band's factor is computed over its taxpayers and applied to them; non-taxpayers take the all-bands factor, as donors below the first band already do. A tape without the flag counts every record, and the receipt says which basis it used. Option F of the #1428 other-income investigation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two FRS-row mappings disagreed with SPI Table 3.7, so the frame's other income below £100k carried the wrong concept. - NS&I Direct Saver and the other NS&I and bank products move from other_investment_income to savings_interest_income. SPI has counted all interest paid gross or net as bank interest since 2016-17 (note 4). Gilts stay in other income, as interest on securities (note 6). - Income from an annuity not bought with pension funds or from a trust or covenant (FRS PENSION.PENTYPE 4 and 5, the ADULT ANYPEN4 and ANYPEN5 sources) moves from private_pension_income to other_investment_income, the other income Table 3.7 counts (note 6). The FRS records the whole annuity payment, of which only the income element is taxable. Both columns are taxable savings income in policyengine-uk, so the savings tax on these rows does not move. The FRS spine test covers both routes. Options A1 and A2 of the #1428 other-income investigation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Other income was the one investment leaf of the SPI stage-1 transport with no FRS analogue. Its quantile came from an identity uniform keyed on the twin, which ignored the twin's holdings. Copies drew large amounts on modest incomes, and the draw was out of line with the rest of each copy's income. The analogue map now names the twin's own other_investment_income: gilts, trust income and purchased-annuity income, the FRS sources of SPI's other income after the previous commit. A copy's draw now ranks with its twin's holdings, as interest, dividends, pensions and property do. The stage spec's analogue_columns (sources.yaml and source_stages.json) and the release input coverage manifest that pins the spec follow, and the spine test's synthetic frame carries the column, as the FRS spine always writes it. Option C of the #1428 other-income investigation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings #1428 up to date with main, including PR B (#1421, the ISA subscriptions and policyengine-uk 2.164.2). Conflicts resolved: - frs_spine: main's ISA interest column, with this branch's NS&I-to- deposit routing. - The FRS spine test: ISA interest apart, plus NS&I as deposit interest (5.0 a week of non-ISA interest). - The SPI engine test: the savings columns on the OBR path. - The vendoring register: both new resources. - The coverage manifest: regenerated. - The data contract: gate digests recomputed for the merged gates.json (main's, plus the UC take-up tolerance of 0.06). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The band factors divide published amounts by the tape's FACT-weighted
amounts, so they assume the tape grosses to the taxpayer population.
The H2 spine fixture's synthetic tape doesn't: its factors ran to
millions, lifted its band donors' income out of their bands, and the
fixture's regeneration refused ("no record whose uprated total income
lies in the band from 200000"). CI's acceptance and staging smoke tests
would meet the same refusal.
Now a column whose banded tape amount, over the bands it carries, is
under a tenth of the published amount is left as it is, and the receipt
says so. The production tape holds about 0.37 of the published interest
and 0.6 of the other income. A test covers the skip.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI's engine-uk graph test and integration-uk smoke test refused the SPI income stage: the fixture's stage-spec snapshot still named no analogue for other_investment_income, against the reviewed map (C in b822474). tools/graph_uk_spine_fixture.py regenerates it on the merged tree: - fixture.json carries the new analogue entry; - the fixture's derived CGT and property fact sources follow the synthetic frame they're built from; - uk_spine.json's pin follows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI's engine test test_spi_stage2_does_not_require_frs_other_investment_income runs the SPI income stage on a frame whose FRS rows carry no other_investment_income. The stage fills that output with the stage-time zero, but only after the stage-1 draw. Since b822474 the transport reads the column as the twin's own analogue before the draw, so the stage refused the frame. Now a stage-1 output that is its own analogue and that the frame doesn't carry gets the same zero before the transport. Every twin locates as a non-carrier, and the final values are unchanged. Production FRS frames always write the column. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
e5ac7a5 built the fixture in a local environment still on policyengine-uk 2.162.14. The lock has pinned 2.164.2 since the merge of main (9ee6fbd), so CI's engine-uk graph test found the stamped engine version differ from the graph it builds. Rebuilt with the environment synced to the lock: the graph's version stamp moves, and so do the fixture's synthetic property rental facts, which the engine uprates. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's choice of 2026-10-30 (microcosm#1418, the trading allowance measures). With Table 3.10's self-employment counts alone, the solve can meet each GBP 1 to GBP 10,000 band with traders who have no pay. In the table, more than half of the traders below GBP 3,000 also have employment income, and an employee's side profit is taxed at their marginal rate. - hmrc.spi.employment_income.count_by_self_employment_income_band counts each person with pay over the year (employment_income_annual, which Table 3.6's employment rows read) in their own self-employment band, taxpayer or not, over the same eight bands as the self-employment counts. Table 3.10's column also holds miscellaneous income, benefits in kind and the contributory benefits, which the binding leaves out, as Table 3.6's does. The counts grow to 2025 with HMRC's projected growth of all taxpayers, as the self-employment counts do. - The zero band and the bands from GBP 10,000 are signed out row by row. The target is declared signature-incomplete beside the other Table 3.10 rows and is unmapped in the registry parity (no uk-data metric). - The national references and membership and the national compile-parity receipt are regenerated on the pinned feed (53815b2): eight new rows. The membership's 13 SPI dividend rows and 8 Table 3.10 self-employment amount rows also take the October EFO engine pins that main's policyengine-uk 2.164.2 lock brought (b89d3aa). The merge left those rows on the 2.162.14 pins. The parity receipt doesn't move for them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's choice of 2026-10-31 (microcosm#1428). The three Income Tax liabilities statistics Table 2.6 rows bound HMRC's own 2024-25 and 2025-26 projections as a calendar-2025 window: GBP 8.40bn of tax on savings, x1.52 on the 2023-24 outturn, against x1.36 for savings income on the OBR's path. With SPI Table 3.7's savings income already on the OBR path, those rows pinned the tax on savings, and so the savings rates' static yield, about 12% above the base the Budget's costing grows. - The rows now read the 2023-24 outturn alone. They are observed facts, so hmrc_itl_savings_rate leaves the source-projection families. - They grow by the OBR's savings income growth (October 2026 EFO Table 4.5), x1.3562 to calendar 2025, the index Table 3.7's savings rows take: GBP 1.61bn, 2.24bn and 3.66bn, GBP 7.51bn in all. - Growing the tax on the income's path leaves out the leverage that HMRC's projection carries from the frozen Personal Savings Allowance and thresholds. - The national references, membership and compile-parity receipt are regenerated on the pinned feed, and only the three rows move. The engine test's comments now name calendar 2025's two tax years. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's ruling of 2026-10-31 (microcosm#1428): fix the take-up pools instead of widening the gate again. The spine from d563f4f stopped at uk_stage_uc_take_up. Single people's caseload ended 0.0674 of DWP's count further from it than before the redraw: 0.0568 on f3cb3dd's spine, against this branch's 0.06. The reachable pools met DWP's counts. The gap was the pools whose DWP count exceeds every entitled unit, which kept a clipped rate, and which bands land there moves with small upstream changes. - After the source-support pooling, the stage joins a pool that cannot meet DWP's count to an adjacent pool (_join_unmet_pools). Such a pool's undrawn units' expected claims exceed the count, or the count exceeds them plus every drawn unit. Lowest first, it joins the neighbour with more room for its gap, the lower on a tie. Joining stops when every pool can meet its count or the family type is one pool. A family type whose count exceeds all its entitled units stays one unreachable pool. - The stage spec restates the pooling rule (sources.yaml, source_stages.json, UC_TAKE_UP_POOLING), and the coverage manifest and the H2 fixture follow. - uk_stage_uc_take_up returns to main's 0.055, undoing this branch's 0.06. The gate note, the committed-parameter test and the data contract's mirrored digests follow. - New tests: an unmet pool joins the neighbour with more room, or the lower one on a tie; an overshooting pool joins one with drawn claims; a family type that cannot meet its count is one unreachable pool, flagged by the side it misses on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's choice of 2026-10-31 (microcosm#1428). policyengine-uk applies
the trading allowance against self_employment_gross_receipts, which
neither the FRS nor the SPI tape records. Without receipts it relieved
profits up to the allowance in full and nothing above. So the Budget's
raise to GBP 3,000 missed its main base, the traders who already claim
the GBP 1,000 allowance against larger receipts.
- A new last spine stage, trading_allowance_receipts, flags each trader
with a positive profit as a claimant, drawn on their own person_id:
0.64 for profits up to GBP 4,500, 0.32 from GBP 4,500 to GBP 9,000,
none above.
- It stores receipts of profit plus GBP 1,000 for a claimant, exact
under current law, and profit plus GBP 3,000 for any other trader
(expenses at the raised allowance; the costing books switchers as
behaviour). A person with no positive profit keeps zero.
- The rates are calibrated to HMRC's policy paper (about 400,000
taxpayers with a lower bill, about GBP 380 on average) until HMRC
publishes claimant counts.
- Every trader with a positive profit is treated as filing, as the
bound Table 3.10 counts are of filers. So a profit of GBP 1,000 or
less is now taxed.
- Wiring:
- the operation kind in the stage schema and the allowed kinds;
- the stage in sources.yaml and source_stages.json, with its mass
receipt;
- the graph reads and cell, the kernel modules, the spine build and
fixture implementations, and the coverage manifest family.
- self_employment_gross_receipts leaves UK_ENGINE_DEFAULT_INPUTS and
joins the export surface's allowed columns (terminal_gates and
gates.json), so the data contract's gate digests follow.
- Tests: the stage's own, and the rosters (51 stages). The H2 fixture
follows.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This reverts commit 5c0137f (María's ruling of 2026-10-31, microcosm#1428). The national built from 6485af4 stopped at uk_target_fit. Under the engine's rules, tax on savings grows faster than savings income: the Personal Savings Allowance and the thresholds are frozen. So the tax rows grown on the income's path asked for less tax than SPI Table 3.7's savings income gives. To meet them, the solve cut the interest rows in the basic-rate bands: the GBP 30,000 band ended 30.7% under target and the GBP 20,000 band 12.0% under. HMRC's own 2024-25 and 2025-26 projections carry that leverage, and the national from b441bb5 met them beside the interest rows. So the savings rates' yield stays pinned to the projection. Its distance from the costing is a difference of basis (the projection's leverage) and of behaviour (ISA shifting), not a fault in the base. The three rows, their references, membership and parity receipt, and the two tests are back as at d563f4f. The changelog fragment goes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The spine from f9676fe stopped in trading_allowance_receipts, and so did CI's graph driver test and the staging smoke test. The stage counts its receipt's support by household_source_id, but its graph declaration read only self_employment_income, so the executor didn't hand it that cell. The stage now declares the read, as regional_home_values and property_purchases do, and the H2 fixture's graph follows. The fixture tool's legacy oracle runs every stage on the whole frame, so it didn't catch this. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The receipts stage now runs after the ISA subscriptions, so the two tests that pinned the ISA subscriptions as the last spine stage name both. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The stage fits no model, but it declared an empty fit_weight_records tuple. The stage evidence then carried an empty list for it, and the release-cut weights audit reads an empty list as a fitting stage that recorded nothing, so every national certified after #1421 failed uk_weights_audit. A regression test reads the stage's evidence. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Gift Aid and charitable investment gifts are sparse leaves that the stage-1 chain draws after, and conditioned on, the income leaves. Rebasing the donor savings on the tape's taxpayers changed which few donors' gifts the copies drew, and the release-cut input-mass check found charitable_investment_gifts beyond its band. Each column's total over the drawn copies, at the stage's household weights, is now scaled to the prepared tape's FACT-weighted total (María's ruling of 2026-10-31). The stage evidence records each column's factor only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's ruling of 2026-10-31 (microcosm#1428). The release-cut input-mass check on the national from 0c2f531 found jsa_income_reported beyond its band. The national met OBR's total JSA by scaling income-based reporters together with contributory claimants, but income-based JSA closes with the move to Universal Credit: DWP's Spring Forecast 2026 puts it at GBP 91.0m in FY2024-25 and under GBP 1m from FY2025-26. - dwp.jsa.amount_income_based binds DWP's income-based expenditure line on jsa_income over Great Britain, at the calendar-2025 window of the two fiscal years, as the pension-age Housing Benefit row is. The solve now meets the rest of OBR's total on contributory claimants. The target is unmapped in the registry parity (no uk-data metric). - The national references and membership and the national compile-parity receipt are regenerated on the pinned feed (53815b2): one new row, resolved at GBP 23.4m. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measurements: the certified national from
|
| Measure | Years | Part 1 (b441bb5a0) |
This PR (a8e25870c) |
OBR line | Table 6.1 |
|---|---|---|---|---|---|
| Trading allowance to £3,000 | 2028 to 2031 | −0.097 / −0.097 / −0.093 / −0.097 | −0.148 / −0.149 / −0.148 / −0.147 | −0.221 / −0.165 / −0.171 / −0.178 | −0.225 / −0.165 / −0.175 / −0.180 |
| Savings income rates +2pp | 2029 to 2031 | +0.610 / +0.646 / +0.678 | +0.582 / +0.611 / +0.636 | +0.443 / +0.475 / +0.510 | +0.520 / +0.555 / +0.590 |
| Property income rates +2pp | 2029 to 2031 | +0.422 / +0.438 / +0.451 | +0.402 / +0.417 / +0.429 | +0.394 / +0.403 / +0.412 | +0.435 / +0.445 / +0.455 |
| Carer's Allowance earnings taper | 2030, 2031 | −0.038 / −0.075 | −0.037 / −0.073 | — | −0.040 / −0.080 |
| Domestic electricity at zero VAT | 2026, 2027 | −0.789 / −0.332 | −0.788 / −0.332 | — | −0.755 / −0.305 |
| Fuel duty rise deferred | 2026, 2027 | −0.290 / −0.501 | −0.290 / −0.501 | — | −0.575 / −0.780 |
What moved
- Trading allowance. The receipts stage lifts the cost by about half, to about HMRC's static base: about 400,000 taxpayers with a lower bill, by about £380 on average. The rest of the costing is behaviour, traders who switch to the allowance, plus miscellaneous income, which this scoring environment doesn't relieve (policyengine-uk#2726).
- Trading allowance, against the OBR line. From 2029-30 the cost trails income tax plus NICs by 10–17%. 2028-29 is the first Self Assessment cash year, with about 1.5 years of liability.
- Savings rates. The yield is 25–31% above the OBR's income tax line (part 1: 33–38%). That compares a static figure with a line after behaviour.
- This PR pins the 2025 tax on savings to HMRC's Table 2.6 projection, by rate (0.98, 1.00, 1.00). On that base, +2 points gives £0.48bn in 2025 and £0.58bn in 2029-30.
- EFO paragraph 3.17 gives the measure about £0.6bn a year after a savers' response of around £0.2bn, mainly ISAs. That implies a static yield of roughly £0.75bn across all taxes, against this PR's static income tax of about £0.61bn.
- The split of that response by tax isn't published. So the gap reads as at least half behaviour, probably most of it, with Max asked to verify.
- The earlier reading here, a difference of base level with ISA shifting small, is withdrawn.
- Behaviour belongs in policyengine-uk, not in this dataset, so no data change is proposed for savings.
- Property income rates. The yield is about 5% below part 1 and 2–4% above the OBR's income tax line.
- Against part 1 the 2029-30 yield falls £20m:
- £8m is extra finance-cost relief. Nothing binds landlords' finance costs, and this branch moved them toward HMRC's published level.
- £10m is a lower tax rate on property income outside Scotland.
- £2m is more property income in Scotland, where the rate doesn't rise.
- Against the income tax line there is no shortfall to explain. The partnership rescale in Certify the Microcosm UK national from microcosm#1428 on policyengine-uk 2.164.2 (0a4c8e2c; local area 6528c865 or c9df4b86) policyengine.py#731 is withdrawn, and the PRIS bounds stay a lead (comment).
- Against part 1 the 2029-30 yield falls £20m:
- The other measures barely move.
- Certification fixes. They move none of these measures by more than £0.001bn against the uncertified build from
0c2f5314f. - The pension tax allowance measure fails in the scoring template on policyengine-uk 2.162.14, not on the data.
🤖 Generated with Claude Code
Independent review of #1428 at
|
| Allowance | Deduction 2027-28 | Deduction 2028-29 |
|---|---|---|
| £3,000 | £0 | £0 |
| £3,500 | £296 | £183 |
| £4,000 | £796 | £683 |
| £5,000 | £1,796 | £1,683 |
- Before this PR they got nothing at any of these levels. A £50,000 trader's 2028-29 income tax at £5,000 falls from £9,547 to £8,874. So any raise beyond the Budget's £3,000 is mis-scored; a rough estimate from public counts is hundreds of millions to about £1.5bn a year at £4,000 to £5,000.
- Later conflict: once pe-uk#2765's claim status lands, any non-claimant full relief it computes would also read the stored expense.
Merge hazards
3. The empty fit-weight-records failure is a class, and #1430 brings it back.
- The chain:
b9bcd709cfixes the ISA stage, but any stage that declares the hook and returns()fails the same way:- it is snapshotted as empty (
uk_runtime/stage_evidence.py:69-79); - it is rehydrated to
()(uk_runtime/release_certification.py:216-219); - it then fails the release-blocking
uk_weights_audit(uk_runtime/battery_bindings.py:1469-1471), with no exemption.
- it is snapshotted as empty (
- UK: company cars from the FRS job table (dashboard #316) #1430: its
company_carsstage (company_cars.py:1373-1375at40968ff53) returns(). A probe with its real class fails the audit, and the two PRs merge without conflict in either order. - The same class elsewhere:
nts_bus_travel's published-shares fallback stores();uc_reporter_redraw,salary_sacrificeandspi_housing_shellmake weighted fits with no hook, so the audit never sees them.
- Suggestion: a roster-wide test that each stage either fits and records or declares nothing.
4. With #1422, the new receipts test fails (packages/microcosm-build/tests/engine_free/uk/test_uk_trading_allowance_receipts.py:121-125).
- It asserts the literal "<10" for five traders. The file is new, so it merges without conflict, and then UK: take up #1095's actionable items on policyengine-uk 2.165.1 #1422's floor of three makes it fail (
'3-9' == '<10'). test_uk_uc_take_up.py:254sits in a conflict hunk that has to be rewritten by hand.- Fix: build the expected labels with
rounded_count.
5. With #1426, the UC take-up gate silently becomes 0.12 (uk/gates.json:638).
- The merge:
maximum_caseload_deviationmerges to UK #1095 A/B fixes: the SPI age band (uk-data#628), households' HRAD from Table 8a receipts, and the SPI pay floor (uk-data#629) #1426's 0.12 outside the conflict markers. Only the notes conflict, and this PR's note says 0.055. So keeping this PR's note ships 0.12 under a text that says 0.055. - What it needs: UK #1095 A/B fixes: the SPI age band (uk-data#628), households' HRAD from Table 8a receipts, and the SPI pay floor (uk-data#629) #1426 scopes 0.12 to the UC award fix of UK: every open data item: uk-data parity, engine locks, the local surface, consumption and high-value homes #1095, and the pool join (7) changes what the gate measures. The value needs a ruling and a re-measured spine.
Modelling
6. The rebase counts band-edge migration twice (uk_runtime/spi_income.py:1925-1926).
- Mechanism:
- Each factor is the published 2023-24 amount, banded on 2023-24 income, over the tape banded on its 2022-23 nominal income. So it absorbs net migration across band edges into the open top bands.
- Every consumer then bands again on uprated income, and counts that migration again: the band-donor pools (
:1471) and the Table 3.7 targets.
- Synthetic tapes: the rebased frame's £1m-and-over interest comes to between 1.02 and 1.35 times published, depending on the tail. The no-growth control gives exactly 1. The £200k–£1m bands also over-carry, and narrow bottom bands under-carry.
- Suggestion: band the tape on income moved to 2023-24, or iterate the factors until the rebased tape, banded on its own uprated income, reproduces the published amounts.
7. The UC pool join moves claims the wrong way between award bands (uk_runtime/uc_take_up.py:547).
- Mechanism: a joined pool applies one realized rate to every drawable unit in both bands. So an overshooting band's drawn units start claiming, and a short band's stop.
- Real stage function, join off and on:
| Case | Join off (claims per band) | Join on (claims per band) | UC spending |
|---|---|---|---|
| A band overshooting on reporters | 200 / 200 | 280 / 70 | −39% |
| A band short of its count | 60 / 100 | 38 / 162 | +45% |
| Reporters above the family total | 150 / 80 | 150 / 0 |
- Why it matters: the gate checks family totals only, but 92 of the 100 family-by-band payment rows bind in the national solve.
- Suggestion: holding the unmet band at its clip and passing its gap to the neighbour gives the same family total, with less per-band error.
8. The receipts encoding also misplaces who gains (uk_runtime/trading_allowance_receipts.py:185-189).
- Full relief: drawn non-claimants whose real receipts are within the allowance lose full relief, which ITTOIA s. 783AE gives automatically. Only partial relief needs the s. 783AM election. With the rates calibrated to HMRC's 400,000 gainers, their gain moves onto drawn claimants.
- Uprating drift: a claimant's £1,000 grows with the index, so the raise's extra relief is £1,932 in 2027-28, falling to £1,758 in 2031-32. That is 3.4% to 12.1% below £2,000, not "slightly", the wording at
:35-37, in the stage spec and in the body. - Capital allowances: the profit + £1,000 identity also fails for SPI copies with capital allowances, which can't have claimed partial relief.
9. The gift anchor fixes the total but not the concentration (uk_runtime/spi_income.py:2851-2852).
- Mechanism: one scalar per column scales every SPI row. So if a few extreme draws dominate a column, they still dominate it, and every other carrier shrinks.
- Example (synthetic): with one large draw holding 98% of the mass, ordinary carriers fall from £1,000 to about £16.
- Coverage: band carriers at full Table 2.5 counts and support copies at half coverage are scaled alike.
- Why it matters: Gift Aid works per person, through rate-band extension and adjusted net income, and no target binds gifts. The stage evidence records factors only, so concentration in the real build can't be read from it.
10. The taxpayer basis double-counts, and the guard falls back silently (uk_runtime/spi_income.py:1933, :1954-1962, :2066-2069).
- Two double counts:
- The factor divides 2023-24 published taxpayers by 2022-23 tape taxpayers, and also scales 2022-23 non-taxpayers by the all-bands factor. So savers who became taxpayers count twice: two donors, one a non-taxpayer, who are both taxpayers next year give 800 where the truth is 400.
- Table 3.7's lowest band includes taxpayers below the Personal Allowance (note 8), and they are double-counted the same way.
- Latent fallbacks:
- a column the grossing guard skips stays at 2022-23 values, yet is still grown from 2023-24 under the published-bands label;
- the guard is per column, so a thin top band can get a factor near a million;
is_taxpayeris always written, so the receipt always says "taxpayers";- TAXPAYER is neither a required column nor validated.
11. The new employment count sits outside the support floor (uk/calibration_measure_exclusions.json:34-45).
- The gap: the floor's counts scope lists Table 3.10's self-employment rows, but not the new employment count. Yet its reason (
:76) says Table 3.10's bands are dropped and receipted, not refused. - Effect: a zero-support employment cell would abort the solve's initialisation, or fail
uk_target_fit, instead of being dropped. It's latent.
Tests and texts
12. The new targets and claimant rates have no tests or evidence.
- Targets: in-memory mutations pass all fifteen engine-free target test files. These are changing the JSA row's
value_variable, dropping the employment filter, and removingband_upper_bound. Comparetest_uk_pension_age_benefit_targets.py:88-111for the Housing Benefit sibling. - Claimant rates: 0.64 and 0.32, and the £4,500 and £9,000 edges, are calibrated off-repo. No script or evidence is committed, and no gate re-measures gainers or the average gain.
13. Texts:
- the £1m-and-over interest hold tracks this PR (
uk/calibration_measure_exclusions.json:177), so its expiry message will point to a merged PR; docs/uk-income-anchors-280.md::65-69contradicts:87;:88-90leaves out the taxpayer basis and the guard;:401-402describes the old other-income analogue;
tests/engine/uk/test_uk_hmrc_uprating.py:125-149describes the removed donor path;- the "NS&I and gilt" texts are now stale:
uk_runtime/spi_support.py:139-143, which is also the stated reason other income stays out of the stage-2 predictors;uk_runtime/spi_spine.py:218-219;uk/uk_population_targets.json:14875and:22071;
- the employment count's note (
:3901) says benefits in kind are excluded, but SPI rows count PAY + EPB + TAXTERM; - the changelog fragments describe branch history, including a 0.06 tolerance that never shipped, and fixes to code that never shipped;
uk_runtime/frs_spine.py:750reads PENTYPE with a silent zero, where the file refuses other missing columns. The H2 fixture has no PENTYPE column, so it never exercises the new routing.
Checked and found true
- The option A revert is exact:
git diff 5c0137fb4^ f9676fed9is empty over all seven files option A touched. - The vendored Table 3.7 rows match the pinned feed: 26 rows, and the sha matches.
- The eight employment-count rows check out: their uprating, signed exclusions, membership and parity rows. So does the JSA window: income-based plus contributory comes within 0.1% of OBR's total.
- Contract and ledger: the contract digests, the coverage manifest and the ledger accounting are consistent.
- The receipts stage's wiring is consistent, and the claimant draw is deterministic and independent of other draws.
- Merging with current main is clean, and UK: take up #1095's actionable items on policyengine-uk 2.165.1 #1422's floor-of-ten guards find nothing in this PR's added lines.
- CI is green at
a8e25870c.
Disclosure-control items went to the author separately.
🤖 Generated with Claude Code
The #1428 review's modelling findings 1, 6, 7 and 10. - Annuity and trust payments (FRS PENTYPE 4 and 5) stay in private_pension_income, where policyengine-uk's means tests, household income and tax read them. A calibration-only descriptor, annuity_and_trust_income, stores their part, and SPI Table 3.7's other-income rows read other_investment_income_spi_concept: the engine's other income plus that share of its private pension. Rows the SPI stage draws carry a zero descriptor. The FRS spine refuses a pension tab without PENTYPE. - The SPI savings rebase bands the tape on its income moved to 2023-24, repeating until no record crosses a band edge, so consumers that re-band on uprated income count the migration once. A taxpayer is a record the tape flags, or one whose income crosses the first band by 2023-24, and flagged taxpayers below the first band sit in it. A band under a tenth of its published amount takes the all-bands factor, a column the rebase skips grows on its engine index from 2022-23, and TAXPAYER codes other than 1 and 2 are refused. - The UC take-up stage holds a pool that can't meet DWP's count at its clip and passes its gap to neighbouring pools, offsetting opposite gaps first, so every band keeps its own rate. The export surface allows the descriptor, and the release-cut contract digests follow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The #1428 review's findings 3, 4, 8, 11, 12 and 13. - A roster test pins the eight stage transforms that declare fit_weight_records, read from the source, so a stage that declares the hook and records nothing can't reach the weights audit unreviewed (3). - The receipts test builds its small-cell labels with rounded_count, so it holds whatever floor main sets (4). - The trading allowance receipts stage states its two limits precisely: the raise relieves £1,932 more in 2027-28, falling to £1,758 by 2031-32, and the other traders' stored £3,000 mis-scores allowances above £3,000. Both wait on policyengine-uk#2765's claim status (microcosm#1439) (8). - The support floor's counts scope takes Table 3.10's employment count by self-employment band (11). - Tests pin the income-based JSA and employment-count rows (12). - Texts: the £1m-and-over interest hold tracks microcosm#1418; the income anchors doc, the uprating test and the NS&I comments describe what ships; the employment count's note says SPI rows count benefits in kind; the changelog fragments describe the change, not the branch's history (13). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The synthetic H2 fixture's pension rows now carry PENTYPE, so the spine fixture exercises the annuity and trust routing; the spine and the coverage manifest's source digests follow the two commits before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The rebase banded the tape on its income moved to 2023-24 with each record's savings at its own band's factor, repeating until no record changed band. That has no fixed point: the band sets the factor and the factor the band, and on the production tape records kept changing band from pass to pass, so the spine from 59ef8ec took the twentieth pass. The tape is now banded once: each income leaf at its 2022-23 to 2023-24 factor and each rebased column at its factor over all bands, on the tape's 2022-23 bands. Banded so, the rebased tape reproduces the published amounts band by band, and the band-edge migration the consumers re-band on uprated income is counted once. A record whose band's factor differs from all bands' can still sit across an edge on its rebased income. The receipt records each column's banding factor in place of the pass count. A column the tape carries none of takes a factor of 1, as the all-bands factor does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
91775db added annuity_and_trust_income to the export surface in gates.json and moved the data contract's release-cut part digests, but not the full-manifest pins the release contract checks a certificate's spec against (policy, gates manifest and spec fingerprint), nor their mirror in the contract tests. The national from 9f70c71 certified, and its release directory then failed the contract on those three pins. They now match the committed spec, as f3540cb moved them for the receipts column. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Drop the stage's empty fit-weight records, which the release-cut weights audit read as a fitting stage that recorded none (as #1428 does for ISA), with a test on the stage's evidence. - Cap the tape rows' carve at EPB within the pay the employee pension contribution leaves, so salary_sacrifice never meets a contribution above pay. The review's two probes are tests, one of them running salary_sacrifice on the stage's output, and the invariants count a contribution above the carved pay. - Take a carved row's UC gainful self-employment again on its carved pay, by the root's rule; the flag is a declared output and rewrite. - Draw the unknown fuels to Table 2C together: the holders without private fuel draw electricity at Table 2C's share over the share the private-fuel holders leave. - Say the calibrated-file comparison with Tables 1, 3A and 4 is a manual step (doc, stage note, fact fence, BIK purpose text, gate note). - Restate the obr.ni_employer exclusion for the ni_class_1_employer binding, with a test pinning both; give obr.fuel_duties the edition map; give the council tax nation rows a parity reason that names the main-residence scaling. - Restate the stale texts: the gate note and stage health, the salary_sacrifice comment, the ISA doc, the docs that counted 19 mapped rows, the engine tests and the doc's baseline-only sentences. Gate digests re-pinned. Generated surfaces follow in the next commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The review of
|
| Measure | Years | a8e25870c |
9f70c71e7 |
OBR line | Table 6.1 |
|---|---|---|---|---|---|
| Trading allowance to £3,000 | 2028 to 2031 | −0.148 / −0.149 / −0.148 / −0.147 | −0.173 / −0.175 / −0.173 / −0.170 | −0.221 / −0.165 / −0.171 / −0.178 | −0.225 / −0.165 / −0.175 / −0.180 |
| Savings income rates +2pp | 2029 to 2031 | +0.582 / +0.611 / +0.636 | +0.587 / +0.615 / +0.640 | +0.443 / +0.475 / +0.510 | +0.520 / +0.555 / +0.590 |
| Property income rates +2pp | 2029 to 2031 | +0.402 / +0.417 / +0.429 | +0.414 / +0.429 / +0.441 | +0.394 / +0.403 / +0.412 | +0.435 / +0.445 / +0.455 |
| Carer's Allowance earnings taper | 2030, 2031 | −0.037 / −0.073 | −0.035 / −0.070 | — | −0.040 / −0.080 |
| Domestic electricity at zero VAT | 2026, 2027 | −0.788 / −0.332 | −0.789 / −0.331 | — | −0.755 / −0.305 |
| Fuel duty rise deferred | 2026, 2027 | −0.290 / −0.501 | −0.290 / −0.500 | — | −0.575 / −0.780 |
What moved:
- Trading allowance: the cost rises about a sixth, all of it income tax.
- NICs don't move. Every drawn claimant's profit stays under Class 4's lower profits limit, so only income tax changes.
- The rise sits in the SPI copies. Their income draws read the rebased tape, and more of them gain, by more. FRS rows don't move.
- Against HMRC's policy paper: on this national the raise gives about 436,000 taxpayers a lower bill in 2028-29, by about £407 on average. HMRC gives about 400,000 and £380, and
a8e25870cgave 409,000 and £370. - So the claimant rates now overshoot. They were calibrated on
b441bb5a0, and on this national they overshoot HMRC's static base by about a sixth. That, not a closed behaviour gap, is why the cost now meets the OBR's income tax plus NICs line from 2029-30. - UK trading allowance: store the claim status once policyengine-uk#2765 is released #1439 makes re-measuring gainers after each build a committed check.
- Savings rates: up under 1%, still 25–33% above the OBR's income tax line. The tax on savings in 2025 sits at 0.99, 0.99 and 1.00 of HMRC's Table 2.6 projection, by rate. Max's check on how much of that gap is behaviour still stands.
- Property rates: up about 3%, to 5–7% above the OBR's income tax line. The rise also sits in the SPI copies: their property draws read the rebased tape, and they carry more property income. FRS rows barely move.
- The other measures barely move.
- The pension tax allowance measure still fails in the scoring template on policyengine-uk 2.162.14, not on the data.
Fix before the national is used
1. Annuity and trust income: back in pension income (91775db4a). Done as you suggested.
- FRS PENTYPE 4 and 5 stay in
private_pension_income, so the means tests, household income and tax read them as they did before this PR. - A calibration-only descriptor,
annuity_and_trust_income, stores their part. The export carries it. - Table 3.7's other-income rows read a new measure,
other_investment_income_spi_concept: the engine's other income plus a share of the engine's private pension. The share is each person's descriptor over their stored private pension, so a reform that changes private pension moves the measure with it. - Rows the SPI stage draws carry a zero descriptor. Their private pension is the tape's, which books these payments as other income.
- The FRS spine now refuses a pension tab without PENTYPE. The H2 fixture's pension rows carry PENTYPE, so the fixture exercises the routing.
- The body's "the tax doesn't move" now covers NS&I only, which stays in interest.
2. Non-claimants' receipts above £3,000: kept and documented, with a follow-up.
- María kept the encoding for now. It scores the Budget's £3,000 measure, and storing the claim status from Add a trading allowance claim status, so allowance reforms reach claimants without turnover data policyengine-uk#2765 will replace it once that is released.
- The stage spec, the module and the changelog state the limit. Any allowance above £3,000 × the index relieves every non-claimant from the stored expense, so such reforms shouldn't be scored on this national.
- UK trading allowance: store the claim status once policyengine-uk#2765 is released #1439 tracks storing the claim status once #2765 is released, which removes the limit.
Merge hazards
3. Empty fit-weight records: a roster test (24e2329f6).
-
test_uk_fit_weight_records_roster.pyreads eachuk_runtimemodule's source. It pins the eight stage transforms that declarefit_weight_records. -
UK: company cars from the FRS job table (dashboard #316) #1430's
company_carsstage fails it on merge. Its author then has to record each fit or drop the hook. -
Not fixed here:
- the NTS stage's published-shares fallback still stores
(). It's latent, because the production donor takes the interview path; uc_reporter_redraw,salary_sacrificeandspi_housing_shellstill fit without the hook, so the audit doesn't see them.
Both are follow-ups for after this PR. A run that made no fit should say so in a way the audit accepts, and the three stages should record their fits.
- the NTS stage's published-shares fallback still stores
4. The receipts test's labels (24e2329f6).
- They're built with
rounded_count, so the test holds under either floor. test_uk_uc_take_up.pychanged again for finding 7, so its conflict with UK: take up #1095's actionable items on policyengine-uk 2.165.1 #1422 still needs resolving by hand at that merge.
5. The UC take-up gate with #1426: not changed here.
- The value needs a ruling when the two PRs meet. The body's "Not in this PR" says so.
- On the spine from
9f70c71e7, the UC take-up gate passes at 0.055.
Modelling
6. Band-edge migration: counted once (91775db4a, 9f70c71e7). This is your first suggestion.
- The tape is banded on its income moved to 2023-24. That is each income leaf times its 2022-23 to 2023-24 factor, leaving out finance costs and the loss draws, plus any part of total income the leaves don't itemise.
- Iterating, your second suggestion, didn't converge on the production tape.
91775db4amoved each record's savings by its own band's factor and repeated the banding. Records kept changing band from pass to pass, because a record's band sets its factor and the factor sets its band. The spine from59ef8eceetook the twentieth pass, and its receipt said it hadn't converged. 9f70c71e7bands once. Each savings column moves by its factor over all bands, on the tape's 2022-23 bands. Banded so, the rebased tape reproduces the published amounts band by band. A record whose band's factor differs from all bands' can still sit across an edge on its rebased income.- I checked this locally against your measure: the tape's rebased savings, re-banded on each record's own rebased 2023-24 income, against the published bands. The single pass came closer than both the 2022-23 banding and the twentieth pass, with the top bands near their published amounts. The figures rest on the tape, so they stay out of this thread.
- The receipt records each column's banding factor in place of the pass count.
- Tests:
- one builds a tape whose income grows across an edge, and checks the record is banded where the consumers will find it;
- another checks that a saver whose savings are a large part of their income is banded on them.
7. The UC pool join: replaced by passing gaps (91775db4a). This is your suggestion.
- A pool that can't meet its count is held at its clip, and its gap passes on.
- Opposite gaps in the family type offset first. What remains goes to the other pools, nearest first, each taking what its room allows. At equal distance the pool with more room goes first, and on a tie the lower band.
- Every band keeps its own rate.
- The tests now cover an overshooting pool, a short pool, and gaps that offset before passing.
8. Who gains, and the drift (24e2329f6, #1439).
- The module, the spec and the changelog give the drift exactly: £1,932 more in 2027-28, falling to £1,758 by 2031-32 (3.4% to 12.1% below £2,000). That replaces "slightly".
- Full relief for drawn non-claimants whose receipts are within the allowance moves with the claim status in UK trading allowance: store the claim status once policyengine-uk#2765 is released #1439. So do SPI copies with capital allowances, which can't have claimed partial relief. The frame doesn't carry capital allowances today, so that needs a flag from the SPI draw. Both are now noted on UK trading allowance: store the claim status once policyengine-uk#2765 is released #1439.
9. The gift anchor's concentration: acknowledged.
- The changelog says the anchor fixes totals, not concentration.
- The stage evidence keeps factors only. A concentration statistic for columns resting on few SPI records would be disclosive.
- A published total for the charity reliefs on gifts of shares and property would let a target bind them. That is in "Not in this PR".
10. The taxpayer basis and the guard (91775db4a).
- A taxpayer is a record with TAXPAYER 1, or one whose income crosses the first band by 2023-24. So a saver who becomes a taxpayer counts once.
- A flagged taxpayer below the first band sits in it (note 8).
- A band under a tenth of its published amount takes the all-bands factor, and the receipt names it.
- A column the grossing guard skips grows on its engine index from 2022-23, not on the published path.
is_taxpayeris written only when the tape has TAXPAYER, and the receipt's label follows it.- TAXPAYER codes other than 1 and 2, or blanks, are refused.
11. The support floor (24e2329f6). Its counts scope now takes Table 3.10's employment count by self-employment band. A cell without support is dropped and receipted.
Tests and texts
12. Target tests (24e2329f6).
- New tests pin the income-based JSA row's selector, window and binding, and the employment count's variable, grouping, filter and bands.
- The claimant rates' tool and a post-build gainers check are in UK trading allowance: store the claim status once policyengine-uk#2765 is released #1439's "Done when".
13. Texts (24e2329f6).
- The £1m-and-over interest hold now tracks UK: data fixes for the Budget 2026 household measures #1418.
- The income anchors doc no longer contradicts itself. It covers the taxpayer basis and the guard, and its other-income analogue is gilt interest.
- The uprating test describes the two-leg savings factor.
- The NS&I and gilt comments, and the two target notes, describe what ships.
- The employment count's note says SPI rows count benefits in kind (PAY + EPB + TAXTERM).
- The changelog fragments describe the change, not the branch's history.
- The FRS spine refuses a missing PENTYPE (see 1).
🤖 Generated with Claude Code
The rates calibrated on b441bb5 (0.64 for 2024-25 profits up to £4,500, 0.32 to £9,000) give about 436,000 taxpayers with a lower bill in 2028-29 on the national from 9f70c71, by about £407 on average, against the policy paper's 400,000 and £380: the SPI copies' small traders moved with the savings rebase. Measured on policyengine-uk on that national, the rates that meet both figures are 0.72 and 0.14 (María's choice of 2026-11-01). The draw keeps each trader's uniform, so a lower rate keeps a subset of the claimants and the mean gain holds. The rates drift with each national; microcosm#1439 re-measures them after each build. The H2 fixture and the coverage manifest follow with the merge of main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The savings follow-up to #1418, part 1: fix the low savings base at its source instead of making the solve stretch weights. This branch starts from PR C's last commit (
0b276e571, merged as #1420).Draft. The national built from
9f70c71e7is certified (cutfd0d4bc2): all 50 spine gates, 7 calibration-seam gates and 19 release-cut gates pass. Its release directory assembled once4dd83e999moved the contract's pins, and the publisher pre-flight passes. It is staged, not published. The reply to the independent review has its measurements, and an earlier comment hasa8e25870c's, the cut (0a4c8e2c) PolicyEngine/policyengine.py#731 still bundles.91775db4ato4dd83e999address the review (below).Earlier builds:
d563f4fbcstopped at the UC take-up gate.6485af4afpassed every gate, but its national stopped at the fit gate on the tax on savings rows.f9676fed9stopped in the new receipts stage, which hadn't declared its source-household read;0c2f5314fdeclares it.0c2f5314fstopped at the release-cut certification on three gates;3702769f0toa8e25870cremove their causes (below).a8e25870ccertified. The review's fixes move the data, so the build from9f70c71e7replaces it.59ef8eceepassed every gate, but its savings rebase hadn't converged (below).9f70c71e7bands the tape once, and the national from59ef8eceewas stopped.9f70c71e7certified. Its release directory failed the data contract on three pins that91775db4ahadn't moved;4dd83e999moves them, and assembly and the pre-flight pass.The problem
PR C binds SPI Table 3.7's interest and other income by total-income band. Interest moves to the OBR's savings path from 2023-24.
The PR C national built from
5aba83cd2stopped at certification on that band. Its figures stay out of this thread, because the band rests on the SPI tape.The change
spi_income.rebase_spi_donor_savingsruns before any draw. It scales each donor's interest and other income to HMRC's published 2023-24 Table 3.7 amounts:9f70c71e7). A record whose band's factor differs from all bands' can still sit across an edge on its rebased income;The stage-1 forest and the band-donor resample both read the rebased tape. The receipt records factors only, never counts.
_spi_income_uprating_factorsthen grows the two columns from 2023-24 to the frame's year on the OBR's savings-income growth (October 2026 EFO, Table 4.5), through the newhmrc_uprating.uk_obr_savings_income_level. This is the path the Table 3.7 band targets already take. A column the rebase leaves as it is grows on its engine index from 2022-23 instead (since91775db4a). From the frame's year on, the engine projects both at GDP per head, so both columns are now listed inSPI_HISTORY_INDEX_DIFFERS_FROM_PROJECTION.uk/hmrc_spi_table_3_7_savings_rows.jsonvendors the 26 published rows (13 bands × 2 columns) from the feed pin. It is declared in the vendoring register and the country package.Docs and changelog:
docs/uk-income-anchors-280.mdandchangelog.d/1418-spi-donor-savings-rebase.changed.md.The UC take-up gate: 0.055 to 0.06 (second commit)
The first spine built from
667fe532astopped atuk_stage_uc_take_up. After the take-up redraw, single people's caseload ended 0.0568 of DWP's count further from it than before. PR C's spine was at 0.0523, and the tolerance is 0.055. The cause:The reachable pools still realize DWP's counts within 0.005. The tolerance goes to 0.06 (María's ruling of 2026-10-30), as PR A took it from 0.05 to 0.055 for the gilt reshuffle. Pool boundaries that jump with small upstream changes are a follow-up on #1334. The gate's note, the committed-parameter test and the data contract's mirrored digests are updated.
Superseded by
6485af4af, which fixes the pools and returns the gate to 0.055 (below).The national from
f3cb3ddd3: what movedThe spine passed every gate. The national's fit gate blocked on one row. Below £1m, every Table 3.7 interest and other-income band fits within 1% after the solve, and the £1m-and-over other-income row that blocked PR C's national now passes the fit gate.
The share of each group's solved mass sitting on households stretched beyond 3× their design weight, PR C's blocked national against this one:
The last row is the FRS under-reporting part 2 would address.
At design weights, the rebased frame carries more savings than SPI in the £200k–£1m bands, where its top earners are already over-represented, so the solve now cuts those savers' weights instead. The figures for the £1m-and-over band stay out of this thread, because it rests on the SPI tape.
The £1m-and-over interest hold (third commit)
The one row the fit gate blocked on, Table 3.7's £1m-and-over interest amount, is held as a measured distortion, as the ten HMRC £1m-and-over cells are (María's ruling of 2026-10-30), and is still measured on every evaluation. How the rebase sits in the top bands at design weights is a follow-up.
Why other income is short below £100k, and the fixes (fourth to sixth commits)
A read-only investigation of the national from
f3cb3ddd3found that below £100k the frame carries 0.40 of Table 3.7's other income at design weights. It ranked three causes:By María's choice (2026-10-30), three changes address the first two, plus a fix to part 1's rebase:
4a6dc728f, the rebase on taxpayers (F). Table 3.7 counts taxpayers only, so each band's factor is now computed over the tape's taxpayers (TAXPAYER 1), and non-taxpayers take the all-bands factor. Over all records, the two lowest bands' taxpayers fell short of the published amounts. Since91775db4a, a saver who becomes a taxpayer by 2023-24 counts once, as a taxpayer, and a flagged taxpayer below the first band sits in it (the review's finding 10).092de13b9, FRS savings income on SPI's definitions (A1, A2).savings_interest_income, since SPI has counted gross-paid interest as bank interest since 2016-17 (note 4). Gilts stay in other income.91775db4ait stays inprivate_pension_income, and Table 3.7's other-income rows read it through a calibration-only descriptor (below).b82247443, a coherent other-income draw on SPI copies (C). The stage-1 transport locates each copy's draw at its twin's own other income (gilts, trust and annuity income) rather than at an identity uniform that ignored it. The stage spec's analogue map and the coverage manifest follow.The investigation's frame arithmetic puts other income below £100k at design weights at about 0.66 of its target with these changes, against 0.40 before. The next build measures it.
FRS rows keep their own reported incomes: the investigation's larger lever, an SPI fill of other income on FRS-row taxpayers, is not taken.
The small traders' pay: SPI Table 3.10's employment-income counts (
d563f4fbc)The national from
b441bb5a0costs the Budget's trading allowance measures (Table 6.1 line 15) below PR A's, and both sit below the costing. A read-only investigation found two things:By María's choice (2026-10-30),
hmrc.spi.employment_income.count_by_self_employment_income_bandbinds Table 3.10's individuals with employment income over the same eight bands from £1 to £10,000 as the self-employment counts:employment_income_annual, the variable Table 3.6's employment rows read) counts in their own self-employment band, taxpayer or not.target_reference_signed_exclusions.json.b89d3aab8). The merge had left them on the 2.162.14 pins. The parity receipt doesn't move for them, and builds compile live from the pins.The tax on savings rows: the OBR path (
5c0137fb4, reverted inf9676fed9)The three ITL Table 2.6 rows bound HMRC's own 2024-25 and 2025-26 projections as a calendar-2025 window: £8.40bn of tax on savings, ×1.52 on the 2023-24 outturn. Table 3.7's savings income grows only ×1.36 on the OBR's path. Bound together, the rows pinned the tax on savings, and so the savings rates' static yield, about 12% above the base the Budget's costing grows.
By María's choice (2026-10-31, option A):
hmrc_itl_savings_rateleaves the source-projection families. Only the three rows move in the references, the membership and the parity receipt.Reverted (
f9676fed9, María's ruling of 2026-10-31). The national from6485af4afstopped atuk_target_fit, as the leverage caveat predicted:HMRC's own projection carries that leverage, and
b441bb5a0's national met it beside the interest rows. So the savings rates' yield stays pinned to the projection. Its distance from the costing is a difference of basis and of behaviour (ISA shifting), not a fault in the base.UC take-up pools that cannot meet DWP's count (
6485af4af)The spine from
d563f4fbcstopped atuk_stage_uc_take_up. Single people's caseload ended 0.0674 of DWP's count further from it than before the redraw (0.0568 onf3cb3ddd3's spine).By María's ruling (2026-10-31), the stage is fixed instead of the gate widened again:
The join (replaced). After the source-support pooling, a pool that cannot meet its count joined the adjacent pool with more room for its gap. The review found that a joined pool's single rate moved claims the wrong way between award bands (finding 7).
Passing gaps (
91775db4a). A pool that cannot meet its count is held at its clip, and its gap passes on:Every band keeps its own rate, so no band's drawn units claim against its own count to meet another's. A family type's caseload still lands on DWP's wherever its entitled units allow. UC results move in later builds.
The gate returns to main's 0.055.
What follows: the stage spec, the coverage manifest, the H2 fixture and the data contract's digests. New stage tests cover the gap passing.
The trading allowance's base: the self-employed's gross receipts (
f3540cbc6)policyengine-uk applies the trading allowance against
self_employment_gross_receipts, and neither the FRS nor the SPI tape records turnover. Without receipts, the engine relieved profits up to the allowance in full and nothing above. So the Budget's raise to £3,000 missed its main base: traders who already claim the £1,000 allowance against larger receipts and gain the larger deduction. No published source gives turnover beside profit. But under current law a claimant's receipts are their profit plus £1,000 exactly, so only who claims is missing.By María's choice (2026-10-31), a new last spine stage,
trading_allowance_receipts, sets each trader's receipts:Who claims. Each trader with a positive profit is flagged as a claimant, drawn on their own
person_id: 0.64 for profits up to £4,500, 0.32 from £4,500 to £9,000, none above.Receipts. A claimant's receipts are their profit plus £1,000. Any other trader's are their profit plus £3,000, expenses at the raised allowance; the costing books traders who switch to the allowance as behaviour. A person with no positive profit keeps zero, which the engine reads as unknown.
The rates are calibrated to HMRC's policy paper (about 400,000 taxpayers with a lower bill, by about £380 on average) until HMRC publishes claimant counts. An HMRC table of claimants by profit band would replace them.
Filing. Every trader with a positive profit is treated as filing, as the bound Table 3.10 counts are of filers. So a profit of £1,000 or less is now taxed where the engine used to relieve it in full.
Two limits, until Add a trading allowance claim status, so allowance reforms reach claimants without turnover data policyengine-uk#2765's claim status is released (UK trading allowance: store the claim status once policyengine-uk#2765 is released #1439; María's ruling of 2026-11-01 on the review's findings 2 and 8). The engine uprates receipts with profit, on the mixed income index, while the allowance is nominal:
Who gains is approximate too: a drawn non-claimant whose receipts are within the allowance loses the full relief the law gives automatically, and the calibrated rates move that gain onto claimants.
Wiring:
self_employment_gross_receiptsalso leaves the register of engine inputs left at their default and joins the export surface's allowed columns, so the data contract's gate digests follow.Certifying the national from
0c2f5314f(3702769f0toa8e25870c)The release-cut certification of the national from
0c2f5314fstopped on three gates. By María's rulings (2026-10-31), all three are fixed at their cause, and no gate is widened.uk_weights_audit: a bug on main since Budget 2026 ISA data, PR B of #1418: ISA subscriptions per adult, the Lifetime ISA anchor, additive ISA income, policyengine-uk 2.164.2 #1421 (b9bcd709c).fit_weight_recordstuple.3702769f0also pins the receipts stage in the two stage-order tests that named the ISA subscriptions as the last stage.uk_input_mass_parityoncharitable_investment_gifts(c5ef12756).uk_input_mass_parityonjsa_income_reported(a8e25870c).dwp.jsa.amount_income_basedbinds DWP's income-based line at the calendar-2025 window of the two years, about £23.4m, as the pension-age Housing Benefit row is. The solve now meets the rest of OBR's total on contributory claimants.The review of
a8e25870c(91775db4ato4dd83e999)An independent review of
a8e25870craised thirteen findings. By María's rulings (2026-11-01):91775db4a).private_pension_income, where policyengine-uk's means tests (Pension Credit, Universal Credit, Housing Benefit, Council Tax Reduction, Income Support), household income and tax read them.annuity_and_trust_income, stores their part. It is calibration-only, and the export carries it.other_investment_income_spi_concept: the engine's other income plus that share of the engine's private pension, as the table counts both payments as other income (note 6).24e2329f6). A roster test pins the eight stage transforms that declarefit_weight_records, read from the source. A new one has to be reviewed, and either record each fit it makes or declare nothing.24e2329f6). They are built withrounded_count, so the test holds whatever small-cell floor main sets.91775db4aand9f70c71e7). Banding on income moved to 2023-24 and the taxpayer basis, as above.91775db4amoved each record's savings by its own band's factor and repeated the banding until no record changed band. On the production tape that didn't converge, because a record's band sets its factor and the factor sets its band.9f70c71e7bands once, moving the savings by each column's factor over all bands. Its receipt records that factor in place of the pass count.91775db4a). Gaps pass between pools instead of joining them, as above.24e2329f6). Its counts scope takes Table 3.10's employment count by self-employment band, so a cell without support is dropped and receipted instead of stopping the solve.24e2329f6). Tests pin the income-based JSA row and the employment count's binding. The claimant rates' calibration, as a committed tool with a post-build check, is part of UK trading allowance: store the claim status once policyengine-uk#2765 is released #1439.24e2329f6).4dd83e999moves the data contract's full-manifest pins (policy, gates manifest and spec fingerprint) with the descriptor's export surface, asf3540cbc6did for the receipts column.59ef8eceeregenerates the H2 fixture and the coverage manifest. The national references, membership and parity receipt don't move. The H2 fixture's synthetic tape doesn't gross to the published amounts, so9f70c71e7leaves it as it is.Up to date with main, and CI (30 Oct)
9ee6fbde0merges main into the branch. The main change it brings is PR B (Budget 2026 ISA data, PR B of #1418: ISA subscriptions per adult, the Lifetime ISA anchor, additive ISA income, policyengine-uk 2.164.2 #1421: ISA subscriptions, policyengine-uk 2.164.2). Conflicts resolved:gates.json;995969f02leaves the savings rebase off any tape whose weights don't gross to the population.e5ac7a5d8regenerates the H2 spine fixture for the other-income analogue (C). CI's engine-uk graph test and the integration-uk smoke test refused its stale stage-spec snapshot.53ad29934starts a missing self-analogue output at zero before the SPI transport. Since C, the transport readsother_investment_incomeas each twin's own analogue before the stage-1 draw, so CI's engine test of a frame whose FRS rows carry no other income was refused. That column now gets the stage's zero before the transport, as it did after the draw, and the final values are unchanged. Production FRS frames always write it.45c2888faregenerates the H2 spine fixture on policyengine-uk 2.164.2, the lock's pin since the merge of main.e5ac7a5d8had built it in a local environment still on 2.162.14, which CI's engine-uk graph test caught.Not in this PR
Checks run locally
All pass. Engine-free tests:
test_uk_spi_savings_rebase.py;test_uk_trading_allowance_receipts.py, the stage rosters, the country spec, the graph and the terminal gates;test_uk_spi_savings_rebase.py, and the SPI spine, support and rank-transport tests;test_uk_fit_weight_records_roster.py, the receipts and support floor tests, and the pension-age targets;The engine tests
engine/uk/test_uk_spi_income.pyandengine/uk/test_uk_spi_state_pension_age.pyalso pass locally for the gift anchor, and the first again for the review's fixes.The updated engine tests (
engine/uk/test_uk_spi_income.py,engine/uk/test_uk_hmrc_uprating.py,engine/uk/test_uk_itl_savings_tax_rates.py,engine/uk/test_uk_release_input_coverage.py) and the H2 acceptance test run in CI.🤖 Generated with Claude Code