[MaxGhenis]

Fixes #2633

Draft. WIP: the coverage value is re-derived on the next certified Microcosm cut before this lands. This branch now carries main, including #2189 (energy-bill VAT is added after grossing and is not grossed up; the coverage is derived against receipts net of energy VAT). It ships main's coverage, 0.67, plus test, doc and provenance fixes. The value it lands with comes from a run with #2189's formula on the cut that follows microcosm#1383's rebuild, after that cut's fit:

• if that run puts 2024-25 VAT within ±2% of receipts at 0.67, 0.67 stays and this PR lands with only the fixes below;
• otherwise the coverage moves to the value that run gives, and the impact section is filled from runs at that value.

At #1383's merged head, obr.vat is held out of the fit (María's ruling of 24 October; calibration_measure_exclusions.json), so that measurement is not circular. Every figure marked PENDING comes from a run, not from arithmetic on other runs.

What changed and why

gov.simulation.vat.survey_consumption_coverage was derived in #2187 on the Microcosm UK 2024-25 cut f9d1922c (2 Oct) and re-derived with the energy split in #2189. Its metadata says to re-derive it when the dataset it should match changes.

The certified 6cba5d71 (policyengine.py 6.10.2) and a133f37a (the current default; policyengine.py#715) cuts record about 10% more standard-rated consumption: £435.60bn against £396.15bn in 2024-25. Before the energy split, with 0.67, total vat on a133f37a (main c0252415a) came out as follows:

	2024-25	2025-26	2026-27
Model vat	£188.61bn	£196.43bn	£201.71bn
OBR receipts (EFO March 2026, table 3.8)	£170.994bn	£180.169bn	£187.704bn
Gap	+10.3%	+9.0%	+7.5%

On 6cba5d71, a reform run that raises the standard rate from 20% to 21% changed total vat and gov_balance by £9.94bn in 2026-27 (PolicyEngine-UK 2.135.7 and 2.152.6 agree), 12.9% above HMRC's £8.8bn ready reckoner. These pre-split runs are recorded in the doc as the reason to re-derive; they do not set the value.

Changes in this branch against main:

• Tests: new Hypothesis properties and fixed band-edge cases in test_vat_grossing_properties.py; a 2024 case in vat_grossing.yaml (2,050 / 0.469 = 4,371.00, the same product as later years).
• Docs: docs/engineering/vat-coverage.md gains a "Later Microcosm cuts (24 October 2026)" section with the a133f37a and 6cba5d71 inputs (tags, revisions, SHA-256), versions and the pre-split rows, and a pointer from the "Which datasets" paragraph.
• Parameters: none yet. survey_consumption_coverage stays 0.67 and microdata_vat_coverage 0.469 until the run above.

Impact

With the parameters unchanged, this branch moves no model output: it adds tests and docs. PENDING, if the run above moves the coverage, runs at the new value of:

• total vat on the new cut and on Enhanced FRS 1.58.1, 2024-25 to 2026-27;
• a one-point standard-rate reform run on the new cut (reform minus baseline vat and gov_balance);
• the Raise VAT standard rate by 2pp regression fixture (reforms_config.yaml), updated from that run.

Invariants

These hold for every input:

1. With the deprecated factor at its declared value, the grossing factor equals survey_consumption_coverage × household_share_of_receipts.
2. A reform that moves the deprecated factor out of the closeness band around its declared value (1e-8 + 1e-5 × the value, NumPy's default isclose tolerance) overrides the two; a value inside the band counts as untouched.
3. Each household's vat with no energy consumption equals (standard rate × full_rate_vat_consumption + reduced rate × reduced_rate_vat_consumption) / shipped product. It is linear in each household's consumption, with no cross-household term.
4. The deprecated default equals the product of the two factors in every year from 2015 to 2030.
5. The deprecated factor has a single undated value, so 2024 takes the product like every later year (vat_grossing.yaml, 2024 case). Adding a dated entry to it, which would send earlier years down the legacy path, fails that case.

Invariants 1–3 are Hypothesis tests in policyengine_uk/tests/test_vat_grossing_properties.py. Invariant 4 is in test_vat_grossing_legacy.py (main). A mutation that drops the household share from vat_grossing_factor fails invariant 1. For invariant 2, fixed cases sit at 0.99 and 1.01 bands either side of the declared value; changing the helper's np.isclose to exact equality fails the inside cases. The helper properties have no Hypothesis deadline, since they check algebra, not speed.

The dataset-level check (total vat within a few percent of receipts on the certified Microcosm cut) can't run in CI, because the tests' certified dataset is Enhanced FRS 1.58.1. It is reproduced by the doc's snippet.

Interaction with open work

PolicyEngine/microcosm#1383 (merged) locks policyengine-uk 2.151.0 and holds obr.vat out of the fit as a diagnostic (commit 94d4102fd; exclusion approved 2026-10-24, expiring 2027-01-24). The exclusion says the row rebinds once policyengine-uk re-derives the factor on a file that does not bind it. If a later cut binds obr.vat under 0.67, its fit would absorb the factor, so this PR's value is paired only with a cut on which a run supports it. Any lock bump is María's call.

Reproducer: n/a: tests and docs only; model source and dependencies are unchanged until the coverage is re-derived

axiom: n/a: a microsimulation calibration factor for survey under-recording, not law

🤖 Generated with Claude Code

[github-actions]

Aggregate impact on certified data: FAIL

The report did not run, so this change has no report. See the workflow run.

[MaxGhenis]

Review r1 (GPT-6.1 Sol, REQUEST_CHANGES at b6d8af5): all five findings are fixed in e1ff493.

1. £9.94bn evidence (P2). That figure is a real reform run, but the PR didn't cite it. The doc and body now cite it: on 6cba5d71 (revision bb225848), reform (standard rate 21%) minus baseline gives vat +£9.94bn in 2026-27 and +£9.68bn in 2025-26, with the same change in gov_balance. It ran on PolicyEngine-UK 2.135.7 and again on 2.152.6, and both give the same result. The baseline-only driver the review read is a different script; its static figure is not cited anywhere.
2. Legacy-override oracle (P2). The band is now stated once: 1e-8 + 1e-5 × the declared value, which is np.isclose's default.
   • The moved-legacy property draws values outside it, and a new property draws values inside it.
   • Fixed cases sit at ±0.99 and ±1.01 bands.
   • The review's counterexample (0.518005185) is now classified as untouched and passes.
   • Mutating the helper to exact equality fails the inside cases.
3. Deadlines (P2). The two helper properties now have deadline=None.
4. Version provenance (P3). The a133f37a rows: 2.152.7 / core 3.33.2, and again 2.147.2 / core 3.33.3, with the same totals. The 6cba5d71 rows: 2.147.2 / core 3.33.3.
5. "Same to 3 dp" (P3). The body now says 6cba5d71 gives products of 0.504 to 0.518, and so gives 0.74 once the 0.739 coverage is rounded. The parameter metadata now says only that the 2024-25 coverage matches.

Local runs at e1ff493:

• test_vat_grossing_properties.py + test_vat_grossing_legacy.py: 12 passed.
• vat_grossing.yaml: 5 passed.
• ruff: clean.

Still PENDING, and the PR stays a draft until they are done:

• the totals at the new product, and the one-point reform run on a133f37a;
• the Raise VAT standard rate by 2pp fixture, which is updated only from a measured run.

These run in the hub's next allowed host window. A delta review follows them.

[MaxGhenis]

Plan change (24 Oct): this PR no longer lands on a133f37a's figures. Two changes move what the coverage should be:

• microcosm#1383, María's rebuild tonight, changes the consumption base: it levels the LCFS COICOP divisions' over-draw.
• Model the temporary 0% VAT on GB domestic electricity #2189 changes the formula: energy-bill VAT is no longer grossed up, and the coverage is derived against receipts net of energy VAT.

So the value is re-derived on Monday's final cut, after its fit, with the formula on main. The 2pp fixture is updated from a run on the same code, and a delta review follows before this is marked ready. If that run puts VAT within ±2% of receipts at 0.67, this keeps 0.67.

At #1383's head e570d65ff, obr.vat is held out of the fit (calibration_measure_exclusions.json; the exclusion is removed from the calibration registry by apply_uk_calibration_measure_exclusions), so that measurement is not circular.

The figures run today are a reference only, on a133f37a at e1ff493 and before #2189:

Run	Result
Total vat at 0.74 vs receipts	−0.13% (2024-25), −1.29% (2025-26), −2.70% (2026-27)
1pp reform run, reform minus baseline	£8.99bn in 2026-27, +2.2% vs HMRC's £8.8bn
Enhanced FRS 1.58.1, total vat vs receipts	+29.0% (2024-25), +27.5% (2025-26)
2pp fixture	£24.97bn

[MaxGhenis]

Merged main (with #2189) at 87911eb58. The branch now ships main's coverage, 0.67 (product 0.469), and #2189's formula. Its own changes are the property tests, the 2024 pin case and the later-cut provenance in the doc; test expectations are back at 2,050 / 0.469 = 4,371.00. The VAT pytest files (38 passed) and the gov/hmrc/vat YAML tests (10 passed) pass locally.

The coverage is re-derived with #2189's formula on the next certified cut, after its fit. Title and body updated to match.