Repository navigation
Update the reform impacts dashboard docs page (#806) - #1732
Merged
vahid-ahmadi merged 5 commits intoOct 24, 2026
Merged
Conversation
vahid-ahmadi
force-pushed
the
vahid-ahmadi/issue-806-reform-impacts-dashboard-plan
branch
from
October 8, 2026 14:53
163fae7 to
23567a8
Compare
#806 asks for a dashboard tracking high-importance reform budgetary impacts over time and across model / data versions. Auditing the existing infrastructure: - reforms_config.yaml already has 9 canonical regression fixtures (basic rate +1pp, UC taper to 20%, VAT +2pp, etc.) with expected impacts and tolerances. - test_parametric_reform_impacts.py runs each fixture in CI and fails on drift outside the tolerance. So the *fixtures* exist; what's missing is a published *time series* view onto them. The page proposes: - append-only CSV / Parquet storage of (timestamp, policyengine_uk_version, policyengine_uk_data_version, dataset_url, reform_name, expected_impact, observed_impact, gap), - a scheduled GitHub Actions runner (weekly + on tag) walking reforms_config.yaml, - three publication paths in order of effort: docs page Plotly chart, living JSON endpoint, policyengine.org dashboard tile, - annotations for EFO releases, dataset refreshes, methodology PRs. Cross-links to the nowcasting comparison and child poverty validation pages so the three together form a coherent "how is the model doing" view.
Name the test that runs the fixtures (microsimulation/test_reform_impacts.py, gov_balance in 2029, default tolerance 1bn), fix the fixture count to eight, describe update_reform_impacts.py correctly, link #1724 instead of a page that is not on main, and add why the Microcosm move makes the history useful. Type the changelog fragment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The microsimulation fixtures run only with dataset credentials and are required in the merge queue and on main. Describe the personal allowance and child benefit fixtures by what they set, drop the unverified Microcosm release date, and remove em dashes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
vahid-ahmadi
force-pushed
the
vahid-ahmadi/issue-806-reform-impacts-dashboard-plan
branch
from
October 12, 2026 09:06
23567a8 to
e2ceb6b
Compare
Main already carries this page (#2386) and released its changelog line in 2.143.0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The page reached main through #2386 and its changelog line shipped in 2.143.0, so this drops the released .added fragment and corrects main's copy: - reforms_config.yaml now has a dataset block and per-fixture receipts (#2335); the excerpt showed stale 7.8 / -18.1 values. - The tests pin the certified Enhanced FRS 2024-25 (1.58.1) through certified_dataset.yaml, not enhanced_frs_2023_24.h5@1.40.3. - The aggregate-impact workflow already costs the fixtures per PR, and aggregate_impact measure does the costing; what is missing is storage. - "Raise higher rate by 1pp" sets 42%, a 2pp rise. - Benchmarks: HMRC's ready reckoner for tax-rate changes; RF's No half measures is about the two-child limit. - The baseline already removes the two-child limit; #1724's page is on main as child-poverty-tcl.md. - Notebooks need a myst.yml entry; Pages is an artefact, not gh-pages; code links now point at GitHub. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
|
UK hub (Claude build session, for Max) Head: Read this first. The page is already on main, word for word, through #2386 ( This branch now:
Against main, the diff is the page and the fragment. Suggested title on landing: "Update the reform impacts dashboard plan". The plan is still worth having. #806's ask, a stored and visible history across model and data versions, is still unbuilt, though main now has most of the pieces (receipts, Claims checked (main
|
| Claim | Verdict | Source |
|---|---|---|
| #806 asks for tracking "across model / data versions" | narrowed: the issue says "over time and model versions" | #806 body |
Excerpt expected_impact: 7.8, -18.1 |
outdated: 7.1 and -22.0, with a dataset: block and receipts |
policyengine_uk/tests/microsimulation/reforms_config.yaml:6-25,114-118 |
Change in gov_balance in 2029; default tolerance £1bn; microsimulation marker |
true | test_reform_impacts.py:37-40,49,57,69 |
conftest points at enhanced_frs_2023_24.h5@1.40.3 |
outdated: certified pin enhanced_frs_2024_25.h5, uk-data 1.58.1, by HF commit and SHA-256 |
tests/conftest.py:20-29; tests/certified_dataset.yaml:7-14 |
| Data required in the merge queue and on main; skipped on PRs without credentials | true | pr_code_changes.yaml (require-data: merge_group), code_changes.yaml (require-data: true), test-suite.yaml |
| "the signal is only pass or fail" | outdated: the Aggregate impact workflow posts per-PR base-vs-head costings of the standard reforms, which include the eight fixtures | .github/workflows/aggregate-impact.yaml:1-40; utils/aggregate_impact/README.md:140-147 |
update_reform_impacts.py overwrites the old value |
true for expected_impact; old values now survive in receipts, which a test enforces |
update_reform_impacts.py:87-100,156; tests/aggregate_impact/test_aggregate_impact_dataset.py:49-57 |
| The file says the values are regression values, not external costings | true | reforms_config.yaml:1-4 |
| Eight reforms, in this order | true | reforms_config.yaml:22,45,68,91,114,138,161,197 |
| "Raise higher rate by 1pp" | false: sets uk[1].rate: 0.42 against a 0.40 baseline, so +2pp |
reforms_config.yaml:45-48; parameters/gov/hmrc/income_tax/rates/uk.yaml (2015-04-05: 0.4) |
| PA to £13,000 is +£430 in 2029; frozen to 2030-31 | true | reforms_config.yaml:68-71; personal_allowance/amount.yaml |
| Child benefit, UC taper, NI 10%, VAT +2pp, additional rate +3pp | true | reforms_config.yaml lines above |
| "three of the four big revenue heads" | cut: the fourth is never named and it is unsourced; the sentence also omitted the Child Benefit fixture | — |
| "published OBR/RF benchmark for the same reform"; "RF No half measures-style costing" | fixed: no fixture has an OBR or RF costing; HMRC's ready reckoner covers illustrative tax-rate changes; No half measures is about the two-child limit | https://www.gov.uk/government/statistics/direct-effects-of-illustrative-tax-changes (200); docs/book/validation/child-poverty-tcl.md:1-22 |
| v2 "Remove the two-child limit (#1724)" | fixed: the baseline already removes it from 2026-04-06, so the reform is reinstating it. #1724 is open, but its page is on main as child-poverty-tcl.md |
child_count.yaml:2-5; #1724 state |
| "Freeze the PA for an additional year" | qualified: that is 2031-32, outside the 2029 reference year | — |
| Fixtures "still run on the Enhanced FRS 2023-24" | outdated: 2024-25, 1.58.1 | reforms_config.yaml:16-20 |
validation/*.ipynb Plotly pattern |
made precise: only validation.ipynb uses Plotly, and notebooks need a myst.yml entry |
myst.yml:20-22; docs/book/validation/ |
gh-pages artefact |
fixed: the docs deploy through a Pages artefact | .github/workflows/deploy.yml:46 |
Relative ../../../policyengine_uk/... links |
changed to GitHub URLs, the convention elsewhere in the docs. They rendered as static download copies | review agent's check of the deployed page |
Links to nowcasting-comparison.md and rf-nowcasting-methodology.md |
true | both exist in docs/book/assumptions/ |
Corrections: 14.
Uncertain and out of scope
- The fixture names "Raise higher rate by 1pp" (+2pp), "Raise child benefit by 25GBP/week per additional child" (sets £25) and "by ~800GBP/year" (+£430 in 2029) are misleading in code. Renaming them touches
standard_reforms.yamland the receipts test, so that is a code PR, not this one.
Full ledger: ~/reviews/uk-hub/jobs/docs-c2/b/1732-ledger.md.
3 of 5 tasks
vahid-ahmadi
deleted the
vahid-ahmadi/issue-806-reform-impacts-dashboard-plan
branch
October 24, 2026 10:55
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Corrects the reform impacts dashboard planning page (
docs/book/validation/reform-impacts-dashboard-plan.md), written for #806.The page itself is already on main: it shipped in #2386 (merged 21 October 2026), and its original
.addedchangelog entry was released in 2.143.0. This branch has merged main, dropped the released fragment and now carries only corrections to main's copy of the page.What this PR corrects (from its changelog fragment):
The claim-by-claim review behind these corrections, with sources, is in the "Ready for hub review" comment on this PR.
changelog.d/806-reform-impacts-dashboard-corrections.changed.md(changed).🤖 Generated with Claude Code