Skip to content

Update the reform impacts dashboard docs page (#806) - #1732

Merged
vahid-ahmadi merged 5 commits into
mainfrom
vahid-ahmadi/issue-806-reform-impacts-dashboard-plan
Oct 24, 2026
Merged

vahid-ahmadi merged 5 commits into
mainfrom
vahid-ahmadi/issue-806-reform-impacts-dashboard-plan

Conversation

@vahid-ahmadi

@vahid-ahmadi vahid-ahmadi commented May 27, 2026 •

Copy link
Copy Markdown
Collaborator

Corrects the reform impacts dashboard planning page (docs/book/validation/reform-impacts-dashboard-plan.md), written for #806.

The page itself is already on main: it shipped in #2386 (merged 21 October 2026), and its original .added changelog entry was released in 2.143.0. This branch has merged main, dropped the released fragment and now carries only corrections to main's copy of the page.

What this PR corrects (from its changelog fragment):

Updated the reform impacts dashboard plan (docs/book/validation/reform-impacts-dashboard-plan.md) for the fixture receipts, the certified Enhanced FRS 2024-25 pin and the per-pull-request aggregate-impact costings now on main, and corrected the higher-rate fixture, the external benchmarks and the two-child limit reform.

The claim-by-claim review behind these corrections, with sources, is in the "Ready for hub review" comment on this PR.

  • Changelog fragment: changelog.d/806-reform-impacts-dashboard-corrections.changed.md (changed).
  • Docs and changelog only; no code changes.

🤖 Generated with Claude Code

@vahid-ahmadi
vahid-ahmadi force-pushed the vahid-ahmadi/issue-806-reform-impacts-dashboard-plan branch from 163fae7 to 23567a8 Compare October 8, 2026 14:53
vahid-ahmadi and others added 3 commits October 12, 2026 10:59
#806 asks for a dashboard tracking high-importance reform budgetary
impacts over time and across model / data versions.

Auditing the existing infrastructure:

- reforms_config.yaml already has 9 canonical regression fixtures
  (basic rate +1pp, UC taper to 20%, VAT +2pp, etc.) with expected
  impacts and tolerances.
- test_parametric_reform_impacts.py runs each fixture in CI and
  fails on drift outside the tolerance.

So the *fixtures* exist; what's missing is a published *time series*
view onto them.

The page proposes:

- append-only CSV / Parquet storage of (timestamp,
  policyengine_uk_version, policyengine_uk_data_version, dataset_url,
  reform_name, expected_impact, observed_impact, gap),
- a scheduled GitHub Actions runner (weekly + on tag) walking
  reforms_config.yaml,
- three publication paths in order of effort: docs page Plotly chart,
  living JSON endpoint, policyengine.org dashboard tile,
- annotations for EFO releases, dataset refreshes, methodology PRs.

Cross-links to the nowcasting comparison and child poverty validation
pages so the three together form a coherent "how is the model doing"
view.
Name the test that runs the fixtures (microsimulation/test_reform_impacts.py, gov_balance in 2029, default tolerance 1bn), fix the fixture count to eight, describe update_reform_impacts.py correctly, link #1724 instead of a page that is not on main, and add why the Microcosm move makes the history useful. Type the changelog fragment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The microsimulation fixtures run only with dataset credentials and are
required in the merge queue and on main. Describe the personal allowance
and child benefit fixtures by what they set, drop the unverified
Microcosm release date, and remove em dashes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vahid-ahmadi
vahid-ahmadi force-pushed the vahid-ahmadi/issue-806-reform-impacts-dashboard-plan branch from 23567a8 to e2ceb6b Compare October 12, 2026 09:06
MaxGhenis and others added 2 commits October 23, 2026 02:20
Main already carries this page (#2386) and released its changelog line in 2.143.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The page reached main through #2386 and its changelog line shipped in
2.143.0, so this drops the released .added fragment and corrects main's
copy:

- reforms_config.yaml now has a dataset block and per-fixture receipts
  (#2335); the excerpt showed stale 7.8 / -18.1 values.
- The tests pin the certified Enhanced FRS 2024-25 (1.58.1) through
  certified_dataset.yaml, not enhanced_frs_2023_24.h5@1.40.3.
- The aggregate-impact workflow already costs the fixtures per PR, and
  aggregate_impact measure does the costing; what is missing is storage.
- "Raise higher rate by 1pp" sets 42%, a 2pp rise.
- Benchmarks: HMRC's ready reckoner for tax-rate changes; RF's No half
  measures is about the two-child limit.
- The baseline already removes the two-child limit; #1724's page is on
  main as child-poverty-tcl.md.
- Notebooks need a myst.yml entry; Pages is an artefact, not gh-pages;
  code links now point at GitHub.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Collaborator

UK hub (Claude build session, for Max)
Ready for hub review

Head: 551399d02b264a11b0f22adb8ef6f718d9b4699d

Read this first. The page is already on main, word for word, through #2386 (d50ee85a9, merged 2026-10-21). Its changelog line shipped in 2.143.0 (CHANGELOG.md:194).

This branch now:

  • merges main (cc63d6e32);
  • deletes changelog.d/806.added.md;
  • corrects main's copy;
  • adds changelog.d/806-reform-impacts-dashboard-corrections.changed.md.

Against main, the diff is the page and the fragment. Suggested title on landing: "Update the reform impacts dashboard plan".

The plan is still worth having. #806's ask, a stored and visible history across model and data versions, is still unbuilt, though main now has most of the pieces (receipts, aggregate_impact measure, per-PR comments).

Claims checked (main a45baf5e4)

Claim Verdict Source
#806 asks for tracking "across model / data versions" narrowed: the issue says "over time and model versions" #806 body
Excerpt expected_impact: 7.8, -18.1 outdated: 7.1 and -22.0, with a dataset: block and receipts policyengine_uk/tests/microsimulation/reforms_config.yaml:6-25,114-118
Change in gov_balance in 2029; default tolerance £1bn; microsimulation marker true test_reform_impacts.py:37-40,49,57,69
conftest points at enhanced_frs_2023_24.h5@1.40.3 outdated: certified pin enhanced_frs_2024_25.h5, uk-data 1.58.1, by HF commit and SHA-256 tests/conftest.py:20-29; tests/certified_dataset.yaml:7-14
Data required in the merge queue and on main; skipped on PRs without credentials true pr_code_changes.yaml (require-data: merge_group), code_changes.yaml (require-data: true), test-suite.yaml
"the signal is only pass or fail" outdated: the Aggregate impact workflow posts per-PR base-vs-head costings of the standard reforms, which include the eight fixtures .github/workflows/aggregate-impact.yaml:1-40; utils/aggregate_impact/README.md:140-147
update_reform_impacts.py overwrites the old value true for expected_impact; old values now survive in receipts, which a test enforces update_reform_impacts.py:87-100,156; tests/aggregate_impact/test_aggregate_impact_dataset.py:49-57
The file says the values are regression values, not external costings true reforms_config.yaml:1-4
Eight reforms, in this order true reforms_config.yaml:22,45,68,91,114,138,161,197
"Raise higher rate by 1pp" false: sets uk[1].rate: 0.42 against a 0.40 baseline, so +2pp reforms_config.yaml:45-48; parameters/gov/hmrc/income_tax/rates/uk.yaml (2015-04-05: 0.4)
PA to £13,000 is +£430 in 2029; frozen to 2030-31 true reforms_config.yaml:68-71; personal_allowance/amount.yaml
Child benefit, UC taper, NI 10%, VAT +2pp, additional rate +3pp true reforms_config.yaml lines above
"three of the four big revenue heads" cut: the fourth is never named and it is unsourced; the sentence also omitted the Child Benefit fixture —
"published OBR/RF benchmark for the same reform"; "RF No half measures-style costing" fixed: no fixture has an OBR or RF costing; HMRC's ready reckoner covers illustrative tax-rate changes; No half measures is about the two-child limit https://www.gov.uk/government/statistics/direct-effects-of-illustrative-tax-changes (200); docs/book/validation/child-poverty-tcl.md:1-22
v2 "Remove the two-child limit (#1724)" fixed: the baseline already removes it from 2026-04-06, so the reform is reinstating it. #1724 is open, but its page is on main as child-poverty-tcl.md child_count.yaml:2-5; #1724 state
"Freeze the PA for an additional year" qualified: that is 2031-32, outside the 2029 reference year —
Fixtures "still run on the Enhanced FRS 2023-24" outdated: 2024-25, 1.58.1 reforms_config.yaml:16-20
validation/*.ipynb Plotly pattern made precise: only validation.ipynb uses Plotly, and notebooks need a myst.yml entry myst.yml:20-22; docs/book/validation/
gh-pages artefact fixed: the docs deploy through a Pages artefact .github/workflows/deploy.yml:46
Relative ../../../policyengine_uk/... links changed to GitHub URLs, the convention elsewhere in the docs. They rendered as static download copies review agent's check of the deployed page
Links to nowcasting-comparison.md and rf-nowcasting-methodology.md true both exist in docs/book/assumptions/

Corrections: 14.

Uncertain and out of scope

  • The fixture names "Raise higher rate by 1pp" (+2pp), "Raise child benefit by 25GBP/week per additional child" (sets £25) and "by ~800GBP/year" (+£430 in 2029) are misleading in code. Renaming them touches standard_reforms.yaml and the receipts test, so that is a code PR, not this one.

Full ledger: ~/reviews/uk-hub/jobs/docs-c2/b/1732-ledger.md.

@vahid-ahmadi vahid-ahmadi changed the title Add reform impacts dashboard planning doc (#806) Update the reform impacts dashboard docs page (#806) Oct 23, 2026
@vahid-ahmadi
vahid-ahmadi added this pull request to the merge queue Oct 23, 2026
Merged via the queue into main with commit f548e39 Oct 24, 2026
10 checks passed
@vahid-ahmadi
vahid-ahmadi deleted the vahid-ahmadi/issue-806-reform-impacts-dashboard-plan branch October 24, 2026 10:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants