Repository navigation
UK output-area-anchored geography ladder (US block-ladder pattern) #349
Description
UK output-area-anchored geography ladder (the US block-ladder pattern for the UK)
Goal
Give the UK artifact the same local-geography spine the US artifact has (us_runtime/geography_ladder.py): anchor every household at the finest published census geography, derive every other layer deterministically from that anchor, and keep one national dataset, filterable at any grain (the #275 standing rule — no per-area files).
Design (mirroring the US block ladder)
Anchor = census output area (OA). The UK's block equivalent: 2021 Census OAs for England & Wales, Scotland's Census 2022 OAs, NI small areas. Assignment ladder:
- Region is already assigned (FRS carries it; it is a calibrated dimension of the national build).
- Assign a Westminster parliamentary constituency (2024 boundaries) within region, sampled proportional to constituency household counts — the politically salient rung, and the rung where per-area calibrated weights (Publish a Populace UK local variant with rowwise H5 and long weights #146 long weights) can later bind, exactly as the US anchors its ladder inside the assigned congressional district.
- Assign one OA within the constituency, sampled proportional to OA population (ONS OA→PCON24 best-fit lookup defines membership).
- Every other layer derives from the OA — no independent randomness:
- LSOA and MSOA: structural nesting (OA → LSOA → MSOA).
- Local authority district: ONS OA→LAD lookup.
- Ward: OA→ward lookup.
- ITL levels / combined authorities: LAD-level lookups.
UK geographies famously do not nest cleanly (constituencies cross LAD boundaries); the ONS Open Geography Portal best-fit lookups are the sanctioned resolution and should be carried in the ladder artifact itself, sha-pinned, like the US ladder artifact carries its block crosswalks.
Discipline ported from the US implementation
- Vintage policy = error: the ladder artifact records one vintage per derived layer; the loader refuses an artifact missing any; assignment refuses a ladder whose constituency vintage differs from the vintage the constituencies were assigned under (the Translate old-vintage CD targets to current district geography #205 lesson — no silent partial joins on mismatched vintages).
- Engine inputs only: exported columns are policyengine-uk household inputs (or plain data columns where no engine input exists); never persisted formula outputs (the Fix
in_nycbefore enforcing no-formula exports #34 lesson — derived quantities recompute from the rung). - Seeded draws: constituency and OA sampling under the build's seed discipline, reproducible from the release manifest.
Relationship to the existing UK local machinery
Complementary, not competing, with #146/#147:
- The ladder gives the spine: every household locatable at any grain, local analysis = filter (or aggregate) the national file.
- The long weights (Publish a Populace UK local variant with rowwise H5 and long weights #146,
local_solver/local_runner, already ported) give calibrated per-area estimate quality on top of the spine where area targets exist. - The credibility gates (Port UK local target surface and credibility gates from uk-data #147) score whether a given area's filtered/weighted estimate is publishable — random-within-region assignment is only as good as region-level joint structure, and the gates are what say so per area.
Acceptance
- Ladder artifact (OA anchor + lookups + vintages) built from ONS sources, sha-pinned, with provenance in the ledger.
- Assignment wired into the UK build behind the country-spec geography-spine schema; gate output mirrors
us_geography_ladder_assignment_summary. - National artifact round-trips: filter by constituency/LAD/region reproduces the calibrated region totals; assignment summary reports per-rung coverage.
- Cross-checks: constituency household counts vs ONS, LAD population shares vs mid-year estimates (report-only at first; gates per Port UK local target surface and credibility gates from uk-data #147).
First milestone: England & Wales ladder end-to-end on the current populace-uk 2023 artifact; Scotland/NI rungs follow once their lookup vintages are pinned.
Refs: #146 (local variant publication), #147 (credibility gates), #145 (source refresh), #275 (one-national-dataset rule), populace#8 / policyengine.py#427 (the artifact this extends).
Activity
UK output-area-anchored geography ladder (US block-ladder pattern) (#354
juaristi22 commentedon Oct 16, 2026
Done in #354 (the output-area ladder, England and Wales, July) and #932 (#931 item (a): identity-keyed atomic assignment for all four nations, now the full build's default). The two-stage draw this issue specified remains as
--geography-assignment legacyfor measurement builds, and release candidates refuse it. The constituency and local-authority household cross-checks now bind as calibration targets. What remains is tracked in #931: (d) rosters from the supports and retiring the ladder file, (e) the 2025 local-authority recode, and (f) movinggeography.yamltophase: f3. The enrichment reorder, item (b), is in #1278.