Skip to content

Build the national + local-area pair on policyengine-uk 2.151.0, and drop UK model-figure withholding - #1396

Draft
juaristi22 wants to merge 15 commits into
mainfrom
uk-pair-release-2151
Draft

juaristi22 wants to merge 15 commits into
mainfrom
uk-pair-release-2151

Conversation

@juaristi22

@juaristi22 juaristi22 commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

The UK national + local-area (dense) pair on policyengine-uk 2.151.0, built after #1383 merged (main at fe1e2ad), and the fixes it needed. Both releases are published to the inspect lane in policyengine/populace-uk-private:

Line Cut tag Built at
National microcosm-uk-2024-25-national-20261025T091414Z-a6299504 67ee2dd
Local area (dense, K=15, 150,000 households) microcosm-uk-2024-25-dense-20261026T105309Z-c9df4b86 ee178eb

Both stand on one release-candidate spine. The national is promoted (latest-national.json); the dense release moves no pointer. Merge with a merge commit, so both build commits stay in main's history.

The last eight commits change the next build, not these releases. By María's rulings of 2026-10-26:

  • the UK builds stop withholding model figures and per-area support figures, since their releases go only to the licensed private repository;
  • they stop publishing to the public Orrery by default;
  • a stage receipt's small-cell rule can flag, but never fail, a build.

After Vahid's review, the Orrery summaries carry no row's figures or failure lines. Publishing to Hugging Face is otherwise unchanged: a check that refused to stage to a repository not reported as private was reverted (María's ruling of 2026-10-26).

Commits

  1. 802c9c2: the under-2 usage signal takes its own bound of 0.10. The first national at fe1e2ad failed release-cut certification on uk_take_up_signal alone. The under-2 usage share sat 0.0516 from its rates' expectation, just past ±0.05. On the spine's own weights the draw meets its rates; DfE's working-parent row at 9 to 11 months binds a thin cell of that draw's children, and the calibrated weights move the share. By María's ruling of 2026-10-25 the row stays bound and this one share takes ±0.10; every other flag keeps ±0.05. The battery and release-cut digests are re-pinned. The national rebuilt at 67ee2dd sits 0.0521 away and passes. After Vahid's review María kept the bound at 0.10.

  2. 67ee2dd: the childcare take-up rates are refitted on 2.151.0 (María's ruling of 2026-10-25):

    Rate Was Now
    Tax-Free Childcare 0.9697 0.9134
    Working-parent hours 0.8262 0.8217
    Targeted hours, 2-year-olds 0.6551 0.8687
    Under-2 usage, DfE age 0 0.6962 0.7000

    Age-1 usage and universal hours stay at their bounds. The national was rebuilt here and certified, 19 of 19 release-cut gates.

  3. 7dac4c3: --withhold-thin-rows-only for the dense build (María's ruling of 2026-10-26). The first dense build's calibrated gates ran the model-figure withholding for hours, then failed in an eigendecomposition that did not converge: the differencing check, which since Lock policyengine-uk 2.151.0 and write the UK inputs it reads (#1095, #1278) #1383 runs over each region's rows of every grain and each local family with its parents. The switch kept the thin rows' withholding and skipped the differencing check, so the retry reused the stored solve. Commit 8 removes the switch with the rest of the withholding.

  4. d6449fc: the sign-off drafter reads withheld national rows. It refused any (thin national rows were withheld from María's rule of 2026-10-21), and held the national fit gate's withheld count to national rows, where the gate counts the surface's whole set. It still reads the builds made while rows were withheld.

  5. ee178eb: the gate-only sign-off of the retried dense build 1:

    Register Added Kept Retired
    National target-fit rows 1 5 10
    Local target-fit cells 6 17 14
    Area support 0 6 0

    Entries are local-line only. The assembly deadline is 2026-11-16, when the earliest of the kept entries expires.

  6. 9b71d42: main merged in after both releases were published. The gate digests are re-pinned on the merged gates.json and the H2 fixture is regenerated.

  7. 5ff449d: UK graph evidence stays off the public Orrery unless asked. A build that staged to Hugging Face published its graph by default. It now publishes only on --publish-orrery, and --no-publish-orrery states the default. Its help warned that, after commits 8 and 9, the summaries would carry every row's model figures and the gates' failure lines; commit 11 takes them out.

  8. 58560fd: UK builds no longer withhold model figures (UK private rent at the assigned area (#1123 item 4, "PR B"): what is in place and the pieces still missing #1262, Lock policyengine-uk 2.151.0 and write the UK inputs it reads (#1095, #1278) #1383):

    • A target row resting on fewer than ten source households, and a row that would give one back by difference, keep their figures in the calibration diagnostics, targets.csv, the dense and national gate reports, the national evidence and, until commit 11, the evidence summaries.
    • The fit gates judge every row.
    • Totals, losses and the size receipt are the solve's own, and staged epochs carry their loss.
    • BRMA rows carry no carrier verdict, and the release contract no longer warns about one.

    This removes the thin-row count, the differencing check and --withhold-thin-rows-only. Release certification refuses a run built while rows were withheld, rather than judging it without them. The release contract still checks the withheld rows of earlier releases.

  9. 6ed063a: dense releases ship per-area support figures (UK local-area release: publish a K=15, 150,000-household dense dataset in at most two builds #1301):

    • The area-support gate states an area's rows, Kish ESS and distinct sources beside the floors it misses, and each level's minima.
    • The diagnostics' area rows carry the figures, and the candidate manifest states the minima.
    • The release contract, the candidate pre-flight and the drafter's verify no longer walk the shipped files for them.
    • The drafter still walks the entries it drafts, since they go into a public pull request.
  10. dc50c4b: a stage receipt's small-cell rule flags, never fails.

    • Stage refusals: the gambling top-up and the annual-income stage refused on fewer than ten donor or reporting households. They now refuse only when their fits cannot run.
    • Gates: five spine gates failed or errored on a figure their receipt prints "<10": gambling, property purchases, UC take-up, the annual-income ratios and regional home values. The excise and legacy-closure gates already passed on theirs, silently. All seven now pass on what they cannot read and name it in their details.
    • Unchanged: counts a gate requires to be zero, and the declared donor-support floors, fail as before.
  11. 0862fc5: the graph summaries carry no row's figures or failure lines (Vahid's review, M1). --publish-orrery stays opt-in, and the summaries are what it submits.

    • A calibration summary keeps each target row's target, tolerance and source, but no row's estimates or errors.
    • The size summary keeps the losses over every row, but not the largest change in one row.
    • A gate summary keeps each gate's verdict and its count of failure lines, but not the lines.
    • The export checks the summaries before it preserves anything for publication. If one carries a row's estimates or a failure line, it stops with unpublishable_summary and writes no inventory, so neither the build nor the emitter's retry can submit it.
    • The private diagnostics, targets.csv and the gate reports keep everything.
  12. eddb636: dataset staging refuses a repository that is not private (Vahid's review, S1). Reverted by commit 14.

  13. 93694c6: a thin pay-concept ratio is held to its bounds (Vahid's second review). On one to nine reporting households, a ratio past its plausibility bounds is held to them and flagged, rather than stopping the build. On ten or more it still refuses.

  14. 2e01592: commit 12 is reverted. By María's ruling of 2026-10-26, a new Hugging Face request must not keep a build from staging, and disclosure is handled with the outputs' consumers. The dataset pre-flight and upload behave as before commit 12.

Builds

All local and licensed; the outputs stay with the runs.

  • National: spine, then build and release-cut certification at 67ee2dd. Published 2026-10-25, promoted 2026-10-26.
  • Dense: build 1 at 67ee2dd stopped at the crashed gates. It was retried at 7dac4c3 from the stored solve (--require-solve-reuse) and blocked only by gate-only fit failures. The reuse build at ee178eb passed every release-blocking gate. Verify found the solve reused and no row differing, the pre-flights and assembly passed, and it was published 2026-10-26.

The next build

  • The merge brought main's target and gate declarations (Write policyengine-uk's alcohol, tobacco and gambling duty inputs from the LCFS stage #1381 and others), so the next pair needs a new spine and a full solve whatever this PR does. Commits 8 to 10 also re-key the spine stage, target and gate nodes.
  • Without the withholding the dense calibrated gates take under a minute (53 s in the replay below). Before, the differencing check ran for about 3.5 hours and then failed.
  • The fit gates now judge the rows that were withheld, so a row resting on few households that misses its bound fails like any other and needs a fix or a sign-off.
  • Local launchers that pass --withhold-thin-rows-only must drop it; the flag is gone.

Tests

Run locally on the touched surfaces:

  • the take-up gate and contract, the gate-battery pins, the data contract, the battery bindings and the terminal-gate pins;
  • the sign-off drafter, the build CLI, the diagnostics, the full gates, the terminal kernel, the national graph, release certification and the Orrery contract and publication;
  • the candidate pre-flight, the assembler, and the coverage manifest and H2 fixture checks;
  • the stage-health gates, the gambling top-up and the annual-income stage, with the H2 spine parity check rerun after commit 10;
  • for commits 11 and 12: the Orrery contract and publication (including a synthetic refused build with --publish-orrery, and a build whose summary provider regresses), the calibration graph's summaries, dataset staging, the rowwise candidate and national role, and the build CLI;
  • for commits 13 and 14: the annual-income stage, dataset staging, the rowwise candidate and the build CLI, with the H2 spine parity check rerun.

The withholding tests are replaced by ones that a thin row keeps its figures and is judged, through the real bindings and the gate kernels. CI runs the full suites.

Replays on real data. The merge stops the stored solves from being reused at this head. So the gates were replayed with commits 7 to 9 applied on each published build's own commit, on its stored solve: the solve came from the graph store and the gates ran on the new code. All the outputs stayed local.

  • Dense (build 2 at ee178eb, 24,303 rows): the calibrated gates ran in 53 s, and every row carries its figures.
    • The fit gates judged the 1,650 local cells and 37 national rows build 2 had withheld. Two of those cells miss the local bound; every national row passes.
    • Every other gate reads as build 2's.
  • National (a6299504 at 67ee2dd, 1,268 rows): the national gates ran in 9 s. uk_target_fit judged every row, the 69 that were withheld included, and passed. No gate changed.

Disclosure

No figure in this description rests on fewer than ten source households or names a record. The rates are the contract's, the under-2 deviations are aggregate shares, and the sign-off counts are register entries.

From commit 8 on, the builds' private outputs carry figures for rows and areas resting on fewer than ten source households. They go to policyengine/populace-uk-private, which only people with a UKDS licence can read; the telemetry's fit summary goes to policyengine/populace-uk-staging. María handles disclosure with the outputs' consumers, the calibration-diagnostics dashboard included, separately.

Public outputs keep their checks: the Orrery is opt-in and its summaries carry no row's figures or failure lines (commit 11), and the drafter refuses a figure in a public register entry.

🤖 Generated with Claude Code

juaristi22 and others added 2 commits October 25, 2026 08:45
The first national on policyengine-uk 2.151.0 failed release-cut certification on uk_take_up_signal alone: the under-2 usage share (max_free_entitlement_hours_used, over the children of DfE age 0 or 1 in units that claim the working-parent hours) sat just past 0.05 from its ages' weighted contract rate. On the spine's own weights the draw meets its rates. DfE's working-parent row at 9 to 11 months binds a thin cell of that draw's children, and the calibrated weights that fit the row move the share further from its rates than they move any other flag's. By María's ruling of 2026-10-25 the row stays bound and this one share takes its own bound (under_two_usage_maximum_share_deviation 0.10); every other flag keeps 0.05. The gate report records each flag's bound, and the binding routes the new parameter.

The battery and release-cut gate digests are re-pinned; the spine, calibration-seam and dense local-gate digests do not move. The battery-bindings fixture's take-up stub takes the real binding's parameter keys.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#1383 moved the engine to 2.151.0 without refitting the six childcare rates, which were fitted on 2.147.2. The tool-version-7 fitter, run on the release-candidate spine built at fe1e2ad (feed abf0921), gives Tax-Free Childcare 0.9134 (was 0.9697), working-parent hours 0.8217 (0.8262), targeted hours for 2-year-olds 0.8687 (0.6551) and under-2 usage at DfE age 0 0.7000 (0.6962); age-1 usage stays at its upper bound and the conditional universal rate at its lower bound, with their reasons restated on the new fit. The targeted rate rises because 2.148.1 reads Universal Credit earnings before the work allowance, so fewer families qualify for the targeted two-year-old offer against the same DfE count; on the old rate the national calibration fitted that row by reweighting. Folded into the pair's rebuild by María's ruling of 2026-10-25.

The fit overrides the take-up columns and reads only the stored ages and months, so the spine the rebuild draws on this contract leaves it valid. The H2 spine fixture's stage-contract digest moves; the coverage manifest does not. The contract test's rate pins follow the refit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor

@juaristi22, a heads-up before I touch anything near this PR. Max ruled on 25 October (d2013, d2014) that CenTax's HMRC tabulations of parents' incomes become Microcosm UK calibration targets before the £100k childcare dashboard is republished, so I'm adding them. The work goes on a new branch stacked on this one (head 67ee2dd). I won't push here.

What I'm adding:

  • A committed resource, uk/centax_childcare_parent_facts.json. It is built from the workbook Arun sent Vahid on 21 October (sha256 8c238557…), with the report URL and figure and table ids. Its rows ride beside the Chronicle feed the same way high_value_home_stock_facts.json does.

  • Person measures, in a new uk_runtime/centax_parents.py. A parent is an adult in the benefit unit of a child eligible for the 3-and-4 hours at some point in the model year (DfE age 3 or 4, or 5 while still before reception), in England.

  • A new target family in uk_population_targets.json:

    • parents by adjusted net income in wide bands (Fig. 3.1), and the same split by employee or not (3.3), by sex (3.6) and by region (3.7, as shares);
    • live DfE accounts below £100k (2.1);
    • the Table A.1 totals.

    Each share becomes a count. None of CenTax's model outputs is a target: no bunching, no elasticities, no costs.

  • Your files: I don't expect to touch take_up_contract.json, frs_take_up.py or battery_bindings.py. If the new family needs a gate in gates.json, I'll say so here first and rebase onto your version.

The design calls go in the PR body for your sign-off: the bands, ageing the 2021-22 figures to 2025, CenTax's parent identification against the FRS, and its child definition against dfe_spring_term.py.

For numbers, I'll rebuild the spine at 67ee2dd on our side and check it against your a6299504 input (593713a6…). Then I'll build the national with and without the new family and rerun the dashboard's static job on both. Nothing gets published or released.

@MaxGhenis

Copy link
Copy Markdown
Contributor

@juaristi22, a question about the pair's certification, from the UK hub (25 Oct, 23:30Z).

Max ruled on d2011 (option A): certify your 2.151.0 national and its local-area cut together, by Tue 27 Oct 18:00Z. If that misses, the fallback is a133 with #699's area routing (policyengine.py#715).

Your national is now a Hugging Face tag: microcosm-uk-2024-25-national-20261025T091414Z-a6299504, at commit 994cac89. Its microcosm_uk_2024_25.h5 hashes b96a8cd6…064e, the same as the staged copy. latest-national.json on main still points at a133f37a.

Two questions:

  1. The local-area cut. When do you expect its tag, and what will it be called?
  2. The policyengine.py certification PR. Are you opening it, or shall the UK hub? I don't see a branch for it. The hub will open it as soon as the local-area cut is tagged, unless you've started one, so there's one writer.

For whoever writes it, the triple-lock repo (uk-triple-lock#39) checks two things in the bundle:

  • region_datasets is exactly national → microcosm_uk_2024_25.h5, with constituency and local_authority → the local-area file;
  • the local-area file's path_template.

It also asks whether the bundle can declare the calibration year. The build's run_config.calibration_year is 2025, but the release manifest at the tag declares only household_weight_year (dataset_year 2024).

The first dense build on policyengine-uk 2.151.0 produced no gate report: the differencing check behind the model-figure withholding, which since #1383 runs over each region's rows of every grain and each local family with its parents, ran for hours on the dense surface and then failed to converge in an eigendecomposition. By María's ruling of 2026-10-26, --withhold-thin-rows-only keeps the withholding of the rows resting on fewer than ten source households and skips that check, in the calibrated gates, the diagnostics writer and the run's evidence summaries, so each BRMA row's carrier verdict stays true to its count.

The gate code it would otherwise change is shared with the source preflight, which keys the solve, so the build command arms the switch at run time and records it on the calibrated gates node and in the build request: only that node and what follows it re-key, and a retry with --require-solve-reuse replays the stored solve. It needs --no-publish-orrery, and the national role refuses it; a default request is unchanged byte for byte. Tests cover the refusals and that only the calibrated gates node is marked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 and others added 2 commits October 26, 2026 10:51
Since María's rule of 2026-10-21 a thin national or regional row is withheld as a local cell is, so targets.csv leaves its figures blank. The drafter refused any such row ("targets.csv withholds a national row"), and it held the national fit gate's withheld count to the national rows alone, where the gate counts the surface's whole withheld set (its parity evidence carries it). It now splits withheld national rows out beside the local ones, signs none of them, matches each gate's count and compares both sets between the builds. The test fixture's national gate counts the whole set, as the binding does, and a new test withholds a national row.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drafted by tools/draft_uk_dense_signoff.py from the dense build-1 attempt uk-local-candidate-f100-s42-20261026T091056Z-824870f6 (the retry at 7dac4c3 with --withhold-thin-rows-only), approved on 2026-10-26 by juaristi22 under her standing instruction for this release: one national target-fit row and six local target-fit cells added, the entries still earned kept, and ten national and fourteen local entries the build no longer earns retired. Every entry is scoped to the local line and suppresses only its release fence; the rows stay bound in the solve. The reuse build runs at this commit with gate clock 2026-10-26.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 and others added 4 commits October 26, 2026 13:30
The national (a6299504, built at 67ee2dd) and the local-area release (c9df4b86, built at ee178eb) are published, so the branch takes main by a merge, keeping both build commits in its history. Main and the branch had each re-pinned the gate-battery and release-cut digests and regenerated the H2 spine fixture: the digests are re-pinned on the merged gates.json (the spine, calibration-seam and dense local-gate digests do not move) and the fixture is regenerated on the merged stages; the coverage manifest is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A UK build that staged to Hugging Face submitted its saved graph to the
public Orrery by default, and --no-publish-orrery turned that off. By
María's ruling of 2026-10-26 the submission now happens only on
--publish-orrery, whatever the staging; --no-publish-orrery states the
default. Staging to the private repositories is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
By María's ruling of 2026-10-26 the UK releases go only to the licensed
private repository, whose users need the full diagnostics, so a build no
longer withholds model figures (microcosm#1262, #1383):

- A target row resting on fewer than ten source households, and a row
  that would give one back by difference, keep their estimates and errors
  in the calibration diagnostics, targets.csv, the dense and national gate
  reports, the national calibration evidence and the run's evidence
  summaries.
- The fit gates judge every row; no binding counts withheld rows, and no
  report guard looks for one.
- Totals, losses and the size receipt are the solve's own, and staged
  epochs carry their loss again.
- BRMA rows carry no carrier verdict, and the release contract no longer
  warns about one.

This removes the thin-row count, the differencing check (its block spans,
residual search and eigendecompositions, which ran for hours on the dense
surface and then failed to converge) and --withhold-thin-rows-only. The
withholding code is gone from the gate modules, so the next dense build
re-solves. Release certification refuses a run built while rows were
withheld, rather than judging it without them; the release contract still
checks the withheld rows of earlier releases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
By María's ruling of 2026-10-26 the dense releases ship per-area support
figures again (microcosm#1301), since they go only to the licensed private
repository:

- The area-support gate states an area's rows, Kish ESS and distinct
  sources beside the floors it misses, in its failures and its reviewed
  exclusions, and reports each level's minima beside its counts.
- The diagnostics' weakest-area and support-limited rows carry the figures
  beside support_below_floor.
- The candidate manifest's support block states the minima.

The release contract no longer walks a release directory for these
figures, and neither does the candidate pre-flight or the sign-off
drafter's verify over the signed report, so no dense release raises a
contract warning. The walk itself stays for the drafter, whose drafted
entries go into a public pull request. --publish-orrery's help says the
gate summaries now carry the figures too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vahid-ahmadi

Copy link
Copy Markdown
Contributor

@juaristi22 round 1, at 6ed063a58.

Verdict: nothing record-level is newly published, and the default path is safer than before. One must: --publish-orrery is now the only route to the public Orrery, and with the withholding gone it would publish thin-row figures, guarded only by help text. The refit itself looks sound. I have not repeated Max's two comments (CenTax targets; certification timing).

Privacy and disclosure

What I checked:

  • Record-level data. No commit changes what goes into the h5 or where it goes. UK staging still defaults to policyengine/populace-uk-private (uk_runtime/staging.py:24-27). I checked with a read-only login: it and populace-uk-staging are private on Hugging Face, and an anonymous request gets 401.
  • Orrery default. publication_enabled now returns true only on --publish-orrery (orrery_publication.py:58-66). Before 5ff449d, any build that staged to Hugging Face published by default. Good change.
  • Public sign-off PRs. The drafter still refuses per-area figures in drafted entries (tools/draft_uk_dense_signoff.py:1870), so the committed reviewed-exclusion registers stay free of them.
  • What the private release now carries. Thin-row estimates and errors (58560fd), plus each area's rows, ESS and distinct-source count beside the floor it misses, and each level's minima (gates.py:2532, :2727, :2756). These are aggregates, not records. Inside a repository limited to licensed users that is a licence question, not a breach. Note this reverses the 17 Oct ruling (UK private rent at the assigned area (#1123 item 4, "PR B"): what is in place and the pieces still missing #1262), which withheld the same figures from the same private destination. So who can read that repository is now the question everything rests on (S2).

Must

  • M1. Make --publish-orrery safe in code, not in help text. _uk_result_summary no longer strips thin rows (orrery_contract.py:38-40; _withhold_summary_rows is gone), and the gate summaries now carry per-area support figures. The only guard is the help text at full_build_cli.py:856-866 and docs/orrery-publication.md:59-67, which says "check them against the UKDS licence before passing it". One flag on a dense or national build would put rows resting on fewer than ten source households on a public site. Suggest either refusing --publish-orrery on UK builds now that nothing is withheld (as --withhold-thin-rows-only refused it in 7dac4c3), or keeping a public-only strip that nulls thin rows and support figures in the Orrery summaries. Ideally the guard sits where the job is submitted, so the emitter's retry path (microcosm-emitter-publish-graph, docs/orrery-publication.md:76-80) is covered too. Please add a test either way.

Should

  • S1. Staging trusts the repo id, which POPULACE_UK_STAGED_DATASET_REPO_ID can override. Now that unwithheld figures ride along, a check that the target repo is private before upload would cost one API call.
  • S2. The PR body's Disclosure section ("No figure here rests on fewer than ten source households") describes the first two commits. Please update it for the last three, and say who can read populace-uk-private. The ruling rests on everyone with read access being a UKDS-licensed user.
  • S3. The under-2 bound goes from 0.05 to 0.10 (frs_take_up.py:619-675, gate uk_take_up_signal), but the body says the share sat "just past" 0.05. Please state the measured deviation, and consider a tighter bound (say 0.07) or an expiry, so a real drift on this flag still fails.
  • S4. The refit receipt credits the change to policyengine-uk 2.151.0, but the input spine changed too: 6aea1ec before, fe1e2ad after (take_up_contract.json, input_spine). Please say how much of each pool shift is the engine and how much is the other spine changes.

Childcare refit

  • Sources. These are unchanged: DfE spring-term counts via dfe_spring_term.py, and HMRC's June 2025 Tax-Free Childcare (TFC) statistics (1,151,515 children with used accounts; £599.8m top-up). Fitter tool version 7, seed 0, and the optimiser reports success.
  • Fit to DfE and HMRC. This is about the same as before:
    • targeted 2-year-olds 1.000;
    • 9 to 11 months 1.000;
    • working parents aged 2 to 4: 0.996;
    • age 1: 0.952 (was 0.943);
    • universal-only: 1.047;
    • TFC used accounts 1.063, top-up 0.928.
    • The diagnostic rows for working parents aged 2 and 3 stay off: 1.093 and 0.924.
  • What the rates imply about the eligible pools, since the fitted counts barely move:
    • TFC: 0.9697 to 0.9134 (take_up_contract.json:171), with used accounts and top-up nearly unchanged. So the TFC-eligible pool on the new spine is about 6% larger (0.9697 / 0.9134 = 1.062).
    • 30 hours (working parents): 0.8262 to 0.8217 (:268). The pool is about 0.5% larger.
    • Targeted 2-year-olds (15 hours): 0.6551 to 0.8687 (:657). The pool is about a quarter smaller (0.6551 / 0.8687 = 0.75). This is consistent with 2.148.1 reading UC earnings before the work allowance, but it is a large move. A line in the body with the before and after eligible counts would settle it.
    • Under-2 at age 0: 0.6962 to 0.7000 (:363). Age 1 stays at its upper bound and the conditional universal rate at its lower bound.
  • Downstream. Costings that assume newly eligible families take up TFC at the fitted rate, as the receipt's estimand notes, get about 6% less TFC take-up per newly eligible family at an unchanged pool. The £100k childcare dashboard will need a rerun once this lands. The 30-hours change is small.

What I ran

  • Fresh worktree at 6ed063a58, Python 3.14: take-up gate and contract, Orrery publication and contract, full-build CLI, shared gates, dense sign-off drafter, release certification, battery bindings, dense release contract and data contract. 1,122 passed, 1 skipped.
  • Read the five feature commits' code diffs. Skimmed only the large deletions in diagnostics.py and its tests, the docs and runbook edits, and the reviewed-exclusion retirements (ee178eb).
  • No spine or national build, so I could not measure the eligible pools directly. The pool shifts above are inferred from the receipt's rates and achieved values.
  • CI: all ten checks pass, and the PR is mergeable.

Aggregates only.

🤖 Generated with Claude Code

By María's ruling of 2026-10-26 a small-cell rule may only flag. Two
stages refused, and several spine gates failed, on a figure a receipt
prints "<10".

- The gambling top-up refused a key whose donor recorded stakes on fewer
  than ten households, or left fewer than ten without. It now refuses only
  below the two a side its stratified fit needs.
- The annual-income stage refused a pay ratio fewer than ten households
  report. It now refuses only when none does, and its receipt prints the
  ratio "<10".
- The gambling gate failed a key whose national shares the receipt
  withholds. The property-purchase gate errored on a kind's banded
  figures, and the UC take-up gate failed a banded caseload. The
  annual-income gate failed a banded ratio, and the regional home-value
  gate errored on a banded region count.

Each now passes on what it cannot read and names it in its details. The
excise and legacy-closure gates already passed on their banded figures
and now name them too.

Counts a gate requires to be zero, and the declared donor-support floors,
fail as before. A band in either still tells a failure apart.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22 juaristi22 changed the title Bound the under-2 usage signal on its own and refit childcare take-up on policyengine-uk 2.151.0 Build the national + local-area pair on policyengine-uk 2.151.0, and drop UK model-figure withholding Oct 26, 2026
Maria Juaristi and others added 2 commits October 26, 2026 17:47
The summaries are what --publish-orrery submits to the public Orrery.
Since the build stopped withholding thin rows, nothing marks the rows
resting on fewer than ten source households, so the summaries could
publish their figures (Vahid's review of #1396, M1):

- a calibration summary keeps each target row's target, tolerance and
  source, but no row's estimates or errors;
- the size summary keeps the losses over every row, but not the largest
  change in one row;
- a gate summary keeps each gate's verdict and its count of failure
  lines, but not the lines, which can state a row's figures or an
  area's support.

The build's private diagnostics and gate reports keep all of them.

The export checks the summaries before it preserves anything for
publication, and stops with the unpublishable_summary error code if one
carries either. With no preserved inventory, neither the build nor the
emitter's retry command can submit them. The --publish-orrery opt-in
stays, as María chose.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The staged bundle carries the build's full diagnostics, and its
repository id can come from POPULACE_UK_STAGED_DATASET_REPO_ID or
--staged-dataset-repo-id (Vahid's review of #1396, S1):

- the pre-flight refuses to start a remote dataset stage when the Hub
  does not report the repository as private, before the spine is read;
- the upload itself, shared by the build, the failure-evidence bundle
  and the manual stage tool, refuses with REPOSITORY_UNAVAILABLE before
  anything is read or written.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

@vahid-ahmadi Thanks for round 1. M1 and S1 are fixed on this branch (0862fc5, eddb636), and the body is updated for S2 and S3. S4 is below.

M1. --publish-orrery is safe in code. The opt-in stays, as María chose. The figures now come out where the summaries are made, so every path that can submit them only sees the stripped copy.

  • Calibration summary: keeps each target row's target, tolerance and source, but no row's estimates or errors (_uk_result_summary).
  • Size summary: keeps the losses over every row, but not max_target_scaled_change, which is one row's figure.
  • Gate summary: keeps each gate's verdict and a failure_count, but not the lines, which can state a row's figures or an area's support (_gate_rows).
  • The private diagnostics, targets.csv and the gate reports keep all of them.

The guard. finalize_graph checks the recorded summaries (public_summary_findings) before it preserves anything for publication. If a summary carries a row's estimates or a failure line, it stops with unpublishable_summary and writes no orrery.upload.json. That covers the retry: microcosm-emitter-publish-graph submits either that inventory or a job the build already queued. The check can't sit in the emitter itself, because the retry command lives in the microcosm-emitter package.

Tests:

  • the providers on a thin BRMA row and on a failed gate;
  • the summaries of a real calibration graph;
  • a synthetic refused dense build with --publish-orrery, whose preserved publication counts the failure lines without carrying them;
  • a build whose gate provider is made to regress: it leaves no inventory and submits nothing.

S1. Staging checks that the repository is private. HuggingFaceDatasetStorage.private() reads repo_info().private.

  • The pre-flight refuses a remote dataset stage, before the spine is read, when the Hub does not report the repository as private. The same holds when it reports nothing.
  • stage_bundle refuses with REPOSITORY_UNAVAILABLE before it reads or writes anything. The build, the failure-evidence bundle and tools/stage_uk_rowwise_candidate.py all share it.

S2. Disclosure. The body's Disclosure section now covers commits 8 to 12. Only people with a UKDS licence can read populace-uk-private, and commit 12 refuses to stage anywhere the Hub does not report as private. The calibration-diagnostics dashboard is being handled separately.

S3. The under-2 usage signal. The measured absolute deviation from the rates' expectation:

Build Deviation Bound Verdict
fe1e2ad (first national) 0.0516 0.05 failed
67ee2dd (a6299504) 0.0521 0.10 passed

By María's ruling the bound stays at 0.10. The body's commit 1 now states both deviations.

S4. Engine against spine. I ran the same fitter (tool version 7) on the old spine (6aea1ec) under 2.151.0, and compared it with the two committed fits. The pools are the fitter's ceilings at a take-up rate of 1, on the spine's own weights, to the nearest 1,000.

The fitted counts barely move, as you noted, so these pools are what the rates moved against.

Downstream. Agreed: the £100k dashboard reruns once this lands.

Aggregates only.

@vahid-ahmadi

Copy link
Copy Markdown
Contributor

@juaristi22 round 2, at eddb636c2.

Verdict: M1 and S1 are fixed in code and tested, S2 and S3 are in the body, and the S4 answer settles it. Nothing here blocks. Three shoulds and a nit below. One of them is a gap I left in round 1: S1 named only the dataset repository, not the telemetry one.

Round 1 status

  • M1. Fixed. I tried to find a path that still publishes thin-row or per-area figures. I found none in code:
    • The providers strip them at source: target rows lose initial, achieved, residual, relative_error and within_tolerance (orrery_contract.py:42-48, :60-63), and gate rows carry failure_count, not the lines (:216). The other gate fields are ids, verdicts and fixed reason text. The other size fields are counts and totals.
    • finalize_graph runs public_summary_findings before anything is preserved (orrery_publication.py:135-139). Both callers go through it: the success path and the failure-evidence path (full_build_cli.py:1806, :1837). On a finding, names is still empty, so only the receipt is written and no orrery.upload.json exists for the emitter's retry.
    • The check reads every recorded summary in every run, so it covers the per-run summaries.json files that go up as their own files.
    • The other uploaded files carry no row figures. The run manifests (role manifest) hold node receipts: outcomes, mass logs and rounded_count figures. Commits 8 and 9 added nothing to them. Upstream files are the spine's declaration, manifest, schema and evidence index, with no summaries. orrery_reconstruction.py is declaration-only.
    • Only --publish-orrery turns on publication (orrery_publication.py:68, default None at full_build_cli.py:876). No env var, launcher or other caller sets it. Telemetry events carry gate ids, counts and statuses only.
  • S1. Fixed for the dataset bundle. The pre-flight and stage_bundle both refuse a repository the Hub does not report as private (rowwise_staging.py:156-171, staging_dataset.py:603-620). Today, with a read-only login, populace-uk-private and populace-uk-staging both report private, and an anonymous request gets 401. See S5 for the telemetry repository.
  • S2. Done. The Disclosure section now covers commits 8 to 12. I can't check from outside who can read populace-uk-private, so that rests on your statement.
  • S3. Done. 0.0516 and 0.0521 are stated. Keeping 0.10 is your call, and I won't press it.
  • S4. Accepted. You ran the counterfactual I asked for. Your measured pools supersede the figures I inferred from the rates in round 1. Targeted 2-year-olds fall 22%, not "about a quarter". Working-parent hours rise 0.3%, not 0.5%. TFC matches: 1,263,000 to 1,340,000 is 1.061, against my 1.062. Withholding the spine's part of the 2-year-old shift on the small-cell rule is right for a public thread.

New

  • S5 (should). Give the telemetry repository the same private check. fit_summary.json goes to populace-uk-staging, which POPULACE_UK_STAGING_REPO_ID can override (uk_runtime/staging.py:19-22). Its writer has no private() check (staging_v2.py:869). Since commit 8 the summary carries max_abs_relative_error and per-family fit over all rows, thin ones included (rowwise_staging.py:831-841). It also carries dataset_size.max_target_scaled_change (:849-858), the figure commit 11 strips from the Orrery as one row's. The Disclosure line "they go only to populace-uk-private" should then name this repository too.
  • S6 (should). Make the guard an allowlist. public_summary_findings (orrery_contract.py:297-325) looks for five target fields, max_target_scaled_change and failures. The regression test re-adds exactly failures. A provider that regressed under any other name (estimate, lines, an area table) would pass. Refusing any target-row, gate-row or table key outside the projected set would make it the guard the body describes.
  • S7 (should). Preserved directories from commits 8 to 10. At those commits the summaries carried the figures, and finalize_graph wrote orrery-publication/orrery.upload.json whatever the flag. microcosm-emitter-publish-graph --directory will submit such a directory with no check. Did any build or replay at those commits reach finalize_graph? If so, please delete those preserved directories. Code can't close this one.
  • Nit. pay_concept_ratio still applies PAY_CONCEPT_RATIO_BOUNDS when one to nine households carry the ratio (frs_annual_income.py:809), and the receipt then prints it "<10". A noisy thin ratio can still stop a sample build. That may be intended as a plausibility check rather than a small-cell rule. If so, a comment saying so would help.

Childcare effects: unchanged. take_up_contract.json is untouched since 6ed063a58, so the four rates still stand. TFC take-up per newly eligible family is about 6% lower. The targeted 2-year-old pool is about 22% smaller (measured), not a quarter. The £100k dashboard still needs its rerun.

What I ran

  • Fresh clone at eddb636c2, fresh uv venv, Python 3.14, one process: Orrery contract and publication, stage health, annual income, gambling, rowwise candidate, calibration graph, dataset staging, build CLI, take-up gate and contract. 423 passed, 1 skipped (counted from the progress output). No data builds.
  • Read all three new commits' code diffs, and the evidence, receipt and telemetry paths named above.
  • CI: lint, wheels, engine-uk (3.13, 3.14) and integration-uk pass. engine-free and engine-us (3.13, 3.14) were still running at 18:55 UTC, 41 minutes in. This review stands at eddb636c2 either way.

Aggregates only.

🤖 Generated with Claude Code

Maria Juaristi and others added 2 commits October 26, 2026 19:33
The ratio's bounds are a plausibility check on the two pay concepts, but
on one to nine reporting households a ratio past them is noise, and it
could still stop a sample build (Vahid's second review of #1396). Such a
ratio is now held to the bounds; the receipt still prints it "<10" and
the gate reads it unmeasured. On ten or more households a ratio past the
bounds still refuses, since there it means the concepts do not line up.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This reverts commit eddb636. By María's ruling of 2026-10-26 the
build's publishing behaviour stays as it was: a build must not be kept
from staging by a new Hugging Face request, and disclosure is handled
with the outputs' consumers. The dataset pre-flight and stage_bundle no
longer ask the Hub whether the repository is private.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

@vahid-ahmadi Thanks for round 2. The nit is fixed (93694c6) and S7 needs no deletion. S5 and S6 are not taken, and S1's check is reverted (2e01592), by María's ruling: the build's publishing behaviour stays as it was, and she handles disclosure with the outputs' consumers. The body is updated.

S5. Not taken. A new Hugging Face request must not keep a build from staging. For the same reason S1's check is reverted: the dataset pre-flight and the upload no longer ask the Hub whether the repository is private, and behave as before. The body's Disclosure section now names populace-uk-staging as where the telemetry's fit summary goes.

S6. Not taken, on the same ruling. The M1 guard stays as it is. The providers still strip the figures at source.

S7. No preserved directory from commits 8 to 10 exists.

  • No build ran at those commits. The replays ran the gate nodes on stored solves and never reached finalize_graph.
  • No orrery-publication directory on this machine postdates commit 8.
  • The local emitter queue holds no graph job.
  • Every publication receipt here reads either skipped (local only) or failed before enqueue, and no retry status file exists. So nothing from these builds was ever submitted.

Nit. On one to nine reporting households, a pay-concept ratio past its plausibility bounds is now held to them rather than stopping the build. The receipt still prints it "<10", and the gate reads it as unmeasured. On ten or more households it still refuses, since a ratio past the bounds there means the two pay concepts don't line up.

Aggregates only.

@vahid-ahmadi

Copy link
Copy Markdown
Contributor

@juaristi22 round 3, at 2e01592c7.

Verdict: one must, S6. We have raised it since round 2, for the reason below. The nit and S7 are closed. S1 is reopened by the revert and, with S5, stays a should. I won't press either if you keep your ruling, but the reason given doesn't hold for the pre-flight. CI has not run at this head.

Status

  • S6. Must, raised from should. The guard is unchanged (orrery_contract.py:297-325). It still looks for a fixed list of names: the five target fields (:42-48), max_target_scaled_change (:50) and gate failures (:323). I tested it at this head. A summary with a target row carrying estimate, a gate row carrying lines, a new areas table with per-area values, or an unknown overview field all return no findings. The export would go ahead. So the guard fails open. We have raised it to a must because this check is the last thing between record-level figures and a public output. A guard like that has to fail closed: refuse anything it does not know. Your ruling keeps the build's publishing behaviour as it was. An allowlist keeps that too, since today's providers emit only known fields and would pass unchanged. It only changes what happens when a provider regresses. The size provider already lists its fields (orrery_contract.py:132-157). The guard could check each summary's overview keys, table names, and target and gate row keys against declared sets like that one, and refuse the rest. A test that adds an unknown field and expects a finding would pin it.
  • S1. Reopened by the revert, should. 2e01592c7 takes the private check out of the pre-flight and stage_bundle (rowwise_staging.py:152-155, staging_dataset.py:598-599). The repository id can still come from POPULACE_UK_STAGED_DATASET_REPO_ID (uk_runtime/staging.py:26). On the reason given: the pre-flight already calls repo_info through head_revision (staging_storage.py:201-208), and private is in that same response. Reading it there adds no Hub request, and the pre-flight already stops the run if that call fails. It would only refuse a repository that is not private, which is the case it exists for. That is my suggestion. Leaving stage_bundle as it is now is fine.
  • S5. Not taken, should. The telemetry writer still builds its transport with no private check (staging_v2.py:869-873), and POPULACE_UK_STAGING_REPO_ID can still redirect it (uk_runtime/staging.py:21). The Disclosure section now names populace-uk-staging, which helps. Privacy is checked nowhere in code. Today, with a read-only login, the Hub reports both populace-uk-private and populace-uk-staging as private, and an anonymous request cannot see either. That holds only until someone changes a setting or an env var.
  • S7. Closed. Thanks for checking. No build ran at commits 8 to 10, and no orrery-publication directory or queued job exists. Nothing to delete.
  • Nit. Fixed. On one to nine households the ratio is now clamped to PAY_CONCEPT_RATIO_BOUNDS, with a comment saying why. Ten or more still refuse (frs_annual_income.py:809-820). The new test covers both sides (test_uk_frs_annual_income.py:539-547).

New since eddb636c2

No new findings. The revert is clean: no private() caller or test is left behind, and the changed files pass ruff check and ruff format --check.

CI

No workflow has run at 2e01592c7 or 93694c68f. GitHub reports the PR as conflicting with main, so the pull-request checks won't start until that is resolved. The last run, at eddb636c2, passed.

What I ran

  • Fresh clone at 2e01592c7, fresh uv venv, Python 3.14, one process. Annual income, dataset staging, rowwise candidate, Orrery contract and publication: 152 passed. A separate probe of public_summary_findings with the four unknown fields above: all four returned no findings. No data builds.
  • Hub visibility through the huggingface_hub API, read-only.

Aggregates only.

🤖 Generated with Claude Code

The branch takes main by a merge, keeping both published builds' commits
in its history. Main and the branch had each re-pinned the gate-battery
digests and regenerated the H2 spine fixture: the policy, gates-manifest
and spec-fingerprint digests are re-pinned on the merged gates.json (the
under-2 usage bound and #1400's gambling stake scale both in it), and the
fixture is regenerated on the merged stages. The certification part and
dense local-gate digests do not move.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

@vahid-ahmadi Thanks for round 3. Main is merged in (d8ed5a2), so the PR no longer conflicts and CI can run. S6, S1 and S5 are not taken, by María's ruling: the build's publishing behaviour stays as it was, and she handles disclosure with the outputs' consumers.

S6. The guard stays as M1 left it. The providers still strip the figures at source.

S1 and S5. Reading private from the repo_info the pre-flight already makes would add no request, as you say. But it would still change what a build does when the repository isn't private.

CI. The conflict was main's #1400 (policyengine-uk 2.162.3). Both sides had re-pinned the gate-battery digests and regenerated the H2 spine fixture.

  • The three battery digests are re-pinned on the merged gates.json, and the fixture is regenerated. The certification part and dense local-gate digests don't move.
  • On 2.162.3 these tests pass locally: the pins, the battery bindings, gambling, stage health, the take-up contract, annual income, the Orrery and the H2 parity check.

Aggregates only.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants