[MaxGhenis]

Fixes #2401

Reproducer: policyengine_uk/tests/test_hbai_negative_income_properties.py::test_reset_income_and_its_measures

Also guards #2411's gap: policyengine_uk/tests/test_hbai_housing_costs_properties.py::test_service_charges_do_not_change_computed_bhc. This is a computed-BHC check that service charges never reduce BHC. It does not reproduce #2401.

HBAI resets negative household income before housing costs (BHC) to zero, then derives income after housing costs (AHC) from that reset figure. This implements the DWP HBAI negative-income convention.

Behaviour

• hbai_household_net_income = max(0, hbai_household_net_income_before_reset_to_zero). The new before-reset variable preserves the previous component accounting, including trading losses and the Council Tax abolition branch.
• AHC subtracts all five housing costs from reset BHC: rent, water/sewerage, mortgage interest, structural insurance and housing_service_charges (ground rent and property service charges). AHC can remain negative.
• Equivalised/real HBAI incomes, poverty measures and deciles follow the reset. Negative incomes formerly excluded with decile −1 now participate in ranking. Ties at zero land in decile 1 when they hold at most 10% of person weight.
• Labour supply/progression defaults use the before-reset variable to preserve the actual budget constraint. Non-HBAI household net income is unchanged.

Adoption and merge

Adopted under the UK hub's fallback with the author and existing history preserved. Main 9a9b52cb8c25c617a5494b7cb57eaa0df9811906 is merged into head 06f47f7885ac64b3b63107432f382f1debface19, including #2411, #2407, #2415 and #2391; no rebase or force-push.

The two conflicts were reconciled as follows:

• docs/book/validation/hbai.md retains the reset explanation, all five housing costs and Deduct ground rent and service charges from HBAI AHC income #2411's FRS service-charge guidance. It also incorporates the prior round-3 wording correction about people at or below zero.
• hbai_household_net_income_ahc.py retains the reset BHC input, Deduct ground rent and service charges from HBAI AHC income #2411's fifth subtraction, metadata explaining both mechanisms, and references to both DWP methodology sections.

The reset properties now include service charges in their independent oracle and positive generated/pinned inputs. Four YAML cases include charges and adjusted AHC/equivalised/gap expectations. #2411's P3 follow-up is closed by a computed-BHC check in 2024–2026 which computes positive BHC from earnings, raises service charges by £1,200, and verifies before-reset/reset BHC are unchanged while AHC falls by exactly £1,200.

Invariants

For before-reset income b and nonnegative total housing costs HC:

1. Reset BHC is max(0, b); equivalised and real BHC remain nonnegative. The before-reset figure equals the sum of income components less deductions.
2. AHC is max(0, b) − HC, with all five housing costs included exactly once. Service charges do not reduce BHC.
3. BHC/AHC totals increase by the weighted negative part of b. Poverty gaps decline by that amount on the measured data; BHC gaps cannot exceed their poverty lines. AHC gaps can exceed their lines by housing costs.
4. Absolute/deep poverty headcounts are unchanged for positive thresholds. Relative headcounts are unchanged while the respective BHC/AHC median is positive; a nonpositive BHC median becomes zero and no reset BHC income lies below its relative line.
5. All households receive deciles 1–10, ordered by equivalised income. Zero-income households share one rank; decile 1 requires that group to hold at most a tenth of people.
6. Dynamics use b, preserving responses to pay rises while measured HBAI income is floored at zero.

Validation at the adopted head

• Locked environment: UK 2.132.3, Core 3.33.2, Python 3.13.9, UV_LINK_MODE=clone uv sync --frozen --extra dev.

• make format: passed (Ruff formatting and lint). git diff --check: passed.

• Foreground uv run --frozen --no-sync pytest -n 4 --dist worksteal -o xfail_strict=true ... -q: 84 passed, no failures/skips, 111 existing MicroSeries warnings, 1229.55s. Files: test_hbai_negative_income_properties.py, test_hbai_housing_costs_properties.py, test_household_income_decile.py, test_relative_poverty_median.py, test_absolute_poverty_reference_year.py, test_labour_supply_response_formulas.py, test_labour_supply_dynamics_properties.py, test_behavioral_responses.py, test_marginal_rate_precision.py, test_pawhp_household_income_properties.py, test_trading_loss_properties.py (all under policyengine_uk/tests/).

• Foreground YAML with pytest -n 4 --dist worksteal -p hbai_yaml_plugin --junitxml=.job/pytest-yaml.xml: 15 passed, no warnings, 75.81s. Files: policyengine_uk/tests/policy/baseline/household/income/hbai_negative_income.yaml, .../hbai_housing_costs.yaml, policyengine_uk/tests/policy/reforms/parametric/hbai_council_tax_lvt.yaml. Workspace-only harness loads Core's collector per worker and assigns unique IDs; JUnit confirms every intended case passed exactly once. Core CLI -n filters names, so it was not used as a worker flag. An initial 15-pass invocation had duplicate IDs/teardown warnings and was superseded by this clean run.

• Main source provenance: all 2,931 archived tracked blobs match origin/main; local Git bundle verifies.

• Required uvx ruff format --check .: passed (3,420 files already formatted), with uv tool/cache paths inside .job/ for the sandbox.

• Both sequential 2026 microsimulations and the aggregate comparison checks passed. The manifest version compatibility check reports false for this newer model/Core pair, as disclosed below.

Expected-failure evidence: after merging #2411, introducing positive service-charge generated/pinned inputs while retaining the old four-cost oracle caused test_reset_income_and_its_measures to fail at its AHC identity (1 failed in 94.81s). It passes after the fifth oracle cost is restored. The evidence log is retained in the job folder.

The full suite was not run, as requested. The documentation change is prose in an existing page; the full book build was not run.

2026 enhanced-FRS impact

Exactly one real 2026 microsimulation per side, run sequentially using the owner's impact.py metrics, restricted to 2026 and augmented with provenance/resource guards and HBAI child/State Pension age groups. These are static baseline comparisons; the earlier reform-specific labour-supply and decile-impact runs were not repeated.

Data: uk-data 1.58.1, enhanced_frs_2024_25.h5; release manifest at HF commit 89be6b73fc1e6eeba6dae2f9400676d8510bcd51, artifact at 2526edbbb687a3b54dbb11272e5389e4b62d2ac2. Dataset SHA256: 056e80fd8e9151d78f432a25cde863340baee81304c6216a37fb575ce8563784. Manifest SHA256: 545797bf231cd3681e64b677065d921304e9a32f1e46e757a940a412b4136c17. Artifact bytes/hash match the manifest.

Compatibility limitation: the manifest's compatible_model_packages lists UK ==2.122.2 and ==2.130.1; compatible_core_packages lists Core ==3.32.13 and ==3.33.0. Both source revisions run UK 2.132.3 with locked Core 3.33.2, outside those exact certified versions. These measurements use the latest certified data artifact, but this newer model/Core pairing is not certified by that manifest. No version metadata was changed to bypass the check.

Main SHA: 9a9b52cb8c25c617a5494b7cb57eaa0df9811906. Branch SHA: 06f47f7885ac64b3b63107432f382f1debface19.

Outcome	Main	Branch
Negative BHC: survey households	677	0
Negative BHC: weighted households	142,005	0
Negative BHC: people	183,355	0
BHC income total (£bn)	1,514.306647	1,515.139365
AHC income total (£bn)	1,304.388435	1,305.221154
poverty_gap_bhc (£bn)	35.269392	34.436673
poverty_gap_ahc (£bn)	59.931716	59.098997
poverty_gap (£bn)	35.269388	34.436669
deep_poverty_gap (£bn)	5.250857	4.418138

All poverty headcounts below are exactly unchanged (millions of people; values apply to both sides).

Poverty measure	All	HBAI children	SPA pensioners
Absolute BHC	10.426039	3.140323	1.549055
Absolute AHC	14.212630	4.514379	1.445416
Relative BHC	11.455870	3.538076	1.757597
Relative AHC	14.713190	4.712467	1.506160

Children use is_hbai_dependent_child; pensioners use is_SP_age, accounting for the 2026 State Pension age transition. The owner’s under-18 and age-66+ groups are retained in the aggregate JSON and are also exactly unchanged.

Before-reset negative income remains identical: 677 survey households, 142,005 weighted households and 183,355 people. All 677 move from decile −1 to decile 1; every original nonnegative decile is exactly unchanged.

Post-reset zero-BHC people hold 0.2717% of the population, below 10%. The person-weighted equivalised BHC/AHC medians are £40,696.95 / £34,891.94, identical on both sides and positive.

Income-total and gap changes match the £0.832718568bn negative part within a recorded £953.67 aggregate float32 tolerance (largest observed residual £365.20). All provenance, matched-weight, reset, headcount and decile checks pass.

Preflight readings (decimal GB): main compressor 0.812, free disk 62.560; branch compressor 0.808, free disk 62.309. Both satisfy compressor <30 GB and free disk >60 GB. Both runs verify the dataset hash and size against the manifest.

Hand-off

Prior independent rounds 1–3 approved earlier heads, most recently 59d57bdafdb5feda95baa14092cc19bab5f992dc. The merged code/test delta requires the hub's independent delta review; no self-review was performed and this PR is not merged.

Retained as a draft because the manifest does not certify this newer model/Core pairing. All code tests and formatting checks pass. The hub owns independent delta review and certification follow-up.

axiom: n/a: statistics (the HBAI income measure), not law

[MaxGhenis]

UK hub: coordination with #2411 (ground rent and service charges in AHC), which lands first.

#2411 is approved at 6b1e19950, all its checks pass, and it is in the merge queue. Its independent review ran git merge-tree dd6484936 6b1e19950 59d57bdaf against this PR and found two overlaps and one test gap that this PR must take care of when it merges main:

1. docs/book/validation/hbai.md:47: reconcile both explanations. Keep Deduct ground rent and service charges from HBAI AHC income #2411's statement that housing_service_charges is the fifth AHC housing cost.
2. hbai_household_net_income_ahc.py: keep Deduct ground rent and service charges from HBAI AHC income #2411's fifth subtraction (housing_service_charges) and reconcile the metadata and reference lines (around L10).
3. test_hbai_negative_income_properties.py:
   • add housing_service_charges to the HOUSING_COSTS oracle (L54);
   • also add it to the positive generated and pinned inputs (L90), not just the list, so the reset property exercises the charge.

#2411's own tests supply BHC directly, so they stay valid after this PR's reset. The review found no incompatible income mechanism.

The UK hub will pick this PR up after #2411 merges.

[MaxGhenis]

UK hub: adopting under d1280's fallback (owner session gone). A hub build job (20261015-000509-adopt-uk2402) is merging main (with #2411), reconciling hbai.md and the AHC variable as the coordination comment above describes, adding housing_service_charges to the oracle and the generated inputs, and re-measuring on certified data. The PR author is unchanged, and no history is rewritten. A delta review follows.

[MaxGhenis]

UK hub hand-off — adopted #2402

Head: 06f47f7885ac64b3b63107432f382f1debface19. Main baseline: 9a9b52cb8c25c617a5494b7cb57eaa0df9811906. Original approved head: 59d57bdafdb5feda95baa14092cc19bab5f992dc. The author is unchanged; two commits were added without rewriting history, and the branch was pushed once.

Since the last approved head:

• Merged main (Deduct ground rent and service charges from HBAI AHC income #2411, Run each part's modifier at its own time in a sum of scenarios #2407, Remove agent working notes from the repository root #2415, Test winter heating payments for IS, JSA and ESA couples #2391), resolving the two predicted conflicts. AHC is reset BHC less all five housing costs; documentation retains the FRS charge guidance and both methodology references.
• Added service charges to the reset properties' independent oracle and positive generated/pinned inputs; updated four YAML cases.
• Closed Deduct ground rent and service charges from HBAI AHC income #2411's P3 gap: computed positive before-reset/reset BHC stays unchanged when charges rise £1,200; AHC falls £1,200, checked in 2024–2026.
• Incorporated the prior round-3 wording corrections for zero-income shares, relative medians and the conditional bottom-decile claim.

Validation: 84 Python tests passed, 15 distinct YAML cases passed, foreground -n 4, locked Core 3.33.2. Includes both HBAI properties, decile/poverty readers, dynamics/formulas/behaviour/precision, PAWHP and trading losses. The old four-cost oracle failed at the AHC identity after positive charge inputs were introduced, then passed with the corrected oracle. make format, git diff --check, and required uvx ruff format --check . passed. Full suite and full documentation book build were not run. The final YAML run has unique node IDs and JUnit confirmation of all 15 cases.

Exactly one real 2026 enhanced-FRS simulation per side, sequentially, using uk-data 1.58.1:

• 677 survey households / 142,005 weighted households / 183,355 people have negative BHC on main; reset BHC has none. All 677 move from decile −1 to 1; all other deciles are exactly unchanged.
• BHC total: £1,514.306647bn → £1,515.139365bn. AHC total: £1,304.388435bn → £1,305.221154bn. Both rise approximately £0.832719bn.
• BHC poverty gap: £35.269392bn → £34.436673bn; AHC: £59.931716bn → £59.098997bn; deep BHC: £5.250857bn → £4.418138bn. Reset delta invariants pass within aggregate float32 tolerance.
• Absolute and relative poverty BHC/AHC are exactly unchanged for all people, HBAI children and State Pension age pensioners (full headcounts in updated PR body). All other recorded poverty headcounts are also exactly unchanged.

Dataset SHA256: 056e80fd8e9151d78f432a25cde863340baee81304c6216a37fb575ce8563784; manifest HF commit 89be6b73fc1e6eeba6dae2f9400676d8510bcd51. Hash and size match. Preflight compressor/free disk: main 0.812/62.560 GB, branch 0.808/62.309 GB (decimal GB); both satisfy the requested limits.

Open item / draft retained: the manifest permits UK 2.122.2/2.130.1 and Core 3.32.13/3.33.0, whereas both runs use merged UK 2.132.3 / locked Core 3.33.2. The artifact is certified; this newer model/Core pairing is not listed as compatible. This is disclosed in the PR body and job records.

The hub owns the independent delta review and certification follow-up. No self-review, merge, dataset build, or data upload was performed. .job/HANDOFF.md, progress, aggregate outputs and handoffs.jsonl are the local hand-off artifacts; record-level NPZ files remain local and unstaged.

[MaxGhenis]

UK hub merge audit, head 06f47f7885ac64b3b63107432f382f1debface19 (adopted under d1280; author unchanged).

• Independent delta review: APPROVE WITH NITS at this exact head, executed (GPT-6.1 Sol, -s workspace-write, job 20261015-011420-uk2402-review-delta). Rounds 1–3 had approved up to 59d57bd.
  • Merge: nothing from main or Deduct ground rent and service charges from HBAI AHC income #2411 was lost. AHC is the reset BHC less all five housing costs.
  • Tests: 84 Python and 67 YAML pass.
  • Mutation checks: removing the oracle's service charge fails 2 properties. Removing Deduct ground rent and service charges from HBAI AHC income #2411's subtraction fails the computed-BHC check (2024–2026), the reset property and 7 YAML cases. A wrong BHC deduction is caught only by the new computed-BHC check.
  • Impact (2026, uk-data 1.58.1, sha256 056e80fd…3784, main 9a9b52c vs this head, same model, data and weights): the comparison reproduces byte for byte, and the reviewer's own branch rerun matched.
    • 677 survey households (142,005 weighted; 183,355 people) have negative BHC on main.
    • BHC and AHC totals each rise £0.8327bn, and the poverty gaps fall by the same.
    • All 48 headcounts (8 measures × 6 groups) are exactly unchanged.
    • The only decile move is −1 → 1 for those 677 households.
  • Source: DWP's HBAI FYE 2025 methodology resets negative BHC to zero and derives AHC from the adjusted BHC.
  • Certification: the 1.58.1 manifest doesn't list 2.132.3 / core 3.33.2. That doesn't affect this comparison or hold a PE-UK merge; certifying the newer model is the next uk-data release's job.
• Body fixed: the Reproducer now names only the Reset negative HBAI household income to zero before housing costs, as HBAI does #2401 test, and the Deduct ground rent and service charges from HBAI AHC income #2411 guard is listed separately as a computed-BHC check.
• Follow-up (docs layout nit): in docs/book/validation/hbai.md:48-55, Deduct ground rent and service charges from HBAI AHC income #2411's paragraph lacks a blank line, so it renders inside the reset paragraph and sits under the new heading. Queued as a small hub fix.
• Checks: all 15 pass, MERGEABLE CLEAN, marked ready. Enqueued in the merge queue pinned to this head.

[MaxGhenis]

Merged by the merge queue at reviewed head 06f47f788. The audit is in the hub comment above. #2401 is fixed.