Hold BSP flat after the dataset benefit split

changelog.d/2381.fixed.md

@@ -0,0 +1 @@
+Held reported Bereavement Support Payment flat for datasets supplying both separate bereavement-benefit inputs, while preserving CPI uprating for legacy and partial splits and the separate pensions.

docs/book/assumptions/growthfactors.md

@@ -22,6 +22,12 @@ We apply CPI to these variables:
 
 - Other variables– `afcs_reported`, `bsp_reported`, `childcare_expenses`, `free_school_fruit_veg`, `free_school_meals`, `free_school_milk`, `maintenance_expenses`, `statutory_maternity_pay`, `statutory_paternity_pay`, `statutory_sick_pay`, `state_pension`, `war_widows_pension_reported`, `widowed_parents_allowance_reported`
 
+`bsp_reported` stays nominally flat when a dataset explicitly supplies both
+`war_widows_pension_reported` and `widowed_parents_allowance_reported` in
+the same source year, including all-zero columns. Datasets with either
+column missing and household situations retain CPI uprating. See the
+[bereavement-benefit split guide](https://github.com/PolicyEngine/policyengine-uk/blob/main/docs/engineering/skills/dataset-sources.md#bereavement-benefit-split).
+
 ## Road fuel volume
 
 We use HMRC road-fuel clearances and OBR-implied forecast volumes for petrol and diesel spending, because PolicyEngine converts these spending fields into litres by dividing by pump prices.

docs/book/programs/gov/dwp/bereavement-support-plan.md

@@ -55,15 +55,19 @@ That means:
 - Since [#2382](https://github.com/PolicyEngine/policyengine-uk/pull/2382)
   (2.135.7), War Widow's Pension has its own variables,
   `war_widows_pension_reported` and `war_widows_pension`. `bsp_reported`
-  keeps CPI uprating only until the datasets move FRS code 9 out of it
-  and WPA into `widowed_parents_allowance_reported` (code 9 is done on
-  Microcosm's main branch in microcosm#1333; still open for the
-  Enhanced FRS in
-  [policyengine-uk-data#578](https://github.com/PolicyEngine/policyengine-uk-data/issues/578),
-  and [microcosm#1222](https://github.com/PolicyEngine/microcosm/issues/1222)
-  is still open);
-  [#2381](https://github.com/PolicyEngine/policyengine-uk/issues/2381)
-  tracks freezing it after that.
+  stays nominally flat for datasets that explicitly supply both
+  `war_widows_pension_reported` and `widowed_parents_allowance_reported`
+  in the same source year, including all-zero columns. Datasets with
+  either column missing retain CPI uprating, as do household situations.
+  Microcosm's merged
+  [#1383](https://github.com/PolicyEngine/microcosm/pull/1383) supplies both
+  columns and closes
+  [#1222](https://github.com/PolicyEngine/microcosm/issues/1222). Older
+  releases and the Enhanced FRS retain the compatibility path;
+  [policyengine-uk-data#578](https://github.com/PolicyEngine/policyengine-uk-data/issues/578)
+  tracks the Enhanced FRS code-9 move. See the
+  [dataset-source guide](https://github.com/PolicyEngine/policyengine-uk/blob/main/docs/engineering/skills/dataset-sources.md#bereavement-benefit-split)
+  for projection, fallback and off-rate residual treatment.
 - Since [#2572](https://github.com/PolicyEngine/policyengine-uk/issues/2572),
   WPA also has its own variables, `widowed_parents_allowance_reported`
   (CPI-uprated) and `widowed_parents_allowance`. Unlike BSP, WPA is taxable
@@ -74,8 +78,9 @@ That means:
   cap and an overlapping benefit for Carer's Allowance and Carer Support
   Payment. It ends at State Pension age and while the recipient lives with a
   partner ([SSCBA 1992 s. 39A(4) and (5)][sscba-39a]), so it never reaches
-  Pension Credit. The datasets still put it in `bsp_reported` until they
-  split FRS code 6.
+  Pension Credit. Older datasets still put it in `bsp_reported`;
+  Microcosm #1383 separates amounts matching its positive WPA rate rule,
+  while retaining unclassified off-rate amounts in BSP.
 
 ## Proposed scope
 
@@ -180,10 +185,10 @@ November 2024
 ([DWP, July 2025](https://www.gov.uk/government/publications/bereavement-benefit-remedial-order-uptake-february-2023-to-november-2024)).
 Existing awards continue while the surviving parent is entitled to Child
 Benefit and under State Pension age, so some recipients remain. In
-the current data they sit inside `bsp_reported` (see above), so they
-are already counted in income. Once datasets fill
-`widowed_parents_allowance_reported`, they are told apart from BSP; the
-gap left is a rule-based amount for reforms.
+older datasets they sit inside `bsp_reported` (see above), so they are
+already counted in income. Datasets that fill
+`widowed_parents_allowance_reported` distinguish those reported amounts
+from BSP; the gap left is a rule-based amount for reforms.
 
 ### Phase 3: data-side bereavement inputs
 

docs/engineering/skills/dataset-sources.md

@@ -121,6 +121,42 @@ matching that receipt. The producer must carry both period kind and year into
 the H5 artifact; this consumer change does not retrofit that metadata or verify
 the released private file.
 
+## Bereavement-benefit split
+
+Bereavement Support Payment rates remain unchanged under the
+[2026 up-rating order, article 19](https://www.legislation.gov.uk/uksi/2026/148/article/19/made).
+Older datasets combine it with CPI-uprated Widowed Parent's Allowance and
+War Widow's Pension in `bsp_reported`. They keep CPI projection until the
+producer separates those benefits.
+
+Explicitly supplied `widowed_parents_allowance_reported` and
+`war_widows_pension_reported` person columns together declare the completed
+split used by the merged
+[Microcosm PR #1383](https://github.com/PolicyEngine/microcosm/pull/1383),
+including when every value in either column is zero. The automatic projector
+then holds `bsp_reported` nominally flat; the separate allowance and
+war-widow inputs still grow by CPI. Either column alone leaves BSP on the
+legacy CPI path. Unclassified off-rate amounts that the producer retains in
+`bsp_reported` receive the same nominal treatment. These can include Widowed
+Parent's Allowance with an inherited additional pension above its rate;
+freezing that CPI-linked component understates it.
+
+Beyond the materialized projection horizon, the model holds BSP flat only
+when its latest earlier input has both explicitly stored split inputs for
+the same period. A calculated or carried default does not count, nor do
+inputs supplied in different periods.
+This also preserves the distinction when an external multi-year dataset
+changes schema between observations. Supplied annual BSP amounts remain
+authoritative. Disabling economic assumptions preserves those amounts when
+either observation declares the split, rather than replacing separate BSP
+with an older combined report.
+
+UK single-year files, materialized Core datasets and their projected children
+preserve both columns. Household situations retain their existing CPI behavior,
+even when they supply the separate benefits. The change follows dataset
+semantics, without inferring a split from a filename, model version or the
+amount received. See [issue #2381](https://github.com/PolicyEngine/policyengine-uk/issues/2381).
+
 ## Tests
 
 For dataset source routing, prefer stubbed tests that prove dispatch behavior

policyengine_uk/data/economic_assumptions.py

@@ -128,6 +128,10 @@
     }
 )
 DATASET_UPRATING_EXCLUSIONS = DECLARED_ENERGY_INPUTS
+# Both separated benefits must be explicit before BSP can be held nominally flat.
+BEREAVEMENT_SPLIT_INPUTS = frozenset(
+    {"war_widows_pension_reported", "widowed_parents_allowance_reported"}
+)
 # Inputs that describe only the year they are stored for, so are not carried
 # into projected years. Unused annual allowance comes from the three tax years
 # before this one (Finance Act 2004 s. 228A), a window that moves each year; a
@@ -342,7 +346,17 @@ def apply_single_year_uprating(
             for table_name, df in zip(current_year.table_names, current_year.tables):
                 if variable in df.columns:
                     prev_year_value = getattr(previous_year, table_name)[variable]
-                    current_year_value = prev_year_value * (1 + index_rel_change)
+                    split_bsp = (
+                        variable == "bsp_reported"
+                        and BEREAVEMENT_SPLIT_INPUTS.issubset(
+                            previous_year.person.columns
+                        )
+                    )
+                    current_year_value = (
+                        prev_year_value
+                        if split_bsp
+                        else prev_year_value * (1 + index_rel_change)
+                    )
                     getattr(current_year, table_name)[variable] = current_year_value
 
     # Next, apply custom uprating.
@@ -834,13 +848,20 @@ def reset_growthfactor_uprating(
             current_year_dataset.energy_price_basis
             == base_year_dataset.energy_price_basis
         )
+        preserve_bsp = BEREAVEMENT_SPLIT_INPUTS.issubset(
+            base_year_dataset.person.columns
+        ) or BEREAVEMENT_SPLIT_INPUTS.issubset(current_year_dataset.person.columns)
 
         # One declaration describes every stored energy amount. Keep the
         # later group intact if a partial reset would mix different bases.
         for table_name in current_year_dataset.table_names:
             base_table = getattr(base_year_dataset, table_name)
             current_table = getattr(current_year_dataset, table_name)
             for variable in growthfactor_uprated_variables:
+                # Split BSP amounts do not reflect economic growth. In a
+                # schema transition, the base amount can include other benefits.
+                if variable == "bsp_reported" and preserve_bsp:
+                    continue
                 if variable in current_table.columns and variable in base_table.columns:
                     current_table[variable] = base_table[variable].values
         if reset_energy:

policyengine_uk/data/uprating_indices.yaml

@@ -35,11 +35,10 @@ gov.economic_assumptions.yoy_growth.obr.consumer_price_index:
 - alcohol_and_tobacco_consumption
 - bus_fare_spending
 - bus_fare_spending_reported
-# bsp_reported still holds FRS code 9 (War Widow's Pension, CPI-uprated) and
-# the Widowed Parent's Allowance in code 6 (CPI-uprated) as well as
-# Bereavement Support Payment, whose rates are frozen. It moves off this index
-# once datasets put code 9 in war_widows_pension_reported and Widowed
-# Parent's Allowance in widowed_parents_allowance_reported (#2381).
+# Legacy bsp_reported can include War Widow's Pension and Widowed Parent's
+# Allowance, so it retains CPI here. The projector holds it flat when the
+# source person table supplies both widowed_parents_allowance_reported and
+# war_widows_pension_reported. The separate benefits still grow by CPI (#2381).
 - bsp_reported
 - carers_allowance_reported
 - child_benefit_reported

policyengine_uk/simulation.py

@@ -33,6 +33,7 @@
 from policyengine_uk.utils.entity_ids import ENTITY_ID_VARIABLES, check_entity_ids
 from policyengine_uk.utils.scenario import ConstructionReformState, Scenario
 from policyengine_uk.data.economic_assumptions import (
+    BEREAVEMENT_SPLIT_INPUTS,
     extend_single_year_dataset,
     reset_growthfactor_uprating,
     uprating_input_fingerprint,
@@ -54,6 +55,7 @@
     capital_gains_application_metadata,
 )
 from policyengine_uk.utils.dependencies import get_variable_dependencies
+from policyengine_uk.utils.data_source import built_from_data
 from policyengine_uk.utils.input_dataframe_export import input_dataframe
 from policyengine_uk.utils.supplied_inputs import (
     SUPPLIED_INPUT_VARIABLES,
@@ -1359,6 +1361,50 @@ def calculate_all(self, year: int) -> None:
             fiscal_year=year,
         )
 
+    def _calculate(self, variable_name: str, period: Period = None):
+        # Core requires an indexed input to have no formula. Keep the legacy
+        # CPI declaration, but carry split dataset BSP nominally after the
+        # materialized horizon, including through Core's monthly conversion.
+        if (
+            variable_name == "bsp_reported"
+            and period is not None
+            and period.unit == "year"
+            and period.size == 1
+            and built_from_data(self)
+        ):
+            variable = self.tax_benefit_system.get_variable(variable_name)
+            holder = self.get_holder(variable_name)
+            inputs = holder.get_input_periods(self.branch_name)
+            if (
+                not variable.formulas
+                and not variable.is_neutralized
+                and variable.uprating
+                == "gov.economic_assumptions.indices.obr.consumer_price_index"
+                and period not in inputs
+            ):
+                source = max(
+                    (
+                        known
+                        for known in inputs
+                        if known.unit == "year" and known.start < period.start
+                    ),
+                    key=lambda known: (known.start, known.stop),
+                    default=None,
+                )
+                if source is not None:
+                    split = all(
+                        source
+                        in self.get_holder(name).get_input_periods(self.branch_name)
+                        for name in BEREAVEMENT_SPLIT_INPUTS
+                    )
+                    # Track both dependencies even when either input is absent.
+                    # Calculated defaults do not declare a completed split.
+                    for name in BEREAVEMENT_SPLIT_INPUTS:
+                        self.calculate(name, source)
+                    if split:
+                        return self.calculate(variable_name, source)
+        return super()._calculate(variable_name, period)
+
     def calculate(
         self,
         variable_name: str = None,