Skip to content

Publish the model metadata the Public API can only read through a private method - #701

Draft
anth-volk wants to merge 14 commits into
mainfrom
v2plan/1f-114-public-metadata
Draft

anth-volk wants to merge 14 commits into
mainfrom
v2plan/1f-114-public-metadata

Conversation

@anth-volk

@anth-volk anth-volk commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #700

Problem

The Public API stores each .py model version's metadata for the PolicyEngine app. Seven groups of fields the app reads are not in .py's public records, so the only way to read them today is the private _load_system() method. This PR publishes them on the existing Pydantic records, so the Public API can read them from .py's public surface.

Additions only. No existing field, default or behaviour changes for current .py users.

New public fields

Record Field Type Source
policyengine.core.Entity (new model) key, plural, is_person str, str, bool The country package's entities
TaxBenefitModelVersion entities list[Entity], in the package's order; empty by default system.entities
TaxBenefitModelVersion basic_inputs list[str]; empty by default or when the package declares none system.basic_inputs (the household fields shown by default)
Variable is_input_variable Optional[bool] core's is_input_variable(): no formula and no adds or subtracts
Variable hidden_input Optional[bool] The variable's hidden_input
Variable module_name Optional[str] The variable's module_name
policyengine.core.EnumValue (new model) value, label str, str | int | float | bool | None Each enum member's name, and its declared value unchanged (an integer stays an integer, as the Public API's /{country}/metadata sends it)
Variable enum_values Optional[list[EnumValue]], in declaration order; None for non-enum variables The variable's possible_values
Variable adds_parameter_path, subtracts_parameter_path Optional[str] Set when adds or subtracts names a parameter instead of a list (237 US variables add and 2 subtract that way)
Parameter economy, household bool; True when the package does not say The parameter's economy and household metadata, as the Public API's /{country}/metadata reads them
ParameterNode (existing fields) Each bracket schedule (for example gov.hmrc.income_tax.rates.uk) is now a node, listed before its brackets

Fields that are None mean the country model does not provide the value (for example older country packages).

Deliberate behaviour

  • adds and subtracts are unchanged. They still hold the list resolved as of 2025-01-01. The new path fields sit beside them, so a consumer can read the parameter's list for any year.
  • A schedule's label is its stored label, or None when the package stores none (for example gov.hmrc.income_tax.rates.dividends). Consumers generate a label from the name. Bracket nodes (name[0], name[1], …) and their rate, threshold and amount parameters are unchanged.
  • Schedules without brackets are left out, as empty organisational nodes already are.
  • All schedules are published, not only those under gov. That includes the two US calibration.* schedules.
  • UK duplicates are unchanged. The UK package's baseline subtree already makes every UK gov.* parameter and node appear twice in parameters and parameter_nodes, with the same id. That includes the 41 UK schedules. This PR does not change it, because deduplicating would change existing behaviour. Consumers should deduplicate by name; parameter_nodes_by_name and parameters_by_name already hold one entry per name.
  • Entity labels, descriptions and roles, parameter periods, and a second copy of each parameter's label are not added. Nothing reads them.

Tests

  • Stub-based tests for each field, including:
    • enum labels declared as strings, integers, floats, booleans and None;
    • a parameter path that cannot be resolved;
    • schedules with and without a stored label;
    • a schedule without brackets.
  • UK and US tests that compare every variable, parameter and schedule with the installed country package.
  • UK and US tests that pin concrete values from the pinned country packages: entity keys and plurals, parameter flags, input, hidden and module values, parameter-named parts, and schedule labels.
  • The new country tests use shared fixtures (tests/fixtures/country_model_fixtures.py) that skip when a country package is not installed.
  • make lint passes, and the full suite passes locally: 4233 passed, 15 skipped.

Environment variables

None read or added.

Country package pins

Unchanged.

Before merging

This PR waits only for review. After it merges, a .py release is needed before the Public API can depend on these fields. This PR does not publish one.

🤖 Generated with Claude Code

anth-volk and others added 14 commits October 22, 2026 02:05
A model version now records the country package's entities (key, plural,
and whether it is the person entity) and its basic inputs, the household
fields shown by default. Entity labels, descriptions and roles are not
recorded. A country package without basic inputs publishes an empty list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each variable now records whether users enter it rather than it being
calculated (core's is_input_variable), whether it is hidden from input
lists, and the module that defines it. Variables built from older
country models that do not declare these leave them as None.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Enum variables now carry enum_values, each value's stored name with the
label users see, in the order the country model declares them. The
existing possible_values list of names is unchanged. A label that is
not a string, such as an integer, is published as its string form.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Where a variable's adds or subtracts names a parameter instead of a list,
the variable now records that path in adds_parameter_path or
subtracts_parameter_path, so a consumer can read the list for any year.
adds and subtracts keep the list resolved as of 2025-01-01.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each parameter now carries the economy and household flags from its
country package metadata, True when the metadata omits them, as the
Public API's v1 metadata does. Parameters built without a country
package default to True.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A bracket schedule, such as gov.hmrc.income_tax.rates.uk, is now a
parameter node carrying its stored label (None when the country package
has none) and description, listed before its brackets. Its brackets
(name[0], name[1], ...) stay nodes of their own, and their rate,
threshold and amount parameters are unchanged. Schedules with no
brackets are left out, as empty nodes already are.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An enum member declared with an integer, such as the US
ILPersonalExemptionEligibilityStatus values 1, 2 and 3, now publishes
the integer as its label instead of its string form, as the Public
API's /{country}/metadata does. EnumValue.label holds a string or an
integer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The entity and parameter-flag modules imported uk_latest and us_latest at
module level, which the country modules define only when their package
is installed, so a missing package failed collection, stub-only tests
included, and the importorskip calls inside the tests could never skip.
A fixture now skips first and imports the model afterwards. In the
variable and parameter-node modules, whose module-level imports predate
this change, the added tests import the core system directly instead of
appearing to skip.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A schedule whose brackets list is empty is not published as a parameter
node, as empty organizational nodes already are not.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The country tests computed their expected values with the same
expressions as the implementation, so they could not detect a
difference from what the country packages declare. They now also check
fixed values read from the pinned country packages: each country's
entity keys, plurals and person flag; economy and household flags of
parameters declared false, directly or through propagated metadata, and
of one unflagged parameter; input, hidden-input and module values of
known variables; the count of parameter-named adds (UK 4, US 237) and
subtracts (UK 0, US 2); and stored schedule labels, including schedules
without one, which publish None.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
EnumValue.label held only a string or an integer, so a member declared
as True or 1.0 became 1, and one declared as 0.5 or None failed
validation, which would fail building the country model version. The
label now holds any of those scalars and keeps each declared type,
also through JSON.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The country tests this branch added now get the model version through
shared fixtures (country_model for each country, uk_country_model and
us_country_model), which skip when the country package is missing and
import the model version only after that check. They and the empty
model version helper are defined once, in
tests/fixtures/country_model_fixtures.py, and registered in conftest.
Tests that existed before this branch keep their module-level imports.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The exact totals of parameter-named adds and subtracts would fail on a
routine country package update that adds one, although .py's behavior
is unchanged. The pinned variables stay; the counts are now at least
the pinned packages' totals (UK 4 adds; US 237 adds and 2 subtracts),
and every published path must name a published parameter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Publish the model metadata the Public API can only read through a private method

1 participant