Skip to content

v3: name a policy's missing parameters and run nothing with it - #1429

Open
anth-volk wants to merge 23 commits into
mainfrom
v2plan/1k-144-block-missing-parameters
Open

anth-volk wants to merge 23 commits into
mainfrom
v2plan/1k-144-block-missing-parameters

Conversation

@anth-volk

@anth-volk anth-volk commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #1428

What changes

The Public API is adding missing_parameters to GET /{country}/policy/{id} (PolicyEngine/policyengine-api#4039): the parameter paths a policy sets that the current tax-benefit model no longer has, or null when it could not check. v3 now reads it wherever a user chooses or runs a policy. When the list is not empty, v3 names the parameters in a notice and runs nothing with that policy. The notice's wording is plain: it says the reform or report "cannot be run, because it changes parameters that PolicyEngine’s model no longer has:" and lists the paths (they have no labels in the model any more). For the user's own saved reforms and saved reports, a dialog also offers to transfer them to a new policy without those parameters (see "Transfer a saved reform or report to a new policy" below).

This has an effect only once that Public API change is deployed. While the field is absent, and whenever it is null or empty, v3 behaves as before and blocks nothing.

Where it applies

  • Saved reforms ("Your reforms"). Both reform stores read the field with each reform's policy and keep it as Reform.missingParameters. The card reads "Uses parameters the model no longer has"; the reform's page names them and disables "View full impact report". Opening such a reform also opens its transfer dialog.
  • Re-run of a saved report. Re-run is decided when clicked, from the report's own reform and baseline policies (reportMissingParameters), so it covers a report compared with a deleted or edited saved reform. A policy that cannot be read does not hide the others' result; when the report itself cannot be read, the saved comparison reform a re-run would fall back to decides. Re-run is disabled while it checks. When it finds missing parameters, the notice names them and the saved report's transfer dialog opens.
  • Build. The "Compared with" list shows a reform with missing parameters, names them, and cannot select it; a draft already compared with such a reform names the parameters and disables "Save and run". The draft's own changes come from the current model's parameter list and are not checked. Build never opens the transfer dialog.
  • The report page. useFlagshipReport reads each report policy as soon as the simulations name it (not after the parameter metadata) and returns policiesChecked and missingParameters. The page shows "Loading report" until the policies are checked and starts nothing before then. With missing parameters:
    • without stored results, it shows the title and the notice, and calculates nothing;
    • with stored results, it shows them under the notice, for the report's own year only. No other budget-window year, budget window or UK local-area breakdown runs, including the local-area breakdown that main now starts in the background; the Household tab (where all household calculations and their variation views live) says households cannot be calculated; Recompute and Edit in Build are disabled.
    • when the report is the user's own saved report, its transfer dialog opens as the page opens, whichever section the link names. A report the user has not saved, such as another user's shared link, shows the notice only.

Transfer a saved reform or report to a new policy

Like the previous app's DeprecationModal, a dialog names the parameters the model no longer has and offers to move the user's saved item onto a new policy without them. It has two actions: the transfer, and "Not now", which closes it and leaves the item blocked as above. There is no delete action. When nothing would be left without those parameters, the transfer is disabled and the dialog says why. A failed transfer shows its message in the dialog and changes nothing; nothing retries by itself.

  • Saved reform: "Transfer to a new policy". It calls reformStore.update with the reform's other changes only. The store saves them as a v1 policy (POST /{country}/policy, which returns an identical one if it exists) and repoints the reform: PATCH /v2/user-policies/{id} with only legacy_policy_id, or the same change in the browser store. The reform keeps its name, baseline and provenance, and its reports keep their results. The reforms are read again, so the notice and the block lift.
  • Saved report: "Transfer and run again".
    • It runs the report again as Re-run does: the same year, population, budget window and comparison. If the report's record or its baseline simulation cannot be read, it stops rather than guess them.
    • It reads every policy the report ran again and leaves their missing parameters out of the provisions and out of the comparison policy. A policy that cannot be read stops the transfer, and a comparison that would be left with no changes is refused with a message.
    • It repoints the same saved report at the new run instead of saving a new one: PATCH /v2/user-reports/{id} with legacy_report_id, provisions, validation: null and last_run_at, or the same change in the browser store. Its links to saved reforms stay as they are. FlagshipReportStore gains update for this.
    • An account report keeps its link (the row's ID), which then shows the new results; transferred from its own page, the page stays and reads the new run. A report kept in this browser (sign-in off, or outside the US and UK) is linked by its v1 report, so its saved report opens the new run's link instead, and the dialog says so.
    • The repointed report is dated by its new run (an account record's date is now the row's last_run_at when it has one), and its old validation pin is cleared so the new run is pinned.
  • Where it opens. On a saved reform's page, opened from its card in Your library; from Re-run on a reform's reports; and on the user's own saved report's page. Lists open nothing until clicked.

The Public API is unchanged: it already accepts both PATCH bodies.

Behaviour that changes even while the field is absent

  • The report page reads each of its policies as soon as the simulations name them, and waits for those reads before it shows stored results or starts the run. The policy details, which used to read each policy after the parameter metadata, reuse that read, so each policy is still requested once. A read that fails is not retried for this check and counts as nothing missing, so a failure does not hold the page.
  • The browser reform store (sign-in off, or a country without accounts) reads each saved reform's policy when it lists them, through the session's query cache. A policy that cannot be read lists as saved.
  • An account saved report's date is its row's last_run_at when set. A new row sets it when it is created, so only a repointed report reads differently.

Overlap with other open PRs

Environment variables

None. The change reads no environment variables and touches no workflow, build or deployment configuration.

What this PR waits for

PolicyEngine/policyengine-api#4039 (stacked on PolicyEngine/policyengine-api#4036) must be deployed before this has any effect. This PR can merge before or after it.

Testing

Each commit was written test-first; the failing run came before the code change (vitest, Node 26).

  • ef2debaa (stores read the field): 6 store tests failed (the stores did not read missing_parameters); then 49 passed with the store hook tests.
  • 07eeeb3b (saved reforms): 3 page tests failed (no "Uses parameters the model no longer has", no notice); then passed.
  • ab8023a8 (Build): 2 tests failed (Unable to find role="alert"); then 43 passed.
  • eb5c9aab (report page): 6 tests failed (no policiesChecked/missingParameters, the run started, no notice); then passed.
  • Review round 1: 1772c24e (hold every run until checked; no retry delay; one read per policy; browser findById), 244f1754 (Re-run checks the report's policies), 2a1376d5 (UK local-areas reason; helper and notice tests). 5 tests failed first, among them a complete report starting another year and a window before the check, and two reads per policy; then passed.
  • Review round 2: 54021cdd (a settled check stays settled on refetch; renaming reads nothing) and 1ae167b2 (Re-run decided by the report's own policies, disabled while checking; the picker names parameters). 5 tests failed first; then passed.
  • Review round 3: ed1e2b17 (Re-run with a partly failed check or an unreadable report; UK overview card). 3 tests failed first; then passed.
  • On every commit up to ed1e2b17: bun run typecheck, bun run lint (ESLint and Stylelint), bun run prettier, and the flagship, hooks, api and libs tests. After an earlier rebase onto main, the whole app/ suite: 459 files, 5,926 passed, 10 skipped.
  • Rebased onto main again on 2026-10-29 (9f542782). main's new background start of UK local-area breakdowns is held while a report is blocked or its policies are still being checked, with one new ReportRunLifecycle test that fails without it. The type check, lint, Prettier, the changed test files and the local-area tests passed. A whole-suite run had 3 failures, none in a file this PR touches: two header tests also fail on main, and a chart test timed out under load and passes alone.
  • Transfer dialog (dec8c6a4, 3edd38c2, dcfb1975): 28 tests failed first (no store update, no repointing run, no dialog); then passed.
  • Its review round 1: 5b659735 (refuse a transfer that cannot read the run it replaces), 6e567ec4 (dialog on any section), 2c549f2e (link wording for each store, one dialog for both pages), 97a606ba and 4e3e9768 (date and re-pin a repointed report; a refused browser write fails). Review round 2: 5b9c83bc (read every policy of the report before transferring), 1f21b640 (drop the transferred report's cached page data), 391116c8 (re-read the pin only when a run is replaced), 11c86633 (keep fields passed as undefined). Review round 3: f6d346e0 (stay on the page when the link is kept). Each fix had a failing test first.
  • For the transfer dialog and its fixes, only the tests these commits add or change were run (by name for the later fixes), with bun run typecheck, bun run lint and bun run prettier; the whole suite was not run.
  • Earlier GitHub Actions runs on this branch were cancelled at the user's request, until 2026-10-27.

🤖 Generated with Claude Code

@vercel

vercel Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
policyengine-calculator-next Ready Ready Preview Oct 30, 2026 3:52pm UTC
policyengine-website Ready Ready Preview Oct 30, 2026 3:52pm UTC

Request Review

anth-volk and others added 22 commits October 30, 2026 19:41
GET /{country}/policy/{id} is gaining missing_parameters: the parameters
the policy sets that the current model no longer has, or null when the
Public API could not check. The reform stores now read it with the
policy's changes and keep it on the reform as missingParameters, only
when the list is not empty. While the field is absent (an API without
it) or null, a reform reads exactly as before.

The account store reads it with each reform's changes; the query cache
key becomes v1-policy-content because it now holds both. The browser
store, given a policy reader (as useReformStore passes), reads each
reform's policy when it lists them; a policy that cannot be read lists
as saved.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A saved reform whose policy sets parameters the current model no longer
has now says so on Your reforms: its card reads "Uses parameters the
model no longer has", and its page names each parameter in a notice and
disables "View full impact report". Its reports still open, but offer
no Re-run, and neither does a report compared with such a reform.

MissingParametersNotice is the shared notice; the parameters have no
labels in the model any more, so it shows their paths. ReformReports
takes canRerun to decide which reports may run again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In Build's "Compared with" picker, a saved reform whose policy sets
parameters the current model no longer has is listed but cannot be
chosen, and says "Uses parameters the model no longer has". A draft that
already compares with such a reform names its parameters in a notice
and disables "Save and run". Comparing with current law or another
saved reform runs as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
useFlagshipReport now reads each report policy's missing_parameters as
soon as the simulations name the policies, without waiting for the
parameter metadata, and returns policiesChecked and missingParameters.
The report page starts its calculation only once the policies are
checked; a policy that cannot be read counts as checked with nothing
missing, so the page behaves as before when the API does not report
the field.

When the report's reform or baseline policy sets parameters the model
no longer has:
- a report with no stored results shows its title and a notice naming
  the parameters, and calculates nothing;
- a report with stored results shows them under that notice, for its
  own year only: no other budget-window years, budget window or
  local-area breakdown run, the Household tab says households cannot be
  calculated, and the adjust panel's Recompute and Edit in Build are
  disabled.

The cache key for a policy's content moves to queryKeys, where the
report hook and the reform store share it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review found that only the report's automatic run waited for the policy
check: other budget-window years asked for earlier, the budget window
and the stored-results view could start calculations before the check
returned. The report page now has one runsAllowed value (checked, and
nothing missing) that the automatic run, the other years and the budget
window all use, and it keeps showing "Loading report" until the
policies are checked, so nothing below it renders early.

A policy that cannot be read is no longer retried for the check, so a
failed read no longer delays the run by the client's retry back-off.
Each policy is fetched once: the check and the policy details share one
cached read (readV1Policy, key v1-policy), which the reform stores use
too. The browser reform store's findById now checks the policy as
findMine does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Re-run was offered or refused from the saved reforms only, but a re-run
runs the report's stored changes against the baseline policy the report
ran. A report whose comparison reform was deleted, or that ran against
an unnamed comparison, could still re-run with a policy that sets
parameters the model no longer has. Re-run now reads the report's two
policies first (reportMissingParameters); when either sets such
parameters it names them in a notice and runs nothing. A check that
cannot be made leaves Re-run as it was.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… helpers

A UK report whose policies set parameters the model no longer has
showed its local-area maps with no explanation, which for a national
result without area impacts read as an empty map. The tab now says
that local-area results cannot be calculated for the report and that
only the results it holds are shown.

New tests cover the missing-parameter helpers (absent, null and
non-list fields, non-path entries, combining and sorting) and the
notice's wording for one and for several parameters.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hout reading it

A second review found that policiesChecked followed the live state of
each policy read. When a failed read was asked for again (a refetch on
window focus, or the policy details reading the same cache entry), the
entry went back to pending and the report page returned to "Loading
report", losing its tabs and household form. A read that has settled
once now counts as checked while it is asked for again.

The browser reform store's update looked the reform up with findById,
which now reads its v1 policy; update reads the stored reform directly,
so renaming makes no request. Tests also cover a null
missing_parameters in the report hook.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…Build's picker

Re-run was hidden, with nothing said, for a report compared with a
saved reform that sets parameters the model no longer has, and that
decision read the saved reform's current policy rather than the one the
report ran. Re-run is now offered as before and decided when clicked,
from the report's own reform and baseline policies: when either sets
such parameters a notice names them and nothing runs. While that check
runs, Re-run is disabled, so a second click cannot start a second run.
ReformReports no longer needs canRerun.

Build's "Compared with" list now names the missing parameters of a
reform it cannot offer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…local-area runs

The third review found two ways a Re-run could still start with a
policy that sets parameters the model no longer has:
- one of the report's policies could not be read while the other
  reported missing parameters, and the failed read discarded both;
- the report itself could not be read, so the re-run compares with the
  saved comparison reform's own policy, which was not checked.
reportMissingParameters now counts every policy it can read, and when
the report cannot be read the saved comparison reform's missing
parameters decide.

On a UK report that cannot run, the overview card no longer offers to
calculate income changes across local areas. policyChecks gets its own
tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A saved report could be created, read and given a validation pin, but
not changed. FlagshipReportStore.update now changes only the supplied
fields: the v1 report the record points at (apiReportId), and the
provisions and year that report ran. The record keeps its ID, which is
its link, and its saved-reform links.

The account store sends PATCH /v2/user-reports/{id} with
legacy_report_id (and last_run_at, since pointing at another v1 report
is running it again) and provisions; the Public API's
UserReportUpdateInput accepts both. The row keeps no year. The browser
store updates its entry in place, and an unknown record fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
runFlagshipReport takes savedReportId: the run's v1 report, provisions
and year then go onto that saved report through the store's update,
instead of a new record being created. An account report keeps its
row's ID, so its link opens the new results; a browser record's link is
the new v1 report. A failed update fails the run, in either store, so a
saved report is never left pointing at the old run unnoticed.

useRunFlagshipReport passes it through and now answers the report's
link, or null when the run did not start. With a saved report it also
reads the shared report and the session's record again, and clears its
running state, since the page that started it may stay open.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…o longer has

As the old app's DeprecationModal did, one of the user's saved items
whose policy sets parameters the current model no longer has now offers
to move onto a new policy without them. The dialog names the
parameters as the notice does, and offers the transfer and "Not now",
which closes it and leaves the item blocked, as before.

- A saved reform: the dialog opens when the reform's page opens from
  its card in Your library. "Transfer to a new policy"
  calls reformStore.update with the reform's other changes only; the
  store saves them as a v1 policy (POST /{country}/policy answers an
  identical one if it exists) and repoints the reform, keeping its
  name, baseline and provenance. The reforms are read again, so the
  notice and the block lift.
- A saved report: Re-run on a reform's page opens the dialog when the
  check finds such parameters, and so does the report's own page, as it
  opens, when it is this session's saved report. "Transfer and run
  again" runs the report as Re-run does (same year, population, budget
  window and comparison), without those parameters in its provisions
  or in its baseline policy, and repoints the same saved report at the
  new run, so its link keeps working.
- When nothing would be left to transfer, the action is off and the
  dialog says why. A failure shows in the dialog and changes nothing;
  nothing retries by itself. There is no delete action.

Build's "Compared with" list and its draft notice, and reports the user
has not saved (a shared link), keep the notice only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rerunSettings falls back to a nationwide run in the current year when
the API's record of the report or its baseline simulation cannot be
read. Re-run can afford that, because it saves a new report; a transfer
repoints the saved report itself, so a 2028 Maryland report could have
been replaced by a nationwide run for this year. The Reforms page opens
the transfer dialog in exactly that state, when the report is
unreadable and its saved comparison reform has missing parameters.

rerunSettings takes requireRun, which fails instead of falling back,
and fails when no year is known. The transfer uses it, so such a
transfer shows its error and the saved report keeps its run. The
message for a comparison left with no changes now says "this
parameter" when there is one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pens

With stored results, the report page put the dialog in ReportView's
overview lead, which is rendered only once the Overview section has
been open. A link to another section (?tab=economy, from Back, a reload
or a copied link) therefore opened the user's own blocked report with
no dialog until Overview was clicked. The dialog is a portal, so it now
sits beside ReportView instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ages

Both report dialogs said a transfer keeps the report's link. That holds
for an account report, whose link is its row's ID, but not for one kept
in this browser (sign-in off, or outside the US and UK): its link is
its v1 report, which the transfer replaces. SavedReportTransferDialog
now says "It keeps its link, which then shows the new results" for an
account report and "Your saved report then opens the new results"
otherwise.

The Reforms and report pages had the same dialog wiring twice; both now
use SavedReportTransferDialog. The two transfer hooks share one
contract: each is a mutation whose error is a TransferError carrying
the dialog's message. Their module is renamed
useMissingParameterTransfers, after the hooks it holds.

New report-page tests open a saved report's link where accounts exist:
its owner gets the dialog, and another user gets the notice only. The
browser-store transfer test's run now answers a new v1 report, as the
real run does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dation

A saved report moved onto a new run kept its first run's date and
validation pin. Your library then said "Last run" on the old date and
sorted it there. Its report page compared the new run's validation with
the old pin: useReportValidationSnapshot reads a record once per link,
and the link does not change.

- A new apiReportId in FlagshipReportStore.update is a new run. The
  account store also sends validation: null (the Public API clears the
  pin), and the browser store clears it and dates the record now.
- An account record's createdAt is now its row's last_run_at when the
  row has one, so a repointed row reads as run when it ran again.
- useReportValidationSnapshot takes the v1 report the page shows and
  reads the record again when it changes, so the new run is pinned.
- The browser store's update fails when the browser refuses the write,
  instead of reporting a change it did not make.
- FlagshipReportUpdates says that only a browser record keeps year.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The run-report test for moving a browser record onto a new run still
expected the record's first date. Since the store dates a repointed
record by its new run and drops its old validation pin, it expects
that instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A report's transfer left out only the parameters its dialog opened
with, and that list leaves out any policy whose read failed: the report
page combines only the reads that succeeded, and Re-run's check counts
the policies it could read. With the reform policy unread, a provision
for a removed parameter stayed in, the saved report was moved onto a
new run that still set it, and it still could not run.

The transfer now reads the report's policies again
(reportMissingParameters with requireEveryPolicy) and leaves out every
parameter they report, from the provisions and from the comparison
policy. A policy that cannot be read stops the transfer with its
message, and a report left with no provisions says so.

New tests call useSavedReportTransfer directly. The report page's
transfer tests now give the report's policies to read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After a transfer, the run marked the shared report and the session's
record stale and opened the report's link. If that page had been open
in the last five minutes, for example to say "Not now" before using
Re-run, it opened from the cached old run, offered its transfer again
and could start a second one from the stale record, until a refetch
returned the new run.

The run now resets those queries, so the page reads the saved report
afresh. Their keys are exported from useFlagshipStores as
savedReportQueryKeys, which its hooks and the run share, so renaming a
key cannot silently skip the reset.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
useReportValidationSnapshot read the record again whenever the run it
was given changed, which includes the first time the page learns it
(from none to the report's ID). That second read reset the check that a
record is pinned only once, so a record with no pin, whose live checks
settled before the report loaded, could be pinned twice.

It now reads again only when one known run is replaced by another, as
after a transfer, including when the page shows no report in between.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LocalStorageFlagshipReportStore.update spread every key of the update,
so a field passed as undefined replaced the stored value with nothing,
while the account store sends only the fields that are set. The
browser store now leaves out undefined fields too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On an account report's own page, "Transfer and run again" ends with the
run opening the report's link, which is the link already open: the
navigation pushed a second history entry for the same page, so the
first Back press stayed on the report. The report page now passes the
link it shows, and a run that answers that link stays on the page,
which reads the new run once its saved-report queries are reset.

Three comments that said a run always writes a new saved report now
say that a transfer repoints one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

2 active deployments
Preview – policyengine-website — 4013eb51 Deployed Oct 30, 2026 by vercel[bot]
Preview – policyengine-calculator-next — 4013eb51 Deployed Oct 30, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

v3: name a policy's missing parameters and do not run it

1 participant