Skip to content

Run Pyright in CI for every package, in its own locked environment - #864

Merged
anth-volk merged 18 commits into
mainfrom
pyright-ci
Oct 21, 2026
Merged

Run Pyright in CI for every package, in its own locked environment#864
anth-volk merged 18 commits into
mainfrom
pyright-ci

Conversation

@anth-volk

@anth-volk anth-volk commented Oct 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Fixes #863

Same-Repository Draft PR Check

  • This PR is a draft.
  • This PR is from a branch in PolicyEngine/policyengine-sim-api, not a personal fork.

Summary

Runs Pyright on every pull request for every uv package, each in its own locked environment, and holds each package to a registered rule. This is the base of a stack (#867): #866, #869, #871, #873, #875 and #877 above it reduce the remaining errors to zero and make every rule enforced.

Package Errors at ceb30e1 Rule After this PR
contract, observability, Stage 12 persistence, fastapi 0 sim.quality.pyright-libraries-and-entry (enforced) 0
simulation-entry 2 sim.quality.pyright-libraries-and-entry (enforced) 0
simulation-gateway 81 sim.quality.pyright-gateway (ratcheted) 81 in rules/baselines/pyright-gateway.txt
simulation-executor 178 sim.quality.pyright-executor (ratcheted) 178 in rules/baselines/pyright-executor.txt
apis-integ 4 sim.quality.pyright-apis-integ (ratcheted) 4 in rules/baselines/pyright-apis-integ.txt

A fifth rule, sim.quality.every-package-type-checked, fails the quality guards when a package under libs/ or projects/ has no Pyright rule.

How the check works

python -m scripts.rules pyright [--sync] [<package>...] (standard library only, in scripts/rules/pyright.py):

  1. finds the Pyright rule whose enforcer.projects lists the package;
  2. with --sync, runs uv sync --frozen --all-extras in the package;
  3. runs the pinned Pyright (uv tool run --from pyright==1.1.414 pyright --outputjson --pythonpath <package>/.venv/bin/python --project <package>) in the package directory, so the package's [tool.pyright] include decides what is checked and imports resolve against its own dependencies. --project makes Pyright read only the package's own configuration: without it, Pyright also uses a pyrightconfig.json from a parent directory;
  4. under an enforced rule, fails on any error. Under a ratcheted rule, fails on an error the baseline doesn't list, and on a baseline entry for an error that no longer occurs.

The pyright job in pr.yml runs it once per package in a matrix, after generating the API client for apis-integ. CI complete needs the job, and it has the same stacked-PR condition as the other jobs. make check runs it for every package.

Decisions

Baselines: a standard-library comparison, not --baselinefile. None of the pinned versions (1.1.409, 1.1.411, 1.1.414) accepts --baselinefile or --writebaseline: both report Unexpected option. Those options exist only in the basedpyright fork. So the script reads pyright --outputjson and compares it with a text baseline: one line per error, <path> :: <Pyright rule> :: <first line of the message>, sorted, without line numbers so edits elsewhere in a file don't change it. One line per error keeps the registry's [rule.baseline] count equal to the number of tolerated errors, so the guard's "baselines may only shrink" comparison counts errors. --write-baseline rewrites the file as the errors that occur now and refuses when one is new, so it can only remove entries.

Pyright version: one pinned version (1.1.414) for every package, not each lock file's. Results are not stable across the versions in use: for the same two executor errors in stage12_parity.py, 1.1.409 prints Series | Unknown | DataFrame and 1.1.414 prints Series | Unknown | Self@DataFrame, and baseline entries quote messages. Also, the daily dependency update relocks every package with uv lock --upgrade, so the locked Pyright versions move whenever Pyright releases; pinning keeps dependency updates from changing type-check results, as the ruff pin does. 1.1.414 is the newest release and already the stage12-persistence lock's version. Error counts were the same under every version, and 1.1.414 reported exactly the same errors as each package's locked version except for that executor message. The script ignores PYRIGHT_PYTHON_FORCE_VERSION and fails if Pyright reports any other version. Each package's build extra still installs Pyright for editors.

The span scope type. policyengine-observability annotates its scope managers' __exit__ as returning bool although it always returns False. Pyright reads a bool result as "may suppress the exception", so it reports every variable assigned inside with runtime.span(...) as possibly unbound afterwards. This causes the entry's two errors, and most of the gateway's 66 and the executor's 60 possibly-unbound errors. The new policyengine_simulation_observability.scopes.span(runtime, name, attributes=...) returns the runtime's own scope unchanged, typed as AbstractContextManager[object], whose exit type defaults to bool | None (the type contextlib.contextmanager scopes have, which type checkers don't treat as suppressing). The executor's Stage 12 code already types its span wrappers this way (coordination.py). This PR uses it at the entry's one call site; the fix PRs use it in the gateway and executor. No runtime behaviour changes: the same object is returned. A precise annotation upstream (-> Literal[False]) would be the more direct fix, in the policyengine-observability repository.

apis-integ is included. Generating its client takes about 8 seconds once the entry's build environment is installed, the same step test-integration runs. Its code has no src directory, so its [tool.pyright] include is policyengine_api_full_integ and tests. Its 4 errors are the integration tests passing Client | AuthenticatedClient to generated endpoints typed as needing AuthenticatedClient; a PR above this one resolves them.

Each package declares what Pyright checks. Every pyproject.toml now has a [tool.pyright] include. Without one, Pyright checks the whole package directory, tests included: the contract library reported 32 errors in 25 files instead of none in its 14 source files. The rules guard now fails when a package a Pyright rule lists has no include, or is missing from the CI job's matrix.

Changes to the rules tool

  • A pyright enforcer now lists projects (package directories) instead of one config file, since each package configures Pyright in its own pyproject.toml. No existing rule used the old field.
  • The rendered AGENTS.md rules block now lists checked rules by id only, one line per category. The registry had exactly 60 rules on main, and the renderer collapses the block above 60 (AGENTS_COLLAPSE_AT); this PR adds 5.

make check

make check generates the API client, runs ruff, then python -m scripts.rules pyright --sync. It runs every check and fails at the end if any failed; before, a ruff failure counted only in the last directory.

Its ruff step, and make format's, used to run uv run ruff from the repository root, which runs whatever ruff is on PATH. With 0.16.8 there, make check reported 154 CPY001 findings that CI's ruff doesn't select. The Makefile now reads the version from the lint job's uv tool install ruff==<version> step in pr.yml, so the two can't drift, and runs uv tool run ruff@<version>. It stops with a message if it can't read a version.

Commits (red, then green)

  1. Type the observability span scope as never suppressing exceptions. Red: tests/test_scopes.py failed with ImportError: cannot import name 'scopes'; Pyright reported the entry's 2 errors. Green: observability library 35 passed; entry Pyright 0 errors; entry unit tests 652 passed.
  2. Let a Pyright rule name the packages it checks. Red: 7 failures (unknown key 'projects' in enforcer (pyright), and the old description). Green: 120 rules-tool tests passed.
  3. Add a Pyright check that holds each package to its rule. Red: No module named 'scripts.rules.pyright'. Green: 29 tests passed, using a fake uv on PATH; a run on the real entry reported version 1.1.414 and 0 errors with PYRIGHT_PYTHON_FORCE_VERSION=latest set.
  4. Declare in each package what Pyright checks. Red (measured): the contract library checked 25 files, tests included, and reported 32 errors. Green: 14 files, 0 errors; all eight packages measured 0, 0, 0, 0, 0, 81, 178, 4.
  5. Write Pyright baseline entries sorted. Red: the new sorting test failed against the previous module. Green: passes.
  6. Run Pyright for every package in CI, under four registered rules. Red: test_pr_workflow_is_gated failed with ci-complete does not need pyright; the rules guard reported the pyright job as enforcing no registered rule; the three ratchet proofs failed without fixture baselines. Green: test_ci_complete_gate.py, 10 passed; quality guards and python -m scripts.rules guard --base-ref origin/main passed; python -m scripts.rules pyright passed for all eight packages.
  7. Run the Pyright check from make check, syncing each package first. Red: the two --sync tests failed with unrecognized arguments: --sync. Green: 157 rules-tool tests passed; make check exited 0 with ruff 0.15.22 first on PATH.
  8. Fail the quality guards when a package has no Pyright rule. Red: cannot import name 'check_coverage', then the guard runner reported quality guard 'pyright-coverage' ... has no rule. Green: tests and guards pass; a temporary projects/probe-package/pyproject.toml made the guard fail, naming the package.
  9. Rewrite only ratcheted baselines when --write-baseline names no package. Red: the new test failed, because the command tried each enforced package, reported "is enforced, so ... has no baseline" and exited 1. Green: without packages it rewrites each ratcheted package's baseline and leaves enforced ones alone.
  10. Say what --write-baseline adds, and test two untested output paths. Docs and help text now say that --write-baseline refuses an error an existing baseline doesn't list, but records every error when the file doesn't exist yet (adopting a rule). They also say the guard's count comparison is what stops growth. Two new tests pin a run with both a new error and a stale entry, and --sync with --write-baseline; both pass on the existing code.
  11. Run CI's pinned ruff in make check and make format. Red: with ruff 0.16.8 first on PATH, make check failed with 154 CPY001 findings. Green: with the same PATH, make check exits 0 running ruff 0.15.22. A missing version stops make with a message (checked with make require-ruff-version RUFF_VERSION=).
  12. Require a downgrade reason to ratchet an enforced rule. Red: a test moving an enforced rule to ratcheted with a 50-entry baseline got no violation. Green: the guard reports status weakened from enforced to ratcheted unless the rule has a [rule.downgrade] reason.
  13. Reject a package listed twice, or configured outside pyproject.toml. Red: two tests, a package listed by two Pyright rules and a package with a pyrightconfig.json (which Pyright reads instead of [tool.pyright]; typeCheckingMode: off there gave 0 errors on a file with one), got no violation. Green: both are reported.
  14. Refuse option mixes the pyright command ignored, and say when idle. Red: five tests: pyright --write-baselines ran a plain check; --list-packages ignored packages, --sync and --write-baseline; --write-baseline with no ratcheted rule printed only "pyright: passed". Green: usage errors for the first two, and a message for the third.
  15. Run the lint job's ruff commands in make check. Red: make -n check showed ruff check per src directory and no format check, unlike the lint job; and the version sed returned both versions from a workflow with two uv tool install ruff== lines. Green: make check runs ruff format --check projects/*/src libs/*/src and ruff check . with the first pinned version, and exits 0.
  16. Say exactly what the guard checks about Pyright baselines and settings. Docs only: the guard compares counts, not entries, so a swap with the same count is caught only by code-owner review; and the package settings that could weaken the check without failing it are listed.
  17. Run Pyright with --project so a parent pyrightconfig.json is ignored. Red: the new ParentConfigurationTest builds libs/alpha with one return-type error, puts a pyrightconfig.json with typeCheckingMode: off in the repository root and in libs/, and runs the real check with the pinned Pyright; it got no errors. Green: it reports the reportReturnType error in libs/alpha/src/alpha.py. The quality-guards job now installs uv so this test runs in CI. Elsewhere it is skipped when uv is missing; in CI (GITHUB_ACTIONS set) a missing uv fails it.
  18. Spell a package one way in both the guard and the Pyright check. Red: a package listed as libs/alpha by one rule and as ./libs/alpha by another got no violation. Green: normalized_package in scripts/rules/model.py (os.path.normpath, with / separators), used by the duplicate-package check and by the Pyright check, reports it.

Proof fixtures

Under rules/fixtures/<rule id>/{bad,good}/, recorded with Pyright 1.1.414 on main at ceb30e1 (the checkout path replaced by /repository). The enforced rule's bad fixture is the entry's real pre-fix report: invocation_id possibly unbound at stage12_backend.py:550 and :563. Each ratchet's bad fixture has a baseline that omits one real error and lists one that no longer occurs. The proof tests are in scripts/rules/tests/test_pyright.py and run in the quality-guards job.

Environment variables

The pyright job has no secrets and reads:

Variable Provider How it reaches the step
PACKAGE the job's strategy.matrix.package the step's env: (${{ matrix.package }}), so no expression is interpolated into shell
PYRIGHT_PYTHON_FORCE_VERSION not set in CI scripts/rules/pyright.py removes it from Pyright's environment if present
PYRIGHT_PYTHON_IGNORE_WARNINGS scripts/rules/pyright.py set by the script for the Pyright process

The quality-guards job now installs uv with astral-sh/setup-uv@v8.1.0 (no inputs) and still has no secrets. ParentConfigurationTest reads GITHUB_ACTIONS, which GitHub Actions sets in every step, to fail instead of skipping when uv is missing. Its uv tool run --from pyright==1.1.414 downloads the pyright package from PyPI and Pyright from npm, using Node from the ubuntu-latest image; neither download needs credentials.

./scripts/generate-clients.sh (apis-integ only) builds the entry app, whose Settings.from_env() reads APP_ENVIRONMENT, SIMULATION_ENTRYPOINT_AUTH_*, OLD_GATEWAY_*, STAGE12_* and K_REVISION. None is set in this job; each has a default, and the test-integration and dependency-update jobs already run the same script without them. uv, python and Node (used by the pyright package) come from astral-sh/setup-uv, actions/setup-python and the ubuntu-latest image; the pyright package downloads Pyright from npm on first use.

Self-review findings and how they were resolved

review-code ran twice on this PR, the second time from a separate agent over the whole stack. Neither run found a bug. Code-quality findings:

  1. --write-baseline with no package argument failed for every enforced package, although the ratcheted baselines were written. Fixed (commit 9).
  2. The docs said --write-baseline never adds an error, but a missing baseline is recorded in full. Corrected (commit 10).
  3. Two output paths had no tests. Added (commit 10).
  4. The docs overstated what the guard prevents: it compares counts, not entries, and nothing stopped an enforced rule from being ratcheted again with any count. Fixed: the guard now requires a downgrade reason for that move (commit 12), and the docs say a same-count swap is caught only by code-owner review (commit 16). I didn't add entry-by-entry comparison with the base branch: the guard is shared by every ratcheted rule, and the stack ends with no Pyright baselines.
  5. Package settings could switch checking off unseen: a pyrightconfig.json is now rejected (commit 13). ignore, a lower typeCheckingMode or rule severities in [tool.pyright] are documented as review-only (commit 16). Judging those settings would mean encoding a policy for every Pyright option.
  6. A package could be listed by two rules. Now rejected (commit 13).
  7. Command-line edge cases. Fixed (commit 14). Not changed: options placed between package arguments fail on Python 3.11 in argparse. This repository requires Python 3.13, where they work.
  8. make check's sed could read two versions, and its ruff scope differed from CI's. Fixed (commit 15).
  9. The rendered AGENTS.md block now lists checked rules by id only. Not changed: this is the renderer's designed behaviour above 60 rules, and any added rule would cross it.

A later review-code pass over the whole stack found two more problems in this PR. Both are fixed:

  1. A pyrightconfig.json in a parent directory still replaced a package's [tool.pyright] settings, because the guard looked for one only inside the package and Pyright searches parent directories too. Fixed by running Pyright with --project <package>, proven with the real Pyright (commit 17).
  2. The duplicate-package check compared the strings as written, so ./libs/alpha and libs/alpha passed it as two packages. Fixed (commit 18).

A review-code pass over commits 17 and 18 found no bugs. Minor findings, not changed:

  • ParentConfigurationTest proves that a parent pyrightconfig.json is ignored, but not that the package's own [tool.pyright] is read: its only setting, include = ["src"], gives the same result as Pyright's defaults.
  • The test downloads the pyright package from PyPI and Pyright from npm the first time it runs in a job, so a registry outage can fail the quality-guards job.

Suppressions added

None.

Real bugs found

None.

Testing

  • python -m unittest discover -s scripts/rules/tests -t .: 170 tests pass, including ParentConfigurationTest with the real Pyright 1.1.414.
  • python scripts/run_quality_guards.py: rules-registry, ci-scripts and pyright-coverage pass; python -m scripts.rules guard --base-ref origin/main passes.
  • python -m scripts.rules pyright: all eight packages pass.
  • uvx ruff@0.15.22 format --check projects/*/src libs/*/src and uvx ruff@0.15.22 check .: clean.
  • make check: exits 0.
  • Observability library tests: 35 passed. Entry unit tests: 652 passed. Executor test_ci_complete_gate.py and test_modal_scripts.py: 73 passed.

🤖 Generated with Claude Code

anth-volk and others added 11 commits October 21, 2026 18:17
@anth-volk @claude
Type the observability span scope as never suppressing exceptions
e4d282d 
policyengine-observability annotates its scope managers' __exit__ as
returning bool, although it always returns False. Pyright reads a bool
result as "this context manager may suppress the exception", so it
reports every variable assigned inside `with runtime.span(...)` as
possibly unbound afterwards.

scopes.span returns the runtime's own scope, typed as a context manager
whose __exit__ returns bool | None, which type checkers don't treat as
suppressing exceptions. The entry's Stage 12 dispatch uses it, which
fixes its two Pyright errors (invocation_id possibly unbound).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk @claude
Let a Pyright rule name the packages it checks
13d2535 
A Pyright rule's enforcer now lists the package directories it checks
(`projects`), replacing the single `config` file: in this repository
each package configures Pyright in its own pyproject.toml. The guard
checks that each package exists, declares what Pyright checks with a
[tool.pyright] include, and appears in the matrix of the rule's CI job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk @claude
Add a Pyright check that holds each package to its rule
ecd23d8 
`python -m scripts.rules pyright <package>` finds the Pyright rule that
lists the package, runs the pinned Pyright version (1.1.414) in the
package directory against the interpreter of the package's locked
environment, and compares the errors with the rule:

- under an enforced rule the package may have no errors;
- under a ratcheted rule it may have exactly the errors its baseline
  lists. A new error fails, and so does an entry for an error that no
  longer occurs, so the baseline only shrinks.

Pinned Pyright 1.1.409 to 1.1.414 have no --baselinefile option, and
their messages differ for the same error, so the baseline is a text
file of `<path> :: <rule> :: <first message line>` entries, one per
error and without line numbers, compared by this standard-library
script. --write-baseline drops entries that no longer occur and never
adds one. Without a package argument every listed package is checked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk @claude
Declare in each package what Pyright checks
d41cf0b 
Every package's pyproject.toml now has a [tool.pyright] include: src
for the libraries and services, and the integration tests for
apis-integ, whose code has no src directory. Without one, Pyright
checks the whole package directory, tests included: the contract
library reported 32 errors in 25 files instead of none in its 14
source files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk @claude
Write Pyright baseline entries sorted, under a wrapped header
85b7447 
Entries were written in the order Pyright reports errors, which moves
an entry whenever code above it moves. Sorted, a fix's diff only
removes lines, and identical errors sit next to each other.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk @claude
Run Pyright for every package in CI, under four registered rules
2e97499 
The new pyright job in pr.yml checks each uv package in its own locked
environment with `python -m scripts.rules pyright`. It generates the
API client first for apis-integ, which installs it from a path. CI
complete needs the job, and it runs only at the bottom of a stack, like
the other jobs.

Rules:
- sim.quality.pyright-libraries-and-entry (enforced): the contract,
  observability, Stage 12 persistence and fastapi libraries and the
  entry have no errors.
- sim.quality.pyright-gateway (ratcheted, 81 errors),
  sim.quality.pyright-executor (ratcheted, 178) and
  sim.quality.pyright-apis-integ (ratcheted, 4): only the errors their
  baselines list.

Each rule's proof runs the comparison on recorded Pyright reports from
main at ceb30e1. The enforced rule's bad fixture is the entry's real
pre-fix report. docs/engineering/skills/type-checking.md explains the
check, the pinned version and the baselines.

The registry now has 64 rules, past the renderer's limit of 60, so the
AGENTS.md rules block lists checked rules by id per category.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk @claude
Run the Pyright check from make check, syncing each package first
99e340d 
`python -m scripts.rules pyright --sync` runs `uv sync --frozen
--all-extras` in each package before checking it, and fails the package
when the sync fails. make check now runs it for every listed package,
after generating the API client that apis-integ installs, instead of
running `uv run pyright <dir>` from the repository root, which used no
package's environment. It also runs every check and fails at the end if
any failed; before, only the last directory's result counted. The CI
job runs the same command.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk @claude
Fail the quality guards when a package has no Pyright rule
759aeae 
The pyright-coverage guard lists every uv package (a directory under
libs/ or projects/ with a pyproject.toml) and fails for each one that no
Pyright rule lists, naming the registry entry and the CI matrix to
update. Registered as sim.quality.every-package-type-checked, with a
proof fixture holding a package no rule lists.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk @claude
Rewrite only ratcheted baselines when --write-baseline names no package
5ff94f2 
`python -m scripts.rules pyright --write-baseline` with no package
argument tried every listed package, failed for each enforced one
("is enforced, so ... has no baseline") and exited 1, even though it
had rewritten the ratcheted baselines. Without packages it now rewrites
the baseline of every package a ratcheted rule lists, and leaves the
enforced packages alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk @claude
Say what --write-baseline adds, and test two untested output paths
d4f982e 
The docs and help text said --write-baseline never adds an error. It
refuses an error an existing baseline doesn't list, but it records every
error when the baseline file doesn't exist yet, which is how a package
is first ratcheted. The docs now say so, and that the rules guard's
count comparison is what stops a baseline growing.

New tests pin two paths that had none: a run with both a new error and
an entry that no longer occurs reports both and both fixes, and --sync
with --write-baseline syncs before rewriting. Both pass on the existing
code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk @claude
Run CI's pinned ruff in make check and make format
b9f2780 
Both targets ran `uv run ruff` from the repository root, which runs
whatever ruff is on PATH. With ruff 0.16.8 there, make check reported
154 CPY001 findings that CI's ruff 0.15.22 doesn't select, and failed.
The Makefile now reads the version from the lint job's
`uv tool install ruff==<version>` step in pr.yml and runs
`uv tool run ruff@<version>`, and stops with a message if it can't read
one. With 0.16.8 still first on PATH, make check now passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 21, 2026
Closed
Merged
@anth-volk
anth-volk added this pull request to stack #867 October 21, 2026 14:53
This was referenced Oct 21, 2026
Closed
Merged
Open
Draft
Open
Open
Open
anth-volk and others added 5 commits October 21, 2026 20:17
@anth-volk @claude
Require a downgrade reason to ratchet an enforced rule
a93cb17 
The guard compared baseline counts with the base branch only when the
base rule already had a baseline, and counted only a move to advisory or
doc-only as a weakening. So an enforced rule could become ratcheted with
a baseline of any size and nothing would object. Moving from enforced to
ratcheted now needs a [rule.downgrade] reason, like the other
weakenings, and the registry docs say so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk @claude
Reject a package listed twice, or configured outside pyproject.toml
f79e850 
Two gaps in what the guard checks about a Pyright rule's packages:

- A package listed by two Pyright rules was checked against the first
  only, and twice in a default run. Registry validation now reports the
  second listing.
- A pyrightconfig.json in a package takes precedence over the
  [tool.pyright] table the guard inspects, so it could switch checking
  off unseen (`typeCheckingMode: off` gave 0 errors on a file with one).
  The guard now fails when a listed package has one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk @claude
Refuse option mixes the pyright command ignored, and say when idle
508d333 
- `pyright --write-baselines` (the ci-scripts flag) ran a plain check
  without writing anything; it is now a usage error that names
  --write-baseline.
- `--list-packages` ignored package arguments, --sync and
  --write-baseline; combining them is now a usage error.
- `--write-baseline` with no package argument and no ratcheted Pyright
  rule wrote nothing and said only "pyright: passed"; it now says there
  is no baseline to write.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk @claude
Run the lint job's ruff commands in make check
ae82411 
make check ran `ruff check` on each src directory and no format check,
while CI's lint job runs `ruff format --check projects/*/src libs/*/src`
and `ruff check .`, so a passing make check didn't mean a passing job.
It now runs those two commands, with CI's version. The version is read
from the first `uv tool install ruff==` line in pr.yml; before, a second
such line would have produced two versions and a broken command.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk @claude
Say exactly what the guard checks about Pyright baselines and settings
8d64716 
The type-checking skill said deleting and rewriting a baseline would pass
the guard only if the count stayed the same, as though entries were
compared. The guard compares counts, so a change that fixes one error
and adds another passes it; the skill now says so, names code-owner
review of rules/ as what catches that, and lists the package settings
(ignore, typeCheckingMode, rule severities) that could weaken the check
without failing it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 21, 2026
Draft
Draft
anth-volk and others added 2 commits October 21, 2026 20:51
@anth-volk @claude
Run Pyright with --project so a parent pyrightconfig.json is ignored
427b981 
Started in a package directory without --project, Pyright 1.1.414 also
uses a pyrightconfig.json from a parent directory, such as the
repository root, libs/ or projects/, instead of the package's
[tool.pyright]. A parent file with `typeCheckingMode: off` reported 0
errors on a file with one. The check now passes `--project <package>`,
so Pyright reads only the package's own configuration; the guard
already rejects a pyrightconfig.json inside a package.

A new test runs the pinned Pyright for real on a package with parent
pyrightconfig.json files: it found no errors before and finds the one
error now. The quality-guards job installs uv so the test runs there;
it fails in CI if uv is missing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk @claude
Spell a package one way in both the guard and the Pyright check
189541a 
The duplicate-package check normalised paths with rstrip("/"), but the
Pyright check normalises with os.path.normpath, so ./libs/alpha in one
rule and libs/alpha in another passed validation and were then checked
as one package against the first rule only. Both now use one function,
normalized_package, so the second listing is reported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk
anth-volk marked this pull request as ready for review October 21, 2026 20:22
@anth-volk
anth-volk requested a review from a team as a code owner October 21, 2026 20:22
@anth-volk
anth-volk merged commit 07e3ec4 into main Oct 21, 2026
22 checks passed
@anth-volk
anth-volk deleted the pyright-ci branch October 21, 2026 20:30
@anth-volk anth-volk mentioned this pull request Oct 21, 2026
Merged
anth-volk added a commit that referenced this pull request Oct 21, 2026
@anth-volk @claude
Fix the executor Pyright errors this branch adds
aaa97c8 
Main now runs Pyright on every package (#864). Against the executor's
baseline, this branch added seven errors:

- The run's new reads of `dataset_selection`, `region_resolution` and
  `simulation_params` were reported as possibly unbound, because
  `runtime.span(...)` is typed as possibly suppressing exceptions. The
  run's spans now open with `scopes.span(runtime, ...)`, as
  docs/engineering/skills/type-checking.md says. `_build_simulation` is
  annotated with the `StageReportingSimulation` it returns, which the
  progress callback needs.
- `build_state_datasets` couldn't be passed as a `RemoteFunction`,
  because Pyright doesn't resolve `modal` in src/modal to the Modal
  package. The precompute app casts its six Modal functions to that
  protocol.
- The manifest's artifact list is built with `ManifestEntry` elements,
  and the district-code check passes `isin` a sorted list.

54 baseline entries no longer occur, so the baseline drops from 178 to
124 entries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk anth-volk mentioned this pull request Oct 21, 2026
Merged
2 tasks done
anth-volk added a commit that referenced this pull request Oct 21, 2026
@anth-volk @claude
Bake per-state ACS-local year files into the main path's worker image (…
c94bbd3 
…#879)

* Report the US state code when resolving a state or district region

RegionResolution gains state_code: the two-letter code of the US state a
state or congressional-district region lies in, taken from the .py region.
National, UK and region-group resolutions leave it None. The baked
per-state datasets added next are looked up by this code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Let callers choose the folder _load_dataset reads a year file from

_load_dataset takes an optional data_folder that replaces the folder the
dataset selection would otherwise use (the image's data folder for the
default dataset, a temporary folder for any other). Callers are unchanged;
the main path and Stage 12 will each pass the folder of a baked per-state
dataset when one exists.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Key and lay out per-state datasets for the main path's precompute

A per-state dataset is a regional dataset's year file filtered to one
state's households. Its content key (schema sds1) covers the regional
dataset's name, revision-pinned URI and certified SHA-256, the year, the
state code and FIPS, the filter variable, the model and .py versions and
the SPM selection. Its store path is
state-datasets/<country>/<digest>/<stem>_year_<year>.h5, keeping the
basename ensure_datasets resolves.

state_datasets.py gives each state's folder under the image data folder,
state-datasets/<dataset>/<STATE>, so the manifest builder and the runtime
lookup agree on where a state's files live. The default dataset's key and
identity are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Plan a per-state dataset for each state that loads a regional dataset

state_dataset_targets resolves every US state region exactly as a request
for it is resolved and keeps the states whose dataset is not the bundle
default, with their row filter (state_fips) and state code. The plan lists
one StateDatasetPlanEntry per precompute year and target state, year-major
and by state code, with its key, store path, runtime filename, image folder
and store presence; select_work keeps the missing ones. A bundle that sends
states back to the default dataset plans none.

Nothing builds or publishes these entries yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Build, check and upload one year's per-state datasets

build_state_datasets_impl re-derives each planned state's key in its own
container and refuses to build when the planner disagreed, then has .py's
ensure_datasets prepare the regional dataset's full-country year file once,
collects garbage, and reloads that one year. It refuses data in which a
household's congressional district code names another state, since
district requests read their state's file. For each state it keeps the
rows the state region's own scoping strategy keeps, writes them with .py's
save(), copies the year-file stamp that save() drops, confirms that .py's
reuse checks (stamp and renames record, then the legacy SPM record) accept
the file, and uploads it write-once. Weights are unchanged.

The tests write a three-household year file with .py's own save, stamp and
SPM record, and show that .py's real ensure_datasets reuses a state file
without rebuilding it. The identity stubs now import .py's US dataset module
before patching the manifest, so a stubbed test cannot bind the stub into it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Place manifest artifacts that name a folder in that folder of the image

The image's artifact fetch now puts an artifact with a "folder" at
<data folder>/<folder>/<filename>, so per-state datasets, which all share
the year file's basename, land in their own state folders. Artifacts
without a folder still go straight into the data folder. Before anything
downloads, it refuses absolute folders, folders with "." or ".." parts,
empty parts or backslashes, filenames that are not bare names, and two
entries for the same file.

The fetch layer's timeout rises from 900 s to 3,600 s for the several GB
of per-state datasets. No manifest lists such artifacts yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Run the per-state dataset builds in the main path's precompute

The precompute app gains build_state_datasets (8 CPUs, 96 GiB, two hours),
one container per year, validating plan entries in and dumping build
results out like the other wrappers. run_precompute spawns the state builds
with the dataset builds and waits for them only before publishing the
manifest, so baseline computation never waits for them, then reports how
many state datasets were built and uploaded and their total size.

The manifest does not list the state datasets yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* List the per-state datasets in the main path's deploy manifest

build_manifest appends a state_dataset entry for every planned state file,
with its folder and state code, after the datasets and baselines. The new
entries use their own StateDatasetArtifact model, chosen by "type", so a
plan without per-state datasets publishes byte-for-byte the manifest it
always did (the pinned wire-shape test is unchanged). The deploy-time
manifest validation accepts the new entries and hands them to the fetch
layer, which places each in its state's folder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Read a state's baked dataset for US state and district requests

Before loading a non-default dataset, the main path's runtime asks
state_datasets.baked_state_data_folder for the image folder of the
request's state (a district passes its state's code). It returns the folder
only when the image holds that state's year file for the requested year
and .py's reuse checks (stamp and renames record, legacy SPM record) accept
it; a file .py would rebuild is skipped, so ensure_datasets never rebuilds
the full regional dataset inside a baked folder. _load_dataset then reads
from that folder, and the district filter still applies at run time.
Otherwise the request prepares the regional dataset on demand, as before.

Runs whose region loads a non-default dataset record dataset_source as
baked_state or on_demand. National and default-dataset runs are unchanged,
and Stage 12 still loads without a folder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Refuse only district codes that name another state's district

The per-state build refused the whole year whenever any household's
congressional_district_geoid did not start with its own state FIPS,
including codes that name no district, such as an unassigned 0. Such a
household matches no district request on either path, so it cannot change
a result, yet it would have stopped the precompute and the deploy waiting
on it. The build now refuses only codes that name a registered district
of a different state.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Build state datasets per year and dataset, and report what was uploaded

run_precompute grouped state entries by year only, while one build
prepares a single dataset's year file and refuses entries for two. A bundle
sending different states to different datasets would have failed every
year's build; builds are now grouped by year and dataset.

Each state build result records how long the year's build had run when the
file was uploaded, and the precompute log now reports, per build, how many
files were built and uploaded, the bytes actually uploaded (files already
stored no longer count), and the build time, for tuning the two-hour limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Test district loading end to end and state file rows against the source

A district request is now followed from region resolution, with the real
dataset selection and baked-folder lookup over a stamped Utah file, to the
folder _load_dataset is given; the old test only repeated a state lookup.
Each state file's rows are now compared with .py's filter applied to the
full source frames, instead of with the file filtered again, which would
not have noticed another state's rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Type the state target's row filter and refuse a target without one

StateDatasetTarget.scoping_strategy is typed as an optional
RowFilterStrategy instead of Any, and a build whose target has none raises
a clear error before writing anything, instead of failing on None. A
comment in artifact_keys now says that per-state datasets are fetched into
their state's folder rather than straight into the data folder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Load on demand when .py's year-file reuse checks cannot be imported

The baked-folder lookup runs .py's reuse checks, one of which imports a
private .py helper. If a .py upgrade moved it, every state and district
request would have failed with ImportError while loading its dataset. The
lookup now logs a warning and returns no folder, so the request prepares
its dataset on demand as it would without a baked file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Refuse a state without a row filter before preparing the year file

The check for a state target without a row filter ran inside the per-state
loop, after the full-country year file was prepared and earlier states
were uploaded. _checked_state_targets now refuses it with the other
in-container checks, before anything is prepared, and returns each
state's row filter. Docstrings now say builds run one container per year
and dataset.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Fix the executor Pyright errors this branch adds

Main now runs Pyright on every package (#864). Against the executor's
baseline, this branch added seven errors:

- The run's new reads of `dataset_selection`, `region_resolution` and
  `simulation_params` were reported as possibly unbound, because
  `runtime.span(...)` is typed as possibly suppressing exceptions. The
  run's spans now open with `scopes.span(runtime, ...)`, as
  docs/engineering/skills/type-checking.md says. `_build_simulation` is
  annotated with the `StageReportingSimulation` it returns, which the
  progress callback needs.
- `build_state_datasets` couldn't be passed as a `RemoteFunction`,
  because Pyright doesn't resolve `modal` in src/modal to the Modal
  package. The precompute app casts its six Modal functions to that
  protocol.
- The manifest's artifact list is built with `ManifestEntry` elements,
  and the district-code check passes `isin` a sorted list.

54 baseline entries no longer occur, so the baseline drops from 178 to
124 entries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Anthony Volk <14987227+anth-volk@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Run Pyright in CI for every package, in its own locked environment

1 participant

@anth-volk