[vahid-ahmadi]

Why

The five Python shards are at both of their limits.

Time. Shards now take 57 to 75 minutes against a 75-minute timeout-minutes.

• #2649, run 40308966462, first attempt: shards took 63, 72, 68 and 64 minutes, and the limit cancelled shard 4/5 at 75 minutes with every test passed. Its earlier runs took 54 to 65 minutes.
• Merge queue run 40406434741 (26 October): 66 to 69 minutes on all five shards.

Memory. Peaks are 13 to 15.9 GB of the 16 GB runner.

• #2661, run 40393666784: shards 2/5, 3/5 and 5/5 lost their runners at 15,874, 15,586 and 15,752 MB.
• The merge queue run for Uprate the Scottish Carer Supplement by September CPI with 5p rounding #2105, run 40395541241: shard 2/5 lost its runner at 15,501 MB. So main itself is now at the limit, not only one branch.

#2637 deals the slow tests out by recorded duration, so a sixth shard spreads both time and memory. With shard_durations.toml as it stands, the listed slow tests come to about 161 minutes of worker time per shard on five shards and 134 on six; the hashed remainder also falls by a sixth per shard.

What changed

• .github/workflows/test-suite.yaml: the matrix is shard: [1, 2, 3, 4, 5, 6], and the comments say six and K/6. A new paragraph above the job records the numbers above.
• pr_code_changes.yaml, Makefile, docs/engineering/skills/testing.md and docs/book/validation/ubuntu-vs-macos.md: "five" and K/5 become "six" and K/6.
• A changelog fragment.

Nothing else needed editing. The job name and --shard use strategy.job-total, the outcome artifacts are outcomes-K.jsonl, Result downloads test-* and passes outcomes-*.jsonl, and check_test_shards.py takes the shard count from the logs it is given. Historical comments (the 5-shard measurements in test-suite.yaml, conftest.py and shard_durations.toml) are left as records.

Checks

The required checks keep their names: "Test / Result" and "Lint" (checked against the branch rules for main). The per-shard checks become "Test / Python K/6"; none of them is required, so the merge queue is not affected.

Run locally in a fresh uv sync --frozen --extra dev environment:

• test_ci_test_shards.py and test_ci_runner_variables.py: 126 passed.
• Those 126 tests run as --shard K/6 for K = 1 to 6 (22, 20, 24, 20, 18 and 22 tests), then check_test_shards.py on the six outcome logs: every collected test ran in exactly one shard and passed.

Not run locally: the full suite. This PR's own CI run is the test of the timings and memory; I will add the six shards' durations and peaks here when it finishes.

For Max to decide

• Cost. This adds one arm64 runner per run of the suite (pull requests, merge queue groups, main pushes and releases). Total runner minutes should stay roughly flat, since the same tests are spread over more machines, but each run holds six runners instead of five and pays the setup (install, dataset download) one more time. Max, please OK the cost before this merges.
• Limits. This gives headroom; it does not fix a branch that itself uses more memory. If a pull request's tests push a worker's memory up (as Carry unused Income Tax trading losses forward once #2661's did, 0.6 to 0.9 GB above passing runs), it can still run out on six shards. The underlying cause is the per-worker memory growth in The Python test shards run out of memory under sys.monitoring coverage #2330.

🤖 Generated with Claude Code

[vahid-ahmadi]

This PR's own run, 40422010249: all six shards and Result passed.

	Duration	Peak memory used
1/6	60 min	13,844 MB
2/6	64 min	10,810 MB
3/6	60 min	15,612 MB
4/6	57 min	14,219 MB
5/6	53 min	14,968 MB
6/6	53 min	14,400 MB

Against five shards: merge queue run 40406434741 took 66 to 69 minutes with peaks of 12,845 to 15,589 MB, and #2649's run 40308966462 took 63 to 75 minutes (4/5 cancelled at the limit).

So time improves: the slowest shard is 64 minutes instead of 69 to 75, about 11 minutes under the limit. Memory does not improve the way I expected. Shard 3/6 peaked at 15,612 MB, as high as the worst five-shard shard, and four of six shards are still above 14 GB. The peak seems to come from particular tests on a worker rather than from how many tests each worker runs, so a sixth shard does not by itself fix the runner losses. That is one run, so treat it as a first reading.

Max, this still needs your OK on the extra runner. A real memory fix probably needs a look at which tests drive the peak (#2330), separately from this PR.

🤖 Generated with Claude Code

[juaristi22]

Program review

Base repository: PolicyEngine/policyengine-uk
PR number: 2690
Reviewed head SHA: e252830
Merge base SHA: 96315db
Mode: full
Scope: Changed CI six-shard behavior and affected static dependencies; no policy changes or performance audit
Source manifest: /private/tmp/policyengine-command-runs/881387a1b745/pr2690-external-review-20261027-review-sources.json
Review status: COMPLETE

Source Documents

No source documents registered; see scope and validation.

Critical

None.

Should Address

None.

Suggestions

None.

Evidence Gaps

No material gaps reported.

Notes

• CODE-only review: this is pure infrastructure, so policy/source review is inapplicable under the workflow. Static evidence traces strategy.job-total into K/6 selection, generic duration/hash partitioning, numbered artifact logs, and wildcard Result completeness/failure auditing. PR, main, queue and wheel callers use the same reusable workflow; Result/Lint names and data/publication gates are unchanged. See the detailed CODE report for file/line evidence.
• Max’s explicit extra-runner cost decision remains a maintainer landing choice. Live branch-protection settings, author-reported local test counts, historical durations/RSS, memory causes and universal memory relief were not independently certified. This code approval grants no merge, queue, cost or administrative authority.
• The original canonical start at 09:00:54.312239 UTC is retained. A long wall-clock gap before CODE analysis has unproved cause; the complete review exceeds its original 20/25-minute targets. No clock reset, excluded time or workflow timing PASS is claimed. The first CODE dispatch marker was not saved; no exact dispatch/scope60-second success claim is made.

Validation Summary

Exact e252830 capture has 36 raw checks:35SUCCESS and1PublishSKIPPED; all CheckSuite commits join to the reviewed head. Source run40422010249 and release-wheel run40422010259 passed YAML, all six Python shards and Result (source121941740975; release121941296302). This is hosted check-status evidence, not independently read logs/artifacts/test counts/private amounts/RSS. Local tests, collection, country/Core imports, model calculations, environment setup and formatting NOT RUN; no duplicate local run needed for this matrix-only static review. New external source GETs0/sourcebudget0. Detailed CODE evidence: /private/tmp/policyengine-command-runs/881387a1b745/pr2690-external-review-20261027-review-code.md Exact snapshot head/tree/index remained clean, then it was removed normally after CODE DONE; no forced cleanup. Receipt: /private/tmp/policyengine-command-runs/881387a1b745/pr2690-external-review-20261027-Root-normal-cleanup.json

Timing

setup seconds: 2991.97s; code role seconds: 312.43s; policy role seconds: 0.00s; adjudication seconds: 0.00s; consolidation cleanup seconds: 93.70s; elapsed seconds: 3458.29s

Review Severity

APPROVE. Open findings: 0 critical, 0 should address, 0 suggestions.