[dan-petty]

devops roadmap plan-check had no token measure, so a plan above the stage budget roadmap.stage.step_tokens could start a run. It now measures each task group with measure_groups, the function of the stage job. It reports a group-over-budget problem for a group above the budget and a group-no-files problem for a group that changes a file and packs none. The stage job and plan-check build the shared text with one function, shared_texts, from the issue body with no record block, so the two figures are equal. The run record also keeps the input tokens of the first request of each step in implement.spend, so a person can compare a stage figure with what the model received.

Closes #3499

Change: openspec/changes/archive/2026-11-05-3499-plan-check-measures-each-group/

[dan-petty]

Review finding fixed

A review found that plan-check measured the raw GitHub body of the issue. That body ends with the record block of the jobs. The stage job measures item.body, which has no record block. So the figure of plan-check was too high and could give a false group-over-budget problem. I confirmed this before the fix: with a record block of about 600 tokens at the end of the body, the budget of 8597 tokens gave a figure of 9209 tokens. The two review items were the same defect.

Fix (commit fix(roadmap): plan-check measures the issue body with no record block): plan_check calls split_record once. It gives the same body to the gates and to the size measure, as the store gives it to the stage job. Two new tests in tests/roadmap/test_roadmap_plan_check.py use an issue body that has text and a record block. One test sets the budget one token below the stage figure and checks the figure in the message. The other test sets the budget to the stage figure and checks that there is no problem. Both tests failed on the code before the fix.

Fail-to-pass run

uv run devops ci fail-to-pass --base origin/main ended with exit 0 on the head of this branch:

• 3 tests failed on the base: the spend holds the size of the first request of each step, the outcome of a step holds the input tokens of its first request, and the run with more events than the bound keeps those sizes.
• 2 test files errored on the base at collection: tests/roadmap/test_roadmap_plan_check.py and tests/roadmap/test_roadmap_stage_eval_run.py. They need shared_texts and the moved checkout_at, which the base does not have.
• 13 other tests in the changed files passed on the base.

Full uv run devops ci on the final code

All checks pass except test and coverage:

• test: 18811 passed, 4 failed, 1 error. The 4 failures and the error are the host-state tests of the open issues fix(ci): host state fails four tests and one case #3585, fix(tests): four tests fail on the devcontainer host #3590, fix(ci): host-state tests fail and the suite exceeds 5 minutes #3593, fix(tests): four host-dependent tests fail on main #3594 and fix(ci): four tests read host state and fail in a devcontainer #3597: test_get_ssh_key_prefix_resolution, test_find_project_config_path_skips_directory, test_mark_only_commands_under_global_dry_run and test_the_direct_executor_refuses_a_network_profile_before_it_starts_the_tool[direct]. They read the file /tmp/config.yaml and the tools installed on this host. I ran the same four test files in a clean worktree of origin/main: the same 4 failures and the same error. Nothing in this change touches them.
• coverage has no result because test failed on this host. The checks on the GitHub runner give the figure.
• lint, format, typecheck, security, docs, openspec, test-tree, swallowed-failures and the other checks pass.
• The suite took 8m 25s on a host that ran several gates at the same time. fix(ci): host-state tests fail and the suite exceeds 5 minutes #3593 tracks the 5 minute budget.

The OpenSpec change is archived in the last commit. The branch is a fast-forward of origin/main at f6d57ecd.