fix(roadmap): plan-check measures each task group

README.md

@@ -380,7 +380,7 @@ and excessive permissions fail it, and every other finding is shown. |
 |  | `devops roadmap stage [OPTIONS]` | Make Ready items Staged: for each item, push a branch with its OpenSpec change in one commit and open a draft plan pull request. The Service marks the draft ready when its checks pass. Details: docs/roadmap/stage.md. |
 |  | `devops roadmap run [OPTIONS]` | Run the roadmap jobs that are due, in order: metrics, then intake, refine and the alternatives check, and record each one's last success. Intake is due on an issue reopened, and decides at most 5 candidates a run and keeps a record of those it left beside the schedule; refine is due every 31 minutes, on a first run, and after an intake that placed a critical fix or a P0 item; the alternatives check is due every 719 minutes and on a first run, and checks at most roadmap.alternatives.first_pass_limit items while an item has no check, and roadmap.alternatives.limit items after. Without --confirm, or with --dry-run, prints the due list and runs nothing. |
 |  | `devops roadmap implement [OPTIONS]` | Implement a Staged or Planned item in bounded steps: with --group, run one task group in a checkout; without it, run the job, which takes items, runs their steps, and marks the pull request ready. Details: docs/roadmap/implement.md. |
-|  | `devops roadmap plan-check [OPTIONS] <pull_request>` | Check a plan pull request: the stage gates without a model, the title, the body, the `Semantic PR title` check and a base spec that moved since the plan. Details: docs/roadmap/plan-check.md. |
+|  | `devops roadmap plan-check [OPTIONS] <pull_request>` | Check a plan pull request: the stage gates without a model, the token size of each task group, the title, the body, the `Semantic PR title` check and a base spec that moved since the plan. Details: docs/roadmap/plan-check.md. |
 |  | `devops roadmap plan-recheck [OPTIONS]` | Check each change folder on the default branch again. Each folder gets one result: ok, rewrite, new-plan, delivered or cleanup. The command reads GitHub and git, calls no model and writes nothing. Details: docs/roadmap/plan-recheck.md. |
 |  | `devops roadmap explain [OPTIONS] <item>` | Say, for one issue, whether the refine job, the stage job and the implement job take it. For each job the answer is yes, or no with the reason of the first rule that fails: the issue is closed or is no item, the filter, the rules of the job (the Status, the owner, the stop on record, the plan and the branch, and the setting roadmap.stage.planned_limit), and the limit of a run. For refine the reason is the wait of the item, such as a question for a person. The reasons are the texts of the 'Left out' list of stage --plan and implement --plan. The command reads GitHub and git, calls no model and writes nothing. It uses the filter and the limit that the Service uses for each job. A group over the step budget shows only when an earlier run recorded it. |
 |  | `devops roadmap inbox [OPTIONS]` | List what waits for a person in each repository of service.repos, or in the one that --repo names. Each line has its number, its age and the step that clears it, and an old wait is marked. It reads GitHub, calls no model and writes nothing; details: docs/roadmap/inbox.md. |

docs/CLI_REFERENCE.md

@@ -6690,7 +6690,7 @@ devops roadmap implement [OPTIONS]
 
 ### `devops roadmap plan-check`
 
-**Check a plan pull request: the stage gates without a model, the title, the body, the `Semantic PR title` check and a base spec that moved since the plan. Details: docs/roadmap/plan-check.md.**
+**Check a plan pull request: the stage gates without a model, the token size of each task group, the title, the body, the `Semantic PR title` check and a base spec that moved since the plan. Details: docs/roadmap/plan-check.md.**
 
 ```bash
 devops roadmap plan-check [OPTIONS] <pull_request>

docs/ROADMAP_WORKFLOW.md

@@ -114,7 +114,7 @@ These are all the keys of the job record, by the job that writes them:
 | Stage | `stage.needs_fix` | The plan pull request that the Service told about the label `plan/needs-fix` |
 | Stage | `stage.failed_at`, `stage.failed_error` | When the last plan or write of the Item failed and the class of its error. The job leaves the Item out for `roadmap.stage.retry_hours` hours; a staged Item or an Item stopped on the budget clears both |
 | Implement | `implement.group`, `implement.sha`, `implement.branch`, `implement.pr` | The last group that passed, the last commit, the branch and the pull request |
-| Implement | `implement.stopped`, `implement.failure`, `implement.spend`, `implement.trace` | The step that stopped the Item, the class of its failure, the spend of its steps, and the trace of its last run |
+| Implement | `implement.stopped`, `implement.failure`, `implement.spend`, `implement.trace` | The step that stopped the Item, the class of its failure, the spend of its steps with the input tokens of the first request of each step (`steps`, kept apart from the trace, which drops its oldest events), and the trace of its last run |
 | Implement | `implement.check`, `implement.check_log` | The check that failed in the stop, and the masked end of its log (at most 2000 characters). A retry and a clear rule of the Service remove both with the stop |
 | Implement | `implement.wip` | The work in progress that a step kept: its group, the tasks done, the tasks of the group and the last commit as JSON. The job removes it when the group passes |
 | Implement | `implement.job`, `implement.job_state`, `implement.job_digest` | The runner Job that the Service made, its state after a round reported it, and the digest of the runner image that the Job ran |

docs/commands/roadmap.md

@@ -166,7 +166,7 @@ devops roadmap implement [OPTIONS]
 
 ## `devops roadmap plan-check`
 
-**Check a plan pull request: the stage gates without a model, the title, the body, the `Semantic PR title` check and a base spec that moved since the plan. Details: docs/roadmap/plan-check.md.**
+**Check a plan pull request: the stage gates without a model, the token size of each task group, the title, the body, the `Semantic PR title` check and a base spec that moved since the plan. Details: docs/roadmap/plan-check.md.**
 
 ```bash
 devops roadmap plan-check [OPTIONS] <pull_request>

docs/roadmap/plan-check.md

@@ -13,7 +13,13 @@ the issue in the `Plans #<issue>` line of the body. A pull request with no such
 more than one, is a problem.
 
 The GitHub client gives the title, the body, the base and the check runs. The command reads the
-issue that the body plans, for its acceptance criteria. It builds no model client.
+issue that the body plans, for its acceptance criteria and for the text of the item that the size
+measure counts. The command takes the body of the issue with no record block of the jobs, as the
+stage job reads it. It builds no model client.
+
+To measure the groups, the command makes a detached worktree of the head commit in a temporary
+folder and removes it when the measure ends. The files that the tasks name are the files of the head
+commit, whatever the working tree of the checkout holds.
 
 ## What it checks
 
@@ -22,6 +28,15 @@ issue that the body plans, for its acceptance criteria. It builds no model clien
 - The read: the reader reports each change file that it cannot read, each task group that the
   model refuses and each `Verification:` command that the implement step cannot run, as a `read`
   problem. It never ends in a traceback.
+- The size of each task group (#3499): the command measures each group with `measure_groups`, the
+  function of the stage job, on the tracked files of the head commit. The item and the files of the
+  change folder are the shared text, as in the stage job. It reports a `group-over-budget` problem
+  for a group above the setting `roadmap.stage.step_tokens`, and a `group-no-files` problem for a
+  group that changes a file that exists and packs no file, because no code span in its tasks names
+  one. A group that only creates files packs none by nature, and the command accepts it. The plan
+  holds no estimate of the lines of a created file, so the command counts one line for each. For a
+  group that creates files, its figure is lower than the stage figure by 32 tokens for each
+  estimated line above the first. For any other group, the two figures are equal.
 - The title: `docs(openspec): plan #<issue> <summary>`, with at most `SUMMARY_MAX_CHARS`
   characters after the colon.
 - The body: at most `BODY_MAX_CHARS` characters, one `Plans #<issue>` line, and no closing keyword

openspec/changes/archive/2026-11-05-3499-plan-check-measures-each-group/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-11-05

openspec/changes/archive/2026-11-05-3499-plan-check-measures-each-group/design.md

@@ -0,0 +1,25 @@
+# Design
+
+## Decisions
+
+- **Call `measure_groups` from `plan_check` with the groups that `read_change` gives.**: The stage job measures with this function, so one function gives one figure. The test compares the figure of `plan-check` with a call of `measure_groups` on the same plan.
+- **Build the shared text with one function, `shared_texts`, that the stage job and `plan-check` both call.**: The shared text is the title and body of the issue and the files of the change folder. Two copies of that list can drift, and the figures would then differ with no test to show it.
+- **Measure on the head commit of the pull request, in a detached worktree of `checkout_at`.**: `named_files`, `packed_file` and `rules_sections` read a directory, and the working tree of the caller can be at any commit or hold edits. The head commit is the tree that the plan names, and it is the same tree on each run.
+- **Move `checkout_at` to `src/devops_cli/roadmap/clone_git.py` and give it the `operation` of its caller.**: `clone_git.py` already holds `git`. `plan_check.py` must not import an eval module for a git helper, and `stage_eval_run.py` and `stage_verify_eval.py` keep the same behavior.
+- **Add `files` to `MeasuredGroup`.**: The measure already finds the files that the step packs. A second call of `named_files` in `plan_check.py` would repeat that work and could differ from it.
+- **Read the budget with `load_settings().roadmap.stage.step_tokens`, and add no option.**: The issue names this setting. A person who needs another budget sets `DEVOPS_CLI_ROADMAP_STAGE_STEP_TOKENS`.
+- **Report each finding as a problem with the rules `group-over-budget` and `group-no-files`, on the path `tasks.md` of the change folder.**: A problem has a path, a rule and a message, and the output and the exit code already follow from that.
+- **Report `group-no-files` only for a group that changes a file that exists.**: A group whose `changes` are all in `creates` has no file to pack, and the plan is right. The sample plan of `tests/roadmap/test_roadmap_plan_check.py` has such a group, and a valid plan prints no problem.
+- **Keep the one-line estimate of a created file.**: `tasks.md` holds no estimate, and the plan reader gives each created path one line. The figure of `plan-check` is then a lower bound for a group that creates files, and equal to the stage figure for any other group.
+- **Record the size of the first request in `implement.spend`, under `steps`, from the `usage` of the first reply of the step.**: The reply reports the input tokens of the request that it answers, so the first reply gives the size of the first request. `implement.spend` is written apart from `implement.trace`, so the trace limit does not touch it.
+- **Take the first request of the first turn of the first tier only.**: A later turn or tier continues a conversation, and its first request is not the first request of the step.
+
+## Rejected alternatives
+
+- **Measure on the working tree of the checkout that runs `plan-check`.**: The working tree is at any commit and can hold edits, so two runs on one pull request can give two figures. The test of equality with the stage job would then depend on the state of the checkout.
+- **Check out only the named files and the routed documents.**: `rules_sections` reads the routes from `AGENTS.md` and the documents that they name. A second list of paths to extract can differ from what the step reads, and the full checkout of this repository takes about 1 second.
+- **Report every group that packs no file.**: The run over the change folders of `origin/main` found 8 such groups, and 2 of them only create files. A plan with such a group is valid, and the report would give it a false problem.
+- **Add the estimate of created lines to `tasks.md`.**: This changes the `Changes:` line that the renderer, the reader, the path gates and the implement step all read, and plans on the default branch hold the old form. The issue does not name it. The limit is stated in the documents instead.
+- **Add a `--step-tokens` option to `plan-check`.**: The issue asks for the setting of the stage job, and the variable already overrides it.
+- **Put the first request in a `model` event of the trace only.**: `encode` drops the oldest events first, so a long run loses the first event. The record #2636 held 77 events and dropped 154.
+- **Add the figure as a column of the table in the job comment.**: The table shows the totals of a step. The figure is a fact for a person who compares plan and model, and the record holds it.

openspec/changes/archive/2026-11-05-3499-plan-check-measures-each-group/proposal.md

@@ -0,0 +1,51 @@
+# Proposal
+
+## Why
+
+The stage comment gives a token figure for each task group of a plan (#3499). The figure goes stale when the plan or the files that its tasks name change, and no gate measures it again. `devops roadmap plan-check` has no token measure. The implement step refuses a group only above the window minus the turn reserve, so a plan above the stage budget can start a run. The stage figures of four plans were 5150, 5687, 3798 and 31191 tokens. A measure of the same plans now gives about 25778, 12835, 60701 and 43832 tokens. No record holds the size of the first request of a step, so nobody can compare a stage figure with what the model received.
+
+## What Changes
+
+- `plan-check` measures each task group of the plan with `measure_groups`, the function of the stage job, on the tracked files of the pull request head. It reports a `group-over-budget` problem for a group above `roadmap.stage.step_tokens`, and a `group-no-files` problem for a group that changes a file that exists and packs no file. A group that only creates files packs none by nature, and the command accepts it.
+- One function, `shared_texts` in `src/devops_cli/roadmap/stage_change.py`, builds the text that all groups share: the item and the files of the change folder. The stage job and `plan-check` both call it, so their figures stay equal.
+- `MeasuredGroup` gains `files`, the number of files that the step packs for the group, so the stage job and `plan-check` count them in one place.
+- `checkout_at` moves from `src/devops_cli/roadmap/stage_eval_run.py` to `src/devops_cli/roadmap/clone_git.py`, so `plan-check` can read the head commit as a checkout without a dependency on the eval.
+- A step records the input tokens of its first model request. `implement.spend` holds them for each step under `steps`, apart from the trace, so the trace limit never drops them.
+- The change carries a spec delta on `roadmap-plan-check` and on `roadmap-implement-job`, because the command output and the job record are behavior that a consumer observes.
+- The change fits one pull request.
+
+## Capabilities
+
+### New Capabilities
+
+None.
+
+### Modified Capabilities
+
+- `roadmap-plan-check`: Plan-check measures each task group
+- `roadmap-implement-job`: The record keeps the size of the first request of each step
+
+## Impact
+
+- Code: `src/devops_cli/roadmap/plan_check.py`, `src/devops_cli/roadmap/stage_change.py`, `src/devops_cli/roadmap/stage.py`, `src/devops_cli/roadmap/clone_git.py`, `src/devops_cli/roadmap/stage_eval_run.py`, `src/devops_cli/roadmap/stage_verify_eval.py`, `src/devops_cli/roadmap/implement_step.py`, `src/devops_cli/roadmap/implement_run.py`, `src/devops_cli/lang/en/messages.py`, `src/devops_cli/lang/en/help.py`
+- Documents: `docs/roadmap/plan-check.md`, `docs/ROADMAP_WORKFLOW.md`, `docs/commands/roadmap.md`, `docs/CLI_REFERENCE.md`, `README.md`
+- Tests: `tests/roadmap/test_roadmap_plan_check.py`, `tests/roadmap/test_roadmap_stage_eval_run.py`, `tests/roadmap/test_roadmap_implement_step_trace.py`, `tests/roadmap/test_roadmap_implement_job_trace.py`
+
+## Feasibility
+
+- Read of `src/devops_cli/roadmap/plan_check.py`: the module has no call to `measure_groups` and no token count. `grep -rn measure_groups src` finds the callers `stage.py` and `implement_run.py` only. The premise of the issue holds.
+- Read of `src/devops_cli/roadmap/stage_change.py`: `measure_groups(root, groups, shared=...)` counts the shared text, the text of the group, the files and the rules that the step reads under `root`, and 32 tokens for each estimated line of a created file. The stage job calls it with the item and the rendered files of the change folder.
+- Read of `src/devops_cli/roadmap/implement_step.py`: the step refuses inputs above `window - CONST_ROADMAP_IMPLEMENT_TURN_RESERVE_TOKENS`, and no stage budget applies. The statement of the issue holds.
+- Read of `src/devops_cli/roadmap/implement_trace.py`: `encode` drops the oldest events first, so the first model event of a long run is the first to go. A figure of the first request needs a place outside the trace.
+- Command output: a script ran `read_change` and `measure_groups` over the 56 change folders of `origin/main`. 23 folders read, and 65 groups gave 2 groups above 40000 tokens, none above 64000 (the default of `roadmap.stage.step_tokens`), and 8 groups that pack no file. 6 of the 8 change an existing file that no task names in a code span, and 2 only create files.
+- Limit that the plan format sets: `tasks.md` holds no estimate of the lines of a created file. The plan reader gives each created path one line, so the figure of `plan-check` for a group that creates files is lower than the stage figure by 32 tokens for each estimated line above the first. For a group that creates no file the two figures are equal. The proposal keeps the format: a change of the `Changes:` line is a change of the plan format, and the issue does not name it. The docs of the command state the limit.
+- The deployed value of `roadmap.stage.step_tokens` is not in this checkout. The command reads the setting of the machine that runs it, so it uses the value that the deployment sets, 40000 or 64000.
+- `uv run devops ci test tests/roadmap/test_roadmap_plan_check.py` and `uv run devops ci test tests/roadmap/test_roadmap_implement_job_trace.py` run the new tests and fail before the code and pass after it.
+
+## Existing Tool Considered
+
+Project code covers the need: `measure_groups`, `count_tokens`, `named_files`, `packed_file` and `rules_sections` give the figure, and `checkout_at` gives the head commit as a checkout. No outside tool knows the inputs of an implement step. The change reuses all of them and builds no counter. The cost is about 8,000 tokens to build, 1,000 tokens to keep and about 2 hours elapsed. The change deletes the copy of `checkout_at` in `src/devops_cli/roadmap/stage_eval_run.py` by moving it, and deletes no other code.
+
+## Dependencies
+
+The plan reader of #3482 reads a plan with no crash, so `plan-check` has a plan to measure.

openspec/changes/archive/2026-11-05-3499-plan-check-measures-each-group/specs/roadmap-implement-job/spec.md

@@ -0,0 +1,12 @@
+## ADDED Requirements
+
+### Requirement: The record keeps the size of the first request of each step
+The implement job SHALL record in `implement.spend`, under `steps`, the input tokens of the first model request of each step, from the usage of the first reply of the step. The job SHALL write the figure apart from the trace, so the limit of the trace never drops it.
+
+#### Scenario: A step records its first request
+- **WHEN** a step makes two model requests
+- **THEN** `implement.spend` holds for that step the input tokens of the first request, and they equal `tokens_in` of the first `model` event of the step.
+
+#### Scenario: A trace over its limit keeps the figure
+- **WHEN** a run has more events than `CONST_ROADMAP_IMPLEMENT_TRACE_CHARS` allows
+- **THEN** the trace has a `dropped` count above 0, and `implement.spend` still holds the first-request figure of each step.