Skip to content
2 changes: 1 addition & 1 deletion docs/ROADMAP_WORKFLOW.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,7 @@ A stop has a class (`roadmap/failure.py`, #3065). A stop site that knows its cau
The Job runs these actions by itself:
- the fix rounds of `roadmap.implement.repair_rounds` for a failed check,
- `amend` for a request of the kinds `path`, `command` and `task`,
- `return` for a `premise` request,
- `return` for a `premise` request, and for a behavior group that adds a guard which the branch already defines ([the premise of a group](roadmap/implement.md)),
- the split of a group whose inputs exceed the context window.

A plan can order the removal or the rename of a test (#3490). Its `proposal.md` names the old test as `path::name` (`path::Class.test_x` for a test of a class) in the `check` of an entry of `## Weakened checks`, and the entry says what the test stopped and what safeguard takes its place: `- **<check>**: Stopped: <stopped> Safeguard: <safeguard>`. The step guard allows that test. It refuses any other lost test, with the class `tests_lost`. AGENTS.md rule 5 lets only the owner lower a threshold, and only a person merges a pull request, so the Job reads the record from `origin/<default>` and never from its checkout. An entry that the agent writes waives nothing, and a plan that is only on its staged branch records nothing. A step that a person runs with `devops roadmap implement --item <number> --group <group>` reads no record, so it refuses each lost test. The plan reader refuses a line of the section in another form.
Expand Down
2 changes: 2 additions & 0 deletions docs/roadmap/implement.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,8 @@ The job never merges. [Checks of the implement job](../ROADMAP_WORKFLOW.md#check

**A stop.** A step that fails stops the item. The job comments why, shows a table of the steps (the step, its attempts, its tokens in and out, and its last failing command), shows the masked diff of the discarded work, and sets the Status to Blocked. The job keeps the trace of its run in `implement.trace`: each model call, tool call, check and retry, with no argument, no output, no secret and no private host name. Read it with `devops roadmap explain <issue> --trace`. The job keeps no Job artifact. The Service runs the item again after the comment `implement: retry`. For a stop whose class a new attempt can fix, it runs the item again with no comment. This happens after a Job that ended with no result, up to `roadmap.implement.continuations_per_item` times (default 2). It also happens after a new runner image, within the limits of `roadmap.implement.digest_clears_per_class` and `roadmap.implement.digest_clears_per_item`.

**The premise of a behavior group.** Before a behavior group starts, the job checks its premise at the branch tip. It reads each task of the group that has an add verb, the word guard or check, and one name in a code span. When the branch tip defines that name, the job returns the item to New. The comment holds the task and the definition line. The job runs no step, makes no model request and writes nothing. The check uses the same definition lookup as the premise check of the stage job (`src/devops_cli/roadmap/premise.py`). It reads the last commit that is not a `wip:` commit, so a guard that an earlier attempt of the same group kept is no defect. A group of another kind gets no check.

**A plan request.** A stopped step can request a bounded change of its plan in a `plan-request` block. The job judges requests of the kinds `path`, `command` and `task` with deterministic rules. The job commits an accepted amendment, records it in `tasks.md`, and runs the group again. A group whose inputs exceed the limit is split once for each item. A `premise` request returns the item to New with its evidence. Other requests stop the item as a question for a person.

**Claims.** The job leaves out an item that an agent or a person claimed with the label `owner/agent` or `owner/human`. When one claims an item that a running job works on, the job ends it at the next group boundary, with no push and no pull request.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,12 @@
- **Put the shared definition check in the new module `src/devops_cli/roadmap/premise.py`, and import it from the stage probe and the implement job.**: A private import from `stage_probe.py` would couple the implement job to the internals of the stage probe, and a copy would be two checks to keep in step.
- **Read the branch tip of the worktree as the commit of the check.**: The branch tip holds the staged plan and the work of the earlier groups, so the check also catches a guard that an earlier group adds.
- **Detect a task that adds a guard from its text: an add verb, a word such as guard or check, and one backticked identifier.**: The rule is deterministic and needs no model call, so the test asserts it offline with a fixed tasks file.
- **Read the kind of a group from the public `group_kind` of `src/devops_cli/roadmap/plan_read.py`, and run the check for a behavior group only.**: `TaskGroup` carries no kind. The plan reader already infers the kind from a `Kind:` line or the `fails on the base` suffix of the first task, so the implement job reuses that inference. Only a behavior group adds a product guard; a document or test group would return an item for a name it never adds.
- **Read the kind of a group from `TaskGroup.is_behavior` of `src/devops_cli/roadmap/step_files.py`, and run the check for a behavior group only.**: The property applies the rule of the plan reader: a `Kind:` line or the `fails on the base` suffix of the first task. The implement step already uses it. Only a behavior group adds a product guard; a document or test group would return an item for a name it never adds.
- **Return the item to New with the definition line and run no group step.**: CONTEXT.md and the failure table of `docs/ROADMAP_WORKFLOW.md` record that a false premise returns the item to New with its evidence.

## Rejected alternatives

- **Keep the premise check inside the step model and trust it to notice the defect.**: The issue shows the plan of PR #3374 for #3368 passed `check_proposal` and would pass the plan check of #3393, so only a deterministic check at the branch tip catches the duplicate guard.
- **Copy the definition lookup into the implement job and keep the stage probe as it is.**: `src/devops_cli/roadmap/stage_probe.py` already holds `_definition_pattern` and the definition branch of `_Run.identifier`; a copy makes two checks to keep in step, which the issue rejects.
- **Run the premise check for every group, whatever its kind.**: A group whose kind is not behavior adds no product guard, so the check would return an item for a document or test task that names an existing helper.
- **Add a `kind` field to `TaskGroup`, or read the kind with the private `plan_read._kind`.**: The writer drops the kind of a group from `tasks.md`, and the plan reader infers it. A second inference in `step_files.py` would duplicate the first, and `step_files.py` cannot import `stage_change.py` without a cycle. A private import would couple the implement job to the reader internals.
- **Add a second kind inference to the implement job, or read the private `plan_read._kind`.**: `TaskGroup.is_behavior` already gives the kind to the implement job. A second inference would duplicate it, and a private import would couple the implement job to the reader internals.
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,11 @@ The implement job runs the task groups of a Staged plan without a deterministic

- Extract the definition lookup of the stage probe into one shared function in the new module `src/devops_cli/roadmap/premise.py`.
- Call the shared function from `src/devops_cli/roadmap/stage_probe.py` so the stage job and the implement job share one definition check.
- Expose the group-kind inference that the plan reader already uses as `group_kind` in `src/devops_cli/roadmap/plan_read.py`.
- Read the kind of a group from `TaskGroup.is_behavior` in `src/devops_cli/roadmap/step_files.py`, which the implement step already uses.
- Check the premise of each behavior group at the branch tip in `src/devops_cli/roadmap/implement_run.py`, before the group step runs.
- Return the item to New with the definition line when a behavior group task adds a guard that the branch tip defines, and run no step and write no file.
- Add the false-premise and true-premise tests to `tests/roadmap/test_roadmap_implement_job.py`.
- Record the new check on the implement page `docs/roadmap/implement.md`.
- Record the new check on the implement page `docs/roadmap/implement.md` and in `docs/ROADMAP_WORKFLOW.md`.
- Add the shared check and the behavior-group premise check as requirements of the capability `roadmap-implement-job`, so the delta holds the new behavior.
- The change fits one pull request.

Expand All @@ -28,8 +28,8 @@ None.

## Impact

- Code: `src/devops_cli/roadmap/premise.py`, `src/devops_cli/roadmap/stage_probe.py`, `src/devops_cli/roadmap/plan_read.py`, `src/devops_cli/roadmap/implement_run.py`
- Documents: `docs/roadmap/implement.md`
- Code: `src/devops_cli/roadmap/premise.py`, `src/devops_cli/roadmap/stage_probe.py`, `src/devops_cli/roadmap/implement_run.py`, `src/devops_cli/lang/en/messages.py`
- Documents: `docs/roadmap/implement.md`, `docs/ROADMAP_WORKFLOW.md`
- Tests: `tests/roadmap/test_premise.py`, `tests/roadmap/test_roadmap_implement_job.py`

## Feasibility
Expand All @@ -40,8 +40,8 @@ None.
- The probes say of the shared module that this change adds: "The path `src/devops_cli/roadmap/premise.py` is no file of the staging commit. No commit in the history deleted or renamed it." So the module is a path to create, and the premise of the issue holds.
- read_file `src/devops_cli/roadmap/stage_probe.py` showed the definition lookup that the change extracts: `_definition_pattern` builds the expression and `_Run.identifier` runs `git grep -n -E` with it at the staging commit.
- read_file `src/devops_cli/roadmap/implement_run.py` showed the group loop and `_return_premise`, which runs only after the step model reports a premise request, so no deterministic check runs before a group today.
- read_file `src/devops_cli/roadmap/step_files.py` showed that `TaskGroup` holds the tasks and the text of a group, and no kind. So the implement job cannot read the kind from the parsed group.
- read_file `src/devops_cli/roadmap/plan_read.py` showed that `_kind` infers the kind of a group from a `Kind:` line or the `fails on the base` suffix of its first task, and that `_group` already uses the inference. The change exposes it as `group_kind`, so the implement job runs the check for a behavior group only.
- read_file `src/devops_cli/roadmap/step_files.py` showed that `TaskGroup.is_behavior` judges the kind of a group from a `Kind:` line or the `fails on the base` suffix of its first task, the same rule as the plan reader. The implement step already uses it (`implement_step.py`).
- read_file `src/devops_cli/roadmap/plan_read.py` showed that `_kind` applies the same rule for the plan check. So the implement job needs no new inference, and it runs the check for a behavior group only.
- First checks against the real system: `uv run devops ci test tests/roadmap/test_premise.py tests/roadmap/test_roadmap_stage_probe.py`, then `uv run devops ci test tests/roadmap/test_roadmap_implement_job.py`, then `uv run devops ci docs`.

## Existing Tool Considered
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
## ADDED Requirements

### Requirement: The job checks the premise of a behavior group at the branch tip
Before the job runs a behavior group, the job SHALL check each guard or check identifier that a task of the group adds against the branch tip, and SHALL return the item to New with the definition line when the branch tip defines it. The job SHALL read the kind of the group from `group_kind` of `src/devops_cli/roadmap/plan_read.py`.
Before the job runs a behavior group, the job SHALL check each guard or check identifier that a task of the group adds against the branch tip, and SHALL return the item to New with the definition line when the branch tip defines it. The job SHALL read the kind of the group from `TaskGroup.is_behavior` of `src/devops_cli/roadmap/step_files.py`.

#### Scenario: A task adds a guard that the base already runs
- **WHEN** A behavior group task says to add the guard `_delivery_title` and the branch tip defines `_delivery_title`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,25 +4,25 @@

Reads: `src/devops_cli/roadmap/stage_probe.py`, `src/devops_cli/roadmap/refine.py`, `tests/roadmap/test_roadmap_stage_probe.py`. Changes: `src/devops_cli/roadmap/premise.py` (new), `src/devops_cli/roadmap/stage_probe.py`, `tests/roadmap/test_premise.py` (new).

- [ ] 1.1 In `tests/roadmap/test_premise.py`, write `test_the_definition_lines_of_an_identifier_come_from_the_commit`, which runs the shared function on the commit of the checkout and asserts that the definition line of `_delivery_title` names `src/devops_cli/roadmap/implement_run.py`. Run it first and see it fail on the missing module
- [x] 1.1 In `tests/roadmap/test_premise.py`, write `test_the_definition_lines_of_an_identifier_come_from_the_commit`, which runs the shared function on the commit of the checkout and asserts that the definition line of `_delivery_title` names `src/devops_cli/roadmap/implement_run.py`. Run it first and see it fail on the missing module
- Verification: `uv run devops ci test tests/roadmap/test_premise.py tests/roadmap/test_roadmap_stage_probe.py` fails on the base
- [ ] 1.2 Create `src/devops_cli/roadmap/premise.py` with `definition_lines(source, sha, name, git=git_result)`, which runs the extended `git grep` of `_definition_pattern` at `sha` and returns the definition lines, and `defines(source, sha, name)`, which reports whether the list is not empty. Move `_definition_pattern` and the definition branch of `_Run.identifier` of `src/devops_cli/roadmap/stage_probe.py` into the new module. Call `definition_lines` from `_Run.identifier` with `self.git`, so the stage probe keeps its command budget, its messages and its line limit
- [x] 1.2 Create `src/devops_cli/roadmap/premise.py` with `definition_lines(source, sha, name, git=git_result)`, which runs the extended `git grep` of `_definition_pattern` at `sha` and returns the definition lines, and `defines(source, sha, name)`, which reports whether the list is not empty. Move `_definition_pattern` and the definition branch of `_Run.identifier` of `src/devops_cli/roadmap/stage_probe.py` into the new module. Call `definition_lines` from `_Run.identifier` with `self.git`, so the stage probe keeps its command budget, its messages and its line limit
- Verification: `uv run devops ci test tests/roadmap/test_premise.py tests/roadmap/test_roadmap_stage_probe.py` passes

## 2. Check the premise of a behavior group before its step

Reads: `src/devops_cli/roadmap/implement_run.py`, `src/devops_cli/roadmap/premise.py`, `src/devops_cli/roadmap/plan_read.py`, `src/devops_cli/roadmap/step_files.py`, `tests/roadmap/test_roadmap_implement_job.py`, `tests/support/roadmap_implement.py`. Changes: `src/devops_cli/roadmap/implement_run.py`, `src/devops_cli/roadmap/plan_read.py`, `tests/roadmap/test_roadmap_implement_job.py`.
Reads: `src/devops_cli/roadmap/implement_run.py`, `src/devops_cli/roadmap/premise.py`, `src/devops_cli/roadmap/step_files.py`, `tests/roadmap/test_roadmap_implement_job.py`, `tests/support/roadmap_implement.py`. Changes: `src/devops_cli/roadmap/implement_run.py`, `src/devops_cli/roadmap/premise.py`, `src/devops_cli/lang/en/messages.py`, `tests/roadmap/test_premise.py`, `tests/roadmap/test_roadmap_implement_job.py`.

- [ ] 2.1 In `tests/roadmap/test_roadmap_implement_job.py`, add `test_a_behavior_group_that_adds_a_guard_the_base_already_runs_returns_the_item`, whose tasks file has a behavior group (`Kind: behavior.`) that says to add the guard `_delivery_title`, and assert the item is New with the definition line in the comment, that no step ran and that the worktree holds no change. Add `test_a_behavior_group_whose_guard_the_base_lacks_runs`, which asserts the group runs and commits. Run both first and see the first fail
- [x] 2.1 In `tests/roadmap/test_roadmap_implement_job.py`, add `test_a_behavior_group_that_adds_a_guard_the_base_already_runs_returns_the_item`, whose tasks file has a behavior group (a failing test, then a task that adds the guard `_delivery_title`), and assert the item is New with the definition line in the comment, that no model request ran and that the branch tip is as it was. Add `test_a_behavior_group_whose_guard_the_base_lacks_runs`, which asserts the group runs and commits, and `test_a_group_that_is_not_behavior_runs_though_the_base_defines_the_name`. Add the table test of `added_guard` to `tests/roadmap/test_premise.py`. Run them first and see the first fail
- Verification: `uv run devops ci test tests/roadmap/test_roadmap_implement_job.py` fails on the base
- [ ] 2.2 Expose `group_kind(group)` in `src/devops_cli/roadmap/plan_read.py`, the inference that `_group` already uses. Add `guard_names(group)` to `src/devops_cli/roadmap/implement_run.py`, which reads a task of an add verb, a word such as guard or check, and one backticked identifier. Before a behavior group starts, call `defines` of `src/devops_cli/roadmap/premise.py` at `self._head()` for each name, and take `_return_premise` with a `premise` request whose evidence is the definition line when one is found. A group of another kind runs no check
- [x] 2.2 Add `added_guard(task_text)` and `false_premises(source, sha, group)` to `src/devops_cli/roadmap/premise.py`. `added_guard` reads a task of an add verb, then the word guard or check directly followed by one identifier in a code span; an identifier after a preposition names code that the task changes. `false_premises` calls `definition_lines` for each such name. In `src/devops_cli/roadmap/implement_run.py`, call `false_premises` before the step of a group for which `TaskGroup.is_behavior` holds, at the last commit that is no `wip:` commit, and return the item to New with the definition lines when there are any. A group of another kind runs no check
- Verification: `uv run devops ci test tests/roadmap/test_roadmap_implement_job.py` passes

## 3. Record the new premise check on the implement page

Reads: `docs/roadmap/implement.md`, `docs/ROADMAP_WORKFLOW.md`, `CONTEXT.md`. Changes: `docs/roadmap/implement.md`.
Reads: `docs/roadmap/implement.md`, `docs/ROADMAP_WORKFLOW.md`, `CONTEXT.md`. Changes: `docs/roadmap/implement.md`, `docs/ROADMAP_WORKFLOW.md`.

- [ ] 3.1 Add a paragraph to the page `docs/roadmap/implement.md` that says the job checks the premise of a behavior group at the branch tip before the group starts, that a definition of a guard the task adds returns the item to New with the definition line, and that the job writes nothing in that case
- [x] 3.1 Add a paragraph to the page `docs/roadmap/implement.md` that says the job checks the premise of a behavior group at the branch tip before the group starts, that a definition of a guard the task adds returns the item to New with the definition line, and that the job writes nothing in that case
- Verification: `uv run devops ci docs` passes

## Workflow follow-up
Expand Down
Loading
Loading