Repository navigation
fix(hardening): defect PR #3669 missing declared surface hardening #3675
Description
Unhardened Defect PR Audit Finding
Defect Pull Request #3669 was merged without adding guardrails or lessons to any declared hardening surface.
Declared Hardening Surfaces
docs/TESTING.mdARCHITECTURE.mdSECURITY.mdCONTRIBUTING.mdsrc/devops_cli/ai/tasks/*.mdsrc/devops_cli/ai/knowledge_base/.devops/review.tomlsrc/devops_cli/review/rules/
Per project invariants (AGENTS.md), defect fixes must codify root-cause guardrails into at least one declared surface to prevent recurrent regressions.
Proposed design
Problem statement
Pull request #3669 fixed a defect but added no guardrail to a declared hardening surface. The defect can return with no check to stop it. This change adds the missing guardrail to one declared surface. It also makes the hardening audit report the pull request as hardened.
Acceptance criteria
Requirement: The declared surface records the defect of pull request 3669
The change SHALL record the root cause of pull request #3669 in at least one hardening surface that the issue declares.
Scenario: A declared surface cites the defect pull request
- WHEN A reader searches the declared surfaces for the text
3669 - THEN One declared surface names pull request fix(roadmap): plan-check measures each task group #3669 and the defect class that it fixed.
Requirement: The guardrail fails when the defect returns
The guardrail MUST fail a check when the defect that pull request #3669 fixed returns.
Scenario: A person reverts the fix
- WHEN A person reverts the code change of pull request fix(roadmap): plan-check measures each task group #3669 in a worktree
- THEN The guardrail test fails, and its message names the defect.
Requirement: The hardening audit reports the pull request as hardened
The hardening audit SHALL report the pull request that delivers the change as hardened.
Scenario: The audit reads the pull request
- WHEN The hardening audit runs on the pull request that delivers the change
- THEN The job summary reports the pull request as hardened and names the guardrail's surface.
Existing Tool Considered
No check has run for this issue: the alternatives job recorded none. No open, maintained tool or library covers a project-specific root-cause guardrail, and the repository already holds the hardening audit and the surface list. The change builds one regression test and one document line. Written cost: about 20,000 tokens to build, about 5,000 tokens each year to keep and fix, and about 3 hours of elapsed time.
Key questions
- Which defect did pull request fix(roadmap): plan-check measures each task group #3669 fix, and which behaviour must the guardrail pin?: The issue body names no defect. The body, the diff and the files of pull request fix(roadmap): plan-check measures each task group #3669 hold it, and AGENTS.md rule 11 puts the guardrail at the cause. Refine reads them and names the defect class in the test docstring. [sources: AGENTS.md]
- Which declared hardening surface carries the guardrail?: A regression test in the test file of the behaviour that the defect broke, in the mirror directory under
tests/. (best practice) [sources: docs/TESTING.md, openspec/specs/pr-hardening-audit/spec.md, AGENTS.md]- Chosen: A regression test in the mirror test file of the behaviour. docs/TESTING.md puts a regression test in the file of the behaviour it pins, and the pr-hardening-audit spec counts a test as a surface. The test fails when the defect returns.
- Rejected: A requirement in the living spec of the capability. A spec requirement states the rule but runs no check, so the defect can return with no failure.
- Rejected: A rule in
src/devops_cli/review/rules/. A Semgrep rule fits only a defect that a pattern expresses, and the defect of fix(roadmap): plan-check measures each task group #3669 is not stated. - Rejected: A suppression or note in
.devops/review.toml. That file holds suppressions and severity caps. It silences a finding and prevents no defect. - Rejected: A section in
docs/TESTING.mdorARCHITECTURE.md. Prose drifts, and no check fails when the defect returns.
- Does the change cite a fail-to-pass run?: No. The fix of the defect is on
main, so the guardrail passes on the base, and a fix that changes only tests and documents gets no fail-to-pass check. (best practice) [sources: docs/TESTING.md, CONTRIBUTING.md]- Chosen: Cite no fail-to-pass run. The check needs a changed test that fails on the base. The defect is already fixed, so the guardrail passes there.
- Rejected: Cite a fail-to-pass run. The check fails the pull request when no changed test fails on the base.
- Does the hardening audit count a change under
tests/as a hardening surface?: Yes. The pr-hardening-audit spec states that a test counts as a hardening surface, and that a living spec counts as one. [sources: openspec/specs/pr-hardening-audit/spec.md]
Fit
Fits in one pull request.
Suspected block
None.
Dependencies
None.
Activity
devops-cli-service commentedon Nov 5, 2026
Intake placed this item to the backlog: intake puts every new item in the backlog. A person plans the milestones.
fixprefix of the title gives the type.hardening, and .github/labels.yml declares no scope/hardening label. Add the label to the file to give the scope to this item.A process-compliance gap that risks recurrent regressions but does not break a user-facing feature.Codifying guardrails prevents the same class of regression from recurring across the project.The work is to add guardrail text to one or more existing declared surfaces for a single PR.Both follow the same hardening-audit template for a defect PR, but target different PRs (#3669 vs #3663) with different root causes.devops-cli-service commentedon Nov 7, 2026
Refine cannot set this item to Ready at b6af763. A bot opened the item, so a review of the issue and of the proposal must approve it first. The review found the defects below. A person edits the body to answer them. The next refine round looks at the item again only after a body edit.
false_premiseat key_questions, question 4: The plan claims the pr-hardening-audit spec counts a test as a hardening surface. No file in the input shows this, and the issue lists notests/path as a declared surface. Evidence: "The pr-hardening-audit spec states that a test counts as a hardening surface, and that a living spec counts as one."dan-petty commentedon Nov 11, 2026
Closed as not planned. The check that filed this item passes now.
uv run devops pr hardening-report --pr 3669is_hardened: true. The hardened surfaces aretests/andopenspec/specs/.#3684 (merged 2026-11-05) counts tests and specs as hardening guardrails. Thus PR #3669 has the guardrail that the audit did not find. #3661 closes this family of items under the issue stewards plan #4780 (AGENTS.md rule 2). If the check fails again, reopen this item.