Skip to content

The Security review in a Run #550

Closed
Closed

Description

@JacobStephens2

This was generated by AI during triage.

Parent

#427

What to build

Mode decision (2026-10-09)

Jacob chose guidance for the optional Security review after the #549 trial was graded. Use one session to read the relevant security attack-class guidance and review the change with supporting code; do not run the full six-phase audit or delegate auditors for this review. The separate whole-repository Security audit keeps its full-audit workflow. Existing proof-of-concept, private-record, and Self-merge requirements still apply.

Evidence and limits: trial grading and decision. Guidance was cheaper and faster in this sample; detection accuracy and equivalence between modes remain unmeasured. Detailed grades remain private.

Turning it on. The Security review is off by default. It's turned on by security-review on the command, which is passed on to the Runs a command starts, or by review under [security] in the User config. no-security-review forbids it. Setup asks about it.

When it runs. When it's on, Delivery runs the Security review session after the opening session (the implement session, or a Continuation's) and before the push and the Repair loop. A Base fix gets one too.

What the session does. In the chosen mode, it:

  • fixes only what it can show with a failing test, kept as a regression test;
  • lists the unaddressed findings the change introduced under Security, in the pull request's Unaddressed findings;
  • ends on a final line that counts them.

What thirdshift does with the result. If any such finding is left, or the session was refused or cut short, a Merge run doesn't Self-merge. It ends a Failed run whose cause names them, with its pull request left ready for review, as after a Policy refusal. A Run that isn't a Merge run lists the findings and goes on.

Acceptance criteria

  • Off by default, and on with the word or the setting. The word overrides the setting, and Setup asks about it.
  • The session runs after the opening session and before the Repair loop, and a Base fix runs one.
  • A Merge run whose review leaves an introduced finding unaddressed, or is refused, ends a Failed run with its pull request ready, and the cause names why.
  • A Run that isn't a Merge run lists the findings under Security in Unaddressed findings.
  • The Session prompt is added to the Prompts and skills page.

Activity

  1. JacobStephens2 commented on Oct 9, 2026

    @JacobStephens2
    OwnerAuthor

    This was generated by AI during triage.

    Agent Brief

    Category: enhancement
    Summary: Implement the opt-in Security review in guidance mode for Runs and Base fixes.

    Current behavior: The trial and grading are complete. Jacob's mode decision resolves this Ticket's human gate; the behavior below still needs implementation.

    Desired behavior: Run one guidance review after the opening session and before pushing and the Repair loop. Apply the relevant attack classes to the diff and supporting code without delegated auditors or the full audit workflow. Retain this Ticket's command/config precedence, Setup behavior, proof-of-concept requirement, result final line, and Self-merge hold.

    Key interfaces: Delivery opening-to-push ordering; command and User config review selection; Security review prompt and result interpretation; generated Prompts and skills documentation.

    Acceptance criteria:

    • All acceptance criteria in the Ticket body are met.
    • Both the prompt and execution use the chosen guidance mode, without the full audit workflow or delegated auditors.
    • A refused or incomplete review cannot establish success for a Merge run.
    • The separate Security audit retains its full-audit workflow.

    Out of scope: Changing the chosen mode, adding mode selection UI, changing trial measurements, and implementing the Spec-only or private old-finding behavior assigned to #551 and #552.

  2. added
    enhancementNew feature or request
    ready-for-agentFully specified, ready for an AFK agent
    and removed
    needs-triageMaintainer needs to evaluate this issue
    on Oct 9, 2026
  3. JacobStephens2 commented on Oct 9, 2026

    @JacobStephens2
    OwnerAuthor

    Closed by #592, merged into issue-427 by a thirdshift Merge run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestready-for-agentFully specified, ready for an AFK agent

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions