Skip to content

Report Security reproduction failures separately from completed audits - #604

Merged
JacobStephens2 merged 1 commit into
mainfrom
issue-603
Oct 9, 2026
Merged

JacobStephens2 merged 1 commit into
mainfrom
issue-603

Conversation

@JacobStephens2

@JacobStephens2 JacobStephens2 commented Oct 9, 2026 •

Copy link
Copy Markdown
Owner

Summary

Preserve the completed Security audit when a later reproduction fails, and identify the failed reproduction in the Run notification.

- Result:       audit failed
- Audit:        audit failed
+ Result:       reproduction failed
+ Audit:        audit complete
+ Reproduction: 1 refused

Other reproduction failures report their number as failed; safeguard refusals retain the safe cause and Session log and Command log pointers. Genuine audit failures and successful Security runs retain their existing reporting.

Closes #603

Evidence

  • Before: cargo test --test security_run refused_security_reproductions_keep_the_record_and_stop_before_the_next_finding -- --exact failed on the original audit failed subject.
    After: The same regression passes for Claude Code and Codex, on public and private repositories, with fixing allowed: all private records survive, no next reproduction or fix starts, and private diagnostics stay out of the notification.
  • Reproduction 2 coverage includes an incomplete result and a nonzero Harness exit, retaining reproduction 1's high severity and stopping before reproduction 3.
  • cargo test: 1662 passed, 0 failed, 1 ignored across 41 test binaries.
  • cargo test --test security_run: 83 passed.
  • cargo check --all-targets, cargo clippy --all-targets -- -D warnings, and cargo fmt --check: passed.

Test seams

The existing Security run scenario harness: command exit status and progress, captured Run notifications, and private records exposed by the GitHub stand-in. Fake Harness CLIs and Resend are external adapters.

Review

Standards: 0 findings. Spec: 0 findings. Both reviewers read all four changed files; no changed files were left unread.

Security: 0 introduced findings. Guidance-mode review covered 7f13fc162a1b1ad47c5fe862ddb01cb1f36b3617...7d3b03865d4d87e28253337e0c9cc5d9f97b2bc1, all four changed files and supporting failure, reproduction, private-record and notification code. Relevant attack-class guidance and ADR 0015 were read. Sandboxed offline validation passed: 83 Security run tests and 47 unit tests across notification, run_ending and security.

Unaddressed findings

  • Standards: None.
  • Spec: None.
  • Security: None.

Security review outcome

No unaddressed introduced findings.

Merge Danger

Door: two-way

Reporting changes only; no migration or changes to reproduction or fix dispatch rules.

Blast Radius: notifications

Built with codex · gpt-6.1-sol · high

@JacobStephens2
JacobStephens2 merged commit d2f2faf into main Oct 9, 2026
17 checks passed
@JacobStephens2
JacobStephens2 deleted the issue-603 branch October 9, 2026 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Report reproduction failures separately from completed Security audits

1 participant