Security runs and the Security review: thirdshift finds, reproduces and fixes vulnerabilities

[JacobStephens2]

Summary

Adds Fencing with thirdshift secure and opt-in guidance Security reviews. Findings are recorded privately and reproduced before an allowed fix becomes public work.

Security run
  gates: shared Pass lock → Ready issue → failed fix still open
  work: most severe reproduced fix, else wait/unchanged skip, else audit
  audit: origin/Base branch → quick audit → private records → sequential reproductions
  allowed fix: terse Ticket or Spec → dispatched Run or Spec run

Delivery (Run, Base fix, or completed Spec PR; excludes Ticket Runs)
  opening session → optional guidance Security review → push → Repair/CI → ready or Self-merge
  introduced finding left unfixed / refused or incomplete review → hold Self-merge
  reproduced old finding → private record for a later Security run

• Public repositories use draft advisories; private repositories use security-finding issues. Records match fingerprints across states and preserve the Day shift's grades. Failed fixes keep their Claim and pause Fencing until closed.
• Command words and [security] settings control review, fixing and Harness choice; both features default off. Setup asks about review and fixing. Codex security sessions retain refusal checks, answering-Model logs and the raised thread cap; guidance reviews stay in one session, including Resume.
• Security runs share Pass locking, Activity and Command logs, and dispatch notifications. Notifications include severity, title and private link; no private write-up. Undecided fixing is offered explicitly.
• Embeds the MIT-licensed Cloudflare audit skill at c1c8a8c, including its Node validators. Adds README setup and cron examples, generated prompts and the Prompts and skills page. The graded trial records the Day shift's guidance-mode choice and its detection limits.

Closes #427

Evidence

• Before: cargo test --test security_review guidance_security_review_on_codex_does_not_request_delegated_auditors -- --exact --nocapture failed: Codex appended a delegation instruction to the single-session guidance review. After: passes; covers the initial session and Resume. The existing full-audit regression also passes with fresh sub-agents on both launches.
• Before: cargo test --test security_review reproduced_old_review_finding_sets_the_private_advisory_severity -- --exact --nocapture failed with null instead of low. After: passes; new reproduced review records use the existing private-record update rules to write the scored severity and proof-of-concept.
• Validation: cargo nextest run --test-threads 8: 1,632 passed, one existing benchmark skipped. cargo test also passed. Node validators: 65 passed. cargo fmt --check and cargo clippy --all-targets -- -D warnings pass. Generated-page checks pass in the Rust suite.

Test seams

• The end-to-end scenario harness: the real binary and git, fake GitHub and Harness CLIs; observable command results, prompts, records, delivery, logs and notifications.
• The Pass seam's in-memory double: gate order, work selection and waiting permutations.

Unaddressed findings

Standards

None.

Spec

Inherited informational advisory grade decision: #567, originally raised by PR #566, remains open with only needs-triage. The Spec asks for the rubric grade and proof-of-concept on the record, while the public advisory severity field cannot carry informational. src/github/advisories.rs:500–506 deliberately fails before writing an unsupported grade; private finding issues retain it. The Day shift must choose the representation, which the Spec has not settled.

Focused runs both pass and exercise this exact boundary: cargo test --test security_run an_informational_advisory_requires_a_day_shift_decision_without_an_invented_grade -- --exact verifies the deliberate public failure and unchanged evidence; cargo test --test security_run a_private_reproduction_accepts_the_rubrics_informational_severity -- --exact verifies private preservation. These runs document the deferred policy; they do not claim the public requirement is complete.

Review coverage

Review fixed point: main at 3a81a11; original Spec head: 58bb0de. Both axes read all 106 changed files, including generated files, imported skill sources and tests. No changed file was left unread. Reports with each axis's final files-read list are saved as .thirdshift-review-igwNvs/standards.md and .thirdshift-review-igwNvs/spec.md.

Merge Danger

Door: two-way

The code and opt-in settings can be reverted. Enabled Security runs create private GitHub records; explicitly allowed fixing can publish issues and push code, disclosing the fixed vulnerability as described in ADR 0015.

Blast Radius: factory

Touches shared Pass, Harness and Delivery paths used by Runs, Spec runs, Base fixes and scheduled Passes. Security review and fixing stay off by default; the whole-repository audit and guidance review have separate execution policies.

Tickets

• Prefactor: the Pass lock and its skip line name any Pass #536 done
• The security-audit skill ships as a Factory skill #537 done
• thirdshift secure audits the Base branch and records each finding as a draft advisory #538 done
• The Security run's Run notification #539 done
• Private repositories: findings as security-finding issues #540 done
• Audits wait for an unchanged Base branch and for findings waiting on the Day shift #541 done
• Reproducing each finding #542 done
• Fixing a reproduced finding with a single Ticket #543 done
• A failed fix pauses Security runs; the notification offers fixing; Fencing is documented #544 landed with Pause Security after failed fixes and offer fixing #582
• Bigger fixes as a Spec with Tickets, and a private finding's own issue as its fix #545 done
• Choosing the Security run's Harness, with Codex adjustments #546 done
• Safeguard refusals fail a Security run, and each security session's Model is logged #547 done
• Setup asks about fixing #548 done
• Trial: the Security review's two modes #549 done
• The Security review in a Run #550 landed with Add opt-in guidance Security reviews to Runs #592
• The Security review on a Spec PR #551 landed with Run guidance Security review once on Spec PRs #593
• Old vulnerabilities from a Security review go to the private record #552 landed with Record old Security review findings privately #594

Built with codex · gpt-6.1-sol · xhigh