Reproduce recorded Security findings

README.md

@@ -743,7 +743,7 @@ thirdshift secure base main harness claude
 thirdshift secure base main harness codex model gpt-6.1-sol effort high
 ```
 
-The command takes the Harness, Model and Effort words and the shared Pass words (`merge`, `no-merge`, `base-fix`, `no-base-fix`, `parallel`, `email` and `no-email`, with or without dashes). It reproduces and fixes nothing, dispatches no fix, and never commits, pushes, publishes or closes an advisory; the fix-related options are reserved for later Security run work.
+The command takes the Harness, Model and Effort words and the shared Pass words (`merge`, `no-merge`, `base-fix`, `no-base-fix`, `parallel`, `email` and `no-email`, with or without dashes). After recording findings, it tries to reproduce each untriaged one in a fresh throwaway worktree at the recorded audited commit, and keeps the outcome, severity when reproduced, notes and test in the private record. It preserves findings already triaged by the Day shift. It dispatches no fix and never commits, pushes, publishes or closes an advisory; the fix-related options are reserved for later Security run work.
 
 `email`, optionally followed by an address, or `email.always` in the User config asks for one **Run notification** when the Security run ends, whether the audit succeeded, failed or was interrupted. `no-email` overrides the default. The notification says how the audit ended and lists each finding it recorded or matched to an existing private record: its severity when known, title and private link. It includes no finding write-up, trace or evidence, since it passes through Resend. Detailed failure causes stay in the local logs; the notification gives the audit's status and log paths. A recording failure still lists the records reached before it failed. A skipped Security run sends none. The usual address and Resend API key checks run before the skip checks or any work; a failed send is a warning and never changes the run's outcome.
 

prompts/README.md

@@ -26,6 +26,7 @@ With `harness codex`, each session runs `codex exec --json --dangerously-bypass-
 | Spec review | In a Spec run, starts the Spec review once every Ticket has landed on the Spec branch, before the Spec PR, a draft until then, is marked ready. | [spec-review.md](spec-review.md) |
 | Architecture review | Starts the Architecture review, the session an Architect run opens with, in a worktree at the head of the Base branch. The line naming the focus is left out when the command gives none. | [architecture-review.md](architecture-review.md) |
 | Security audit | Starts the report-only Security audit in a throwaway worktree at origin's Base branch head. The threat-model line is included when a conventional document exists; artifacts stay under the repository's audit root. | [security-audit.md](security-audit.md) |
+| Security reproduction | After a Security audit, tries to reproduce one recorded finding in a fresh throwaway worktree at its audited commit. The test is copied into the private record only after a complete outcome. | [security-reproduction.md](security-reproduction.md) |
 | Conflict Repair | Starts a Repair session when merging the Base branch into the Issue branch leaves conflicts. | [conflict-repair.md](conflict-repair.md) |
 | Conflict Repair, on Foreign commits | In a Merge run, starts a Repair session when merging Foreign commits from the Issue branch on origin into the local one leaves conflicts. | [foreign-conflict-repair.md](foreign-conflict-repair.md) |
 | Review Repair | In a Merge run, starts a Repair session once Foreign commits are merged into the Issue branch, to review them from the head the Run last knew as its own before they can be merged. | [review-repair.md](review-repair.md) |

prompts/security-reproduction.md

@@ -0,0 +1,21 @@
+<!-- Generated by src/prompts_page.rs from the prompts in src/prompt.rs and their titles and when sentences in src/prompts_page.rs; don't edit. Regenerate with UPDATE_PROMPTS=1 cargo test prompts_page -->
+
+# Security reproduction
+
+After a Security audit, tries to reproduce one recorded finding in a fresh throwaway worktree at its audited commit. The test is copied into the private record only after a complete outcome.
+
+```
+Reproduce this recorded Security finding at audited commit `<audited commit>`.
+Read the `thirdshift-security-audit` skill's likelihood-and-impact severity rubric.
+Write a proof-of-concept test from the finding's validation plan, then run it against the untouched code in this throwaway worktree. Use harmless payloads only, never a deployed site or a real third-party service. Do not fix or change repository source.
+If the test reproduces the finding, score its severity with the skill's rubric and judge whether one session can hold the fix (single) or it needs a Spec (spec).
+Test file: `<test file>`.
+Write the test's full text to this file, even when not reproduced. In your final message, give reproduction notes: the exact command, its result, and, when reproduced, likelihood, impact and the fix-size reasoning.
+End your final message with exactly `Security reproduction: reproduced <severity> <size>`, where severity is critical, high, medium, low or informational and size is single or spec, or `Security reproduction: not reproduced`.
+This session commits, pushes, opens and publishes nothing.
+
+Recorded finding:
+<recorded Security finding>
+
+You run headless: nobody is watching, and ending your turn ends the session. Run tests and other long commands in the foreground, raising the command's timeout if needed. If a command is moved to the background, wait for that task by its own task id or output file, never by process names or patterns (`pgrep`, `ps | grep`, and the like): other sessions on this machine run the same commands. Never end your turn while a background task you depend on is still running: ending the turn kills it. Before ending your turn, stop every background task you no longer need, by its task id (with the `TaskStop` tool, if you have it): a task still running when your turn ends is taken as work you were waiting on.
+```

site/prompts/index.html

@@ -174,6 +174,25 @@ <h3 id="prompt-security-audit-title">Security audit</h3>
 Write the skill's report artifacts and run-metadata.json; mark run_status complete only when all required artifacts are written. Run both skill validators before finishing.
 End your final message with exactly `Security audit: complete` after writing valid artifacts, or `Security audit: incomplete` when incomplete.
 
+You run headless: nobody is watching, and ending your turn ends the session. Run tests and other long commands in the foreground, raising the command's timeout if needed. If a command is moved to the background, wait for that task by its own task id or output file, never by process names or patterns (`pgrep`, `ps | grep`, and the like): other sessions on this machine run the same commands. Never end your turn while a background task you depend on is still running: ending the turn kills it. Before ending your turn, stop every background task you no longer need, by its task id (with the `TaskStop` tool, if you have it): a task still running when your turn ends is taken as work you were waiting on.
+</code></pre>
+      </article>
+      <article class="job-sheet" id="prompt-security-reproduction" aria-labelledby="prompt-security-reproduction-title">
+        <h3 id="prompt-security-reproduction-title">Security reproduction</h3>
+        <p>After a Security audit, tries to reproduce one recorded finding in a fresh throwaway worktree at its audited commit. The test is copied into the private record only after a complete outcome.</p>
+        <p class="sent-by">Sent by a Security run, before any unit</p>
+        <pre class="job-text"><code>Reproduce this recorded Security finding at audited commit `&lt;audited commit&gt;`.
+Read the `thirdshift-security-audit` skill's likelihood-and-impact severity rubric.
+Write a proof-of-concept test from the finding's validation plan, then run it against the untouched code in this throwaway worktree. Use harmless payloads only, never a deployed site or a real third-party service. Do not fix or change repository source.
+If the test reproduces the finding, score its severity with the skill's rubric and judge whether one session can hold the fix (single) or it needs a Spec (spec).
+Test file: `&lt;test file&gt;`.
+Write the test's full text to this file, even when not reproduced. In your final message, give reproduction notes: the exact command, its result, and, when reproduced, likelihood, impact and the fix-size reasoning.
+End your final message with exactly `Security reproduction: reproduced &lt;severity&gt; &lt;size&gt;`, where severity is critical, high, medium, low or informational and size is single or spec, or `Security reproduction: not reproduced`.
+This session commits, pushes, opens and publishes nothing.
+
+Recorded finding:
+&lt;recorded Security finding&gt;
+
 You run headless: nobody is watching, and ending your turn ends the session. Run tests and other long commands in the foreground, raising the command's timeout if needed. If a command is moved to the background, wait for that task by its own task id or output file, never by process names or patterns (`pgrep`, `ps | grep`, and the like): other sessions on this machine run the same commands. Never end your turn while a background task you depend on is still running: ending the turn kills it. Before ending your turn, stop every background task you no longer need, by its task id (with the `TaskStop` tool, if you have it): a task still running when your turn ends is taken as work you were waiting on.
 </code></pre>
       </article>

src/github.rs

@@ -13,7 +13,7 @@ use crate::labels::{Label, Labels};
 use crate::process::{self, Control, Interruption};
 
 mod advisories;
-pub use advisories::{DraftAdvisory, Package, SecurityRecords};
+pub use advisories::{DraftAdvisory, Package, SecurityRecord, SecurityRecords};
 
 #[cfg(test)]
 mod execution_tests;

src/github/advisories.rs

@@ -8,6 +8,7 @@ use serde_json::{Value, json};
 use super::GitHub;
 use crate::issue::IssueUrl;
 use crate::labels::{Label, NEEDS_TRIAGE};
+use crate::security::reproduction::{Reproduction, Severity};
 
 const SECURITY_FINDING: Label = Label::new(
     "security-finding",
@@ -41,12 +42,95 @@ impl SecurityRecords {
         }
     }
 
-    pub fn remember(&mut self, record: Value) -> &Value {
-        let records = match self {
-            Self::Advisories(records) | Self::Issues(records) => records,
+    pub fn remember(&mut self, record: Value) {
+        match self {
+            Self::Advisories(records) | Self::Issues(records) => records.push(record),
+        }
+    }
+
+    pub fn finding(&self, draft: &DraftAdvisory) -> Option<&Value> {
+        let (records, field) = match self {
+            Self::Advisories(records) => (records, "description"),
+            Self::Issues(records) => (records, "body"),
         };
-        records.push(record);
-        records.last().expect("the new record was just added")
+        let marker = format!("Fingerprint: `{}`", draft.fingerprint);
+        records.iter().find(|record| {
+            record[field]
+                .as_str()
+                .is_some_and(|text| text.lines().any(|line| line == marker))
+        })
+    }
+
+    pub fn record(&self, value: &Value) -> Result<SecurityRecord> {
+        Ok(match self {
+            Self::Advisories(_) => SecurityRecord::Advisory {
+                id: value["ghsa_id"]
+                    .as_str()
+                    .context("Security finding record has no advisory ID")?
+                    .to_string(),
+                description: value["description"]
+                    .as_str()
+                    .context("Security finding record has no description")?
+                    .to_string(),
+                untriaged: value["state"] == "draft" && value["severity"].is_null(),
+            },
+            Self::Issues(_) => SecurityRecord::Issue {
+                number: value["number"]
+                    .as_u64()
+                    .context("Security finding record has no issue number")?,
+                description: value["body"]
+                    .as_str()
+                    .context("Security finding record has no body")?
+                    .to_string(),
+                untriaged: value["state"]
+                    .as_str()
+                    .is_some_and(|state| state.eq_ignore_ascii_case("open"))
+                    && value["labels"].as_array().is_some_and(|labels| {
+                        labels.iter().any(|label| {
+                            label["name"]
+                                .as_str()
+                                .is_some_and(|name| NEEDS_TRIAGE.is_named(name))
+                        })
+                    }),
+            },
+        })
+    }
+}
+
+/// A finding read from the private storage selected for this repository.
+pub enum SecurityRecord {
+    Advisory {
+        id: String,
+        description: String,
+        untriaged: bool,
+    },
+    Issue {
+        number: u64,
+        description: String,
+        untriaged: bool,
+    },
+}
+
+impl SecurityRecord {
+    /// A Day-shift decision is the finding's grade; a repeated fingerprint
+    /// must not publish new proof-of-concept evidence or replace that grade.
+    pub fn untriaged(&self) -> bool {
+        match self {
+            Self::Advisory { untriaged, .. } | Self::Issue { untriaged, .. } => *untriaged,
+        }
+    }
+
+    pub fn description(&self) -> &str {
+        match self {
+            Self::Advisory { description, .. } | Self::Issue { description, .. } => description,
+        }
+    }
+
+    pub fn name(&self) -> String {
+        match self {
+            Self::Advisory { id, .. } => id.clone(),
+            Self::Issue { number, .. } => format!("issue #{number}"),
+        }
     }
 }
 
@@ -64,21 +148,6 @@ pub struct DraftAdvisory {
     pub package: Package,
 }
 
-impl DraftAdvisory {
-    pub fn recorded_in<'a>(&self, records: &'a SecurityRecords) -> Option<&'a Value> {
-        let marker = format!("Fingerprint: `{}`", self.fingerprint);
-        let (records, field) = match records {
-            SecurityRecords::Advisories(records) => (records, "description"),
-            SecurityRecords::Issues(records) => (records, "body"),
-        };
-        records.iter().find(|record| {
-            record[field]
-                .as_str()
-                .is_some_and(|description| description.lines().any(|line| line == marker))
-        })
-    }
-}
-
 impl GitHub {
     /// Every state and every page. A 404 selects issues only when repository
     /// metadata confirms they will be private.
@@ -145,18 +214,85 @@ impl GitHub {
                 }
                 let url = std::str::from_utf8(&output.stdout)
                     .context("creating a private Security finding issue returned invalid UTF-8")?;
-                IssueUrl::parse(url.trim()).map_err(|_| {
+                let issue = IssueUrl::parse(url.trim()).map_err(|_| {
                     anyhow::anyhow!(
                         "creating a private Security finding issue returned no issue URL"
                     )
                 })?;
-                Ok(json!({
-                    "body": draft.description,
-                    "title": draft.summary,
-                    "html_url": url.trim()
-                }))
+                let mut record = self.issue_view(&issue, "number,body,state,labels,title")?;
+                record["html_url"] = json!(url.trim());
+                Ok(record)
+            }
+        }
+    }
+
+    /// Only the reproduction fields change; state, labels and disclosure stay
+    /// with the Day shift. API diagnostics may contain private test evidence.
+    pub fn update_security_record(
+        &self,
+        repo: &str,
+        record: &SecurityRecord,
+        reproduction: &Reproduction,
+    ) -> Result<()> {
+        if !record.untriaged() {
+            bail!("refusing to replace a triaged Security finding record");
+        }
+        let (path, storage) = match record {
+            SecurityRecord::Advisory { id, .. } => (
+                format!("repos/{repo}/security-advisories/{id}"),
+                SecurityRecords::Advisories(Vec::new()),
+            ),
+            SecurityRecord::Issue { number, .. } => (
+                format!("repos/{repo}/issues/{number}"),
+                SecurityRecords::Issues(Vec::new()),
+            ),
+        };
+        let observed = self.output_with_input(&["api", &path], None)?;
+        if !observed.status.success() {
+            bail!(
+                "reading a private Security finding record before its update failed ({})",
+                observed.status
+            );
+        }
+        let observed: Value = serde_json::from_slice(&observed.stdout)
+            .context("reading a private Security finding record returned invalid JSON")?;
+        let current = storage.record(&observed)?;
+        if !current.untriaged() {
+            bail!(
+                "the Security finding was triaged during its reproduction; leaving the record unchanged"
+            );
+        }
+        if current.description() != record.description() {
+            bail!(
+                "the Security finding changed during its reproduction; leaving the record unchanged"
+            );
+        }
+        if matches!(record, SecurityRecord::Advisory { .. })
+            && matches!(reproduction.severity(), Some(Severity::Informational))
+        {
+            bail!(
+                "informational severity has no GitHub advisory field; the Day shift must decide its representation; leaving the record unchanged"
+            );
+        }
+        let description = reproduction.description(record.description());
+        let body = match record {
+            SecurityRecord::Advisory { .. } => {
+                json!({"description": description, "severity": reproduction.severity()})
             }
+            SecurityRecord::Issue { .. } => json!({"body": description}),
+        };
+        let body = serde_json::to_vec(&body)?;
+        let output = self.output_with_input(
+            &["api", "--method", "PATCH", &path, "--input", "-"],
+            Some(&body),
+        )?;
+        if !output.status.success() {
+            bail!(
+                "updating a private Security finding record failed ({})",
+                output.status
+            );
         }
+        Ok(())
     }
 
     /// The create endpoint creates a draft. No severity or version claim.