diff --git a/CONTEXT.md b/CONTEXT.md index c28cca6..236f28f 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -69,7 +69,7 @@ One invocation of the factory on a single issue that is not a **Spec**, from lau A factory-owned checkout, on an **Issue branch** for a **Run** or detached for an **Architecture review**. Acquisition accounts for partial local effects, removing only attempt-owned clean artifacts on failure and retaining and naming work or uncertain artifacts. Failed-acquisition retention survives later Architecture review attempts. Successful acquisition carries checkout and registration identity through its lifetime; ordinary synchronization, observations and Failed run preservation share acquired-instance checks before and after every Git subprocess attempt, including retries, acting only on that acquired checkout and selected Issue branch and refusing changed or uncertain ownership. Failed run preservation retains work when ownership has changed or is uncertain and finishes after Command interruption without clearing it; ordinary synchronization remains interruptible. Confirmed Self-merge branch deletion uses the captured Launch repository independently of checkout validity. Cleanup checks stage-specific disposal authority before and after every destructive Git attempt, including retries: acquired identity and the captured cleanup head for checkout removal, removed-path absence and an unused expected head for local branch removal, then ref absence and the exact original local subsection for branch configuration removal. Only successful, verified transitions advance disposal; partial failures retain and name every remaining or uncertain resource. Live cleanup finishes after Command interruption without clearing it; stale disposal stays interruptible. Stale Architecture review scratch is disposable only with evidence of successful detached acquisition and unchanged ownership; unmarked or uncertain checkouts are retained. **Claim**: -The mark that the factory has taken an issue: thirdshift labels the issue `in-progress`, in place of `ready-for-agent` if it has that, when a **Run** or a **Spec run** starts on it, whether started on its **Issue URL** or by an **Architect run**, a **Pickup run** or a **Security run**. A **Ticket**'s Run in a Spec run and a **Base fix** make none. Ending a Claim finishes even after Command interruption. A Claim is released, `ready-for-agent` put back, only when the Run ends with nothing on `origin` to take over: no **Issue branch** and no pull request. Otherwise it stays while the issue is open, whether its pull request is ready for review or the Run failed, until the **Day shift** relabels it, and thirdshift takes the label off once the issue is closed. +The mark that the factory has taken an issue: thirdshift labels the issue `in-progress`, in place of `ready-for-agent` if it has that, when a **Run** or a **Spec run** starts on it, whether started on its **Issue URL** or by an **Architect run**, a **Pickup run** or a **Security run**. A **Ticket**'s Run in a Spec run and a **Base fix** make none. Ending a Claim finishes even after Command interruption. A failed Security fix keeps its Claim even with nothing on `origin`, so it stays with the Day shift. For other Runs, a Claim is released, `ready-for-agent` put back, only when the Run ends with nothing on `origin` to take over: no **Issue branch** and no pull request. Otherwise it stays while the issue is open, whether its pull request is ready for review or the Run failed, until the **Day shift** relabels it, and thirdshift takes the label off once the issue is closed. _Avoid_: lock (a lock is per machine and ends with its process), assignment **Spec run**: @@ -109,7 +109,7 @@ A **Pickup run**'s removal of `in-progress` from every closed issue in the repos _Avoid_: cleanup, garbage collection **Ready issue**: -An open issue a **Pickup run** may take: labelled `ready-for-agent`, with no label that makes an **Unready Ticket** and no **Claim**, not a sub-issue, not a **Base fix**'s issue, with no open blocker, never started (no **Issue branch** and no pull request), not a **Spec** whose **Tickets** are all closed, and left untouched long enough that whoever is shaping it has finished. On a Spec, the label says its Tickets are published. A `ready-for-agent` Ticket inside a Spec is not one: it is reached through its Spec, when the Spec is itself a Ready issue. +An open issue a **Pickup run** may take: labelled `ready-for-agent`, with no label that makes an **Unready Ticket** and no **Claim**, not a sub-issue, not a **Base fix**'s or a **Security run**'s fix issue, with no open blocker, never started (no **Issue branch** and no pull request), not a **Spec** whose **Tickets** are all closed, and left untouched long enough that whoever is shaping it has finished. On a Spec, the label says its Tickets are published. A `ready-for-agent` Ticket inside a Spec is not one: it is reached through its Spec, when the Spec is itself a Ready issue. _Avoid_: queued issue, backlog item **Architecture review**: @@ -156,7 +156,7 @@ A **Run** asked to end with its pull request merged rather than left for review, _Avoid_: auto-merge (GitHub's own feature, which thirdshift does not use) **Run notification**: -A message thirdshift sends when a **Run**, a **Spec run**, an **Architect run** or a **Security run** ends, whatever its outcome (ready, merged, failed or interrupted; for an Architect run that dispatched nothing, plan published, idea filed, idea already filed or review failed), to the address given with the email flag or the default in the **User config**. Each sends one only when asked to, by the flag or by the User config; a Spec run's notification lists each **Ticket**'s outcome, and a Ticket's **Run** never sends one of its own. After an **Inherited failure** it carries, beside the cause, what the Run says on stderr: where the checks fail on the **Base branch**, and the offer of a **Base fix** if nobody decided against one. An Architect run's notification tells how its **Architecture review** ended, naming the plan or idea issue, and how the Spec run or Run it dispatched ended, which sends none of its own; a skipped one sends none. A **Pickup run** that took a **Ready issue** sends the one the Spec run or Run it dispatched would have sent, which sends none of its own; a skipped one sends none. A **Security run**'s notification lists each **Security finding**'s severity, title and private link, never its write-up, since the message passes through a third party; a skipped one sends none. Failing to send one never changes the outcome. +A message thirdshift sends when a **Run**, a **Spec run**, an **Architect run** or a **Security run** ends, whatever its outcome (ready, merged, failed or interrupted; for an Architect run that dispatched nothing, plan published, idea filed, idea already filed or review failed), to the address given with the email flag or the default in the **User config**. Each sends one only when asked to, by the flag or by the User config; a Spec run's notification lists each **Ticket**'s outcome, and a Ticket's **Run** never sends one of its own. After an **Inherited failure** it carries, beside the cause, what the Run says on stderr: where the checks fail on the **Base branch**, and the offer of a **Base fix** if nobody decided against one. An Architect run's notification tells how its **Architecture review** ended, naming the plan or idea issue, and how the Spec run or Run it dispatched ended, which sends none of its own; a skipped one sends none. A **Pickup run** that took a **Ready issue** sends the one the Spec run or Run it dispatched would have sent, which sends none of its own; a skipped one sends none. A **Security run**'s notification lists each **Security finding**'s severity, title and private link, never its write-up, since the message passes through a third party. When a Run leaves a reproduced finding unfixed and nobody decided against fixing, it offers both the command and the User config setting that allow fixing; a skipped one sends none. Failing to send one never changes the outcome. _Avoid_: completion email, alert **User config**: diff --git a/README.md b/README.md index f0e8099..2da9e45 100644 --- a/README.md +++ b/README.md @@ -41,7 +41,7 @@ The **Claim** is the mark that the factory has taken an issue, so the issue list How the Run or the Spec run ends decides what becomes of its Claim: -- **Released**, when it fails with nothing on `origin` to take over: no Issue branch for the issue there and no pull request from one, open, merged or closed, or for a Spec run, no Spec branch and no Spec PR. That is a session that fails before it changes anything, as on a usage limit or an expired login, a Ctrl-C before any work, or a worktree that can't be created, on an issue that was never started before. A retry that fails that way on an issue an earlier Run left an Issue branch or a pull request for keeps its Claim. The issue's labels go back as they were before the Claim: `in-progress` comes off, and `ready-for-agent` goes back if the issue had it, and only then. So an outage costs a retry and doesn't strand the issue. stderr says `releasing the Claim on #: labelling it ready-for-agent, in place of in-progress`, or `releasing the Claim on #: removing in-progress`. A Failed run whose push failed is released too, since nothing of it reached `origin`: its work is only in the worktree it kept, which stops the next Run on that machine at the pre-flight checks until you push or delete the local Issue branch. +- **Released**, except for a failed Security fix, when it fails with nothing on `origin` to take over: no Issue branch for the issue there and no pull request from one, open, merged or closed, or for a Spec run, no Spec branch and no Spec PR. That is a session that fails before it changes anything, as on a usage limit or an expired login, a Ctrl-C before any work, or a worktree that can't be created, on an issue that was never started before. A retry that fails that way on an issue an earlier Run left an Issue branch or a pull request for keeps its Claim. The issue's labels go back as they were before the Claim: `in-progress` comes off, and `ready-for-agent` goes back if the issue had it, and only then. So an outage costs a retry and doesn't strand the issue. stderr says `releasing the Claim on #: labelling it ready-for-agent, in place of in-progress`, or `releasing the Claim on #: removing in-progress`. A Failed run whose push failed is released too, since nothing of it reached `origin`: its work is only in the worktree it kept, which stops the next Run on that machine at the pre-flight checks until you push or delete the local Issue branch. - **Kept**, on any other ending that leaves the issue open: a pull request ready for review, or a [Failed run](#failed-runs) that pushed its Issue branch or left a pull request. The issue stays `in-progress` until you relabel it, so a failure waits for you. - **Removed**, in a Merge run, once the Self-merge has left the issue closed, whether thirdshift closed it or GitHub did: `in-progress` comes off, after `removing in-progress from issue #` on stderr. @@ -518,7 +518,7 @@ A Failed run: 1. Commits any uncommitted work as `thirdshift: failed run ()`, with a timestamp and the hostname, and pushes the Issue branch, so nothing is lost. If the branch has no changes against the Base branch, nothing is pushed. 2. Converts its open pull request, if any, back to a draft, so a pull request only claims to be ready when the factory stands behind it. The next successful Continuation marks it ready again. The exception is a Merge run's policy refusal: the pull request is ready, mergeable and green and only the Self-merge could not happen, so it stays ready for review, and no failure commit is pushed onto the head whose CI was watched. 3. Cleans up as usual, prints the reason to stderr and exits non-zero. If the push failed, the worktree and local Issue branch are kept instead, and stderr names the branch, its head commit and the worktree path, so you can recover the work or push it by hand. -4. Releases its [Claim](#when-the-claim-ends) if it left nothing on `origin`: no Issue branch and no pull request. Otherwise the issue stays `in-progress`. +4. Releases its [Claim](#when-the-claim-ends), unless it is a failed Security fix, if it left nothing on `origin`: no Issue branch and no pull request. Otherwise the issue stays `in-progress`. Merges, never rebases or force-pushes: a branch worked on from several servers never loses history. @@ -543,7 +543,7 @@ In a [Spec run](#spec-runs), a Ticket's Run says this on its own stderr, relayed A **Spec** is an issue with sub-issues, its **Tickets**. `thirdshift ` on a Spec is a **Spec run**: it works through the Tickets in the order their GitHub "blocked by" links allow, each Ticket's **Run** a **Merge run** into the **Spec branch**, then leaves one **Spec PR** from the Spec branch into the Base branch ready for review ([ADR-0006](docs/adr/0006-spec-runs-merge-tickets-into-a-spec-branch.md)). An issue with no sub-issues is an ordinary Run. -A Spec run makes the [Claim](#the-claim) on the Spec, as a Run does on its issue: the Spec is labelled `in-progress`, in place of `ready-for-agent`. Its Tickets' Runs change no label on their Tickets. The Claim [ends](#when-the-claim-ends) as a Run's does: it is released if the Spec run fails with no Spec branch on `origin` and no Spec PR, removed once a Self-merge of the Spec PR has left the Spec closed, and kept otherwise. The Spec branch is pushed before the first Ticket starts, so a Spec run that got as far as its Tickets keeps its Claim. +A Spec run makes the [Claim](#the-claim) on the Spec, as a Run does on its issue: the Spec is labelled `in-progress`, in place of `ready-for-agent`. Its Tickets' Runs change no label on their Tickets. The Claim [ends](#when-the-claim-ends) as a Run's does: it is released if the Spec run fails with no Spec branch on `origin` and no Spec PR, except for a failed Security fix, removed once a Self-merge of the Spec PR has left the Spec closed, and kept otherwise. The Spec branch is pushed before the first Ticket starts, so a Spec run that got as far as its Tickets keeps its Claim. The Spec PR opens as a draft as soon as the first Ticket lands, titled from the Spec, with `Closes #` and a Tickets checklist: one line per Ticket, ticked once it is done, with its pull request, or saying it is running, failed, blocked or unready. thirdshift rewrites the checklist between its `` markers as each Ticket starts and ends, leaving the rest of the body as it is. Once every Ticket is done, the **Spec review** rewrites the body, and thirdshift puts the checklist back, appending it if the markers are gone, before marking the Spec PR ready. @@ -753,13 +753,15 @@ A publishing session reads the private record and follows the reproduction's fix On a private repository, the finding's own issue is the fix's Ticket or Spec, keeping its write-up and proof-of-concept. A one-session fix needs no publishing session or second issue: thirdshift marks the finding's issue ready and dispatches it. For a bigger fix, the publishing session adds terse Tickets as that issue's native sub-issues, with their blocking links. -`email`, optionally followed by an address, or `email.always` in the User config asks for one **Run notification** when the Security run ends, whether the audit succeeded, failed or was interrupted. `no-email` overrides the default. The notification says how the audit ended and lists each finding it recorded or matched to an existing private record: its severity when known, title and private link. It includes no finding write-up, trace or evidence, since it passes through Resend. Detailed failure causes stay in the local logs; the notification gives the audit's status and log paths. A recording failure still lists the records reached before it failed. A skipped Security run sends none. The usual address and Resend API key checks run before the skip checks or any work; a failed send is a warning and never changes the run's outcome. +A failed fix keeps its **Claim**, even if it pushed nothing, and stays open for the **Day shift**. No later Security run or Pickup run retries it. Security runs pause while that fix's issue is open; closing the issue lets them go on. + +`email`, optionally followed by an address, or `email.always` in the User config asks for one **Run notification** when the Security run ends, whether the audit succeeded, failed or was interrupted. `no-email` overrides the default. The notification says how the audit ended and lists each finding it recorded or matched to an existing private record: its severity when known, title and private link. It includes no finding write-up, trace or evidence, since it passes through Resend. Detailed failure causes stay in the local logs; the notification gives the audit's status and log paths. A recording failure still lists the records reached before it failed. When a run leaves a reproduced finding unfixed and neither the command nor the User config decided against fixing, its notification and stderr offer both `thirdshift secure security-fix` and `fix = true` under `[security]`. With `no-security-fix`, or the setting turned off, it offers nothing. A skipped Security run sends none. The usual address and Resend API key checks run before the skip checks or any work; a failed send is a warning and never changes the run's outcome. A Security run chooses its Harness from the command's `harness` word, then `[security] harness` in the User config, then `harness.default`, then Claude Code. A blank or missing Security setting keeps that default. Model and Effort come from the chosen Harness's own `[harness.]` section, with command words taking precedence. For example, `harness claude` overrides `[security] harness = "codex"` and uses `[harness.claude]`. Codex security sessions and their Resumes set `agents.max_concurrent_threads_per_session=8` and ask for fresh sub-agents with `fork_turns: "none"`, so the skill's verifiers stay independent. -A Security run checks its gates in order: another **Pass** on the repository running on the machine, a **Ready issue**, a **Security finding** waiting for the **Day shift**, then an unchanged **Base branch** since the last completed Security audit. An unreproduced draft advisory with no severity waits; closing it, publishing it or assigning severity ends that wait. A reproduced finding waits while fixing is disabled until its record is closed or published, or its fix Ticket is closed. On a private repository, an open finding issue with `needs-triage` waits until that label is removed or the issue is closed. The audited commit comes from the skill's own `run-metadata.json` under the audit root, with no separate state file. A skip exits `0`, starts no session, keeps no Command log and sends no Run notification; its reason goes into the repository's **Activity log** only when it changes, with the usual `activity.quiet_skips` behavior. Before starting work, thirdshift checks the chosen Harness and that **Node.js** is on `PATH`, since the embedded skill's report validators require it. +A Security run checks its gates in order: another **Pass** on the repository running on the machine, a **Ready issue**, a failed Security fix whose issue is still open, a **Security finding** waiting for the **Day shift**, then an unchanged **Base branch** since the last completed Security audit. An unreproduced draft advisory with no severity waits; closing it, publishing it or assigning severity ends that wait. A reproduced finding waits while fixing is disabled until its record is closed or published, or its fix Ticket is closed. On a private repository, an open finding issue with `needs-triage` waits until that label is removed or the issue is closed. The audited commit comes from the skill's own `run-metadata.json` under the audit root, with no separate state file. A skip exits `0`, starts no session, keeps no Command log and sends no Run notification; its reason goes into the repository's **Activity log** only when it changes, with the usual `activity.quiet_skips` behavior. Before starting work, thirdshift checks the chosen Harness and that **Node.js** is on `PATH`, since the embedded skill's report validators require it. The Security audit runs in a throwaway worktree detached at origin's Base branch head, leaving the Launch directory and local work untouched, including when `launch.pull` is set. Its Session prompt runs `thirdshift-security-audit` in full audit mode with the `quick` profile, auditing the whole repository except vendored and third-party code. It names a `SECURITY.md` or conventional threat-model document when present, reads compatible earlier runs and ends `incomplete` instead of asking for input. @@ -767,6 +769,19 @@ Each audit keeps a new output directory under `///audits/ A finding's title becomes the advisory summary. Its description preserves its write-up, trace, evidence and validation plan, with the fingerprint and audited commit. thirdshift claims no severity, CWE or affected version range. It uses the package in the repository's manifest, or the `other` ecosystem when none is identified. Matching fingerprints in any advisory state are kept rather than recorded again. Rejected candidates stay in the local report and produce no advisory. Draft advisories require GitHub repository security manager or administrator access. On a private repository whose advisory endpoint is unavailable, the private record is an issue labelled `security-finding` and `needs-triage`. It is never replaced by a public finding issue. +### Fencing + +**Fencing** is the factory finding and fixing vulnerabilities with nobody starting it: Security runs started on a schedule with fixing allowed. thirdshift never schedules itself ([ADR-0009](docs/adr/0009-the-operating-system-schedules-thirdshift.md)). This Linux crontab entry starts one every five minutes; use your own paths, with `claude` and `gh` already logged in and Git able to push: + +```cron +PATH=/home/you/.local/bin:/usr/local/bin:/usr/bin:/bin +*/5 * * * * cd /home/you/repos/widgets && thirdshift secure base main harness claude security-fix >> /home/you/thirdshift-secure.log 2>&1 +``` + +Each pass yields to a **Ready issue**, so work the **Day shift** shaped goes first. It is skipped while another Pass on the repository is running, while a fix has failed and its issue stays open, while a finding waits for triage and no reproduced finding can be fixed, or while the Base branch is unchanged and no fix is waiting. A failed fix pauses Fencing for the Day shift until its issue is closed; it is never retried by a later Security run or Pickup run. Triaging a finding lets audits go on; fixing allowed can still take a reproduced finding ahead of a different finding waiting for triage. + +A skip exits `0` and sends no Run notification. Its reason appears in the **Activity log** once per change of reason. Set `activity.quiet_skips = true` in the [User config](#user-config) to keep those skips out of the scheduler's log. `merge` on the command, or `merge.always = true` in the User config, lets each fix's Run merge its pull request; otherwise it stays ready for review. + ## Pickup runs `thirdshift pickup` starts a **Pickup run**: one pass, with no **Issue URL**, that takes the lowest-numbered **Ready issue** in the repository and runs it, so that an issue you have already marked ready needs no command typed for it. Start it from a clone of the repository, on the **Base branch**, or name the Base branch with `base `: @@ -784,7 +799,7 @@ A Ready issue is an open issue that: - has none of the labels that make an **Unready Ticket**, `ready-for-human`, `needs-info`, `wontfix` and `needs-triage`, so a contradictory label errs on the side of not running; - carries no [Claim](#the-claim): it is not labelled `in-progress`; - is not a sub-issue. A sub-issue is a **Ticket** of a **Spec**, and is never run on its own, whatever the Spec's labels: it is reached through its Spec, when the Spec is itself a Ready issue, so its work always goes through the **Spec branch**. Labelling one Ticket never promotes its Spec, either; -- is not labelled `base-fix`: the Run that opened a [Base fix](#base-fix)'s issue owns it; +- is not labelled `base-fix` or `security-fix`: the Run that opened a [Base fix](#base-fix)'s issue, or the Security run that dispatched a fix, owns it. A failed Security fix is still passed over if GitHub prevented its Claim from being made; - has no open blocker, by GitHub's "blocked by" links, never the text of its body. Once every blocker is closed, it can be taken; - was never started: no **Issue branch** for it is on `origin`, and no pull request from one exists, open, merged or closed. So a Pickup run never does a [Continuation](#continuation), and an issue whose Run failed waits for you; - is not a **Spec** whose **Tickets** are all closed. With nothing started on it, such a Spec was done some other way, and the [Spec run](#spec-runs) it would be dispatched as stops with nothing to do, so a Pickup run passes it over rather than take it on every pass. Waiting never makes it a Ready issue, so this comes before the next condition, however lately the Spec was labelled: close it, or take its `ready-for-agent` off. A Spec with at least one open Ticket is taken; @@ -905,7 +920,7 @@ The command's flags and the User config decide what a pass does with the issue i */30 * * * * cd ~/repos/widgets && thirdshift pickup base main no-merge >> ~/.thirdshift/logs/cron.log 2>&1 ``` -An issue whose run failed [keeps its Claim](#when-the-claim-ends) and waits for the **Day shift**: it stays `in-progress`, with the Issue branch and any draft pull request the run left, no later pass takes it, and it counts towards the Claim limit until it is closed or you take the label off. To send it round again by hand, run `thirdshift ` from the clone, with the Base branch checked out, since that command takes no `base `: it picks up where the failed run stopped, as a [Continuation](#continuation) does. Labelling it `ready-for-agent` again doesn't do it: a Pickup run never takes an issue that was started. The one failure a later pass does retry is one that left nothing on `origin`, such as a usage limit or an expired login: its Claim is released, so the issue is a Ready issue again once it has settled. +An issue whose run failed [keeps its Claim](#when-the-claim-ends) and waits for the **Day shift**: it stays `in-progress`, with the Issue branch and any draft pull request the run left, no later pass takes it, and it counts towards the Claim limit until it is closed or you take the label off. To send it round again by hand, run `thirdshift ` from the clone, with the Base branch checked out, since that command takes no `base `: it picks up where the failed run stopped, as a [Continuation](#continuation) does. Labelling it `ready-for-agent` again doesn't do it: a Pickup run never takes an issue that was started. For an ordinary Run, the one failure a later pass does retry is one that left nothing on `origin`, such as a usage limit or an expired login: its Claim is released, so the issue is a Ready issue again once it has settled. A failed Security fix is always left for the Day shift, even with nothing on `origin`. If you shape your issues with the upstream `to-spec` and `to-tickets`, from [mattpocock/skills](https://github.com/mattpocock/skills), unchanged: `ready-for-agent` on a **Spec** must mean its **Tickets** are published, every Ticket attached as a sub-issue and every "blocked by" link in place. Upstream, `to-spec` labels the Spec `ready-for-agent` when it publishes it, minutes or days before `to-tickets` attaches the Tickets, and a Spec with no Tickets yet looks exactly like a standalone Ticket, so a pass would start a plain Run on it. Label such a Spec `needs-triage` until its Tickets are attached, then swap that for `ready-for-agent`, as the copies of the two this repository's own Day shift uses, under `.agents/skills/`, do ([`docs/agents/triage-labels.md`](docs/agents/triage-labels.md)). The ten minutes an issue must be settled for are only a second line of defence: they cover Tickets attached within minutes of the label, not a Spec left labelled and without Tickets for longer. diff --git a/src/base_fix.rs b/src/base_fix.rs index 78d1e27..82ac205 100644 --- a/src/base_fix.rs +++ b/src/base_fix.rs @@ -61,11 +61,10 @@ const RETRY_WITH: &str = "Retry with"; /// What starts the line of [`Advice`] naming the User config's `base.fix`. const OR_SET: &str = "Or set"; -/// A line of the advice a Run gives after its cause when Inherited failures -/// fail it with no Base fix taken: each check where it fails on the Base -/// branch, then, if nobody decided against a Base fix, how to allow one. It -/// reads `