diff --git a/governance/ENFORCEMENT.md b/governance/ENFORCEMENT.md new file mode 100644 index 00000000..93ce9fc0 --- /dev/null +++ b/governance/ENFORCEMENT.md @@ -0,0 +1,74 @@ +# Enforcement + +Every rule in `constitution.md` has a row here. A row records: + +- **Mode**: test, agent or human (see the constitution's "How rules are + enforced"). +- **Check**: the CI check, plus a yes/no review question for any part a test + cannot reach. +- **Today**: what the check finds on `main`, measured. The measurements below + were taken on `f9bfd08` (2026-10-12). +- **Status**. + +Status values: + +- `enforced`: the check runs in CI on every pull request, and a pull request + built to violate it was shown to fail. The row links that demonstration. +- `running`: the check runs in CI, but no failure demonstration is recorded + yet. +- `dormant`: the check runs but asserts nothing until a named precondition is + met. It becomes `enforced` on its own when the precondition holds. +- `planned`: agreed, not yet written. The row names the milestone it lands in. + A row planned for two milestones is either built or struck. +- `unbuilt`: no check exists, and nobody has started one. +- `manual`: enforced by agent or human review only, with the review question + and the owner named. + +A new check starts from today's findings as an allow-list. Each entry links an +issue, and the list can only shrink (A1). + +**No check blocks a merge yet.** `main` requires a pull request, but no status +checks and no approvals, and `enforce_admins` is off (see ENF-R1). Until R1 is +in place, every row below is visible in CI but does not stop a merge. + +| ID | Mode | Check | Today | Status | +|----|------|-------|-------|--------| +| ENF-S1 | test + human | Planned: a committed inventory of every `exec` command, plugin step type and `gh_api` method in a shipped framework, each recorded as reading setup. A step not in the inventory fails. `gh_api` steps must be GET. Adding a row needs maintainer review against the question "Could this step's answer change while nothing in the repository or its platform settings changes (tool version, network, time, history)?" | About 15-20 rows. 5 outcome uses: `zizmor` ×2 (BR-01.01, AC-04.02), `gittuf_verify_policy`, `gittuf_commits_signed`, `repro_witness_attestation`. 2 borderline: `repro_hermetic_build`, `repro_bit_for_bit`. These are cut or reframed per ADR-0001. | planned: scope-reset milestone | +| ENF-S2 | test | CI `Test` job: `tests/darnit/tools/test_compliance_errors.py`, `tests/darnit/assertions/test_compliance_effects.py`, `tests/darnit/harness/test_harness_compliance.py`, `tests/darnit/config/test_strict_framework_loading.py`, and the adversarial corpus gate (`tests/darnit_baseline/corpus/`, including `plugin-redefines-manual`). Planned: a test that every shipped `promotion` matches corpus eligibility. | Known gaps from the wiring audit: the shared-handler cache, list-valued `when` (N/A), `gh_api`/`mcp` truthiness, a plugin registering without its plugin context, a dropped control leaving its level compliant. 0 promotions are shipped. | running (only the corpus gate has a failure demonstration) | +| ENF-S3 | test | CI `Test` job: `tests/darnit/context_integrity/`. | Known gap: raw `.project/project.yaml` values satisfy a remediation `when` guard, and the `project.*` template namespace is unguarded. | running | +| ENF-S4 | test | CI `Test` job: `tests/darnit/context_integrity/test_single_writer.py`, `test_user_files_survive.py`. Planned: an allow-list of filesystem write sites in `packages/*/src`, so a new write site outside the executor fails. | Write sites outside the executor are reachable from remediation: `darnit_baseline/threat_model/remediation.py` (5 sites; overwrites uncommitted user files), `remediation/helpers.py:53`, `darnit_baseline/tools.py:1144-1145`. | running, with known gaps | +| ENF-S5 | test | CI `Test` job: `tests/darnit/config/operator/test_repository_cannot_configure.py`, `test_baseline_toml_removed.py`, `tests/darnit/trust/`. | Known gap: `remediate_audit_findings` builds its trust target from the checkout's origin remote. | running, with known gaps | +| ENF-F1 | test | Planned: an AST check over `packages/darnit/src` that fails on an import of an implementation package, a framework-name literal outside the one default constant, and a `register_control` call outside the control loader. | 0 imports. 5 framework-name literals in 4 files: `agent/graph.py:402,416`, `cli.py:504`, `config/merger.py:691`, `config/context_schema.py:350`. `register_control` is called only by the loader, but CLAUDE.md shows it as a plugin API. | planned: first checks milestone | +| ENF-F2 | test | Today: tier-1 parity tests (`tests/darnit/parity/tier1/`) compare `audit_openssf_baseline` with `HarnessRun` only. Planned: a parity matrix of entry points (CLI, MCP, builtin, harness, `darnit run`) × options over the parity fixtures, plus an AST check that profile, tags and default-framework resolution happen in one module. | Profile resolution happens in 3 places and is absent from the CLI; tags are parsed in 3 places; the default framework is set in 4. CLI `audit` has no `--level`; MCP audit has no include/exclude; the harness has no tags or profile. | planned: second checks milestone | +| ENF-F3 | test + agent | Planned, three ratchets:
(1) Every MCP tool parameter and CLI flag is read (AST); configuration fields later.
(2) Whole-repo mypy against a baseline generated in CI's environment. Any `attr-defined`, `call-arg` or `name-defined` error fails outright.
(3) The set of shipped functions with no executed line, from CI's coverage run, may only shrink. Protocol stubs and test-fixture packages are excluded.
Agent question: "Does a test reach each new or changed default implementation through a real entry point?" | (1) 6 unread: `auto_init_config`, `template`, `get_pending_data` `level` and `profile`, `--profile` on `audit` and `plan`. (2) 201 errors, 21 in the always-fail tier. (3) 101 functions. Today's CI `Type Check` job checks 2 files with `\|\| true` and gates nothing. | planned: first checks milestone | +| ENF-F4 | test | CI `Non-dev install smoke` job (#575, extended by #596): builds every published wheel, installs it without development dependencies, runs one audit, and fails on a control that could not be evaluated or on more than one wheel per package. | Passes on `main`. Shown to fail when `packaging` is removed from darnit-core (#575 review). Plugins loaded only through registration (gittuf, reproducibility) are not imported in full. | running (a failure demonstration exists in review; link it from a closed pull request to mark `enforced`) | +| ENF-C1 | test + human | Planned: committed snapshots of each contract surface: MCP tool schemas, the argparse tree, `model_json_schema()` of both configuration schemas, and JSON/SARIF/predicate output for one fixture. A removal or rename in a snapshot diff requires a BREAKING line. Human question: "Did the ADR or spec that decides this change land with or before it?" | Today's `Spec Sync` check finds 8 hard-coded handler names in a document and enforces almost nothing. | manual (maintainers); snapshots planned | +| ENF-R1 | test + human | `main` branch protection:
- required checks `Test`, `Lint`, `Spec-Implementation Sync`, `Build`, `DCO` and `agent-review`;
- required code-owner review, with CODEOWNERS listing only the owned paths, so a pull request touching them needs an owner other than its author;
- administrators included.

The `agent-review` check runs a fresh agent session that the author cannot steer. It has its own identity, takes the constitution's review questions as input, treats pull request text as untrusted, and reads no secrets from a fork's code. It passes when it raises no blocking finding, or when a maintainer other than the author has approved the pull request.

Human part: a weekly digest of pull requests merged without a second person goes to the maintainers, and anything doubtful becomes an issue.

Planned:
- a scheduled job that fails when protection drifts from this declaration;
- a test that the CI test command has no marker, `-k`, `--deselect` or `--ignore` selection;
- a hermetic job that runs the suite with an empty HOME, CI variables set and the network blocked, and fails if anything is written under HOME. | Protection requires a PR only: 0 approvals, no required checks, `enforce_admins` off. No `agent-review` check exists. CODEOWNERS owns everything (`*`) and names only one maintainer. CI has run the whole suite since #592. With an empty HOME, the suite writes 377 files there, including `~/.darnit/verification_cache`, a pip cache, tree-sitter grammars and sigstore TUF metadata (a network fetch). | unbuilt (settings change pending) | +| ENF-A1 | test + agent | Planned: a committed file of ratchet counts and sets, compared on every pull request:
- suppressions by kind;
- the mypy baseline;
- the unexecuted-function set;
- reader allow-lists;
- the inventory of Protocols, entry-point groups and registries;
- CI's test selection.
An increase fails unless the pull request links an issue and a maintainer approves.
Agent question: "Did this change weaken, skip or suppress anything to make a run pass?" | Baseline: 85 `# noqa` (47 of them `BLE001`), 54 `# type: ignore`, 41 test skips (`mark.skip`/`skipif`, `pytest.skip(`, `importorskip`), 5 `pragma: no cover`, 2 `continue-on-error` and 10 `\|\| true` in workflows and scripts, 3 ruff per-file-ignores. 17 extension points (9 Protocols, 3 entry-point groups, 5 registries). | planned: first checks milestone | +| ENF-A2 | test + human | `DCO` check (not yet required; it cannot tell an agent-added sign-off from a person's). Planned:
- a check that fails on an AI `Co-authored-by` or `Signed-off-by` line, on a PR body without exactly one AI-assistance box ticked, or on AI use declared without an `Assisted-by` trailer;
- a check that fails on a local home-directory path in a tracked file;
- an output test that an audit report contains no home path or user profile.
Human question: "Can the author explain this change?" | Since AI_POLICY landed (38 commits): 1 AI co-author line, 6 AI-disclosed PRs without `Assisted-by`, 6 PRs with no AI box ticked. 21 home-directory paths in tracked files, mostly Spec Kit `tasks.md`. | planned: first checks milestone | +| ENF-A3 | test + agent | CODEOWNERS entry for `governance/` naming the maintainers, and required code-owner review on `main`. Planned: `scripts/governance_lint.py`, which requires every MUST in `governance/` to carry an id, every id to have a row here, and every row to be cited by a rule. Agent question: "Did a maintainer ask for this change to `governance/`?" | CODEOWNERS lists only `@mlieberman85`, plus a stale `/main.py` line. Code-owner review is not required. | unbuilt | + +## Build order + +1. ENF-R1 settings (no code) and a CODEOWNERS pull request that owns only + the expensive paths and names both maintainers. Nothing gates until these + land. Until the `agent-review` check exists, R1 falls back to a second + maintainer's approval on every pull request. +2. ENF-F3 (1), the reader check for MCP parameters and CLI flags, starting + with 6 allow-list entries linked to #528, #541 and #576. +3. ENF-F3 (2), the mypy gate, replacing today's `Type Check` job. +4. ENF-F1, the AST check, either fixing the 5 framework-name literals or + starting with them allow-listed. +5. ENF-A2, the attribution and home-path checks. +6. ENF-A1, the ratchet file. +7. ENF-F4: record the failure demonstration (#575 merged). +8. ENF-R1's hermetic job. The writes to HOME are fixed first, or allow-listed. +9. ENF-F2, the parity matrix and the single-resolver check. +10. ENF-F3 (3), the coverage ratchet. +11. ENF-C1, the contract snapshots. +12. ENF-S1, the inventory, after the scope-reset classification. +13. ENF-A3, governance_lint. + +Prove each new check: open a pull request built to violate it, watch the check +fail, close that pull request unmerged, and link it from the row. That link is +what moves a row from `running` to `enforced`. diff --git a/governance/constitution.md b/governance/constitution.md new file mode 100644 index 00000000..a3426cfd --- /dev/null +++ b/governance/constitution.md @@ -0,0 +1,146 @@ +# Constitution + +These are the non-negotiable rules for this repository. Every rule carries an +enforcement id that maps to a row in `ENFORCEMENT.md`. Each row says how the +rule is enforced and how far that enforcement reaches today. + +Changing a rule requires an ADR in `decisions/`. Do not edit a rule in place +without one. + +This document is deliberately short. Every agent session loads it, so each +rule costs context in every session, forever. Before adding a rule, read +`where-decisions-live.md`: most decisions belong somewhere else. + +What darnit is, and is not, is decided in +[ADR-0001](decisions/0001-what-darnit-is.md). This document sits under the +project's [Technical Charter](../CHARTER.md) and +[GOVERNANCE.md](../GOVERNANCE.md), which decide who decides. This document +decides what a change must satisfy. + +## How rules are enforced + +Every rule names one or more enforcement modes. A deterministic check is +always preferred. + +- **test**: a check in CI that fails on a violation, and has been shown to. +- **agent**: the rule is in every agent's instructions (`AGENTS.md`), and the + agent applies it while writing and reviewing, using the yes/no question in + the rule's row. +- **human**: a maintainer applies it in review, and is accountable for it. + +A rule may rely on agent or human enforcement alone only if its row says why a +test is impossible or disproportionate, gives the yes/no question a reviewer +answers from the diff, and names an owner. A rule enforced that way that +catches nothing for two milestones is reviewed for deletion. A rule whose +check cannot fail is not enforced. + +--- + +## 1. Scope and correctness + +**S1.** A control's result MUST depend only on the audited project's setup: +its files, its CI configuration, its platform settings (read, never changed by +a check) and its confirmed project data. The same setup MUST give the same +result on any machine, at any time, with any tool version darnit supports. A +PASS states that a practice is set up, never that its outcome holds: for +example, that gittuf is set up, not that history satisfies the policy; that a +scanner runs in CI, not that its findings are clean. `[ENF-S1]` + +**S2.** No false PASS. A step MUST conclude only an outcome in its effective +set: its registered ceiling, narrowed by `concludes`, widened only by a +promotion that the corpus backs with zero false PASS results. A model judgment +MUST NOT conclude PASS. No plugin MAY replace a core step type. A level is +compliant only when every applicable control at that level is PASS. WARN, +ERROR, PENDING, an unhonored claim and a control that failed to load all count +against it. `[ENF-S2]` + +**S3.** darnit MUST NOT conclude a value that needs a person's judgment +(`auto_detect = false`). Detection may propose a candidate. Only a person's +confirmation makes a value usable, and an unusable value MUST NOT reach a +verdict, a remediation input, an attestation or a write. `[ENF-S3]` + +**S4.** Remediation MUST write only through the remediation executor, MUST +record every write in the run manifest, and MUST NOT write a path with +uncommitted user changes. `[ENF-S4]` + +**S5.** The audited repository is untrusted input. Nothing in it, including its +own configuration files and its version-control remotes, configures darnit or +grants trust. Trust and tool configuration come only from operator +configuration or verified CI metadata. `[ENF-S5]` + +--- + +## 2. Framework shape + +**F1.** The core (`packages/darnit`) MUST NOT import or name an implementation +package or a framework. The default framework is one named constant. Controls +are data: only a framework's TOML defines them, and Python adds step types, +remediation handlers and tools. `[ENF-F1]` + +**F2.** There is one execution path. Every entry point that audits or +remediates (the CLI, each framework's MCP tools, the builtin tools, the +harness, `darnit run`) MUST resolve framework, level, tags, profile, +include/exclude and output format through one shared resolver, and MUST give +the same controls and statuses on the parity fixtures. `[ENF-F2]` + +**F3.** Shipped code can run. Everything darnit accepts MUST have a production +reader: every CLI flag, MCP tool parameter, step setting and configuration +field. Every function in a shipped package MUST be executed by a test, and a +shipped default implementation of a Protocol by a test that reaches it through +a real entry point. A change MUST NOT add a type error. `[ENF-F3]` + +**F4.** Each published package, installed from its built wheel without +development dependencies, MUST run an audit with no import error. `[ENF-F4]` + +--- + +## 3. Contracts + +**C1.** darnit's public contracts are: the CLI, the MCP tool surface, the +framework TOML schema, the operator configuration schema, the plugin protocol +and the output formats (JSON, SARIF, attestation predicate). A change to a +contract MUST be visible in review, MUST land with or after its ADR or feature +spec, and, when it breaks a user, MUST appear under a **BREAKING** entry in +`CHANGELOG.md`. Breaking changes also follow GOVERNANCE.md's RFC process. +`[ENF-C1]` + +--- + +## 4. Process + +**R1.** A change MUST reach `main` only through a pull request whose +required checks pass on its head. The required checks include: + +- the whole test suite, run hermetically: no real HOME, no network, no + dependence on CI environment variables; +- an independent agent review. + +A pull request that touches a path owned in CODEOWNERS MUST also be approved +by a code owner other than its author. The owned paths are the ones where a +wrong merge is expensive: `governance/`, the release and CI workflows, trust +and confirmation, the remediation executor and platform engine, and plugin +discovery. A pull request whose agent review raises a blocking finding needs +a maintainer other than the author to resolve it. + +Administrators are not exempt. The one exception is a merge from a GitHub +security advisory, which is recorded in the advisory. `[ENF-R1]` + +**A1.** Every ratchet only tightens: suppressions, skips, baselines, +allow-lists, the extension-point inventory and CI's test selection. Loosening +one requires a maintainer-approved pull request that links an issue. A +contributor (person or agent) who cannot satisfy a rule MUST stop and file an +issue that names the rule and the conflict, and MUST NOT weaken a check to +make a run pass. `[ENF-A1]` + +**A2.** Contributions follow [AI_POLICY.md](../AI_POLICY.md). Committed files, +pull request and issue text, and darnit's own output MUST NOT contain local +filesystem paths, an auditor's personal data, credentials, or the names of +private projects. `[ENF-A2]` + +**A3.** A change to `governance/` MUST be approved by a code owner of +`governance/`. Where the Charter gives the decision to the TSC (scope, and +contribution policy), the change MUST be approved by a majority of the TSC's +voting members, recorded as pull request approvals as GOVERNANCE.md +describes. Every MUST in `governance/` carries an enforcement id with a row in +`ENFORCEMENT.md`. An agent MUST NOT change `governance/` unless a maintainer +asked it to. `[ENF-A3]` diff --git a/governance/decisions/0001-what-darnit-is.md b/governance/decisions/0001-what-darnit-is.md new file mode 100644 index 00000000..fbb6b6b9 --- /dev/null +++ b/governance/decisions/0001-what-darnit-is.md @@ -0,0 +1,184 @@ +# ADR-0001: darnit aligns a project's setup with a hygiene goal + +- Status: proposed +- Date: 2026-10-12 +- Decided by: the TSC, because the Charter (2.g.ii) gives it changes to the + project's scope. +- Rule impact: creates S1. Restates the existing conservative, trust and + authority principles as S2-S5. + +## Context + +The maintainers have described darnit's purpose many times, but never in one +place a change could be held to. Without that statement, two kinds of work +kept entering the project, each reasonable on its own. + +The first kind is runtime verification: checks that run a tool and grade what +it says. + +- `gittuf_commits_signed` parses `git log %G?`. It turned a fabricated + signature into a dispositive PASS (wiring audit W-01). +- Witness runtime traces were Sigstore-verified to decide whether a build + touched the network (#553). +- Reproducibility checks infer hermeticity and bit-for-bit output. +- zizmor findings decide CI controls. + +Each brings that tool's trust roots and failure modes into darnit, and each is +a place where darnit can say PASS about something it cannot know. + +The second kind is generic interfaces ahead of any user: check and remediation +adapters, a locator, a storage backend, four store protocols, pass and template +registries, shared handlers, budgets, enhancers. The 2026-10-10 wiring audit +found 149 defects, most of them in exactly this kind of code. It had been +declared, documented and validated, but never connected. #487 removed about +8,000 lines of it, and more remains. + +darnit is an LF project with a [Technical Charter](../../CHARTER.md). This ADR +is not a charter. It decides what the software is for. + +## Decision + +**darnit is a framework for aligning a project's setup with a hygiene goal.** + +A hygiene goal is a published set of practices a project should have in place: +OpenSSF Baseline, the Community Specification, "use gittuf", "be reproducible". +darnit reads a project's **setup**, reports where it falls short of the goal, +and helps set up what is missing. + +Setup means: + +- **Files:** SECURITY.md, LICENSE, CODEOWNERS, lockfiles, a gittuf policy. +- **CI configuration:** which workflows run, what they run, and with what + permissions. +- **Platform settings:** branch protection, required reviews, 2FA + requirements, read through the platform API. +- **Declared project data:** `.project/`, confirmed by a person where a value + needs judgment (S3). + +**darnit does not verify runtime behavior or tool outcomes.** It checks that a +mechanism is set up and in use. Whether the mechanism produces the right +answer at any given moment is the job of the mechanism. + +| Goal | In scope: setup | Out of scope: outcome | +|------|-----------------|-----------------------| +| gittuf | gittuf is initialized; a policy exists with a root of trust; the policy is well formed; CI runs gittuf verification | whether history satisfies the policy; whether recent commits are signed | +| OpenSSF Baseline | SECURITY.md, LICENSE and contribution docs exist; branch protection and review settings are configured; security testing and dependency scanning run in CI | whether today's scan results are clean; whether the security tests pass | +| Reproducibility | dependencies are pinned in lockfiles; the build environment is declared; CI produces signed provenance | whether a build is hermetic at runtime; whether two builds are bit-for-bit identical | + +### The framework + +To serve that purpose, the framework provides: + +- **Evidence readers**: repository files, parsed CI configuration, platform + settings (read-only) and confirmed project data. +- **Checks**: declarative, defined in framework TOML. They produce PASS, FAIL, + or a result that is not yet known together with the reason it is not known. +- **Remediation**: a plan that is previewed, approved and then applied by a + single writer, which sets up what is missing. +- **Plugins**: hygiene goals (frameworks), and integrations that teach darnit + how to read and set up one tool's configuration. +- **Interfaces**: the CLI, MCP tools and agent skills, as thin shells over one + execution path. + +### The admission test + +A control, step type, interface, configuration key or subsystem is admitted only +if all three answers are yes: + +1. Does it read or change a project's setup? +2. Would its answer stay the same across two runs with no change to that + setup, on any machine, at any time, with any tool version darnit supports? +3. Does something that exists today need it? A new extension point (a + Protocol, entry-point group, registry or plugin hook) needs two + implementations, or an ADR that says why the seam is needed before the + second. + +Question 2 catches outcome checks: the answer to "are the scans clean" changes +without anyone touching the setup. Question 3 catches premature interfaces. + +What a control's PASS claims matters as much as how the control measures it. +A step can read only CI configuration and still back a claim about an outcome: +`repro_bit_for_bit` reads CI files, but RE-03.01 claims "build output is +identical across independent builds". S1 therefore says what a PASS may claim: +that a practice is set up, never that its outcome holds. + +## Alternatives + +**darnit as a verifier.** It would run gittuf verification, scanners, +reproducibility builds and Witness, and grade the results. This was rejected +for four reasons: + +- It duplicates tools that already do this and do it better. +- It makes darnit trust what those tools trust, which brings in signing keys, + Sigstore roots, monitors and network access. +- Its answers change without the project changing. +- It is where false PASS results come from. W-01 is a dispositive PASS for a + forged signature. + +**Keep outcome checks, but as advisory evidence.** Feature 044 took this route +for the reproducibility checks, and #565 did for Witness. It was rejected: + +- An advisory check still has to be maintained, tested and kept correct. +- It still shows up in reports. +- It still invites the next change that promotes it to deciding. + +**No written scope; rely on review.** This is the status quo, and the audit +measured its result. + +## Consequences + +- **Some controls are cut or reframed.** A scope-reset classification will + mark every control, step type and subsystem as keep, reframe or cut, for the + maintainers to decide. Expected outcomes: + - gittuf GT-02.01 ("recent commits are signed") is cut, or reframed to + "commit signing is required", by gittuf policy or branch protection. + - GT-01.02 is reframed from "policy passes verification" to "the policy is + well formed and CI runs gittuf verification". + - Witness verification is cut. + - RE-03.01 is reframed from "build output is identical" to "CI runs a + reproducibility check and normalizes build timestamps". + - RE-02.01 is reframed to the parts that read build configuration. + - The Baseline controls that grade zizmor findings are reframed to read the + workflow files themselves. A zizmor result can change with the zizmor + version, and with no workflows it is an exit-code WARN (#584). + - Two areas are deferred to the scope-reset classification: + - The threat-model analysis engine (features 006, 010, 011 and 014: + tree-sitter discovery and opengrep taint analysis). Producing a threat + model document as a remediation is setup. An analysis engine that grades + code is not. + - Feature 023's Scorecard wiring, because several Scorecard checks are + outcome checks. +- **Some audit findings close by deletion, not repair.** W-01 is one; several + P1s about unused interfaces are others. +- **Some OSPS controls are phrased as outcomes.** For those, darnit reports + whether the practice is set up. Reports and the attestation predicate say + "set up for", not "compliant with", wherever that is the truth. The exact + wording and the predicate change are a contract change under C1. +- **Fewer features.** Some contributions that work and were tested will be + declined because they verify an outcome. The PR template and the S1 review + make that visible before the work is done, not after. +- **Model judgments (`llm_eval`) are still allowed,** for a narrow use: whether + a file's content says what a setup practice requires, for example whether + SECURITY.md names a reporting channel. They remain PASS candidates that a + person confirms. That is still setup. + +## Decided in review + +The co-maintainer's review of the draft (#578, 2026-10-10) settled four of the +open questions: + +- The zizmor-graded controls are reframed to read the workflow files. +- Model judgments stay, as PASS candidates a person confirms. +- Reports say "set up for" where the practice, not its outcome, is what darnit + observed. +- Unknown step keys fail loading, and remediation acts on FAIL only. These two + belong to the docs-consolidation ADR, which follows separately. + +## Enforcement + +- **S1 (`ENF-S1`):** a committed inventory of every `exec` command, plugin step + type and `gh_api` method in a shipped framework, plus maintainer review of + each new inventory row. +- **S2-S5:** restate rules that existing tests check (`ENF-S2` to `ENF-S5`). +- **Question 3 of the admission test** is checked by the F3 reader and + coverage ratchets and by the extension-point inventory in A1. diff --git a/governance/decisions/0002-governance-model.md b/governance/decisions/0002-governance-model.md new file mode 100644 index 00000000..dbb70652 --- /dev/null +++ b/governance/decisions/0002-governance-model.md @@ -0,0 +1,195 @@ +# ADR-0002: Governance is a short constitution whose rules name their enforcement + +- Status: proposed +- Date: 2026-10-12 +- Decided by: the TSC, because the Charter (2.g.vii) gives it policies and + processes for contributing. +- Rule impact: creates `governance/` and the rules in `constitution.md`. + Retires the Spec Kit constitution as a separately edited document. + +## Context + +darnit's rules live in several places that disagree: + +- **The Spec Kit constitution** (`.specify/memory/constitution.md`, version + 1.3.2, 347 lines). +- **`CLAUDE.md`** (415 lines, about half of it feature changelog). +- **`docs/architecture/framework-design.md`** (2,369 lines, titled "the + authoritative specification"). +- **About 22,000 lines of docs**, many of which still describe removed or + never-built behavior. +- **`AI_POLICY.md`.** + +None of these documents says which of its statements anything enforces, and +the gaps showed. + +- The 2026-10-10 wiring audit found rules that nothing enforced, and a CI type + check that could not fail. +- Two days later, CI turned out to have been running only 2,390 of 5,164 + tests (#591). It selected tests by markers that most tests did not carry, so + none of the safety suites behind features 040-044 had ever run in CI. +- `main` requires no status checks, so even the checks that did run blocked + nothing. + +darnit's contributors use several coding agents (Claude, Gemini, Grok, +OpenAI Codex), and most of the code is written with one. A rule that only a +person remembers does not reach the agent writing the change. A rule that +only an agent applies can be skipped by the next agent. + +## Decision + +1. **`governance/` holds the rules.** It contains: + - `constitution.md`: the rules; + - `ENFORCEMENT.md`: one row per rule; + - `where-decisions-live.md`; + - `decisions/`: the ADRs. + + A change to it needs a code owner's approval, and the TSC's approval where + the Charter gives the TSC the decision (A3). +2. **Every rule names how it is enforced, and tests come first.** + - **test:** a deterministic CI check, preferred wherever one is possible. + - **agent:** the rule is in every agent's instructions, applied with a + yes/no review question. + - **human:** a maintainer applies it in review. + + A rule enforced only by agents or humans needs a stated reason why a test + is impossible or disproportionate, a review question and an owner, and it + is reviewed for deletion if it catches nothing for two milestones. Most + rules that look like they need judgment have a deterministic part (an + inventory, a count that only shrinks, a snapshot diff), and that part is + built as a test. +3. **A row says `enforced` only when it is proven.** A row is `enforced` only + when its check runs in CI and a pull request built to violate it was shown + to fail. A check that runs but has not been shown to fail is `running`. The + #591 gap is the reason: tests that were assumed to run did not. +4. **Every agent reads the same rules.** `AGENTS.md` at the repository root is + the agent instruction file, because Codex, Cursor, Copilot and others read + it, and Gemini can be pointed at it. It carries: + - a copy of the constitution between markers, which a test keeps identical + to `governance/constitution.md`; + - where to read before touching what; + - the test and lint commands; + - the stop-and-file protocol (A1); + - the attribution lines from AI_POLICY. + + `CLAUDE.md` becomes a short file that imports `AGENTS.md` and adds only + Claude-specific notes. The Spec Kit constitution becomes a pointer to + `governance/`, and nobody runs `/speckit.constitution`. +5. **The pull request template asks questions that can be checked:** + - which rules the change could affect; + - where a new flag, parameter or setting is read; + - whether a contract snapshot changed, and its BREAKING entry; + - the issue behind any new suppression or allow-list entry; + - the AI tool and model used, and the parts it wrote. +6. **An agent review is a required check, but never an approval.** + - An independent agent reviews each pull request against the + constitution's review questions, and reports possible violations citing + the rule, file and line. It runs in a fresh session the author cannot + steer, under its own identity. It treats pull request text as untrusted, + and never reads secrets from a fork's code. + - Its findings are claims, like a model judgment in darnit. A blocking + finding stops the merge until a maintainer other than the author resolves + it. A clean review lets the author merge only on paths that CODEOWNERS + does not own. + - On owned paths, a code owner other than the author always approves. An + agent's review is never enough on its own where a wrong merge is + expensive. This mirrors darnit's rule that a model judgment never + concludes PASS by itself. +7. **The Spec Kit constitution's principles become rules:** + - Plugin Separation and TOML-First become F1. + - Conservative-by-Default and Sieve Pipeline Integrity become S2. The + latter's "4-phase ordering MUST be respected" clause is dropped: the + framework enforces no phase order, and RFC-0001 replaces phases. + - Never Guess User Values becomes S3. + + New rules not in the Spec Kit constitution: + - S1, scope; + - S4, remediation does no harm; + - S5, the audited repository is untrusted; + - F2, one execution path; + - F3, shipped code can run; + - F4, declared dependencies; + - R1, merge gate; + - A1, ratchets. +8. **`framework-design.md` stops being "authoritative".** A later ADR on + consolidating the docs moves its normative content into rules, ADRs or + contract documents, its descriptive content into design notes, and deletes + what describes removed or out-of-scope features. Until then, + `scripts/validate_sync.py` keeps reading it. +9. **`governance/` is not a Spec Kit feature.** + - It is plain files, changed by pull requests, with no `specs/NNN` folder, + tasks or feature number. Spec Kit organizes work that ships behavior; + governance changes when a rule is wrong or a feature needs a decision, + and a feature stream would invent a backlog for it. + - A feature that needs a rule or an ADR records the need in its own spec, + and the rule lands here. + - Building a check is a plain pull request that cites its `ENF-` row and + links the pull request that shows it failing. +10. **RFCs and project governance are unchanged.** An RFC (`docs/rfcs/`) is a + proposal under discussion, and an accepted RFC becomes an ADR. + `CHARTER.md`, `GOVERNANCE.md` and the TSC process still decide who + decides. + +## Open question for the TSC + +R1 lets an author merge a change to a path CODEOWNERS does not own, after the +required checks and an independent agent review pass. GOVERNANCE.md says "one +maintainer approval required" for every pull request. The TSC decides between: + +- **R1 as written.** GOVERNANCE.md is amended to match. +- **A second maintainer on every pull request.** The agent review stays + advisory. + +Whichever it chooses, R1 falls back to a second maintainer on every pull +request until the `agent-review` check exists. + +## Alternatives + +**Keep the Spec Kit constitution as the rule source, and add enforcement ids +to it.** It is the file Spec Kit's commands rewrite, it is 347 lines, and it +mixes rules with process and history. + +**Make `framework-design.md` the rule source.** At 2,369 lines it is the +wallpaper a short constitution exists to avoid, and much of it describes code +that no longer exists. + +**Allow only tested rules.** That is the cleanest version, but some rules have +a judgment part no test reaches. Two examples: whether a new step's answer can +change without the setup changing, and whether a decision landed before the +change. Leaving those unwritten would leave them to memory. The agent and +human modes, with their conditions, keep them written down and small. + +**Instruct agents in `CLAUDE.md` only.** Contributors use agents that do not +read it. + +## Consequences + +- **Statements stop binding.** A MUST in a design doc or a spec no longer + binds; only a rule with a row does. +- **The table is mostly `planned`.** That is honest, but it means the rules + bind review now and CI later. +- **The first checks find real defects.** On `main` today, the planned checks + would flag: + - 6 unread parameters and flags; + - 201 type errors; + - 101 functions no test executes; + - 5 framework-name literals in core; + - a test suite that writes 377 files to HOME. + + Each check starts with today's findings as an allow-list that can only + shrink. That is work, spread over the build order in `ENFORCEMENT.md`. +- **Nobody merges their own change unreviewed.** + - On paths where a wrong merge is expensive, R1 requires the other + maintainer, and includes administrators. With two maintainers, that is a + real constraint on speed. + - Elsewhere, an independent agent review stands in for the second person, + and a weekly digest lets the maintainers check its calls after the + fact. +- **Blocked work stays visible.** Agents stop and file instead of editing a + rule, so some work waits for a person. + +## Enforcement + +This ADR creates the rules in `constitution.md`. Their rows are in +`ENFORCEMENT.md`. The `AGENTS.md` copy test and the Spec Kit pointer check are +part of ENF-A3's governance_lint. diff --git a/governance/decisions/README.md b/governance/decisions/README.md new file mode 100644 index 00000000..e97690bd --- /dev/null +++ b/governance/decisions/README.md @@ -0,0 +1,30 @@ +# Decisions + +One ADR per decision. ADRs are numbered and immutable once accepted: a decision +that turns out wrong gets a new ADR that supersedes it, never an edit in place. + +`../where-decisions-live.md` says when something belongs here rather than in +the constitution, a feature spec or a design note. In short, an ADR records +reasoning and a constitution rule creates an obligation. Most decisions need +the first and not the second. + +Proposals still under discussion are RFCs in `docs/rfcs/`. An accepted RFC +becomes an ADR here. + +## Numbering + +Files are named `NNNN-kebab-title.md` and numbered in order. A number is never +reused. + +## Status values + +- `proposed`: written, not yet agreed +- `accepted`: in force +- `superseded by ADR-NNNN`: replaced, with its text kept for the record + +## Index + +| ADR | Title | Status | +|-----|-------|--------| +| [0001](0001-what-darnit-is.md) | darnit aligns a project's setup with a hygiene goal | proposed | +| [0002](0002-governance-model.md) | Governance is a short constitution whose rules name their enforcement | proposed | diff --git a/governance/decisions/_template.md b/governance/decisions/_template.md new file mode 100644 index 00000000..9a772f96 --- /dev/null +++ b/governance/decisions/_template.md @@ -0,0 +1,34 @@ +# ADR-NNNN: + +- Status: proposed +- Date: YYYY-MM-DD +- Rule impact: <none, or the constitution rule this creates or changes> + +## Context + +What forced a decision: the real constraints, not the aspirational ones. + +## Scope + +How the decision passes ADR-0001's admission test: what setup it reads or +changes, why its answer stays the same when the setup does not change, and +what that exists today needs it. + +## Decision + +What was decided, in the fewest words that are still unambiguous. + +## Alternatives + +What else was considered, and the specific reason each lost. "Simpler" is not +a reason; say what the alternative could not do. + +## Consequences + +What this costs, including what it makes hard. An ADR with only upsides was not +a decision. + +## Enforcement + +The `ENF-` id and its row in `ENFORCEMENT.md`, or a statement that this ADR +creates no rule. Most ADRs create no rule. diff --git a/governance/where-decisions-live.md b/governance/where-decisions-live.md new file mode 100644 index 00000000..f9efd88c --- /dev/null +++ b/governance/where-decisions-live.md @@ -0,0 +1,117 @@ +# Where decisions live + +Read this before writing a rule, a spec or a design doc. Without it, every +decision drifts into whichever document was open, the documents contradict +each other, and nobody, human or agent, can tell which one binds. + +## The first question: does it serve the scope? + +Before deciding where a decision lives, apply +[ADR-0001](decisions/0001-what-darnit-is.md)'s admission test to what is being +decided: + +1. Does it read or change a project's setup: files, CI configuration, + platform settings or confirmed project data? +2. Would its answer stay the same across two runs with no change to that + setup, on any machine, at any time? +3. Does a control, a remediation or an interface that exists today need it? + A new extension point (a Protocol, entry-point group, registry or plugin + hook) needs two implementations, or an ADR that says why the seam is + needed before the second. + +A "no" to any of these means the work is out of scope, or premature. Stop, +and raise it in an issue before writing it down anywhere. + +## The three questions + +1. Is it a **rule** (something a change must satisfy) or a **decision** + (a choice with reasons)? +2. Can you **name the check** that enforces it? +3. What must **already exist** for that check to mean anything, and what does + the rule say before then? + +| | A test can check it | Only review can check it | +|---|---|---| +| **A rule for every change** | `constitution.md` rule, plus an `ENFORCEMENT.md` row in `test` mode | A constitution rule in `agent` or `human` mode, only if it meets the conditions in the constitution's "How rules are enforced". Otherwise an ADR, which creates no rule. | +| **A choice for one feature** | That feature's tests | That feature's `specs/NNN/plan.md` | + +Prefer a test. Most rules that look like they need judgment have a +deterministic part: an inventory, a count that can only shrink, or a +snapshot whose diff makes the change visible. Build that part as a test, and +leave only the remaining judgment to agents and maintainers. + +## The fourth question: can it fail? + +A check that cannot fail is a wish with a green tick. Before a check lands, +build the violation it claims to catch and watch the check fail on it. The +wiring audit of 2026-10-10 found this exact defect twice: + +- CI's `Type Check` job runs with `|| true`; +- its `tools.py` call-argument check cannot see a single darnit signature. + +Both looked like checks, and neither could fail. + +## The precondition test + +A rule that refers to something not yet built must say what it means until that +thing exists. Otherwise it either blocks work or quietly lies. Three shapes, in +order of preference: + +- **Vacuous satisfaction**: the condition is trivially met until the + precondition exists. +- **`dormant`**: the check runs, asserts nothing, and names what it waits for. +- **A named milestone**: `planned: scope-reset milestone`. + +## The naming test + +"Everything must be documented" fails, because there is no check for +"everything". Narrow the statement until a check exists ("every MCP tool +parameter is read by production code"), or move it down a row. + +## The destinations + +**Constitution rule.** It binds every change, it has a check, and violating it +matters. The bar is high, and the constitution should mostly stop growing. If +it keeps gaining rules, something that belongs in an ADR or a plan is leaking +upward. + +**ADR** (`governance/decisions/`). It records the reasoning behind a choice. +ADRs are numbered and immutable: a decision that turns out wrong gets a new ADR +that supersedes it, never an edit in place. Most decisions are an ADR and no +rule. An RFC (`docs/rfcs/`) is a proposal under discussion; when it is +accepted, the decision becomes an ADR. + +**Contract.** The framework TOML schema, the operator configuration schema, +the plugin protocol, the CLI, the MCP tool surface and the output formats. +The code that defines a contract (pydantic models, the Protocol, argparse, +the tool signatures) is normative, and prose describes it without superseding +it. A change follows C1. + +**Feature spec** (`specs/NNN-*/`). Spec Kit's `spec.md`, `plan.md` and +`tasks.md` for one feature. Technology choices and the design of a single +feature belong in `plan.md`, where they stay revisable. A spec cites the ADRs +and rules it depends on by number. It does not restate them. + +**Design note** (`design/`). Describes how a component works today. It is +descriptive: when it and the code disagree, the code is right and the note is +stale. A design note creates no obligation, so it uses no MUST or SHALL. + +**User docs** (`docs/`). How to install, use, write a framework or write a +plugin. Every command, flag, key and API a user doc tells someone to use must +exist. A doc that teaches a removed API is a bug. + +## Exceptions + +A change that cannot satisfy a rule gets an ADR that records the exception and +its reasoning. Never a silent suppression and never a weakened check (A1). + +## Governance changes during feature work + +Changes to `governance/` need a code owner's approval, and the TSC's where +the Charter requires it (A3). An agent whose task needs a governance change +stops and files an issue (A1). The work stays blocked until a maintainer makes +or approves the change. That is the design: a blocked task is visible, and a +rule quietly edited to fit is not. + +At every milestone boundary, ask one standing question: does anything in +`governance/` contradict what we just built?