From 8ea1eab3c257374e9aa5c90c639e3a429dc79f65 Mon Sep 17 00:00:00 2001 From: Michael Lieberman Date: Sat, 10 Oct 2026 11:36:36 -0400 Subject: [PATCH 1/7] governance: add constitution, enforcement table and ADRs 0001-0003 Draft for maintainer review. Adopts the swansong governance model: a short constitution whose rules each carry an enforcement id, an ENFORCEMENT.md recording each check's honest status, where-decisions-live.md, and immutable ADRs. - ADR-0001: darnit aligns a project's setup with a hygiene goal; it does not verify runtime behavior or tool outcomes. Admission test. - ADR-0002: governance model; Spec Kit constitution becomes a pointer; framework-design.md stops being authoritative; governance/ is not a Spec Kit feature. - ADR-0003: docs consolidation plan (contracts, design notes, user docs, archive, deletions) and the contradictions to decide first. Nothing outside governance/ changes in this commit. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Michael Lieberman --- governance/ENFORCEMENT.md | 57 +++++ governance/constitution.md | 117 ++++++++++ governance/decisions/0001-what-darnit-is.md | 155 ++++++++++++++ .../0002-governance-follows-swansong.md | 106 ++++++++++ .../decisions/0003-docs-consolidation.md | 199 ++++++++++++++++++ governance/decisions/README.md | 31 +++ governance/decisions/_template.md | 34 +++ governance/where-decisions-live.md | 109 ++++++++++ 8 files changed, 808 insertions(+) create mode 100644 governance/ENFORCEMENT.md create mode 100644 governance/constitution.md create mode 100644 governance/decisions/0001-what-darnit-is.md create mode 100644 governance/decisions/0002-governance-follows-swansong.md create mode 100644 governance/decisions/0003-docs-consolidation.md create mode 100644 governance/decisions/README.md create mode 100644 governance/decisions/_template.md create mode 100644 governance/where-decisions-live.md diff --git a/governance/ENFORCEMENT.md b/governance/ENFORCEMENT.md new file mode 100644 index 00000000..a095b701 --- /dev/null +++ b/governance/ENFORCEMENT.md @@ -0,0 +1,57 @@ +# Enforcement + +Every normative rule in `governance/` maps to a row here. Once ENF-A4's check +exists, CI fails when a MUST or MUST NOT carries an enforcement id with no row +here, or when a row names a check that does not exist. + +Status values (the same as swansong's): + +- `enforced`: the check exists and runs in CI. The row says in its text what the + check does not yet cover. +- `dormant`: the check exists and runs, but its precondition is unmet, so it + asserts nothing yet. The row names the precondition. The row becomes + `enforced` on its own once the precondition is met. +- `planned`: agreed but not yet written. The row names the milestone it lands + in. A row planned for two milestones is either written or struck. +- `unbuilt`: no check exists, and nobody has started one. +- `manual`: enforced by human review, and the row names who is responsible. + +**A caveat that applies to every row today.** `main`'s branch protection +requires a pull request but requires no status checks and no approvals. Until +ENF-R1 is enforced, an `enforced` row below means the check runs in CI and is +visible. It does not mean a failing run blocks a merge. + +| ID | Rule | Check | Failure mode | Status | +|----|------|-------|--------------|--------| +| ENF-S1 | Controls check setup, not outcomes | Manual: the reviewer of any pull request that adds or changes a control or step type in a shipped framework TOML or plugin applies ADR-0001's admission test. Planned check: every `exec` step and plugin step type in a shipped framework is listed in a reviewed inventory that records the setup it reads, and a step not in the inventory fails. | A control that runs a tool and grades its result, which brings false PASS risk and the tool's trust roots into darnit | manual (maintainers); check planned for the scope-reset milestone | +| ENF-S2 | Unverified is not compliant | CI `Test` job: `tests/darnit/tools/test_compliance_errors.py`, `tests/darnit/assertions/test_compliance_effects.py`, `tests/darnit/harness/test_harness_compliance.py`, and the adversarial corpus gate (`tests/darnit_baseline/corpus/`, `known_false_pass.toml`). Does not cover: a control dropped at load time while its level is still reported compliant (wiring audit W-18), or a list-valued `when` that makes a control N/A (W-05). | A false PASS, or a level reported compliant with an unverified control | enforced, with known gaps | +| ENF-S3 | Judgment values are never concluded | CI `Test` job: `tests/darnit/context_integrity/` (`test_consumers_use_usable`, `test_accept_candidates`, `test_confirmation_records`, `test_reads_do_not_write`). Does not cover: raw `.project/project.yaml` values satisfying a remediation `when` guard, or the `project.*` template namespace (W-09). | A guessed maintainer, contact or license used as if confirmed | enforced, with known gaps | +| ENF-S4 | Remediation writes only through the executor | CI `Test` job: `tests/darnit/context_integrity/test_single_writer.py`, `test_user_files_survive.py`. Does not cover: plugin remediation handlers registered with `supports_plan=False` that write for themselves (W-08). | A user's uncommitted work overwritten; files changed that the run manifest does not list | enforced, with known gaps | +| ENF-S5 | The audited repository is untrusted | CI `Test` job: `tests/darnit/config/operator/test_repository_cannot_configure.py`, `test_baseline_toml_removed.py`, and the trust tests under `tests/darnit/trust/`. Does not cover: `remediate_audit_findings` building its trust target from the checkout's origin remote (W-07). | A repository that configures its own audit, or grants itself trust | enforced, with known gaps | +| ENF-S6 | Steps conclude only within their effective set | CI `Test` job: `tests/darnit/config/test_strict_framework_loading.py`, the orchestrator dispatch tests, and the corpus gate (`plugin-redefines-manual` among its cases). Does not cover: the shared-handler cache (W-04), `gh_api` and `mcp` expressions decided by truthiness (W-16), or a plugin registering without its plugin context (W-17). | A PASS no step was allowed to give | enforced, with known gaps | +| ENF-F1 | Framework does not import implementations | Planned: an import check over `packages/darnit/src` that fails on `import darnit_` or `from darnit_`. | The framework works only with in-tree plugins | unbuilt | +| ENF-F2 | Controls live in TOML | CI `Spec Sync`: `scripts/validate_sync.py` (TOML schema, pass types, SARIF reads TOML). Strict loading rejects unknown control keys. | Control metadata or logic split between Python and TOML | enforced | +| ENF-F3 | No premature extension points | Manual: the reviewer asks for the second implementation or the ADR. | A seam with one implementation and no caller; the stores, adapters and pass registries found in the wiring audit | manual (maintainers) | +| ENF-F4 | Everything accepted has a reader | Planned: a test that walks the framework-schema and operator-config models, the declared step settings, the argparse tree and the registered MCP tool signatures, and requires each field, setting, flag and parameter to be read in `packages/*/src` outside its declaring module, against a reviewed allow-list that gives a reason for each entry. | A documented setting that silently does nothing (W-10, W-13, W-14, W-24, W-29 in the wiring audit) | planned: scope-reset milestone | +| ENF-F5 | Every function runs in a test; default implementations run end to end | Planned: per-function coverage from the CI test run, failing on any function in `packages/*/src` with no executed line unless it is in a reviewed allow-list; plus a test that lists each Protocol's shipped default implementation and requires a test through a real entry point. On 2026-10-10, 129 functions had no executed line. | Code nobody runs; a default implementation that has never worked (#574) | planned: scope-reset milestone | +| ENF-F6 | No new type errors | Planned: whole-repo mypy (`mypy_path` over every `packages/*/src`, pydantic plugin) against a committed baseline file, failing on any new error and on any `attr-defined`, `call-arg` or `name-defined` error. Validated locally on 2026-10-10: 193 baseline entries, about 7 seconds warm, and it fails when a real defect is taken out of the baseline. Today's CI `Type Check` job checks two files with `|| true` and gates nothing. | Missing methods and wrong call signatures shipping (#574, the attestation predicate, the CEL `except` clause) | planned: scope-reset milestone | +| ENF-C1 | Contracts change spec-first; breaking changes are listed | CI `Spec Sync` covers handler names and the TOML schema. The rest is manual: the reviewer checks that the ADR or spec landed with or before the code, and that the BREAKING entry is present. | A contract that changes with no recorded decision; a user broken with no notice | manual (maintainers), partly enforced | +| ENF-A1 | AI attribution | DCO check requires the human submitter's `Signed-off-by`. The rest is manual: the reviewer checks the `Assisted-by` trailer, the absence of AI co-author or sign-off lines, and the PR's AI disclosure. Planned: a commit-message check that fails on an AI `Co-authored-by` or `Signed-off-by`. | AI attribution that misstates who certified the change | manual, partly enforced | +| ENF-A2 | Governance changes are labelled | Planned: a workflow that fails any pull request touching `governance/` unless it carries the `tier-1` label, plus a CODEOWNERS entry naming both maintainers for `governance/`. | Rules changed by whoever was editing nearby | unbuilt | +| ENF-A3 | No weakening checks | Planned: a diff check that fails on an added `# noqa`, `# type: ignore`, `pytest.mark.skip` or `continue-on-error` outside a `tier-1` pull request. | A green build bought by lowering the bar | unbuilt | +| ENF-A4 | Every rule has a row | Planned: `scripts/governance_lint.py`, which requires every MUST in `governance/*.md` to carry an id, every id to have a row here, and every row to be cited by a rule. | Governance drifting back into prose | unbuilt | +| ENF-R1 | Merges go through required checks | `main` branch protection. Today it requires a pull request with zero approvals and no required status checks. Planned: require `Test`, `Lint`, `Spec Sync`, `DCO` and, once built, the ENF-F6 type gate. | A red pull request merged | unbuilt (protection exists, but requires no checks) | + +## Build order + +1. ENF-R1. Nothing else gates until the checks are required. +2. ENF-F6, ENF-A4 and ENF-A2. These are small and already designed, and the + first two stop new defects of the kind the wiring audit found. +3. ENF-F4 and ENF-F5. They will fail on day one; start them with the current + failures as their allow-list and burn it down. +4. ENF-F1 and ENF-A3. +5. ENF-S1's inventory check, once the scope-reset classification has decided + which steps stay. + +Prove each new check the way swansong does: open a pull request built to +violate it, watch the check fail, and close that pull request unmerged. diff --git a/governance/constitution.md b/governance/constitution.md new file mode 100644 index 00000000..cc167d86 --- /dev/null +++ b/governance/constitution.md @@ -0,0 +1,117 @@ +# Constitution + +These are the non-negotiable rules for this repository. Every rule carries an +enforcement id that maps to a row in `ENFORCEMENT.md`. A rule with no +enforcing check is a wish, not a rule, and it will be ignored. + +Changing a rule requires an ADR in `decisions/`. Do not edit a rule in place +without one. + +This document is deliberately short. Every agent session imports it, so each +rule here costs context in every session, forever. Before adding anything, +read `where-decisions-live.md`: most decisions belong somewhere else. + +What darnit is, and is not, is decided in +[ADR-0001](decisions/0001-what-darnit-is.md). The S rules below are the parts +of that decision a reviewer or a check can hold a change to. + +This document sits under the project's [Technical Charter](../CHARTER.md) and +[GOVERNANCE.md](../GOVERNANCE.md), which govern who decides. This document +governs what a change must satisfy. + +--- + +## 1. Scope + +**S1.** A control MUST check or change the setup of the project it audits: +its files, its CI configuration, its platform settings and its declared +project data. A control MUST NOT run a tool to judge that tool's result, or +judge the project's runtime behavior. Out of scope, for example: whether +history satisfies a gittuf policy, whether recent commits are signed, whether +a scanner comes back clean, whether a build is hermetic or reproducible. +Whether gittuf is set up, a scanner is configured to run, or a lockfile and a +provenance step exist is in scope. `[ENF-S1]` + +**S2.** Unverified is not compliant. A level is compliant only when every +applicable control at that level is PASS. WARN, ERROR, PENDING, an unhonored +claim and a control that failed to load all count against it. `[ENF-S2]` + +**S3.** darnit MUST NOT conclude a value that needs a person's judgment +(`auto_detect = false`). Detection may propose a candidate. Only a person's +confirmation makes a value usable, and an unusable value MUST NOT reach a +verdict, a remediation input, an attestation or a write. `[ENF-S3]` + +**S4.** Remediation MUST write only through the remediation executor, MUST +record every write in the run manifest, and MUST NOT write a path with +uncommitted user changes. `[ENF-S4]` + +**S5.** The audited repository is untrusted input. Nothing in it, including its +own configuration files and its version-control remotes, configures darnit or +grants trust. Trust and tool configuration come only from operator +configuration or verified CI metadata. `[ENF-S5]` + +**S6.** A step MUST conclude only an outcome in its effective set: its +registered ceiling, narrowed by `concludes`, widened only by a promotion that +the corpus backs with zero false PASS results. A model judgment MUST NOT +conclude PASS. No plugin MAY replace a core step type. `[ENF-S6]` + +--- + +## 2. Framework shape + +**F1.** The `darnit` package MUST NOT import an implementation package. The +framework reaches implementations only through plugin discovery and the +`ComplianceImplementation` protocol. `[ENF-F1]` + +**F2.** Controls are defined in a framework's TOML. Python adds step types, +remediation handlers and tools; it does not define controls. `[ENF-F2]` + +**F3.** No new extension point (a Protocol, entry-point group, registry, store +kind or plugin hook) until two implementations need it, or an ADR records why +the seam is needed before the second. `[ENF-F3]` + +**F4.** Everything darnit accepts MUST have a reader: every field of the +framework TOML schema and of operator configuration, every step setting, +every CLI flag and every MCP tool parameter is read by production code. A +setting that changes nothing is removed, not documented. `[ENF-F4]` + +**F5.** Every function in `packages/*/src` MUST be executed by a test, and +every shipped default implementation of a Protocol MUST be executed by a test +that reaches it through a real entry point (the CLI, an MCP tool or +`run_sieve_audit`), not through a test double. `[ENF-F5]` + +**F6.** A change MUST NOT add a type error. An `attr-defined`, `call-arg` or +`name-defined` error MUST NOT be added even to the baseline. `[ENF-F6]` + +--- + +## 3. Contracts + +**C1.** darnit's public contracts are: the framework TOML schema, the operator +configuration schema, the plugin protocol, the CLI, the MCP tool surface and +the output formats (JSON, SARIF, attestation predicate). A change to one MUST +land its ADR or feature spec before or with the code, and a breaking change +MUST appear under a **BREAKING** entry in `CHANGELOG.md`. `[ENF-C1]` + +--- + +## 4. Contributors and agents + +**A1.** AI-assisted contributions follow [AI_POLICY.md](../AI_POLICY.md): +attribution by an `Assisted-by` trailer, never an AI `Co-authored-by` or +`Signed-off-by`, and disclosure in the pull request. `[ENF-A1]` + +**A2.** `governance/` is tier-1. A change there MUST carry the `tier-1` label, +which records that a maintainer decided it, before it merges. An agent MUST NOT +change `governance/` unless a maintainer asked for that change. `[ENF-A2]` + +**A3.** A contributor (person or agent) who cannot satisfy a rule MUST stop and +file an issue describing the conflict. They MUST NOT weaken a check, skip a +test or add a suppression to make a run pass. `[ENF-A3]` + +**A4.** Every normative statement in `governance/` MUST carry an enforcement id +with a row in `ENFORCEMENT.md`. If you cannot name the check, you do not have a +rule. `[ENF-A4]` + +**R1.** A change MUST reach `main` only through a pull request whose required +checks passed on its head. `[ENF-R1]` diff --git a/governance/decisions/0001-what-darnit-is.md b/governance/decisions/0001-what-darnit-is.md new file mode 100644 index 00000000..159d696b --- /dev/null +++ b/governance/decisions/0001-what-darnit-is.md @@ -0,0 +1,155 @@ +# ADR-0001: darnit aligns a project's setup with a hygiene goal + +- Status: proposed +- Date: 2026-10-10 +- Rule impact: creates S1. Restates the existing conservative, trust and + authority principles as S2, S3, S5 and S6. + +## Context + +The maintainers have described darnit's purpose many times, but never in one +place a change could be held to. Without that statement, two kinds of work +kept entering the project, each reasonable on its own. + +The first kind is runtime verification: checks that run a tool and grade what +it says. + +- `gittuf_commits_signed` parses `git log %G?`. It turned a fabricated + signature into a dispositive PASS (wiring audit W-01). +- Witness runtime traces were Sigstore-verified to decide whether a build + touched the network (#553). +- Reproducibility checks infer hermeticity and bit-for-bit output. +- zizmor findings decide CI controls. + +Each brings that tool's trust roots and failure modes into darnit, and each is +a place where darnit can say PASS about something it cannot know. + +The second kind is generic interfaces ahead of any user: check and remediation +adapters, a locator, a storage backend, four store protocols, pass and template +registries, shared handlers, budgets, enhancers. The 2026-10-10 wiring audit +found 149 defects, most of them in exactly this kind of code. It had been +declared, documented and validated, but never connected. #487 removed about +8,000 lines of it, and more remains. + +darnit is an LF project with a [Technical Charter](../../CHARTER.md). This ADR +is not a charter. It decides what the software is for. + +## Decision + +**darnit is a framework for aligning a project's setup with a hygiene goal.** + +A hygiene goal is a published set of practices a project should have in place: +OpenSSF Baseline, the Community Specification, "use gittuf", "be reproducible". +darnit reads a project's **setup**, reports where it falls short of the goal, +and helps set up what is missing. + +Setup means: + +- **Files:** SECURITY.md, LICENSE, CODEOWNERS, lockfiles, a gittuf policy. +- **CI configuration:** which workflows run, what they run, and with what + permissions. +- **Platform settings:** branch protection, required reviews, 2FA + requirements, read through the platform API. +- **Declared project data:** `.project/`, confirmed by a person where a value + needs judgment (S3). + +**darnit does not verify runtime behavior or tool outcomes.** It checks that a +mechanism is set up and in use. Whether the mechanism produces the right +answer at any given moment is the job of the mechanism. + +| Goal | In scope: setup | Out of scope: outcome | +|------|-----------------|-----------------------| +| gittuf | gittuf is initialized; a policy exists with a root of trust; the policy is well formed; CI runs gittuf verification | whether history satisfies the policy; whether recent commits are signed | +| OpenSSF Baseline | SECURITY.md, LICENSE and contribution docs exist; branch protection and review settings are configured; security testing and dependency scanning run in CI | whether today's scan results are clean; whether the security tests pass | +| Reproducibility | dependencies are pinned in lockfiles; the build environment is declared; CI produces signed provenance | whether a build is hermetic at runtime; whether two builds are bit-for-bit identical | + +### The framework + +To serve that purpose, the framework provides: + +- **Evidence readers**: repository files, parsed CI configuration, platform + settings (read-only) and confirmed project data. +- **Checks**: declarative, defined in framework TOML. They produce PASS, FAIL, + or a result that is not yet known together with the reason it is not known. +- **Remediation**: a plan that is previewed, approved and then applied by a + single writer, which sets up what is missing. +- **Plugins**: hygiene goals (frameworks), and integrations that teach darnit + how to read and set up one tool's configuration. +- **Interfaces**: the CLI, MCP tools and agent skills, as thin shells over one + execution path. + +### The admission test + +A control, step type, interface, configuration key or subsystem is admitted only +if all three answers are yes: + +1. Does it read or change a project's setup? +2. Would its answer stay the same across two runs with no change to that setup? +3. Does something that exists today need it? + +Question 2 catches outcome checks: the answer to "are the scans clean" changes +without anyone touching the setup. Question 3 catches premature interfaces (F3). + +## Alternatives + +**darnit as a verifier.** It would run gittuf verification, scanners, +reproducibility builds and Witness, and grade the results. This was rejected +for four reasons: + +- It duplicates tools that already do this and do it better. +- It makes darnit trust what those tools trust, which brings in signing keys, + Sigstore roots, monitors and network access. +- Its answers change without the project changing. +- It is where false PASS results come from. W-01 is a dispositive PASS for a + forged signature. + +**Keep outcome checks, but as advisory evidence.** Feature 044 took this route +for the reproducibility checks, and #565 did for Witness. It was rejected: + +- An advisory check still has to be maintained, tested and kept correct. +- It still shows up in reports. +- It still invites the next change that promotes it to deciding. + +**No written scope; rely on review.** This is the status quo, and the audit +measured its result. + +## Consequences + +- **Some controls are cut or reframed.** A scope-reset classification will + mark every control, step type and subsystem as keep, reframe or cut, for the + maintainers to decide. Expected outcomes: + - gittuf GT-02.01 is cut. + - GT-01.02 is reframed from "policy passes verification" to "the policy is + well formed and CI runs gittuf verification". + - Witness verification and RE-03.01 are cut. + - RE-02.01 is reframed to the parts that read build configuration. + - The Baseline controls that grade zizmor findings need a decision. Reading + a workflow file for a dangerous trigger is setup; running a scanner and + grading its findings is an outcome. + - Two areas are borderline and need a decision: + - The threat-model analysis engine (features 006, 010, 011 and 014: + tree-sitter discovery and opengrep taint analysis). Producing a threat + model document as a remediation is setup. An analysis engine that grades + code is not. + - Feature 023's Scorecard wiring, because several Scorecard checks are + outcome checks. +- **Some audit findings close by deletion, not repair.** W-01 is one; several + P1s about unused interfaces are others. +- **Some OSPS controls are phrased as outcomes.** For those, darnit reports + whether the practice is set up. Reports and the attestation predicate must + say "set up for", not "compliant with", wherever that is the truth. That is + a wording and predicate change, decided separately. +- **Fewer features.** Some contributions that work and were tested will be + declined because they verify an outcome. The PR template and the S1 review + make that visible before the work is done, not after. +- **Model judgments (`llm_eval`) are still allowed,** for a narrow use: whether + a file's content says what a setup practice requires, for example whether + SECURITY.md names a reporting channel. They remain PASS candidates that a + person confirms. That is still setup. + +## Enforcement + +- **S1 (`ENF-S1`):** manual review now. A planned inventory check covers every + `exec` step and plugin step type in a shipped framework. +- **S2 and S3:** restate existing tested rules (`ENF-S2`, `ENF-S3`). +- **F3, F4, F5:** cover question 3 of the admission test. diff --git a/governance/decisions/0002-governance-follows-swansong.md b/governance/decisions/0002-governance-follows-swansong.md new file mode 100644 index 00000000..637b9c93 --- /dev/null +++ b/governance/decisions/0002-governance-follows-swansong.md @@ -0,0 +1,106 @@ +# ADR-0002: Governance is a short enforced constitution, ADRs and an enforcement table + +- Status: proposed +- Date: 2026-10-10 +- Rule impact: creates `governance/` and rules A1-A4 and R1. Retires the Spec + Kit constitution as a separately edited document. + +## Context + +darnit's rules live in several places that disagree: + +- **The Spec Kit constitution** (`.specify/memory/constitution.md`, version + 1.3.2, 347 lines). +- **`CLAUDE.md`** (415 lines, about half of it feature changelog). +- **`docs/architecture/framework-design.md`** (2,369 lines, titled "the + authoritative specification"). +- **About 22,000 lines of docs**, many of which still describe removed or + never-built behavior. +- **`AI_POLICY.md`.** + +None of these documents says which of its statements a check enforces. The +wiring audit found rules that nothing enforced: dead code, single writer, +confirmation. It also found a CI type check that could not fail. + +The swansong project governs itself with a short constitution in which every +rule carries an enforcement id, an `ENFORCEMENT.md` that records each check's +honest status, immutable ADRs for reasoning, and a document that says where +each kind of decision lives. The maintainers asked darnit to follow it. + +## Decision + +1. **`governance/` is normative and tier-1.** It holds: + - `constitution.md`: the rules; + - `ENFORCEMENT.md`: one row per rule; + - `where-decisions-live.md`; + - `decisions/`: the ADRs. +2. **Every rule has an enforcement id and a row.** A statement no check can + enforce is an ADR, not a rule. +3. **The Spec Kit constitution becomes a pointer to `governance/`,** so + `/speckit.*` commands read the same rules. Nobody edits it directly, and + nobody runs `/speckit.constitution`. Its principles map to rules: + - Plugin Separation becomes F1. + - Conservative-by-Default becomes S2. + - TOML-First becomes F2. + - Never Guess User Values becomes S3. + - Sieve Pipeline Integrity becomes S6, the authority ceiling. Its + "4-phase ordering MUST be respected" clause is dropped: the spec already + says the framework enforces no phase order, and RFC-0001 replaces + phases. + - New rules not in the Spec Kit constitution: S1 (scope), S4 (remediation + does no harm, from framework-design section 15) and S5 (the audited + repository is untrusted, from section 14). +4. **`framework-design.md` stops being "authoritative".** + - Its normative content becomes rules, ADRs or the contracts (C1). + - Its descriptive content becomes design notes under `design/`. + - Content about removed or out-of-scope features is deleted. + - The docs consolidation plan (a later ADR) says where each section goes. + - Until that plan lands, `scripts/validate_sync.py` keeps reading the + handler-name registry from it. +5. **`CLAUDE.md` imports the constitution.** It keeps routing ("read X before + touching Y") and a short "never" list. The feature changelog + ("Active Technologies", "Recent Changes") is removed, because git and + `CHANGELOG.md` already record it. +6. **`governance/` is not a Spec Kit feature.** It is plain files changed by + tier-1 pull requests, with no `specs/NNN` folder, no tasks and no feature + number. This follows swansong's ADR-0006. Spec Kit organizes work that + ships behavior, as specify, plan, tasks, implement. Governance changes when + a rule turns out wrong, or a feature needs a decision. It has no backlog of + its own, and a feature stream would invent one. A feature that needs a + rule or ADR records the need in its own spec, and the rule lands here. + Building an enforcement check is code: a plain pull request that cites + its `ENF-` row and shows the check failing on a built violation. It becomes + a Spec Kit feature only if it is large enough to need a plan. +7. **RFCs stay where they are.** An RFC (`docs/rfcs/`) is a proposal under + discussion; once accepted, it is recorded as an ADR. +8. **Project governance is unchanged.** `CHARTER.md`, `GOVERNANCE.md` and the + TSC process keep deciding who decides. Adopting this ADR is an operational + change made by maintainer consensus under `GOVERNANCE.md`. + +## Alternatives + +- **Keep the Spec Kit constitution as the rule source and add enforcement ids + to it.** It stays the place Spec Kit's own commands rewrite, it is 347 + lines, and it mixes rules with process and history. Rejected. +- **Make `framework-design.md` the rule source.** At 2,369 lines it is the + wallpaper a short constitution exists to avoid, and much of it describes + code that no longer exists. Rejected. + +## Consequences + +- **Many current statements stop being rules.** A MUST in a design doc or a + spec no longer binds; only a rule with a row does. That is the point, but it + means reading `ENFORCEMENT.md` honestly. Most of darnit's checks are + `planned` or `unbuilt` today, and CI blocks no merge until ENF-R1 is + enforced. +- **The work is real.** Building checks F4, F5 and F6, consolidating the docs, + and requiring CI on `main` are each more than a day of work. +- **Speed changes.** Agents that need a governance change stop and file an + issue instead of editing the rule. That is slower, and the slowdown is + deliberate. + +## Enforcement + +- **This ADR creates A1-A4 and R1.** Their rows are in `ENFORCEMENT.md`. +- **The Spec Kit pointer is planned under ENF-A4.** `governance_lint` will + check that `.specify/memory/constitution.md` contains only the pointer. diff --git a/governance/decisions/0003-docs-consolidation.md b/governance/decisions/0003-docs-consolidation.md new file mode 100644 index 00000000..241e00c3 --- /dev/null +++ b/governance/decisions/0003-docs-consolidation.md @@ -0,0 +1,199 @@ +# ADR-0003: Consolidate the docs into governance, contracts, design notes and user docs + +- Status: proposed +- Date: 2026-10-10 +- Rule impact: none directly. It moves normative text under A4, so every MUST + that survives carries an enforcement id. + +## Context + +The repository has about 28,500 lines of active documentation in more than 100 +files: + +- root documents: 4,469 lines; +- `docs/`: 22,691 lines; +- package READMEs and skills: about 1,000 lines; +- the Spec Kit constitution: 347 lines. + +Separately, `specs/` holds about 44,700 lines of Markdown. + +Normative statements (MUST, SHALL, "authoritative") are spread across about 30 +files, and three of those files call themselves authoritative: + +- the Spec Kit constitution; +- `docs/architecture/framework-design.md`, which alone has 265 normative + statements; +- `docs/architecture/README.md`. + +An inventory on 2026-10-10 found nine places where these documents contradict +each other or the code: + +- the phase ordering; +- whether unknown step keys load; +- whether WARN is remediated; +- a deleted `rules/catalog.py` described as live; +- stale handler lists; +- `.baseline.toml` compatibility; +- `[plugins]` placement; +- `[framework]` versus `[metadata]`; +- a stale Spec Kit pointer. + +Many user docs still teach removed APIs. Some examples: `DeterministicPass`, +`main.py`, `baseline-mcp`, `PENDING_LLM`, and "62 controls / v2025.10.10" +(the baseline now has 66 controls, at v2026.02.19). + +## Decision + +### Layout + +``` +governance/ normative, tier-1 + constitution.md + ENFORCEMENT.md + where-decisions-live.md + contracts/ behavior every implementation and plugin must keep + results-and-authority.md outcomes, ceilings, disposition, compliance, corpus + framework-toml.md schema, strict loading, step and remediation fields, CEL, when + context-values.md standing, confirmation, lapse, single writer + remediation-safety.md plan/apply, approval, platform engine, outcomes, manifest, VCS + operator-config-and-trust.md operator config, trust, claims, plugin trust + plugin-protocol.md protocol members, registration, module-path policy, packaging + decisions/ ADRs +design/ descriptive notes on how components work today; no MUST + overview.md, sieve.md, audit-pipeline.md, context.md, reporting.md, + remediation-engine.md, stores.md, agent-graph.md, workflow.md +docs/ user and author docs; every command and key they name exists (F4) + install/ user/ plugin-authoring/ contributing/ archive/ +``` + +### Rules for this layout + +- **Contracts.** A contract keeps only statements that carry an enforcement + id (A4). An unenforced MUST becomes either a row with status `planned` or + `unbuilt`, or descriptive text in `design/`. That keeps the contracts from + regrowing into another 2,369-line specification. +- **Specs.** `specs/NNN-*/` folders become implementation records. A spec's + open tasks are either finished or closed with a note. Specs are not + rewritten. +- **Dogfood files stay where they are.** These files are darnit's own + compliance evidence: + - in `docs/`: `DEPENDENCIES.md`, `SAST-POLICY.md`, `SCA-POLICY.md`, + `SECRETS-POLICY.md`, `SECURITY-ASSESSMENT.md`, `TEST-REQUIREMENTS.md`, and + `threatmodel/`; + - at the root: `SECURITY.md`, `THREAT_MODEL.md`, `CONTRIBUTING.md`, + `GOVERNANCE.md`, `README.md`, `SUPPORT.md`. + + `openssf-baseline.toml` and `.project/project.yaml` look for them at these + exact paths. +- **LF documents are unchanged.** `CHARTER.md`, `GOVERNANCE.md`, + `TECHNICAL-STEERING-COMMITTEE.md`, `AI_POLICY.md` and `CHANGELOG.md` stay as + they are. The one change is that GOVERNANCE.md's "breaking changes require an + RFC" now points at the RFC-then-ADR flow. + +### `framework-design.md` + +The specification is about 75% contract, 15% description and 10% history or +out-of-scope material. Its section numbers survive as anchors in the new +contracts, because test docstrings cite them. + +| Sections | Goes to | +|---|---| +| 1 Purpose, philosophy | replaced by ADR-0001 and the constitution | +| 1.3 diagram, 5.1 and 5.3 execution model, 11.1-11.3, 12, 13 | `design/` | +| 2 schema, 3.2-3.8 field and ceiling tables, 4.3, 4.4, 4.6, 4.7 | `contracts/framework-toml.md` | +| 3.0-3.1 outcomes, ceilings, disposition, order; 5.2, 5.4, 5.5 | `contracts/results-and-authority.md` (without the Witness passages) | +| 4.1, 4.2, 4.5, 15 | `contracts/remediation-safety.md` | +| 6.3-6.5, 10.1-10.3, 10.5 | `contracts/plugin-protocol.md` | +| 6.6, 14 | `contracts/operator-config-and-trust.md` | +| 7.4-7.11 | `contracts/context-values.md` | +| 7.1-7.3, 4.8, 6.1-6.2, Appendix A | `docs/plugin-authoring/` | +| 8 SARIF mapping | `design/reporting.md` | +| 9 sync enforcement | `ENFORCEMENT.md`. Its check finds 8 hard-coded handler names in the file, so it enforces almost nothing. | +| Appendix B (rules catalog migration) | deleted; the catalog no longer exists | +| Appendix C, version history | context for the ADRs below; otherwise removed (git keeps it) | + +### Everything else + +| Disposition | Files | +|---|---| +| Into `governance/contracts/` | `docs/architecture/`: `cel-expressions`, `conditional-controls`, `control-dependencies`, `external-templates`, `shared-handlers` (if shared handlers survive), `remediation-manual-guidance`, `github-api-remediation`, `remediation-audit-filtering` (after the WARN contradiction is decided), and the rule from `implementation-provided-tools` | +| Into `design/` | `ARCHITECTURE.md` (as `overview.md`); `docs/architecture/`: `audit-pipeline`, `audit-context-collection`, `context-collection`, `dot-project-integration`, `org-project-resolution`, `repo-identity-resolution`, `framework-agnostic-reporting`; `docs/`: `WORKFLOW.md`, `agent-graph.md`, a short version of `design/CONTEXT_SIEVE_DESIGN.md` | +| Into the darnit-baseline package docs | `ci-workflow-templates`, `declarative-file-templates`, `policy-doc-templates`, and `ATTESTATION_SPEC.md` (refreshed as the predicate contract) | +| Merged into `docs/plugin-authoring/` | `IMPLEMENTATION_GUIDE.md` (cut by about half), `HANDLER_AUTHORING.md`, `CEL_CONTEXT.md` together with `getting-started/cel-reference.md`, `getting-started/implementation-development.md`, both tutorials, `packaging-plugins.md` (the entry point), `plugin-authoring/stores.md` | +| Merged into `docs/contributing/` | `GETTING_STARTED.md`, `getting-started/{README,framework-development,development-workflow,environment-setup,testing,troubleshooting}`, the MCP development section of CONTRIBUTING.md | +| Into `docs/user/` | `using-skills`, `SECURITY_GUIDE.md` (linking the contracts instead of restating them), `MIGRATION_GUIDE.md`, `CNCF_METADATA.md`, `CSL_ONBOARDING.md` | +| Recorded as ADRs | `docs/decisions/cel-library-evaluation.md`; the accepted parts of RFC-0001; and `docs/design/builtin-tool-factories.md` as a proposed ADR once its owner wants it reviewed | +| `docs/archive/` (excluded from `test_doc_step_examples`) | `DECISION_FLOWS.md`, `ARCHITECTURE_RESEARCH_BRIEF.md`, `TEST_REPO_DESIGN.md`, `plugin-discovery-design.md`, `SARIF_DESIGN.md`, `design/CONTEXT_PROMPTS.md`, `design/cncf-project-spec-analysis.md` | +| Deleted | `design/reproducibility-attestation-system.md` (2,131 lines; out of scope under ADR-0001); `USAGE_GUIDE.md`; `GOVERNANCE_DOCS_DESIGN.md`; `TODO.md` (its items move to issues); `docs/examples/`; `docs/architecture/`: `context-documentation`, `sieve-handler-authoring`, `handler-pipeline`, `plugin-registry`, `README` | +| Rewritten in place | `README.md` (to about 300 lines, describing the scope as ADR-0001 states it); `CLAUDE.md` (to about 100 lines: import the constitution, routing, AI rules, test and lint commands); the package READMEs (control counts; the gittuf and reproducibility scope) | + +Estimated result: about 12,700 lines of active docs, down from about 28,500, +with about 2,950 of them normative and in one place, instead of about 5,800 +spread across 30 files. + +### Decisions needed before contract text is written + +1. Is phase ordering enforced? The proposal is no: dispatch follows the + step list. Constitution V asserted otherwise. +2. Do unknown step keys load? They fail loading since feature 044. + `handler-pipeline.md` is stale. +3. Is WARN remediated? `framework-design.md` 10.4 says only FAIL is. + `remediation-audit-filtering.md` says everything not PASS. The proposal + is FAIL only, which is what the code does. +4. Does `[plugins]` belong in framework TOML at all? The proposal is no: it + moves to operator config, and the unused framework-schema field is + removed (wiring audit CONFIG-18). +5. RFC-0001 promises that `.baseline.toml` keeps loading. That was superseded + by #556, and is recorded as such in the ADR that records RFC-0001's + accepted parts. + +### ADRs to backfill + +These record decisions that already shipped, so that the contracts can cite +them instead of restating the reasoning: + +- CEL via cel-python +- authority per claim (RFC-0001 stages 0-1, features 041 and 044) +- propose-only judgment keys (features 018 and 042) +- the audited repository is untrusted, and `.baseline.toml` is removed (040, #556) +- remediation plan/apply and `platform_setting` replacing `api_call` (043) +- removal of dead subsystems and the protocol trim (#487, #451) +- PENDING and ERROR statuses (036, 041) + +### Order of work + +1. ADR-0001, ADR-0002 and this ADR are accepted, after the maintainers decide + the open items. +2. CLAUDE.md is rewritten and the Spec Kit constitution becomes a pointer. + This is small, and it stops agents from reading stale rules. +3. The contracts are written from `framework-design.md`, one file per pull + request. `validate_sync.py`, the PR template and the test docstrings + follow. +4. `design/` notes are written and moved. +5. The user docs are merged, archived, deleted and rewritten. The doc-example + test is extended so that it also runs over the merged docs. + +## Alternatives + +**Keep `framework-design.md` and fix it.** It would keep being the place where +every feature appends a section. Rejected for the reasons in ADR-0002. + +**Treat the tests as the only contract and make all prose descriptive.** This +is honest, but plugin authors and reviewers then have no readable statement of +the behavior they must keep. Rejected in favor of contracts restricted to +enforced statements. + +## Consequences + +- **Links break.** Many links and test-docstring citations change. Keeping the + section numbers as anchors limits the breakage, but some will still be + missed. +- **Removed docs lose history in place.** Their history stays in git and + `docs/archive/`. +- **The work is large.** About five pull requests for the contracts, and two + or three for the user docs. + +## Enforcement + +None new. Contracts are subject to A4 (`ENF-A4`) and user docs to F4 +(`ENF-F4`). diff --git a/governance/decisions/README.md b/governance/decisions/README.md new file mode 100644 index 00000000..5567e135 --- /dev/null +++ b/governance/decisions/README.md @@ -0,0 +1,31 @@ +# Decisions + +One ADR per decision. ADRs are numbered and immutable once accepted: a decision +that turns out wrong gets a new ADR that supersedes it, never an edit in place. + +`../where-decisions-live.md` says when something belongs here rather than in +the constitution, a feature spec or a design note. In short, an ADR records +reasoning and a constitution rule creates an obligation. Most decisions need +the first and not the second. + +Proposals still under discussion are RFCs in `docs/rfcs/`. An accepted RFC +becomes an ADR here. + +## Numbering + +Files are named `NNNN-kebab-title.md` and numbered in order. A number is never +reused. + +## Status values + +- `proposed`: written, not yet agreed +- `accepted`: in force +- `superseded by ADR-NNNN`: replaced, with its text kept for the record + +## Index + +| ADR | Title | Status | +|-----|-------|--------| +| [0001](0001-what-darnit-is.md) | darnit aligns a project's setup with a hygiene goal | proposed | +| [0002](0002-governance-follows-swansong.md) | Governance is a short enforced constitution, ADRs and an enforcement table | proposed | +| [0003](0003-docs-consolidation.md) | Consolidate the docs into governance, contracts, design notes and user docs | proposed | diff --git a/governance/decisions/_template.md b/governance/decisions/_template.md new file mode 100644 index 00000000..9a772f96 --- /dev/null +++ b/governance/decisions/_template.md @@ -0,0 +1,34 @@ +# ADR-NNNN: + +- Status: proposed +- Date: YYYY-MM-DD +- Rule impact: <none, or the constitution rule this creates or changes> + +## Context + +What forced a decision: the real constraints, not the aspirational ones. + +## Scope + +How the decision passes ADR-0001's admission test: what setup it reads or +changes, why its answer stays the same when the setup does not change, and +what that exists today needs it. + +## Decision + +What was decided, in the fewest words that are still unambiguous. + +## Alternatives + +What else was considered, and the specific reason each lost. "Simpler" is not +a reason; say what the alternative could not do. + +## Consequences + +What this costs, including what it makes hard. An ADR with only upsides was not +a decision. + +## Enforcement + +The `ENF-` id and its row in `ENFORCEMENT.md`, or a statement that this ADR +creates no rule. Most ADRs create no rule. diff --git a/governance/where-decisions-live.md b/governance/where-decisions-live.md new file mode 100644 index 00000000..adc807ac --- /dev/null +++ b/governance/where-decisions-live.md @@ -0,0 +1,109 @@ +# Where decisions live + +Read this before writing a rule, a spec or a design doc. Without it, every +decision drifts into whichever document was open, the documents contradict +each other, and nobody, human or agent, can tell which one binds. + +Adapted from swansong's `governance/where-decisions-live.md`. + +## The first question: does it serve the scope? + +Before deciding where a decision lives, apply +[ADR-0001](decisions/0001-what-darnit-is.md)'s admission test to what is being +decided: + +1. Does it read or change a project's setup: files, CI configuration, + platform settings or declared project data? +2. Would its answer stay the same across two runs with no change to that setup? +3. Does a control, a remediation or an interface that exists today need it? + +A "no" to any of these means the work is out of scope, or premature. Stop, +and raise it in an issue before writing it down anywhere. + +## The three questions + +1. Is it a **rule** (something a change must satisfy) or a **decision** + (a choice with reasons)? +2. Can you **name the check** that enforces it? +3. What must **already exist** for that check to mean anything, and what does + the rule say before then? + +| | Enforceable by a check | Not enforceable | +|---|---|---| +| **A rule for every change** | `constitution.md` rule plus an `ENFORCEMENT.md` row | ADR only; it creates no rule | +| **A choice for one feature** | That feature's tests | That feature's `specs/NNN/plan.md` | + +## The fourth question: can it fail? + +A check that cannot fail is a wish with a green tick. Before a check lands, +build the violation it claims to catch and watch the check fail on it. The +wiring audit of 2026-10-10 found this exact defect twice: + +- CI's `Type Check` job runs with `|| true`; +- its `tools.py` call-argument check cannot see a single darnit signature. + +Both looked like checks, and neither could fail. + +## The precondition test + +A rule that refers to something not yet built must say what it means until that +thing exists. Otherwise it either blocks work or quietly lies. Three shapes, in +order of preference: + +- **Vacuous satisfaction**: the condition is trivially met until the + precondition exists. +- **`dormant`**: the check runs, asserts nothing, and names what it waits for. +- **A named milestone**: `planned: scope-reset milestone`. + +## The naming test + +"Everything must be documented" fails, because there is no check for +"everything". Narrow the statement until a check exists ("every MCP tool +parameter is read by production code"), or move it down a row. + +## The destinations + +**Constitution rule.** It binds every change, it has a check, and violating it +matters. The bar is high, and the constitution should mostly stop growing. If +it keeps gaining rules, something that belongs in an ADR or a plan is leaking +upward. + +**ADR** (`governance/decisions/`). It records the reasoning behind a choice. +ADRs are numbered and immutable: a decision that turns out wrong gets a new ADR +that supersedes it, never an edit in place. Most decisions are an ADR and no +rule. An RFC (`docs/rfcs/`) is a proposal under discussion; when it is +accepted, the decision becomes an ADR. + +**Contract.** The framework TOML schema, the operator configuration schema, +the plugin protocol, the CLI, the MCP tool surface and the output formats. +The code that defines a contract (pydantic models, the Protocol, argparse, +the tool signatures) is normative, and prose describes it without superseding +it. A change follows C1. + +**Feature spec** (`specs/NNN-*/`). Spec Kit's `spec.md`, `plan.md` and +`tasks.md` for one feature. Technology choices and the design of a single +feature belong in `plan.md`, where they stay revisable. A spec cites the ADRs +and rules it depends on by number. It does not restate them. + +**Design note** (`design/`). Describes how a component works today. It is +descriptive: when it and the code disagree, the code is right and the note is +stale. A design note creates no obligation, so it uses no MUST or SHALL. + +**User docs** (`docs/`). How to install, use, write a framework or write a +plugin. Every command, flag, key and API a user doc tells someone to use must +exist (F4). A doc that teaches a removed API is a bug. + +## Exceptions + +A change that cannot satisfy a rule gets an ADR that records the exception and +its reasoning. Never a silent suppression and never a weakened check (A3). + +## Governance changes during feature work + +`governance/` is tier-1 (A2). An agent whose task needs a governance change +stops and files an issue (A3). The work stays blocked until a maintainer makes +or approves the change. That is the design: a blocked task is visible, and a +rule quietly edited to fit is not. + +At every milestone boundary, ask one standing question: does anything in +`governance/` contradict what we just built? From 99d9fb9b52d69e5406cb1d1a8b50229cfe7e9aa2 Mon Sep 17 00:00:00 2001 From: Michael Lieberman <mlieberman85@gmail.com> Date: Mon, 12 Oct 2026 14:11:04 -0400 Subject: [PATCH 2/7] governance: S rows are planned until CI runs the whole suite (#591, #592) Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Michael Lieberman <mlieberman85@gmail.com> --- governance/ENFORCEMENT.md | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/governance/ENFORCEMENT.md b/governance/ENFORCEMENT.md index a095b701..c9945b3e 100644 --- a/governance/ENFORCEMENT.md +++ b/governance/ENFORCEMENT.md @@ -16,6 +16,11 @@ Status values (the same as swansong's): - `unbuilt`: no check exists, and nobody has started one. - `manual`: enforced by human review, and the row names who is responsible. +**Tests are not yet run in CI.** Until #592 merges, the CI `Test` job +selects tests only by the `unit` and `integration` markers. 2,754 of 5,164 +tests carry neither marker, and that includes every test the S rows cite +(#591). The S rows are `planned` until then. + **A caveat that applies to every row today.** `main`'s branch protection requires a pull request but requires no status checks and no approvals. Until ENF-R1 is enforced, an `enforced` row below means the check runs in CI and is @@ -24,11 +29,11 @@ visible. It does not mean a failing run blocks a merge. | ID | Rule | Check | Failure mode | Status | |----|------|-------|--------------|--------| | ENF-S1 | Controls check setup, not outcomes | Manual: the reviewer of any pull request that adds or changes a control or step type in a shipped framework TOML or plugin applies ADR-0001's admission test. Planned check: every `exec` step and plugin step type in a shipped framework is listed in a reviewed inventory that records the setup it reads, and a step not in the inventory fails. | A control that runs a tool and grades its result, which brings false PASS risk and the tool's trust roots into darnit | manual (maintainers); check planned for the scope-reset milestone | -| ENF-S2 | Unverified is not compliant | CI `Test` job: `tests/darnit/tools/test_compliance_errors.py`, `tests/darnit/assertions/test_compliance_effects.py`, `tests/darnit/harness/test_harness_compliance.py`, and the adversarial corpus gate (`tests/darnit_baseline/corpus/`, `known_false_pass.toml`). Does not cover: a control dropped at load time while its level is still reported compliant (wiring audit W-18), or a list-valued `when` that makes a control N/A (W-05). | A false PASS, or a level reported compliant with an unverified control | enforced, with known gaps | -| ENF-S3 | Judgment values are never concluded | CI `Test` job: `tests/darnit/context_integrity/` (`test_consumers_use_usable`, `test_accept_candidates`, `test_confirmation_records`, `test_reads_do_not_write`). Does not cover: raw `.project/project.yaml` values satisfying a remediation `when` guard, or the `project.*` template namespace (W-09). | A guessed maintainer, contact or license used as if confirmed | enforced, with known gaps | -| ENF-S4 | Remediation writes only through the executor | CI `Test` job: `tests/darnit/context_integrity/test_single_writer.py`, `test_user_files_survive.py`. Does not cover: plugin remediation handlers registered with `supports_plan=False` that write for themselves (W-08). | A user's uncommitted work overwritten; files changed that the run manifest does not list | enforced, with known gaps | -| ENF-S5 | The audited repository is untrusted | CI `Test` job: `tests/darnit/config/operator/test_repository_cannot_configure.py`, `test_baseline_toml_removed.py`, and the trust tests under `tests/darnit/trust/`. Does not cover: `remediate_audit_findings` building its trust target from the checkout's origin remote (W-07). | A repository that configures its own audit, or grants itself trust | enforced, with known gaps | -| ENF-S6 | Steps conclude only within their effective set | CI `Test` job: `tests/darnit/config/test_strict_framework_loading.py`, the orchestrator dispatch tests, and the corpus gate (`plugin-redefines-manual` among its cases). Does not cover: the shared-handler cache (W-04), `gh_api` and `mcp` expressions decided by truthiness (W-16), or a plugin registering without its plugin context (W-17). | A PASS no step was allowed to give | enforced, with known gaps | +| ENF-S2 | Unverified is not compliant | Tests (not yet run in CI; see #591): `tests/darnit/tools/test_compliance_errors.py`, `tests/darnit/assertions/test_compliance_effects.py`, `tests/darnit/harness/test_harness_compliance.py`, and the adversarial corpus gate (`tests/darnit_baseline/corpus/`, `known_false_pass.toml`). Does not cover: a control dropped at load time while its level is still reported compliant (wiring audit W-18), or a list-valued `when` that makes a control N/A (W-05). | A false PASS, or a level reported compliant with an unverified control | planned: #592 (the tests exist; CI does not select them, #591) | +| ENF-S3 | Judgment values are never concluded | Tests (not yet run in CI; see #591): `tests/darnit/context_integrity/` (`test_consumers_use_usable`, `test_accept_candidates`, `test_confirmation_records`, `test_reads_do_not_write`). Does not cover: raw `.project/project.yaml` values satisfying a remediation `when` guard, or the `project.*` template namespace (W-09). | A guessed maintainer, contact or license used as if confirmed | planned: #592 (the tests exist; CI does not select them, #591) | +| ENF-S4 | Remediation writes only through the executor | Tests (not yet run in CI; see #591): `tests/darnit/context_integrity/test_single_writer.py`, `test_user_files_survive.py`. Does not cover: plugin remediation handlers registered with `supports_plan=False` that write for themselves (W-08). | A user's uncommitted work overwritten; files changed that the run manifest does not list | planned: #592 (the tests exist; CI does not select them, #591) | +| ENF-S5 | The audited repository is untrusted | Tests (not yet run in CI; see #591): `tests/darnit/config/operator/test_repository_cannot_configure.py`, `test_baseline_toml_removed.py`, and the trust tests under `tests/darnit/trust/`. Does not cover: `remediate_audit_findings` building its trust target from the checkout's origin remote (W-07). | A repository that configures its own audit, or grants itself trust | planned: #592 (the tests exist; CI does not select them, #591) | +| ENF-S6 | Steps conclude only within their effective set | Tests (not yet run in CI; see #591): `tests/darnit/config/test_strict_framework_loading.py`, the orchestrator dispatch tests, and the corpus gate (`plugin-redefines-manual` among its cases). Does not cover: the shared-handler cache (W-04), `gh_api` and `mcp` expressions decided by truthiness (W-16), or a plugin registering without its plugin context (W-17). | A PASS no step was allowed to give | planned: #592 (the tests exist; CI does not select them, #591) | | ENF-F1 | Framework does not import implementations | Planned: an import check over `packages/darnit/src` that fails on `import darnit_<implementation>` or `from darnit_<implementation>`. | The framework works only with in-tree plugins | unbuilt | | ENF-F2 | Controls live in TOML | CI `Spec Sync`: `scripts/validate_sync.py` (TOML schema, pass types, SARIF reads TOML). Strict loading rejects unknown control keys. | Control metadata or logic split between Python and TOML | enforced | | ENF-F3 | No premature extension points | Manual: the reviewer asks for the second implementation or the ADR. | A seam with one implementation and no caller; the stores, adapters and pass registries found in the wiring audit | manual (maintainers) | @@ -44,7 +49,7 @@ visible. It does not mean a failing run blocks a merge. ## Build order -1. ENF-R1. Nothing else gates until the checks are required. +1. #592 (run the whole suite in CI), then ENF-R1. Nothing gates until both land. 2. ENF-F6, ENF-A4 and ENF-A2. These are small and already designed, and the first two stop new defects of the kind the wiring audit found. 3. ENF-F4 and ENF-F5. They will fail on day one; start them with the current From 692630045034a947f73a03cc2587419abef96459 Mon Sep 17 00:00:00 2001 From: Michael Lieberman <mlieberman85@gmail.com> Date: Mon, 12 Oct 2026 15:48:04 -0400 Subject: [PATCH 3/7] governance: S rows are enforced now that CI runs the whole suite (#592) Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Michael Lieberman <mlieberman85@gmail.com> --- governance/ENFORCEMENT.md | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/governance/ENFORCEMENT.md b/governance/ENFORCEMENT.md index c9945b3e..76812bf6 100644 --- a/governance/ENFORCEMENT.md +++ b/governance/ENFORCEMENT.md @@ -16,11 +16,6 @@ Status values (the same as swansong's): - `unbuilt`: no check exists, and nobody has started one. - `manual`: enforced by human review, and the row names who is responsible. -**Tests are not yet run in CI.** Until #592 merges, the CI `Test` job -selects tests only by the `unit` and `integration` markers. 2,754 of 5,164 -tests carry neither marker, and that includes every test the S rows cite -(#591). The S rows are `planned` until then. - **A caveat that applies to every row today.** `main`'s branch protection requires a pull request but requires no status checks and no approvals. Until ENF-R1 is enforced, an `enforced` row below means the check runs in CI and is @@ -29,11 +24,11 @@ visible. It does not mean a failing run blocks a merge. | ID | Rule | Check | Failure mode | Status | |----|------|-------|--------------|--------| | ENF-S1 | Controls check setup, not outcomes | Manual: the reviewer of any pull request that adds or changes a control or step type in a shipped framework TOML or plugin applies ADR-0001's admission test. Planned check: every `exec` step and plugin step type in a shipped framework is listed in a reviewed inventory that records the setup it reads, and a step not in the inventory fails. | A control that runs a tool and grades its result, which brings false PASS risk and the tool's trust roots into darnit | manual (maintainers); check planned for the scope-reset milestone | -| ENF-S2 | Unverified is not compliant | Tests (not yet run in CI; see #591): `tests/darnit/tools/test_compliance_errors.py`, `tests/darnit/assertions/test_compliance_effects.py`, `tests/darnit/harness/test_harness_compliance.py`, and the adversarial corpus gate (`tests/darnit_baseline/corpus/`, `known_false_pass.toml`). Does not cover: a control dropped at load time while its level is still reported compliant (wiring audit W-18), or a list-valued `when` that makes a control N/A (W-05). | A false PASS, or a level reported compliant with an unverified control | planned: #592 (the tests exist; CI does not select them, #591) | -| ENF-S3 | Judgment values are never concluded | Tests (not yet run in CI; see #591): `tests/darnit/context_integrity/` (`test_consumers_use_usable`, `test_accept_candidates`, `test_confirmation_records`, `test_reads_do_not_write`). Does not cover: raw `.project/project.yaml` values satisfying a remediation `when` guard, or the `project.*` template namespace (W-09). | A guessed maintainer, contact or license used as if confirmed | planned: #592 (the tests exist; CI does not select them, #591) | -| ENF-S4 | Remediation writes only through the executor | Tests (not yet run in CI; see #591): `tests/darnit/context_integrity/test_single_writer.py`, `test_user_files_survive.py`. Does not cover: plugin remediation handlers registered with `supports_plan=False` that write for themselves (W-08). | A user's uncommitted work overwritten; files changed that the run manifest does not list | planned: #592 (the tests exist; CI does not select them, #591) | -| ENF-S5 | The audited repository is untrusted | Tests (not yet run in CI; see #591): `tests/darnit/config/operator/test_repository_cannot_configure.py`, `test_baseline_toml_removed.py`, and the trust tests under `tests/darnit/trust/`. Does not cover: `remediate_audit_findings` building its trust target from the checkout's origin remote (W-07). | A repository that configures its own audit, or grants itself trust | planned: #592 (the tests exist; CI does not select them, #591) | -| ENF-S6 | Steps conclude only within their effective set | Tests (not yet run in CI; see #591): `tests/darnit/config/test_strict_framework_loading.py`, the orchestrator dispatch tests, and the corpus gate (`plugin-redefines-manual` among its cases). Does not cover: the shared-handler cache (W-04), `gh_api` and `mcp` expressions decided by truthiness (W-16), or a plugin registering without its plugin context (W-17). | A PASS no step was allowed to give | planned: #592 (the tests exist; CI does not select them, #591) | +| ENF-S2 | Unverified is not compliant | CI `Test` job (the whole suite since #592): `tests/darnit/tools/test_compliance_errors.py`, `tests/darnit/assertions/test_compliance_effects.py`, `tests/darnit/harness/test_harness_compliance.py`, and the adversarial corpus gate (`tests/darnit_baseline/corpus/`, `known_false_pass.toml`). Does not cover: a control dropped at load time while its level is still reported compliant (wiring audit W-18), or a list-valued `when` that makes a control N/A (W-05). | A false PASS, or a level reported compliant with an unverified control | enforced, with known gaps | +| ENF-S3 | Judgment values are never concluded | CI `Test` job (the whole suite since #592): `tests/darnit/context_integrity/` (`test_consumers_use_usable`, `test_accept_candidates`, `test_confirmation_records`, `test_reads_do_not_write`). Does not cover: raw `.project/project.yaml` values satisfying a remediation `when` guard, or the `project.*` template namespace (W-09). | A guessed maintainer, contact or license used as if confirmed | enforced, with known gaps | +| ENF-S4 | Remediation writes only through the executor | CI `Test` job (the whole suite since #592): `tests/darnit/context_integrity/test_single_writer.py`, `test_user_files_survive.py`. Does not cover: plugin remediation handlers registered with `supports_plan=False` that write for themselves (W-08). | A user's uncommitted work overwritten; files changed that the run manifest does not list | enforced, with known gaps | +| ENF-S5 | The audited repository is untrusted | CI `Test` job (the whole suite since #592): `tests/darnit/config/operator/test_repository_cannot_configure.py`, `test_baseline_toml_removed.py`, and the trust tests under `tests/darnit/trust/`. Does not cover: `remediate_audit_findings` building its trust target from the checkout's origin remote (W-07). | A repository that configures its own audit, or grants itself trust | enforced, with known gaps | +| ENF-S6 | Steps conclude only within their effective set | CI `Test` job (the whole suite since #592): `tests/darnit/config/test_strict_framework_loading.py`, the orchestrator dispatch tests, and the corpus gate (`plugin-redefines-manual` among its cases). Does not cover: the shared-handler cache (W-04), `gh_api` and `mcp` expressions decided by truthiness (W-16), or a plugin registering without its plugin context (W-17). | A PASS no step was allowed to give | enforced, with known gaps | | ENF-F1 | Framework does not import implementations | Planned: an import check over `packages/darnit/src` that fails on `import darnit_<implementation>` or `from darnit_<implementation>`. | The framework works only with in-tree plugins | unbuilt | | ENF-F2 | Controls live in TOML | CI `Spec Sync`: `scripts/validate_sync.py` (TOML schema, pass types, SARIF reads TOML). Strict loading rejects unknown control keys. | Control metadata or logic split between Python and TOML | enforced | | ENF-F3 | No premature extension points | Manual: the reviewer asks for the second implementation or the ADR. | A seam with one implementation and no caller; the stores, adapters and pass registries found in the wiring audit | manual (maintainers) | @@ -49,7 +44,7 @@ visible. It does not mean a failing run blocks a merge. ## Build order -1. #592 (run the whole suite in CI), then ENF-R1. Nothing gates until both land. +1. ENF-R1. CI has run the whole suite since #592, but nothing gates until the checks are required. 2. ENF-F6, ENF-A4 and ENF-A2. These are small and already designed, and the first two stop new defects of the kind the wiring audit found. 3. ENF-F4 and ENF-F5. They will fail on day one; start them with the current From e74301d574ca915f838e09bc39977f092e082e0b Mon Sep 17 00:00:00 2001 From: Michael Lieberman <mlieberman85@gmail.com> Date: Mon, 12 Oct 2026 15:57:32 -0400 Subject: [PATCH 4/7] governance: drop references to a private project; rename ADR-0002 Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Michael Lieberman <mlieberman85@gmail.com> --- governance/ENFORCEMENT.md | 4 ++-- ...rnance-follows-swansong.md => 0002-governance-model.md} | 7 ++----- governance/decisions/README.md | 2 +- governance/where-decisions-live.md | 2 -- 4 files changed, 5 insertions(+), 10 deletions(-) rename governance/decisions/{0002-governance-follows-swansong.md => 0002-governance-model.md} (93%) diff --git a/governance/ENFORCEMENT.md b/governance/ENFORCEMENT.md index 76812bf6..05740200 100644 --- a/governance/ENFORCEMENT.md +++ b/governance/ENFORCEMENT.md @@ -4,7 +4,7 @@ Every normative rule in `governance/` maps to a row here. Once ENF-A4's check exists, CI fails when a MUST or MUST NOT carries an enforcement id with no row here, or when a row names a check that does not exist. -Status values (the same as swansong's): +Status values: - `enforced`: the check exists and runs in CI. The row says in its text what the check does not yet cover. @@ -53,5 +53,5 @@ visible. It does not mean a failing run blocks a merge. 5. ENF-S1's inventory check, once the scope-reset classification has decided which steps stay. -Prove each new check the way swansong does: open a pull request built to +Prove each new check: open a pull request built to violate it, watch the check fail, and close that pull request unmerged. diff --git a/governance/decisions/0002-governance-follows-swansong.md b/governance/decisions/0002-governance-model.md similarity index 93% rename from governance/decisions/0002-governance-follows-swansong.md rename to governance/decisions/0002-governance-model.md index 637b9c93..5b886bf6 100644 --- a/governance/decisions/0002-governance-follows-swansong.md +++ b/governance/decisions/0002-governance-model.md @@ -22,10 +22,7 @@ None of these documents says which of its statements a check enforces. The wiring audit found rules that nothing enforced: dead code, single writer, confirmation. It also found a CI type check that could not fail. -The swansong project governs itself with a short constitution in which every -rule carries an enforcement id, an `ENFORCEMENT.md` that records each check's -honest status, immutable ADRs for reasoning, and a document that says where -each kind of decision lives. The maintainers asked darnit to follow it. +The maintainers want a model in which the constitution is short, every rule carries an enforcement id, an `ENFORCEMENT.md` records each check's honest status, immutable ADRs hold the reasoning, and one document says where each kind of decision lives. ## Decision @@ -63,7 +60,7 @@ each kind of decision lives. The maintainers asked darnit to follow it. `CHANGELOG.md` already record it. 6. **`governance/` is not a Spec Kit feature.** It is plain files changed by tier-1 pull requests, with no `specs/NNN` folder, no tasks and no feature - number. This follows swansong's ADR-0006. Spec Kit organizes work that + number. Spec Kit organizes work that ships behavior, as specify, plan, tasks, implement. Governance changes when a rule turns out wrong, or a feature needs a decision. It has no backlog of its own, and a feature stream would invent one. A feature that needs a diff --git a/governance/decisions/README.md b/governance/decisions/README.md index 5567e135..f3e86b39 100644 --- a/governance/decisions/README.md +++ b/governance/decisions/README.md @@ -27,5 +27,5 @@ reused. | ADR | Title | Status | |-----|-------|--------| | [0001](0001-what-darnit-is.md) | darnit aligns a project's setup with a hygiene goal | proposed | -| [0002](0002-governance-follows-swansong.md) | Governance is a short enforced constitution, ADRs and an enforcement table | proposed | +| [0002](0002-governance-model.md) | Governance is a short enforced constitution, ADRs and an enforcement table | proposed | | [0003](0003-docs-consolidation.md) | Consolidate the docs into governance, contracts, design notes and user docs | proposed | diff --git a/governance/where-decisions-live.md b/governance/where-decisions-live.md index adc807ac..7f87b17f 100644 --- a/governance/where-decisions-live.md +++ b/governance/where-decisions-live.md @@ -4,8 +4,6 @@ Read this before writing a rule, a spec or a design doc. Without it, every decision drifts into whichever document was open, the documents contradict each other, and nobody, human or agent, can tell which one binds. -Adapted from swansong's `governance/where-decisions-live.md`. - ## The first question: does it serve the scope? Before deciding where a decision lives, apply From 44b25fb492bd55aeeb6b0fb0e1abd10d1a276b70 Mon Sep 17 00:00:00 2001 From: Michael Lieberman <mlieberman85@gmail.com> Date: Mon, 12 Oct 2026 16:47:11 -0400 Subject: [PATCH 5/7] governance: 14 rules, each naming how it is enforced Revise the draft after a measured review of every rule: - Rules name an enforcement mode: test (preferred), agent or human. Agent- and human-only rules need a reason, a yes/no review question, an owner, and a deletion review after two quiet milestones. - `enforced` now requires a recorded failure demonstration; a check that runs but has not been shown to fail is `running`. - Merge S6 into S2 (no false PASS), F1+F2 (framework-agnostic core, controls are data), F4-F6 into F3 (shipped code can run), A2+A4 into A3 (governance changes; TSC where the Charter requires). Old F3 becomes ADR-0001's admission question 3. - New: F2 one execution path, F4 declared dependencies, R1 merge gate incl. whole-suite hermetic CI and a second-maintainer approval, A1 ratchets only tighten, A2 nothing private or local is published. - ENFORCEMENT rows carry today's measured findings on f9bfd08. - ADR-0001 rewords S1 to cover what a PASS claims, and records the co-maintainer's answers from review. ADR-0002 adds the enforcement modes, AGENTS.md as the agent instruction file, and TSC authority. - The docs consolidation ADR moves to its own later pull request. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Michael Lieberman <mlieberman85@gmail.com> --- governance/ENFORCEMENT.md | 111 ++++++---- governance/constitution.md | 163 ++++++++------ governance/decisions/0001-what-darnit-is.md | 65 ++++-- governance/decisions/0002-governance-model.md | 205 ++++++++++++------ .../decisions/0003-docs-consolidation.md | 199 ----------------- governance/decisions/README.md | 3 +- governance/where-decisions-live.md | 26 ++- 7 files changed, 356 insertions(+), 416 deletions(-) delete mode 100644 governance/decisions/0003-docs-consolidation.md diff --git a/governance/ENFORCEMENT.md b/governance/ENFORCEMENT.md index 05740200..16720e15 100644 --- a/governance/ENFORCEMENT.md +++ b/governance/ENFORCEMENT.md @@ -1,57 +1,72 @@ # Enforcement -Every normative rule in `governance/` maps to a row here. Once ENF-A4's check -exists, CI fails when a MUST or MUST NOT carries an enforcement id with no row -here, or when a row names a check that does not exist. +Every rule in `constitution.md` has a row here. A row records: + +- **Mode**: test, agent or human (see the constitution's "How rules are + enforced"). +- **Check**: the CI check, plus a yes/no review question for any part a test + cannot reach. +- **Today**: what the check finds on `main`, measured. The measurements below + were taken on `f9bfd08` (2026-10-12). +- **Status**. Status values: -- `enforced`: the check exists and runs in CI. The row says in its text what the - check does not yet cover. -- `dormant`: the check exists and runs, but its precondition is unmet, so it - asserts nothing yet. The row names the precondition. The row becomes - `enforced` on its own once the precondition is met. -- `planned`: agreed but not yet written. The row names the milestone it lands - in. A row planned for two milestones is either written or struck. +- `enforced`: the check runs in CI on every pull request, and a pull request + built to violate it was shown to fail. The row links that demonstration. +- `running`: the check runs in CI, but no failure demonstration is recorded + yet. +- `dormant`: the check runs but asserts nothing until a named precondition is + met. It becomes `enforced` on its own when the precondition holds. +- `planned`: agreed, not yet written. The row names the milestone it lands in. + A row planned for two milestones is either built or struck. - `unbuilt`: no check exists, and nobody has started one. -- `manual`: enforced by human review, and the row names who is responsible. - -**A caveat that applies to every row today.** `main`'s branch protection -requires a pull request but requires no status checks and no approvals. Until -ENF-R1 is enforced, an `enforced` row below means the check runs in CI and is -visible. It does not mean a failing run blocks a merge. - -| ID | Rule | Check | Failure mode | Status | -|----|------|-------|--------------|--------| -| ENF-S1 | Controls check setup, not outcomes | Manual: the reviewer of any pull request that adds or changes a control or step type in a shipped framework TOML or plugin applies ADR-0001's admission test. Planned check: every `exec` step and plugin step type in a shipped framework is listed in a reviewed inventory that records the setup it reads, and a step not in the inventory fails. | A control that runs a tool and grades its result, which brings false PASS risk and the tool's trust roots into darnit | manual (maintainers); check planned for the scope-reset milestone | -| ENF-S2 | Unverified is not compliant | CI `Test` job (the whole suite since #592): `tests/darnit/tools/test_compliance_errors.py`, `tests/darnit/assertions/test_compliance_effects.py`, `tests/darnit/harness/test_harness_compliance.py`, and the adversarial corpus gate (`tests/darnit_baseline/corpus/`, `known_false_pass.toml`). Does not cover: a control dropped at load time while its level is still reported compliant (wiring audit W-18), or a list-valued `when` that makes a control N/A (W-05). | A false PASS, or a level reported compliant with an unverified control | enforced, with known gaps | -| ENF-S3 | Judgment values are never concluded | CI `Test` job (the whole suite since #592): `tests/darnit/context_integrity/` (`test_consumers_use_usable`, `test_accept_candidates`, `test_confirmation_records`, `test_reads_do_not_write`). Does not cover: raw `.project/project.yaml` values satisfying a remediation `when` guard, or the `project.*` template namespace (W-09). | A guessed maintainer, contact or license used as if confirmed | enforced, with known gaps | -| ENF-S4 | Remediation writes only through the executor | CI `Test` job (the whole suite since #592): `tests/darnit/context_integrity/test_single_writer.py`, `test_user_files_survive.py`. Does not cover: plugin remediation handlers registered with `supports_plan=False` that write for themselves (W-08). | A user's uncommitted work overwritten; files changed that the run manifest does not list | enforced, with known gaps | -| ENF-S5 | The audited repository is untrusted | CI `Test` job (the whole suite since #592): `tests/darnit/config/operator/test_repository_cannot_configure.py`, `test_baseline_toml_removed.py`, and the trust tests under `tests/darnit/trust/`. Does not cover: `remediate_audit_findings` building its trust target from the checkout's origin remote (W-07). | A repository that configures its own audit, or grants itself trust | enforced, with known gaps | -| ENF-S6 | Steps conclude only within their effective set | CI `Test` job (the whole suite since #592): `tests/darnit/config/test_strict_framework_loading.py`, the orchestrator dispatch tests, and the corpus gate (`plugin-redefines-manual` among its cases). Does not cover: the shared-handler cache (W-04), `gh_api` and `mcp` expressions decided by truthiness (W-16), or a plugin registering without its plugin context (W-17). | A PASS no step was allowed to give | enforced, with known gaps | -| ENF-F1 | Framework does not import implementations | Planned: an import check over `packages/darnit/src` that fails on `import darnit_<implementation>` or `from darnit_<implementation>`. | The framework works only with in-tree plugins | unbuilt | -| ENF-F2 | Controls live in TOML | CI `Spec Sync`: `scripts/validate_sync.py` (TOML schema, pass types, SARIF reads TOML). Strict loading rejects unknown control keys. | Control metadata or logic split between Python and TOML | enforced | -| ENF-F3 | No premature extension points | Manual: the reviewer asks for the second implementation or the ADR. | A seam with one implementation and no caller; the stores, adapters and pass registries found in the wiring audit | manual (maintainers) | -| ENF-F4 | Everything accepted has a reader | Planned: a test that walks the framework-schema and operator-config models, the declared step settings, the argparse tree and the registered MCP tool signatures, and requires each field, setting, flag and parameter to be read in `packages/*/src` outside its declaring module, against a reviewed allow-list that gives a reason for each entry. | A documented setting that silently does nothing (W-10, W-13, W-14, W-24, W-29 in the wiring audit) | planned: scope-reset milestone | -| ENF-F5 | Every function runs in a test; default implementations run end to end | Planned: per-function coverage from the CI test run, failing on any function in `packages/*/src` with no executed line unless it is in a reviewed allow-list; plus a test that lists each Protocol's shipped default implementation and requires a test through a real entry point. On 2026-10-10, 129 functions had no executed line. | Code nobody runs; a default implementation that has never worked (#574) | planned: scope-reset milestone | -| ENF-F6 | No new type errors | Planned: whole-repo mypy (`mypy_path` over every `packages/*/src`, pydantic plugin) against a committed baseline file, failing on any new error and on any `attr-defined`, `call-arg` or `name-defined` error. Validated locally on 2026-10-10: 193 baseline entries, about 7 seconds warm, and it fails when a real defect is taken out of the baseline. Today's CI `Type Check` job checks two files with `|| true` and gates nothing. | Missing methods and wrong call signatures shipping (#574, the attestation predicate, the CEL `except` clause) | planned: scope-reset milestone | -| ENF-C1 | Contracts change spec-first; breaking changes are listed | CI `Spec Sync` covers handler names and the TOML schema. The rest is manual: the reviewer checks that the ADR or spec landed with or before the code, and that the BREAKING entry is present. | A contract that changes with no recorded decision; a user broken with no notice | manual (maintainers), partly enforced | -| ENF-A1 | AI attribution | DCO check requires the human submitter's `Signed-off-by`. The rest is manual: the reviewer checks the `Assisted-by` trailer, the absence of AI co-author or sign-off lines, and the PR's AI disclosure. Planned: a commit-message check that fails on an AI `Co-authored-by` or `Signed-off-by`. | AI attribution that misstates who certified the change | manual, partly enforced | -| ENF-A2 | Governance changes are labelled | Planned: a workflow that fails any pull request touching `governance/` unless it carries the `tier-1` label, plus a CODEOWNERS entry naming both maintainers for `governance/`. | Rules changed by whoever was editing nearby | unbuilt | -| ENF-A3 | No weakening checks | Planned: a diff check that fails on an added `# noqa`, `# type: ignore`, `pytest.mark.skip` or `continue-on-error` outside a `tier-1` pull request. | A green build bought by lowering the bar | unbuilt | -| ENF-A4 | Every rule has a row | Planned: `scripts/governance_lint.py`, which requires every MUST in `governance/*.md` to carry an id, every id to have a row here, and every row to be cited by a rule. | Governance drifting back into prose | unbuilt | -| ENF-R1 | Merges go through required checks | `main` branch protection. Today it requires a pull request with zero approvals and no required status checks. Planned: require `Test`, `Lint`, `Spec Sync`, `DCO` and, once built, the ENF-F6 type gate. | A red pull request merged | unbuilt (protection exists, but requires no checks) | +- `manual`: enforced by agent or human review only, with the review question + and the owner named. + +A new check starts from today's findings as an allow-list. Each entry links an +issue, and the list can only shrink (A1). + +**No check blocks a merge yet.** `main` requires a pull request, but no status +checks and no approvals, and `enforce_admins` is off (see ENF-R1). Until R1 is +in place, every row below is visible in CI but does not stop a merge. + +| ID | Mode | Check | Today | Status | +|----|------|-------|-------|--------| +| ENF-S1 | test + human | Planned: a committed inventory of every `exec` command, plugin step type and `gh_api` method in a shipped framework, each recorded as reading setup. A step not in the inventory fails. `gh_api` steps must be GET. Adding a row needs maintainer review against the question "Could this step's answer change while nothing in the repository or its platform settings changes (tool version, network, time, history)?" | About 15-20 rows. 5 outcome uses: `zizmor` ×2 (BR-01.01, AC-04.02), `gittuf_verify_policy`, `gittuf_commits_signed`, `repro_witness_attestation`. 2 borderline: `repro_hermetic_build`, `repro_bit_for_bit`. These are cut or reframed per ADR-0001. | planned: scope-reset milestone | +| ENF-S2 | test | CI `Test` job: `tests/darnit/tools/test_compliance_errors.py`, `tests/darnit/assertions/test_compliance_effects.py`, `tests/darnit/harness/test_harness_compliance.py`, `tests/darnit/config/test_strict_framework_loading.py`, and the adversarial corpus gate (`tests/darnit_baseline/corpus/`, including `plugin-redefines-manual`). Planned: a test that every shipped `promotion` matches corpus eligibility. | Known gaps from the wiring audit: the shared-handler cache, list-valued `when` (N/A), `gh_api`/`mcp` truthiness, a plugin registering without its plugin context, a dropped control leaving its level compliant. 0 promotions are shipped. | running (only the corpus gate has a failure demonstration) | +| ENF-S3 | test | CI `Test` job: `tests/darnit/context_integrity/`. | Known gap: raw `.project/project.yaml` values satisfy a remediation `when` guard, and the `project.*` template namespace is unguarded. | running | +| ENF-S4 | test | CI `Test` job: `tests/darnit/context_integrity/test_single_writer.py`, `test_user_files_survive.py`. Planned: an allow-list of filesystem write sites in `packages/*/src`, so a new write site outside the executor fails. | Write sites outside the executor are reachable from remediation: `darnit_baseline/threat_model/remediation.py` (5 sites; overwrites uncommitted user files), `remediation/helpers.py:53`, `darnit_baseline/tools.py:1144-1145`. | running, with known gaps | +| ENF-S5 | test | CI `Test` job: `tests/darnit/config/operator/test_repository_cannot_configure.py`, `test_baseline_toml_removed.py`, `tests/darnit/trust/`. | Known gap: `remediate_audit_findings` builds its trust target from the checkout's origin remote. | running, with known gaps | +| ENF-F1 | test | Planned: an AST check over `packages/darnit/src` that fails on an import of an implementation package, a framework-name literal outside the one default constant, and a `register_control` call outside the control loader. | 0 imports. 5 framework-name literals in 4 files: `agent/graph.py:402,416`, `cli.py:504`, `config/merger.py:691`, `config/context_schema.py:350`. `register_control` is called only by the loader, but CLAUDE.md shows it as a plugin API. | planned: first checks milestone | +| ENF-F2 | test | Today: tier-1 parity tests (`tests/darnit/parity/tier1/`) compare `audit_openssf_baseline` with `HarnessRun` only. Planned: a parity matrix of entry points (CLI, MCP, builtin, harness, `darnit run`) × options over the parity fixtures, plus an AST check that profile, tags and default-framework resolution happen in one module. | Profile resolution happens in 3 places and is absent from the CLI; tags are parsed in 3 places; the default framework is set in 4. CLI `audit` has no `--level`; MCP audit has no include/exclude; the harness has no tags or profile. | planned: second checks milestone | +| ENF-F3 | test + agent | Planned, three ratchets:<br>(1) Every MCP tool parameter and CLI flag is read (AST); configuration fields later.<br>(2) Whole-repo mypy against a baseline generated in CI's environment. Any `attr-defined`, `call-arg` or `name-defined` error fails outright.<br>(3) The set of shipped functions with no executed line, from CI's coverage run, may only shrink. Protocol stubs and test-fixture packages are excluded.<br>Agent question: "Does a test reach each new or changed default implementation through a real entry point?" | (1) 6 unread: `auto_init_config`, `template`, `get_pending_data` `level` and `profile`, `--profile` on `audit` and `plan`. (2) 201 errors, 21 in the always-fail tier. (3) 101 functions. Today's CI `Type Check` job checks 2 files with `\|\| true` and gates nothing. | planned: first checks milestone | +| ENF-F4 | test | Pull request #575: build every published wheel, install it without development dependencies, run one audit, and fail on an import error. | Not on `main` (#575 is open). The #573 class of bug is uncaught. | planned: merge #575 | +| ENF-C1 | test + human | Planned: committed snapshots of each contract surface: MCP tool schemas, the argparse tree, `model_json_schema()` of both configuration schemas, and JSON/SARIF/predicate output for one fixture. A removal or rename in a snapshot diff requires a BREAKING line. Human question: "Did the ADR or spec that decides this change land with or before it?" | Today's `Spec Sync` check finds 8 hard-coded handler names in a document and enforces almost nothing. | manual (maintainers); snapshots planned | +| ENF-R1 | test | `main` branch protection: required checks `Test`, `Lint`, `Spec-Implementation Sync`, `Build` and `DCO`; 1 approval from a maintainer other than the author; administrators included. Planned:<br>- a scheduled job that fails when protection drifts from this declaration;<br>- a test that the CI test command has no marker, `-k`, `--deselect` or `--ignore` selection;<br>- a hermetic job that runs the suite with an empty HOME, CI variables set and the network blocked, and fails if anything is written under HOME. | Protection requires a PR only: 0 approvals, no required checks, `enforce_admins` off. CI has run the whole suite since #592. With an empty HOME, the suite writes 377 files there, including `~/.darnit/verification_cache`, a pip cache, tree-sitter grammars and sigstore TUF metadata (a network fetch). | unbuilt (settings change pending) | +| ENF-A1 | test + agent | Planned: a committed file of ratchet counts and sets, compared on every pull request:<br>- suppressions by kind;<br>- the mypy baseline;<br>- the unexecuted-function set;<br>- reader allow-lists;<br>- the inventory of Protocols, entry-point groups and registries;<br>- CI's test selection.<br>An increase fails unless the pull request links an issue and a maintainer approves.<br>Agent question: "Did this change weaken, skip or suppress anything to make a run pass?" | Baseline: 85 `# noqa` (47 of them `BLE001`), 54 `# type: ignore`, 41 test skips (`mark.skip`/`skipif`, `pytest.skip(`, `importorskip`), 5 `pragma: no cover`, 2 `continue-on-error` and 10 `\|\| true` in workflows and scripts, 3 ruff per-file-ignores. 17 extension points (9 Protocols, 3 entry-point groups, 5 registries). | planned: first checks milestone | +| ENF-A2 | test + human | `DCO` check (not yet required; it cannot tell an agent-added sign-off from a person's). Planned:<br>- a check that fails on an AI `Co-authored-by` or `Signed-off-by` line, on a PR body without exactly one AI-assistance box ticked, or on AI use declared without an `Assisted-by` trailer;<br>- a check that fails on a local home-directory path in a tracked file;<br>- an output test that an audit report contains no home path or user profile.<br>Human question: "Can the author explain this change?" | Since AI_POLICY landed (38 commits): 1 AI co-author line, 6 AI-disclosed PRs without `Assisted-by`, 6 PRs with no AI box ticked. 21 home-directory paths in tracked files, mostly Spec Kit `tasks.md`. | planned: first checks milestone | +| ENF-A3 | test + agent | CODEOWNERS entry for `governance/` naming the maintainers, and required code-owner review on `main`. Planned: `scripts/governance_lint.py`, which requires every MUST in `governance/` to carry an id, every id to have a row here, and every row to be cited by a rule. Agent question: "Did a maintainer ask for this change to `governance/`?" | CODEOWNERS lists only `@mlieberman85`, plus a stale `/main.py` line. Code-owner review is not required. | unbuilt | ## Build order -1. ENF-R1. CI has run the whole suite since #592, but nothing gates until the checks are required. -2. ENF-F6, ENF-A4 and ENF-A2. These are small and already designed, and the - first two stop new defects of the kind the wiring audit found. -3. ENF-F4 and ENF-F5. They will fail on day one; start them with the current - failures as their allow-list and burn it down. -4. ENF-F1 and ENF-A3. -5. ENF-S1's inventory check, once the scope-reset classification has decided - which steps stay. - -Prove each new check: open a pull request built to -violate it, watch the check fail, and close that pull request unmerged. +1. ENF-R1 settings (no code) and a CODEOWNERS pull request. Nothing gates + until these land. +2. ENF-F3 (1), the reader check for MCP parameters and CLI flags, starting + with 6 allow-list entries linked to #528, #541 and #576. +3. ENF-F3 (2), the mypy gate, replacing today's `Type Check` job. +4. ENF-F1, the AST check, either fixing the 5 framework-name literals or + starting with them allow-listed. +5. ENF-A2, the attribution and home-path checks. +6. ENF-A1, the ratchet file. +7. ENF-F4: merge #575. +8. ENF-R1's hermetic job. The writes to HOME are fixed first, or allow-listed. +9. ENF-F2, the parity matrix and the single-resolver check. +10. ENF-F3 (3), the coverage ratchet. +11. ENF-C1, the contract snapshots. +12. ENF-S1, the inventory, after the scope-reset classification. +13. ENF-A3, governance_lint. + +Prove each new check: open a pull request built to violate it, watch the check +fail, close that pull request unmerged, and link it from the row. That link is +what moves a row from `running` to `enforced`. diff --git a/governance/constitution.md b/governance/constitution.md index cc167d86..65746263 100644 --- a/governance/constitution.md +++ b/governance/constitution.md @@ -1,40 +1,58 @@ # Constitution These are the non-negotiable rules for this repository. Every rule carries an -enforcement id that maps to a row in `ENFORCEMENT.md`. A rule with no -enforcing check is a wish, not a rule, and it will be ignored. +enforcement id that maps to a row in `ENFORCEMENT.md`. Each row says how the +rule is enforced and how far that enforcement reaches today. Changing a rule requires an ADR in `decisions/`. Do not edit a rule in place without one. -This document is deliberately short. Every agent session imports it, so each -rule here costs context in every session, forever. Before adding anything, -read `where-decisions-live.md`: most decisions belong somewhere else. +This document is deliberately short. Every agent session loads it, so each +rule costs context in every session, forever. Before adding a rule, read +`where-decisions-live.md`: most decisions belong somewhere else. What darnit is, and is not, is decided in -[ADR-0001](decisions/0001-what-darnit-is.md). The S rules below are the parts -of that decision a reviewer or a check can hold a change to. +[ADR-0001](decisions/0001-what-darnit-is.md). This document sits under the +project's [Technical Charter](../CHARTER.md) and +[GOVERNANCE.md](../GOVERNANCE.md), which decide who decides. This document +decides what a change must satisfy. -This document sits under the project's [Technical Charter](../CHARTER.md) and -[GOVERNANCE.md](../GOVERNANCE.md), which govern who decides. This document -governs what a change must satisfy. +## How rules are enforced + +Every rule names one or more enforcement modes. A deterministic check is +always preferred. + +- **test**: a check in CI that fails on a violation, and has been shown to. +- **agent**: the rule is in every agent's instructions (`AGENTS.md`), and the + agent applies it while writing and reviewing, using the yes/no question in + the rule's row. +- **human**: a maintainer applies it in review, and is accountable for it. + +A rule may rely on agent or human enforcement alone only if its row says why a +test is impossible or disproportionate, gives the yes/no question a reviewer +answers from the diff, and names an owner. A rule enforced that way that +catches nothing for two milestones is reviewed for deletion. A rule whose +check cannot fail is not enforced. --- -## 1. Scope +## 1. Scope and correctness -**S1.** A control MUST check or change the setup of the project it audits: -its files, its CI configuration, its platform settings and its declared -project data. A control MUST NOT run a tool to judge that tool's result, or -judge the project's runtime behavior. Out of scope, for example: whether -history satisfies a gittuf policy, whether recent commits are signed, whether -a scanner comes back clean, whether a build is hermetic or reproducible. -Whether gittuf is set up, a scanner is configured to run, or a lockfile and a -provenance step exist is in scope. `[ENF-S1]` +**S1.** A control's result MUST depend only on the audited project's setup: +its files, its CI configuration, its platform settings (read, never changed by +a check) and its confirmed project data. The same setup MUST give the same +result on any machine, at any time, with any tool version darnit supports. A +PASS states that a practice is set up, never that its outcome holds: for +example, that gittuf is set up, not that history satisfies the policy; that a +scanner runs in CI, not that its findings are clean. `[ENF-S1]` -**S2.** Unverified is not compliant. A level is compliant only when every -applicable control at that level is PASS. WARN, ERROR, PENDING, an unhonored -claim and a control that failed to load all count against it. `[ENF-S2]` +**S2.** No false PASS. A step MUST conclude only an outcome in its effective +set: its registered ceiling, narrowed by `concludes`, widened only by a +promotion that the corpus backs with zero false PASS results. A model judgment +MUST NOT conclude PASS. No plugin MAY replace a core step type. A level is +compliant only when every applicable control at that level is PASS. WARN, +ERROR, PENDING, an unhonored claim and a control that failed to load all count +against it. `[ENF-S2]` **S3.** darnit MUST NOT conclude a value that needs a person's judgment (`auto_detect = false`). Detection may propose a candidate. Only a person's @@ -50,68 +68,69 @@ own configuration files and its version-control remotes, configures darnit or grants trust. Trust and tool configuration come only from operator configuration or verified CI metadata. `[ENF-S5]` -**S6.** A step MUST conclude only an outcome in its effective set: its -registered ceiling, narrowed by `concludes`, widened only by a promotion that -the corpus backs with zero false PASS results. A model judgment MUST NOT -conclude PASS. No plugin MAY replace a core step type. `[ENF-S6]` - --- ## 2. Framework shape -**F1.** The `darnit` package MUST NOT import an implementation package. The -framework reaches implementations only through plugin discovery and the -`ComplianceImplementation` protocol. `[ENF-F1]` - -**F2.** Controls are defined in a framework's TOML. Python adds step types, -remediation handlers and tools; it does not define controls. `[ENF-F2]` - -**F3.** No new extension point (a Protocol, entry-point group, registry, store -kind or plugin hook) until two implementations need it, or an ADR records why -the seam is needed before the second. `[ENF-F3]` +**F1.** The core (`packages/darnit`) MUST NOT import or name an implementation +package or a framework. The default framework is one named constant. Controls +are data: only a framework's TOML defines them, and Python adds step types, +remediation handlers and tools. `[ENF-F1]` -**F4.** Everything darnit accepts MUST have a reader: every field of the -framework TOML schema and of operator configuration, every step setting, -every CLI flag and every MCP tool parameter is read by production code. A -setting that changes nothing is removed, not documented. `[ENF-F4]` +**F2.** There is one execution path. Every entry point that audits or +remediates (the CLI, each framework's MCP tools, the builtin tools, the +harness, `darnit run`) MUST resolve framework, level, tags, profile, +include/exclude and output format through one shared resolver, and MUST give +the same controls and statuses on the parity fixtures. `[ENF-F2]` -**F5.** Every function in `packages/*/src` MUST be executed by a test, and -every shipped default implementation of a Protocol MUST be executed by a test -that reaches it through a real entry point (the CLI, an MCP tool or -`run_sieve_audit`), not through a test double. `[ENF-F5]` +**F3.** Shipped code can run. Everything darnit accepts MUST have a production +reader: every CLI flag, MCP tool parameter, step setting and configuration +field. Every function in a shipped package MUST be executed by a test, and a +shipped default implementation of a Protocol by a test that reaches it through +a real entry point. A change MUST NOT add a type error. `[ENF-F3]` -**F6.** A change MUST NOT add a type error. An `attr-defined`, `call-arg` or -`name-defined` error MUST NOT be added even to the baseline. `[ENF-F6]` +**F4.** Each published package, installed from its built wheel without +development dependencies, MUST run an audit with no import error. `[ENF-F4]` --- ## 3. Contracts -**C1.** darnit's public contracts are: the framework TOML schema, the operator -configuration schema, the plugin protocol, the CLI, the MCP tool surface and -the output formats (JSON, SARIF, attestation predicate). A change to one MUST -land its ADR or feature spec before or with the code, and a breaking change -MUST appear under a **BREAKING** entry in `CHANGELOG.md`. `[ENF-C1]` +**C1.** darnit's public contracts are: the CLI, the MCP tool surface, the +framework TOML schema, the operator configuration schema, the plugin protocol +and the output formats (JSON, SARIF, attestation predicate). A change to a +contract MUST be visible in review, MUST land with or after its ADR or feature +spec, and, when it breaks a user, MUST appear under a **BREAKING** entry in +`CHANGELOG.md`. Breaking changes also follow GOVERNANCE.md's RFC process. +`[ENF-C1]` --- -## 4. Contributors and agents - -**A1.** AI-assisted contributions follow [AI_POLICY.md](../AI_POLICY.md): -attribution by an `Assisted-by` trailer, never an AI `Co-authored-by` or -`Signed-off-by`, and disclosure in the pull request. `[ENF-A1]` - -**A2.** `governance/` is tier-1. A change there MUST carry the `tier-1` label, -which records that a maintainer decided it, before it merges. An agent MUST NOT -change `governance/` unless a maintainer asked for that change. `[ENF-A2]` - -**A3.** A contributor (person or agent) who cannot satisfy a rule MUST stop and -file an issue describing the conflict. They MUST NOT weaken a check, skip a -test or add a suppression to make a run pass. `[ENF-A3]` - -**A4.** Every normative statement in `governance/` MUST carry an enforcement id -with a row in `ENFORCEMENT.md`. If you cannot name the check, you do not have a -rule. `[ENF-A4]` - -**R1.** A change MUST reach `main` only through a pull request whose required -checks passed on its head. `[ENF-R1]` +## 4. Process + +**R1.** A change MUST reach `main` only through a pull request. The required +checks MUST pass on its head, and they include the whole test suite, run +hermetically: no real HOME, no network, no dependence on CI environment +variables. The pull request MUST be approved by a maintainer other than its +author, and administrators are not exempt. The one exception is a merge from +a GitHub security advisory, which is recorded in the advisory. `[ENF-R1]` + +**A1.** Every ratchet only tightens: suppressions, skips, baselines, +allow-lists, the extension-point inventory and CI's test selection. Loosening +one requires a maintainer-approved pull request that links an issue. A +contributor (person or agent) who cannot satisfy a rule MUST stop and file an +issue that names the rule and the conflict, and MUST NOT weaken a check to +make a run pass. `[ENF-A1]` + +**A2.** Contributions follow [AI_POLICY.md](../AI_POLICY.md). Committed files, +pull request and issue text, and darnit's own output MUST NOT contain local +filesystem paths, an auditor's personal data, credentials, or the names of +private projects. `[ENF-A2]` + +**A3.** A change to `governance/` MUST be approved by a code owner of +`governance/`. Where the Charter gives the decision to the TSC (scope, and +contribution policy), the change MUST be approved by a majority of the TSC's +voting members, recorded as pull request approvals as GOVERNANCE.md +describes. Every MUST in `governance/` carries an enforcement id with a row in +`ENFORCEMENT.md`. An agent MUST NOT change `governance/` unless a maintainer +asked it to. `[ENF-A3]` diff --git a/governance/decisions/0001-what-darnit-is.md b/governance/decisions/0001-what-darnit-is.md index 159d696b..fbb6b6b9 100644 --- a/governance/decisions/0001-what-darnit-is.md +++ b/governance/decisions/0001-what-darnit-is.md @@ -1,9 +1,11 @@ # ADR-0001: darnit aligns a project's setup with a hygiene goal - Status: proposed -- Date: 2026-10-10 +- Date: 2026-10-12 +- Decided by: the TSC, because the Charter (2.g.ii) gives it changes to the + project's scope. - Rule impact: creates S1. Restates the existing conservative, trust and - authority principles as S2, S3, S5 and S6. + authority principles as S2-S5. ## Context @@ -84,11 +86,21 @@ A control, step type, interface, configuration key or subsystem is admitted only if all three answers are yes: 1. Does it read or change a project's setup? -2. Would its answer stay the same across two runs with no change to that setup? -3. Does something that exists today need it? +2. Would its answer stay the same across two runs with no change to that + setup, on any machine, at any time, with any tool version darnit supports? +3. Does something that exists today need it? A new extension point (a + Protocol, entry-point group, registry or plugin hook) needs two + implementations, or an ADR that says why the seam is needed before the + second. Question 2 catches outcome checks: the answer to "are the scans clean" changes -without anyone touching the setup. Question 3 catches premature interfaces (F3). +without anyone touching the setup. Question 3 catches premature interfaces. + +What a control's PASS claims matters as much as how the control measures it. +A step can read only CI configuration and still back a claim about an outcome: +`repro_bit_for_bit` reads CI files, but RE-03.01 claims "build output is +identical across independent builds". S1 therefore says what a PASS may claim: +that a practice is set up, never that its outcome holds. ## Alternatives @@ -118,15 +130,18 @@ measured its result. - **Some controls are cut or reframed.** A scope-reset classification will mark every control, step type and subsystem as keep, reframe or cut, for the maintainers to decide. Expected outcomes: - - gittuf GT-02.01 is cut. + - gittuf GT-02.01 ("recent commits are signed") is cut, or reframed to + "commit signing is required", by gittuf policy or branch protection. - GT-01.02 is reframed from "policy passes verification" to "the policy is well formed and CI runs gittuf verification". - - Witness verification and RE-03.01 are cut. + - Witness verification is cut. + - RE-03.01 is reframed from "build output is identical" to "CI runs a + reproducibility check and normalizes build timestamps". - RE-02.01 is reframed to the parts that read build configuration. - - The Baseline controls that grade zizmor findings need a decision. Reading - a workflow file for a dangerous trigger is setup; running a scanner and - grading its findings is an outcome. - - Two areas are borderline and need a decision: + - The Baseline controls that grade zizmor findings are reframed to read the + workflow files themselves. A zizmor result can change with the zizmor + version, and with no workflows it is an exit-code WARN (#584). + - Two areas are deferred to the scope-reset classification: - The threat-model analysis engine (features 006, 010, 011 and 014: tree-sitter discovery and opengrep taint analysis). Producing a threat model document as a remediation is setup. An analysis engine that grades @@ -136,9 +151,9 @@ measured its result. - **Some audit findings close by deletion, not repair.** W-01 is one; several P1s about unused interfaces are others. - **Some OSPS controls are phrased as outcomes.** For those, darnit reports - whether the practice is set up. Reports and the attestation predicate must - say "set up for", not "compliant with", wherever that is the truth. That is - a wording and predicate change, decided separately. + whether the practice is set up. Reports and the attestation predicate say + "set up for", not "compliant with", wherever that is the truth. The exact + wording and the predicate change are a contract change under C1. - **Fewer features.** Some contributions that work and were tested will be declined because they verify an outcome. The PR template and the S1 review make that visible before the work is done, not after. @@ -147,9 +162,23 @@ measured its result. SECURITY.md names a reporting channel. They remain PASS candidates that a person confirms. That is still setup. +## Decided in review + +The co-maintainer's review of the draft (#578, 2026-10-10) settled four of the +open questions: + +- The zizmor-graded controls are reframed to read the workflow files. +- Model judgments stay, as PASS candidates a person confirms. +- Reports say "set up for" where the practice, not its outcome, is what darnit + observed. +- Unknown step keys fail loading, and remediation acts on FAIL only. These two + belong to the docs-consolidation ADR, which follows separately. + ## Enforcement -- **S1 (`ENF-S1`):** manual review now. A planned inventory check covers every - `exec` step and plugin step type in a shipped framework. -- **S2 and S3:** restate existing tested rules (`ENF-S2`, `ENF-S3`). -- **F3, F4, F5:** cover question 3 of the admission test. +- **S1 (`ENF-S1`):** a committed inventory of every `exec` command, plugin step + type and `gh_api` method in a shipped framework, plus maintainer review of + each new inventory row. +- **S2-S5:** restate rules that existing tests check (`ENF-S2` to `ENF-S5`). +- **Question 3 of the admission test** is checked by the F3 reader and + coverage ratchets and by the extension-point inventory in A1. diff --git a/governance/decisions/0002-governance-model.md b/governance/decisions/0002-governance-model.md index 5b886bf6..6a520b09 100644 --- a/governance/decisions/0002-governance-model.md +++ b/governance/decisions/0002-governance-model.md @@ -1,9 +1,11 @@ -# ADR-0002: Governance is a short enforced constitution, ADRs and an enforcement table +# ADR-0002: Governance is a short constitution whose rules name their enforcement - Status: proposed -- Date: 2026-10-10 -- Rule impact: creates `governance/` and rules A1-A4 and R1. Retires the Spec - Kit constitution as a separately edited document. +- Date: 2026-10-12 +- Decided by: the TSC, because the Charter (2.g.vii) gives it policies and + processes for contributing. +- Rule impact: creates `governance/` and the rules in `constitution.md`. + Retires the Spec Kit constitution as a separately edited document. ## Context @@ -18,86 +20,151 @@ darnit's rules live in several places that disagree: never-built behavior. - **`AI_POLICY.md`.** -None of these documents says which of its statements a check enforces. The -wiring audit found rules that nothing enforced: dead code, single writer, -confirmation. It also found a CI type check that could not fail. +None of these documents says which of its statements anything enforces, and +the gaps showed. -The maintainers want a model in which the constitution is short, every rule carries an enforcement id, an `ENFORCEMENT.md` records each check's honest status, immutable ADRs hold the reasoning, and one document says where each kind of decision lives. +- The 2026-10-10 wiring audit found rules that nothing enforced, and a CI type + check that could not fail. +- Two days later, CI turned out to have been running only 2,390 of 5,164 + tests (#591). It selected tests by markers that most tests did not carry, so + none of the safety suites behind features 040-044 had ever run in CI. +- `main` requires no status checks, so even the checks that did run blocked + nothing. + +darnit's contributors use several coding agents (Claude, Gemini, Grok, +OpenAI Codex), and most of the code is written with one. A rule that only a +person remembers does not reach the agent writing the change. A rule that +only an agent applies can be skipped by the next agent. ## Decision -1. **`governance/` is normative and tier-1.** It holds: +1. **`governance/` holds the rules.** It contains: - `constitution.md`: the rules; - `ENFORCEMENT.md`: one row per rule; - `where-decisions-live.md`; - `decisions/`: the ADRs. -2. **Every rule has an enforcement id and a row.** A statement no check can - enforce is an ADR, not a rule. -3. **The Spec Kit constitution becomes a pointer to `governance/`,** so - `/speckit.*` commands read the same rules. Nobody edits it directly, and - nobody runs `/speckit.constitution`. Its principles map to rules: - - Plugin Separation becomes F1. - - Conservative-by-Default becomes S2. - - TOML-First becomes F2. + + A change to it needs a code owner's approval, and the TSC's approval where + the Charter gives the TSC the decision (A3). +2. **Every rule names how it is enforced, and tests come first.** + - **test:** a deterministic CI check, preferred wherever one is possible. + - **agent:** the rule is in every agent's instructions, applied with a + yes/no review question. + - **human:** a maintainer applies it in review. + + A rule enforced only by agents or humans needs a stated reason why a test + is impossible or disproportionate, a review question and an owner, and it + is reviewed for deletion if it catches nothing for two milestones. Most + rules that look like they need judgment have a deterministic part (an + inventory, a count that only shrinks, a snapshot diff), and that part is + built as a test. +3. **A row says `enforced` only when it is proven.** A row is `enforced` only + when its check runs in CI and a pull request built to violate it was shown + to fail. A check that runs but has not been shown to fail is `running`. The + #591 gap is the reason: tests that were assumed to run did not. +4. **Every agent reads the same rules.** `AGENTS.md` at the repository root is + the agent instruction file, because Codex, Cursor, Copilot and others read + it, and Gemini can be pointed at it. It carries: + - a copy of the constitution between markers, which a test keeps identical + to `governance/constitution.md`; + - where to read before touching what; + - the test and lint commands; + - the stop-and-file protocol (A1); + - the attribution lines from AI_POLICY. + + `CLAUDE.md` becomes a short file that imports `AGENTS.md` and adds only + Claude-specific notes. The Spec Kit constitution becomes a pointer to + `governance/`, and nobody runs `/speckit.constitution`. +5. **The pull request template asks questions that can be checked:** + - which rules the change could affect; + - where a new flag, parameter or setting is read; + - whether a contract snapshot changed, and its BREAKING entry; + - the issue behind any new suppression or allow-list entry; + - the AI tool and model used, and the parts it wrote. +6. **An AI review step is advisory.** + - If added, an agent reviews each pull request against the constitution and + comments on possible violations, citing the rule, file and line. + - Its comment is a claim, like a model judgment in darnit: a maintainer + confirms or dismisses it. + - It never approves, never labels, is never a required check, and never + reads secrets from a fork's code. +7. **The Spec Kit constitution's principles become rules:** + - Plugin Separation and TOML-First become F1. + - Conservative-by-Default and Sieve Pipeline Integrity become S2. The + latter's "4-phase ordering MUST be respected" clause is dropped: the + framework enforces no phase order, and RFC-0001 replaces phases. - Never Guess User Values becomes S3. - - Sieve Pipeline Integrity becomes S6, the authority ceiling. Its - "4-phase ordering MUST be respected" clause is dropped: the spec already - says the framework enforces no phase order, and RFC-0001 replaces - phases. - - New rules not in the Spec Kit constitution: S1 (scope), S4 (remediation - does no harm, from framework-design section 15) and S5 (the audited - repository is untrusted, from section 14). -4. **`framework-design.md` stops being "authoritative".** - - Its normative content becomes rules, ADRs or the contracts (C1). - - Its descriptive content becomes design notes under `design/`. - - Content about removed or out-of-scope features is deleted. - - The docs consolidation plan (a later ADR) says where each section goes. - - Until that plan lands, `scripts/validate_sync.py` keeps reading the - handler-name registry from it. -5. **`CLAUDE.md` imports the constitution.** It keeps routing ("read X before - touching Y") and a short "never" list. The feature changelog - ("Active Technologies", "Recent Changes") is removed, because git and - `CHANGELOG.md` already record it. -6. **`governance/` is not a Spec Kit feature.** It is plain files changed by - tier-1 pull requests, with no `specs/NNN` folder, no tasks and no feature - number. Spec Kit organizes work that - ships behavior, as specify, plan, tasks, implement. Governance changes when - a rule turns out wrong, or a feature needs a decision. It has no backlog of - its own, and a feature stream would invent one. A feature that needs a - rule or ADR records the need in its own spec, and the rule lands here. - Building an enforcement check is code: a plain pull request that cites - its `ENF-` row and shows the check failing on a built violation. It becomes - a Spec Kit feature only if it is large enough to need a plan. -7. **RFCs stay where they are.** An RFC (`docs/rfcs/`) is a proposal under - discussion; once accepted, it is recorded as an ADR. -8. **Project governance is unchanged.** `CHARTER.md`, `GOVERNANCE.md` and the - TSC process keep deciding who decides. Adopting this ADR is an operational - change made by maintainer consensus under `GOVERNANCE.md`. + + New rules not in the Spec Kit constitution: + - S1, scope; + - S4, remediation does no harm; + - S5, the audited repository is untrusted; + - F2, one execution path; + - F3, shipped code can run; + - F4, declared dependencies; + - R1, merge gate; + - A1, ratchets. +8. **`framework-design.md` stops being "authoritative".** A later ADR on + consolidating the docs moves its normative content into rules, ADRs or + contract documents, its descriptive content into design notes, and deletes + what describes removed or out-of-scope features. Until then, + `scripts/validate_sync.py` keeps reading it. +9. **`governance/` is not a Spec Kit feature.** + - It is plain files, changed by pull requests, with no `specs/NNN` folder, + tasks or feature number. Spec Kit organizes work that ships behavior; + governance changes when a rule is wrong or a feature needs a decision, + and a feature stream would invent a backlog for it. + - A feature that needs a rule or an ADR records the need in its own spec, + and the rule lands here. + - Building a check is a plain pull request that cites its `ENF-` row and + links the pull request that shows it failing. +10. **RFCs and project governance are unchanged.** An RFC (`docs/rfcs/`) is a + proposal under discussion, and an accepted RFC becomes an ADR. + `CHARTER.md`, `GOVERNANCE.md` and the TSC process still decide who + decides. ## Alternatives -- **Keep the Spec Kit constitution as the rule source and add enforcement ids - to it.** It stays the place Spec Kit's own commands rewrite, it is 347 - lines, and it mixes rules with process and history. Rejected. -- **Make `framework-design.md` the rule source.** At 2,369 lines it is the - wallpaper a short constitution exists to avoid, and much of it describes - code that no longer exists. Rejected. +**Keep the Spec Kit constitution as the rule source, and add enforcement ids +to it.** It is the file Spec Kit's commands rewrite, it is 347 lines, and it +mixes rules with process and history. + +**Make `framework-design.md` the rule source.** At 2,369 lines it is the +wallpaper a short constitution exists to avoid, and much of it describes code +that no longer exists. + +**Allow only tested rules.** That is the cleanest version, but some rules have +a judgment part no test reaches. Two examples: whether a new step's answer can +change without the setup changing, and whether a decision landed before the +change. Leaving those unwritten would leave them to memory. The agent and +human modes, with their conditions, keep them written down and small. + +**Instruct agents in `CLAUDE.md` only.** Contributors use agents that do not +read it. ## Consequences -- **Many current statements stop being rules.** A MUST in a design doc or a - spec no longer binds; only a rule with a row does. That is the point, but it - means reading `ENFORCEMENT.md` honestly. Most of darnit's checks are - `planned` or `unbuilt` today, and CI blocks no merge until ENF-R1 is - enforced. -- **The work is real.** Building checks F4, F5 and F6, consolidating the docs, - and requiring CI on `main` are each more than a day of work. -- **Speed changes.** Agents that need a governance change stop and file an - issue instead of editing the rule. That is slower, and the slowdown is - deliberate. +- **Statements stop binding.** A MUST in a design doc or a spec no longer + binds; only a rule with a row does. +- **The table is mostly `planned`.** That is honest, but it means the rules + bind review now and CI later. +- **The first checks find real defects.** On `main` today, the planned checks + would flag: + - 6 unread parameters and flags; + - 201 type errors; + - 101 functions no test executes; + - 5 framework-name literals in core; + - a test suite that writes 377 files to HOME. + + Each check starts with today's findings as an allow-list that can only + shrink. That is work, spread over the build order in `ENFORCEMENT.md`. +- **R1 slows merges.** It requires a second maintainer's approval and + includes administrators. That is slower, and it is the point. +- **Blocked work stays visible.** Agents stop and file instead of editing a + rule, so some work waits for a person. ## Enforcement -- **This ADR creates A1-A4 and R1.** Their rows are in `ENFORCEMENT.md`. -- **The Spec Kit pointer is planned under ENF-A4.** `governance_lint` will - check that `.specify/memory/constitution.md` contains only the pointer. +This ADR creates the rules in `constitution.md`. Their rows are in +`ENFORCEMENT.md`. The `AGENTS.md` copy test and the Spec Kit pointer check are +part of ENF-A3's governance_lint. diff --git a/governance/decisions/0003-docs-consolidation.md b/governance/decisions/0003-docs-consolidation.md deleted file mode 100644 index 241e00c3..00000000 --- a/governance/decisions/0003-docs-consolidation.md +++ /dev/null @@ -1,199 +0,0 @@ -# ADR-0003: Consolidate the docs into governance, contracts, design notes and user docs - -- Status: proposed -- Date: 2026-10-10 -- Rule impact: none directly. It moves normative text under A4, so every MUST - that survives carries an enforcement id. - -## Context - -The repository has about 28,500 lines of active documentation in more than 100 -files: - -- root documents: 4,469 lines; -- `docs/`: 22,691 lines; -- package READMEs and skills: about 1,000 lines; -- the Spec Kit constitution: 347 lines. - -Separately, `specs/` holds about 44,700 lines of Markdown. - -Normative statements (MUST, SHALL, "authoritative") are spread across about 30 -files, and three of those files call themselves authoritative: - -- the Spec Kit constitution; -- `docs/architecture/framework-design.md`, which alone has 265 normative - statements; -- `docs/architecture/README.md`. - -An inventory on 2026-10-10 found nine places where these documents contradict -each other or the code: - -- the phase ordering; -- whether unknown step keys load; -- whether WARN is remediated; -- a deleted `rules/catalog.py` described as live; -- stale handler lists; -- `.baseline.toml` compatibility; -- `[plugins]` placement; -- `[framework]` versus `[metadata]`; -- a stale Spec Kit pointer. - -Many user docs still teach removed APIs. Some examples: `DeterministicPass`, -`main.py`, `baseline-mcp`, `PENDING_LLM`, and "62 controls / v2025.10.10" -(the baseline now has 66 controls, at v2026.02.19). - -## Decision - -### Layout - -``` -governance/ normative, tier-1 - constitution.md - ENFORCEMENT.md - where-decisions-live.md - contracts/ behavior every implementation and plugin must keep - results-and-authority.md outcomes, ceilings, disposition, compliance, corpus - framework-toml.md schema, strict loading, step and remediation fields, CEL, when - context-values.md standing, confirmation, lapse, single writer - remediation-safety.md plan/apply, approval, platform engine, outcomes, manifest, VCS - operator-config-and-trust.md operator config, trust, claims, plugin trust - plugin-protocol.md protocol members, registration, module-path policy, packaging - decisions/ ADRs -design/ descriptive notes on how components work today; no MUST - overview.md, sieve.md, audit-pipeline.md, context.md, reporting.md, - remediation-engine.md, stores.md, agent-graph.md, workflow.md -docs/ user and author docs; every command and key they name exists (F4) - install/ user/ plugin-authoring/ contributing/ archive/ -``` - -### Rules for this layout - -- **Contracts.** A contract keeps only statements that carry an enforcement - id (A4). An unenforced MUST becomes either a row with status `planned` or - `unbuilt`, or descriptive text in `design/`. That keeps the contracts from - regrowing into another 2,369-line specification. -- **Specs.** `specs/NNN-*/` folders become implementation records. A spec's - open tasks are either finished or closed with a note. Specs are not - rewritten. -- **Dogfood files stay where they are.** These files are darnit's own - compliance evidence: - - in `docs/`: `DEPENDENCIES.md`, `SAST-POLICY.md`, `SCA-POLICY.md`, - `SECRETS-POLICY.md`, `SECURITY-ASSESSMENT.md`, `TEST-REQUIREMENTS.md`, and - `threatmodel/`; - - at the root: `SECURITY.md`, `THREAT_MODEL.md`, `CONTRIBUTING.md`, - `GOVERNANCE.md`, `README.md`, `SUPPORT.md`. - - `openssf-baseline.toml` and `.project/project.yaml` look for them at these - exact paths. -- **LF documents are unchanged.** `CHARTER.md`, `GOVERNANCE.md`, - `TECHNICAL-STEERING-COMMITTEE.md`, `AI_POLICY.md` and `CHANGELOG.md` stay as - they are. The one change is that GOVERNANCE.md's "breaking changes require an - RFC" now points at the RFC-then-ADR flow. - -### `framework-design.md` - -The specification is about 75% contract, 15% description and 10% history or -out-of-scope material. Its section numbers survive as anchors in the new -contracts, because test docstrings cite them. - -| Sections | Goes to | -|---|---| -| 1 Purpose, philosophy | replaced by ADR-0001 and the constitution | -| 1.3 diagram, 5.1 and 5.3 execution model, 11.1-11.3, 12, 13 | `design/` | -| 2 schema, 3.2-3.8 field and ceiling tables, 4.3, 4.4, 4.6, 4.7 | `contracts/framework-toml.md` | -| 3.0-3.1 outcomes, ceilings, disposition, order; 5.2, 5.4, 5.5 | `contracts/results-and-authority.md` (without the Witness passages) | -| 4.1, 4.2, 4.5, 15 | `contracts/remediation-safety.md` | -| 6.3-6.5, 10.1-10.3, 10.5 | `contracts/plugin-protocol.md` | -| 6.6, 14 | `contracts/operator-config-and-trust.md` | -| 7.4-7.11 | `contracts/context-values.md` | -| 7.1-7.3, 4.8, 6.1-6.2, Appendix A | `docs/plugin-authoring/` | -| 8 SARIF mapping | `design/reporting.md` | -| 9 sync enforcement | `ENFORCEMENT.md`. Its check finds 8 hard-coded handler names in the file, so it enforces almost nothing. | -| Appendix B (rules catalog migration) | deleted; the catalog no longer exists | -| Appendix C, version history | context for the ADRs below; otherwise removed (git keeps it) | - -### Everything else - -| Disposition | Files | -|---|---| -| Into `governance/contracts/` | `docs/architecture/`: `cel-expressions`, `conditional-controls`, `control-dependencies`, `external-templates`, `shared-handlers` (if shared handlers survive), `remediation-manual-guidance`, `github-api-remediation`, `remediation-audit-filtering` (after the WARN contradiction is decided), and the rule from `implementation-provided-tools` | -| Into `design/` | `ARCHITECTURE.md` (as `overview.md`); `docs/architecture/`: `audit-pipeline`, `audit-context-collection`, `context-collection`, `dot-project-integration`, `org-project-resolution`, `repo-identity-resolution`, `framework-agnostic-reporting`; `docs/`: `WORKFLOW.md`, `agent-graph.md`, a short version of `design/CONTEXT_SIEVE_DESIGN.md` | -| Into the darnit-baseline package docs | `ci-workflow-templates`, `declarative-file-templates`, `policy-doc-templates`, and `ATTESTATION_SPEC.md` (refreshed as the predicate contract) | -| Merged into `docs/plugin-authoring/` | `IMPLEMENTATION_GUIDE.md` (cut by about half), `HANDLER_AUTHORING.md`, `CEL_CONTEXT.md` together with `getting-started/cel-reference.md`, `getting-started/implementation-development.md`, both tutorials, `packaging-plugins.md` (the entry point), `plugin-authoring/stores.md` | -| Merged into `docs/contributing/` | `GETTING_STARTED.md`, `getting-started/{README,framework-development,development-workflow,environment-setup,testing,troubleshooting}`, the MCP development section of CONTRIBUTING.md | -| Into `docs/user/` | `using-skills`, `SECURITY_GUIDE.md` (linking the contracts instead of restating them), `MIGRATION_GUIDE.md`, `CNCF_METADATA.md`, `CSL_ONBOARDING.md` | -| Recorded as ADRs | `docs/decisions/cel-library-evaluation.md`; the accepted parts of RFC-0001; and `docs/design/builtin-tool-factories.md` as a proposed ADR once its owner wants it reviewed | -| `docs/archive/` (excluded from `test_doc_step_examples`) | `DECISION_FLOWS.md`, `ARCHITECTURE_RESEARCH_BRIEF.md`, `TEST_REPO_DESIGN.md`, `plugin-discovery-design.md`, `SARIF_DESIGN.md`, `design/CONTEXT_PROMPTS.md`, `design/cncf-project-spec-analysis.md` | -| Deleted | `design/reproducibility-attestation-system.md` (2,131 lines; out of scope under ADR-0001); `USAGE_GUIDE.md`; `GOVERNANCE_DOCS_DESIGN.md`; `TODO.md` (its items move to issues); `docs/examples/`; `docs/architecture/`: `context-documentation`, `sieve-handler-authoring`, `handler-pipeline`, `plugin-registry`, `README` | -| Rewritten in place | `README.md` (to about 300 lines, describing the scope as ADR-0001 states it); `CLAUDE.md` (to about 100 lines: import the constitution, routing, AI rules, test and lint commands); the package READMEs (control counts; the gittuf and reproducibility scope) | - -Estimated result: about 12,700 lines of active docs, down from about 28,500, -with about 2,950 of them normative and in one place, instead of about 5,800 -spread across 30 files. - -### Decisions needed before contract text is written - -1. Is phase ordering enforced? The proposal is no: dispatch follows the - step list. Constitution V asserted otherwise. -2. Do unknown step keys load? They fail loading since feature 044. - `handler-pipeline.md` is stale. -3. Is WARN remediated? `framework-design.md` 10.4 says only FAIL is. - `remediation-audit-filtering.md` says everything not PASS. The proposal - is FAIL only, which is what the code does. -4. Does `[plugins]` belong in framework TOML at all? The proposal is no: it - moves to operator config, and the unused framework-schema field is - removed (wiring audit CONFIG-18). -5. RFC-0001 promises that `.baseline.toml` keeps loading. That was superseded - by #556, and is recorded as such in the ADR that records RFC-0001's - accepted parts. - -### ADRs to backfill - -These record decisions that already shipped, so that the contracts can cite -them instead of restating the reasoning: - -- CEL via cel-python -- authority per claim (RFC-0001 stages 0-1, features 041 and 044) -- propose-only judgment keys (features 018 and 042) -- the audited repository is untrusted, and `.baseline.toml` is removed (040, #556) -- remediation plan/apply and `platform_setting` replacing `api_call` (043) -- removal of dead subsystems and the protocol trim (#487, #451) -- PENDING and ERROR statuses (036, 041) - -### Order of work - -1. ADR-0001, ADR-0002 and this ADR are accepted, after the maintainers decide - the open items. -2. CLAUDE.md is rewritten and the Spec Kit constitution becomes a pointer. - This is small, and it stops agents from reading stale rules. -3. The contracts are written from `framework-design.md`, one file per pull - request. `validate_sync.py`, the PR template and the test docstrings - follow. -4. `design/` notes are written and moved. -5. The user docs are merged, archived, deleted and rewritten. The doc-example - test is extended so that it also runs over the merged docs. - -## Alternatives - -**Keep `framework-design.md` and fix it.** It would keep being the place where -every feature appends a section. Rejected for the reasons in ADR-0002. - -**Treat the tests as the only contract and make all prose descriptive.** This -is honest, but plugin authors and reviewers then have no readable statement of -the behavior they must keep. Rejected in favor of contracts restricted to -enforced statements. - -## Consequences - -- **Links break.** Many links and test-docstring citations change. Keeping the - section numbers as anchors limits the breakage, but some will still be - missed. -- **Removed docs lose history in place.** Their history stays in git and - `docs/archive/`. -- **The work is large.** About five pull requests for the contracts, and two - or three for the user docs. - -## Enforcement - -None new. Contracts are subject to A4 (`ENF-A4`) and user docs to F4 -(`ENF-F4`). diff --git a/governance/decisions/README.md b/governance/decisions/README.md index f3e86b39..e97690bd 100644 --- a/governance/decisions/README.md +++ b/governance/decisions/README.md @@ -27,5 +27,4 @@ reused. | ADR | Title | Status | |-----|-------|--------| | [0001](0001-what-darnit-is.md) | darnit aligns a project's setup with a hygiene goal | proposed | -| [0002](0002-governance-model.md) | Governance is a short enforced constitution, ADRs and an enforcement table | proposed | -| [0003](0003-docs-consolidation.md) | Consolidate the docs into governance, contracts, design notes and user docs | proposed | +| [0002](0002-governance-model.md) | Governance is a short constitution whose rules name their enforcement | proposed | diff --git a/governance/where-decisions-live.md b/governance/where-decisions-live.md index 7f87b17f..f9efd88c 100644 --- a/governance/where-decisions-live.md +++ b/governance/where-decisions-live.md @@ -11,9 +11,13 @@ Before deciding where a decision lives, apply decided: 1. Does it read or change a project's setup: files, CI configuration, - platform settings or declared project data? -2. Would its answer stay the same across two runs with no change to that setup? + platform settings or confirmed project data? +2. Would its answer stay the same across two runs with no change to that + setup, on any machine, at any time? 3. Does a control, a remediation or an interface that exists today need it? + A new extension point (a Protocol, entry-point group, registry or plugin + hook) needs two implementations, or an ADR that says why the seam is + needed before the second. A "no" to any of these means the work is out of scope, or premature. Stop, and raise it in an issue before writing it down anywhere. @@ -26,11 +30,16 @@ and raise it in an issue before writing it down anywhere. 3. What must **already exist** for that check to mean anything, and what does the rule say before then? -| | Enforceable by a check | Not enforceable | +| | A test can check it | Only review can check it | |---|---|---| -| **A rule for every change** | `constitution.md` rule plus an `ENFORCEMENT.md` row | ADR only; it creates no rule | +| **A rule for every change** | `constitution.md` rule, plus an `ENFORCEMENT.md` row in `test` mode | A constitution rule in `agent` or `human` mode, only if it meets the conditions in the constitution's "How rules are enforced". Otherwise an ADR, which creates no rule. | | **A choice for one feature** | That feature's tests | That feature's `specs/NNN/plan.md` | +Prefer a test. Most rules that look like they need judgment have a +deterministic part: an inventory, a count that can only shrink, or a +snapshot whose diff makes the change visible. Build that part as a test, and +leave only the remaining judgment to agents and maintainers. + ## The fourth question: can it fail? A check that cannot fail is a wish with a green tick. Before a check lands, @@ -89,17 +98,18 @@ stale. A design note creates no obligation, so it uses no MUST or SHALL. **User docs** (`docs/`). How to install, use, write a framework or write a plugin. Every command, flag, key and API a user doc tells someone to use must -exist (F4). A doc that teaches a removed API is a bug. +exist. A doc that teaches a removed API is a bug. ## Exceptions A change that cannot satisfy a rule gets an ADR that records the exception and -its reasoning. Never a silent suppression and never a weakened check (A3). +its reasoning. Never a silent suppression and never a weakened check (A1). ## Governance changes during feature work -`governance/` is tier-1 (A2). An agent whose task needs a governance change -stops and files an issue (A3). The work stays blocked until a maintainer makes +Changes to `governance/` need a code owner's approval, and the TSC's where +the Charter requires it (A3). An agent whose task needs a governance change +stops and files an issue (A1). The work stays blocked until a maintainer makes or approves the change. That is the design: a blocked task is visible, and a rule quietly edited to fit is not. From 33029cf7c9d1c7bc1023e7b2f8157807d9cb6df0 Mon Sep 17 00:00:00 2001 From: Michael Lieberman <mlieberman85@gmail.com> Date: Mon, 12 Oct 2026 18:31:21 -0400 Subject: [PATCH 6/7] governance: R1 tiers review by risk; agent review as a required check On CODEOWNERS-owned paths (governance, release and CI workflows, trust, the remediation executor and platform engine, plugin discovery) a code owner other than the author approves. Elsewhere the author may merge once the required checks and an independent agent review pass; a blocking finding needs a second maintainer. Recorded in ADR-0002 as an open question for the TSC, since GOVERNANCE.md says one maintainer approval on every pull request. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Michael Lieberman <mlieberman85@gmail.com> --- governance/ENFORCEMENT.md | 8 ++-- governance/constitution.md | 22 +++++++--- governance/decisions/0002-governance-model.md | 43 +++++++++++++++---- 3 files changed, 55 insertions(+), 18 deletions(-) diff --git a/governance/ENFORCEMENT.md b/governance/ENFORCEMENT.md index 16720e15..14e9fae8 100644 --- a/governance/ENFORCEMENT.md +++ b/governance/ENFORCEMENT.md @@ -43,15 +43,17 @@ in place, every row below is visible in CI but does not stop a merge. | ENF-F3 | test + agent | Planned, three ratchets:<br>(1) Every MCP tool parameter and CLI flag is read (AST); configuration fields later.<br>(2) Whole-repo mypy against a baseline generated in CI's environment. Any `attr-defined`, `call-arg` or `name-defined` error fails outright.<br>(3) The set of shipped functions with no executed line, from CI's coverage run, may only shrink. Protocol stubs and test-fixture packages are excluded.<br>Agent question: "Does a test reach each new or changed default implementation through a real entry point?" | (1) 6 unread: `auto_init_config`, `template`, `get_pending_data` `level` and `profile`, `--profile` on `audit` and `plan`. (2) 201 errors, 21 in the always-fail tier. (3) 101 functions. Today's CI `Type Check` job checks 2 files with `\|\| true` and gates nothing. | planned: first checks milestone | | ENF-F4 | test | Pull request #575: build every published wheel, install it without development dependencies, run one audit, and fail on an import error. | Not on `main` (#575 is open). The #573 class of bug is uncaught. | planned: merge #575 | | ENF-C1 | test + human | Planned: committed snapshots of each contract surface: MCP tool schemas, the argparse tree, `model_json_schema()` of both configuration schemas, and JSON/SARIF/predicate output for one fixture. A removal or rename in a snapshot diff requires a BREAKING line. Human question: "Did the ADR or spec that decides this change land with or before it?" | Today's `Spec Sync` check finds 8 hard-coded handler names in a document and enforces almost nothing. | manual (maintainers); snapshots planned | -| ENF-R1 | test | `main` branch protection: required checks `Test`, `Lint`, `Spec-Implementation Sync`, `Build` and `DCO`; 1 approval from a maintainer other than the author; administrators included. Planned:<br>- a scheduled job that fails when protection drifts from this declaration;<br>- a test that the CI test command has no marker, `-k`, `--deselect` or `--ignore` selection;<br>- a hermetic job that runs the suite with an empty HOME, CI variables set and the network blocked, and fails if anything is written under HOME. | Protection requires a PR only: 0 approvals, no required checks, `enforce_admins` off. CI has run the whole suite since #592. With an empty HOME, the suite writes 377 files there, including `~/.darnit/verification_cache`, a pip cache, tree-sitter grammars and sigstore TUF metadata (a network fetch). | unbuilt (settings change pending) | +| ENF-R1 | test + human | `main` branch protection:<br>- required checks `Test`, `Lint`, `Spec-Implementation Sync`, `Build`, `DCO` and `agent-review`;<br>- required code-owner review, with CODEOWNERS listing only the owned paths, so a pull request touching them needs an owner other than its author;<br>- administrators included.<br><br>The `agent-review` check runs a fresh agent session that the author cannot steer. It has its own identity, takes the constitution's review questions as input, treats pull request text as untrusted, and reads no secrets from a fork's code. It passes when it raises no blocking finding, or when a maintainer other than the author has approved the pull request.<br><br>Human part: a weekly digest of pull requests merged without a second person goes to the maintainers, and anything doubtful becomes an issue.<br><br>Planned:<br>- a scheduled job that fails when protection drifts from this declaration;<br>- a test that the CI test command has no marker, `-k`, `--deselect` or `--ignore` selection;<br>- a hermetic job that runs the suite with an empty HOME, CI variables set and the network blocked, and fails if anything is written under HOME. | Protection requires a PR only: 0 approvals, no required checks, `enforce_admins` off. No `agent-review` check exists. CODEOWNERS owns everything (`*`) and names only one maintainer. CI has run the whole suite since #592. With an empty HOME, the suite writes 377 files there, including `~/.darnit/verification_cache`, a pip cache, tree-sitter grammars and sigstore TUF metadata (a network fetch). | unbuilt (settings change pending) | | ENF-A1 | test + agent | Planned: a committed file of ratchet counts and sets, compared on every pull request:<br>- suppressions by kind;<br>- the mypy baseline;<br>- the unexecuted-function set;<br>- reader allow-lists;<br>- the inventory of Protocols, entry-point groups and registries;<br>- CI's test selection.<br>An increase fails unless the pull request links an issue and a maintainer approves.<br>Agent question: "Did this change weaken, skip or suppress anything to make a run pass?" | Baseline: 85 `# noqa` (47 of them `BLE001`), 54 `# type: ignore`, 41 test skips (`mark.skip`/`skipif`, `pytest.skip(`, `importorskip`), 5 `pragma: no cover`, 2 `continue-on-error` and 10 `\|\| true` in workflows and scripts, 3 ruff per-file-ignores. 17 extension points (9 Protocols, 3 entry-point groups, 5 registries). | planned: first checks milestone | | ENF-A2 | test + human | `DCO` check (not yet required; it cannot tell an agent-added sign-off from a person's). Planned:<br>- a check that fails on an AI `Co-authored-by` or `Signed-off-by` line, on a PR body without exactly one AI-assistance box ticked, or on AI use declared without an `Assisted-by` trailer;<br>- a check that fails on a local home-directory path in a tracked file;<br>- an output test that an audit report contains no home path or user profile.<br>Human question: "Can the author explain this change?" | Since AI_POLICY landed (38 commits): 1 AI co-author line, 6 AI-disclosed PRs without `Assisted-by`, 6 PRs with no AI box ticked. 21 home-directory paths in tracked files, mostly Spec Kit `tasks.md`. | planned: first checks milestone | | ENF-A3 | test + agent | CODEOWNERS entry for `governance/` naming the maintainers, and required code-owner review on `main`. Planned: `scripts/governance_lint.py`, which requires every MUST in `governance/` to carry an id, every id to have a row here, and every row to be cited by a rule. Agent question: "Did a maintainer ask for this change to `governance/`?" | CODEOWNERS lists only `@mlieberman85`, plus a stale `/main.py` line. Code-owner review is not required. | unbuilt | ## Build order -1. ENF-R1 settings (no code) and a CODEOWNERS pull request. Nothing gates - until these land. +1. ENF-R1 settings (no code) and a CODEOWNERS pull request that owns only + the expensive paths and names both maintainers. Nothing gates until these + land. Until the `agent-review` check exists, R1 falls back to a second + maintainer's approval on every pull request. 2. ENF-F3 (1), the reader check for MCP parameters and CLI flags, starting with 6 allow-list entries linked to #528, #541 and #576. 3. ENF-F3 (2), the mypy gate, replacing today's `Type Check` job. diff --git a/governance/constitution.md b/governance/constitution.md index 65746263..a3426cfd 100644 --- a/governance/constitution.md +++ b/governance/constitution.md @@ -108,12 +108,22 @@ spec, and, when it breaks a user, MUST appear under a **BREAKING** entry in ## 4. Process -**R1.** A change MUST reach `main` only through a pull request. The required -checks MUST pass on its head, and they include the whole test suite, run -hermetically: no real HOME, no network, no dependence on CI environment -variables. The pull request MUST be approved by a maintainer other than its -author, and administrators are not exempt. The one exception is a merge from -a GitHub security advisory, which is recorded in the advisory. `[ENF-R1]` +**R1.** A change MUST reach `main` only through a pull request whose +required checks pass on its head. The required checks include: + +- the whole test suite, run hermetically: no real HOME, no network, no + dependence on CI environment variables; +- an independent agent review. + +A pull request that touches a path owned in CODEOWNERS MUST also be approved +by a code owner other than its author. The owned paths are the ones where a +wrong merge is expensive: `governance/`, the release and CI workflows, trust +and confirmation, the remediation executor and platform engine, and plugin +discovery. A pull request whose agent review raises a blocking finding needs +a maintainer other than the author to resolve it. + +Administrators are not exempt. The one exception is a merge from a GitHub +security advisory, which is recorded in the advisory. `[ENF-R1]` **A1.** Every ratchet only tightens: suppressions, skips, baselines, allow-lists, the extension-point inventory and CI's test selection. Loosening diff --git a/governance/decisions/0002-governance-model.md b/governance/decisions/0002-governance-model.md index 6a520b09..dbb70652 100644 --- a/governance/decisions/0002-governance-model.md +++ b/governance/decisions/0002-governance-model.md @@ -81,13 +81,20 @@ only an agent applies can be skipped by the next agent. - whether a contract snapshot changed, and its BREAKING entry; - the issue behind any new suppression or allow-list entry; - the AI tool and model used, and the parts it wrote. -6. **An AI review step is advisory.** - - If added, an agent reviews each pull request against the constitution and - comments on possible violations, citing the rule, file and line. - - Its comment is a claim, like a model judgment in darnit: a maintainer - confirms or dismisses it. - - It never approves, never labels, is never a required check, and never - reads secrets from a fork's code. +6. **An agent review is a required check, but never an approval.** + - An independent agent reviews each pull request against the + constitution's review questions, and reports possible violations citing + the rule, file and line. It runs in a fresh session the author cannot + steer, under its own identity. It treats pull request text as untrusted, + and never reads secrets from a fork's code. + - Its findings are claims, like a model judgment in darnit. A blocking + finding stops the merge until a maintainer other than the author resolves + it. A clean review lets the author merge only on paths that CODEOWNERS + does not own. + - On owned paths, a code owner other than the author always approves. An + agent's review is never enough on its own where a wrong merge is + expensive. This mirrors darnit's rule that a model judgment never + concludes PASS by itself. 7. **The Spec Kit constitution's principles become rules:** - Plugin Separation and TOML-First become F1. - Conservative-by-Default and Sieve Pipeline Integrity become S2. The @@ -123,6 +130,19 @@ only an agent applies can be skipped by the next agent. `CHARTER.md`, `GOVERNANCE.md` and the TSC process still decide who decides. +## Open question for the TSC + +R1 lets an author merge a change to a path CODEOWNERS does not own, after the +required checks and an independent agent review pass. GOVERNANCE.md says "one +maintainer approval required" for every pull request. The TSC decides between: + +- **R1 as written.** GOVERNANCE.md is amended to match. +- **A second maintainer on every pull request.** The agent review stays + advisory. + +Whichever it chooses, R1 falls back to a second maintainer on every pull +request until the `agent-review` check exists. + ## Alternatives **Keep the Spec Kit constitution as the rule source, and add enforcement ids @@ -158,8 +178,13 @@ read it. Each check starts with today's findings as an allow-list that can only shrink. That is work, spread over the build order in `ENFORCEMENT.md`. -- **R1 slows merges.** It requires a second maintainer's approval and - includes administrators. That is slower, and it is the point. +- **Nobody merges their own change unreviewed.** + - On paths where a wrong merge is expensive, R1 requires the other + maintainer, and includes administrators. With two maintainers, that is a + real constraint on speed. + - Elsewhere, an independent agent review stands in for the second person, + and a weekly digest lets the maintainers check its calls after the + fact. - **Blocked work stays visible.** Agents stop and file instead of editing a rule, so some work waits for a person. From 8af507d73038ced5c064c7b94857f16c1b547626 Mon Sep 17 00:00:00 2001 From: Michael Lieberman <mlieberman85@gmail.com> Date: Thu, 15 Oct 2026 15:27:43 -0400 Subject: [PATCH 7/7] governance: ENF-F4 runs since #575 Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Michael Lieberman <mlieberman85@gmail.com> --- governance/ENFORCEMENT.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/governance/ENFORCEMENT.md b/governance/ENFORCEMENT.md index 14e9fae8..93ce9fc0 100644 --- a/governance/ENFORCEMENT.md +++ b/governance/ENFORCEMENT.md @@ -41,7 +41,7 @@ in place, every row below is visible in CI but does not stop a merge. | ENF-F1 | test | Planned: an AST check over `packages/darnit/src` that fails on an import of an implementation package, a framework-name literal outside the one default constant, and a `register_control` call outside the control loader. | 0 imports. 5 framework-name literals in 4 files: `agent/graph.py:402,416`, `cli.py:504`, `config/merger.py:691`, `config/context_schema.py:350`. `register_control` is called only by the loader, but CLAUDE.md shows it as a plugin API. | planned: first checks milestone | | ENF-F2 | test | Today: tier-1 parity tests (`tests/darnit/parity/tier1/`) compare `audit_openssf_baseline` with `HarnessRun` only. Planned: a parity matrix of entry points (CLI, MCP, builtin, harness, `darnit run`) × options over the parity fixtures, plus an AST check that profile, tags and default-framework resolution happen in one module. | Profile resolution happens in 3 places and is absent from the CLI; tags are parsed in 3 places; the default framework is set in 4. CLI `audit` has no `--level`; MCP audit has no include/exclude; the harness has no tags or profile. | planned: second checks milestone | | ENF-F3 | test + agent | Planned, three ratchets:<br>(1) Every MCP tool parameter and CLI flag is read (AST); configuration fields later.<br>(2) Whole-repo mypy against a baseline generated in CI's environment. Any `attr-defined`, `call-arg` or `name-defined` error fails outright.<br>(3) The set of shipped functions with no executed line, from CI's coverage run, may only shrink. Protocol stubs and test-fixture packages are excluded.<br>Agent question: "Does a test reach each new or changed default implementation through a real entry point?" | (1) 6 unread: `auto_init_config`, `template`, `get_pending_data` `level` and `profile`, `--profile` on `audit` and `plan`. (2) 201 errors, 21 in the always-fail tier. (3) 101 functions. Today's CI `Type Check` job checks 2 files with `\|\| true` and gates nothing. | planned: first checks milestone | -| ENF-F4 | test | Pull request #575: build every published wheel, install it without development dependencies, run one audit, and fail on an import error. | Not on `main` (#575 is open). The #573 class of bug is uncaught. | planned: merge #575 | +| ENF-F4 | test | CI `Non-dev install smoke` job (#575, extended by #596): builds every published wheel, installs it without development dependencies, runs one audit, and fails on a control that could not be evaluated or on more than one wheel per package. | Passes on `main`. Shown to fail when `packaging` is removed from darnit-core (#575 review). Plugins loaded only through registration (gittuf, reproducibility) are not imported in full. | running (a failure demonstration exists in review; link it from a closed pull request to mark `enforced`) | | ENF-C1 | test + human | Planned: committed snapshots of each contract surface: MCP tool schemas, the argparse tree, `model_json_schema()` of both configuration schemas, and JSON/SARIF/predicate output for one fixture. A removal or rename in a snapshot diff requires a BREAKING line. Human question: "Did the ADR or spec that decides this change land with or before it?" | Today's `Spec Sync` check finds 8 hard-coded handler names in a document and enforces almost nothing. | manual (maintainers); snapshots planned | | ENF-R1 | test + human | `main` branch protection:<br>- required checks `Test`, `Lint`, `Spec-Implementation Sync`, `Build`, `DCO` and `agent-review`;<br>- required code-owner review, with CODEOWNERS listing only the owned paths, so a pull request touching them needs an owner other than its author;<br>- administrators included.<br><br>The `agent-review` check runs a fresh agent session that the author cannot steer. It has its own identity, takes the constitution's review questions as input, treats pull request text as untrusted, and reads no secrets from a fork's code. It passes when it raises no blocking finding, or when a maintainer other than the author has approved the pull request.<br><br>Human part: a weekly digest of pull requests merged without a second person goes to the maintainers, and anything doubtful becomes an issue.<br><br>Planned:<br>- a scheduled job that fails when protection drifts from this declaration;<br>- a test that the CI test command has no marker, `-k`, `--deselect` or `--ignore` selection;<br>- a hermetic job that runs the suite with an empty HOME, CI variables set and the network blocked, and fails if anything is written under HOME. | Protection requires a PR only: 0 approvals, no required checks, `enforce_admins` off. No `agent-review` check exists. CODEOWNERS owns everything (`*`) and names only one maintainer. CI has run the whole suite since #592. With an empty HOME, the suite writes 377 files there, including `~/.darnit/verification_cache`, a pip cache, tree-sitter grammars and sigstore TUF metadata (a network fetch). | unbuilt (settings change pending) | | ENF-A1 | test + agent | Planned: a committed file of ratchet counts and sets, compared on every pull request:<br>- suppressions by kind;<br>- the mypy baseline;<br>- the unexecuted-function set;<br>- reader allow-lists;<br>- the inventory of Protocols, entry-point groups and registries;<br>- CI's test selection.<br>An increase fails unless the pull request links an issue and a maintainer approves.<br>Agent question: "Did this change weaken, skip or suppress anything to make a run pass?" | Baseline: 85 `# noqa` (47 of them `BLE001`), 54 `# type: ignore`, 41 test skips (`mark.skip`/`skipif`, `pytest.skip(`, `importorskip`), 5 `pragma: no cover`, 2 `continue-on-error` and 10 `\|\| true` in workflows and scripts, 3 ruff per-file-ignores. 17 extension points (9 Protocols, 3 entry-point groups, 5 registries). | planned: first checks milestone | @@ -61,7 +61,7 @@ in place, every row below is visible in CI but does not stop a merge. starting with them allow-listed. 5. ENF-A2, the attribution and home-path checks. 6. ENF-A1, the ratchet file. -7. ENF-F4: merge #575. +7. ENF-F4: record the failure demonstration (#575 merged). 8. ENF-R1's hermetic job. The writes to HOME are fixed first, or allow-listed. 9. ENF-F2, the parity matrix and the single-resolver check. 10. ENF-F3 (3), the coverage ratchet.