Repository navigation
CI skips more than half the test suite: tests are selected by unit/integration markers most tests don't have #591
Description
CI's Test job selects tests by marker (pytest tests/ -m unit, then pytest tests/ -m integration), plus the corpus directory. Most of the suite carries neither marker, so it never runs in CI:
$ uv run pytest tests/ --collect-only -m "unit or integration" | tail -1
2390/5164 tests collected (2774 deselected)
The 2,754 tests that never run include every safety suite behind features 040-044:
| Path | Run by CI / total |
|---|---|
tests/darnit/context_integrity/ (single writer, candidates, confirmations) |
0 / 105 |
tests/darnit/remediation/ (plan/apply, approval, platform engine, manifest, git safety) |
0 / 569 |
tests/darnit/trust/ |
0 / 116 |
tests/darnit/stores/ |
0 / 166 |
tests/darnit/server/ |
0 / 133 |
tests/darnit/config/test_strict_framework_loading.py |
0 / 21 |
compliance calculation (tools/test_compliance_errors.py, assertions/test_compliance_effects.py, harness/test_harness_compliance.py) |
0 / 27 |
A green CI run has therefore never covered these tests. Main is green under the full suite today (5145 passed, 19 skipped locally with --all-extras and CI's environment variables), because maintainers have been running the full suite locally. A contributor's PR, however, can break any of these and still show green.
Fix: run pytest tests/ without a marker selection. live and upstream tests stay opt-in through tests/conftest.py (#548). The full run takes about 4 minutes.
Follow-up: nothing stops a marker selection from coming back. The proposed governance (#578, ENF-R1) would make Test a required check on main; today main requires no status checks at all.
Found while reviewing #588 and #567.
Drafted with Claude Code; reviewed and posted by me.
Activity
governance: S rows are planned until CI runs the whole suite (darnitd…
test: run every test as a local run, wherever the suite runs (darnitd…
ci: run the whole test suite instead of marker selections (darnitdevo…