Skip to content

check-ci-status step 5 classifier is not total; check-merge-readiness defaults to READY on the gap #352

Description

@dean0x

Problem

check-ci-status's classification step is a three-arm priority list over the CI checks it
fetches. As of PR #326 it lives at dist/skills/git/references/pr/check-ci-status.md (moved
verbatim from src/assets/agents/git.mds, unchanged by that move — this is a pre-existing
defect, not a regression it introduced):

5. Classify in priority order: if any check has state `IN_PROGRESS` or `PENDING` → `PENDING`;
   else if any conclusion is `FAILURE` → `FAILING`; else if all conclusions are `SUCCESS` →
   `PASSING`

Three arms, but the declared output contract has five values (PASSING | FAILING | PENDING | NO_CI | NO_PR). A GitHub check that has been requested but not yet started reports an empty
conclusion and a state outside {IN_PROGRESS, PENDING} — it matches none of the three arms,
so the operation has no defined answer for it.

That gap is inert on its own if nothing consumes the undefined case optimistically. Something
does: check-merge-readiness reuses the same logic —

3. Fetch CI status (same logic as `check-ci-status`)
4. Classify (first matching rule wins):
   - `NOT_READY (unresolved threads: {n})` — unresolved_threads > 0
   - `NOT_READY (changes requested)` — reviewDecision == `CHANGES_REQUESTED`
   - `NOT_READY (CI failing: {checks})` — ci_status == `FAILING`
   - `NOT_READY (CI pending)` — ci_status == `PENDING` (expected after a push; non-alarming)
   - `NOT_READY (no approving review)` — reviewDecision == `REVIEW_REQUIRED` or null
   - `READY` — no rule above matched (unresolved_threads == 0, reviewDecision == `APPROVED`,
     ci_status == `PASSING` or `NO_CI`)

READY is the terminal default arm, reached by exhaustion ("no rule above matched"), not by
a positive check that CI actually passed. Every NOT_READY arm requires a positive match; an
unclassifiable ci_status (the requested-but-not-started gap above) satisfies none of them and
falls straight through to READY.

The gate whose entire purpose is to be conservative resolves to its most permissive value
exactly when it is least informed. The window is only a few seconds — right after a push — which
is precisely the moment a merge decision is likely to be made.

Why this matters in practice

.devflow/learning/pitfalls.md already records the lived instance as PF-075: an early
CI-poll loop treated an unstarted check as "not failing yet" and let an admin-bypass squash merge
proceed before CI had actually finished. PF-075 states the transferable rule: a classifier's arms
must cover its own declared output domain, and a safety gate's permissive terminal value should
be reached by a positive conjunction, never by "nothing else matched."

Proposed fix (not in scope for #326)

  1. Add an explicit arm in check-ci-status step 5 for "started but unconcluded" (empty
    conclusion, state outside the two-value set) and map it to PENDING (or a distinct
    INDETERMINATE, per PF-075's third rule, if collapsing it into PENDING loses information
    a caller needs).
  2. Invert check-merge-readiness's terminal arm so READY is a positive conjunction
    (unresolved_threads == 0 AND reviewDecision == APPROVED AND ci_status IN {PASSING, NO_CI})
    and everything unmatched falls through to NOT_READY (or a new NOT_READY (CI status unknown) reason) rather than to READY.
  3. Treat "(same logic as check-ci-status)" as inheriting whatever gap check-ci-status has —
    fix once, at the source of the classification, not at each reuse site.

Where it lives now

Moved (verbatim) by #326 to dist/skills/git/references/pr/check-ci-status.md; the
check-merge-readiness operation lives in the same module, references/pr/, generated from
src/assets/mds/git/_pr.mds.

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions