[Fleet incident] Govern orphaned GitHub Actions workflow lifecycle organization-wide

Systemic fleet defect

Live Actions inventories now prove the same recurrence mechanism in multiple ContextualWisdomLab repositories:

appguardrail: 42 registered workflows, including numerous historical apply-*, finalize-*, and *-once.yml identities still reported as state: active although sampled paths return 404 on the protected default branch (appguardrail#929);
clearfolio: 23 registered workflows, including several one-shot-*, PR-specific reconciliation, and repair identities still active although sampled source paths are absent (clearfolio#423);
disksage: 54 registered workflows, including many PR-specific/one-shot/finalizer identities still active although sampled source paths are absent (disksage#191).

This is not isolated repository clutter. The organization has repeatedly used temporary branch-writing/finalizer workflows, deleted their YAML after use, and assumed source deletion completed the workflow lifecycle. GitHub persists workflow registry records independently, so deleted source paths can remain advertised as active identities.

Root cause

The central governance model validates repository trees and workflow source, but it does not reconcile the GitHub Actions control-plane registry against the exact protected default-branch tree. No organization-level invariant currently requires:

active repository workflow identity
=> source file exists on exact protected default branch
   OR explicit reviewed exception

Consequently the fleet cannot reliably answer which workflow writers are genuinely enabled, and file-only tests cannot detect orphaned registry state.

Required central architecture

Implement a centrally owned, read-only-first workflow lifecycle inventory using the existing organization GitHub App/OIDC authority rather than a new PAT. It should:

enumerate every visible non-archived organization repository with pagination;
resolve each exact default-branch SHA;
paginate each repository's Actions workflow registry;
distinguish repository-path workflows from GitHub-owned dynamic workflows;
check exact source-file presence on the resolved default-branch SHA;
classify records as present_active, present_disabled, orphan_active, orphan_disabled, dynamic_owned, or unresolved;
emit immutable evidence with repository, workflow ID, name, path, state, default-branch SHA, observation time, and API-pagination receipts;
fail closed on incomplete visibility, 403/404 ambiguity, pagination truncation, or branch movement;
open/update bounded repository issues for orphan_active findings through existing governance;
permit disablement only in a separate reviewed operator step or explicit bounded central action after exact evidence is revalidated.

Do not recreate deleted workflows, add a temporary self-deleting cleanup workflow, or give an untrusted scanner workflow organization-wide write authority.

Prevention policy

Add a central reusable contract for repository workflows and AppGuardrail integration so that future bounded repair workflows must either:

be implemented through a normal branch/PR and deleted before merge without ever becoming a protected-default-branch workflow identity; or
include an explicit lifecycle owner, expiry condition, and post-use registry-disable action verified through a normal reviewed control-plane path.

A source file named once is not alone proof of invalidity; classification must be evidence-based. Conversely a benign name must not hide branch-writing or workflow-state mutation.

Acceptance criteria

complete paginated inventory of every visible eligible repository;
exact default-branch SHA binding and branch-movement retry/abort behavior;
no conflation of GitHub dynamic workflows with deleted repository files;
all known active orphan identities in AppGuardrail, Clearfolio, and DiskSage routed to their owner issues and ultimately disabled;
no production scheduled, reusable, security, release, Dependabot, or GitHub-owned workflow disabled by heuristic name matching;
tests cover partial pagination, API permission loss, transient 404/5xx, default-branch changes, path case/encoding, renamed workflows, reused workflow IDs, and active files with one-shot-like names;
AppGuardrail detects the same control-plane class from live evidence;
protected-main central integration through normal review/check policy;
a post-integration organization sweep produces a reproducible ledger and demonstrates that the known orphan records are no longer active;
no COPILOT_GITHUB_TOKEN, guessed PAT, or broad secrets: inherit is introduced.

This issue is part of the Hourly Product Development fleet incident. The central .github dedicated writer loop owns implementation; repository fleet audit remains read-only and tracks downstream operational evidence.

Additional confirmed blast radius

Fresh repository-specific inventory extends the proven recurrence beyond the initial three repositories:

pg-erd-cloud: 52 workflow identities; multiple active apply-pr-*, autofix-pr-*, bootstrap-*, cleanup-*, probe and PR-specific repair records. Protected-main contents lookup returns 404 for sampled apply-pr-696-review-fixes-v4.yml. Owner issue: ContextualWisdomLab/pg-erd-cloud#865.
bandscope: 44 workflow identities; multiple active bootstrap, finalizer, diagnosis and PR-specific records. Protected-default-branch contents lookup returns 404 for sampled bootstrap-handoff-import.yml. Owner issue: ContextualWisdomLab/bandscope#847.
clearfolio: its active hourly-product-development.yml identity is itself absent from protected main; recorded runs are feature-branch push runs from Draft Guard OpenCode approval with commit check-runs #271, so it must be classified as an orphan pending intended supported integration rather than as proof of an operating schedule.

The central solution must therefore preserve owner intent as well as source presence. An active absent identity may be:

a completed temporary writer that should be disabled;
a production workflow awaiting normal protected-branch integration and therefore not yet operational;
an unresolved/ambiguous record requiring owner review.

Do not auto-disable solely from path absence or naming. The inventory should open/update exact owner issues and require a second exact-state/intent check before any disablement.

Critical scope update: the central control plane itself has 256 workflow identities

Fresh live Actions inventory for ContextualWisdomLab/.github reports 256 workflow records. Many historical bootstrap, apply, finalizer, cleanup, branch-patch, and PR-specific records are still state: active, including examples such as:

bootstrap-npm-workspace-wiring.yml;
bootstrap-org-sweep-pr.yml;
bootstrap-pr608-fixes.yml;
bootstrap-trusted-uv-materializer.yml;
apply-pr782-artifact.yml;
apply-pr785-hash-boundary.yml;
agent-mention-payload-binding-finalize.yml;
numerous historical cleanup/patch/finalizer identities.

A fresh protected-main contents lookup returns 404 Not Found for sampled active path .github/workflows/bootstrap-npm-workspace-wiring.yml. The source is absent, while the central workflow registry still advertises it as active.

This elevates the issue from downstream repository hygiene to a central control-plane integrity incident. The central repository cannot serve as authoritative fleet workflow inventory until its own registry is reconciled.

Additional constraints

Inventory and remediation must be paginated beyond the first 100 records; total_count=256 proves a single-page implementation is incomplete.
The central sweep must distinguish live required/reusable workflows from deleted repair identities and GitHub-owned dynamic workflows.
Before disabling any central record, re-fetch exact protected-main SHA, path presence, current workflow state, and known caller references. A deleted central reusable workflow identity may still be pinned by leaf repositories; those callers need a supported migration path rather than a silent disablement.
Preserve current agent-mention-*, required review/security, merge-scheduler, ruleset-audit, documentation-quality, and other supported control-plane workflows.
Do not grant the read-only inventory job organization-wide disable authority. Disablement remains a separate reviewed operator phase.

Acceptance now requires central self-reconciliation plus the downstream owner issues already linked. A fleet sweep that omits the second and third workflow pages is a false success.

Further confirmed fleet blast radius

Additional live repository inventories and protected-default-branch source checks confirm the same lifecycle defect in five more repositories:

fast-mlsirm: 102 workflow records; sampled active apply-reviewed-ho-mc-source-fix.yml is absent from protected main. Owner issue fast-mlsirm#809. The supported read-only Hourly PR Queue Governance workflow remains present and its scheduled run 31585657618 succeeded, so it must be preserved.
naruon: 99 records; sampled active bootstrap-update-disksage-v4.yml is absent from protected develop. Owner issue naruon#1324.
contextual-orchestrator: 28 records; sampled active one-shot-nim-default-request-cap.yml is absent from protected main. Owner issue contextual-orchestrator#122.
inkspan: 48 records; sampled active apply-preparse-envelope-limits.yml is absent from protected main. Owner issue inkspan#278.
newsdom-api: 22 records; sampled active one-shot-finalize-review-fixes.yml is absent from protected develop. Owner issue newsdom-api#604.

The central inventory must now support repositories whose default branch is not main, distinguish current supported evidence/read-only workflows from deleted writers, and explicitly handle repositories with more than 100 records. The recurrence mechanism is organization-wide and cannot be closed by sampling a few high-profile repositories.

Additional confirmed owner repository

ContextualWisdomLab/codec-carver now joins the proven blast radius:

live registry: 19 workflow records;
sampled active rebuild-pr-317-clean.yml is absent from protected main;
several PR-specific rebuild/repair identities remain active;
owner issue: codec-carver#401.

Preserve CI, fuzzing, Dependabot, CodeQL, and GitHub-owned dynamic agent workflows. This further confirms that lifecycle classification must use exact source presence and owner intent rather than workflow names.

Expanded live blast radius and near-complete first-pass repository sweep

Fresh Actions-registry versus exact-default-branch checks now confirm the same orphan-workflow lifecycle defect in additional active product repositories:

OriginWeave: 37 records; sampled active format-pr1-once.yml absent from protected main. Owner issue OriginWeave#123.
ThreadWeave: 27 records; sampled active apply-pr20-concurrency-fix.yml absent from protected main. Owner issue ThreadWeave#31.
RankWeave: 30 records; sampled active apply-hourly-loop-repair.yml absent from protected main. Owner issue RankWeave#38.
DiagramWeave: 31 records; sampled active one-shot-definition-docs.yml absent from protected main. Owner issue DiagramWeave#27.
EgressWeave: 23 records; sampled active merge-main-pr39.yml absent from protected main. Owner issue EgressWeave#202.
four-pillars: 23 records; sampled active one-shot-calendar-repair.yml absent from protected main. Owner issue four-pillars#33.
keyverse: 42 records; sampled active bootstrap-federation-preflight.yml absent from protected main. Owner issue keyverse#99.
noema: 34 records; sampled active one-shot-noema-mode-metadata-repair.yml absent from protected main. Owner issue noema#226.
saju-caldav: active regenerate-radicale-runtime-lock.yml absent from protected main. Owner issue saju-caldav#33.
scopeweave, semantic-data-portal, xtrmLLMBatchPython, and EmbedRelay have also received exact owner findings; EmbedRelay additionally has a normal retirement PR removing a live protected-main branch-writing materializer.

The first-pass sweep also found clean/small registries in repositories whose inventories contain only current repository workflows and GitHub-owned dynamic identities. Therefore the central implementation must record both positive and negative evidence rather than assuming every repository is affected.

Required central data model refinement

A binary present/orphan flag is insufficient. The fleet now contains at least these states:

present_supported_active — current production workflow, e.g. DiagramWeave/ThreadWeave hourly paths;
orphan_active_retire — completed temporary writer, source absent, owner confirms disablement;
orphan_active_pending_integration — intended production workflow exists only on an active PR, e.g. Clearfolio hourly development;
present_temporary_active_pr_owned — bounded repair source still present and attached to live work;
dynamic_github_owned — CodeQL, Dependabot, Pages, Copilot/agent identities with no repository path;
disabled_historical;
unresolved_visibility_or_race — incomplete pagination, permission loss, branch movement, or ambiguous 404/403/5xx.

Disablement must require an exact second refetch plus owner intent and caller-impact analysis. In particular, central deleted reusable-workflow identities may still be pinned by leaf repositories; caller migration must precede disablement. A read-only detector should never hold organization-wide mutation authority.

The systemic RCA is now strongly supported across repositories with main, develop, and legacy default branches, registries above 100 records, and both current and historical hourly/product workflows. Closure requires a complete paginated organization ledger, owner-path remediation, central self-reconciliation of its 256 records, and post-disablement operational proof that supported hourly/review/security/release workflows still run.

BandScope downstream evidence is now complete for this fleet class. Against protected ContextualWisdomLab/bandscope develop@acdbea6344fe1231c39535b575f4de35e4c607c9, the live Actions registry was paginated to 59 active identities (20 + 20 + 19). Exact-tree reconciliation yields 10 present repository workflow sources, 7 GitHub-owned dynamic/* identities, and 42 active repository-path identities whose exact source path is absent from that protected tree. The full workflow-ID → path ledger and revalidation caveats are recorded on bandscope#847 (comment 5305656768).

The BandScope pass remained read-only, as required here: no registry state was mutated. The ledger deliberately classifies by exact path presence rather than workflow-name heuristics. Two entries (dependency-review.yml and opencode-review.yml) especially require provider-ownership revalidation immediately before an authorized disable because current required checks with similar names may now be centrally supplied even though the stale repository workflow identity/path is absent. The operator step should refetch workflow ID/state, exact protected default-branch SHA/tree, and provider ownership immediately before each mutation, then publish a complete after-inventory proving supported CI/security/release/dynamic workflows remain intact.

Fresh Inkspan owner evidence adds ContextualWisdomLab/inkspan#278 to this fleet class. On protected inkspan/main@e8109ec2a17de8bd6594487aa12c8c8a93cb2c03, the Actions registry currently reports 63 workflow identities. Workflow id 326648609 (.github/workflows/apply-preparse-envelope-limits.yml) remains state: active, while an exact contents read of that path at the protected SHA returns 404. Inkspan #278 owns repository-level operational remediation; this central issue should include Inkspan in the read-only inventory/ledger and lifecycle-prevention acceptance. Do not disable by name heuristic: revalidate each workflow id/path/state against the exact protected tree immediately before any reviewed operator mutation.

BandScope exact-evidence refresh — live dynamic-path contract and expanded orphan blast radius

Fresh BandScope evidence on 2026-08-17 materially updates the earlier #847 handoff:

protected default branch remains develop@acdbea6344fe1231c39535b575f4de35e4c607c9;
the live Actions registry now reports 59 workflow identities (the earlier central comment recorded 44);
protected develop contains only the current checked-in workflow set, while many historical bootstrap/finalizer/repair identities remain state: active with paths absent from that exact tree;
the live GitHub API also proves GitHub-managed workflows are represented without a separate source field, using paths such as:
- dynamic/agents/anthropic-code-agent;
- dynamic/agents/openai-code-agent;
- dynamic/dependabot/dependabot-updates;
- dynamic/dependabot/update-graph;
- dynamic/github-code-scanning/codeql.

This matters to the central contract: a classifier that depends on a synthetic source == github field will falsely label real GitHub-managed identities as unresolved. Conversely, accepting an arbitrary source: github field as an ownership override is unsafe because registry input is untrusted and it could mask an absent repository workflow. The reliable observed discriminator is the API's dynamic/ path namespace, while any other active non-.github/workflows/*.yml|yaml path should fail closed unless separately proven.

BandScope PR #895 now carries a repository-owned read-only detector and two regression layers for this exact contract. Current exact head 7f1e5cae02f497b58c51744e596633f8d3a95e20:

recognizes a real-shaped dynamic/agents/openai-code-agent record with no source field as GitHub-managed;
rejects forged source: github metadata on an absent .github/workflows/deleted-repair.yml record and still classifies it as an orphan;
keeps unknown external/... active paths unresolved.

Actual organization-wide inventory and disablement authority remain owned here; BandScope does not add privileged cleanup or suppress the control-plane defect.

Central acceptance tests added by this evidence

{id: 297018635, name: "OpenAI Codex", path: "dynamic/agents/openai-code-agent", state: "active"} with no source field must classify as GitHub-owned dynamic evidence, never orphan_active and never an unresolved repository-path failure.
{id: 1, name: "workflow", path: ".github/workflows/deleted-repair.yml", state: "active", source: "github"} with that path absent from the exact protected tree must remain an orphan; auxiliary metadata cannot override path/tree authority.
An active unknown path such as external/workflows/example must fail closed pending explicit ownership evidence.

feat(actions): inventory orphaned workflow identities

fast-mlsirm downstream evidence — 2026-08-20

Protected ContextualWisdomLab/fast-mlsirm/main@04d0bc21a2a20693bcf16108cd76d394fe844d23 was reconciled against the complete Actions registry (100 + 11 pages = 111 identities).

100 active repository-path orphan identities were revalidated by workflow ID/path and disabled through the normal Actions API; the resulting state is disabled_manually.
The post-mutation registry has 100 disabled_manually and 11 active.
The 11 active identities are exactly six protected-tree workflows plus five GitHub-owned dynamic/* identities; active_paths_missing_local=0.
The six protected workflows include CI, CodeQL, ClusterFuzzLite, Hourly PR Queue Governance, Release Tag, and Statistical Studies; none was disabled.
The hourly governance workflow continues to run successfully on the pinned protected SHA.

The repository-side audit and adversarial tests already cover pagination, branch movement, transient transport errors, exact path classification, dynamic identities, and read-only recurrence. This evidence should be included in the central fleet ledger; the mutation authority remains separate from the read-only detector as required by this issue.

fast-mlsirm review-control-plane follow-up

The repository currently retains protected required contexts strix and opencode-review, but historical workflow IDs 303795245 and 303795244 are absent from the current Actions registry. Existing failed/pending PR runs reference those IDs; direct log/rerun endpoints return HTTP 404.

Please include this as a central lifecycle/control-plane reconciliation item. The repository gate remains strict and unchanged; no source YAML was restored and no required context was weakened.

Storage allocation correction for the recovery receipt above: the 2026-09-27 S1 plan budgeted 160 GiB of virtual disk for three new runners (48/64/32 GiB). The observed full disk led to a 96 GiB OpenCode guest; current commitments are 48/96/32 = 176 GiB, 16 GiB above that historical cap. The S1 host had 240 GiB free on /data after the sparse-image resize, so this extension did not immediately allocate another 32 GiB of physical space. CPU and RAM assignments did not change. Treat the old 160 GiB storage cap as superseded for this guest and remeasure host free space before further growth.

2026-09-28 central OpenCode runner RCA and repair progress:

The prior No space left on device occurred before review code on cwlab-s1-04. After the guest filesystem was expanded to 96 GiB, live inspection found 15 completed-run coverage tool images and 29 completed-run coverage work directories still on that runner. The coverage sandbox writes as uid 65532, which also made the runner's normal temp cleanup report permission errors.
Confirmed the only active coverage run was 36383485414 and excluded it. Removed the 15 old images and 29 old attempt directories only. Guest free space increased from 32 GiB to 79 GiB; the active coverage job subsequently completed successfully.
PR #2491 makes each coverage attempt clean its own Docker image and work directory on all step outcomes. Local GITHUB_ACTIONS=true suite: 5,095 passed, 5 skipped, 40 subtests passed. It is a separate central CI repair, not part of feat(actions): inventory orphaned workflow identities #1026's diff.
feat(actions): inventory orphaned workflow identities #1026 remains open at 117a5b359d808dc2fe7b35bc335287a720529bc7 with required checks queued and no qualifying new approval. No merge claim or bypass is being made.

2026-09-28 central review queue capacity receipt:

At 08:25 UTC, the queue held 23 queued Required OpenCode Review runs and 30 queued Required Noema Review runs. The sole CWL MCP remediation runner cwlab-s1-02 was online and busy. This was runner admission pressure, not a feat(actions): inventory orphaned workflow identities #1026 source failure.
S1 host had 30 GiB available RAM and 234 GiB free on /data. Added isolated cwlab-s1-06 (3 vCPUs, 2-CPU service quota, 12 GiB RAM, 48 GiB virtual disk) using the existing verified Ubuntu image and guarded VM preparation script. The bounded allocation record is /data/contextualwisdomlab/actions-runners/central-runner-allocation-20260928.json on S1; the historical 2026-09-27 plan is preserved. This uses the available 2-CPU quota increment and 12 GiB of the observed memory headroom. It is a capacity choice, not a guaranteed queue-time forecast.
Registered the runner only in the restricted CWL MCP remediation group, which already selects trusted central-main review workflows and approved repositories. Registration token was passed by stdin and not stored. GitHub runner ID 1078916 is online and busy; the guest journal confirms Connected to GitHub, Listening for Jobs, and Running job: noema-review at 08:37:54 UTC.
Existing group runner cwlab-s1-02 remained online and busy. feat(actions): inventory orphaned workflow identities #1026 still requires exact-head checks and independent approval before normal merge.

fix(ci): scope coverage build cleanup to each attempt

2026-09-28 central control lane receipt:

#1026's Noema admission, OpenCode bootstrap, and PR queue scan were still queued after adding the reviewer lane. The Noema admission job requests self-hosted/linux/x64/cwlab-control; CodeQL's PR-authored language-detection job deliberately remains GitHub-hosted under the current trust policy, so this change does not claim to solve that separate queue.

At 08:40 UTC S1 had about 23 GiB available RAM, 228 GiB free on /data, 12 logical CPUs, and 7.41 load average. Added isolated cwlab-s1-07 as a restricted CWL central control runner (1 vCPU, 1-CPU quota, 4 GiB RAM, 32 GiB virtual disk). The resulting service quota sum can exceed 12 logical CPUs by one if every VM saturates at once; current load provided headroom, and the host will be monitored. The allocation record is /data/contextualwisdomlab/actions-runners/central-runner-allocation-control-20260928.json on S1. Existing VMs and runner groups were preserved.

GitHub runner ID 1079034 is online and busy. Guest service logs show Connected to GitHub, Listening for Jobs, and Running job: cancel-superseded-pr-runs at 08:48:08 UTC. Registration token was passed by stdin and not stored. This is observed job admission, not proof that #1026's required checks or independent review have passed.

2026-09-28 exact-ID workflow retirement receipt:

Registry ID 362687040, .github/workflows/windows-worktree-diagnostic.yml, was active before the write. DiskSage default main remained at 1f2d89ffc1c36a6cbaf003f21ad6c8248caa3230; the path is absent there. Open PR fix(opencode): prioritize current-head evidence #444's current head 07d81e8b03f165f67b48d5dc550e7a8d3ac6f74c and base a2e059bced3fa250927a0e92e5bfc7640437f898 also lack the path. The historical YAML had only a pull_request trigger, not workflow_call.
The workflow had exactly two recorded runs, both completed on 2026-09-20, with no queued or running run at the write boundary.
Issued only PUT /repos/ContextualWisdomLab/DiskSage/actions/workflows/362687040/disable. A fresh API read returned state: disabled_manually; default SHA was unchanged and no active run appeared.
If an owner proves this identity is still needed, the exact rollback is PUT /repos/ContextualWisdomLab/DiskSage/actions/workflows/362687040/enable after confirming current source and owner intent. No other workflow ID was changed.

2026-09-28 owner revalidation of two default-branch-absent registry IDs:

352036915 (actions-poll-analyzer-coverage.yml) is active in the registry and absent from default develop at 2949d30718752ea5915c7713ba227e8c19d9e5bf. Its file is present at open PR fix(opencode): retry OpenCode after coverage blockers clear #1152 head 255cfd866efd971396e132992a474074487f1939; that PR targets the feature branch carrying the analyzer. Its recorded runs are completed, but the open PR still owns the source.
338547094 (python-shell-ast-coverage.yml) is active in the registry and absent from that same default head. Its file is present at open PR fix(review): strip leftover range interiors from posted suggestions #998 current head efa3fe5271ca9b7ffe9a905cfe9ceef21953487b; a run for that exact head (36387342728) was queued today.

These IDs are default-branch-absent inventory findings, not safe disable targets while those PRs use the workflow files. No AppGuardrail workflow state was changed. Recheck the current PR heads, active runs, owner intent, and default branch after the PR lifecycles complete before considering exact-ID retirement. The earlier ledger evidence remains valid for default-branch absence only.

Correction and final exact-ID receipt (2026-09-28): the first disablement checked PR #444 but not every open PR branch, so I promptly re-enabled ID 362687040 while completing the broader audit. No other workflow ID was touched.

I paginated all 123 open DiskSage PRs and checked .github/workflows/windows-worktree-diagnostic.yml at every distinct current head and base SHA: 153 absent, 0 present, 0 API errors. The complete PR list was fetched again immediately afterward and its (PR number, head SHA, base SHA) set was unchanged. The local audit receipt is /private/tmp/disksage-open-pr-workflow-audit-20260928.json (SHA-256 5d56b33b07fc5c22844fae9cffcb1bae8bc01344d6cfc0a19c6aa93f2a9d5cdc); the second PR listing is /private/tmp/disksage-open-prs-final-20260928.json (SHA-256 b4bae3df575dda50d52ca555d8d091d8bdee2b69f83acc9e1d34057da2754dae). These local files are evidence pointers, not GitHub-hosted artifacts.

Immediately before the final write, default main was still 1f2d89ffc1c36a6cbaf003f21ad6c8248caa3230, the registry ID/path was active, and its only two historical runs were completed. The historical YAML had only a pull_request trigger, not workflow_call. I then issued the exact-ID disable call. A fresh API read now reports state: disabled_manually for 362687040; default SHA is unchanged. This is the verified final state. If the owner later restores the source and needs the same ID, use the exact-ID enable endpoint only after reviewing that new intent.

2026-09-28 live workflow registry check: Clearfolio has 25 workflow identities, of which 6 are active. The two active repository-owned YAML paths are .github/workflows/ci.yml (ID 311068487) and .github/workflows/fuzz.yml (ID 309141912); both exist at the unchanged default main SHA 06633a25109c62e24a7015ae04fb9f6e0a246f7e. The other four active identities use GitHub-owned dynamic/ paths (Copilot, two Dependabot entries, and code scanning). Thus this snapshot has zero active repository-owned default-branch-absent IDs. No Clearfolio workflow state was changed; historical disabled identities remain in the registry for audit.

2026-09-28 S1-07 routing correction:

The new control VM initially inherited ubuntu-24.04 and ubuntu-latest custom labels from the registration helper. A central noema-review job for seedream_evasepic#347 (run 36278117325, still the PR's current head) selected the workflow's generic Ubuntu fallback and landed on S1-07. This was an unintended long model job on a control lane. It is a legitimate current-head review, so it was not cancelled.

Removed only those two generic labels from org runner ID 1079034. Its verified labels now match the existing control VMs: self-hosted, Linux, X64, cwlab, cwlab-control, cwlab-s1-07. Updated and syntax-checked the S1 registration scripts so future control registration omits generic hosted-runner labels. The runner remains online; already-running work continues. Reviewer-lane S1-06 keeps its separate restricted group.

A read-only, fully successful 87-repository sample counted 52 active jobs (46 GitHub-hosted, 6 CWL runners) against the documented Team hosted concurrency limit of 60; this single snapshot does not prove a concurrency-cap blockage. #1026's current-head required jobs remain queued, and the PR remains unmerged.

Pre-merge central workflow-registry audit (2026-09-28 09:47 UTC; not the post-integration fleet ledger):

Current .github default branch: 3295c259bcb688673170a1902f46d1d6c775bad4. The pre-merge ledger had 32 active registry IDs whose files were absent there.
Revalidated all 360 open PR head/base identities. Exact GitHub tree-object lookups found 30 candidate paths on open PR heads identified from PR file changes. An exhaustive lookup of the other two paths across all 360 heads found one more on PR fix(opencode): replace probe quota with exact changed-path evidence #1701 (source-fix-1629-no-heuristic-compute.yml), despite its absence from that PR's changed-file list. Preserved all 31 branch-owned IDs.
The sole remaining candidate, workflow ID 357075215 (verify-security-scan-default-branch-contract.yml), was absent from all 360 open PR heads, all 103 distinct open PR bases, current main, and the tip of its original push-trigger branch. Commit 77a4eacb64014650d2818ced739c30e334185ddc explicitly retired its one-off source. The workflow had only three historical runs, all completed, and no current-main source references.
Disabled only ID 357075215 via the workflow-ID API. A follow-up GET confirms disabled_manually; the complete run listing remains three completed runs and zero active runs.

Reproducible local receipts: /tmp/945-github-open-prs-20260928.json; head-presence receipt SHA-256 4a15bf4bbd80b9966190a7da6bf85995ad40db689c81587cc92c1fed3738afe0; exhaustive two-path receipt SHA-256 1e38b2bb157541602cbafd97a75f279650dc9e3ed6cbd079a439c975a5a13131. PR #1026 remains open; a fresh complete post-integration fleet ledger and remaining owner routing are still required.

Pre-merge singleton candidate audit (2026-09-28 09:55 UTC; reviewed against the same SHA-bound pre-merge ledger, not a post-integration fleet claim):

Checked the one-candidate repositories beyond DiskSage: 21 repositories and 668 open PR heads. Exact source-object lookups found 17 candidate workflows present on at least one open PR head; these IDs were preserved. A second PR-identity sweep found one codec-carver head update, but its positive source proof is on the unchanged PR 🛡️ Sentinel: [HIGH] Fix Information Disclosure in CI Subprocess Logs #631 head.
The four without a source on a sampled open PR head received deeper checks. mcp-shared-gateway already restored standalone-gateway.yml on current main (3d5740695b2ca083e2d4fc7edb575c526201e906); the saved ledger was stale. codex ID 368448552 remains present on its active candidate branch. opencode ID 368561598 remains present on its build branch and has queued run 36347853396. Both were preserved.
pg-llm-batch ID 353220716 (lock-diagnostic.yml) was absent from current main (5913c4bad79d6bc29d7cc1c624abb7db2ea6a77c), its original branch tip, all 115 revalidated open PR heads, and all 71 distinct PR bases. The one-off, path-scoped push workflow was removed by commit e1045b6ed74e848cd99a50b02b42fe731fcc8b9b after the lock was admitted; its four historical runs are completed. Disabled only this ID. A follow-up GET confirms disabled_manually and zero active runs.

Local receipt: /tmp/945-singleton-head-presence-20260928.json, SHA-256 9ef0afeede8c0d4fce1bf1e083806325697358c479d741b6f093a03683318bec. Further review must use a fresh post-integration ledger and revalidate every candidate before owner routing or mutation.

Pre-merge lifecycle follow-up (2026-09-28 10:13 UTC):

Audited 20 default-branch-absent active IDs across eight repositories with exact open-PR head objects. Fifteen source files were present on at least one open PR head and were preserved. The five absent from all observed open PR heads and bases received separate branch/run review.
fast-mlsirm ID 353842644 still has its E2E source on the original branch tip; preserved. TEPP ID 351264644 was explicitly removed by commit b90f1d9e2c0d846288722530673c37c72453027e as purpose-complete, had 64 completed runs and no active run; disabled and verified disabled_manually. argos ID 351190665 was deleted from its branch, had one completed run, and the separate GitHub-owned CodeQL ID 276742938 remains active; disabled only the old repo-owned ID and verified both states.
mhtml-etl-gateway IDs 349182448 and 349719996 were left on closed PR Require fresh OpenCode reviews after branch updates #62's branch as success-reporting placeholders for dependency and Noema checks. Neither was on current main or any of 51 open PR heads/three bases; both had only completed runs. Disabled both exact IDs, verified disabled_manually, and routed owner follow-up to [Security] Retire branch-only placeholder review workflows mhtml-etl-gateway#121.

Queue RCA snapshot at 09:58–10:04 UTC: .github returned 654 queued runs (653 unique IDs after pagination overlap), 585 older than one hour and 67 older than one day. Among names that bind a target PR, queued CodeQL dispatch (124), Noema (29), and OpenCode (27) each had one run per PR, so a blanket duplicate cancellation is unsupported. A separate non-atomic organization snapshot saw 60 running workflow runs and 55 running jobs (52 GitHub-hosted, three self-hosted); it does not prove the Team 60-job hosted ceiling was reached. All four central-control group runners were busy when checked. Historic Noema run 32214648177 binds the old head of now-closed .github#1131, but both normal and force cancellation returned HTTP 409 while GitHub still reported it queued; no cancellation was claimed. Current-head #1026 and #2491 jobs remain queued. No current-head run or unrelated inference was cancelled.

Local read-only receipts: /tmp/945-smallgroups-head-presence-20260928.json SHA-256 b81071f921b7cabcc917a401251019cebc6335f7648b47e9790017967a5351b8; /tmp/945-central-queued-runs-20260928.ndjson SHA-256 31df565888f6914a6226d6e79851c537460d56cbaa85e58fe9652f9ff8e63ba6; /tmp/945-org-inprogress-job-snapshot-20260928.json SHA-256 919503940028bfe50b02e51fd37d26a66961338a5c6b0f206888468bd62325ea. A new post-integration fleet ledger remains required after #1026 merges.

Pre-merge owner routing update (2026-09-28 10:26 UTC): exact current default-branch SHA, complete registry ID/state, and open PR head/base tree checks were used before routing. The following 639 distinct active, default-branch-absent candidate IDs from the saved fleet ledger are now listed by ID/path in their owning repository's issue. This is owner handoff, not a claim that all 639 are retired or disabled:

life-os: 313 IDs in #202; 10 additional IDs had source on open PR heads and were preserved.
LineageWeave: 209 IDs in #1134.
quarantine-sandbox-runtime: 66 IDs in #147; two open-PR-head IDs preserved.
wardnet: 27 IDs in #452.
naruon: 15 IDs in #1324; five open-PR-head IDs preserved.
pingora-gateway: two IDs in #121; seven open-PR-head IDs preserved.
supply-chain-control-plane: seven IDs in #3; one open-PR-head ID preserved.

Additional exact-ID cleanup this turn: .github 357075215, pg-llm-batch 353220716, TEPP 351264644, argos 351190665, and mhtml-etl-gateway 349182448/349719996 were separately revalidated, disabled, and re-read as disabled_manually; the latter two have owner follow-up #121. Current branch-owned IDs, GitHub-owned dynamic CodeQL, and queued legitimate model jobs were preserved.

PR #1026 and central CI repair PR #2491 remain open with current-head checks queued. Post-integration full-fleet ledger, per-ID owner disposition, qualifying review, terminal checks, and normal merge remain outstanding.

2026-09-28 11:12 UTC update for #1026 (7e8106e49a9c45004c7e5bd73f8f15701d89f315) and central cleanup #2491 (1769d25a82ca4929ec46497ccb7ba60b5caacf17): both remain open and protected-merge blocked. #1026 has no new current-head approval, 14 queued checks, and no current-head failed conclusion.

The native Strix run 36401126412 was cancelled at 10:13:13 UTC because a same-PR, same-head repository_dispatch replacement, run 36408376060, was created at 10:13:11 UTC. The replacement remains queued. No further duplicate was dispatched.

Exact job API evidence at 10:58 UTC: CodeQL run 36401129119, job 108859001714 (Detect CodeQL languages) requests ubuntu-24.04 and has runner_id=0; Noema run 36401126378, jobs 108858993168 and 108858993444, request self-hosted,linux,x64,cwlab-control and have runner_id=0; replacement Strix run 36408376060, jobs 108882443011 and 108882443394, request the same control labels and have runner_id=0. OpenCode run 36401126381 has three queued self-hosted,linux,x64 jobs, all runner_id=0. These are admission/assignment observations, not source-test failures.

Runner group CWL central control allows the trusted main workflow paths and had four online members; its occupancy changed from all busy to partly/all idle during sampling while the above jobs remained queued. This does not isolate GitHub scheduling versus organization-wide concurrency as the sole cause. The S1 host had 7.7 GiB available RAM and all 2 GiB swap used at 10:50 UTC, so adding another VM there is not a safe capacity fix. #2491's per-attempt coverage cleanup remains under checks/review.

No current-head run was cancelled, no review/security gate was bypassed, and neither PR was merged. Continue exact-head review/check tracking and diagnose any terminal failure from logs.

Navigation Menu

Repository navigation

[Fleet incident] Govern orphaned GitHub Actions workflow lifecycle organization-wide #945

Systemic fleet defect

Root cause

Required central architecture

Prevention policy

Acceptance criteria

Additional confirmed blast radius

Critical scope update: the central control plane itself has 256 workflow identities

Additional constraints

Further confirmed fleet blast radius

Additional confirmed owner repository

Expanded live blast radius and near-complete first-pass repository sweep

Required central data model refinement

BandScope exact-evidence refresh — live dynamic-path contract and expanded orphan blast radius

Central acceptance tests added by this evidence

fast-mlsirm downstream evidence — 2026-08-20

fast-mlsirm review-control-plane follow-up

24 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[Fleet incident] Govern orphaned GitHub Actions workflow lifecycle organization-wide #945

Description

Systemic fleet defect

Root cause

Required central architecture

Prevention policy

Acceptance criteria

Activity

seonghobae commented on Aug 12, 2026

Additional confirmed blast radius

seonghobae commented on Aug 12, 2026

Critical scope update: the central control plane itself has 256 workflow identities

Additional constraints

seonghobae commented on Aug 12, 2026

Further confirmed fleet blast radius

seonghobae commented on Aug 12, 2026

Additional confirmed owner repository

seonghobae commented on Aug 12, 2026

Expanded live blast radius and near-complete first-pass repository sweep

Required central data model refinement

seonghobae commented on Aug 16, 2026

seonghobae commented on Aug 16, 2026

seonghobae commented on Aug 17, 2026

BandScope exact-evidence refresh — live dynamic-path contract and expanded orphan blast radius

Central acceptance tests added by this evidence

seonghobae commented on Aug 20, 2026

fast-mlsirm downstream evidence — 2026-08-20

seonghobae commented on Aug 20, 2026

fast-mlsirm review-control-plane follow-up

24 remaining items

seonghobae commented on Sep 28, 2026

seonghobae commented on Sep 28, 2026

seonghobae commented on Sep 28, 2026

seonghobae commented on Sep 28, 2026

seonghobae commented on Sep 28, 2026

seonghobae commented on Sep 28, 2026

seonghobae commented on Sep 28, 2026

seonghobae commented on Sep 28, 2026

seonghobae commented on Sep 28, 2026

seonghobae commented on Sep 28, 2026

seonghobae commented on Sep 28, 2026

seonghobae commented on Sep 28, 2026

seonghobae commented on Sep 28, 2026

seonghobae commented on Sep 28, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions