Repository navigation
Description
User Story
As an OpenShell maintainer labeling accepted issues and PRs, I need one managed runner that previews and applies classifications through /label-issue so I can validate the workflow before enabling automatic dispatch.
Description
Deliver the managed classifier, invocation selector, and one-item runner in .agents/agents/labeler/. Accept an issue/PR number or URL; prepare evidence, expand it as needed, classify against the trusted skill and current result contract, and preview the proposed changes. Explicit application uses the saved exact preview. Human force bypasses only unchanged detection.
Keep eligibility, evidence handling, semantic validation, label mutations, checkpoints, and reporting in the shared helper. Ineligible, excluded, held, or unchanged items stop before model invocation. Lookup failures remain errors; partial evidence remains partial. Recheck current evidence and eligibility before writes, preserve label-event checkpoint verification, and reconcile uncertain writes without blind replay. Ambiguity stays in local reports; the labeler posts no comments.
The classifier receives prepared evidence without tools, MCP servers, or supplied GitHub credentials. Pin its configuration, bound cost and execution time, preserve session references, and recover failed sessions. Consolidate request preview and authorized application under the runner's entry point.
Select explicit issue/PR acceptance events and authorized human PR-comment/manual invocations, including drafts and force, without independently deciding acceptance. Install no automatic dispatch or backlog execution in this issue.
Suggested UX
Invoke /label-issue <number-or-URL> for a preview, inspect the exact changes, then authorize one-item application. The same runner serves later automated callers.
Context
- Parent: chore: create an invocable agent for workflow and topic labeling #5276. Implementation: feat(review): add labeling agent in Claude Managed Agents #5407.
- Reuse feat(review): automatically label new PRs for accepted work #5260's accepted-work policy and feat(skills): extract issue and PR labeling #5355's shared helper. chore: prepare area and topic labels for migration #5151 supplies the label/routing prerequisites for live application, including the approved checkpoint label.
- Automatic dispatch and coordination across entry points remain in chore: create an invocable agent for workflow and topic labeling #5276. The
GITHUB_TOKENacceptance writer needs an explicit hook, including drafts and independently of reviewer-routing success, because its label write does not trigger another workflow. - Follow the migration experiment's validation guidance. Distinguish model confidence, validator acceptance, and human-reviewed correctness; do not repair invalid proposals into apparent success or weaken validation.
Definition of Done
- One entry point supports issue/PR numbers or URLs, preview, explicitly authorized saved-preview application, human force, and model-request preview.
- Deterministic skips make no model call; evidence expansion, classification, and shared validation/application work end to end without duplicate helper or result-schema logic.
- The pinned classifier enforces tool/credential isolation, budget/deadline bounds, completion checks, and failed-session recovery.
- Invocation selection covers acceptance events, bot-applied acceptance, drafts, and authorized human commands without activating automatic dispatch.
- Failures retain execution references and distinguish stale inputs, missing prerequisites, invalid output, and uncertain writes; no false successful checkpoint is recorded.
- CI-registered offline tests cover orchestration, authorization, pagination, completion, evidence coverage, and failure/recovery boundaries, including the linked experiment's failure cases.
- Observed live runs verify a dedicated CMA configuration, trusted labeling identity, scoped credentials, and budget. Cover legacy migration, missing evidence, unchanged skips, force, and partial failures, starting with pinned Sonnet. Measure CMA cost/latency rather than importing CLI estimates; never commit credentials.
- Document setup, invocation, the concise human report, and recovery.
Activity
[-]feat(review): add managed labeling runtime foundation[/-][+]feat(review): deliver managed labeling runtime and one-item runner[/+]purp commentedon Nov 19, 2026
Combined #5411 into this issue after its runner implementation was merged into the branch for #5407. This issue now covers the managed classifier, invocation selector, and end-to-end one-item runner, including their shared-helper boundaries and live validation. The description was tightened with the omit-needless-words skill; #5411's migration-experiment guidance remains linked.
#5411 is superseded by this issue. Automatic dispatch and coordination across entry points remain tracked in #5276.