From 772aecd6363dabfadf804788b654d47be08bd426 Mon Sep 17 00:00:00 2001 From: Rafael Brandao Date: Thu, 10 Sep 2026 14:22:00 -0700 Subject: [PATCH] Add agent-authored metadata to the Reviews skill An agent with no model key and no ArchDev session can now freeze the diff with `reviews manifest`, write archdev.review-metadata.v1 annotations, and serve them with `reviews local --metadata`, or publish the same file with `reviews publish --metadata`. The reference explains the file, the rules the CLI enforces (changed-range overlap, per-path coverage, the diff_sha256 fence), and the publish caveats. Bootstrap now also probes `reviews manifest --help`, which ArchDev v0.36.0 provides. --- README.md | 4 +- skills/reviews/SKILL.md | 13 ++- skills/reviews/references/agent-metadata.md | 118 ++++++++++++++++++++ skills/reviews/scripts/bootstrap.ps1 | 4 +- skills/reviews/scripts/bootstrap.sh | 3 +- tests/reviews-skill.sh | 2 + 6 files changed, 138 insertions(+), 6 deletions(-) create mode 100644 skills/reviews/references/agent-metadata.md diff --git a/README.md b/README.md index 0d6ee5d..e986d20 100644 --- a/README.md +++ b/README.md @@ -120,7 +120,9 @@ npx skills add ArchAstro/archdev --skill reviews ``` Ask your coding agent to open local review and iterate on your feedback. No -Task, PR, or daemon is required for that local loop. +Task, PR, or daemon is required for that local loop, and the agent can author +the risk/theme annotations itself with `reviews manifest` and +`reviews local --metadata`, with no model key or ArchDev login. ## Repository scope diff --git a/skills/reviews/SKILL.md b/skills/reviews/SKILL.md index 1a51817..da27679 100644 --- a/skills/reviews/SKILL.md +++ b/skills/reviews/SKILL.md @@ -42,6 +42,7 @@ Do not install/start Jobs simply to open a local review. | Intent | Command | | --- | --- | | Human review of local code | `reviews local --feedback-format jsonl --no-open` | +| Author the review metadata yourself, no model or login | `reviews manifest`, then `reviews local --metadata ` | | Open the Needs review inbox | `reviews inbox` (or `reviews`) | | Open an existing PR | `reviews open ` | | Discover/inspect AI review DAGs | `reviews workflows list` / `show ` | @@ -49,9 +50,11 @@ Do not install/start Jobs simply to open a local review. | Generate publication JSON only | `reviews generate pull-request` / `metadata` | | Publish/update a branch PR | `reviews publish` | -Read [workflows.md](references/workflows.md) for AI review DAG features and +Read [workflows.md](references/workflows.md) for AI review DAG features, [site-and-publication.md](references/site-and-publication.md) for site/GitHub -access, browser features, publication, and the Jobs handoff. Task-plan review +access, browser features, publication, and the Jobs handoff, and +[agent-metadata.md](references/agent-metadata.md) to write the risk/theme +annotations yourself instead of generating them with a model. Task-plan review uses the separate Tasks workflow; do not use its session-file/revision commands for code review. @@ -64,7 +67,11 @@ for code review. 2. Run `"$archdev" auth status`; if signed out, run `auth login`, let the human finish browser sign-in, and verify it. Model metadata generation requires usable model access; use the Agents skill's provider setup when available. - `--model ` chooses the metadata model. + `--model ` chooses the metadata model. Without a + session or provider key, or when the human wants your judgment on the + diff, skip this step and follow [agent-metadata.md](references/agent-metadata.md): + `reviews manifest` plus `reviews local --metadata ` need neither. + The CLI refuses a model it cannot reach before freezing anything. 3. Choose a valid local base ref. Default is `origin/main`; use `--base HEAD` for current working changes without the branch's earlier committed diff, or the requested branch base. Fetch a known remote ref when needed; do not diff --git a/skills/reviews/references/agent-metadata.md b/skills/reviews/references/agent-metadata.md new file mode 100644 index 0000000..f0d858d --- /dev/null +++ b/skills/reviews/references/agent-metadata.md @@ -0,0 +1,118 @@ +# Author review metadata yourself + +Use this path when no model access or ArchDev session is available, or when +the human wants your judgment on the diff rather than a model's. You freeze +the change set, write sparse risk, theme, and note annotations against it, and +serve them through the same `reviews local` browser flow. The CLI validates +every annotation against the frozen diff; you supply what a reviewer must look +at. Only publishing the same file to a pull request needs a login. + +## 1. Freeze and read the change set + +```sh +"$archdev" reviews manifest --base origin/main > manifest.json +``` + +The manifest captures committed, staged, unstaged, and non-ignored untracked +changes against the merge base of `--base` and HEAD, the same snapshot rules +as `reviews local`. It prints `files[]` and `changes[]`: the annotatable text +ranges with `path`, `side`, 1-based `start_line` and `end_line`, and their +`patch`. Binary, rename-only, and mode-only changes appear in `files[]` +without a range and cannot carry annotations. Keep `diff_sha256`; it fences your +metadata to this exact tree. + +Read the patches before annotating. Open the file when a patch is not enough +to judge a change. Do not annotate from memory of what you intended to change, +and do not edit any file between this step and step 3. + +## 2. Write the metadata file + +Write one UTF-8 JSON file outside the repository, for example in the harness +scratch directory, with schema `archdev.review-metadata.v1`. Read +`"$archdev" reviews guide` for the installed CLI's exact contract; it wins +when this page and the CLI disagree. + +```json +{ + "schema": "archdev.review-metadata.v1", + "diff_sha256": "", + "summary": { + "intent": "One paragraph: what the change does and why.", + "overall_risk": "medium" + }, + "pull_request": { + "title": "Short title", + "body_markdown": "PR body a reviewer would want.", + "draft": false + }, + "annotations": [ + { "kind": "risk", "path": "src/auth.ts", "side": "modified", "start_line": 10, "end_line": 22, "risk": "high", "note": "Audience is compared before the issuer is verified." }, + { "kind": "semantic_group", "path": "src/auth.ts", "side": "modified", "start_line": 30, "end_line": 41, "semantic_group": "Auth boundary", "note": "Callback validation and its tests." }, + { "kind": "semantic_group", "path": "test/auth.test.ts", "side": "modified", "start_line": 5, "end_line": 40, "semantic_group": "Auth boundary", "note": "Covers the rejected-audience path." }, + { "kind": "note", "path": "src/legacy.ts", "side": "original", "start_line": 3, "end_line": 3, "note": "Removed export had no remaining callers." } + ] +} +``` + +Rules the CLI enforces, and how to satisfy them: + +1. Every annotation must overlap a `changes[]` entry on the same `path` and + `side`. `modified` is the new file for added or changed lines; `original` + is the old file for removed lines. A rejected file names the failing index. +2. `risk` is `critical`, `high`, `medium`, or `low`, and its `note` states a + concrete way the change can be wrong: auth, data loss, concurrency, + compatibility, cost. Do not label a range high risk without such a reason. +3. Ranges that belong together across files share one `semantic_group` + label. The reviewer can accept a whole group at once, so group by what one + person would verify in one sitting. +4. `note` carries context the diff cannot show: an invariant, a follow-up, a + reason a suspicious-looking change is safe. +5. Cover every path listed in `changes[]` with at least one entry; the CLI + rejects a file that leaves a changed path unannotated. For generated or + mechanical files add one `note` saying why they need little review. + Beyond that, be sparse: annotate what a reviewer must look at, not every + hunk. +6. `summary` and `pull_request` are optional for a local review. + `pull_request` is required for publishing. `overall_risk` defaults to the + highest annotation risk when `summary` is absent. + +## 3. Serve it + +```sh +"$archdev" reviews local --base origin/main --metadata metadata.json --feedback-format jsonl --no-open +``` + +`--metadata` replaces model generation: no `auth status`, no provider, no +`--model`. The server starts with the analysis already ready, and the browser +shows your summary banner, the risk labels, and the themes. From the `ready` +record onward, follow the main skill's launch, listen, and iterate steps +unchanged. Rerun steps 1 to 3 for another pass after edits. + +If the CLI reports that the working tree changed since the manifest ran, +regenerate the manifest and the metadata; do not edit the fence by hand. +`reviews local --no-metadata` serves the diff with no annotations when the +human only wants to read the change. + +## 4. Publish the same file to a pull request + +Publishing writes GitHub and ArchDev's hosted review store, so it needs +`"$archdev" auth login` and a working `gh auth status`. With `pull_request` +present in the file: + +```sh +"$archdev" reviews publish --base main --metadata metadata.json +``` + +Publish validates the annotations against the exact PR head, sets the PR +title and body from `pull_request`, and stores the annotations for the hosted +reviewer. The `diff_sha256` fence is checked here too: publish rejects the +file before pushing when the committed diff differs from the diff you +reviewed. Committing identical content keeps the digest valid. If content +changed, regenerate the annotations and the digest together; copying a new +digest onto old annotations defeats the fence. Publish sets the title and +body only for a PR it creates, and does not replace annotations already +stored for the same head. Tell the human before running it that it pushes +the branch, and follow the repository's own commit and push rules. + +Never put secrets, tokens, or customer data in annotations, summaries, or PR +bodies. diff --git a/skills/reviews/scripts/bootstrap.ps1 b/skills/reviews/scripts/bootstrap.ps1 index 550013e..273f65f 100644 --- a/skills/reviews/scripts/bootstrap.ps1 +++ b/skills/reviews/scripts/bootstrap.ps1 @@ -37,7 +37,9 @@ function Test-Reviews([string]$Binary) { $helpText = & $Binary reviews local --help 2>$null if ($LASTEXITCODE -ne 0 -or (($helpText -join "`n") -notmatch "(?m)^Usage: archdev reviews local ")) { return $false } $helpText = & $Binary reviews workflows run --help 2>$null - return ($LASTEXITCODE -eq 0 -and (($helpText -join "`n") -match "(?m)^Usage: archdev reviews workflows run ")) + if ($LASTEXITCODE -ne 0 -or (($helpText -join "`n") -notmatch "(?m)^Usage: archdev reviews workflows run ")) { return $false } + $helpText = & $Binary reviews manifest --help 2>$null + return ($LASTEXITCODE -eq 0 -and (($helpText -join "`n") -match "(?m)^Usage: archdev reviews manifest ")) } if (-not (Test-Reviews $archdev)) { diff --git a/skills/reviews/scripts/bootstrap.sh b/skills/reviews/scripts/bootstrap.sh index 20e160a..ac9267e 100755 --- a/skills/reviews/scripts/bootstrap.sh +++ b/skills/reviews/scripts/bootstrap.sh @@ -68,7 +68,8 @@ fi supports_reviews() { "$1" reviews local --help 2>/dev/null | grep -Fq "Usage: archdev reviews local " && - "$1" reviews workflows run --help 2>/dev/null | grep -Fq "Usage: archdev reviews workflows run " + "$1" reviews workflows run --help 2>/dev/null | grep -Fq "Usage: archdev reviews workflows run " && + "$1" reviews manifest --help 2>/dev/null | grep -Fq "Usage: archdev reviews manifest " } if ! supports_reviews "$executable"; then diff --git a/tests/reviews-skill.sh b/tests/reviews-skill.sh index a6c9055..0c4576e 100755 --- a/tests/reviews-skill.sh +++ b/tests/reviews-skill.sh @@ -18,6 +18,7 @@ bootstrap="$root/project/.agents/skills/reviews/scripts/bootstrap.sh" test -x "$bootstrap" test -f "$root/project/.agents/skills/reviews/references/site-and-publication.md" test -f "$root/project/.agents/skills/reviews/references/workflows.md" +test -f "$root/project/.agents/skills/reviews/references/agent-metadata.md" # Substitute only the release boundary: no real installation or account writes. cat > "$root/installer/install.sh" <<'INSTALLER' @@ -31,6 +32,7 @@ cat > "$ARCHDEV_INSTALL_DIR/archdev" <<'CLI' if [[ "$*" == '--version' ]]; then echo fixture; exit 0; fi if [[ "$*" == 'reviews local --help' ]]; then echo 'Usage: archdev reviews local [options]'; exit 0; fi if [[ "$*" == 'reviews workflows run --help' ]]; then echo 'Usage: archdev reviews workflows run [options] '; exit 0; fi +if [[ "$*" == 'reviews manifest --help' ]]; then echo 'Usage: archdev reviews manifest [options]'; exit 0; fi exit 1 CLI chmod +x "$ARCHDEV_INSTALL_DIR/archdev"