Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
107 changes: 107 additions & 0 deletions .agents/agents/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
# Agents

Agents live in `.agents/agents/<name>/`. Names are descriptive nouns specifying the job, optionally qualified by adjectives: `labeler`, `reviewer`, or `security-reviewer`.

## Layout

```text
.agents/
|-- skills/
| `-- agent-behavior -> ../agents/agent-name/skills/agent-behavior
`-- agents/
|-- README.md
`-- agent-name/
|-- README.md
|-- run.py
|-- bin/
| |-- helper.py
| `-- tool.py
|-- prompts/
| `-- task.md
|-- skills/
| `-- agent-behavior/
| `-- SKILL.md
|-- references/
| `-- result-contract.md
|-- tests/
| |-- run_test.py
| `-- tool_test.py
|-- deploy/
| |-- Dockerfile
| `-- policy.yaml
`-- evals/
`-- cases.json
```

Only `README.md` and a `run.*` entry point are required. Add other directories when needed; do not create empty scaffolding.

- `run.*` is the agent-root entry point for humans, workflows, and interactive callers. Use the implementation language's extension, such as `run.py` or `run.sh`. It can delegate to a shared launcher.
- `bin/` holds supporting scripts, tools, and implementation modules.
- `prompts/` holds model or harness prompts.
- `skills/` holds canonical agent-owned skills. A symlink in [`.agents/skills/`](../skills) can expose the same skill for discovery. Reference canonical instructions and contracts rather than copying them.
- `references/` holds contracts and supporting guidance.
- `tests/` holds explicitly selected deterministic and integration tests.
- `evals/` holds model-quality cases and scoring code.
- `deploy/` holds execution-specific packaging assets, such as a Dockerfile and sandbox policy. Manifests and dependency files live at the agent root where their tooling expects them. Never commit secrets.
- `README.md` covers setup, invocation, permissions, outputs, and recovery.

## Deterministic orchestration

The `labeler` agent checks authorization, acceptance, lifecycle, and material changes before calling a model-based classifier. The classifier receives prepared evidence and no tools; deterministic code validates and applies the result.

```text
labeler/
|-- README.md
|-- run.py
|-- bin/
| |-- acceptance.py
| |-- classifier.py
| |-- evidence.py
| `-- policy.py
|-- prompts/
| `-- classify.md
|-- skills/
| `-- label-issue/
| |-- SKILL.md
| `-- references/
| `-- result-contract.md
`-- tests/
|-- run_test.py
|-- acceptance_test.py
|-- classifier_test.py
|-- evidence_test.py
`-- policy_test.py
```

Execution: `run.py` -> deterministic gates -> prepared evidence -> model-based classifier -> validation and application. Deterministic skips create no model session. Acceptance recognizes established policy and human disposition; the model does not accept work.

## Harness-driven execution

A reviewer launches a model/harness with a prompt and selected tools. Its entry point enforces invocation scope and permissions; sandbox policy, guarded commands, or SDK tool handlers enforce tool access.

```text
reviewer/
|-- README.md
|-- run.sh
|-- agent.yaml
|-- bin/
| |-- read-item
| `-- submit-findings
|-- prompts/
| `-- review.md
|-- skills/
| `-- review-item/
| `-- SKILL.md
|-- references/
| `-- result-contract.md
|-- deploy/
| |-- Dockerfile
| `-- policy.yaml
`-- tests/
|-- run_test.sh
`-- submit_findings_test.sh
```

Execution: `run.sh` -> scope checks -> harness with prompt and tools -> validated result. Tools can be registered SDK functions or guarded sandbox commands. Supervision and repeated cycles belong outside the model.

These examples illustrate execution styles, not an inventory of agents.
File renamed without changes.
File renamed without changes.
32 changes: 14 additions & 18 deletions scripts/agents/gator/README.md → .agents/agents/gator/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

Launch a headless sandbox agent that runs the `gator-gate` skill against OpenShell issues and pull requests. The default and currently only supported harness is Codex.

The agent-root entry point is `.agents/agents/gator/run.sh`.

## Prerequisites

- `gh` is authenticated on the host and has access to `NVIDIA/OpenShell`.
Expand All @@ -13,14 +15,13 @@ Launch a headless sandbox agent that runs the `gator-gate` skill against OpenShe
## Usage

```shell
./scripts/agents/run.sh \
--agent gator \
./.agents/agents/gator/run.sh \
--gateway docker-dev \
--harness codex \
"Run gator on PR 1536 and keep watching until it closes or merges."
```

By default the launcher uses `scripts/agents/gator/Dockerfile` as the sandbox image source. It builds `scripts/agents/gator/` as the image context, so gator-specific image files such as `policy.yaml` and `bin/gh` stay with the gator agent. The launcher bakes a task-independent prompt template, skills, subagents, and shared runtime files into `/etc/openshell/agent-payload`, then passes the resulting image reference to `openshell sandbox create`.
By default the launcher uses `.agents/agents/gator/Dockerfile` as the sandbox image source. It builds `.agents/agents/gator/` as the image context, so gator-specific image files such as `policy.yaml` and `bin/gh` stay with the gator agent. The launcher bakes a task-independent prompt template, skills, subagents, and shared runtime files into `/etc/openshell/agent-payload`, then passes the resulting image reference to `openshell sandbox create`.

The PR assignment, operator test authorization, harness selection, watch mode, and polling interval are runtime inputs, not rendered image contents. The launcher supplies the non-secret assignment as `OPENSHELL_AGENT_TASK` in the canonical process command. The supervisor assembles the prompt once in memory and sends it to each harness cycle on stdin; cycles do not read task instructions from an agent-writable file. Assignment text is literal data, so shell syntax and template tokens inside it are not evaluated. Do not put credentials in assignments; use providers. Runtime command metadata can contain the assignment. This separates reusable tooling from per-run instructions; it does not make prompt instructions a security boundary.

Expand All @@ -41,8 +42,8 @@ requires the matching current kube context and a running local cluster, builds
with Docker, and imports the image into that cluster before creating a sandbox:

```shell
OPENSHELL_WORKSPACE=gator mise exec k3d@5.9.0 -- ./scripts/agents/run.sh \
--agent gator --gateway openshell-dev-main \
OPENSHELL_WORKSPACE=gator mise exec k3d@5.9.0 -- ./.agents/agents/gator/run.sh \
--gateway openshell-dev-main \
--k3d-cluster openshell-dev-main --name gtr-pr-1234 --watch \
"Review and monitor PR #1234 through the gator-gate workflow."
```
Expand All @@ -57,8 +58,8 @@ changing gateway settings, importing providers, reading provider credentials,
or launching a sandbox:

```shell
./scripts/agents/run.sh --agent gator --gateway remote-kube \
--publish-image registry.example.com/team/gator:payload-13 --build-only
./.agents/agents/gator/run.sh --gateway remote-kube \
--publish-image registry.example.com/team/gator:payload-14 --build-only
```

The builder uses existing Docker/Podman registry authentication. Both local and
Expand All @@ -71,8 +72,8 @@ The launcher prints `Tooling image: REGISTRY/REPOSITORY@sha256:DIGEST` after a
successful push. Reuse that full reference for subsequent tasks:

```shell
OPENSHELL_WORKSPACE=gator ./scripts/agents/run.sh \
--agent gator --gateway remote-kube --name gtr-pr-1234 --watch \
OPENSHELL_WORKSPACE=gator ./.agents/agents/gator/run.sh \
--gateway remote-kube --name gtr-pr-1234 --watch \
--image "$GATOR_IMAGE" \
"Review and monitor PR #1234 through the gator-gate workflow."
```
Expand Down Expand Up @@ -122,7 +123,7 @@ The launcher:
Gator-owned GitHub review thread without touching human review threads.
- Installs `gator/bin/validate-review-findings` to downgrade blockers that lack the required reachability, ownership, base-vs-head, impact, and reproducer evidence.
- Keeps that normalized evidence as Gator's internal review contract, then renders validated blockers for people as a read-aloud `Summary`, an actionable `Fix`, and a deterministic `Verify`. Exact paths and only the additional provenance an implementation agent needs appear in collapsed `Agent context`; raw evidence headings such as `Base` and `Head` are not posted publicly. Review-process provenance, docs and E2E disposition, SHAs, and state codes appear at the end of the summary in collapsed `Gator metadata`, while required human actions remain visible.
- Bakes `scripts/agents/gator/skills/gator-gate/SKILL.md` into `/etc/openshell/agent-payload`.
- Bakes `.agents/agents/gator/skills/gator-gate/SKILL.md` into `/etc/openshell/agent-payload`.
- Bakes `.claude/agents/principal-engineer-reviewer.md` so the selected harness can run a deterministic independent reviewer execution through `/etc/openshell/agent-payload/runtime/subagent.sh principal-engineer-reviewer < task.md`.
- For `--harness codex`, optionally bakes a host Codex executable as `/etc/openshell/agent-payload/runtime/harnesses/codex/codex`.
- Starts the selected harness without a TTY.
Expand All @@ -138,13 +139,8 @@ The launcher preserves existing gateway-owned Codex refresh material by default

## Tests

The test suite requires Bash, Ruby, Node.js, jq, and ripgrep on the host. It mocks image, gateway, credential, and GitHub operations.

```shell
bash scripts/agents/gator/bin/gh_guard_test.sh
bash scripts/agents/gator/bin/review_feedback_ledger_test.sh
bash scripts/agents/gator/bin/resolve_gator_review_threads_test.sh
bash scripts/agents/runtime/harnesses/codex/exec_test.sh
bash scripts/agents/run_test.sh
bash scripts/agents/image_test.sh
bash scripts/agents/runtime/launch_test.sh
bash scripts/agents/runtime/supervisor_test.sh
mise run test:gator
```
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# SPDX-License-Identifier: Apache-2.0

id: gator
payload_version: 13
payload_version: 14
display_name: Gator Gate Agent
description: Validate and monitor OpenShell GitHub issues and pull requests through the gator state machine.

Expand Down
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Important sandbox constraints:
actions and use GraphQL-backed `gh` reads when useful.
- Keep watching active PRs until they close, merge, or the operator stops the sandbox.
- At the start of every watch cycle, read `payload_version` from
`scripts/agents/gator/agent.yaml` on the default branch through the GitHub
`.agents/agents/gator/agent.yaml` on the default branch through the GitHub
contents API. If the published integer is greater than
`{{PAYLOAD_VERSION}}`, do not write to GitHub or run the reviewer. Finish with
`OPENSHELL_AGENT_RESULT {"status":"terminal_failure","reason":"stale_gator_payload","notes":"This sandbox is using an outdated Gator payload. Relaunch it before monitoring continues."}` so the operator relaunches the immutable watcher.
Expand Down
8 changes: 8 additions & 0 deletions .agents/agents/gator/run.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/usr/bin/env bash

set -euo pipefail

AGENT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT_DIR="$(cd "$AGENT_DIR/../../.." && pwd)"

exec bash "$ROOT_DIR/scripts/agents/run.sh" --agent "$AGENT_DIR" "$@"
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ This skill is a gating workflow. It can start from any issue or PR state, inspec

## Skill Location

Codex and other agent harnesses should load this skill from the repository path `scripts/agents/gator/skills/gator-gate/SKILL.md`. After this branch is merged, the canonical GitHub location is <https://github.com/NVIDIA/OpenShell/blob/main/scripts/agents/gator/skills/gator-gate/SKILL.md>.
Codex and other agent harnesses should load this skill from the repository path `.agents/agents/gator/skills/gator-gate/SKILL.md`. After this branch is merged, the canonical GitHub location is <https://github.com/NVIDIA/OpenShell/blob/main/.agents/agents/gator/skills/gator-gate/SKILL.md>.

## Prerequisites

Expand Down Expand Up @@ -796,7 +796,7 @@ Use the `principal-engineer-reviewer` sub-agent. Include:
`LEFT` for a deleted line), severity, finding ID, and concise comment body
- Instruction not to rely on local test execution

When running inside the `scripts/agents/gator` sandbox launcher, invoke the reviewer command specified in the sandbox prompt. Use `task.md` for the subagent input. Put the review feedback ledger, review mode, PR metadata, linked issue context, and mode-appropriate diff/file context in `task.md`. Require the reviewer to emit only the JSON envelope described in `references/review-findings-schema.md` to `review-findings.raw.json`, then run `validate-review-findings review-findings.raw.json > review-findings.json`. Only normalized entries with `blocking: true` may affect labels or public review comments. Missing evidence downgrades a proposed Critical or Warning to a non-blocking hypothesis; do not repair the reviewer output by guessing. The main gator process remains responsible for labels, comments, docs gates, and CI monitoring. Before posting, compare every proposed finding with all open, resolved, and waived ledger findings plus prior review summaries. Remove semantically equivalent findings unless the new diff reintroduces the defect or newly available evidence meets the Critical unchanged-code exception above. If the reviewer command exits nonzero or the saved reviewer output is absent, malformed, or fails envelope validation, stop the cycle with the `reviewer_subagent_failed` transient result described above without changing GitHub labels or posting a public disposition.
When running inside the `.agents/agents/gator` sandbox launcher, invoke the reviewer command specified in the sandbox prompt. Use `task.md` for the subagent input. Put the review feedback ledger, review mode, PR metadata, linked issue context, and mode-appropriate diff/file context in `task.md`. Require the reviewer to emit only the JSON envelope described in `references/review-findings-schema.md` to `review-findings.raw.json`, then run `validate-review-findings review-findings.raw.json > review-findings.json`. Only normalized entries with `blocking: true` may affect labels or public review comments. Missing evidence downgrades a proposed Critical or Warning to a non-blocking hypothesis; do not repair the reviewer output by guessing. The main gator process remains responsible for labels, comments, docs gates, and CI monitoring. Before posting, compare every proposed finding with all open, resolved, and waived ledger findings plus prior review summaries. Remove semantically equivalent findings unless the new diff reintroduces the defect or newly available evidence meets the Critical unchanged-code exception above. If the reviewer command exits nonzero or the saved reviewer output is absent, malformed, or fails envelope validation, stop the cycle with the `reviewer_subagent_failed` transient result described above without changing GitHub labels or posting a public disposition.

Post findings using these rules:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@

set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
GATOR_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
SCRIPT_DIR="$GATOR_DIR/bin"
WRAPPER="$SCRIPT_DIR/gh"

assert_status() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@

set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
GATOR_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
GATOR_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
SCRIPT_DIR="$GATOR_DIR/bin"
RESOLVER="$SCRIPT_DIR/resolve-gator-review-threads"

tmp="$(mktemp -d)"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,9 @@

set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
GATOR_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
GATOR_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
SCRIPT_DIR="$GATOR_DIR/bin"
ROOT_DIR="$(cd "$GATOR_DIR/../../.." && pwd)"
LEDGER="$SCRIPT_DIR/review-feedback-ledger"

tmp="$(mktemp -d)"
Expand Down Expand Up @@ -406,14 +407,14 @@ rg -q 'COPY bin/validate-review-findings /usr/local/bin/validate-review-findings
"$GATOR_DIR/Dockerfile"
ruby -ryaml -e '
manifest = YAML.load_file(ARGV.fetch(0))
abort unless manifest.fetch("payload_version") == 13
abort unless manifest.fetch("payload_version") == 14
resource = manifest.fetch("resources").find {
|entry| entry.fetch("id") == "gator-review-findings-schema"
}
abort unless resource.fetch("destination") ==
"skills/gator-gate/references/review-findings-schema.md"
' "$GATOR_DIR/agent.yaml"
rg -Fq 'manifest.fetch("resources", [])' "$GATOR_DIR/../run.sh"
rg -Fq 'manifest.fetch("resources", [])' "$ROOT_DIR/scripts/agents/run.sh"
rg -Fq 'Gator payload version: {{PAYLOAD_VERSION}}' \
"$GATOR_DIR/prompts/gator.md"
rg -q 'review-feedback-ledger NVIDIA OpenShell <pr-number>' \
Expand All @@ -428,7 +429,7 @@ rg -Fq 'private_security_review_required' "$GATOR_DIR/skills/gator-gate/SKILL.md
rg -Fq 'Do not initiate external disclosure' "$GATOR_DIR/skills/gator-gate/SKILL.md"
rg -Fq 'not a generic private-security blocker' "$GATOR_DIR/prompts/gator.md"
rg -Fq 'Security classification alone is not a private-triage gate' \
"$GATOR_DIR/../../../.claude/agents/principal-engineer-reviewer.md"
"$ROOT_DIR/.claude/agents/principal-engineer-reviewer.md"
rg -q '### Pragmatic review calibration' \
"$GATOR_DIR/skills/gator-gate/SKILL.md"
rg -q 'A new commit permits a delta review' \
Expand All @@ -440,9 +441,9 @@ rg -q 'available evidence demonstrates a Critical' \
rg -q 'Keep reviews pragmatic and convergent' \
"$GATOR_DIR/prompts/gator.md"
rg -q '### Pragmatic review calibration' \
"$GATOR_DIR/../../../.claude/agents/principal-engineer-reviewer.md"
"$ROOT_DIR/.claude/agents/principal-engineer-reviewer.md"
rg -q 'Do not mine unchanged code for new findings' \
"$GATOR_DIR/../../../.claude/agents/principal-engineer-reviewer.md"
"$ROOT_DIR/.claude/agents/principal-engineer-reviewer.md"
rg -q 'three finding-bearing rounds' \
"$GATOR_DIR/skills/gator-gate/SKILL.md"
rg -q 'alone is not a process blocker' \
Expand All @@ -452,7 +453,7 @@ rg -q '`test_dispatch_required`' \
rg -q 'Apply `test:windows` whenever a PR affects Windows support' \
"$GATOR_DIR/skills/gator-gate/SKILL.md"
rg -q 'require the `test:windows` label' \
"$GATOR_DIR/../../../.claude/agents/principal-engineer-reviewer.md"
"$ROOT_DIR/.claude/agents/principal-engineer-reviewer.md"
rg -q 'attacker_or_operator_prerequisite' \
"$GATOR_DIR/skills/gator-gate/references/review-findings-schema.md"
rg -Fq 'Write `Summary` as natural prose that can be read aloud' \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@

set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
GATOR_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
SCRIPT_DIR="$GATOR_DIR/bin"
VALIDATOR="$SCRIPT_DIR/validate-review-findings"
tmp="$(mktemp -d)"
trap 'rm -rf "$tmp"' EXIT
Expand Down
Loading
Loading