🤖 Usable PR Validator

Validate Pull Requests against your Usable knowledge base standards using AI (OpenCode with OpenRouter, Anthropic, OpenAI, or Google Gemini)

✨ Features

🧠 AI-Powered Validation: Uses OpenCode with OpenRouter (default), Anthropic, OpenAI, or Google Gemini to understand context and architectural patterns
🔀 Multi-Provider Support: Choose between OpenRouter (75+ models), Anthropic, OpenAI, or Google Gemini
📚 Usable Integration: Validate PRs against your team's knowledge base stored in Usable
🔌 MCP Protocol: Connects directly to Usable's MCP server for real-time standards
🎯 System Prompts: Organization-wide validation standards fetched from Usable and auto-merged
🚀 Dynamic Prompts: Fetch latest validation prompts from Usable API (no static files needed!)
💬 Comment-Triggered Revalidation: Mention @usable in PR comments to trigger revalidation with context
📝 Automatic Deviation Documentation: AI creates knowledge base fragments when deviations are approved
⚙️ Highly Configurable: Customizable prompts, severity levels, and validation rules
🔄 Reliable: Automatic retry logic with exponential backoff for API failures
💬 Smart PR Comments: Updates existing comments to avoid spam
📊 Controlled Reports: Metadata-only artifacts and comments by default; detailed prose requires an explicit publication-approved opt-in
🔒 Secure: Proper secret handling with automatic cleanup

🚀 Quick Start (5 Minutes)

Self-hosted runners need jq on PATH to generate the MCP JSON configuration; it is preinstalled on GitHub-hosted runners.

Prerequisites

For OpenCode (default, supports OpenRouter, Anthropic, OpenAI):

An API key for your chosen provider (e.g., OpenRouter, Anthropic, or OpenAI)
A Usable account with API token (get one at usable.dev)
GitHub repository with pull requests

For Google Gemini (alternative provider):

A Google Cloud project with Vertex AI API enabled
A service account key with Vertex AI permissions
A Usable account with API token (get one at usable.dev)
GitHub repository with pull requests

Choosing a Provider

Provider selection works in two levels:

provider chooses the CLI tool: opencode (default) or gemini
opencode-provider chooses which AI service OpenCode connects to (only applies when provider is opencode)

provider
├── opencode (default)
│   └── opencode-provider
│       ├── openrouter (default) ─ 75+ models via OpenRouter
│       ├── anthropic             ─ Claude models direct
│       └── openai                ─ GPT models direct
└── gemini
    └── Uses Google Gemini directly (configured via gemini-model)

Step 1: Create Validation Prompt

Create .github/prompts/pr-validation.md in your repository:

# PR Validation Instructions

## CRITICAL OUTPUT INSTRUCTION
**START YOUR OUTPUT DIRECTLY WITH:** `# PR Validation Report`

## PR Context
{{PR_CONTEXT}}

## Your Task
Analyze the changes and validate against standards.

### Get PR Changes
```bash
git diff origin/{{BASE_BRANCH}}...{{HEAD_BRANCH}}

[See templates/ directory for complete examples]

Step 2: Add GitHub Secrets

Go to your repository Settings → Secrets → Actions and add:

For OpenCode (default):

OPENCODE_API_KEY: Your AI provider API key (e.g., OpenRouter, Anthropic, or OpenAI)
USABLE_API_TOKEN: Your Usable API token (get from usable.dev → Settings → API Tokens)

If using a non-default provider, also set opencode-provider and opencode-model in your workflow:

Provider opencode-provider opencode-model example

OpenRouter (default) openrouter moonshotai/kimi-k2.5

Anthropic anthropic claude-sonnet-4-5

OpenAI openai gpt-4o

For Gemini (alternative):

GEMINI_SERVICE_ACCOUNT_KEY: Base64-encoded service account JSON key
```
cat service-account.json | base64
```
USABLE_API_TOKEN: Your Usable API token (get from usable.dev → Settings → API Tokens)

Step 3: Create Workflow

Create .github/workflows/pr-validation.yml:

name: PR Validation

on:
  pull_request:
    branches: [main, develop]

permissions:
  contents: read
  pull-requests: write

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          fetch-depth: 0

      - uses: flowcore-io/usable-pr-validator@v2.2.4
        with:
          prompt-file: '.github/prompts/pr-validation.md'
          workspace-id: 'your-workspace-uuid'
        env:
          OPENCODE_API_KEY: ${{ secrets.OPENCODE_API_KEY }}
          USABLE_API_TOKEN: ${{ secrets.USABLE_API_TOKEN }}

That's it! Your PRs will now be validated automatically. 🎉

📖 Configuration

Inputs

Input	Description	Required	Default
`prompt-file`	Path to validation prompt markdown file (optional if `use-dynamic-prompts` is enabled)
`use-dynamic-prompts`	Fetch latest prompt from Usable API instead of using static file		`false`
`prompt-fragment-id`	Usable fragment UUID to use as prompt (required when `use-dynamic-prompts` is `true`)	✓ (with dynamic prompts)
`workspace-id`	Usable workspace UUID (required - used to fetch MCP system prompt)	✓
`required-fragment-ids`	Comma- or whitespace-separated fragment UUIDs to retrieve and verify before validation. Empty keeps legacy discovery behavior but reports grounding as `not-required`, not `complete`.
`merge-custom-prompt`	Merge fetched Usable prompt with custom `prompt-file` (only when both are provided)		`true`
`provider`	AI provider to use (`opencode` or `gemini`)		`opencode`
`opencode-provider`	AI provider for OpenCode (e.g., `openrouter`, `anthropic`, `openai`)		`openrouter`
`opencode-model`	Model ID for OpenCode		`moonshotai/kimi-k2.5`
`opencode-api-key-secret`	Secret name for the configured opencode-provider API key		`OPENCODE_API_KEY`
`gemini-model`	Gemini model to use (when provider is `gemini`)		`gemini-2.5-flash`
`service-account-key-secret`	Secret name for Gemini service account key		`GEMINI_SERVICE_ACCOUNT_KEY`
`mcp-server-url`	Usable MCP server URL		`https://usable.dev/api/mcp`
`mcp-token-secret`	Secret name for Usable API token		`USABLE_API_TOKEN`
`fail-on-critical`	Fail build on critical violations		`true`
`comment-mode`	PR comment behavior (`update`/`new`/`none`)		`update`
`comment-title`	Title for PR comment (for multi-stage validation)		`Automated Standards Validation`
`artifact-retention-days`	Days to retain reports		`30`
`report-visibility`	`full` publishes the validated assistant report in comments/artifacts. `metadata-only` publishes only action-generated verdict/count/grounding; use it for confidential repositories or Usable context.		`full`
`max-retries`	Maximum whole-review retry attempts for retryable provider failures or incomplete final reports (0-10)		`2`
`timeout-minutes`	Hard shared time limit for provider attempts, backoff, and fallback (1-360 minutes)		`15`
`base-ref`	Base reference for diff comparison. Useful for release-please branches to compare against last release tag instead of base branch.		PR base branch
`head-ref`	Head reference for diff comparison		PR head branch
`allow-web-fetch`	Allow AI to use web_fetch tool for external resources (security consideration)		`false`

Note: You must set the USABLE_API_TOKEN secret and either OPENCODE_API_KEY (default provider) or GEMINI_SERVICE_ACCOUNT_KEY (gemini provider). Usable MCP integration is required for this action.

🧠 System Prompts (Automatic)

The action automatically includes system prompts to ensure high-quality validations. The final prompt is assembled in this order:

Action System Prompt (hardcoded in system-prompt.md)
- Critical guidelines (no hallucination, verify file contents)
- Output format requirements
- Severity definitions
- Usable MCP integration instructions
Workspace MCP System Prompt (fetched from Usable API)
- Your workspace-specific Usable MCP guidelines
- Fetched from /api/workspaces/{workspace-id}/mcp-system-prompt
- Defines how to search and use your knowledge base
User Prompt (your validation rules)
- Repository-specific validation criteria
- Can be a static file or dynamic from Usable

This three-layer approach ensures:

✅ Consistent validation behavior across all repositories
✅ Proper Usable MCP integration
✅ Accurate, hallucination-free reports
✅ Flexibility for repo-specific rules

📋 Smart Diff Summary (Efficient Validation)

The action provides the AI with a compact summary of changed files rather than dumping massive diffs into the prompt. This makes validation more reliable, scalable, and cost-effective.

What the AI Receives:

## 📋 Changed Files Summary

**Total files changed**: 15

### `src/app/api/users/route.ts`
- **Changes**: +45 lines, -12 lines
- **Modified ranges**: Line 10-25, Line 45-67, Line 89-102

### `src/lib/services/user.service.ts`
- **Changes**: +23 lines, -8 lines
- **Modified ranges**: Line 15-30, Line 55-61

How the AI Uses It:

Reviews the summary to understand scope and which files changed
Reads specific files on-demand using cat or git show HEAD:path/to/file.ts
Focuses on modified line ranges mentioned in the summary
Checks dependencies when needed (imports, configs, related files)
Validates intelligently without needing to process full diffs

Benefits:

✅ Scalable: Works with PRs of any size (even 100+ files)
✅ Reliable: AI doesn't need to run git diff (which can fail)
✅ Efficient: Reads only what it needs, not entire file contents upfront
✅ Cost-effective: Smaller prompt sizes = lower API costs
✅ Intelligent: Agentic approach lets the AI decide what to fetch

Example Validation Flow:

Summary shows: src/app/api/subscription/route.ts changed (lines 10-25)

AI's approach:
1. cat src/app/api/subscription/route.ts          # Read the changed file
2. Focus on lines 10-25                            # That's what changed
3. cat src/lib/services/subscription.service.ts   # Check the imported service
4. grep "subscription-updated" flowcore.yml       # Verify event type exists
5. Report any violations found

This agent-driven approach is more robust than trying to inject massive git diffs into the prompt, especially for large PRs where diffs can exceed token limits.

🚀 Dynamic Prompts

Instead of maintaining static prompt files, you can now fetch prompts dynamically from your Usable workspace. This ensures you're always using the latest validation standards without manual updates.

Benefits:

✅ Always use the latest validation best practices
✅ Reduced setup complexity - no prompt file needed
✅ Automatic updates when your team's standards evolve
✅ Centralized prompt management in Usable

How it works:

Enable use-dynamic-prompts: true
Provide the prompt-fragment-id with your Usable fragment UUID
The action fetches that specific prompt from your Usable workspace
Automatically merges with system prompts (action + MCP)

Examples:

# Dynamic user prompt from Usable
- uses: flowcore-io/usable-pr-validator@v2.2.4
  with:
    use-dynamic-prompts: true
    prompt-fragment-id: 'user-prompt-uuid'
    workspace-id: 'your-workspace-uuid'
  env:
    OPENCODE_API_KEY: ${{ secrets.OPENCODE_API_KEY }}
    USABLE_API_TOKEN: ${{ secrets.USABLE_API_TOKEN }}

# Static user prompt file (most common)
- uses: flowcore-io/usable-pr-validator@v2.2.4
  with:
    prompt-file: '.github/prompts/pr-validation.md'
    workspace-id: 'your-workspace-uuid'
  env:
    OPENCODE_API_KEY: ${{ secrets.OPENCODE_API_KEY }}
    USABLE_API_TOKEN: ${{ secrets.USABLE_API_TOKEN }}

Outputs

Output	Description
`validation-status`	Result: `passed` or `failed`
`validation-passed`	Boolean: `true` or `false`
`critical-issues`	Count of critical violations
`grounding-status`	Deterministic grounding result: `complete`, `incomplete`, or `not-required`
`report-artifact-name`	Name of report artifact

🎯 Usage Examples

Minimal Setup (OpenCode - default)

- uses: flowcore-io/usable-pr-validator@v2.2.4
  with:
    prompt-file: '.github/prompts/validate.md'
    workspace-id: 'your-workspace-uuid'
  env:
    OPENCODE_API_KEY: ${{ secrets.OPENCODE_API_KEY }}
    USABLE_API_TOKEN: ${{ secrets.USABLE_API_TOKEN }}

Using Gemini Instead

- uses: flowcore-io/usable-pr-validator@v2.2.4
  with:
    prompt-file: '.github/prompts/validate.md'
    workspace-id: 'your-workspace-uuid'
    provider: 'gemini'
  env:
    GEMINI_SERVICE_ACCOUNT_KEY: ${{ secrets.GEMINI_SERVICE_ACCOUNT_KEY }}
    USABLE_API_TOKEN: ${{ secrets.USABLE_API_TOKEN }}

With Custom MCP Server

- uses: flowcore-io/usable-pr-validator@v2.2.4
  with:
    prompt-file: '.github/prompts/validate.md'
    workspace-id: 'your-workspace-uuid'
    mcp-server-url: 'https://your-custom-mcp.com/api/mcp'
    mcp-token-secret: 'YOUR_CUSTOM_TOKEN'
  env:
    OPENCODE_API_KEY: ${{ secrets.OPENCODE_API_KEY }}
    YOUR_CUSTOM_TOKEN: ${{ secrets.YOUR_MCP_TOKEN }}

Advanced Configuration

- uses: flowcore-io/usable-pr-validator@v2.2.4
  with:
    prompt-file: '.github/validation/standards.md'
    workspace-id: 'your-workspace-uuid'
    provider: 'opencode'
    opencode-provider: 'anthropic'
    opencode-model: 'claude-sonnet-4-5'
    mcp-server-url: 'https://confluence.company.com/api/mcp'
    mcp-token-secret: 'CONFLUENCE_TOKEN'
    fail-on-critical: true
    comment-mode: 'update'
    artifact-retention-days: 90
    max-retries: 3
  env:
    OPENCODE_API_KEY: ${{ secrets.OPENCODE_API_KEY }}
    CONFLUENCE_TOKEN: ${{ secrets.CONF_API_TOKEN }}

Multiple Validation Stages

Use comment-title to create separate PR comments for each validation stage:

jobs:
  validate-backend:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          fetch-depth: 0
      
      - uses: flowcore-io/usable-pr-validator@v2.2.4
        with:
          prompt-file: '.github/prompts/backend-standards.md'
          workspace-id: 'your-workspace-uuid'
          comment-title: 'Backend Validation'  # Creates unique comment
        env:
          OPENCODE_API_KEY: ${{ secrets.OPENCODE_API_KEY }}
          USABLE_API_TOKEN: ${{ secrets.USABLE_API_TOKEN }}

  validate-frontend:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          fetch-depth: 0
      
      - uses: flowcore-io/usable-pr-validator@v2.2.4
        with:
          prompt-file: '.github/prompts/frontend-standards.md'
          workspace-id: 'your-workspace-uuid'
          comment-title: 'Frontend Validation'  # Creates unique comment
        env:
          OPENCODE_API_KEY: ${{ secrets.OPENCODE_API_KEY }}
          USABLE_API_TOKEN: ${{ secrets.USABLE_API_TOKEN }}

Note: Each comment-title creates a separate PR comment that updates independently. Artifacts are also uniquely named based on the title.

Release-Please Integration

Validate all changes since the last release for release-please PRs:

name: PR Validation

on:
  pull_request:
    branches: [main]

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          fetch-depth: 0
      
      # Determine base reference for release-please branches
      - name: Get base reference
        id: base-ref
        run: |
          if [[ "${{ github.head_ref }}" == release-please--* ]]; then
            # For release-please branches, compare against last release tag
            LAST_TAG=$(git describe --tags --abbrev=0 2>/dev/null || echo "${{ github.event.pull_request.base.ref }}")
            echo "ref=${LAST_TAG}" >> $GITHUB_OUTPUT
            echo "Comparing against last release: ${LAST_TAG}"
          else
            # For regular PRs, use base branch
            echo "ref=${{ github.event.pull_request.base.ref }}" >> $GITHUB_OUTPUT
            echo "Comparing against base branch: ${{ github.event.pull_request.base.ref }}"
          fi
      
      - uses: flowcore-io/usable-pr-validator@v2.2.4
        with:
          prompt-file: '.github/prompts/pr-validation.md'
          workspace-id: 'your-workspace-uuid'
          base-ref: ${{ steps.base-ref.outputs.ref }}  # Custom base reference
        env:
          OPENCODE_API_KEY: ${{ secrets.OPENCODE_API_KEY }}
          USABLE_API_TOKEN: ${{ secrets.USABLE_API_TOKEN }}

Tip: This ensures release-please PRs validate all accumulated changes since the previous release, not just the most recent commit.

📝 Prompt Engineering

Best Practices

Clear Output Format: Always specify exact markdown structure
Suppress Preamble: Explicitly instruct to start with report header
Context Limits: Remind AI not to output fetched content or logs
Severity Levels: Define clear criteria for Critical/Important/Suggestion
Examples: Include good/bad patterns in prompt

Placeholder Variables

Available in your prompt file:

{{PR_CONTEXT}} - Full PR context (number, title, description, URL)
{{BASE_BRANCH}} - Target branch (e.g., main)
{{HEAD_BRANCH}} - Source branch (e.g., feature/new-thing)
{{PR_TITLE}} - Just the PR title
{{PR_DESCRIPTION}} - Just the PR description
{{PR_NUMBER}} - Just the PR number
{{PR_URL}} - Direct link to PR
{{PR_AUTHOR}} - GitHub username
{{PR_LABELS}} - Comma-separated labels

Report Structure

Your prompt should instruct the AI to output this structure:

# PR Validation Report

## Summary
[Brief overview of findings]

## Critical Violations ❌
[Must-fix issues - build fails if present]

## Important Issues ⚠️
[Should-fix issues - build passes but flagged]

## Suggestions ℹ️
[Nice-to-have improvements]

## Validation Outcome
- **Status**: PASS ✅ | FAIL ❌
- **Critical Issues**: [count]
- **Important Issues**: [count]
- **Suggestions**: [count]

The action isolates only the provider's completed final assistant response before parsing it. For OpenCode 1.18.17, it selects the terminal step_start/step_finish span by messageID and joins only completed text parts from that span; earlier assistant text, tool output, errors, and incomplete terminal events cannot become the report. Gemini CLI 0.7.0 uses only its top-level JSON response string. A single conservative preamble (for example, Here is the requested PR validation report:) and a full outer Markdown fence are normalized away, but multiple report headers remain invalid. Provider transcripts stay runner-local; logs contain only metadata counts and error codes.

Publication boundary

The default public report is the full validated report (report-visibility: full), as before v2.2.9. v2.2.9 briefly changed the default to metadata-only. The rollback preserves the default for consumers on v2.2.8 or earlier. If you pinned v2.2.9 and rely on its metadata-only default, set report-visibility: metadata-only explicitly before upgrading; otherwise detailed assistant reports will resume in comments and artifacts. Parsing a valid final answer does not certify its prose as non-confidential, so confidential repositories should opt in to report-visibility: metadata-only. It constructs a new report from fixed action-owned text, the PASS/FAIL verdict, a bounded critical-issue count (0–1000), and the independently enforced grounding status. It never copies assistant summaries, findings, source excerpts, links, filenames, or retrieved standards into the report. Unknown visibility values, invalid UTF-8, oversized reports, invalid counts, and inconsistent grounding fail closed. The input report limit is 1 MiB.

Comments, artifacts, and action outputs use this publication boundary; the log never previews assistant prose. Comments also independently ignore supplied report text in metadata mode. comment-mode: none disables only comments, not artifacts. Each composite invocation has a unique report path and an explicit ready flag, so a setup failure cannot publish a previous invocation's report. Artifacts upload before comments; comments retain the action verdict and artifact link within a 60,000-byte UTF-8 limit.

report-visibility: full publishes validated assistant Markdown in comments and artifacts. It is not redaction or a confidentiality guarantee: a prompt-injected pull request can steer what the assistant writes. Use metadata-only for confidential customer repositories or confidential Usable context. In both modes, provider transcripts, prompts, config files, and assistant prose are not previewed in logs, and API publication failures never print exception/request bodies.

Action-owned private prompts/configuration and raw transcripts are owner-readable and cleaned up after the run. Use ephemeral runners and least-privilege Usable access: publication filtering does not prevent a provider from seeing authorized inputs, sandbox malicious code, remove provider-owned caches, certify model findings, or retroactively remove old comments/artifacts/logs. Verdicts and issue counts remain model-derived operational metadata, not deterministic proof of compliance. Required grounding and deterministic CI gates remain authoritative.

🔌 Usable Integration (Required)

What is Usable?

Usable is a team knowledge base and memory system that stores your:

Coding standards and conventions
Architecture patterns and decisions
Security requirements and best practices
Project-specific documentation

This action requires Usable and connects to your Usable workspace via MCP (Model Context Protocol) to validate PRs against your living documentation. The integration is always enabled and provides the AI with access to your team's knowledge base.

Setup

Get Your Usable API Token
- Go to usable.dev
- Navigate to Settings → API Tokens
- Create a new token with fragments.read permission

Add GitHub Secrets

# In your repo: Settings → Secrets → Actions
USABLE_API_TOKEN=your_usable_token_here
OPENCODE_API_KEY=your_api_key_here

Configure Workflow

- uses: flowcore-io/usable-pr-validator@v2.2.4
  with:
    prompt-file: '.github/prompts/pr-validation.md'
    workspace-id: 'your-workspace-uuid'
  env:
    OPENCODE_API_KEY: ${{ secrets.OPENCODE_API_KEY }}
    USABLE_API_TOKEN: ${{ secrets.USABLE_API_TOKEN }}

Note: Usable MCP integration is always enabled and uses https://usable.dev/api/mcp by default. You can customize the server URL with the mcp-server-url input if needed.

Update Prompt to Use Usable

### Fetch Standards from Usable

Use agentic-search-fragments to find relevant standards:
- Coding standards for {{BASE_BRANCH}}
- Architecture patterns
- Security requirements
- repo:your-repo-name tag

Use MCP search for discovery. Read full details with the action-owned UUID-only helper described in the injected Action-Owned Usable Grounding section.

💬 Comment-Triggered Revalidation

⚡ 2-Minute Setup - Add comment-triggered validation to any repository!

@usable Mentions

Trigger revalidations and approve deviations by commenting on a PR with @usable:

@usable This PR intentionally uses console.log in debug utilities.
These files are specifically for debugging and need console output.

What Happens:

✅ The action automatically triggers a revalidation
📝 Your comment is passed to the AI validator
🧠 The AI can:
- Understand the context you've provided
- Approve deviations from standards
- Document the decision by creating a fragment in Usable
- Link the fragment in the validation report
📊 A new validation report is posted with the override applied

Setting Up Comment Revalidation

Create .github/workflows/comment-revalidation.yml:

name: Comment Revalidation

on:
  issue_comment:
    types: [created]

jobs:
  revalidate:
    # Only run for trusted commenters on a PR who mention @usable
    if: |
      github.event.issue.pull_request &&
      contains(github.event.comment.body, '@usable') &&
      contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association)
    
    # Use the reusable workflow - it handles everything!
    uses: flowcore-io/usable-pr-validator/.github/workflows/comment-revalidation.yml@v2.2.4
    with:
      workspace-id: 'your-workspace-uuid'  # REQUIRED
      prompt-file: '.github/prompts/pr-validation.md'  # Optional
      # Customize as needed:
      # use-dynamic-prompts: true
      # prompt-fragment-id: 'fragment-uuid'
      # provider: 'opencode'
      # opencode-model: 'moonshotai/kimi-k2.5'
      # comment-title: '🔄 Custom Title'
      # fail-on-critical: false
    secrets:
      OPENCODE_API_KEY: ${{ secrets.OPENCODE_API_KEY }}
      USABLE_API_TOKEN: ${{ secrets.USABLE_API_TOKEN }}
    permissions:
      contents: read
      pull-requests: write

That's it! Just 15 lines and you're done. The reusable workflow handles:

✅ Extracting PR details
✅ Checking out the correct commit
✅ Determining base ref (branch vs tag)
✅ Running validation with override context
✅ Posting results as a comment
✅ Adding reaction emoji to acknowledge

Security: Comment revalidation runs with your secrets, so only comments from OWNER, MEMBER, or COLLABORATOR authors start it, and pull requests from forks are refused. The validator runs from a pinned release, never from the pull request checkout. Keep the author_association condition in your caller workflow too: reusable workflow versions before this fix do not enforce it. Organization members whose membership is private can show up as CONTRIBUTOR. Make the membership public, or add them as collaborators.

Access behavior: COLLABORATOR includes read-only and triage collaborators; this gate does not require write permission. A trusted comment on a fork PR fails explicitly before checkout rather than silently skipping.

Release pin maintenance: After publishing a stable release, update the full SHA and version comment in .github/workflows/comment-revalidation.yml to that release commit. Revalidation Release Pin checks both against GitHub's latest published stable release on publication, daily, and when its policy files change. v2.2.9 predates the comment publication and PR-context fixes in this PR; comment revalidation needs a release containing them before it works end to end. Update caller/template reusable-workflow refs to that release too. Shared latest promotion remains a separate protected operation.

Tip: Copy templates/comment-revalidation-workflow.yml for a ready-to-use template

How It Works

Example: Approving a Deviation

PR has a violation: Validator flags console.log usage

Developer comments:

@usable This is intentional. The debug utility needs console output 
for troubleshooting production issues. Only used in /debug/ directory.

Validator understands and documents:
- Creates a fragment in Usable titled "Approved Deviation: console.log in debug utilities"
- Includes justification, conditions, PR link, and approver
- Tags it as deviation, approved, repo:your-repo

Report shows:

## Override Applied

A deviation from standards has been approved and documented:

- **Deviation**: console.log usage in debug utilities
- **Justification**: Required for production troubleshooting
- **Documentation**: Fragment created - "Approved Deviation: console.log in debug utilities" (ID: abc-123)
- **Approved by**: @developer

This deviation has been recorded in the knowledge base for future reference.

Use Cases:

✅ Approve Deviations: "This violation is acceptable because..."
🎯 Focus Validation: "@usable Focus on security issues only"
💡 Provide Context: "@usable This API change was approved by the architecture team"
🔄 Re-run After Fixes: "@usable Please revalidate now that I've fixed the issues"

Benefits:

📚 Knowledge Base Grows: Approved deviations are automatically documented
🔗 Traceability: Every deviation links back to the approving PR and person
🤝 Team Communication: Decisions are visible to everyone
🚀 No Manual Work: AI handles documentation automatically

🔒 Security

Secret Handling

Service account keys are base64-decoded to /tmp with 600 permissions
Temporary files automatically cleaned up in always() block
Never logged or exposed in outputs
Use GitHub encrypted secrets for storage

Permissions Required

permissions:
  contents: read        # Read repository code
  pull-requests: write  # Post comments

Security Best Practices

Rotate API keys regularly (OpenRouter or Google Cloud)
Use least-privilege service accounts (Gemini provider)
Review validation prompts for sensitive data
Keep allow-web-fetch disabled (default) unless you specifically need it
Use GitHub encrypted secrets for all API keys

Web Fetch Security

The allow-web-fetch input controls whether the AI can download external resources during validation.

Default: false (DISABLED) - Recommended for most use cases

✅ When to keep it disabled (default):
- Standard PR validation using only your codebase and Usable knowledge base
- Security-sensitive environments
- When you want to ensure validation is fully reproducible
- When you don't need external documentation or references
⚠️ When you might enable it:
- Validating against external API documentation
- Checking compliance with published standards (e.g., OWASP, RFC specs)
- Verifying links in documentation PRs
- Fetching external schema definitions

Security implications when enabled:

AI can make HTTP requests to arbitrary URLs
Could potentially expose internal URLs if mentioned in PR
May introduce non-deterministic validation results
Consider network egress policies and firewall rules

Example of enabling:

- uses: flowcore-io/usable-pr-validator@v2.2.4
  with:
    prompt-file: '.github/prompts/validate-api-docs.md'
    allow-web-fetch: true  # Only enable when needed
  env:
    OPENCODE_API_KEY: ${{ secrets.OPENCODE_API_KEY }}
    USABLE_API_TOKEN: ${{ secrets.USABLE_API_TOKEN }}

🐛 Troubleshooting

OpenCode installation is reproducible

The action installs a pinned OpenCode CLI release through npm and verifies the reported version before validation starts. This avoids depending on an unverified remote installer archive and prevents transient CDN responses from being passed to tar. Consumers do not need to install OpenCode themselves.

Provider CLI exits without a report

Symptom: The provider exits non-zero before producing a publishable validation report.

The action keeps provider stdout and stderr private because they may contain prompts, fragment content, tool output, tokens, or raw API responses. It does not stream or upload those files. Instead, the job log emits one content-free Provider failure metadata JSON object with allowlisted fields only:

structured error name (or unrecognized)
numeric HTTP status code, when present
allowlisted machine error code, when present
provider-declared retryability
generic missing-field names and presence/count flags

Messages, response bodies, response headers, URLs, tool output, and transcript text are never included. Use the safe metadata to distinguish authentication, rate-limit/server, context-window, model/configuration, and unknown failures. If all identifying fields are missing, reproduce with the exact pinned action and provider version rather than enabling raw transcript logging.

Also verify the Git Diff Setup section completed before provider execution:

🔍 Verifying Git Diff Setup
✅ Base ref available: origin/main
✅ Head ref available: origin/feature-branch
✅ Three-dot diff works: origin/main...origin/feature-branch

The execution header confirms the selected model and prompt size without printing prompt content:

🤖 Running OpenCode CLI
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Model: openrouter/moonshotai/kimi-k2.5
Prompt size: XXXX bytes
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

For git-only diagnosis, run:

./scripts/test-git-diff.sh main feature-branch

Git Diff Not Working

Symptom: Error messages like "revisions or paths not found" or AI reports inability to analyze changes

Root Cause: Git refs not properly fetched in GitHub Actions environment

Automated Fix (already implemented in v1.3.1+):

Enhanced git setup with retry logic for both base and head refs
Multiple ref format attempts (branch, tag, explicit refspec)
Pre-flight validation that tests git diff before AI starts

Manual Verification:

# Run diagnostic script
./scripts/test-git-diff.sh <base-branch> <head-branch>

# Expected output:
# ✅ Base ref available: origin/main
# ✅ Head ref available: origin/feature
# ✅ Three-dot diff works
# 📊 5 files changed

Common Mistake - Detached HEAD:

If you're checking out with a specific SHA, this creates a detached HEAD state that breaks diff operations:

# ❌ WRONG - Creates detached HEAD
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
  with:
    ref: ${{ github.event.pull_request.head.sha }}
    fetch-depth: 0

# ✅ CORRECT - Let checkout handle the ref automatically
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
  with:
    fetch-depth: 0  # This is all you need for PRs

If you MUST use a specific ref:

# Option 1: Don't specify ref for PR workflows
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
  with:
    fetch-depth: 0

# Option 2: Explicitly fetch both refs after checkout
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
  with:
    ref: ${{ github.event.pull_request.head.sha }}
    fetch-depth: 0

- name: Fetch branch refs
  run: |
    git fetch origin ${{ github.event.pull_request.base.ref }}:refs/remotes/origin/${{ github.event.pull_request.base.ref }}
    git fetch origin ${{ github.event.pull_request.head.ref }}:refs/remotes/origin/${{ github.event.pull_request.head.ref }}

MCP Tools Not Available

Symptom: Error messages like Tool "search_memory_fragments" not found in registry or AI sees wrong tools (search_file_content, read_many_files instead of Usable Memory tools)

Root Cause: MCP server not receiving workspace-id context, so it returns generic tools instead of Usable-specific memory tools

Solution (Fixed in v1.6.0+):

The action now automatically passes workspace-id to the MCP server via the x-workspace-id header. Ensure you're using the latest version:

- uses: flowcore-io/usable-pr-validator@v2.2.4
  with:
    workspace-id: '60c10ca2-4115-4c1a-b6d7-04ac39fd3938'  # Required!

If using older versions, update to v1.6.0+ or manually verify:

workspace-id input is provided in your workflow
Action passes WORKSPACE_ID environment variable to MCP setup
MCP settings include x-workspace-id header in configuration

Expected MCP tools when working correctly:

agentic-search-fragments
search-memory-fragments
get-memory-fragment-content is deliberately excluded from both CLI configurations; the action-owned UUID-only HTTPS helper performs full reads and records them in a per-run ledger
explore-workspace-graph
create-memory-fragment
update-memory-fragment

Validation Fails Immediately

Symptom: Action fails before running Gemini

Causes:

Prompt file not found
Missing required secrets
Invalid MCP configuration

Solution:

# Check prompt file exists
ls -la .github/prompts/

# Verify secrets are set
# Go to repo Settings → Secrets → Actions

Report Not Extracted

Symptom: Warning about report extraction

Cause: AI didn't follow output format instructions

Solution: Strengthen prompt instructions:

## CRITICAL OUTPUT INSTRUCTION
**YOU MUST START WITH:** `# PR Validation Report`
DO NOT include thinking process or explanations!

API Rate Limit Errors

Symptom: 429 errors from AI API

Solution:

Increase max-retries (automatic exponential backoff is built in)
For OpenRouter: check your rate limits at openrouter.ai
For Gemini: use Vertex AI (higher limits) and check Google Cloud quotas

MCP Connection Failures

Symptom: Can't connect to MCP server

Solutions:

# 1. Verify URL is correct
mcp-server-url: 'https://correct-url.com/api/mcp'

# 2. Check token is valid
# Ensure MCP_API_TOKEN secret is set

# 3. Test connectivity
curl -H "Authorization: Bearer $TOKEN" $MCP_URL

📊 Cost Estimation

OpenRouter Pricing (default)

Model	Input (per 1M tokens)	Output (per 1M tokens)	Typical PR Cost
moonshotai/kimi-k2.5	$0.45	$2.25	$0.02-0.10
anthropic/claude-sonnet-4-5	$3.00	$15.00	$0.15-0.75
openai/gpt-4o	$2.50	$10.00	$0.10-0.50

Estimate: ~$0.02-0.10 per PR with moonshotai/kimi-k2.5 (default)

Google Gemini Pricing (Vertex AI)

Model	Input (per 1M tokens)	Output (per 1M tokens)	Typical PR Cost
gemini-2.5-flash	$0.075	$0.30	$0.01-0.03
gemini-2.0-flash	$0.10	$0.40	$0.02-0.05
gemini-2.5-pro	$1.25	$5.00	$0.25-1.00

Estimate: ~$0.01-0.05 per PR with gemini-2.5-flash

MCP Costs

MCP server costs vary by provider:

Usable: Check pricing at usable.dev
Self-hosted: Server infrastructure costs
Confluence: Included in license

🔖 Versioning

This action follows Semantic Versioning and uses automated releases via release-please.

Using Specific Versions

# Moving compatibility alias (use only when centrally managed)
- uses: flowcore-io/usable-pr-validator@latest

# Specific provider-protected version (recommended)
- uses: flowcore-io/usable-pr-validator@v2.2.4

# Exact reviewed commit (strongest supply-chain binding; substitute the reviewed SHA)
- uses: flowcore-io/usable-pr-validator@<resolved-v2.2.4-commit-sha>

# Latest commit on main (not recommended for production)
- uses: flowcore-io/usable-pr-validator@main

Version Strategy

Major (v1.x.x → v2.x.x): Breaking changes requiring user action
Minor (v1.0.x → v1.1.x): New features, backward compatible
Patch (v1.0.0 → v1.0.1): Bug fixes, backward compatible

Prefer an exact reviewed commit SHA, or an exact version when centralized update automation is required. Publishing a version does not move latest automatically. After representative consumer canaries pass, a maintainer must dispatch Promote latest validator alias with the exact stable version and approve its protected validator-rollout environment. This keeps moving-alias consumers on their last approved validator until the company-wide rollout is deliberate.

🤝 Contributing

Contributions welcome! See CONTRIBUTING.md for guidelines.

All commits must follow Conventional Commits for automated releases:

feat: for new features
fix: for bug fixes
docs: for documentation
feat!: or BREAKING CHANGE: for breaking changes

Development Setup

git clone https://github.com/flowcore-io/usable-pr-validator.git
cd usable-pr-validator

# Test locally (requires act)
act pull_request -s OPENCODE_API_KEY="your-key-here" -s USABLE_API_TOKEN="your-token"

📜 License

MIT License - see LICENSE for details.

🙏 Acknowledgments

OpenCode CLI for multi-provider AI capabilities
OpenRouter for unified model access
Google Gemini for AI capabilities
Model Context Protocol (MCP) community
GitHub Actions ecosystem

📞 Support

Made with ❤️ by Flowcore

Name		Name	Last commit message	Last commit date
.github		.github
scripts		scripts
templates		templates
tests		tests
.gitignore		.gitignore
.release-please-manifest.json		.release-please-manifest.json
AGENTS.md		AGENTS.md
CHANGELOG.md		CHANGELOG.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
PROJECT_SUMMARY.md		PROJECT_SUMMARY.md
README.md		README.md
action.yml		action.yml
release-please-config.json		release-please-config.json
system-prompt.md		system-prompt.md

Provider	`opencode-provider`	`opencode-model` example
OpenRouter (default)	`openrouter`	`moonshotai/kimi-k2.5`
Anthropic	`anthropic`	`claude-sonnet-4-5`
OpenAI	`openai`	`gpt-4o`

Latest commit

History

Folders and files

Repository files navigation