Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/labeler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,12 @@ contract:tracker:
- 'tools/jira/**'
- 'tools/sourcehut/**'

contract:typed-decision:
- any:
- changed-files:
- any-glob-to-any-file:
- 'tools/typed-decision/**'

substrate:action-guard:
- any:
- changed-files:
Expand Down
1 change: 1 addition & 0 deletions docs/adapters/registry.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ extension point = a documented, labelled slot with a tracking issue.
| [`tools/chat`](../../tools/chat/) | [`chat-slack`](../../tools/chat-slack/) | Discord [#1421](https://github.com/apache/magpie/issues/1421) |
| Agent harness | Claude Code, [Codex](codex.md) `experimental` ([#313](https://github.com/apache/magpie/issues/313)), [Gemini CLI](gemini.md) `experimental` ([#314](https://github.com/apache/magpie/issues/314)), [Local LLM (Ollama / llama.cpp / vLLM)](local-llm.md) ([#315](https://github.com/apache/magpie/issues/315)), [Cursor](cursor.md) ([#316](https://github.com/apache/magpie/issues/316)), [Goose](goose.md) `guide only` ([#319](https://github.com/apache/magpie/issues/319)), [Aider](aider.md) `guide only` ([#317](https://github.com/apache/magpie/issues/317)), [GitHub Copilot](copilot.md) `guide only` ([#318](https://github.com/apache/magpie/issues/318)), Grok `reviewer backend only` ([#1416](https://github.com/apache/magpie/issues/1416)) | Amazon Q [#320](https://github.com/apache/magpie/issues/320)–OpenHands [#322](https://github.com/apache/magpie/issues/322) |
| Security cross-ref | [`tools/osv`](../../tools/osv/) | — |
| [`tools/typed-decision`](../../tools/typed-decision/) | TypeSafe Jev API | Local models / llama.cpp / Ollama ([#1431](https://github.com/apache/magpie/issues/1431)) |

## In-tree organizations

Expand Down
2 changes: 2 additions & 0 deletions docs/labels-and-capabilities.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,7 @@ framework substrate:
| `contract:scan-format` | contract | Security-scanner report parsing. |
| `contract:project-metadata` | contract | Governance rosters / people / releases. |
| `contract:security-cross-ref` | contract | Vulnerability database / cross-reference alias lookup (OSV.dev / NVD). |
| `contract:typed-decision` | contract | Provider-agnostic typed decision backend (Choice / Score / Noul). |
| `substrate:analytics` | substrate | Read-only metrics / dashboards / renderers. |
| `substrate:sandbox` | substrate | Agent isolation, egress control, settings audit. |
| `substrate:action-guard` | substrate | Deterministic pre-tool-use command guards. |
Expand Down Expand Up @@ -367,6 +368,7 @@ or a contract-free mix of substrates (e.g. `tools/spec-inventory` is
| [`tools/skill-reconciler-diff`](../tools/skill-reconciler-diff/) | `substrate:framework-dev` | Deterministic structural diff between two skill trees — parses frontmatter, section headings, step inventory, placeholders, support files, and safety-baseline clauses into a JSON diff object for the `skill-reconciler` skill |
| [`tools/vcs`](../tools/vcs/) | `contract:source-control` | Backend-dispatching implementation of the source-control (VCS) capability ([`tools/github/source-control.md`](../tools/github/source-control.md)); complete Git, Mercurial (Hg), and Fossil backends, plus detected extension point for SVN (#602) |
| [`tools/sourcehut`](../tools/sourcehut/) | `contract:tracker` + `contract:source-control` + `contract:mail-archive` | SourceHut (sr.ht) forge bridge: todo.sr.ht, lists.sr.ht, builds.sr.ht, and git/hg repository reads |
| [`tools/typed-decision`](../tools/typed-decision/) | `contract:typed-decision` | Provider-agnostic typed decision tool contract (Choice / Score / Noul) with TypeSafe Jev API as initial backend |

A tool's capability is the **interface it provides**, not which skills
happen to consume it (RFC-AI-0005). `tools/github` provides the
Expand Down
10 changes: 9 additions & 1 deletion docs/vendor-neutrality.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,7 @@ with pluggable backends already include:
| [`tools/forwarder-relay`](../tools/forwarder-relay/) | ASF Security relay, huntr.com, HackerOne triagers |
| [`tools/scan-format`](../tools/scan-format/) | security-scanner report formats (ASVS reference) |
| [`tools/vcs`](../tools/vcs/) | Git (complete), Mercurial (complete), Subversion, … (extension points) |
| [`tools/typed-decision`](../tools/typed-decision/) | TypeSafe Jev API (cloud), local models via llama.cpp / Ollama (extension point) |

The security-team surface follows the same pattern: CNA backends live
behind [`tools/cve-tool`](../tools/cve-tool/) (the ASF Vulnogram adapter
Expand All @@ -173,6 +174,11 @@ reference), inbound report relays behind
behind [`tools/scan-format`](../tools/scan-format/), and an OSV.dev
vulnerability cross-reference client [`tools/osv`](../tools/osv/)
implements the `contract:security-cross-ref` capability.
Similarly, structured low-latency decisions sit behind the
`contract:typed-decision` contract ([`tools/typed-decision`](../tools/typed-decision/)),
with TypeSafe's Jev API as the initial reference backend and local
inference (via llama.cpp, Ollama, or vLLM) tracked as the second backend to
ensure cross-vendor choice.

The distinction Magpie enforces: **vendor-specific *integrations* are
expected and welcome; vendor-specific *workflows* are forbidden.** A
Expand Down Expand Up @@ -209,6 +215,7 @@ contract for one vendor:
| [`tools/forwarder-relay`](../tools/forwarder-relay/) | ASF-security ([`tools/gmail/asf-relay.md`](../tools/gmail/asf-relay.md)) | huntr.com, HackerOne |
| [`tools/scan-format`](../tools/scan-format/) | ASVS | other scanner formats |
| [`tools/vcs`](../tools/vcs/) | Git | Mercurial, Subversion, … |
| [`tools/typed-decision`](../tools/typed-decision/) | TypeSafe Jev API | Local models (llama.cpp, Ollama, vLLM) ([#1431](https://github.com/apache/magpie/issues/1431)) |

A project selects an adapter per capability in its config
(`cve_authority.tool: vulnogram`, `archive_system.kind: ponymail`,
Expand Down Expand Up @@ -566,7 +573,7 @@ generated block below.

<!-- BEGIN vendor-neutrality-score — generated by `uv run --project tools/vendor-neutrality-score vendor-neutrality-score --markdown`; do not edit by hand -->

**Overall vendor-neutrality score: 10/12 capability contracts (83%).** Generated by [`tools/vendor-neutrality-score`](../tools/vendor-neutrality-score/); re-run it to refresh this section.
**Overall vendor-neutrality score: 10/13 capability contracts (77%).** Generated by [`tools/vendor-neutrality-score`](../tools/vendor-neutrality-score/); re-run it to refresh this section.

| Capability contract | Neutral? | Class | Backends today | Basis |
|---|---|---|---|---|
Expand All @@ -582,6 +589,7 @@ generated block below.
| `contract:scan-format` | ✅ | agnostic | — | vendor-neutral by construction — one spec serves every backend |
| `contract:project-metadata` | ✅ | single-org | ASF | single-organisation capability (ASF); no vendor choice to make |
| `contract:security-cross-ref` | ❌ | vendor-backed | OSV.dev | only 1 backend vendor (OSV.dev); needs 1 more |
| `contract:typed-decision` | ❌ | vendor-backed | TypeSafe | only 1 backend vendor (TypeSafe); needs 1 more |

**Per-skill assessment: 78/78 skills carry no vendor lock-in.** A skill is *capability-pure* when it names no backend at all, *portable* when every backend it names has an alternative (its contract is green), and *vendor-coupled* only when it reaches for a backend that is the sole implementation of a capability.

Expand Down
3 changes: 3 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,8 @@ package = false
# here once so every member can name it in its `dev` group without repeating
# the source; workspace members inherit the root's sources.
magpie-dev = { workspace = true }
checker = { workspace = true }


[tool.uv.workspace]
members = [
Expand Down Expand Up @@ -154,4 +156,5 @@ members = [
"tools/fossil",
"tools/sourcehut",
"tools/gitlab",
"tools/typed-decision",
]
11 changes: 11 additions & 0 deletions tools/privacy-llm/checker/src/checker/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,15 @@
Skills shell out to this command at Step 0 (pre-flight) when they
may read ``<private-list>`` content. The contract is documented
in ``tools/privacy-llm/wiring.md``.

:func:`check_endpoint` is the runtime counterpart for a single
outbound endpoint, used by tools that make their own LLM calls.
"""

from checker.check import Verdict, check_endpoint, check_stack

__all__ = [
"Verdict",
"check_endpoint",
"check_stack",
]
213 changes: 213 additions & 0 deletions tools/privacy-llm/checker/src/checker/check.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,12 +35,15 @@

import argparse
import dataclasses
import re
import sys
import urllib.parse

from checker.config import (
LLMEntry,
OptInEntry,
ParsedConfig,
_first_url,
host_of,
locate_config_path,
parse_config,
Expand Down Expand Up @@ -185,6 +188,216 @@ def check_stack(config: ParsedConfig) -> list[Verdict]:
return out


_HOSTNAME_LABEL_RE = re.compile(r"^[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?$")


def _is_valid_hostname(token: str) -> bool:
"""Validate that candidate token is a genuine hostname.

Rejects tokens containing userinfo (@), fragments (#), path slashes (/),
version numbers (all-numeric dot-separated segments like 3.5, 4.0, 1.0.0),
TLDs without ASCII alphabetical characters, or invalid label syntax.
"""
if not token or len(token) > 253:
return False
if any(c in token for c in ("@", "#", "/", ":", "?")):
return False
labels = token.split(".")
if len(labels) < 2:
return False
# Reject tokens where all dot-separated segments are purely numeric
if all(lbl.isdigit() for lbl in labels):
return False
# Require at least one ASCII alphabetical character in top-level domain / rightmost label
if not any(c.isascii() and c.isalpha() for c in labels[-1]):
return False
# Require valid hostname label syntax for each label
for lbl in labels:
if not lbl or len(lbl) > 63:
return False
if not _HOSTNAME_LABEL_RE.match(lbl):
return False
return True


def _extract_opt_in_host(name: str) -> str | None:
"""Extract destination hostname from an opt-in entry name.

Recognises full URLs (https://api.example.com/v1) and bare hostnames
at the start of the entry (api.example.com (Provider)). Returns None
for name-only opt-ins (e.g. 'TypeSafe — Jev API', 'AWS Bedrock',
'3.5 Sonnet (AWS Bedrock)').
"""
url = _first_url(name)
if url is not None:
return host_of(url)

# Hyphens are not separators here: they are legal inside hostname labels
# (``bedrock-runtime.eu-central-1.amazonaws.com``).
first_token = re.split(r"[\s—,(:]+", name.strip())[0]
if _is_valid_hostname(first_token):
return host_of("//" + first_token)
return None


_URL_IN_TEXT_RE = re.compile(r"[a-zA-Z][a-zA-Z0-9+.-]*://\S+")


def _name_matches_description(opt_name: str, raw_desc: str) -> bool:
"""Match a name-only opt-in against the provider description.

URLs are stripped from the description first, and the opt-in name (or its
short name) must appear as whole words. Otherwise a short opt-in name such
as ``AI (internal)`` would match inside ``https://api.typesafe.ai/...`` and
approve a provider nobody opted in to.
"""
desc_lc = _URL_IN_TEXT_RE.sub(" ", raw_desc).strip().lower()
if not desc_lc:
return False
name_lc = opt_name.lower().strip()
for candidate in (name_lc, _shortname(name_lc)):
if candidate and re.search(rf"(?<!\w){re.escape(candidate)}(?!\w)", desc_lc):
return True
return False


def _approve_endpoint_by_opt_in(
entry: LLMEntry,
ep_host: str,
opt_in: list[OptInEntry],
*,
default_endpoint: str | None = None,
raw_desc: str | None = None,
) -> Verdict:
"""Match endpoint against opt-in entries.

For URL opt-in entries: requires exact host_of(endpoint) == host_of(opt_url).
For name-only opt-in entries: matches only if endpoint matches default_endpoint
and the provider name matches the opt-in entry name.
"""
for opt in opt_in:
opt_host = _extract_opt_in_host(opt.name)
matched = False

if opt_host is not None:
if ep_host == opt_host:
matched = True
else:
# Name-only opt-in: matches only if endpoint is the default endpoint
if default_endpoint is not None:
def_host = host_of(default_endpoint)
if (entry.url == default_endpoint) or (def_host is not None and ep_host == def_host):
matched = _name_matches_description(opt.name, raw_desc or "")

if not matched:
continue

if not opt.data_residency:
return Verdict(
entry,
False,
f"opt-in entry {opt.name!r} matches but is missing the 'Data-residency contract' sub-bullet",
)
if not opt.approved_by or _is_placeholder(opt.approved_by):
return Verdict(
entry,
False,
f"opt-in entry {opt.name!r} matches but the 'Approved-by' "
f"sub-bullet is missing or still has placeholder text "
f"({opt.approved_by!r}); a real PMC member must sign off.",
)
return Verdict(entry, True, f"opt-in entry {opt.name!r} (data-residency + approved-by present)")

return Verdict(
entry,
False,
f"third-party endpoint {entry.url} denied: no opt-in entry was declared "
f"with matching host in <project-config>/privacy-llm.md",
)


def check_endpoint(
endpoint: str,
config: ParsedConfig | None = None,
*,
default_endpoint: str | None = None,
raw_desc: str | None = None,
) -> Verdict:
"""Verify whether a single outbound LLM endpoint URL is approved.

Checks default-approval rules (e.g. localhost, *.apache.org), falling back
to host-bound opt-in entries in ``<project-config>/privacy-llm.md``.
URLs containing userinfo or fragments are strictly rejected. The free-text
Claude Code rule does not apply to endpoint URL checks.
"""
desc = raw_desc if raw_desc is not None else endpoint
entry = LLMEntry(raw=desc, url=endpoint)
Comment thread
potiuk marked this conversation as resolved.

# 1. URL validation: reject userinfo, fragments, or malformed URLs
try:
parsed = urllib.parse.urlsplit(endpoint)
except ValueError as err:
return Verdict(entry, False, f"unparsable URL {endpoint!r}: {err}")

if not parsed.scheme or not parsed.netloc:
return Verdict(entry, False, f"invalid URL (missing scheme or host): {endpoint!r}")

if parsed.scheme.lower() not in ("http", "https"):
return Verdict(entry, False, f"unsupported URL scheme {parsed.scheme!r} in {endpoint!r}")

if parsed.fragment or "#" in endpoint:
return Verdict(entry, False, f"endpoint URL must not contain a fragment: {endpoint!r}")

if parsed.username is not None or parsed.password is not None or "@" in parsed.netloc:
return Verdict(entry, False, f"endpoint URL must not contain userinfo: {endpoint!r}")

ep_host = parsed.hostname
if ep_host is None:
return Verdict(entry, False, f"unparsable URL host in {endpoint!r}")
ep_host = ep_host.lower()

# 2. Default rules for URL-only check (no Claude Code free-text rule!)
if ep_host in _LOCAL_HOSTS:
return Verdict(entry, True, f"local-only inference at {ep_host} (default-approved)")
if ep_host in _APACHE_ORG_CARVE_OUTS:
return Verdict(
entry,
False,
f"{ep_host} is carved out of the *.apache.org default approval "
f"({_APACHE_ORG_CARVE_OUTS[ep_host]}); declare it as an opt-in "
f"entry if the PMC accepts the residency terms",
)
if ep_host.endswith(".apache.org") or ep_host == "apache.org":
return Verdict(entry, True, f"*.apache.org-hosted endpoint at {ep_host} (default-approved)")

# 3. Locate and parse config if not passed
if config is None:
try:
path = locate_config_path()
except FileNotFoundError as err:
return Verdict(
entry,
False,
f"Third-party endpoint {endpoint} denied (no privacy-llm config found): {err}",
)
try:
config = parse_config(path)
except Exception as err:
return Verdict(
entry,
False,
f"Failed to parse privacy-llm config at {path}: {err}",
)

return _approve_endpoint_by_opt_in(
entry,
ep_host,
config.opt_in,
default_endpoint=default_endpoint,
raw_desc=raw_desc,
)


def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(
prog="privacy-llm-check",
Expand Down
1 change: 1 addition & 0 deletions tools/privacy-llm/checker/src/checker/py.typed
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# Marker file for PEP 561.
Loading
Loading