Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ $ sentinel deprecations --endpoint http://localhost:9000/mcp

## Limitations, stated plainly

**Five MUST requirements cannot be verified black-box.** `sentinel` reports them as
**Four MUST requirements, and one beyond-spec rule, cannot be verified black-box.** `sentinel` reports them as
`INDETERMINATE` — never as passes — excludes them from the gate, and prints them on every scan:

| Rule | Why a scan cannot settle it | What would |
Expand All @@ -82,12 +82,12 @@ $ sentinel deprecations --endpoint http://localhost:9000/mcp
| `no-token-passthrough` | What a server sends to its own dependencies is invisible from the client side. | Capture the server's egress during an authenticated call. |
| `handle-possession-is-not-authentication` | Needs **two** authenticated principals; a scan has one. | Mint a handle as one, present it as the other. |
| `mrtr-retries-are-idempotent` | A duplicate retry that returns the right answer is indistinguishable from one that performed the effect twice *and* returned the right answer. | Count the effect at its source, not in the reply. |
| `invocations-are-audited` | An audit log is not exposed over MCP; a server claiming to keep one would be believed on its own word. | Read the log and verify its chain. |
| `SENTINEL/SECURITY/invocations-are-audited` *(beyond-spec: the security page has no audit-logging requirement)* | An audit log is not exposed over MCP; a server claiming to keep one would be believed on its own word. | Read the log and verify its chain. |

**A clean scan is not a clean bill of health.** A harness that scored these as passes would be
lying, and a tool that implied otherwise would be worse than no tool.

The broker implements all five, and proves it with tests that *do* have the access a scan lacks —
The broker implements all five properties, and proves it with tests that *do* have the access a scan lacks —
`TestTokenForAnotherAudienceRejected`, `TestInboundTokenNeverForwarded`,
`TestCrossPrincipalHandleRefused`, `TestDuplicateRetryIsIdempotent`, `TestChainDetectsTampering`.
That is the point of shipping both: the scan says what it can see, and the server's own suite
Expand Down
4 changes: 2 additions & 2 deletions docs/MIGRATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -319,7 +319,7 @@ and a scanner cannot mint a token signed by your issuer.

## Limits: what a scan cannot tell you

Five MUST rules cannot be settled from the wire, and `sentinel` reports them as **`INDETERMINATE`**
Four MUST rules, and one beyond-spec security rule, cannot be settled from the wire, and `sentinel` reports them as **`INDETERMINATE`**
rather than as passes:

| Rule | What would settle it |
Expand All @@ -328,7 +328,7 @@ rather than as passes:
| `no-token-passthrough` | Capture your server's egress while it serves an authenticated call; confirm the inbound token appears in none of it. |
| `handle-possession-is-not-authentication` | Mint a handle as one principal, present it as another; confirm the refusal. |
| `mrtr-retries-are-idempotent` | Retry a completed flow and count the effect **at its source**, not in the reply. |
| `invocations-are-audited` | Read the log directly and verify its chain. |
| `SENTINEL/SECURITY/invocations-are-audited` (beyond-spec) | Read the log directly and verify its chain. |

A clean scan is not a clean bill of health, and a tool that implied otherwise would be worse than
no tool. These five are where you still have to look yourself.
Expand Down
4 changes: 2 additions & 2 deletions harness/src/sentinel/catalog/beyond/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,6 @@

from __future__ import annotations

from sentinel.catalog.beyond import budget, style
from sentinel.catalog.beyond import budget, security, style

__all__ = ["budget", "style"]
__all__ = ["budget", "security", "style"]
41 changes: 41 additions & 0 deletions harness/src/sentinel/catalog/beyond/security.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
"""Beyond-spec security checks.

These are `SENTINEL/SECURITY/*`: properties this project holds a server to that
the specification's security page does not state as requirements. They are the
rules that used to be filed under the spec namespace on the strength of a
citation that, read closely, never said what the rule claimed.
"""

from __future__ import annotations

from sentinel.catalog.base import RuleResult, Severity, Verifiability, rule
from sentinel.probe.client import Probe


@rule(
id="SENTINEL/SECURITY/invocations-are-audited",
title="Every tool invocation is recorded in an append-only log",
severity=Severity.MUST,
rationale=(
"The 2026-07-28 security best-practices page has no audit-logging section, so "
"no MUST in the specification requires this. It is still the property that "
"lets an operator answer 'what did this agent do, and as whom' after the "
"fact, and a server that cannot answer is hard to defend in an incident. "
"Sentinel holds servers to it as an opinion, not as conformance."
),
verifiability=Verifiability.UNVERIFIABLE,
remediation=(
"Write an append-only, hash-chained row for every invocation, and fail the "
"invocation if the write fails. Enforce append-only with database grants rather "
"than convention, so the property does not depend on every future code path "
"remembering it."
),
introduced_in="0.3.0",
)
def invocations_audited(_: Probe) -> RuleResult:
return RuleResult.indeterminate(
"An audit log is not exposed over MCP, and a server that claimed to keep one would "
"be believed on its own word -- which is the opposite of what an audit is for. "
"To settle it: read the log directly and verify its chain, e.g. with a "
"verification command the server ships."
)
10 changes: 5 additions & 5 deletions harness/src/sentinel/catalog/deprecations.py
Original file line number Diff line number Diff line change
Expand Up @@ -200,7 +200,7 @@ def months_remaining(self, detection: Detection) -> int | None:
id="roots",
name="Roots",
replacement="explicit tool arguments naming the paths a tool may touch",
citation=f"{SPEC_BASE}/changelog#sep-2577-roots-sampling-logging",
citation=f"{SPEC_BASE}/changelog#deprecated",
sep="SEP-2577",
deprecated_on=date(2026, 7, 28),
removal=_SEP_2577,
Expand All @@ -214,7 +214,7 @@ def months_remaining(self, detection: Detection) -> int | None:
id="sampling",
name="Sampling",
replacement="multi round-trip requests (resultType: input_required)",
citation=f"{SPEC_BASE}/changelog#sep-2577-roots-sampling-logging",
citation=f"{SPEC_BASE}/changelog#deprecated",
sep="SEP-2577",
deprecated_on=date(2026, 7, 28),
removal=_SEP_2577,
Expand All @@ -227,7 +227,7 @@ def months_remaining(self, detection: Detection) -> int | None:
id="logging",
name="Logging",
replacement="_meta.io.modelcontextprotocol/logLevel, set per request",
citation=f"{SPEC_BASE}/changelog#sep-2577-roots-sampling-logging",
citation=f"{SPEC_BASE}/changelog#deprecated",
sep="SEP-2577",
deprecated_on=date(2026, 7, 28),
removal=_SEP_2577,
Expand All @@ -240,7 +240,7 @@ def months_remaining(self, detection: Detection) -> int | None:
id="http-sse",
name="HTTP+SSE transport",
replacement="Streamable HTTP",
citation=f"{SPEC_BASE}/basic/transports#streamable-http",
citation=f"{SPEC_BASE}/basic/transports/streamable-http#backward-compatibility",
sep="SEP-2596",
deprecated_on=date(2025, 3, 26),
# No date, and no way to ask for one. SEP-2596 has not reached Final,
Expand Down Expand Up @@ -270,7 +270,7 @@ def months_remaining(self, detection: Detection) -> int | None:
id="include-context",
name='includeContext: "thisServer" / "allServers"',
replacement="explicit arguments carrying whatever context the call needs",
citation=f"{SPEC_BASE}/changelog#includecontext",
citation=f"{SPEC_BASE}/changelog#deprecated",
sep="SEP-2596",
deprecated_on=date(2025, 11, 25),
# The registry says "Follows Sampling (SEP-2577)" — not a date of its
Expand Down
20 changes: 10 additions & 10 deletions harness/src/sentinel/catalog/must/discovery.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,14 @@
from sentinel.probe.client import KEY_SERVER_INFO, Probe

CHANGELOG = f"{SPEC_BASE}/changelog"
LIFECYCLE = f"{SPEC_BASE}/basic/lifecycle"
LIFECYCLE = f"{SPEC_BASE}/basic/versioning"


@rule(
id="MCP/2026-07-28/MUST/discover-implemented",
title="server/discover is implemented",
severity=Severity.MUST,
citation=f"{CHANGELOG}#server-discover-replaces-initialize",
citation=f"{CHANGELOG}#major-changes",
verifiability=Verifiability.BLACK_BOX,
remediation=(
"Implement server/discover. It replaced the initialize handshake in this revision "
Expand Down Expand Up @@ -61,7 +61,7 @@ def discover_implemented(probe: Probe) -> RuleResult:
id="MCP/2026-07-28/MUST/discover-without-negotiated-version",
title="server/discover answers without a negotiated protocol version",
severity=Severity.MUST,
citation=f"{LIFECYCLE}#version-negotiation",
citation=f"{LIFECYCLE}#protocol-version-negotiation",
verifiability=Verifiability.BLACK_BOX,
remediation=(
"Exempt server/discover from version negotiation. Its purpose is to let a client "
Expand Down Expand Up @@ -99,7 +99,7 @@ def discover_without_version(probe: Probe) -> RuleResult:
id="MCP/2026-07-28/MUST/discover-reports-supported-versions",
title="server/discover reports supportedVersions",
severity=Severity.MUST,
citation=f"{LIFECYCLE}#version-negotiation",
citation=f"{LIFECYCLE}#protocol-version-negotiation",
verifiability=Verifiability.BLACK_BOX,
remediation=(
"Include a non-empty supportedVersions array in the server/discover result. "
Expand All @@ -125,7 +125,7 @@ def discover_reports_versions(probe: Probe) -> RuleResult:
id="MCP/2026-07-28/MUST/discover-reports-server-info",
title="server/discover identifies the server",
severity=Severity.MUST,
citation=f"{LIFECYCLE}#server-identity",
citation=f"{SPEC_BASE}/server/discover#response",
verifiability=Verifiability.BLACK_BOX,
remediation=(
"Report serverInfo with at least a name from server/discover. Clients key cache "
Expand Down Expand Up @@ -158,7 +158,7 @@ def discover_reports_server_info(probe: Probe) -> RuleResult:
id="MCP/2026-07-28/MUST/discover-reports-capabilities",
title="server/discover reports capabilities",
severity=Severity.MUST,
citation=f"{LIFECYCLE}#capabilities",
citation=f"{SPEC_BASE}/server/discover#response",
verifiability=Verifiability.BLACK_BOX,
remediation=(
"Report a capabilities object from server/discover. A client cannot tell an "
Expand All @@ -183,7 +183,7 @@ def discover_reports_capabilities(probe: Probe) -> RuleResult:
id="MCP/2026-07-28/MUST/unsupported-version-rejected",
title="An unsupported protocol version is rejected with -32022",
severity=Severity.MUST,
citation=f"{LIFECYCLE}#version-negotiation",
citation=f"{LIFECYCLE}#protocol-version-negotiation",
verifiability=Verifiability.BLACK_BOX,
remediation=(
"Return -32022 UnsupportedProtocolVersion, with supportedVersions in the error "
Expand Down Expand Up @@ -214,7 +214,7 @@ def unsupported_version_rejected(probe: Probe) -> RuleResult:
id="MCP/2026-07-28/MUST/list-changed-advertised-truthfully",
title="A declared listChanged capability is backed by subscriptions/listen",
severity=Severity.MUST,
citation=f"{CHANGELOG}#subscriptions",
citation=f"{CHANGELOG}#major-changes",
verifiability=Verifiability.BLACK_BOX,
remediation=(
"Advertise listChanged: false unless subscriptions/listen is implemented. A client "
Expand Down Expand Up @@ -279,7 +279,7 @@ def list_changed_is_truthful(probe: Probe) -> RuleResult:
id="MCP/2026-07-28/MUST/initialize-removed",
title="The initialize handshake is gone",
severity=Severity.MUST,
citation=f"{CHANGELOG}#server-discover-replaces-initialize",
citation=f"{CHANGELOG}#major-changes",
verifiability=Verifiability.BLACK_BOX,
remediation=(
"Remove the initialize method and answer method-not-found for it. This revision is "
Expand Down Expand Up @@ -315,7 +315,7 @@ def _removed_method_rule(method: str, slug: str, why: str, replacement: str) ->
id=f"MCP/2026-07-28/MUST/{slug}",
title=f"{method} is removed",
severity=Severity.MUST,
citation=f"{CHANGELOG}#removed-methods",
citation=f"{CHANGELOG}#major-changes",
verifiability=Verifiability.BLACK_BOX,
remediation=(
f"Stop serving {method}; it was removed in 2026-07-28. {why} "
Expand Down
14 changes: 7 additions & 7 deletions harness/src/sentinel/catalog/must/envelope.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
id="MCP/2026-07-28/MUST/result-type-present",
title="Every result carries a non-empty resultType",
severity=Severity.MUST,
citation=f"{BASIC}/index#result-envelope",
citation=f"{BASIC}/index#resulttype",
verifiability=Verifiability.BLACK_BOX,
remediation=(
"Set resultType on every result — 'complete', or 'input_required' when the call "
Expand Down Expand Up @@ -60,7 +60,7 @@ def result_type_present(probe: Probe) -> RuleResult:
id="MCP/2026-07-28/MUST/server-info-echoed",
title="Every result echoes serverInfo",
severity=Severity.MUST,
citation=f"{BASIC}/index#result-envelope",
citation=f"{BASIC}/index#resulttype",
verifiability=Verifiability.BLACK_BOX,
remediation=(
"Echo io.modelcontextprotocol/serverInfo in each result's _meta. Clients key cache "
Expand All @@ -83,7 +83,7 @@ def server_info_echoed_deprecated(probe: Probe) -> RuleResult:
id="MCP/2026-07-28/MUST/cacheable-results-carry-ttl",
title="Every list and read result carries ttlMs",
severity=Severity.MUST,
citation=f"{CHANGELOG}#cacheableresult-is-required",
citation=f"{CHANGELOG}#minor-changes",
verifiability=Verifiability.BLACK_BOX,
remediation=(
"Add ttlMs to every list and read result. CacheableResult became required in this "
Expand Down Expand Up @@ -117,7 +117,7 @@ def cacheable_ttl(probe: Probe) -> RuleResult:
id="MCP/2026-07-28/MUST/cacheable-results-carry-scope",
title="Every list and read result carries cacheScope",
severity=Severity.MUST,
citation=f"{CHANGELOG}#cacheableresult-is-required",
citation=f"{CHANGELOG}#minor-changes",
verifiability=Verifiability.BLACK_BOX,
remediation=(
"Add cacheScope ('private' or 'public') to every list and read result. Choose it "
Expand Down Expand Up @@ -157,7 +157,7 @@ def cacheable_scope(probe: Probe) -> RuleResult:
id="MCP/2026-07-28/MUST/tools-list-is-deterministic",
title="tools/list is byte-stable across repeated calls",
severity=Severity.MUST,
citation=f"{CHANGELOG}#deterministic-tool-ordering",
citation=f"{CHANGELOG}#minor-changes",
verifiability=Verifiability.BLACK_BOX,
remediation=(
"Build the tool manifest once, sort it by name byte-wise, and serve the "
Expand All @@ -180,7 +180,7 @@ def tools_list_deterministic_deprecated(probe: Probe) -> RuleResult:
id="MCP/2026-07-28/MUST/tools-declare-input-schema",
title="Every advertised tool declares an input schema",
severity=Severity.MUST,
citation=f"{BASIC}/index#tools",
citation=f"{SPEC_BASE}/server/tools#listing-tools",
verifiability=Verifiability.BLACK_BOX,
remediation=(
"Give every tool an inputSchema. A model calling a tool with no schema has to "
Expand Down Expand Up @@ -215,7 +215,7 @@ def tools_declare_schema(probe: Probe) -> RuleResult:
id="MCP/2026-07-28/MUST/tools-are-named",
title="Every advertised tool has a name",
severity=Severity.MUST,
citation=f"{BASIC}/index#tools",
citation=f"{SPEC_BASE}/server/tools#listing-tools",
verifiability=Verifiability.BLACK_BOX,
remediation=(
"Give every tool a non-empty name. It is the only handle a client has for calling "
Expand Down
10 changes: 5 additions & 5 deletions harness/src/sentinel/catalog/must/headers.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ def _named_tool(probe: Probe) -> str | None:
id="MCP/2026-07-28/MUST/mcp-method-header-required",
title="Mcp-Method is required on Streamable HTTP POST",
severity=Severity.MUST,
citation=f"{TRANSPORT}#streamable-http",
citation=f"{TRANSPORT}#request-metadata",
verifiability=Verifiability.BLACK_BOX,
remediation=(
"Reject a POST with no Mcp-Method header. The header exists so a gateway or WAF "
Expand Down Expand Up @@ -90,7 +90,7 @@ def mcp_method_required(probe: Probe) -> RuleResult:
id="MCP/2026-07-28/MUST/mcp-name-header-required",
title="Mcp-Name is required on a tools/call, resources/read or prompts/get POST",
severity=Severity.MUST,
citation=f"{TRANSPORT}#streamable-http",
citation=f"{TRANSPORT}#request-metadata",
verifiability=Verifiability.BLACK_BOX,
remediation=(
"Reject a tools/call, resources/read or prompts/get with no Mcp-Name header. "
Expand Down Expand Up @@ -150,7 +150,7 @@ def mcp_name_required(probe: Probe) -> RuleResult:
id="MCP/2026-07-28/MUST/mcp-name-not-required-where-undefined",
title="Mcp-Name is not demanded on a method the header table does not define it for",
severity=Severity.MUST,
citation=f"{TRANSPORT}#streamable-http",
citation=f"{TRANSPORT}#request-metadata",
verifiability=Verifiability.BLACK_BOX,
introduced_in="0.2.0",
remediation=(
Expand Down Expand Up @@ -211,7 +211,7 @@ def mcp_name_not_required_where_undefined(probe: Probe) -> RuleResult:
id="MCP/2026-07-28/MUST/header-body-mismatch-rejected",
title="A header disagreeing with the body is rejected with -32020",
severity=Severity.MUST,
citation=f"{TRANSPORT}#header-contract",
citation=f"{TRANSPORT}#request-metadata",
verifiability=Verifiability.BLACK_BOX,
remediation=(
"Compare Mcp-Method and Mcp-Name against the JSON-RPC body and return -32020 "
Expand Down Expand Up @@ -262,7 +262,7 @@ def header_mismatch_rejected(probe: Probe) -> RuleResult:
id="MCP/2026-07-28/MUST/resource-not-found-is-invalid-params",
title="Resource not found is -32602, not -32002",
severity=Severity.MUST,
citation=f"{CHANGELOG}#error-code-reallocation",
citation=f"{ERRORS}",
verifiability=Verifiability.BLACK_BOX,
remediation=(
"Return -32602 InvalidParams for a resource that does not exist. It moved from "
Expand Down
6 changes: 3 additions & 3 deletions harness/src/sentinel/catalog/must/meta.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ def _rejected_as_invalid_params(response: RawResponse, what: str) -> RuleResult:
id="MCP/2026-07-28/MUST/missing-client-capabilities-rejected",
title="A request without clientCapabilities is rejected with -32602 and HTTP 400",
severity=Severity.MUST,
citation=f"{BASIC}#meta",
citation=f"{BASIC}#_meta",
verifiability=Verifiability.BLACK_BOX,
remediation=(
"Reject a request whose _meta omits io.modelcontextprotocol/clientCapabilities "
Expand All @@ -68,7 +68,7 @@ def missing_client_capabilities_rejected(probe: Probe) -> RuleResult:
id="MCP/2026-07-28/MUST/missing-protocol-version-rejected",
title="A request without a declared protocol version is rejected with -32602 and HTTP 400",
severity=Severity.MUST,
citation=f"{BASIC}#meta",
citation=f"{BASIC}#_meta",
verifiability=Verifiability.BLACK_BOX,
remediation=(
"Reject a request whose _meta omits io.modelcontextprotocol/protocolVersion with "
Expand All @@ -92,7 +92,7 @@ def missing_protocol_version_rejected(probe: Probe) -> RuleResult:
id="MCP/2026-07-28/MUST/missing-capability-error-shape",
title="A -32021 error names the capabilities it needed",
severity=Severity.MUST,
citation=f"{BASIC}#meta",
citation=f"{BASIC}#_meta",
verifiability=Verifiability.BLACK_BOX,
remediation=(
"When returning -32021 MissingRequiredClientCapability, populate "
Expand Down
Loading
Loading