From ca23118d77b176d19b10788029a1f258af24b0f3 Mon Sep 17 00:00:00 2001 From: Tommaso Bolis Date: Tue, 22 Sep 2026 13:25:59 +0200 Subject: [PATCH 1/6] chore: gitignore /.claude/workflows (#530) Closes #529 --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index 89eebdf0..dd4f130e 100644 --- a/.gitignore +++ b/.gitignore @@ -38,6 +38,7 @@ node_modules/ /CLAUDE.md /.claude/skills /.claude/settings.json +/.claude/workflows # Playwright MCP scratch output (local agent tooling) .playwright-mcp/ From 641195e7a0e34351cdff5538437857f05140ed33 Mon Sep 17 00:00:00 2001 From: Tommaso Bolis Date: Tue, 22 Sep 2026 14:08:30 +0200 Subject: [PATCH 2/6] chore: gitignore /.claude/commands in the public repo (#532) Claude slash-command definitions are canonical in the private donkey-development-kit-claude companion repo and symlinked into the public tree under .claude/commands/. Add the matching ignore line so the symlink does not show as untracked and cannot be accidentally committed, mirroring the /.claude/skills, /.claude/settings.json, and /.claude/workflows lines. Co-authored-by: Claude Opus 4.8 (1M context) --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index dd4f130e..7ada22d1 100644 --- a/.gitignore +++ b/.gitignore @@ -39,6 +39,7 @@ node_modules/ /.claude/skills /.claude/settings.json /.claude/workflows +/.claude/commands # Playwright MCP scratch output (local agent tooling) .playwright-mcp/ From f6bacc5653f822a2973044c58263604ed646b20f Mon Sep 17 00:00:00 2001 From: Tommaso Bolis Date: Tue, 22 Sep 2026 14:41:20 +0200 Subject: [PATCH 3/6] docs: group Typed refusals under the six-piece-minimum sidebar separator (#534) Closes #533 --- website/content/_meta.js | 2 +- website/public/llms.txt | 5 +---- 2 files changed, 2 insertions(+), 5 deletions(-) diff --git a/website/content/_meta.js b/website/content/_meta.js index 1615dbfa..8beeb1a2 100644 --- a/website/content/_meta.js +++ b/website/content/_meta.js @@ -9,11 +9,11 @@ export default { title: 'Live today', }, frameworks: 'Model access', - errors: 'Typed refusals', '-- six-piece': { type: 'separator', title: 'Phase 1 — the six-piece minimum', }, + errors: 'Typed refusals', budget: 'Budget & pacing', simulator: 'Local simulator', testing: 'Testing & conformance', diff --git a/website/public/llms.txt b/website/public/llms.txt index 44f9c75b..355ca51f 100644 --- a/website/public/llms.txt +++ b/website/public/llms.txt @@ -22,12 +22,9 @@ - [CrewAI](https://donkey-development-kit.github.io/donkey-development-kit/frameworks/crewai.md): CrewAI gets a governed `LLM`, backed by LiteLLM under the hood, pointed at the Agent Fabric LLM proxy through LiteLLM's own model-string and kwarg conventions rather than raw OpenAI ones. - [LlamaIndex](https://donkey-development-kit.github.io/donkey-development-kit/frameworks/llamaindex.md): LlamaIndex gets a governed `OpenAILike` LLM pointed at the Agent Fabric LLM proxy, with the one flag that a chat-only gateway absolutely requires forced on for you. -## Live today - -- [Typed refusals](https://donkey-development-kit.github.io/donkey-development-kit/errors.md): The proxy doesn't just pass model calls through — it enforces policy. When it rejects a call, the SDK turns the response into a **typed exception** so you branch on the governance outcome instead of… - ## Phase 1 — the six-piece minimum +- [Typed refusals](https://donkey-development-kit.github.io/donkey-development-kit/errors.md): The proxy doesn't just pass model calls through — it enforces policy. When it rejects a call, the SDK turns the response into a **typed exception** so you branch on the governance outcome instead of… - [Budget & pacing](https://donkey-development-kit.github.io/donkey-development-kit/budget.md): Remaining token budget as a first-class object, with pacing that slows down before it hits the wall. - [Local simulator](https://donkey-development-kit.github.io/donkey-development-kit/simulator.md): A local server that replays real gateway rejections, so you can develop against a governed gateway without having one. - [Testing & conformance](https://donkey-development-kit.github.io/donkey-development-kit/testing.md): In-process refusal injection, a pytest plugin you run against your own agent, and an out-of-process gateway fixture. From 556a5c3b405fbf254fbddc7891907a773b38da2c Mon Sep 17 00:00:00 2001 From: Tommaso Bolis Date: Tue, 22 Sep 2026 14:44:39 +0200 Subject: [PATCH 4/6] Tighten docs-site sidebar items and hero tagline size (#536) Closes #535 --- website/styles/globals.css | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/website/styles/globals.css b/website/styles/globals.css index e503e5ea..6f49f9ad 100644 --- a/website/styles/globals.css +++ b/website/styles/globals.css @@ -246,11 +246,11 @@ body { .nextra-sidebar a, .nextra-sidebar button { color: var(--af-fg-muted); - font-size: 1rem; + font-size: 0.9rem; font-weight: 500; line-height: 1.5; letter-spacing: -0.006em; - padding-block: 0.5rem; + padding-block: 0.2rem; } .nextra-sidebar a:hover, @@ -317,7 +317,7 @@ body { .af-hero-tagline { margin-top: 1.05rem; - font-size: 1.12rem; + font-size: 1rem; line-height: 1.6; color: var(--af-fg-muted); max-width: 44rem; From 601e37497f882e9c2f7b9fa75347a664df385c89 Mon Sep 17 00:00:00 2001 From: Tommaso Bolis Date: Tue, 22 Sep 2026 15:20:14 +0200 Subject: [PATCH 5/6] chore: bump version to 0.1.0.dev6 (#538) --- python/pyproject.toml | 2 +- python/src/donkey_kit/__init__.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/python/pyproject.toml b/python/pyproject.toml index c8e30051..2b9987a5 100644 --- a/python/pyproject.toml +++ b/python/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "hatchling.build" [project] name = "donkey-kit" -version = "0.1.0.dev5" +version = "0.1.0.dev6" description = "An SDK for consuming Agent Fabric capabilities (governed model and tool access) from your own agent framework." readme = "README.md" requires-python = ">=3.10" diff --git a/python/src/donkey_kit/__init__.py b/python/src/donkey_kit/__init__.py index b16e1a15..6cbaff34 100644 --- a/python/src/donkey_kit/__init__.py +++ b/python/src/donkey_kit/__init__.py @@ -52,7 +52,7 @@ PublicationAssetType, ) -__version__ = "0.1.0.dev5" +__version__ = "0.1.0.dev6" __all__ = [ "STRICT", From 57727821ed423e2bac5f0d08f1af1b0d1e0a1ddb Mon Sep 17 00:00:00 2001 From: Tommaso Bolis Date: Tue, 22 Sep 2026 15:21:51 +0200 Subject: [PATCH 6/6] =?UTF-8?q?chore(verify):=20live-capture=20regex-promp?= =?UTF-8?q?t-guard=20+=20Azure=20content-safety=20rejections=20(=C2=A70.3,?= =?UTF-8?q?=20#253)=20(#537)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Refs #253 (partial — issue remains open for the three uncaptured shapes). --- docs/verified-apis.md | 60 ++++++++++++------- python/src/donkey_kit/core/errors.py | 31 ++++++---- python/tests/fixtures/fixtures.lock | 4 +- python/tests/fixtures/rejections/README.md | 56 ++++++++++------- .../reject.content-safety.body.json | 2 +- .../reject.content-safety.headers.txt | 2 +- python/tests/unit/test_rejection_contract.py | 17 ++++-- website/content/concepts/verification.mdx | 8 ++- website/content/errors.mdx | 33 +++++----- website/content/feature-overview.mdx | 9 +-- website/content/index.mdx | 5 +- website/public/concepts/verification.md | 8 ++- website/public/errors.md | 33 +++++----- website/public/feature-overview.md | 9 +-- website/public/index.md | 5 +- website/public/llms-full.txt | 55 +++++++++-------- 16 files changed, 202 insertions(+), 135 deletions(-) diff --git a/docs/verified-apis.md b/docs/verified-apis.md index c062733e..14ab48e3 100644 --- a/docs/verified-apis.md +++ b/docs/verified-apis.md @@ -14,9 +14,14 @@ > (§2) and the token-rate-limit (`429`, empty body, header-only budget) + PII > (`403`, `type:pii_detected`) rejection contracts (§4) are now also > `VERIFIED (LIVE)` — both policies were applied to `openai-sdk` for capture and -> removed. Still `UNVERIFIED`: prompt-injection / regex-prompt-guard / -> content-safety rejection bodies (§4, typed from the policy pages, pending live -> capture #253) and the **framework constructor/binding names** (§§8–10). §11 +> removed. The regex-prompt-guard (`403` + +> `matched_patterns`) and Azure content-safety (`403` + `…-action: reject`) +> rejection shapes were also `VERIFIED (LIVE)` on 2026-09-22 against the deployed +> `ddk-injection-guard` / `ddk-azure-content-safety` proxies (#253). Still +> `UNVERIFIED`: the Injection Protection body (`x-injection-protection: blocked`, +> a distinct policy — no proxy deployed) and any other content-moderation / +> Bedrock-guardrail shape (§4, pending live capture #253), and the **framework +> constructor/binding names** (§§8–10). §11 > records A2D shapes > (`VERIFIED-SHAPE-ONLY`), used only to validate SDK value types; no blocked > code path was wired to them. @@ -208,9 +213,14 @@ point the SDK at it via the `cost_*_header` overrides. ## 4. Policy rejection response shapes (capture as fixtures, BG §1.5) -**Four rejection shapes** are LIVE-VERIFIED (2026-08-28) from the `openai-sdk` -proxy (client-id-enforcement, upstream passthrough, PII, token-rate-limit). -Fixtures in `tests/fixtures/anypoint/llm_proxy/reject.*`. The critical lesson: +**Six rejection shapes** are now LIVE-VERIFIED: four from the `openai-sdk` proxy +(client-id-enforcement, upstream passthrough, PII, token-rate-limit; 2026-08-28), +plus **Regex Prompt Guard** and **Azure Content Safety** — both `403` policy +blocks confirmed 2026-09-22 against the deployed provisioning proxies +`ddk-injection-guard` (instance 21179713) and `ddk-azure-content-safety` +(instance 21180957), rows added to the policy table below (#253). +Fixtures in `tests/fixtures/anypoint/llm_proxy/reject.*` and +`tests/fixtures/rejections/reject.*`. The critical lesson: **neither the status code nor the mere shape of the `error` value is a sufficient discriminator** — the same nested-object envelope is emitted by both the upstream provider AND a gateway policy (PII), and a policy block can be a @@ -253,26 +263,32 @@ whose message names the observed status and any `x-llm-proxy-*` policy headers a states the **shape is unconfirmed** (#184), rather than being mis-typed. The full eight-shape taxonomy is indexed in `tests/fixtures/rejections/README.md`. -The Regex Prompt Guard and content-safety/guardrail shapes below are -**DOCUMENTED, pending live capture (#253)** — typed from the policy pages, not -from a live sandbox round-trip, so **no `_verify.py` row flips to -`verified=True`** for them: +The Regex Prompt Guard and content-safety/guardrail shapes below are now +**VERIFIED (LIVE), 2026-09-22 (#253)** — confirmed against the deployed +provisioning proxies (`ddk-injection-guard`, `ddk-azure-content-safety`), not +only typed from the policy pages. They have **no `_verify.py` constant** — +`classify()` reads them straight from the response — so the verification record +is these rows, not an `Unverified(...)` flip: | Shape | HTTP | Discriminator | Maps to | Source | |---|---|---|---|---| -| Regex Prompt Guard | `403` | top-level `matched_patterns` list (flat-string `error`) | `PromptInjectionBlocked` (`policy="regex-prompt-guard"`) | [Regex Prompt Guard policy](https://docs.mulesoft.com/gateway/latest/policies-included-regex-prompt-guard); DOCUMENTED, pending live capture (#253) | -| Content safety / guardrails | `403` | `x-llm-proxy-azure-content-safety-action` / `x-llm-proxy-bedrock-guardrail-action` == `reject`; reasons in the sibling `…-reason` header | `ContentSafetyBlocked` (parses `categories`) | [Azure Content Safety policy](https://docs.mulesoft.com/gateway/latest/policies-included-azure-content-safety); [Amazon Bedrock Guardrails policy](https://docs.mulesoft.com/gateway/latest/policies-included-bedrock-guardrails); DOCUMENTED, pending live capture (#253) | +| Regex Prompt Guard | `403` | top-level `matched_patterns` list (flat-string `error`) | `PromptInjectionBlocked` (`policy="regex-prompt-guard"`) | [Regex Prompt Guard policy](https://docs.mulesoft.com/gateway/latest/policies-included-regex-prompt-guard) (v1.11.4); VERIFIED (LIVE) 2026-09-22 on `ddk-injection-guard` (instance 21179713) | +| Content safety / guardrails | `403` | `x-llm-proxy-azure-content-safety-action` / `x-llm-proxy-bedrock-guardrail-action` == `reject`; reasons in the sibling `…-reason` header | `ContentSafetyBlocked` (parses `categories`) | [Azure Content Safety policy](https://docs.mulesoft.com/gateway/latest/policies-included-azure-content-safety) (v1.13.0); [Amazon Bedrock Guardrails policy](https://docs.mulesoft.com/gateway/latest/policies-included-bedrock-guardrails) (v1.13.0); VERIFIED (LIVE) 2026-09-22 on `ddk-azure-content-safety` (instance 21180957, Azure verified; Bedrock still documented-only) | | Unrecognised refusal (fall-through) | any non-429 `4xx` | matches **none** of the discriminators above; no nested `error` envelope; no `www-authenticate` (e.g. an unrecognised `403`) | generic `PolicyViolation` (`policy="unknown"`), message says **shape unconfirmed** and names the observed status + `x-llm-proxy-*` headers | UNVERIFIED — no known contract; capture + type via #184/#253 | -The **injection body** and any **other content-moderation / federated-guardrail** -shape remain uncaptured. Rather than guess a type for them, `classify()` surfaces -any such unrecognised refusal **honestly** — a `PolicyViolation` whose message -states the shape is unconfirmed and whose remediation asks the operator to file -the observed status/headers/body so the shape can be typed (#184). This is the -last row above; it is deliberately **not** an `AuthError`, even for a `403`, once -the verified `www-authenticate` auth shape is excluded. The official policy -pages above establish the documented contracts; direct sandbox capture remains -tracked in #253 and is still required before any verification status changes. +The Regex Prompt Guard (row 7) and Azure Content Safety (row 8) shapes are now +LIVE-captured (2026-09-22, #253). Still **uncaptured**: the **Injection +Protection** body (`x-injection-protection: blocked`, a policy *distinct* from +Regex Prompt Guard — no proxy running it is deployed) and any **other +content-moderation / federated-guardrail** shape (e.g. Bedrock Guardrails). +Rather than guess a type for them, `classify()` surfaces any such unrecognised +refusal **honestly** — a `PolicyViolation` whose message states the shape is +unconfirmed and whose remediation asks the operator to file the observed +status/headers/body so the shape can be typed (#184). This is the last row +above; it is deliberately **not** an `AuthError`, even for a `403`, once the +verified `www-authenticate` auth shape is excluded. Closing these last shapes +needs an Injection Protection (and a Bedrock Guardrails) proxy deployed to +capture against; that residual keeps #253 open. **Budget window emission — two forms, keyed on outcome not status class (LIVE-VERIFIED).** The gateway signals its token-rate-limit window in two @@ -326,6 +342,8 @@ the prose parser from #352) now populates identically from a simulated or a live | LLM proxy core | `llm-proxy-core` `1.0.5` | applied VERIFIED (LIVE) | on `openai-sdk`; rejection body not yet triggered | 2026-08-28 | `policy:list` | | Token rate limiting | interface `llm-token-rate-limit` `1.0.2` (impl `-policy-flex` `1.0.4`) | VERIFIED (LIVE) | Two emission forms: `429` limit-exceeded → **empty body**, numeric trio `x-token-limit`/`x-token-remaining`/`x-token-reset`(ms), no `retry-after`; `200`/`403` under the same policy → the window as prose in a single `x-llm-proxy-ratelimit` header (ms reset). See "Budget window emission" above. | 2026-08-28 | applied + live probe | | PII detection | interface `llm-pii-detection-policy` `1.0.0` (impl `-flex` `1.0.2`) | VERIFIED (LIVE) | `403`, nested `{"error":{message,type:"pii_detected"}}`, no `www-authenticate` | 2026-08-28 | applied + live probe | +| Regex Prompt Guard | `regex-prompt-guard-policy` `1.0.0` | VERIFIED (LIVE) | `403`, flat-string `error` + top-level `matched_patterns` list; body matched the committed fixture byte-for-byte | 2026-09-22 | live probe, `ddk-injection-guard` instance 21179713 | +| Azure Content Safety | `azure-content-safety-policy` `1.0.0` | VERIFIED (LIVE) | `403`, `x-llm-proxy-azure-content-safety-action: reject` + `-phase` + `-reason` headers; body `{"error":…,"categories":[…]}` (categories prompt-dependent, e.g. `severity_hate,severity_violence`, `prompt_shield`) | 2026-09-22 | live probe, `ddk-azure-content-safety` instance 21180957 (private-space gateway) | **Apply note (verified):** these LLM policies apply against the schema-bearing **interface** asset id/version (`llm-token-rate-limit` `1.0.2`, diff --git a/python/src/donkey_kit/core/errors.py b/python/src/donkey_kit/core/errors.py index 657ef874..258a9335 100644 --- a/python/src/donkey_kit/core/errors.py +++ b/python/src/donkey_kit/core/errors.py @@ -473,11 +473,16 @@ def classify( (flagged reasons parsed from the sibling ``...-reason`` header). Also checked before the auth rule. - These two shapes are **documented, not yet live-captured** (same posture as - #181's header-based injection typing; docs/verified-apis.md §4). Exact strings are pinned from - the policy pages; no ``_verify.py`` row flips to ``verified=True`` until a - live sandbox round-trip confirms them (#253). Any other content-moderation / - federated-guardrail shape still falls through to a generic + Both shapes are now **live-verified** (2026-09-22, #253): Regex Prompt Guard + against ``ddk-injection-guard`` and Azure Content Safety against + ``ddk-azure-content-safety`` (docs/verified-apis.md §4). These shapes have no + ``_verify.py`` constant — ``classify()`` reads them straight from the + response — so the record is the §4 rows, not a ``verified=True`` flip. Amazon + Bedrock Guardrails (same ``...-action: reject`` header family) and the + Injection Protection body (``x-injection-protection: blocked``, a distinct + policy) stay documented-only — no proxy running them is deployed to capture + (#253). Any other content-moderation / federated-guardrail shape still falls + through to a generic :class:`PolicyViolation` whose message names what was observed and says the shape is unconfirmed (#184). Because auth is the verified 401 / ``www-authenticate`` shape (docs/verified-apis.md §4), a **403 carrying no ``www-authenticate`` header** and matching @@ -522,9 +527,10 @@ def classify( # Content-safety / guardrails policy: 403 + a vendor `...-action: reject` # header (Azure Content Safety / Amazon Bedrock Guardrails, docs/verified-apis.md §4). - # Documented, pending live capture (#253). Checked before the 401/403 → auth - # rule because a moderation block is not an auth failure. Keyed on the - # header, not the body, so the body-less Bedrock reject is caught too. + # Azure verified LIVE 2026-09-22 (#253); Bedrock still documented-only. + # Checked before the 401/403 → auth rule because a moderation block is not an + # auth failure. Keyed on the header, not the body, so the body-less Bedrock + # reject is caught too. cs = _content_safety_reject(response) if cs is not None: vendor, categories = cs @@ -542,7 +548,8 @@ def classify( # Regex Prompt Guard policy: 403 + a top-level `matched_patterns` list # (flat-string `error`, so NOT the nested upstream envelope; docs/verified-apis.md §4). - # Documented, pending live capture (#253). Checked before the 401/403 → auth + # Verified LIVE 2026-09-22 against ddk-injection-guard (#253); body matched + # the committed fixture byte-for-byte. Checked before the 401/403 → auth # rule so a deny-list block is not mis-typed as an auth failure. matched = body.get("matched_patterns") if body is not None else None if isinstance(matched, list): @@ -730,9 +737,9 @@ def _json_body(response: httpx.Response) -> dict[str, Any] | None: # Content-safety / guardrails policies report their verdict in a pair of vendor # headers — an ``...-action`` (``allow``|``reject``) and a comma-separated -# ``...-reason``. Both are ``x-llm-proxy--...`` (docs/verified-apis.md §4). Pinned from the -# Azure Content Safety and Amazon Bedrock Guardrails policy pages; documented, -# pending live capture (#253). +# ``...-reason``. Both are ``x-llm-proxy--...`` (docs/verified-apis.md §4). +# Azure Content Safety verified LIVE 2026-09-22 against ddk-azure-content-safety +# (#253); Amazon Bedrock Guardrails still documented-only (no proxy deployed). _CONTENT_SAFETY_VENDORS: tuple[tuple[str, str, str], ...] = ( ( "x-llm-proxy-azure-content-safety-action", diff --git a/python/tests/fixtures/fixtures.lock b/python/tests/fixtures/fixtures.lock index 88716753..d62303db 100644 --- a/python/tests/fixtures/fixtures.lock +++ b/python/tests/fixtures/fixtures.lock @@ -11,8 +11,8 @@ "anypoint/llm_proxy/responses.success.body.json": "sha256:c47727c6ffb0d2dec80ed37d182825d5707de043517a890999fefa4e407dc90e", "anypoint/llm_proxy/responses.success.headers.txt": "sha256:263ce673cad339d8cc06e095924fb5ff1310f84d47f591f18eb1a8c90fb8027b", "rejections/reject.content-moderation.headers.txt": "sha256:5df6a849d1ea245db54617e66f1d380566270f23102ca0cb8046a1c8b990cbda", - "rejections/reject.content-safety.body.json": "sha256:4c9c71529bd564d3c72e4e6dc383fa527088290c9016b3afb2f8706e5e32fbf1", - "rejections/reject.content-safety.headers.txt": "sha256:9bfd3b7d8e34b08a1df5771ce4e545dc000473ab9f240da623892e049bc587fa", + "rejections/reject.content-safety.body.json": "sha256:678a01904e36159a5089f4e0784fc3a61b9fcdeefda04e6a16c07b8e81fea267", + "rejections/reject.content-safety.headers.txt": "sha256:d991dbc094133c574968b5f735d000398fb556bbae5c65139fba11ce7e3bfdec", "rejections/reject.injection-protection.headers.txt": "sha256:7964c425ed91c8498b6f25474d6cd91c0350ec3803496f75c40fb71004930ea2", "rejections/reject.regex-prompt-guard.body.json": "sha256:9bf2ca369b9250edbac362f3d74904d76d8c6e338c1c9ef53fe33b9848db7f23", "rejections/reject.regex-prompt-guard.headers.txt": "sha256:01c9848a2470e700602713dc0530656c4ff6be647b69441308b24e78ab3e11ae", diff --git a/python/tests/fixtures/rejections/README.md b/python/tests/fixtures/rejections/README.md index 23072a49..e5343a73 100644 --- a/python/tests/fixtures/rejections/README.md +++ b/python/tests/fixtures/rejections/README.md @@ -9,16 +9,28 @@ the `.headers.txt` / `.body.json` / `.body.empty` convention). ## Provenance & verification status (verification discipline) -The build guide describes these contracts as "public in Omni Gateway -v1.11–v1.13", but **no public docs URL for the rejection contracts is recorded -anywhere in this repo**, and `v1.11–v1.13` has no concrete in-repo source (the -only live `v1.13` is the Flex Gateway *runtime* v1.13.2; docs/verified-apis.md §4 cites policy -*asset* versions). Writing a docs URL or a "docs version" into these fixtures -would be inventing an endpoint, which the verification discipline forbids. So each row below records -only what is **defensible**, and the "cite the public docs URL + version" -portion of the #181 acceptance criteria is left open under **#253** (`verify: -re-confirm the v1.11-v1.13 rejection contracts against current docs and a -sandbox`) rather than fabricated to green the checkbox. +These contracts are now **public in the Omni Gateway policy reference**, so each +row below cites its documented source (docs URL + the product line that +introduced the policy) — no longer left open as "no URL recorded" (#253): + +| Policy (LLM lane) | Docs page | Introduced | +|---|---|---| +| Injection Protection (row 3) | https://docs.mulesoft.com/gateway/latest/policies-included-injection-protection | v1.12.0 | +| Regex Prompt Guard (row 7) | https://docs.mulesoft.com/gateway/latest/policies-included-regex-prompt-guard | v1.11.4 | +| LLM PII Detection (row 2) | https://docs.mulesoft.com/gateway/latest/policies-included-llm-pii-detection | v1.13.0 | +| LLM Token Rate Limit (row 1) | https://docs.mulesoft.com/gateway/latest/policies-included-llm-token-rate-limit | v1.11.0 | +| Azure Content Safety (row 8) | https://docs.mulesoft.com/gateway/latest/policies-included-azure-content-safety | v1.13.0 | +| Amazon Bedrock Guardrails (row 8, sibling) | https://docs.mulesoft.com/gateway/latest/policies-included-bedrock-guardrails | v1.13.0 | + +**Live-capture status (#253, 2026-09-22):** rows 7 and 8 were re-confirmed +against the deployed provisioning proxies — `ddk-injection-guard` (Regex Prompt +Guard, instance 21179713, shared-omni-gateway) and `ddk-azure-content-safety` +(Azure Content Safety, instance 21180957, private-space-omni-gateway). Row 3 +(Injection Protection, `x-injection-protection: blocked`) and row 4 (the +fall-through) stay documented-only: **no Injection Protection policy proxy is +deployed** to capture them, so their bytes remain honest placeholders (the +deployed injection-lane proxy runs Regex Prompt Guard = row 7, a distinct +policy). Rows 1/2/5 remain LIVE from the original 2026-08-28 captures. ## The eight rows @@ -26,12 +38,12 @@ sandbox`) rather than fabricated to green the checkbox. |---|-------|---------|--------------|---------------|------------| | 1 | Token rate limit | `../anypoint/llm_proxy/reject.token-rate-limit.{headers.txt,body.empty}` | `TokenBudgetExceeded` | 429 + `x-token-limit`/`-remaining`/`-reset`, empty body | LIVE, 2026-08-28 (`llm-token-rate-limit` 1.0.2) | | 2 | PII detection | `../anypoint/llm_proxy/reject.pii-detected.{headers.txt,body.json}` | `PIIDetected` | nested `error.type == "pii_detected"`, no `www-authenticate` | LIVE, 2026-08-28 (`llm-pii-detection-policy` 1.0.0) | -| 3 | Injection protection | `reject.injection-protection.{headers.txt,body.empty}` | `PromptInjectionBlocked` | header `x-injection-protection: blocked` (not status) | **Shape UNVERIFIED** — header discriminator only; body unknown → empty placeholder. Blocked on #253. | +| 3 | Injection protection | `reject.injection-protection.{headers.txt,body.empty}` | `PromptInjectionBlocked` | header `x-injection-protection: blocked` (not status) | **Shape UNVERIFIED** — header discriminator only; body unknown → empty placeholder. [Injection Protection policy](https://docs.mulesoft.com/gateway/latest/policies-included-injection-protection) (v1.12.0); still open on #253 — no Injection Protection proxy deployed. | | 4 | Content moderation / federated guardrails | `reject.content-moderation.{headers.txt,body.empty}` | generic `PolicyViolation` | falls through (no nested error, no injection/guard/safety discriminator) | **UNDER-DOCUMENTED** — minimal 4xx proving the fall-through stays generic. Blocked on #253. | | 5 | Upstream provider 4xx | `../anypoint/llm_proxy/reject.model-not-found.body.json` | `UpstreamRequestError` | non-429 4xx, nested `error` with `code`/`type`/`param` | LIVE, 2026-08-28 (OpenAI passthrough) | | 6 | Upstream 5xx | `reject.upstream-5xx.{headers.txt,body.empty}` | `UpstreamModelError` (retryable) | 5xx status range (no competing discriminator) | **SYNTHETIC** — status-range classification only; no live capture, no invented body. | -| 7 | Regex Prompt Guard | `reject.regex-prompt-guard.{headers.txt,body.json}` | `PromptInjectionBlocked` (`policy="regex-prompt-guard"`) | 403 + top-level `matched_patterns` list (flat-string `error`) | **DOCUMENTED, pending live capture (#253)** — body carries a representative `matched_patterns` list because the discriminator *is* a body field; pinned from the Regex Prompt Guard policy page, not a live capture. | -| 8 | Content safety / guardrails | `reject.content-safety.{headers.txt,body.json}` | `ContentSafetyBlocked` (parses `categories`) | 403 + `x-llm-proxy--…-action: reject` (Azure Content Safety / Bedrock Guardrails) | **DOCUMENTED, pending live capture (#253)** — header discriminator pinned from the Azure Content Safety / Bedrock Guardrails policy pages; the `.body.json` is illustrative, not a live capture. | +| 7 | Regex Prompt Guard | `reject.regex-prompt-guard.{headers.txt,body.json}` | `PromptInjectionBlocked` (`policy="regex-prompt-guard"`) | 403 + top-level `matched_patterns` list (flat-string `error`) | **VERIFIED (LIVE), 2026-09-22 (#253)** — body matches the live capture byte-for-byte against `ddk-injection-guard` (instance 21179713). [Regex Prompt Guard policy](https://docs.mulesoft.com/gateway/latest/policies-included-regex-prompt-guard) (v1.11.4). | +| 8 | Content safety / guardrails | `reject.content-safety.{headers.txt,body.json}` | `ContentSafetyBlocked` (parses `categories`) | 403 + `x-llm-proxy--…-action: reject` (Azure Content Safety / Bedrock Guardrails) | **VERIFIED (LIVE), 2026-09-22 (#253)** — discriminator headers + body shape confirmed against `ddk-azure-content-safety` (instance 21180957); `.body.json`/`-reason` hold a live-captured category set (`severity_hate,severity_violence`; categories are prompt-dependent). [Azure Content Safety policy](https://docs.mulesoft.com/gateway/latest/policies-included-azure-content-safety) (v1.13.0). | Rows 1, 2, 5 **alias** the existing live captures in `../anypoint/llm_proxy/` (referenced, not copied — moving them would break that directory's contract-test @@ -39,15 +51,17 @@ helpers and provenance chain). Rows 3, 4, 6 live here because they are not (yet) live-captured; their bodies are zero-byte `.body.empty` placeholders — the honest "shape unknown" marker, never a guessed body. -Rows 7 and 8 (#289) also live here and are **DOCUMENTED, pending live capture +Rows 7 and 8 (#289) also live here and are now **VERIFIED (LIVE), 2026-09-22 (#253)**: their discriminators (the `matched_patterns` list, the vendor -`…-action: reject` header) are pinned from the policy pages, and `classify()` -types them from those discriminators, but they have **not** been confirmed -against a live sandbox — no `_verify.py` row flips to `verified=True` for them. -Unlike rows 3/4/6, row 7 needs a non-empty body: its discriminator is a body -field (`matched_patterns`), so an empty placeholder could not exercise the path. -The bodies are the minimal representative shapes needed to drive the -discriminator, explicitly not presented as live captures. +`…-action: reject` header) were confirmed against the deployed provisioning +proxies, not just pinned from the policy pages. The regex-prompt-guard body +matched the committed bytes exactly; the content-safety fixture now carries a +live-captured category set. These shapes have **no `_verify.py` constant** to +flip — `classify()` reads them straight from the response, so the verification +record is the docs/verified-apis.md §4 rows plus this table, not an +`Unverified(...)` guard. Unlike rows 3/4/6, row 7 needs a non-empty body: its +discriminator is a body field (`matched_patterns`), so an empty placeholder could +not exercise the path. Client-ID enforcement (401 + `www-authenticate` → `AuthError`) is a separate **consumer-auth** case, deliberately **not** one of the eight policy-rejection diff --git a/python/tests/fixtures/rejections/reject.content-safety.body.json b/python/tests/fixtures/rejections/reject.content-safety.body.json index cc61b461..ebb1059b 100644 --- a/python/tests/fixtures/rejections/reject.content-safety.body.json +++ b/python/tests/fixtures/rejections/reject.content-safety.body.json @@ -1 +1 @@ -{"error":"Content blocked by Azure Content Safety","categories":["severity_hate","blocklist"]} +{"error":"Content blocked by Azure Content Safety","categories":["severity_hate","severity_violence"]} diff --git a/python/tests/fixtures/rejections/reject.content-safety.headers.txt b/python/tests/fixtures/rejections/reject.content-safety.headers.txt index dbeeb88d..3576c577 100644 --- a/python/tests/fixtures/rejections/reject.content-safety.headers.txt +++ b/python/tests/fixtures/rejections/reject.content-safety.headers.txt @@ -2,4 +2,4 @@ HTTP/1.1 403 Forbidden content-type: application/json x-llm-proxy-azure-content-safety-action: reject x-llm-proxy-azure-content-safety-phase: request -x-llm-proxy-azure-content-safety-reason: severity_hate,blocklist +x-llm-proxy-azure-content-safety-reason: severity_hate,severity_violence diff --git a/python/tests/unit/test_rejection_contract.py b/python/tests/unit/test_rejection_contract.py index 2e26454c..a4ee8224 100644 --- a/python/tests/unit/test_rejection_contract.py +++ b/python/tests/unit/test_rejection_contract.py @@ -4,9 +4,14 @@ Rows 1/2/5 alias the live captures under ``anypoint/llm_proxy/`` (referenced, not moved); rows 3/4/6 (injection, content-moderation, upstream-5xx) and rows 7/8 -(regex-prompt-guard, content-safety — #289) live in ``rejections/``. The #289 -shapes are documented (pinned from the policy pages) but not yet live-captured; -their live capture is blocked on #253 (see ``rejections/README.md`` and docs/verified-apis.md §4). +(regex-prompt-guard, content-safety — #289) live in ``rejections/``. Rows 7/8 are +now LIVE-VERIFIED (2026-09-22, #253): the regex-prompt-guard body matched the +committed fixture byte-for-byte against ``ddk-injection-guard`` (instance +21179713), and the content-safety discriminator headers + body shape were +confirmed against ``ddk-azure-content-safety`` (instance 21180957) — see +``rejections/README.md`` and docs/verified-apis.md §4. Rows 3 (injection-protection +``x-injection-protection: blocked``) and 4 (fall-through) stay documented-only: +no Injection Protection policy proxy is deployed to capture them (#253). The discriminator is the error ``type`` + specific headers, NEVER the status code alone — which is exactly why rows 7/8 (both 403) must not be swallowed by the 401/403 → auth rule. @@ -109,6 +114,8 @@ def test_row8_content_safety_is_content_safety_blocked_not_auth() -> None: err = classify(_response(REJECTIONS, "content-safety", 403)) assert isinstance(err, ContentSafetyBlocked) assert err.policy == "content-safety" - # Flagged reasons are parsed from the sibling `...-reason` header. - assert err.categories == ["severity_hate", "blocklist"] + # Flagged reasons are parsed from the sibling `...-reason` header. The values + # are the live-captured set (Azure returned severity_hate,severity_violence + # on a hate-speech probe against ddk-azure-content-safety, #253). + assert err.categories == ["severity_hate", "severity_violence"] assert err.remediation # required, non-empty diff --git a/website/content/concepts/verification.mdx b/website/content/concepts/verification.mdx index 59bab7f2..8b9e16ea 100644 --- a/website/content/concepts/verification.mdx +++ b/website/content/concepts/verification.mdx @@ -24,8 +24,10 @@ how a blocked value flips to verified — is documented for contributors in and the absence of a `/models` endpoint (`404`). - **Attribution** (live-verified): the `client_id` credential is the per-agent attribution unit; the gateway emits identity/telemetry on the response. -- **The four policy rejection shapes** (live-verified): auth `401`, PII `403`, - token-budget `429`, upstream passthrough. See [Error taxonomy](/errors). +- **Six policy rejection shapes** (live-verified): auth `401`, PII `403`, + token-budget `429`, upstream passthrough, plus the **regex prompt-guard** and + **Azure content-safety** `403` blocks (the last two confirmed 2026-09-22 + against the deployed provisioning proxies, #253). See [Error taxonomy](/errors). ## What is still blocked — and why that's a feature @@ -39,7 +41,7 @@ how a blocked value flips to verified — is documented for contributors in | Budget query endpoint ([Phase 1](/budget)) | **Missing upstream** | Budget is reported in-band on response headers only, so `remaining` is last-known-good; `observed_at` exposes the staleness. | | Whether Anypoint Monitoring ingests OTLP GenAI spans ([Phase 1](/telemetry)) | Unverified | The SDK promises "exports OTLP" and lets the sink be your choice. | | A provisioning control plane | **Cut** | Not built. It would compete with API Manager and Terraform — see [Roadmap](/roadmap). | -| Prompt-injection / regex-guard / content-safety bodies | Not captured | Injection is typed by the `x-injection-protection: blocked` header, regex prompt-guard by a top-level `matched_patterns` list (both → `PromptInjectionBlocked`), and content-safety by the Azure Content Safety / Bedrock Guardrails `…-action: reject` header (→ `ContentSafetyBlocked`) — all pinned from the policy pages, but their **bodies** are uncaptured and no verification row flips until a live round-trip confirms them. Any other moderation shape still falls through to a generic `PolicyViolation`. No invented discriminator — bodies tracked in #253. | +| Injection-protection / Bedrock-guardrails / fall-through bodies | Not captured | The **regex prompt-guard** (`matched_patterns` list) and **Azure content-safety** (`…-action: reject` header) bodies are now **live-captured** (2026-09-22, #253). Still uncaptured: the distinct **Injection Protection** policy's body (typed by the `x-injection-protection: blocked` header; no such proxy is deployed), the **Bedrock Guardrails** sibling of content-safety, and the undiscriminated content-moderation fall-through to a generic `PolicyViolation` — all pinned from the policy pages, and no verification row flips until a live round-trip confirms them. No invented discriminator — tracked in #253. | | First-party **TypeScript SDK** | Planned | TypeScript examples reach the proxy over its OpenAI-compatible API via the `openai` / `@anthropic-ai/sdk` clients; no first-party TS package is implied until it ships. | ## What this means for you diff --git a/website/content/errors.mdx b/website/content/errors.mdx index ff65c289..45878971 100644 --- a/website/content/errors.mdx +++ b/website/content/errors.mdx @@ -8,12 +8,13 @@ branch on the governance outcome instead of parsing bodies. ## The rejection shapes `classify()` types -`classify()` types eight rejection shapes. **Four are live-verified** against the -sandbox proxy (auth, PII, token rate limit, upstream); **injection** is typed by -its header discriminator but its body is pending live capture; and three more — -**regex prompt guard**, **content safety / guardrails**, and the -**undiscriminated content-moderation** fall-through — are typed from the policy -pages, pending sandbox confirmation, all tracked in [#253]. The critical lesson: +`classify()` types eight rejection shapes. **Six are live-verified** against the +sandbox proxies (auth, PII, token rate limit, upstream, **regex prompt guard**, +and **Azure content safety** — the last two confirmed 2026-09-22, [#253]). +Still pending live capture: **injection protection** (typed by its header +discriminator, body uncaptured — a distinct policy from regex prompt guard, with +no proxy deployed), Amazon **Bedrock guardrails**, and the **undiscriminated +content-moderation** fall-through — all tracked in [#253]. The critical lesson: **neither the status code nor the shape of the `error` value alone is a sufficient discriminator** — a `403` can be PII, a regex-guard block, a content-safety block (all policy blocks) *or* auth, and the same nested-object @@ -25,8 +26,8 @@ authoritative discriminator is the error **`type`** plus specific headers. | Client-ID enforcement (auth) | `401` | flat `{"error":"…"}` + `www-authenticate: Client-ID-Enforcement` | `AuthError` | live | | PII detected | `403` | nested `{"error":{type:"pii_detected"}}`, **no** `www-authenticate` | `PIIDetected` (parses `entities`) | live | | Injection protection | `400` | header `x-injection-protection: blocked` (**not** the status) | `PromptInjectionBlocked` | pending (#253) | -| Regex prompt guard | `403` | top-level `matched_patterns` list (flat `error`) | `PromptInjectionBlocked` (`policy="regex-prompt-guard"`) | pending (#253) | -| Content safety / guardrails | `403` | header `x-llm-proxy--…-action: reject` (Azure Content Safety / Bedrock Guardrails) | `ContentSafetyBlocked` (parses `categories`) | pending (#253) | +| Regex prompt guard | `403` | top-level `matched_patterns` list (flat `error`) | `PromptInjectionBlocked` (`policy="regex-prompt-guard"`) | live (#253) | +| Content safety / guardrails | `403` | header `x-llm-proxy--…-action: reject` (Azure Content Safety / Bedrock Guardrails) | `ContentSafetyBlocked` (parses `categories`) | live — Azure; Bedrock pending (#253) | | Token rate limit | `429` | **empty body**; `x-token-limit`/`-remaining`/`-reset` headers (ms) | `TokenBudgetExceeded` (`retry_after` derived) | live | | Content moderation (undiscriminated) | `4xx` | falls through — no nested `error`, no injection/guard/safety discriminator | generic `PolicyViolation` | pending (#253) | | Upstream provider 4xx | `4xx` | nested `error` object **with** `code`/`type`/`param` | `UpstreamRequestError` | live | @@ -216,13 +217,15 @@ surfaces immediately as `AuthError`. ## Documented-but-not-live-captured shapes -Four discriminators — the `x-injection-protection` header, the regex -`matched_patterns` list, and the Azure Content Safety / Bedrock Guardrails -`…-action: reject` headers — are **pinned from the policy pages** and typed by -`classify()`, but have **not yet been confirmed against a live sandbox**; their -exact bodies are tracked in [#253]. `classify()` therefore types them from the -header/field discriminator only, and no verification row flips to `verified` -until a live round-trip confirms the shape. Any *other* content-moderation / +The regex `matched_patterns` list and the **Azure Content Safety** +`…-action: reject` header are now **live-captured** (2026-09-22, [#253]) against +the deployed provisioning proxies. Two documented discriminators remain pinned +from the policy pages but **not yet confirmed against a live sandbox**: the +`x-injection-protection` header of the distinct **Injection Protection** policy +(no such proxy is deployed to capture it) and the **Bedrock Guardrails** sibling +of the content-safety header. `classify()` types those from the header +discriminator only, and no verification row flips to `verified` until a live +round-trip confirms the shape. Any *other* content-moderation / federated-guardrail shape still falls through to a generic `PolicyViolation` rather than an invented type — this is deliberate (verification discipline): we don't invent a discriminator we haven't observed. See [Verification policy](/concepts/verification). diff --git a/website/content/feature-overview.mdx b/website/content/feature-overview.mdx index 50c7d8e7..e21d17e9 100644 --- a/website/content/feature-overview.mdx +++ b/website/content/feature-overview.mdx @@ -144,10 +144,11 @@ and `registered_tools` are exported from `donkey_kit`. header; parses `categories`), a generic `PolicyViolation` fall-through for undiscriminated content-moderation, `UpstreamRequestError` (4xx) and `UpstreamModelError` (5xx), plus `AuthError` (401) for the separate consumer-auth -case — so you branch on governance outcomes instead of parsing bodies. Four are -live-verified; the injection, regex-prompt-guard, content-safety and -content-moderation bodies are pinned from the policy pages, pending sandbox -capture (#253). See [Error taxonomy](/errors). +case — so you branch on governance outcomes instead of parsing bodies. Six are +live-verified (including the regex-prompt-guard and Azure content-safety `403` +blocks, confirmed 2026-09-22); the injection-protection, Bedrock-guardrails and +content-moderation fall-through bodies are pinned from the policy pages, pending +sandbox capture (#253). See [Error taxonomy](/errors). Those exception types are the client-side mirror of the policies the gateway enforces. The **LLM** lane below is the one this SDK targets today; the same diff --git a/website/content/index.mdx b/website/content/index.mdx index 220f8d84..3e535a6e 100644 --- a/website/content/index.mdx +++ b/website/content/index.mdx @@ -179,8 +179,9 @@ simulation, and refusal handlers can all attach *without you wiring each one*. **Governed model access is live-verified** against a real Anypoint sandbox: the proxy base-URL shape (`https:////`, no `/v1`), the `client_id` + `client_secret` consumer-auth header pair, the attribution - headers, and the four policy rejection shapes (auth `401`, PII `403`, - token-budget `429`, upstream passthrough). The framework-free client and the + headers, and six policy rejection shapes (auth `401`, PII `403`, + token-budget `429`, upstream passthrough, plus the regex prompt-guard and + Azure content-safety `403` blocks). The framework-free client and the adapters are wired to that verified contract. diff --git a/website/public/concepts/verification.md b/website/public/concepts/verification.md index 06efac4a..a062e5bd 100644 --- a/website/public/concepts/verification.md +++ b/website/public/concepts/verification.md @@ -20,8 +20,10 @@ how a blocked value flips to verified — is documented for contributors in and the absence of a `/models` endpoint (`404`). - **Attribution** (live-verified): the `client_id` credential is the per-agent attribution unit; the gateway emits identity/telemetry on the response. -- **The four policy rejection shapes** (live-verified): auth `401`, PII `403`, - token-budget `429`, upstream passthrough. See [Error taxonomy](https://donkey-development-kit.github.io/donkey-development-kit/errors.md). +- **Six policy rejection shapes** (live-verified): auth `401`, PII `403`, + token-budget `429`, upstream passthrough, plus the **regex prompt-guard** and + **Azure content-safety** `403` blocks (the last two confirmed 2026-09-22 + against the deployed provisioning proxies, #253). See [Error taxonomy](https://donkey-development-kit.github.io/donkey-development-kit/errors.md). ## What is still blocked — and why that's a feature @@ -35,7 +37,7 @@ how a blocked value flips to verified — is documented for contributors in | Budget query endpoint ([Phase 1](https://donkey-development-kit.github.io/donkey-development-kit/budget.md)) | **Missing upstream** | Budget is reported in-band on response headers only, so `remaining` is last-known-good; `observed_at` exposes the staleness. | | Whether Anypoint Monitoring ingests OTLP GenAI spans ([Phase 1](https://donkey-development-kit.github.io/donkey-development-kit/telemetry.md)) | Unverified | The SDK promises "exports OTLP" and lets the sink be your choice. | | A provisioning control plane | **Cut** | Not built. It would compete with API Manager and Terraform — see [Roadmap](https://donkey-development-kit.github.io/donkey-development-kit/roadmap.md). | -| Prompt-injection / regex-guard / content-safety bodies | Not captured | Injection is typed by the `x-injection-protection: blocked` header, regex prompt-guard by a top-level `matched_patterns` list (both → `PromptInjectionBlocked`), and content-safety by the Azure Content Safety / Bedrock Guardrails `…-action: reject` header (→ `ContentSafetyBlocked`) — all pinned from the policy pages, but their **bodies** are uncaptured and no verification row flips until a live round-trip confirms them. Any other moderation shape still falls through to a generic `PolicyViolation`. No invented discriminator — bodies tracked in #253. | +| Injection-protection / Bedrock-guardrails / fall-through bodies | Not captured | The **regex prompt-guard** (`matched_patterns` list) and **Azure content-safety** (`…-action: reject` header) bodies are now **live-captured** (2026-09-22, #253). Still uncaptured: the distinct **Injection Protection** policy's body (typed by the `x-injection-protection: blocked` header; no such proxy is deployed), the **Bedrock Guardrails** sibling of content-safety, and the undiscriminated content-moderation fall-through to a generic `PolicyViolation` — all pinned from the policy pages, and no verification row flips until a live round-trip confirms them. No invented discriminator — tracked in #253. | | First-party **TypeScript SDK** | Planned | TypeScript examples reach the proxy over its OpenAI-compatible API via the `openai` / `@anthropic-ai/sdk` clients; no first-party TS package is implied until it ships. | ## What this means for you diff --git a/website/public/errors.md b/website/public/errors.md index 72e01fe2..251725e8 100644 --- a/website/public/errors.md +++ b/website/public/errors.md @@ -6,12 +6,13 @@ branch on the governance outcome instead of parsing bodies. ## The rejection shapes `classify()` types -`classify()` types eight rejection shapes. **Four are live-verified** against the -sandbox proxy (auth, PII, token rate limit, upstream); **injection** is typed by -its header discriminator but its body is pending live capture; and three more — -**regex prompt guard**, **content safety / guardrails**, and the -**undiscriminated content-moderation** fall-through — are typed from the policy -pages, pending sandbox confirmation, all tracked in [#253]. The critical lesson: +`classify()` types eight rejection shapes. **Six are live-verified** against the +sandbox proxies (auth, PII, token rate limit, upstream, **regex prompt guard**, +and **Azure content safety** — the last two confirmed 2026-09-22, [#253]). +Still pending live capture: **injection protection** (typed by its header +discriminator, body uncaptured — a distinct policy from regex prompt guard, with +no proxy deployed), Amazon **Bedrock guardrails**, and the **undiscriminated +content-moderation** fall-through — all tracked in [#253]. The critical lesson: **neither the status code nor the shape of the `error` value alone is a sufficient discriminator** — a `403` can be PII, a regex-guard block, a content-safety block (all policy blocks) *or* auth, and the same nested-object @@ -23,8 +24,8 @@ authoritative discriminator is the error **`type`** plus specific headers. | Client-ID enforcement (auth) | `401` | flat `{"error":"…"}` + `www-authenticate: Client-ID-Enforcement` | `AuthError` | live | | PII detected | `403` | nested `{"error":{type:"pii_detected"}}`, **no** `www-authenticate` | `PIIDetected` (parses `entities`) | live | | Injection protection | `400` | header `x-injection-protection: blocked` (**not** the status) | `PromptInjectionBlocked` | pending (#253) | -| Regex prompt guard | `403` | top-level `matched_patterns` list (flat `error`) | `PromptInjectionBlocked` (`policy="regex-prompt-guard"`) | pending (#253) | -| Content safety / guardrails | `403` | header `x-llm-proxy--…-action: reject` (Azure Content Safety / Bedrock Guardrails) | `ContentSafetyBlocked` (parses `categories`) | pending (#253) | +| Regex prompt guard | `403` | top-level `matched_patterns` list (flat `error`) | `PromptInjectionBlocked` (`policy="regex-prompt-guard"`) | live (#253) | +| Content safety / guardrails | `403` | header `x-llm-proxy--…-action: reject` (Azure Content Safety / Bedrock Guardrails) | `ContentSafetyBlocked` (parses `categories`) | live — Azure; Bedrock pending (#253) | | Token rate limit | `429` | **empty body**; `x-token-limit`/`-remaining`/`-reset` headers (ms) | `TokenBudgetExceeded` (`retry_after` derived) | live | | Content moderation (undiscriminated) | `4xx` | falls through — no nested `error`, no injection/guard/safety discriminator | generic `PolicyViolation` | pending (#253) | | Upstream provider 4xx | `4xx` | nested `error` object **with** `code`/`type`/`param` | `UpstreamRequestError` | live | @@ -212,13 +213,15 @@ surfaces immediately as `AuthError`. ## Documented-but-not-live-captured shapes -Four discriminators — the `x-injection-protection` header, the regex -`matched_patterns` list, and the Azure Content Safety / Bedrock Guardrails -`…-action: reject` headers — are **pinned from the policy pages** and typed by -`classify()`, but have **not yet been confirmed against a live sandbox**; their -exact bodies are tracked in [#253]. `classify()` therefore types them from the -header/field discriminator only, and no verification row flips to `verified` -until a live round-trip confirms the shape. Any *other* content-moderation / +The regex `matched_patterns` list and the **Azure Content Safety** +`…-action: reject` header are now **live-captured** (2026-09-22, [#253]) against +the deployed provisioning proxies. Two documented discriminators remain pinned +from the policy pages but **not yet confirmed against a live sandbox**: the +`x-injection-protection` header of the distinct **Injection Protection** policy +(no such proxy is deployed to capture it) and the **Bedrock Guardrails** sibling +of the content-safety header. `classify()` types those from the header +discriminator only, and no verification row flips to `verified` until a live +round-trip confirms the shape. Any *other* content-moderation / federated-guardrail shape still falls through to a generic `PolicyViolation` rather than an invented type — this is deliberate (verification discipline): we don't invent a discriminator we haven't observed. See [Verification policy](https://donkey-development-kit.github.io/donkey-development-kit/concepts/verification.md). diff --git a/website/public/feature-overview.md b/website/public/feature-overview.md index 15f3add2..49af6415 100644 --- a/website/public/feature-overview.md +++ b/website/public/feature-overview.md @@ -129,10 +129,11 @@ and `registered_tools` are exported from `donkey_kit`. header; parses `categories`), a generic `PolicyViolation` fall-through for undiscriminated content-moderation, `UpstreamRequestError` (4xx) and `UpstreamModelError` (5xx), plus `AuthError` (401) for the separate consumer-auth -case — so you branch on governance outcomes instead of parsing bodies. Four are -live-verified; the injection, regex-prompt-guard, content-safety and -content-moderation bodies are pinned from the policy pages, pending sandbox -capture (#253). See [Error taxonomy](https://donkey-development-kit.github.io/donkey-development-kit/errors.md). +case — so you branch on governance outcomes instead of parsing bodies. Six are +live-verified (including the regex-prompt-guard and Azure content-safety `403` +blocks, confirmed 2026-09-22); the injection-protection, Bedrock-guardrails and +content-moderation fall-through bodies are pinned from the policy pages, pending +sandbox capture (#253). See [Error taxonomy](https://donkey-development-kit.github.io/donkey-development-kit/errors.md). Those exception types are the client-side mirror of the policies the gateway enforces. The **LLM** lane below is the one this SDK targets today; the same diff --git a/website/public/index.md b/website/public/index.md index a7e05202..038c6028 100644 --- a/website/public/index.md +++ b/website/public/index.md @@ -122,8 +122,9 @@ simulation, and refusal handlers can all attach *without you wiring each one*. **Governed model access is live-verified** against a real Anypoint sandbox: the proxy base-URL shape (`https:////`, no `/v1`), the `client_id` + `client_secret` consumer-auth header pair, the attribution - headers, and the four policy rejection shapes (auth `401`, PII `403`, - token-budget `429`, upstream passthrough). The framework-free client and the + headers, and six policy rejection shapes (auth `401`, PII `403`, + token-budget `429`, upstream passthrough, plus the regex prompt-guard and + Azure content-safety `403` blocks). The framework-free client and the adapters are wired to that verified contract. Beyond the live-verified contract above, shipped SDK capability pages say what diff --git a/website/public/llms-full.txt b/website/public/llms-full.txt index 34122bfe..31069526 100644 --- a/website/public/llms-full.txt +++ b/website/public/llms-full.txt @@ -132,8 +132,9 @@ simulation, and refusal handlers can all attach *without you wiring each one*. **Governed model access is live-verified** against a real Anypoint sandbox: the proxy base-URL shape (`https:////`, no `/v1`), the `client_id` + `client_secret` consumer-auth header pair, the attribution - headers, and the four policy rejection shapes (auth `401`, PII `403`, - token-budget `429`, upstream passthrough). The framework-free client and the + headers, and six policy rejection shapes (auth `401`, PII `403`, + token-budget `429`, upstream passthrough, plus the regex prompt-guard and + Azure content-safety `403` blocks). The framework-free client and the adapters are wired to that verified contract. Beyond the live-verified contract above, shipped SDK capability pages say what @@ -478,10 +479,11 @@ and `registered_tools` are exported from `donkey_kit`. header; parses `categories`), a generic `PolicyViolation` fall-through for undiscriminated content-moderation, `UpstreamRequestError` (4xx) and `UpstreamModelError` (5xx), plus `AuthError` (401) for the separate consumer-auth -case — so you branch on governance outcomes instead of parsing bodies. Four are -live-verified; the injection, regex-prompt-guard, content-safety and -content-moderation bodies are pinned from the policy pages, pending sandbox -capture (#253). See [Error taxonomy](https://donkey-development-kit.github.io/donkey-development-kit/errors.md). +case — so you branch on governance outcomes instead of parsing bodies. Six are +live-verified (including the regex-prompt-guard and Azure content-safety `403` +blocks, confirmed 2026-09-22); the injection-protection, Bedrock-guardrails and +content-moderation fall-through bodies are pinned from the policy pages, pending +sandbox capture (#253). See [Error taxonomy](https://donkey-development-kit.github.io/donkey-development-kit/errors.md). Those exception types are the client-side mirror of the policies the gateway enforces. The **LLM** lane below is the one this SDK targets today; the same @@ -1949,12 +1951,13 @@ branch on the governance outcome instead of parsing bodies. ## The rejection shapes `classify()` types -`classify()` types eight rejection shapes. **Four are live-verified** against the -sandbox proxy (auth, PII, token rate limit, upstream); **injection** is typed by -its header discriminator but its body is pending live capture; and three more — -**regex prompt guard**, **content safety / guardrails**, and the -**undiscriminated content-moderation** fall-through — are typed from the policy -pages, pending sandbox confirmation, all tracked in [#253]. The critical lesson: +`classify()` types eight rejection shapes. **Six are live-verified** against the +sandbox proxies (auth, PII, token rate limit, upstream, **regex prompt guard**, +and **Azure content safety** — the last two confirmed 2026-09-22, [#253]). +Still pending live capture: **injection protection** (typed by its header +discriminator, body uncaptured — a distinct policy from regex prompt guard, with +no proxy deployed), Amazon **Bedrock guardrails**, and the **undiscriminated +content-moderation** fall-through — all tracked in [#253]. The critical lesson: **neither the status code nor the shape of the `error` value alone is a sufficient discriminator** — a `403` can be PII, a regex-guard block, a content-safety block (all policy blocks) *or* auth, and the same nested-object @@ -1966,8 +1969,8 @@ authoritative discriminator is the error **`type`** plus specific headers. | Client-ID enforcement (auth) | `401` | flat `{"error":"…"}` + `www-authenticate: Client-ID-Enforcement` | `AuthError` | live | | PII detected | `403` | nested `{"error":{type:"pii_detected"}}`, **no** `www-authenticate` | `PIIDetected` (parses `entities`) | live | | Injection protection | `400` | header `x-injection-protection: blocked` (**not** the status) | `PromptInjectionBlocked` | pending (#253) | -| Regex prompt guard | `403` | top-level `matched_patterns` list (flat `error`) | `PromptInjectionBlocked` (`policy="regex-prompt-guard"`) | pending (#253) | -| Content safety / guardrails | `403` | header `x-llm-proxy--…-action: reject` (Azure Content Safety / Bedrock Guardrails) | `ContentSafetyBlocked` (parses `categories`) | pending (#253) | +| Regex prompt guard | `403` | top-level `matched_patterns` list (flat `error`) | `PromptInjectionBlocked` (`policy="regex-prompt-guard"`) | live (#253) | +| Content safety / guardrails | `403` | header `x-llm-proxy--…-action: reject` (Azure Content Safety / Bedrock Guardrails) | `ContentSafetyBlocked` (parses `categories`) | live — Azure; Bedrock pending (#253) | | Token rate limit | `429` | **empty body**; `x-token-limit`/`-remaining`/`-reset` headers (ms) | `TokenBudgetExceeded` (`retry_after` derived) | live | | Content moderation (undiscriminated) | `4xx` | falls through — no nested `error`, no injection/guard/safety discriminator | generic `PolicyViolation` | pending (#253) | | Upstream provider 4xx | `4xx` | nested `error` object **with** `code`/`type`/`param` | `UpstreamRequestError` | live | @@ -2155,13 +2158,15 @@ surfaces immediately as `AuthError`. ## Documented-but-not-live-captured shapes -Four discriminators — the `x-injection-protection` header, the regex -`matched_patterns` list, and the Azure Content Safety / Bedrock Guardrails -`…-action: reject` headers — are **pinned from the policy pages** and typed by -`classify()`, but have **not yet been confirmed against a live sandbox**; their -exact bodies are tracked in [#253]. `classify()` therefore types them from the -header/field discriminator only, and no verification row flips to `verified` -until a live round-trip confirms the shape. Any *other* content-moderation / +The regex `matched_patterns` list and the **Azure Content Safety** +`…-action: reject` header are now **live-captured** (2026-09-22, [#253]) against +the deployed provisioning proxies. Two documented discriminators remain pinned +from the policy pages but **not yet confirmed against a live sandbox**: the +`x-injection-protection` header of the distinct **Injection Protection** policy +(no such proxy is deployed to capture it) and the **Bedrock Guardrails** sibling +of the content-safety header. `classify()` types those from the header +discriminator only, and no verification row flips to `verified` until a live +round-trip confirms the shape. Any *other* content-moderation / federated-guardrail shape still falls through to a generic `PolicyViolation` rather than an invented type — this is deliberate (verification discipline): we don't invent a discriminator we haven't observed. See [Verification policy](https://donkey-development-kit.github.io/donkey-development-kit/concepts/verification.md). @@ -4426,8 +4431,10 @@ how a blocked value flips to verified — is documented for contributors in and the absence of a `/models` endpoint (`404`). - **Attribution** (live-verified): the `client_id` credential is the per-agent attribution unit; the gateway emits identity/telemetry on the response. -- **The four policy rejection shapes** (live-verified): auth `401`, PII `403`, - token-budget `429`, upstream passthrough. See [Error taxonomy](https://donkey-development-kit.github.io/donkey-development-kit/errors.md). +- **Six policy rejection shapes** (live-verified): auth `401`, PII `403`, + token-budget `429`, upstream passthrough, plus the **regex prompt-guard** and + **Azure content-safety** `403` blocks (the last two confirmed 2026-09-22 + against the deployed provisioning proxies, #253). See [Error taxonomy](https://donkey-development-kit.github.io/donkey-development-kit/errors.md). ## What is still blocked — and why that's a feature @@ -4441,7 +4448,7 @@ how a blocked value flips to verified — is documented for contributors in | Budget query endpoint ([Phase 1](https://donkey-development-kit.github.io/donkey-development-kit/budget.md)) | **Missing upstream** | Budget is reported in-band on response headers only, so `remaining` is last-known-good; `observed_at` exposes the staleness. | | Whether Anypoint Monitoring ingests OTLP GenAI spans ([Phase 1](https://donkey-development-kit.github.io/donkey-development-kit/telemetry.md)) | Unverified | The SDK promises "exports OTLP" and lets the sink be your choice. | | A provisioning control plane | **Cut** | Not built. It would compete with API Manager and Terraform — see [Roadmap](https://donkey-development-kit.github.io/donkey-development-kit/roadmap.md). | -| Prompt-injection / regex-guard / content-safety bodies | Not captured | Injection is typed by the `x-injection-protection: blocked` header, regex prompt-guard by a top-level `matched_patterns` list (both → `PromptInjectionBlocked`), and content-safety by the Azure Content Safety / Bedrock Guardrails `…-action: reject` header (→ `ContentSafetyBlocked`) — all pinned from the policy pages, but their **bodies** are uncaptured and no verification row flips until a live round-trip confirms them. Any other moderation shape still falls through to a generic `PolicyViolation`. No invented discriminator — bodies tracked in #253. | +| Injection-protection / Bedrock-guardrails / fall-through bodies | Not captured | The **regex prompt-guard** (`matched_patterns` list) and **Azure content-safety** (`…-action: reject` header) bodies are now **live-captured** (2026-09-22, #253). Still uncaptured: the distinct **Injection Protection** policy's body (typed by the `x-injection-protection: blocked` header; no such proxy is deployed), the **Bedrock Guardrails** sibling of content-safety, and the undiscriminated content-moderation fall-through to a generic `PolicyViolation` — all pinned from the policy pages, and no verification row flips until a live round-trip confirms them. No invented discriminator — tracked in #253. | | First-party **TypeScript SDK** | Planned | TypeScript examples reach the proxy over its OpenAI-compatible API via the `openai` / `@anthropic-ai/sdk` clients; no first-party TS package is implied until it ships. | ## What this means for you