Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
2845ac9
Mark Phase 1 telemetry features as shipped (#470)
tbolis-at-mulesoft Sep 21, 2026
df5fc5a
Close Donkey-owned auth transport during teardown (#466)
tbolis-at-mulesoft Sep 21, 2026
cbdaac2
fix(tools): return independent filtered views (#468)
tbolis-at-mulesoft Sep 21, 2026
9ec944a
test(adapters): cover Agent Framework middleware and guard paths (#469)
tbolis-at-mulesoft Sep 21, 2026
cd39257
Fix duplicate public AssetType contracts (#467)
tbolis-at-mulesoft Sep 21, 2026
742ebda
docs(adapters): correct framework verification comments (#483)
tbolis-at-mulesoft Sep 21, 2026
69e54fd
docs(cli): replace stale provisioning citations (#482)
tbolis-at-mulesoft Sep 21, 2026
204fc0d
Fix budget pacing recovery after window reset (#480)
tbolis-at-mulesoft Sep 21, 2026
1a3302d
Fix AuthError consumer credential guidance (#478)
tbolis-at-mulesoft Sep 21, 2026
0d9a70f
Distinguish shipped roadmap capabilities from proposals (#472)
tbolis-at-mulesoft Sep 21, 2026
b620fc3
Replace stale milestone citations outside provisioning (#490)
tbolis-at-mulesoft Sep 21, 2026
c431b2e
Remove provisioning-as-code from the README headline (#481)
tbolis-at-mulesoft Sep 21, 2026
bdd71d9
Populate the platform support boundary ledger (#473)
tbolis-at-mulesoft Sep 21, 2026
be46737
Test connected-app OAuth token flow (#493)
tbolis-at-mulesoft Sep 21, 2026
07e888f
Demote legacy provisioning scaffolding from public API (#474)
tbolis-at-mulesoft Sep 21, 2026
6d61501
Inject the clock into pace and guard the e2e retry (#492)
tbolis-at-mulesoft Sep 21, 2026
c86a49b
Clarify export verification boundary (#500)
tbolis-at-mulesoft Sep 21, 2026
290f4e3
Retire obsolete milestone vocabulary outside Python source (#499)
tbolis-at-mulesoft Sep 21, 2026
52ca259
chore(release): bump version to 0.1.0.dev3 (#502)
tbolis-at-mulesoft Sep 21, 2026
6afb932
test(cli): cover hidden provisioning command contracts (#476)
tbolis-at-mulesoft Sep 21, 2026
c62faff
Correct OAuth token path verification wording (#477)
tbolis-at-mulesoft Sep 21, 2026
545d148
Correct budget reset pass-through wording (#495)
tbolis-at-mulesoft Sep 21, 2026
36c8f69
Protect verification-blocked registry and tool guards (#475)
tbolis-at-mulesoft Sep 21, 2026
9197896
Make fixture-lock helpers wheel-safe (#498)
tbolis-at-mulesoft Sep 21, 2026
f5d9709
Stop budget retry loops when reset time is unknown (#494)
tbolis-at-mulesoft Sep 21, 2026
5d461ba
docs(testing): publish gateway identity exemptions (#479)
tbolis-at-mulesoft Sep 21, 2026
bb5483b
Test remaining verification-blocked surfaces (#505)
tbolis-at-mulesoft Sep 21, 2026
19189de
Fix AuthError remediation for control-plane failures (#507)
tbolis-at-mulesoft Sep 21, 2026
176503b
Assert header-only adapter correlation exemptions (#506)
tbolis-at-mulesoft Sep 21, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 27 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ defaults:
jobs:
base-only:
# Working instruction #9: install ONLY the base package and run the base
# suite, to catch accidental top-level framework imports.
# suite to catch accidental top-level framework imports. The installed-wheel
# check runs afterward in its own environment.
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
Expand All @@ -27,6 +28,30 @@ jobs:
- name: import donkey_kit with no framework extras installed
run: python -c "import donkey_kit; from donkey_kit import Donkey; print(donkey_kit.__version__)"
- run: pytest -q tests/unit
- name: installed wheel resolves packaged simulator fixtures (BG §1.4)
run: |
wheel_dir="$(mktemp -d)"
venv_dir="$(mktemp -d)"
trap 'rm -rf "$wheel_dir" "$venv_dir"' EXIT
pip install build
python -m build --wheel --outdir "$wheel_dir"
set -- "$wheel_dir"/*.whl
test "$#" -eq 1
python -m venv "$venv_dir"
"$venv_dir/bin/pip" install "$1"
"$venv_dir/bin/python" - <<'PY'
from donkey_kit.simulator import fixtures

# Prove load() cannot silently fall back to checkout-only resources.
assert not fixtures._SOURCE_ROOT.exists(), fixtures._SOURCE_ROOT
for shape in fixtures.SHAPES:
fixtures.load(shape)
assert fixtures.LOCK_PATH.exists(), fixtures.LOCK_PATH
assert fixtures.compute_manifest() == fixtures.read_lock()
before = fixtures.LOCK_PATH.read_bytes()
fixtures.write_lock()
assert fixtures.LOCK_PATH.read_bytes() == before
PY

typecheck-and-lint:
runs-on: ubuntu-latest
Expand Down Expand Up @@ -134,7 +159,7 @@ jobs:

# Nightly framework matrix (§8.4) — runs the conformance kit against the
# LATEST release of each framework and opens an issue on failure. Scheduled
# separately once adapters are conformance-wired (M1+).
# separately during Phase 1 and later.

docs-llms-drift:
# #205 / BG §1.10: llms.txt, llms-full.txt, and the per-page .md files are
Expand Down
7 changes: 3 additions & 4 deletions .github/workflows/nightly-matrix.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,9 @@
# 3. pytest — the unit + ergonomics suites.
#
# NOTE (known constraint): the extras carry no `openai` upper bound ("floors,
# never ceilings"), so a fresh resolve may pull openai 3.x / vendored httpx2,
# which breaks the httpx transport injection the SDK is verified against (openai
# 1.x). If a framework job fails on that, it is EXPECTED signal, not a flake —
# see the pending openai-version decision in docs/m1-completion-checklist.md §3.
# never ceilings"). OpenAI 3.x retyped its injected client to vendored httpx2,
# but runtime injection is re-verified and test-pinned; see
# docs/verified-apis.md §8.1.
#
# Live verification (a real proxy round-trip) is intentionally NOT in the nightly
# run: it needs sandbox credentials. Trigger it manually via workflow_dispatch
Expand Down
30 changes: 19 additions & 11 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,9 +114,10 @@ framework that may not be installed.
— span consumers must close in a `finally`, never relying on the response hook.
A hookless client behaves exactly as it did before the hooks were added. The
full contracts live in the `core/transport.py` docstrings.
- **`Governance`** (`governance.py`) is a second top-level object alongside
`Donkey`, outside the linear import stack — it depends only on `core`. It is
ONE object behind three verbs: `simulate()` (an ephemeral local
- **`Governance`** (`governance.py`) is legacy scaffolding outside the linear
import stack — it depends only on `core` and remains reachable from its module,
but it is not exported as first-class `donkey_kit` API. It is ONE object behind
three verbs: `simulate()` (an ephemeral local
gateway harness), `export()` (emit the governed-state manifest), and `resolve()`
(reconcile a running `Donkey` against it, raising `GovernanceDrift` on
mismatch); a separate platform-team-only `apply()` is the deliberate escape
Expand Down Expand Up @@ -182,7 +183,10 @@ the SDK's clearest value over raw HTTP. Two invariants govern the taxonomy in
2. **Every error carries a `remediation`.** On `PolicyViolation` the human-readable
next step is a *required* field — the concrete action to take (e.g. "the budget
window resets in 42m; request an increase in API Manager") is worth more than a
stack trace.
stack trace. `AuthError` uses its class-level LLM-proxy guidance by default and
canonical class-level overrides for connected-app and provider-chain failures,
so its message and remediation always name the same credential plane and auth
provider.

**`classify()` is fixture-driven, not guessed.** The HTTP-response → exception
mapping in `classify()` is populated from real rejection captures taken against a
Expand Down Expand Up @@ -240,18 +244,22 @@ up front. The demotion landed in #197; deepening LangGraph is tracked in #198.
(`python/tests/conformance/suite.py`) runs identically against every adapter. A
framework is "supported" only when it passes every scenario **or** records an
*asserted exemption* in `KNOWN_LIMITATIONS` — never a silent skip. Those
exemptions are published in the README as credibility (e.g. adapters that reach
models through LiteLLM cannot propagate a per-run correlation ID, because LiteLLM
owns the transport).
exemptions are published in the README as credibility. ADK and CrewAI cannot
propagate a per-run correlation ID or populate `donkey.last_call`, because
LiteLLM owns the transport. LlamaIndex and Microsoft Agent Framework have the
same two exemptions because they receive only a static `default_headers`
snapshot, which deliberately excludes the per-run correlation ID, rather than
the SDK's shared HTTP client.

The centre of gravity moves with the roster cut (`BG §1.5`): the internal
matrix shrinks to LangGraph, and the deliverable becomes the **customer-facing
pytest plugin** users run against their own agent (#191).

Two adapters carry documented conformance exemptions: CrewAI (per-run
correlation degrades to per-client because it reaches models through LiteLLM,
like ADK) and the Anthropic SDK (depends on the proxy exposing an
Anthropic-native Messages API route, an open verification item).
Four adapters carry documented conformance exemptions for per-run correlation
and gateway-identity observation: ADK and CrewAI because LiteLLM owns their
transport, plus LlamaIndex and Microsoft Agent Framework because they receive
only static headers. The conformance suite pins those exemptions to each
adapter's actual transport behavior.

---

Expand Down
38 changes: 27 additions & 11 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -223,14 +223,14 @@ don't rely on GitHub auto-close, which can silently miss. Closing the issue is
what advances its milestone's completed count, which is how release readiness is
tracked. A `develop → main` promotion happens when a milestone reaches **0 open
issues**; the release PR's title carries the milestone and version (e.g.
`Release: M1 — Model access (0.1.0)`).
`Release: Phase 1 — Build the MVP (0.1.0)`).

---

## 2. Testing strategy

The repo has **five distinct test surfaces**, each with its own job and its own
CI gate. Getting the surface wrong either weakens a real gate (a framework test
The repo has **six distinct test surfaces**, each mapped to a specific CI or
local gate. Getting the surface wrong either weakens a real gate (a framework test
slipped into `tests/unit`) or produces a false negative (a skipped conformance
scenario nobody reviews). Pick by what the change exercises:

Expand All @@ -242,8 +242,9 @@ scenario nobody reviews). Pick by what the change exercises:
| A running local Omni Gateway (docker) | `@pytest.mark.local_gateway` (off by default) |
| A real Anypoint sandbox | `@pytest.mark.sandbox` (off by default, gated by `DONKEY_SANDBOX_TESTS=1`) |
| A framework's constructor signature/kwargs | `scripts/verify_frameworks.py` (not pytest) |
| A downstream package-root import/call combination that source-only analysis cannot exercise | **`tests/typecheck/`** — checked by `mypy`, not pytest |

If a change fits none of these, stop and ask — don't invent a sixth surface.
If a change fits none of these, stop and ask — don't invent a seventh surface.

### `tests/unit/` — the framework-free gate

Expand All @@ -257,6 +258,20 @@ provable: `PolicyViolation` stays distinct from the retryable
`remediation` (assert it directly). See [`ARCHITECTURE.md`](ARCHITECTURE.md#error-taxonomy-design-bg-12)
for why.

That job also builds and installs the wheel into an isolated environment, then
verifies the packaged simulator fixtures against their shipped integrity lock.
This is the packaging-path gate; source-checkout tests alone cannot prove those
resources landed in the wheel.

### `tests/typecheck/` — downstream static contracts

`mypy` strict-checks these small, non-pytest modules alongside `src/donkey_kit`.
Use this surface when the contract depends on how a consumer combines public
package-root imports and annotated calls — a composition source-only analysis
cannot exercise. Keep a matching runtime assertion under `tests/unit/` when the
contract also has runtime behavior; a typecheck fixture is not a substitute for
a runtime test.

### The conformance kit — "never a silent skip"

One suite (`python/tests/conformance/suite.py`) runs identically against every
Expand Down Expand Up @@ -345,13 +360,14 @@ python scripts/verify_frameworks.py [--live] [--only <fw>] [--emit-verified]
The full pre-write checklist lives in the `ddk-coding-conventions` skill and the
build plan; the load-bearing rules:

- **`mypy --strict`, blocking.** The whole `src/donkey_kit` tree is
strict-checked. Annotate every public signature; no untyped defs, no implicit
`Any`. Prefer `X | None` over `Optional[X]` (ruff `UP` rewrites the old form),
and put `from __future__ import annotations` at the top of every module (house
style). Don't silence a real signature mismatch with an unexplained
`# type: ignore` — the single `[[tool.mypy.overrides]]` block already handles
optional/absent framework deps.
- **`mypy --strict`, blocking.** The whole `src/donkey_kit` tree and the
downstream public-API contracts under `tests/typecheck/` are strict-checked.
Annotate every public signature; no untyped defs, no implicit `Any`. Prefer
`X | None` over `Optional[X]` (ruff `UP` rewrites the old form), and put
`from __future__ import annotations` at the top of every module (house style).
Don't silence a real signature mismatch with an unexplained `# type: ignore`
— the single `[[tool.mypy.overrides]]` block already handles optional/absent
framework deps.
- **Framework-free core & lazy imports.** `core/` depends on **httpx + pydantic
only** — no agent framework, ever. Adapters import their framework **lazily,
inside the method that uses it**, never at module top level; import the
Expand Down
10 changes: 6 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,10 @@

# Donkey Development Kit

An SDK for consuming **Agent Fabric** capabilities — governed model access,
governed tool access, and provisioning-as-code — from your own agent framework,
in your own IDE, without adopting Mule.
An SDK for consuming **Agent Fabric** capabilities — governed model and tool
access — from your own agent framework, in your own IDE, without adopting Mule.

> **Project status — alpha, pre-release.** This is `v0.1.0.dev2`
> **Project status — alpha, pre-release.** This is `v0.1.0.dev3`
> (`Development Status :: 3 - Alpha`). The **LLM data plane is live-verified**;
> most other surfaces are verification-gated (see
> [What's verified](#whats-verified-verification-discipline) below). **Not yet published to PyPI** —
Expand Down Expand Up @@ -125,3 +124,6 @@ legitimately cannot satisfy a scenario, the reason is asserted in code
| Framework | Scenario | Why it's exempt |
| --- | --- | --- |
| ADK, CrewAI | correlation ID propagated | LiteLLM owns the transport, so the SDK's `httpx` client cannot be injected — the correlation ID ends up per-client, not per-run. A LiteLLM logger callback may recover trace correlation later. |
| LlamaIndex, Microsoft Agent Framework | correlation ID propagated | These adapters receive a static `default_headers` snapshot, which deliberately excludes the per-run correlation ID. Without the SDK's `httpx` client, `donkey.run(id=...)` cannot update their request headers. |
| ADK, CrewAI | gateway identity observed | LiteLLM owns the transport, so no response reaches the SDK's `_on_response` hook. When every resolved adapter is non-observing, `donkey.last_call` reports `UNAVAILABLE` and names them in `surface`. |
| LlamaIndex, Microsoft Agent Framework | gateway identity observed | These adapters receive `default_headers`, not the SDK's `httpx` client, so no response reaches `_on_response`. When every resolved adapter is non-observing, `donkey.last_call` reports `UNAVAILABLE` and names them in `surface`. |
1 change: 0 additions & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ against a real Anypoint sandbox versus what is still assumed or blocked.
| File | What it is |
| --- | --- |
| [`verified-apis.md`](verified-apis.md) | **The verification ledger** — the single source of truth for every endpoint, header, and class name the SDK touches and its status (`VERIFIED (LIVE)` / `VERIFIED (CLI)` / `VERIFIED (plugin)` / `UNVERIFIED` / blocked). When a value is confirmed against a sandbox, flip its row here **and** set `verified=True` in `core/_verify.py` — the two edits move together. |
| [`m1-completion-checklist.md`](m1-completion-checklist.md) | **The 0.1.0 milestone tracker** (build plan Phase 1, "M1 — Model access"). A status-keyed checklist toward a truthful, publishable release; a box flips to ✅ only when the fact is confirmed against the installed framework / real sandbox, never just because the code that assumes it was written. |
| [`unsupported-boundary.md`](unsupported-boundary.md) | **The procurement doc** — classifies every platform API the SDK calls (public / no-SLA / undocumented), so enterprise-buyer questions get a five-minute answer instead of a two-week stall. Its "Undocumented surfaces" section is designed to stay empty. |
| [`releasing.md`](releasing.md) | **How a release reaches PyPI** — the Trusted Publishing (OIDC) workflow wired as `.github/workflows/release.yml` (#206), the public API surface semver governs, and the one-time human step to register the trusted publisher. The version scheme and tagging live in the `ddk-release` skill; this is the publish half. |

Expand Down
Loading
Loading