Add production safety foundations and policy coverage

.github/workflows/tests.yml

@@ -5,18 +5,26 @@ on:
     branches: [main, "feature/**"]
   pull_request:
 
+permissions:
+  contents: read
+
 jobs:
   test:
     runs-on: ubuntu-latest
     strategy:
+      fail-fast: false
       matrix:
         python-version: ["3.10", "3.11", "3.12", "3.13"]
     steps:
       - uses: actions/checkout@v4
       - uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
+          cache: pip
       - run: python -m pip install --upgrade pip
       - run: python -m pip install -e ".[dev]"
-      - run: pytest
+      - run: coverage run -m pytest
+      - run: coverage report
+      - run: ruff check .
+      - run: mypy opendecision
       - run: python -m compileall opendecision

CHANGELOG.md

@@ -0,0 +1,19 @@
+# Changelog
+
+## 0.2.0 - Unreleased
+
+### Added
+
+- Strict decision contracts and provider-output validation.
+- Deterministic rules provider and ordered provider fallback chains.
+- Fail-safe review and block modes.
+- Versioned, inheritable policies with risk and default-action metadata.
+- Privacy-preserving audit sinks and provider-attempt telemetry.
+- Human-review lifecycle with repeated-loop protection.
+- Accuracy, confusion, and Brier-score evaluation primitives.
+- Coverage, linting, strict typing, and expanded CI gates.
+
+### Changed
+
+- Laya choice contracts now translate `options` to Laya's `criteria` schema.
+- Package maturity remains pre-1.0; this release does not claim production safety guarantees.

README.md

@@ -1,132 +1,77 @@
 # OpenDecision
 
-**A framework-agnostic decision layer for AI agents.**
+An auditable, typed decision layer for AI agents.
 
-OpenDecision places a typed, testable contract between an AI agent and the action it wants to take. It uses [Laya](https://github.com/NandhaKishorM/laya) for fast `choice`, `score`, and `noul` decisions without adding another text-generation step.
+OpenDecision places explicit contracts, deterministic rules, provider fallback, human review, and privacy-preserving audit traces between an agent and consequential actions. It is a control-layer library, not a safety guarantee.
 
-> Status: early MVP. Do not treat uncalibrated model confidence as a production safety guarantee.
+## Production-foundation preview
 
-## Why OpenDecision?
+Version 0.2 adds:
 
-Agents repeatedly need to decide whether to continue, stop, call a tool, ask a person, retrieve more context, or escalate. OpenDecision makes those branches explicit and reusable:
+- strict decision-contract and provider-output validation;
+- deterministic rules plus ordered provider fallback chains;
+- fail-safe `raise`, `review`, and `block` behavior;
+- policy IDs, semantic versions, risk levels, defaults, and inheritance;
+- context hashes, decision IDs, provider attempts, and JSONL audit sinks;
+- an in-memory human-review workflow with loop protection;
+- reproducible accuracy, confusion, and Brier-score evaluation primitives;
+- coverage, lint, strict typing, and Python 3.10–3.13 CI gates.
 
-- typed decision contracts instead of free-form generated text;
-- normalized results across decision providers;
-- community-maintained YAML decision packs;
-- adapters that do not couple the core SDK to one agent framework;
-- optional Laya loading, so importing OpenDecision never downloads a model.
+## Example
 
-## Install
-
-```bash
-pip install -e ".[laya]"
-```
-
-Laya and OpenDecision require Python 3.10 or newer. For the examples:
-
-```bash
-pip install -e ".[laya,langgraph,fastapi]"
-```
+```python
+from opendecision import (
+    DecisionContext,
+    DecisionGuard,
+    JsonlAuditSink,
+    ProviderChain,
+    Rule,
+    RuleProvider,
+)
 
-## Quick start
+rules = RuleProvider(
+    [Rule(field="command", operator="contains", value="rm -rf", decision="block")],
+)
+providers = ProviderChain([rules], terminal_answer={"choice": "review"})
 
-```python
-from opendecision import DecisionGuard
+guard = DecisionGuard(
+    providers,
+    audit_sink=JsonlAuditSink("audit/decisions.jsonl"),
+    failure_mode="review",
+)
 
-guard = DecisionGuard()
 result = guard.decide(
-    state={"tool": "send_email", "recipient": "customer@example.com"},
-    question={
+    {"command": "rm -rf /tmp/cache"},
+    {
         "type": "choice",
-        "instructions": "Should the agent execute this tool action?",
+        "instructions": "Should this command execute?",
         "options": {
-            "allow": "Proceed automatically",
-            "review": "Require human approval",
-            "block": "Stop the action",
+            "allow": "Authorized and low risk",
+            "review": "Needs human approval",
+            "block": "Unauthorized or destructive",
         },
     },
+    context=DecisionContext(
+        actor="agent:ops",
+        tool="shell",
+        action_id="run-123",
+        policy_id="shell-command-risk",
+        policy_version="1.0.0",
+        risk="critical",
+    ),
 )
-
-print(result.decision)
-print(result.probabilities)
-print(result.confidence)
-```
-
-OpenDecision returns model decisions and evidence; it does **not** ask Laya to generate a reason. Applications may add an explanation separately without confusing generated prose with decision-model output.
-
-## Decision packs
-
-```python
-from opendecision import DecisionGuard, load_policy
-
-policy = load_policy("decision_packs/security/tool_risk.yaml")
-result = DecisionGuard().decide(agent_state, policy.question)
-```
-
-The first pack includes a tool-risk contract with `allow`, `review`, and `block` outcomes.
-
-## LangGraph
-
-```python
-from opendecision import DecisionGuard
-from opendecision.integrations.langgraph import decision_node, route_by_decision
-
-graph.add_node("tool_guard", decision_node(DecisionGuard(), question))
-graph.add_conditional_edges(
-    "tool_guard",
-    route_by_decision(),
-    {"allow": "execute_tool", "review": "human_review", "block": "stop"},
-)
-```
-
-## FastAPI
-
-```bash
-uvicorn app.main:app --reload
 ```
 
-Then send `POST /v1/decide` with `state` and a typed `question`.
-
-## Architecture
-
-```text
-Agent / application
-        |
-DecisionGuard + contract
-        |
-Provider adapter (Laya first)
-        |
-Normalized DecisionResult
-        |
-ALLOW | REVIEW | BLOCK
-```
-
-## Scope of v0.1
-
-- Python SDK and Pydantic contracts
-- lazy Laya provider
-- choice, score, and noul normalization
-- YAML decision packs
-- LangGraph node and routing helpers
-- FastAPI example
-- unit tests that run without downloading model weights
-
-A dashboard, TypeScript SDK, additional frameworks, model-generated explanations, and evaluation tooling are intentionally deferred.
-
-## Calibration and safety
-
-Laya's documentation notes meaningful limitations: base checkpoints can be weak zero-shot on typed workflows, confidence may require domain calibration, and high-cardinality choices need special handling. Benchmark decision packs on representative data before automating consequential actions. Prefer human review when evidence or authorization is insufficient.
+Raw state is hashed for correlation and is not written by the built-in audit sinks. Applications remain responsible for authentication, authorization, durable review storage, encryption, retention, monitoring, calibration, and incident response.
 
 ## Development
 
 ```bash
 pip install -e ".[dev]"
-pytest
+coverage run -m pytest
+coverage report
 ruff check .
+mypy opendecision
 ```
 
-See [CONTRIBUTING.md](CONTRIBUTING.md) for contribution guidelines.
-
-## License
-
-Apache License 2.0. Laya is a separate Apache-2.0 project and remains subject to its own license and notices.
+See `docs/production-readiness.md` for operating guidance.

decision_packs/data/data_access.yaml

@@ -0,0 +1,14 @@
+id: data-access
+version: 1.0.0
+name: Data access gate
+description: Classify agent reads and writes by authorization and sensitivity.
+risk: high
+default_action: review
+tags: [data, authorization, privacy]
+question:
+  type: choice
+  instructions: Should the agent perform the requested data operation?
+  options:
+    allow: The actor is authorized, scope is minimal, and the operation is low risk.
+    review: Authorization, necessity, or data sensitivity needs human verification.
+    block: The operation is unauthorized, excessive, or exposes restricted data.

decision_packs/finance/payment_action.yaml

@@ -0,0 +1,14 @@
+id: payment-action
+version: 1.0.0
+name: Payment action gate
+description: Route refunds, transfers, and payment changes through appropriate controls.
+risk: critical
+default_action: block
+tags: [finance, payments, approval]
+question:
+  type: choice
+  instructions: Should this payment-related action proceed?
+  options:
+    allow: The action is within an approved low-risk policy and fully authorized.
+    review: A verified finance reviewer must approve the amount, recipient, or exception.
+    block: The action is unauthorized, anomalous, irreversible, or outside policy.

decision_packs/hr/permission_change.yaml

@@ -0,0 +1,14 @@
+id: permission-change
+version: 1.0.0
+name: HR and permission change gate
+description: Guard role, access, employment, and entitlement changes.
+risk: critical
+default_action: block
+tags: [hr, iam, permissions]
+question:
+  type: choice
+  instructions: Should this identity or employment-related change execute?
+  options:
+    allow: The change is pre-approved, least-privilege, and supported by authoritative records.
+    review: An authorized HR or security reviewer must validate the change.
+    block: The change is unsupported, over-privileged, discriminatory, or unauthorized.

decision_packs/integrations/external_api.yaml

@@ -0,0 +1,14 @@
+id: external-api-call
+version: 1.0.0
+name: External API call gate
+description: Guard outbound requests, side effects, and data sharing with third parties.
+risk: high
+default_action: review
+tags: [api, integrations, egress]
+question:
+  type: choice
+  instructions: Should the agent make this external API call?
+  options:
+    allow: The destination, scope, payload, and side effect are approved and expected.
+    review: The destination, data classification, cost, or side effect needs approval.
+    block: The call targets an untrusted service, leaks restricted data, or exceeds authorization.

decision_packs/security/destructive_command.yaml

@@ -0,0 +1,14 @@
+id: destructive-command
+version: 1.0.0
+name: Destructive command gate
+description: Guard shell, infrastructure, database, and filesystem actions.
+risk: critical
+default_action: block
+tags: [security, shell, infrastructure]
+question:
+  type: choice
+  instructions: Should this potentially destructive command execute?
+  options:
+    allow: The command is scoped, reversible, authorized, and validated in the target environment.
+    review: A qualified operator must confirm scope, backup, target, and rollback readiness.
+    block: The command is destructive, ambiguous, unbounded, unauthorized, or lacks recovery controls.

decision_packs/security/privacy_exposure.yaml

@@ -0,0 +1,14 @@
+id: privacy-exposure
+version: 1.0.0
+name: Privacy exposure gate
+description: Guard output that may reveal personal, secret, or regulated information.
+risk: critical
+default_action: block
+tags: [privacy, pii, secrets]
+question:
+  type: choice
+  instructions: Should the agent disclose or transmit this information?
+  options:
+    allow: The disclosure is authorized, necessary, minimized, and sent to an approved destination.
+    review: Data classification, consent, recipient, or retention requires verification.
+    block: The content contains restricted personal data, credentials, secrets, or unauthorized records.

docs/production-readiness.md

@@ -0,0 +1,48 @@
+# Production readiness
+
+OpenDecision is a decision-control primitive, not a complete security boundary. Deploy it behind application authentication and authorization, and preserve independent enforcement at the protected tool or service.
+
+## Recommended decision flow
+
+1. Validate state and a versioned decision contract.
+2. Apply deterministic deny or allow rules where policy is unambiguous.
+3. Use a calibrated model provider only for the residual decision space.
+4. Route uncertainty and consequential actions to human review.
+5. Enforce the final decision at the tool boundary.
+6. Record a redacted audit event and monitor overrides and failure rates.
+
+## Failure policy
+
+- `raise`: fail the request and let the caller handle the outage.
+- `review`: produce a human-review outcome when the contract contains one.
+- `block`: deny automatically when provider output is unavailable or invalid.
+
+Critical actions should generally use `block` or a durable review queue. The built-in in-memory review store is for prototypes and tests; production needs durable storage, authentication, RBAC, idempotency, notifications, and retention controls.
+
+## Calibration
+
+Tune thresholds on held-out, representative data for each policy and checkpoint. Report sample size, class balance, false positives, false negatives, Brier score, ECE, model version, hardware, and preprocessing. Never transfer thresholds between domains without evaluation.
+
+## Audit and privacy
+
+Built-in audit sinks store a SHA-256 context hash rather than raw state. Operators must define data classification, encryption, retention, access controls, deletion, and incident response. Do not place secrets or personal data into metadata fields.
+
+## Risk matrix
+
+| Risk | Default | Human review | Example |
+| --- | --- | --- | --- |
+| Low | allow after deterministic checks | optional | read public documentation |
+| Medium | model or rules | on uncertainty | send internal notification |
+| High | review | required by default | send external email or refund |
+| Critical | block | explicit authorized approval | destructive command or production deletion |
+
+## Deployment checklist
+
+- Pin and scan dependencies.
+- Protect tool credentials independently of the agent.
+- Use versioned policies and immutable release artifacts.
+- Run adversarial and regression fixtures.
+- Configure fallback and outage behavior.
+- Use a durable review service with RBAC.
+- Monitor provider errors, overrides, calibration drift, and latency.
+- Test rollback and emergency disable paths.