Skip to content

Add observed-effect references integration - #246

Open
astrogilda wants to merge 3 commits into
agentrust-io:mainfrom
astrogilda-forks:probityai-observed-effect
Open

astrogilda wants to merge 3 commits into
agentrust-io:mainfrom
astrogilda-forks:probityai-observed-effect

Conversation

@astrogilda

Copy link
Copy Markdown

This adds integrations/probityai-observed-effect, the integration @imran-siddique asked about on agentrust-io/awesome-ai-governance#108.

Five signed TRACE Trust Records each carry a references entry with rel observed-effect, pointing at an in-toto statement that an observer outside the agent signed. I copied the two observer statements those records cite byte for byte from vectors-observed-effect at v0.15.0. The tests verify every record with released agentrust-trace 0.11.0. They also recompute each relying-party verdict from the committed bytes: verified, digest mismatch, unresolved, and observer key not configured.

To reproduce for the Verified tier:

pip install -e "integrations/probityai-observed-effect[test]"
python -m pytest integrations/probityai-observed-effect/tests
pip install agent-evidence-vectors==0.15.0
agent-evidence-vectors --corpus vectors-observed-effect

trace-spec#403 proposes the observed-effect value and the registry does not list it yet, so I claim no conformance level.

Five signed TRACE Trust Records cite observer-signed in-toto statements through a references entry with rel observed-effect. The tests verify each record with released agentrust-trace and recompute the relying-party verdict from the committed bytes; CI also replays the published vectors-observed-effect corpus at v0.15.0.

Signed-off-by: Sankalp Gilda <sankalp.gilda@gmail.com>
@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

🔴 Contributor Check: HIGH

Check Result
Profile HIGH
Credential LOW
Overall HIGH

Automated check by AgenTrust Contributor Check.

@github-actions github-actions Bot added the needs-review:HIGH Contributor check flagged HIGH risk label Oct 1, 2026
@astrogilda

Copy link
Copy Markdown
Author

@imran-siddique -- trace-spec#403 is merged and its merge checks are green. This integration makes the accepted examples runnable against the pinned Probity corpus. The remaining blocker is corpus installation on Python 3.11/3.12. I'd like this to become a recurring CI check for the observed-effect relation once that is resolved.

@imran-siddique

Copy link
Copy Markdown
Member

@astrogilda the corpus step fails on 3.11 and 3.12 because every agent-evidence-vectors release from 0.11.1 on declares Requires-Python >=3.13; the integration's own tests pass on both. Gate only the "Replay the published observed-effect corpus" step on matrix.python >= 3.13, say in integration.yaml and the README that the replay runs on 3.13 and later, and keep 3.11 and 3.12 running the integration tests. If the corpus does not actually need 3.13, lowering its floor in a release is the other route. Once the matrix is green this merges as community tier; Verified is a separate review afterwards.

Signed-off-by: Sankalp Gilda <sankalp.gilda@gmail.com>
@imran-siddique

Copy link
Copy Markdown
Member

@astrogilda running the replay under its own 3.13 on every leg is better than gating it: 3.11 and 3.12 now get the replay too, and all eight jobs pass. One mechanical step left: validate reports marketplace/catalog.json is stale, because main gained integrations since you generated it. Merge main and rerun scripts/generate_marketplace_catalog.py; once validate is green this merges as community tier.

@astrogilda

Copy link
Copy Markdown
Author

I merged current main and regenerated marketplace/catalog.json at fff10d6. GitHub is now holding the refreshed workflows for maintainer approval. Could you approve them? Once Validate integrations is green, this satisfies the final condition you set for the community-tier merge.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-review:HIGH Contributor check flagged HIGH risk

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants