From 586eccb56bcd62a371121a4e63d99c43a31fb115 Mon Sep 17 00:00:00 2001 From: ecrum19 Date: Fri, 25 Sep 2026 13:05:42 +0200 Subject: [PATCH 1/4] Design the v0.1.0 policy demonstrator A first, small slice of privacy-policy-design.md: ODRL policies attached to files, regions and variants of synthetic single-sample VCFs, a release view per request, and a check against a VCF-side oracle. Governed release, not anonymization. Later versions are mapped onto the full design's build order. Co-Authored-By: Claude Opus 5.5 --- docs/README.md | 4 + docs/policy-demonstrator.md | 522 ++++++++++++++++++++++++++++++++++ docs/privacy-policy-design.md | 6 + docs/roadmap.md | 5 +- 4 files changed, 536 insertions(+), 1 deletion(-) create mode 100644 docs/policy-demonstrator.md diff --git a/docs/README.md b/docs/README.md index 8d6e44d..c41b411 100644 --- a/docs/README.md +++ b/docs/README.md @@ -54,6 +54,10 @@ deciding whether the tool fits your problem, read graph: an ODRL profile with graph selectors, three enforcement tiers, and verification — plus a candid account of why access control is not anonymization when the genotypes are themselves identifiers. +- **[Policy demonstrator v0.1.0](policy-demonstrator.md)** — *design, not yet + implemented.* The first slice of the privacy design: ODRL policies attached + to files, regions and variants of single-sample VCFs, per-request release + views, and an oracle that checks each view withholds exactly what it should. ### What the graph looks like diff --git a/docs/policy-demonstrator.md b/docs/policy-demonstrator.md new file mode 100644 index 0000000..2e5e547 --- /dev/null +++ b/docs/policy-demonstrator.md @@ -0,0 +1,522 @@ +# Policy attachment demonstrator — v0.1.0 + +*Part of the [VCF-RDFizer documentation](README.md). Status: **design, not yet +implemented.** This is the first, deliberately small slice of +[`privacy-policy-design.md`](privacy-policy-design.md): it uses that document's +vocabulary and rules, implements a subset of them, and states plainly what it +leaves out.* + +v0.1.0 shows three things on a small cohort of synthetic single-sample VCFs: + +1. **Attach.** ODRL policies are attached to files, genomic regions and + individual variants, and they sit in the same graph as the data, so a + single SPARQL query can ask about both. +2. **Evaluate.** One set of policies gives each requester a different release + view, according to who they are and the purpose they state. +3. **Verify.** Each view is checked against an independent oracle computed + from the source VCFs. It must withhold exactly what the policy says: no + more and no less. + +The rest of this document specifies the fixture, the subset of the profile, +the decision semantics, the command, the tests and what goes into the paper, +and it ends with the plan for later versions. + +It is **governed release, not anonymization**, and every string the tool emits +must say so. See [`privacy-policy-design.md` §1](privacy-policy-design.md#1-the-uncomfortable-premise). + +--- + +## 1. Why this is worth demonstrating + +Today, consent for genomic data is attached at the level of a whole dataset. A +repository record carries one data-use code, and VCF itself has no place to +state a policy at all. Real policies are finer than that: + +- *this participant* consented to health research but not to clinical use; +- *this region* holds actionable incidental findings and may only be read for + clinical care; +- *this variant* is sensitive enough to be restricted to disease-specific + research; +- *this participant* withdrew, and that overrides everything else. + +An RDF graph can hold every one of those statements next to the data it +governs, in standard vocabularies (ODRL for the rules, GA4GH DUO for the +purposes, PROV for what was released), and it can answer questions that mix +the two. A VCF-based workflow would need an external spreadsheet, and its +enforcement would live in someone's scripts. The demonstrator makes that +difference concrete, and it measures it rather than just asserting it. + +--- + +## 2. The fixture: a cohort of single-sample files + +Five synthetic participants, **one small VCF each**: + +| File | Participant | Consent (file-level policy) | +| --- | --- | --- | +| `P001.vcf` | P001 | General research use **and** clinical care use | +| `P002.vcf` | P002 | General research use **and** clinical care use | +| `P003.vcf` | P003 | Health/medical/biomedical research only | +| `P004.vcf` | P004 | **Withdrawn**: prohibited for every requester | +| `P005.vcf` | P005 | Disease-specific research only | + +Each file is `VCFv4.3` with `##reference=GRCh38`, about 40 records, and one +sample column. The records sit at real GRCh38 positions in three groups: + +| Group | Where | Why | +| --- | --- | --- | +| *BRCA1* | chr17:43,044,295–43,125,483 | Actionable incidental findings; the region rule applies here | +| *APOE* | chr19, including rs429358 (44,908,684 T>C) and rs7412 (44,908,822 C>T) | rs429358 defines the ε4 allele; the variant rule applies to it alone, and rs7412 is its unrestricted neighbour | +| Background | a few loci on chr1 and chr20 | Governed only by the file-level consent | + +**The genotypes are synthetic and generated from a fixed seed.** The design +doc's first premise is that genotypes identify people, and a privacy +demonstrator should not be built on real individuals. Positions and alleles +are real, so the region and variant selectors are tested against real +coordinates. Everything else is invented. + +The generator is `examples/policy/make_fixture.py`. It is deterministic, and +it writes the five VCFs together with a `fixture.json` that records the seed, +the loci and each file's intended consent. + +**Why single-sample files help:** + +- **Consent is per participant, so it becomes a file-level policy.** The + policy travels with the participant's file, which is how consent actually + works. +- **Withdrawing a participant means withholding one file.** No per-sample + surgery inside a shared graph is needed. +- **Both sample profiles work.** In the condensed profile, a record's sample + vector has exactly one entry, so withholding a record removes whole + literals and never has to rewrite one. The masking problem in + [`privacy-policy-design.md` §8](privacy-policy-design.md#8-the-condensed-representation-problem) + does not arise. Multi-sample files, where it does, are v0.2 (§10). +- **The cohort is a union of graphs.** Each file's IRIs start with + `file://P00n.vcf`, so the five graphs merge without collisions (see + [conversion §6](conversion.md#6-iri-templates)). Region and variant rules + are then cohort-wide rules that cut across files. + +--- + +## 3. The profile subset + +Namespace `vcfp:` = `https://w3id.org/vcf-rdfizer/policy#`, as in the full +design. v0.1.0 implements exactly this: + +| Construct | v0.1.0 | Notes | +| --- | --- | --- | +| `odrl:Set` / `odrl:Policy` | yes | One policy per participant, plus one cohort policy | +| `odrl:permission`, `odrl:prohibition` | yes | | +| `odrl:target` = a file IRI | yes | The file resource *is* the asset; no selector is needed | +| `odrl:target` = `vcfp:GraphSelection` with `vcfp:RegionSelector` | yes | `vcfp:assembly` is required and checked against `vcfc:referenceGenome` | +| `odrl:target` = `vcfp:GraphSelection` with `vcfp:VariantSelector` | yes, **new** | chrom, pos, ref, alt and assembly. The full design has no mutation-level selector; this adds one | +| `odrl:assignee` | yes | An IRI, or `odrl:All` | +| `odrl:constraint` on `odrl:purpose` with `odrl:isAnyOf` / `odrl:isNoneOf` | yes | Right-hand sides are DUO terms, matched through the DUO hierarchy (§3.1) | +| `odrl:action odrl:read` | yes | The only action evaluated | +| `odrl:duty` (`odrl:attribute`, `odrl:inform`) | recorded | Copied into the release manifest as obligations the requester accepts. **Not enforced** | +| `odrl:conflict odrl:prohibit` | yes, required | Deny wins; any other value is rejected | +| `vcfp:transform vcfp:drop` | yes | The only effect | +| `vcfp:SampleSelector`, `FieldSelector`, `HeaderSelector`, `ClassSelector`, `PredicateSelector`, `PatternSelector` | **no** | v0.2 and later (§10). A policy that uses one is rejected, not ignored | +| `generalize`, `pseudonymize`, `threshold`, `aggregateOnly`, `maskVectorPositions` | **no** | Later versions. Rejected if present | + +**Anything v0.1.0 cannot evaluate stops the run with an error.** That applies +to unknown selectors, unsupported effects, missing assemblies, and assemblies +that don't match the data. This is the design doc's "the residual set must +abort" rule (§6), and it matters most in the smallest version: a demonstrator +that silently skipped a rule would demonstrate the opposite of its purpose. + +### 3.1 Purposes and the DUO hierarchy + +Purposes are GA4GH Data Use Ontology terms: + +| Code | IRI | Role in the fixture | +| --- | --- | --- | +| GRU, general research use | `obo:DUO_0000042` | P001, P002 consent; the general-research requester | +| HMB, health/medical/biomedical research | `obo:DUO_0000006` | P003 consent | +| DS, disease-specific research | `obo:DUO_0000007` | P005 consent; the Alzheimer's consortium's purpose | +| CC, clinical care use | `obo:DUO_0000043` | P001, P002 consent; the clinical requester | + +Matching uses subsumption. **A requester's purpose satisfies a consent when +the purpose is the consented term or a narrower one**, so a disease-specific +study falls within a general-research consent, but not the other way round. +v0.1.0 bundles the four terms and their `rdfs:subClassOf` links as +`vcf_rdfizer_data/policy/duo-subset.ttl`, with the DUO release they were taken +from recorded in the file. Full DUO, with disease qualifiers via MONDO and +release pinning in the policy (`vcfp:duoVersion`), is later (§10). + +--- + +## 4. Decision semantics (normative for v0.1.0) + +A **request** is `(assignee a, purpose p)`. The unit of decision is the +**record**. File-level resources (the header, the file, the sample set) are +decided with their file. + +A rule *applies* to a request when its assignee is `odrl:All` or equals `a`, +and every constraint holds for `p`: + +- `purpose isAnyOf S` holds when `p ⊑ s` for some `s` in `S`; +- `purpose isNoneOf S` holds when `p ⊑ s` for no `s` in `S`. + +A rule *covers* a record when its target selects the record: the record's +file, a region containing the record's `vcfc:pos` on its `vcfc:chrom`, or a +variant matching its chrom, pos, ref and alt. + +**Record `r` in file `f` is released to request `q` if and only if:** + +1. some **permission** that applies to `q` covers `f` — the default-deny + rule: no permission, no release; **and** +2. no **prohibition** that applies to `q` covers `r` or `f` — deny wins. + +A file's own resources are released when (1) and (2) hold for the file +itself. A released triple must have both its subject and any IRI object +released, so no triple in a view points at something the view withholds. + +The fixture's cohort policy has two rules, and both are prohibitions with an +exception. That keeps the semantics above complete without needing +"most-specific target wins", which ODRL doesn't define: + +```turtle +@prefix odrl: . +@prefix vcfp: . +@prefix obo: . + +<#brca1> a odrl:Asset , vcfp:GraphSelection ; + vcfp:selector [ a vcfp:RegionSelector ; vcfp:assembly "GRCh38" ; + vcfp:chrom "chr17" ; vcfp:start 43044295 ; vcfp:end 43125483 ] . + +<#apoe-e4> a odrl:Asset , vcfp:GraphSelection ; + vcfp:selector [ a vcfp:VariantSelector ; vcfp:assembly "GRCh38" ; + vcfp:chrom "chr19" ; vcfp:pos 44908684 ; vcfp:ref "T" ; vcfp:alt "C" ] . + +<#cohort-policy> a odrl:Set ; + odrl:uid ; + odrl:profile ; + odrl:conflict odrl:prohibit ; + # Incidental findings: readable only for clinical care. + odrl:prohibition [ odrl:target <#brca1> ; odrl:action odrl:read ; odrl:assignee odrl:All ; + odrl:constraint [ odrl:leftOperand odrl:purpose ; odrl:operator odrl:isNoneOf ; + odrl:rightOperand obo:DUO_0000043 ] ] ; + # APOE e4: readable only for disease-specific research. + odrl:prohibition [ odrl:target <#apoe-e4> ; odrl:action odrl:read ; odrl:assignee odrl:All ; + odrl:constraint [ odrl:leftOperand odrl:purpose ; odrl:operator odrl:isNoneOf ; + odrl:rightOperand obo:DUO_0000007 ] ] . + +# A participant's consent, attached to their file. +<#consent-P003> a odrl:Set ; + odrl:uid ; + odrl:profile ; + odrl:conflict odrl:prohibit ; + odrl:permission [ odrl:target ; odrl:action odrl:read ; + odrl:assignee odrl:All ; + odrl:constraint [ odrl:leftOperand odrl:purpose ; odrl:operator odrl:isAnyOf ; + odrl:rightOperand obo:DUO_0000006 ] ; + odrl:duty [ odrl:action odrl:attribute ] ] . +``` + +### 4.1 What the three requesters get + +| Requester | Purpose | P001 | P002 | P003 | P004 | P005 | *BRCA1* | rs429358 | +| --- | --- | --- | --- | --- | --- | --- | --- | --- | +| General-research consortium | GRU | ✓ | ✓ | — | — | — | withheld | withheld | +| Alzheimer's consortium | DS | ✓ | ✓ | ✓ | — | ✓ | withheld | **released** | +| Clinical genetics lab | CC | ✓ | ✓ | — | — | — | **released** | withheld | + +Every cell follows from §4 alone: + +- P003 is out for general research, because GRU is broader than P003's + health-research consent. +- P003 and P005 are in for the Alzheimer's consortium, because DS falls + within both HMB and DS. +- P004 is out everywhere: the withdrawal is a prohibition, and prohibitions + win. +- *BRCA1* and rs429358 are withheld unless the purpose is the one + exception each rule allows. + +This grid is the demonstrator's headline result, and §7 makes it the oracle. + +--- + +## 5. What `attach` adds to the graph + +`attach` merges the policy graph into the data. It then materializes what each +selection selects, so policies can be queried alongside data without +re-implementing the selectors in SPARQL: + +```turtle + odrl:hasPolicy <#consent-P003> . +<#brca1> vcfp:selects , … . + odrl:hasPolicy <#cohort-policy> . +``` + +It also writes an attachment manifest: the policy's digest, and the selection +counts per asset. + +The query that makes the argument: *which records could I not share with a +general-research collaborator, and why?* + +```sparql +PREFIX odrl: +PREFIX vcfc: +PREFIX obo: +SELECT ?record ?chrom ?pos ?rule WHERE { + ?record a vcfc:VCFRecord ; vcfc:chrom ?chrom ; vcfc:pos ?pos ; + odrl:hasPolicy ?policy . + ?policy odrl:prohibition ?rule . + ?rule odrl:constraint [ odrl:operator odrl:isNoneOf ; odrl:rightOperand ?exempt ] . + FILTER(?exempt != obo:DUO_0000042) +} +``` + +That query runs over the annotated graph unchanged, under any of the four +engines the validation stage already uses. No VCF tool can answer it. + +--- + +## 6. What `evaluate` writes + +For one request it writes a directory: + +| File | Contents | +| --- | --- | +| `view.nt` | The release view: the released triples, sorted | +| `manifest.ttl` | A `vcfp:ReleaseView` (§6.1) | +| `decisions.csv` | One row per record: file, chrom, pos, ref, alt, released or withheld, and the rule that decided it | +| `summary.json` | Counts per file and per rule, for the paper figure | + +### 6.1 The manifest + +This follows [`privacy-policy-design.md` §10](privacy-policy-design.md#10-the-release-manifest), +with the fields v0.1.0 can fill: + +```turtle +<#release> a vcfp:ReleaseView ; + vcfp:derivedFrom , , , , ; + vcfp:policy ; + vcfp:policyDigest "sha256:…" ; + vcfp:request [ odrl:assignee ; + odrl:purpose obo:DUO_0000007 ] ; + vcfp:recordsReleased 142 ; vcfp:recordsWithheld 58 ; + vcfp:filesWithheld 1 ; vcfp:triplesWithheld 9412 ; + vcfp:obligation [ odrl:action odrl:attribute ] ; + vcfp:disclosureModel "governed release; not anonymization" ; + prov:wasGeneratedBy [ prov:used ] ; + prov:generatedAtTime "…"^^xsd:dateTime . +``` + +The numbers in the example are placeholders. The policy is referenced by +digest as well as by IRI, and the view keeps the original IRIs, since +pseudonymization is v0.4 (§10). The manifest says so in +`vcfp:disclosureModel`. + +### 6.2 Implementation + +The evaluator loads the union graph with `rdflib`, which is already a runtime +dependency, so no image change is needed. It then does three things: + +- **Resolves selectors to records** with one SPARQL query per selector, over + `vcfc:chrom`, `vcfc:pos`, `vcfc:ref` and `vcfc:alt`, with the assembly + checked against each file's `vcfc:referenceGenome`. +- **Decides each record** by §4. +- **Expands each withheld record to its subtree by IRI prefix**, using the + hierarchy in [conversion §6](conversion.md#6-iri-templates): + - `#record/{ROW}` and everything under it (alleles, events); + - `#call/{ROW}` and everything under it (INFO values; the condensed matrix + and vectors); + - `#sample/{ROW}/` and everything under it (expanded sample calls, + genotypes and FORMAT values). + + A withheld file removes everything under `file://{FILE}`. A final pass + drops any triple whose IRI object falls in a withheld subtree. + +This is in memory by design. A size guard refuses graphs above 5M triples, +with a message that names the demonstrator's scope. The streaming version is +v0.3 (§10). + +--- + +## 7. How `check` verifies a view + +The checks follow the validation methodology: a claim is tested against an +independent computation, not taken on trust. + +1. **Oracle agreement.** `check` reparses the source VCFs directly, without + the graph or any SPARQL, and computes which records each request should + see from `fixture.json` and §4. The view's released records must equal + that set **exactly**: + - an extra record is a leak; + - a missing record is over-withholding. + + Both are failures, because a redactor that withheld everything would + otherwise pass. +2. **Prohibition ASKs.** Each prohibition that applies is compiled into a + SPARQL `ASK` that must be false on the view: for example, any record in + the *BRCA1* window, or any subject under `file://P004.vcf`. +3. **No dangling references.** No IRI object in the view points into a + withheld subtree. +4. **Header and IRI leakage.** No IRI or header literal in the view names a + withheld participant. In v0.1.0 that can only occur through a + whole-file withdrawal, which the prefix rule removes, so the check + guards against regressions in that rule. + +**Mutation tests** (`test/test_policy_unit.py`) break a correct view on +purpose and assert that `check` catches each break: + +- reinsert one withheld *BRCA1* triple; +- restore one triple of P004; +- delete one record that should have been released; +- leave one dangling allele reference; +- swap two requesters' views. + +A check that has never caught a planted leak is not evidence of anything. + +--- + +## 8. The command + +`vcf-rdfizer-policy`, a separate console script in the pattern of +`vcf-rdfizer-link`. It runs on the host and needs no Docker. + +```bash +vcf-rdfizer-policy attach --rdf P00*.nt.gz --policy policy.ttl -o cohort-annotated.nt +vcf-rdfizer-policy evaluate --rdf P00*.nt.gz --policy policy.ttl \ + --assignee https://example.org/party/alz-consortium \ + --purpose obo:DUO_0000007 -o views/alz +vcf-rdfizer-policy check --view views/alz --policy policy.ttl --vcf P00*.vcf +vcf-rdfizer-policy explain --policy policy.ttl # the §4.1 grid, from the policy alone +``` + +| Exit code | Meaning | +| --- | --- | +| 0 | Success; for `check`, every check passed | +| 1 | `check` found a leak, over-withholding or a dangling reference | +| 2 | The policy uses something v0.1.0 cannot evaluate, the assembly doesn't match, or the graph is over the size guard | + +**Layout:** + +```text +vcf_rdfizer_policy_cli.py console entry point (pyproject: vcf-rdfizer-policy) +vcf_rdfizer_policy/ + __init__.py version 0.1.0 + profile.py parse + validate a policy against the v0.1.0 subset + purposes.py DUO subsumption over the bundled subset + selectors.py file, region and variant selectors -> record IRIs + decide.py the §4 semantics; the one place they live + attach.py evaluate.py check.py manifest.py +vcf_rdfizer_data/policy/ + vcfp-0.1.ttl the profile terms v0.1.0 defines + duo-subset.ttl four DUO terms and their hierarchy, release recorded +examples/policy/ + make_fixture.py fixture.json P001.vcf … P005.vcf + policy.ttl requests/{gru,alz,clinical}.ttl + converted/ the five converted graphs, committed so the demo runs without Docker + run_demo.sh README.md +test/test_policy_unit.py runs in CI (matches test_*_unit.py) +``` + +`decide.py` holds the semantics, and **both `evaluate` and `check`'s oracle +call it**, so the §4 rules exist in exactly one place. The oracle's +independence comes from its input, the VCF text rather than the graph, not +from a second implementation of the rules. + +--- + +## 9. What goes into the paper + +The capabilities table's *Provenance/policy annotations* row currently reads +"Not implemented / Not evaluated". It becomes: + +> **Demonstrator (v0.1.0)** · **Exercised on a fixture** · ODRL policies +> attached to files, regions and variants; per-request release views +> verified against a VCF-side oracle; not anonymization, and not enforced at +> conversion time. + +Plus one figure and one short passage (Discussion, or a Results subsection +next to linking): + +**Figure: policies attached at three granularities, and what they do.** + +- **(a)** Where the policies attach. A schematic, in the style of Figure 2: a + participant's consent on `VCFFile`, the *BRCA1* rule on a region selection, + the ε4 rule on one variant, and each linked to its policy by + `odrl:hasPolicy`. +- **(b)** What each requester receives. The §4.1 grid, with participants and + the two cohort rules as rows and the three requesters as columns, cells + marked released or withheld, and triples withheld per requester along the + bottom. The data come from `summary.json`, drawn by `make_figures.py`. +- **(c)** A policy and the question it enables. The 10-line ODRL rule for + rs429358 beside the §5 query. This can be a listing instead, if the figure + gets crowded. + +**The passage states the advantages, each with what supports it:** + +- Granularity finer than a dataset-level consent code: file, region and + variant, each governed separately (panel b). +- Policies that are queryable alongside the data they govern (the §5 query). +- Standards rather than bespoke configuration: ODRL, DUO and PROV. +- Releases that are auditable: a digest-pinned manifest per view. +- A claim that is measured: every view matches an independent VCF-side + oracle, and the mutation tests show the check catches planted leaks. + +**And it states the limits,** in the same paragraph: + +- the fixture is synthetic, and small; +- there is no pseudonymization, so a view keeps its original IRIs; +- enforcement happens after conversion, not during it; +- duties are recorded but not enforced; +- it is governed release, not anonymization. + +--- + +## 10. Beyond v0.1.0 + +Each version maps onto the full design's build order +([`privacy-policy-design.md` §13](privacy-policy-design.md#13-build-order)). + +| Version | Adds | Design § | +| --- | --- | --- | +| **v0.2** | Multi-sample files: `vcfp:SampleSelector` (expanded first), then `vcfp:maskVectorPositions` for condensed, with the single-position-mask warning; `FieldSelector` and `HeaderSelector`; the `generalize` effect (genotype → carrier status, which suits *APOE*) | §4.1, §4.2, §8 | +| **v0.3** | Enforcement during conversion: TSV-level (Tier 1) and emitter-level (Tier 2), including `CHROM`/`POS` on `ParsedSampleRecord`; a streaming post-hoc `--mode redact` (Tier 3) with the two-pass region map; the size guard lifted | §5 | +| **v0.4** | Pseudonymization with IRI re-minting and per-release keys, and sorting views by token so row order cannot be reconstructed | §7 | +| **v0.5** | Full DUO with release pinning and MONDO disease qualifiers; `policy diff` and a fuller `policy check` (lint); enforced duties, with an audit sink for `odrl:inform` | §4.3, §12 | +| later | `threshold`; query-time rewriting for an operated endpoint, paired with aggregate controls | §5, §9 | + +Two rules carry forward unchanged from v0.1.0: + +- **Anything that can't be enforced stops the run.** A version may widen what + can be enforced, but never replace that error with a warning. +- **Every version ships its oracle checks and mutation tests with it.** The + full design says it directly: "A crude redaction that is verified is worth + more than a sophisticated one that is not." + +--- + +## 11. Work plan + +| Step | Delivers | Estimate | +| --- | --- | --- | +| 1 | `make_fixture.py`, the five VCFs, `fixture.json`; convert and commit the graphs | 0.5 day | +| 2 | `profile.py` (subset validation, rejection of everything else), `purposes.py`, `vcfp-0.1.ttl`, `duo-subset.ttl` | 0.5–1 day | +| 3 | `selectors.py`, `decide.py`, `evaluate.py`: views, manifest, `decisions.csv`, `summary.json` | 1–1.5 days | +| 4 | `attach.py`, and the §5 query checked under the validation engines | 0.5 day | +| 5 | `check.py` (oracle, ASKs, dangling, leakage) and `test/test_policy_unit.py`, including the mutation tests | 1 day | +| 6 | `vcf_rdfizer_policy_cli.py`, `explain`, `run_demo.sh`, `examples/policy/README.md`, links from `limitations.md` and `roadmap.md` | 0.5 day | +| 7 | Paper: the capabilities row, the passage, the figure (panel b from `summary.json`) | 0.5 day | +| | **Total** | **about 4–5 days** | + +**Done means all of the following:** + +- `run_demo.sh` produces three views from a clean checkout without Docker. +- `check` passes on all three views. +- Every mutation test fails the check it targets. +- The CI suite passes. +- The paper's figure is regenerated from the demo's own output. + +--- + +## See also + +- [Privacy policy design](privacy-policy-design.md) — the full design this is a slice of +- [Data linking](datalinking.md) — the sibling plug-in; the same manifest and provenance pattern +- [Conversion §6](conversion.md#6-iri-templates) — the IRI hierarchy the evaluator relies on +- [Validation methodology](validation-methodology.md) — why a policy claim needs an oracle diff --git a/docs/privacy-policy-design.md b/docs/privacy-policy-design.md index 228663a..180174c 100644 --- a/docs/privacy-policy-design.md +++ b/docs/privacy-policy-design.md @@ -6,6 +6,12 @@ contract before code depends on it — and, in this case, to be explicit about what a policy layer can and cannot achieve, because the failure mode of a privacy feature is a false sense of safety.* +*The first slice to be built is specified separately: +[`policy-demonstrator.md`](policy-demonstrator.md), v0.1.0 — ODRL attached to +files, regions and variants over single-sample fixtures, with per-request views +verified against an oracle. It implements a subset of this document and cites +the sections it defers.* + VCF-RDFizer currently makes one decision about disclosure: it converts everything. Every sample, every genotype, every header line, every free-text `Description` goes into the graph, and every artifact is all-or-nothing. The diff --git a/docs/roadmap.md b/docs/roadmap.md index 649efe3..1658386 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -141,7 +141,10 @@ The proposal is an ODRL profile whose assets are **graph selectors** (by class, predicate, sample, genomic region, declared field or pattern), compiled to a release plan and enforced at the cheapest available point in the existing pipeline — TSV pre-filtering, emitter-time filtering, or a post-hoc pass. -Full design in [`privacy-policy-design.md`](privacy-policy-design.md). +Full design in [`privacy-policy-design.md`](privacy-policy-design.md). The first +slice, a v0.1.0 demonstrator over single-sample fixtures, is specified in +[`policy-demonstrator.md`](policy-demonstrator.md); its §10 maps later versions +onto the full design's build order. Two findings from that design are worth surfacing here because they affect work outside it: From 136b3cdd25f04e86639b8af4ef6c7c3b837234d4 Mon Sep 17 00:00:00 2001 From: ecrum19 Date: Fri, 25 Sep 2026 13:24:23 +0200 Subject: [PATCH 2/4] Implement the v0.1.0 policy demonstrator vcf-rdfizer-policy attaches ODRL policies to VCF-RDFizer graphs, evaluates them per request, and checks each release view against an oracle read from the source VCFs: - profile.py reads a policy and refuses anything outside the v0.1.0 subset: file, region and variant targets; purpose constraints over DUO; drop only; deny wins. - decide.py holds the decision rules, the one place they live; both evaluate and the oracle call it. - release.py writes the view, per-record decisions, counts and a manifest with the policy digest and 'governed release; not anonymization'. - check.py compares the view with the VCF-side oracle, and checks for prohibited content, dangling references and withheld-file mentions. examples/policy has a deterministic synthetic cohort of five single-sample VCFs, their converted graphs in both profiles, the policy, and run_demo.sh. The check caught a real leak on its first run: withheld files kept their header subtrees until split() learned that every IRI under a file belongs to it. Co-Authored-By: Claude Opus 5.5 --- conda-recipe/meta.yaml | 5 + docs/README.md | 4 +- docs/limitations.md | 6 +- docs/policy-demonstrator.md | 69 ++-- docs/privacy-policy-design.md | 2 +- examples/policy/P001.vcf | 46 +++ examples/policy/P002.vcf | 35 ++ examples/policy/P003.vcf | 39 ++ examples/policy/P004.vcf | 38 ++ examples/policy/P005.vcf | 41 ++ examples/policy/README.md | 69 ++++ examples/policy/converted/PROVENANCE.json | 10 + .../policy/converted/condensed/P001.nt.gz | Bin 0 -> 11799 bytes .../policy/converted/condensed/P002.nt.gz | Bin 0 -> 8875 bytes .../policy/converted/condensed/P003.nt.gz | Bin 0 -> 9881 bytes .../policy/converted/condensed/P004.nt.gz | Bin 0 -> 9522 bytes .../policy/converted/condensed/P005.nt.gz | Bin 0 -> 10509 bytes examples/policy/converted/expanded/P001.nt.gz | Bin 0 -> 15035 bytes examples/policy/converted/expanded/P002.nt.gz | Bin 0 -> 11097 bytes examples/policy/converted/expanded/P003.nt.gz | Bin 0 -> 12475 bytes examples/policy/converted/expanded/P004.nt.gz | Bin 0 -> 12073 bytes examples/policy/converted/expanded/P005.nt.gz | Bin 0 -> 13303 bytes examples/policy/fixture.json | 95 +++++ examples/policy/make_fixture.py | 168 ++++++++ examples/policy/policy.ttl | 73 ++++ examples/policy/run_demo.sh | 49 +++ pyproject.toml | 6 +- test/test_policy_unit.py | 382 ++++++++++++++++++ vcf_rdfizer_data/policy/__init__.py | 1 + vcf_rdfizer_data/policy/duo-subset.ttl | 26 ++ vcf_rdfizer_data/policy/vcfp-0.1.ttl | 34 ++ vcf_rdfizer_policies/__init__.py | 28 ++ vcf_rdfizer_policies/check.py | 114 ++++++ vcf_rdfizer_policies/decide.py | 73 ++++ vcf_rdfizer_policies/graphs.py | 86 ++++ vcf_rdfizer_policies/profile.py | 172 ++++++++ vcf_rdfizer_policies/purposes.py | 47 +++ vcf_rdfizer_policies/release.py | 181 +++++++++ vcf_rdfizer_policy.py | 132 ++++++ 39 files changed, 2002 insertions(+), 29 deletions(-) create mode 100644 examples/policy/P001.vcf create mode 100644 examples/policy/P002.vcf create mode 100644 examples/policy/P003.vcf create mode 100644 examples/policy/P004.vcf create mode 100644 examples/policy/P005.vcf create mode 100644 examples/policy/README.md create mode 100644 examples/policy/converted/PROVENANCE.json create mode 100644 examples/policy/converted/condensed/P001.nt.gz create mode 100644 examples/policy/converted/condensed/P002.nt.gz create mode 100644 examples/policy/converted/condensed/P003.nt.gz create mode 100644 examples/policy/converted/condensed/P004.nt.gz create mode 100644 examples/policy/converted/condensed/P005.nt.gz create mode 100644 examples/policy/converted/expanded/P001.nt.gz create mode 100644 examples/policy/converted/expanded/P002.nt.gz create mode 100644 examples/policy/converted/expanded/P003.nt.gz create mode 100644 examples/policy/converted/expanded/P004.nt.gz create mode 100644 examples/policy/converted/expanded/P005.nt.gz create mode 100644 examples/policy/fixture.json create mode 100644 examples/policy/make_fixture.py create mode 100644 examples/policy/policy.ttl create mode 100755 examples/policy/run_demo.sh create mode 100644 test/test_policy_unit.py create mode 100644 vcf_rdfizer_data/policy/__init__.py create mode 100644 vcf_rdfizer_data/policy/duo-subset.ttl create mode 100644 vcf_rdfizer_data/policy/vcfp-0.1.ttl create mode 100644 vcf_rdfizer_policies/__init__.py create mode 100644 vcf_rdfizer_policies/check.py create mode 100644 vcf_rdfizer_policies/decide.py create mode 100644 vcf_rdfizer_policies/graphs.py create mode 100644 vcf_rdfizer_policies/profile.py create mode 100644 vcf_rdfizer_policies/purposes.py create mode 100644 vcf_rdfizer_policies/release.py create mode 100644 vcf_rdfizer_policy.py diff --git a/conda-recipe/meta.yaml b/conda-recipe/meta.yaml index 385fc12..81d5506 100644 --- a/conda-recipe/meta.yaml +++ b/conda-recipe/meta.yaml @@ -17,6 +17,7 @@ build: - vcf-rdfizer=vcf_rdfizer:main - vcf-rdfizer-rules=vcf_rdfizer_rules:main - vcf-rdfizer-link=vcf_rdfizer_link:main + - vcf-rdfizer-policy=vcf_rdfizer_policy:main requirements: host: @@ -36,16 +37,20 @@ test: - vcf-rdfizer --help - vcf-rdfizer-rules --help - vcf-rdfizer-link list + - vcf-rdfizer-policy --version # The vendored vocabulary is package data, not code, so no import touches # it: a dropped package-data glob would pass every check above and only # surface when SHACL validation runs. One file per glob in pyproject.toml. - python -c "import importlib.resources as r; d = r.files('vcf_rdfizer_data'); assert d.joinpath('VOCABULARY_PROVENANCE.json').is_file() and d.joinpath('shacl').joinpath('vcf-4.5.shacl.ttl').is_file() and d.joinpath('ontology').joinpath('vcf-core-vocabulary.bundle.ttl').is_file()" + # The policy demonstrator's DUO subset is package data too. + - python -c "import importlib.resources as r; assert r.files('vcf_rdfizer_data.policy').joinpath('duo-subset.ttl').is_file()" imports: - vcf_rdfizer # The vocabulary terms, VCF-version model and lexical parsers. Importing it # here catches a packaging miss that would otherwise only surface at # conversion time, since vcf_rdfizer imports it at module load. - vcf_rdfizer_vocab + - vcf_rdfizer_policies about: home: https://github.com/ecrum19/VCF-RDFizer diff --git a/docs/README.md b/docs/README.md index c41b411..89c9b28 100644 --- a/docs/README.md +++ b/docs/README.md @@ -54,8 +54,8 @@ deciding whether the tool fits your problem, read graph: an ODRL profile with graph selectors, three enforcement tiers, and verification — plus a candid account of why access control is not anonymization when the genotypes are themselves identifiers. -- **[Policy demonstrator v0.1.0](policy-demonstrator.md)** — *design, not yet - implemented.* The first slice of the privacy design: ODRL policies attached +- **[Policy demonstrator v0.1.0](policy-demonstrator.md)** — *implemented; + walkthrough in `examples/policy/`.* The first slice of the privacy design: ODRL policies attached to files, regions and variants of single-sample VCFs, per-request release views, and an oracle that checks each view withholds exactly what it should. diff --git a/docs/limitations.md b/docs/limitations.md index d265cb1..1eb21a3 100644 --- a/docs/limitations.md +++ b/docs/limitations.md @@ -267,7 +267,11 @@ record what an artifact was permitted to contain. Three consequences today: amount of label removal makes a released genotype graph non-identifying. The plan is [`privacy-policy-design.md`](privacy-policy-design.md), which is -explicit that what it offers is *governed release*, not anonymization. +explicit that what it offers is *governed release*, not anonymization. Its first +slice exists as a demonstrator, [`vcf-rdfizer-policy`](policy-demonstrator.md) +v0.1.0: ODRL policies on files, regions and variants, verified release views, +over single-sample fixtures only. It changes none of the points above for a +graph produced by a normal conversion. **No clinical claims.** The tool transcribes a VCF. It does not interpret, annotate, prioritize, or assess pathogenicity, and its output should not be diff --git a/docs/policy-demonstrator.md b/docs/policy-demonstrator.md index 2e5e547..36643eb 100644 --- a/docs/policy-demonstrator.md +++ b/docs/policy-demonstrator.md @@ -1,7 +1,7 @@ # Policy attachment demonstrator — v0.1.0 -*Part of the [VCF-RDFizer documentation](README.md). Status: **design, not yet -implemented.** This is the first, deliberately small slice of +*Part of the [VCF-RDFizer documentation](README.md). Status: **implemented +(v0.1.0)**; walkthrough in [`examples/policy/`](../examples/policy/README.md). This is the first, deliberately small slice of [`privacy-policy-design.md`](privacy-policy-design.md): it uses that document's vocabulary and rules, implements a subset of them, and states plainly what it leaves out.* @@ -60,8 +60,8 @@ Five synthetic participants, **one small VCF each**: | `P004.vcf` | P004 | **Withdrawn**: prohibited for every requester | | `P005.vcf` | P005 | Disease-specific research only | -Each file is `VCFv4.3` with `##reference=GRCh38`, about 40 records, and one -sample column. The records sit at real GRCh38 positions in three groups: +Each file is `VCFv4.3` with `##reference=GRCh38`, 23–34 records (139 in all), +and one sample column. The records sit at real GRCh38 positions in three groups: | Group | Where | Why | | --- | --- | --- | @@ -71,9 +71,17 @@ sample column. The records sit at real GRCh38 positions in three groups: **The genotypes are synthetic and generated from a fixed seed.** The design doc's first premise is that genotypes identify people, and a privacy -demonstrator should not be built on real individuals. Positions and alleles -are real, so the region and variant selectors are tested against real -coordinates. Everything else is invented. +demonstrator should not be built on real individuals. Positions fall in real +GRCh38 loci, and rs429358 and rs7412 are the real variants, so the region and +variant selectors are tested against real coordinates. Every other allele, and +every genotype, is invented. Those REF alleles are not checked against the +reference. + +The site catalogue also plants the boundary cases the tests need: +- records one base outside each end of the *BRCA1* window, and records on + each end of it, so the bounds are tested as inclusive; +- a decoy T>G at rs429358's position, so the variant rule is tested on its + alleles and not on its position alone. The generator is `examples/policy/make_fixture.py`. It is deterministic, and it writes the five VCFs together with a `fixture.json` that records the seed, @@ -136,12 +144,16 @@ Purposes are GA4GH Data Use Ontology terms: | DS, disease-specific research | `obo:DUO_0000007` | P005 consent; the Alzheimer's consortium's purpose | | CC, clinical care use | `obo:DUO_0000043` | P001, P002 consent; the clinical requester | +In DUO the first three form a chain, DS ⊑ HMB ⊑ GRU. Clinical care use sits +elsewhere: DUO files it as a data use *modifier*, not a permission. v0.1.0 +treats it as a purpose like the others, and it matches only itself. + Matching uses subsumption. **A requester's purpose satisfies a consent when the purpose is the consented term or a narrower one**, so a disease-specific study falls within a general-research consent, but not the other way round. v0.1.0 bundles the four terms and their `rdfs:subClassOf` links as -`vcf_rdfizer_data/policy/duo-subset.ttl`, with the DUO release they were taken -from recorded in the file. Full DUO, with disease qualifiers via MONDO and +`vcf_rdfizer_data/policy/duo-subset.ttl`, copied from DUO release 2021-02-23, +which the file records. Full DUO, with disease qualifiers via MONDO and release pinning in the policy (`vcfp:duoVersion`), is later (§10). --- @@ -296,17 +308,17 @@ with the fields v0.1.0 can fill: vcfp:policyDigest "sha256:…" ; vcfp:request [ odrl:assignee ; odrl:purpose obo:DUO_0000007 ] ; - vcfp:recordsReleased 142 ; vcfp:recordsWithheld 58 ; - vcfp:filesWithheld 1 ; vcfp:triplesWithheld 9412 ; + vcfp:recordsReleased 76 ; vcfp:recordsWithheld 63 ; + vcfp:filesWithheld 1 ; vcfp:triplesWithheld 4649 ; vcfp:obligation [ odrl:action odrl:attribute ] ; vcfp:disclosureModel "governed release; not anonymization" ; prov:wasGeneratedBy [ prov:used ] ; prov:generatedAtTime "…"^^xsd:dateTime . ``` -The numbers in the example are placeholders. The policy is referenced by -digest as well as by IRI, and the view keeps the original IRIs, since -pseudonymization is v0.4 (§10). The manifest says so in +The numbers are the Alzheimer's consortium's view of the expanded fixture. The +policy is referenced by digest as well as by IRI, and the view keeps the +original IRIs, since pseudonymization is v0.4 (§10). The manifest says so in `vcfp:disclosureModel`. ### 6.2 Implementation @@ -319,7 +331,9 @@ dependency, so no image change is needed. It then does three things: checked against each file's `vcfc:referenceGenome`. - **Decides each record** by §4. - **Expands each withheld record to its subtree by IRI prefix**, using the - hierarchy in [conversion §6](conversion.md#6-iri-templates): + hierarchy in [conversion §6](conversion.md#6-iri-templates). Every IRI under + `file://{FILE}` belongs to that file, header and sample set included; only + the three row subtrees also carry a row: - `#record/{ROW}` and everything under it (alleles, events); - `#call/{ROW}` and everything under it (INFO values; the condensed matrix and vectors); @@ -359,6 +373,12 @@ independent computation, not taken on trust. whole-file withdrawal, which the prefix rule removes, so the check guards against regressions in that rule. +**The check proved its worth on its first run.** An early `split()` recognised +only the three row subtrees, so a withdrawn file's header and sample set stayed +in every view. Every record decision was correct, so the leak was invisible from the +record counts. Check 4 flagged it at once, naming the IRI that should not +have been there. + **Mutation tests** (`test/test_policy_unit.py`) break a correct view on purpose and assert that `check` catches each break: @@ -395,25 +415,28 @@ vcf-rdfizer-policy explain --policy policy.ttl # the §4.1 grid, from the **Layout:** ```text -vcf_rdfizer_policy_cli.py console entry point (pyproject: vcf-rdfizer-policy) -vcf_rdfizer_policy/ - __init__.py version 0.1.0 +vcf_rdfizer_policy.py console entry point: vcf-rdfizer-policy +vcf_rdfizer_policies/ (named like vcf_rdfizer_link.py / vcf_rdfizer_linking/) + __init__.py version, namespaces, PolicyError profile.py parse + validate a policy against the v0.1.0 subset purposes.py DUO subsumption over the bundled subset - selectors.py file, region and variant selectors -> record IRIs + graphs.py load graphs; records; which file and row an IRI belongs to decide.py the §4 semantics; the one place they live - attach.py evaluate.py check.py manifest.py + release.py evaluate, attach, and the manifest + check.py the VCF-side oracle and the structural checks vcf_rdfizer_data/policy/ vcfp-0.1.ttl the profile terms v0.1.0 defines duo-subset.ttl four DUO terms and their hierarchy, release recorded examples/policy/ - make_fixture.py fixture.json P001.vcf … P005.vcf - policy.ttl requests/{gru,alz,clinical}.ttl - converted/ the five converted graphs, committed so the demo runs without Docker + make_fixture.py fixture.json P001.vcf … P005.vcf policy.ttl + converted/{expanded,condensed}/ the converted graphs, committed so the demo runs without Docker run_demo.sh README.md test/test_policy_unit.py runs in CI (matches test_*_unit.py) ``` +The requesters live in `fixture.json`, which both `run_demo.sh` and the +tests read. + `decide.py` holds the semantics, and **both `evaluate` and `check`'s oracle call it**, so the §4 rules exist in exactly one place. The oracle's independence comes from its input, the VCF text rather than the graph, not diff --git a/docs/privacy-policy-design.md b/docs/privacy-policy-design.md index 180174c..ce0a55d 100644 --- a/docs/privacy-policy-design.md +++ b/docs/privacy-policy-design.md @@ -6,7 +6,7 @@ contract before code depends on it — and, in this case, to be explicit about what a policy layer can and cannot achieve, because the failure mode of a privacy feature is a false sense of safety.* -*The first slice to be built is specified separately: +*The first slice is implemented and specified separately: [`policy-demonstrator.md`](policy-demonstrator.md), v0.1.0 — ODRL attached to files, regions and variants over single-sample fixtures, with per-request views verified against an oracle. It implements a subset of this document and cites diff --git a/examples/policy/P001.vcf b/examples/policy/P001.vcf new file mode 100644 index 0000000..06fc7eb --- /dev/null +++ b/examples/policy/P001.vcf @@ -0,0 +1,46 @@ +##fileformat=VCFv4.3 +##fileDate=20260925 +##source=vcf-rdfizer-policy-demonstrator-fixture +##reference=GRCh38 +##contig= +##contig= +##contig= +##contig= +##INFO= +##FORMAT= +##FORMAT= +#CHROM POS ID REF ALT QUAL FILTER INFO FORMAT P001 +chr1 11913838 . A T 87 PASS DP=38 GT:DP 1/1:38 +chr1 33711661 . G C 61 PASS DP=37 GT:DP 1/1:37 +chr1 45899293 . C T 74 PASS DP=52 GT:DP 0/1:52 +chr1 138834739 . T G 70 PASS DP=46 GT:DP 0/1:46 +chr1 139602254 . G T 84 PASS DP=49 GT:DP 1/1:49 +chr1 162068500 . A G 57 PASS DP=33 GT:DP 1/1:33 +chr1 227621759 . C G 88 PASS DP=51 GT:DP 0/1:51 +chr17 43044294 . A C 62 PASS DP=45 GT:DP 0/1:45 +chr17 43044295 . C G 91 PASS DP=46 GT:DP 0/1:46 +chr17 43055276 . G A 86 PASS DP=51 GT:DP 0/1:51 +chr17 43059666 . G A 55 PASS DP=24 GT:DP 0/1:24 +chr17 43061329 . G A 75 PASS DP=40 GT:DP 1/1:40 +chr17 43075071 . C G 70 PASS DP=50 GT:DP 1/1:50 +chr17 43087473 . A C 52 PASS DP=45 GT:DP 0/1:45 +chr17 43095759 . C T 52 PASS DP=51 GT:DP 1/1:51 +chr17 43100876 . G T 93 PASS DP=36 GT:DP 0/1:36 +chr17 43108081 . T C 60 PASS DP=22 GT:DP 0/1:22 +chr17 43111166 . C A 71 PASS DP=29 GT:DP 0/1:29 +chr17 43118559 . T C 56 PASS DP=50 GT:DP 0/1:50 +chr17 43125483 . C T 76 PASS DP=40 GT:DP 1/1:40 +chr17 43125484 . G C 89 PASS DP=59 GT:DP 0/1:59 +chr19 44906345 . T G 62 PASS DP=52 GT:DP 0/1:52 +chr19 44907272 . T G 85 PASS DP=29 GT:DP 0/1:29 +chr19 44907527 . G C 78 PASS DP=23 GT:DP 1/1:23 +chr19 44908684 rs429358 T C 56 PASS DP=58 GT:DP 0/1:58 +chr20 12657015 . G A 94 PASS DP=39 GT:DP 0/1:39 +chr20 13974064 . C G 73 PASS DP=22 GT:DP 0/1:22 +chr20 22188117 . A C 99 PASS DP=47 GT:DP 0/1:47 +chr20 30588233 . T G 55 PASS DP=22 GT:DP 0/1:22 +chr20 36112198 . C A 50 PASS DP=38 GT:DP 1/1:38 +chr20 38576612 . C G 94 PASS DP=26 GT:DP 0/1:26 +chr20 46050645 . A G 71 PASS DP=36 GT:DP 0/1:36 +chr20 47351745 . C A 81 PASS DP=24 GT:DP 1/1:24 +chr20 50908083 . G C 89 PASS DP=49 GT:DP 0/1:49 diff --git a/examples/policy/P002.vcf b/examples/policy/P002.vcf new file mode 100644 index 0000000..f0e257c --- /dev/null +++ b/examples/policy/P002.vcf @@ -0,0 +1,35 @@ +##fileformat=VCFv4.3 +##fileDate=20260925 +##source=vcf-rdfizer-policy-demonstrator-fixture +##reference=GRCh38 +##contig= +##contig= +##contig= +##contig= +##INFO= +##FORMAT= +##FORMAT= +#CHROM POS ID REF ALT QUAL FILTER INFO FORMAT P002 +chr1 11913838 . A T 90 PASS DP=47 GT:DP 0/1:47 +chr1 45899293 . C T 68 PASS DP=54 GT:DP 0/1:54 +chr1 139602254 . G T 56 PASS DP=34 GT:DP 1/1:34 +chr1 162068500 . A G 61 PASS DP=35 GT:DP 1/1:35 +chr1 231965065 . C T 94 PASS DP=57 GT:DP 0/1:57 +chr1 232595915 . T C 57 PASS DP=27 GT:DP 0/1:27 +chr17 43061329 . G A 98 PASS DP=32 GT:DP 0/1:32 +chr17 43075071 . C G 80 PASS DP=52 GT:DP 0/1:52 +chr17 43087473 . A C 53 PASS DP=58 GT:DP 1/1:58 +chr17 43095759 . C T 96 PASS DP=37 GT:DP 1/1:37 +chr17 43111166 . C A 92 PASS DP=32 GT:DP 0/1:32 +chr17 43118500 . C T 67 PASS DP=29 GT:DP 0/1:29 +chr17 43124696 . T A 52 PASS DP=56 GT:DP 1/1:56 +chr19 44907675 . A C 77 PASS DP=22 GT:DP 0/1:22 +chr19 44908684 . T G 50 PASS DP=28 GT:DP 1/1:28 +chr19 44908822 rs7412 C T 55 PASS DP=38 GT:DP 0/1:38 +chr20 12657015 . G A 82 PASS DP=31 GT:DP 0/1:31 +chr20 13974064 . C G 65 PASS DP=56 GT:DP 0/1:56 +chr20 36112198 . C A 55 PASS DP=20 GT:DP 0/1:20 +chr20 38576612 . C G 63 PASS DP=39 GT:DP 0/1:39 +chr20 46050645 . A G 86 PASS DP=34 GT:DP 1/1:34 +chr20 47351745 . C A 51 PASS DP=20 GT:DP 0/1:20 +chr20 50908083 . G C 72 PASS DP=60 GT:DP 0/1:60 diff --git a/examples/policy/P003.vcf b/examples/policy/P003.vcf new file mode 100644 index 0000000..cb1f6ba --- /dev/null +++ b/examples/policy/P003.vcf @@ -0,0 +1,39 @@ +##fileformat=VCFv4.3 +##fileDate=20260925 +##source=vcf-rdfizer-policy-demonstrator-fixture +##reference=GRCh38 +##contig= +##contig= +##contig= +##contig= +##INFO= +##FORMAT= +##FORMAT= +#CHROM POS ID REF ALT QUAL FILTER INFO FORMAT P003 +chr1 33711661 . G C 53 PASS DP=20 GT:DP 0/1:20 +chr1 45899293 . C T 70 PASS DP=20 GT:DP 0/1:20 +chr1 138834739 . T G 92 PASS DP=35 GT:DP 1/1:35 +chr1 152491455 . A G 56 PASS DP=42 GT:DP 0/1:42 +chr1 231965065 . C T 69 PASS DP=22 GT:DP 0/1:22 +chr1 232595915 . T C 62 PASS DP=47 GT:DP 0/1:47 +chr17 43055276 . G A 59 PASS DP=50 GT:DP 1/1:50 +chr17 43059666 . G A 91 PASS DP=43 GT:DP 1/1:43 +chr17 43061329 . G A 85 PASS DP=35 GT:DP 1/1:35 +chr17 43075071 . C G 52 PASS DP=50 GT:DP 0/1:50 +chr17 43087473 . A C 65 PASS DP=38 GT:DP 1/1:38 +chr17 43095759 . C T 69 PASS DP=24 GT:DP 0/1:24 +chr17 43100876 . G T 69 PASS DP=33 GT:DP 0/1:33 +chr17 43108081 . T C 79 PASS DP=46 GT:DP 0/1:46 +chr17 43124696 . T A 54 PASS DP=22 GT:DP 1/1:22 +chr19 44906345 . T G 75 PASS DP=57 GT:DP 0/1:57 +chr19 44907272 . T G 95 PASS DP=35 GT:DP 0/1:35 +chr19 44907527 . G C 51 PASS DP=33 GT:DP 1/1:33 +chr19 44907675 . A C 52 PASS DP=27 GT:DP 0/1:27 +chr19 44908684 rs429358 T C 60 PASS DP=33 GT:DP 0/1:33 +chr19 44908822 rs7412 C T 58 PASS DP=39 GT:DP 0/1:39 +chr20 13974064 . C G 68 PASS DP=27 GT:DP 0/1:27 +chr20 20513631 . T A 51 PASS DP=44 GT:DP 0/1:44 +chr20 22188117 . A C 68 PASS DP=44 GT:DP 0/1:44 +chr20 30588233 . T G 58 PASS DP=47 GT:DP 0/1:47 +chr20 36112198 . C A 60 PASS DP=40 GT:DP 1/1:40 +chr20 38576612 . C G 85 PASS DP=23 GT:DP 1/1:23 diff --git a/examples/policy/P004.vcf b/examples/policy/P004.vcf new file mode 100644 index 0000000..cc609b5 --- /dev/null +++ b/examples/policy/P004.vcf @@ -0,0 +1,38 @@ +##fileformat=VCFv4.3 +##fileDate=20260925 +##source=vcf-rdfizer-policy-demonstrator-fixture +##reference=GRCh38 +##contig= +##contig= +##contig= +##contig= +##INFO= +##FORMAT= +##FORMAT= +#CHROM POS ID REF ALT QUAL FILTER INFO FORMAT P004 +chr1 11913838 . A T 76 PASS DP=43 GT:DP 0/1:43 +chr1 33711661 . G C 64 PASS DP=29 GT:DP 0/1:29 +chr1 45899293 . C T 74 PASS DP=48 GT:DP 0/1:48 +chr1 138834739 . T G 61 PASS DP=45 GT:DP 0/1:45 +chr1 139602254 . G T 89 PASS DP=31 GT:DP 1/1:31 +chr1 231965065 . C T 98 PASS DP=25 GT:DP 0/1:25 +chr1 232595915 . T C 76 PASS DP=28 GT:DP 1/1:28 +chr17 43055276 . G A 81 PASS DP=31 GT:DP 0/1:31 +chr17 43059666 . G A 82 PASS DP=49 GT:DP 0/1:49 +chr17 43075071 . C G 52 PASS DP=50 GT:DP 0/1:50 +chr17 43087473 . A C 65 PASS DP=37 GT:DP 0/1:37 +chr17 43095759 . C T 65 PASS DP=24 GT:DP 0/1:24 +chr17 43108081 . T C 99 PASS DP=31 GT:DP 0/1:31 +chr17 43118500 . C T 83 PASS DP=27 GT:DP 0/1:27 +chr17 43118559 . T C 73 PASS DP=23 GT:DP 1/1:23 +chr17 43124696 . T A 51 PASS DP=52 GT:DP 0/1:52 +chr19 44907272 . T G 96 PASS DP=50 GT:DP 0/1:50 +chr19 44907675 . A C 68 PASS DP=30 GT:DP 1/1:30 +chr19 44908684 rs429358 T C 80 PASS DP=58 GT:DP 0/1:58 +chr20 12657015 . G A 79 PASS DP=28 GT:DP 1/1:28 +chr20 13974064 . C G 98 PASS DP=27 GT:DP 0/1:27 +chr20 20513631 . T A 85 PASS DP=27 GT:DP 0/1:27 +chr20 30588233 . T G 85 PASS DP=26 GT:DP 0/1:26 +chr20 36112198 . C A 54 PASS DP=35 GT:DP 0/1:35 +chr20 47351745 . C A 55 PASS DP=31 GT:DP 0/1:31 +chr20 50908083 . G C 95 PASS DP=58 GT:DP 0/1:58 diff --git a/examples/policy/P005.vcf b/examples/policy/P005.vcf new file mode 100644 index 0000000..079e72e --- /dev/null +++ b/examples/policy/P005.vcf @@ -0,0 +1,41 @@ +##fileformat=VCFv4.3 +##fileDate=20260925 +##source=vcf-rdfizer-policy-demonstrator-fixture +##reference=GRCh38 +##contig= +##contig= +##contig= +##contig= +##INFO= +##FORMAT= +##FORMAT= +#CHROM POS ID REF ALT QUAL FILTER INFO FORMAT P005 +chr1 11913838 . A T 71 PASS DP=25 GT:DP 0/1:25 +chr1 33711661 . G C 72 PASS DP=56 GT:DP 0/1:56 +chr1 45899293 . C T 87 PASS DP=47 GT:DP 0/1:47 +chr1 138834739 . T G 51 PASS DP=47 GT:DP 1/1:47 +chr1 139602254 . G T 87 PASS DP=26 GT:DP 1/1:26 +chr1 152491455 . A G 66 PASS DP=40 GT:DP 0/1:40 +chr1 231965065 . C T 50 PASS DP=25 GT:DP 0/1:25 +chr1 232595915 . T C 58 PASS DP=50 GT:DP 1/1:50 +chr17 43055276 . G A 75 PASS DP=54 GT:DP 0/1:54 +chr17 43059666 . G A 98 PASS DP=55 GT:DP 0/1:55 +chr17 43087473 . A C 94 PASS DP=23 GT:DP 1/1:23 +chr17 43095759 . C T 91 PASS DP=27 GT:DP 1/1:27 +chr17 43100876 . G T 51 PASS DP=35 GT:DP 0/1:35 +chr17 43111166 . C A 68 PASS DP=31 GT:DP 0/1:31 +chr17 43118500 . C T 56 PASS DP=47 GT:DP 0/1:47 +chr17 43118559 . T C 68 PASS DP=22 GT:DP 0/1:22 +chr17 43124696 . T A 54 PASS DP=58 GT:DP 1/1:58 +chr19 44907675 . A C 82 PASS DP=57 GT:DP 0/1:57 +chr19 44908684 rs429358 T C 67 PASS DP=43 GT:DP 0/1:43 +chr19 44908822 rs7412 C T 84 PASS DP=34 GT:DP 1/1:34 +chr20 12657015 . G A 95 PASS DP=60 GT:DP 0/1:60 +chr20 13974064 . C G 56 PASS DP=43 GT:DP 0/1:43 +chr20 20513631 . T A 81 PASS DP=21 GT:DP 0/1:21 +chr20 22188117 . A C 80 PASS DP=44 GT:DP 0/1:44 +chr20 36112198 . C A 74 PASS DP=28 GT:DP 0/1:28 +chr20 38576612 . C G 92 PASS DP=55 GT:DP 0/1:55 +chr20 46050645 . A G 59 PASS DP=53 GT:DP 1/1:53 +chr20 47351745 . C A 82 PASS DP=28 GT:DP 0/1:28 +chr20 50908083 . G C 99 PASS DP=20 GT:DP 1/1:20 diff --git a/examples/policy/README.md b/examples/policy/README.md new file mode 100644 index 0000000..584e05d --- /dev/null +++ b/examples/policy/README.md @@ -0,0 +1,69 @@ +# Policy demonstrator (v0.1.0) + +One synthetic cohort, one set of ODRL policies, three requesters, and three +different release views, each checked against the source VCFs. The design, and +what v0.1.0 deliberately leaves out, is +[`docs/policy-demonstrator.md`](../../docs/policy-demonstrator.md). + +This is **governed release, not anonymization.** The views keep the original +IRIs, and a released genotype still identifies the person it came from. + +## Run it + +```bash +examples/policy/run_demo.sh /tmp/policy-demo # needs rdflib; no Docker +PROFILE=condensed examples/policy/run_demo.sh /tmp/policy-demo-condensed +``` + +It explains the policy, attaches it to the graph, and then for each requester +evaluates a view, checks it, and finally prints the decision grid: + +```text +requester P001.vcf P002.vcf P003.vcf P004.vcf P005.vcf triples withheld +alz 22 16 18 withheld 20 4649 +clinical 33 23 withheld withheld withheld 6539 +gru 21 16 withheld withheld withheld 7869 +``` + +Each number is the records released from that file. Where P001 and P002 differ +between requesters, that's the cohort rules at work. *BRCA1* records are +released only for clinical care, and the APOE ε4 variant (rs429358) only for +disease-specific research. + +## What is here + +| File | What it is | +| --- | --- | +| `make_fixture.py` | Deterministic generator for the five VCFs and `fixture.json` | +| `P001.vcf` … `P005.vcf` | One synthetic participant each, GRCh38, 23–34 records | +| `fixture.json` | Seed, loci, each participant's consent, and the three requesters | +| `policy.ttl` | Five consent policies (one per file) and the cohort policy (*BRCA1*, ε4) | +| `converted/` | The VCFs converted once per sample profile, so nothing needs Docker; see `PROVENANCE.json` | +| `run_demo.sh` | The walkthrough above | + +Positions are real GRCh38 coordinates in real loci. rs429358 and rs7412 are +the real variants. Every other allele, and every genotype, is synthetic. To +regenerate the fixture, run `python3 examples/policy/make_fixture.py`. A test +checks that the committed files are byte-identical to what it writes. + +## The pieces, one at a time + +```bash +vcf-rdfizer-policy explain --policy policy.ttl +vcf-rdfizer-policy attach --rdf converted/expanded/P00*.nt.gz --policy policy.ttl -o annotated.nt +vcf-rdfizer-policy evaluate --rdf converted/expanded/P00*.nt.gz --policy policy.ttl \ + --assignee https://example.org/party/alz-consortium --purpose DUO:0000007 -o views/alz +vcf-rdfizer-policy check --view views/alz --policy policy.ttl --vcf P00*.vcf +``` + +`evaluate` writes the following into its output directory: +- `view.nt`, the released triples; +- `decisions.csv`, one row per record with the rule that decided it; +- `summary.json`, the counts; +- `manifest.ttl`, the policy's digest, the request, what was withheld, the + obligations the requester accepted, and `"governed release; not + anonymization"`. + +`check` re-derives the expected release straight from the VCF text and fails +on any difference. An extra record counts as a leak, and a missing one as +over-withholding. diff --git a/examples/policy/converted/PROVENANCE.json b/examples/policy/converted/PROVENANCE.json new file mode 100644 index 0000000..521badd --- /dev/null +++ b/examples/policy/converted/PROVENANCE.json @@ -0,0 +1,10 @@ +{ + "description": "The five fixture VCFs converted once per sample profile, committed so the demonstrator and its tests run without Docker.", + "host": "vcf-bench-1", + "converted": "2026-09-25", + "image": "ecrum19/vcf-rdfizer@sha256:1904e96dde12ab2e2e70d8ee1267765c293ab100a8bd1b14d5b009b2bf8e34aa", + "image_tag": "ecrum19/vcf-rdfizer:3.1.0", + "wrapper_commit": "3da1a5a4a85204c74ed986a953f9098286d1bc80", + "command": "vcf_rdfizer.py --mode full --input --sample-representation {expanded|condensed} --representations none --rdf-storage-mode plain --rdf-compression gzip --image ecrum19/vcf-rdfizer --image-version 3.1.0 --no-build", + "outputs": {"expanded": "expanded/P00n.nt.gz", "condensed": "condensed/P00n.nt.gz"} +} diff --git a/examples/policy/converted/condensed/P001.nt.gz b/examples/policy/converted/condensed/P001.nt.gz new file mode 100644 index 0000000000000000000000000000000000000000..49e88901f48db9c79e19b70ee5b2edc9950d3984 GIT binary patch literal 11799 zcmbVQc|4Tw*GAEZGJ|XhGh|8GLdZHZ*0LvLU!ttpB}_CJB!py->}wLqzD!xNMY8YN z_w2)rcPuTx@3;5!dq3|#^T+c%_kFJWI_EmqxraFz0BE~iVnuL5jgOC?$Ih9cJHa6I zd_$rCa(e;acfHS_XLrty6FhVN&Wd&=R|GhMPQG&Evi2EDt4rGpH$pT{KEKl@SU!@S zHOGTtmv+6i;h3JClQf&3C{E=UinU$Z&vB~n-S*$_(|T*tMpI}H?N0_5>`$XF zh7OmsdG9XHk2IfhlO5Tjz3MhLxYpghJH3S%%$u&3sxIGm#5Ob#zglMIHC&8JaNg66 z^L?D*HoZGv?x%Xn*rM3ITIb3V5ADd6r5EqdbGKJLS{sM;Nle6)8+qR`;2Gu_DL}*( zlzb~ejOh*)xo;Yt@~JGvy_0L z;)3OTnreHdhw2i$Th$U<^|m*S_SOQHM-07n2HUDDs`PlhMZ>cNr{f0p7xavf;R~Mn zcAt=zTB7s)*;gWP$j=lxlZ&5rrVzn281Q)b=2R`AQ$;_#7dH|R`N+EmOf>PjxgPby z0yp3)=(&{jzfHfF&jEFwfl+N|xp=#BwjBPbo9OZ@J4OrAF@yK&r?0H2A{;>jGb1r5IWlch+2Fo=%Q9ZPw!A zoE`MCZ_-4!MD~lI=?T(t`c{}*{==Eje6LZ(spQTpbwA$3Ck9O3+HH8qxoJx2bipg+ z$;*fF3EvDx$|Ywhq}Sev_lU2{_eid=t*On@m^|cs(*n|;e8uaC%5xaK!6p4fvI%I2u)8{h8DTTHZ$E^jIQQw z)A_4m^BJ?fFsbhglt18+7a(fR&thGk&z-c$;t^5y6DN1nut_{67COyll3mu+^sU&j z{3E$n2_l=(N(9IBUvWuuYje(*Ia7AkVjh~sDi-tb$-MHEx%CHd%2 zh(0W#>Y-A1MM-~&Z@KPvuJ`4H?|CN{iYM`raoc0(o$6ek<3EpxFz9MJ!~HcxbSz$c zAVicS*n(mVo8%smikm(NAErYYF`Rz0a;i6m&NLK;7Dud)`T(BP?Rb91n|nU;rOv zsm*6$!Ljp0%KYgMSnn`vagOq$Rz<`COM)CxOO!7oDy3C6BP0$;(xq@_npQ-&P25h< z9dgN!eB|=9^pn$DssAXS^R5jMK$iFR9JuF! z3R7-^fyyDbN>i!d41&J1P3rq(NUU*?8w}duJk+<}6HxF5Vg?N#_L@t2IdS&wFEQ_# zd9U3c2;ZOX;jMnS)1@utU50&Fh)Lg`8=4~ZT=SS-&WR4*{oK9Lu(9!DDGnj=sAwm^ zNdJmtzSQ1)pHW47eAc9Wu#~WpAr{+8F@hm0>&drK@C#Mx2;|&Hx;!A}N=&DCacgyUouV85d16OW?GD zPq-DnT)m)}HC*{cbA5Lvt33T)TuQP$#Zo-Agjj(o4amGgI>S2k6(@3^bJKv`U*mP*nE8VS7<>2=j94taBjtmE6KqM;Cm z-lvx;Xoco)-||VwtPROr_sTT2hp!l9)(#0Fl`}))gbXC8;_Nh417O~t`YxIppUPe# z`I@_{N?eje;+E8TEh)&BlB(Y$Xj$AP-&P;&?s{oMLFBt-ya9xyB;oY3SZmqrE#oxS z(heD;X`tQ|u-{%u_xnq&sQHf;=_%HXmgQ)W&KS^ew6UxT{H6k=KlF;%4%JQNd7o7W z#KnD{B9N2hLkih>%kq4X&NrZ8cVk(mI^ePub*=1~#(Lvq^`~#8r&pcOT)_-QWv;%{ zs9+u|YRmFWheI05I$pY{pBNaUYie4_`uNl=c(UyZ&7H507d+dVsz8HbtYw+W@74rm zRs2KPBR-suYB%_tZx<(*{}qiV%Lq5QY#h@3Qxm+p{E=N|?j7L;^5!{a6G$~13& zHODe!5Y%wc4BB0nJdX4Ju(}?+3VyZ{Ekc(x9=03l_q9O;;M7x+8+@COBhaZwK_zi< zaO1GQ3UaKq84y1c;dvs@2*X<&S4quTvCyiCgI4o4absZlTJsrhH0v`~eAAR?OgcVu zBI`4)I|Eo)-)K-~Lq^_W7c6cLTUzY3+t7-w#x0v}>`nZb!u$xG)(#$+9Kw1C8S!ej zj~MM&;ojYWpYL#JFR$9%oy_RIXm?xK)iFM3w7sulHzF^`>z}d=a)Y*NNB7Cr!msROX8*P%i^Fxjj z)jOuLVua+h?}D>>q^dC2ZPxcDnns3%OMR(TyGF9qDtFuM`e(?yG&Z4mF`~6RG^p{_cQ+t1?*l~VzO0ova@?(4@ zv{BXZJr%&HZQG=ZzSUc*I8$AJP!fkt&8a_kT7AuQ$K=BusVLHB0DS^t&)uoA6!HH1 zGZIq&L2U|NB|`&VyL>~QVWSa)B!;cVYVWmPw@=JUgHkln^OKi0*LjCW@_DNs(QYA1 z)fQhcW~Dcg(pTTnA8yeO&Z)-L#!c}J;mWW=`;Xi{RP^8Ee51ZTG`>~5w`^o1bUB}C zX{kP$mKMh^R9o0b0^6Td9jzQTyvMmt;b;z(y7i#ZP7^7~W=@CUppkQJuvYBWqRq=` zlQVE3oxju#zL<33PO}{CRI6n*Ok62f zZb`_|G6a=4cRyJYmVH;4Ex)L8>byy?W2wijFLUaBnM_)9b21MUnx;?ZF})^Rn<_GT z=CSdrT&vl#45+c80Qx5gEs1P)Z^n)9bL#Rr^0u!%zGmf|s>Pqd`klv9o|d?FZ_zn? zsM}mgvrq;5d=0o>CjPk~we4*7sUh2-dlOorlHUH9?{i<^Ir3(&Jp_-Jpa0A9ZdTcM ze%TE(P}r9j`5qlZ=_ujhvVdxng<7#OR=TmdFLF8Z)~`L@9dDucmo5H=An(yh-;kBb z7Y{aOQo3IksY{f7?S3*E_PqVV9XwY@X_%8aN<->gIHVuiZRVVueD8*~#EhLL{QPoj z)IOi@sd;4k8NVEP<~*j#khw1z`AjDRB0ZPB;8VtDFS$FJsZCL)m*$yH2G$Ct{({fN zYqB%4dS&>H;~@QL?N+Xl#?#TErZjoXsyt11r2WO#xd9(pvjVOCzlE(vJ?z3DnKmxf zt>kn!%4$QiS?`68Wt0a zE8j3Q*40(P!(cQ88DqYpROX7u>esc1z0${9r7^|LvtWG}K#&zbm#)O~9jTa-aQETZN93 z2WVM^_vyO~3KzCacBL!z#DmF_RRT^HmUHTs+uh9zvj%fxR?pd{e7n&)uQ8+&m~QmV z95GnajIwrSEJ`SccGYg9PN5v5Chu#N@)fFIb76jh`>LgDdZLTJQC`dKr8}xrPPJC1 zxeiz)6V}Q!-6bIz`8FU2Xl=+lJ~ZzazPXY7>|{4QE3NeL_1*?;i?)TE5)0(jcVhAe zKPIvt#%0|-ipzZ+Mtc98VQx$%%B_f2uD>rpjPe#$8qEaJ(miIQ7}27u8}F{}S>4@S z&)?*WV>>!_@A0ww@yDLCrC0>gSli(vFXAI_9z~wSM;<(mymS(aZ7ChmNCvCt7kpj$UyPy~MHSQS3{y+f-&D%N*-7+EiEbmdP)^wxSd% zE(BV8<6|%5V{aYBzWCws_0i+lD@U=3la5clcYJDb80lTrXsUTXw^c)T%T!wLOjvcS zn)q``#Zy~#p#?2*eH`nV+Eo5VX)P9<$zH_#iY>&|PwwT?;_$U+RksqyHih0*j@RL> z2;Qo@&RV$wyFi*9sA2T>$)mBO(3Qub$&N$w9*5=?hc%&;iR{bb2Lt+ic#F=@`Z3Qb zs42gbj1EC04C(yw9cfMNUwBDX~r&`*+lC`hoMD=w&8mpf_ zlE%}Gx}=WRM#?Sgbyp*wW-E^B1*zoH4u5wn!F}bUe%Vod zxzDrvXl-lc(pEj)s*9HM7(3EN)pyDzzhiCG9chEt^GF+&D@WSEm)=EIVbXX7qo9qt z=vBk*P6gh*qh=DVU-zHc`s=ZgKfEIT4}pKUjdh{*-^9|*`$;9sSI54b^|R!pFH}4S z5G^#^78oWoV)&h@8~-S?=8?bg{VwD4h`pmTuQSv7r->z#{gEdH%h&QNs@JoqGuS65 zJu6uNTw2Hq1N=W&8s@hy2@GEz=Di&!oy?gP#ZGu|O{5cmyqE4?k+7ck%qkpPo z7W=CA7aua&O)QKjNqO#X{-_^!#Z7Jud8~e}lNx0Y5|H;u7n>N18EmG0if z`cR)>!B}qeI%975Q7nyV5=lIyYq~c$c9! z-%=FkJGyrl8E0k=nGBKmEtLj8CBt zdm`O^8kS}yClh)6&R)K7zvOF7+?5^bCOj^t#FiKF`@wY(yMxV4@SC|DZuSGeS+rNC zQ1?`IN?-)vKt!93B7gsQQCi!e4gS<-CFo69{FOrNOv{U~d)E`MfG4hbl{hzk6PHqP zDfybi);Z5?giYYnCTdrK1w=I5+s(MH|vnvJh(4(`fBR%AbC>;3TUw`oUH(=iM>!cETJbMFV@BS zQ7uP-;=jm_am(euAdKd3sP}@02S&Sm*%e+ypuW$ zCLTG{M(xQgrk2cLT0`vk{NeYkq<05@wllhM?%wIqD9CZSxO&(Sfyi^#G& zwOgqt&&3j+l9n~0xa9f#-7DWkYi2ilK*5u~w#YX`q!OJT#CGAn=islcbO`6wbl%n^ zFgwYLh=b$GHQ$s(xlYAS7vVnSB=33%ZE;+$sGtA%qig66+njcA$SU_ef%5Wo@M!h$ z?iqvqiHx|oEXnC>14*+Pn*mfqZvqfS-Y85`mDjFoY*xLD55 zzGHN`WZ|AIR8Qz%kc2z&2{Dx_arTa4`3hc zU{wdM^l5jA8;w*NRmJRY@`Y}*uj}%ldl8bY_9a7$y~7i&LzPtts!VCTQn@eQf8L$6 zU1!V7N)+aLC}OiZ5Eu6`q@a8xnMgyEfgHsgLS>lUg$K!_X_0#P=4|*{q zAJ5ro3+Ge5le1~A&$&=lQC(F&{N*$He(SfX39N)x`g<)d_p*(M8-j?9*EetZHR`8C zuV!4;Wyg$Us6Q~R6~OG+67`y#d`i_AsTK=McAKYQ#6VVB-9*n4OGVSlQUy3<7S$w6 zg0eJ&vt^JlAtu+yums|>8-WU~$+ylx7=tJ%I~jxAC8yL7IifZT;jr_UuMwV=jc6O3Y=VLmW=hE0dbq`lR} z=1=GD3V_n2XfvcNeR2Mm-pz)Nt6By%r&X(oCMhAh+CR9rK+L-W1+g;c4NP;wP@phe zP5XeH&nxsy!8>WhG0rJ#&(jpwr4ebwn6*F!WyGyB0!g)>AOVXRcQt_@w+j!mu(e@?#M&_U4JN$;vPL-CaVF-{4EQ#!GJ4G$AX$M8x^ z!QX%wbK>LMdAeUjN&eltxr>h$Ko;>;4>7=Y^k#ptDbN~)c$M6H-!>d z6LEqL=J8wpxSB}hY=Q4gfdCA_dXau6G?_3q?dn2y2Q!@+0ago$)gqa(PX(XA@(W5^ zH)OiukPb5%FmvpSbSbo@^CAU5 zQ77~E5QY=@l@8lYGtNl0|Pya$VCxY$%6QMqaENU%lz30YV%LpGFFzHLN z6dIs_KyjS}Fg(RE9Fiqgd_b1!gvqIyg2F>SXG8H7id&I^#^QkO4PJ=JcZ74{IIF@n zOJIh;`l)obn}{=TL%&jdVMO-D$oer(jzOS<8Fm7W_(fg62)6i7>ISP0q|(2|@gl*1 z1OxMwYu|%EeY_s<68iX6{Y*j8KPyk{FO^rZR+K0#z1Y5oW}=lwpfq3NVzUxC*L!5v zdt!d&H4n@Gv+`VkSeM_GXMc{)^&DP1*_>Dx2h9A`nC#*QH>So^x%g z3$Orb*UiU7_YV;E;DD?wSa3R!WriO!f2AWy6+ZVh#R`}#6%ugaRrN3tb_brCE|pCJ z!qlH4#V--B&!}Sg#57w#yE}g3mo5Tk3a8+a`M76 z^LrEErsU2AEfXisuIx`s9Yl%{2Pj(LnjDI*(-?{B^>t4sAT8vRK}nu#a(a3R>JQXP zbVmWnsE{+7;qkyMh&cen%!?)kIe5WQ`LQB8i>g7juFqMJDED&^INvn~=w&|WQoNa?c^i?w0a4SGPWB>&hSB%bk7f*o&;Ks;C zzFZgzPu}5k8vJv}ujeu-W1EkpUkH&iHo$Q;e=)EIdmz9AjsPIH+PUxcG+&}8QnP*J4N z^(=)%vs)kmWabYv45<^t0#6G1j#M}YhM~6I!73<}^|pdWW50KFR&Pvdm^K)C(us-# zeVP^Bd=~sH^*amPA7B#|@GF~jfWo)GviWGghA;Psu|gvJ3mFuQeh#VNkSX{`OC&w= zzScy1XG)t43e|N1Ttzl0i$1M{LR~@8T7?0v4=YQ&AroI&>XU&1fI{Ja=W<(72qp9o z3a>Z~g1n3q0e_vd46_MDc%VAK=0l*=@DM`s#R$?0tDij;P!LIhRz$;i!;9{!~y0{Gmc= z$C0Q~NeEit0j03H?f%_hji*3R0Tcr{~}E2--I<-#*_s<>1G8B zJ9?o|Ka)wrivIRHnP=O7R)4Q4viA-PIvjOSem;3>SATM5LkP9`ld}(&F(fzkz82lg zQ0?CA`#O1`fW6DV(20csum20(-=vA`Rb|ZGQ524(t%xni@+eCyq*CG_;foSc>OW(`gx>CWU>m1IzXvNn+J?XcMH1kvGZN~ zA#n^4(_;-tD4;6;se#_Oe{102H2qZ+paEOoKae#53g7=irUx^p`v!*dbpoLa{LqCf z_+AG;>~+IZd_Rd>AcQXbNt{rdz@%=s$mg$M*gHWmROPi7D(qYs5UdB_Ev1)3)zAM} z@x|K$BWo}a3{T)-VnL6&kR-bEK-mr+)VYI|QDN5Gk5FO7ScVijFx2oa0Mx4d5_pX; z066~T?n6?C%NI|89pTbW?$1olLE)@tSkUd&TnZ@p=t&5=5Jiq`1QAvuM(J*|zy(WE zkfl2k>QGM)WjQEgQwKZzIlk}a)NKJ{ZVOcf%!IK~a=4KwfGVFwH-E zuBnMH(CgB3Du@=696UbZhgVR3XbEIJi_tw-H&EQD>7jGg`XFa+9`xHE zjIR&+Z4?I+1aA2kl%)I}pU&zIB;8`&EGBE@zKL2i*Jgt{(}BgVBSoG+e*&b3ywPI% zDO`8}D5dCct?pmufTX*;s!6K8kjkKaXbyNA;^qWHN>~gzWsa@j4`|13!2v&T3v>e+ z{z&@u9Np62Hmy;;(N3V>?7hIdz%TaxaIFK|_^n$Ir2Y*F*thT;2bz)v-AMvIdpDB` z^6|j!$5I*6wL&7S^rM~tK^dzC>8Rtx-eLjY{#Rn}{*AK2c)Zu|0-{@j00$SN4^_G4#{LlD2?CUk z3?ZN|}&z=0=Rs3NbBLND8ZMR^U!HxjUuC792Ax%Hd)A;HGT%*II1co*L$ZQ6l8 zg%bE<4ZNlbdK5U z7ZN8MpQuBa5#jI$nx?}aXfFqUpuKDCqa*kO?PA=FjdC#dPuNOlCNI(4v?-Zr!k);2C=lU0T+c#b;qHC?H{W-T1D1TuK0GV3; z6`KjNSNE3#{)vqc+1vLgwrX>FyvVD~`-0N$CmpIu2>#QZJ;H#G{^fty{_W1&cwe#| z3E*@)edU+={fEWO|8Exa?f2NQNdVsbvq3Ugqr`LYKNkhyH>e=r1^WX4x`d>lw|*;7 z(RzS`y8)BoO146RnK=l8%v?jCK#Ekf)ljv|q7EJ?(D9~{f*idD1L*FV1>`Q0g6fGM z1HlFOX=jT4CMxbR46r)$wgM^g;<*n{6!J=o>3_GGJT)hBZXkp~PaQhAm&gg0tYl{e z<8K~Rkb|2?)Ih*Nm)r$@RLSNF|9?6S)Dq2<2NtF}#l0F~#qx9VB9=q>{hGY!zyg2u z7-?%1@kq+&-r1A!SoGU)c}TW7-6dvJF;-DX=Q-r}&FKGTFn`>P&a%KQp8x{{zLJKItKHvhHh_`8*v_&> zIU;oq{lzny>*suPQ2~|q&-q3t_FwbO{DctNYTS530hPcAh8X$*KrPDgz#C72H-0;x z5dLa*JWf-xqC0;z@&)XPUyb}91`}28e~{E3U|}UpA@=s6W+gsT|GXKUWc|6qZ7xz_UESVWn6vm7#WKANCVP@<*iEIUEHZg zQq1q3S9ue%)LAT!9~v0<>AW~jWwART5iJwKN9|n-P|)CcLZ#%OCPe5gM(ly^+|L;# z-Ow`haCEDj*XxzT=NI`_*Vy=0<28r;@OU4#$S>1=X&Cao-lt;UKDM|^I_iX{rR}*_ zn2cQPKkL)Nf3|b_NvFX4lhl;XvkNVwtE(%y5b3k^$>!x4Ki}Wwo;&@RA;f^${^aLX zvB3?oFWnbpxBHhGQc^J${Sy%WGQ-^}^S!;F+xvS`Uw0RG5Y0V94;S`xJ&G}Sj7T&j zcj4FS+?-NUzn0#V{mz7El~XZ7tD(+wccs_x#{S;*j_u-w%8L2bZKuoIjLGo5fan~1 zZRJ_DBzaRaGkH_rA;@uBB&oRyoII`!OHf+u5a+6|Y(c%f=|gRs9&@ zw&v!Cqy{m@jk803#`}y-LoyJZ+#x@=oq?Bg%`5TiS&zIoqWU2bos(UuzgV{2Y7-)C z3yd7kHhDj~m-|VKQ0BJBbkb|U+&k@MW6z5zA5Li<3_Aq;v7_CmiV$OT^ZE7dWi|1S z>s}uu8@{)37oC{}RD;Ltyw`2g3>;(=Z@NmdbhKQw(}sCWw5qKZE3I_3v}j&TmzYk* z`#7GuL+lV0c%pYHU!`Ky^qrf&;I}n5$^V3@N zOjp9HmUex4h@{oDsKBbt_1M^fVOCd?xWM@tWiE_%QoNc67*{A^;HqzXpS@kp?&I+D zm?_MY@r;-l*JF{g9{p;>j!k^wXA>7=MY9RnSEro`_-n>A`C-ET6*KhQ<0EhDo;}?d zVz^UJ784ZS**Lz4FR#-rxYi94##VR-_@PHIY%MP0+~nP3Ya73sH!J(ozI}Z1v$<@2 zX@A_kBB4!wf2-O&RgC{iL5t7E_^8e(p=^6psW_x3H-J74*eE1l{EAma3QdB$q*3nh>2M&z>Za)yJi z)!(<9mJZT zFljnv<#yr9E)vhG<)bJR8t7G*;gRaIZxYpR(tRByqK%GcyuR72o_=V%LhA7LC57q5 zSOsDSd1f|GFh6s}i(#wV;HBN_vzV!nI*O(m4lnRuZ>po*m`%9de;(QO=Q7RBBsULj z*u%xBpyGYuN$GKLzo)j4QgSH1yj+t1!|RC6jSoMw6@(|AA6c|n5^KT^l^b=_cNL^{ zjhz0O<$R}{<04qzhhms90$K4V%Q4}1t- zUnC_XZe>LwQX^tiGuCZByOu3GxOIKB38H)ZKxml_lvR(|3d z9&>6R{OUV!+UUfw4sc8aUO*^=gH&?Nf7v(XL#%f;72--kYrMpd4lNp6ssqW+w> zrP9a*^c*mS$Yb^ISR~#g^y-r7K7cG%Vc?G+|*R{>f+Rqk*N-DBT zXH6t0&t{vawpV2ZgvKW>YTv9)Ebn>Jzc6VnEi1kFe94zI;L%j+DCb9*E~uJiYzi2R z`8?m-GIXhOf0v=xH~K`cl&ti`*@bQQT01fFc6+JM?F9+G8j`o$m3=w6G>oQATB+Vz zA*njLVgF%PM%@6pI@!KshFH6{IM!|m4k6D2f84FB63^^vi05J1Aw2eZV+zRt(#?kW zy2yW|yBmXiaQQ)Buhi4(B(K7P>!B}vw7d2DYQ&x7lPB}yC96$e`s6SuY9v$(#sQ-<@K|^fc8C< zFU_soYfY3fxzk2=t~Y4$hG5z(Qnj~HrF<}7W$6PhvmHY@c^&LrJE!+{bWHB8*XfQq zJ)$0*ezZ5}$Ft~88WXtJ*^qYIGLCf$Y{838cW8F3Fr?e2fP%&oF6~D-v`?jcuVI?5zBVPN&H(Ks5yFUW-n8t88~_7Q-(6I%-C*7{=IO5^3hnmwk0yV5%@Bs80BH zxNKBikb8rL)zix8JAFutg)1G86Inp@V@dSQZPdCODhcjreN)7O}e z<$~D3R`Xe3E6QG&qb4F#r6CF3lhmR6MJ}t?7^z(-3%~D?EAYHL*W-a_9>2OsBi(Su z_Xf#(Hnekkwp+cePLrY!5E8bp`JZ3ThjeFQv@-1tZ7<9&B)^Dz#^QKGXLw{zjdx+r zoRcd*OrPQYagV~yVn+O6fL9;8Z-u0{XMc5yRxJ@x+)^y&RVFASENanV%3V zS~|5(9H~o1f2sf8JC>1jq2jc_JzDSm)^~}Vd7&v|M#5#G9;HKrM2)ta1!~rjzf{&{ zjAHCe2HDlUdWjEnGzt?b@MRSEvLpB=y5a64c>BNLiyj`q6OQ1W58yux<%b!&d5v50Tqrkr(1a4Y2 zKce#&_R}3SI~&MtmmMn{ex+X92;;+P8WEsvG|4!a4E3T zh(+u#%|9Gr2Onyl&>&iWIW2tsL++x*mQ=LGRuL1)0lA{O!^=6$FxnvLWJ7t}LW|8e zRx&vQyTOX4oI+x z*bzo>W$Ns27PmeJ=9^-BVE9fiZ<2*)2y<&C*q{nnb^>D)b+o&C$4r|1VB?e5Skz2+ zF@N{khf6QI29DBpaQ8vlyv8W;3!DG{UHm8h^`rP_58`h!voGSOZ3ylNvhW^;|H2UJ zeNAg5zt5rsUnRZhpD@$_MZW^bri_Yq2uixLvO|VGKP+WB9UhIeXBIcrlD}jvqud-{ z@SxA9+PvbaWx5ReQt>s3C_r2Rt8dhZ*op!i_56WXdBaZsLpW0Y@#Z&KnG(2@N_s`q zYC&+-T$tyhD>ZfvE^(w(@_PK(qwFj_*WbmXesb^zhvH_0rcJ?ynxb-Njux785w1>aX%VVF**D-tFp7MWVX;K^7!Id+;grX?cVduE`X(nADnB= z2>od}qpErezsrDcDwckLcYMigb=&;Wv5|erR6jy`)9F=&3&C1HTf)?87updw;OlOqp<4dH zM&sR5*%BFD3snnyTpA-zSU0vZh z$ycq!;hqcQYPN)IHs{4beby1DmTcAA1mXq?FH%aHX^VSiH{TZLzmhC(THV=uC}@2jIL2Q!DSmP?N;s zeGT<1bY<|_wz$(-?eL?W_HC4qaAkZ1NXj5HvvX(}pY>i&XWU{LMbK9rapz*uUYq20 z2)2rOi*VlE*k0A`C4Y5QwA%AuW?>7Ki1b==`ZUTmGn?)9VwgV52)ZR32fDpfKtFS}x z7c?WphgihoP&_obZRJ{))MlHlz+LrCW%#eR+(Iq+D!aQ>K}F>t`7` zSz^F+eAV$P;zH;6T0T&ZekA=SCT6m5;&mgXuIRdpMTvY_dltBK(w4p+$eE{mO8Ux; zgD(fl-vusNLmYC4lw_1tu3BfDyX!)jvEVYT73`rI?SBk+wq0eF{lN2s{=#y75RKGb zU2cYaJ(2Qv5B!}<&lhu-zk3XML&KG<{&>?mhicS=4j4igDHy$#@PSt?L2y76Aa~kF zKwUF~-JZ&TcSRytE&8}Y$p|PRZ~+x2C>3<-RyePGj<_1LEj@u{&V^<)hWWbPNe(Jj zH6LBe=xiERC(Q2KT~%Wy8##jy^t@}TyR2`O=x=}`^DSu#QI?;Nd-#50-B`vtEq)KI zc>K}YME?Bc&M(Jz%*C3ftV4A?>yk!0Zou>!?E0I;(p!ElesO+>7~QQf-zPu3$~otD zZDcqn@tX(c!+Wl;4(Hb)7OAOBmmchAzFYA3B6hS)lB0Rv+Ei->nKcN%#NCT}WF_AC zg?p7G9vte-&);)r3?hB}G%e6uMzciflh0cQE?#Y;iQWa~?W)RnF@)1oYYgM+wM6p{ z#M1_+X|R~6wAJ5z#b9o*Eq&-t`U!OTgczZ6Ci3V}+CpzZzM3>Y_rpsZ=DJIuXNsivu^&B|E zq;>m;ZVX_QL%o-K6^gyZ5$-AfqwB*4oirY$QMh^fhc59IV`!itotqu~eYofQz@d5oGUd5Ys!&NO1&-Dd;qyi>TCg!>{VSxGdR-W! zU}=)x-mIox;+n0<^oiMG&`>a%DMhsh;@4A^&07%=BF zFlIJfJhwA1NT=Y(cxp36lW_BDV-YcxSkxhT59EM+EC?$_f~Epzi`k?V7B)3|3zC@VHLe%|?8+o3kdE!)zg`{^G*9DPRr8s1K+n=njQKb;EpyO*5 zd+Z;Gbln*CzY*=?I{v6=gj_RH{x21cs%MKM{;6U%!ci8Y>i3F`WO!@_xKfpcSj~uD z(7~}Cn*~Y)Qz-^_V^~j+646`QBKar5J3W+&*(tC}M^GweO{JF`>9@rV)$+qlH^!vR zmK<|v*c904J4+PQ^qbVxIlWWBy;G2x(U2NVwbtB4mC|*4z%TBZvT1Kb#;nz#)P>cc_<#!+~CD zO^t*u4Cf(m4&nf~k(SPW2XzgSx>x>!fjGcP>a`k8Rlv7)K@y_HNxdLaZ+90)M~!Od zww?cm9i^Np1tH;2cEFM605ipuawbVgDHPB%*3hrfD#yl<9j=gi|5?se0tn|xpP3dN zi5*8OnlEdiTkffQXo?9#6OjJt?`Mj`{G9*-)(krw+km~Ty$dH# zO64!7SXzT6N7IZRjT6MWqO7PCv*<^8RnI_I!CpSKpnZDy_|7tS-hyK(u7aedTPmiY zU~%Ujs9kGMV~yvR!CICbu0)KFirE#d;O6nvGNrRS=y`}wlrOdA*!w#fZ%f2rRBW#3 zTcg>wY&R58QiS2rK*%vj5JX8a6wdz5Q`I$3%6pQFsUmJGBNz42>jVpL7?BSA*-XWQ z+nR`SL|7n5OQ@CwW{72kj!y*3Kw(8=Hxxg8 zO9hpxgn(e01G$}#12b` z+ds8Dg`~dySqn#}fK%hZ4#Z|?<4%XFml*|bDCQVOlHXG&riZb>#=q8u;CNwGF;9w}=IrR7t%9^c@ z8!b*4#wHx6EGXj4+2^pT_YZWdxAn-MNMS*45ujBzil(qbP368aHlD)*n&Of{ zZvLQE89=EpDg`9S6l+sF3`aYLL8?P9c(j%S6md9%X%8W8;R{e0f!pGO0`g)eH#&xX zNmiK^ioyx$3@hW54lS4hdPT7y7$Qb7>HWv5xVTNcV&8^0&I)*;Dy%W_pW-oRRuF-F=qiuII4Pvr#e!=R zjba}&Yg$01f#VP>hQI)3VuK|8-lbQ|sS1=_Zsq%2l!FExDq8~b%Ctcl%l0r zZ)aQ3veR)kZqVY9WYxX4wT<|^XGE7;lIZDPOTI2|@7;63j8l2*tJ~}CK7lY!q^}-+)8Nl5UfP>|;i(k&v!tnBakJ zC46;_3Rw8U4iAAFRH(34oT=brjt9Nw^SuCVeQD1b2mb>hi^16%PLhyX03;e;y$PZycIZ+&1*Frz zT&ndSE**ehJtt$_k|wG2$Dz_}P|80$e;f*UZfP6}dasHCl)eX)h6RAUVTaw45K3uD zinv=&Rdl7tC<{TL8p*G8&&PG`H(D3C4m$QomX#8>Qiz>B5{tjO34#i9Ly-iKg{lf0 z^kFeWBZD$)y<5aZBo_1_^=&YyreA59hyufIVgXR*CI-qTGf1jrh0i$s8^S26EgJy- zEri0^xqQIBWs<>yfT9c1rj zfMi2h0pLgx0cg=uvl#Y)DYP{MwM_y ze>*RN(giM4T9n8YC_fdH|31iyQWa2r4xx67Dy*Hz3W`qRtcHPgAiwrOD-tyLM6Zs+ ziQvxaKRX&I-cR)?10}PSr4np)js}`^-BSRa4`4?Un;GsS(PoXgno!4GgB;^%7#8Qr z3PX!uxsTM8Kx+QqF1Y}eWSpmX%pL&EVT0yKQ1XvH>`;9Wwm)n+RM4D%vaxC>O1{p= zOSV%CG48H{n?jcDvi>pPCk15NzYKWuKMbgVd->XjyN^<{%-->WKmUy8Mu5bVzeh9S pbK`%{7*oGnf7)ci|IC;>%$apWATKf6jG7E_k-73~E%rFo{{eD*^y&Zr literal 0 HcmV?d00001 diff --git a/examples/policy/converted/condensed/P003.nt.gz b/examples/policy/converted/condensed/P003.nt.gz new file mode 100644 index 0000000000000000000000000000000000000000..215a812af64efc58099cc9c2f580e667203d3765 GIT binary patch literal 9881 zcma)=c|6o>*#E5&$yl>x%vdWTS+Wje>|2qg?6Q?DTh@$aFo`TFYh*cuGDKu26Uv@s zC;Q0G*v)u`ROdP8Jg?vH?|Z&JpX+^JpX<6`fR8CDe;AhEBO<*nB69Vzizm_O3o}=C z)wkg*T_};X;cJuF0O3g@^ZjdAB!J;Uf?L*fL2)X~#F1D%PmT-=_zvjvq*m$voAiR^ zFS9O2Qw51>1^IphW^&^e9kZk3ot{j0BJ4hGv=&WCLX6f!M%6^IP)jn{X$vkG5dR*{w zb$y`EdfCIh{grBXaV>shZbg2s8g9k+baBz5{@?&VSyi1Jc-7=}HSMr&)AxsP93*c; zH&u5ZyDy1KulL(uLS?w%F@{~Gu3hy3i|O@))%#lp-u@*i4^6wCKb|`b^s6(~ozpc; zh2y8+?!T|YZ}$Sz*H7=MZrgItb)Zo2&}dG$?XBT*dj=1?_R_wYzS3dg$D3G`{o`TR zZ>ce4tv)Pd{Ny4n`pdQ1QdiR5zxhCGfGxdSF{vo{7JT%_V9QLvgSDAKTklfSks_ns zP4~^kdt3c`y#*U%cSbhk@4F;s7@m7S*aDOe+n!}YcV948qF4*$kcQOx`@WDIGeNSC zsWul>|1caSp?oq~&v%V&W$7-mv8VVH#pFZdyvs#KcI!(uCIQA45L9Fb{FVV+rnYv; z1)l`>t&0J|wLiRAA;nl%8h6Pz*a6|#gt2`i`s_TLXijvsxZhi))ugPuZ3QCXoO16y z^sots;ywDlD;CO8>5<}ofR!ywKJ7&Bbna_OU7#QKd#8Gj{@s}L#J1hM)Q`FmaJR`) z)5W5yP5&ue+F`)%-pfVob^uCSp`poj?rT3yQss(g;39JyySFBJ(s0Gsl|-Y3uHL@Z zG0yuHxO1q)!qj=cse2xpc)_|hQteSlkxqEZ6uVu_8!)`qjj=boC1bA8zuy_Z{nt#P z$rAql^TGYl$MXSo`>BUIE6eNE^K;IK^uwh-lZB0?0nQC?F|&@`$D3`@)C=~$tLm9~ z!=0V;x*}hi@Iy-C?ah+|v|EQ%_m{hMCe|Z4xM#Si^&DO@{Zp>qI_SzI zTbFvK!{gYeTSCBnO4tMC>+X7X!eoR`Ica}?`eYoaLSq#Lw-jntv&;B75!y$D=%J!<4L!$oQpg!+RIOz^NYItUKEAzUrd$ zkvvTyIUaKHbsV3Vn|G{p7&zD7sXeuKHqa?}l|iJ9R;{L=H%8i+M`%@@`BY3bZ;Hr4 zDZX7}?xo_XBf9IZUvAr0Q4(B5^e7-oH4*uwPj&WVOO9jA8O@l^XR3Q2xQDB_>lQHw z^Wnuml5{_RS;e_4nR$9bC#TFjJP(|j4hpru0G(L^OIaQ3w;$6Cqt0v9XMT8;H0Sl) zz~C!w5HrC*C&t*$#@tq`swp?fJwsJNlGc%a{$Luzx zmM`Fq&(>eUl*7H0=iY)J8pH`jt%)o1gC;d2Z;grh-k5V*IQYt|Jbg#D3MN(**=k1L z`wjLa3sV$HY#WEV?9QZ8BWmYsa_(uQGwUgL92a*7*&bJnF#q|!9+5=Vr)BccjON6r zom8iNF*7_Cg)KLEKr-I-**c#EQSK7=5T;X7yFLp!cb<)l2ydk3EF5&_tv9;+w&O)(f+GVTlA=|h&de_hHs(#T!&3h`2%HF zUUY5+)}{!_W6#q!_G8r@Qcpvk;-q|q?DV18w6T2kr{3#dO!wCh4EJ~SpWKvkM2{$<#f zZ2YYPmWMjjU)-g)N?B)VocZRTNML$*ZtKjYOSCGXW^9HdNu;;$aMhuNG7_}!xXZ`$ zE<0|NrNun3Pjz#0zS{0Tto5bC`DeZU?9-WvPo0MK6z1?5nrlKXzRzs=99uH7)ZL=z zrn5(A=9(pzV;dUQSabz!9I0jvA*7b|v< z0H1R~tv%Ie_WtNRXM++0AN19lwVMK; zrx|+i<&kF7pcPv#o~(9u)&>}2?8<}=Y+q8j7@mf78onvgKL&%fkMLVRo8kLG5yWsq z;4^rUs6tq0aa)!f(yYG~)OJ7_<3bYgoLRQIVQHNCqmnCdXUB^p#pCd_B=W_6)ay3& zjtV9Yw`kpEYGo5dnrB%)t4Br(uk+2!qe)?3NOe@2Nhy`|wyr&3sBdXLiWhhQEn4ek z1v)pSNJ)Q-s=L z88wYmr{vULBA}};mi9MlRa!Y`SU$$&xoq)>2V0by=2Q^wddT_C+iN!GW|?xUdW%|# zRr&=9^@zx0OLN9ojxH!b=Jnbq>+J%H3C1aB9f_z?{;z}9-a33qmzjr1XU9XxePoA- zo2!Y>?AEC6*K@4cP?gn;cf5P|4$jq>>zmE}ky&wHTV?a$x5pKox8S9?4kI-<7f%mZUh>N#sotaPtUl4rTy|S|O@$mBcXk74W`6LcI zJ5}R9{zj<8w|YMn*XO?tXV`fFbXfNbmmiz&D&L2gx-4Ig&4*BUCHW_o;AZM|yj^#O z@$8P)WURKr;w`S)*pY^&Gfi##a9G{b)qH7hxhdh^O~TUuWMSOtvWyS>(uZRrxv}By zTWk6T_?gW1eUuI)Fs-Xf)|fP*^eHq8+ePGwCl)iw`K%-3~x7k8!Tju ze)PNRuRa|&SX!96IT$k+c({Eo@*%Hk!CiOLX5s3`yPkb!c-&)DzL$_q{3yN>Cw5r< z-q^g88CaZ$@!jrEIo$9zp&6Z79v&6_GTk2dT?sKk5|(-RLukj$Rrn_3-nP3BblfY7 zeb70oI|6}KIz;Efb`k_%6|uRRiA(<|cw}spfA+$zQ@5VPdl6^(;TdG2Xq8E3KrXeS z=B+XZ4}+M-u*fVS^Uasl(R9v>eWCY99#{KKkNt>Sc*W_v_(So&+NU{=H=Lo=y|ble zPkac!WBP1g6&omTU-W~Hd&so!&i%|RhsQ>87?(RRlk1zUOO6yW3y9E1BYoMchA~jp zO#y2o+7KlT`Ew4NET%~k8B6ZyLW z9LiLy7{z=sk<&QemmOs$M@cBbG>af*H55?ZMu+0`tbp!nor4x|Udie(kUMqmVr02b zU+A)gN?91redZM>qW3;Qf88)fLv{1jiN4W)=ohF){gZ*!q7^WH>=u98@N^O9y9&Rr z(gmvLtNj2Rdh3-cg)QGdlsXj3`_FFA4(Z3uh6f>;w;MhA}Ftn+_)) zB>%rXWE>6>Jcv#falWbWDZTpF?q4mBr0m6SNF)V9V*#bkI2LA&!rn}tyM{?E#DPqg2iXb+rd4;^XewOtub zFQKWh$IowRee&_QP(|DWi@B#YGAuV;Vz7UZ#kx&Yua_(r_<7V!I}4pwnj*mN3CUEg zOL@iWMSC5ZB5B3`pzEM_IXT{KyY2~PAIyPFDwKaL!icYP5q5QjIPfn~6Q3V)I>*o& zw|_$G*(T!6G4Ls$*lvg5(l4;WR~C@hwGk@uSA9qVeQYP&b;L-4-;FMow&*ri5B?&) zb3%NFh&R-Wp+(p%+?jc7$9^I@{?<<2+1-24H8nrp^Rm>C8c}CE0v~g2k^?H+2SM~h zCV*^o+J+qRv)zO)h7QF?eAH?(sVwEUe5NwOBsJ_N_NBo$88RzvNV`7JqO{TmAK6Wi z9%$H>k1R}ZHy>4<8uqj5h(Ww`l~_Zq-m|3R1N)(1%?(lN@#8-2;AQt=KB=4fxb86f ziK%1Q|4TN>+%xU9^?=<(G@N-%AUW(Owb^)@59H0-!jK9sJy9k)rxho5cI{((XO zWWezME_aoE^xwKr->WFA)G$f>EuU-~<~kHNRGcnFQ({%{;Aev|%O6E6CL_$dGZp7oX{w7IU$u_AfJsdTj&LhYK+{^4hTdwe>T9tJc|zE;Z6@quFJ*=MH-$-0vc z{rlI&RIcNo0IXT-2qwA(*MYw~w=frdX@UQtz(=IIueg~v8K@E&yWOT+NF5mkg*8_% z7nnGcgk|M0-XmdN7>podP0yQpyvgT?-zi&`jR{_N@_WDDj)&bh6lz`W-ZiiD6D%>z6Xxi*2u5nuQ5X+9NMx~ zhd&lwuCGL7j2Cxgc+-YvJ85rBn2&^g>fc9V-j9x?I7h6+-I&KbZ+J(6ZPbga`b)#C z!bTrGEJW__Gaxi*^wfBywM;X_YSuOJqSsQqHXXd0ad8H4Put~lmB0b*wxzDyuN^W{?UjtGXO=WORk>3yC^?^)he8`aA$ zGI@Jpa_M=9oGj*f)I(R+$Lz^2>LTq2zm8U(htzUF-&~Y9w0)L=2$lQfsTtBUyYigP zYcyUv4pzaq7%@O7EN=dJtb@$*^Vm7bT9c}-Z(=~5GLw7C^z%pm$l=Cnd^aKKZqd;E z@$9x&EdiA)ig9sbRl#@Durop{n}_8NUM1h4Uto}t&U&$5R}$`dN$(7qxCXwj!t$Kx z`FP8uU6~w1vj`h~7-i{3^hG@~GA~uvt!=~3I4yr3mV4y$&7T)Pa5CI$4OXqa?%y|R z{@M=MUpf=QNXcUqft0Z*%G=;CjQ+%*II0?SEmV#Zib8Vw8I`&ORywxF*%6Ds7nU`) zeK{E5v9oxZE#^g(M{&j2{-{OA%)TNv0%0#f=Sxu<}K8=N?AE}WRb?D+hZg%|r&;`OW5^`7Te$Il;B6&-H&2FQP#^Q^-q3F~I8M~oiq z`kmW!wWxtZI=|u+gR3cioR+aSiqO>gp0Xr*+8tMr0RC{;ekU}Ui$-wHQe(tP=E~q@ zO?WBY%WGAGQ&wO#(l+9R9i>T}Ulrc-almMGo9nnsg^k6N4OtUYp>b`<${uo#UxKHp z8NYE?dc4!o>dnsV@QYC`Rmfk$MMiC@GfV5u{_)?8byE{+AWmx^1I9X>Icc{#bE_|P zF2)eI8IatSx~_SB@^aL$ChaFFs|8XF+j2kPbuhcjHn8*~@BOv$yo%BNWk^QD3JdJ> zF58pK`yMUs&lEi6y4AabTKaaVfJGAkqpextk;|^8$}G_X2@(DN0hK92cK&{>gWsH- z#~#+_ea4nwl+vvTy#HuAukPui0?T>c+njLG4$+=NoK<>JGq@GgZ^3ZJ1gCf#>T)(L z&}7{0)x%96_}gp12yiYy>ADtq(mw<5@V_8_AFBALI)xdAC-ZRqS{+1jLn5aLXw#cg76=rG`fLr=Mky+ zm(iDH_DT;MD)*`AxVdnkTmV<=mAy9c?hI|%;@7NumYI-8sY@^GO-XTsFmx`Y;1u&- z6Oeuc8{s3LRbRug9aNQF&s=RrgahgLVrU5l@$N{xyEsD3LZ8#%bROlWTPO5EJZLh6F z2&M1G8k010$hTwWIm=KI1IlU zK_-a}U#r3O95)TGKBWs0-UP90pXmP<8PWM_6_gG|x<4Ryd;<$928F<&ZYsb@X-;E=6-di+IV_EW z@C~olzgW!Y)$5VtHVxEzfgdxp9_Sgl~72w2dFE7hA6j9=Wubk?(An&pGD+qsz?Fj8Dj8Pa-H3^nN zVY>Lh61W@U#oa)B?Dt#)IxHn|2T$@QWY~3?BWPAlDvs@g*qSGKv`6uYm0rcv&ADethj&^}hkKL2%;# z4bTAPed{X;)zNj$|Mfd2x1;d4{}q$If5&9vDunKiH?}_r)v!tE#)JrbLjqzR6^20> zD8Wg2kDCkzCw`!ep^dr{guji%-zLXO5(dptzOe+RC(XBzS8w>Y=DT=*);{URqpOgs z)A{Yu@L2to{ES#801?Qfd!d#8d)pj88%uWq_(6D_@@fs-eELcCm;$7SD8!JdL_Jn8 zWR4fRvk`(5!Xh1gxCLyMiWw4|&jQvzhlV=1OuA-eSfa8rUV%PjCXlJIfJyuveS)l# z`+Gv;!fRpzNvtM%Z1dmW7XTBpgd>9j-6*dJ28BB~?T(W}h|NzyZYV~OvVQSVa~Jqp zOXYNyrtD&gjF?h9@&%xzeKU=Q^p@gjx)Sm==qY;Z+z^2C#jH6N12S4rZ6qA8MN@n=cXSmQ>6re3zBm?+k=Jjp1Dg;Jw?M^>{ zy_mgYBpp7_LF2=VEjS6r~vNQ{&UQ=<|R>^-84R3Qfq-GuSO3Me_D=!|oV z2;_VDz&O~Ib*qnfzSFIj7(gMFj~KEVDSI%#tu0KDo@oPsS?*GT*rEW@r=T22HAN&b z_~phur8Am@1Cv0g7!io=JYpabq=RIPfQ~w1+IZ7=0~d#RsmjGV^a0?>uewMiQv!g_ z1Ry-k1)4pE;3JNcUDSkHOo1qnG~ZW{;{gKcV+)3fNV-AW zuRz;30h02Xkbp)y+%q%{u)JTxwqiwPY=^VhpzL(U6BGoz__)sTNtLCf-y>343@$sz>YTg(u~ z1;_V`P&>L`LK}LkA7BbNbz zizbYK&~M9>B{SSG^hp$~qv@>QL;xuC^Z?dDlu>}+QTWCJ`u68mDX;tvDH|91_TNH! zaa9_vPzYX8aysrdNCeXOWO|U&TW9}sdPw|xdWZ+=(O54Pw=rt!ApmqJEd=E(pAM2n z1sTl=UN@6jmd_(aWP)J-jz&e!9Iy)FZ0Y3KYboP(0Q(S13zx#62(@rvodwOq( z5rMq6$pE%KX*3}<;`?-@u&o1b;g_I{5AE zh~&xaa3Y2}dHp>G9>`95$b++S&WZVooISLZZeAuy62eQu{y76!5-uBZGKE&=5K_Ak zV#~uSv1^tFbTVPBU}K`|vrlA*is3IpWYVdKK7pG_xMnI#g+3*YTFVUwtoO|ax^A%A zB`%w<;OXg}4?x`Nsg%90v0}%@GBvDXJig*TuFp=nm#j77Db<^|VjX=p&%J%%qkH<4 zAJMRqx$gj{wqnwfWMf1zuE+N7V4wgm@o)q(Xp16(h1tqq>!Wb8{ zqy$rN0$LjfElsF=XxSS6GJj`3qMyfRKlseua(*dB5{BTuafL$q!unI{S={BQ$L)au z^dGaf#h=A8B`gxAaY}zF38`BIc(~P^{_XzvO6v#!g-zPw5<^F7EMB>8LpoOdFO~(^KotuQI zZcP7qlW;N57fLEVgYHIpj*V1Tf%3+lW^!E;*yAqe0*G<#u`^*@Ph!?>L5NxR_$viC zVFh7ynHt{~rmi0EL-CqI_O}m=caLMHh3s%J;(>~>fY{^^*J9&C0HiQW1dF9{IG|BY ziL%EM;UM;FJP~U@i}*F39EhM|f6Kp7u-Ry^q4EQ-x#R3VJQVmfIHlB(mH+JLq{4p> zPE~|4BR9}+kzYyCJeiWNT{{in5e4x0W2#V=INX*ne8dpP2Q-LlfVd;K3i>mkUsGw8 z2sHaQSN#f5&%~WLgt6tydDSmvgtdv(KeQD26*Y1-WY|Bg8UNhb|7)gS1A)Ko^#2Jz zxe7AuKf@nPXPx;wVoqVwzlRbD7aIEa@GBy%43#NMw2lp3*8epqHDUpp{~DAx|2-&U zPWQYcCogFxOtW9B`Dl;)S=h(sz4Qc-8Xx`V9>rCV?|yp|f&PBY7~eq{ r|9Q=rC=m`$ke#9|`Sy2Y={KhcFF@S_P<{6FA@RcHME<&M5~BYB+d;vN literal 0 HcmV?d00001 diff --git a/examples/policy/converted/condensed/P004.nt.gz b/examples/policy/converted/condensed/P004.nt.gz new file mode 100644 index 0000000000000000000000000000000000000000..776cec57086035114aeaf0c7f77d80a887243972 GIT binary patch literal 9522 zcmbVxcRba5{C|XweGnNL=a_|z63Q`-laVsYOdO6Pdt|R;92_!>Y$1`APRU+zM7CsQ zWo6~q+d2IXb-R7zvQwegpvQ>X+G^IH@fze(8dP>xr>+ zlb8{eH=!%tC4jUM8D9TKk7NP{<-6GTB<(CPXC9qsqp_z>imPFl4eu^d*k_I6f76fG z#}rgsJ}kfL+OL5rDCBzFe~249tU4HJZZ7O6mRZs4|Eq<&Onh@H_Q;OD~}Fn1jZ`zxvw3XG@M zEq1(kr<+SkkpBBDz6WsD@~qNhp*e}Dse0;gWMT2v!%t0lHK*mW8?19 z^>C#Bdef2>cZ9W!yVKmkQ03hV0~-TDQ`OF zdxNhEUem(EnILWzQXgLy>j;dng9yX3-=*?n8A^Mqo`0_w$Slialo<(S)sA`~2O1w+ z((y5jDX>nLa?|e{~mdX7l>* z?^y0R;OFe#y(_~Xd{a&P0WbGHboPSOA0F-&R2yAz7%Uw2uJk^1APnyJ68Z>5+exJizembkl)`bl+K-b8uhF*Jihr+)bk{+?!Bw9;o0^5zRoHAX8hZ9 z`}d`?mp$d3aa!kQZXUJ}RE080=XR<_4p)R*l8fDu6|G~x42y$cMZ zo7mA|G{t&Q?=B5L8=2bJcI<}BcF%s!SWWKG-xx;J#(2C->!>x5JJb4cjmuj$`4Rc0 zZ@#HdaIc;rte!pD%V~?KiZ#{;4ti>S`{=?bOMhMSe7bsknn;a74e)1)5t>*cm-Ymm z?x)G}#O6xpql&zMxqERUf^i#}GdZ7Ba3#CQ0@YI3s6ZH(O1H^F4T}FI2YkfO->h{bK)X=6&3K*5lIm#w5?Rp6Q}QU z%WouKoBWnl9(a0QTXjUCQyDW(x1w3=;k^2-v?$j;wOmK*L}$~z^n53a{n&$B$Z4V8 z>DF*x2-gG`qrF_8(vPo24?Ie5}%&mlKlR3pnWS7}2&QyPPOR9eIWqUdc4cKXgV5WdzQHf`1*4{U$1 zFxu&F2jzMmwAb=*aqBQDTOAhC=&s~tJtT-;%ANGPH(HOQF!tYluXBFDd(Jkj1<7f$ z)Z#PWzrW_#{3$KW7J`_2HITTR^N~XqrFfTnaG&K$-0nLrqEf{MSDpyrcRu?x=aI_F zjw_1^&Soefx|(+Kfmc;!jijelTZCS%rKJh9tIrUd%6Apearp!(@Z22i{q=E&b+fJ} zH%80`zhK^$?svbttwav4@K&g4g6EVzs=v4xzV09Fa~3~e@_fRHbtP*jx@otQ=|NYM zqb!d*&Q!W>sUZ0t!i~`-s{SDFZ8zz9?nxJ}cmvM(F23uUSB$oHOP)V;sX1bh){l78 zLt%7!wL#3HNc&s;B7ErGY1#aT^b{ZeKv%2N*b7&koAIQtO^X+nOxkeL1iRzB2XkM( z$&K`(^3+_kI}tR)N4z9Wm)N}Q!npqw5!}y9q9-^*zR8VNgWb1fjc$HfzNlYW$&5Yf zXrzEgHazVVzhFWaUmJwW(JGm8^A#zV_m1a`Lo8tH710GdTyLzU7?csMQiO5kH{;A# z=-VI}r$(zS$Gaq%=1s`uHeGJvlt{;`_gAVJy$4M+sp`1ZzMGtsO)l~X=?Z@QgucR~ zvnKt%KuPwY6lldv8Btnb6@}bCz{8TNn)5RX!S(n$qWUV4cVIK_0j%xm&cV?WRphQS#}QtfA}%@ z63m{&+?2AO+5X7W#Z8tG%%7?d&=tIXR^;R1`-Sp&v@{1wq2^lM@Yg*T+P-yyX5+vB zq3wMxH(?C5>QNfonG)S(aaaNAKd|cw1xu2>LTTk|cWa}3RT%c&PJ@8mE? z(Vrbth$|aOo7_I>S)$)wO!;*JV(bv~z4qWF%y#X%D?d0D=)3@_bv_8uE^iA~-!0@dAy zCe^pk#(wawC!nMe^-D-(MzO(0eV6gUA zo*dgqWabwwm+TaQ7OTU8>Y9w=s=|6Cos^n`$h|7Oh0o<5)fIP5JI}0O^fv8x2Ja9K ztBXT%HaklWUXCu4{sO?s0}O@ok3Hel@^P*QLYphIt*ANw>f&NhylTO|Ny#nRy@^cT)ahy%M{A|qSvj*E(@h$njkjxh%cS`pPTD~eO_p+3D z_l$i&U)JAh9ZNM>E@E#G8ih(rNp(f>UGmkq<2NVj7w3P{}69t(U?5 z&&zZmi$ndvUt=G4OWh%2&F!=*KfAT4J1=r-4MXWGGS4IRzD^`YEdHUXAg4M3R}K5lga(L1_Ar;~0 z?pq15=`wY5)g=#!`>~gMq;j5 z-~+`-_k3@OfThX^+jWW+*V&9%CJu-0#=hm`xV7!NXh1*AdPXK#a5PL+pzAyAB95m1 zPBO&5Z47=h`{(VhzixN45(3qNL@s~N2+13}$UBYm<76lZ{{(QM(5j)2xsf&;ZaL28 z|4!29@N@MKO@sKKoUi=ieC3Gq)jAvS*Q0u4FDiM>4zI#+ zjMDB!@6E#SWF61Y$0Vtq+mmh0mXjN+BEAmySn5iY>r=Gl9?Pqr>TV>Krw&_fzBFJ8 zg}>;q6arz?iIV%D>Kzg}BpR?EH7R|)BYb5s=}~9n+@#U&Xsd|tPstL~n%=Jdl+4KU zr(_|&Bx51`l#C@K3$-5VbYaKb{cHPo70bzEqZN1GpN3mWU#3R>GzE3HHndPc2hnAG zGqSVsS?_cB&5n-7i}A4q;X@+6Tp1zVQ5}sx#d`QrR*qWXr&u9B#WLFcDVF8rPq8c~ ziDLEVY@^cte~VXKz3PZ5`+xS!r9iz>@*lXhw-^2g+ehI@(dZ(Q_cI(Ka>wE5Zu}LF zKX@Rm1Np8S&)=PoYQZ-XTDk_cI1IWjLY_V@RQHlM_8Pbl(jxk2ha)61U&8>3X)TO6) zBkoo6mxWC`e8lIbqB+vt)g?L_mv7m>GI2Tg*fDX)lrHPC=z77q_SLDb?VLW*b1KFa z#s!^j8W*myk1>wWV~i08y$ercA1t|PH0in4Rt`QaGKsm~G+Qg9kR{7_o zKQ7AefB2mEoKT|*W1Ehn=d2T-(+>K4{Hyz$zoQ}ciDr2YtNo-yzQmwtSB~9w`Ybw& z8xy=XI`66xVa|ekn)|JaKM}k~e9SD+#;tNstgwl%<99Z`BZ$>Re4t+U zFQTA7Z+dq=d~-eVOg*DFzZ2Zb)yP`C_Ll7{10qe!2}4|T)NLCE-tw;O{LQS?($|YC zRX)Bz11;)(tM+P*;}8M;LLtxUFedY^YDaKl?>yvutib^47s)#N-U`XAb*%$?T9twP=1YeqBC(Fvcx9dmywgJDGe__i4bH`Z~hif-$Mu z=o0V_(_2bP{{1ixL+13y_soMKiTt!D zyn%F)<8*6~jV`sQ6DN=H_d#KRsMAEAU0_S@r%_1qxdyT7hDYWq77-m>w=4K4FOrXt zPg0UkgfrfLpu*m8IvJAdq<+hcs*Hp5_G>>Oi4e=zBsb!QEvORtoO8{tX2}_fb>~ix=KjY z8XUCn>v~n5w~)Jymwtw!em8y%Uzq1O9(mENUgxWW62xL|YHTw0V#T*BH-hih3%x)C zO1u!_O`mw)(C2<=c4vCB^m_9Q2T8<}M-VOnL-u>}I`m?UjstJE@{DKZ#yR~DY!0q( z_seu|G*{0O-@;VOPpKP48fOwVtjmmCxuu432EGlZPjsl1d`jfjZ%>_FUKgl%GGM5c z8ecPEx3;k5=CQo)`PxHbgwuJ_l;W)Fxtv)=QKj&DaR+(`-yE!nKGRmF^gt^;u)X|X z+Si%uUQKO zZ6<~-h)F9U{g}UOs;=erWlJt}8+WOGn)pzr?G6@8SL@^6%+{Dc$A=)xQNW%iB_mfs zhLG<+3`eKm(WM1%<~d^^n+ArN3W?~UP1SxLC~_AJx`{;3_2tFlil3QTgyqE|Hp65R zvJ{9Q_l0@(g*$pCh#>bZ9t2dmhMF+%pR}OKtS8N`Z#;pgy~LNP5K2M3uIUviK2V1_ zryhI*8A7?gRM?ZN5TTIHl5p_7+n}pO{-DYI4A~nduNC7ILZ(Nj$L3sJam4{811}FA zlndk$lGqm(?RlmF`~ErG2AHJ~^=LV({u3#_5{AuHD4<~2Z#qM1y{|0y>=RtdA@GXa zL6ZwprUEvf1|N8ivJXz#3%3p>P+Vv1;Z{ER`NhTW1wGuX7ANotKzstlykjPa1TS$( zu{)m4oq3);DJL=ARh|EY0L1V7&+x|~*{Xl&G-{2hmCI2Gu z=-_VxH(Op8Dk9Bv$rnoK`r!6Kg#GiS3o%n#dDmvU@Oy20LzM~`E96bmePv$?4?m^I zfX%QZXh7!s`4Y@Oxnuti?ye&^xABq?o6_(82`-mqzMCIRw1yPAhr3RVn5ii?{|zPych8O6b~t)x zsy;YmfGLEK3&h{-{GE>R!+U?hONDMqODQ5x(o)XHWRdLWQvQyIbrK<0V7{aTno1BN zxZcqXkyh|hks3d=BO(8^8MaIFiTkhMimD)s{~6rDJO2?}6S$l)jQ0cGW|+wZVvBi_ zo2PR5ZXiWr$YQpA4k2|#Bn>SPFY%Y8?|}Gs|0yYLAKdm&NpbsAMCv^J7!IbCLyOuz z?VggQ#qCoXk&po2ko!gZlKT7lBLYd%&m*i@jZhssb9%2TR;V|bsCnjf8B)nUg73*g zV@l3U!LwBFK5Howz7363>jI}3Zr^n(#kc5oVxrD33Z!KVm~2F%w&qRrF0 zdxPUbePjGd%_l#a=T|rhfk{|hV2U1RWkQ*IfKkP(79)~1j7~rK|z(_Oo&hhgk>1;)U8`Uurk@E zYnX*$81TKCGC-mUAn{faeUjlEAPz)Y{i8v=NC|Cb7eXmiF(Mp-fF|d&=UDYnG@ydJ&fK7H0)XBZ z;47e8*II$#I#Vi`HQ z{s{=}kbriG0;d#VRM30q0C$lo)&U-vcv&MV=_D#?ga=zk;|9%<(OopHB(}2{GK;1ABu+d-R33oz6-^>cY(SnFLjFoK*PhGoH0 z^Q)m>P(oev(HR!H43emqF3o-@!xSD^V#GX3jTxkj-t`7)qR~rzR_HndZJ=$Ki8Yl3 z6jnpU08UDk0X<~}J#_~-^YD{{Bw=O{xu(~2prE)=AR?RrVI2rO1wNM9{YRj=VKDIh z5ET=G5{R%41|CR$ri`1?&i9#>SKcJ5*$tc>mn{A#XH}sIWuP%qr*va_CvKni86ZVF-UJ z)SMPdW+8k7&8&z{poM;K*A|3DD8IlZXrRX#6`leyRWUi%=|Mn-y!Vbqiu&46n4MCM zb}-5?^_V4Dl_QqU8;q)=-yIXFi9R9{O^hlET_Hc#eu=x%kaJs-`QR03Mh^{qKZ>Qq zd;nm|l+h>89ebS@7HZB3CF7+9VcB`HUr0f8uwx6_ar6DgCZ&je_b0aM#Qbt>%vigt zZ=ri@t^hd*tl0#wRxHXyp@)Jzvq&g05=!U`%)JjI&qG6j#8DCh`@iZSc|DR@)kvaxS8FvJ9Z3~(PWEjl5 z+P~X&3bFh;aX~i{$WQ`B0lo=nqHjk6^ZquQmKSTN8-OA55(=2d3ex^&}Fb7iG72!(^%IlH|&lUE?G((wc4HX8G6eKfJ z&N@==KIBi#(a1%ZnHG{Q)NP21*xIsjH(r)S}bSKr5f41jO-@dIvwTFbo0qs+|O^waa=@uFbsGMai6w3joT3Gd-w@ z8wS7-Rm=l&=n_;zx3uu_VFPLGp?tDOqtb`Qi#RGFp)&vmb+9uOD=96m^m~R=Fd{7f zIm0dgHN!XUSn!&Z_m;(#&TspR|CO!dJ@PwS-fc}1MapQi91tqW9F=6mgAGI<4}>wK z*uxue7ObT|K5N}`loEFTP_$G9D+pZ1W7?_&?YMcg>^9FnKU#JP z!Z_mAIG$00u%ms59i(=K!tPF^(1urQ?%xS#08jl2y+jK@qWo`Q?eV`i-_gPEDee>6 z`d6?dP7x3O3>FaFT|fn5IS2R&>Q}6wB#a32f6~SBFS@4o=2xb$oR2ci8^~M*|HFUy1XOUs@zZc(aXa^D8DQLV;=x=OA;?Sb+zp(-0e(&Q(0OrkaC6DI%|9!R0 zek#>eLZkb>hTY&17CJWAOC@O0zYO-}-v%R2`+y=&_kuXyzrR{VF(B-Jf3@1?;`zOd kj}HEG8Pft0_J52=z1=V5eTzvYCWi{sW|t;Uj+37FKMcGl%>V!Z literal 0 HcmV?d00001 diff --git a/examples/policy/converted/condensed/P005.nt.gz b/examples/policy/converted/condensed/P005.nt.gz new file mode 100644 index 0000000000000000000000000000000000000000..c4004db6758b8cad9a763f1cd19fa42e3331eec2 GIT binary patch literal 10509 zcmb7|dpy(s`~O914zVGpFe6mT`Iz%IQKAr2LZ&FkoP{h5%ONF&oDWfmq@0;p2sxjR zQO;+^7@P0t_<6tIfBb&`?sd2xugBy5ysqmua2Nn^2vKB9MRQp~LRQStjcPd2%;}ic zyWpkPd_cmW!nkki=_uNY4dE45b|?t0QRy6B6gSW7L9Bi0 z=~hd|?o{p+-FQJ%z>5y!a8)TX|w>ak8}q5hIUivO7v>b#hojk-;1~$xQR- zRb{+Z@A;L!d@R2L-d+-+nN3-6*_KYYC0#wV*obHfBey!x-z*ftl4l<$v|9J~WBrIL z3tRPMTk<-xbVf~kYjpRc+Ek2ux$o{`ci0lji}0mY9)Vq-TPk!KaX>v?8g=Xmn+S0- z$M;UneaiZR{IEkFz2rgDvdFPbc#17EcrN-KOqzNV9)qpen(0>>%A03p5C0yL8Bokp zGWdv7H}SRvWTd_TKfccY+Hj$ZPsDF{>= zt#8?~Exqdd;EO7G`c7hA%T!E^&Aq_pc}E*}r%BYjb;c5Tb^OCxu~L!9^-RiV_?F(Q zav!URamQ9}QkTaMM}O~WnNf%gbSmD2ES6Ft~cLnvmr7o>QxJIX)sj+>e)VKjMp`W@LCnNHtoNnT#Vt*~ah)%(Sx`dEXxPZr^ zi&X1?T+?^+Oo6Sp54Dk^(bKCx@7PmLwO=Z7Y4n##H-PUt%wXg&Y(cG%;m zizf#2Z-1|`Q@+=jfL;`SU-$IVhJ?vC5X0@ZdlDiLC4!x@S@#vBlM8?ECU>O4drgyX z&RYGd)Iq&o?=tL^+bDK*l-JSR!qhUAe+Rh zUlgf6CYyA6hMU<Gh&k+`3J94F2Cm z`nZ{=l@p$Ba<__!4B848FZh?#u{M0xDz9@DXnZuFec+N7Z~eO?^$qv=*75gVkmH^c zj`%2cg@r7fl4Bpku5czq0`RdXmGdKYO|)-EHt?s*i!5TJKGh6~IMSHfp+ZqSSkS{LRvg%$w^z14?h!-hV6vodHcq8T$g zd|sXxFs~SEi+cM_ecqw6*@tbX+ge|y{HST^n|A^CHgAvGd7InOW4kXB4Gg~~W)hwW zlOi(v7<6~;XL=?+Up`xCTQ0VgHbV6xByz(nbIB-kS+3&rjGXecBh=7vNlv+;_J+$T z0v~WRc*+6dKd1 zjy+uwLriqDmMXk%f|24PDpTu#r5c?wx=S~hlRO#t?jKI-JEZKpn^>$qKDhKQ%^~dF zNu2PTt_9Bq2j#Mh`1wlK5$_ncsl@6!rKKt*Vv$n8ivXvxHL>h*pT&1+IX3sqA083v zQrW*9j+Zf_7oR+C)&w$XT*>?1#^Hi{z+h3gqIYSc(1l(s1l}2dAV5;1Yjls+c zGqTE{o~+l3SCZ!F?{Vf1HdOLdM|!O1zhr6gI8%PrwAf|Xx~tZVG$$9i;nqtzB%^Ku zz7=*!crOGci^01U_2p2$L&uzP#01LuO;-cdmoK`!`!XTNsvqgOKABC^z+~-v3^sF_g1Z%Y}f{WF5 z$FU^hOjh^A_|95q7}@qi#YpZ{d^d4s`%`j!ynBfg7ombVajyFdkw<=e6INfTPZ&Ge zkTV^UDqX2?xe^@G;&+8k6*{!6DpeHp-h!aRui!RPoCRQ-V(_}%v2thFf-tBwBgKK4)k#}A)oL(t!j=%Iirp1l zeTw$Ezob5uwPjtox%S7;(;1bn*7x zF6TFe?1a_luPwxgO<^3Q?N2VwwnKuWq>X~5h|!*U)W+1{jN$Z5awT>kfwU|(ze|RG zNdB%ZtNW;RQxoNySRdDcb8PJAl7b} zeR1gORz{k=Je%>|_0jWp?Ll|e`p;-pMz;^2nvgaFTx(3zjL-KDr@Y2tCHwHOL-=hg zqsVL~Fi5i0o9*MQe%~uT&G}i4yV{?psNe9VGjvWCnm+PgPboKQv?&7`tYbh9RkM<1 zZI|DF%HeY>mQ0HuslUymmJe9EmC zwk~o=HBU7oRFLq0NMOz9LuVpU>#qX3)8FZl_540p>1?a`F!qgy}RVHwRp?DfjkDQrRQ?$ zr)%DC%|A~!r`-B%HBJou-sR}OUJ$7=l*n1!Lsu`qnWM#;%NG$b+m6fQI~Wq}>)1y5 zSP2K!@`+an<}%ZlzL^XVqe#y7$sCF|M>I_}D#YEb(=YpBV&R`l*`bobCoq1TJ26% z))ev43s!#3Exzxx46MH`EcXz{3}tbHGnnKNmL;>!P*&FwTjU#%ijhcJqOqI5IlhL~ z1^H$AYRNGb>5fOLU7*w#Cq4jd?rIS?8*y=ouS62Z5Tb=yo-em^&PT_t8M9hN%!uHp z5v7T)#rw$4zw|4I-lt8dHI@|9_Q<=I1NCH=^=wfV-@qMono zA1XS@O7^_+gGTzvN_J+qT$Tzittsvh>d8$CD({vST$qLNg0dePFgPFb%#F;yn9%>w zyNAo~gOrbVZyELl{KxYpKc6q7JfAIG^_bPpytQU{F7C<3dg>!UH~jTWg+QggdW3Mx zJX~d-5qm98r7~>TLN^VUQ2c_^?xtF*7WPH#om)01QWMZo4`mB&#vC`AwUxFnYoTq^ zx{wJUc^AHkvf6m0C3yr}ylFOGw2j>NEVqD!<|El6Ge0Dz-o%d_C!iBdTz%LV9b3cS z7c;xVUkP70Q_nRtrdlQCY%MbDY_r+jI6q^yNG@L1N4A*ihRc>Ex20R?Hoq^Ejt=(# zlw&nZ=azJpNy4Qj;=X)~L9G$^ z8fJG&qM=$HA~RpjHz~+TZ&Q%7r63pG&nT4fERXUE5>r}%`6jD(R}NjRQ*`XNg_+fa z+^DeinyE4K%^_{^ogW6qB-BIxCM&veCgMAED#UKwMR8>q-JO>@xg4s@gqo>y z-eVzW1+K8ZtoqKg7zDP6Qz&FzuaVxdy;bBQ?wjzhy6L<%6lUgvS}7_^pO|m{U`AXq z#AoaWGZb+YX5I`jfJOdU_A^G;cS#rLJ+^jN-1nDe{aKwBiUB`h7gyegolx@=b_UcB z*cn8FeFn-xe8V;|H#|Ig^5t}Gw-pzyiH3vRP~zgvGPs{>I8>N^hW`wqElCHGB;P zwd{vWXDa@VdJon6pB0A~eu5J80~GV0pm_fT#hU_(wx}7ha`|2fhpK!BgW7&`SQnbQ zE57riIdCNHHwQw^cK`7U{}dgiFY8Q*=OMA5W%izP-hI34H#b-RMszAsH99Ki{5{E0 z{6R=i=;Pv?_;p3ibFT%y>F!opIMD>9Wf;6Tcv?P;nUTxCeWl+zUKTrSPpp=>ocM(q zkyNU6u(g{WTThPhN=Zs>P*OR2ofPS(1XaDaE})?BR%Pgsx+!=H+`E`9GCtA0nzrkT zxR$&_%=vcjd*M}&jr?RkUmtP-_64fa#F5Bc=~ErSc1v+4ZZJs3vikJc`&Q*`agdY2SG|I};^HBtY@nL?VdJAGsZ9#g5!*I7Q`$D(8JxC4GaqN? zYm!)O8!U4JFZ;AxBp=FLNGqX1I^;tYfid;?(HaQ%+rMxp!Y3&n-6wc zT-;G)vf<_d*%EPOlUphMzV0Q2gRGIJTuarZ>my5K+moZrY00cf{TFWmytbO+lane1 zAK)sPQY9thpIk#ohS*z zw`RbR%gc!;B!jFo4qSZFZ$%$-oHp}DIYfeXFoebM_Mv0dY;h`?ch9rf&=;}J4$@bN z2)$}!RG>N~<#Tx~Duec@y+ufg56pzgTFJ16{y1KHN%U?QTP{DhQ7yemNR24N(3T`( zvzGN;sT%Fv#^tvaerM;BCXeFF&6|fmGc7H9SCw6Lgs1rpJ4INiKkX;IEIzQi@G*ah znCv7bo<&TsSaj^vT__eQvt-_NaXq%4pImug=}Q+$!@rnjmQKlLi8&^rTDJEV_jjV^ zNlmy{GixB%^FvQuC1m-E;$yGJyhQZ@F7v3&&q#di;CG=CQq`^)ZkCttZD_skv$Z}q z{Cqy_P7+5GA&2rOtK$@CbSjDYHGdu1ReF@E>=O5B`C{zixUK5FXw}^kbF(*j2*k&y z?=!szGQX27!&=3}w{>Fb11k%+7LM91T2?zhgI;^o#S*>A@NKaJ=F)g-Ggzs=T%xCLkOEdUJ_@`l2wZL#4f#-s?51C()k8_J?&nvr7C0k`l?w$zx)Y6bs`bT1Xzu9;#d+NP2^CnprxB!g1UKXSh9=npd!KxVNU z1vV2n+4zb=V6F^2vn-%^*lX^as_4hzFi@)&&$UwKN&g651va1SR3QR-w-neeo{xpO zVqmV)JZBA}9>BP`7YXmSy!-xg1 z8pNhrd`x@_HkICin7-K=mnJpSYT6>wVIfPp^0xdmR4!4w6a2ni2+z(UN!9%AYxIHo57MQB$`}lFD8g30ldbSu?cf-mP--{IC zKY$NWc$TUrA!JDutRZe$^fg3MIJCc2_ zu55_F!yn*7VxbRGaMStk)lp)g0>iLu4K!JXVkr3n1AdBPDESpPe3}`2JY$A=g|QQi zoPR_6!3Xj6 zZB&(p)_q1PY7*H2ctr3_zKjM6c60!*wUQ;RN%=WR{|FJLDsx5`g=zg-u7(3>qrQ61 z1*%`R#3QbzsKsFVUTb#qLs8t!gD})HTB7Sij4tZR5dnM%GfM|P#7lKi9dCbBBm5zD z8|s5BzlEgbyG z*MA@gAl*^6XC`&CPqzvQ$dUAAF>PQ~Z3+(KTbskl+kvrf3zUdC1&vf~;PETLK!q#W$Z0AXv_y=TQ@<(Yk&@Ww*M7uM4{Qs@1=ij=($y+m5Ps77`r}Lksso`ET zlKfB7by_gATi6AY&s|_?1H))!bZmYLgAnbG{1#?>l+)wrG;?@1Kk0)d(N4E*^l}g- zl%1K2`lU90#Kx^IX|+%1NKBXc_i$4omC8RGBBL>&$7G-(0O-a^a^KI#&HSo=$wCm<^cg&T^Q$lAMu6DeD}di0~jL3bUp=H zf#|9q3sMWf9$C`Lz^z1g&TMimOK!%|o#px5RtJlMz3dm|09Fe5oq~ieFnxJ^oJ9nF z+LdM-1`$005h&s}Z%~IeUj(M8jTatXXuRIYu?1POQ&3`7Qlkky zL#>hixhZH&Uw{|nFrB2l21e{?yKDFNz~VSMkn{bE zn4~2pjUFcSi49(jh8!Ryp!6Xt@(~E>7y{HX*cTxeV#C=3flG7%q0N%e<}<)?Z8$yj zCgy>+v0|_+02Ka40y0SjnUn)KpMWI-Pcl8T&c6=b6GwTe!^d204GgAJ0wQ31849=8 zsJQ$9Z4%25j|rVtJa-fV!%*_o!UWMl-Ckq#FjqiQXiOx>I2u!;DuAmgrll?*j>smHnE{G3P%Wkgax2YUi?$o%l{B2jI$rV;l9b2dL1kU@ovb_PR?g#qm^1p@_L?AU(8Fv1q4c*76mbbx2|RJ z!*O>uGq`c|7ciJJ7!KRVK)ZdgRO(+*z|x<1xD6pN$sT(vE6@ZWsJH$;W()W6 z0-5)ZXLW#rqTimy*Ge!X2x1>G@#Bn7HIbXXm<=v zSzYunY#|p45su)rV%((#Sc0mN(oMss0YFgJm{}&~XfzOH3KW;&hsf=__Qn!8u+DC=%>-#%UtjE}y6+^6t{~JcZI|G^fzVQkmX@%f8b{xYQZOof} zxRR{)sDHJJyN~Ys!5>>V7=#Q{kwODO`4SMhr)Ez81)l%~ zAO6d|B2GxrANQ7kT)z%!5f!B9zYb|})BAMLII*m`eEcBUz&`i@upoe%AIVVT&vdZ5 zw z3Y|J#0CvU#G~Kp~X zh1f$9Vp=8Mfz4xr*Ah-4MSxe)fc`mUXXbw9eCcJJ;S-gsXkdSOD-UFQK?U7FCP(3J zqH+T-Y>a(WmhEMw_m}B~NhaOG*S0w;+M%aH3Rx2xCkz&huv z*D1s2o-yS?vEf!lJf5eG?N=> zlfK{57HED;J1qe{{h$5xG!^vp|MXLfn3cVL8uLV_Q;L>iTcDPf#;@`C|2l>ILZ#mx1Jyn^49OIr-)qrTO2i@;N5?T`&6a&qIW;p#Xsr}(fu4ee=R*f zX4C&JJv*d!Tgn7iczteuJN+>N0zK%;DuUw{#0@Y(9%W?k!-E0gl(y+k*==OdK)Z0D z9Rpf{(suS%HPt`6g{(T}!{56FVLN^A$Zn_i(Yb5CklR-$4F5GJxgP_M{cBE2{>Pj| zPup2(NeEKvy|o`D=*VOF-@aKnHkY`D`%!-1xMTw6c< zdm}rej(PK!YdLjdhDk=x%redBDJyL7rEHB~tkKXFNdJ>HrT<{+hS$L+OufDIc_M1_Z%jX7RvLGvX)W~2$81t?IB1&!BJrx6F;uy5QOR0 gD^`I|sU7prbTE`Op8-E>e%+;Nvr+K*Btk>=e{RYSfdBvi literal 0 HcmV?d00001 diff --git a/examples/policy/converted/expanded/P001.nt.gz b/examples/policy/converted/expanded/P001.nt.gz new file mode 100644 index 0000000000000000000000000000000000000000..a42f6da55519b7e79042c80ec523b873627b1bbb GIT binary patch literal 15035 zcmbVycR1Dk|G!zsh;t4y!#R$ukYts496C`bw8SwIvLag?m>{`LaMh= zp=c;>n2|8EIk<*GZ+xBoI=e$$sE?4pKe|3#vsLB3eW8l({u+9RSR%v9Q_R!4;JRc!r7nA5fL(wCl28l-f7uCN@?H)xJGb8CnZ=z9%BvUoP$F(Kg%O1jW}_)w zs4M2o$V@EoCNE-)pXv)Dem%pSc#B7v5S*uI)47#*Fk82gu98O_OFe`)TXiuknS@ExPn_4(b{ ziwZaCr5mz>1{b&0;{AL&Yg&fIa!Wow9e734-#@qdZmoW9wYh#yb+FQV$s{Pg@~*2) zlD&Abz1`68j<@#~aPG^n>+r(29b5Yvg3~=`_oe!hg<>B2oP~M&)&+aJ8ZY!l?pAAw z!R_)O?MjcA8-`hfvD<}&9FxF4mYZo^6ljyBzB}<(D-d#$ITL)Z@m6vsqxnMBnpvEg^iL z?dBO?m=*B8_%>#NZhCZbllV&)}lZq45Eu@`S#Az3w{OM=w!dR<5R#Z$ew=UDF>5 zpS_YiEvoap2D}z;PaUP3CO|0FXdGEd>&;C!{x;26qmv-6Am?H@l&3nv6$3qs@Z`JWVigEHD&oHB0o(1!Lm+(1)u^jV_W`8Sg`Vi}7cxO8zD<{PR5#-aijVZIQG8Bqhqt6s<9dng zLpzPVWWpts%)=BMFdo5{&X6Ik_-d<8^xNlYiiYh1)1mS;EDA-8dLDc)X0+OF7>igQ z8=#{#zw2?Cfpc*#`AZ@Ue|T_pT!+!|1Ff^FWvM{YTwL28O_thE=V@P27YFpzdZp~a z&m$k^;5g26-+tOm+hsL!Rb)hA!^SlzNAGh!g_>u_ZU5;n@$mIHK82YYckNwL6PD_t z>vLm}yPU51)u{7kE+uxA%_RLLK-i-wQEyYkSAk|QKVl^ftx3i2-x*_C5?DLFGz$Yy z&9H}*F9crXFL}wEsqIlPq>LKhSWT;GdcY#s7ek^p6a@&Hws)djVtwmGhisl7&OLq1 z8-jmd&9Rx4u>;2n2oT`1FJc+j=u&(dX+2JqzXKc4I{35P{OEwzm(s1toYnehe2KvF zO)7CvgZt#ww?j=`H)v$Dlk-!zge43DDzZj=g{XA9KgE0FrA_UDyIFfBASkGUhMCQc`V5?sJdfQ(stOTjm|z5)5 zcN082A_Zz1+5RfvYJGmX0Ps#OdbG{XqYO0~{OMgou6OHNQ|aFN5V~7eMz7J`JT^Nr zL3)9-($wG!+*NS61B20Pj{2-)kvT2N|cxRFHd7!)~w{pt+kv#905 zRk2LC&kiX(w82$Z11vXctT%a?;H-G#^Rq{!BfsYq9dnD5#L zN!XeDpKd07`qhT8n>F{|WE4)Hbly5b~?XKHepdPEAS{@C+>|SmGxYwxorc*602oE0p*dCK2gV3yj_N^RifA~hoSAQ{9S#VdL_3plnV+!YG z`SU)n=<}7VGVJl9z<>Ao4`HTfP+g%Q{nhGHq>y&1X|SEi!|F8iRBw!G3H!L%YL_lW z9mzGcX>Mi~m{EtUrOgQWhWohM`VFx(jo|rNL@ZCb-xn8}H;m8c+%%fM%=rBMdbFa! zLiz3j#^TSvssZ=LPz=4M@wmE+$B{bR^V7)WF_QB+7YsYa8_i>^1*L{!T6-#f+EQ1A z^go|B9b&n_U84oAoNaUJ=IxC^EO5*;TFe^$WPR3*zJDj9h$N67`R!^(~h5Tn_q6{o7@YY;WMo& zRb@3*mBeOu`meiHp>A85QRNHVl2K)5z<^Gvp6u2AB)byVe#5>Xr5^DT8AgatD`5fc zb!PbC+lo(xt;?P}%^UUFSNmo9J+(!L6^Q-MO0W-h(iiI$+L@c_y{c9h*1n@L9r8!o zu9q@?&09$^-uj}HguXxB7S!s->Cd9K9b6U~pWVunTT3JV5+eS92iiJLbk)$PdU-9! zE4@(o%ISL0ovM6~W6LyT>E8`2?<_v9$X2MCJN0rfmwT`w`@>v!#iRc9N1q;9iwJJS zFVI@@Y|q?aXK-61juAdb*pEGlpmg40(@wRz1J56}70+q0FIgZG7!xm!UmHD|p-GuKxhQx3Vp5Ks*f0U|Z9oiF zmxFLY;hS5cywt_)cBVGJFd2xKEa~In#JkR~-{qLpzp;q;giNDjP11SU-zol<`6z6C z@Qr`j`HU<*sYk9#!}yx7pB7qk=ZhngQTBE@#ZzzIg+iwD&$yzKK7=lB#Sgj?H&0YY zp0Y66vQz5zHR!XLir~AV<#*DGH6*t135{ruVRC$tz+7|-MdqD?VmhZS`6|P<>qxst z^cgs2^8yr+V)H|g@5E#LUmWZ);V-Y=?R(N_r=Hxyo`ODef|rr~^3?=FyQ`xTWmkI= z`Pa_q;_vLq>YgEsl4M+xGZtibPB~qAap{}OAr5g`MJe`%_@{C3q1epu7}xNab<-Ch z_bun%Xc@y8d3VXvf{`+Y)L5{Vu}|eEMUw|$6_d}6@vqeB_6626CUIymcu`c$WUS)X zFE`fOF{!ZCeA*;B1fu=VgCM6W^|29Xm&F#`+W+PceS5cvw2~(h9Q`dWw zl3Ej2i7tQ(;S4gbA}Y|Z{J49sxuuSeVL5GE)mdNiUc)U+nI|Ld2Zjn8sT=dpM6r`Y zF}ES}L-GEe8C@?NE65U4lzWf-cz(TKjoT&Wwr`u*?P_WIik3Hy_ieR$pQk_D=*v0O zBtqj-;UvC#t$U=?34kxYq-U+I*+=1GP?Q14qpOegF_dL57c3IQpP>wArA~TEpQm|I z)At9+cRxYem^{ik&+Wp_!EF2}hr($=%b3iC#Nzyr|J#B(F@x|pPYb%#vh}p<*oyU| zeuI^B6mEeOCNhF2&f*KYZ`rcC_g;KT4X=2-(RY$`&WEMfawd+lojrih=^(z(?=L**lf}Ep;>7|vYI9IQbJDl}n8LFSRa3U4 z1$8w37r@Uh?MtssdYuo&jalD&D<(C^?BR!}1iHk}Ip zwy{mkMA_sMyJe-|_pVz$hIei>N#EXGLDx6?(=Czc{8M;}%KeX*i|Iz#X)%MzdHB2! zS7Kfpj+@9>9RcFb^^;e)@6_%2;u}P}!1I62%#4Xkjc@EiHh*+}Tnh;Y#Ap)@=Tk zv1_#dpis!2&OK8l)#0&(5wkJZQ7P`FI;Zmbr8kkoD(L$c6LasjB?nD z(Sqw#)FRnh4&^dRh2dI|v#YfN0*Ar6%a>c+jyH?@;H1cdSW90l1A+w>*3PO@^5si*cz7)~Hu{oYgTc~2lJFh*i4VsXJd?^C`4|GH44yfq zW50kkTN_e*aY6}PIPt>WvlyCMqWu)`reEf9N$Y?#OoAHX@B=w%sIjw{f8e; zoHH%vlCp2Tf>>EQ&qVIFb+ljhS(z=6?-CUJ<7q;r?O#5)^GqsMoT?u^M zkY)sa5Uqt3MaMaH(?A_qGs@=oNK-vP+TjDFrB&c!H>e{^`FNL1U$TI=0@#N6(?c&S zn+5`oz|ub`S5+~wtBk&m==I$CF}1%Y=RSvEhZL=#WcI@*LyGtMBXP_aoQnBI)MHTr z`+h-QhYkd?Bs_W|oHt*xfcHZ@OK_&UP`&k%nFv>IHyzZ0C!>tGhwIz{T#UcPCGbDt zGII@kKcqa)%1&0)mI=n)HkryR z!KFtP{KaN*^W+=S5)>Kn+;?>3m5>sr2^LzW<6Tl#9({LB1hz&ep>A3(NDaG@AepWPJ$ zQ*fg@%>qA3r8`EW9490J!P&CS`g;b#a=<`@4;aYv0scMnW!X27H8Ba*KQ$2Aa?ON_ zKQfS(1@K`REdiWjndXTihw>cisL}_WH@$B39Y@~$t67bdSuX(Ky3Zr4DS=VF>{eBZ z-^a(5P45_i!Mc;I%A-$xaH08Eka$+pFr3pRTI5GyiZ9NI{Y8l@ln+_C2J*D$`4LTB zl?n!+_I^@0jm%lo|EXBzz=tC~X<)tlaPWHT?r>hPjKfpwy-?*m5UQtt7peom|0PuY z|1?xJWsO82#oJ3aoJFuW+l0x!Juw_TAO_KcVo2owS15eH{^uqlIypzOxbqiO{+0F^ zr+&ff|I+D^U7Zd9APxdpoMFOb`5rWe1JI5ggf{O$7m=Vr{+0o0Y?L4< z_(x6A8Y;e|Jen4qToj&iAQq-LGtTWLeFPji@Tbx6cg`#9OX2^J^XqJ23p-96G8X%< z#M0?|-*%vcNPg(Y$Y?E_p;boJ?~JYRVsWYolYx88ksn}=@*s0R9FFmSaya8Zb+{H8 zt)PfcqxFiPWksDUdjzpKvxLd_d*DC^!14VKT*Utb&h(GL4QtBt>c_1ajj8~G1yo>( zqo-w_?Piwtfy{br{JYG8{x6wD^QW2h$nnu*9s{D7ga1utWtg4xrKe{ZBvePsC?K%+ z{u;aziAM~15}_1G{Dad*Je7{8cu6gO0%z5+{XXh@GKndk48J>9=`~*r2^KmWT-Q{< zCxIgAA?n;{QE;RkC;tadbyL^f{w4s9S#C+L|ECTtGA-sX|7TZ+8qEPd{WzCLrL6GL8^GXM+2-!O z@RB+ZUUv?J7e7S8+?yPhLxZejvkt&{L*64Vus$b6Du@Gl;gPS0r@$#coHbAW`r3~c zd9Jy?kOfrb)qx-zs(v3Sd#|5YJrx0C_|c@?`_;*VV~+cENC(Er_~BX_{7%?kA$9C$ z!~6_<5=v{0hNeO=JF2MC;Le-(c7tmm`9N^_9tf_E_g1?%u3y4IMypfp53e-#C;#|L zQ-KpDnXqfH0R(183f1t=o1S~vQV(DYIEc+~?>0C9-Vd$?@SkJLulPT-%n{FECex$@ z*=Izf(a$JVi*#4^^+EzqL>aMIljY&%?DOvK-ZlIHD?LnQ*~WcOm8kZt2Dd7&nyNvg zm{=!a4WogZ!zkkWNKheY{VZ-{b-Bv^dP2b2vr1t%ug6jT#X{uXs=)<&1g7e2k4@Lu z@#xHDJQ{kf0jo1nsQi9+cISvPEHr_>7I@wG!1{*EW1pK$o{k;sDk z8xkG~<<*9A!^7xc3VrIvXzkf%1!qKOL~yKY&diFQIx5iM zt2t|2;&|NV=s>h~)Jp?X3t4AP;?<+UY3=!%A?8O*G`$>L8Z(eXV)=z!8n!lH?JWDh zkG8a=jm0r%i_ZW3UZ8>L6r`Nrh5J|uMlXIW&Sy2=?$LwY1N8iQa zI3%fGtz){B|J|w3isvb^CF8oVg}C1949|PQy`)wi?tqk|E}XGDpWioXXC>Goy1{H! zN;)8wbW*#6Ci1nM9nH0x8M~R@SJEi0Zqab_(&v|4OxVxcS)JJZa3cv#AXAB+{mT9s z`M;h~Qn8bzsPz`Vf5hHf2ajkT1Q&RB4tC@l*j6WG89deFn8`>&5&TCU9{+4^4t0N? zDVs||J^Qeioq?|D18T)_nvMS*M<^%h!C&LLh!^=uO6YB8Pt7dyxJx0!US=6>4oDKG zu1H-6hQsP@uy?pO3Tc2mjE~GpAyerb;^RvsIWmOfWRohNz z_j?zGsV--!zN@M#BmH9Fs|)M5Y}bBmGKT_5ZVholEZLnI1;r;zmLEZdK8FTSZh0b< zqLYuN&VA#IILB_UVqev|r|}8^JUCjHoB6y#b{5%dkGTZ0PexvQH9gE zx{+t6UX4C+VNsV0o#>YR7;#uYPehb~d491&_#&gwcdNVUBU5d@wB4qmW&vj_i%dHy z=)9v5Jzt8`HXQQ0V5}6O;(Jolkz@(!+XY#q*;VYPH~12DPneR4h-a4=bzbbL)MF zS*cK;{gg6q#N^FzTR`_M?XCU6*NxA!hLR)bvitE*WvH#6#1B5S^QPn*@EgAyKWEl)j+;h>Rg&E>96 zPwM-!Oy$Z`iGI0>@^)K3F50ye_NJcNnZM=pR$U~^&E+GVyr0B1grK9=iEJP9PN#9> zUP*uUHvM*_Fv*5W@OI2I2Bs57GSBINS!uo}&t8qFTkv-8jh<7Q<4~Gj-6S-Ve(uBL zg+X9q2!}Q)^JVU5c)%^gc{aTwku3i=W%Me`x1D?`5OG zy6y>-tNdTgS+fWM6-uM4GgsZJ1IKz~UgRv67C1lr(vBUR_|nad8bTSN=4bfZd*pF9 z&rWR4oFd%Szj1b`?^x=_t54dm*Rp_1;Ft@F#|kfeMBo4H*l;b+FR{{`-SaKrngCE! zusNm8el~(@)0Ci%-LS%{M(F^acCiKZ@ZWu*jhOFAy#xHB6W+*h*V2$8XcI9$sgrdG z#PE)l#)?7hRI6Oq&I>s%e6XtLOBr$owdkh1g1ylvgL;5LJ@4*L!-L6#PMzjjF9>k+ z1zinLt>vJUEeez^67x}I<)D|1P5Gj8)qA- z)AO3P3HgV_j@B~S%8m~{#N=?X{EA2xefu<~0qE1$VWzrLTMQJ)wl8dV**yTN{rm<#-3X zgZ?z*O9>w$2bTDhV%k2X)`!T(kz5fOR!X~a_=`5em=w4XH7(+iAgLxJD8t9i`WRS{{z(Ew?J2GVksG1l9j>^lWnR- zH?{GQ*d)9L5?;S+8z!+ykT|U@_KZX&@eK#%bZ)?OuBZ?3C2W@{w+d((!bqDxT#Ot0$<{rtdI^AYer2DVDp~7WhW1R(gU@L4%D<`q6Ruf&vot4 zO3--N(3Yy7&q@#m?vq?Uoh$xtXq&%*MjSxJ6e}yVqXb-t^wD~O#5Zx6ncN`K+kwQR z8had_K?m-WxBK9?(0DRxs(*#1xXxOhkfFH#$a`Q}m-WaVybc~PRhklvnSzQ^9j!{U zGQ|{C@%jY^Vp`j*fO179RvE1bDw~POQo#@x)!-zyKQ2!Zw=Evj)iSIT^OoQIGTkFU z!jXTtYb8jGX0`ELGb1DHKAC8Yme<5*7;?+j!9c|hAp~DTTHVp?LNDM zl~ju28xZ^@a0eteDztL*GOog&3@JJ;08O1z4>jNkrTjaib^mtT?uBfGPKVhlR8@W zKm0xXFMo@f;>bLK&3N3l zMEqR4VdjjQ%wJ}!FS{DRsyEj4i1zx!=Sz$lD7&f3IqvJRD2rXT(x=KJo;7l7Hp92ZVUk1141oVh(ZXr3o<ebRSddtK5N298s;d44^cLU}2_xnjiwaheKyqF9l<+nN9ig*@m#4c7P+I-_w93OW7dM zd|sm$D)5(~yojU_yn;FlL=`J6$qmJo1K=x}4O&<|&?yWymvt0_O-r;F9E6Eo30em< zfuPa)SYYiCjwb&-AYUv9$a?t%APPdkhfp^@2B#7^0>N?c;mWDtH?euVh(JIXklj}b z0DAdK7Sclo=`jbUil{yV3eh|q5OIyMS_CJRf6$wUwvoPI^y$r52s1VR9&tyI*o*%r zu9x}0#4Q`9r+hdJCQWl$=L!T)<;f&~JH&@e=Yrov=I@?A-KeVd6WH#9ePH>sa16Y-9D2Bu`3oi}e@e7xw`^u0tNujV z(;+`#R2pf^Qj;-QoS$3DVVZpLJp}7)y9lyAx?zD^dmzt3E|Wk)tBtTs&}a{J?8On7 zjbPuG=A+@H-Mk(E0O2nt06oKiTaEw+$U8qEJ2cPBXI}sc>R7Me2>?$H$A9>*0D2p$ z=1fWZ2(!BXNC^D*j!`19s{h_G+28G0;e}a}ID>(nWmCYYmv9VV?YI_}QXQ)*hbfIT z-Id_uAPcG<2qLco2B0^QL(!!tz?X1*B|r3N_8Me z_UUXl&JF!(Y=4>v%br+YQ9;s=z2RI3`4b5PX7BTt9CJt3*O_Fmr!m0Y2Vjy`-tK8- zn1}7B?_ME;T=~^^*SSqezU#qw$>xz&_>WAOr)fv0eBNP^@DBw+!E~bm9I(H0V90DY z2VR^SgZzvN401OrNYSc=MShio9O5<$NBr?Hvi}r5Vdc?lt5G6UoAL6ua0EFmoE<9*v+i!56r=S;62Wc!f?fV z+4&1+Hb3I^Uz|w?hjxFF!(9sd$4*4mv9`an6B`M*jpn|cxIEbxtPMHb<~M@HRvl)y z>U^M%EpfF~kQ!ve>%eYvkwpm#l92+RYg(BMA3~-~3Vcxn3xv{#6!;WxasXtcz-RE$ z17(sSue)-KH7*o?T7Ohw@6EaBIrw>^0{iJK{UwuNLdceyU>09Pa=lwnHgj>pUrXY4 zir(dALlZz09N)`*USXIAzhgGqj?+#FD4@+(==GEBcV+8$oya4=L#X>D5yq})c3i-aUVK0OB0#37pXo=na$HmEXIoHE#(MqQ z7GStVe#IytXBnZ^f6v9WBllh02lIEL{Nmz2Lg%215A$%H@gWO#>E#KVV}b)fjlu}5 zvHd7U))*rjtBR!)I*Mrk`VLN*{9M0T-^%sbDvrx4HLa&VzWvuQmL(OtvjAGRv<2Lm_(cXap@P{h~0o2H8Kp_E(@WNU^`!S0;I?_GEGwcK$ z19ZMEuZ*29rMNv10&IjL08x2HNG52k2a(HkM8ZX|&lf0hlMDcA5>~}p+P}q=ipXMs z7Iii>4npsIfO62}3i*UL!Wiro4rMH=S~rsP>?OcK=qXffEHE3rV=VF3+n zh8aDL14K0*cz(A+ugl>^_!U>>5cx58IMRYigG zGHu&!D;xrdk0T`Y?N-A>H2Grxu76Y+yAf0Sl=+vtQ8>ud@V8}p_g=DJ zrg!fpe~(y78SDJZh?}Z{*hnpZNHvxZ_kOpwE*YjHnPY)b*MQ=uc_Gq!oqmn`uiZ}n zkev}^s=HVF1rWQnANPrl)N(JkE0s}^u(k;>{U4k+N1*AkqE8U2I zH2fy#IUYT~^T+f2%ZKm2*XMm**L}ZU8)Fmz@KmS44hvVEkM9Hg}GYyF**gKz%lJKY1jtyI11CmE9+bd`J=BBHbPEhi&n5*K6eq^r&bHYOeC zr}gOO-j+(yj71xtb9WDyqrI)kP2cmAk;zuZvDuegSdRh&ch+(phIgI`;bRwde5lfG zBuXOf?6Y%e*5h^>pz7dqN)e{LsTSaKFy}UO?@L;%!A#W$s_m4N^{U;`PZQs_GfrBM z2h8{PdcPg7olcHKz3Kn<-KjmjDy6I_MYp4BTbHP?b(69xxh&(zciwJ<^@1mmC$&%r zdl?|u%ge=JV6|g6f1{wABQRX=zky&^ARTDocKT)0I>?8)*g5MPfGQW0;ni%yvkMRmIM0v2?V@VF^lt_~W`P69E#Xv9ghLXB2&ksF-{pB{{L zoF5)9)}}ZI`nm<~y43h4pY56LW{CN<9~_MIzL`E+TkAcx^EugIRGV?+IsIb3V|8}K zsCK?!I0m&H=S^!r-khGz&cGeD)onw1_{Bw@tsBvBzD zQ8poBzgB+!Fu<#^>B#$Rb3EY9D7|FU?xWL8Ugt609Mjs_z(avZL6}VmNaPDmUX(sF zlh=GHsITMNo332h6%?CZU-K@+0GA^-YI)$ z_vL>`!VtI9T^A$pq5fEDzRR*QWz;&iYx$tBrL<(>z4M&ds@~OOr{#AM?DQu9ifs0$ z48Xj*9DHi3r}`9RA(<2z>{82HnJj%tG7U$%y%qi7Wd%9afR0^Dr+4oR@38L{-;J1x zxyj!Dg8>u!8&hKUtq%&Q4;HG5POG8+k>fi2B7={7Mm1GtS`Cx3o9q>$gb>V!KgqEC zsuKD=6Xiv@^~rkD&`B?Ei?gn(KcM4;FlL>L420y;)Ns|b@gVG1uzlB`khEkHHD?x| zbgq;xu>8Sm5OekImkS=FeRz%jOqDzJz~{C^@kte|>;_*l9iA<*xC&Qh3S3;?SO|}{`rO_#<8?P~Fw#!(QJp1@5oLD_`{ZK=eQRyIYcEp79p9Z+G&Fy~ zmAW=fC*BY7pMX}Szvx>T3K3bo|moWQ!gf^ylbUDwv*x*(3gwSuh7yj2xaxE^P8ne zk9r!-L{QFuR3Ys#LY2M&Ro|&7JWYsR%R0+m4`1{RIO~esYq-f{Yhut@9rZDCI=AwWuKM7zT(D-iT=O7$gkb@!Yq_rN(#t@Z&pV)P4T`77WR7z0-Rk#Z!&Ob zb6Q>Av}+c`HawW^2uyPhUYax7_C4?X8W6C%=(`h8BZNAh-f~`Y9-_Y}{5qL!>*^ck zsUFN+V=)>Iu$i9nn`l>xUC>?}LC$!RM$rmzd| zT6$U=8=Jdtn%!{yzFuS@y{sZGq`fj}f);VmAyYXR_=7QEF1!R=!>vhGb0lSbarBLF zf}FZPNd|Cy3Vos?;~A&ybm7+8Zf<5AY*~g*^$&Q|-nmr$>`U!5U4-SA6~Z@7;_aY9 zzMC)y8iq*P3oEdRpsts5@`=GC*Sj4r4J3o_@uZAJ-R1M)$J@+z+dVPNa_-=6P4HM^ zkWXu`ZbEzQ@9joxIJ~r%iqu$i=a-Q5GEA^;krXXb9e$I8?>~yCLS?Iz5!jO3E-6oS z0F4`dbx(nEoOd>Of3d+#MCQ9jjB-Luv~KmJae`60NzqrKvtz!;4&sNgcfMYe|H?19 zjdeHuKB60|TBWKROUq^@H~!gzxGlAhI$sHa-_eD=fp(FgK-U+Ag`7{|uq6_s2KuvLP5j0~3v+_kJ)ft1Ggt zt1zlQF{=gI?;asL;~*$CB9qHAeuX~SSK^gofyqMm{8+6Z@x#A>ZpAwY*_<%Pf^UT@`^El9m8V8hzboYpi=8LoFi zbGxAF1!M49t?6CNQYsY~Jg?JGn5I;ZTPmI&ADj&#AIrh1$e5>vq%*ASTr!?Ka$4A_`^^tO@%&V;Wb9JxIGwt?T z{Gn2>D&s$Nw<+sl?mD#S-5f8uIjo#wi#aA2Fj_DhE29Fe;-HFEjqJlQZ*yOph_6=i zn4Ep<@W=)O(UP`yP_%VWH;WHlkyWgN2_!MPV(2%opZ(@F1^W39k@i1D8ls=uCx#ng z;)s4;hE;YDWQOHgREachH@GB4Q?ZuCjnrxP+YfH_t%9IQGI%~IdagwZjIh#RGJt83 z4|;zMAJ&4!UwVIR!pv)sFRMj}1>@NFwTUUhR{xQ>>I|Y+PRv(ZW5`F-N8)3#TS?OI z-EdKV(4*t|N(B|9glec&%n5 zZ}i`(8ezH)5Bb|@OvjmIJ$?`3xsWwhoLD3O$(Uwqao9@$#^`l;{r5|wqkc2`+V4hp zUUp@2?ys)Qz5XAHf?jzz7dVWM)W~Z2MA{k>w)(Li zVwSSj&=GUJlX<|g4^}<-?v>LryngFaUHEV6NPkyHr!KRqFTCM-XCpRUHrS|^6uc41 zs4g?cLe@%5LRhV&i>^R#+}oysd$RUiQfP;ibPe}|cAr`VB;MTjV(bw3TF zh#heqDiEK_gIlqAH5tT`-NF*!=&XH>A!6M1qn%#^)ZB4<6eQ`GBbAXWT@=&)KC6Xg zs4gNL<#5+*L>4=uT_iYBNWOscX&WgEL!EN*0!bw1U5K$jafjZHi;N(%D-f5h!sb(f zx@Sx-v20ZM5oXVz^2B~2-Qjeyb5gzN`49zF088O9D}?`fAU4Yf0(mhzTB3-47?hdc z=m`$^Hr$qp-{^||BpDa8yqJf=>BLMaml(Kqv3T^mgA8h7ri}UV#J%qLJAX`|%ce?{ zz$zKu@_otVLXH&i=rZOT4@2Okn(x*nG4H}4%#~N(pOiRnmd@|nK3!G?L=Z`_@JtEl zX8yp(^BX?(&foAA#Ed+_;A>*~iLcEio#(&BXJT6a=yM$;LIDs_@%}Y4zkw>cPD9eX zDmGq~_Icu15Go{@;AxvwLh4%M1*c04JZ-iWRSyKG1j`$?JggKW*}NfdnEy^j${bw` z0C&xN-pNBGcn{wCerTFVjuTN5piqXoRLAnSIz?rw=Z{guAR2k)l%P-FaeDz^H6w6G zLn9|q9L7DndT|3#O~b(&Ah&_HMtiRFp+%Kz5s!j^;~eB~dAS87;1WgFD%^;x+~k}- zAyEEN=fJ1bU)Tq<%}ZiNo-%=Tozlxxr9}zBy0)$w@zg?`!Ki9sZ~tObOdO)c@1-sv zB+W}rhB+_YtpxmZw+cKW!@%0|=x}Soo+WeDGtiI!b4!nG`T?D6i z^JaMiS*8#RKLMmh`++x$buU4N5TX3WC}{ZMGp%nT@fZ?$iNZI)_s_m3pnX5rg>RCx zB#XGTT!YU|sJQJvw3U;Zqfy}@1b4~G#8ZF25=SN&jUisseKjc)h)IN!K5O9~?_-o8 zc7&Xx1*GPI{DQW5%ME-MYviD$c?)q}1EZd7y!Tq-VSW3b0yN4t$0L#T?YeSU^^amI zz*kji{ho??DC9pRC&M)7bzuw+*v|ux)iGXPo70CLlcY5DceE53r%t-<#}iVWPZqqn zIar+V5}I5_4BcoZG~%12lA*7-ErK|-ulQIQK%yJ3%@6IQc&%Z1rH|7yI`!&>X;sGGV^Qh z_(E#}Ted@5DdBUURNOW%0bctJAouS8Vdjb#IPz2Ac<+xw*T5N}aM}NXUlhdUb6t&Y z^ZgW-bfKO2RBp`lyTMeWE2lnuB|_`>dZf6vU7Kxqy`!JS1om=DU%!l=*Kg5V`8|4I z@cm;d$+_6asAml|@n|lVTmM7yM2RDC|LduD<`rQh|KzA$X+uJ3D_B&H4Idb>o0hZ` ztCA0KF$n^K=Ve*rsSmCO2@S#P8 zD?2Zyzxfb;6U=x?kTdUy{j0z95#c}jD-GozcMhDd+iyiNG^h!w555-#cg{Sq>zqN~5OnhcTkyc$iGj$>P~sCiIzB!j9ABx8w?noRqA*$&Kq}YQ)zoZ<)7C zWyT%Yk5r>?dkIO@Wl|?|S5OATo$XB2lhFalQSfA(YCQrz znX8ZU*4ek-z3=H3CMqO2U{w8~$*r<6M&*K0#}Ss+ zs&98GdZCQ9A!VK|)peJ*Y?`I-=av3E{ryUxs*>X|SK|KbN?D4~%dfd$6?g$}!Sp&J zniU^eb#JCoy!)i+Ko*X@xM(2GC=!}z9^FjzoTnM~t^f(NaNcsh$D>m~ZAz)QH7eXd zltuW1Fr{XSVW_1h98gO}t6cyv)s`a@?cS$beIgf`r;{k6V?))TR{V5N-LJU`^NYlz z2PWLk3PF;t9>^xL-u=_#@X@ymWK>bSSS~Nb8DIk!pT+D}HfQ?KluPn8k7=gV#L#_` zwMU2N@`tZ2*ynNe47Se#QhlA%7f)D@?|w;JZ8JMrYxD~AI}gGFU;jqKZl>I4&GLj= zNX=ABKru0t;APbOd<0pfs{0=>!Qm-Meur%Rf>j9qEXfB7e?}4YJQ@e5a?F2oqw6|@ zw-vB23gRl3tslf0 z!+!B5pe=Y1=V01~NKEMIOe_0s{KF#wZ@xZT<%H+m6l@Bwm0x*E>E(SE)i5XQm&Z+o zD#dXqCsL#aZ-#YJI4~+I=UsQex^KD4AF7ae=YDmTy6(-x>?gUf+n=$Ddxk>@lUTTm zO;1En)1t5EO&e~rmP^9o>WqU73QX~bifME@MJSTF{#brgSsl)9=@v-%X-j#tEkI;F zdx>(d)wr+!9qIO0-wZYrE@{vH)CMBGWyvf?GP^vOckpntqkxkuKEpW(ys`xZ+ZGKBI zYIb}uVz(%3lZ>2;-tL_&XpnKg$WTIgWb(noHScvZ z^8VKBd!rM@u!7_j%hOegUid}0z>90wxSqwSkt<#ZXOV`t$bD8{8E=V_uJNUrQy+e? ze@);WKWYE{&+19Y!4jEKrq_8iS&m-@#qdc5BLMw*S|m`qNHXv+U8MJxi>>xCi8>jb zzQuMn4qfD(r(e|hSj|X1{C7SLa`J@7K7oH$4-1pd+kO3(h`5!Lj_@sYXI?B>7mfa# zxxs`m0NwjRESDQyl@5j)kp%2#i%!SANx2<)gpSA04o5gN()Mh!K8UYYfek|8LRs=7 zjCDhAWCD?(`18&xPSK#;&Xp?ppmSv?tmv9{2;l6Ydqe1XB*n@A3sm4tLl8O0cuyYl z&|I@zV5n7#WVHC8NoW=o!64(DR@iyx6U6Gkj*G5zS<{}dCOGJvnkLry)_AFVm=WbM zt_5+fM7OXyNrm~)th_Ydv4!p9l{hoqp!3dR*I|<{g&k0WW7^WeLWxL;Ebvo>z)!1F zN#}(8$I{;)DPd%e`O5_^MOYn>LuDn8rIjI)>gId8g_416puGE0rBaA<0g*{qpcUbs z4H%SuUQ7dDA!B4ISS8z;D(ZOe{8*&?%&hqromGgHD&??qSw-j|qcm>a)u(29?(tkQwZte};+JnV_1chzOEK+|aKKMy1WnMp1W^iM3%+PM;ZrI~xW z&8t<4!RKL|b*pi6YDMtHt_@#hz}_KOuVj3&dv~Q@4zm z-ja$+)TCkON}x)J&w@AzModVCOtOg2VxGWrCBm*0E83v7^CbA*VjY)3j?2h%J1~LV z>mI}s*Eanrv30dd^%sd&!#N%z2L&C5(dX%az3{`SksP1<@3z?C(U14SQ(=#DU#(Or z{4i{^iPu8%yQz*})HfZ&|*3rMO_QhQQj4mjD*>Ya}5e=T&gsD$H4^Y#Zd z{^NBld`zAW-M3DE!s}Y8Qu-&nmVbe#7FaQh<$*qmc!Wpyi@R5cOj3TjYa0{xlU@ks zbAQtNwIs`h9X!9%`>P~p;yqqc2^tS*E|`k8cn6quB(V?Qs&)` z=7KGq>~e}bo#32t;R6Lq0V=_kpj(W2-Q7exRfEixC$v(ICQ&lw^q#?ykFasRafXE0 zI?b8n=a}W>=dem6EI2^%@ys}-(|5x>?#YqK(tHy?zK#fOx2|dWgA0m7?Fu)YEwG{1 zk(3r_n3@gF!DbA}VU-GJzCPhC?^>+Y?kXIvU*tVIooBv%#abR5K@LrBj%~z++`IvS zHZU3YNW%IOnczv0pYO_2LZp!xrOYTU0+>JvKozM4x`jZ-QOzNcO<^XKh;n)WX#Exf z2&#f1084DD$m_Dm2O@|&iQo|6btoUqlKK+_&NmE%_3^>_G?`EdNNPk65W7Objo2Ot zr8aPa^CoBxOvjfWng2v9C!9@pNt$n4Js;PxT{*rcA z7U}b^()eM0_y3X>Q~|m5QyMm`Z{!zg&D~Kyxb4b_)(CK$1EF&t+ryB0QUstH{2drJ z*gOFblCFg}z+Vcu4%1q7p@m#i0Ol*m`~fS7jEBJe+kotFl*h%>t_inlkv6v*jbTV{ zN+_I9*agPL2jd!spc0<^pc6FumAFd;0g$DEA@OCAGK7dbV9*bo)q?OaGZGk{MXw4H zUlJ)pgShh~PZdcKt~h%0KoGu8v30VO|u$KieQ~OfMYxN?saJg96kj2La`v7#h-e4X}Q`aw(*#Lo-1Q9_Anf7=G&` zLBgDRiSJ@12}Q|c3`J<{By;&B@x z2%4I42ckm_(OCt~$tx1WERmg`fY-x30HA0A1n_1wu#5uW!UDYlxdoRrss2p;i3!fv zFon>=M(914LP}p>L&&@Q=mWu!$eR;z{?vm)!5E;fmr7dNmlX4%vqXqF{u>F@H9TBF zWS|H5&;}E{AWZPSs$qh6sLLJ{hzZ_K9OrAQiyQMFJYg^OrYft1yk5T0^4lLD4ShB& zDxd+0IP*TaK{0B^I{2);i!PLSsd#^Pu|H*T^KHl>7C3Gllh)l(hopPvo1OblsmGX& zLz@_sB{SFUqu1Bnn*vx^uZ-P>o7%f!B4LQB`s;ncxBDK1h7y=atYackBum^yV;GkV zQmw4QuB;mQe2i&QWzik2*Wj%q*e8ye1AoZfN@w%(Q>f~+a~xDheNWCoiV6LAWtJuA z9HZYYniF>AOxt%V>|6!B;;adxJ1c=&>ID3!N{?WDVpq$q!W^{D z$UKP@5?g>`f`$I z86`vyqf>~GLK1_}G|;YUQb=$@7|=WlWS)ToiG^PZ-Z;YTs=fjVu32P5U1LHCk-+HQ ztFyx+rJt{9&-+8NwV&>WwooIGcRZ>f7+^5CgH+pK5CYA5DdGpPFdzn)EYj#wXC@E> z3;<%}MG=5prUU)lm+tDl+rB09S3o_fA)bE+6p-)}Ru`Z~>2FwJJwMBXi#hnUJTJxV zvgRJmGEipVxmqHV+2B1TZX}==IAEwH(U(yFFwk7J5uGt#DIm+j5(GsQkIY|#(KRUL zc3Xg;4-fzd_$|>x7{c(tD^35N3;qMqa>zAJ1in_kTY!x?^j6b>Z3srmAT2;3Kms$4zVrgM4ij9UArsNm%*YPOZitXaVipVH z&&7gy0jA|fh2|zTeF-dJb~u-t5dsMzkwqf8kyN+hfwh-2$2(R7GjsA4<0b+8N|$4a z9QtD{&CQ42fxwn8_5d||bmgSm5c5_QAPk{;JrT1+Fk|w^68SkMu`9m)LA;JxA`bx& zYifu!2WAGqFK56H#Q%xi6`*`~A!PJkRfXuIs+8`yLxxH~{d{pv)GJQ2WxQ%fgRb@kY}uT}GOU z8^_n)IJFigRqj*eSQ|=rCr*Xk$>$vHeF}SJt5Lvd5Nl?mru-eV?UCGOE<_TT(jz;1 zS@B4yEAvS3jkl9;J>LG-a9e&~Mti%3@P)qe6KVL-&Qaf{b?E^%U#_kWb>xJ?By07} zG?p)YLzS%5U6-|VG`N3aoFM`~I>c=57Vh1F*H#-lG!GM|nLC{9ygS0?JD#*jeol3k zTz``q-nTXm-^uabEQF7;6)~SI^pt1h9dUeV%$7PDuRA$DJjN_~9qi0}8(kb-?FZZ) zk3MPMT%6kNzkaZ#)lnw{uj31LEHIEEyaw}C&dkpoHQ6arWJ0@^!2S~R5OkhNcOj(4BNDZ>-wS{vD?S5cC^IFc#b3w zn;*MWR#$qLeY3)FS^9Es_78=m^s!xl%WZe+J$2HJy;V1=5##n4aQpf@$MV~Ywq4{7 z!%fHd!@HK2YlqG#a@Up)V8Ms`B9xIieaHqM4$2RaQaScY)8$W?y0FO*G@IlkyMD8g zts@8Ic2wlR1%1YybM=vL^=u9!jRa?9ocA2#-am;*&Y{n;oO;-lQg?x?p@JnmNKZPj z)q~>TtA4BYi#w|B-?9a=u|ySIrFj~4JZ+C_c|1V5lCEv(Go)x&z4U(5U z0>uGMacf?iktGSSQb*~6^cp&)i36UX8-Z)}mR4NVQ?h`N^Gwb(@6ePko;H z9;qH{VH{>^ZbYe$S^A7O4vD~=9tU)TrB`pY4m z{QSpEtkNeRvweN;==Cn4zRi4-d70^JHu}z^hcJ;w7-67!m44QGL{9jkWX$;qnHJ@4 z)0WP{4RaxnF{NNJ2E(?-M;w8zHRXkwQdEX*OSikkUokk#=V}!uQFT&zx*Fn>HP=i- z_wPN9-F_stZRaDqP@O?;bII%0a9`Fhxizy19FGigf(u%zf3&J;sa}CUTYOlv)ep+&@u8O$N7Z;$DIRR=54yUUcjlkd(#%WWrP7t% z4Tm=YZtWl}F)#0x8)#KZ>1mldo(K#@S;#(ld~0Xk46UvsyGwER?4mXHm-pBVJ674s z;n=nlQ-^e4BTMsixUW}nmBskTm_w4s(zl-SvBm0hPlPPE?#(*zV#r z7crHX8$8D*MB+ykuDkhR9%LK!M_&fSCT&Zq@WT}E%iKzkad$soU3Vc$O8(0zKFdnQWySY`rGtx#_Vo)EWxb`eU5tA3w9r?agv7lKoh+9(~<-Gc)2;99z1BUFA`Fygu zciwiTBX~dc#P8zl8`u^9m^hMaZ;ul+*Ou5)mN%0`$0_dHteHCm!RevQbVQ~!#-XsF zmPQZDlqId9YxwKFVQA4bHeG4*#fY+qOfia3`%ET{<<FvXB!i9doLeh z2GQ_Yg~4ka7fbm7QkW&8%+IlI?lSC!quRuXVa1)Lone$#n+E$RNBGh%l78>}yCZxe zGPRDR=Tjnr7N%}H)~USrRJ3Y4d)`h7^80`^yeR3Uf`XeQH8ZVw*!H&<#u&&lWb^X` z!pb5K2{pCSTMD1Jx_{pxA)a_ItmaV#uy-YmbhLM&9)>_Zwi0_3Ysit$ugSdkd(M8`jmTNM34CG`uoX$VP0x)?Ebd$at(SAr>W>?`8HTP1@YD)@8bn6#nv-6R zzP0-P*++4WB@M2xxx;B<%egOV8YuFP_((smcGz}#3`a#A(ySk`M}N<`K$&a;819AS zY6YFfSeho?m(va>RtJ}mHA?SJce+N7G#Kik7FSzU?TpTc+z~FkJ2_jjmfE&P`bp?vEH zsmjbd^qGD85Q}zJFh0QVPSD@M$27!`%cw{%D*a$h+4;ardcl>q-;YORwW-knO#? z5Q{`9?QX8kt(RmNCQGfW@A(rx&8rpbl0O@%Q=lx**EglL5k=M@=x~!``la~0R}IBU zpOZ>&E*j%T+mhNFyC5+JS-w4L;7CQ4H^`$vf85nUpga*Dclq7)F}IS#BID75GGmWI zR_{vVH7H&)W^auxX*j#`iTKs@x=Ym-7Vrv_8jDe5Tg)zYe7VD$pEha0heQFF*^!^B=pvpMFUjoI}ydfolg zb;%s*T`vGup!;4EYy)@Y;G~CbqxC*FcXc8oT2;OaNUgNo?J)B{Pl@TdjUT)Jg8X=v zcwJ0>p@v*LJ;^n!4UcO`7tx02^fj{$ulMeJws!Wk1NqHr-ho~vc1K$iGEd#t-?hfL zG{@qzLR*FlID|ZV#9QMf6$1%8d-VNyZxfNUc&{R-c_(Nuq*V{ve$);SLI#5rO8!s{ z1wqy-Rwi9PxTsDK3b(eTzUQ_@wZwaL3Id0S-|DbfE(M$4vASL>!M%K|2V|AZX2RTG_n~SybRyXNOYhE&T=* z+7=2$ra^KLvbs%D#OP4C?ao8BH-4i`D|`gYpGCQ>=rx0d$;8P}P4(#T01XYIfmg3z zjSRlJ`JT<)?xH)7BF(1)bJZakl&i*OQCnvw9?F$2(_4~&ut>&3>G9qw?FANh9DO4) zroeAC=e`Au*YtO$jsC))Rq`2s4UhhjzbGFEbAlNHCIy;xzG5LPNSl13`Cg)C<2LH>WkBb&vE9r6*2(4k%LBQ1|~+{u0W_z3MKLn=rdK^eBLwTjyl*MdWJhlrnbcH zq)NwWK3)bj;kaH`XuG~d1Xy;r&|A_}ze>F9{y>Y`iav@DALS(Gsxz;R>ke)UT6NKy;UEYX+ko4jP5K`i70S$Af5*648o5QRf81$lRZi`tpf7R`RYTu*ct)b^ zZzOvCBZR~vu!`_0CDMd~Rg za?6xb%bLO50p_Tvk45VJoKz@s3bG<~L;NZaj!Sqb7gbJ=HO<>yx0BziwAGXlca3;= z3hs{{;Z^QctUzPqsFtq=DTn+eFZ;0Y)!OeAI4k=2INZb;k*hi~h0O%(@Rj)S1-ysl zMo(_9NO3Z*MbQE9MOgI$o^05!^Hk9DZawFq5ur4Yrhc3JB6)LL_@OTbcT$+&nyWoz zGmScKm((Jd*K>CUsqJSOOy1d^8jF9NCd*lwfp1LNX?*yQM`4n{L6{8Gpr9PX&`wfFMI3={YUy1H)X*g7d_DVWgp~W6h##65 zKa}T=F1XknHiY^_u0sa<3e56HyE48lmxOS@u6e+`9M|+Nu!Rmj?yZ(JrS2L&r$O|` z%K5Lus*jlCo<+nA_r0P^}aD?77GB`*(q;bMczBi981laF2h+fvN!8Rt^j=UW#p?hG#pGXaL$$$jg4^D2!f z)b3*?FO2*gor1L#bM^J!OSo6kEu@l=7L9rKJQ;ybI3SLx;AY>rYtPSFc3v$4?fP$} zwIdJZD32VTX=jOhIGm$sUG}2F;iVs1fkxpWbIz?q!NPMmA{dRw-r{JG@lA9M$El0T zF$HrzB8sip=rMCK8B&X>P)H3AKs#wua|V}uT2dtBC>9mlIkbwWAq>s+RO`^E@(p{7RH*oX68o6gp*y|R)c z+ZAfRa>ruS{~&IibCqJA;BsV78W7|W02?`YSRf;`!U-G6`lgH%_ALO^F!;w#R$%M^ zLyFhaqSjDla-3y-rjFM?8Ck$ zy_{Vc3`34xe*nRA9KQ{_Sm`h#?6DeibC{du7vtU-W>3t!>nV%#;n$W;6=j?YKi>}} zv=;D5uAo<%9Mf{aoMj5@Z#~dDjvWR{+q()h_)m`TQR@Giv@u*OuSWJ95Abw=&O5vij*TZk zqGvv?r7`tu{IUe(6W>c6P zaF}wfED5Kv>>@(pT z_@RuDD>2W633ql??h=L0QL3%2q5H*c@UVdKy;&qf6idX&AAM89`=8PJtR1|Ya zb;b20O}PVYjMM`NuS1|-Ufj$MJ1!qM+L-BZ>Cg|HTqb3w{G+%x^x41;$it_L#Yyg&p+Q)>9<@z4c%>Q!Md9 z1qeV|O|PNWs2DWt-%qm7F;b6*E6i6%z$s-KS8-7jIxIm3yLg27NGd2=M(svj|Lu;_Sp9Um3>?}D2>sJ8KURR4N6a{@D&U-lvnoH% zcTurV{Pm0(Kp`1id$18wi1XZ=%GEQshWzGM%71dJ$N$T%A)ZGq4vL_7YS5oy_+tS8 zk4oUy&R>_4I*aoYZ|eK)g5ZdU+x?v>3l2>f`;(pP{T;)47JRRsC(N~>=oFBkibA0$Lhk!P^o`Y?>!$5o%J~u#P0OI~ zURvepv6rq#m?PMH*>RLg!!-GoMV?}1kmVFo@6m2sepFXVBC|Cj^0DdISU~|Cb$Bz6 z;8wcgC(Qoh;89+Go6B1XV^oOV@Bzm202_r@=G!0&ueBss8GSz6j9J&_ww$Hza+ZNF zZ0o6{hL{p~_-CJ2SAVST;QpxfHSD`k@#Pm)Tt^ajVy|3vS1bN{?4J9rU3ieTb(qUN zNilcl^nd6pmLr2Ht|M`{@9;)%rKXvF;Oll$U%sYPj&6TBl@w!pc?4AC+q2SLY}#`y zrC;7VGm%tJ0iFNc9XoO>`SkKQH}z%mH;-De{;IfR9-SWZD7UR$2zLzX>=^NS2Ae9_r?2;2MOAw}7J{ndm4}Xx=v`_4-2%5B26p8e3C(qqj0bOc!nFlhUV? z4wpybG-BVGTHlv)R)6)ht`^3`Ql4XU&$qny{#=rM*fj1DZCSlTUPqQ)hicE#)S89( z3{uC1b_OfCQXv$$w@`G&z0TD~q`76qrqJ(>Ri;e>P=9Xqx^y?88PAs}Pv$@QmCR4hdkDGlX=_J{@9$1~+G^k$ko z@L$3kw63#Ic;G+BT0(4*dZS}BK83(JHKDO6Q&c&YO?QBk^fIvw@dPEYTqsEQfdNxJ zt+MaU+wnQ|w98kBUR%v+Ta$gWCVZCN63u5HhfWDdLwJ=3WT=tyUpY_d7Y31gTcp+< zK;~YcI9dZSqrAtX`UqW#C9I9%@bC7 z8rMJCXVxt>c8@KuQx@J|ik9?%=%1{M$xb^~uz6Xok3$@rF*8$NQ<+GmlMPWB2VaLs z*N=$G&y9^5V%pd&>L|~<`KbPZ3Ajo#x4d$@g}h)e_+iWv{VQ5(thg5WJtHQlCHlR@ zpu6=43U#?bfOZ78%th|gzbdGiasEMXURS~9d-S-RYZzN*cdT3}>DXVGl(G60?Qxko zZo=vWTQO22TK4JP{%)bSXAQZFw!K`6gN3v?N@5FKH>HXM6SC zl|3&z7vyYVIy}ErKM0x3aCdXznT<7es_mM2QvlueG-l9xjtvdE;lk_cfJs+e@rd5D zY>1oE=A;ea=4#5ap}c5ILyanZ%cLZwcjp7kMU&b3ED7pj>4ZE5gyM!W^toc}*GnOg zPpncO8`{b7LLh23QndDJ&lJ^>if#5%uGyUUA!1kLtLu^ZpR=qY@h_S{FPhw|&!V9q zl}gB(*auwDGt8Aru;#AbHauJ?{2GA0NjJ2u+WO{>4&8h-yU`cfnH#>at8E#0)B-9|`4=P#^X@vp$?eE|C>)1z+fZTK z&_*7P!#E$Eg07L#)d$=m!EY4{w$QVizyYcW^@_FgoJMPPSSY ztcNYmQwd&*2`qFfYGBn{w!_E6fYof~scQRZ%U zN?AtJU77j9Z)dFbu${6x5stWm0jD(YVV5(W1qHIqYuV57#OVcN>yr=*g+aKBgR$H< zK%H!ggwx%O&ieCI??nBuCP?fMIgVNIDYNp1bPUdPxB|ZjrquV)t|U`VHwY)u=7N#l zYK>g~!nN%&%l*CgY*la7<>9b)SZbUEIT^}oCRc`zjV0tI8!baIq+V&j~xnU zieJgB6FzrX5{lG->rp_qLqtH`+aY&ZyDX=k+`LHYK5)S2zDA`+)?>;1JQNCNQG%QQ z?8HnKuZRl%?Q`E58KRHQ+83ejDb8Gm=l&Of+hf6S7uOrB-zFkpup_ma05 z4+%{{uj^Q|>#j2&rt+%E!^tUt7||c%?f@}6e~CLEO>O9ptk7jVqj)a=|1>>`#3u zd~rAYqHmCg?`D8)^;U_zfz89?z&Nk*LsDgYd|(*y(k$^ChJMx`2|04IH)78 z!@EvW*AhU27Y@d6l|Gv=r&ucHlBvR0_p4nP>)Q$R}4b$LhdLxX|( z_ePYFHWv^Nbg9#Txsba6Y8-u})M~QW7$uepqfodS!B1$`2&37b(6-hi>aQXI224;S zmkQE`5+Ow$8vta4WbXAnCa?oSj7)u@g7{FuEp{c{;H%(UApT0j)n=|2IB13Hz19vU zuh^A1b2EUkyu4sYF}uZEdFZ`JL&^Rg;*U_~VVaVzM=gdx7Sc~5 zkU6a5imunPD3#-J6y^iL0X3mC)EI3O_?cvc_;Ybu=; zB7ltk;sEM*LULdOJg|W$Fq=;?4ai5DI3lpen!QEK=0&BmEObRVEdWOu*8DY$HUSuG z1Qh9I2Y)}4Z=3byseFly9@N%pXA1Kuohpp(Kkp1xx)p7yHbf$9K!q4+rG~`mBY~gB z(4@spfa3NLAkVE^K!_sI>;$8~8xF!11kf%5xO5TV41o}X#gwc(-u}e)9S{8dH*Dgs z+I4;Ay>FNnOE}=364!h5`W7$Q4J1quNzW7mOcexly@q&2>nOJXv%f^30L85VKnMP8 z8i+oE96Y%cBn5`#HluE~)i*A#zh<&8Cxs$m24Eg_b@{KGoG1s`LXJA!%{j-?5?o8ygdL8%(?64?HIgRr)|i z4H-z40=qMS-4Af|+&H5L5KC}LbcQaALYKF|=+FdOL<`X0^NETT z3zSHJ0*q$mKwm!xeSw6X$~@rm7cOr<;L)$RrZmK>-|{==F`hM=7V?J&$l1Y^fA_#- z%s+S_L($biK_Qz3K%FXv0LH$9A^^MGnn*$wq~djiRBYU7L=^EfGW@W?xd>p$Up6?E z_|H^x#sfJ2mTDc~t6Cdb9pJDPF`-zlLj+K=+)Y|95_2ay;h6;ab;P%AWV|xRd9^ zu2(adU$EHHxelQ0CdIhTu;zsOA&kov?7`t-!{NPDMITbNrd@5@{Z$d#mGv|b`8Hlz z$cHaQvO@uvh8@GHZXb0OeYBTLjPqNOhGb$sy{PB6kH&o-+qLwKcdXIs*u@Q%fpiW6 zYi~^};^0=-psq43R#s{bI7sjNK|~CE z1VJog!aW}sG;0w}no(i&Rn6px(t{}Xmc2y_LS{9Cmp+l8D{Jd3YFk2twU&A%I9fJzhV(r&A3sI+t+KS%hfY!TF!J z|Bhv%;t;L#SU0LFD`!knQ9(z0Th&^D-6&w*@}58TKl6qJgPS)&Sepy;j=Iz=m|TdG zyBp2-v-E7h-S{9xrtUGWor6e=sWg=#Xb=Zl?4Qypi2`R9J5lMw-KME#OZ zWT>hX9esBzb93QN}XpP6(zfqrI^d&qNYqd?%Np-mcV3ZQk`Iq-9_ zUeee&+)iT)goxykfhEqk8n+2k`p-T?2(pg<`#vLa76!lXGtYqLrrI$NHaq^p#2f)k z{R`6`ZUFyh`kfa3vOVhN&*>+D2bTDC`X!$(GuqIEVVj}Xrwao-nePWtLNKJm--#Od zbGP{|D(^teh=dI`9W#}G#CI73xa_a^UfsO(W9*$4el}M9s`lj{n~Xg^)c)5^M))j^ z5WF6s`Lm>iQ&EQofFPjrlR=c6XyLQ9W$e!B+VWOxN*KEQ=VpU|oo+V!USLfmY;hyi z)=cD*YPKxUdK~%7!f-l5el85BBji6@7K;kf{O6Wc zQ+!Ud^ElR69I^bVP_*r##e&G8~ z93KHi^z*9r#|PCAZ&mc^41C|69w$zHe{t%Q>Wf0jPc>(1tnCUt+{;s^P*r{;(n|c& zOzZ9Y?%$)W7FySD;Puzzxe7_VLTjz&KKCddR-B6CP_Gsca691(`%M&EhZh4KB zFE0Axkv7h$bLR-G#Oai!b`C%gjRlryluo zu{ry}cPe|^=;}((x!OxBa-)lZDtw7jsayQJg)eD*x>2uoPdr>#D<#%uxJ#ZzFDhoD zHww|`sFS0$JU434-?x{x8$YkFEVQI9&4-z6)Ni513ssg=iTI2QL1>bvxny;P%}`)( z&|dWm-|xr!38_;{OY=MB`BEeNXOmS29^ARV=aDak@19vmU7Dg(c)l3Jc$RQQWCV9J z&If$<6WB%>v2kU+GLYCRl~IVcb*q-D_AB3=K!5Lo49QDbmCvNM7!S0pw{Ct9+;Fe5 zyKw7OA8{`~`t#GJMzqSR6}#lDiK!}aW!_3*wyW^zTct-F#pz-l{_v9c1<_i6=@a~n zGovA+MaDJWV#|}C8#;5f^y!*aB-8v@KQaeCi=lhTq7$_9&Q!_Zxo~6ZXTo=T>v0bP zY3zB?dljNp({ahd0`b#rzTeM5>n|=A7e#;l@I-Kgn43{mbdN`{?dkeb_3`}d+0=K? zdC214gjQ zk8j@Yg)q-dj69Vxclz7eX6@&h#gn_~fivRHu(m74r^bfU5|1u73!O8DHmc9*J{E}T zOnTw<-Zmti`IEz%)A6(I);G|`P)|PplIH^u0%TOVPiR4RQb9yH*Bd6|Um||~MD_Qc z_H_0pX+m>Jg4kv0iXJ@N^RV(c3k@qN8<|~}$0~EpHz;`#Yw4J{u=XSy9nfQ8{h65L zKY`5lfreLM>@)32Q`w?(tVW(|vvA-iwV4NBB`XEOCgOLClk%L!UnRqE(wjUROR*Tg*B|=joP|+H z9oc#=Z|`(L>Z8EZjBo)lUY!{2Qq|Ipt@y0iBgfE#SR|gt`l^Qj^dcXR=VjYsZB5zw z*k`K%p>}MODc~si7TK;o;t#nxxu|S;$GBr&9Ilv2DViiL>A>1FR8g(jNdxn0W)rr& z(H6W6ymZd_8gq#3wH&WH8bw*T>W+BiD`wwEyR^KoxucOHdDWnoecc0I36HQNQFXCf z-n;F$K9$#oceii7ceW_jPk?ioeye()cl>tQU3|v*BG#@U1Ia+*+T2Fg>ILqQRL@Yd zE+($WDQbX?b4CoCV{%04I~v`G)OMx&3>E4&M`0B-E^A2ES;9b9@*DcWFvGS_LG|4{ zo%qM}Hitbz{ni&!YrZ>-yeN8qH*K1IIpluD*DI=kgB z2`+RtG1nob?+`OKh{ShI{xYVsCDC3yRe;{7Tulz={ec5{bV<9*JMZNe7W}QforO20 z8GK#_1s{fyXpW2#{a*+~-4N}rcyxi@wU24vV1ygcYPaQTuwHNZ$fnakbCq@U8_#7z zvy;;1A0}HK;%aMMhdXRu0EcxX-H6{EQ`#b|!Y(qQdy;Mqv{VHrzKPsFKXvPm^F1b# z+YC3%9C+-(Va0ZkvRGE3O%LnQfi11RQ$uiqin++ipdQ~6x4>Bi5j>~yO`ef$g{<1u z+i$-$5=Br!Z!;{t(;?q^BiI3Z&P- zcSd4china>N$G6)?637v##?=wGhty=#zM4>{PNkohFx3c zQEjn-dsz)7Vi9HMN|E{u>ss%dE^2ycjtPU3&Lk)dvt+qHY$l}Q%3F`|?>K6!EQT#P z@(LY3ETMEox2&#`4Kt{Yvf&$3lIdQYSP|iJ#vl3cUMcG4Y~#+&iouon7+` zn`YgM8}oB4_`_0e&vtvw~1TbAdD<#q~F?-5BEFMG1aSKTd|o=jdu7qrrAEeV7Yik=thl7CmZdx-XU>|e;FcL@0ed8a z#ltr@Nd5sE6H4gm$dEVan1BwslLft8`K$R;mwdh@OQ}@8TrQXGs#ZOn;u3vgcb)rv zH(xSzxiaYHky+(ujaiIdy8^b6$LBp~$2doKRlgB)DNVih(sG^{TP?IiOF^wa^%Nb) z4TV$9+fD;z7fT!fRZ`@E`ayT}^yiO_~aa2W+i2pStGyzsFmLDuD> zk7vsK_RyViS3mhsi$@=pkToK6+gj;MRLURGoswhXfIOkAcr}l6^d4N^ob*fMHP;K) z)-IeC+MT-AZcm__b}ZrVZc1@=stWCjGeEhGOfw!Pc6n0vxzO8rj&iXCCsXPzI|r|n z=9r(S73oNao~I3Vq_v~X6w=@d2MKyOI*e<*eIi8VXWnJjlq&lwXhqUY zr8%`gH@%YkD0@?C0hK0$fVA4MR}CT#WOT8kIrW7$lVES?Wf7npQ_vj|$V%LRq>-@F zX7rf>L2kd7pEcdM{8kP>pn>BlPUcNq8!EcS{IRj;J;v^8Kdw+>DIQB38H5><-5cZE za=QRtJbS?+-$-APX7TK0W1s_9oG2y!x~#Y9Gn3wvv_coyU9UoGB~MDwphV3VB8M0w zhnm|pi>~$HI8k#CCNCyusZ*ln#MCz3sc9X230%!Xcynv-Ug!Ld8_TZha+`c)#LWxp zMLgwp4~{Vm(UoQ)EddquQJ|Gtgt z;unQQqX?l;JA7Pek$w-170=AVSfp=7`Nmg3ngUOjkT9H9mt8Mu8+$Lng(d-%8pVcB62SLz(BmrA#J|6sy0m;NRB<2D%yaK7PVmUqEyd zf173OAiwtx`t{luwO+T%XI~!P^JeUIYfmSObB}cUGGg}Hl)U*cwxN~ITw=)boo%^Q9C3TP%2St|v_gwD?eAmqOYl)ney=2i*Hu;N_CczcYE?E-TG$GGG?Gh`M00a9~8# zZ$_N`-H6UE2O2K@(C~`yzjxxWIzibTZ;~^42v1JZ#pD;{gg4c@w~IHGMaLET)RP^> zZ_mR<_ETfgLf|DbA2ljGy>DsCp?ovj27TZ{o!?v-|GNv7@ON_8>)qq)2~L-xy{{G! z_?Q3QhJ5{fvA;xlvE(4iy)6F}Wt%#;k9+Ma8IymcQ$c`Ei**iZIJHaq+j4$x2Y%#_!DA;Qkl zL8bbX(4tvaoW04^-Wou4t(hqgqoQm9cA-o_MTc{J;fsJx3>8yBM&E_$0i@DdNYtBD z&KJC##^1da6Q){ETs137ma7ROC|}j8^;J9t+-{@Sjjm=kU0G}aW>M5Ib7!W>3mn=~ z+D)2_7=9tHNU=I~`J8_>^ubfsRQj6Sk#{;^1=N6oGGWnJ)-BZu{4trW+-=fS!4)dv z7G{x`IOQ`@bXr!Rg@+B>?5A_D{2bVl6VqV0b#BFE5aII(aenJcfr7*m3~@fQRZAVI z@QfZ*-#5CR6^!dWa^L?}QFByCLw6+%_T-~fa03lxl(3^Yq}UFDgguEiQ$t43v%sFT zs8v+~xz)!jGIBOIjGVvWQY3U*2VQajt8TSxQVdtuCMet(cx^`*+$Z!#e7r{{WKU4I z{1@RmxL9VNkRvMBuhCc27jZXy2sFe1@f5{2JkFT@(A8-a+923}o*fnub{j17kmUlu z-jS$STy76t>cb-lgI0Yw^1~h#;yftfrtB#W&_Fm$8~H|r26hSCu7-TW+&#d91H#gG zb^Se-Ejan3`fhYrD%%1(2Ivf_N0;=OqXf!h2d3JHu&y%Q$OhNbo&(yHQDH)%ou?3w z#m-$|1@Q=+D5*5Xr9AN6(-y~MP}xUWGC@2Au?=@Krt=QyIDVrO`$sz9|3Y{Czo1M0 zI{ZAicSWjAtX4KG;q^w>`CB&qx(TnfQ=>y0uA2a15ij2^Xs`&7)n4CmNMbug)>{{0 zaNnLH>KYZu`27<&5}(k3FyML|+`tSxiZDnH@edG?4Q=pJ)m>gn`!FsbV;lFN+~qjb ze>XFVqViTO#B;LD6F*py;naU>E+Iui*nL0*?Afkq(odUfA=t8@j&yX=z9%mRGGC4J zjT(|ST6N4=1nCbc4ADb9C!SVJmzE}hJ;P3M(L{}9LJ`a^VLg$w+T@Go*WU^W(~>VU ztL8FzKnCRb3lIPTMWauoK=df1xS?_hDUyfQH?*Coqh^88$43 zhB_PN&0X5n4ctHkOh^ER5{r)iIbHZ0w>#p6bYvTGpBsKgHj$F{IAd;!G^_xV@R~z& z{EM;I(v>U1>f@Yll8m5wb*+RWZM21~5-|<$o4Oam42_s!Po@QzK<3k2YUB5U8R zES3sHXNd705cykX{UziU}!;I&Y8o**ckA)LF3MVfTtx}yUxpX=2`oDqO6Ny=#4IRo<26d`lc&Mvh!QGh;?U=`)Q$D`vK0#Dlvrx#+U zm+FP;mY00L0X_KfZV2RxIyb6&VY9F${QT{US#h@k?%CobL})v0ld+? z@G2+aNUl436q9Nz$J57Y)KN^O6(R^`Iu7#Y;Hgw$mnL_mc_Hhm(=rJu50Xpt53~huz^tA(A?UJ60 z|F&H~Ds}7tTati*^Z&b@ksR8v*UN}F&j9tDdRj4m(0Jj$H6G({jhC2^B3nZNvZ98i z#~6k;pdeAW6v=PsY3NO{>QnyZsjIGgNnW@@%W>jV=i&?JTy`C^~eZ`1- zSdhz$SHLsc;@lBXIUK=!j~mnQDr0*7V40o#ZJAyD$7RO$zbvzB|HU%%-FrVrn_E&O zZzK;#4F217@uzt5|C4yM|CM+$<2MiC6e@>U5rd3S&xNNI8waC}{%y3J|1{e2|A~(C zzn~Kg#E62+W_41u(JX2Ahn>%S+^}E1L9EoDx@(rF>sK*YKo1w=fOzy|$j`8IEB3Gs zKOZhUmCuuysIAo;z23U9a!4cMleVibc6DIph1$k&5Mm^8(sPfHiC&`uldr^gS~~6SSQ>GHC*0vSZi*Z{}D6iCLZosG5o<757=i4 z(Uy7veIHdhH@kcOxp7mRL9qg66_z>8pHDA@-{Dy)V|z4sue>~eVWglsAussOjP-TZ zYQJyAB?HR1GmTgCv#kcUhy1qPbtr6W#!^&ABxoPG8+u(=9ZYVXI0IF5pOH#S|8RNr zy8h@rt~W)cu=4IpGxuC?I}L%9>T_*NYGU+Yv2k{0;wNhCuD*P!e@bfw6_qw+ef5^* zDLcDeE?<-oZ+VWjThO1^$qj;(SUV(&S7%=f=JYd{Q$HW^#;>)%xN_L&vxJY|>&Hf@ zi*pF~+?FZ9iIBPou_^E?NIK`PY}Zr#V^meFO+KaaBh^3W1Tw5 zM?e2KT2KD(05V>DoGm=B^}2rTj-jtU>(GPgQHvq7j7ZygXRdqcllOKO2NMl(&ux(| z3ZD8?aH-fh+x@4ooqy@I{}k>APvKUzYu~Y>o06v6C?R?_bwpyluR<^t%2M9Yrs1u> zct+8kA-Am97XHQKjjjLe7dg78^Y>`)YVXV?Th)LnJHhzp%&wk0Z|TBk%OLd|uj~Ug_GnaZ5vt+APoUU6rjz zvX#9X3R3gN`W6*7DearJzt37HK0tP&Ek)oGCCg2PZw@ZBHXj>jbjEq&mAIGHJZ?4Z zx)Um{oFyd9&)jjzTE#5o+mGxKBYmk8L=qYm7_K3U!C9EoXTmHy#B@1@wnX_PjVsL! zvcuT&?oQJ&dfjxt`y`TmPfo%7tJkbr_|@X=;<{t$w#8RMXjGevyAOMSObv20IP=!0 zU6{xxuRk&1+B?rW)3xie#$i5poyOv3yaVMnElo)#?PoaCdAcUf$aO|I)0xc_iLd<@ z{)F_8?AU=E@d%n&(c60|bwau+qP?Q@O6+?uJ%g-sS18Qcw`IZ&6RFILx&SGmX&O@BSXecd8A-O2ORsAJ)BVf^ATujh(J82_kt4#dcgHpg=Iy_K-EgyhMpkrK z;@Jt$wqgFrkE0LVQfMnw#2#b67WyHj>pKBEQ1@5OWo$`q);rDG!AQBjG3mGDUm5kr zp_DbVJD-QX4)pBWmMnST-`lVGuLcn>ZSSVVZ&er6aXv$%FAzqK3|ut6x<7?OGMM?0U`FK3Tmi3E`^@dV|96i6#~0fvYEvEs=(Y#v94$5;n;Xm0E?rTwt|@Oi#j!S*kUj%`&_J0!U()#I&I!~Msy{km5@UX|qmzJbCF zRCr$+d_=nnJS}#hxmEeVB+`CMsk&R4G7{eNWm#eE-ChP@wF?jl%>u}Y${d#P8Z}A9 z_P)^i46e&MnzS_9E=za#RTQ>0%Yp@Z?6wrB^Vn@CKF`Zn(6RK#7SlCs7a0X>vk+$7 zhiPm<)9s+~c3GaoX(GC{S5IzdJE}~IiT(CGD z&u>*$AabcrAN@Fk^g(>~lSP&x`c9xC=UrWNTbCc0xP)Z#5x&tCB}Y`3f3t;x;J{!a zBntq3CcTq?BUANHPB@y(nOP%7xAQUzYltRuu0@hL*CL(xmf`*OL~1%E0{nIdf1*VM zsrDXbt}t}A@IwSiNDtJ`uW@LyliSmBmi4(zFi8wc%444O$c9i7W#rWG2{Er3*XYjP zO(uz{5-rcQXXL_DNg~_c;$ptbpL`Tqhjy91qX@USAe*=JyZF5ew|R+uQpbmh4Evb4 zYV#$E(4&uP6c4b5Z1%C9%Jd!H$@e-SU(@*vhNHKj5JNO#T84P?B2%~LGV6k2FexEN zc8#tKW`I7aiS{|QCU2S;OgeHDoZN=Zhoc>0Dc7tt)~sCA&}=LWv+-G!{pOb%EHhP$ z@+#M?cIOJ)LP-W3{cAeSPaRDw)@BsbB0KpvY6aTUnj)V(o?~sE8`{+f4PK)QDj#N| zX3<6)9$s@{1L9vjJE}Qr>{!CP)-$PwJ_RhEi_cTXkiKfcACMA03hd-BF7pP54RzVa z4{=bn|I{*Mwy^o1v^J@~1m6~_@z4zkE%QpA%%JJ(7wRn5as5lvU-tSy9yr;TSq z)~up@TvEx;{_w)iK3*gS9-R4Zwy-sX zAiDSy{q_bkCXf)&VmgU-u>kOkfD)Q$w;uv#DyjaMv<;Tj|C4U55u2}ucChrDD_rlg zPo*K=7b~5MPc&}KzLBx(mDZ+_vCB5FyFj)}V%P2$E*L!mjS!)xSBxROIiNvZ+o#ER z@1!(a*m3~e#lH_c7LAc3Lhd)N&D=DhU=Sj0n8I?Ii6IgUUG{Vx_Q@aHlK9 zm5#`Ax;?{C({<<3RX~<%ATHsFDr|M@g;boCroh{{wMauQH8iGdfw`d#@^zyZIu+gdISUW!*m0#5LMD!*yCZ(^6~kut0ReHmUMM48B^eriqgJ zg%~mkd=B$bXZG7VeDC-gVjjqRD!TUFd^#Q?eBSMSqVu0Ft+g&q}b z5*thyn4tc$eN5TaohSCUV1Hc24ED;H#j8-iKBt^TWBuJP%{8jgBnE)0a}Lv-uADsAt?b!93A9= zY&Qcg(WJTRh6mUueOzN1ywuRoE^~PtgN3bEE3@AY1wlbfS}gp0ofYo^|w#UMB!8rUs=h!GYs|Ec^%MPb4Xnds2*w(Tv7T1LvH?ZuaS$qhN1$*KEhO~SlyCzG+#t&emver`1{KUo z44A5Rc<*SW{xGr_`}en0??mFpC?7j8a(_wLjlvGBkY( z)`aDvS%Fc1>U9*>p!j#ap8u0xeAx?}?6CfCFXZkXfk4lN(*qX`9|Eq$02Tu2J)wd$ z5LrzJzI5PU<9ttF-1d*-WC0-4ejVo~AnoZ^yRv2l-XN8If0)Y}rgGT@?FAZv84LLEF&4FovT&NR0&|a#60jR4i)F8I+wU;JwKgY?UjWztI zarQc3dMAI36Hlt8f3A*oaNA}mrlc=!4g(l+&tnj}R5!? za=CcspX0o&4tM=SrC-K*>OaTHSJP6;eExadvMk4L0pyM)7VvhsfuZ;veQ|3fK-lcS z?Aq9vT15=~*`si*gf`agFh&80+xKA6CRm2*8USi-8w7WyguAYSHO(UwA!Z=43M&HM zU|%-h`w_OU9OZpP>4CcQ-}`AmG=iEi&ITr41lI0Ar>e$hWSvZ%-`in&oe!jB!0k&( zu9AlUT=!OwWWwdf8Gx+}T(hJO(@XOd3PeT%k;y+I8Ux@C@u3Gkm%{**VgUGKfow6S--(1D%zV4p&B4MQ-8S_K+bp> zzjPV@4ToKF#$7)FLW-3Iif9KOY1g%E9ID;W;B)o(VB!*>2N#qcBajr0zh&WTdQaZU zG}L#Bx;IP=PxQ|Khi3@i?F z@56_+Vi@e%qiWdr#z|O1N09|afddx|I1SAVU{VO-pa;H?kb)568uhVH9YLUR&4^Qw z!~_2-MLgX1FLN^x+Nnt{KA(aa1~U;xJuE^T%f*-gh!&^scmnm0H`QtaWRrIxc{_vx z+{Cikpq3bB$oN9I0t6oLa@JQb^Kzpw;vIoCqJ$9$?M+pHa2|nhZUe?O^%&t!*kF{O z!e9PniUCCb^luOUk&u&QV@s$)I4K~Uet>LIjWmEL!`;FBI$ymM)05TaW>8vk#gQGX ztod;iiNQ96-przADE@c=vUy}5B=6(vXh6#WD><2DR%rDFEE_cOD0v6=f0OLej``V0 zzb%+dyusp<4%?l_6|mmU>OGRGqt2I*4r*$L={Alm8 z$fh41Nku;J4;{I&;g@5$@VELl3By>s{Jp+?{-M5unB3T2Tl3a9o%=QH`y1p(*ip%% z0!4=*p?4zy$XFmU?GVIB^WYW2m6c5W=bOUcSIY=|VL$BQoOs!o(kn^(WfTP0%+CB{ zl9UX^&3~U{>VHns-vFyMiu8eLRw(~`iwOfFlm4k9d13&mKPQR#soOvAmhAJXWe3$w zOkiODv$`b^Shv3qv-i?}R`=H)NwW3jKR5l-kH4)I6hHWfWYIum>JP~lcMg$DxB6*LTy5%+B=J-GFt4k;Jw)+;G1%S$ literal 0 HcmV?d00001 diff --git a/examples/policy/converted/expanded/P005.nt.gz b/examples/policy/converted/expanded/P005.nt.gz new file mode 100644 index 0000000000000000000000000000000000000000..fb33bfbeeb38029a10df1a3f88f34f1c67add673 GIT binary patch literal 13303 zcmbWdc|4T+|2}Rv)@DT6nL)N}W#5epWh$KoF z22+FVBn%nD_oj24I`8xQJjRZR1hTg(gF}`jCA(o`EEaQI~Ka--j{e+i*LWu)}T!6TP*d~&Kj*(qs*J03#GJRP@${K1Ls1? zawt|jMr94}P1x$!RB$kr37T5&W6PNW?yJwT?rk_jV1rwQy^8`-f1s_bG4Xz77;+zSO&0 z+fyfN>kDz_GI^F{`1%Jqvo~N=_A%~j(*dX0S^c**SKYzyJDL>5lQD4<<4JQ(9UXdya}*|*G7xi`6Rsnql;N_7 zS8Z2MNe&$u&((QLf~Yc2j=MihX&=G*`ko|x9v_!D&~ty8oUy6R{T7}1Cc`tot7o5m zzpd)~4yM}V$+VO$3H-6oq8TYE7m=`l{hb|>gOMpIzR^NoHMfA5gmiS23*n1jhEL|6 zs&s2NMx|;Y9P5+2I>dtysWFY1h>96p7Q{dvnJJpctqR2P{8ulSzw9Od#8%pAKhr`caZ$-_U*9lm-;b&?SsB5o*R60g2<|FERt{$&?C`^cJ_ zGbSV6dA0HWu*(}KLh97(@y#*WD8KclwO0nm5^k+Lj-Lq?6hLh*^#!b1Y!K`&mxk4o zzuAx6vKV-IjnXweKTjHyOKU`}wWOyBJzO&VaJemWacpvKV`hP9gPU8voh3VjYHw>8 zn=&ReAXXP{XMF){yI2J1Ywnle)!AG>h%iNVUciiY4z-LhHLppOsk&W!MBt$qb- zg<{AytF&4{S0;0(KIoaoPc1!`FjT^)&-V?IV(0r--;f>f0db4@wC_e7Pz`GMhWaqn z8h*N~zKYzq7>14C#*fym`-KMjkjU}N^KWL0@R;;o%xwC>MY*lOwI6M^F2Ypx7~-lv z6JJS+qz-9jpTK%_t3rX7!y^C0%B!!7V7ChcY1ujsBZaR}KOAl4y>79Uh`;ZZX|0Rf z#-SY8o>A=LzD1Qxhm^ebyL?4CP5Xhga$3CU1jEbOFGp)uOEw?Y8eit4xyC5t%h!GR zy@Y(Cv}XJz^*CwySiX$rr`?}#ntW#NB5+i9DW{hdF_sUUc!oJ2>7AyD@4&y#8a#fz z+taP~!hza!oHprm)|vLm7q{&_yAMxkKD?cjDaWL?T0cy7_@wpX%UXTbeAeJyz==8E zvIgJ!bszRqDbHYCco40rswP=|;eekYioilRis|g|ac3-7LbfYqrF8pu&mB({FlB^d z@^0lb_rA<9uP2K?%x|%DQX%#A*l2|of&-8br9KU+oma4$A{hG*KZY8b2E|y^cRvw# zx|2K|D5OpvL;@=B%RDp`D5K!_QNj8#(4F5qslt)|y5U}HG2N)OS;&GWL+Ti=4v znqNdFPMW6fS2dHpp2bg5RV%-!yD4AuIhQHyo?Tzxyk%7;&J3<~g1sj3LK^x;HHjP1~uOLH6wenA=E@LQ&& zlcyknIv((EUMN7Fhr*<4ouBID_eB-((V3nRYnT^0V{*OPn_RjRV#z$_$B{E9qaUj= zbuAkDe$=5qz63*Iw{UR5$7Vo=B#7)_E@qekM}(ZXzwHB0`QU zZDwF6Ef-BiN~M}IXGBlf&YX7DY*n;^Gsx*xD}Gw5Y|I zmM%1c%aZYq$1)w*CV@zD1R`mWg)lxqi6*S&H-1Ak@_g;AnfZ~f*1l4t?nooUb%K_e>KC~Cm)sY&{h}G0 z_zSnwg>AQQKamWjiJHmOQm@A}Xw+}c#3&kmi6AMgj+vHjv-5>+QF8?uz+W*a$>`Bb z^;nXT6iI7L$6-m1Te2m2g*<#Ua>v9jNhq$82Ds<-Ld;J>Pc(?N> zmgp1mcortz#*uml-?d!81VK*MfgLdN_O;;CH6U5Lo&g!`J(+J-@t-|eojyF~%P+{P z7^Qj`+wI>9D|0kw5qI`_7zbz~7}sPMrK!5lpyn!K$&Yn`I(ANTp>ojAs#nFg+rSxa zcncm{Em^Z$6Ap-ClY@|-H35O_)WmRrI1%sfGYL~yWOIY zgZ1adqj$A`g$C`k(E@A8BLhVHdl4Oym6c7(a2g05k50}Asg6fWn^)h~{j#R$(R)xS zXQzz5S+TVPh3u?hr30EMihles|ieY0yWXp5~ui?TlLOZE?*ta*cvZT!)l8M2WMZxpzAGbrY=H)T|C z>sWi23vK3<=ChXJ*o67?ZNH5xM;0hbqkb+tT-$Zn&8U7OM#26c=d|q+3&z(!EJ-?8 zXGsT}id!9a&pP{TqHTuulQ`8BaryeE&&mOA3_9~i=1Orw*v$>+;J_^p@)mK%*^5q$ z@IyA7)F!kh9L1JZw|za}dkf~(WPVJoPTLdrhYyqHm*)p3rn^X+gs{nZZKv8$l;6>F z-pI{mTtbN!tHFmb%4&hrN?wDa)f8R`rw+H?XR?N`5JVB*9HV{w^-5-Lo@v^^!O6z> z;@a;@b8y!H7w-VB!3*PGNJ2JiHlFpZlIL3AkFTYr+0OBhT-V3H@Wi=(Au_kV(rSR@ zl?i9f@e1BdU&*{(;30b%RXa+U8CW!x^M09>bFQRR?q&Y#mU!qCR6U@vyLFH<=&Er> zeCbB&APasu)P}LTJR?K9Qn>Y*Z|(Zaeng4HQTv?dr;bhEc~4W|QIW?|87re@-{T;h z%Tjj2PFx~hs^T1lwLfi*ty8ih`%^pTVEc7?OeWW$w9x7aWy+WnRq|=@jWx}PUaD(; zygAzJ7vnF6^L9x#D^htLy~rCr!z#e(ky@7-(-s+FYQpe6ul8w@%F|0@s z(J+DPagza2(@|D7=5Z4n>c>$o+`@}h((lY+5{EScU-Y>@qO%3FiQNf;-r`U~j0X0K zo$e^HH$Pu{KUu}rq~9-Dd@*+iF8GQ1H7{=e%$SK!RccOYGyc{C{#sxM<1+lyMa777 zhiRN%#yviJmH*>`GJKAjLAX6Gk8ZB{eUIKFJ*R<-?wS(opIpQ33vHR_PSq;2W7ya+ z@p*=2vja{n7(bKmWt}}aR2V-l$JGUH2FHMT=H(eJ0N-3ipS#tfr(WfZR>47HUodUc zxxS@sSjk&mvvgFDA|uYsT3RpP19hXA?^8M?DL(}u6ZGGPir&#s-AD_wz>A6 zb9`xn3Rs2~HrZT);JltZ>lcizq@SyHM?xGAR!dT2%8vS(xtrsz@$&9t)>h(i1TLp6 zVd;c>>gUDpZ_Dq0&?M1upfr*Wb60Whqf$qUtbQcT+()bMK8y=@_z1vd(OOQ~9DwGD zRs9S44~1*F4rrT28O!=flY=)?eH~SP-o(}9U?9$9hJMiRL&2q%vRTE$`6XRU7|WM* zxQX<8vG($>!MORy&-w*+(E0y{Zp!ReboXd5sU?Xmmt$KN9v497Hm>2CW$jZm=Z+{! zyq2}6KV7d8Lyfb?G}i^~WA0?fcq*Ok)360U4>FyJzP>uD*B3nM_sB?+(Lb`eqD6LB zSL=FCQZ^D#W4N`52l?N z0Q_fq4FvsDf^HL$#5s&WFzxMwl&HOjsa|Kn}R_WBRk-ec03>27T7Gr@7k0QBRb zD?4+j@!K3a>HgCk(*9rOkWBR-=1?r7?VXAYig#Ip5#z$y*EW_^9XW}F)KUWmV(qJWpU-5Qq&ktRyZEv}JO0GRG{%;1! zZsWHboJiMJPdwx97H}tWMDnIbuaA-K>xHPVoYekK=^SM&8v2aUOwsoFkCt*iTcl9? z7hIN<-krXMsJ-bc`46txUs;i^VjppLZive*FP;Z5_@B0b*HS4IS`5ShDxF5!r|$<| zHCcwI`Er*5^PF|gy4`{*cf@0VbXESL^K}|F!gO`7%$bO-^-bBIM|0lIRx1kf*ZCeX zlN*$y4>qRoO^!xd(}kpYCAJ3`pk+3WAUa;i9gCvbJmMNYQFn>wHKFgsJC=>IiaDxx zX-}*^E^~eG*r>dPKDequ2GLC-(;X8rLB@|Rx-c|{s0H>K3P$&;N8C|1%9XlX=LCep z0n6`S>6|MFY7mjlR5TWzo+&L_Yc5(sRJ0L@t!Cn|arZS(;#QW5z?P|84|(P&>VNj^ z?S0y6YDxEJ(kkX3d}^fPu%Q!msH}oJH-hocyIIYdjoyJD-N5?sxO*+0HcYxc&p{s2 zes)-gqKSfb7DwO_R4!}w!i1y4)5I4AFK>t-!R zbCs2XLLt7s!U}zIOmUJF7GWO~U7~@lv?z&1!O*!@f?o7Ix<}W!mbJ1v;IN2Moqt@Y zmyqQ>p}0ZcPaeV5FlVpqPY_+Nb1qe(Un`+;lAyTAx8Fq+Y>}gt=DNh(TS-X1r*~8$ zu*lAc2p+Zwlo}aLuaAo=g*|bC<|D6`Mxy?me-_zum91K1Kz(?1j=JT6>K7$)T~K8DBTXdX)8u zf+n)e94&?hN!%oc+xxTLt=NJ&1XkMr8 zrQkP5BCwaza04Vgw;##(WsW_sz90?jCm9P+eDtb@HA^u;JxbL)AwwjlbGc%m{ za{^LSyf~9o3q*2z4GYE-h0A*@a_*=|15~`%X;y@p-y=ct%t3L-0DNPLDuO%sZvT$Y z>9=U9Jia@C$MyzrNlM{g`$MSQ>c31N@mt%)hItXudqb*POt2-+{F-UiN|U7P@RWEU z_RJxoG4)MjEp#H~y&r=@zIAlmz*;tnithS%|DJ!jOu6)xoFd zobRS5=4}lmMP}m({w-Mr0_g(=3nz$`B*)UAv8#uUUo|y(mvp^E7Zm67gu^HySpQAz zv3M#Dj7;Ji#?lN{qm!f)gQ_Mo1XT{9N|;fd=cOpDG~81Kc@)thq5u>K$5Gets`^Vj zx|270mbF7HN|2k%*1IcV0ZpKE#`8r~2 zMs_SPAA$UujUAa^8U8A>K5(u7m09WkB6Dcse~?+sGXq)nyv(N-f*>A3UhUdV=*Zs^ zn&F=kns1N0|B}!Q|0SWfLHkcF9SVz*)EZec8fJ$1@a5N#c9}EyjX9QoVov7 aJ; zU@o(iBbFfc6|a3(>|RhBH|tSQ97eV4+HM64{8qv0zgI9+VJ86>_7bqo=;Xnt;aFA( zuhGf08~vnPZgg=2vgM`oKUaWTXT7?u5ykT<>NL;hHCW@0=9|2FYmdq8sVvw3YkAQ# z<`X@2_xLCXs4Et!z!xxuzQvHcs_XVdI{dMsT?Ys2i7t#O4hZdJ;M#8)IQ4r5MqyL8 z=#yTF3oku>&c#8F<`(()5xA%z`aVPXi&Vh;9e8gWJFwq;&r77?018mdE6K=8Do_Mi zmJUA~@45+u+43R>u>AoTk)XJL-5@KFLpvR;3I4N#jf@U&Lk|7x9ZSdYKkQiFmd=LY zj~N}lR@X0q#=2yDZQHG2$Zr)a@_Pka?=FB#MtcjOgV?|CV3F$M8ERh5RtrO7(Rqmi z!0tM3!bdNPoB76({BD?pyLiT|Bb^J_mL&P|e^hfRBTXZWG+QrTR^Qo8vwD8r zP34<@+f6Ut-CYk%_tt|ktAD?o>Nx!u;sWA+RNqcgrAVrgmHdK&CVdEaRM@xGfy|<3 zsb@{z7WHCe^o*Xs#^+Zj(~sUvur_E*pC`|Ek#6Q9E*u)0SnV5WEoJ2?KSI15tJ)aS zu*?>J^s0D%>^jTc3_{M$hR~ZFQ0a`@%vd1d0%v{B12S_Ni>CQ9CN&aii|@Gj|%1?c)m*pi@%k9Y123wD%?ud(b=2wQXs#?d-iO1A9VQRr0S7qTm2ck z%zTIAub|m+e$DeMB!BW!S|`EST=)A`&?}+Qy3L+-8(qXE|6`d@3t^?9_kvlS`=n`b z>umVV*%n9nPV4k^@|MEMh82;=WB#EtE2j<58CKC{aYeBo)+#tV6JK;MN4;Kb}EIO3o!NlD-J@L_^dT2Pqqe?N=T;|3bqa4~!-NJh@#6q!b*q}$;{Fhf} zAFd2fS)RD*bfEN)&wl`)PfSlVmY2pf%&clo1BV~>4ikaH*wZrQ6yO=7O0_-?`LN!i zr}>^$HEMkx7Y{! z$vdU@a1QK``d?yDRzl)V)#8~R#tK8kzK^Hvs>UsH&4G7-0V0GlJ(%LE1FNDE2rC}b zlD1x#kyU3OT5?Rk$#&SzG?Ttjn)RsVEu!#l9?@gxQfGln$>-EY=SvMP@fk8_MmcERA-qos>tE63vFe@7LoCUOsS5C6h+^ znx|N+?tLm($CO#XpFfQqDw>}k>}t)&#Lr5f5>B!cfGv!yj!aM3sA#0@{8678+d;Yx zs$Y@2Wud?Lr7Iz}nP2jNStOZpAIUXmaP#|?(N^=AUYkZ)^(Z;?C23T#bIp9^YV(Va zfi0v}0%3EW96*=!ENMWM$;G(}`X~xt7jFJR+W5esb=j;u@li?j4^2#cM)9=K__|7? zUVY{Uz6{BAZND725{`KKi?W;yuJU!n;yBoYoPyvKm`;lJsB0Z;_HsB|C)rC+80cMI zSrP~}uu-?8rXM|67Cys3Uy;Mx2WPQkY(GF=VurH}uVu)5!&wF`X1(JysxszbmFBgu zoKT21y(4c?W)HTOSVyMbhade&mG|te3v;$8eV!u*Do>A(<*;yP;Csz!6?pPXis*Yz zPw`KE%njmV&w5x@siX?O=QN8Ovq<3@*nQSw9koO=+UF4gSu6_B{?^`{t^aCMX zitVfMr*5Lx?~5}WJp9I{SV|*PI3)_02OE@>}Rm=+8h>v3fMxQyZ)L%lPcY zm#$<}OxwPo`XChL>&nYmbs}bHyNU{j+YZyN*HQ3)?R#}?)*(dGMDS_%&UXvy8tS*W zwTz|vdS%73*=-X9z_!tiisKibCf#GHq7(bVI2LMAW6|(~mwjSuZS?g7p3>Rcr`EGh zKzhIQez{HN*-X^St*eWC(y6m>{j4NnAQ>6Kpjq-U`&$T`sVyPrRrCDYHmevL5&zq< z1HmDID4%QP`DxgSucKdwBnNKao+jsi99rtXyuSZ<9))-kJg2uU60U{JhA>;k%5jbjO?woET7G=1+1|Vl1x|K*?ipX zSz~2Wxvxm*xAZzxgXQGs3a?0TyHLgUgJb*YuS~%ssbh~Q@)O^l+gt)}!}at&`r|o) z{Ki_)##Jl#UZK`(WZc`_#?ND#$y>E67EOXLK!biT6I7338cs^onJ!lc%Sl7WW{5Ij zTAAFRtWouh7cph}zf|@;TE|2=*A@qV`5XA;IuausY^4p z&qqoh<#@b#Wk7djfKwmEBF3=triaREniV6p!q~>Gi(*5dOsH30_BiUISlN1{L|aN| zC^yWxo;DsrqD&wwlZ8EbIyZGPIqn*xn%)G$C{@$>{MzRs6^phHw{q&g?pvA6whmcm zf`|xHpm|_ncZH2hCvtQE{t68GgyAUOOo5!nYPyZ(xONj1K=DfKK8}wovDag!_bHH! zP?Xi*~7*7;hfDs;kpKA^x z3+6pswW^eBzv5a5Mm>5HI1`L=*3b#2-Ovv)9M}T?LD-bS#_azP*6_cCWtCEX=%AVS zf^!=MgO5iJ2Cb!y)dk3CsGA$%qFK0Ms< z^^~{7jja%vZ~g(RND$?spMUsP498TrI823Nu7vIp(l#>c6(aIzhjJ?T<9@_x@lb8( zXT0qZx_?KW!Um?WwF1J$?FzR@QTS8QorAxayM)?fE=HBY_A7JIM|A=XmELdiv+dy0 zTHdcf^HIo|nK}aUq_O<(dh-3`@pnBRWZ&q3V{tdO3?=^pmI$UK{sqiRy`_YiazlZk zhXDFOI88;7QQRFt#iG|`MYjzIzMt;^0N#z;S zaGHyNrVG>EjRoUh#1Vf|*F5Dd^>^y>{+GHyB+6@u>e@PycAaThl>$Wql3Hk)PkBcG z&&O;XIL$maFk;L^@@t#iVBx5iiK_0%JUNEaaK7x@`aCAR9dIKr6RWM zxQ14aE#+H0_MMhnQ8j(4Q?vM9jSKtg|7v1QQg;gID-KEEl?pUtN)Grealt2_G5fw${~G ztR@m72vkCm1WvtJ7M`K&M!wH1>wT#e*3qV7i1Lv~Tz3$}D74#USCoV+y%+*}0L#K< zgtmf2O<5#sy?F+61(`p6qaSseOAoEhHOQ1K1&;wQEx|Lukbz|z`gCm5j4m2y2ZN*0 zHc&X-)&1gFW)W;5Kitl!To@4pDvr6p-48{`e+0ve<>AH0g|X2f7Pu=q-2av@>m@LZ z+tvqOOa(7~D~zy-QHP%5m$E7y9$<{Tz5Z#9k*{3QfLYN8j>s_VzI~%OCkl;=xc-QN zsa!S!94b{Y$!!(I3N2i@jt0-dKp*m98qiY)?O>R;1A0G<7=%RMSY+4hXOEH~3Fx&K zfuRvKar>PUAlrn6()k;i*^G6CQ=*+~w(N!VXVXL7E7($u5shl1-hy>g7 z%tRUa?2H@J&vC0!oIebL`0$?l3m7d^x%A(GVg3QwR?|k$0Z#0Y6Px*5{A$vlv&Jrh zE!l&X0OpCg27)9gMu9aFz!(mY541S~%mXV^`inFHU9|nLq>&-y&m!8Z(jT>wE(>!)gT zaxO_B(2kdhQL2tns;T;D7Rduq?cf^_SW6i@G$J_$Obmw*&1j&qh+S{&j$kCX33qcp zkqS&SV}RBk`NWh=G%E<22%Coi`9T??S3a7MJMF&Xf^}Y)^B?&kVuW`2dw!@M_}~1v z3TO4w(<|2EAfm7F>;%YtaICA1D4bsb&ff{eMsw~2#cPl2F-`(-IuSM~79xTzr-R#} z%XX4wcM$);2LcmM|B8>p`5PqrI2zDYoEJJeT)q0K__JK=d44L0v;-BbTA2U7E>*@& zD&6dxbCL)yC~dfumLSdR0ua?oK|6c6C8U7Lh$mn{sACdegoz)?B+$*&PmdEdPt6KE zd{%}H!%|?JeGu0k@!;5ePUVazr0x=(u-b;9A7CN)98S9*)3n(L~@5h z+{jFhwkA|HtiJat1J;3 znA|NVu99OSY@ave);Rg*=WD_wo5N?-U^!<*ZaLAH#CD>&2M9XqF2FaU+mdDv9;vL! z8aHD)@#+F2BQV()7+tzUMNN`L?~L}bsU;p9%L)iV_!G;BzW*k$oaiBrc#G43C-;6vHYEa>aqA6Tn4#?#o%c z4V2{1khkWBTK^LAtXKZOD(EDfRfo8rsXQ$hBuxJhNOipOG*{(hLdATpCszuh~urUOVsAGn}?x(9lmARqH6F>*v z-046^GSyz3@+E-N|1nPg-a+JR4|d{o_K5Grp|fBZQ8CDm^Ut+H#t7~8_gKLFLo6&S zVS7$6qpN74&ac9O-(iIYHw|N%%6Hl(7JTs3ZVwiH*#l>r0LK3Xj>}`OAZ2vXUcY8N zCWbo%n_mKbTOW%%vjeUn2K`5%!WlW7ZT_yN|NpAF8p5XV&F4t_kO-9BSG-U7&%EF1 zo4tuE`z7yNK;_c^nD^9CxVyjR{UvnMv5*0-a`_Q2%&hLN3JsYS7mcClo+>|lBJaSRIQzqt!H%4i4f`gJH0jhv& zal}rWW8bg_X2~p-3ty(hdArT0avk$$q!cMs|0zh%yQTZR$MLf8S{;Mo0Dk03nR}?EnA( literal 0 HcmV?d00001 diff --git a/examples/policy/fixture.json b/examples/policy/fixture.json new file mode 100644 index 0000000..34c8865 --- /dev/null +++ b/examples/policy/fixture.json @@ -0,0 +1,95 @@ +{ + "description": "Synthetic single-sample cohort for the v0.1.0 policy demonstrator. Genotypes are synthetic; see make_fixture.py.", + "seed": 20260925, + "assembly": "GRCh38", + "participants": { + "P001": { + "consent": [ + "DUO_0000042", + "DUO_0000043" + ] + }, + "P002": { + "consent": [ + "DUO_0000042", + "DUO_0000043" + ] + }, + "P003": { + "consent": [ + "DUO_0000006" + ] + }, + "P004": { + "consent": [ + "DUO_0000042" + ], + "withdrawn": true + }, + "P005": { + "consent": [ + "DUO_0000007" + ] + } + }, + "files": { + "P001.vcf": { + "participant": "P001", + "records": 34 + }, + "P002.vcf": { + "participant": "P002", + "records": 23 + }, + "P003.vcf": { + "participant": "P003", + "records": 27 + }, + "P004.vcf": { + "participant": "P004", + "records": 26 + }, + "P005.vcf": { + "participant": "P005", + "records": 29 + } + }, + "loci": { + "brca1": { + "chrom": "chr17", + "start": 43044295, + "end": 43125483 + }, + "apoe_e4": { + "chrom": "chr19", + "pos": 44908684, + "ref": "T", + "alt": "C", + "id": "rs429358" + }, + "apoe_e2": { + "chrom": "chr19", + "pos": 44908822, + "ref": "C", + "alt": "T", + "id": "rs7412" + } + }, + "requesters": { + "gru": { + "assignee": "https://example.org/party/research-consortium", + "purpose": "DUO_0000042", + "label": "General-research consortium" + }, + "alz": { + "assignee": "https://example.org/party/alz-consortium", + "purpose": "DUO_0000007", + "label": "Alzheimer's consortium" + }, + "clinical": { + "assignee": "https://example.org/party/clinical-genetics", + "purpose": "DUO_0000043", + "label": "Clinical genetics lab" + } + } +} diff --git a/examples/policy/make_fixture.py b/examples/policy/make_fixture.py new file mode 100644 index 0000000..eb5c1b3 --- /dev/null +++ b/examples/policy/make_fixture.py @@ -0,0 +1,168 @@ +#!/usr/bin/env python3 +"""Generate the policy demonstrator's synthetic cohort. + + python3 examples/policy/make_fixture.py [OUT_DIR] # writes P001.vcf ... P005.vcf, fixture.json + +Five participants, one single-sample VCF each. Positions are GRCh38 and fall in +real loci -- BRCA1, APOE, and background sites on chr1 and chr20 -- so the region +and variant selectors are exercised against real coordinates. Two variants are +real and named: rs429358 (the APOE e4-defining SNP) and rs7412. Every other +allele, and every genotype, is synthetic and drawn from a fixed seed: the +genotypes identify no one, which is the point of using a fixture for a privacy +demonstration. REF alleles other than the two named variants are not checked +against the reference genome. + +The site catalogue also plants the cases the tests need: + * records one base outside each end of the BRCA1 window, so the region + bounds are tested as inclusive; + * a decoy at rs429358's position with a different ALT, so the variant + selector is tested on alleles and not on position alone. + +The output is deterministic: rerunning it rewrites byte-identical files. +""" + +from __future__ import annotations + +import json +import random +import sys +from pathlib import Path + +HERE = Path(__file__).resolve().parent +SEED = 20260925 +ASSEMBLY = "GRCh38" +FILE_DATE = "20260925" + +CONTIGS = {"chr1": 248956422, "chr17": 83257441, "chr19": 58617616, "chr20": 64444167} + +BRCA1 = {"chrom": "chr17", "start": 43044295, "end": 43125483} +APOE_E4 = {"chrom": "chr19", "pos": 44908684, "ref": "T", "alt": "C", "id": "rs429358"} +APOE_E2 = {"chrom": "chr19", "pos": 44908822, "ref": "C", "alt": "T", "id": "rs7412"} + +# Consent per participant, as DUO terms. P004 withdrew. +PARTICIPANTS = { + "P001": {"consent": ["DUO_0000042", "DUO_0000043"]}, + "P002": {"consent": ["DUO_0000042", "DUO_0000043"]}, + "P003": {"consent": ["DUO_0000006"]}, + "P004": {"consent": ["DUO_0000042"], "withdrawn": True}, + "P005": {"consent": ["DUO_0000007"]}, +} + +REQUESTERS = { + "gru": {"assignee": "https://example.org/party/research-consortium", "purpose": "DUO_0000042", + "label": "General-research consortium"}, + "alz": {"assignee": "https://example.org/party/alz-consortium", "purpose": "DUO_0000007", + "label": "Alzheimer's consortium"}, + "clinical": {"assignee": "https://example.org/party/clinical-genetics", "purpose": "DUO_0000043", + "label": "Clinical genetics lab"}, +} + +BASES = "ACGT" + + +def site_catalogue(rng: random.Random) -> list[dict]: + """Every site any participant may carry, before sampling.""" + sites = [] + + def snv(chrom, pos, tag, ident="."): + ref = rng.choice(BASES) + alt = rng.choice([b for b in BASES if b != ref]) + sites.append({"chrom": chrom, "pos": pos, "ref": ref, "alt": alt, "id": ident, "tag": tag}) + + # BRCA1: twelve sites inside the window, and one just outside each end. + for pos in sorted(rng.sample(range(BRCA1["start"] + 1, BRCA1["end"]), 12)): + snv("chr17", pos, "brca1") + snv("chr17", BRCA1["start"] - 1, "brca1-edge-outside") + snv("chr17", BRCA1["end"] + 1, "brca1-edge-outside") + snv("chr17", BRCA1["start"], "brca1-edge-inside") + snv("chr17", BRCA1["end"], "brca1-edge-inside") + + # APOE: the two named variants, a decoy at the e4 position, and neighbours. + sites.append({**APOE_E4, "tag": "apoe-e4"}) + sites.append({**APOE_E2, "tag": "apoe-e2"}) + sites.append({"chrom": "chr19", "pos": APOE_E4["pos"], "ref": "T", "alt": "G", "id": ".", + "tag": "apoe-e4-decoy"}) + for pos in sorted(rng.sample(range(44905791, 44909393), 4)): + if pos not in (APOE_E4["pos"], APOE_E2["pos"]): + snv("chr19", pos, "apoe-region") + + # Background, governed by consent alone. + for chrom, lo, hi in (("chr1", 1_000_000, 240_000_000), ("chr20", 1_000_000, 60_000_000)): + for pos in sorted(rng.sample(range(lo, hi), 10)): + snv(chrom, pos, "background") + return sites + + +def carried(rng: random.Random, participant: str, sites: list[dict]) -> list[dict]: + """The sites one participant carries, with the scenario's cases forced in.""" + forced = { + "apoe-e4": participant in ("P001", "P003", "P004", "P005"), + "apoe-e4-decoy": participant == "P002", + "brca1-edge-outside": participant == "P001", + "brca1-edge-inside": participant == "P001", + } + chosen = [] + for site in sites: + if site["tag"] in forced: + keep = forced[site["tag"]] + elif site["tag"] == "apoe-e2": + keep = participant in ("P002", "P003", "P005") + else: + keep = rng.random() < 0.7 + if keep: + chosen.append(site) + return chosen + + +def write_vcf(path: Path, participant: str, records: list[dict], rng: random.Random) -> None: + order = {c: i for i, c in enumerate(CONTIGS)} + records = sorted(records, key=lambda s: (order[s["chrom"]], s["pos"], s["alt"])) + lines = [ + "##fileformat=VCFv4.3", + f"##fileDate={FILE_DATE}", + "##source=vcf-rdfizer-policy-demonstrator-fixture", + f"##reference={ASSEMBLY}", + ] + lines += [f"##contig=" for c, n in CONTIGS.items()] + lines += [ + '##INFO=', + '##FORMAT=', + '##FORMAT=', + "#CHROM\tPOS\tID\tREF\tALT\tQUAL\tFILTER\tINFO\tFORMAT\t" + participant, + ] + for site in records: + depth = rng.randint(20, 60) + genotype = "1/1" if rng.random() < 0.3 else "0/1" + qual = rng.randint(50, 99) + lines.append("\t".join([site["chrom"], str(site["pos"]), site["id"], site["ref"], site["alt"], + str(qual), "PASS", f"DP={depth}", "GT:DP", f"{genotype}:{depth}"])) + path.write_text("\n".join(lines) + "\n", encoding="utf-8") + + +def main(out_dir: Path = HERE) -> None: + out_dir = Path(out_dir) + rng = random.Random(SEED) + sites = site_catalogue(rng) + files = {} + for participant in PARTICIPANTS: + records = carried(rng, participant, sites) + write_vcf(out_dir / f"{participant}.vcf", participant, records, rng) + files[f"{participant}.vcf"] = {"participant": participant, "records": len(records)} + + fixture = { + "description": "Synthetic single-sample cohort for the v0.1.0 policy demonstrator. " + "Genotypes are synthetic; see make_fixture.py.", + "seed": SEED, + "assembly": ASSEMBLY, + "participants": PARTICIPANTS, + "files": files, + "loci": {"brca1": BRCA1, "apoe_e4": APOE_E4, "apoe_e2": APOE_E2}, + "requesters": REQUESTERS, + } + (out_dir / "fixture.json").write_text(json.dumps(fixture, indent=2) + "\n", encoding="utf-8") + for name, info in files.items(): + print(f"wrote {name}: {info['records']} records") + + +if __name__ == "__main__": + main(Path(sys.argv[1]) if len(sys.argv) > 1 else HERE) diff --git a/examples/policy/policy.ttl b/examples/policy/policy.ttl new file mode 100644 index 0000000..4c73a37 --- /dev/null +++ b/examples/policy/policy.ttl @@ -0,0 +1,73 @@ +# Policies for the demonstrator cohort (docs/policy-demonstrator.md §2, §4). +# +# One consent policy per participant, attached to that participant's file, and +# one cohort policy with two rules that cut across files. Every rule reads the +# graph (odrl:read); deny wins (odrl:conflict odrl:prohibit); anything no +# permission covers is withheld. + +@prefix odrl: . +@prefix vcfp: . +@prefix obo: . +@prefix ex: . + +# --- Cohort-wide rules ------------------------------------------------------ + +ex:brca1 a odrl:Asset , vcfp:GraphSelection ; + vcfp:selector [ a vcfp:RegionSelector ; vcfp:assembly "GRCh38" ; + vcfp:chrom "chr17" ; vcfp:start 43044295 ; vcfp:end 43125483 ] . + +ex:apoe-e4 a odrl:Asset , vcfp:GraphSelection ; + vcfp:selector [ a vcfp:VariantSelector ; vcfp:assembly "GRCh38" ; + vcfp:chrom "chr19" ; vcfp:pos 44908684 ; vcfp:ref "T" ; vcfp:alt "C" ] . + +ex:cohort a odrl:Set ; + odrl:uid ex:cohort ; + odrl:profile ; + odrl:conflict odrl:prohibit ; + # Actionable incidental findings: readable only for clinical care. + odrl:prohibition [ odrl:target ex:brca1 ; odrl:action odrl:read ; odrl:assignee odrl:All ; + odrl:constraint [ odrl:leftOperand odrl:purpose ; odrl:operator odrl:isNoneOf ; + odrl:rightOperand obo:DUO_0000043 ] ] ; + # APOE e4: readable only for disease-specific research. + odrl:prohibition [ odrl:target ex:apoe-e4 ; odrl:action odrl:read ; odrl:assignee odrl:All ; + odrl:constraint [ odrl:leftOperand odrl:purpose ; odrl:operator odrl:isNoneOf ; + odrl:rightOperand obo:DUO_0000007 ] ] . + +# --- Participant consents --------------------------------------------------- + +ex:consent-P001 a odrl:Set ; odrl:uid ex:consent-P001 ; + odrl:profile ; odrl:conflict odrl:prohibit ; + odrl:permission [ odrl:target ; odrl:action odrl:read ; odrl:assignee odrl:All ; + odrl:constraint [ odrl:leftOperand odrl:purpose ; odrl:operator odrl:isAnyOf ; + odrl:rightOperand obo:DUO_0000042 , obo:DUO_0000043 ] ; + odrl:duty [ odrl:action odrl:attribute ] ] . + +ex:consent-P002 a odrl:Set ; odrl:uid ex:consent-P002 ; + odrl:profile ; odrl:conflict odrl:prohibit ; + odrl:permission [ odrl:target ; odrl:action odrl:read ; odrl:assignee odrl:All ; + odrl:constraint [ odrl:leftOperand odrl:purpose ; odrl:operator odrl:isAnyOf ; + odrl:rightOperand obo:DUO_0000042 , obo:DUO_0000043 ] ; + odrl:duty [ odrl:action odrl:attribute ] ] . + +ex:consent-P003 a odrl:Set ; odrl:uid ex:consent-P003 ; + odrl:profile ; odrl:conflict odrl:prohibit ; + odrl:permission [ odrl:target ; odrl:action odrl:read ; odrl:assignee odrl:All ; + odrl:constraint [ odrl:leftOperand odrl:purpose ; odrl:operator odrl:isAnyOf ; + odrl:rightOperand obo:DUO_0000006 ] ; + odrl:duty [ odrl:action odrl:attribute ] ] . + +# P004 consented to general research and later withdrew. The withdrawal is a +# prohibition, so it overrides the permission under deny-wins. +ex:consent-P004 a odrl:Set ; odrl:uid ex:consent-P004 ; + odrl:profile ; odrl:conflict odrl:prohibit ; + odrl:permission [ odrl:target ; odrl:action odrl:read ; odrl:assignee odrl:All ; + odrl:constraint [ odrl:leftOperand odrl:purpose ; odrl:operator odrl:isAnyOf ; + odrl:rightOperand obo:DUO_0000042 ] ] ; + odrl:prohibition [ odrl:target ; odrl:action odrl:read ; odrl:assignee odrl:All ] . + +ex:consent-P005 a odrl:Set ; odrl:uid ex:consent-P005 ; + odrl:profile ; odrl:conflict odrl:prohibit ; + odrl:permission [ odrl:target ; odrl:action odrl:read ; odrl:assignee odrl:All ; + odrl:constraint [ odrl:leftOperand odrl:purpose ; odrl:operator odrl:isAnyOf ; + odrl:rightOperand obo:DUO_0000007 ] ; + odrl:duty [ odrl:action odrl:attribute ] ] . diff --git a/examples/policy/run_demo.sh b/examples/policy/run_demo.sh new file mode 100755 index 0000000..7b56556 --- /dev/null +++ b/examples/policy/run_demo.sh @@ -0,0 +1,49 @@ +#!/usr/bin/env bash +# The v0.1.0 policy demonstrator end to end, without Docker: +# attach the policies, evaluate one release view per requester, check each view +# against the source VCFs, and print the decision grid. +# +# examples/policy/run_demo.sh [OUT_DIR] # default: ./policy-demo-out (must not exist) +# +# PROFILE=condensed runs it on the condensed graphs instead of the expanded ones. +set -euo pipefail + +HERE="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)" +OUT="${1:-policy-demo-out}" +PROFILE="${PROFILE:-expanded}" +POLICY="$HERE/policy.ttl" +RDF=("$HERE"/converted/"$PROFILE"/P00*.nt.gz) +VCF=("$HERE"/P00*.vcf) +POLICY_CMD=(python3 "$HERE/../../vcf_rdfizer_policy.py") + +[[ -e "$OUT" ]] && { echo "error: $OUT exists; choose a new directory" >&2; exit 2; } +mkdir -p "$OUT" + +echo "== the policy" +"${POLICY_CMD[@]}" explain --policy "$POLICY" + +echo; echo "== attach: policies in the graph" +"${POLICY_CMD[@]}" attach --rdf "${RDF[@]}" --policy "$POLICY" -o "$OUT/cohort-annotated.nt" + +# One view per requester in fixture.json: key, assignee IRI, DUO purpose. +while read -r key assignee purpose; do + echo; echo "== $key ($purpose)" + "${POLICY_CMD[@]}" evaluate --rdf "${RDF[@]}" --policy "$POLICY" \ + --assignee "$assignee" --purpose "$purpose" -o "$OUT/views/$key" + "${POLICY_CMD[@]}" check --view "$OUT/views/$key" --policy "$POLICY" --vcf "${VCF[@]}" +done < <(python3 -c 'import json,sys +for k, r in json.load(open(sys.argv[1]))["requesters"].items(): print(k, r["assignee"], r["purpose"])' "$HERE/fixture.json") + +echo; echo "== decision grid (records released per file)" +python3 - "$OUT/views" <<'PY' +import json, sys +from pathlib import Path +views = sorted(Path(sys.argv[1]).iterdir()) +summaries = {v.name: json.loads((v / "summary.json").read_text()) for v in views} +files = sorted(next(iter(summaries.values()))["files"]) +print("requester".ljust(10) + "".join(f.split("//")[1].ljust(10) for f in files) + "triples withheld") +for name, s in summaries.items(): + cells = "".join((str(s["files"][f].get("released", 0)) if s["files"][f]["released"] else "withheld").ljust(10) + for f in files) + print(name.ljust(10) + cells + str(s["triples_withheld"])) +PY diff --git a/pyproject.toml b/pyproject.toml index 0db051c..fffcc40 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -37,6 +37,7 @@ Issues = "https://github.com/ecrum19/VCF-RDFizer/issues" vcf-rdfizer = "vcf_rdfizer:main" vcf-rdfizer-rules = "vcf_rdfizer_rules:main" vcf-rdfizer-link = "vcf_rdfizer_link:main" +vcf-rdfizer-policy = "vcf_rdfizer_policy:main" [project.optional-dependencies] dev = [ @@ -45,13 +46,14 @@ dev = [ ] [tool.setuptools] -py-modules = ["vcf_rdfizer", "vcf_rdfizer_gzip", "vcf_rdfizer_rules", "vcf_rdfizer_link", "vcf_rdfizer_vocab", "vcf_rdfizer_cottas"] -packages = ["vcf_rdfizer_data", "vcf_rdfizer_data.rules", "vcf_rdfizer_data.linkers", "vcf_rdfizer_linking"] +py-modules = ["vcf_rdfizer", "vcf_rdfizer_gzip", "vcf_rdfizer_rules", "vcf_rdfizer_link", "vcf_rdfizer_policy", "vcf_rdfizer_vocab", "vcf_rdfizer_cottas"] +packages = ["vcf_rdfizer_data", "vcf_rdfizer_data.rules", "vcf_rdfizer_data.linkers", "vcf_rdfizer_data.policy", "vcf_rdfizer_linking", "vcf_rdfizer_policies"] include-package-data = true [tool.setuptools.package-data] "vcf_rdfizer_data.rules" = ["default_rules.ttl"] "vcf_rdfizer_data.linkers" = ["*/linker.ttl", "*/resolver.py", "*/genes.gff3", "*/README.md"] +"vcf_rdfizer_data.policy" = ["*.ttl"] # Vendored from the published vocabulary so SHACL validation works from an # installed package with no separate checkout. Digests are pinned in # vcf_rdfizer_data/VOCABULARY_PROVENANCE.json. diff --git a/test/test_policy_unit.py b/test/test_policy_unit.py new file mode 100644 index 0000000..65b29b5 --- /dev/null +++ b/test/test_policy_unit.py @@ -0,0 +1,382 @@ +"""The v0.1.0 policy demonstrator: profile, decisions, release views and their check. + +The decision tests pin docs/policy-demonstrator.md §4.1 cell by cell. The +release tests run the real committed fixture graphs in both sample profiles. +The mutation tests break a correct view in each way a redactor can go wrong +and require `check` to say so -- a check that has never caught a planted +leak is not evidence of anything. +""" + +import contextlib +import filecmp +import importlib.util +import io +import json +from pathlib import Path +import shutil +import tempfile +import unittest + +try: + import rdflib +except ModuleNotFoundError: # pragma: no cover - exercised only without rdflib + rdflib = None + +from test.helpers import VerboseTestCase + +ROOT = Path(__file__).resolve().parent.parent +EXAMPLE = ROOT / "examples" / "policy" +POLICY = EXAMPLE / "policy.ttl" +VCFS = sorted(EXAMPLE.glob("P00*.vcf")) +FIXTURE = json.loads((EXAMPLE / "fixture.json").read_text(encoding="utf-8")) +DUO = "http://purl.obolibrary.org/obo/DUO_" + +PREFIXES = """@prefix odrl: . +@prefix vcfp: . +@prefix obo: . +@prefix ex: . +""" + + +def request(key): + from vcf_rdfizer_policies.decide import Request + + spec = FIXTURE["requesters"][key] + return Request(spec["assignee"], DUO + spec["purpose"].split("_")[1]) + + +def write_policy(directory, body, conflict="odrl:prohibit"): + """A one-policy file around `body` (the policy's rules), for rejection tests.""" + path = Path(directory) / "policy.ttl" + path.write_text(f"{PREFIXES}\nex:p a odrl:Set ; odrl:conflict {conflict} ;\n{body} .\n", encoding="utf-8") + return path + + +FILE_PERMISSION = """odrl:permission [ odrl:target ; odrl:action odrl:read ]""" + + +@unittest.skipIf(rdflib is None, "rdflib is required") +class PurposeTests(VerboseTestCase): + def test_every_accepted_spelling_names_the_same_term(self): + from vcf_rdfizer_policies.purposes import purpose_iri + + for spelling in ("DUO:0000007", "DUO_0000007", "obo:DUO_0000007", DUO + "0000007"): + self.assertEqual(purpose_iri(spelling), DUO + "0000007") + + def test_a_term_outside_the_bundled_subset_is_refused(self): + from vcf_rdfizer_policies import PolicyError + from vcf_rdfizer_policies.purposes import purpose_iri + + with self.assertRaises(PolicyError): + purpose_iri("DUO:0000019") + with self.assertRaises(PolicyError): + purpose_iri("GRU") + + def test_narrower_purposes_fall_within_broader_consents_and_not_the_reverse(self): + from vcf_rdfizer_policies.purposes import within + + gru, hmb, ds, cc = (DUO + n for n in ("0000042", "0000006", "0000007", "0000043")) + self.assertTrue(within(ds, gru) and within(ds, hmb) and within(hmb, gru)) + self.assertFalse(within(gru, hmb) or within(hmb, ds)) + self.assertFalse(within(cc, gru) or within(gru, cc)) + + +@unittest.skipIf(rdflib is None, "rdflib is required") +class ProfileTests(VerboseTestCase): + def test_the_example_policy_loads_as_eight_rules(self): + from vcf_rdfizer_policies.profile import FileTarget, RegionTarget, VariantTarget, load_policy + + _, rules = load_policy(POLICY) + kinds = sorted((r.kind, type(r.target).__name__) for r in rules) + self.assertEqual(kinds.count(("permission", "FileTarget")), 5) + self.assertEqual(kinds.count(("prohibition", "FileTarget")), 1) # P004's withdrawal + self.assertIn(("prohibition", "RegionTarget"), kinds) + self.assertIn(("prohibition", "VariantTarget"), kinds) + region = next(r.target for r in rules if isinstance(r.target, RegionTarget)) + self.assertEqual((region.chrom, region.start, region.end), ("chr17", 43044295, 43125483)) + self.assertTrue(all(isinstance(r.target, (FileTarget, RegionTarget, VariantTarget)) for r in rules)) + + def test_anything_outside_the_v010_subset_is_refused(self): + from vcf_rdfizer_policies import PolicyError + from vcf_rdfizer_policies.profile import load_policy + + region = ('ex:a a vcfp:GraphSelection ; vcfp:selector [ a vcfp:RegionSelector ; ' + 'vcfp:chrom "chr1" ; vcfp:start 1 ; vcfp:end 2 {extra} ] .\n') + cases = { + "deny-wins is required": (FILE_PERMISSION, "odrl:perm"), + "only read": (FILE_PERMISSION.replace("odrl:read", "odrl:distribute"), "odrl:prohibit"), + "unknown rule property": (FILE_PERMISSION[:-1] + " ; odrl:refinement [] ]", "odrl:prohibit"), + "unsupported effect": (FILE_PERMISSION[:-1] + " ; odrl:duty [ odrl:action odrl:anonymize ; " + "vcfp:transform vcfp:generalize ] ]", "odrl:prohibit"), + "non-DUO purpose": (FILE_PERMISSION[:-1] + " ; odrl:constraint [ odrl:leftOperand odrl:purpose ; " + "odrl:operator odrl:isAnyOf ; odrl:rightOperand ex:x ] ]", "odrl:prohibit"), + "target is not a file": (FILE_PERMISSION.replace("", "ex:thing"), "odrl:prohibit"), + } + for name, (body, conflict) in cases.items(): + with self.subTest(name), tempfile.TemporaryDirectory() as td: + with self.assertRaises(PolicyError): + load_policy(write_policy(td, body, conflict)) + + selector_cases = { + "missing assembly": region.format(extra=""), + "start after end": region.format(extra='; vcfp:assembly "GRCh38"').replace("vcfp:end 2", "vcfp:end 0"), + "unsupported selector": region.format(extra='; vcfp:assembly "GRCh38"').replace( + "RegionSelector", "SampleSelector"), + } + for name, asset in selector_cases.items(): + with self.subTest(name), tempfile.TemporaryDirectory() as td: + path = write_policy(td, "odrl:prohibition [ odrl:target ex:a ; odrl:action odrl:read ]") + path.write_text(path.read_text() + asset, encoding="utf-8") + with self.assertRaises(PolicyError): + load_policy(path) + + +@unittest.skipIf(rdflib is None, "rdflib is required") +class DecisionTests(VerboseTestCase): + """The §4.1 grid, and the boundary cases the fixture plants.""" + + @classmethod + def setUpClass(cls): + from vcf_rdfizer_policies.profile import load_policy + + _, cls.rules = load_policy(POLICY) + + def decide(self, subject, key): + from vcf_rdfizer_policies.decide import decide + + return decide(subject, self.rules, request(key)).released + + def record(self, file="P001", chrom="chr1", pos=1, ref="A", alt="C"): + from vcf_rdfizer_policies.graphs import Record + + return Record(f"file://{file}.vcf", 1, chrom, pos, ref, (alt,)) + + def test_files_follow_each_participants_consent(self): + grid = {"gru": "✓✓———", "alz": "✓✓✓—✓", "clinical": "✓✓———"} + for key, expected in grid.items(): + got = "".join("✓" if self.decide(f"file://P00{n}.vcf", key) else "—" for n in range(1, 6)) + self.assertEqual(got, expected, key) + + def test_the_region_and_variant_rules_each_exempt_one_purpose(self): + brca1 = self.record(chrom="chr17", pos=43100000) + e4 = self.record(chrom="chr19", pos=44908684, ref="T", alt="C") + self.assertEqual([self.decide(brca1, k) for k in ("gru", "alz", "clinical")], [False, False, True]) + self.assertEqual([self.decide(e4, k) for k in ("gru", "alz", "clinical")], [False, True, False]) + + def test_region_bounds_are_inclusive(self): + at = lambda pos: self.decide(self.record(chrom="chr17", pos=pos), "gru") # noqa: E731 + self.assertEqual([at(43044294), at(43044295), at(43125483), at(43125484)], + [True, False, False, True]) + + def test_the_variant_rule_matches_alleles_not_just_position(self): + decoy = self.record(chrom="chr19", pos=44908684, ref="T", alt="G") + self.assertTrue(self.decide(decoy, "gru")) + + def test_withdrawal_overrides_every_permission(self): + for key in ("gru", "alz", "clinical"): + self.assertFalse(self.decide(self.record(file="P004"), key), key) + + def test_a_rule_for_one_party_does_not_bind_another(self): + from dataclasses import replace + from vcf_rdfizer_policies.decide import applies + + withdrawal = next(r for r in self.rules if r.kind == "prohibition" and hasattr(r.target, "iri")) + only_alz = replace(withdrawal, assignee=FIXTURE["requesters"]["alz"]["assignee"]) + self.assertTrue(applies(only_alz, request("alz"))) + self.assertFalse(applies(only_alz, request("gru"))) + + def test_an_assembly_mismatch_stops_evaluation(self): + from vcf_rdfizer_policies import PolicyError + from vcf_rdfizer_policies.decide import check_assemblies + + check_assemblies(self.rules, {"file://P001.vcf": "GRCh38"}) + with self.assertRaises(PolicyError): + check_assemblies(self.rules, {"file://P001.vcf": "GRCh37"}) + + +class FixtureViews(VerboseTestCase): + """Shared setup: every requester's view of both fixture profiles. No tests of its own.""" + + @classmethod + def setUpClass(cls): + from vcf_rdfizer_policies.graphs import load + from vcf_rdfizer_policies.profile import load_policy, policy_digest + from vcf_rdfizer_policies.release import evaluate, write_release + + cls.policy_graph, cls.rules = load_policy(POLICY) + cls.tmp = tempfile.TemporaryDirectory() + cls.graphs, cls.views = {}, {} + for profile in ("expanded", "condensed"): + graph = load(sorted((EXAMPLE / "converted" / profile).glob("*.nt.gz"))) + cls.graphs[profile] = graph + for key in FIXTURE["requesters"]: + release = evaluate(graph, cls.rules, request(key)) + out = Path(cls.tmp.name) / profile / key + write_release(release, out, policies={r.policy for r in cls.rules}, digest=policy_digest(POLICY)) + cls.views[profile, key] = (release, out) + + @classmethod + def tearDownClass(cls): + cls.tmp.cleanup() + + +@unittest.skipIf(rdflib is None, "rdflib is required") +class ReleaseTests(FixtureViews): + """Evaluate the committed fixture graphs, in both profiles, and check every view.""" + + def test_every_view_passes_its_check(self): + from vcf_rdfizer_policies.check import check_view + + for (profile, key), (_, out) in self.views.items(): + with self.subTest(profile=profile, requester=key): + self.assertEqual(check_view(out, POLICY, self.rules, VCFS), []) + + def test_a_view_partitions_the_graph(self): + for (profile, key), (release, _) in self.views.items(): + self.assertEqual(len(release.view) + release.triples_withheld, len(self.graphs[profile])) + + def test_the_withdrawn_file_leaves_nothing_behind(self): + for (_, key), (release, _) in self.views.items(): + self.assertFalse([t for t in release.view if "P004.vcf" in str(t[0]) + str(t[2])], key) + + def test_the_manifest_records_the_request_and_what_was_withheld(self): + release, out = self.views["expanded", "alz"] + manifest = (out / "manifest.ttl").read_text(encoding="utf-8") + for expected in ("governed release; not anonymization", "DUO_0000007", "odrl:attribute", + f"vcfp:recordsWithheld {sum(not d.released for _, d in release.decisions)}"): + self.assertIn(expected, manifest) + self.assertEqual(json.loads((out / "summary.json").read_text())["files"]["file://P004.vcf"]["released"], False) + + def test_a_release_is_never_written_over_another(self): + from vcf_rdfizer_policies.release import write_release + + release, out = self.views["expanded", "gru"] + with self.assertRaises(FileExistsError): + write_release(release, out, policies=set(), digest="") + + def test_attach_makes_policies_queryable_alongside_the_data(self): + from vcf_rdfizer_policies.release import attach + + graph = rdflib.Graph() + for triple in self.graphs["expanded"]: + graph.add(triple) + counts = attach(graph, self.policy_graph, self.rules) + self.assertEqual(counts["https://example.org/policy/demo-cohort/apoe-e4"], 4) # P001, P003-P005 + # docs/policy-demonstrator.md §5: records a general-research collaborator may not receive. + rows = graph.query(""" + PREFIX odrl: PREFIX vcfc: + SELECT DISTINCT ?chrom WHERE { ?r a vcfc:VCFRecord ; vcfc:chrom ?chrom ; odrl:hasPolicy ?p . + ?p odrl:prohibition ?rule . }""") + self.assertEqual(sorted(str(c) for (c,) in rows), ["chr17", "chr19"]) + + +@unittest.skipIf(rdflib is None, "rdflib is required") +class MutationTests(FixtureViews): + """Break a correct view each way a redactor can fail; `check` must catch every one.""" + + def mutate(self, key, edit, *, policy=POLICY): + """Copy the expanded `key` view, apply `edit` to its view.nt lines, and check it.""" + from vcf_rdfizer_policies.check import check_view + + _, out = self.views["expanded", key] + with tempfile.TemporaryDirectory() as td: + copy = Path(td) / "view" + shutil.copytree(out, copy) + lines = (copy / "view.nt").read_text(encoding="utf-8").splitlines() + (copy / "view.nt").write_text("\n".join(edit(lines)) + "\n", encoding="utf-8") + return "\n".join(check_view(copy, policy, self.rules, VCFS)) + + def full_graph_lines(self, needle): + return [line for line in self.graphs["expanded"].serialize(format="nt").splitlines() if needle in line] + + def test_a_reinstated_withheld_region_record_is_a_leak(self): + brca1 = FIXTURE["loci"]["brca1"] + record = next(r for r, _ in self.views["expanded", "gru"][0].decisions + if r.file.endswith("P001.vcf") and r.chrom == "chr17" and brca1["start"] < r.pos < brca1["end"]) + subject = f"<{record.file}#record/{record.row}>" + report = self.mutate("gru", lambda lines: lines + self.full_graph_lines(subject + " ")) + self.assertIn("leak", report) + self.assertIn("prohibited content present", report) + + def test_a_restored_triple_of_the_withdrawn_file_is_caught(self): + report = self.mutate("alz", lambda lines: lines + self.full_graph_lines(" ")[:1]) + self.assertIn("P004.vcf", report) + + def test_a_deleted_released_record_is_over_withholding(self): + report = self.mutate("alz", lambda lines: [l for l in lines if not l.startswith("")]) + self.assertIn("over-withheld: file://P003.vcf#record/1", report) + + def test_a_reference_to_a_missing_resource_is_dangling(self): + report = self.mutate("alz", lambda lines: [l for l in lines if not l.startswith("")]) + self.assertIn("dangling reference", report) + + def test_one_requesters_view_does_not_pass_as_anothers(self): + _, alz = self.views["expanded", "alz"] + alz_lines = (alz / "view.nt").read_text(encoding="utf-8").splitlines() + self.assertIn("leak", self.mutate("gru", lambda _: alz_lines)) + + def test_a_view_is_checked_against_the_policy_it_was_made_under(self): + with tempfile.TemporaryDirectory() as td: + changed = Path(td) / "policy.ttl" + changed.write_text(POLICY.read_text(encoding="utf-8") + "\n# edited\n", encoding="utf-8") + self.assertIn("different policy", self.mutate("gru", lambda lines: lines, policy=changed)) + + +@unittest.skipIf(rdflib is None, "rdflib is required") +class CommandTests(VerboseTestCase): + def run_cli(self, *argv): + from vcf_rdfizer_policy import main + + stdout, stderr = io.StringIO(), io.StringIO() + with contextlib.redirect_stdout(stdout), contextlib.redirect_stderr(stderr): + code = main([str(a) for a in argv]) + return code, stdout.getvalue() + stderr.getvalue() + + def test_evaluate_then_check_round_trip(self): + rdf = sorted((EXAMPLE / "converted" / "condensed").glob("*.nt.gz")) + spec = FIXTURE["requesters"]["clinical"] + with tempfile.TemporaryDirectory() as td: + out = Path(td) / "clinical" + code, text = self.run_cli("evaluate", "--rdf", *rdf, "--policy", POLICY, + "--assignee", spec["assignee"], "--purpose", "DUO:0000043", "-o", out) + self.assertEqual(code, 0, text) + code, text = self.run_cli("check", "--view", out, "--policy", POLICY, "--vcf", *VCFS) + self.assertEqual((code, text.strip().splitlines()[-1]), (0, "PASS")) + code, text = self.run_cli("evaluate", "--rdf", *rdf, "--policy", POLICY, + "--assignee", spec["assignee"], "--purpose", "DUO:0000043", "-o", out) + self.assertEqual(code, 2) # never overwrites + + def test_an_unsupported_policy_exits_2_with_the_reason(self): + with tempfile.TemporaryDirectory() as td: + path = write_policy(td, FILE_PERMISSION, conflict="odrl:perm") + code, text = self.run_cli("explain", "--policy", path) + self.assertEqual(code, 2) + self.assertIn("deny wins", text) + + def test_explain_and_attach(self): + code, text = self.run_cli("explain", "--policy", POLICY) + self.assertEqual(code, 0) + self.assertIn("prohibition on ", text) + with tempfile.TemporaryDirectory() as td: + out = Path(td) / "annotated.nt" + code, _ = self.run_cli("attach", "--rdf", EXAMPLE / "converted" / "expanded" / "P001.nt.gz", + "--policy", POLICY, "-o", out) + self.assertEqual(code, 0) + self.assertIn("http://www.w3.org/ns/odrl/2/hasPolicy", out.read_text(encoding="utf-8")) + + +class FixtureTests(VerboseTestCase): + def test_the_committed_fixture_is_what_the_generator_writes(self): + spec = importlib.util.spec_from_file_location("make_fixture", EXAMPLE / "make_fixture.py") + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + with tempfile.TemporaryDirectory() as td, contextlib.redirect_stdout(io.StringIO()): + module.main(Path(td)) + names = [p.name for p in VCFS] + ["fixture.json"] + match, mismatch, errors = filecmp.cmpfiles(EXAMPLE, td, names, shallow=False) + self.assertEqual((mismatch, errors), ([], [])) + self.assertEqual(len(match), 6) + + +if __name__ == "__main__": + unittest.main() diff --git a/vcf_rdfizer_data/policy/__init__.py b/vcf_rdfizer_data/policy/__init__.py new file mode 100644 index 0000000..faba12d --- /dev/null +++ b/vcf_rdfizer_data/policy/__init__.py @@ -0,0 +1 @@ +"""Bundled data for the policy demonstrator: the vcfp profile terms and a DUO subset.""" diff --git a/vcf_rdfizer_data/policy/duo-subset.ttl b/vcf_rdfizer_data/policy/duo-subset.ttl new file mode 100644 index 0000000..371b7e9 --- /dev/null +++ b/vcf_rdfizer_data/policy/duo-subset.ttl @@ -0,0 +1,26 @@ +# The four GA4GH Data Use Ontology terms the v0.1.0 policy demonstrator uses, +# with their subclass links exactly as DUO states them. +# +# Source: http://purl.obolibrary.org/obo/duo/releases/2021-02-23/duo.owl +# (EBISPOT/DUO master, read 2026-09-25). Only rdfs:subClassOf and labels are +# copied. A purpose satisfies a consented term when it is that term or a +# subclass of it, so disease-specific research falls within a +# general-research consent and not the reverse. +# +# Clinical care use is, in DUO, a data use *modifier* rather than a permission. +# v0.1.0 treats it as a purpose like the others; it matches only itself. + +@prefix obo: . +@prefix rdfs: . +@prefix owl: . + + a owl:Ontology . + +obo:DUO_0000042 rdfs:label "general research use" ; + rdfs:subClassOf obo:DUO_0000001 . +obo:DUO_0000006 rdfs:label "health or medical or biomedical research" ; + rdfs:subClassOf obo:DUO_0000042 . +obo:DUO_0000007 rdfs:label "disease specific research" ; + rdfs:subClassOf obo:DUO_0000006 . +obo:DUO_0000043 rdfs:label "clinical care use" ; + rdfs:subClassOf obo:DUO_0000017 . diff --git a/vcf_rdfizer_data/policy/vcfp-0.1.ttl b/vcf_rdfizer_data/policy/vcfp-0.1.ttl new file mode 100644 index 0000000..3e38f11 --- /dev/null +++ b/vcf_rdfizer_data/policy/vcfp-0.1.ttl @@ -0,0 +1,34 @@ +# The VCF-RDFizer policy profile terms that v0.1.0 defines. The full profile is +# specified in docs/privacy-policy-design.md; docs/policy-demonstrator.md lists +# which parts v0.1.0 implements. Anything a policy uses beyond these terms is +# rejected by the evaluator rather than ignored. + +@prefix vcfp: . +@prefix odrl: . +@prefix rdfs: . +@prefix xsd: . +@prefix rdf: . + + a odrl:Profile ; + rdfs:label "VCF-RDFizer policy profile, v0.1.0 subset" . + +vcfp:GraphSelection a rdfs:Class ; rdfs:subClassOf odrl:Asset ; + rdfs:comment "An asset that is a selection of a VCF graph, described by exactly one vcfp:selector." . +vcfp:selector a rdf:Property ; rdfs:domain vcfp:GraphSelection . +vcfp:selects a rdf:Property ; rdfs:domain vcfp:GraphSelection ; + rdfs:comment "Written by `vcf-rdfizer-policy attach`: a record the selection selects." . + +vcfp:RegionSelector a rdfs:Class ; + rdfs:comment "Records whose POS lies in [vcfp:start, vcfp:end] on vcfp:chrom, 1-based and inclusive." . +vcfp:VariantSelector a rdfs:Class ; + rdfs:comment "Records with exactly this chrom, pos, ref and alt." . +vcfp:assembly a rdf:Property ; rdfs:comment "Required. Must equal the file's vcfc:referenceGenome." . +vcfp:chrom a rdf:Property . +vcfp:start a rdf:Property ; rdfs:range xsd:integer . +vcfp:end a rdf:Property ; rdfs:range xsd:integer . +vcfp:pos a rdf:Property ; rdfs:range xsd:integer . +vcfp:ref a rdf:Property . +vcfp:alt a rdf:Property . + +vcfp:ReleaseView a rdfs:Class ; + rdfs:comment "A graph produced from source files under a policy for one request. Governed release, not anonymization." . diff --git a/vcf_rdfizer_policies/__init__.py b/vcf_rdfizer_policies/__init__.py new file mode 100644 index 0000000..78c2db8 --- /dev/null +++ b/vcf_rdfizer_policies/__init__.py @@ -0,0 +1,28 @@ +"""Policy demonstrator v0.1.0: ODRL policies attached to VCF-RDFizer graphs. + +Policies attach to files, genomic regions and single variants. One policy set +yields a different release view per request (who asks, and for what purpose), +and each view can be checked against an oracle computed from the source VCFs. + +This is governed release, not anonymization. The specification, including what +v0.1.0 deliberately leaves out, is docs/policy-demonstrator.md. +""" + +VERSION = "0.1.0" + +ODRL = "http://www.w3.org/ns/odrl/2/" +VCFP = "https://w3id.org/vcf-rdfizer/policy#" +VCFC = "https://w3id.org/vcf-core/vocab#" +OBO = "http://purl.obolibrary.org/obo/" + +#: Stated in every release manifest, so no artifact can be mistaken for more. +DISCLOSURE_MODEL = "governed release; not anonymization" + + +class PolicyError(ValueError): + """A policy, graph or request that v0.1.0 cannot evaluate. + + Always fatal. A rule that is parsed and then not applied would leave the + operator believing a release is governed when it is not, so the tool stops + instead of guessing or skipping. + """ diff --git a/vcf_rdfizer_policies/check.py b/vcf_rdfizer_policies/check.py new file mode 100644 index 0000000..25ed4ad --- /dev/null +++ b/vcf_rdfizer_policies/check.py @@ -0,0 +1,114 @@ +"""Verify a release view against an oracle computed from the source VCFs. + +The oracle reads records straight from the VCF text -- no graph, no SPARQL -- +and decides them with decide.py. A view passes only if its records equal the +oracle's released set exactly (an extra record is a leak; a missing one is +over-withholding) and it survives three structural checks: no triple inside +a prohibited target, no reference to anything withheld, and no mention of a +withheld file. Each failure is returned as one human-readable line. +""" + +import gzip +from pathlib import Path +import re + +from . import ODRL, VCFC, VCFP +from .decide import Request, applies, check_assemblies, decide +from .graphs import Record, records, split +from .profile import FileTarget, RegionTarget, policy_digest + + +def read_vcf(path: Path): + """(file IRI, assembly, [Record]) from a VCF, numbering rows as the converter does.""" + path = Path(path) + opener = gzip.open if path.name.endswith(".gz") else open + name = re.sub(r"\.gz$", "", path.name) + file_iri, assembly, found = f"file://{name}", None, [] + with opener(path, "rt", encoding="utf-8") as handle: + for line in handle: + if line.startswith("##reference="): + assembly = line.split("=", 1)[1].strip() + if line.startswith("#"): + continue + chrom, pos, _, ref, alt = line.rstrip("\n").split("\t")[:5] + found.append(Record(file_iri, len(found) + 1, chrom, int(pos), ref, + tuple(sorted(a for a in alt.split(",") if a != ".")))) + return file_iri, assembly, found + + +def oracle(vcf_paths, rules, request): + """The (file, row) pairs a correct view releases, and the files it withholds.""" + parsed = [read_vcf(p) for p in vcf_paths] + check_assemblies(rules, {f: a for f, a, _ in parsed}) + released, withheld_files = set(), set() + for file_iri, _, file_records in parsed: + if not decide(file_iri, rules, request).released: + withheld_files.add(file_iri) + continue + released |= {(r.file, r.row) for r in file_records if decide(r, rules, request).released} + return released, withheld_files + + +def _only(graph, predicate): + """The object of the one triple with this predicate (Graph.value needs a subject).""" + import rdflib + + return next(graph.objects(None, rdflib.URIRef(predicate)), None) + + +def read_request(manifest_graph) -> Request: + import rdflib + + request = _only(manifest_graph, VCFP + "request") + return Request(str(manifest_graph.value(request, rdflib.URIRef(ODRL + "assignee"))), + str(manifest_graph.value(request, rdflib.URIRef(ODRL + "purpose")))) + + +def check_view(view_dir: Path, policy_path: Path, rules, vcf_paths) -> list: + """Every way the view in `view_dir` departs from the policy; empty means it passes.""" + import rdflib + + view_dir = Path(view_dir) + manifest = rdflib.Graph().parse(str(view_dir / "manifest.ttl"), format="turtle") + view = rdflib.Graph().parse(str(view_dir / "view.nt"), format="nt") + request = read_request(manifest) + failures = [] + + recorded = str(_only(manifest, VCFP + "policyDigest")) + if recorded != policy_digest(policy_path): + failures.append(f"view was produced under a different policy ({recorded})") + + expected, withheld_files = oracle(vcf_paths, rules, request) + actual = {(r.file, r.row) for r in records(view)} + failures += [f"leak: {f}#record/{row} is released but the policy withholds it" + for f, row in sorted(actual - expected)] + failures += [f"over-withheld: {f}#record/{row} should have been released" + for f, row in sorted(expected - actual)] + + for rule in rules: + if rule.kind == "prohibition" and applies(rule, request) and view.query(_ask(rule.target)).askAnswer: + failures.append(f"prohibited content present: {rule.label}") + + subjects = {s for s in view.subjects() if isinstance(s, rdflib.URIRef)} + for s, p, o in view: + if isinstance(o, rdflib.URIRef) and split(str(o))[0] and o not in subjects: + failures.append(f"dangling reference: <{s}> <{p}> <{o}>") + + names = {f.split("://", 1)[1] for f in withheld_files} + for term in {t for triple in view for t in triple}: + for name in names: + if name in str(term): + failures.append(f"withheld file {name} is named by {term.n3()}") + return failures + + +def _ask(target) -> str: + """A SPARQL ASK that is true when anything the target selects is in the graph.""" + if isinstance(target, FileTarget): + return f'ASK {{ ?s ?p ?o FILTER(STRSTARTS(STR(?s), "{target.iri}#") || STR(?s) = "{target.iri}") }}' + if isinstance(target, RegionTarget): + where = f"FILTER(?pos >= {target.start} && ?pos <= {target.end})" + else: + where = f'FILTER(?pos = {target.pos}) ?r vcfc:ref "{target.ref}" ; vcfc:alt "{target.alt}" .' + return (f'PREFIX vcfc: <{VCFC}> ASK {{ ?r a vcfc:VCFRecord ; vcfc:chrom "{target.chrom}" ; ' + f"vcfc:pos ?pos . {where} }}") diff --git a/vcf_rdfizer_policies/decide.py b/vcf_rdfizer_policies/decide.py new file mode 100644 index 0000000..623a803 --- /dev/null +++ b/vcf_rdfizer_policies/decide.py @@ -0,0 +1,73 @@ +"""The v0.1.0 decision rules (docs/policy-demonstrator.md §4) -- the one place they live. + +Both `evaluate` and the oracle in `check` call these functions, so the rules +cannot drift apart. The oracle's independence comes from its input (the VCF +text, not the graph), not from a second copy of the rules. +""" + +from dataclasses import dataclass + +from . import PolicyError +from .profile import FileTarget, RegionTarget, VariantTarget +from .purposes import within + + +@dataclass(frozen=True) +class Request: + assignee: str + purpose: str # a DUO IRI + + +@dataclass(frozen=True) +class Decision: + released: bool + reason: str + + +def applies(rule, request) -> bool: + """Does the rule bind this request: assignee matches, and every constraint holds?""" + if rule.assignee is not None and rule.assignee != request.assignee: + return False + for constraint in rule.constraints: + inside = any(within(request.purpose, term) for term in constraint.purposes) + if inside != (constraint.operator == "isAnyOf"): + return False + return True + + +def covers(target, subject) -> bool: + """Does the target select this subject (a Record, or a file IRI)?""" + file_iri = subject if isinstance(subject, str) else subject.file + if isinstance(target, FileTarget): + return target.iri == file_iri + if isinstance(subject, str): + return False # region and variant targets select records, never whole files + if isinstance(target, RegionTarget): + return subject.chrom == target.chrom and target.start <= subject.pos <= target.end + if isinstance(target, VariantTarget): + return (subject.chrom, subject.pos, subject.ref) == (target.chrom, target.pos, target.ref) \ + and target.alt in subject.alts + raise PolicyError(f"unknown target {target!r}") + + +def check_assemblies(rules, file_assemblies: dict) -> None: + """Every region or variant rule must name the assembly every file declares.""" + for rule in rules: + wanted = getattr(rule.target, "assembly", None) + for file_iri, declared in file_assemblies.items(): + if wanted is not None and declared != wanted: + raise PolicyError(f"{rule.label} is for {wanted}, but <{file_iri}> declares " + f"{declared or 'no reference genome'}") + + +def decide(subject, rules, request) -> Decision: + """Release iff a binding permission covers the file and no binding prohibition covers the subject.""" + binding = [rule for rule in rules if applies(rule, request)] + for rule in binding: + if rule.kind == "prohibition" and covers(rule.target, subject): + return Decision(False, f"withheld: {rule.label}") + file_iri = subject if isinstance(subject, str) else subject.file + for rule in binding: + if rule.kind == "permission" and covers(rule.target, file_iri): + return Decision(True, f"released: {rule.label}") + return Decision(False, f"withheld: no permission covers <{file_iri}> for this purpose") diff --git a/vcf_rdfizer_policies/graphs.py b/vcf_rdfizer_policies/graphs.py new file mode 100644 index 0000000..188c69b --- /dev/null +++ b/vcf_rdfizer_policies/graphs.py @@ -0,0 +1,86 @@ +"""Load converted graphs and read the records the selectors act on. + +A record is identified by its file IRI and row number, which is how the +converter mints its IRIs (docs/conversion.md §6). Everything that belongs to +row N lives under one of three IRI subtrees -- #record/N, #call/N and +#sample/N -- so withholding a record means withholding those subtrees. +""" + +from dataclasses import dataclass +import gzip +from pathlib import Path +import re + +from . import VCFC, PolicyError + +#: v0.1.0 evaluates in memory. Above this the demonstrator is the wrong tool. +MAX_TRIPLES = 5_000_000 + +_ROW = re.compile(r"(?:record|call|sample)/(\d+)(?:/.*)?") + + +@dataclass(frozen=True) +class Record: + file: str # file IRI, e.g. file://P001.vcf + row: int + chrom: str + pos: int + ref: str + alts: tuple + + +def load(paths) -> "rdflib.Graph": + """Parse .nt / .nt.gz files into one graph, after a size check.""" + import rdflib + + total = 0 + for path in map(Path, paths): + opener = gzip.open if path.name.endswith(".gz") else open + with opener(path, "rt", encoding="utf-8") as handle: + total += sum(1 for _ in handle) + if total > MAX_TRIPLES: + raise PolicyError(f"{total:,} triples exceeds the v0.1.0 in-memory limit of " + f"{MAX_TRIPLES:,}; the demonstrator is for fixtures, not cohorts") + graph = rdflib.Graph() + for path in map(Path, paths): + opener = gzip.open if path.name.endswith(".gz") else open + with opener(path, "rb") as handle: + graph.parse(handle, format="nt") + return graph + + +def records(graph) -> list: + """Every vcfc:VCFRecord in the graph, with the fields selectors use.""" + rows = graph.query(f""" + PREFIX vcfc: <{VCFC}> + SELECT ?record ?chrom ?pos ?ref (GROUP_CONCAT(?alt; separator=",") AS ?alts) + WHERE {{ ?record a vcfc:VCFRecord ; vcfc:chrom ?chrom ; vcfc:pos ?pos ; vcfc:ref ?ref . + OPTIONAL {{ ?record vcfc:alt ?alt }} }} + GROUP BY ?record ?chrom ?pos ?ref""") + found = [] + for record, chrom, pos, ref, alts in rows: + file_iri, row = split(str(record)) + found.append(Record(file_iri, row, str(chrom), int(pos), str(ref), + tuple(sorted(str(alts).split(","))) if alts else ())) + return sorted(found, key=lambda r: (r.file, r.row)) + + +def assemblies(graph) -> dict: + """Each file IRI mapped to its declared vcfc:referenceGenome.""" + rows = graph.query(f"""PREFIX vcfc: <{VCFC}> + SELECT ?file ?assembly WHERE {{ ?file a vcfc:VCFFile . OPTIONAL {{ ?file vcfc:referenceGenome ?assembly }} }}""") + return {str(f): (str(a) if a is not None else None) for f, a in rows} + + +def split(iri: str): + """(file IRI, row or None) for any IRI the converter mints; (None, None) otherwise. + + Every IRI under file://X belongs to file X -- header lines and the sample set + included, which is what lets a withheld file take its header with it. Only + the #record/, #call/ and #sample/ subtrees also carry a row. + """ + if not iri.startswith("file://"): + return None, None + file_iri, _, fragment = iri.partition("#") + match = _ROW.fullmatch(fragment) + return file_iri, int(match.group(1)) if match else None diff --git a/vcf_rdfizer_policies/profile.py b/vcf_rdfizer_policies/profile.py new file mode 100644 index 0000000..d662e4c --- /dev/null +++ b/vcf_rdfizer_policies/profile.py @@ -0,0 +1,172 @@ +"""Read an ODRL policy file into rules, rejecting anything v0.1.0 cannot evaluate. + +The supported subset is docs/policy-demonstrator.md §3. Everything outside it -- +an unknown selector, an effect other than drop, a conflict strategy other than +deny-wins, an unrecognised property on a rule -- raises PolicyError. Silently +ignoring a rule would be worse than refusing the policy. +""" + +from dataclasses import dataclass +import hashlib +from pathlib import Path + +from . import ODRL, VCFP, PolicyError +from .purposes import purpose_iri + +RDF_TYPE = "http://www.w3.org/1999/02/22-rdf-syntax-ns#type" +POLICY_CLASSES = {ODRL + name for name in ("Policy", "Set", "Offer", "Agreement")} +#: Properties a rule may carry. Anything else (odrl:refinement, odrl:remedy, +#: ...) could change what the rule means, so it is refused, not dropped. +RULE_PROPERTIES = {RDF_TYPE, ODRL + "target", ODRL + "action", ODRL + "assignee", + ODRL + "assigner", ODRL + "constraint", ODRL + "duty"} + + +@dataclass(frozen=True) +class FileTarget: + """A whole converted file, named by its file IRI, e.g. .""" + iri: str + + +@dataclass(frozen=True) +class RegionTarget: + """Records with POS in [start, end] on chrom, 1-based and inclusive.""" + asset: str + assembly: str + chrom: str + start: int + end: int + + +@dataclass(frozen=True) +class VariantTarget: + """Records with exactly this chrom, pos, ref and alt.""" + asset: str + assembly: str + chrom: str + pos: int + ref: str + alt: str + + +@dataclass(frozen=True) +class Constraint: + """A purpose constraint: odrl:isAnyOf or odrl:isNoneOf a set of DUO terms.""" + operator: str + purposes: frozenset + + +@dataclass(frozen=True) +class Rule: + kind: str # "permission" or "prohibition" + policy: str # the policy IRI the rule belongs to + target: object # FileTarget | RegionTarget | VariantTarget + assignee: str = None # None means odrl:All + constraints: tuple = () + duties: tuple = () # ODRL action IRIs; recorded, not enforced + + @property + def label(self) -> str: + """Short, stable name for reports: kind and target.""" + target = getattr(self.target, "asset", None) or self.target.iri + return f"{self.kind} on <{target}>" + + +def policy_digest(path: Path) -> str: + """sha256 of the policy file's bytes, as recorded in every manifest.""" + return "sha256:" + hashlib.sha256(Path(path).read_bytes()).hexdigest() + + +def load_policy(path: Path): + """Parse and validate a policy file; returns (rdflib graph, [Rule, ...]).""" + import rdflib + + graph = rdflib.Graph().parse(str(path), format="turtle") + policies = {s for s, o in graph.subject_objects(rdflib.RDF.type) if str(o) in POLICY_CLASSES} + if not policies: + raise PolicyError(f"{path}: no odrl:Policy, Set, Offer or Agreement found") + rules = [] + for policy in sorted(policies, key=str): + conflict = graph.value(policy, rdflib.URIRef(ODRL + "conflict")) + if str(conflict) != ODRL + "prohibit": + raise PolicyError(f"<{policy}>: v0.1.0 requires odrl:conflict odrl:prohibit (deny wins)") + if graph.value(policy, rdflib.URIRef(ODRL + "obligation")) is not None: + raise PolicyError(f"<{policy}>: odrl:obligation is not supported in v0.1.0") + for kind in ("permission", "prohibition"): + for node in graph.objects(policy, rdflib.URIRef(ODRL + kind)): + rules.append(_rule(graph, str(policy), kind, node)) + return graph, rules + + +def _rule(graph, policy, kind, node): + import rdflib + + unknown = {str(p) for p in graph.predicates(node)} - RULE_PROPERTIES + if unknown: + raise PolicyError(f"{kind} in <{policy}> uses unsupported properties: {sorted(unknown)}") + if str(graph.value(node, rdflib.URIRef(ODRL + "action"))) != ODRL + "read": + raise PolicyError(f"{kind} in <{policy}>: the only supported action is odrl:read") + assignee = graph.value(node, rdflib.URIRef(ODRL + "assignee")) + return Rule( + kind=kind, + policy=policy, + target=_target(graph, graph.value(node, rdflib.URIRef(ODRL + "target"))), + assignee=None if assignee is None or str(assignee) == ODRL + "All" else str(assignee), + constraints=tuple(_constraint(graph, c) for c in graph.objects(node, rdflib.URIRef(ODRL + "constraint"))), + duties=tuple(sorted(_duty(graph, d) for d in graph.objects(node, rdflib.URIRef(ODRL + "duty")))), + ) + + +def _target(graph, node): + import rdflib + + if node is None: + raise PolicyError("a rule has no odrl:target") + selectors = list(graph.objects(node, rdflib.URIRef(VCFP + "selector"))) + if not selectors: + iri = str(node) + if isinstance(node, rdflib.URIRef) and iri.startswith("file://") and "#" not in iri: + return FileTarget(iri) + raise PolicyError(f"target <{node}> is neither a file IRI nor a vcfp:GraphSelection") + if len(selectors) != 1: + raise PolicyError(f"<{node}>: a GraphSelection needs exactly one vcfp:selector") + selector = selectors[0] + kind = str(graph.value(selector, rdflib.RDF.type) or "") + + def get(name, cast=str): + value = graph.value(selector, rdflib.URIRef(VCFP + name)) + if value is None: + raise PolicyError(f"<{node}>: selector is missing vcfp:{name}") + return cast(value) + + if kind == VCFP + "RegionSelector": + target = RegionTarget(str(node), get("assembly"), get("chrom"), get("start", int), get("end", int)) + if target.start > target.end: + raise PolicyError(f"<{node}>: vcfp:start is after vcfp:end") + return target + if kind == VCFP + "VariantSelector": + return VariantTarget(str(node), get("assembly"), get("chrom"), get("pos", int), get("ref"), get("alt")) + raise PolicyError(f"<{node}>: selector type {kind or '(none)'} is not supported in v0.1.0") + + +def _constraint(graph, node): + import rdflib + + left = str(graph.value(node, rdflib.URIRef(ODRL + "leftOperand"))) + operator = str(graph.value(node, rdflib.URIRef(ODRL + "operator"))) + if left != ODRL + "purpose": + raise PolicyError(f"only odrl:purpose constraints are supported, not <{left}>") + if operator not in (ODRL + "isAnyOf", ODRL + "isNoneOf"): + raise PolicyError(f"only odrl:isAnyOf and odrl:isNoneOf are supported, not <{operator}>") + values = list(graph.objects(node, rdflib.URIRef(ODRL + "rightOperand"))) + if not values: + raise PolicyError("a purpose constraint has no odrl:rightOperand") + return Constraint(operator.rsplit("/", 1)[1], frozenset(purpose_iri(str(v)) for v in values)) + + +def _duty(graph, node): + import rdflib + + transform = graph.value(node, rdflib.URIRef(VCFP + "transform")) + if transform is not None and str(transform) != VCFP + "drop": + raise PolicyError(f"effect <{transform}> is not supported in v0.1.0; only vcfp:drop") + return str(graph.value(node, rdflib.URIRef(ODRL + "action"))) diff --git a/vcf_rdfizer_policies/purposes.py b/vcf_rdfizer_policies/purposes.py new file mode 100644 index 0000000..d9ffe75 --- /dev/null +++ b/vcf_rdfizer_policies/purposes.py @@ -0,0 +1,47 @@ +"""DUO purposes, and the subsumption rule used to match them. + +A requester's purpose satisfies a consented term when it is that term or a +narrower one: disease-specific research falls within a general-research consent, +not the reverse. The hierarchy is the bundled DUO subset +(vcf_rdfizer_data/policy/duo-subset.ttl), not a live ontology. +""" + +from functools import lru_cache +from importlib.resources import files +import re + +from . import OBO, PolicyError + +_TERM = re.compile(r"(?:obo:|http://purl\.obolibrary\.org/obo/)?DUO[:_](\d{7})") + + +def purpose_iri(value: str) -> str: + """Normalise obo:DUO_0000007, DUO:0000007, DUO_0000007 or a full IRI.""" + match = _TERM.fullmatch(value.strip()) + if not match: + raise PolicyError(f"not a DUO term: {value!r}") + iri = f"{OBO}DUO_{match.group(1)}" + if iri not in _parents(): + raise PolicyError(f"{value} is not in the bundled DUO subset; v0.1.0 knows " + + ", ".join(sorted(t.rsplit("/", 1)[1] for t in _parents()))) + return iri + + +@lru_cache(maxsize=1) +def _parents() -> dict: + """Each bundled term mapped to its one DUO parent.""" + import rdflib + + graph = rdflib.Graph().parse(files("vcf_rdfizer_data.policy") / "duo-subset.ttl") + return {str(s): str(o) for s, o in graph.subject_objects(rdflib.RDFS.subClassOf)} + + +def within(purpose: str, term: str) -> bool: + """True when `purpose` is `term` or a descendant of it.""" + parents = _parents() + node = purpose + while node is not None: + if node == term: + return True + node = parents.get(node) + return False diff --git a/vcf_rdfizer_policies/release.py b/vcf_rdfizer_policies/release.py new file mode 100644 index 0000000..8056e4e --- /dev/null +++ b/vcf_rdfizer_policies/release.py @@ -0,0 +1,181 @@ +"""Evaluate a policy for one request, and attach policies to a graph. + +`evaluate` decides every file and record (decide.py), withholds the IRI subtrees +of whatever is refused, and drops any remaining triple that points into a +withheld subtree, so a view never references something it does not contain. +`attach` writes the policies into the data instead, so they can be queried +alongside it. +""" + +from dataclasses import dataclass, field +from datetime import datetime, timezone +import csv +import json +from pathlib import Path + +from . import DISCLOSURE_MODEL, ODRL, VCFP, VERSION +from .decide import applies, check_assemblies, covers, decide +from .graphs import assemblies, records, split +from .profile import FileTarget + + +@dataclass +class Release: + request: object + view: list = field(default_factory=list) # (s, p, o) rdflib terms + decisions: list = field(default_factory=list) # (Record, Decision) + files: dict = field(default_factory=dict) # file IRI -> Decision + duties: tuple = () # of the permissions that released a file + triples_withheld: int = 0 + + +def evaluate(graph, rules, request) -> Release: + import rdflib + + file_assemblies = assemblies(graph) + check_assemblies(rules, file_assemblies) + release = Release(request) + release.files = {f: decide(f, rules, request) for f in sorted(file_assemblies)} + withheld_rows = set() + for record in records(graph): + file_decision = release.files[record.file] + decision = decide(record, rules, request) if file_decision.released else file_decision + release.decisions.append((record, decision)) + if not decision.released: + withheld_rows.add((record.file, record.row)) + withheld_files = {f for f, d in release.files.items() if not d.released} + release.duties = tuple(sorted({ + duty for rule in rules if rule.kind == "permission" and applies(rule, request) + and any(covers(rule.target, f) for f in release.files if f not in withheld_files) + for duty in rule.duties})) + + def withheld(term) -> bool: + if not isinstance(term, rdflib.URIRef): + return False + file_iri, row = split(str(term)) + return file_iri in withheld_files or (file_iri, row) in withheld_rows + + for triple in graph: + if withheld(triple[0]) or withheld(triple[2]): + release.triples_withheld += 1 + else: + release.view.append(triple) + return release + + +def summary(release) -> dict: + """Counts per file and per deciding reason, for the manifest and the paper figure.""" + per_file = {} + for record, decision in release.decisions: + counts = per_file.setdefault(record.file, {"released": 0, "withheld": 0}) + counts["released" if decision.released else "withheld"] += 1 + reasons = {} + for _, decision in release.decisions: + reasons[decision.reason] = reasons.get(decision.reason, 0) + 1 + return { + "request": {"assignee": release.request.assignee, "purpose": release.request.purpose}, + "files": {f: {"released": d.released, "reason": d.reason, **per_file.get(f, {})} + for f, d in release.files.items()}, + "records_released": sum(d.released for _, d in release.decisions), + "records_withheld": sum(not d.released for _, d in release.decisions), + "triples_released": len(release.view), + "triples_withheld": release.triples_withheld, + "reasons": reasons, + } + + +def write_release(release, out_dir: Path, *, policies, digest: str) -> None: + """Write view.nt, decisions.csv, summary.json and manifest.ttl into a new directory.""" + import rdflib + + out_dir = Path(out_dir) + if out_dir.exists() and any(out_dir.iterdir()): + raise FileExistsError(f"{out_dir} is not empty; a release is never overwritten") + out_dir.mkdir(parents=True, exist_ok=True) + + view = rdflib.Graph() + for triple in release.view: + view.add(triple) + lines = sorted(line for line in view.serialize(format="nt").splitlines() if line.strip()) + (out_dir / "view.nt").write_text("\n".join(lines) + "\n", encoding="utf-8") + + with (out_dir / "decisions.csv").open("w", newline="", encoding="utf-8") as handle: + writer = csv.writer(handle) + writer.writerow(["file", "row", "chrom", "pos", "ref", "alts", "released", "reason"]) + for record, decision in release.decisions: + writer.writerow([record.file, record.row, record.chrom, record.pos, record.ref, + ",".join(record.alts), decision.released, decision.reason]) + + counts = summary(release) + (out_dir / "summary.json").write_text(json.dumps(counts, indent=2) + "\n", encoding="utf-8") + (out_dir / "manifest.ttl").write_text( + manifest(counts, policies=policies, digest=digest, sources=release.files, duties=release.duties), + encoding="utf-8") + + +def manifest(counts, *, policies, digest, sources, duties) -> str: + """The vcfp:ReleaseView description (docs/policy-demonstrator.md §6.1), as Turtle.""" + import rdflib + from rdflib.namespace import PROV, XSD + + vcfp, odrl = rdflib.Namespace(VCFP), rdflib.Namespace(ODRL) + g = rdflib.Graph() + g.bind("vcfp", vcfp), g.bind("odrl", odrl), g.bind("prov", PROV) + view, request = rdflib.URIRef("#release"), rdflib.BNode() + g.add((view, rdflib.RDF.type, vcfp.ReleaseView)) + for source in sorted(sources): + g.add((view, vcfp.derivedFrom, rdflib.URIRef(source))) + for policy in sorted(policies): + g.add((view, vcfp.policy, rdflib.URIRef(policy))) + g.add((view, vcfp.policyDigest, rdflib.Literal(digest))) + g.add((view, vcfp.request, request)) + g.add((request, odrl.assignee, rdflib.URIRef(counts["request"]["assignee"]))) + g.add((request, odrl.purpose, rdflib.URIRef(counts["request"]["purpose"]))) + for name in ("records_released", "records_withheld", "triples_withheld"): + g.add((view, vcfp[_camel(name)], rdflib.Literal(counts[name]))) + g.add((view, vcfp.filesWithheld, rdflib.Literal(sum(not f["released"] for f in counts["files"].values())))) + for duty in sorted(duties): + obligation = rdflib.BNode() + g.add((view, vcfp.obligation, obligation)) + g.add((obligation, odrl.action, rdflib.URIRef(duty))) + g.add((view, vcfp.disclosureModel, rdflib.Literal(DISCLOSURE_MODEL))) + g.add((view, PROV.wasGeneratedBy, rdflib.URIRef(f"urn:vcf-rdfizer-policy:{VERSION}"))) + g.add((view, PROV.generatedAtTime, + rdflib.Literal(datetime.now(timezone.utc).replace(microsecond=0).isoformat(), datatype=XSD.dateTime))) + return g.serialize(format="turtle") + + +def _camel(name: str) -> str: + head, *rest = name.split("_") + return head + "".join(part.title() for part in rest) + + +def attach(graph, policy_graph, rules) -> dict: + """Merge the policies into `graph` and link each governed resource to its policy. + + Files get odrl:hasPolicy directly. A region or variant selection records what + it selects (vcfp:selects), and each selected record gets odrl:hasPolicy too, + so a SPARQL query needs no knowledge of the selectors. Returns counts per asset. + """ + import rdflib + + has_policy, selects = rdflib.URIRef(ODRL + "hasPolicy"), rdflib.URIRef(VCFP + "selects") + check_assemblies(rules, assemblies(graph)) + all_records = records(graph) + counts = {} + for triple in policy_graph: + graph.add(triple) + for rule in rules: + policy = rdflib.URIRef(rule.policy) + if isinstance(rule.target, FileTarget): + graph.add((rdflib.URIRef(rule.target.iri), has_policy, policy)) + counts[rule.target.iri] = 1 + continue + asset = rdflib.URIRef(rule.target.asset) + selected = [r for r in all_records if covers(rule.target, r)] + for record in selected: + record_iri = rdflib.URIRef(f"{record.file}#record/{record.row}") + graph.add((asset, selects, record_iri)) + graph.add((record_iri, has_policy, policy)) + counts[rule.target.asset] = len(selected) + return counts diff --git a/vcf_rdfizer_policy.py b/vcf_rdfizer_policy.py new file mode 100644 index 0000000..ce70e6b --- /dev/null +++ b/vcf_rdfizer_policy.py @@ -0,0 +1,132 @@ +#!/usr/bin/env python3 +"""vcf-rdfizer-policy: attach ODRL policies to VCF graphs, evaluate them, and check the result. + + vcf-rdfizer-policy attach --rdf P*.nt.gz --policy policy.ttl -o annotated.nt + vcf-rdfizer-policy evaluate --rdf P*.nt.gz --policy policy.ttl --assignee IRI --purpose DUO:0000007 -o views/alz + vcf-rdfizer-policy check --view views/alz --policy policy.ttl --vcf P*.vcf + vcf-rdfizer-policy explain --policy policy.ttl + +v0.1.0 demonstrator: governed release, not anonymization. Runs on the host; no +Docker. Exit codes: 0 success, 1 a check failed, 2 the policy, graph or request +cannot be evaluated. See docs/policy-demonstrator.md. +""" + +import argparse +from pathlib import Path +import sys + +from vcf_rdfizer_policies import VERSION, PolicyError + + +def _require_rdflib(): + try: + import rdflib # noqa: F401 + except ModuleNotFoundError: + raise PolicyError("vcf-rdfizer-policy requires 'rdflib': python -m pip install rdflib") from None + + +def cmd_attach(args): + from vcf_rdfizer_policies.graphs import load + from vcf_rdfizer_policies.profile import load_policy + from vcf_rdfizer_policies.release import attach + + policy_graph, rules = load_policy(args.policy) + graph = load(args.rdf) + out = Path(args.out) + if out.exists(): + raise FileExistsError(f"{out} exists; attach never overwrites") + counts = attach(graph, policy_graph, rules) + out.write_text(graph.serialize(format="nt"), encoding="utf-8") + for asset, n in sorted(counts.items()): + print(f"{asset}: {n} resource(s) linked") + print(f"wrote {out}") + return 0 + + +def cmd_evaluate(args): + from vcf_rdfizer_policies.decide import Request + from vcf_rdfizer_policies.graphs import load + from vcf_rdfizer_policies.profile import load_policy, policy_digest + from vcf_rdfizer_policies.purposes import purpose_iri + from vcf_rdfizer_policies.release import evaluate, summary, write_release + + _, rules = load_policy(args.policy) + request = Request(args.assignee, purpose_iri(args.purpose)) + release = evaluate(load(args.rdf), rules, request) + write_release(release, args.out, policies={r.policy for r in rules}, + digest=policy_digest(args.policy)) + counts = summary(release) + print(f"released {counts['records_released']} record(s), withheld {counts['records_withheld']}; " + f"{counts['triples_withheld']} triple(s) withheld -> {args.out}") + return 0 + + +def cmd_check(args): + from vcf_rdfizer_policies.check import check_view + from vcf_rdfizer_policies.profile import load_policy + + _, rules = load_policy(args.policy) + failures = check_view(args.view, args.policy, rules, args.vcf) + for failure in failures: + print(f"FAIL {failure}") + print("PASS" if not failures else f"{len(failures)} failure(s)") + return 0 if not failures else 1 + + +def cmd_explain(args): + from vcf_rdfizer_policies.profile import load_policy + + _, rules = load_policy(args.policy) + for rule in rules: + who = "anyone" if rule.assignee is None else f"<{rule.assignee}>" + terms = [f"purpose {c.operator} {', '.join(sorted(p.rsplit('/', 1)[1] for p in c.purposes))}" + for c in rule.constraints] + print(f"{rule.label}: applies to {who}" + (" when " + " and ".join(terms) if terms else "") + + (f"; duties: {', '.join(d.rsplit('/', 1)[1] for d in rule.duties)}" if rule.duties else "")) + print("Deny wins; anything no permission covers is withheld.") + return 0 + + +def build_parser(): + parser = argparse.ArgumentParser(prog="vcf-rdfizer-policy", description=__doc__.split("\n\n")[0]) + parser.add_argument("--version", action="version", version=f"%(prog)s {VERSION}") + sub = parser.add_subparsers(dest="command", required=True) + + attach = sub.add_parser("attach", help="write the policies into the graph") + attach.add_argument("--rdf", nargs="+", required=True, help="converted .nt / .nt.gz files") + attach.add_argument("--policy", required=True, type=Path) + attach.add_argument("-o", "--out", required=True, help="annotated .nt to create") + attach.set_defaults(run=cmd_attach) + + evaluate = sub.add_parser("evaluate", help="write one request's release view") + evaluate.add_argument("--rdf", nargs="+", required=True, help="converted .nt / .nt.gz files") + evaluate.add_argument("--policy", required=True, type=Path) + evaluate.add_argument("--assignee", required=True, help="the requesting party's IRI") + evaluate.add_argument("--purpose", required=True, help="a DUO term, e.g. DUO:0000007") + evaluate.add_argument("-o", "--out", required=True, type=Path, help="new or empty directory") + evaluate.set_defaults(run=cmd_evaluate) + + check = sub.add_parser("check", help="verify a view against the source VCFs") + check.add_argument("--view", required=True, type=Path, help="a directory written by evaluate") + check.add_argument("--policy", required=True, type=Path) + check.add_argument("--vcf", nargs="+", required=True, help="the source VCFs") + check.set_defaults(run=cmd_check) + + explain = sub.add_parser("explain", help="list the policy's rules in plain language") + explain.add_argument("--policy", required=True, type=Path) + explain.set_defaults(run=cmd_explain) + return parser + + +def main(argv=None): + args = build_parser().parse_args(argv) + try: + _require_rdflib() + return args.run(args) + except (PolicyError, FileExistsError, FileNotFoundError) as error: + print(f"error: {error}", file=sys.stderr) + return 2 + + +if __name__ == "__main__": + sys.exit(main()) From abcf5fcaa7ab30d0edf11efa8126b361b48355c6 Mon Sep 17 00:00:00 2001 From: ecrum19 Date: Fri, 25 Sep 2026 13:41:26 +0200 Subject: [PATCH 3/4] Keep a file's released flag separate from its record counts in summary.json Each file's per-record counts were merged into its entry under the key 'released', overwriting the boolean decision with a count. The demo grid only read correctly because a count of 0 is falsy. The counts are now records_released and records_withheld, and a test pins the schema. Co-Authored-By: Claude Opus 5.5 --- examples/policy/run_demo.sh | 2 +- test/test_policy_unit.py | 6 +++++- vcf_rdfizer_policies/release.py | 4 ++-- 3 files changed, 8 insertions(+), 4 deletions(-) diff --git a/examples/policy/run_demo.sh b/examples/policy/run_demo.sh index 7b56556..9710167 100755 --- a/examples/policy/run_demo.sh +++ b/examples/policy/run_demo.sh @@ -43,7 +43,7 @@ summaries = {v.name: json.loads((v / "summary.json").read_text()) for v in views files = sorted(next(iter(summaries.values()))["files"]) print("requester".ljust(10) + "".join(f.split("//")[1].ljust(10) for f in files) + "triples withheld") for name, s in summaries.items(): - cells = "".join((str(s["files"][f].get("released", 0)) if s["files"][f]["released"] else "withheld").ljust(10) + cells = "".join((str(s["files"][f]["records_released"]) if s["files"][f]["released"] else "withheld").ljust(10) for f in files) print(name.ljust(10) + cells + str(s["triples_withheld"])) PY diff --git a/test/test_policy_unit.py b/test/test_policy_unit.py index 65b29b5..d0d99ad 100644 --- a/test/test_policy_unit.py +++ b/test/test_policy_unit.py @@ -245,7 +245,11 @@ def test_the_manifest_records_the_request_and_what_was_withheld(self): for expected in ("governed release; not anonymization", "DUO_0000007", "odrl:attribute", f"vcfp:recordsWithheld {sum(not d.released for _, d in release.decisions)}"): self.assertIn(expected, manifest) - self.assertEqual(json.loads((out / "summary.json").read_text())["files"]["file://P004.vcf"]["released"], False) + files = json.loads((out / "summary.json").read_text())["files"] + self.assertEqual(files["file://P004.vcf"]["released"], False) # the decision, not a count + self.assertEqual(files["file://P001.vcf"]["released"], True) + self.assertEqual(sum(f["records_released"] for f in files.values()), + sum(d.released for _, d in release.decisions)) def test_a_release_is_never_written_over_another(self): from vcf_rdfizer_policies.release import write_release diff --git a/vcf_rdfizer_policies/release.py b/vcf_rdfizer_policies/release.py index 8056e4e..214fb7c 100644 --- a/vcf_rdfizer_policies/release.py +++ b/vcf_rdfizer_policies/release.py @@ -67,8 +67,8 @@ def summary(release) -> dict: """Counts per file and per deciding reason, for the manifest and the paper figure.""" per_file = {} for record, decision in release.decisions: - counts = per_file.setdefault(record.file, {"released": 0, "withheld": 0}) - counts["released" if decision.released else "withheld"] += 1 + counts = per_file.setdefault(record.file, {"records_released": 0, "records_withheld": 0}) + counts["records_released" if decision.released else "records_withheld"] += 1 reasons = {} for _, decision in release.decisions: reasons[decision.reason] = reasons.get(decision.reason, 0) + 1 From f1dcc935fa98e9f78c7423731dfd340f27413cab Mon Sep 17 00:00:00 2001 From: ecrum19 Date: Fri, 25 Sep 2026 14:46:51 +0200 Subject: [PATCH 4/4] Generalise the policy plug-in: selectors and partitions are declarations The engine is now three generic steps, each configured in Turtle: - select: a target is a resource IRI or a selection computed by a declared selector type, a SPARQL SELECT with named parameters; - partition: a profile's ownership rule (a property path, plus IRI subtrees) says what a withheld resource takes with it; - decide: ODRL, deny wins, default-deny. VCF knowledge moves out of the code into the bundled VCF Core profile (vcf_rdfizer_data/policy/vcf-core-profile.ttl): the region and variant selectors, the ownership rule, and the per-record report. Purposes come from any RDFS or SKOS vocabulary (--purposes); the DUO subset is the default, with a DUO: prefix. Selector types can also be declared in the policy file; examples/policy/custom-selector.ttl adds a QUAL threshold with no code. check keeps generic structural checks (digest, prohibited content, default-deny, dangling) and moves the VCF-text oracle into vcf_oracle.py, which now models every fixed column except INFO. The demo reproduces v0.1.0's grid and triple counts exactly, in both profiles. The tests move to vcf-rdfizer-testing (plugin-tests/policy), kept apart from this code and run by hand. Co-Authored-By: Claude Opus 5.5 --- conda-recipe/meta.yaml | 4 +- docs/README.md | 10 +- docs/limitations.md | 5 +- docs/policy-demonstrator.md | 751 ++++++++----------- examples/policy/README.md | 75 +- examples/policy/custom-selector.ttl | 53 ++ examples/policy/fixture.json | 6 +- examples/policy/make_fixture.py | 6 +- examples/policy/run_demo.sh | 6 +- test/test_policy_unit.py | 386 ---------- vcf_rdfizer_data/policy/duo-subset.ttl | 2 + vcf_rdfizer_data/policy/vcf-core-profile.ttl | 63 ++ vcf_rdfizer_data/policy/vcfp-0.1.ttl | 51 +- vcf_rdfizer_policies/__init__.py | 17 +- vcf_rdfizer_policies/check.py | 142 ++-- vcf_rdfizer_policies/decide.py | 73 -- vcf_rdfizer_policies/engine.py | 136 ++++ vcf_rdfizer_policies/graphs.py | 84 +-- vcf_rdfizer_policies/policy.py | 157 ++++ vcf_rdfizer_policies/profile.py | 224 ++---- vcf_rdfizer_policies/purposes.py | 47 -- vcf_rdfizer_policies/release.py | 168 ++--- vcf_rdfizer_policies/vcf_oracle.py | 73 ++ vcf_rdfizer_policies/vocabulary.py | 59 ++ vcf_rdfizer_policy.py | 117 +-- 25 files changed, 1267 insertions(+), 1448 deletions(-) create mode 100644 examples/policy/custom-selector.ttl delete mode 100644 test/test_policy_unit.py create mode 100644 vcf_rdfizer_data/policy/vcf-core-profile.ttl delete mode 100644 vcf_rdfizer_policies/decide.py create mode 100644 vcf_rdfizer_policies/engine.py create mode 100644 vcf_rdfizer_policies/policy.py delete mode 100644 vcf_rdfizer_policies/purposes.py create mode 100644 vcf_rdfizer_policies/vcf_oracle.py create mode 100644 vcf_rdfizer_policies/vocabulary.py diff --git a/conda-recipe/meta.yaml b/conda-recipe/meta.yaml index 81d5506..ea05eff 100644 --- a/conda-recipe/meta.yaml +++ b/conda-recipe/meta.yaml @@ -42,8 +42,8 @@ test: # it: a dropped package-data glob would pass every check above and only # surface when SHACL validation runs. One file per glob in pyproject.toml. - python -c "import importlib.resources as r; d = r.files('vcf_rdfizer_data'); assert d.joinpath('VOCABULARY_PROVENANCE.json').is_file() and d.joinpath('shacl').joinpath('vcf-4.5.shacl.ttl').is_file() and d.joinpath('ontology').joinpath('vcf-core-vocabulary.bundle.ttl').is_file()" - # The policy demonstrator's DUO subset is package data too. - - python -c "import importlib.resources as r; assert r.files('vcf_rdfizer_data.policy').joinpath('duo-subset.ttl').is_file()" + # The policy plug-in's profile and default purpose vocabulary are package data too. + - python -c "import importlib.resources as r; d = r.files('vcf_rdfizer_data.policy'); assert d.joinpath('vcf-core-profile.ttl').is_file() and d.joinpath('duo-subset.ttl').is_file()" imports: - vcf_rdfizer # The vocabulary terms, VCF-version model and lexical parsers. Importing it diff --git a/docs/README.md b/docs/README.md index 89c9b28..b05531e 100644 --- a/docs/README.md +++ b/docs/README.md @@ -54,10 +54,12 @@ deciding whether the tool fits your problem, read graph: an ODRL profile with graph selectors, three enforcement tiers, and verification — plus a candid account of why access control is not anonymization when the genotypes are themselves identifiers. -- **[Policy demonstrator v0.1.0](policy-demonstrator.md)** — *implemented; - walkthrough in `examples/policy/`.* The first slice of the privacy design: ODRL policies attached - to files, regions and variants of single-sample VCFs, per-request release - views, and an oracle that checks each view withholds exactly what it should. +- **[Policy attachment v0.1.0](policy-demonstrator.md)** — *implemented; + walkthrough in `examples/policy/`.* The first slice of the privacy design: + ODRL policies attached to any resource or declared graph selection, a + partition rule that says what a withheld resource takes with it, per-request + release views, and checks — including a VCF-text oracle — that each view + withholds exactly what it should. Selectors and partitions are Turtle, not code. ### What the graph looks like diff --git a/docs/limitations.md b/docs/limitations.md index 1eb21a3..9869791 100644 --- a/docs/limitations.md +++ b/docs/limitations.md @@ -269,8 +269,9 @@ record what an artifact was permitted to contain. Three consequences today: The plan is [`privacy-policy-design.md`](privacy-policy-design.md), which is explicit that what it offers is *governed release*, not anonymization. Its first slice exists as a demonstrator, [`vcf-rdfizer-policy`](policy-demonstrator.md) -v0.1.0: ODRL policies on files, regions and variants, verified release views, -over single-sample fixtures only. It changes none of the points above for a +v0.1.0: ODRL policies on files and on declared graph selections (region and +variant ship; others are Turtle declarations), verified release views, in +memory and after conversion. It changes none of the points above for a graph produced by a normal conversion. **No clinical claims.** The tool transcribes a VCF. It does not interpret, diff --git a/docs/policy-demonstrator.md b/docs/policy-demonstrator.md index 36643eb..78e8fe2 100644 --- a/docs/policy-demonstrator.md +++ b/docs/policy-demonstrator.md @@ -1,539 +1,440 @@ -# Policy attachment demonstrator — v0.1.0 +# Policy attachment — v0.1.0 *Part of the [VCF-RDFizer documentation](README.md). Status: **implemented -(v0.1.0)**; walkthrough in [`examples/policy/`](../examples/policy/README.md). This is the first, deliberately small slice of -[`privacy-policy-design.md`](privacy-policy-design.md): it uses that document's -vocabulary and rules, implements a subset of them, and states plainly what it -leaves out.* - -v0.1.0 shows three things on a small cohort of synthetic single-sample VCFs: - -1. **Attach.** ODRL policies are attached to files, genomic regions and - individual variants, and they sit in the same graph as the data, so a - single SPARQL query can ask about both. -2. **Evaluate.** One set of policies gives each requester a different release - view, according to who they are and the purpose they state. -3. **Verify.** Each view is checked against an independent oracle computed - from the source VCFs. It must withhold exactly what the policy says: no - more and no less. - -The rest of this document specifies the fixture, the subset of the profile, -the decision semantics, the command, the tests and what goes into the paper, -and it ends with the plan for later versions. - -It is **governed release, not anonymization**, and every string the tool emits -must say so. See [`privacy-policy-design.md` §1](privacy-policy-design.md#1-the-uncomfortable-premise). +(v0.1.0)**; walkthrough in [`examples/policy/`](../examples/policy/README.md); +tests in [vcf-rdfizer-testing `plugin-tests/policy/`](https://github.com/ecrum19/vcf-rdfizer-testing/tree/main/plugin-tests). +This is the first slice of [`privacy-policy-design.md`](privacy-policy-design.md): +it uses that document's vocabulary and rules, implements a subset of them, and +states plainly what it leaves out.* + +`vcf-rdfizer-policy` attaches ODRL policies to an RDF graph and produces one +release view per request. It is built from three generic steps, and each is +configured in Turtle, not code: + +1. **Select.** A rule's target is either one resource IRI or a *selection*. A + selection is computed by a selector type, declared as a SPARQL `SELECT`. +2. **Partition.** A *profile* says what a selected resource owns, so that + withholding a record also withholds its call, its alleles and its genotypes. +3. **Decide.** ODRL semantics: a binding permission must own a resource, no + binding prohibition may, and deny wins. + +The bundled **VCF Core profile** makes this work on graphs written by +VCF-RDFizer, with region and variant selectors. Anyone can add a selector, such +as FILTER state, QUAL, an INFO key or genes from a linkset, by declaring it. +The same engine partitions any other RDF graph with a different profile. + +This is **governed release, not anonymization**, and every string the tool +emits says so. See [`privacy-policy-design.md` §1](privacy-policy-design.md#1-the-uncomfortable-premise). --- -## 1. Why this is worth demonstrating +## 1. Why -Today, consent for genomic data is attached at the level of a whole dataset. A -repository record carries one data-use code, and VCF itself has no place to -state a policy at all. Real policies are finer than that: +Consent for genomic data is recorded per dataset. A repository record carries +one data-use code, and VCF itself has no place to state a policy at all. Real +policies are finer than that: - *this participant* consented to health research but not to clinical use; -- *this region* holds actionable incidental findings and may only be read for +- *this region* holds actionable incidental findings, readable only for clinical care; -- *this variant* is sensitive enough to be restricted to disease-specific - research; +- *this variant* is restricted to disease-specific research; - *this participant* withdrew, and that overrides everything else. An RDF graph can hold every one of those statements next to the data it -governs, in standard vocabularies (ODRL for the rules, GA4GH DUO for the -purposes, PROV for what was released), and it can answer questions that mix -the two. A VCF-based workflow would need an external spreadsheet, and its -enforcement would live in someone's scripts. The demonstrator makes that -difference concrete, and it measures it rather than just asserting it. +governs, in standard vocabularies: ODRL for the rules, GA4GH DUO for the +purposes, PROV for what was released. It can also answer questions that mix +the two (§6). A VCF-based workflow would need an external spreadsheet, and its +enforcement would live in someone's scripts. --- -## 2. The fixture: a cohort of single-sample files +## 2. How it works -Five synthetic participants, **one small VCF each**: +```text +policy.ttl ──► rules ─────────────────────────────┐ + │ target │ kind, assignee, purpose constraints + ▼ ▼ + ┌───────────┐ resources ┌───────────┐ owned sets ┌──────────┐ +graph ────►│ select │──────────────►│ partition │───────────────►│ decide │──► view, decisions, + └───────────┘ └───────────┘ └──────────┘ manifest + selector types ownership rule ODRL, deny wins + (SPARQL, profile) (profile) (engine.py) +``` -| File | Participant | Consent (file-level policy) | +| Layer | What it is | Where | | --- | --- | --- | -| `P001.vcf` | P001 | General research use **and** clinical care use | -| `P002.vcf` | P002 | General research use **and** clinical care use | -| `P003.vcf` | P003 | Health/medical/biomedical research only | -| `P004.vcf` | P004 | **Withdrawn**: prohibited for every requester | -| `P005.vcf` | P005 | Disease-specific research only | +| Engine | Select, partition, decide; knows only ODRL, selector declarations, ownership rules and a purpose hierarchy. It never mentions VCF | `vcf_rdfizer_policies/engine.py` | +| Profile | Selector types, the ownership rule, and the reporting-unit query, in Turtle | `vcf_rdfizer_data/policy/vcf-core-profile.ttl` (bundled), or `--profile` | +| Purposes | An RDFS or SKOS hierarchy | `vcf_rdfizer_data/policy/duo-subset.ttl` (bundled), or `--purposes` | +| VCF oracle | An independent check of a view against the source VCF text (§7) | `vcf_rdfizer_policies/vcf_oracle.py` | -Each file is `VCFv4.3` with `##reference=GRCh38`, 23–34 records (139 in all), -and one sample column. The records sit at real GRCh38 positions in three groups: +### 2.1 Selector types -| Group | Where | Why | -| --- | --- | --- | -| *BRCA1* | chr17:43,044,295–43,125,483 | Actionable incidental findings; the region rule applies here | -| *APOE* | chr19, including rs429358 (44,908,684 T>C) and rs7412 (44,908,822 C>T) | rs429358 defines the ε4 allele; the variant rule applies to it alone, and rs7412 is its unrestricted neighbour | -| Background | a few loci on chr1 and chr20 | Governed only by the file-level consent | - -**The genotypes are synthetic and generated from a fixed seed.** The design -doc's first premise is that genotypes identify people, and a privacy -demonstrator should not be built on real individuals. Positions fall in real -GRCh38 loci, and rs429358 and rs7412 are the real variants, so the region and -variant selectors are tested against real coordinates. Every other allele, and -every genotype, is invented. Those REF alleles are not checked against the -reference. - -The site catalogue also plants the boundary cases the tests need: -- records one base outside each end of the *BRCA1* window, and records on - each end of it, so the bounds are tested as inclusive; -- a decoy T>G at rs429358's position, so the variant rule is tested on its - alleles and not on its position alone. - -The generator is `examples/policy/make_fixture.py`. It is deterministic, and -it writes the five VCFs together with a `fixture.json` that records the seed, -the loci and each file's intended consent. - -**Why single-sample files help:** - -- **Consent is per participant, so it becomes a file-level policy.** The - policy travels with the participant's file, which is how consent actually - works. -- **Withdrawing a participant means withholding one file.** No per-sample - surgery inside a shared graph is needed. -- **Both sample profiles work.** In the condensed profile, a record's sample - vector has exactly one entry, so withholding a record removes whole - literals and never has to rewrite one. The masking problem in - [`privacy-policy-design.md` §8](privacy-policy-design.md#8-the-condensed-representation-problem) - does not arise. Multi-sample files, where it does, are v0.2 (§10). -- **The cohort is a union of graphs.** Each file's IRIs start with - `file://P00n.vcf`, so the five graphs merge without collisions (see - [conversion §6](conversion.md#6-iri-templates)). Region and variant rules - are then cohort-wide rules that cut across files. +A selector type is a SPARQL `SELECT` that projects `?resource`, plus the +parameters a policy supplies. Each `vcfp:parameter` is a property the +selector node must carry, and its value is bound to the query variable named +after the property's local name: `vcfp:start` binds `?start`. An optional +`vcfp:violations` query lists reasons the selector cannot apply to a graph. +Any row it returns stops evaluation; the VCF Core selectors use it to require +that every file declares the policy's assembly. + +This is the shipped region selector, in full: + +```turtle +vcfp:RegionSelector a vcfp:SelectorType ; + vcfp:parameter vcfp:assembly , vcfp:chrom , vcfp:start , vcfp:end ; + vcfp:query """ + PREFIX vcfc: + SELECT ?resource WHERE { + ?resource a vcfc:VCFRecord ; vcfc:chrom ?c ; vcfc:pos ?pos . + FILTER(STR(?c) = STR(?chrom) && ?pos >= ?start && ?pos <= ?end) }""" ; + vcfp:violations """ … files whose vcfc:referenceGenome is not ?assembly … """ . +``` + +A policy uses it like this: + +```turtle +ex:brca1 a odrl:Asset , vcfp:GraphSelection ; + vcfp:selector [ a vcfp:RegionSelector ; vcfp:assembly "GRCh38" ; + vcfp:chrom "chr17" ; vcfp:start 43044295 ; vcfp:end 43125483 ] . +``` + +Selector types are read from the profile files, and also from the policy file +itself, so a policy can bring its own (§8). + +### 2.2 Partitioning + +A withheld resource takes with it everything it **owns**: + +- the resources its `vcfp:ownershipPath` (a SPARQL property path) reaches; +- when `vcfp:iriSubtree` is true, every IRI beneath any of those, after a `#` + or `/`. + +The VCF Core profile sets the path to `vcfc:hasCall/vcfc:hasSampleCall?` and +turns the subtree rule on, which matches the converter's IRIs +([conversion §6](conversion.md#6-iri-templates)): + +- a file owns everything under `file://NAME#`: its header, sample set and + records; +- a record `…#record/9` owns its alleles (`…#record/9/allele/…`), its call + `…#call/9` with that call's INFO values and condensed matrix, and, through + the call, its expanded per-sample calls `…#sample/9/P001`. + +A triple is released when its subject is released, and its object too if the +object is a node of the graph. So no view ever points at something it doesn't +contain. + +### 2.3 Reporting units + +`vcfp:unitQuery` names what the per-unit report counts. It must project +`?resource` and `?group`, and any other variables become report columns. The +VCF Core profile reports records grouped by file, with chrom, pos, ref and alt. --- -## 3. The profile subset +## 3. Policies -Namespace `vcfp:` = `https://w3id.org/vcf-rdfizer/policy#`, as in the full -design. v0.1.0 implements exactly this: +Namespace `vcfp:` = `https://w3id.org/vcf-rdfizer/policy#`. -| Construct | v0.1.0 | Notes | +| Construct | Supported | Notes | | --- | --- | --- | -| `odrl:Set` / `odrl:Policy` | yes | One policy per participant, plus one cohort policy | +| `odrl:Set` / `Policy` / `Offer` / `Agreement` | yes | | | `odrl:permission`, `odrl:prohibition` | yes | | -| `odrl:target` = a file IRI | yes | The file resource *is* the asset; no selector is needed | -| `odrl:target` = `vcfp:GraphSelection` with `vcfp:RegionSelector` | yes | `vcfp:assembly` is required and checked against `vcfc:referenceGenome` | -| `odrl:target` = `vcfp:GraphSelection` with `vcfp:VariantSelector` | yes, **new** | chrom, pos, ref, alt and assembly. The full design has no mutation-level selector; this adds one | -| `odrl:assignee` | yes | An IRI, or `odrl:All` | -| `odrl:constraint` on `odrl:purpose` with `odrl:isAnyOf` / `odrl:isNoneOf` | yes | Right-hand sides are DUO terms, matched through the DUO hierarchy (§3.1) | | `odrl:action odrl:read` | yes | The only action evaluated | -| `odrl:duty` (`odrl:attribute`, `odrl:inform`) | recorded | Copied into the release manifest as obligations the requester accepts. **Not enforced** | -| `odrl:conflict odrl:prohibit` | yes, required | Deny wins; any other value is rejected | +| `odrl:target` = a resource IRI | yes | The resource and everything it owns, e.g. `` | +| `odrl:target` = a `vcfp:GraphSelection` | yes | One `vcfp:selector`, whose type the profile or policy declares | +| `odrl:assignee` | yes | An IRI, or `odrl:All` | +| `odrl:constraint` on `odrl:purpose`, `isAnyOf` / `isNoneOf` | yes | Terms of the purpose vocabulary (§3.1) | +| `odrl:duty` | recorded | Copied into the manifest as obligations. **Not enforced** | +| `odrl:conflict odrl:prohibit` | required | Deny wins; any other value is refused | | `vcfp:transform vcfp:drop` | yes | The only effect | -| `vcfp:SampleSelector`, `FieldSelector`, `HeaderSelector`, `ClassSelector`, `PredicateSelector`, `PatternSelector` | **no** | v0.2 and later (§10). A policy that uses one is rejected, not ignored | -| `generalize`, `pseudonymize`, `threshold`, `aggregateOnly`, `maskVectorPositions` | **no** | Later versions. Rejected if present | +| anything else on a rule (`odrl:refinement`, `odrl:remedy`, …) | **refused** | It could change what the rule means | +| `generalize`, `pseudonymize`, `threshold`, `maskVectorPositions` | **refused** | Later versions (§10) | -**Anything v0.1.0 cannot evaluate stops the run with an error.** That applies -to unknown selectors, unsupported effects, missing assemblies, and assemblies -that don't match the data. This is the design doc's "the residual set must -abort" rule (§6), and it matters most in the smallest version: a demonstrator -that silently skipped a rule would demonstrate the opposite of its purpose. +**What the engine cannot evaluate, it refuses.** A missing parameter, an +undeclared selector type, a purpose outside the vocabulary, a selector whose +`violations` query returns rows: each stops the run with an error, never a +warning. A tool that silently skipped a rule would demonstrate the opposite of +its purpose. -### 3.1 Purposes and the DUO hierarchy +### 3.1 Purposes -Purposes are GA4GH Data Use Ontology terms: +A requester's purpose satisfies a term when it **is that term or a narrower +one**, following `rdfs:subClassOf` or `skos:broader`. So any RDFS or SKOS +vocabulary works, passed with `--purposes`. Names resolve as full IRIs or +through the vocabulary's own prefixes. -| Code | IRI | Role in the fixture | +The default is four GA4GH DUO terms (`duo-subset.ttl`), copied from DUO release +2021-02-23 together with their subclass links: + +| Code | IRI | In the example | | --- | --- | --- | -| GRU, general research use | `obo:DUO_0000042` | P001, P002 consent; the general-research requester | -| HMB, health/medical/biomedical research | `obo:DUO_0000006` | P003 consent | -| DS, disease-specific research | `obo:DUO_0000007` | P005 consent; the Alzheimer's consortium's purpose | -| CC, clinical care use | `obo:DUO_0000043` | P001, P002 consent; the clinical requester | - -In DUO the first three form a chain, DS ⊑ HMB ⊑ GRU. Clinical care use sits -elsewhere: DUO files it as a data use *modifier*, not a permission. v0.1.0 -treats it as a purpose like the others, and it matches only itself. - -Matching uses subsumption. **A requester's purpose satisfies a consent when -the purpose is the consented term or a narrower one**, so a disease-specific -study falls within a general-research consent, but not the other way round. -v0.1.0 bundles the four terms and their `rdfs:subClassOf` links as -`vcf_rdfizer_data/policy/duo-subset.ttl`, copied from DUO release 2021-02-23, -which the file records. Full DUO, with disease qualifiers via MONDO and -release pinning in the policy (`vcfp:duoVersion`), is later (§10). +| GRU, general research use | `DUO:0000042` | P001, P002 consent; the general-research requester | +| HMB, health/medical/biomedical research | `DUO:0000006` | P003 consent | +| DS, disease-specific research | `DUO:0000007` | P005 consent; the Alzheimer's study | +| CC, clinical care use | `DUO:0000043` | P001, P002 consent; the clinical requester | + +DS ⊑ HMB ⊑ GRU, so a disease-specific study falls within a general-research +consent, but not the reverse. DUO files CC as a data use *modifier*, not a +permission. Here it is a purpose like the others, and matches only itself. --- -## 4. Decision semantics (normative for v0.1.0) +## 4. Decision rules (normative) -A **request** is `(assignee a, purpose p)`. The unit of decision is the -**record**. File-level resources (the header, the file, the sample set) are -decided with their file. +A **request** is `(assignee a, purpose p)`. A rule *binds* a request when its +assignee is `odrl:All` or `a`, and each constraint holds: -A rule *applies* to a request when its assignee is `odrl:All` or equals `a`, -and every constraint holds for `p`: +- `isAnyOf S` holds when `p ⊑ s` for some `s` in `S`; +- `isNoneOf S` holds when `p ⊑ s` for none. -- `purpose isAnyOf S` holds when `p ⊑ s` for some `s` in `S`; -- `purpose isNoneOf S` holds when `p ⊑ s` for no `s` in `S`. +For each binding rule, the engine selects its target's resources (§2.1) and +takes everything they own (§2.2). -A rule *covers* a record when its target selects the record: the record's -file, a region containing the record's `vcfc:pos` on its `vcfc:chrom`, or a -variant matching its chrom, pos, ref and alt. +**A resource is released if and only if:** -**Record `r` in file `f` is released to request `q` if and only if:** +1. some binding **permission** owns it — the default-deny rule: no + permission, no release; **and** +2. no binding **prohibition** owns it — deny wins. -1. some **permission** that applies to `q` covers `f` — the default-deny - rule: no permission, no release; **and** -2. no **prohibition** that applies to `q` covers `r` or `f` — deny wins. +A permission on a file and a prohibition on a region inside it combine as +"everything in the file except the region". No "most-specific target wins" +rule is needed, which is fortunate, since ODRL does not define one. -A file's own resources are released when (1) and (2) hold for the file -itself. A released triple must have both its subject and any IRI object -released, so no triple in a view points at something the view withholds. +--- -The fixture's cohort policy has two rules, and both are prohibitions with an -exception. That keeps the semantics above complete without needing -"most-specific target wins", which ODRL doesn't define: +## 5. The example: one cohort, three requesters -```turtle -@prefix odrl: . -@prefix vcfp: . -@prefix obo: . - -<#brca1> a odrl:Asset , vcfp:GraphSelection ; - vcfp:selector [ a vcfp:RegionSelector ; vcfp:assembly "GRCh38" ; - vcfp:chrom "chr17" ; vcfp:start 43044295 ; vcfp:end 43125483 ] . - -<#apoe-e4> a odrl:Asset , vcfp:GraphSelection ; - vcfp:selector [ a vcfp:VariantSelector ; vcfp:assembly "GRCh38" ; - vcfp:chrom "chr19" ; vcfp:pos 44908684 ; vcfp:ref "T" ; vcfp:alt "C" ] . - -<#cohort-policy> a odrl:Set ; - odrl:uid ; - odrl:profile ; - odrl:conflict odrl:prohibit ; - # Incidental findings: readable only for clinical care. - odrl:prohibition [ odrl:target <#brca1> ; odrl:action odrl:read ; odrl:assignee odrl:All ; - odrl:constraint [ odrl:leftOperand odrl:purpose ; odrl:operator odrl:isNoneOf ; - odrl:rightOperand obo:DUO_0000043 ] ] ; - # APOE e4: readable only for disease-specific research. - odrl:prohibition [ odrl:target <#apoe-e4> ; odrl:action odrl:read ; odrl:assignee odrl:All ; - odrl:constraint [ odrl:leftOperand odrl:purpose ; odrl:operator odrl:isNoneOf ; - odrl:rightOperand obo:DUO_0000007 ] ] . - -# A participant's consent, attached to their file. -<#consent-P003> a odrl:Set ; - odrl:uid ; - odrl:profile ; - odrl:conflict odrl:prohibit ; - odrl:permission [ odrl:target ; odrl:action odrl:read ; - odrl:assignee odrl:All ; - odrl:constraint [ odrl:leftOperand odrl:purpose ; odrl:operator odrl:isAnyOf ; - odrl:rightOperand obo:DUO_0000006 ] ; - odrl:duty [ odrl:action odrl:attribute ] ] . -``` +`examples/policy/` holds five synthetic participants, one small single-sample +VCF each: 23–34 records, 139 in all. The positions are GRCh38 and fall in real +loci: *BRCA1* (chr17:43,044,295–43,125,483), *APOE* (including the real +rs429358 T>C and rs7412 C>T), and background sites on chr1 and chr20. + +Every other allele, and every genotype, is synthetic, drawn from a fixed seed. +The design's first premise is that genotypes identify people, so a privacy +example should not be built on real ones. + +The generator also plants boundary cases: +- records one base inside and one base outside each end of the *BRCA1* window; +- a decoy T>G at rs429358's position, to test that the variant rule matches + alleles and not just position. -### 4.1 What the three requesters get +Single-sample files make each participant's consent a policy on their own +file, and a withdrawal the withholding of one file. They also mean the +condensed profile's per-sample vectors have one entry each, so nothing ever +has to be masked *inside* a literal. + +`policy.ttl` holds: +- five consents, one per file (P004's includes the withdrawal, a prohibition); +- a cohort policy with two prohibitions, *BRCA1* unless the purpose is CC and + rs429358 unless the purpose is DS. | Requester | Purpose | P001 | P002 | P003 | P004 | P005 | *BRCA1* | rs429358 | | --- | --- | --- | --- | --- | --- | --- | --- | --- | -| General-research consortium | GRU | ✓ | ✓ | — | — | — | withheld | withheld | -| Alzheimer's consortium | DS | ✓ | ✓ | ✓ | — | ✓ | withheld | **released** | -| Clinical genetics lab | CC | ✓ | ✓ | — | — | — | **released** | withheld | - -Every cell follows from §4 alone: +| General-research consortium | GRU | 21 | 16 | — | — | — | withheld | withheld | +| Alzheimer's consortium | DS | 22 | 16 | 18 | — | 20 | withheld | **released** | +| Clinical genetics lab | CC | 33 | 23 | — | — | — | **released** | withheld | -- P003 is out for general research, because GRU is broader than P003's - health-research consent. -- P003 and P005 are in for the Alzheimer's consortium, because DS falls - within both HMB and DS. -- P004 is out everywhere: the withdrawal is a prohibition, and prohibitions - win. -- *BRCA1* and rs429358 are withheld unless the purpose is the one - exception each rule allows. +The numbers are records released from each file. Every cell follows from §4: +- P003's HMB consent admits DS but not GRU; +- P004's withdrawal is a prohibition, so it wins everywhere; +- each cohort rule is lifted for exactly one purpose. -This grid is the demonstrator's headline result, and §7 makes it the oracle. +Released records total 37, 76 and 56 of 139, with 7,869, 4,649 and 6,539 +triples withheld in the expanded profile. The condensed profile releases the +same records. --- -## 5. What `attach` adds to the graph +## 6. What it writes -`attach` merges the policy graph into the data. It then materializes what each -selection selects, so policies can be queried alongside data without -re-implementing the selectors in SPARQL: +### 6.1 `attach`: policies in the graph + +`attach` merges the policies into the data. It links each directly targeted +resource, and each resource a selection selects, to its policy: ```turtle - odrl:hasPolicy <#consent-P003> . -<#brca1> vcfp:selects , … . - odrl:hasPolicy <#cohort-policy> . + odrl:hasPolicy ex:consent-P003 . +ex:brca1 vcfp:selects , … . + odrl:hasPolicy ex:cohort . ``` -It also writes an attachment manifest: the policy's digest, and the selection -counts per asset. - -The query that makes the argument: *which records could I not share with a -general-research collaborator, and why?* +Policy and data are then one graph, and one query asks which records a +general-research collaborator may not receive, and under which rule. It needs +no knowledge of the selectors: ```sparql PREFIX odrl: PREFIX vcfc: -PREFIX obo: SELECT ?record ?chrom ?pos ?rule WHERE { - ?record a vcfc:VCFRecord ; vcfc:chrom ?chrom ; vcfc:pos ?pos ; - odrl:hasPolicy ?policy . - ?policy odrl:prohibition ?rule . - ?rule odrl:constraint [ odrl:operator odrl:isNoneOf ; odrl:rightOperand ?exempt ] . - FILTER(?exempt != obo:DUO_0000042) -} + ?record a vcfc:VCFRecord ; vcfc:chrom ?chrom ; vcfc:pos ?pos ; odrl:hasPolicy ?policy . + ?policy odrl:prohibition ?rule . } ``` -That query runs over the annotated graph unchanged, under any of the four -engines the validation stage already uses. No VCF tool can answer it. - ---- - -## 6. What `evaluate` writes - -For one request it writes a directory: +### 6.2 `evaluate`: one request's release | File | Contents | | --- | --- | -| `view.nt` | The release view: the released triples, sorted | -| `manifest.ttl` | A `vcfp:ReleaseView` (§6.1) | -| `decisions.csv` | One row per record: file, chrom, pos, ref, alt, released or withheld, and the rule that decided it | -| `summary.json` | Counts per file and per rule, for the paper figure | - -### 6.1 The manifest - -This follows [`privacy-policy-design.md` §10](privacy-policy-design.md#10-the-release-manifest), -with the fields v0.1.0 can fill: +| `view.nt` | The released triples, sorted | +| `decisions.csv` | One row per reporting unit: the unit query's columns, released or not, and the rule that decided it | +| `summary.json` | Counts per group and per deciding reason | +| `manifest.ttl` | The `vcfp:ReleaseView`, below | ```turtle <#release> a vcfp:ReleaseView ; - vcfp:derivedFrom , , , , ; - vcfp:policy ; + vcfp:derivedFrom , … , ; + vcfp:policy ex:cohort , ex:consent-P001 , … ; vcfp:policyDigest "sha256:…" ; vcfp:request [ odrl:assignee ; odrl:purpose obo:DUO_0000007 ] ; vcfp:recordsReleased 76 ; vcfp:recordsWithheld 63 ; - vcfp:filesWithheld 1 ; vcfp:triplesWithheld 4649 ; + vcfp:groupsWithheld 1 ; vcfp:triplesWithheld 4649 ; vcfp:obligation [ odrl:action odrl:attribute ] ; vcfp:disclosureModel "governed release; not anonymization" ; - prov:wasGeneratedBy [ prov:used ] ; + prov:wasGeneratedBy ; prov:generatedAtTime "…"^^xsd:dateTime . ``` -The numbers are the Alzheimer's consortium's view of the expanded fixture. The -policy is referenced by digest as well as by IRI, and the view keeps the -original IRIs, since pseudonymization is v0.4 (§10). The manifest says so in -`vcfp:disclosureModel`. - -### 6.2 Implementation - -The evaluator loads the union graph with `rdflib`, which is already a runtime -dependency, so no image change is needed. It then does three things: - -- **Resolves selectors to records** with one SPARQL query per selector, over - `vcfc:chrom`, `vcfc:pos`, `vcfc:ref` and `vcfc:alt`, with the assembly - checked against each file's `vcfc:referenceGenome`. -- **Decides each record** by §4. -- **Expands each withheld record to its subtree by IRI prefix**, using the - hierarchy in [conversion §6](conversion.md#6-iri-templates). Every IRI under - `file://{FILE}` belongs to that file, header and sample set included; only - the three row subtrees also carry a row: - - `#record/{ROW}` and everything under it (alleles, events); - - `#call/{ROW}` and everything under it (INFO values; the condensed matrix - and vectors); - - `#sample/{ROW}/` and everything under it (expanded sample calls, - genotypes and FORMAT values). - - A withheld file removes everything under `file://{FILE}`. A final pass - drops any triple whose IRI object falls in a withheld subtree. - -This is in memory by design. A size guard refuses graphs above 5M triples, -with a message that names the demonstrator's scope. The streaming version is -v0.3 (§10). +The policy is recorded by digest as well as by IRI, since an IRI's content can +change after the fact. Views keep the source IRIs, as there is no +pseudonymization yet (§10), and `vcfp:disclosureModel` says so. --- -## 7. How `check` verifies a view - -The checks follow the validation methodology: a claim is tested against an -independent computation, not taken on trust. - -1. **Oracle agreement.** `check` reparses the source VCFs directly, without - the graph or any SPARQL, and computes which records each request should - see from `fixture.json` and §4. The view's released records must equal - that set **exactly**: - - an extra record is a leak; - - a missing record is over-withholding. - - Both are failures, because a redactor that withheld everything would - otherwise pass. -2. **Prohibition ASKs.** Each prohibition that applies is compiled into a - SPARQL `ASK` that must be false on the view: for example, any record in - the *BRCA1* window, or any subject under `file://P004.vcf`. -3. **No dangling references.** No IRI object in the view points into a - withheld subtree. -4. **Header and IRI leakage.** No IRI or header literal in the view names a - withheld participant. In v0.1.0 that can only occur through a - whole-file withdrawal, which the prefix rule removes, so the check - guards against regressions in that rule. - -**The check proved its worth on its first run.** An early `split()` recognised -only the three row subtrees, so a withdrawn file's header and sample set stayed -in every view. Every record decision was correct, so the leak was invisible from the -record counts. Check 4 flagged it at once, naming the IRI that should not -have been there. - -**Mutation tests** (`test/test_policy_unit.py`) break a correct view on -purpose and assert that `check` catches each break: - -- reinsert one withheld *BRCA1* triple; -- restore one triple of P004; -- delete one record that should have been released; -- leave one dangling allele reference; -- swap two requesters' views. - -A check that has never caught a planted leak is not evidence of anything. +## 7. Checking a view + +`vcf-rdfizer-policy check --view DIR --rdf SOURCE --policy P [--vcf …]` fails +on each of the following. + +**Structural checks, for any graph:** +1. The manifest's policy digest doesn't match the policy. +2. Anything a binding prohibition owns appears in the view, as subject or + object. +3. A subject in the view is owned by no binding permission (default-deny). +4. A triple points at a node of the source that the view does not contain. + +**The VCF oracle (`--vcf`):** it reads the source VCFs as text and builds a +minimal VCF Core graph of the fixed columns except INFO: CHROM, POS, ID, REF, +ALT, QUAL and FILTER. It then evaluates the same policy on that graph. The +view's records must equal the records released there **exactly**: an extra +record is a leak, and a missing one is over-withholding. + +The structural checks reuse the engine's selectors, so they confirm that a +view honours the policy, but they cannot catch a wrong selector. The oracle +can, because its input never went through the converter. A selector that +reads INFO, FORMAT or the header is outside what the oracle models, so don't +pass `--vcf` for such a policy. + +**The checks earned their place early.** In development, a withdrawn file's +header and sample set stayed in every view even though every record decision +was right, so nothing looked wrong from the counts. The check named the leaked +IRI. That failure is why the partition is defined by ownership (§2.2) rather +than by a list of record subtrees. + +**The tests** are in +[vcf-rdfizer-testing `plugin-tests/policy/`](https://github.com/ecrum19/vcf-rdfizer-testing/tree/main/plugin-tests), +kept apart from this code and run by hand. They cover: +- the §5 grid cell by cell, in both profiles; +- the refusals; +- the generality cases: a selector declared in a policy, a non-VCF graph with + a property-path partition, a SKOS vocabulary; +- mutation tests, where each planted fault must fail `check`: a reinstated + prohibited record, a restored triple of the withdrawn file, a deleted + record, a dangling reference, a swapped view, and a changed policy. --- -## 8. The command - -`vcf-rdfizer-policy`, a separate console script in the pattern of -`vcf-rdfizer-link`. It runs on the host and needs no Docker. - -```bash -vcf-rdfizer-policy attach --rdf P00*.nt.gz --policy policy.ttl -o cohort-annotated.nt -vcf-rdfizer-policy evaluate --rdf P00*.nt.gz --policy policy.ttl \ - --assignee https://example.org/party/alz-consortium \ - --purpose obo:DUO_0000007 -o views/alz -vcf-rdfizer-policy check --view views/alz --policy policy.ttl --vcf P00*.vcf -vcf-rdfizer-policy explain --policy policy.ttl # the §4.1 grid, from the policy alone -``` +## 8. Extending it without code -| Exit code | Meaning | -| --- | --- | -| 0 | Success; for `check`, every check passed | -| 1 | `check` found a leak, over-withholding or a dangling reference | -| 2 | The policy uses something v0.1.0 cannot evaluate, the assembly doesn't match, or the graph is over the size guard | +**A new selector.** Declare a `vcfp:SelectorType`, either in the policy file +or in a file passed with `--profile`. +[`examples/policy/custom-selector.ttl`](../examples/policy/custom-selector.ttl) +adds "records whose QUAL is below a threshold" as one short declaration, and +withholds 38 of the example's 139 records at a threshold of 60. Selectors for +FILTER state, an INFO value, a sample (for multi-sample files), or genes from a +linkset all follow the same pattern. -**Layout:** +**Another kind of graph.** Pass a profile with a different ownership rule and +unit query. For a graph of documents that own their sections through +`ex:hasSection`, with opaque IRIs: -```text -vcf_rdfizer_policy.py console entry point: vcf-rdfizer-policy -vcf_rdfizer_policies/ (named like vcf_rdfizer_link.py / vcf_rdfizer_linking/) - __init__.py version, namespaces, PolicyError - profile.py parse + validate a policy against the v0.1.0 subset - purposes.py DUO subsumption over the bundled subset - graphs.py load graphs; records; which file and row an IRI belongs to - decide.py the §4 semantics; the one place they live - release.py evaluate, attach, and the manifest - check.py the VCF-side oracle and the structural checks -vcf_rdfizer_data/policy/ - vcfp-0.1.ttl the profile terms v0.1.0 defines - duo-subset.ttl four DUO terms and their hierarchy, release recorded -examples/policy/ - make_fixture.py fixture.json P001.vcf … P005.vcf policy.ttl - converted/{expanded,condensed}/ the converted graphs, committed so the demo runs without Docker - run_demo.sh README.md -test/test_policy_unit.py runs in CI (matches test_*_unit.py) +```turtle +ex:Docs a vcfp:Profile ; + vcfp:ownershipPath "*" ; + vcfp:iriSubtree false ; + vcfp:unitQuery "SELECT ?resource ?group WHERE { ?group ?resource }" . ``` -The requesters live in `fixture.json`, which both `run_demo.sh` and the -tests read. - -`decide.py` holds the semantics, and **both `evaluate` and `check`'s oracle -call it**, so the §4 rules exist in exactly one place. The oracle's -independence comes from its input, the VCF text rather than the graph, not -from a second implementation of the rules. +**Other purposes.** Pass `--purposes` a vocabulary: full DUO, or a local SKOS +scheme. --- -## 9. What goes into the paper +## 9. The command -The capabilities table's *Provenance/policy annotations* row currently reads -"Not implemented / Not evaluated". It becomes: - -> **Demonstrator (v0.1.0)** · **Exercised on a fixture** · ODRL policies -> attached to files, regions and variants; per-request release views -> verified against a VCF-side oracle; not anonymization, and not enforced at -> conversion time. - -Plus one figure and one short passage (Discussion, or a Results subsection -next to linking): - -**Figure: policies attached at three granularities, and what they do.** - -- **(a)** Where the policies attach. A schematic, in the style of Figure 2: a - participant's consent on `VCFFile`, the *BRCA1* rule on a region selection, - the ε4 rule on one variant, and each linked to its policy by - `odrl:hasPolicy`. -- **(b)** What each requester receives. The §4.1 grid, with participants and - the two cohort rules as rows and the three requesters as columns, cells - marked released or withheld, and triples withheld per requester along the - bottom. The data come from `summary.json`, drawn by `make_figures.py`. -- **(c)** A policy and the question it enables. The 10-line ODRL rule for - rs429358 beside the §5 query. This can be a listing instead, if the figure - gets crowded. +```bash +vcf-rdfizer-policy explain --policy policy.ttl +vcf-rdfizer-policy attach --rdf P00*.nt.gz --policy policy.ttl -o annotated.nt +vcf-rdfizer-policy evaluate --rdf P00*.nt.gz --policy policy.ttl \ + --assignee https://example.org/party/alz-consortium --purpose DUO:0000007 -o views/alz +vcf-rdfizer-policy check --view views/alz --rdf P00*.nt.gz --policy policy.ttl --vcf P00*.vcf +``` -**The passage states the advantages, each with what supports it:** +Every subcommand takes: +- `--profile`, repeatable; a file or `vcf-core` (the default); +- `--purposes`, the purpose vocabulary. -- Granularity finer than a dataset-level consent code: file, region and - variant, each governed separately (panel b). -- Policies that are queryable alongside the data they govern (the §5 query). -- Standards rather than bespoke configuration: ODRL, DUO and PROV. -- Releases that are auditable: a digest-pinned manifest per view. -- A claim that is measured: every view matches an independent VCF-side - oracle, and the mutation tests show the check catches planted leaks. +The command runs on the host and needs `rdflib`, not Docker. It evaluates in +memory, and refuses graphs over 5M triples. -**And it states the limits,** in the same paragraph: +| Exit code | Meaning | +| --- | --- | +| 0 | Success; for `check`, every check passed | +| 1 | `check` found a failure | +| 2 | The policy, profile, graph or request cannot be evaluated | -- the fixture is synthetic, and small; -- there is no pseudonymization, so a view keeps its original IRIs; -- enforcement happens after conversion, not during it; -- duties are recorded but not enforced; -- it is governed release, not anonymization. +```text +vcf_rdfizer_policy.py the command +vcf_rdfizer_policies/ + engine.py select, partition, decide -- generic + policy.py ODRL -> rules; refuses what it cannot evaluate + profile.py selector types and the ownership rule, from Turtle + vocabulary.py purpose hierarchies (RDFS / SKOS) + release.py evaluate, attach, manifest + check.py the structural checks -- generic + vcf_oracle.py the VCF-text oracle + graphs.py loading, IRI hierarchy +vcf_rdfizer_data/policy/ + vcf-core-profile.ttl the VCF Core profile: region and variant selectors, ownership, units + duo-subset.ttl the default purpose vocabulary + vcfp-0.1.ttl the profile terms +examples/policy/ the cohort, policy.ttl, custom-selector.ttl, run_demo.sh +``` --- ## 10. Beyond v0.1.0 -Each version maps onto the full design's build order +Mapped onto the full design's build order ([`privacy-policy-design.md` §13](privacy-policy-design.md#13-build-order)). +Selectors are now declarations, so the full design's sample, field, header and +class selectors are Turtle, not engine work. What remains needs code: | Version | Adds | Design § | | --- | --- | --- | -| **v0.2** | Multi-sample files: `vcfp:SampleSelector` (expanded first), then `vcfp:maskVectorPositions` for condensed, with the single-position-mask warning; `FieldSelector` and `HeaderSelector`; the `generalize` effect (genotype → carrier status, which suits *APOE*) | §4.1, §4.2, §8 | -| **v0.3** | Enforcement during conversion: TSV-level (Tier 1) and emitter-level (Tier 2), including `CHROM`/`POS` on `ParsedSampleRecord`; a streaming post-hoc `--mode redact` (Tier 3) with the two-pass region map; the size guard lifted | §5 | -| **v0.4** | Pseudonymization with IRI re-minting and per-release keys, and sorting views by token so row order cannot be reconstructed | §7 | -| **v0.5** | Full DUO with release pinning and MONDO disease qualifiers; `policy diff` and a fuller `policy check` (lint); enforced duties, with an audit sink for `odrl:inform` | §4.3, §12 | -| later | `threshold`; query-time rewriting for an operated endpoint, paired with aggregate controls | §5, §9 | - -Two rules carry forward unchanged from v0.1.0: +| **v0.2** | Multi-sample files: a sample selector (a declaration) for expanded graphs, and `maskVectorPositions` for condensed ones, which rewrites a literal and so needs code; the `generalize` effect (genotype → carrier status) | §4.2, §8 | +| **v0.3** | Enforcement during conversion (TSV and emitter tiers), and a streaming evaluator, lifting the size limit | §5 | +| **v0.4** | Pseudonymization: IRI re-minting with per-release keys | §7 | +| **v0.5** | Full DUO with release pinning and MONDO qualifiers; `policy diff`; enforced duties with an audit sink | §4.3, §12 | +| later | `threshold`; query-time rewriting for an operated endpoint | §5, §9 | -- **Anything that can't be enforced stops the run.** A version may widen what - can be enforced, but never replace that error with a warning. -- **Every version ships its oracle checks and mutation tests with it.** The - full design says it directly: "A crude redaction that is verified is worth - more than a sophisticated one that is not." +Two rules carry forward unchanged: ---- - -## 11. Work plan - -| Step | Delivers | Estimate | -| --- | --- | --- | -| 1 | `make_fixture.py`, the five VCFs, `fixture.json`; convert and commit the graphs | 0.5 day | -| 2 | `profile.py` (subset validation, rejection of everything else), `purposes.py`, `vcfp-0.1.ttl`, `duo-subset.ttl` | 0.5–1 day | -| 3 | `selectors.py`, `decide.py`, `evaluate.py`: views, manifest, `decisions.csv`, `summary.json` | 1–1.5 days | -| 4 | `attach.py`, and the §5 query checked under the validation engines | 0.5 day | -| 5 | `check.py` (oracle, ASKs, dangling, leakage) and `test/test_policy_unit.py`, including the mutation tests | 1 day | -| 6 | `vcf_rdfizer_policy_cli.py`, `explain`, `run_demo.sh`, `examples/policy/README.md`, links from `limitations.md` and `roadmap.md` | 0.5 day | -| 7 | Paper: the capabilities row, the passage, the figure (panel b from `summary.json`) | 0.5 day | -| | **Total** | **about 4–5 days** | - -**Done means all of the following:** - -- `run_demo.sh` produces three views from a clean checkout without Docker. -- `check` passes on all three views. -- Every mutation test fails the check it targets. -- The CI suite passes. -- The paper's figure is regenerated from the demo's own output. +- **What cannot be enforced stops the run.** A version may widen what can be + enforced, but never turn that error into a warning. +- **Every version ships with its checks and mutation tests.** "A crude + redaction that is verified is worth more than a sophisticated one that is + not." --- @@ -541,5 +442,5 @@ Two rules carry forward unchanged from v0.1.0: - [Privacy policy design](privacy-policy-design.md) — the full design this is a slice of - [Data linking](datalinking.md) — the sibling plug-in; the same manifest and provenance pattern -- [Conversion §6](conversion.md#6-iri-templates) — the IRI hierarchy the evaluator relies on +- [Conversion §6](conversion.md#6-iri-templates) — the IRIs the VCF Core ownership rule follows - [Validation methodology](validation-methodology.md) — why a policy claim needs an oracle diff --git a/examples/policy/README.md b/examples/policy/README.md index 584e05d..1ef7b65 100644 --- a/examples/policy/README.md +++ b/examples/policy/README.md @@ -1,12 +1,12 @@ -# Policy demonstrator (v0.1.0) +# Policy attachment example (v0.1.0) One synthetic cohort, one set of ODRL policies, three requesters, and three -different release views, each checked against the source VCFs. The design, and -what v0.1.0 deliberately leaves out, is +different release views, each checked against the source VCFs. How the engine +works (select → partition → decide, each configured in Turtle) is in [`docs/policy-demonstrator.md`](../../docs/policy-demonstrator.md). -This is **governed release, not anonymization.** The views keep the original -IRIs, and a released genotype still identifies the person it came from. +This is **governed release, not anonymization.** Views keep the original IRIs, +and a released genotype still identifies the person it came from. ## Run it @@ -16,7 +16,7 @@ PROFILE=condensed examples/policy/run_demo.sh /tmp/policy-demo-condensed ``` It explains the policy, attaches it to the graph, and then for each requester -evaluates a view, checks it, and finally prints the decision grid: +evaluates a view and checks it. Finally it prints the decision grid: ```text requester P001.vcf P002.vcf P003.vcf P004.vcf P005.vcf triples withheld @@ -28,7 +28,7 @@ gru 21 16 withheld withheld withheld 7869 Each number is the records released from that file. Where P001 and P002 differ between requesters, that's the cohort rules at work. *BRCA1* records are released only for clinical care, and the APOE ε4 variant (rs429358) only for -disease-specific research. +disease-specific research. P004 withdrew. ## What is here @@ -37,14 +37,15 @@ disease-specific research. | `make_fixture.py` | Deterministic generator for the five VCFs and `fixture.json` | | `P001.vcf` … `P005.vcf` | One synthetic participant each, GRCh38, 23–34 records | | `fixture.json` | Seed, loci, each participant's consent, and the three requesters | -| `policy.ttl` | Five consent policies (one per file) and the cohort policy (*BRCA1*, ε4) | +| `policy.ttl` | Five consents (one per file) and the cohort policy (*BRCA1*, ε4) | +| `custom-selector.ttl` | A selector type declared in the policy itself (below) | | `converted/` | The VCFs converted once per sample profile, so nothing needs Docker; see `PROVENANCE.json` | | `run_demo.sh` | The walkthrough above | -Positions are real GRCh38 coordinates in real loci. rs429358 and rs7412 are -the real variants. Every other allele, and every genotype, is synthetic. To -regenerate the fixture, run `python3 examples/policy/make_fixture.py`. A test -checks that the committed files are byte-identical to what it writes. +The positions are real GRCh38 coordinates in real loci, and rs429358 and +rs7412 are the real variants. Every other allele, and every genotype, is +synthetic. To regenerate the fixture, run +`python3 examples/policy/make_fixture.py`. ## The pieces, one at a time @@ -53,17 +54,53 @@ vcf-rdfizer-policy explain --policy policy.ttl vcf-rdfizer-policy attach --rdf converted/expanded/P00*.nt.gz --policy policy.ttl -o annotated.nt vcf-rdfizer-policy evaluate --rdf converted/expanded/P00*.nt.gz --policy policy.ttl \ --assignee https://example.org/party/alz-consortium --purpose DUO:0000007 -o views/alz -vcf-rdfizer-policy check --view views/alz --policy policy.ttl --vcf P00*.vcf +vcf-rdfizer-policy check --view views/alz --rdf converted/expanded/P00*.nt.gz \ + --policy policy.ttl --vcf P00*.vcf ``` `evaluate` writes the following into its output directory: - `view.nt`, the released triples; - `decisions.csv`, one row per record with the rule that decided it; - `summary.json`, the counts; -- `manifest.ttl`, the policy's digest, the request, what was withheld, the - obligations the requester accepted, and `"governed release; not - anonymization"`. +- `manifest.ttl`, the policy's digest, the request, what was withheld and the + obligations accepted. -`check` re-derives the expected release straight from the VCF text and fails -on any difference. An extra record counts as a leak, and a missing one as -over-withholding. +`check` runs the structural checks against the source graph. With `--vcf`, it +also re-derives the expected records straight from the VCF text and fails on +any difference. + +## Write your own selector + +Selector types are SPARQL, declared in Turtle, and there is nothing to code. +`custom-selector.ttl` declares one inside the policy that uses it: + +```turtle +ex:QualityBelow a vcfp:SelectorType ; + vcfp:parameter ex:threshold ; # bound to ?threshold + vcfp:query """ + PREFIX vcfc: + SELECT ?resource WHERE { + ?resource a vcfc:VCFRecord ; vcfc:hasCall ?call . + ?call vcfc:qual ?qual . + FILTER(?qual < ?threshold) }""" . + +ex:low-quality a odrl:Asset , vcfp:GraphSelection ; + vcfp:selector [ a ex:QualityBelow ; ex:threshold 60 ] . +``` + +With every file released for general research except that selection, 38 of +the 139 records are withheld: + +```bash +vcf-rdfizer-policy evaluate --rdf converted/expanded/P00*.nt.gz --policy custom-selector.ttl \ + --assignee https://example.org/party/anyone --purpose DUO:0000042 -o views/quality +``` + +The query must project `?resource`. Each `vcfp:parameter` names a property of +the selector node, bound to the variable of the same local name. A selector +type can also carry a `vcfp:violations` query, and any row it returns stops +evaluation; the shipped region and variant selectors use one to check the +assembly. + +To share a selector across policies, put it in a file and pass +`--profile vcf-core --profile my-selectors.ttl`. diff --git a/examples/policy/custom-selector.ttl b/examples/policy/custom-selector.ttl new file mode 100644 index 0000000..3282a83 --- /dev/null +++ b/examples/policy/custom-selector.ttl @@ -0,0 +1,53 @@ +# Extending the policy engine without code: a selector type declared in the +# policy file itself. +# +# ex:QualityBelow selects records whose call has QUAL under a threshold. The +# declaration is a SPARQL SELECT projecting ?resource; its one parameter, +# ex:threshold, is bound to ?threshold from the selector node below. The engine +# loads it alongside the bundled VCF Core profile and treats it exactly like +# the shipped region and variant selectors. +# +# vcf-rdfizer-policy evaluate --rdf converted/expanded/P00*.nt.gz \ +# --policy custom-selector.ttl --assignee https://example.org/party/anyone \ +# --purpose DUO:0000042 -o views/quality + +@prefix odrl: . +@prefix vcfp: . +@prefix obo: . +@prefix ex: . + +# --- The new selector type -------------------------------------------------- +ex:QualityBelow a vcfp:SelectorType ; + vcfp:parameter ex:threshold ; + vcfp:query """ + PREFIX vcfc: + SELECT ?resource WHERE { + ?resource a vcfc:VCFRecord ; vcfc:hasCall ?call . + ?call vcfc:qual ?qual . + FILTER(?qual < ?threshold) }""" . + +# --- A policy that uses it ------------------------------------------------------ +ex:low-quality a odrl:Asset , vcfp:GraphSelection ; + vcfp:selector [ a ex:QualityBelow ; ex:threshold 60 ] . + +ex:release a odrl:Set ; + odrl:uid ex:release ; + odrl:conflict odrl:prohibit ; + # Every file may be read for general research ... + odrl:permission [ odrl:target ; odrl:action odrl:read ; + odrl:constraint [ odrl:leftOperand odrl:purpose ; odrl:operator odrl:isAnyOf ; + odrl:rightOperand obo:DUO_0000042 ] ] ; + odrl:permission [ odrl:target ; odrl:action odrl:read ; + odrl:constraint [ odrl:leftOperand odrl:purpose ; odrl:operator odrl:isAnyOf ; + odrl:rightOperand obo:DUO_0000042 ] ] ; + odrl:permission [ odrl:target ; odrl:action odrl:read ; + odrl:constraint [ odrl:leftOperand odrl:purpose ; odrl:operator odrl:isAnyOf ; + odrl:rightOperand obo:DUO_0000042 ] ] ; + odrl:permission [ odrl:target ; odrl:action odrl:read ; + odrl:constraint [ odrl:leftOperand odrl:purpose ; odrl:operator odrl:isAnyOf ; + odrl:rightOperand obo:DUO_0000042 ] ] ; + odrl:permission [ odrl:target ; odrl:action odrl:read ; + odrl:constraint [ odrl:leftOperand odrl:purpose ; odrl:operator odrl:isAnyOf ; + odrl:rightOperand obo:DUO_0000042 ] ] ; + # ... except records below the quality threshold, which no one receives. + odrl:prohibition [ odrl:target ex:low-quality ; odrl:action odrl:read ] . diff --git a/examples/policy/fixture.json b/examples/policy/fixture.json index 34c8865..cb70b27 100644 --- a/examples/policy/fixture.json +++ b/examples/policy/fixture.json @@ -78,17 +78,17 @@ "requesters": { "gru": { "assignee": "https://example.org/party/research-consortium", - "purpose": "DUO_0000042", + "purpose": "DUO:0000042", "label": "General-research consortium" }, "alz": { "assignee": "https://example.org/party/alz-consortium", - "purpose": "DUO_0000007", + "purpose": "DUO:0000007", "label": "Alzheimer's consortium" }, "clinical": { "assignee": "https://example.org/party/clinical-genetics", - "purpose": "DUO_0000043", + "purpose": "DUO:0000043", "label": "Clinical genetics lab" } } diff --git a/examples/policy/make_fixture.py b/examples/policy/make_fixture.py index eb5c1b3..8e51a8d 100644 --- a/examples/policy/make_fixture.py +++ b/examples/policy/make_fixture.py @@ -49,11 +49,11 @@ } REQUESTERS = { - "gru": {"assignee": "https://example.org/party/research-consortium", "purpose": "DUO_0000042", + "gru": {"assignee": "https://example.org/party/research-consortium", "purpose": "DUO:0000042", "label": "General-research consortium"}, - "alz": {"assignee": "https://example.org/party/alz-consortium", "purpose": "DUO_0000007", + "alz": {"assignee": "https://example.org/party/alz-consortium", "purpose": "DUO:0000007", "label": "Alzheimer's consortium"}, - "clinical": {"assignee": "https://example.org/party/clinical-genetics", "purpose": "DUO_0000043", + "clinical": {"assignee": "https://example.org/party/clinical-genetics", "purpose": "DUO:0000043", "label": "Clinical genetics lab"}, } diff --git a/examples/policy/run_demo.sh b/examples/policy/run_demo.sh index 9710167..68eb549 100755 --- a/examples/policy/run_demo.sh +++ b/examples/policy/run_demo.sh @@ -30,7 +30,7 @@ while read -r key assignee purpose; do echo; echo "== $key ($purpose)" "${POLICY_CMD[@]}" evaluate --rdf "${RDF[@]}" --policy "$POLICY" \ --assignee "$assignee" --purpose "$purpose" -o "$OUT/views/$key" - "${POLICY_CMD[@]}" check --view "$OUT/views/$key" --policy "$POLICY" --vcf "${VCF[@]}" + "${POLICY_CMD[@]}" check --view "$OUT/views/$key" --policy "$POLICY" --rdf "${RDF[@]}" --vcf "${VCF[@]}" done < <(python3 -c 'import json,sys for k, r in json.load(open(sys.argv[1]))["requesters"].items(): print(k, r["assignee"], r["purpose"])' "$HERE/fixture.json") @@ -40,10 +40,10 @@ import json, sys from pathlib import Path views = sorted(Path(sys.argv[1]).iterdir()) summaries = {v.name: json.loads((v / "summary.json").read_text()) for v in views} -files = sorted(next(iter(summaries.values()))["files"]) +files = sorted(next(iter(summaries.values()))["groups"]) print("requester".ljust(10) + "".join(f.split("//")[1].ljust(10) for f in files) + "triples withheld") for name, s in summaries.items(): - cells = "".join((str(s["files"][f]["records_released"]) if s["files"][f]["released"] else "withheld").ljust(10) + cells = "".join((str(s["groups"][f]["records_released"]) if s["groups"][f]["released"] else "withheld").ljust(10) for f in files) print(name.ljust(10) + cells + str(s["triples_withheld"])) PY diff --git a/test/test_policy_unit.py b/test/test_policy_unit.py deleted file mode 100644 index d0d99ad..0000000 --- a/test/test_policy_unit.py +++ /dev/null @@ -1,386 +0,0 @@ -"""The v0.1.0 policy demonstrator: profile, decisions, release views and their check. - -The decision tests pin docs/policy-demonstrator.md §4.1 cell by cell. The -release tests run the real committed fixture graphs in both sample profiles. -The mutation tests break a correct view in each way a redactor can go wrong -and require `check` to say so -- a check that has never caught a planted -leak is not evidence of anything. -""" - -import contextlib -import filecmp -import importlib.util -import io -import json -from pathlib import Path -import shutil -import tempfile -import unittest - -try: - import rdflib -except ModuleNotFoundError: # pragma: no cover - exercised only without rdflib - rdflib = None - -from test.helpers import VerboseTestCase - -ROOT = Path(__file__).resolve().parent.parent -EXAMPLE = ROOT / "examples" / "policy" -POLICY = EXAMPLE / "policy.ttl" -VCFS = sorted(EXAMPLE.glob("P00*.vcf")) -FIXTURE = json.loads((EXAMPLE / "fixture.json").read_text(encoding="utf-8")) -DUO = "http://purl.obolibrary.org/obo/DUO_" - -PREFIXES = """@prefix odrl: . -@prefix vcfp: . -@prefix obo: . -@prefix ex: . -""" - - -def request(key): - from vcf_rdfizer_policies.decide import Request - - spec = FIXTURE["requesters"][key] - return Request(spec["assignee"], DUO + spec["purpose"].split("_")[1]) - - -def write_policy(directory, body, conflict="odrl:prohibit"): - """A one-policy file around `body` (the policy's rules), for rejection tests.""" - path = Path(directory) / "policy.ttl" - path.write_text(f"{PREFIXES}\nex:p a odrl:Set ; odrl:conflict {conflict} ;\n{body} .\n", encoding="utf-8") - return path - - -FILE_PERMISSION = """odrl:permission [ odrl:target ; odrl:action odrl:read ]""" - - -@unittest.skipIf(rdflib is None, "rdflib is required") -class PurposeTests(VerboseTestCase): - def test_every_accepted_spelling_names_the_same_term(self): - from vcf_rdfizer_policies.purposes import purpose_iri - - for spelling in ("DUO:0000007", "DUO_0000007", "obo:DUO_0000007", DUO + "0000007"): - self.assertEqual(purpose_iri(spelling), DUO + "0000007") - - def test_a_term_outside_the_bundled_subset_is_refused(self): - from vcf_rdfizer_policies import PolicyError - from vcf_rdfizer_policies.purposes import purpose_iri - - with self.assertRaises(PolicyError): - purpose_iri("DUO:0000019") - with self.assertRaises(PolicyError): - purpose_iri("GRU") - - def test_narrower_purposes_fall_within_broader_consents_and_not_the_reverse(self): - from vcf_rdfizer_policies.purposes import within - - gru, hmb, ds, cc = (DUO + n for n in ("0000042", "0000006", "0000007", "0000043")) - self.assertTrue(within(ds, gru) and within(ds, hmb) and within(hmb, gru)) - self.assertFalse(within(gru, hmb) or within(hmb, ds)) - self.assertFalse(within(cc, gru) or within(gru, cc)) - - -@unittest.skipIf(rdflib is None, "rdflib is required") -class ProfileTests(VerboseTestCase): - def test_the_example_policy_loads_as_eight_rules(self): - from vcf_rdfizer_policies.profile import FileTarget, RegionTarget, VariantTarget, load_policy - - _, rules = load_policy(POLICY) - kinds = sorted((r.kind, type(r.target).__name__) for r in rules) - self.assertEqual(kinds.count(("permission", "FileTarget")), 5) - self.assertEqual(kinds.count(("prohibition", "FileTarget")), 1) # P004's withdrawal - self.assertIn(("prohibition", "RegionTarget"), kinds) - self.assertIn(("prohibition", "VariantTarget"), kinds) - region = next(r.target for r in rules if isinstance(r.target, RegionTarget)) - self.assertEqual((region.chrom, region.start, region.end), ("chr17", 43044295, 43125483)) - self.assertTrue(all(isinstance(r.target, (FileTarget, RegionTarget, VariantTarget)) for r in rules)) - - def test_anything_outside_the_v010_subset_is_refused(self): - from vcf_rdfizer_policies import PolicyError - from vcf_rdfizer_policies.profile import load_policy - - region = ('ex:a a vcfp:GraphSelection ; vcfp:selector [ a vcfp:RegionSelector ; ' - 'vcfp:chrom "chr1" ; vcfp:start 1 ; vcfp:end 2 {extra} ] .\n') - cases = { - "deny-wins is required": (FILE_PERMISSION, "odrl:perm"), - "only read": (FILE_PERMISSION.replace("odrl:read", "odrl:distribute"), "odrl:prohibit"), - "unknown rule property": (FILE_PERMISSION[:-1] + " ; odrl:refinement [] ]", "odrl:prohibit"), - "unsupported effect": (FILE_PERMISSION[:-1] + " ; odrl:duty [ odrl:action odrl:anonymize ; " - "vcfp:transform vcfp:generalize ] ]", "odrl:prohibit"), - "non-DUO purpose": (FILE_PERMISSION[:-1] + " ; odrl:constraint [ odrl:leftOperand odrl:purpose ; " - "odrl:operator odrl:isAnyOf ; odrl:rightOperand ex:x ] ]", "odrl:prohibit"), - "target is not a file": (FILE_PERMISSION.replace("", "ex:thing"), "odrl:prohibit"), - } - for name, (body, conflict) in cases.items(): - with self.subTest(name), tempfile.TemporaryDirectory() as td: - with self.assertRaises(PolicyError): - load_policy(write_policy(td, body, conflict)) - - selector_cases = { - "missing assembly": region.format(extra=""), - "start after end": region.format(extra='; vcfp:assembly "GRCh38"').replace("vcfp:end 2", "vcfp:end 0"), - "unsupported selector": region.format(extra='; vcfp:assembly "GRCh38"').replace( - "RegionSelector", "SampleSelector"), - } - for name, asset in selector_cases.items(): - with self.subTest(name), tempfile.TemporaryDirectory() as td: - path = write_policy(td, "odrl:prohibition [ odrl:target ex:a ; odrl:action odrl:read ]") - path.write_text(path.read_text() + asset, encoding="utf-8") - with self.assertRaises(PolicyError): - load_policy(path) - - -@unittest.skipIf(rdflib is None, "rdflib is required") -class DecisionTests(VerboseTestCase): - """The §4.1 grid, and the boundary cases the fixture plants.""" - - @classmethod - def setUpClass(cls): - from vcf_rdfizer_policies.profile import load_policy - - _, cls.rules = load_policy(POLICY) - - def decide(self, subject, key): - from vcf_rdfizer_policies.decide import decide - - return decide(subject, self.rules, request(key)).released - - def record(self, file="P001", chrom="chr1", pos=1, ref="A", alt="C"): - from vcf_rdfizer_policies.graphs import Record - - return Record(f"file://{file}.vcf", 1, chrom, pos, ref, (alt,)) - - def test_files_follow_each_participants_consent(self): - grid = {"gru": "✓✓———", "alz": "✓✓✓—✓", "clinical": "✓✓———"} - for key, expected in grid.items(): - got = "".join("✓" if self.decide(f"file://P00{n}.vcf", key) else "—" for n in range(1, 6)) - self.assertEqual(got, expected, key) - - def test_the_region_and_variant_rules_each_exempt_one_purpose(self): - brca1 = self.record(chrom="chr17", pos=43100000) - e4 = self.record(chrom="chr19", pos=44908684, ref="T", alt="C") - self.assertEqual([self.decide(brca1, k) for k in ("gru", "alz", "clinical")], [False, False, True]) - self.assertEqual([self.decide(e4, k) for k in ("gru", "alz", "clinical")], [False, True, False]) - - def test_region_bounds_are_inclusive(self): - at = lambda pos: self.decide(self.record(chrom="chr17", pos=pos), "gru") # noqa: E731 - self.assertEqual([at(43044294), at(43044295), at(43125483), at(43125484)], - [True, False, False, True]) - - def test_the_variant_rule_matches_alleles_not_just_position(self): - decoy = self.record(chrom="chr19", pos=44908684, ref="T", alt="G") - self.assertTrue(self.decide(decoy, "gru")) - - def test_withdrawal_overrides_every_permission(self): - for key in ("gru", "alz", "clinical"): - self.assertFalse(self.decide(self.record(file="P004"), key), key) - - def test_a_rule_for_one_party_does_not_bind_another(self): - from dataclasses import replace - from vcf_rdfizer_policies.decide import applies - - withdrawal = next(r for r in self.rules if r.kind == "prohibition" and hasattr(r.target, "iri")) - only_alz = replace(withdrawal, assignee=FIXTURE["requesters"]["alz"]["assignee"]) - self.assertTrue(applies(only_alz, request("alz"))) - self.assertFalse(applies(only_alz, request("gru"))) - - def test_an_assembly_mismatch_stops_evaluation(self): - from vcf_rdfizer_policies import PolicyError - from vcf_rdfizer_policies.decide import check_assemblies - - check_assemblies(self.rules, {"file://P001.vcf": "GRCh38"}) - with self.assertRaises(PolicyError): - check_assemblies(self.rules, {"file://P001.vcf": "GRCh37"}) - - -class FixtureViews(VerboseTestCase): - """Shared setup: every requester's view of both fixture profiles. No tests of its own.""" - - @classmethod - def setUpClass(cls): - from vcf_rdfizer_policies.graphs import load - from vcf_rdfizer_policies.profile import load_policy, policy_digest - from vcf_rdfizer_policies.release import evaluate, write_release - - cls.policy_graph, cls.rules = load_policy(POLICY) - cls.tmp = tempfile.TemporaryDirectory() - cls.graphs, cls.views = {}, {} - for profile in ("expanded", "condensed"): - graph = load(sorted((EXAMPLE / "converted" / profile).glob("*.nt.gz"))) - cls.graphs[profile] = graph - for key in FIXTURE["requesters"]: - release = evaluate(graph, cls.rules, request(key)) - out = Path(cls.tmp.name) / profile / key - write_release(release, out, policies={r.policy for r in cls.rules}, digest=policy_digest(POLICY)) - cls.views[profile, key] = (release, out) - - @classmethod - def tearDownClass(cls): - cls.tmp.cleanup() - - -@unittest.skipIf(rdflib is None, "rdflib is required") -class ReleaseTests(FixtureViews): - """Evaluate the committed fixture graphs, in both profiles, and check every view.""" - - def test_every_view_passes_its_check(self): - from vcf_rdfizer_policies.check import check_view - - for (profile, key), (_, out) in self.views.items(): - with self.subTest(profile=profile, requester=key): - self.assertEqual(check_view(out, POLICY, self.rules, VCFS), []) - - def test_a_view_partitions_the_graph(self): - for (profile, key), (release, _) in self.views.items(): - self.assertEqual(len(release.view) + release.triples_withheld, len(self.graphs[profile])) - - def test_the_withdrawn_file_leaves_nothing_behind(self): - for (_, key), (release, _) in self.views.items(): - self.assertFalse([t for t in release.view if "P004.vcf" in str(t[0]) + str(t[2])], key) - - def test_the_manifest_records_the_request_and_what_was_withheld(self): - release, out = self.views["expanded", "alz"] - manifest = (out / "manifest.ttl").read_text(encoding="utf-8") - for expected in ("governed release; not anonymization", "DUO_0000007", "odrl:attribute", - f"vcfp:recordsWithheld {sum(not d.released for _, d in release.decisions)}"): - self.assertIn(expected, manifest) - files = json.loads((out / "summary.json").read_text())["files"] - self.assertEqual(files["file://P004.vcf"]["released"], False) # the decision, not a count - self.assertEqual(files["file://P001.vcf"]["released"], True) - self.assertEqual(sum(f["records_released"] for f in files.values()), - sum(d.released for _, d in release.decisions)) - - def test_a_release_is_never_written_over_another(self): - from vcf_rdfizer_policies.release import write_release - - release, out = self.views["expanded", "gru"] - with self.assertRaises(FileExistsError): - write_release(release, out, policies=set(), digest="") - - def test_attach_makes_policies_queryable_alongside_the_data(self): - from vcf_rdfizer_policies.release import attach - - graph = rdflib.Graph() - for triple in self.graphs["expanded"]: - graph.add(triple) - counts = attach(graph, self.policy_graph, self.rules) - self.assertEqual(counts["https://example.org/policy/demo-cohort/apoe-e4"], 4) # P001, P003-P005 - # docs/policy-demonstrator.md §5: records a general-research collaborator may not receive. - rows = graph.query(""" - PREFIX odrl: PREFIX vcfc: - SELECT DISTINCT ?chrom WHERE { ?r a vcfc:VCFRecord ; vcfc:chrom ?chrom ; odrl:hasPolicy ?p . - ?p odrl:prohibition ?rule . }""") - self.assertEqual(sorted(str(c) for (c,) in rows), ["chr17", "chr19"]) - - -@unittest.skipIf(rdflib is None, "rdflib is required") -class MutationTests(FixtureViews): - """Break a correct view each way a redactor can fail; `check` must catch every one.""" - - def mutate(self, key, edit, *, policy=POLICY): - """Copy the expanded `key` view, apply `edit` to its view.nt lines, and check it.""" - from vcf_rdfizer_policies.check import check_view - - _, out = self.views["expanded", key] - with tempfile.TemporaryDirectory() as td: - copy = Path(td) / "view" - shutil.copytree(out, copy) - lines = (copy / "view.nt").read_text(encoding="utf-8").splitlines() - (copy / "view.nt").write_text("\n".join(edit(lines)) + "\n", encoding="utf-8") - return "\n".join(check_view(copy, policy, self.rules, VCFS)) - - def full_graph_lines(self, needle): - return [line for line in self.graphs["expanded"].serialize(format="nt").splitlines() if needle in line] - - def test_a_reinstated_withheld_region_record_is_a_leak(self): - brca1 = FIXTURE["loci"]["brca1"] - record = next(r for r, _ in self.views["expanded", "gru"][0].decisions - if r.file.endswith("P001.vcf") and r.chrom == "chr17" and brca1["start"] < r.pos < brca1["end"]) - subject = f"<{record.file}#record/{record.row}>" - report = self.mutate("gru", lambda lines: lines + self.full_graph_lines(subject + " ")) - self.assertIn("leak", report) - self.assertIn("prohibited content present", report) - - def test_a_restored_triple_of_the_withdrawn_file_is_caught(self): - report = self.mutate("alz", lambda lines: lines + self.full_graph_lines(" ")[:1]) - self.assertIn("P004.vcf", report) - - def test_a_deleted_released_record_is_over_withholding(self): - report = self.mutate("alz", lambda lines: [l for l in lines if not l.startswith("")]) - self.assertIn("over-withheld: file://P003.vcf#record/1", report) - - def test_a_reference_to_a_missing_resource_is_dangling(self): - report = self.mutate("alz", lambda lines: [l for l in lines if not l.startswith("")]) - self.assertIn("dangling reference", report) - - def test_one_requesters_view_does_not_pass_as_anothers(self): - _, alz = self.views["expanded", "alz"] - alz_lines = (alz / "view.nt").read_text(encoding="utf-8").splitlines() - self.assertIn("leak", self.mutate("gru", lambda _: alz_lines)) - - def test_a_view_is_checked_against_the_policy_it_was_made_under(self): - with tempfile.TemporaryDirectory() as td: - changed = Path(td) / "policy.ttl" - changed.write_text(POLICY.read_text(encoding="utf-8") + "\n# edited\n", encoding="utf-8") - self.assertIn("different policy", self.mutate("gru", lambda lines: lines, policy=changed)) - - -@unittest.skipIf(rdflib is None, "rdflib is required") -class CommandTests(VerboseTestCase): - def run_cli(self, *argv): - from vcf_rdfizer_policy import main - - stdout, stderr = io.StringIO(), io.StringIO() - with contextlib.redirect_stdout(stdout), contextlib.redirect_stderr(stderr): - code = main([str(a) for a in argv]) - return code, stdout.getvalue() + stderr.getvalue() - - def test_evaluate_then_check_round_trip(self): - rdf = sorted((EXAMPLE / "converted" / "condensed").glob("*.nt.gz")) - spec = FIXTURE["requesters"]["clinical"] - with tempfile.TemporaryDirectory() as td: - out = Path(td) / "clinical" - code, text = self.run_cli("evaluate", "--rdf", *rdf, "--policy", POLICY, - "--assignee", spec["assignee"], "--purpose", "DUO:0000043", "-o", out) - self.assertEqual(code, 0, text) - code, text = self.run_cli("check", "--view", out, "--policy", POLICY, "--vcf", *VCFS) - self.assertEqual((code, text.strip().splitlines()[-1]), (0, "PASS")) - code, text = self.run_cli("evaluate", "--rdf", *rdf, "--policy", POLICY, - "--assignee", spec["assignee"], "--purpose", "DUO:0000043", "-o", out) - self.assertEqual(code, 2) # never overwrites - - def test_an_unsupported_policy_exits_2_with_the_reason(self): - with tempfile.TemporaryDirectory() as td: - path = write_policy(td, FILE_PERMISSION, conflict="odrl:perm") - code, text = self.run_cli("explain", "--policy", path) - self.assertEqual(code, 2) - self.assertIn("deny wins", text) - - def test_explain_and_attach(self): - code, text = self.run_cli("explain", "--policy", POLICY) - self.assertEqual(code, 0) - self.assertIn("prohibition on ", text) - with tempfile.TemporaryDirectory() as td: - out = Path(td) / "annotated.nt" - code, _ = self.run_cli("attach", "--rdf", EXAMPLE / "converted" / "expanded" / "P001.nt.gz", - "--policy", POLICY, "-o", out) - self.assertEqual(code, 0) - self.assertIn("http://www.w3.org/ns/odrl/2/hasPolicy", out.read_text(encoding="utf-8")) - - -class FixtureTests(VerboseTestCase): - def test_the_committed_fixture_is_what_the_generator_writes(self): - spec = importlib.util.spec_from_file_location("make_fixture", EXAMPLE / "make_fixture.py") - module = importlib.util.module_from_spec(spec) - spec.loader.exec_module(module) - with tempfile.TemporaryDirectory() as td, contextlib.redirect_stdout(io.StringIO()): - module.main(Path(td)) - names = [p.name for p in VCFS] + ["fixture.json"] - match, mismatch, errors = filecmp.cmpfiles(EXAMPLE, td, names, shallow=False) - self.assertEqual((mismatch, errors), ([], [])) - self.assertEqual(len(match), 6) - - -if __name__ == "__main__": - unittest.main() diff --git a/vcf_rdfizer_data/policy/duo-subset.ttl b/vcf_rdfizer_data/policy/duo-subset.ttl index 371b7e9..24051c9 100644 --- a/vcf_rdfizer_data/policy/duo-subset.ttl +++ b/vcf_rdfizer_data/policy/duo-subset.ttl @@ -11,6 +11,8 @@ # v0.1.0 treats it as a purpose like the others; it matches only itself. @prefix obo: . +# So DUO:0000007 resolves as a prefixed name, with no special-case code. +@prefix DUO: . @prefix rdfs: . @prefix owl: . diff --git a/vcf_rdfizer_data/policy/vcf-core-profile.ttl b/vcf_rdfizer_data/policy/vcf-core-profile.ttl new file mode 100644 index 0000000..096bac9 --- /dev/null +++ b/vcf_rdfizer_data/policy/vcf-core-profile.ttl @@ -0,0 +1,63 @@ +# The VCF Core profile for vcf-rdfizer-policy: everything the engine needs to know +# about graphs written by VCF-RDFizer, declared here rather than coded. +# +# * vcfp:SelectorType -- a kind of graph selection. Its vcfp:query is a SPARQL +# SELECT that must project ?resource. Each vcfp:parameter is a property the +# policy's selector node must carry; its value is bound to the query variable +# named after the property's local name (vcfp:start -> ?start). An optional +# vcfp:violations query lists reasons the selector cannot be applied to this +# graph; any row stops evaluation. +# * vcfp:Profile -- how the graph is partitioned. A withheld resource takes with +# it everything it owns: the resources reached by vcfp:ownershipPath (a SPARQL +# property path), and, when vcfp:iriSubtree is true, every IRI beneath any of +# them (…#record/9 owns …#record/9/allele/0). vcfp:unitQuery names what the +# per-record report counts: ?resource and ?group are required, other +# variables become report columns. +# +# New selectors need no code: declare another vcfp:SelectorType here, in a +# file passed with --profile, or in the policy file itself. + +@prefix vcfp: . +@prefix rdfs: . + +vcfp:VCFCore a vcfp:Profile ; + rdfs:label "VCF Core graphs as VCF-RDFizer writes them" ; + # A record owns its call, and through the call its per-sample calls + # (expanded profile); the condensed matrix already sits under the call's IRI. + vcfp:ownershipPath "/?" ; + vcfp:iriSubtree true ; + vcfp:unitQuery """ + PREFIX vcfc: + SELECT ?resource ?group ?chrom ?pos ?ref (GROUP_CONCAT(?a; separator=",") AS ?alt) + WHERE { ?group vcfc:hasRecord ?resource . + ?resource vcfc:chrom ?chrom ; vcfc:pos ?pos ; vcfc:ref ?ref . + OPTIONAL { ?resource vcfc:alt ?a } } + GROUP BY ?resource ?group ?chrom ?pos ?ref""" . + +vcfp:RegionSelector a vcfp:SelectorType ; + rdfs:comment "Records whose POS lies in [start, end] on chrom, 1-based and inclusive." ; + vcfp:parameter vcfp:assembly , vcfp:chrom , vcfp:start , vcfp:end ; + vcfp:query """ + PREFIX vcfc: + SELECT ?resource WHERE { + ?resource a vcfc:VCFRecord ; vcfc:chrom ?c ; vcfc:pos ?pos . + FILTER(STR(?c) = STR(?chrom) && ?pos >= ?start && ?pos <= ?end) }""" ; + vcfp:violations """ + PREFIX vcfc: + SELECT ?file ?declared WHERE { + ?file a vcfc:VCFFile . OPTIONAL { ?file vcfc:referenceGenome ?declared } + FILTER(!BOUND(?declared) || STR(?declared) != STR(?assembly)) }""" . + +vcfp:VariantSelector a vcfp:SelectorType ; + rdfs:comment "Records with exactly this chrom, pos, ref and one ALT equal to alt." ; + vcfp:parameter vcfp:assembly , vcfp:chrom , vcfp:pos , vcfp:ref , vcfp:alt ; + vcfp:query """ + PREFIX vcfc: + SELECT ?resource WHERE { + ?resource a vcfc:VCFRecord ; vcfc:chrom ?c ; vcfc:pos ?p ; vcfc:ref ?r ; vcfc:alt ?a . + FILTER(STR(?c) = STR(?chrom) && ?p = ?pos && STR(?r) = STR(?ref) && STR(?a) = STR(?alt)) }""" ; + vcfp:violations """ + PREFIX vcfc: + SELECT ?file ?declared WHERE { + ?file a vcfc:VCFFile . OPTIONAL { ?file vcfc:referenceGenome ?declared } + FILTER(!BOUND(?declared) || STR(?declared) != STR(?assembly)) }""" . diff --git a/vcf_rdfizer_data/policy/vcfp-0.1.ttl b/vcf_rdfizer_data/policy/vcfp-0.1.ttl index 3e38f11..4969f8f 100644 --- a/vcf_rdfizer_data/policy/vcfp-0.1.ttl +++ b/vcf_rdfizer_data/policy/vcfp-0.1.ttl @@ -1,34 +1,41 @@ -# The VCF-RDFizer policy profile terms that v0.1.0 defines. The full profile is -# specified in docs/privacy-policy-design.md; docs/policy-demonstrator.md lists -# which parts v0.1.0 implements. Anything a policy uses beyond these terms is -# rejected by the evaluator rather than ignored. +# Terms of the VCF-RDFizer policy profile that v0.1.0 defines. The engine reads +# ODRL policies; these terms describe the three things it is configured with: +# selections (what a rule targets), profiles (how a graph is partitioned), and +# release views (what it writes). docs/policy-demonstrator.md explains each. @prefix vcfp: . @prefix odrl: . -@prefix rdfs: . -@prefix xsd: . @prefix rdf: . +@prefix rdfs: . a odrl:Profile ; - rdfs:label "VCF-RDFizer policy profile, v0.1.0 subset" . + rdfs:label "VCF-RDFizer policy profile, v0.1.0" . +# --- Selections -------------------------------------------------------------- vcfp:GraphSelection a rdfs:Class ; rdfs:subClassOf odrl:Asset ; - rdfs:comment "An asset that is a selection of a VCF graph, described by exactly one vcfp:selector." . -vcfp:selector a rdf:Property ; rdfs:domain vcfp:GraphSelection . + rdfs:comment "An asset that is a selection of a graph, described by exactly one vcfp:selector." . +vcfp:selector a rdf:Property ; rdfs:domain vcfp:GraphSelection ; + rdfs:comment "The selector node: typed with a vcfp:SelectorType, carrying its parameters." . vcfp:selects a rdf:Property ; rdfs:domain vcfp:GraphSelection ; - rdfs:comment "Written by `vcf-rdfizer-policy attach`: a record the selection selects." . + rdfs:comment "Written by `attach`: a resource the selection selects." . +vcfp:SelectorType a rdfs:Class ; + rdfs:comment "A kind of selection, declared by a SPARQL SELECT that projects ?resource." . +vcfp:query a rdf:Property ; rdfs:domain vcfp:SelectorType . +vcfp:parameter a rdf:Property ; rdfs:domain vcfp:SelectorType ; + rdfs:comment "A required property of the selector node, bound to the query variable named after its local name." . +vcfp:violations a rdf:Property ; rdfs:domain vcfp:SelectorType ; + rdfs:comment "A SPARQL SELECT whose rows are reasons the selector cannot be applied; any row stops evaluation." . -vcfp:RegionSelector a rdfs:Class ; - rdfs:comment "Records whose POS lies in [vcfp:start, vcfp:end] on vcfp:chrom, 1-based and inclusive." . -vcfp:VariantSelector a rdfs:Class ; - rdfs:comment "Records with exactly this chrom, pos, ref and alt." . -vcfp:assembly a rdf:Property ; rdfs:comment "Required. Must equal the file's vcfc:referenceGenome." . -vcfp:chrom a rdf:Property . -vcfp:start a rdf:Property ; rdfs:range xsd:integer . -vcfp:end a rdf:Property ; rdfs:range xsd:integer . -vcfp:pos a rdf:Property ; rdfs:range xsd:integer . -vcfp:ref a rdf:Property . -vcfp:alt a rdf:Property . +# --- Partitioning -------------------------------------------------------------- +vcfp:Profile a rdfs:Class ; + rdfs:comment "How a graph is partitioned, and what the per-unit report counts." . +vcfp:ownershipPath a rdf:Property ; rdfs:domain vcfp:Profile ; + rdfs:comment "A SPARQL property path: a withheld resource also withholds what it reaches." . +vcfp:iriSubtree a rdf:Property ; rdfs:domain vcfp:Profile ; + rdfs:comment "When true, a withheld resource also withholds every IRI beneath it (after '#' or '/')." . +vcfp:unitQuery a rdf:Property ; rdfs:domain vcfp:Profile ; + rdfs:comment "A SPARQL SELECT of the reported units: ?resource and ?group required." . +# --- Release views --------------------------------------------------------------- vcfp:ReleaseView a rdfs:Class ; - rdfs:comment "A graph produced from source files under a policy for one request. Governed release, not anonymization." . + rdfs:comment "A graph produced under a policy for one request. Governed release, not anonymization." . diff --git a/vcf_rdfizer_policies/__init__.py b/vcf_rdfizer_policies/__init__.py index 78c2db8..855dc0f 100644 --- a/vcf_rdfizer_policies/__init__.py +++ b/vcf_rdfizer_policies/__init__.py @@ -1,11 +1,16 @@ -"""Policy demonstrator v0.1.0: ODRL policies attached to VCF-RDFizer graphs. +"""Policy demonstrator v0.1.0: attach ODRL policies to RDF graphs and release governed views. -Policies attach to files, genomic regions and single variants. One policy set -yields a different release view per request (who asks, and for what purpose), -and each view can be checked against an oracle computed from the source VCFs. +Three generic steps, each configured in Turtle rather than code: -This is governed release, not anonymization. The specification, including what -v0.1.0 deliberately leaves out, is docs/policy-demonstrator.md. + select a rule's target is an IRI, or a selection computed by a declared + SPARQL selector type (profile.py) + partition the profile's ownership rule extends each selection to what it owns + decide ODRL: a binding permission must own a resource, and no binding + prohibition may; deny wins (engine.py) + +The bundled VCF Core profile makes it work on VCF-RDFizer graphs; vcf_oracle.py +adds an independent check against the source VCFs. This is governed release, not +anonymization. The specification is docs/policy-demonstrator.md. """ VERSION = "0.1.0" diff --git a/vcf_rdfizer_policies/check.py b/vcf_rdfizer_policies/check.py index 25ed4ad..8aaa750 100644 --- a/vcf_rdfizer_policies/check.py +++ b/vcf_rdfizer_policies/check.py @@ -1,114 +1,66 @@ -"""Verify a release view against an oracle computed from the source VCFs. - -The oracle reads records straight from the VCF text -- no graph, no SPARQL -- -and decides them with decide.py. A view passes only if its records equal the -oracle's released set exactly (an extra record is a leak; a missing one is -over-withholding) and it survives three structural checks: no triple inside -a prohibited target, no reference to anything withheld, and no mention of a -withheld file. Each failure is returned as one human-readable line. -""" - -import gzip -from pathlib import Path -import re - -from . import ODRL, VCFC, VCFP -from .decide import Request, applies, check_assemblies, decide -from .graphs import Record, records, split -from .profile import FileTarget, RegionTarget, policy_digest +"""Verify a release view: structural checks for any graph, plus an optional oracle. +Given the view, the policy, and the source graph the view was made from: -def read_vcf(path: Path): - """(file IRI, assembly, [Record]) from a VCF, numbering rows as the converter does.""" - path = Path(path) - opener = gzip.open if path.name.endswith(".gz") else open - name = re.sub(r"\.gz$", "", path.name) - file_iri, assembly, found = f"file://{name}", None, [] - with opener(path, "rt", encoding="utf-8") as handle: - for line in handle: - if line.startswith("##reference="): - assembly = line.split("=", 1)[1].strip() - if line.startswith("#"): - continue - chrom, pos, _, ref, alt = line.rstrip("\n").split("\t")[:5] - found.append(Record(file_iri, len(found) + 1, chrom, int(pos), ref, - tuple(sorted(a for a in alt.split(",") if a != ".")))) - return file_iri, assembly, found +1. the view was produced under this policy (its manifest's digest matches); +2. nothing a binding prohibition owns appears in the view, as subject or object; +3. every subject in the view is owned by a binding permission (default-deny); +4. no triple points at a node of the source that the view does not contain. +Checks 2-4 reuse the engine's selectors and partition, so they confirm the view +honours the policy; they cannot catch a mistake in a selector itself. That is +what an oracle is for: `vcf_oracle.compare` re-derives the expected records +from the VCF text, an input the graph and its conversion never touched. +Each failure is returned as one human-readable line. +""" -def oracle(vcf_paths, rules, request): - """The (file, row) pairs a correct view releases, and the files it withholds.""" - parsed = [read_vcf(p) for p in vcf_paths] - check_assemblies(rules, {f: a for f, a, _ in parsed}) - released, withheld_files = set(), set() - for file_iri, _, file_records in parsed: - if not decide(file_iri, rules, request).released: - withheld_files.add(file_iri) - continue - released |= {(r.file, r.row) for r in file_records if decide(r, rules, request).released} - return released, withheld_files +from pathlib import Path +from . import ODRL, VCFP +from .engine import Partition, Request, applies, select +from .policy import policy_digest -def _only(graph, predicate): - """The object of the one triple with this predicate (Graph.value needs a subject).""" - import rdflib +#: Enough to diagnose; a broken view can otherwise fail thousands of times. +LIMIT = 20 - return next(graph.objects(None, rdflib.URIRef(predicate)), None) - -def read_request(manifest_graph) -> Request: +def read_view(view_dir: Path): + """(view graph, manifest graph, request) from a directory written by evaluate.""" import rdflib - request = _only(manifest_graph, VCFP + "request") - return Request(str(manifest_graph.value(request, rdflib.URIRef(ODRL + "assignee"))), - str(manifest_graph.value(request, rdflib.URIRef(ODRL + "purpose")))) + view_dir = Path(view_dir) + manifest = rdflib.Graph().parse(str(view_dir / "manifest.ttl"), format="turtle") + request_node = next(manifest.objects(None, rdflib.URIRef(VCFP + "request"))) + request = Request(str(manifest.value(request_node, rdflib.URIRef(ODRL + "assignee"))), + str(manifest.value(request_node, rdflib.URIRef(ODRL + "purpose")))) + return rdflib.Graph().parse(str(view_dir / "view.nt"), format="nt"), manifest, request -def check_view(view_dir: Path, policy_path: Path, rules, vcf_paths) -> list: - """Every way the view in `view_dir` departs from the policy; empty means it passes.""" +def check_view(view, manifest, request, *, policy_path, rules, profile, vocabulary, source) -> list: + """Every way `view` departs from the policy, given the `source` it was made from.""" import rdflib - view_dir = Path(view_dir) - manifest = rdflib.Graph().parse(str(view_dir / "manifest.ttl"), format="turtle") - view = rdflib.Graph().parse(str(view_dir / "view.nt"), format="nt") - request = read_request(manifest) failures = [] - - recorded = str(_only(manifest, VCFP + "policyDigest")) + recorded = str(next(manifest.objects(None, rdflib.URIRef(VCFP + "policyDigest")), None)) if recorded != policy_digest(policy_path): failures.append(f"view was produced under a different policy ({recorded})") - expected, withheld_files = oracle(vcf_paths, rules, request) - actual = {(r.file, r.row) for r in records(view)} - failures += [f"leak: {f}#record/{row} is released but the policy withholds it" - for f, row in sorted(actual - expected)] - failures += [f"over-withheld: {f}#record/{row} should have been released" - for f, row in sorted(expected - actual)] - - for rule in rules: - if rule.kind == "prohibition" and applies(rule, request) and view.query(_ask(rule.target)).askAnswer: - failures.append(f"prohibited content present: {rule.label}") - - subjects = {s for s in view.subjects() if isinstance(s, rdflib.URIRef)} - for s, p, o in view: - if isinstance(o, rdflib.URIRef) and split(str(o))[0] and o not in subjects: - failures.append(f"dangling reference: <{s}> <{p}> <{o}>") - - names = {f.split("://", 1)[1] for f in withheld_files} - for term in {t for triple in view for t in triple}: - for name in names: - if name in str(term): - failures.append(f"withheld file {name} is named by {term.n3()}") + partition = Partition(source, profile) + binding = [(rule, partition.owned(select(source, rule.target))) + for rule in rules if applies(rule, request, vocabulary)] + terms = {t for triple in view for t in (triple[0], triple[2]) + if isinstance(t, (rdflib.URIRef, rdflib.BNode))} + for rule, owned in binding: + if rule.kind == "prohibition": + present = sorted(str(t) for t in terms if partition.contains(owned, t)) + failures += [f"prohibited content present: {rule.label} owns <{t}>" for t in present[:LIMIT]] + + granted = [owned for rule, owned in binding if rule.kind == "permission"] + ungoverned = sorted(str(s) for s in set(view.subjects()) + if not any(partition.contains(owned, s) for owned in granted)) + failures += [f"no permission covers <{s}>" for s in ungoverned[:LIMIT]] + + nodes, present = set(source.subjects()), set(view.subjects()) + dangling = sorted((str(s), str(o)) for s, _, o in view if o in nodes and o not in present) + failures += [f"dangling reference: <{s}> -> <{o}>" for s, o in dangling[:LIMIT]] return failures - - -def _ask(target) -> str: - """A SPARQL ASK that is true when anything the target selects is in the graph.""" - if isinstance(target, FileTarget): - return f'ASK {{ ?s ?p ?o FILTER(STRSTARTS(STR(?s), "{target.iri}#") || STR(?s) = "{target.iri}") }}' - if isinstance(target, RegionTarget): - where = f"FILTER(?pos >= {target.start} && ?pos <= {target.end})" - else: - where = f'FILTER(?pos = {target.pos}) ?r vcfc:ref "{target.ref}" ; vcfc:alt "{target.alt}" .' - return (f'PREFIX vcfc: <{VCFC}> ASK {{ ?r a vcfc:VCFRecord ; vcfc:chrom "{target.chrom}" ; ' - f"vcfc:pos ?pos . {where} }}") diff --git a/vcf_rdfizer_policies/decide.py b/vcf_rdfizer_policies/decide.py deleted file mode 100644 index 623a803..0000000 --- a/vcf_rdfizer_policies/decide.py +++ /dev/null @@ -1,73 +0,0 @@ -"""The v0.1.0 decision rules (docs/policy-demonstrator.md §4) -- the one place they live. - -Both `evaluate` and the oracle in `check` call these functions, so the rules -cannot drift apart. The oracle's independence comes from its input (the VCF -text, not the graph), not from a second copy of the rules. -""" - -from dataclasses import dataclass - -from . import PolicyError -from .profile import FileTarget, RegionTarget, VariantTarget -from .purposes import within - - -@dataclass(frozen=True) -class Request: - assignee: str - purpose: str # a DUO IRI - - -@dataclass(frozen=True) -class Decision: - released: bool - reason: str - - -def applies(rule, request) -> bool: - """Does the rule bind this request: assignee matches, and every constraint holds?""" - if rule.assignee is not None and rule.assignee != request.assignee: - return False - for constraint in rule.constraints: - inside = any(within(request.purpose, term) for term in constraint.purposes) - if inside != (constraint.operator == "isAnyOf"): - return False - return True - - -def covers(target, subject) -> bool: - """Does the target select this subject (a Record, or a file IRI)?""" - file_iri = subject if isinstance(subject, str) else subject.file - if isinstance(target, FileTarget): - return target.iri == file_iri - if isinstance(subject, str): - return False # region and variant targets select records, never whole files - if isinstance(target, RegionTarget): - return subject.chrom == target.chrom and target.start <= subject.pos <= target.end - if isinstance(target, VariantTarget): - return (subject.chrom, subject.pos, subject.ref) == (target.chrom, target.pos, target.ref) \ - and target.alt in subject.alts - raise PolicyError(f"unknown target {target!r}") - - -def check_assemblies(rules, file_assemblies: dict) -> None: - """Every region or variant rule must name the assembly every file declares.""" - for rule in rules: - wanted = getattr(rule.target, "assembly", None) - for file_iri, declared in file_assemblies.items(): - if wanted is not None and declared != wanted: - raise PolicyError(f"{rule.label} is for {wanted}, but <{file_iri}> declares " - f"{declared or 'no reference genome'}") - - -def decide(subject, rules, request) -> Decision: - """Release iff a binding permission covers the file and no binding prohibition covers the subject.""" - binding = [rule for rule in rules if applies(rule, request)] - for rule in binding: - if rule.kind == "prohibition" and covers(rule.target, subject): - return Decision(False, f"withheld: {rule.label}") - file_iri = subject if isinstance(subject, str) else subject.file - for rule in binding: - if rule.kind == "permission" and covers(rule.target, file_iri): - return Decision(True, f"released: {rule.label}") - return Decision(False, f"withheld: no permission covers <{file_iri}> for this purpose") diff --git a/vcf_rdfizer_policies/engine.py b/vcf_rdfizer_policies/engine.py new file mode 100644 index 0000000..1a73721 --- /dev/null +++ b/vcf_rdfizer_policies/engine.py @@ -0,0 +1,136 @@ +"""The engine: select, partition, decide (docs/policy-demonstrator.md §4). + +Nothing here knows about VCF. A rule's target selects resources (a declared +SPARQL selector, or one IRI); the profile's ownership rule extends each +selection to everything it owns; and a resource is released when some binding +permission owns it and no binding prohibition does. +""" + +from dataclasses import dataclass + +from . import PolicyError +from .graphs import ancestors +from .policy import Direct + + +@dataclass(frozen=True) +class Request: + assignee: str + purpose: str # a term IRI of the purpose vocabulary + + +def applies(rule, request, vocabulary) -> bool: + """Does the rule bind this request: assignee matches, and every constraint holds?""" + if rule.assignee is not None and rule.assignee != request.assignee: + return False + for constraint in rule.constraints: + inside = any(vocabulary.within(request.purpose, term) for term in constraint.purposes) + if inside != (constraint.operator == "isAnyOf"): + return False + return True + + +def select(graph, target) -> set: + """The resources a target selects in `graph`.""" + import rdflib + + if isinstance(target, Direct): + return {rdflib.URIRef(target.iri)} + rows = graph.query(target.selector.query, initBindings=dict(target.bindings)) + return {row.resource for row in rows} + + +def check_preconditions(graph, rules) -> None: + """Run each selection's declared violations query; any row stops evaluation.""" + for rule in rules: + selector = getattr(rule.target, "selector", None) + if selector is None or selector.violations is None: + continue + rows = list(graph.query(selector.violations, initBindings=dict(rule.target.bindings))) + if rows: + shown = "; ".join(" ".join(str(v) for v in row if v is not None) for row in rows[:3]) + raise PolicyError(f"{rule.label} cannot be applied to this graph: {shown}") + + +class Partition: + """The profile's ownership rule, applied to one graph.""" + + def __init__(self, graph, profile): + self.graph, self.profile = graph, profile + + def owned(self, roots) -> frozenset: + """The roots and everything their ownership path reaches (IRI subtrees are implicit).""" + found = set(roots) + if self.profile.ownership_path: + query = (f"SELECT DISTINCT ?owned WHERE {{ ?root {self.profile.ownership_path} ?owned }}") + for root in roots: + found.update(row.owned for row in self.graph.query(query, initBindings={"root": root})) + return frozenset(found) + + def contains(self, owned, term) -> bool: + """Is `term` in `owned`, or (with iriSubtree) beneath an IRI that is?""" + if term in owned: + return True + if not self.profile.iri_subtree or not hasattr(term, "startswith"): + return False + from rdflib import URIRef + + return any(URIRef(a) in owned for a in ancestors(str(term))) + + +@dataclass +class Evaluation: + """Per-rule ownership, and the release decision for any term.""" + partition: Partition + binding: list # (rule, owned) for every rule that binds the request + + def decide(self, term): + """(released, reason) for one resource; deny wins, and default-deny.""" + for rule, owned in self.binding: + if rule.kind == "prohibition" and self.partition.contains(owned, term): + return False, f"withheld: {rule.label}" + for rule, owned in self.binding: + if rule.kind == "permission" and self.partition.contains(owned, term): + return True, f"released: {rule.label}" + return False, "withheld: no permission covers it for this purpose" + + +def evaluation(graph, rules, request, profile, vocabulary) -> Evaluation: + check_preconditions(graph, rules) + partition = Partition(graph, profile) + binding = [(rule, partition.owned(select(graph, rule.target))) + for rule in rules if applies(rule, request, vocabulary)] + return Evaluation(partition, binding) + + +def view(graph, evaluation) -> tuple: + """(released triples, number withheld). A triple is released when its subject is, + and when its object, if it is a node of the graph, is released too -- so a view + never points at something it does not contain.""" + import rdflib + + nodes = set(graph.subjects()) + cache = {} + + def released(term): + if term not in cache: + cache[term] = evaluation.decide(term)[0] + return cache[term] + + kept, withheld = [], 0 + for s, p, o in graph: + if released(s) and (not isinstance(o, (rdflib.URIRef, rdflib.BNode)) or o not in nodes or released(o)): + kept.append((s, p, o)) + else: + withheld += 1 + return kept, withheld + + +def units(graph, profile) -> list: + """The profile's reporting units: dicts with 'resource', 'group' and any other columns.""" + if profile.unit_query is None: + return [] + rows = graph.query(profile.unit_query) + names = [str(v) for v in rows.vars] + found = [{n: row[n] for n in names} for row in rows] + return sorted(found, key=lambda u: (str(u["group"]), str(u["resource"]))) diff --git a/vcf_rdfizer_policies/graphs.py b/vcf_rdfizer_policies/graphs.py index 188c69b..fdc79b5 100644 --- a/vcf_rdfizer_policies/graphs.py +++ b/vcf_rdfizer_policies/graphs.py @@ -1,86 +1,46 @@ -"""Load converted graphs and read the records the selectors act on. +"""Load RDF inputs, and walk an IRI's hierarchy.""" -A record is identified by its file IRI and row number, which is how the -converter mints its IRIs (docs/conversion.md §6). Everything that belongs to -row N lives under one of three IRI subtrees -- #record/N, #call/N and -#sample/N -- so withholding a record means withholding those subtrees. -""" - -from dataclasses import dataclass import gzip from pathlib import Path -import re -from . import VCFC, PolicyError +from . import PolicyError -#: v0.1.0 evaluates in memory. Above this the demonstrator is the wrong tool. +#: The engine evaluates in memory. Above this, the demonstrator is the wrong tool. MAX_TRIPLES = 5_000_000 -_ROW = re.compile(r"(?:record|call|sample)/(\d+)(?:/.*)?") - -@dataclass(frozen=True) -class Record: - file: str # file IRI, e.g. file://P001.vcf - row: int - chrom: str - pos: int - ref: str - alts: tuple +def _open(path: Path, mode: str): + return gzip.open(path, mode) if path.name.endswith(".gz") else open(path, mode) def load(paths) -> "rdflib.Graph": """Parse .nt / .nt.gz files into one graph, after a size check.""" import rdflib + paths = [Path(p) for p in paths] total = 0 - for path in map(Path, paths): - opener = gzip.open if path.name.endswith(".gz") else open - with opener(path, "rt", encoding="utf-8") as handle: + for path in paths: + with _open(path, "rb") as handle: total += sum(1 for _ in handle) if total > MAX_TRIPLES: - raise PolicyError(f"{total:,} triples exceeds the v0.1.0 in-memory limit of " - f"{MAX_TRIPLES:,}; the demonstrator is for fixtures, not cohorts") + raise PolicyError(f"{total:,} triples exceeds the in-memory limit of {MAX_TRIPLES:,}; " + "the demonstrator is for fixtures, not cohorts") graph = rdflib.Graph() - for path in map(Path, paths): - opener = gzip.open if path.name.endswith(".gz") else open - with opener(path, "rb") as handle: + for path in paths: + with _open(path, "rb") as handle: graph.parse(handle, format="nt") return graph -def records(graph) -> list: - """Every vcfc:VCFRecord in the graph, with the fields selectors use.""" - rows = graph.query(f""" - PREFIX vcfc: <{VCFC}> - SELECT ?record ?chrom ?pos ?ref (GROUP_CONCAT(?alt; separator=",") AS ?alts) - WHERE {{ ?record a vcfc:VCFRecord ; vcfc:chrom ?chrom ; vcfc:pos ?pos ; vcfc:ref ?ref . - OPTIONAL {{ ?record vcfc:alt ?alt }} }} - GROUP BY ?record ?chrom ?pos ?ref""") - found = [] - for record, chrom, pos, ref, alts in rows: - file_iri, row = split(str(record)) - found.append(Record(file_iri, row, str(chrom), int(pos), str(ref), - tuple(sorted(str(alts).split(","))) if alts else ())) - return sorted(found, key=lambda r: (r.file, r.row)) - - -def assemblies(graph) -> dict: - """Each file IRI mapped to its declared vcfc:referenceGenome.""" - rows = graph.query(f"""PREFIX vcfc: <{VCFC}> - SELECT ?file ?assembly WHERE {{ ?file a vcfc:VCFFile . OPTIONAL {{ ?file vcfc:referenceGenome ?assembly }} }}""") - return {str(f): (str(a) if a is not None else None) for f, a in rows} - - -def split(iri: str): - """(file IRI, row or None) for any IRI the converter mints; (None, None) otherwise. +def ancestors(iri: str): + """The IRI itself, then each prefix of it that ends just before a '#' or '/'. - Every IRI under file://X belongs to file X -- header lines and the sample set - included, which is what lets a withheld file take its header with it. Only - the #record/, #call/ and #sample/ subtrees also carry a row. + file://P1.vcf#record/9/allele/0 -> itself, file://P1.vcf#record/9/allele, + file://P1.vcf#record/9, file://P1.vcf#record, file://P1.vcf. The scheme's own + slashes are skipped, so nothing shorter than the authority is yielded. """ - if not iri.startswith("file://"): - return None, None - file_iri, _, fragment = iri.partition("#") - match = _ROW.fullmatch(fragment) - return file_iri, int(match.group(1)) if match else None + yield iri + start = iri.find("://") + 3 if "://" in iri else 0 + for index in range(len(iri) - 1, start, -1): + if iri[index] in "#/": + yield iri[:index] diff --git a/vcf_rdfizer_policies/policy.py b/vcf_rdfizer_policies/policy.py new file mode 100644 index 0000000..35cd2b4 --- /dev/null +++ b/vcf_rdfizer_policies/policy.py @@ -0,0 +1,157 @@ +"""Read an ODRL policy file into rules, refusing anything the engine cannot evaluate. + +Supported (docs/policy-demonstrator.md §3): odrl:permission and odrl:prohibition +with action odrl:read; a target that is either a resource IRI or a +vcfp:GraphSelection whose selector type the profile declares; an assignee; +purpose constraints with odrl:isAnyOf / odrl:isNoneOf; duties, which are +recorded but not enforced; and odrl:conflict odrl:prohibit. Anything else -- +an unknown selector type, a missing parameter, an effect other than drop, an +unrecognised property on a rule -- raises PolicyError. Silently ignoring a rule +would be worse than refusing the policy. +""" + +from dataclasses import dataclass +import hashlib +from pathlib import Path + +from . import ODRL, VCFP, PolicyError +from .profile import variable + +RDF_TYPE = "http://www.w3.org/1999/02/22-rdf-syntax-ns#type" +POLICY_CLASSES = {ODRL + name for name in ("Policy", "Set", "Offer", "Agreement")} +#: Properties a rule may carry. Anything else (odrl:refinement, odrl:remedy, ...) +#: could change what the rule means, so it is refused rather than dropped. +RULE_PROPERTIES = {RDF_TYPE, ODRL + "target", ODRL + "action", ODRL + "assignee", + ODRL + "assigner", ODRL + "constraint", ODRL + "duty"} + + +@dataclass(frozen=True) +class Direct: + """A target that is one resource, e.g. .""" + iri: str + + +@dataclass(frozen=True) +class Selection: + """A target computed by a declared selector type with these parameter bindings.""" + asset: str + selector: object # profile.SelectorType + bindings: tuple # ((variable name, rdflib term), ...) + + +@dataclass(frozen=True) +class Constraint: + operator: str # "isAnyOf" or "isNoneOf" + purposes: frozenset # term IRIs + + +@dataclass(frozen=True) +class Rule: + kind: str # "permission" or "prohibition" + policy: str # the IRI of the policy the rule belongs to + target: object # Direct | Selection + assignee: str = None # None means odrl:All + constraints: tuple = () + duties: tuple = () # ODRL action IRIs; recorded, not enforced + + @property + def label(self) -> str: + """Short, stable name for reports: kind and target.""" + return f"{self.kind} on <{getattr(self.target, 'asset', None) or self.target.iri}>" + + +def policy_digest(path: Path) -> str: + """sha256 of the policy file's bytes, as recorded in every manifest.""" + return "sha256:" + hashlib.sha256(Path(path).read_bytes()).hexdigest() + + +def read_graph(path: Path): + import rdflib + + return rdflib.Graph().parse(str(path), format="turtle") + + +def load_rules(graph, profile, vocabulary) -> list: + """Every rule of every policy in `graph`, validated against the profile and vocabulary.""" + import rdflib + + policies = {s for s, o in graph.subject_objects(rdflib.RDF.type) if str(o) in POLICY_CLASSES} + if not policies: + raise PolicyError("no odrl:Policy, Set, Offer or Agreement found") + rules = [] + for policy in sorted(policies, key=str): + if str(graph.value(policy, rdflib.URIRef(ODRL + "conflict"))) != ODRL + "prohibit": + raise PolicyError(f"<{policy}>: odrl:conflict odrl:prohibit (deny wins) is required") + if graph.value(policy, rdflib.URIRef(ODRL + "obligation")) is not None: + raise PolicyError(f"<{policy}>: odrl:obligation is not supported") + for kind in ("permission", "prohibition"): + for node in graph.objects(policy, rdflib.URIRef(ODRL + kind)): + rules.append(_rule(graph, str(policy), kind, node, profile, vocabulary)) + return rules + + +def _rule(graph, policy, kind, node, profile, vocabulary): + import rdflib + + where = f"{kind} in <{policy}>" + unknown = {str(p) for p in graph.predicates(node)} - RULE_PROPERTIES + if unknown: + raise PolicyError(f"{where} uses unsupported properties: {sorted(unknown)}") + if str(graph.value(node, rdflib.URIRef(ODRL + "action"))) != ODRL + "read": + raise PolicyError(f"{where}: the only supported action is odrl:read") + assignee = graph.value(node, rdflib.URIRef(ODRL + "assignee")) + constraints = tuple(_constraint(graph, c, vocabulary) + for c in graph.objects(node, rdflib.URIRef(ODRL + "constraint"))) + return Rule(kind, policy, _target(graph, graph.value(node, rdflib.URIRef(ODRL + "target")), profile, where), + None if assignee is None or str(assignee) == ODRL + "All" else str(assignee), + constraints, + tuple(sorted(_duty(graph, d) for d in graph.objects(node, rdflib.URIRef(ODRL + "duty"))))) + + +def _target(graph, node, profile, where): + import rdflib + + if node is None: + raise PolicyError(f"{where} has no odrl:target") + selectors = list(graph.objects(node, rdflib.URIRef(VCFP + "selector"))) + if not selectors: + if not isinstance(node, rdflib.URIRef): + raise PolicyError(f"{where}: a target must be an IRI or a vcfp:GraphSelection") + return Direct(str(node)) + if len(selectors) != 1: + raise PolicyError(f"<{node}>: a GraphSelection needs exactly one vcfp:selector") + kind = str(graph.value(selectors[0], rdflib.RDF.type) or "") + if kind not in profile.selectors: + raise PolicyError(f"<{node}>: selector type <{kind or '(none)'}> is not declared by the profile") + selector = profile.selectors[kind] + bindings = [] + for prop in selector.parameters: + value = graph.value(selectors[0], rdflib.URIRef(prop)) + if value is None: + raise PolicyError(f"<{node}>: a {kind.rsplit('#', 1)[-1]} needs <{prop}>") + bindings.append((variable(prop), value)) + return Selection(str(node), selector, tuple(bindings)) + + +def _constraint(graph, node, vocabulary): + import rdflib + + left = str(graph.value(node, rdflib.URIRef(ODRL + "leftOperand"))) + operator = str(graph.value(node, rdflib.URIRef(ODRL + "operator"))) + if left != ODRL + "purpose": + raise PolicyError(f"only odrl:purpose constraints are supported, not <{left}>") + if operator not in (ODRL + "isAnyOf", ODRL + "isNoneOf"): + raise PolicyError(f"only odrl:isAnyOf and odrl:isNoneOf are supported, not <{operator}>") + values = list(graph.objects(node, rdflib.URIRef(ODRL + "rightOperand"))) + if not values: + raise PolicyError("a purpose constraint has no odrl:rightOperand") + return Constraint(operator.rsplit("/", 1)[1], frozenset(vocabulary.resolve(str(v)) for v in values)) + + +def _duty(graph, node): + import rdflib + + transform = graph.value(node, rdflib.URIRef(VCFP + "transform")) + if transform is not None and str(transform) != VCFP + "drop": + raise PolicyError(f"effect <{transform}> is not supported; only vcfp:drop") + return str(graph.value(node, rdflib.URIRef(ODRL + "action"))) diff --git a/vcf_rdfizer_policies/profile.py b/vcf_rdfizer_policies/profile.py index d662e4c..d980ed3 100644 --- a/vcf_rdfizer_policies/profile.py +++ b/vcf_rdfizer_policies/profile.py @@ -1,172 +1,102 @@ -"""Read an ODRL policy file into rules, rejecting anything v0.1.0 cannot evaluate. +"""Profiles: the selector types and the partitioning rule, read from Turtle. -The supported subset is docs/policy-demonstrator.md §3. Everything outside it -- -an unknown selector, an effect other than drop, a conflict strategy other than -deny-wins, an unrecognised property on a rule -- raises PolicyError. Silently -ignoring a rule would be worse than refusing the policy. +A profile is what makes the engine specific to a kind of graph without any +code. It declares the selector types a policy may use -- each a SPARQL SELECT +that projects ?resource -- and how a withheld resource takes others with it. +The bundled VCF Core profile (vcf_rdfizer_data/policy/vcf-core-profile.ttl) +covers graphs written by VCF-RDFizer; its comments document every term. """ -from dataclasses import dataclass -import hashlib +from dataclasses import dataclass, field +from importlib.resources import files from pathlib import Path -from . import ODRL, VCFP, PolicyError -from .purposes import purpose_iri +from . import VCFP, PolicyError -RDF_TYPE = "http://www.w3.org/1999/02/22-rdf-syntax-ns#type" -POLICY_CLASSES = {ODRL + name for name in ("Policy", "Set", "Offer", "Agreement")} -#: Properties a rule may carry. Anything else (odrl:refinement, odrl:remedy, -#: ...) could change what the rule means, so it is refused, not dropped. -RULE_PROPERTIES = {RDF_TYPE, ODRL + "target", ODRL + "action", ODRL + "assignee", - ODRL + "assigner", ODRL + "constraint", ODRL + "duty"} +#: The name `--profile vcf-core` refers to, and the default when none is given. +BUNDLED = {"vcf-core": "vcf-core-profile.ttl"} @dataclass(frozen=True) -class FileTarget: - """A whole converted file, named by its file IRI, e.g. .""" +class SelectorType: iri: str + query: str + parameters: tuple # property IRIs; each binds ? + violations: str = None # optional SELECT; any row means "cannot apply here" @dataclass(frozen=True) -class RegionTarget: - """Records with POS in [start, end] on chrom, 1-based and inclusive.""" - asset: str - assembly: str - chrom: str - start: int - end: int - - -@dataclass(frozen=True) -class VariantTarget: - """Records with exactly this chrom, pos, ref and alt.""" - asset: str - assembly: str - chrom: str - pos: int - ref: str - alt: str - - -@dataclass(frozen=True) -class Constraint: - """A purpose constraint: odrl:isAnyOf or odrl:isNoneOf a set of DUO terms.""" - operator: str - purposes: frozenset - - -@dataclass(frozen=True) -class Rule: - kind: str # "permission" or "prohibition" - policy: str # the policy IRI the rule belongs to - target: object # FileTarget | RegionTarget | VariantTarget - assignee: str = None # None means odrl:All - constraints: tuple = () - duties: tuple = () # ODRL action IRIs; recorded, not enforced - - @property - def label(self) -> str: - """Short, stable name for reports: kind and target.""" - target = getattr(self.target, "asset", None) or self.target.iri - return f"{self.kind} on <{target}>" +class Profile: + iri: str + ownership_path: str = None + iri_subtree: bool = False + unit_query: str = None + selectors: dict = field(default_factory=dict) # type IRI -> SelectorType -def policy_digest(path: Path) -> str: - """sha256 of the policy file's bytes, as recorded in every manifest.""" - return "sha256:" + hashlib.sha256(Path(path).read_bytes()).hexdigest() +def variable(prop: str) -> str: + """The query variable a parameter property binds: vcfp:start -> 'start'.""" + return prop.rstrip("/#").replace("#", "/").rsplit("/", 1)[1] -def load_policy(path: Path): - """Parse and validate a policy file; returns (rdflib graph, [Rule, ...]).""" - import rdflib +def load_profile(sources=(), extra_graph=None) -> Profile: + """Merge profile files (paths or bundled names) and any declarations in `extra_graph`. - graph = rdflib.Graph().parse(str(path), format="turtle") - policies = {s for s, o in graph.subject_objects(rdflib.RDF.type) if str(o) in POLICY_CLASSES} - if not policies: - raise PolicyError(f"{path}: no odrl:Policy, Set, Offer or Agreement found") - rules = [] - for policy in sorted(policies, key=str): - conflict = graph.value(policy, rdflib.URIRef(ODRL + "conflict")) - if str(conflict) != ODRL + "prohibit": - raise PolicyError(f"<{policy}>: v0.1.0 requires odrl:conflict odrl:prohibit (deny wins)") - if graph.value(policy, rdflib.URIRef(ODRL + "obligation")) is not None: - raise PolicyError(f"<{policy}>: odrl:obligation is not supported in v0.1.0") - for kind in ("permission", "prohibition"): - for node in graph.objects(policy, rdflib.URIRef(ODRL + kind)): - rules.append(_rule(graph, str(policy), kind, node)) - return graph, rules - - -def _rule(graph, policy, kind, node): + `extra_graph` is the policy's own graph, so a policy can declare the selector + types it uses. Exactly one vcfp:Profile must result. + """ import rdflib - unknown = {str(p) for p in graph.predicates(node)} - RULE_PROPERTIES - if unknown: - raise PolicyError(f"{kind} in <{policy}> uses unsupported properties: {sorted(unknown)}") - if str(graph.value(node, rdflib.URIRef(ODRL + "action"))) != ODRL + "read": - raise PolicyError(f"{kind} in <{policy}>: the only supported action is odrl:read") - assignee = graph.value(node, rdflib.URIRef(ODRL + "assignee")) - return Rule( - kind=kind, - policy=policy, - target=_target(graph, graph.value(node, rdflib.URIRef(ODRL + "target"))), - assignee=None if assignee is None or str(assignee) == ODRL + "All" else str(assignee), - constraints=tuple(_constraint(graph, c) for c in graph.objects(node, rdflib.URIRef(ODRL + "constraint"))), - duties=tuple(sorted(_duty(graph, d) for d in graph.objects(node, rdflib.URIRef(ODRL + "duty")))), + graph = rdflib.Graph() + for source in sources or ("vcf-core",): + path = files("vcf_rdfizer_data.policy") / BUNDLED[source] if source in BUNDLED else Path(source) + graph.parse(str(path), format="turtle") + if extra_graph is not None: + graph += extra_graph + + def one(node, name, cast=str): + value = graph.value(node, rdflib.URIRef(VCFP + name)) + return None if value is None else cast(value) + + profiles = list(graph.subjects(rdflib.RDF.type, rdflib.URIRef(VCFP + "Profile"))) + if len(profiles) != 1: + raise PolicyError(f"expected exactly one vcfp:Profile, found {len(profiles)}") + node = profiles[0] + selectors = {} + for kind in graph.subjects(rdflib.RDF.type, rdflib.URIRef(VCFP + "SelectorType")): + selectors[str(kind)] = _selector(graph, kind, one) + return Profile( + iri=str(node), + ownership_path=one(node, "ownershipPath"), + iri_subtree=bool(one(node, "iriSubtree", lambda v: v.toPython())), + unit_query=_checked(one(node, "unitQuery"), f"<{node}> vcfp:unitQuery", ("resource", "group")), + selectors=selectors, ) -def _target(graph, node): - import rdflib - - if node is None: - raise PolicyError("a rule has no odrl:target") - selectors = list(graph.objects(node, rdflib.URIRef(VCFP + "selector"))) - if not selectors: - iri = str(node) - if isinstance(node, rdflib.URIRef) and iri.startswith("file://") and "#" not in iri: - return FileTarget(iri) - raise PolicyError(f"target <{node}> is neither a file IRI nor a vcfp:GraphSelection") - if len(selectors) != 1: - raise PolicyError(f"<{node}>: a GraphSelection needs exactly one vcfp:selector") - selector = selectors[0] - kind = str(graph.value(selector, rdflib.RDF.type) or "") - - def get(name, cast=str): - value = graph.value(selector, rdflib.URIRef(VCFP + name)) - if value is None: - raise PolicyError(f"<{node}>: selector is missing vcfp:{name}") - return cast(value) - - if kind == VCFP + "RegionSelector": - target = RegionTarget(str(node), get("assembly"), get("chrom"), get("start", int), get("end", int)) - if target.start > target.end: - raise PolicyError(f"<{node}>: vcfp:start is after vcfp:end") - return target - if kind == VCFP + "VariantSelector": - return VariantTarget(str(node), get("assembly"), get("chrom"), get("pos", int), get("ref"), get("alt")) - raise PolicyError(f"<{node}>: selector type {kind or '(none)'} is not supported in v0.1.0") - - -def _constraint(graph, node): - import rdflib - - left = str(graph.value(node, rdflib.URIRef(ODRL + "leftOperand"))) - operator = str(graph.value(node, rdflib.URIRef(ODRL + "operator"))) - if left != ODRL + "purpose": - raise PolicyError(f"only odrl:purpose constraints are supported, not <{left}>") - if operator not in (ODRL + "isAnyOf", ODRL + "isNoneOf"): - raise PolicyError(f"only odrl:isAnyOf and odrl:isNoneOf are supported, not <{operator}>") - values = list(graph.objects(node, rdflib.URIRef(ODRL + "rightOperand"))) - if not values: - raise PolicyError("a purpose constraint has no odrl:rightOperand") - return Constraint(operator.rsplit("/", 1)[1], frozenset(purpose_iri(str(v)) for v in values)) - - -def _duty(graph, node): +def _selector(graph, kind, one) -> SelectorType: import rdflib - transform = graph.value(node, rdflib.URIRef(VCFP + "transform")) - if transform is not None and str(transform) != VCFP + "drop": - raise PolicyError(f"effect <{transform}> is not supported in v0.1.0; only vcfp:drop") - return str(graph.value(node, rdflib.URIRef(ODRL + "action"))) + query = one(kind, "query") + if query is None: + raise PolicyError(f"selector type <{kind}> has no vcfp:query") + parameters = tuple(sorted(str(p) for p in graph.objects(kind, rdflib.URIRef(VCFP + "parameter")))) + return SelectorType(str(kind), _checked(query, f"<{kind}> vcfp:query", ("resource",)), + parameters, _checked(one(kind, "violations"), f"<{kind}> vcfp:violations", ())) + + +def _checked(query, where, required): + """Parse a declared query now, so a typo fails at load time, not mid-evaluation.""" + if query is None: + return None + from rdflib.plugins.sparql import prepareQuery + + try: + prepared = prepareQuery(query) + except Exception as error: # rdflib raises several parser exception types + raise PolicyError(f"{where} does not parse: {error}") from None + projected = {str(v) for v in prepared.algebra.get("PV", [])} + missing = [v for v in required if v not in projected] + if missing: + raise PolicyError(f"{where} must project {', '.join('?' + v for v in missing)}") + return query diff --git a/vcf_rdfizer_policies/purposes.py b/vcf_rdfizer_policies/purposes.py deleted file mode 100644 index d9ffe75..0000000 --- a/vcf_rdfizer_policies/purposes.py +++ /dev/null @@ -1,47 +0,0 @@ -"""DUO purposes, and the subsumption rule used to match them. - -A requester's purpose satisfies a consented term when it is that term or a -narrower one: disease-specific research falls within a general-research consent, -not the reverse. The hierarchy is the bundled DUO subset -(vcf_rdfizer_data/policy/duo-subset.ttl), not a live ontology. -""" - -from functools import lru_cache -from importlib.resources import files -import re - -from . import OBO, PolicyError - -_TERM = re.compile(r"(?:obo:|http://purl\.obolibrary\.org/obo/)?DUO[:_](\d{7})") - - -def purpose_iri(value: str) -> str: - """Normalise obo:DUO_0000007, DUO:0000007, DUO_0000007 or a full IRI.""" - match = _TERM.fullmatch(value.strip()) - if not match: - raise PolicyError(f"not a DUO term: {value!r}") - iri = f"{OBO}DUO_{match.group(1)}" - if iri not in _parents(): - raise PolicyError(f"{value} is not in the bundled DUO subset; v0.1.0 knows " - + ", ".join(sorted(t.rsplit("/", 1)[1] for t in _parents()))) - return iri - - -@lru_cache(maxsize=1) -def _parents() -> dict: - """Each bundled term mapped to its one DUO parent.""" - import rdflib - - graph = rdflib.Graph().parse(files("vcf_rdfizer_data.policy") / "duo-subset.ttl") - return {str(s): str(o) for s, o in graph.subject_objects(rdflib.RDFS.subClassOf)} - - -def within(purpose: str, term: str) -> bool: - """True when `purpose` is `term` or a descendant of it.""" - parents = _parents() - node = purpose - while node is not None: - if node == term: - return True - node = parents.get(node) - return False diff --git a/vcf_rdfizer_policies/release.py b/vcf_rdfizer_policies/release.py index 214fb7c..a7f2245 100644 --- a/vcf_rdfizer_policies/release.py +++ b/vcf_rdfizer_policies/release.py @@ -1,10 +1,9 @@ -"""Evaluate a policy for one request, and attach policies to a graph. +"""Evaluate a policy for one request, write the release, and attach policies to a graph. -`evaluate` decides every file and record (decide.py), withholds the IRI subtrees -of whatever is refused, and drops any remaining triple that points into a -withheld subtree, so a view never references something it does not contain. -`attach` writes the policies into the data instead, so they can be queried -alongside it. +`evaluate` produces the view (engine.view), a decision for every reporting unit +the profile names, and a decision for each unit group. `write_release` writes +them with a manifest. `attach` writes the policies into the data instead, so +they can be queried alongside it. """ from dataclasses import dataclass, field @@ -14,70 +13,49 @@ from pathlib import Path from . import DISCLOSURE_MODEL, ODRL, VCFP, VERSION -from .decide import applies, check_assemblies, covers, decide -from .graphs import assemblies, records, split -from .profile import FileTarget +from .engine import check_preconditions, evaluation, select, units, view +from .policy import Direct @dataclass class Release: request: object - view: list = field(default_factory=list) # (s, p, o) rdflib terms - decisions: list = field(default_factory=list) # (Record, Decision) - files: dict = field(default_factory=dict) # file IRI -> Decision - duties: tuple = () # of the permissions that released a file + view: list = field(default_factory=list) # released (s, p, o) triples_withheld: int = 0 + units: list = field(default_factory=list) # (unit dict, released, reason) + groups: dict = field(default_factory=dict) # group IRI -> (released, reason) + duties: tuple = () # of the permissions that released something -def evaluate(graph, rules, request) -> Release: - import rdflib - - file_assemblies = assemblies(graph) - check_assemblies(rules, file_assemblies) +def evaluate(graph, rules, request, profile, vocabulary) -> Release: + decided = evaluation(graph, rules, request, profile, vocabulary) release = Release(request) - release.files = {f: decide(f, rules, request) for f in sorted(file_assemblies)} - withheld_rows = set() - for record in records(graph): - file_decision = release.files[record.file] - decision = decide(record, rules, request) if file_decision.released else file_decision - release.decisions.append((record, decision)) - if not decision.released: - withheld_rows.add((record.file, record.row)) - withheld_files = {f for f, d in release.files.items() if not d.released} + release.view, release.triples_withheld = view(graph, decided) + for unit in units(graph, profile): + release.units.append((unit, *decided.decide(unit["resource"]))) + for group in sorted({u["group"] for u, _, _ in release.units}, key=str): + release.groups[str(group)] = decided.decide(group) + released_subjects = {s for s, _, _ in release.view} release.duties = tuple(sorted({ - duty for rule in rules if rule.kind == "permission" and applies(rule, request) - and any(covers(rule.target, f) for f in release.files if f not in withheld_files) + duty for rule, owned in decided.binding if rule.kind == "permission" + and any(decided.partition.contains(owned, s) for s in released_subjects) for duty in rule.duties})) - - def withheld(term) -> bool: - if not isinstance(term, rdflib.URIRef): - return False - file_iri, row = split(str(term)) - return file_iri in withheld_files or (file_iri, row) in withheld_rows - - for triple in graph: - if withheld(triple[0]) or withheld(triple[2]): - release.triples_withheld += 1 - else: - release.view.append(triple) return release def summary(release) -> dict: - """Counts per file and per deciding reason, for the manifest and the paper figure.""" - per_file = {} - for record, decision in release.decisions: - counts = per_file.setdefault(record.file, {"records_released": 0, "records_withheld": 0}) - counts["records_released" if decision.released else "records_withheld"] += 1 + """Counts per group and per deciding reason, for the manifest and the paper figure.""" + counts = {g: {"released": ok, "reason": why, "records_released": 0, "records_withheld": 0} + for g, (ok, why) in release.groups.items()} reasons = {} - for _, decision in release.decisions: - reasons[decision.reason] = reasons.get(decision.reason, 0) + 1 + for unit, ok, why in release.units: + counts[str(unit["group"])]["records_released" if ok else "records_withheld"] += 1 + reasons[why] = reasons.get(why, 0) + 1 return { "request": {"assignee": release.request.assignee, "purpose": release.request.purpose}, - "files": {f: {"released": d.released, "reason": d.reason, **per_file.get(f, {})} - for f, d in release.files.items()}, - "records_released": sum(d.released for _, d in release.decisions), - "records_withheld": sum(not d.released for _, d in release.decisions), + "groups": counts, + "records_released": sum(ok for _, ok, _ in release.units), + "records_withheld": sum(not ok for _, ok, _ in release.units), "triples_released": len(release.view), "triples_withheld": release.triples_withheld, "reasons": reasons, @@ -93,89 +71,81 @@ def write_release(release, out_dir: Path, *, policies, digest: str) -> None: raise FileExistsError(f"{out_dir} is not empty; a release is never overwritten") out_dir.mkdir(parents=True, exist_ok=True) - view = rdflib.Graph() + graph = rdflib.Graph() for triple in release.view: - view.add(triple) - lines = sorted(line for line in view.serialize(format="nt").splitlines() if line.strip()) + graph.add(triple) + lines = sorted(line for line in graph.serialize(format="nt").splitlines() if line.strip()) (out_dir / "view.nt").write_text("\n".join(lines) + "\n", encoding="utf-8") + columns = list(release.units[0][0]) if release.units else ["resource", "group"] with (out_dir / "decisions.csv").open("w", newline="", encoding="utf-8") as handle: writer = csv.writer(handle) - writer.writerow(["file", "row", "chrom", "pos", "ref", "alts", "released", "reason"]) - for record, decision in release.decisions: - writer.writerow([record.file, record.row, record.chrom, record.pos, record.ref, - ",".join(record.alts), decision.released, decision.reason]) + writer.writerow(columns + ["released", "reason"]) + for unit, ok, why in release.units: + writer.writerow([("" if unit[c] is None else str(unit[c])) for c in columns] + [ok, why]) counts = summary(release) (out_dir / "summary.json").write_text(json.dumps(counts, indent=2) + "\n", encoding="utf-8") (out_dir / "manifest.ttl").write_text( - manifest(counts, policies=policies, digest=digest, sources=release.files, duties=release.duties), + manifest(counts, policies=policies, digest=digest, sources=release.groups, duties=release.duties), encoding="utf-8") def manifest(counts, *, policies, digest, sources, duties) -> str: - """The vcfp:ReleaseView description (docs/policy-demonstrator.md §6.1), as Turtle.""" + """The vcfp:ReleaseView description (docs/policy-demonstrator.md §6), as Turtle.""" import rdflib from rdflib.namespace import PROV, XSD vcfp, odrl = rdflib.Namespace(VCFP), rdflib.Namespace(ODRL) g = rdflib.Graph() - g.bind("vcfp", vcfp), g.bind("odrl", odrl), g.bind("prov", PROV) - view, request = rdflib.URIRef("#release"), rdflib.BNode() - g.add((view, rdflib.RDF.type, vcfp.ReleaseView)) + for prefix, namespace in (("vcfp", vcfp), ("odrl", odrl), ("prov", PROV)): + g.bind(prefix, namespace) + node, request = rdflib.URIRef("#release"), rdflib.BNode() + g.add((node, rdflib.RDF.type, vcfp.ReleaseView)) for source in sorted(sources): - g.add((view, vcfp.derivedFrom, rdflib.URIRef(source))) + g.add((node, vcfp.derivedFrom, rdflib.URIRef(source))) for policy in sorted(policies): - g.add((view, vcfp.policy, rdflib.URIRef(policy))) - g.add((view, vcfp.policyDigest, rdflib.Literal(digest))) - g.add((view, vcfp.request, request)) + g.add((node, vcfp.policy, rdflib.URIRef(policy))) + g.add((node, vcfp.policyDigest, rdflib.Literal(digest))) + g.add((node, vcfp.request, request)) g.add((request, odrl.assignee, rdflib.URIRef(counts["request"]["assignee"]))) g.add((request, odrl.purpose, rdflib.URIRef(counts["request"]["purpose"]))) - for name in ("records_released", "records_withheld", "triples_withheld"): - g.add((view, vcfp[_camel(name)], rdflib.Literal(counts[name]))) - g.add((view, vcfp.filesWithheld, rdflib.Literal(sum(not f["released"] for f in counts["files"].values())))) - for duty in sorted(duties): + for key, prop in (("records_released", "recordsReleased"), ("records_withheld", "recordsWithheld"), + ("triples_withheld", "triplesWithheld")): + g.add((node, vcfp[prop], rdflib.Literal(counts[key]))) + g.add((node, vcfp.groupsWithheld, rdflib.Literal(sum(not c["released"] for c in counts["groups"].values())))) + for duty in duties: obligation = rdflib.BNode() - g.add((view, vcfp.obligation, obligation)) + g.add((node, vcfp.obligation, obligation)) g.add((obligation, odrl.action, rdflib.URIRef(duty))) - g.add((view, vcfp.disclosureModel, rdflib.Literal(DISCLOSURE_MODEL))) - g.add((view, PROV.wasGeneratedBy, rdflib.URIRef(f"urn:vcf-rdfizer-policy:{VERSION}"))) - g.add((view, PROV.generatedAtTime, + g.add((node, vcfp.disclosureModel, rdflib.Literal(DISCLOSURE_MODEL))) + g.add((node, PROV.wasGeneratedBy, rdflib.URIRef(f"urn:vcf-rdfizer-policy:{VERSION}"))) + g.add((node, PROV.generatedAtTime, rdflib.Literal(datetime.now(timezone.utc).replace(microsecond=0).isoformat(), datatype=XSD.dateTime))) return g.serialize(format="turtle") -def _camel(name: str) -> str: - head, *rest = name.split("_") - return head + "".join(part.title() for part in rest) - - def attach(graph, policy_graph, rules) -> dict: """Merge the policies into `graph` and link each governed resource to its policy. - Files get odrl:hasPolicy directly. A region or variant selection records what - it selects (vcfp:selects), and each selected record gets odrl:hasPolicy too, - so a SPARQL query needs no knowledge of the selectors. Returns counts per asset. + A direct target gets odrl:hasPolicy. A selection records what it selects + (vcfp:selects), and each selected resource gets odrl:hasPolicy too, so a + SPARQL query needs no knowledge of the selectors. Returns counts per asset. """ import rdflib has_policy, selects = rdflib.URIRef(ODRL + "hasPolicy"), rdflib.URIRef(VCFP + "selects") - check_assemblies(rules, assemblies(graph)) - all_records = records(graph) + check_preconditions(graph, rules) + selections = {rule.target: select(graph, rule.target) for rule in rules} # before merging + graph += policy_graph counts = {} - for triple in policy_graph: - graph.add(triple) for rule in rules: policy = rdflib.URIRef(rule.policy) - if isinstance(rule.target, FileTarget): - graph.add((rdflib.URIRef(rule.target.iri), has_policy, policy)) - counts[rule.target.iri] = 1 - continue - asset = rdflib.URIRef(rule.target.asset) - selected = [r for r in all_records if covers(rule.target, r)] - for record in selected: - record_iri = rdflib.URIRef(f"{record.file}#record/{record.row}") - graph.add((asset, selects, record_iri)) - graph.add((record_iri, has_policy, policy)) - counts[rule.target.asset] = len(selected) + chosen = selections[rule.target] + asset = None if isinstance(rule.target, Direct) else rdflib.URIRef(rule.target.asset) + for resource in chosen: + graph.add((resource, has_policy, policy)) + if asset is not None: + graph.add((asset, selects, resource)) + counts[getattr(rule.target, "asset", None) or rule.target.iri] = len(chosen) return counts diff --git a/vcf_rdfizer_policies/vcf_oracle.py b/vcf_rdfizer_policies/vcf_oracle.py new file mode 100644 index 0000000..2456d8f --- /dev/null +++ b/vcf_rdfizer_policies/vcf_oracle.py @@ -0,0 +1,73 @@ +"""The VCF-side oracle: which records a correct view of VCF-RDFizer graphs releases. + +It reads the source VCFs as text -- no converted graph, no conversion -- and +builds a small VCF Core graph of the fixed columns except INFO: each file with +its reference genome; each record with CHROM, POS, ID, REF and ALT; and its +call with QUAL and FILTER, in the shapes and under the IRIs VCF-RDFizer uses +(file://NAME, #record/N, #call/N). A selector that reads INFO, FORMAT or the +header is outside what the oracle models; its views are covered by the +structural checks in check.py only, so do not pass --vcf for such a policy. +The same policy is then evaluated on that graph. The view's records must equal +the records released there exactly: an extra one is a leak, a missing one is +over-withholding. Its independence comes from its input, not from a second +copy of the rules. +""" + +from decimal import Decimal +import gzip +from pathlib import Path +import re + +from . import VCFC +from .engine import evaluation, units + + +def graph_from_vcfs(paths): + """A minimal VCF Core graph built directly from VCF text.""" + import rdflib + + vcfc = rdflib.Namespace(VCFC) + graph = rdflib.Graph() + for path in map(Path, paths): + name = re.sub(r"\.gz$", "", path.name) + file_iri = rdflib.URIRef(f"file://{name}") + graph.add((file_iri, rdflib.RDF.type, vcfc.VCFFile)) + row = 0 + opener = gzip.open if path.name.endswith(".gz") else open + with opener(path, "rt", encoding="utf-8") as handle: + for line in handle: + if line.startswith("##reference="): + graph.add((file_iri, vcfc.referenceGenome, rdflib.Literal(line.split("=", 1)[1].strip()))) + if line.startswith("#"): + continue + row += 1 + chrom, pos, ident, ref, alt, qual, filters = line.rstrip("\n").split("\t")[:7] + record = rdflib.URIRef(f"{file_iri}#record/{row}") + call = rdflib.URIRef(f"{file_iri}#call/{row}") + graph.add((file_iri, vcfc.hasRecord, record)) + graph.add((record, rdflib.RDF.type, vcfc.VCFRecord)) + graph.add((record, vcfc.hasCall, call)) + graph.add((record, vcfc.chrom, rdflib.Literal(chrom))) + graph.add((record, vcfc.pos, rdflib.Literal(int(pos)))) + graph.add((record, vcfc.ref, rdflib.Literal(ref))) + for allele in alt.split(","): + if allele != ".": + graph.add((record, vcfc.alt, rdflib.Literal(allele))) + for value in ident.split(";"): + if value != ".": + graph.add((record, vcfc.recordId, rdflib.Literal(value))) + if qual != ".": + graph.add((call, vcfc.qual, rdflib.Literal(Decimal(qual)))) + if filters != ".": + graph.add((call, vcfc.filter, rdflib.Literal(filters))) + return graph + + +def compare(view, vcf_paths, *, rules, request, profile, vocabulary) -> list: + """Failures where the view's records differ from the oracle's released records.""" + oracle = graph_from_vcfs(vcf_paths) + decided = evaluation(oracle, rules, request, profile, vocabulary) + expected = {str(u["resource"]) for u in units(oracle, profile) if decided.decide(u["resource"])[0]} + actual = {str(u["resource"]) for u in units(view, profile)} + return ([f"leak: <{r}> is released but the policy withholds it" for r in sorted(actual - expected)] + + [f"over-withheld: <{r}> should have been released" for r in sorted(expected - actual)]) diff --git a/vcf_rdfizer_policies/vocabulary.py b/vcf_rdfizer_policies/vocabulary.py new file mode 100644 index 0000000..1346388 --- /dev/null +++ b/vcf_rdfizer_policies/vocabulary.py @@ -0,0 +1,59 @@ +"""Purpose vocabularies: which purposes fall within which. + +A purpose satisfies a term when it is that term or narrower, following +rdfs:subClassOf or skos:broader, so any RDFS or SKOS vocabulary works. The +bundled four-term DUO subset (vcf_rdfizer_data/policy/duo-subset.ttl) is only +the default; full DUO or a local vocabulary is passed with --purposes. +""" + +from importlib.resources import files +from pathlib import Path + +from . import PolicyError + +BROADER = ("http://www.w3.org/2000/01/rdf-schema#subClassOf", + "http://www.w3.org/2004/02/skos/core#broader") + + +class Vocabulary: + def __init__(self, graph): + import rdflib + + self.parents = {} + for predicate in BROADER: + for narrow, broad in graph.subject_objects(rdflib.URIRef(predicate)): + self.parents.setdefault(str(narrow), set()).add(str(broad)) + self.terms = set(self.parents) | {b for broads in self.parents.values() for b in broads} + self._namespaces = graph.namespace_manager + + @classmethod + def load(cls, path=None) -> "Vocabulary": + """The vocabulary at `path`, or the bundled DUO subset.""" + import rdflib + + source = Path(path) if path else files("vcf_rdfizer_data.policy") / "duo-subset.ttl" + return cls(rdflib.Graph().parse(str(source), format="turtle")) + + def resolve(self, value: str) -> str: + """A full IRI, or a prefixed name the vocabulary binds (DUO:0000007), as a known term.""" + iri = value.strip() + if ":" in iri and "://" not in iri: + try: + iri = str(self._namespaces.expand_curie(iri)) + except ValueError: + raise PolicyError(f"{value!r}: prefix is not bound by the purpose vocabulary") from None + if iri not in self.terms: + raise PolicyError(f"{value} is not a term of the purpose vocabulary") + return iri + + def within(self, purpose: str, term: str) -> bool: + """True when `purpose` is `term` or a descendant of it.""" + seen, frontier = set(), [purpose] + while frontier: + node = frontier.pop() + if node == term: + return True + if node not in seen: + seen.add(node) + frontier.extend(self.parents.get(node, ())) + return False diff --git a/vcf_rdfizer_policy.py b/vcf_rdfizer_policy.py index ce70e6b..876ea12 100644 --- a/vcf_rdfizer_policy.py +++ b/vcf_rdfizer_policy.py @@ -1,11 +1,14 @@ #!/usr/bin/env python3 -"""vcf-rdfizer-policy: attach ODRL policies to VCF graphs, evaluate them, and check the result. +"""vcf-rdfizer-policy: attach ODRL policies to RDF graphs, evaluate them, and check the result. + vcf-rdfizer-policy explain --policy policy.ttl vcf-rdfizer-policy attach --rdf P*.nt.gz --policy policy.ttl -o annotated.nt vcf-rdfizer-policy evaluate --rdf P*.nt.gz --policy policy.ttl --assignee IRI --purpose DUO:0000007 -o views/alz - vcf-rdfizer-policy check --view views/alz --policy policy.ttl --vcf P*.vcf - vcf-rdfizer-policy explain --policy policy.ttl + vcf-rdfizer-policy check --view views/alz --rdf P*.nt.gz --policy policy.ttl [--vcf P*.vcf] +Selectors and the ownership rule come from a profile (--profile, default the +bundled VCF Core profile; selector types may also be declared in the policy +file), and purposes from a vocabulary (--purposes, default a DUO subset). v0.1.0 demonstrator: governed release, not anonymization. Runs on the host; no Docker. Exit codes: 0 success, 1 a check failed, 2 the policy, graph or request cannot be evaluated. See docs/policy-demonstrator.md. @@ -25,36 +28,57 @@ def _require_rdflib(): raise PolicyError("vcf-rdfizer-policy requires 'rdflib': python -m pip install rdflib") from None +def _setup(args): + """(policy graph, profile, vocabulary, rules) from the common arguments.""" + from vcf_rdfizer_policies.policy import load_rules, read_graph + from vcf_rdfizer_policies.profile import load_profile + from vcf_rdfizer_policies.vocabulary import Vocabulary + + graph = read_graph(args.policy) + profile = load_profile(args.profile, extra_graph=graph) + vocabulary = Vocabulary.load(args.purposes) + return graph, profile, vocabulary, load_rules(graph, profile, vocabulary) + + +def cmd_explain(args): + _, profile, _, rules = _setup(args) + for rule in rules: + who = "anyone" if rule.assignee is None else f"<{rule.assignee}>" + terms = [f"purpose {c.operator} {', '.join(sorted(p.rsplit('/', 1)[1] for p in c.purposes))}" + for c in rule.constraints] + what = f" (a {rule.target.selector.iri.rsplit('#', 1)[-1]})" if hasattr(rule.target, "selector") else "" + print(f"{rule.label}{what}: applies to {who}" + (" when " + " and ".join(terms) if terms else "") + + (f"; duties: {', '.join(d.rsplit('/', 1)[1] for d in rule.duties)}" if rule.duties else "")) + print(f"Deny wins; anything no permission covers is withheld. Profile: <{profile.iri}>.") + return 0 + + def cmd_attach(args): from vcf_rdfizer_policies.graphs import load - from vcf_rdfizer_policies.profile import load_policy from vcf_rdfizer_policies.release import attach - policy_graph, rules = load_policy(args.policy) - graph = load(args.rdf) + policy_graph, _, _, rules = _setup(args) out = Path(args.out) if out.exists(): raise FileExistsError(f"{out} exists; attach never overwrites") - counts = attach(graph, policy_graph, rules) - out.write_text(graph.serialize(format="nt"), encoding="utf-8") - for asset, n in sorted(counts.items()): + graph = load(args.rdf) + for asset, n in sorted(attach(graph, policy_graph, rules).items()): print(f"{asset}: {n} resource(s) linked") + out.write_text(graph.serialize(format="nt"), encoding="utf-8") print(f"wrote {out}") return 0 def cmd_evaluate(args): - from vcf_rdfizer_policies.decide import Request + from vcf_rdfizer_policies.engine import Request from vcf_rdfizer_policies.graphs import load - from vcf_rdfizer_policies.profile import load_policy, policy_digest - from vcf_rdfizer_policies.purposes import purpose_iri + from vcf_rdfizer_policies.policy import policy_digest from vcf_rdfizer_policies.release import evaluate, summary, write_release - _, rules = load_policy(args.policy) - request = Request(args.assignee, purpose_iri(args.purpose)) - release = evaluate(load(args.rdf), rules, request) - write_release(release, args.out, policies={r.policy for r in rules}, - digest=policy_digest(args.policy)) + _, profile, vocabulary, rules = _setup(args) + request = Request(args.assignee, vocabulary.resolve(args.purpose)) + release = evaluate(load(args.rdf), rules, request, profile, vocabulary) + write_release(release, args.out, policies={r.policy for r in rules}, digest=policy_digest(args.policy)) counts = summary(release) print(f"released {counts['records_released']} record(s), withheld {counts['records_withheld']}; " f"{counts['triples_withheld']} triple(s) withheld -> {args.out}") @@ -62,59 +86,52 @@ def cmd_evaluate(args): def cmd_check(args): - from vcf_rdfizer_policies.check import check_view - from vcf_rdfizer_policies.profile import load_policy - - _, rules = load_policy(args.policy) - failures = check_view(args.view, args.policy, rules, args.vcf) + from vcf_rdfizer_policies.check import check_view, read_view + from vcf_rdfizer_policies.graphs import load + from vcf_rdfizer_policies.vcf_oracle import compare + + _, profile, vocabulary, rules = _setup(args) + view, manifest, request = read_view(args.view) + failures = check_view(view, manifest, request, policy_path=args.policy, rules=rules, + profile=profile, vocabulary=vocabulary, source=load(args.rdf)) + if args.vcf: + failures += compare(view, args.vcf, rules=rules, request=request, profile=profile, vocabulary=vocabulary) for failure in failures: print(f"FAIL {failure}") print("PASS" if not failures else f"{len(failures)} failure(s)") return 0 if not failures else 1 -def cmd_explain(args): - from vcf_rdfizer_policies.profile import load_policy - - _, rules = load_policy(args.policy) - for rule in rules: - who = "anyone" if rule.assignee is None else f"<{rule.assignee}>" - terms = [f"purpose {c.operator} {', '.join(sorted(p.rsplit('/', 1)[1] for p in c.purposes))}" - for c in rule.constraints] - print(f"{rule.label}: applies to {who}" + (" when " + " and ".join(terms) if terms else "") - + (f"; duties: {', '.join(d.rsplit('/', 1)[1] for d in rule.duties)}" if rule.duties else "")) - print("Deny wins; anything no permission covers is withheld.") - return 0 - - def build_parser(): parser = argparse.ArgumentParser(prog="vcf-rdfizer-policy", description=__doc__.split("\n\n")[0]) parser.add_argument("--version", action="version", version=f"%(prog)s {VERSION}") + common = argparse.ArgumentParser(add_help=False) + common.add_argument("--policy", required=True, type=Path, help="ODRL policy file (Turtle)") + common.add_argument("--profile", action="append", default=[], + help="profile file, or 'vcf-core' (default); repeatable") + common.add_argument("--purposes", type=Path, help="RDFS/SKOS purpose vocabulary (default: a DUO subset)") sub = parser.add_subparsers(dest="command", required=True) - attach = sub.add_parser("attach", help="write the policies into the graph") - attach.add_argument("--rdf", nargs="+", required=True, help="converted .nt / .nt.gz files") - attach.add_argument("--policy", required=True, type=Path) + explain = sub.add_parser("explain", parents=[common], help="list the policy's rules in plain language") + explain.set_defaults(run=cmd_explain) + + attach = sub.add_parser("attach", parents=[common], help="write the policies into the graph") + attach.add_argument("--rdf", nargs="+", required=True, help=".nt / .nt.gz inputs") attach.add_argument("-o", "--out", required=True, help="annotated .nt to create") attach.set_defaults(run=cmd_attach) - evaluate = sub.add_parser("evaluate", help="write one request's release view") - evaluate.add_argument("--rdf", nargs="+", required=True, help="converted .nt / .nt.gz files") - evaluate.add_argument("--policy", required=True, type=Path) + evaluate = sub.add_parser("evaluate", parents=[common], help="write one request's release view") + evaluate.add_argument("--rdf", nargs="+", required=True, help=".nt / .nt.gz inputs") evaluate.add_argument("--assignee", required=True, help="the requesting party's IRI") - evaluate.add_argument("--purpose", required=True, help="a DUO term, e.g. DUO:0000007") + evaluate.add_argument("--purpose", required=True, help="a vocabulary term, e.g. DUO:0000007") evaluate.add_argument("-o", "--out", required=True, type=Path, help="new or empty directory") evaluate.set_defaults(run=cmd_evaluate) - check = sub.add_parser("check", help="verify a view against the source VCFs") + check = sub.add_parser("check", parents=[common], help="verify a view against its source") check.add_argument("--view", required=True, type=Path, help="a directory written by evaluate") - check.add_argument("--policy", required=True, type=Path) - check.add_argument("--vcf", nargs="+", required=True, help="the source VCFs") + check.add_argument("--rdf", nargs="+", required=True, help="the source the view was made from") + check.add_argument("--vcf", nargs="+", help="source VCFs, for the independent record oracle") check.set_defaults(run=cmd_check) - - explain = sub.add_parser("explain", help="list the policy's rules in plain language") - explain.add_argument("--policy", required=True, type=Path) - explain.set_defaults(run=cmd_explain) return parser