Skip to content

chore: feature integration branch for the ENVITED-X pipeline - #3

Draft
jdsika wants to merge 2 commits into
mainfrom
feat/envited-x-pipeline
Draft

jdsika wants to merge 2 commits into
mainfrom
feat/envited-x-pipeline

Conversation

@jdsika

@jdsika jdsika commented Sep 14, 2026 •

Copy link
Copy Markdown

Integration branch. Tracking only, do not merge.

What this branch is

feat/envited-x-pipeline is upstream RDFLib/rdflib@main with every contribution this
organisation is still carrying cherry-picked on top, and nothing else. It is the branch to
check out when you need an rdflib that behaves the way our pipeline expects before
upstream has merged the relevant PRs.

It is rebased on upstream main, never merged into. An empty cherry-pick list is the goal
state: it would mean upstream has merged everything, and the branch could be deleted.

Same convention as ASCS-eV/linkml#14.

What it currently carries

Commit Upstream PR Fork mirror Change
844ac19d RDFLib/rdflib#3543 #2 JSON-LD collection syntax can silently duplicate shared RDF lists
7dd4a556 RDFLib/rdflib#3543 #2 Follow-up by Edmond Chuc, merged into the PR branch from #7: blank nodes behind @id references keep their triples, and explicit chains are written iteratively in linear time

Three of the four changes this branch used to carry are now merged upstream and have
been dropped from it:
#3546 (deterministic N-Triples via an opt-in
canon parameter), #3564 (SPARQL IN /
NOT IN value equality) and #3574 (relative
IRI resolution against a base without its fragment).

The branch is rebased onto upstream main at 58c3700e. It carries the two commits of
RDFLib#3543 without the PR branch's merges of main, and its tree is identical to RDFLib#3543's head
merged into that main, with no conflicts. Snapshot tag: envited-x/2026.10.26.

Last verified on this branch at 7dd4a556, Python 3.12: ruff (check and format) and mypy
are clean for the whole project, and the full test suite gives 8368 passed, 0 failed.

What consumes it

Nothing yet, deliberately. asam-openx-standards pins released rdflib==7.6.0
through scripts/requirements.txt, and diffable-rdf depends on released rdflib too.

That pin is the point: rdflib performs the final serialization of every generated OWL and
SHACL artifact, so a change here moves artifact bytes. Consuming this branch would mean
building from a fork, which the pipeline's reproducibility rules do not allow: the lock
records released versions precisely so a third party can obtain them.

The relationship is therefore one-directional. What lands upstream here eventually arrives
as a released rdflib that the pipeline can bump to, with regenerated artifacts in the same
change. The merged #3504 (the Turtle variant
of the RDFLib#3543 defect) is the precedent: diffable-rdf had to work around it until it was
released.

When a release contains RDFLib#3543, diffable-rdf's sentinel test
test_rdflib_jsonld_serializer_does_not_preserve_shared_list_tail will fail by design;
that is the signal to retire the workaround.

jdsika and others added 2 commits October 26, 2026 17:16
Converter.to_collection walked an rdf:List chain checking only that each cell
carries one rdf:first/rdf:rest pair and that the chain is acyclic. It did not
check how many statements point into the chain.

The @list form inlines the whole chain at a single reference and writes its
cells nowhere else, so when a cell is referenced from more than one place each
referring list rendered the shared cells again. The round trip then gained
triples:

    _:tail rdf:first "b" ; rdf:rest rdf:nil .
    _:head rdf:first "a" ; rdf:rest _:tail .
    ex:s1  ex:p _:head .
    ex:s2  ex:p _:tail .            # second reference into the chain

serialized as two independent @list values, so the tail was emitted twice:
6 triples in, 8 out, not isomorphic. A head referenced twice was duplicated
the same way: 4 in, 6 out.

The Serialize RDF as JSON-LD Algorithm (JSON-LD 1.1 Processing Algorithms and
API, section 8.4) only turns a chain into @list when every cell, the head
included, is a blank node that is "referenced once", counted over the whole
dataset. The converter now applies that condition. It counts blank-node
references once per conversion, over every graph it writes, and a chain with
a cell referenced more than once is written with explicit rdf:first/rdf:rest,
which can express any arrangement of sharing. Counting across graphs also
covers a cell that another named graph points at, whose reference was
otherwise left dangling.

add_to_node offered a term declared with "@container": "@list" for any object
with an rdf:first, then iterated to_collection without checking for None. That
already raised TypeError for a chain the converter refuses, such as one whose
cell carries an extra property, and would have for every shared chain. The
term is now chosen from to_collection's result instead.

This is the defect RDFLib#3504 fixed in the Turtle and longturtle serializers; the
JSON-LD converter has its own chain walk.

Fixes RDFLib#3542.

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
Queue descriptions behind blank-node references, including coercing contexts, so rejected collections retain their triples without recursive expansion.

Cache rejected prefixes for each graph conversion to avoid repeatedly walking shared-tail suffixes. Add round-trip, traversal-count, and converter-reuse regressions.

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
@jdsika
jdsika force-pushed the feat/envited-x-pipeline branch from be5ea8f to 7dd4a55 Compare October 26, 2026 16:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants