Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ All commands use `uv run` prefix (e.g., `uv run pytest`).
* Do not "fix" issues by changing or weakening test conditions. Try harder, or ask questions if a test fails.
* Avoid try/except blocks, these can mask bugs
* Failing fast is a good principle
* Optional dependencies are the documented exception to the two rules above. Guard the import, not the spec: `try: import x` / `except ImportError as exc: raise ImportError("x is required. Install with: pip install 'linkml[extra]'") from exc` - see `generators/bigquerygen.py`. `importlib.util.find_spec` raises under the test fixture below, which hides your message.
* Testing that an optional dependency is absent is not a mock test: use the `mock_missing_import` fixture in `tests/conftest.py`. Do not build isolated environments in CI for this.
* Follow the DRY principle
* Avoid repeating chunks of code, but also avoid premature over-abstraction
* Declarative principles are favored
Expand Down
19 changes: 19 additions & 0 deletions docs/faq/tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,25 @@ There are a few strategies:

You can use any of the generator tools distributed as part of linkml to check for errors in your schema.

## Is there a master list of which generators or validators support which LinkML features?

There is no separately curated, complete matrix of every generator and validator against every metamodel slot.

The closest source of truth is the [Generator Feature Dashboard](../generators/dashboard). It is generated from the [compliance test suite](https://github.com/linkml/linkml/tree/main/tests/linkml/test_compliance) and shows which metamodel features each tested generator implements, partially implements, ignores, or has not yet been tested for.

That dashboard is the outcome of [issue 2333](https://github.com/linkml/linkml/issues/2333) (Generator Support Matrix). It is not exhaustive:

* Only generators that participate in the compliance suite appear as columns.
* Cells can be unknown or partial; a result reflects the current tests, not a guarantee of every edge case.
* There is no equivalent complete matrix for validators such as JSON Schema, Pydantic, SHACL, or ShEx. Those validators typically consume generated artifacts, so generator coverage on the dashboard is the relevant proxy.

For more detail, see:

* [Generators](../generators/index) and the individual generator pages (JSON Schema, Pydantic, SHACL, ShEx, and others)
* [Validating data](../data/validating-data) for validation strategies and plugins

If a feature you need is missing or marked unknown, please open an issue or add a compliance test rather than treating the dashboard as a finished catalog.

## Is there a linter for LinkML?

Yes! See the documentation for [the schema linter](https://linkml.io/linkml/schemas/linter.html).
Expand Down
2 changes: 1 addition & 1 deletion docs/get-involved/Community-Meetings.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Join the LinkML community for regular sessions featuring presentations on LinkML

| Date | Presenter 1 | Topic 1 | Presenter 2 | Topic 2 |
| :---: | :---: | :---: | :----: | :---: |
| October 15, 2026|Piotr Sowiński|[LinkML-Scala](https://github.com/NeverBlink-OSS/linkml-scala)| | Open slot!<br> Volunteers welcome|
| October 15, 2026|Piotr Sowiński|[LinkML-Scala](https://github.com/NeverBlink-OSS/linkml-scala)| Maia Kapur| Autonomous science development at EMSL & model viewer in the BASALT GitHub docs|
| [September 17, 2026](https://docs.google.com/presentation/d/1b_XGol4vnEKsAp0zwCKludfVP_IOFnYoxwW3gBRbbLU/edit?slide=id.g36e69bd970c_1_50#slide=id.g36e69bd970c_1_50)| Matt Gehring| How sniff.world uses LinkML: from consuming Biolink to publishing its own schemas| Nolan Nicols |[Provenance as a Primitive: Building Knowledge Graphs with LOKF](https://docs.google.com/presentation/d/155JuRM4nBrLsFOFkMv3ZQfgQleqwieP27c0vWAo9SZM/edit?slide=id.p1#slide=id.p1)|
| [August 20, 2026](https://docs.google.com/presentation/d/1_DgJTclLsi9XEcnmgAfftVD5HZ1Aa0TF6abGB_YCGx0/edit?slide=id.g36e69bd970c_1_50#slide=id.g36e69bd970c_1_50)| Patrick Golden| Working across boundaries: LinkML in the [Zebrafish Phenotype Atlas Project](https://zappfish.org/)|Alex Anderson| Semantic Converters for Power Systems Data Interoperability and Agentic AI|
| [July 16, 2026](https://docs.google.com/presentation/d/1A05qfTbmI8RXyvoBSplpjSU9e4OP6BlIKJvQ1-eWhGw/edit?slide=id.g36e69bd970c_1_50#slide=id.g36e69bd970c_1_50)| Sierra Moxon | LinkML Microschemas |Stephan Heunis |[The ORINOCO project at Research Center Jülich - building self-hostable research information infrastructure](https://pool.psychoinformatics.de/ui/?sh%3ANodeShape=xyzri%3AXYZProject&pid=xyzrins%3Aprojects%2Forinoco) |
Expand Down
30 changes: 30 additions & 0 deletions docs/howtos/collaborative-development.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,36 @@ Some projects may opt to manage their data alongside the schema in the same repo
for "registry" style projects, but in other cases the data may be best managed separately in a dedicated
store or a repository like Zenodo or Figshare.

(rdf-in-version-control)=
### Keep generated RDF diffs small

When you track generated RDF in Git, adding a class or property can change many
blank-node labels. The default RDFC-1.0 numbering is repeatable for the same graph,
but an insertion can renumber existing nodes and obscure the actual schema edit.

The optional `--diff-stable` flag uses
[diffable-rdf](https://github.com/ASCS-eV/diffable-rdf) to reduce this label churn.
In a uv project containing your schema, install LinkML with the extra and generate
SHACL as follows (using a release that includes this option):

```bash
uv add 'linkml[diff-stable]'
uv run gen-shacl --diff-stable schema.yaml > schema.shacl.ttl
```

The flag also works with `gen-owl`, `gen-rdf`, and `gen-shex --format rdf`.
Python callers of `canonicalize_rdf_graph(..., diff_stable=True)` can install
`linkml-runtime[diff-stable]`. The feature is disabled by default; enabling it
changes existing labels once, so keep that regeneration separate from schema edits.

Labels depend on blank-node neighborhoods. Edits within connected structures or
among symmetric nodes can still affect other labels; minimum diffs are not
guaranteed. Keep tool versions consistent when comparing generated files.
The relabeled output is not the standard RDFC canonical representation.
ShExC, ShEx JSON and instance-data conversion are outside this option's scope.
Non-standard RDF takes the existing fallback path, which warns that it cannot
apply diff-stable labels.

### Permissively License Your Code and Data

(adapted from [O3 guidelines](https://osf.io/vuzt3/))
Expand Down
4 changes: 4 additions & 0 deletions packages/linkml/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,9 @@ dependencies = [ # Specifier syntax: https://peps.python.org/pep-0631/
"platformdirs>=4.12.2",
]

[project.optional-dependencies]
diff-stable = ["linkml-runtime[diff-stable]>=1.10.0,<2.0.0"]

[dependency-groups]
lint = [
"black >= 24.0.0",
Expand All @@ -79,6 +82,7 @@ shacl = [
"pyshacl>=0.40.1",
]
tests = [
"linkml-runtime[diff-stable]",
{ include-group = "lint" },
{ include-group = "typing" },
{ include-group = "shacl" },
Expand Down
14 changes: 13 additions & 1 deletion packages/linkml/src/linkml/generators/owlgen.py
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,9 @@ class OwlSchemaGenerator(Generator):
"""Suffix to add to the schema name to create the ontology URI, e.g. .owl.ttl"""

# ObjectVars
diff_stable: bool = False
"""Reduce blank-node label churn. See :ref:`rdf-in-version-control`."""

metadata_profile: MetadataProfile | None = None
"""Deprecated - use metadata_profiles."""

Expand Down Expand Up @@ -353,7 +356,7 @@ def serialize(self, **kwargs: Any) -> str:
"""
self.as_graph()
fmt = "turtle" if self.format in ["owl", "ttl"] else self.format
return canonicalize_rdf_graph(self.graph, output_format=fmt)
return canonicalize_rdf_graph(self.graph, output_format=fmt, diff_stable=self.diff_stable)

def add_metadata(self, e: Definition | PermissibleValue, uri: URIRef) -> None:
"""
Expand Down Expand Up @@ -1844,6 +1847,15 @@ def slot_owl_type(self, slot: SlotDefinition) -> URIRef:
"specified language tag. Element-level in_language overrides this."
),
)
@click.option(
"--diff-stable/--no-diff-stable",
default=False,
show_default=True,
help=(
"Reduce blank-node label churn across edits. See "
"https://linkml.io/linkml/howtos/collaborative-development.html#rdf-in-version-control"
),
)
@click.version_option(__version__, "-V", "--version")
def cli(yamlfile: str, metadata_profile: str, **kwargs: Any) -> None:
"""Generate an OWL representation of a LinkML model
Expand Down
14 changes: 13 additions & 1 deletion packages/linkml/src/linkml/generators/rdfgen.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,9 @@ class RDFGenerator(Generator):
uses_schemaloader = True

# ObjectVars
diff_stable: bool = False
"""Reduce blank-node label churn. See :ref:`rdf-in-version-control`."""

emit_metadata: bool = False
context: list[str] = None
original_schema: SchemaDefinition = None
Expand All @@ -89,7 +92,7 @@ def __post_init__(self):

def _data(self, g: Graph) -> str:
fmt = "turtle" if self.format == "ttl" else self.format
return canonicalize_rdf_graph(g, output_format=fmt)
return canonicalize_rdf_graph(g, output_format=fmt, diff_stable=self.diff_stable)

def end_schema(self, output: str | None = None, context: str = None, **_) -> str:
gen = JSONLDGenerator(
Expand Down Expand Up @@ -137,6 +140,15 @@ def end_schema(self, output: str | None = None, context: str = None, **_) -> str
multiple=True,
help="JSONLD context file (default: vendored meta.context.jsonld)",
)
@click.option(
"--diff-stable/--no-diff-stable",
default=False,
show_default=True,
help=(
"Reduce blank-node label churn across edits. See "
"https://linkml.io/linkml/howtos/collaborative-development.html#rdf-in-version-control"
),
)
@click.version_option(__version__, "-V", "--version")
def cli(yamlfile, **kwargs):
"""Generate an RDF representation of a LinkML model"""
Expand Down
14 changes: 13 additions & 1 deletion packages/linkml/src/linkml/generators/shaclgen.py
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,9 @@ class ShaclGenerator(Generator):
ignores any per-slot ``in_language``.
"""

diff_stable: bool = False
"""Reduce blank-node label churn. See :ref:`rdf-in-version-control`."""

emit_rules: bool = True
"""Emit ``sh:sparql`` constraints from LinkML ``rules:`` blocks.

Expand Down Expand Up @@ -196,7 +199,7 @@ def generate_header(self) -> str:
def serialize(self, **args) -> str:
g = self.as_graph()
fmt = "turtle" if self.format in ["owl", "ttl"] else self.format
return canonicalize_rdf_graph(g, output_format=fmt)
return canonicalize_rdf_graph(g, output_format=fmt, diff_stable=self.diff_stable)

def as_graph(self) -> Graph:
sv = self.schemaview
Expand Down Expand Up @@ -929,6 +932,15 @@ def add_simple_data_type(func: Callable, r: ElementName) -> None:
"sh:NodeShape. Use --no-emit-rules to suppress rule generation."
),
)
@click.option(
"--diff-stable/--no-diff-stable",
default=False,
show_default=True,
help=(
"Reduce blank-node label churn across edits. See "
"https://linkml.io/linkml/howtos/collaborative-development.html#rdf-in-version-control"
),
)
@click.version_option(__version__, "-V", "--version")
def cli(yamlfile, **args):
"""Generate SHACL turtle from a LinkML model"""
Expand Down
14 changes: 13 additions & 1 deletion packages/linkml/src/linkml/generators/shexgen.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,9 @@ class ShExGenerator(Generator):
uses_schemaloader = True

# ObjectVars
diff_stable: bool = False
"""Reduce blank-node label churn in RDF output. See :ref:`rdf-in-version-control`."""

shex: Schema = field(default_factory=lambda: Schema()) # ShEx Schema being generated
shapes: list = field(default_factory=lambda: [])
shape: Shape | None = None # Current shape being defined
Expand Down Expand Up @@ -177,7 +180,7 @@ def end_schema(self, output: str | None = None, **_) -> str:
g = Graph()
g.parse(data=shex, format="json-ld", version="1.1")
g.bind("owl", OWL)
shex = canonicalize_rdf_graph(g, output_format="turtle")
shex = canonicalize_rdf_graph(g, output_format="turtle", diff_stable=self.diff_stable)
elif self.format == "shex":
g = Graph()
self.namespaces.load_graph(g)
Expand Down Expand Up @@ -258,6 +261,15 @@ def _get_subproperty_values(self, slot: SlotDefinition) -> list:
help="If --expand-subproperty-of (default), slots with subproperty_of will generate NodeConstraint "
"values containing all slot descendants. Use --no-expand-subproperty-of to disable this behavior.",
)
@click.option(
"--diff-stable/--no-diff-stable",
default=False,
show_default=True,
help=(
"Reduce blank-node label churn across edits (--format rdf). See "
"https://linkml.io/linkml/howtos/collaborative-development.html#rdf-in-version-control"
),
)
@click.version_option(__version__, "-V", "--version")
def cli(yamlfile, **args):
"""Generate a ShEx Schema for a LinkML model"""
Expand Down
1 change: 1 addition & 0 deletions packages/linkml_runtime/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ dev = [
linkml-normalize = "linkml_runtime.processing.referencevalidator:cli"

[project.optional-dependencies]
diff-stable = ["diffable-rdf>=0.4.0"]
dev = [
"coverage",
"requests-cache>=1.3.3",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -287,6 +287,7 @@ def _is_safe_prefix_iri(iri: str) -> bool:
def canonicalize_rdf_graph(
graph: rdflib.Graph,
output_format: str = "turtle",
diff_stable: bool = False,
) -> str:
"""Serialize an rdflib Graph deterministically using RDFC-1.0 canonicalization.

Expand All @@ -300,8 +301,22 @@ def canonicalize_rdf_graph(

:param graph: The rdflib Graph to serialize.
:param output_format: Target serialization format (e.g. ``"turtle"``, ``"nt"``).
:param diff_stable: Use diffable-rdf labels to reduce blank-node churn across
edits. Requires the ``diff-stable`` extra; disabled by default. The
non-standard RDF fallback warns when it cannot apply these labels.
See https://linkml.io/linkml/howtos/collaborative-development.html#rdf-in-version-control
:return: Deterministic string serialization of the graph.
:raises ImportError: If diff-stable labels are requested without diffable-rdf.
"""
if diff_stable:
try:
from diffable_rdf import wl_relabel_quads
except ImportError as exc:
raise ImportError(
"diff_stable=True requires diffable-rdf. Install "
"'linkml-runtime[diff-stable]' or 'linkml[diff-stable]'."
) from exc

ox_format = _FORMAT_MAP.get(output_format.lower())
if ox_format is None:
warnings.warn(
Expand Down Expand Up @@ -330,6 +345,16 @@ def canonicalize_rdf_graph(
RDFCanonicalizationWarning,
stacklevel=2,
)
if diff_stable:
warnings.warn(
"diff_stable=True was requested but this graph took the rdflib fallback, "
"which cannot apply Weisfeiler-Lehman blank-node labels. Output is "
"deterministic but NOT diff-stable: an unrelated edit may still renumber "
"blank nodes. Make the offending terms standard RDF (absolute IRIs, IRI "
"predicates) to get diff-stable labels.",
RDFCanonicalizationWarning,
stacklevel=2,
)
return _deterministic_fallback_serialize(graph, output_format)

dataset = ox.Dataset()
Expand All @@ -339,13 +364,17 @@ def canonicalize_rdf_graph(
# 3. Canonicalize blank node labels with RDFC-1.0.
dataset.canonicalize(ox.CanonicalizationAlgorithm.RDFC_1_0)

quads = list(dataset)

if diff_stable:
quads = wl_relabel_quads(quads)

# 4. Sort triples for deterministic ordering.
# RDFC-1.0 stabilizes blank-node labels but pyoxigraph's Dataset
# iteration order is not sorted and varies across processes (verified
# empirically against pyoxigraph 0.5.8). The explicit string-key sort
# is load-bearing for byte-identical output across runs; see
# tests/linkml_runtime/test_utils/test_rdf_canonicalize.py::test_sort_is_load_bearing.
quads = list(dataset)
sorted_triples = sorted(
(ox.Triple(q.subject, q.predicate, q.object) for q in quads),
key=lambda t: (str(t.subject), str(t.predicate), str(t.object)),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -880,7 +880,7 @@ def get_slot(
:param slot_name: name of the slot to be retrieved
:param imports: include import closure
:param attributes: include attributes
:param strict: raise ValueError is not found
:param strict: raise ValueError if not found
:return: slot definition
"""
slot = self.all_slots(imports=imports, attributes=False).get(slot_name, None)
Expand All @@ -904,8 +904,9 @@ def get_subset(
) -> SubsetDefinition | None:
"""Retrieve a subset from the schema.

:param subset_name: name of the subsey to be retrieved
:param subset_name: name of the subset to be retrieved
:param imports: include import closure
:param strict: raise ValueError if not found
:return: subset definition
"""
s = self.all_subsets(imports).get(subset_name, None)
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,7 @@ markers = [
"yarrrml: End-to-end tests for the YARRRML generator",
"typedbgen: Tests for the TypeDB generator",
"arrays: Array specifications!",
"diffable_rdf: tests that exercise the diffable-rdf integration (--diff-stable); run as a canary against linkml in diffable-rdf's own CI",
]

# https://docs.astral.sh/ruff/settings/
Expand Down
37 changes: 36 additions & 1 deletion tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -422,7 +422,8 @@ class MockImportErrorFinder(MetaPathFinder):
"""
Fake like we don't have a module when we really do.

see the ``mock_black_import`` fixture for example usage.
see the ``mock_missing_import`` fixture to hide any module, or ``mock_black_import``
for a single-purpose example.

.. note::

Expand All @@ -442,6 +443,40 @@ def find_spec(self, fullname, path, target):
return None


@pytest.fixture(scope="function")
def mock_missing_import():
"""Pretend an optional dependency is not installed, for any module.

Yields a callable taking a module name, so a test for one optional
dependency does not need a fixture of its own::

def test_requires_the_extra(mock_missing_import):
mock_missing_import("diffable_rdf")
with pytest.raises(ImportError, match="linkml-runtime[diff-stable]"):
...

See :class:`MockImportErrorFinder` for the caveat about reimporting
modules that already imported the one being hidden.
"""
removed = {}
finders = []

def hide(module: str) -> None:
for name, loaded in list(sys.modules.items()):
if name.startswith(module):
removed[name] = loaded
del sys.modules[name]
finder = MockImportErrorFinder(module)
finders.append(finder)
sys.meta_path.insert(0, finder)

yield hide

sys.modules.update(removed)
for finder in finders:
sys.meta_path.remove(finder)


@pytest.fixture(scope="function")
def mock_black_import():
"""
Expand Down
Loading
Loading