Add CoRE MOF v26.0.2 Lux schemas by 100mingyu-rgb · Pull Request #2156 · materialsproject/MPContribs

100mingyu-rgb · 2026-09-13T13:21:36Z

Proposed PR title

Add CoRE MOF v26.0.2 Lux schemas

Summary

Add a core_mofs project package with seven released-artifact schemas.
Preserve the release structure_id as the canonical cross-artifact key.
Model CrystalNets topology as nested Pydantic records, including successful,
partial, and failed executions.
Preserve checker-specific JSON payloads as validated JSON strings in the
combined checker table.
Enforce release invariants such as identity-field agreement, availability
flags, diagnostic mappings, and checker vote consistency.

Validation performed locally

29 relationally complete example structures validated across all artifacts.
All five one-to-one artifacts validated for 42,574 structures each.
212,870 checker-finding rows validated with exactly five unique checkers per
structure.
49,028 calculation-diagnostic rows validated with unique
(structure_id, calculation) keys.
Cross-artifact identifier and compound-key checks passed.
The repository-standard emmet.core.arrow.arrowize check passed for all 12
top-level and nested Pydantic models.

Maintainer review questions

Is core_mofs the preferred project namespace?
Should the combined checker JSON columns remain validated strings, or should
each checker become a separate nested schema?
Should the seven artifacts remain separate models, or should selected
one-to-one fields be composed into one contribution model?

No source CIFs or internal-review data are included in this schema-only change.

100mingyu-rgb · 2026-09-13T13:22:02Z

@bfoley12 The CoRE MOF v26.0.2 Lux schema PR is ready for your review. GitHub did not allow the fork author to add a formal review request, so I am tagging you here as requested. The PR is schema-only and includes the design questions in the description.

bfoley12

In addition to comments in the PR:

Unclear what the "Contribution" is
All field names must be camelCase, not snake_case
Field names should not include units - use client.init_columns to set the units of the data
Lots of Literals that go unused
Possibility for non-final contributions to be submitted

We will go through more rounds of review. There are things that will need to be refactored, but this needs to get to a more final state before that.

bfoley12 · 2026-09-14T17:27:57Z

This is not needed - we save structures as a Structure object internally. You can submit structures by using submit_contributions and having structures described in the submitted data.

Note: You will also need to include sites, lattice, and optionally charge fields in addition to cif.

Sites that use CifManifestRecord can reference pymatgen.Structure instead.

bfoley12 · 2026-09-14T17:31:22Z

It looks like this accepts Errors from a pipeline? Only successfully ran pipelines should have final data submitted.

bfoley12 · 2026-09-14T17:32:15Z

It looks like this accepts Errors from a pipeline? Only successfully ran pipelines should have final data submitted.

bfoley12 · 2026-09-14T17:34:35Z

+StructureId = Annotated[
+    str,
+    Field(
+        pattern=r"^(ASR|FSR|ION)-(COD|CSD|SI)-(?:\d{4}|UNKN)-\d{4}$",


Duplication of _STRUCTURE_ID_RE

bfoley12 · 2026-09-14T17:43:23Z

It looks like this accepts Errors from a pipeline? Only successfully ran pipelines should have final data submitted.

bfoley12 · 2026-09-14T18:09:41Z

+
+    parsed = parse_structure_id(structure_id)
+    if cif_file is not None and cif_file != f"cifs/{structure_id}.cif":
+        raise ValueError("cif_file must equal cifs/{structure_id}.cif")


missing f-string

bfoley12 · 2026-09-14T18:10:48Z

+PositiveSize = Annotated[
+    int,
+    Field(gt=0, description="File size in bytes."),
+]
+NonNegativeInt = Annotated[int, Field(ge=0)]
+NonNegativeFloat = Annotated[float, Field(ge=0)]


These exist as pydantic.* - use them directly

bfoley12 · 2026-09-14T18:12:23Z

+def _parse_dimension(value: object) -> object:
+    """Coerce integer CSV cells while retaining a bounded JSON integer type."""
+
+    if isinstance(value, str) and value in {"0", "1", "2", "3"}:
+        return int(value)
+    return value
+
+
+Dimension = Annotated[Literal[0, 1, 2, 3], BeforeValidator(_parse_dimension)]


Pydantic handles string coercion. Can drop the validator

bfoley12 · 2026-09-14T18:13:16Z

+    metal_element_count: NonNegativeInt = Field(
+        description="Number of distinct detected metal elements."
+    )
+    metal_atom_count: NonNegativeFloat = Field(


count should be Int

bfoley12 · 2026-09-14T18:23:44Z

This will most likely become a Table.

100mingyu-rgb · 2026-09-16T05:31:51Z

@bfoley12 Thank you for the review. I revised the schema so that one contribution represents one final CoRE MOF structure identified by a unique structureId. The previous artifact-specific schemas were replaced with one minimal CoreMofContribution using StructureType. Fields are camelCase and unit-free, optional fields default to None, and operational or non-final pipeline data were removed. The updated tests pass locally. Could you please review the revised structure?

bfoley12

This was a major change from the last commit. Make sure you are including enough information. The information just has to be a well-structured final datapoint (with some associated data (tables, structures).

Overall good step in the right direction, just make sure you aren't leaving out additional meaningful data.

bfoley12 · 2026-09-16T21:36:31Z

+    structureId: StructureId = Field(
+        description="Release-preserved identifier for this CoRE MOF structure."
+    )


Since these are your unique identifiers, they should not live in this class. when submitting contributions, you should set identifiers to your structureId

bfoley12 · 2026-09-16T21:44:32Z

+    structure: StructureType = Field(
+        description="Pymatgen Structure containing lattice, sites, and charge."
+    )
+    cif: Annotated[str, StringConstraints(min_length=1)] = Field(
+        description="Original Crystallographic Information File text."
+    )


We can remove these. Every Contribution carries with it a structures field. When submitting you contributions, you can also submit a Structure there.

See this Quickstart example. Except in that example, identifier would be your structureId and data is what we are describing in this model. Then, structures would be a list of pymatgen Structures.

Let me know if a further example would help!

bfoley12 · 2026-09-16T21:46:10Z

+    sourceDatabase: SourceDatabase = Field(
+        description="COD, CSD, or supporting-information provenance category."
+    )
+    sourceId: Annotated[str, StringConstraints(min_length=1)] = Field(


Can you put a max_length on this? It helps prevent abuse. You can make it something like 1.5x the max length of the longest source ID that you have. Even better would be to more strongly enforce the format of the sourceId

bfoley12 · 2026-09-16T21:47:39Z

+    formula: str | None = Field(
+        default=None,
+        description="Chemical formula recorded for the release structure."
+    )


This will also go in the top level of the Contribution. Here, we are just describing Contribution.data. See the Quickstart link in an earlier comment

Remove the project-specific README and schema test file from this PR. Keep all six schema and import modules and the package-level README unchanged.

Add CoRE MOF v26.0.2 Lux schemas

b6d52ea

100mingyu-rgb force-pushed the feat/core-mofs-lux-schema branch from c3f184c to b6d52ea Compare September 14, 2026 00:00

bfoley12 self-requested a review September 14, 2026 16:19

bfoley12 reviewed Sep 14, 2026

View reviewed changes

Simplify CoRE MOF contribution schema

45ed790

bfoley12 requested changes Sep 16, 2026

View reviewed changes

100mingyu-rgb added 2 commits October 2, 2026 21:31

Refactor CoRE MOF data and table schemas

15ede6d

Limit CoRE MOF PR to schema modules

a2d7993

Remove the project-specific README and schema test file from this PR. Keep all six schema and import modules and the package-level README unchanged.

Conversation

100mingyu-rgb commented Sep 13, 2026

Proposed PR title

Summary

Validation performed locally

Maintainer review questions

Uh oh!

100mingyu-rgb commented Sep 13, 2026

Uh oh!

bfoley12 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

100mingyu-rgb commented Sep 16, 2026

Uh oh!

bfoley12 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants