test: retire the apps-parity dumps, name tools, fixtures and tests by subject

.github/workflows/ci.yml

@@ -7,10 +7,9 @@
 # the body. Package.resolved is not tracked, so CI resolves the newest ImagoKit the manifest's range allows, as a consumer does.
 #
 # THE PINS' PYTHON ORACLES ARE NOT RUN HERE. The tests read committed fixtures (Tests/*/Fixtures/*.json, generated by
-# tools/*/pins.py against pymatgen 2026.9.24, pymatgen-core 2026.9.23, spglib 2.8.0, NumPy 2.5.4, SciPy 1.18.1 and py4DSTEM
+# tools/pins/ against pymatgen 2026.9.24, pymatgen-core 2026.10.16, spglib 2.8.0, NumPy 2.5.4, SciPy 1.18.1 and py4DSTEM
 # f050d207's table) and compare the kit with them. Regenerating a fixture needs the gitignored References/ (the pinned
-# virtual environment and clones) on the maintainer's machine; the parity dumps against the original applications need those applications checked out
-# beside this repository. CI therefore proves "the kit equals the committed pins", not "the pins equal the oracle".
+# virtual environment and clones) on the maintainer's machine. CI therefore proves "the kit equals the committed pins", not "the pins equal the oracle".
 #
 # Never `swift build --build-tests -c release` (fails on this toolchain's Swift Build backend); `swift test -c release` is the gate.
 name: CI

CONTRIBUTING.md

@@ -18,21 +18,37 @@ read committed fixtures and need no Python.
 Every number a scientist reads is covered by a test against a reference: pymatgen and spglib for cells, symmetry and
 d-spacings, py4DSTEM for scattering factors and diffraction geometry, SciPy and NumPy for numerical results, or a closed form
 or planted truth where no package applies. The reference outputs are committed as JSON fixtures under `Tests/*/Fixtures/`,
-with the generator scripts under `tools/`. `docs/parity.md` lists what is held against which reference.
+with the generator scripts under `tools/pins/`. `docs/parity.md` lists what is held against which reference.
 
 A test's tolerance is the one the Python package's own tests use for that function; a result outside it is a bug. A change
 that moves a fixture says why in the pull request and in the CHANGELOG; regenerate a fixture with its script and the pinned
 versions of the reference.
 
 ## Regenerating fixtures
 
-The generators run in pinned Python virtual environments outside the repository (Python 3.12, NumPy 2.5.4, SciPy 1.18.1;
-pymatgen, spglib and orix in one, py4DSTEM in the other), placed at `../References/python/orix` and
-`../References/python/py4dstem` beside the checkout; each script's header names the one it needs. The current `orix` venv
-holds pymatgen-core 2026.10.16 under a `pymatgen` 2026.9.24 metapackage; the README and `docs/parity.md` say 2026.9.24, and the
-older fixtures record pymatgen-core 2026.9.23 (a previous venv). `cif-symmetry-pins.json` was regenerated on the current
-install on 2026-10-21 and came out byte-identical; the other fixtures were not regenerated. Regenerate and re-pin them
-together at the next fixture change.
+Every fixture generator lives under `tools/pins/<subject>/`, and each script's header names the command that runs it. They
+run in three Python 3.12 virtual environments outside the repository, beside the checkout under `../References/python/`,
+one per oracle, each built from its requirements file:
+
+| Venv | Requirements | Runs |
+|---|---|---|
+| `../References/python/pymatgen` | `tools/pins/requirements/pymatgen.txt` (pymatgen-core 2026.10.16, spglib 2.8.0, NumPy 2.5.4, SciPy 1.18.1) | every generator except the two below |
+| `../References/python/orix` | `tools/pins/requirements/orix.txt` (orix 0.15.0) | `tools/pins/orix` |
+| `../References/python/py4dstem` | `tools/pins/requirements/py4dstem.txt` (py4DSTEM f050d207) | `tools/pins/acom-matcher`, and the py4DSTEM halves the pymatgen scripts of `phase-vectors` and `orientation-symmetry` start themselves |
+
+```sh
+for oracle in pymatgen orix py4dstem; do
+  python3.12 -m venv ../References/python/$oracle
+  ../References/python/$oracle/bin/pip install -r tools/pins/requirements/$oracle.txt
+done
+tools/pins/structure-factors/fetch-table.sh                  # py4DSTEM's scattering table, sha256-locked
+python3 tools/crystal-structures/make_beta_double_prime.py   # the beta'' CIF the cell pins read
+```
+
+The last two write into the checkout's own gitignored `References/`.
+
+Every fixture that records a pymatgen version records pymatgen-core 2026.10.16 (the `pymatgen` metapackage is 2026.9.24).
+A regenerated fixture that moves is its own commit with the reason and a CHANGELOG line.
 
 ## Pull requests
 

NOTICE

@@ -49,7 +49,7 @@ without 2 pi; structure factors in A) follows py4DSTEM's Crystal class
 (py4DSTEM/process/diffraction/crystal.py). The Lobato & Van Dyck electron
 scattering table of DiffractoScattering is py4DSTEM's (commit f050d207,
 scattering_factors.txt, sha256 be845d1d..., fetched by
-tools/slice-2/fetch-table.sh into the gitignored References/ and pinned bit for
+tools/pins/structure-factors/fetch-table.sh into the gitignored References/ and pinned bit for
 bit), and the ring fit's models follow py4DSTEM's fit_ellipse_1D and
 fit_ellipse_amorphous_ring (process/calibration/ellipse.py) through
 mac4DSTEM's port.
@@ -64,8 +64,8 @@ pymatgen (used as a reference for checking; no code copied)
 
 Cells, expanded CIF sites, symmetry operations (spglib behind pymatgen),
 d-spacings and the reflections within a sphere are checked against pymatgen
-2026.9.24 / pymatgen-core 2026.9.23 by tools/slice-1/pins.py, whose output is
-the committed fixture slice-1-pins.json. The Laue orbit and the absence rule are
+2026.9.24 / pymatgen-core 2026.10.16 by tools/pins/cells/pins.py, whose output
+is the committed fixture pymatgen-cells-pins.json. The Laue orbit and the absence rule are
 computed from pymatgen's operations in NumPy.
 
     pymatgen - https://github.com/materialsproject/pymatgen (MIT License)
@@ -109,15 +109,16 @@ Py4DSTEMIPFKey.swift reimplements the full-mask beam colour part of py4DSTEM
 f050d207851f8accd3aa550365baaf1f38230428, crystal_viz.py
 plot_orientation_maps (GPL-3.0); its automatic sector bases are generated by
 calling that version's orientation_plan. pymatgen PointGroup (2026.9.24 /
-core 2026.9.23, MIT) is an oracle for all eleven proper rotation tables;
+core 2026.10.16, MIT) is an oracle for all eleven proper rotation tables;
 SciPy Rotation is an oracle for the minimum-angle representative. No orix or
-MTEX code is copied into the library. tools/symmetry/pins.py records provenance.
+MTEX code is copied into the library. tools/pins/orientation-symmetry/pins.py records
+provenance.
 
 ================================================================================
 orix (used as a reference for checking; no code copied)
 ================================================================================
 
-tools/orix/*-pins.py run orix 0.15.0 as the oracle for the IPF colour key,
+tools/pins/orix/*-pins.py run orix 0.15.0 as the oracle for the IPF colour key,
 direction reduction, disorientation and misorientation (docs/parity.md P26 and
 P36); their output is committed as Tests/DiffractoOrientationTests/Fixtures/
 orix-*-pins.json. No orix code is copied into the library.

README.md

@@ -208,12 +208,12 @@ F(hkl) is in A^-2 (f in A over V in A^3); a pattern's rows are `qx` and its colu
 
 Every number has a reference, and the references are pinned (`CONTRIBUTING.md`):
 
-- **pymatgen 2026.9.24** with pymatgen-core 2026.9.23, spglib 2.8.0, NumPy 2.5.4, Python 3.14.7 (cells, expanded CIF sites,
+- **pymatgen 2026.9.24** with pymatgen-core 2026.10.16, spglib 2.8.0, NumPy 2.5.4, Python 3.14.7 (cells, expanded CIF sites,
   symmetry operations, d-spacings, reflections within a sphere, the X-ray peak sums), SciPy 1.18.1 (the ring fit's
   least-squares optimum), **py4DSTEM** f050d207's `scattering_factors.txt` (sha256 `be845d1d...`, bit for bit), and the
   code of the earlier GPL-3.0 applications this library was ported from (pinned by commit in `NOTICE`) where a port must be
   bit identical.
-- The pins are generated before the Swift code (`tools/*/pins.py`, committed as JSON under `Tests/*/Fixtures/`); the
+- The pins are generated before the Swift code (`tools/pins/<subject>/`, committed as JSON under `Tests/*/Fixtures/`); the
   Swift tests read the pins and a planted truth (a closed form, a group-theory count, a cell content). Numerics are held
   to the tolerance the Python package's own tests use; readers, I/O, indices and metadata are exact. Agreement with a
   package is agreement, not truth.
@@ -232,8 +232,8 @@ swift run -c release diffracto-example
 ```
 
 `swift build --build-tests -c release` fails on current toolchains; use `swift test -c release`. The tests read committed
-pins and need no Python. The Python environment that generates the pins is not in the repository: the versions are listed
-under "Oracles and pins" above, and `tools/*/pins.py` are the generators.
+pins and need no Python. The Python environment that generates the pins is described in `CONTRIBUTING.md`, with a requirements
+file per oracle under `tools/pins/requirements/`; the generators are under `tools/pins/<subject>/`.
 
 ## Licence
 

Sources/DiffractoGeometry/EllipseCalibration.swift

@@ -10,8 +10,8 @@
 //  0.30 px ring-width floor guard. NOT ported, because they never landed there: the annulus guard (`ringOutsideAnnulus`)
 //  and the seed check. The only changes are the access (`public`), `DiffractionPattern`'s field names (`pattern.height` is
 //  the seed's `pattern.qy`, the ROW count; `pattern.width` its `pattern.qx`, the COLUMN count) and the removal of
-//  `EllipseCalibrationModel`, which `RingCalibration.swift` holds. The tests prove the arithmetic by bit-identity against
-//  the seed compiled as it was (Tests/.../EllipseAppsParityTests.swift, tools/slice-3b/apps-parity).
+//  `EllipseCalibrationModel`, which `RingCalibration.swift` holds. The tests hold the fit to SciPy's optimum of the same
+//  objective (EllipseCalibrationTests) and to planted truth (the noisy and stop-rule rings).
 //
 //  COORDINATES (py4DSTEM's order, as the seed): `qx` is the ROW index and `qy` the COLUMN index everywhere, so the loops
 //  below read `for qx in 0..<pattern.height` and `pixels[qx * pattern.width + qy]`. `theta` lies in [0, pi) and is measured

Sources/DiffractoGeometry/RingCalibration+Q.swift

@@ -23,7 +23,7 @@ extension RingCalibration {
     /// NaN unless R, L and λ are finite and positive, and NaN where the result is not a finite positive number: an overflow (a
     /// subnormal λ) or an underflow to 0 (a positive R so small against L that atan(R/L) is 0) is never given as infinity or 0.
     /// R may be as large as you like against L: 2θ stays below 180° and q below 2/λ, so there is no other limit.
-    /// VALIDITY: against 60-digit arithmetic (tools/kit-0.0.4/q-pins.py, 100 cases, V 60 to 300 kV, L 500 px to 1e5 px, R 1 px
+    /// VALIDITY: against 60-digit arithmetic (tools/pins/ring-calibration/q-pins.py, 100 cases, V 60 to 300 kV, L 500 px to 1e5 px, R 1 px
     /// to 2000 px) the worst relative error is 8.2e-16: atan2 and sin each round once and neither amplifies (the form has no
     /// subtraction). The 1/d identity on the 320 ring-calibration pin cases holds to 5.5e-16. The trig-free form
     /// 2 sin(½ atan(R/L)) = (R/h) / √((1 + L/h)/2), h = hypot(R, L), was measured too and is no better (both 2.2e-16 in
@@ -42,7 +42,7 @@ extension RingCalibration {
     /// underflow to 0), never an infinity.
     /// VALIDITY: the form is exact, but qλ/2 is a double and tan 2θ diverges at 90°. The rounding of qλ/2 (1.1e-16 relative)
     /// reaches R multiplied by 1/((1 − s²)(1 − 2s²)), s = qλ/2 and 1 − 2s² = cos 2θ, so against 60-digit arithmetic
-    /// (tools/kit-0.0.4/q-pins.py, 120 cases) the worst relative error is 3.3e-15 on the 100 grid cases (V 60 to 300 kV, L
+    /// (tools/pins/ring-calibration/q-pins.py, 120 cases) the worst relative error is 3.3e-15 on the 100 grid cases (V 60 to 300 kV, L
     /// 500 px to 1e5 px, R 1 px to 2000 px, 2θ up to 76°) and 4.5e-14 at 0.99 of the limit (the 20 cases at s√2 = 0.99, the
     /// factor about 100, 2θ = 88.9°), about 1e-15/cos 2θ in general. That passes the 1e-12 band for 2θ below about 89.9°;
     /// closer to the limit the number is right only to that error, and the `maximumLambdaOver2d` guard is the only refusal (one

Sources/DiffractoGeometry/RingIndexing.swift

@@ -13,7 +13,7 @@
 //  gives a confident wrong index. k is a Gaussian coverage factor (default 3). `kMax` is the k_max the list was built to
 //  (Å⁻¹): the list is complete only below it, which is what `beyondKMax` guards.
 //
-//  THE DEFINITION (tools/slice-3/index-pins.py is written from the same text):
+//  THE DEFINITION (tools/pins/ring-indexing/pins.py is written from the same text):
 //    z, δ       z = |q − g| / σ_q; the signed deviation δ = (d_meas − d)/d = g/q − 1; |Δd|/d = |δ|. Candidates order by
 //               (z, g).
 //    margin     between two candidates, their log-likelihood ratio (z₂² − z₁²)/2; accepting needs ≥ ln 19 (the next at

Sources/DiffractoOrientation/NativeIPFKeys.swift

@@ -1,5 +1,5 @@
 // Native IPF keys and direction samplers, ported from GPL-3.0 source c809679c.
-// Oracle: tools/symmetry/pins.py compiles the original git-show source.
+// Oracle: tools/pins/orientation-symmetry/pins.py compiles the original git-show source.
 import Foundation
 import simd
 /// Cubic crystal-symmetry operations and the m-3m inverse-pole-figure color

Sources/DiffractoOrientation/OrixIPFKey.swift

@@ -1,5 +1,5 @@
 // The EDAX/TSL inverse-pole-figure colour key of orix 0.15.0 (itself adapted from MTEX), for all eleven Laue classes.
-// Sectors: OrixIPFSectors.swift, generated by tools/orix/ipf-pins.py. Pinned in OrixIPFKeyTests (orix-ipf-pins.json).
+// Sectors: OrixIPFSectors.swift, generated by tools/pins/orix/ipf-pins.py. Pinned in OrixIPFKeyTests (orix-ipf-pins.json).
 import Foundation
 import simd
 import DiffractoCore

Sources/DiffractoOrientation/OrixIPFSectors.swift

@@ -1,4 +1,4 @@
-// Generated by tools/orix/ipf-pins.py from orix 0.15.0 (fundamental sectors, taken from MTEX). Do not edit.
+// Generated by tools/pins/orix/ipf-pins.py from orix 0.15.0 (fundamental sectors, taken from MTEX). Do not edit.
 import simd
 import DiffractoCore
 

Sources/DiffractoOrientation/Py4DSTEMDirectionFamily.swift

@@ -1,5 +1,5 @@
 // py4DSTEM f050d207 `Crystal.symmetry_reduce_directions` (crystal_ACOM.py:2581), for the eleven Laue classes.
-// Pinned in Py4DSTEMFamilyTests (py4dstem-family-pins.json, generated by tools/symmetry/family-pins.py).
+// Pinned in Py4DSTEMFamilyTests (py4dstem-family-pins.json, generated by tools/pins/orientation-symmetry/family-pins.py).
 import Foundation
 import simd
 import DiffractoCore
@@ -14,7 +14,7 @@ public enum Py4DSTEMOperatorOrder: Sendable, Equatable, CaseIterable {
     /// centred groups) gave identical families on every pinned matrix. The default.
     case pymatgen
     /// The proper operators sorted by their fractional matrices, then their negatives: the harness order of
-    /// `tools/symmetry/family-pins.py` (py4DSTEM's function on an operator list that is not a real crystal's).
+    /// `tools/pins/orientation-symmetry/family-pins.py` (py4DSTEM's function on an operator list that is not a real crystal's).
     case sortedFractional
 }
 

Sources/DiffractoOrientation/Py4DSTEMIPFKey.swift

@@ -1,4 +1,4 @@
-// Generated by tools/symmetry/color-table.py from the pinned py4DSTEM automatic plan sectors.
+// Generated by tools/pins/orientation-symmetry/color-table.py from the pinned py4DSTEM automatic plan sectors.
 // Inverse-transpose sector matrices are pinned to avoid platform-dependent inversion at runtime.
 import Foundation
 import simd

Sources/DiffractoOrientation/PymatgenOperatorTable.swift

@@ -1,4 +1,4 @@
-// Generated by tools/symmetry/family-real-pins.py. Do not edit.
+// Generated by tools/pins/orientation-symmetry/family-real-pins.py. Do not edit.
 // The order in which pymatgen (SpacegroupAnalyzer.get_point_group_operations, as py4DSTEM 0.14.19 reads it) lists the
 // operations of the point group of a centrosymmetric structure: rotation matrices in the fractional basis, row major.
 import DiffractoCore

Tests/DiffractoCoreTests/CellStandardisationTests.swift

@@ -1,7 +1,7 @@
 //
 //  CellStandardisationTests.swift
 //  Materials Project import numerics against pymatgen SpacegroupAnalyzer (fixtures/mp-import/pymatgen-pins.json,
-//  tools/mp-import/pins.py: pymatgen 2026.9.24, pymatgen-core 2026.9.23, spglib 2.8.0, NumPy 2.5.4). The inputs are
+//  tools/pins/mp-import/pins.py: pymatgen 2026.9.24, pymatgen-core 2026.10.16, spglib 2.8.0, NumPy 2.5.4). The inputs are
 //  committed MP-shaped JSON; nothing here touches the network.
 //
 

Tests/DiffractoCoreTests/Slice1bSupport.swift → Tests/DiffractoCoreTests/ConventionalCellPinSupport.swift

@@ -1,6 +1,6 @@
 //
-//  Slice1bSupport.swift
-//  Role: the pins of tools/slice-1b/pins.py (spglib 2.8.0 and pymatgen as oracles for the reduced cell, the lattice's
+//  ConventionalCellPinSupport.swift
+//  Role: the pins of tools/pins/conventional-cell/pins.py (spglib 2.8.0 and pymatgen as oracles for the reduced cell, the lattice's
 //        Bravais name and conventional cell, the operation counts and the rings in the conventional setting), and the
 //        Niggli conditions written out from the text, so the tests check the reduced cell directly and not only the
 //        oracle.
@@ -12,21 +12,21 @@ import Testing
 import simd
 import DiffractoCore
 
-struct Pins1b: Decodable {
+struct ConventionalCellPins: Decodable {
     let versions: [String: String]
     let generatedAt: String
     let randomSummary: RandomSummary
-    let structures: [StructurePin1b]
+    let structures: [ConventionalStructurePin]
     let randomCells: [RandomCellPin]
 
-    static let fileName = "slice-1b-pins.json"
-    static let shared: Pins1b = {
+    static let fileName = "spglib-conventional-cell-pins.json"
+    static let shared: ConventionalCellPins = {
         do {
             let data = try Data(contentsOf: Fixtures.directory.appendingPathComponent(fileName))
             let decoder = JSONDecoder()
             decoder.keyDecodingStrategy = .convertFromSnakeCase
-            return try decoder.decode(Pins1b.self, from: data)
-        } catch { fatalError("slice-1b-pins.json: \(error)") }
+            return try decoder.decode(ConventionalCellPins.self, from: data)
+        } catch { fatalError("spglib-conventional-cell-pins.json: \(error)") }
     }()
 
     struct RandomSummary: Decodable { let rejectedDraws: Int; let worstOracleDifference: Double }
@@ -55,7 +55,7 @@ struct RingsPin: Decodable {
     let families: [RingPin]
 }
 
-struct StructurePin1b: Decodable {
+struct ConventionalStructurePin: Decodable {
     let name: String
     let source: String
     let cell: CellParams

Tests/DiffractoCoreTests/ConventionalCellTests.swift

@@ -13,7 +13,7 @@ import DiffractoCore
 
 struct ConventionalCellTests {
 
-    static let pins = Pins1b.shared
+    static let pins = ConventionalCellPins.shared
 
     /// The name is the lattice each cell was built as (truth) and spglib's (oracle), on 2800 random cells under random
     /// unimodular transformations, 200 per lattice.

Tests/DiffractoCoreTests/ConventionalSymmetryTests.swift

@@ -14,10 +14,10 @@ import DiffractoCore
 
 struct ConventionalSymmetryTests {
 
-    static let pins = Pins1b.shared
+    static let pins = ConventionalCellPins.shared
 
     /// Every pinned structure, detected once (default `conventional: true`).
-    static let detected: [(pin: StructurePin1b, structure: CrystalStructure)] = pins.structures.map {
+    static let detected: [(pin: ConventionalStructurePin, structure: CrystalStructure)] = pins.structures.map {
         ($0, CrystalStructure.detectingSymmetry(cell: $0.cell.unitCell, sites: $0.atomSites))
     }
 

Tests/DiffractoCoreTests/CrystalPresetsTests.swift

@@ -4,7 +4,7 @@ import Testing
 import simd
 import DiffractoCore
 
-/// PresetPins: tools/presets/pins.py (pymatgen 2026.9.24, spglib 2.8.0). The pins build each structure independently, from the
+/// PresetPins: tools/pins/presets/pins.py (pymatgen 2026.9.24, spglib 2.8.0). The pins build each structure independently, from the
 /// published cell and Wyckoff positions with `Structure.from_spacegroup`. Tolerances, written per quantity: lattice
 /// parameters, volume and d(hkl) 1e-7 relative (the reference's tolerance; was 1e-12, closed forms in float64 on both sides); fractional site positions 1e-7
 /// (the kit's hand lists carry 2/3 as a Double, pymatgen's orbit expansion computes it); counts, Z, space-group number